You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-05 - 14:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Sun Apr 05, 2026 00:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21756995517,22cyclictest0-21swapper/219:40:122
2175699531,32cyclictest28623-21diskmemload23:10:132
2175699531,32cyclictest28623-21diskmemload23:10:122
21756995123,24cyclictest29282-21ssh21:40:112
2174899513,41cyclictest25236-21ssh22:50:141
2176499505,33cyclictest28366irq/58-eth022:15:133
2174099503,36cyclictest3818-21ssh22:20:110
21748994832,7cyclictest15267-21sh00:15:141
2175699462,29cyclictest20352-21ssh21:35:142
2174099466,8cyclictest131rcu_preempt22:45:150
21740994610,7cyclictest131rcu_preempt22:00:130
21740994536,5cyclictest131rcu_preempt22:15:130
21740994523,9cyclictest131rcu_preempt20:25:140
2174099452,36cyclictest24337-21ssh00:20:140
2174099441,39cyclictest7258-21ssh22:40:110
2176499422,23cyclictest28366irq/58-eth022:30:123
2175699404,29cyclictest25919-21ssh21:20:122
2174099401,31cyclictest29296-21ssh21:40:120
2176499391,22cyclictest28366irq/58-eth023:40:133
21756993918,16cyclictest0-21swapper/221:55:122
21756993910,20cyclictest27-21ksoftirqd/222:00:132
2174099391,31cyclictest18680-21ssh00:35:120
2174099391,31cyclictest18680-21ssh00:35:110
2176499381,7cyclictest0-21swapper/300:25:143
21756993823,9cyclictest22346irq/40-dwc2_hso23:40:122
2174099382,28cyclictest20327-21ssh21:35:130
2174899371,30cyclictest1741-21sh22:55:131
2174099371,26cyclictest12-21ksoftirqd/023:35:110
2176499363,21cyclictest28366irq/58-eth021:40:123
2175699352,26cyclictest0-21swapper/200:15:122
2175699351,18cyclictest6648-21ntpq23:50:272
2174899353,24cyclictest30689-21ssh22:35:121
2176499342,19cyclictest28366irq/58-eth021:45:113
2174099341,27cyclictest10720-21ssh23:00:140
2174099341,27cyclictest10720-21ssh23:00:130
2176499332,17cyclictest28366irq/58-eth023:10:123
2176499332,17cyclictest28366irq/58-eth023:10:113
2175699332,21cyclictest0-21swapper/220:45:142
21756993312,4cyclictest0-21swapper/220:25:132
2174899335,6cyclictest11980-21ssh23:55:141
2174899333,21cyclictest23833-21ssh21:55:111
2174099332,23cyclictest809-21ssh00:25:120
2176499322,6cyclictest0-21swapper/300:15:133
2175699322,20cyclictest22346irq/40-dwc2_hso20:20:132
21756993217,9cyclictest0-21swapper/221:05:132
2174099323,18cyclictest12-21ksoftirqd/022:50:120
2174099320,23cyclictest0-21swapper/022:30:120
2175699313,19cyclictest22346irq/40-dwc2_hso20:55:132
2175699311,17cyclictest7844-21kworker/u8:021:14:112
2175699311,13cyclictest14412-21diskstats23:20:142
2176499303,17cyclictest28366irq/58-eth021:20:133
2176499303,14cyclictest28366irq/58-eth020:45:133
2176499302,4cyclictest0-21swapper/322:35:133
21748993018,4cyclictest21-21ksoftirqd/122:00:131
2174099302,6cyclictest131rcu_preempt20:20:120
2174099302,21cyclictest12-21ksoftirqd/000:10:130
21740993012,15cyclictest12-21ksoftirqd/023:30:130
2176499296,20cyclictest0-21swapper/323:55:143
2176499295,19cyclictest0-21swapper/321:35:143
2176499293,18cyclictest28366irq/58-eth022:40:113
2175699291,23cyclictest11382-21ssh21:30:142
2175699291,22cyclictest7274-21ssh22:40:132
2175699291,21cyclictest0-21swapper/223:35:132
2175699291,15cyclictest14567-21latency_hist21:50:022
2175699290,15cyclictest13347-21cut23:55:202
2174899291,16cyclictest21-21ksoftirqd/119:20:121
2174099292,20cyclictest14875-21ssh21:50:120
2174099292,19cyclictest29855-21ssh00:05:140
2175699283,15cyclictest22346irq/40-dwc2_hso19:50:132
2175699282,19cyclictest23213-21ssh23:25:132
21756992813,12cyclictest0-21swapper/221:00:132
2175699281,12cyclictest25393-21ntp_states22:30:262
2175699280,18cyclictest7501-21diskmemload23:08:552
2175699280,17cyclictest0-21swapper/223:33:252
2174899281,24cyclictest29848-21sh00:05:151
21748992810,11cyclictest0-21swapper/120:15:121
2174099282,21cyclictest2501-21ssh21:25:150
2174099282,21cyclictest2501-21ssh21:25:140
2174099282,10cyclictest131rcu_preempt20:40:110
2176499272,5cyclictest0-21swapper/319:30:143
2176499271,5cyclictest0-21swapper/323:38:483
2175699272,16cyclictest22346irq/40-dwc2_hso19:35:112
2175699272,12cyclictest16248-21ssh22:45:132
2175699271,23cyclictest924-21snmpd22:35:132
2175699271,22cyclictest22052-21cut19:10:142
2175699270,19cyclictest7501-21diskmemload00:10:132
2175699270,18cyclictest7501-21diskmemload22:18:302
2174899279,10cyclictest0-21swapper/100:35:121
2174899279,10cyclictest0-21swapper/100:35:121
2174899274,13cyclictest32205-21ssh23:30:131
2174899271,6cyclictest0-21swapper/121:25:131
2174899271,6cyclictest0-21swapper/121:25:131
21740992714,4cyclictest0-21swapper/023:20:180
2176499261,5cyclictest0-21swapper/322:52:573
2176499261,5cyclictest0-21swapper/320:40:123
2176499261,3cyclictest0-21swapper/323:15:173
2176499261,16cyclictest28366irq/58-eth020:35:133
2175699262,9cyclictest0-21swapper/219:15:142
2175699261,8cyclictest5019-21ssh00:25:292
2175699261,10cyclictest0-21swapper/220:10:142
2175699260,12cyclictest0-21swapper/221:17:592
2175699260,12cyclictest0-21swapper/200:06:522
2175699260,11cyclictest7501-21diskmemload22:21:492
21748992611,11cyclictest21-21ksoftirqd/119:15:121
2174099261,21cyclictest9766-21ssh00:30:140
2176499251,8cyclictest0-21swapper/322:25:013
2176499251,6cyclictest0-21swapper/323:50:223
2176499251,6cyclictest0-21swapper/300:30:233
2176499251,6cyclictest0-21swapper/300:05:013
2176499251,5cyclictest0-21swapper/322:45:203
2176499251,5cyclictest0-21swapper/322:10:203
2176499251,5cyclictest0-21swapper/321:59:403
2176499251,5cyclictest0-21swapper/300:13:283
2176499251,4cyclictest0-21swapper/300:23:413
2175699252,14cyclictest1503-21latency_hist22:55:022
2175699251,12cyclictest27255-21latency_hist00:40:022
2175699251,12cyclictest27255-21latency_hist00:40:012
2175699251,11cyclictest924-21snmpd23:02:542
2175699251,11cyclictest924-21snmpd23:02:532
2175699250,7cyclictest0-21swapper/223:45:272
2175699250,12cyclictest9419-21kernelversion19:30:202
21748992513,9cyclictest25508-21cron21:20:001
2174099256,12cyclictest18319-21ssh22:10:110
2174099255,12cyclictest21729-21cyclictest19:45:120
2174099252,15cyclictest17011-21ssh21:15:130
2174099251,18cyclictest22327-21ssh00:18:130
2176499243,3cyclictest0-21swapper/300:05:203
2176499241,7cyclictest0-21swapper/323:25:233
2176499241,6cyclictest0-21swapper/323:30:273
2176499241,6cyclictest0-21swapper/321:10:053
2176499241,5cyclictest0-21swapper/323:04:073
2176499241,5cyclictest0-21swapper/323:04:063
2176499241,5cyclictest0-21swapper/322:06:223
2176499241,5cyclictest0-21swapper/322:06:213
2176499241,5cyclictest0-21swapper/321:34:053
2176499241,5cyclictest0-21swapper/321:26:173
2176499241,5cyclictest0-21swapper/321:26:163
2176499241,4cyclictest0-21swapper/323:20:163
2176499241,4cyclictest0-21swapper/322:55:173
2176499241,4cyclictest0-21swapper/322:25:203
2176499241,4cyclictest0-21swapper/322:00:293
2176499241,4cyclictest0-21swapper/320:15:023
2176499241,3cyclictest0-21swapper/320:00:293
2176499241,3cyclictest0-21swapper/300:35:173
2176499241,3cyclictest0-21swapper/300:35:163
2175699242,16cyclictest20902-21ssh00:00:142
21756992414,5cyclictest0-21swapper/220:05:142
21756992410,12cyclictest0-21swapper/222:25:162
2175699240,16cyclictest12609-21ssh23:18:302
2175699240,11cyclictest7501-21diskmemload21:53:532
2175699240,10cyclictest23258-21ssh22:10:372
2174099248,7cyclictest28928-21cat23:10:170
2174099248,7cyclictest28928-21cat23:10:160
2174099244,12cyclictest12-21ksoftirqd/019:25:100
2174099243,17cyclictest151rcuc/000:00:130
2174099243,16cyclictest7997-21ssh21:10:140
2174099241,3cyclictest131rcu_preempt21:20:130
21740992413,3cyclictest0-21swapper/023:15:060
2176499231,9cyclictest0-21swapper/323:05:323
2176499231,6cyclictest0-21swapper/320:25:193
2176499231,5cyclictest0-21swapper/323:45:313
2176499231,5cyclictest0-21swapper/319:40:203
2176499231,5cyclictest0-21swapper/319:20:163
2176499231,4cyclictest0-21swapper/321:50:293
2176499231,4cyclictest0-21swapper/319:50:003
2176499231,4cyclictest0-21swapper/319:35:153
2175699238,12cyclictest0-21swapper/220:30:252
2175699231,7cyclictest0-21swapper/222:55:172
21756992315,5cyclictest6294-21ssh21:25:292
21756992315,5cyclictest6294-21ssh21:25:282
21756992315,4cyclictest0-21swapper/219:25:132
21756992313,5cyclictest0-21swapper/220:35:142
2175699230,11cyclictest15916-21ssh00:32:072
2175699230,10cyclictest0-21swapper/200:22:002
2174899232,15cyclictest5839-21ssh21:45:121
2174899231,18cyclictest20875-21apt-get00:00:121
2174899231,16cyclictest28630-21ssh23:10:131
2174899231,16cyclictest28630-21ssh23:10:121
2174899231,13cyclictest6441-21apt-key20:35:021
2174099238,4cyclictest131rcu_preempt20:00:290
2174099238,10cyclictest9315-21ssh22:05:130
2174099238,10cyclictest9315-21ssh22:05:120
2174099235,10cyclictest0-21swapper/020:30:130
2176499221,6cyclictest0-21swapper/321:05:203
2176499221,6cyclictest0-21swapper/320:50:173
2176499221,6cyclictest0-21swapper/319:25:143
2176499221,6cyclictest0-21swapper/319:10:153
2176499221,5cyclictest0-21swapper/321:00:263
2176499221,5cyclictest0-21swapper/320:30:193
2176499221,5cyclictest0-21swapper/320:15:203
2176499221,5cyclictest0-21swapper/320:05:193
2176499221,5cyclictest0-21swapper/319:50:173
2176499221,5cyclictest0-21swapper/319:15:213
2176499221,4cyclictest0-21swapper/320:20:153
21764992214,5cyclictest28366irq/58-eth021:15:133
2175699221,17cyclictest5773-21irqrtprio20:00:192
2175699221,12cyclictest27917-21grep19:50:012
2175699220,15cyclictest0-21swapper/222:05:412
2175699220,15cyclictest0-21swapper/222:05:412
2174899227,3cyclictest131rcu_preempt21:30:261
2174899226,7cyclictest131rcu_preempt19:10:131
2174899223,12cyclictest0-21swapper/123:00:131
2174899223,12cyclictest0-21swapper/123:00:121
21748992212,7cyclictest8746-21apt-get23:35:131
2174099223,16cyclictest19732-21ssh23:05:150
2174099222,5cyclictest131rcu_preempt20:45:140
2174099221,12cyclictest1734-21ssh22:55:120
21740992210,6cyclictest7501-21diskmemload23:50:170
2176499211,5cyclictest0-21swapper/320:55:153
2176499211,5cyclictest0-21swapper/319:55:163
2175699212,9cyclictest21581-21kthreadcore20:15:192
2175699211,11cyclictest1283-21kthreadcore19:55:192
2175699210,7cyclictest24459-21ps20:50:272
2174899218,10cyclictest29075-21cron20:25:011
2174899211,9cyclictest24564-21users23:05:341
21748992114,5cyclictest0-21swapper/121:20:151
21748992111,7cyclictest0-21swapper/121:10:021
2174899211,14cyclictest14946-21cpuspeed21:50:151
21748992110,8cyclictest13345-21cron19:35:001
2174099214,11cyclictest12816-21ssh22:25:130
2174099212,13cyclictest12-21ksoftirqd/023:45:130
2174099211,16cyclictest13581-21apt-get19:35:110
2176499201,5cyclictest0-21swapper/319:10:023
2175699201,15cyclictest3855-21latency_hist19:25:012
2175699201,10cyclictest13131-21kthreadcore20:40:182
21748992012,5cyclictest0-21swapper/120:05:291
2174899201,14cyclictest14305-21ssh23:20:141
21748992010,7cyclictest20264-21sed22:45:311
2174099201,14cyclictest12000-21ssh23:55:140
2174099201,11cyclictest28647-21kernelversion19:50:180
2174099200,3cyclictest131rcu_preempt23:26:210
2175699191,15cyclictest21843-21latency_hist19:10:022
2174899197,8cyclictest21-21ksoftirqd/120:55:131
2174899196,9cyclictest1-21systemd20:42:381
2174899194,3cyclictest131rcu_preempt21:45:021
2174899194,3cyclictest131rcu_preempt20:00:021
2174899192,7cyclictest21-21ksoftirqd/122:20:211
2174899192,15cyclictest10429-21unixbench_singl20:35:281
2174899191,8cyclictest24974-21kthreadcore22:30:221
21748991914,3cyclictest19068-21tail22:10:191
2174899191,12cyclictest27993-21find19:50:111
2174099197,9cyclictest30352-21cut21:00:010
2174099195,10cyclictest0-21swapper/019:30:140
2174099192,13cyclictest2919-21apt-get21:05:130
2174099192,13cyclictest0-21swapper/021:30:140
2174099191,9cyclictest342-21acpi19:55:110
21740991913,4cyclictest0-21swapper/022:35:190
21740991913,4cyclictest0-21swapper/021:47:390
2174099191,15cyclictest23363-21ls20:15:270
2174899189,6cyclictest23989-21ntpq21:35:251
2174899188,7cyclictest7131-21cut00:10:191
2174899188,2cyclictest0-21swapper/123:49:021
2174899185,4cyclictest4492-21ntp_states00:25:261
2174899183,3cyclictest0-21swapper/119:25:131
2174899183,10cyclictest0-21swapper/122:25:131
2174899183,10cyclictest0-21swapper/121:00:151
2174899182,11cyclictest0-21swapper/120:00:041
2174899181,9cyclictest0-21swapper/123:53:411
2174899181,6cyclictest5868-21irqrtprio23:15:171
21748991811,5cyclictest27417-21usage_Anybus23:25:291
2174899181,15cyclictest0-21swapper/100:30:141
2174899181,14cyclictest20044-21kthreadcore19:40:191
2174899181,14cyclictest16147-21/usr/sbin/munin20:45:291
2174899181,13cyclictest7369-21df_inode22:40:141
2174899181,13cyclictest23199-21apt-get19:45:121
2174099189,6cyclictest17703-21sh23:40:160
2174099185,10cyclictest23070-21latency_hist19:45:020
2174099181,4cyclictest131rcu_preempt20:05:200
2174899178,7cyclictest21043-21entropy20:50:161
2174899175,3cyclictest131rcu_preempt23:40:291
2174899171,7cyclictest1427-21sh20:30:011
2174899171,12cyclictest30992-21ssh22:15:271
2174899171,12cyclictest23829-21apt-get00:20:041
2174899171,12cyclictest11712-21ntp_states21:10:281
21748991710,5cyclictest10076-21kthreadcore22:05:221
21748991710,5cyclictest10076-21kthreadcore22:05:211
2174099177,7cyclictest23902-21cpuspeed21:55:140
2174099173,9cyclictest0-21swapper/020:50:140
2174099171,3cyclictest131rcu_preempt19:15:140
21740991711,4cyclictest0-21swapper/019:10:150
2174099171,13cyclictest30817-21idleruntime21:00:180
2174099171,12cyclictest19416-21apt-get20:15:010
2174899167,6cyclictest21812-21latency_hist19:10:011
2174899162,10cyclictest0-21swapper/120:10:141
2174099161,11cyclictest31659-21sort19:20:150
21740991610,4cyclictest7073-21irqrtprio20:35:180
2174899158,5cyclictest13780-21tune2fs19:35:161
2174099151,11cyclictest21766-21latency_hist19:10:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional