You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-09 - 08:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Thu Apr 09, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2390725343,0sleep30-21swapper/318:57:463
2379723723,0sleep00-21swapper/018:56:310
2377823329,0sleep10-21swapper/118:56:161
2377323026,0sleep20-21swapper/218:56:132
240869976,0cyclictest5014-21munin-run22:49:001
339960,0migration/35979-21sh21:08:353
339960,0migration/34816-21sh22:27:303
339960,0migration/329759-21sh22:03:253
339960,0migration/328264-21sh22:19:523
339960,0migration/320017-21sh22:35:133
339960,0migration/313175-21munin-node23:14:113
339960,0migration/311778-21sh22:50:263
29725060,0irq/27-eth0-tx-12848-21munin-node19:39:200
29715060,0irq/26-eth0-rx-31449-21grep21:24:113
29715060,0irq/26-eth0-rx-27194-21sh23:19:453
29715060,0irq/26-eth0-rx-21945-21grep20:34:123
29715060,0irq/26-eth0-rx-20782-21cpuspeed_turbos23:39:113
259960,0migration/212916-21switchtime19:59:222
240879960,0cyclictest22240-21diskmemload22:56:482
240879960,0cyclictest0-21swapper/223:13:122
240869960,0cyclictest0-21swapper/121:59:471
149960,0migration/029102-21sh21:40:410
149960,0migration/028454-21sh23:02:280
149960,0migration/013973-21kthreadcore23:54:160
8215050,0irq/24-0000:00:5914-21latency_hist21:49:002
8215050,0irq/24-0000:00:30668-21cat21:24:002
8215050,0irq/24-0000:00:22229-21crond20:59:002
8215050,0irq/24-0000:00:22003-21turbostat21:59:002
4150,0ktimersoftd/00-21swapper/022:18:200
339950,0migration/39091-21munin-node00:09:173
339950,0migration/36343-21aten_r1power_vo21:49:113
339950,0migration/36041-21idleruntime19:09:153
339950,0migration/34953-21munin-node23:29:113
339950,0migration/331510-21kthreadcore00:24:173
339950,0migration/331200-21cpuspeed_turbos19:04:123
339950,0migration/330548-21taskset19:02:313
339950,0migration/328940-21munin-node23:24:103
339950,0migration/326794-21munin-node19:49:133
339950,0migration/323786-21munin-node21:19:153
339950,0migration/321336-21munin-node20:54:193
339950,0migration/32095-21kthreadcore23:04:163
339950,0migration/32075-21taskset20:43:183
339950,0migration/317271-21kthreadcore20:29:173
339950,0migration/315710-21kthreadcore21:54:163
339950,0migration/315336-21kthreadcore00:14:183
339950,0migration/314486-21taskset20:27:493
339950,0migration/313388-21munin-node19:39:123
339950,0migration/312499-21df_abs23:54:123
339950,0migration/312097-21taskset22:30:253
29725050,0irq/27-eth0-tx-5344-21ssh21:26:362
29725050,0irq/27-eth0-tx-3940-21ssh22:45:572
29725050,0irq/27-eth0-tx-29030-21ssh21:00:382
29725050,0irq/27-eth0-tx-21585-21munin-node22:39:222
29725050,0irq/27-eth0-tx-1804-21ssh00:25:342
29725050,0irq/27-eth0-tx-0-21swapper/222:52:572
29725050,0irq/27-eth0-tx-0-21swapper/221:38:512
29725050,0irq/27-eth0-tx-0-21swapper/200:05:122
29725050,0irq/27-eth0-tx-0-21swapper/020:51:490
29725050,0irq/27-eth0-tx-0-21swapper/020:07:480
29725050,0irq/27-eth0-tx-0-21swapper/019:44:120
29725050,0irq/27-eth0-tx-0-21swapper/019:22:400
29715050,0irq/26-eth0-rx-6718-21if_err_eth022:09:133
29715050,0irq/26-eth0-rx-6559-21runrttasks23:12:013
29715050,0irq/26-eth0-rx-31317-21hwlatdetect21:44:143
29715050,0irq/26-eth0-rx-31171-21kthreadcore00:04:163
29715050,0irq/26-eth0-rx-30374-21munin-node22:04:113
29715050,0irq/26-eth0-rx-20861-21grep23:59:123
29715050,0irq/26-eth0-rx-17092-21kthreadcore22:14:203
29715050,0irq/26-eth0-rx-14612-21kernelversion22:54:163
29715050,0irq/26-eth0-rx-0-21swapper/323:49:373
29715050,0irq/26-eth0-rx-0-21swapper/323:36:413
29715050,0irq/26-eth0-rx-0-21swapper/323:01:273
29715050,0irq/26-eth0-rx-0-21swapper/322:44:223
29715050,0irq/26-eth0-rx-0-21swapper/322:39:193
29715050,0irq/26-eth0-rx-0-21swapper/321:41:323
29715050,0irq/26-eth0-rx-0-21swapper/321:34:223
29715050,0irq/26-eth0-rx-0-21swapper/321:09:193
29715050,0irq/26-eth0-rx-0-21swapper/321:02:543
29715050,0irq/26-eth0-rx-0-21swapper/320:44:173
29715050,0irq/26-eth0-rx-0-21swapper/320:14:153
27150,0ktimersoftd/20-21swapper/222:18:202
259950,0migration/22722-21sh23:45:572
259950,0migration/225903-21kthreadcore21:59:182
259950,0migration/222113-21aten_r1power_vo22:19:122
259950,0migration/221535-21kthreadcore19:44:172
240879954,0cyclictest3241-21irqbalance19:00:292
240879950,0cyclictest0-21swapper/222:36:562
240869955,0cyclictest22240-21diskmemload21:44:491
240869955,0cyclictest0-21swapper/123:31:231
240869955,0cyclictest0-21swapper/123:02:131
240869955,0cyclictest0-21swapper/120:49:191
240869950,0cyclictest0-21swapper/123:48:061
240869950,0cyclictest0-21swapper/122:09:341
240869950,0cyclictest0-21swapper/121:03:491
240869950,0cyclictest0-21swapper/120:39:141
19150,0ktimersoftd/10-21swapper/122:18:201
179950,0migration/18370-21kthreadcore21:29:161
179950,0migration/14891-21munin-node19:29:181
179950,0migration/132456-21kthreadcore23:24:161
179950,0migration/130638-21munin-node22:04:121
179950,0migration/118274-21sh23:54:231
179950,0migration/115263-21kthreadcore22:54:161
179950,0migration/113291-21sh21:11:001
179950,0migration/112497-21hwlatdetect00:14:141
169950,0watchdog/11353-21kthreadcore21:04:191
149950,0migration/08811-21kthreadcore22:09:160
149950,0migration/06228-21munin-node19:09:160
149950,0migration/05668-21gltestperf23:09:140
149950,0migration/04923-21netstat19:29:180
149950,0migration/04229-21aten_r1power_vo23:49:110
149950,0migration/031821-21munin-node21:44:160
149950,0migration/031725-21kthreadcore23:44:160
149950,0migration/031289-21kthreadcore20:39:170
149950,0migration/030374-21kthreadcore19:24:190
149950,0migration/028418-21taskset23:23:260
149950,0migration/028104-21kthreadcore19:49:160
149950,0migration/027856-21kthreadcore18:59:170
149950,0migration/023491-21kthreadcore00:19:180
149950,0migration/021054-21fschecks_time23:39:120
149950,0migration/020626-21taskset23:18:540
149950,0migration/01918-21kthreadcore21:24:150
149950,0migration/018806-21munin-node22:54:190
149950,0migration/015943-21kthreadcore21:54:160
149950,0migration/015370-21fschecks_time21:14:130
149950,0migration/012483-21sh22:31:330
149950,0migration/012401-21kthreadcore19:59:180
149950,0migration/01225-21idleruntime19:54:150
149950,0migration/01187-21kthreadcore23:24:170
149950,0migration/010586-21kthreadcore20:24:170
149950,0migration/01005-21kthreadcore22:04:170
339940,0migration/32016-21kthreadcore19:29:163
339940,0migration/318695-21kthreadcore20:04:173
339940,0migration/313380-21taskset20:01:243
29725040,0irq/27-eth0-tx-5852-21munin-node22:09:212
29725040,0irq/27-eth0-tx-21602-21telnet20:34:110
29725040,0irq/27-eth0-tx-12075-21tail19:14:130
29725040,0irq/27-eth0-tx-0-21swapper/223:49:142
29725040,0irq/27-eth0-tx-0-21swapper/223:24:042
29725040,0irq/27-eth0-tx-0-21swapper/222:24:092
29725040,0irq/27-eth0-tx-0-21swapper/222:04:242
29725040,0irq/27-eth0-tx-0-21swapper/221:49:112
29725040,0irq/27-eth0-tx-0-21swapper/221:39:092
29725040,0irq/27-eth0-tx-0-21swapper/221:29:032
29725040,0irq/27-eth0-tx-0-21swapper/200:14:132
29725040,0irq/27-eth0-tx-0-21swapper/020:59:170
29725040,0irq/27-eth0-tx-0-21swapper/019:04:130
29715040,0irq/26-eth0-rx-0-21swapper/323:44:563
29715040,0irq/26-eth0-rx-0-21swapper/321:29:303
29715040,0irq/26-eth0-rx-0-21swapper/321:14:193
29715040,0irq/26-eth0-rx-0-21swapper/319:54:183
29715040,0irq/26-eth0-rx-0-21swapper/319:19:203
29715040,0irq/26-eth0-rx-0-21swapper/300:19:043
259940,0migration/29332-21kthreadcore22:29:162
259940,0migration/29238-21aten_r1power_po20:49:122
259940,0migration/25691-21kthreadcore19:54:182
259940,0migration/24595-21kthreadcore19:29:172
259940,0migration/23726-21kthreadcore20:19:162
259940,0migration/2344-21kthreadcore19:04:172
259940,0migration/230873-21kthreadcore23:04:152
259940,0migration/230117-21kthreadcore19:24:182
259940,0migration/223723-21kthreadcore19:19:172
259940,0migration/223305-21kthreadcore23:19:162
259940,0migration/220915-21munin-node23:59:122
259940,0migration/21984-21runrttasks20:42:142
259940,0migration/219568-21latency_hist00:19:002
259940,0migration/214547-21munin-node20:04:142
259940,0migration/214324-21kthreadcore23:14:162
259940,0migration/214247-21kthreadcore23:34:172
259940,0migration/210020-21kthreadcore21:09:182
240879943,0cyclictest6062-21awk19:09:152
240879943,0cyclictest4601-21awk00:09:142
240879943,0cyclictest3556-21lldpd22:59:412
240879943,0cyclictest3556-21lldpd19:36:392
240879943,0cyclictest3244-21awk20:44:152
240879943,0cyclictest3241-21irqbalance19:14:592
240879943,0cyclictest28026-21awk20:14:162
240879943,0cyclictest22316-21awk20:34:142
240879943,0cyclictest15704-21awk21:14:152
240879943,0cyclictest13028-21awk23:54:142
240869944,0cyclictest0-21swapper/123:42:161
240869944,0cyclictest0-21swapper/123:09:381
240869944,0cyclictest0-21swapper/100:23:531
240869943,0cyclictest3514-21snmpd20:55:181
240869943,0cyclictest3514-21snmpd20:30:181
240869943,0cyclictest3514-21snmpd20:07:181
240869943,0cyclictest3514-21snmpd19:50:181
240869943,0cyclictest31827-21awk19:04:151
240869943,0cyclictest30839-21awk22:24:151
240869943,0cyclictest30267-21awk23:04:141
240869943,0cyclictest28946-21awk00:04:131
240869943,0cyclictest28138-21awk20:14:171
240869943,0cyclictest2265-21awk20:19:141
240869943,0cyclictest20370-21awk19:44:151
240869943,0cyclictest15471-21awk21:34:161
240869943,0cyclictest13527-21awk23:34:161
240859943,0cyclictest4530-21awk00:09:130
240859943,0cyclictest3556-21lldpd22:20:110
240859943,0cyclictest3556-21lldpd21:59:410
240859943,0cyclictest3556-21lldpd21:10:110
240859943,0cyclictest3514-21snmpd20:10:480
240859943,0cyclictest31512-21awk21:04:160
240859943,0cyclictest30120-21awk23:04:130
240859943,0cyclictest22590-21awk22:39:140
240859943,0cyclictest2250-21awk20:19:140
240859943,0cyclictest15364-21awk21:34:150
249940,0watchdog/245342chrt23:29:062
249940,0watchdog/230642-21munin-node21:04:192
249940,0watchdog/212848-21munin-node19:39:112
249940,0watchdog/20-21swapper/223:39:102
179940,0migration/15464-21cpuspeed_turbos19:09:121
179940,0migration/14128-21kthreadcore19:54:181
179940,0migration/131700-21kthreadcore00:24:171
179940,0migration/125190-21latency_hist19:24:011
179940,0migration/12510-21aten2.4-expect20:44:121
179940,0migration/124293-21taskset18:59:091
179940,0migration/124078-21kthreadcore23:59:161
179940,0migration/122850-21kthreadcore20:09:171
179940,0migration/122833-21kthreadcore23:19:161
179940,0migration/12012-21kthreadcore21:24:151
179940,0migration/118691-21kthreadcore22:34:231
179940,0migration/115297-21fschecks_time21:14:131
179940,0migration/113628-21munin-node20:24:191
179940,0migration/113052-21irqcore19:14:171
179940,0migration/112426-21sh21:49:331
179940,0migration/112289-21runrttasks22:51:541
169940,0watchdog/13709-21aten_r1power_en00:09:101
169940,0watchdog/129984-21ssh21:22:191
169940,0watchdog/127948-21missed_timers21:39:191
169940,0watchdog/127185-21kthreadcore22:19:181
169940,0watchdog/113011-21cpu19:39:111
169940,0watchdog/110919-21ssh23:51:101
169940,0watchdog/10-21swapper/119:19:151
149940,0migration/012774-21runrttasks21:29:450
339930,0migration/331266-21runrttasks19:24:423
339930,0migration/325828-21taskset19:47:593
339930,0migration/316648-21kthreadcore19:14:193
29715030,0irq/26-eth0-rx-0-21swapper/320:51:093
29715030,0irq/26-eth0-rx-0-21swapper/320:19:143
29715030,0irq/26-eth0-rx-0-21swapper/320:09:113
29715030,0irq/26-eth0-rx-0-21swapper/319:34:113
240879932,0cyclictest9079-21awk20:24:152
240879932,0cyclictest15715-21awk20:29:152
240879930,0cyclictest20273-21latency_hist20:09:012
240869932,0cyclictest3514-21snmpd20:37:541
240869932,0cyclictest3514-21snmpd19:38:031
240869930,0cyclictest0-21swapper/120:01:041
240859933,0cyclictest7928-21kworker/0:020:29:100
240859933,0cyclictest7303-21kworker/0:120:14:080
240859933,0cyclictest7289-21kworker/0:023:59:470
240859933,0cyclictest7289-21kworker/0:023:34:590
240859933,0cyclictest7289-21kworker/0:023:29:110
240859933,0cyclictest7289-21kworker/0:022:49:020
240859933,0cyclictest7289-21kworker/0:022:34:120
240859933,0cyclictest7289-21kworker/0:022:24:320
240859933,0cyclictest7289-21kworker/0:000:24:050
240859933,0cyclictest7289-21kworker/0:000:14:050
240859933,0cyclictest22254-21kworker/0:221:19:060
240859933,0cyclictest22254-21kworker/0:220:54:500
240859933,0cyclictest22254-21kworker/0:220:44:060
240859932,0cyclictest7459-21irqstats19:34:170
240859932,0cyclictest7111-21awk21:49:150
240859932,0cyclictest29027-21awk00:04:140
240859931,0cyclictest29652-21munin-node22:44:250
249930,0watchdog/20-21swapper/219:49:112
179930,0migration/114940-21kthreadcore23:14:161
169930,0watchdog/122240-21diskmemload22:29:101
169930,0watchdog/10-21swapper/122:39:101
169930,0watchdog/10-21swapper/121:54:231
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional