You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-23 - 11:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Mon Mar 23, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
211826968,0sleep20-21swapper/218:59:282
216123626,0sleep30-21swapper/318:59:573
191123624,0sleep10-21swapper/118:57:021
203523430,0sleep00-21swapper/018:58:300
339960,0migration/36906-21sh23:04:113
339960,0migration/36580-21kthreadcore21:40:543
339960,0migration/325723-21munin-node20:05:533
29715060,0irq/26-eth0-rx-17100-21cpu22:10:483
29715060,0irq/26-eth0-rx-13590-21diskstats20:45:493
22919960,0cyclictest18387-21cut23:10:522
22909965,0cyclictest25044-21cut22:55:521
22909960,0cyclictest474-21diskmemload21:58:301
8215050,0irq/24-0000:00:9025-21latency_hist21:45:362
4150,0ktimersoftd/00-21swapper/022:40:300
339950,0migration/39263-21idleruntime22:25:493
339950,0migration/37989-21munin-node23:05:463
339950,0migration/3543-21latency_hist21:00:363
339950,0migration/33664-21irqcore19:00:553
339950,0migration/332038-21idleruntime20:10:503
339950,0migration/331934-21kthreadcore19:45:533
339950,0migration/325749-21kthreadcore19:15:543
339950,0migration/325445-21fschecks_time22:15:503
339950,0migration/324531-21aten_r1power_vo22:35:483
339950,0migration/322570-21kthreadcore20:00:543
339950,0migration/321790-21timerwakeupswit19:11:013
339950,0migration/319219-21kthreadcore22:30:543
339950,0migration/318776-21kthreadcore19:35:533
339950,0migration/318360-21taskset20:22:353
339950,0migration/316978-21switchtime19:56:003
339950,0migration/316553-21sh21:28:353
339950,0migration/315922-21latency_hist22:50:363
339950,0migration/315601-21kthreadcore00:10:523
339950,0migration/314896-21taskset22:48:063
339950,0migration/311599-21kthreadcore21:05:533
29725050,0irq/27-eth0-tx-9256-21munin-node19:05:530
29725050,0irq/27-eth0-tx-7461-21ssh21:41:062
29725050,0irq/27-eth0-tx-6788-21sed22:00:592
29725050,0irq/27-eth0-tx-17173-21munin-node21:10:552
29725050,0irq/27-eth0-tx-16266-21ssh21:09:002
29725050,0irq/27-eth0-tx-11235-21munin-node19:55:490
29725050,0irq/27-eth0-tx-11074-21ssh21:25:512
29725050,0irq/27-eth0-tx-0-21swapper/222:43:202
29725050,0irq/27-eth0-tx-0-21swapper/222:28:502
29725050,0irq/27-eth0-tx-0-21swapper/221:02:322
29725050,0irq/27-eth0-tx-0-21swapper/200:17:352
29725050,0irq/27-eth0-tx-0-21swapper/021:00:470
29725050,0irq/27-eth0-tx-0-21swapper/020:35:260
29725050,0irq/27-eth0-tx-0-21swapper/019:22:560
29715050,0irq/26-eth0-rx-8118-21ssh21:22:323
29715050,0irq/26-eth0-rx-5830-21cut22:00:543
29715050,0irq/26-eth0-rx-5292-21ssh23:42:023
29715050,0irq/26-eth0-rx-27986-21sensors00:15:593
29715050,0irq/26-eth0-rx-17702-21diskstats21:50:493
29715050,0irq/26-eth0-rx-11936-21tr23:25:533
29715050,0irq/26-eth0-rx-0-21swapper/323:51:213
29715050,0irq/26-eth0-rx-0-21swapper/323:45:533
29715050,0irq/26-eth0-rx-0-21swapper/323:36:573
29715050,0irq/26-eth0-rx-0-21swapper/323:10:523
29715050,0irq/26-eth0-rx-0-21swapper/322:56:063
29715050,0irq/26-eth0-rx-0-21swapper/322:05:393
29715050,0irq/26-eth0-rx-0-21swapper/321:48:333
29715050,0irq/26-eth0-rx-0-21swapper/321:35:463
29715050,0irq/26-eth0-rx-0-21swapper/321:15:483
29715050,0irq/26-eth0-rx-0-21swapper/321:00:383
29715050,0irq/26-eth0-rx-0-21swapper/319:20:543
29715050,0irq/26-eth0-rx-0-21swapper/300:28:483
29715050,0irq/26-eth0-rx-0-21swapper/300:07:493
29715050,0irq/26-eth0-rx-0-21swapper/300:01:003
29715050,0irq/26-eth0-rx-0-21swapper/300:00:203
259950,0migration/28416-21gltestperf23:05:472
259950,0migration/27257-21sh23:25:462
259950,0migration/231976-21aten_r1power_cu23:00:452
259950,0migration/230230-21mdstat22:15:552
259950,0migration/229811-21sh23:16:042
259950,0migration/22453-21irqcore21:20:532
259950,0migration/223212-21aten_r1power_cu23:35:472
259950,0migration/222871-21sh22:31:572
259950,0migration/221249-21sh23:52:162
259950,0migration/220769-21munin-node23:30:552
259950,0migration/216672-21ntp_kernel_pll_19:55:572
259950,0migration/216145-21sh22:09:172
249950,0watchdog/20-21swapper/223:20:542
22919955,0cyclictest8927-21munin-node22:45:482
22919955,0cyclictest0-21swapper/200:20:522
22919954,0cyclictest1403-21ps22:20:512
22909955,0cyclictest0-21swapper/123:47:361
22909955,0cyclictest0-21swapper/121:33:351
22909955,0cyclictest0-21swapper/100:16:461
22909954,0cyclictest24248-21ps23:35:531
22909950,0cyclictest0-21swapper/123:53:261
22909950,0cyclictest0-21swapper/123:13:041
22909950,0cyclictest0-21swapper/121:42:341
22899955,0cyclictest0-21swapper/023:55:520
22899955,0cyclictest0-21swapper/022:15:360
22899955,0cyclictest0-21swapper/022:06:180
22899955,0cyclictest0-21swapper/000:23:560
22899955,0cyclictest0-21swapper/000:15:340
22899954,0cyclictest14426-21idleruntime-cro23:50:360
22899950,0cyclictest0-21swapper/021:25:280
19150,0ktimersoftd/10-21swapper/121:04:061
179950,0migration/1767-21kthreadcore20:10:531
179950,0migration/16471-21latency_hist20:40:361
179950,0migration/127022-21kthreadcore19:40:541
179950,0migration/124785-21taskset22:15:411
179950,0migration/123912-21cpuspeed_turbos23:15:491
179950,0migration/121959-21munin-node22:30:561
179950,0migration/120946-21kthreadcore21:50:541
179950,0migration/116587-21fschecks_time22:50:491
179950,0migration/11548-21irqcore22:20:511
179950,0migration/113589-21munin-node20:45:491
179950,0migration/111387-21kthreadcore23:05:511
179950,0migration/110067-21kthreadcore22:25:511
169950,0watchdog/122633-21python22:10:581
169950,0watchdog/120652-21kthreadcore21:10:541
149950,0migration/09849-21kthreadcore20:40:540
149950,0migration/09069-21kthreadcore00:25:550
149950,0migration/07119-21idleruntime00:05:490
149950,0migration/06885-21taskset22:42:540
149950,0migration/06671-21mdstat21:40:550
149950,0migration/06258-21kthreadcore19:50:530
149950,0migration/05390-21kthreadcore22:00:540
149950,0migration/04320-21gltestperf19:25:500
149950,0migration/03522-21taskset19:45:580
149950,0migration/03219-21kthreadcore23:00:520
149950,0migration/030116-21sh23:38:260
149950,0migration/028903-21taskset19:19:000
149950,0migration/027543-21latency00:15:550
149950,0migration/027313-21hwlatdetect20:55:520
149950,0migration/027184-21kthreadcore21:55:530
149950,0migration/026231-21kthreadcore19:40:540
149950,0migration/025438-21latency_hist20:30:360
149950,0migration/023423-21pmu-power20:00:560
149950,0migration/021923-21taskset23:11:220
149950,0migration/021453-21munin-node19:10:580
149950,0migration/019574-21kthreadcore21:30:530
149950,0migration/019389-21kthreadcore21:10:540
149950,0migration/019082-21kthreadcore21:50:530
149950,0migration/018800-21kthreadcore22:50:530
149950,0migration/017667-21gltestperf19:35:500
149950,0migration/014741-21kthreadcore20:45:520
149950,0migration/012628-21missed_timers23:25:540
149950,0migration/011850-21kthreadcore22:25:520
149950,0migration/011790-21kthreadcore22:45:510
339940,0migration/38557-21kthreadcore19:25:543
339940,0migration/329338-21timerandwakeup19:40:593
339940,0migration/324913-21taskset20:26:083
339940,0migration/321452-21runrttasks23:31:143
29725040,0irq/27-eth0-tx-4786-21ssh00:02:002
29725040,0irq/27-eth0-tx-0-21swapper/223:55:452
29725040,0irq/27-eth0-tx-0-21swapper/223:43:062
29725040,0irq/27-eth0-tx-0-21swapper/222:35:372
29725040,0irq/27-eth0-tx-0-21swapper/221:35:372
29725040,0irq/27-eth0-tx-0-21swapper/221:34:412
29725040,0irq/27-eth0-tx-0-21swapper/221:15:412
29725040,0irq/27-eth0-tx-0-21swapper/200:10:552
29725040,0irq/27-eth0-tx-0-21swapper/200:05:402
29725040,0irq/27-eth0-tx-0-21swapper/020:50:480
29725040,0irq/27-eth0-tx-0-21swapper/019:30:560
29715040,0irq/26-eth0-rx-0-21swapper/323:20:573
29715040,0irq/26-eth0-rx-0-21swapper/323:19:343
29715040,0irq/26-eth0-rx-0-21swapper/322:40:513
29715040,0irq/26-eth0-rx-0-21swapper/321:55:533
29715040,0irq/26-eth0-rx-0-21swapper/321:30:543
29715040,0irq/26-eth0-rx-0-21swapper/321:10:373
29715040,0irq/26-eth0-rx-0-21swapper/300:20:543
259940,0migration/2815-21kthreadcore19:45:532
259940,0migration/2325-21fschecks_time20:35:502
259940,0migration/231785-21kthreadcore19:20:532
259940,0migration/229723-21runrttasks19:42:242
259940,0migration/228816-21kthreadcore20:30:542
259940,0migration/227984-21kthreadcore21:55:532
259940,0migration/221257-21kthreadcore19:35:542
259940,0migration/219714-21kthreadcore21:50:532
259940,0migration/219401-21fschecks_count20:25:492
259940,0migration/217940-21kthreadcore22:50:532
259940,0migration/216919-21aten_r1power_en22:10:472
259940,0migration/214697-21kthreadcore20:45:522
259940,0migration/213414-21kthreadcore19:05:542
249940,0watchdog/228300-21python19:15:592
249940,0watchdog/212617-21awk20:41:432
249940,0watchdog/20-21swapper/223:45:502
22919944,0cyclictest0-21swapper/219:50:482
22919943,0cyclictest4573-21awk19:25:522
22919943,0cyclictest3556-21lldpd20:22:572
22919943,0cyclictest3514-21snmpd20:51:272
22919943,0cyclictest3514-21snmpd20:15:422
22919943,0cyclictest3514-21snmpd19:35:122
22919943,0cyclictest3514-21snmpd19:13:272
22919943,0cyclictest3514-21snmpd00:25:432
22919943,0cyclictest3407-21awk19:00:532
22919943,0cyclictest27500-21awk20:55:532
22919943,0cyclictest25417-21awk20:05:512
22919943,0cyclictest18726-21awk20:00:512
22919941,0cyclictest0-21swapper/222:55:482
22909944,0cyclictest0-21swapper/120:06:431
22909944,0cyclictest0-21swapper/100:25:451
22909943,0cyclictest9845-21awk21:05:501
22909943,0cyclictest8101-21awk23:25:501
22909943,0cyclictest687-21awk20:35:511
22909943,0cyclictest3556-21lldpd22:44:271
22909943,0cyclictest3556-21lldpd00:13:571
22909943,0cyclictest3556-21lldpd00:05:571
22909943,0cyclictest3514-21snmpd20:32:121
22909943,0cyclictest3514-21snmpd19:50:421
22909943,0cyclictest3241-21irqbalance21:47:051
22909943,0cyclictest31222-21awk19:45:521
22899944,0cyclictest23387-21kworker/0:220:10:480
22899944,0cyclictest0-21swapper/023:18:580
22899944,0cyclictest0-21swapper/000:00:550
22899943,0cyclictest3556-21lldpd21:48:260
22899943,0cyclictest3556-21lldpd20:37:270
22899943,0cyclictest32507-21sadc22:20:360
22899943,0cyclictest25010-21awk22:55:520
22899941,0cyclictest0-21swapper/020:15:490
179940,0migration/17738-21kthreadcore19:25:541
179940,0migration/16939-21kthreadcore19:00:561
179940,0migration/14315-21unixbench-2d00:00:581
179940,0migration/125503-21irqcore22:35:521
179940,0migration/123785-21kthreadcore20:25:531
179940,0migration/122679-21runrttasks19:35:591
179940,0migration/117155-21irqrtprio19:10:541
179940,0migration/117054-21kthreadcore23:30:521
179940,0migration/116525-21runrttasks19:33:231
179940,0migration/116131-21kthreadcore20:20:541
179940,0migration/114095-21kthreadcore21:25:521
179940,0migration/112586-21kthreadcore19:05:541
179940,0migration/110405-21kthreadcore22:05:531
169940,0watchdog/16559-21runrttasks23:55:421
169940,0watchdog/16559-21runrttasks23:20:381
169940,0watchdog/131960-21ssh21:17:101
169940,0watchdog/130031-21kthreadcore20:55:551
169940,0watchdog/127876-21latency19:15:551
169940,0watchdog/10-21swapper/121:25:021
169940,0watchdog/10-21swapper/120:50:591
169940,0watchdog/10-21swapper/119:55:511
149940,0migration/04567-21kthreadcore19:00:550
149940,0migration/031506-21latency_hist23:20:360
149940,0migration/029780-21kthreadcore21:35:540
149940,0migration/027419-21kthreadcore20:05:540
149940,0migration/022389-21runrttasks23:34:010
149940,0migration/015210-21kthreadcore20:20:540
29715030,0irq/26-eth0-rx-0-21swapper/322:21:343
29715030,0irq/26-eth0-rx-0-21swapper/320:52:023
29715030,0irq/26-eth0-rx-0-21swapper/320:40:493
29715030,0irq/26-eth0-rx-0-21swapper/320:35:503
29715030,0irq/26-eth0-rx-0-21swapper/320:30:483
29715030,0irq/26-eth0-rx-0-21swapper/320:15:493
29715030,0irq/26-eth0-rx-0-21swapper/319:50:483
29715030,0irq/26-eth0-rx-0-21swapper/319:32:523
29715030,0irq/26-eth0-rx-0-21swapper/319:05:493
22919932,0cyclictest3514-21snmpd20:10:422
22909932,0cyclictest30201-21telnet00:20:471
22909932,0cyclictest2560-21sensors19:20:581
22909932,0cyclictest18796-21awk20:00:511
22909932,0cyclictest1802-21awk22:00:521
22899933,0cyclictest6247-21kworker/0:123:50:390
22899933,0cyclictest6247-21kworker/0:123:40:390
22899933,0cyclictest6247-21kworker/0:123:05:410
22899933,0cyclictest6247-21kworker/0:122:30:370
22899933,0cyclictest23387-21kworker/0:221:25:420
22899933,0cyclictest23387-21kworker/0:221:15:440
22899932,0cyclictest9687-21grep21:05:490
22899932,0cyclictest3514-21snmpd22:20:430
179930,0migration/12331-21kthreadcore23:00:511
169930,0watchdog/10-21swapper/123:40:581
169930,0watchdog/10-21swapper/122:45:501
169930,0watchdog/10-21swapper/121:36:301
169930,0watchdog/10-21swapper/120:15:551
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional