You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-22 - 15:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sun Mar 22, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2550927473,0sleep20-21swapper/206:56:562
304450410,0irq/25-eth10-21swapper/306:55:583
2574923925,0sleep00-21swapper/006:59:420
2577523627,0sleep10-21swapper/107:00:031
259039976,0cyclictest0-21swapper/210:15:562
259019976,0cyclictest0-21swapper/009:49:540
339960,0migration/332012-21sh09:44:573
339960,0migration/323955-21ksmtuned10:00:383
339960,0migration/321846-21sh10:57:023
339960,0migration/316998-21cpuspeed_turbos08:30:513
339960,0migration/315259-21sh09:13:153
339960,0migration/314994-21sh10:13:483
29715060,0irq/26-eth0-rx-15995-21tr08:50:563
29715060,0irq/26-eth0-rx-1458-21cat09:05:553
29715060,0irq/26-eth0-rx-13128-21python10:50:593
259029966,0cyclictest0-21swapper/110:21:501
259029960,0cyclictest0-21swapper/111:44:551
259029960,0cyclictest0-21swapper/111:35:371
259029960,0cyclictest0-21swapper/111:01:241
259019966,0cyclictest0-21swapper/012:02:200
339950,0migration/37900-21taskset11:10:533
339950,0migration/35732-21kthreadcore08:20:553
339950,0migration/332735-21kthreadcore11:25:553
339950,0migration/331245-21taskset10:03:333
339950,0migration/329002-21munin-node08:36:023
339950,0migration/32898-21latency08:40:583
339950,0migration/328822-21kthreadcore07:25:553
339950,0migration/327388-21kthreadcore09:20:553
339950,0migration/32712-21kthreadcore12:05:573
339950,0migration/326434-21kthreadcore12:00:573
339950,0migration/326265-21kthreadcore07:45:573
339950,0migration/324309-21munin-node10:20:513
339950,0migration/324208-21kthreadcore08:10:563
339950,0migration/324179-21latency_hist09:00:393
339950,0migration/322252-21sh11:39:383
339950,0migration/320658-21kthreadcore10:15:563
339950,0migration/318640-21kthreadcore11:55:563
339950,0migration/315795-21irqcore07:40:563
339950,0migration/314147-21latency_hist12:15:383
339950,0migration/312747-21rt-features07:10:593
339950,0migration/311182-21kthreadcore09:30:553
29725050,0irq/27-eth0-tx-4315-21uptime12:06:032
29725050,0irq/27-eth0-tx-32521-21munin-node09:45:562
29725050,0irq/27-eth0-tx-29669-21mdstat10:00:582
29725050,0irq/27-eth0-tx-23285-21ssh10:40:302
29725050,0irq/27-eth0-tx-10058-21munin-node08:26:030
29725050,0irq/27-eth0-tx-0-21swapper/009:00:500
29725050,0irq/27-eth0-tx-0-21swapper/007:41:000
29715050,0irq/26-eth0-rx-7660-21aten_r1power_fr10:30:503
29715050,0irq/26-eth0-rx-5291-21ssh11:06:203
29715050,0irq/26-eth0-rx-4415-21kthreadcore10:45:563
29715050,0irq/26-eth0-rx-4145-21ssh12:26:163
29715050,0irq/26-eth0-rx-31732-21ssh09:04:493
29715050,0irq/26-eth0-rx-29860-21sshd12:25:143
29715050,0irq/26-eth0-rx-29857-21sh08:15:533
29715050,0irq/26-eth0-rx-23181-21sh11:20:493
29715050,0irq/26-eth0-rx-23147-21cpuspeed_turbos11:40:513
29715050,0irq/26-eth0-rx-17685-21irqcore09:35:553
29715050,0irq/26-eth0-rx-13350-21cut09:50:553
29715050,0irq/26-eth0-rx-0-21swapper/311:50:553
29715050,0irq/26-eth0-rx-0-21swapper/311:45:483
29715050,0irq/26-eth0-rx-0-21swapper/311:32:073
29715050,0irq/26-eth0-rx-0-21swapper/311:00:573
29715050,0irq/26-eth0-rx-0-21swapper/310:35:413
29715050,0irq/26-eth0-rx-0-21swapper/310:29:293
29715050,0irq/26-eth0-rx-0-21swapper/309:47:503
29715050,0irq/26-eth0-rx-0-21swapper/309:30:233
29715050,0irq/26-eth0-rx-0-21swapper/308:25:523
27150,0ktimersoftd/20-21swapper/210:24:312
259039955,0cyclictest0-21swapper/211:00:522
259039950,0cyclictest0-21swapper/211:13:572
259029955,0cyclictest24138-21diskmemload12:09:021
259029955,0cyclictest0-21swapper/111:23:381
259029955,0cyclictest0-21swapper/109:41:091
259019955,0cyclictest24138-21diskmemload10:42:340
259019955,0cyclictest0-21swapper/009:57:570
259019950,0cyclictest0-21swapper/012:16:220
259019950,0cyclictest0-21swapper/011:07:470
259019950,0cyclictest0-21swapper/010:21:450
259019950,0cyclictest0-21swapper/009:38:240
259019950,0cyclictest0-21swapper/009:16:100
259019950,0cyclictest0-21swapper/007:06:060
259950,0migration/28018-21munin-node10:10:502
259950,0migration/26131-21latency_hist12:10:382
259950,0migration/24933-21pmu-power10:45:582
259950,0migration/23218-21latency_hist08:20:392
259950,0migration/22733-21kthreadcore07:05:572
259950,0migration/223794-21sh09:19:112
249950,0watchdog/20-21swapper/212:16:012
179950,0migration/18762-21idleruntime10:10:541
179950,0migration/126169-21kthreadcore08:35:581
179950,0migration/124469-21sh10:00:511
179950,0migration/121833-21munin-node07:45:521
179950,0migration/119561-21kthreadcore09:35:561
179950,0migration/110042-21kthreadcore09:10:541
149950,0migration/022251-21latency_hist08:10:400
149950,0migration/018689-21kthreadcore11:55:560
149950,0migration/015810-21kthreadcore07:15:580
149950,0migration/013484-21kthreadcore08:50:550
339940,0migration/38609-21kthreadcore08:45:563
339940,0migration/36347-21sh12:10:493
339940,0migration/33782-21kthreadcore07:05:583
339940,0migration/328638-21kthreadcore10:40:563
339940,0migration/323340-21kthreadcore07:20:563
339940,0migration/317899-21kthreadcore07:15:593
339940,0migration/316389-21diskmemload09:15:433
339940,0migration/31061-21irqrtprio10:05:553
29725040,0irq/27-eth0-tx-7332-21ssh10:50:462
29725040,0irq/27-eth0-tx-30870-21ssh11:45:442
29725040,0irq/27-eth0-tx-30620-21sed09:41:022
29725040,0irq/27-eth0-tx-28732-21ssh12:01:052
29725040,0irq/27-eth0-tx-2421-21telnet07:55:510
29725040,0irq/27-eth0-tx-23550-21kthreadcore12:20:552
29725040,0irq/27-eth0-tx-16551-21ssh09:35:502
29725040,0irq/27-eth0-tx-0-21swapper/211:55:532
29725040,0irq/27-eth0-tx-0-21swapper/211:52:042
29725040,0irq/27-eth0-tx-0-21swapper/211:40:432
29725040,0irq/27-eth0-tx-0-21swapper/211:30:492
29725040,0irq/27-eth0-tx-0-21swapper/211:20:512
29725040,0irq/27-eth0-tx-0-21swapper/211:15:392
29725040,0irq/27-eth0-tx-0-21swapper/211:05:412
29725040,0irq/27-eth0-tx-0-21swapper/210:05:492
29725040,0irq/27-eth0-tx-0-21swapper/209:20:472
29725040,0irq/27-eth0-tx-0-21swapper/008:22:230
29725040,0irq/27-eth0-tx-0-21swapper/008:00:500
29725040,0irq/27-eth0-tx-0-21swapper/007:35:530
29715040,0irq/26-eth0-rx-0-21swapper/311:16:303
29715040,0irq/26-eth0-rx-0-21swapper/308:05:543
259039944,0cyclictest0-21swapper/211:35:422
259039943,0cyclictest9254-21awk09:10:532
259039943,0cyclictest8488-21awk10:30:542
259039943,0cyclictest5283-21awk08:45:542
259039943,0cyclictest3556-21lldpd11:26:232
259039943,0cyclictest3556-21lldpd09:50:482
259039943,0cyclictest3556-21lldpd09:30:492
259039943,0cyclictest3556-21lldpd09:27:192
259039943,0cyclictest3556-21lldpd08:20:492
259039943,0cyclictest3556-21lldpd07:56:192
259039943,0cyclictest3556-21lldpd07:50:502
259039943,0cyclictest3556-21lldpd07:35:502
259039943,0cyclictest3556-21lldpd07:16:192
259039943,0cyclictest3556-21lldpd07:15:192
259039943,0cyclictest31223-21awk12:25:542
259039943,0cyclictest24139-21sadc09:00:392
259039943,0cyclictest17089-21awk09:55:532
259039943,0cyclictest16337-21awk10:55:542
259039943,0cyclictest1145-21sadc07:30:402
259029944,0cyclictest0-21swapper/109:35:001
259029943,0cyclictest7981-21awk11:10:531
259029943,0cyclictest681-21awk10:05:531
259029943,0cyclictest3514-21snmpd10:36:101
259029943,0cyclictest3514-21snmpd10:18:101
259029943,0cyclictest3514-21snmpd09:25:401
259029943,0cyclictest3514-21snmpd07:59:241
259029943,0cyclictest3514-21snmpd07:53:241
259029943,0cyclictest3514-21snmpd07:36:541
259029943,0cyclictest3514-21snmpd07:27:241
259029943,0cyclictest3514-21snmpd07:21:241
259029943,0cyclictest3514-21snmpd07:05:541
259029943,0cyclictest3241-21irqbalance12:25:571
259029943,0cyclictest3241-21irqbalance11:51:071
259029943,0cyclictest3241-21irqbalance11:45:471
259029943,0cyclictest3241-21irqbalance11:16:071
259029943,0cyclictest3241-21irqbalance10:31:271
259029943,0cyclictest3241-21irqbalance09:57:071
259029943,0cyclictest32327-21awk11:05:531
259029943,0cyclictest25228-21awk09:00:531
259029943,0cyclictest11888-21awk08:50:531
259019944,0cyclictest0-21swapper/012:22:590
259019944,0cyclictest0-21swapper/010:26:000
259019943,0cyclictest8864-21awk10:10:540
259019943,0cyclictest8320-21awk10:50:530
259019943,0cyclictest8059-21awk07:10:540
259019943,0cyclictest7558-21awk11:50:530
259019943,0cyclictest7237-21awk12:10:530
259019943,0cyclictest3556-21lldpd11:46:530
259019943,0cyclictest3514-21snmpd08:31:240
259019943,0cyclictest3514-21snmpd07:54:090
259019943,0cyclictest3241-21irqbalance11:30:470
259019943,0cyclictest3241-21irqbalance11:26:370
259019943,0cyclictest3241-21irqbalance10:45:570
259019943,0cyclictest3241-21irqbalance10:33:170
259019943,0cyclictest3241-21irqbalance10:06:070
259019943,0cyclictest3241-21irqbalance09:51:070
259019943,0cyclictest3241-21irqbalance09:30:570
259019943,0cyclictest3241-21irqbalance09:28:570
259019943,0cyclictest3241-21irqbalance07:31:080
259019943,0cyclictest29994-21awk08:15:540
259019943,0cyclictest27043-21awk07:00:550
259019943,0cyclictest25528-21awk09:20:530
259019943,0cyclictest24295-21awk08:35:560
259019943,0cyclictest24249-21awk11:00:540
259019943,0cyclictest23619-21sadc10:20:390
259019943,0cyclictest16002-21awk11:15:550
259019940,0cyclictest0-21swapper/007:49:270
259940,0migration/24459-21kthreadcore10:25:562
259940,0migration/227923-21kthreadcore09:00:562
259940,0migration/225879-21kthreadcore07:45:572
259940,0migration/225670-21kthreadcore08:35:572
259940,0migration/225211-21kthreadcore10:40:552
259940,0migration/221177-21netstat08:05:592
259940,0migration/210962-21latency_hist08:50:392
249940,0watchdog/2835-21ssh09:05:532
249940,0watchdog/2710-21cut08:40:572
249940,0watchdog/20-21swapper/208:25:572
249940,0watchdog/20-21swapper/207:40:582
21942244,0sleep021151-21kworker/0:010:57:200
179940,0migration/18977-21hddtemp_smartct09:50:511
179940,0migration/16890-21kthreadcore08:45:561
179940,0migration/15155-21taskset11:26:351
179940,0migration/14208-21kthreadcore07:30:571
179940,0migration/126924-21kthreadcore12:20:561
179940,0migration/122556-21runrttasks12:00:431
179940,0migration/122349-21kthreadcore08:55:591
179940,0migration/122073-21runrttasks11:59:581
179940,0migration/119342-21kthreadcore09:15:531
179940,0migration/11809-21kthreadcore08:15:561
179940,0migration/117600-21kthreadcore08:05:561
179940,0migration/116779-21kthreadcore07:15:581
179940,0migration/116551-21kthreadcore12:15:551
179940,0migration/116229-21runrttasks08:27:511
179940,0migration/114016-21taskset10:52:321
169940,0watchdog/16559-21runrttasks11:36:251
169940,0watchdog/15354-21unixbench_multi10:46:011
169940,0watchdog/130175-21munin-node09:20:571
169940,0watchdog/115337-21ssh10:55:451
169940,0watchdog/11402-21irqstats09:45:531
149940,0migration/032010-21taskset09:44:570
149940,0migration/031030-21kthreadcore07:25:550
149940,0migration/02656-21kthreadcore12:05:570
149940,0migration/023850-21fschecks_time11:20:520
149940,0migration/016740-21irqcore10:35:580
149940,0migration/01673-21kthreadcore08:40:580
149940,0migration/010410-21taskset08:46:270
339930,0migration/310850-21kthreadcore08:00:553
29715030,0irq/26-eth0-rx-0-21swapper/307:55:513
29715030,0irq/26-eth0-rx-0-21swapper/307:35:523
29715030,0irq/26-eth0-rx-0-21swapper/307:00:523
259039932,0cyclictest3556-21lldpd08:30:492
259039932,0cyclictest3556-21lldpd08:00:492
259039930,0cyclictest22806-21diskstats08:10:522
259029933,0cyclictest3556-21lldpd12:11:231
259029932,0cyclictest9828-21ls08:25:391
259029932,0cyclictest3514-21snmpd09:07:251
259029932,0cyclictest3241-21irqbalance07:10:581
259029932,0cyclictest22360-21sensors08:31:001
259019932,0cyclictest8024-21awk11:10:530
259019932,0cyclictest3718-21sensors12:26:010
259019932,0cyclictest23293-21awk08:10:540
259019932,0cyclictest15759-21awk11:35:530
259019930,0cyclictest20432-21munin-node07:20:530
259019930,0cyclictest17713-21munin-node08:55:570
249930,0watchdog/20-21swapper/207:31:382
249930,0watchdog/20-21swapper/207:20:502
179930,0migration/125369-21kthreadcore08:10:561
179930,0migration/112635-21kthreadcore08:00:561
169930,0watchdog/10-21swapper/110:43:131
169930,0watchdog/10-21swapper/110:25:531
169930,0watchdog/10-21swapper/108:40:571
169930,0watchdog/10-21swapper/107:40:581
169930,0watchdog/10-21swapper/107:00:541
149930,0migration/024979-21gltestperf10:00:520
149930,0migration/02463-21kthreadcore09:05:550
149930,0migration/012566-21kthreadcore09:10:550
29715020,0irq/26-eth0-rx-0-21swapper/307:50:493
29715020,0irq/26-eth0-rx-0-21swapper/307:30:523
259019921,0cyclictest0-21swapper/011:40:390
249920,0watchdog/20-21swapper/207:00:502
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional