You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-13 - 21:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Mon Apr 13, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450460,0irq/25-eth10-21swapper/306:53:583
132524233,0sleep20-21swapper/206:57:412
134323423,0sleep00-21swapper/006:57:550
129023324,0sleep10-21swapper/106:57:201
29715070,0irq/26-eth0-rx-15143-21munin-run10:48:353
4160,0ktimersoftd/00-21swapper/012:09:390
339960,0migration/37599-21sh10:03:153
339960,0migration/36941-21sh09:20:163
339960,0migration/36349-21sh11:02:033
339960,0migration/331575-21sh09:16:403
339960,0migration/329853-21sh10:34:033
339960,0migration/329159-21sh12:16:043
339960,0migration/323984-21sh10:13:433
339960,0migration/314941-21sh09:45:503
339960,0migration/314155-21sh10:04:053
29715060,0irq/26-eth0-rx-4650-21switchtime08:33:553
14789966,0cyclictest0-21swapper/208:38:542
14789960,0cyclictest0-21swapper/207:59:002
14779966,0cyclictest0-21swapper/110:03:531
14769960,0cyclictest0-21swapper/011:47:440
8215050,0irq/24-0000:00:0-21swapper/211:28:352
339950,0migration/38564-21sh09:23:453
339950,0migration/38009-21munin-node11:03:483
339950,0migration/37650-21irqcore11:43:503
339950,0migration/35065-21kthreadcore07:23:523
339950,0migration/34693-21taskset12:20:463
339950,0migration/34693-21taskset12:20:463
339950,0migration/33976-21pmu-power11:58:553
339950,0migration/329800-21ksmtuned08:05:273
339950,0migration/327100-21kthreadcore08:53:523
339950,0migration/326691-21kthreadcore09:33:523
339950,0migration/323849-21irqrtprio11:53:493
339950,0migration/322336-21taskset07:36:533
339950,0migration/322025-21taskset10:50:163
339950,0migration/321771-21kthreadcore08:23:523
339950,0migration/318771-21kthreadcore09:08:533
339950,0migration/31759-21aten_r1power_fr06:58:463
339950,0migration/31287-21latency07:18:533
339950,0migration/310547-21munin-node07:53:473
29725050,0irq/27-eth0-tx-6559-21runrttasks10:37:112
29725050,0irq/27-eth0-tx-32523-21munin-node10:58:522
29725050,0irq/27-eth0-tx-28164-21grep10:53:532
29725050,0irq/27-eth0-tx-24001-21munin-node10:13:512
29725050,0irq/27-eth0-tx-18843-21kthreadcore10:28:542
29725050,0irq/27-eth0-tx-17033-21munin-node07:58:580
29725050,0irq/27-eth0-tx-0-21swapper/210:10:052
29725050,0irq/27-eth0-tx-0-21swapper/209:06:442
29715050,0irq/26-eth0-rx-8047-21ssh09:03:393
29715050,0irq/26-eth0-rx-4380-21kthreadcore09:38:523
29715050,0irq/26-eth0-rx-29791-21ssh11:15:253
29715050,0irq/26-eth0-rx-24251-21cut09:53:463
29715050,0irq/26-eth0-rx-17416-21cut10:08:513
29715050,0irq/26-eth0-rx-15559-21kthreadcore08:43:543
29715050,0irq/26-eth0-rx-12294-21kthreadcore10:23:543
29715050,0irq/26-eth0-rx-0-21swapper/312:04:593
29715050,0irq/26-eth0-rx-0-21swapper/311:37:143
29715050,0irq/26-eth0-rx-0-21swapper/311:31:473
29715050,0irq/26-eth0-rx-0-21swapper/311:23:383
29715050,0irq/26-eth0-rx-0-21swapper/311:09:193
29715050,0irq/26-eth0-rx-0-21swapper/310:53:503
29715050,0irq/26-eth0-rx-0-21swapper/310:41:303
29715050,0irq/26-eth0-rx-0-21swapper/310:22:303
29715050,0irq/26-eth0-rx-0-21swapper/309:52:233
29715050,0irq/26-eth0-rx-0-21swapper/309:29:373
29715050,0irq/26-eth0-rx-0-21swapper/308:18:543
29715050,0irq/26-eth0-rx-0-21swapper/307:48:463
27150,0ktimersoftd/20-21swapper/211:38:562
259950,0migration/232186-21latency_hist08:58:352
259950,0migration/224145-21latency_hist09:53:352
259950,0migration/223345-21aten2.4-expect11:13:462
259950,0migration/222913-21cpuspeed_turbos11:53:452
259950,0migration/216996-21munin-node09:08:502
259950,0migration/214995-21latency_hist11:08:352
249950,0watchdog/2628-21irqcore10:18:532
249950,0watchdog/215467-21aten_r1power_en10:48:452
179950,0migration/18178-21fschecks_count10:23:491
179950,0migration/16933-21cpuspeed_turbos11:43:471
179950,0migration/16039-21taskset12:03:301
179950,0migration/15408-21latency_hist08:38:351
179950,0migration/131878-21taskset09:58:191
179950,0migration/131182-21kthreadcore07:18:521
179950,0migration/129229-21taskset11:34:361
179950,0migration/126923-21kthreadcore08:28:511
179950,0migration/122160-21sh10:50:401
179950,0migration/121203-21taskset07:09:301
179950,0migration/117303-21irqrtprio07:33:521
179950,0migration/11601-21kthreadcore08:58:511
169950,0watchdog/1620-21ps10:18:531
169950,0watchdog/118168-21cut11:28:531
14789955,0cyclictest32098-21diskmemload09:44:572
14789955,0cyclictest0-21swapper/211:49:302
14789954,0cyclictest15374-21ps08:43:542
14789950,0cyclictest0-21swapper/210:02:272
14789950,0cyclictest0-21swapper/209:27:082
14779950,0cyclictest0-21swapper/110:44:521
14779950,0cyclictest0-21swapper/109:15:571
14769955,0cyclictest0-21swapper/011:21:240
14769955,0cyclictest0-21swapper/011:02:190
14769955,0cyclictest0-21swapper/010:54:080
14769955,0cyclictest0-21swapper/010:14:140
14769955,0cyclictest0-21swapper/009:40:300
14769955,0cyclictest0-21swapper/009:05:300
14769954,0cyclictest3514-21snmpd10:32:520
14769954,0cyclictest0-21swapper/009:48:510
14769950,0cyclictest0-21swapper/012:25:530
14769950,0cyclictest0-21swapper/012:18:280
14769950,0cyclictest0-21swapper/011:12:250
149950,0migration/05898-21sh10:19:080
149950,0migration/03751-21kthreadcore10:38:520
149950,0migration/029530-21sh11:56:010
149950,0migration/028063-21kthreadcore11:13:520
149950,0migration/025735-21aten_r1power_fr08:53:470
149950,0migration/02326-21missed_timers07:43:550
149950,0migration/022952-21sh10:11:240
149950,0migration/022514-21sh10:51:410
149950,0migration/016388-21sh09:08:480
149950,0migration/014556-21runrttasks10:46:550
149950,0migration/013757-21sh12:07:390
149950,0migration/013209-21kthreadcore08:18:540
149950,0migration/010141-21kthreadcore10:23:530
339940,0migration/35246-21runrttasks11:19:453
339940,0migration/31640-21kthreadcore08:58:513
29725040,0irq/27-eth0-tx-31006-21ssh11:58:472
29725040,0irq/27-eth0-tx-22785-21ssh09:29:182
29725040,0irq/27-eth0-tx-10405-21munin-node07:53:530
29725040,0irq/27-eth0-tx-0-21swapper/212:23:462
29725040,0irq/27-eth0-tx-0-21swapper/212:03:472
29725040,0irq/27-eth0-tx-0-21swapper/211:43:472
29725040,0irq/27-eth0-tx-0-21swapper/211:08:442
29725040,0irq/27-eth0-tx-0-21swapper/210:39:252
29725040,0irq/27-eth0-tx-0-21swapper/209:48:382
29725040,0irq/27-eth0-tx-0-21swapper/209:39:082
29725040,0irq/27-eth0-tx-0-21swapper/008:23:480
29725040,0irq/27-eth0-tx-0-21swapper/007:03:460
29715040,0irq/26-eth0-rx-0-21swapper/312:27:393
29715040,0irq/26-eth0-rx-0-21swapper/312:08:373
29715040,0irq/26-eth0-rx-0-21swapper/311:49:063
29715040,0irq/26-eth0-rx-0-21swapper/311:39:383
29715040,0irq/26-eth0-rx-0-21swapper/310:29:013
29715040,0irq/26-eth0-rx-0-21swapper/308:28:523
29715040,0irq/26-eth0-rx-0-21swapper/308:18:303
29715040,0irq/26-eth0-rx-0-21swapper/308:01:033
29715040,0irq/26-eth0-rx-0-21swapper/307:43:473
29715040,0irq/26-eth0-rx-0-21swapper/307:08:493
259940,0migration/29588-21cpu07:28:472
259940,0migration/28006-21fschecks_time10:43:492
259940,0migration/26464-21runrttasks09:19:092
259940,0migration/225080-21gltestperf09:13:502
259940,0migration/224617-21aten_r1power_fr09:33:472
259940,0migration/224412-21gltestperf08:03:492
259940,0migration/219303-21df_abs08:48:472
259940,0migration/210275-21processes08:13:562
249940,0watchdog/23547-21munin-node07:23:502
249940,0watchdog/2297250irq/27-eth0-tx-11:18:372
249940,0watchdog/226800-21kthreadcore12:13:532
249940,0watchdog/21042-21kthreadcore07:43:542
249940,0watchdog/20-21swapper/212:18:532
249940,0watchdog/20-21swapper/212:18:532
249940,0watchdog/20-21swapper/211:28:492
179940,0migration/17179-21needreboot06:58:531
179940,0migration/130686-21kthreadcore08:53:531
179940,0migration/12966-21kthreadcore11:18:531
179940,0migration/125649-21kthreadcore12:13:521
179940,0migration/12472-21kthreadcore08:08:531
179940,0migration/124291-21kthreadcore07:13:521
179940,0migration/122588-21sh10:10:211
179940,0migration/119848-21kthreadcore09:48:511
179940,0migration/118183-21kthreadcore11:08:501
179940,0migration/11738-21kthreadcore09:18:521
179940,0migration/116976-21fschecks_count09:28:481
179940,0migration/116913-21irqrtprio10:28:521
179940,0migration/112156-21kthreadcore07:53:521
179940,0migration/112009-21kthreadcore11:03:521
179940,0migration/111808-21switchtime12:23:571
179940,0migration/111227-21kthreadcore09:43:521
169940,0watchdog/16871-21munin-node11:23:571
169940,0watchdog/14590-21ssh12:20:291
169940,0watchdog/14590-21ssh12:20:291
169940,0watchdog/126414-21cut10:33:531
169940,0watchdog/118801-21if_err_eth008:23:491
169940,0watchdog/114958-21grep12:08:491
169940,0watchdog/111971-21cut09:03:531
169940,0watchdog/110233-21idleruntime07:28:501
169940,0watchdog/10-21swapper/109:08:531
169940,0watchdog/10-21swapper/108:43:531
14789943,0cyclictest9371-21awk07:03:502
14789943,0cyclictest3556-21lldpd10:03:452
14789943,0cyclictest3556-21lldpd08:59:452
14789943,0cyclictest3556-21lldpd08:37:442
14789943,0cyclictest3556-21lldpd08:30:442
14789943,0cyclictest3556-21lldpd08:11:442
14789943,0cyclictest3556-21lldpd07:40:142
14789943,0cyclictest3556-21lldpd07:35:142
14789943,0cyclictest3556-21lldpd07:16:432
14789943,0cyclictest3514-21snmpd10:23:522
14789943,0cyclictest29277-21awk07:18:492
14789943,0cyclictest23865-21awk11:33:492
14789943,0cyclictest15215-21awk12:08:502
14789943,0cyclictest12206-21awk08:18:522
14779944,0cyclictest0-21swapper/110:56:321
14779943,0cyclictest711-21awk09:58:501
14779943,0cyclictest3556-21lldpd07:40:441
14779943,0cyclictest3514-21snmpd10:13:521
14779943,0cyclictest3241-21irqbalance11:49:131
14779943,0cyclictest32336-21awk10:58:501
14779943,0cyclictest30318-21awk07:43:511
14779943,0cyclictest30082-21sadc08:08:351
14779943,0cyclictest19780-21awk08:48:491
14779940,0cyclictest0-21swapper/111:55:211
14769944,0cyclictest0-21swapper/011:48:500
14769944,0cyclictest0-21swapper/010:35:030
14769943,0cyclictest9165-21awk09:43:500
14769943,0cyclictest5629-21awk08:13:500
14769943,0cyclictest3556-21lldpd11:07:160
14769943,0cyclictest3514-21snmpd07:21:070
14769943,0cyclictest3514-21snmpd07:14:220
14769943,0cyclictest3241-21irqbalance11:38:430
14769943,0cyclictest3241-21irqbalance09:35:330
14769943,0cyclictest3241-21irqbalance09:32:030
14769943,0cyclictest3241-21irqbalance09:14:430
14769943,0cyclictest3241-21irqbalance08:58:440
14769943,0cyclictest3241-21irqbalance08:33:440
14769943,0cyclictest3241-21irqbalance08:29:040
14769943,0cyclictest3241-21irqbalance07:27:340
14769943,0cyclictest31545-21awk11:58:490
14769943,0cyclictest31272-21awk08:08:500
14769943,0cyclictest25146-21awk09:53:500
14769943,0cyclictest24591-21awk08:03:510
14769943,0cyclictest15861-21awk11:28:510
14769940,0cyclictest22788-21telnet07:38:460
149940,0migration/09039-21kthreadcore08:38:530
149940,0migration/024734-21kthreadcore11:33:510
149940,0migration/021650-21kthreadcore08:48:510
149940,0migration/017233-21kthreadcore08:43:540
149940,0migration/015093-21latency_hist07:08:350
149940,0migration/013790-21kthreadcore07:28:530
149940,0migration/010268-21kthreadcore09:23:510
339930,0migration/315754-21runrttasks07:32:263
29715030,0irq/26-eth0-rx-0-21swapper/308:48:463
29715030,0irq/26-eth0-rx-0-21swapper/308:08:463
29715030,0irq/26-eth0-rx-0-21swapper/307:38:503
29715030,0irq/26-eth0-rx-0-21swapper/307:13:513
29715030,0irq/26-eth0-rx-0-21swapper/307:03:463
249930,0watchdog/20-21swapper/206:58:582
169930,0watchdog/12014-21kthreadcore11:38:531
169930,0watchdog/10-21swapper/112:03:411
169930,0watchdog/10-21swapper/111:14:011
169930,0watchdog/10-21swapper/110:38:451
169930,0watchdog/10-21swapper/109:33:531
169930,0watchdog/10-21swapper/109:24:011
169930,0watchdog/10-21swapper/107:59:101
169930,0watchdog/10-21swapper/107:49:021
14789933,0cyclictest3556-21lldpd08:26:142
14789933,0cyclictest3556-21lldpd07:49:442
14789932,0cyclictest3556-21lldpd07:08:432
14789930,0cyclictest11102-21perl07:53:492
14779933,0cyclictest0-21swapper/109:38:391
14779932,0cyclictest9473-21irqcore07:03:511
14779932,0cyclictest6561-21awk08:38:511
14779932,0cyclictest5921-21kernelversion08:13:511
14779932,0cyclictest12254-21awk08:18:531
14769932,0cyclictest6858-21sh11:23:430
14769932,0cyclictest3556-21lldpd09:18:450
14769932,0cyclictest3514-21snmpd07:49:520
14769932,0cyclictest3514-21snmpd07:03:220
14769932,0cyclictest32648-21kthreadcore12:18:520
14769932,0cyclictest32648-21kthreadcore12:18:520
14769932,0cyclictest32175-21cat09:58:350
14769932,0cyclictest17032-21awk07:33:510
14769932,0cyclictest12739-21grep10:03:540
29715020,0irq/26-eth0-rx-0-21swapper/308:39:533
169920,0watchdog/10-21swapper/107:23:381
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional