You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-12 - 21:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sun Apr 12, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2413523624,0sleep00-21swapper/006:56:320
304450350,0irq/25-eth10-21swapper/306:54:063
2403123526,0sleep20-21swapper/206:55:182
2395923228,0sleep10-21swapper/106:54:291
4160,0ktimersoftd/00-21swapper/008:03:580
339960,0migration/35460-21sh09:46:053
339960,0migration/35009-21sh09:04:543
339960,0migration/33054-21taskset11:44:143
339960,0migration/329899-21sh10:22:483
29725060,0irq/27-eth0-tx-0-21swapper/210:18:352
29715060,0irq/26-eth0-rx-8062-21kernelversion09:08:573
29715060,0irq/26-eth0-rx-31455-21kernelversion10:43:563
29715060,0irq/26-eth0-rx-1569-21grep09:23:583
29715060,0irq/26-eth0-rx-11737-21ssh11:30:003
19160,0ktimersoftd/10-21swapper/112:06:231
19160,0ktimersoftd/10-21swapper/109:07:261
8215050,0irq/24-0000:00:5539-21munin-run10:48:402
8215050,0irq/24-0000:00:14438-21munin-run09:53:402
4150,0ktimersoftd/00-21swapper/011:32:190
35150,0ktimersoftd/30-21swapper/312:06:233
339950,0migration/38046-21kthreadcore07:08:583
339950,0migration/37486-21kthreadcore10:48:583
339950,0migration/37115-21fschecks_time07:33:553
339950,0migration/36848-21fschecks_count10:08:543
339950,0migration/332414-21irqrtprio07:03:563
339950,0migration/332069-21kthreadcore12:23:593
339950,0migration/327791-21munin-node07:48:573
339950,0migration/327659-21kthreadcore06:58:583
339950,0migration/326051-21kthreadcore08:33:583
339950,0migration/325581-21munin-node07:44:023
339950,0migration/325580-21kthreadcore08:08:593
339950,0migration/323400-21fschecks_count09:38:533
339950,0migration/322781-21aten2.4-expect09:58:513
339950,0migration/322133-21taskset09:57:543
339950,0migration/317755-21kthreadcore08:53:593
339950,0migration/316884-21kthreadcore10:13:563
339950,0migration/315469-21fschecks_time09:13:543
339950,0migration/315281-21kthreadcore11:53:563
339950,0migration/315003-21idleruntime10:33:543
339950,0migration/31483-21sh08:18:413
339950,0migration/313670-21aten_r1power_vo11:13:503
339950,0migration/313495-21unixbench_singl07:59:043
339950,0migration/311913-21taskset12:12:213
339950,0migration/311425-21sh11:50:053
29725050,0irq/27-eth0-tx-20972-21unixbench_multi09:34:032
29725050,0irq/27-eth0-tx-12644-21munin-node12:13:552
29715050,0irq/26-eth0-rx-28500-21ssh12:03:073
29715050,0irq/26-eth0-rx-27248-21processes11:18:583
29715050,0irq/26-eth0-rx-25402-21kthreadcore10:58:583
29715050,0irq/26-eth0-rx-21185-21ssh10:57:343
29715050,0irq/26-eth0-rx-2091-21kthreadcore07:28:573
29715050,0irq/26-eth0-rx-20500-21ntp_states09:34:003
29715050,0irq/26-eth0-rx-0-21swapper/312:19:023
29715050,0irq/26-eth0-rx-0-21swapper/312:14:253
29715050,0irq/26-eth0-rx-0-21swapper/311:41:053
29715050,0irq/26-eth0-rx-0-21swapper/311:37:323
29715050,0irq/26-eth0-rx-0-21swapper/311:23:423
29715050,0irq/26-eth0-rx-0-21swapper/311:08:563
29715050,0irq/26-eth0-rx-0-21swapper/310:29:583
29715050,0irq/26-eth0-rx-0-21swapper/309:29:363
29715050,0irq/26-eth0-rx-0-21swapper/309:01:173
27150,0ktimersoftd/20-21swapper/212:25:302
27150,0ktimersoftd/20-21swapper/212:12:162
27150,0ktimersoftd/20-21swapper/212:06:232
27150,0ktimersoftd/20-21swapper/209:07:262
259950,0migration/2446-21kthreadcore11:03:582
259950,0migration/215158-21kthreadcore11:53:562
243989955,0cyclictest0-21swapper/210:00:282
243989955,0cyclictest0-21swapper/209:21:592
243989954,0cyclictest6651-21crond09:28:402
243989954,0cyclictest3514-21snmpd10:04:592
243989954,0cyclictest3514-21snmpd09:40:442
243989954,0cyclictest3514-21snmpd08:13:592
243989950,0cyclictest0-21swapper/211:40:252
243989950,0cyclictest0-21swapper/211:13:082
243989950,0cyclictest0-21swapper/211:02:062
243989950,0cyclictest0-21swapper/210:30:342
243989950,0cyclictest0-21swapper/209:46:452
243989950,0cyclictest0-21swapper/209:32:242
243909955,0cyclictest22564-21diskmemload11:56:131
243909955,0cyclictest12345-21memory09:28:581
243909955,0cyclictest0-21swapper/112:23:111
243909954,0cyclictest14747-21awk10:53:531
243909950,0cyclictest0-21swapper/111:25:101
243909950,0cyclictest0-21swapper/111:23:161
243909950,0cyclictest0-21swapper/111:10:401
243909950,0cyclictest0-21swapper/109:55:081
243899955,0cyclictest0-21swapper/012:11:160
243899955,0cyclictest0-21swapper/009:05:040
243899950,0cyclictest0-21swapper/012:24:300
243899950,0cyclictest0-21swapper/010:47:040
243899950,0cyclictest0-21swapper/010:22:000
249950,0watchdog/226842-21kthreadcore07:23:592
19150,0ktimersoftd/10-21swapper/108:44:021
179950,0migration/19508-21aten_r1power_fr08:48:531
179950,0migration/17948-21munin-node09:08:561
179950,0migration/130467-21munin-node06:59:041
179950,0migration/130195-21taskset10:23:391
179950,0migration/127382-21kthreadcore10:38:581
179950,0migration/126811-21kthreadcore10:58:581
179950,0migration/120985-21cpu08:08:531
179950,0migration/118682-21missed_timers11:33:581
179950,0migration/115300-21irqcore08:03:561
149950,0migration/09303-21latency_hist08:48:400
149950,0migration/05176-21kthreadcore07:53:590
149950,0migration/04998-21latency_hist11:28:400
149950,0migration/029968-21aten2.4-expect11:03:520
149950,0migration/023647-21kthreadcore10:58:570
149950,0migration/018970-21aten2.4-expect07:18:510
149950,0migration/015756-21hwlatdetect09:33:540
149950,0migration/014510-21aten2.4-expect10:13:500
339940,0migration/36616-21taskset09:48:423
339940,0migration/33893-21kthreadcore08:18:593
339940,0migration/331744-21kthreadcore08:38:573
339940,0migration/324404-21irqrtprio09:18:563
339940,0migration/318693-21latency_hist07:18:413
339940,0migration/317205-21kthreadcore07:38:583
29725040,0irq/27-eth0-tx-5288-21aten2.4-expect11:28:512
29725040,0irq/27-eth0-tx-21628-21if_eth1.312:18:542
29725040,0irq/27-eth0-tx-0-21swapper/211:59:162
29725040,0irq/27-eth0-tx-0-21swapper/211:50:502
29725040,0irq/27-eth0-tx-0-21swapper/211:34:012
29725040,0irq/27-eth0-tx-0-21swapper/211:24:132
29725040,0irq/27-eth0-tx-0-21swapper/211:22:232
29725040,0irq/27-eth0-tx-0-21swapper/210:50:172
29725040,0irq/27-eth0-tx-0-21swapper/209:54:072
29725040,0irq/27-eth0-tx-0-21swapper/209:13:412
29725040,0irq/27-eth0-tx-0-21swapper/008:38:530
29725040,0irq/27-eth0-tx-0-21swapper/007:58:550
29725040,0irq/27-eth0-tx-0-21swapper/007:23:560
29725040,0irq/27-eth0-tx-0-21swapper/007:03:520
29715040,0irq/26-eth0-rx-0-21swapper/311:04:263
29715040,0irq/26-eth0-rx-0-21swapper/310:38:523
29715040,0irq/26-eth0-rx-0-21swapper/310:25:493
29715040,0irq/26-eth0-rx-0-21swapper/310:03:523
29715040,0irq/26-eth0-rx-0-21swapper/307:53:523
259940,0migration/224541-21kthreadcore07:43:572
259940,0migration/216956-21kthreadcore11:13:552
259940,0migration/21653-21kthreadcore11:43:582
259940,0migration/214175-21aten_r1power_cu10:33:502
259940,0migration/210971-21kthreadcore09:08:582
259940,0migration/210307-21kthreadcore07:58:582
243989944,0cyclictest0-21swapper/210:55:382
243989943,0cyclictest7388-21awk10:08:562
243989943,0cyclictest6529-21awk07:08:562
243989943,0cyclictest3556-21lldpd08:44:342
243989943,0cyclictest3556-21lldpd08:04:332
243989943,0cyclictest3514-21snmpd08:35:292
243989943,0cyclictest3514-21snmpd07:31:592
243989943,0cyclictest29317-21awk08:38:552
243989943,0cyclictest27361-21awk07:48:562
243989943,0cyclictest2605-21awk08:18:572
243989943,0cyclictest23647-21awk08:58:532
243989943,0cyclictest22549-21sadc08:58:402
243989943,0cyclictest21700-21awk08:08:562
243989943,0cyclictest15961-21awk08:28:552
243989943,0cyclictest14068-21awk07:38:562
243989940,0cyclictest0-21swapper/210:38:592
243909944,0cyclictest22564-21diskmemload09:52:281
243909943,0cyclictest7409-21awk07:33:571
243909943,0cyclictest6102-21awk11:28:551
243909943,0cyclictest3556-21lldpd09:25:031
243909943,0cyclictest3514-21snmpd11:16:001
243909943,0cyclictest3514-21snmpd08:43:141
243909943,0cyclictest3514-21snmpd08:24:591
243909943,0cyclictest3514-21snmpd08:02:591
243909943,0cyclictest3514-21snmpd07:38:591
243909943,0cyclictest3514-21snmpd07:16:591
243909943,0cyclictest3514-21snmpd07:08:591
243909943,0cyclictest3514-21snmpd07:03:591
243909943,0cyclictest3241-21irqbalance10:10:491
243909943,0cyclictest31042-21awk10:43:541
243909943,0cyclictest15708-21awk09:13:551
243899944,0cyclictest0-21swapper/011:49:400
243899944,0cyclictest0-21swapper/011:47:460
243899944,0cyclictest0-21swapper/009:30:180
243899943,0cyclictest9349-21awk08:23:560
243899943,0cyclictest7763-21awk09:08:560
243899943,0cyclictest7634-21awk09:48:550
243899943,0cyclictest703-21awk07:28:560
243899943,0cyclictest3556-21lldpd12:14:330
243899943,0cyclictest3556-21lldpd10:05:330
243899943,0cyclictest3556-21lldpd08:50:040
243899943,0cyclictest3514-21snmpd12:05:150
243899943,0cyclictest3514-21snmpd07:36:290
243899943,0cyclictest3241-21irqbalance12:00:080
243899943,0cyclictest3241-21irqbalance10:09:590
243899943,0cyclictest3241-21irqbalance08:13:590
243899943,0cyclictest27391-21awk07:48:560
243899943,0cyclictest2659-21awk08:18:580
243899943,0cyclictest25470-21awk06:58:560
243899943,0cyclictest22683-21awk08:33:560
243899943,0cyclictest22161-21awk11:38:550
243899943,0cyclictest15053-21awk10:33:540
243899943,0cyclictest13084-21awk07:13:550
249940,0watchdog/232237-21interrupts07:03:552
249940,0watchdog/23184-21kthreadcore10:23:592
249940,0watchdog/229913-21sort06:58:592
249940,0watchdog/212747-21cut08:48:592
249940,0watchdog/20-21swapper/210:18:552
249940,0watchdog/20-21swapper/207:13:552
179940,0migration/18902-21kthreadcore12:08:571
179940,0migration/18318-21kthreadcore10:48:581
179940,0migration/15062-21aten_r1power_po11:48:551
179940,0migration/132269-21aten_r1power_fr07:28:521
179940,0migration/131594-21fschecks_time09:43:541
179940,0migration/129708-21irqcore12:23:571
179940,0migration/128175-21kthreadcore07:48:581
179940,0migration/12620-21sensors_temp11:44:021
179940,0migration/125629-21kthreadcore08:33:581
179940,0migration/125195-21kthreadcore11:38:571
179940,0migration/124934-21kthreadcore09:18:571
179940,0migration/122678-21sh09:58:491
179940,0migration/121025-21irqrtprio07:43:561
179940,0migration/119838-21missed_timers10:13:571
179940,0migration/118995-21kthreadcore10:33:571
179940,0migration/117819-21kthreadcore09:33:571
179940,0migration/117158-21kthreadcore08:28:571
179940,0migration/116612-21kthreadcore12:13:571
169940,0watchdog/16559-21runrttasks11:03:421
169940,0watchdog/13812-21needreboot10:03:591
169940,0watchdog/131232-21grep10:23:551
169940,0watchdog/12111-21diskstats08:18:551
169940,0watchdog/10-21swapper/110:29:471
149940,0migration/07208-21sh10:28:560
149940,0migration/04564-21taskset10:24:550
149940,0migration/04026-21kthreadcore09:23:590
149940,0migration/026761-21unixbench_multi08:09:050
149940,0migration/024936-21kthreadcore12:18:560
149940,0migration/024664-21kthreadcore10:38:570
149940,0migration/022512-21if_eth1.311:18:530
149940,0migration/020433-21ksmtuned10:55:170
149940,0migration/019682-21kthreadcore08:28:580
149940,0migration/017289-21kthreadcore11:13:560
149940,0migration/015271-21fschecks_time09:53:530
149940,0migration/014829-21kthreadcore11:53:560
149940,0migration/014112-21idleruntime11:33:540
29725030,0irq/27-eth0-tx-0-21swapper/007:43:510
29715030,0irq/26-eth0-rx-0-21swapper/308:48:543
29715030,0irq/26-eth0-rx-0-21swapper/308:28:523
29715030,0irq/26-eth0-rx-0-21swapper/308:24:243
29715030,0irq/26-eth0-rx-0-21swapper/308:04:583
29715030,0irq/26-eth0-rx-0-21swapper/307:23:533
243989932,0cyclictest19856-21awk07:18:552
243989930,0cyclictest0-21swapper/207:33:562
243909932,0cyclictest6485-21sensors07:54:021
243909932,0cyclictest32103-21tr08:13:581
243909932,0cyclictest21842-21awk11:58:541
243909932,0cyclictest19759-21awk07:18:551
243899933,0cyclictest3556-21lldpd11:09:330
243899932,0cyclictest3514-21snmpd08:53:590
243899932,0cyclictest31576-21tune2fs09:43:540
243899932,0cyclictest23999-21awk09:18:540
243899932,0cyclictest23928-21awk09:38:550
243899932,0cyclictest23556-21awk09:58:540
243899932,0cyclictest17675-21cut09:13:580
243899932,0cyclictest13161-21munin-node07:39:040
243899930,0cyclictest23440-21perl08:58:520
249930,0watchdog/20-21swapper/208:23:592
249930,0watchdog/20-21swapper/207:53:552
169930,0watchdog/10-21swapper/109:38:511
169930,0watchdog/10-21swapper/108:58:511
169930,0watchdog/10-21swapper/108:53:591
169930,0watchdog/10-21swapper/107:23:591
149930,0migration/09646-21kthreadcore07:08:580
29715020,0irq/26-eth0-rx-0-21swapper/308:44:023
29715020,0irq/26-eth0-rx-0-21swapper/307:18:523
243899921,0cyclictest5532-21latency_hist10:48:400
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional