You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-23 - 16:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sat May 23, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
906724536,0sleep10-21swapper/106:50:331
928023524,0sleep00-21swapper/006:53:040
304450350,0irq/25-eth10-21swapper/306:50:133
914323026,0sleep20-21swapper/206:51:262
95019970,0cyclictest0-21swapper/110:30:021
95029960,0cyclictest1753-21diskmemload11:00:532
94999960,0cyclictest0-21swapper/012:07:430
339960,0migration/3557-21sh09:31:413
339960,0migration/3557-21sh09:31:413
339960,0migration/323723-21latency09:25:063
339960,0migration/318177-21sh12:07:223
339960,0migration/316906-21sh10:44:223
339960,0migration/310034-21crond09:19:483
29715060,0irq/26-eth0-rx-0-21swapper/311:45:493
149960,0migration/027352-21sh11:34:170
149960,0migration/026078-21sh09:09:340
149960,0migration/025855-21sh12:11:270
149960,0migration/025007-21sh10:49:380
149960,0migration/015864-21sh10:41:200
149960,0migration/01320-21sh09:33:430
149960,0migration/01320-21sh09:33:430
149960,0migration/010100-21sh08:59:540
95029955,0cyclictest0-21swapper/210:01:272
95029950,0cyclictest0-21swapper/210:41:112
95019955,0cyclictest0-21swapper/112:07:001
95019955,0cyclictest0-21swapper/111:31:501
95019955,0cyclictest0-21swapper/111:04:241
95019955,0cyclictest0-21swapper/110:08:261
95019953,0cyclictest3867-21awk12:20:021
95019950,0cyclictest0-21swapper/109:03:011
95019950,0cyclictest0-21swapper/108:25:581
94999950,0cyclictest0-21swapper/011:56:400
4150,0ktimersoftd/00-21swapper/011:05:500
35150,0ktimersoftd/30-21swapper/311:21:453
339950,0migration/39413-21taskset08:58:373
339950,0migration/3577-21taskset08:28:183
339950,0migration/332713-21taskset10:53:383
339950,0migration/330070-21kthreadcore09:50:053
339950,0migration/328444-21needreboot07:05:083
339950,0migration/328090-21sh11:14:593
339950,0migration/328061-21munin-node08:50:003
339950,0migration/328059-21munin-node12:15:013
339950,0migration/328039-21munin-node11:34:593
339950,0migration/327472-21taskset11:54:523
339950,0migration/326574-21sh12:13:293
339950,0migration/325825-21taskset10:09:533
339950,0migration/319909-21kthreadcore10:05:063
339950,0migration/319155-21munin-node09:05:033
339950,0migration/318090-21taskset11:25:343
339950,0migration/315012-21munin-node06:55:073
339950,0migration/313345-21munin-node07:50:043
339950,0migration/311987-21latency08:10:063
339950,0migration/311330-21latency_hist07:24:493
339950,0migration/310932-21idleruntime09:40:033
339950,0migration/310842-21kthreadcore08:35:053
339950,0migration/310208-21taskset07:16:423
339950,0migration/310050-21taskset11:41:193
29725050,0irq/27-eth0-tx-9176-21ssh08:58:032
29725050,0irq/27-eth0-tx-3837-21cut10:35:052
29725050,0irq/27-eth0-tx-3588-21ssh11:39:592
29725050,0irq/27-eth0-tx-25974-21tune2fs10:50:012
29725050,0irq/27-eth0-tx-2346-21if_err_eth1.210:15:042
29725050,0irq/27-eth0-tx-2050-21ssh11:15:512
29725050,0irq/27-eth0-tx-19302-21munin-node12:10:072
29725050,0irq/27-eth0-tx-17860-21munin-node10:05:122
29725050,0irq/27-eth0-tx-16758-21ssh09:41:422
29725050,0irq/27-eth0-tx-0-21swapper/212:24:272
29725050,0irq/27-eth0-tx-0-21swapper/211:30:422
29725050,0irq/27-eth0-tx-0-21swapper/210:31:452
29725050,0irq/27-eth0-tx-0-21swapper/209:28:542
29725050,0irq/27-eth0-tx-0-21swapper/007:07:280
29715050,0irq/26-eth0-rx-8279-21taskset09:56:023
29715050,0irq/26-eth0-rx-6559-21runrttasks09:36:033
29715050,0irq/26-eth0-rx-3924-21aten_r1power_cu10:59:583
29715050,0irq/26-eth0-rx-29184-21cut09:10:063
29715050,0irq/26-eth0-rx-22810-21kthreadcore11:10:053
29715050,0irq/26-eth0-rx-19498-21df_inode07:55:023
29715050,0irq/26-eth0-rx-11987-21chrt11:04:573
29715050,0irq/26-eth0-rx-0-21swapper/312:22:383
29715050,0irq/26-eth0-rx-0-21swapper/311:50:053
29715050,0irq/26-eth0-rx-0-21swapper/311:33:593
29715050,0irq/26-eth0-rx-0-21swapper/310:39:093
29715050,0irq/26-eth0-rx-0-21swapper/310:18:053
29715050,0irq/26-eth0-rx-0-21swapper/309:20:583
29715050,0irq/26-eth0-rx-0-21swapper/309:00:023
27150,0ktimersoftd/20-21swapper/211:21:452
27150,0ktimersoftd/20-21swapper/211:05:502
259950,0migration/227877-21kthreadcore08:00:052
259950,0migration/223765-21needreboot09:05:072
259950,0migration/221006-21munin-node11:10:032
259950,0migration/218316-21munin-node10:45:022
259950,0migration/212433-21latency_hist07:49:492
259950,0migration/211342-21munin-node09:20:052
259950,0migration/210316-21runrttasks07:17:572
249950,0watchdog/26434-21kthreadcore09:55:062
249950,0watchdog/20-21swapper/207:45:032
19150,0ktimersoftd/10-21swapper/111:25:131
19150,0ktimersoftd/10-21swapper/111:23:291
179950,0migration/1835-21missed_timers11:35:061
179950,0migration/17822-21sh08:55:111
179950,0migration/16918-21kthreadcore09:55:061
179950,0migration/16461-21kthreadcore09:35:061
179950,0migration/132526-21sh09:50:551
179950,0migration/128406-21kthreadcore10:50:051
179950,0migration/127423-21irqcore09:30:041
179950,0migration/127423-21irqcore09:30:041
179950,0migration/120919-21latency_hist08:44:491
179950,0migration/116937-21taskset10:23:421
179950,0migration/11629-21aten2.4-expect10:15:011
169950,0watchdog/130921-21cut10:10:061
149950,0migration/09863-21sh12:01:340
149950,0migration/08785-21sh08:56:590
149950,0migration/07312-21munin-node08:10:020
149950,0migration/07194-21kthreadcore11:40:050
149950,0migration/06657-21kthreadcore10:35:060
149950,0migration/06110-21taskset08:05:180
149950,0migration/04560-21open_files07:40:090
149950,0migration/03854-21kthreadcore08:30:050
149950,0migration/032349-21kthreadcore11:15:050
149950,0migration/029966-21kthreadcore10:30:060
149950,0migration/026163-21munin-node10:50:020
149950,0migration/024903-21kthreadcore11:10:050
149950,0migration/02143-21sh11:36:510
149950,0migration/021209-21kthreadcore11:50:040
149950,0migration/02019-21latency_hist09:14:480
149950,0migration/019065-21kthreadcore07:00:060
149950,0migration/018568-21fschecks_count07:30:020
149950,0migration/018214-21latency_hist09:24:480
149950,0migration/017750-21taskset07:28:120
149950,0migration/016549-21taskset10:22:340
149950,0migration/010480-21munin-node06:55:020
95029943,0cyclictest8659-21awk08:35:032
95029943,0cyclictest3556-21lldpd10:55:092
95029943,0cyclictest3556-21lldpd09:35:092
95029943,0cyclictest3556-21lldpd09:31:102
95029943,0cyclictest3556-21lldpd09:31:102
95029943,0cyclictest3556-21lldpd09:15:092
95029943,0cyclictest3556-21lldpd08:46:092
95029943,0cyclictest3556-21lldpd08:30:392
95029943,0cyclictest3556-21lldpd08:13:382
95029943,0cyclictest3556-21lldpd07:35:372
95029943,0cyclictest3556-21lldpd06:56:372
95029943,0cyclictest3241-21irqbalance09:10:072
95029943,0cyclictest27716-21awk08:25:042
95029943,0cyclictest19139-21awk09:45:052
95029943,0cyclictest17219-21awk07:00:032
95019944,0cyclictest0-21swapper/111:17:581
95019943,0cyclictest4932-21awk07:15:031
95019943,0cyclictest3514-21snmpd11:10:521
95019943,0cyclictest3514-21snmpd07:57:361
95019943,0cyclictest3241-21irqbalance11:45:061
95019943,0cyclictest32306-21awk07:40:041
95019943,0cyclictest28600-21awk08:50:021
95019943,0cyclictest19331-21awk09:25:031
95019943,0cyclictest19186-21awk09:05:031
95019943,0cyclictest18967-21awk07:30:031
95019943,0cyclictest10969-21awk09:40:041
95019943,0cyclictest10671-21awk06:55:041
94999944,0cyclictest0-21swapper/011:29:050
94999943,0cyclictest4884-21awk07:15:030
94999943,0cyclictest3556-21lldpd09:45:100
94999943,0cyclictest3514-21snmpd08:01:210
94999943,0cyclictest3241-21irqbalance09:20:270
94999943,0cyclictest26984-21awk09:50:020
94999943,0cyclictest23174-21kworker/0:111:02:490
94999943,0cyclictest23174-21kworker/0:109:43:410
94999943,0cyclictest10743-21awk10:00:040
4140,0ktimersoftd/00-21swapper/010:56:500
339940,0migration/33708-21runrttasks07:14:233
339940,0migration/321354-21cpuspeed_turbos08:45:003
339940,0migration/318219-21aten_r1power_fr09:45:003
339940,0migration/315762-21kthreadcore07:25:063
29725040,0irq/27-eth0-tx-723-21processes11:55:062
29725040,0irq/27-eth0-tx-27275-21ssh12:14:512
29725040,0irq/27-eth0-tx-1719-21ssh11:35:362
29725040,0irq/27-eth0-tx-15154-21munin-node10:20:112
29725040,0irq/27-eth0-tx-0-21swapper/211:59:572
29725040,0irq/27-eth0-tx-0-21swapper/211:50:022
29725040,0irq/27-eth0-tx-0-21swapper/211:45:102
29725040,0irq/27-eth0-tx-0-21swapper/209:49:592
29725040,0irq/27-eth0-tx-0-21swapper/209:00:042
29725040,0irq/27-eth0-tx-0-21swapper/008:20:130
29725040,0irq/27-eth0-tx-0-21swapper/007:47:340
29715040,0irq/26-eth0-rx-0-21swapper/311:59:563
29715040,0irq/26-eth0-rx-0-21swapper/310:57:513
29715040,0irq/26-eth0-rx-0-21swapper/310:45:253
29715040,0irq/26-eth0-rx-0-21swapper/310:30:223
29715040,0irq/26-eth0-rx-0-21swapper/310:25:283
29715040,0irq/26-eth0-rx-0-21swapper/310:24:053
29715040,0irq/26-eth0-rx-0-21swapper/309:59:513
29715040,0irq/26-eth0-rx-0-21swapper/307:40:013
259940,0migration/231931-21kthreadcore08:50:052
259940,0migration/224904-21kthreadcore07:05:062
259940,0migration/222958-21kthreadcore07:30:072
259940,0migration/222333-21kthreadcore08:20:072
259940,0migration/221428-21kthreadcore07:55:062
259940,0migration/219291-21unixbench-2d08:15:112
259940,0migration/217812-21sshd07:28:552
259940,0migration/216548-21kthreadcore12:05:052
259940,0migration/215111-21kthreadcore11:25:052
259940,0migration/21187-21kthreadcore07:40:062
249940,0watchdog/222618-21kthreadcore10:25:062
249940,0watchdog/214963-21perl08:40:022
249940,0watchdog/21231-21kthreadcore07:10:072
249940,0watchdog/20-21swapper/210:10:062
179940,0migration/1412-21kthreadcore07:10:071
179940,0migration/131903-21kthreadcore12:15:061
179940,0migration/126482-21kthreadcore07:05:071
179940,0migration/126273-21runrttasks12:12:381
179940,0migration/12457-21taskset10:56:311
179940,0migration/122394-21kthreadcore08:20:071
179940,0migration/114624-21kthreadcore07:50:061
179940,0migration/112396-21sendmail_mailst08:10:101
179940,0migration/111962-21sh11:04:531
179940,0migration/110690-21latency_hist07:19:491
179940,0migration/110577-21fschecks_time10:00:021
169940,0watchdog/19665-21ssh11:40:141
169940,0watchdog/129270-21kthreadcore09:10:061
169940,0watchdog/123582-21ssh10:45:341
169940,0watchdog/112975-21kthreadcore09:20:061
169940,0watchdog/10-21swapper/112:00:021
169940,0watchdog/10-21swapper/110:24:581
169940,0watchdog/10-21swapper/109:45:021
169940,0watchdog/10-21swapper/107:45:031
149940,0migration/07747-21kthreadcore11:20:050
149940,0migration/05213-21kthreadcore09:55:060
149940,0migration/03164-21if_eth109:15:030
149940,0migration/029958-21cpuspeed_turbos07:10:020
149940,0migration/021729-21kthreadcore10:05:070
149940,0migration/017997-21kthreadcore08:15:060
95029932,0cyclictest3556-21lldpd07:23:372
95019933,0cyclictest22021-21awk08:45:031
95019932,0cyclictest4614-21cut08:30:061
95019932,0cyclictest3514-21snmpd07:37:361
95019932,0cyclictest28890-21kthreadcore11:55:021
95019932,0cyclictest19832-21ls08:19:481
95019932,0cyclictest17299-21awk07:00:041
95019932,0cyclictest13064-21grep07:25:051
95019932,0cyclictest11016-21kthreadcore10:40:051
94999933,0cyclictest23174-21kworker/0:112:19:530
94999933,0cyclictest23174-21kworker/0:112:14:490
94999933,0cyclictest23174-21kworker/0:111:44:510
94999933,0cyclictest23174-21kworker/0:110:24:590
94999933,0cyclictest23174-21kworker/0:110:14:510
94999933,0cyclictest23174-21kworker/0:110:09:570
94999933,0cyclictest23174-21kworker/0:109:34:510
94999933,0cyclictest23174-21kworker/0:108:45:000
94999933,0cyclictest23174-21kworker/0:108:40:000
94999933,0cyclictest23174-21kworker/0:108:35:040
94999933,0cyclictest23174-21kworker/0:108:24:500
94999933,0cyclictest23174-21kworker/0:107:54:560
94999933,0cyclictest23174-21kworker/0:107:50:000
94999933,0cyclictest23174-21kworker/0:107:34:580
35130,0ktimersoftd/30-21swapper/307:47:373
339930,0migration/324810-21aten_r1power_en07:35:003
29725030,0irq/27-eth0-tx-0-21swapper/008:49:580
29715030,0irq/26-eth0-rx-0-21swapper/308:29:593
29715030,0irq/26-eth0-rx-0-21swapper/308:16:283
29715030,0irq/26-eth0-rx-0-21swapper/308:00:003
29715030,0irq/26-eth0-rx-0-21swapper/307:30:013
29715030,0irq/26-eth0-rx-0-21swapper/307:00:103
259930,0migration/25870-21timerandwakeup08:05:092
179930,0migration/12642-21kthreadcore08:05:041
169930,0watchdog/13121-21munin-node09:15:021
169930,0watchdog/125947-21ssh11:51:021
169930,0watchdog/10-21swapper/110:35:021
169930,0watchdog/10-21swapper/108:00:111
149930,0migration/010711-21runrttasks07:19:560
95019921,0cyclictest7668-21sed08:34:491
339920,0migration/3338-21diskstats08:05:003
29715020,0irq/26-eth0-rx-0-21swapper/308:40:083
29715020,0irq/26-eth0-rx-0-21swapper/308:19:513
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional