You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-03 - 05:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Wed Jun 03, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3192823723,0sleep00-21swapper/018:51:490
304450350,0irq/25-eth10-21swapper/318:49:133
3186223425,0sleep20-21swapper/218:51:032
3180723425,0sleep10-21swapper/118:50:221
321729977,0cyclictest18791-21sensors21:54:091
339960,0migration/329654-21munin-node20:14:123
339960,0migration/327201-21sh22:40:263
339960,0migration/32580-21timerandwakeup22:04:123
321739960,0cyclictest0-21swapper/222:19:062
29725060,0irq/27-eth0-tx-0-21swapper/019:13:570
29715060,0irq/26-eth0-rx-20469-21diskstats23:39:013
27160,0ktimersoftd/212394-21diskmemload21:57:392
149960,0migration/027284-21sh22:40:410
149960,0migration/021025-21aten_r1power_en20:59:000
149960,0migration/020906-21sh21:18:050
8215050,0irq/24-0000:00:0-21swapper/221:13:482
339950,0migration/36759-21kthreadcore21:09:053
339950,0migration/3656-21gltestperf18:54:013
339950,0migration/35299-21munin-node23:29:033
339950,0migration/331546-21latency_hist20:43:483
339950,0migration/331508-21kthreadcore19:29:053
339950,0migration/331242-21taskset20:42:443
339950,0migration/329656-21munin-node23:04:043
339950,0migration/32899-21aten_r1power_cu19:33:583
339950,0migration/327868-21taskset00:03:513
339950,0migration/324883-21kthreadcore20:59:073
339950,0migration/324506-21taskset20:36:273
339950,0migration/322353-21kthreadcore23:59:073
339950,0migration/320969-21taskset19:17:183
339950,0migration/320189-21kthreadcore19:44:063
339950,0migration/32008-21kthreadcore21:24:063
339950,0migration/315877-21kthreadcore22:34:053
339950,0migration/315714-21kthreadcore20:29:073
339950,0migration/313057-21fschecks_time22:54:013
321739955,0cyclictest0-21swapper/222:04:452
321739954,0cyclictest6100-21awk21:29:032
321739950,0cyclictest0-21swapper/222:31:332
321739950,0cyclictest0-21swapper/222:16:532
321739950,0cyclictest0-21swapper/221:45:312
321729955,0cyclictest0-21swapper/123:02:381
321729955,0cyclictest0-21swapper/122:04:461
321729955,0cyclictest0-21swapper/121:10:511
321729954,0cyclictest4258-21ssh21:47:451
321719955,0cyclictest0-21swapper/021:58:520
321719955,0cyclictest0-21swapper/021:57:460
321719950,0cyclictest0-21swapper/023:46:140
29725050,0irq/27-eth0-tx-5350-21irqrtprio23:09:042
29725050,0irq/27-eth0-tx-3841-21ssh22:48:002
29725050,0irq/27-eth0-tx-28464-21munin-node23:04:042
29725050,0irq/27-eth0-tx-21731-21ssh21:59:012
29725050,0irq/27-eth0-tx-18994-21ssh22:34:572
29725050,0irq/27-eth0-tx-13612-21munin-node20:54:040
29725050,0irq/27-eth0-tx-0-21swapper/223:29:132
29725050,0irq/27-eth0-tx-0-21swapper/222:59:582
29725050,0irq/27-eth0-tx-0-21swapper/221:17:282
29725050,0irq/27-eth0-tx-0-21swapper/020:24:120
29725050,0irq/27-eth0-tx-0-21swapper/019:57:140
29715050,0irq/26-eth0-rx-9255-21munin-node00:09:083
29715050,0irq/26-eth0-rx-7919-21kthreadcore22:09:043
29715050,0irq/26-eth0-rx-5648-21cut22:29:043
29715050,0irq/26-eth0-rx-5189-21tr21:49:013
29715050,0irq/26-eth0-rx-28015-21ssh21:41:073
29715050,0irq/26-eth0-rx-26150-21kthreadcore21:19:053
29715050,0irq/26-eth0-rx-22363-21kthreadcore21:59:043
29715050,0irq/26-eth0-rx-20432-21ssh21:37:163
29715050,0irq/26-eth0-rx-2043-21pmu-power22:24:083
29715050,0irq/26-eth0-rx-17504-21port_sshpriv23:54:093
29715050,0irq/26-eth0-rx-14742-21cut23:14:053
29715050,0irq/26-eth0-rx-14307-21if_eth1.321:14:123
29715050,0irq/26-eth0-rx-1323-21munin-node23:24:063
29715050,0irq/26-eth0-rx-10572-21mdstat21:29:063
29715050,0irq/26-eth0-rx-0-21swapper/323:38:243
29715050,0irq/26-eth0-rx-0-21swapper/323:23:473
29715050,0irq/26-eth0-rx-0-21swapper/323:09:053
29715050,0irq/26-eth0-rx-0-21swapper/322:48:503
29715050,0irq/26-eth0-rx-0-21swapper/322:16:173
29715050,0irq/26-eth0-rx-0-21swapper/321:54:053
29715050,0irq/26-eth0-rx-0-21swapper/321:46:413
29715050,0irq/26-eth0-rx-0-21swapper/321:04:113
29715050,0irq/26-eth0-rx-0-21swapper/300:18:183
259950,0migration/27175-21kthreadcore22:09:042
259950,0migration/25684-21sh21:49:022
259950,0migration/228794-21sh21:22:352
259950,0migration/227876-21sh21:01:122
259950,0migration/22664-21sh23:46:322
259950,0migration/226416-21kthreadcore20:14:062
259950,0migration/226084-21unixbench_multi23:39:112
259950,0migration/22416-21munin-node21:04:092
259950,0migration/222674-21kthreadcore23:59:072
259950,0migration/215352-21fschecks_time19:13:582
259950,0migration/214357-21kthreadcore23:54:052
259950,0migration/213330-21fschecks_time20:54:032
259950,0migration/213197-21gltestperf22:54:012
249950,0watchdog/225975-21irqcore20:39:022
249950,0watchdog/215661-21cut23:34:062
179950,0migration/15814-21munin-node20:24:041
179950,0migration/15746-21irqrtprio22:29:041
179950,0migration/14527-21sh21:27:461
179950,0migration/117678-21kthreadcore19:44:051
179950,0migration/117352-21latency23:34:061
179950,0migration/110580-21taskset20:03:521
149950,0migration/08918-21kthreadcore23:49:100
149950,0migration/08523-21kthreadcore00:09:060
149950,0migration/04855-21munin-node22:08:580
149950,0migration/04436-21latency_hist22:28:480
149950,0migration/04354-21latency_hist22:28:480
149950,0migration/04348-21kthreadcore19:34:040
149950,0migration/03801-21sh23:08:350
149950,0migration/031812-21cpuspeed_turbos20:44:010
149950,0migration/03155-21unixbench_multi21:24:110
149950,0migration/030931-21kthreadcore19:29:040
149950,0migration/029656-21fschecks_time22:04:020
149950,0migration/02852-21unixbench-2d21:44:120
149950,0migration/028220-21latency_hist23:23:480
149950,0migration/026475-21unixbench_multi22:59:090
149950,0migration/026086-21kthreadcore19:24:020
149950,0migration/024807-21kthreadcore22:19:040
149950,0migration/024428-21kthreadcore23:39:050
149950,0migration/024266-21kthreadcore00:19:050
149950,0migration/023692-21gltestperf19:49:050
149950,0migration/020685-21df_abs23:18:590
149950,0migration/020340-21kthreadcore20:09:050
149950,0migration/019266-21sh23:37:180
149950,0migration/019141-21kthreadcore19:44:060
149950,0migration/018796-21sh22:34:230
149950,0migration/017799-21ntp_kernel_err23:14:070
149950,0migration/01712-21munin-node22:44:050
149950,0migration/016362-21munin-node21:34:060
149950,0migration/015253-21smart_sda19:39:110
149950,0migration/015079-21kthreadcore22:54:050
149950,0migration/012364-21taskset21:52:490
149950,0migration/012004-21df_abs20:29:020
149950,0migration/010973-21df_abs20:04:010
8215040,0irq/24-0000:00:0-21swapper/200:18:482
339940,0migration/3939-21kthreadcore19:54:053
339940,0migration/37489-21kthreadcore23:49:093
339940,0migration/3313-21kthreadcore20:19:053
339940,0migration/316720-21taskset20:05:013
321739943,0cyclictest807-21awk18:54:022
321739943,0cyclictest3514-21snmpd20:34:172
321739943,0cyclictest3514-21snmpd20:19:472
321739943,0cyclictest3514-21snmpd20:10:462
321739943,0cyclictest3514-21snmpd19:20:012
321739943,0cyclictest30444-21awk19:54:022
321739943,0cyclictest23832-21awk19:49:062
321739943,0cyclictest13161-21awk23:14:032
321729944,0cyclictest0-21swapper/123:28:201
321729944,0cyclictest0-21swapper/122:24:381
321729944,0cyclictest0-21swapper/121:03:271
321729943,0cyclictest4794-21awk23:49:071
321729943,0cyclictest3556-21lldpd22:18:441
321729943,0cyclictest3556-21lldpd19:29:131
321729943,0cyclictest3514-21snmpd19:51:461
321729943,0cyclictest3514-21snmpd19:21:461
321729943,0cyclictest3514-21snmpd19:14:311
321729943,0cyclictest3514-21snmpd19:09:011
321729943,0cyclictest3514-21snmpd19:06:311
321729943,0cyclictest3514-21snmpd19:01:311
321729943,0cyclictest32491-21awk20:44:041
321729943,0cyclictest3241-21irqbalance20:14:071
321729943,0cyclictest30570-21awk19:54:031
321729943,0cyclictest29541-21awk22:44:021
321729943,0cyclictest29415-21awk23:04:041
321729943,0cyclictest21473-21awk22:39:031
321729943,0cyclictest16013-21sadc19:43:481
321729943,0cyclictest13582-21awk22:34:031
321719944,0cyclictest0-21swapper/021:04:190
321719943,0cyclictest3514-21snmpd23:11:460
321719943,0cyclictest3514-21snmpd22:16:310
321719943,0cyclictest3241-21irqbalance21:30:460
321719943,0cyclictest30436-21kworker/0:219:21:580
321719943,0cyclictest22129-21awk21:39:020
321719943,0cyclictest21081-21kworker/0:023:56:390
29725040,0irq/27-eth0-tx-3785-21ssh23:48:572
29725040,0irq/27-eth0-tx-1528-21sed00:04:122
29725040,0irq/27-eth0-tx-0-21swapper/223:23:592
29725040,0irq/27-eth0-tx-0-21swapper/223:18:512
29725040,0irq/27-eth0-tx-0-21swapper/222:38:572
29725040,0irq/27-eth0-tx-0-21swapper/222:28:042
29725040,0irq/27-eth0-tx-0-21swapper/221:34:302
29725040,0irq/27-eth0-tx-0-21swapper/221:24:022
29725040,0irq/27-eth0-tx-0-21swapper/200:14:052
29725040,0irq/27-eth0-tx-0-21swapper/020:49:030
29715040,0irq/26-eth0-rx-0-21swapper/323:43:493
29715040,0irq/26-eth0-rx-0-21swapper/322:59:043
29715040,0irq/26-eth0-rx-0-21swapper/322:46:543
29715040,0irq/26-eth0-rx-0-21swapper/322:19:133
29715040,0irq/26-eth0-rx-0-21swapper/320:55:043
29715040,0irq/26-eth0-rx-0-21swapper/319:49:033
29715040,0irq/26-eth0-rx-0-21swapper/300:21:163
259940,0migration/222711-21kthreadcore21:39:042
259940,0migration/216359-21aten_r1power_po19:44:002
259940,0migration/213201-21munin-node19:04:062
259940,0migration/212520-21kthreadcore19:39:062
259940,0migration/211585-21munin-node20:49:102
259940,0migration/210953-21sh22:50:192
259940,0migration/210701-21switchtime20:24:112
259940,0migration/210587-21runrttasks20:03:572
249940,0watchdog/27876-21kthreadcore00:09:062
249940,0watchdog/26559-21runrttasks19:03:152
249940,0watchdog/232057-21fschecks_count20:44:022
249940,0watchdog/229566-21if_eth119:29:032
249940,0watchdog/215272-21kthreadcore20:29:062
249940,0watchdog/20-21swapper/219:59:072
179940,0migration/16525-21kthreadcore19:34:041
179940,0migration/16262-21taskset18:56:311
179940,0migration/15783-21kthreadcore22:49:061
179940,0migration/15755-21kthreadcore00:09:051
179940,0migration/15421-21kthreadcore23:29:031
179940,0migration/15150-21munin-node22:08:591
179940,0migration/130686-21kthreadcore00:04:071
179940,0migration/128002-21runrttasks19:24:141
179940,0migration/127838-21kthreadcore20:39:031
179940,0migration/126986-21smart_sda21:39:101
179940,0migration/122216-21kthreadcore22:19:031
179940,0migration/121904-21kthreadcore23:39:051
179940,0migration/121803-21kthreadcore23:59:061
179940,0migration/120875-21fschecks_time23:19:001
179940,0migration/119745-21kthreadcore20:09:051
179940,0migration/117816-21runrttasks20:31:191
179940,0migration/115972-21kthreadcore21:14:141
179940,0migration/113184-21sh23:14:031
179940,0migration/112664-21idleruntime23:54:031
179940,0migration/112181-21taskset20:51:311
179940,0migration/111347-21sshd21:29:151
169940,0watchdog/19601-21python19:59:111
169940,0watchdog/127783-21ssh21:19:421
169940,0watchdog/118605-21ssh22:54:181
169940,0watchdog/116440-21kthreadcore20:54:061
169940,0watchdog/110495-21ssh23:09:421
169940,0watchdog/110049-21kthreadcore21:49:061
169940,0watchdog/10-21swapper/123:44:021
169940,0watchdog/10-21swapper/121:34:061
149940,0migration/02672-21sh00:07:000
149940,0migration/012680-21missed_timers19:04:020
149940,0migration/012583-21sh21:12:320
149940,0migration/012121-21df_abs00:14:010
321739932,0cyclictest25593-21kthreadcore19:24:022
321719933,0cyclictest30436-21kworker/0:221:18:490
321719933,0cyclictest30436-21kworker/0:220:39:120
321719933,0cyclictest30436-21kworker/0:220:33:580
321719933,0cyclictest30436-21kworker/0:220:13:560
321719933,0cyclictest30436-21kworker/0:219:59:060
321719933,0cyclictest30436-21kworker/0:219:08:520
321719933,0cyclictest30436-21kworker/0:218:59:040
321719933,0cyclictest21081-21kworker/0:023:59:070
321719933,0cyclictest21081-21kworker/0:023:28:550
321719932,0cyclictest4390-21ssh22:48:540
321719930,0cyclictest30988-21diskstats20:19:010
29725030,0irq/27-eth0-tx-0-21swapper/018:53:580
29715030,0irq/26-eth0-rx-0-21swapper/320:25:153
29715030,0irq/26-eth0-rx-0-21swapper/320:08:593
29715030,0irq/26-eth0-rx-0-21swapper/320:01:013
29715030,0irq/26-eth0-rx-0-21swapper/319:39:043
29715030,0irq/26-eth0-rx-0-21swapper/319:23:563
29715030,0irq/26-eth0-rx-0-21swapper/319:21:423
29715030,0irq/26-eth0-rx-0-21swapper/319:03:563
29715030,0irq/26-eth0-rx-0-21swapper/318:59:283
169930,0watchdog/132368-21sh21:04:061
169930,0watchdog/10-21swapper/120:34:061
169930,0watchdog/10-21swapper/100:19:011
169930,0watchdog/10-21swapper/100:14:021
35120,0ktimersoftd/38321-21kthreadcore20:49:063
321739920,0cyclictest0-21swapper/219:09:022
321729921,0cyclictest20946-21latency_hist21:58:481
29715020,0irq/26-eth0-rx-0-21swapper/319:08:523
249920,0watchdog/20-21swapper/219:33:512
169920,0watchdog/10-21swapper/120:18:541
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional