You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-08 - 03:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Tue Sep 08, 2026 00:43:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
32909910531050,3cyclictest1784-21kworker/0:3+events23:26:380
32979910521049,3cyclictest1118-21turbostat20:29:552
33019910491045,2cyclictest7962-21turbostat00:39:563
32949910471045,2cyclictest19335-21turbostat21:44:541
33019910451044,1cyclictest19205-21kworker/3:0+events20:03:593
32949910431041,2cyclictest15609-21turbostat23:44:561
32979910421039,3cyclictest30345-21kworker/2:0+events20:20:202
32949910421041,1cyclictest8880-21kworker/1:0+events21:57:351
32979910401038,2cyclictest30345-21kworker/2:0+events20:16:062
32979910391037,2cyclictest19942-21kworker/2:2+events00:15:442
32949910371036,1cyclictest8880-21kworker/1:0+events21:36:051
33019910361034,2cyclictest12914-21turbostat22:09:553
32979910341032,2cyclictest8247-21kworker/2:0+events22:01:272
32909910321029,3cyclictest24749-21turbostat00:24:560
33019910281024,3cyclictest10346-21kworker/3:2+events23:38:453
33019910261024,2cyclictest15961-21kworker/3:1+events23:42:563
33019910251023,2cyclictest3373-21turbostat21:29:553
33019910251022,3cyclictest7706-21turbostat22:04:553
32949910251023,2cyclictest28489-21turbostat22:24:551
33019910231022,1cyclictest19205-21kworker/3:0+events20:06:493
33019910221020,2cyclictest6459-21turbostat22:34:543
33019910221020,2cyclictest11644-21turbostat22:39:553
32979910211019,2cyclictest30575-21turbostat21:24:542
32909910201018,2cyclictest24006-21kworker/0:1+events22:57:360
32949910191017,2cyclictest25156-21kworker/1:2+events00:22:321
32949910181016,2cyclictest27295-21kworker/1:2+events22:08:501
32909910181016,2cyclictest31177-21turbostat23:59:560
32949910171015,2cyclictest28931-21kworker/1:2+events21:30:201
32949910171015,2cyclictest20116-21kworker/1:0+events21:14:001
32949910171015,2cyclictest12828-21kworker/1:1+events22:38:151
32949910161014,2cyclictest20116-21kworker/1:0+events21:16:061
3301991000998,2cyclictest27443-21turbostat20:14:553
329499998996,2cyclictest8880-21kworker/1:0+events21:46:041
329499998996,2cyclictest14382-21turbostat00:14:561
329499993992,1cyclictest27295-21kworker/1:2+events21:51:251
329099993992,1cyclictest30944-21kworker/0:2+events21:21:260
329099991990,1cyclictest1784-21kworker/0:3+events23:32:120
329099988987,1cyclictest30944-21kworker/0:2+events22:10:210
329099982981,1cyclictest24006-21kworker/0:1+events23:09:250
329099977976,1cyclictest1007-21kworker/0:0+events23:12:460
330199976975,1cyclictest15961-21kworker/3:1+events23:55:233
329099975973,2cyclictest5156-21turbostat20:39:550
329099974973,1cyclictest17235-21kworker/0:0+events22:40:440
329099971970,1cyclictest25276-21kworker/0:1+events22:19:350
329499965962,3cyclictest23272-21turbostat22:19:541
330199963962,1cyclictest9931-21kworker/3:0+events00:05:433
329099959957,2cyclictest1007-21kworker/0:0+events23:15:430
329099958957,1cyclictest30944-21kworker/0:2+events22:46:250
330199949948,1cyclictest15961-21kworker/3:1+events23:54:283
330199947945,2cyclictest2503-21turbostat21:59:543
330199946945,1cyclictest29897-21kworker/3:2+events00:26:273
330199946943,2cyclictest15961-21kworker/3:1+events00:10:443
329799942940,2cyclictest29969-21turbostat00:29:552
330199931929,2cyclictest28391-21kworker/3:0+events22:26:073
329099931928,3cyclictest19378-21turbostat19:54:560
330199929928,1cyclictest21262-21kworker/3:2+events19:56:523
330199926925,1cyclictest15561-21kworker/3:2+events21:01:143
330199926924,2cyclictest8737-21kworker/3:1+events22:51:503
329799925923,2cyclictest5239-21turbostat23:34:552
330199924924,0cyclictest15961-21kworker/3:1+events00:02:593
330199921920,1cyclictest28391-21kworker/3:0+events22:20:453
330199919918,1cyclictest30559-21kworker/3:2+events19:35:443
330199918917,1cyclictest9257-21kworker/3:1+events20:42:453
330199918917,1cyclictest3111-21kworker/3:0+events20:45:443
329099918916,2cyclictest15680-21turbostat21:04:540
330199915914,1cyclictest15961-21kworker/3:1+events00:32:143
330199915913,2cyclictest25363-21turbostat21:19:543
330199914913,1cyclictest30559-21kworker/3:2+events19:22:343
330199913912,1cyclictest3111-21kworker/3:0+events20:35:523
330199913912,1cyclictest19205-21kworker/3:0+events19:50:233
330199912911,1cyclictest30559-21kworker/3:2+events19:45:453
330199911910,1cyclictest15561-21kworker/3:2+events20:59:243
330199910910,0cyclictest30559-21kworker/3:2+events19:26:483
330199910909,1cyclictest9257-21kworker/3:1+events20:54:253
330199910909,1cyclictest4879-21kworker/3:0+events19:16:513
330199910909,1cyclictest11069-21kworker/3:1+events19:32:163
329099909908,1cyclictest24942-21kworker/0:1+events23:21:060
330199905904,1cyclictest30559-21kworker/3:2+events19:10:223
330199905904,1cyclictest15961-21kworker/3:1+events23:45:223
330199905904,1cyclictest11069-21kworker/3:1+events19:42:563
330199902899,1cyclictest3111-21kworker/3:0+events20:33:413
329099902901,1cyclictest24006-21kworker/0:1+events23:04:250
329099902901,1cyclictest17622-21kworker/0:0+events21:08:170
330199900898,2cyclictest17700-21turbostat21:09:553
329799893889,4cyclictest25428-21turbostat20:09:542
330199879876,3cyclictest19564-21turbostat00:19:563
329099871868,3cyclictest1274-21turbostat22:29:550
329799866863,3cyclictest7185-21turbostat20:44:542
329799686683,3cyclictest27443-21turbostat20:14:552
329499679677,2cyclictest3373-21turbostat21:29:551
329499678675,3cyclictest7706-21turbostat22:04:541
329499676674,2cyclictest6459-21turbostat22:34:551
330199674673,1cyclictest1118-21turbostat20:29:553
329099661656,4cyclictest25363-21turbostat21:19:550
330199647643,4cyclictest30575-21turbostat21:24:543
329799632630,2cyclictest2503-21turbostat21:59:542
329099632629,3cyclictest27222-21turbostat22:54:560
330199618617,1cyclictest23272-21turbostat22:19:543
330199599595,4cyclictest24749-21turbostat00:24:563
329799585582,3cyclictest14382-21turbostat00:14:562
329099579577,2cyclictest12914-21turbostat22:09:550
329099566563,3cyclictest11644-21turbostat22:39:540
329499552551,1cyclictest17700-21turbostat21:09:551
330199550547,3cyclictest5239-21turbostat23:34:553
329499530529,1cyclictest19564-21turbostat00:19:561
330199499497,2cyclictest22026-21turbostat22:49:553
3294993938,1cyclictest25498-21fwupd22:47:371
3301991918,1cyclictest67750irq/123-lan23:26:213
3301991918,1cyclictest67750irq/123-lan23:20:333
3301991918,1cyclictest67750irq/123-lan22:13:503
3301991918,1cyclictest67750irq/123-lan21:54:103
3301991918,1cyclictest67750irq/123-lan21:32:493
3301991818,0cyclictest67750irq/123-lan23:17:133
3301991818,0cyclictest67750irq/123-lan23:05:163
3301991817,1cyclictest67750irq/123-lan22:40:133
3301991817,1cyclictest67750irq/123-lan21:40:233
3301991717,0cyclictest67750irq/123-lan23:12:123
3301991717,0cyclictest67750irq/123-lan21:36:003
3301991716,1cyclictest67750irq/123-lan21:14:243
3301991616,0cyclictest67750irq/123-lan22:59:263
3301991615,1cyclictest67750irq/123-lan23:00:173
3301991515,0cyclictest67750irq/123-lan20:23:163
3301991514,1cyclictest67750irq/123-lan21:45:173
3301991514,1cyclictest67750irq/123-lan20:17:143
329499108,2cyclictest782-21lldpd00:37:361
329499100,2cyclictest5766-21grep20:35:151
32979998,1cyclictest784-21snmpd21:36:262
32979998,1cyclictest507-21systemd-resolve22:10:372
32949997,2cyclictest784-21snmpd20:51:491
32949997,2cyclictest784-21snmpd20:24:351
32949996,3cyclictest784-21snmpd22:52:041
32949996,3cyclictest27092-21needreboot23:50:191
32949995,4cyclictest1830-21cat20:25:141
32949990,2cyclictest24782-21ssh23:16:361
32949990,2cyclictest15603-21sh23:08:521
32909998,1cyclictest507-21systemd-resolve22:34:490
32979988,0cyclictest784-21snmpd21:50:092
32979988,0cyclictest784-21snmpd21:09:052
32979988,0cyclictest782-21lldpd23:45:512
32979988,0cyclictest782-21lldpd20:01:072
32979987,1cyclictest782-21lldpd22:52:352
32979987,1cyclictest526-21avahi-daemon21:25:522
32979987,1cyclictest24488-21ntp_kernel_pll_22:15:162
32979986,2cyclictest784-21snmpd19:47:582
32949988,0cyclictest784-21snmpd19:51:021
32949988,0cyclictest782-21lldpd23:10:421
32949988,0cyclictest782-21lldpd22:41:441
32949988,0cyclictest782-21lldpd00:25:491
32949987,1cyclictest782-21lldpd19:39:481
32949983,5cyclictest8309-21cat20:40:141
32949983,5cyclictest30282-21memory20:15:161
32949982,5cyclictest11492-21cat19:30:001
32949981,7cyclictest547-21NetworkManager20:31:051
32949980,2cyclictest20135-21diskmemload22:11:151
32949980,2cyclictest10703-21wc00:05:181
32949980,1cyclictest31698-21ssh23:24:001
32909988,0cyclictest784-21snmpd21:43:560
32909988,0cyclictest784-21snmpd21:29:570
32909988,0cyclictest784-21snmpd19:35:010
32909988,0cyclictest784-21snmpd00:31:480
32909987,1cyclictest784-21snmpd19:22:550
32909987,1cyclictest784-21snmpd19:11:270
32909987,1cyclictest507-21systemd-resolve23:35:420
32909987,1cyclictest11630-21cpuspeed_turbos19:30:130
32909986,2cyclictest784-21snmpd20:21:350
32909986,2cyclictest784-21snmpd00:14:310
32979977,0cyclictest784-21snmpd19:55:262
32979977,0cyclictest784-21snmpd00:37:062
32979977,0cyclictest784-21snmpd00:22:022
32979977,0cyclictest782-21lldpd22:42:502
32979977,0cyclictest782-21lldpd19:53:182
32979977,0cyclictest782-21lldpd19:15:112
32979976,1cyclictest784-21snmpd21:41:152
32979976,1cyclictest784-21snmpd20:57:152
32979976,1cyclictest5494-21cpuspeed_turbos20:35:132
32979976,1cyclictest526-21avahi-daemon23:52:482
32979976,1cyclictest507-21systemd-resolve23:10:412
32979976,1cyclictest245-20systemd-journal22:25:092
32979976,1cyclictest12279-21fschecks_time22:35:132
32979975,2cyclictest784-21snmpd23:04:152
32949977,0cyclictest784-21snmpd19:32:541
32949977,0cyclictest782-21lldpd22:58:051
32949975,2cyclictest784-21snmpd20:55:371
32949975,2cyclictest0-21swapper/100:00:021
32949974,3cyclictest0-21swapper/123:25:131
32949970,4cyclictest2410-21ntpq22:25:171
32949970,4cyclictest16343-21hddtemp_smartct21:00:151
32949970,3cyclictest7211-21ntpq23:00:161
32949970,3cyclictest6080-21irqstats23:30:131
32949970,3cyclictest3760-21perl19:10:131
32949970,3cyclictest32722-21switchtime23:55:201
32949970,3cyclictest25890-21grep20:05:131
32949970,3cyclictest23966-21fw_conntrack20:00:121
32949970,3cyclictest21157-21cat23:45:121
32949970,3cyclictest1845-21ssh21:22:531
32949970,2cyclictest7433-21cat19:20:011
32949970,2cyclictest16482-21nvmesmart_nvme019:40:151
32949970,0cyclictest230322sleep119:55:411
32909977,0cyclictest784-21snmpd21:48:060
32909977,0cyclictest784-21snmpd20:54:550
32909976,1cyclictest784-21snmpd22:20:010
32909976,1cyclictest784-21snmpd00:37:000
32909976,1cyclictest784-21snmpd00:06:140
32909976,1cyclictest526-21avahi-daemon20:30:010
32909976,1cyclictest4721-21gltestperf00:00:140
32909975,2cyclictest784-21snmpd21:55:390
32909975,2cyclictest784-21snmpd19:16:160
32909972,5cyclictest547-21NetworkManager21:50:220
32979966,0cyclictest9858-21fschecks_time00:05:112
32979966,0cyclictest784-21snmpd21:16:312
32979966,0cyclictest784-21snmpd19:11:572
32979966,0cyclictest784-21snmpd00:30:092
32979965,1cyclictest784-21snmpd20:50:492
32979965,1cyclictest784-21snmpd19:35:532
32979965,1cyclictest526-21avahi-daemon23:15:282
32979965,1cyclictest507-21systemd-resolve23:27:122
32979965,1cyclictest17422-21unixbench_singl23:40:222
32979963,3cyclictest9272-21perl21:30:172
32979963,2cyclictest6661-21ssh00:01:282
32979963,2cyclictest245-20systemd-journal22:23:192
32979963,2cyclictest17937-21ssh22:09:492
32979963,2cyclictest10789-21ssh22:33:492
32979962,3cyclictest8744-21sed19:20:202
32979962,3cyclictest31252-21sh23:23:232
32979962,3cyclictest27199-21ssh21:46:312
32979962,3cyclictest23585-21sed22:45:212
32979962,3cyclictest20853-21fw_forwarded_lo21:10:142
32979962,2cyclictest12425-21grep23:05:172
32949965,1cyclictest10868-21cpuspeed_turbos23:35:111
32949960,3cyclictest7816-21fw_conntrack19:20:121
32949960,3cyclictest10340-21interrupts20:45:141
32949960,2cyclictest3337-21grep00:30:151
32949960,2cyclictest18004-21/usr/sbin/munin19:45:141
32949960,2cyclictest0-21swapper/120:10:211
32909966,0cyclictest782-21lldpd22:00:110
32909966,0cyclictest782-21lldpd20:15:220
32909966,0cyclictest782-21lldpd20:13:480
32909966,0cyclictest782-21lldpd19:45:430
32909965,1cyclictest784-21snmpd19:25:150
32909965,1cyclictest762-21ntpd23:43:020
32909965,1cyclictest526-21avahi-daemon00:16:380
32909962,3cyclictest24403-21ssh21:13:420
32979955,0cyclictest526-21avahi-daemon19:41:562
32979954,1cyclictest10868-21cpuspeed_turbos23:35:112
32979954,1cyclictest0-21swapper/219:30:222
32979953,2cyclictest9968-21cat19:25:132
32979953,2cyclictest3855-21cat20:30:142
32979952,2cyclictest574-21rs:main2
32979952,2cyclictest1204-21rm23:56:092
32979952,2cyclictest11485-21sshd20:48:152
32979951,3cyclictest3353-21ssh22:56:512
32909955,0cyclictest784-21snmpd20:26:170
32909953,2cyclictest784-21snmpd20:02:190
32909953,2cyclictest547-21NetworkManager23:50:110
32909952,3cyclictest547-21NetworkManager21:33:020
32909952,2cyclictest30351-21dpkg00:25:110
32909951,3cyclictest22987-21awk19:55:220
32909950,4cyclictest26503-21ntpq20:05:170
32909943,1cyclictest791-21runrttasks20:45:100
32909942,2cyclictest8473-21wc20:40:150
32909941,3cyclictest16350-21ld19:40:150
32909941,2cyclictest15071-21cat21:35:160
32909941,2cyclictest15039-21cat20:55:200
32909940,4cyclictest21978-21ntpq23:45:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional