You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-29 - 09:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Sat Aug 29, 2026 00:43:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
286739910531050,3cyclictest1736-21kworker/3:2+events23:02:453
286739910531050,2cyclictest15173-21kworker/3:2+events21:17:533
286739910521050,2cyclictest28605-21kworker/3:1+events23:05:513
286739910521050,2cyclictest16617-21kworker/3:0+events21:07:223
286739910511049,2cyclictest28605-21kworker/3:1+events23:16:253
286739910511049,2cyclictest16617-21kworker/3:0+events21:02:583
286739910511049,2cyclictest14568-21kworker/3:1+events19:59:333
286739910511049,2cyclictest14568-21kworker/3:1+events19:53:413
286739910501048,2cyclictest4955-21kworker/3:2+events19:48:463
286739910501048,2cyclictest25162-21kworker/3:2+events00:38:473
286739910491047,2cyclictest6589-21kworker/3:1+events20:55:443
286739910491047,2cyclictest4955-21kworker/3:2+events19:38:443
286739910491047,2cyclictest21775-21kworker/3:2+events22:53:383
286679910491047,2cyclictest19172-21turbostat23:54:552
286739910481046,2cyclictest8762-21turbostat23:44:543
286739910471045,2cyclictest11654-21kworker/3:2+events22:19:343
286739910461045,1cyclictest11654-21kworker/3:2+events22:14:243
286739910451043,2cyclictest4955-21kworker/3:2+events19:43:053
286739910451043,2cyclictest11244-21kworker/3:0+events19:26:243
286739910431041,2cyclictest25162-21kworker/3:2+events00:21:023
286639910401038,2cyclictest32625-21turbostat19:24:551
286739910341033,1cyclictest30491-21kworker/3:1+events00:33:233
286639910321031,1cyclictest6275-21kworker/1:1+events23:37:361
286739910311029,2cyclictest28928-21turbostat20:34:553
286639910291028,1cyclictest6275-21kworker/1:1+events23:29:491
286599910291027,2cyclictest29597-21turbostat00:04:540
286739910281025,3cyclictest30933-21turbostat20:39:553
286639910281026,2cyclictest4597-21turbostat20:54:551
286639910261025,1cyclictest30913-21kworker/1:2+events20:44:461
286739910241022,2cyclictest17493-21kworker/3:1+events22:37:233
286599910231021,2cyclictest31626-21kworker/0:1+events00:11:270
286739910211020,1cyclictest7314-21kworker/3:0+events23:14:343
286599910211020,1cyclictest28299-21kworker/0:0+events20:10:220
286599910211019,2cyclictest3568-21kworker/0:2+events22:08:370
286639910191018,1cyclictest5820-21kworker/1:2+events23:52:121
286639910101009,1cyclictest30527-21kworker/1:2+events19:15:451
286639910071006,1cyclictest30913-21kworker/1:2+events20:37:101
286639910061005,1cyclictest30527-21kworker/1:2+events19:14:441
286639910061005,1cyclictest22482-21kworker/1:0+events00:06:501
286639910051004,1cyclictest6275-21kworker/1:1+events23:34:261
286679910041002,1cyclictest7180-21kworker/2:0+events21:46:082
286639910031002,1cyclictest6275-21kworker/1:1+events23:48:411
28673991000999,1cyclictest15173-21kworker/3:2+events21:13:163
28663991000999,1cyclictest21817-21kworker/1:1+events20:03:171
286639910001000,0cyclictest6275-21kworker/1:1+events23:43:271
2866799998997,1cyclictest24398-21kworker/2:2+events21:38:172
2866399998997,1cyclictest21817-21kworker/1:1+events20:09:131
2866799995994,1cyclictest7180-21kworker/2:0+events22:29:332
2866399995994,1cyclictest30913-21kworker/1:2+events20:46:581
2866799994992,1cyclictest12834-21turbostat00:19:552
2865999994993,1cyclictest28299-21kworker/0:0+events20:16:370
2866799993992,1cyclictest24398-21kworker/2:2+events21:40:212
2867399990988,2cyclictest25162-21kworker/3:2+events00:29:543
2865999990989,1cyclictest28299-21kworker/0:0+events21:32:430
2866799988987,1cyclictest24398-21kworker/2:2+events21:21:482
2867399987986,1cyclictest17493-21kworker/3:1+events22:47:093
2867399987983,4cyclictest28350-21turbostat21:29:553
2866799985984,1cyclictest22708-21kworker/2:1+events22:21:462
2867399974973,1cyclictest4955-21kworker/3:2+events19:34:113
2867399972971,1cyclictest28605-21kworker/3:1+events22:59:313
2867399971969,2cyclictest18817-21turbostat20:09:553
2866799970967,3cyclictest10663-21turbostat21:09:552
2866399966964,2cyclictest10752-21turbostat19:49:561
2865999966964,2cyclictest30562-21kworker/0:2+events20:33:070
2865999957956,1cyclictest29167-21kworker/0:1+events22:03:570
2865999956954,1cyclictest29167-21kworker/0:1+events21:53:150
2865999951950,1cyclictest29167-21kworker/0:1+events21:29:220
2865999948947,1cyclictest32490-21kworker/0:0+events22:31:260
2865999948947,1cyclictest24518-21kworker/0:1+events20:28:060
2865999943942,1cyclictest31626-21kworker/0:1+events00:16:500
2865999942941,1cyclictest28299-21kworker/0:0+events20:21:070
2865999942939,2cyclictest32490-21kworker/0:0+events23:58:040
2865999941939,2cyclictest14773-21turbostat19:59:550
2865999937936,1cyclictest29167-21kworker/0:1+events21:59:220
2865999901899,2cyclictest11014-21turbostat22:44:550
2867399900899,1cyclictest4369-21kworker/3:0+events22:44:113
2867399893892,1cyclictest28605-21kworker/3:1+events23:20:323
2867399886884,2cyclictest1599-21turbostat21:34:553
2865999874872,2cyclictest20285-21turbostat23:24:540
2867399867865,2cyclictest17643-21turbostat21:49:553
2867399866864,2cyclictest32577-21turbostat22:34:543
2867399856854,2cyclictest27366-21turbostat22:29:543
2866399695692,3cyclictest28928-21turbostat20:34:551
2866399690687,3cyclictest20285-21turbostat23:24:541
2867399687684,3cyclictest32625-21turbostat19:24:553
2867399675672,3cyclictest4597-21turbostat20:54:553
2865999669668,1cyclictest17643-21turbostat21:49:550
2865999658657,1cyclictest27366-21turbostat22:29:540
2867399651648,3cyclictest12834-21turbostat00:19:553
2866399646643,3cyclictest29597-21turbostat00:04:541
2867399635633,2cyclictest6527-21turbostat22:09:553
2865999607605,1cyclictest2428-21turbostat00:09:540
2866799594591,3cyclictest17897-21turbostat21:19:562
2865999573572,1cyclictest18817-21turbostat20:09:560
2866399558554,4cyclictest14773-21turbostat19:59:551
2865999557553,4cyclictest19172-21turbostat23:54:550
2866799537536,1cyclictest1599-21turbostat21:34:552
2866799535532,3cyclictest16910-21turbostat22:19:552
2865999435434,1cyclictest23125-21turbostat21:24:560
28673992525,0cyclictest25607-21fwupd21:51:023
28663992019,1cyclictest67750irq/123-lan22:46:111
28663991919,0cyclictest67750irq/123-lan21:10:121
28663991919,0cyclictest6275-21kworker/1:1+events_freezable21:58:151
28663991918,1cyclictest67750irq/123-lan23:55:211
28663991918,1cyclictest67750irq/123-lan22:05:131
28663991918,1cyclictest67750irq/123-lan21:20:141
28663991918,1cyclictest6275-21kworker/1:1+events_freezable22:36:441
28663991918,1cyclictest6275-21kworker/1:1+events21:43:141
28663991818,0cyclictest67750irq/123-lan22:25:191
28663991818,0cyclictest67750irq/123-lan22:17:311
28663991818,0cyclictest67750irq/123-lan22:10:451
28663991818,0cyclictest67750irq/123-lan00:15:261
28663991817,1cyclictest67750irq/123-lan23:05:151
28663991817,1cyclictest67750irq/123-lan23:00:131
28663991817,1cyclictest67750irq/123-lan21:53:171
28663991817,1cyclictest67750irq/123-lan21:45:201
28663991817,1cyclictest67750irq/123-lan21:36:311
28663991817,1cyclictest67750irq/123-lan21:25:201
28663991817,1cyclictest67750irq/123-lan21:18:461
28663991817,1cyclictest6275-21kworker/1:1+events_freezable22:33:111
28663991717,0cyclictest67750irq/123-lan23:17:331
28663991717,0cyclictest67750irq/123-lan22:53:371
28663991717,0cyclictest67750irq/123-lan22:02:361
28663991717,0cyclictest67750irq/123-lan21:00:171
28663991717,0cyclictest67750irq/123-lan20:55:221
28663991717,0cyclictest67750irq/123-lan20:25:521
28663991717,0cyclictest67750irq/123-lan20:15:181
28663991717,0cyclictest67750irq/123-lan19:35:131
28663991717,0cyclictest67750irq/123-lan00:37:411
28663991717,0cyclictest67750irq/123-lan00:32:321
28663991717,0cyclictest67750irq/123-lan00:26:221
28663991717,0cyclictest67750irq/123-lan00:20:361
28663991716,1cyclictest67750irq/123-lan22:20:121
28663991716,1cyclictest67750irq/123-lan19:50:201
28663991716,1cyclictest67750irq/123-lan00:13:291
28663991616,0cyclictest67750irq/123-lan22:43:431
28663991615,1cyclictest67750irq/123-lan23:10:141
28663991615,1cyclictest67750irq/123-lan22:59:331
28663991515,0cyclictest67750irq/123-lan21:05:111
28663991515,0cyclictest67750irq/123-lan20:20:181
28663991515,0cyclictest67750irq/123-lan20:10:201
28663991515,0cyclictest67750irq/123-lan19:28:281
28663991514,1cyclictest67750irq/123-lan21:30:171
28663991414,0cyclictest67750irq/123-lan19:40:401
28667991313,0cyclictest784-21snmpd21:51:482
28667991313,0cyclictest784-21snmpd00:33:162
28667991311,1cyclictest784-21snmpd19:14:022
28663991313,0cyclictest21817-21kworker/1:1+events_freezable19:30:131
28667991212,0cyclictest784-21snmpd22:49:052
28667991212,0cyclictest784-21snmpd21:58:012
28667991212,0cyclictest784-21snmpd19:16:412
28667991110,0cyclictest784-21snmpd23:11:312
2866799109,1cyclictest784-21snmpd20:29:432
28667991010,0cyclictest784-21snmpd23:00:292
286739998,1cyclictest784-21snmpd23:48:473
286679999,0cyclictest784-21snmpd22:01:182
286679999,0cyclictest784-21snmpd21:10:112
286679999,0cyclictest784-21snmpd20:40:292
286679999,0cyclictest784-21snmpd20:36:132
286679999,0cyclictest784-21snmpd19:31:472
286679999,0cyclictest784-21snmpd19:28:562
286679998,1cyclictest784-21snmpd23:42:102
286679998,1cyclictest784-21snmpd19:38:202
286679998,1cyclictest784-21snmpd00:25:152
286739988,0cyclictest782-21lldpd20:02:583
286739988,0cyclictest782-21lldpd00:01:173
286679988,0cyclictest784-21snmpd23:55:012
286679988,0cyclictest784-21snmpd23:38:582
286679988,0cyclictest784-21snmpd22:38:332
286679988,0cyclictest784-21snmpd21:29:412
286679988,0cyclictest784-21snmpd19:57:422
286679988,0cyclictest784-21snmpd19:52:542
286679988,0cyclictest784-21snmpd00:39:542
286679988,0cyclictest782-21lldpd22:11:132
286679988,0cyclictest782-21lldpd21:00:482
286679988,0cyclictest17510-21cat20:00:132
286679987,1cyclictest784-21snmpd20:47:462
286679987,1cyclictest784-21snmpd00:04:172
286679987,1cyclictest5910-21ntp_kernel_pll_23:05:172
286599988,0cyclictest782-21lldpd23:37:530
286599988,0cyclictest782-21lldpd23:01:520
286599988,0cyclictest782-21lldpd22:15:040
286599988,0cyclictest782-21lldpd21:12:120
286599987,1cyclictest784-21snmpd21:07:470
286599987,1cyclictest784-21snmpd00:36:550
286599987,1cyclictest784-21snmpd00:33:400
286739977,0cyclictest782-21lldpd23:26:523
286739977,0cyclictest782-21lldpd22:24:113
286739977,0cyclictest782-21lldpd00:06:263
286739976,1cyclictest784-21snmpd20:10:393
286679977,0cyclictest784-21snmpd22:57:332
286679977,0cyclictest784-21snmpd22:43:352
286679977,0cyclictest784-21snmpd22:08:012
286679977,0cyclictest784-21snmpd20:17:182
286679977,0cyclictest784-21snmpd20:13:362
286679977,0cyclictest784-21snmpd19:45:122
286679977,0cyclictest784-21snmpd19:23:202
286679977,0cyclictest782-21lldpd19:44:412
286679977,0cyclictest782-21lldpd00:20:072
286679976,1cyclictest784-21snmpd22:31:402
286679976,1cyclictest784-21snmpd20:30:522
286679976,1cyclictest507-21systemd-resolve23:34:382
286679975,2cyclictest784-21snmpd20:08:422
286679973,3cyclictest11529-21ssh00:13:122
286599977,0cyclictest784-21snmpd22:50:260
286599977,0cyclictest784-21snmpd22:48:110
286599977,0cyclictest784-21snmpd22:21:290
286599977,0cyclictest784-21snmpd00:28:080
286599977,0cyclictest782-21lldpd21:35:440
286599977,0cyclictest782-21lldpd21:18:540
286599977,0cyclictest782-21lldpd19:31:190
286599976,1cyclictest784-21snmpd20:02:330
286599976,1cyclictest784-21snmpd19:23:080
286599976,1cyclictest526-21avahi-daemon23:41:050
286599976,1cyclictest507-21systemd-resolve22:56:520
286599976,1cyclictest31490-21gltestperf20:35:120
286599975,2cyclictest784-21snmpd23:19:280
286599975,2cyclictest784-21snmpd20:41:460
286599975,2cyclictest784-21snmpd19:29:380
286599974,3cyclictest547-21NetworkManager23:08:050
286739966,0cyclictest784-21snmpd21:21:053
286739966,0cyclictest782-21lldpd23:39:003
286739966,0cyclictest782-21lldpd23:31:353
286739966,0cyclictest782-21lldpd21:58:283
286739966,0cyclictest782-21lldpd20:46:143
286739965,1cyclictest784-21snmpd20:44:353
286739964,2cyclictest784-21snmpd00:13:333
286739963,3cyclictest784-21snmpd22:03:363
286679965,1cyclictest784-21snmpd23:18:182
286679965,1cyclictest784-21snmpd20:56:082
286679965,1cyclictest14415-21cpuspeed_turbos23:45:102
286679963,3cyclictest3085-21perl00:05:122
286599966,0cyclictest784-21snmpd19:10:540
286599966,0cyclictest782-21lldpd22:35:120
286599966,0cyclictest782-21lldpd20:52:560
286599966,0cyclictest526-21avahi-daemon20:58:350
286599965,1cyclictest8976-21cpuspeed_turbos21:00:120
286599964,2cyclictest784-21snmpd23:31:190
286599963,3cyclictest20889-21ssh00:21:420
286599963,3cyclictest10429-21diskstats23:10:110
286599963,2cyclictest574-21rs:main0
286599962,3cyclictest16587-21ssh23:46:250
286739954,1cyclictest784-21snmpd19:18:413
286739953,2cyclictest13173-21head21:40:123
286739952,3cyclictest25084-21fw_packets23:55:133
286739952,2cyclictest28031-21smartctl20:25:183
286739952,2cyclictest24476-21sshd20:15:283
286739952,2cyclictest21340-21ssh23:50:483
286679955,0cyclictest526-21avahi-daemon20:53:272
286679954,1cyclictest25218-21cpuspeed_turbos20:20:122
286679952,2cyclictest29578-21ssh23:28:242
286679952,2cyclictest24578-21ssh23:23:422
286679952,2cyclictest21739-21ssh22:50:092
286599955,0cyclictest526-21avahi-daemon20:48:500
286599953,2cyclictest762-21ntpd19:53:280
286599952,3cyclictest547-21NetworkManager22:13:460
286599952,2cyclictest28342-21ssh23:26:410
286599952,2cyclictest245-20systemd-journal19:44:080
286599952,2cyclictest11667-21grep19:45:150
286599951,3cyclictest7709-21collect219:35:150
286739944,0cyclictest784-21snmpd19:10:293
286739943,1cyclictest784-21snmpd21:36:523
286739941,2cyclictest25390-21cut20:20:133
286599941,2cyclictest31053-21fschecks_count19:15:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional