You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-08 - 15:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack8slot6s.osadl.org (updated Tue Sep 08, 2026 12:43:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
144039910541049,5cyclictest16114-21kworker/3:2+events07:45:213
144039910531050,3cyclictest22808-21kworker/3:1+events07:36:043
144039910531049,4cyclictest4439-21kworker/3:0+events11:02:553
144039910531049,4cyclictest17258-21kworker/3:0+events10:05:443
144039910521050,2cyclictest31071-21kworker/3:0+events07:54:133
144039910521050,2cyclictest16114-21kworker/3:2+events07:43:473
144039910521049,3cyclictest9719-21kworker/3:2+events11:09:233
143869910521050,2cyclictest17829-21kworker/0:2+events09:44:220
143869910521050,2cyclictest14504-21kworker/0:1+events11:18:080
144039910511048,3cyclictest4855-21kworker/3:1+events10:12:123
144039910511048,3cyclictest4855-21kworker/3:1+events08:07:073
144039910511048,3cyclictest16114-21kworker/3:2+events07:15:233
144039910511048,2cyclictest31071-21kworker/3:0+events08:02:333
144039910511047,2cyclictest31071-21kworker/3:0+events07:55:223
143869910511049,2cyclictest29972-21kworker/0:1+events11:27:330
143869910511049,2cyclictest29301-21kworker/0:1+events09:50:440
143869910511049,2cyclictest17217-21kworker/0:0+events09:37:090
143869910511048,3cyclictest27629-21kworker/0:0+events11:22:360
143869910511048,3cyclictest27629-21kworker/0:0+events10:55:220
144039910501047,2cyclictest22491-21kworker/3:0+events07:13:163
144039910501046,2cyclictest16114-21kworker/3:2+events07:31:503
143869910501048,2cyclictest23627-21kworker/0:0+events12:33:370
144039910491047,2cyclictest23112-21turbostat10:49:563
144039910491046,2cyclictest27354-21kworker/3:1+events12:04:283
143869910491047,2cyclictest27629-21kworker/0:0+events11:12:240
143869910491047,2cyclictest17217-21kworker/0:0+events09:34:470
144039910481045,2cyclictest4855-21kworker/3:1+events10:24:143
143989910481046,2cyclictest8919-21kworker/2:1+events10:01:052
143869910471045,2cyclictest27629-21kworker/0:0+events11:34:350
144039910461044,2cyclictest13308-21turbostat09:29:563
143939910451044,1cyclictest21624-21kworker/1:1+events08:59:311
143869910451043,2cyclictest23627-21kworker/0:0+events12:35:530
144039910431039,3cyclictest4855-21kworker/3:1+events10:17:413
144039910391035,4cyclictest32040-21turbostat12:09:543
143989910301026,4cyclictest16337-21turbostat07:19:552
143939910291027,2cyclictest16116-21kworker/1:0+events10:36:281
143939910281027,1cyclictest31391-21kworker/1:2+events10:33:051
143989910261024,2cyclictest3996-21kworker/2:2+events09:59:532
143989910251022,2cyclictest32384-21kworker/2:1+events12:07:422
143939910211019,1cyclictest7819-21kworker/1:0+events09:21:491
143939910201019,1cyclictest4551-21kworker/1:1+events12:16:081
143939910201019,1cyclictest16184-21kworker/1:1+events11:47:561
143939910201018,1cyclictest31391-21kworker/1:2+events10:43:151
143989910191017,2cyclictest19610-21turbostat08:44:552
143939910191018,1cyclictest24074-21kworker/1:0+events09:12:551
143939910191017,2cyclictest4551-21kworker/1:1+events12:22:111
143869910191018,1cyclictest18760-21kworker/0:0+events08:24:040
143989910181016,2cyclictest21656-21turbostat08:49:552
143939910171016,1cyclictest31391-21kworker/1:2+events12:28:361
143939910171016,1cyclictest31391-21kworker/1:2+events11:50:441
143869910101009,1cyclictest29301-21kworker/0:1+events10:54:000
143869910071004,3cyclictest13578-21turbostat10:04:550
143869910031002,1cyclictest17217-21kworker/0:0+events08:42:580
1438699994992,2cyclictest27683-21turbostat09:04:550
1439399992991,1cyclictest31391-21kworker/1:2+events09:15:211
1439399991990,1cyclictest31391-21kworker/1:2+events12:10:201
1438699991990,1cyclictest22460-21kworker/0:2+events07:28:190
14386999900,989cyclictest24105-21sendmail_mailqu07:30:220
1440399982977,3cyclictest18490-21turbostat12:29:543
1439399981980,1cyclictest31391-21kworker/1:2+events09:25:551
1439399968967,1cyclictest31391-21kworker/1:2+events10:29:351
1439399963961,2cyclictest373-21kworker/1:1+events11:38:401
1440399962960,2cyclictest4226-21turbostat12:14:553
1439399962961,1cyclictest21624-21kworker/1:1+events09:00:231
1439399958957,1cyclictest24074-21kworker/1:0+events08:51:281
1439399955954,1cyclictest7301-21kworker/1:0+events08:17:441
1439399954952,2cyclictest22769-21turbostat11:59:561
1439399953952,1cyclictest24074-21kworker/1:0+events09:08:191
1439399952951,1cyclictest25910-21kworker/1:0+events10:48:191
1439399949948,1cyclictest11255-21kworker/1:0+events11:42:451
1439399947946,1cyclictest7301-21kworker/1:0+events08:13:251
1439399945943,2cyclictest28840-21kworker/1:2+events08:45:211
1439899944941,3cyclictest9159-21turbostat11:09:552
1440399940937,3cyclictest3953-21turbostat09:54:553
1438699935933,2cyclictest11499-21kworker/0:1+events08:25:220
1438699927924,3cyclictest5061-21turbostat08:09:550
1439399915913,2cyclictest27706-21turbostat12:39:551
1439399909907,2cyclictest32297-21turbostat10:59:551
1440399899898,1cyclictest18347-21turbostat11:19:553
1438699898897,1cyclictest15275-21kworker/0:1+events07:23:350
1438699895894,1cyclictest11499-21kworker/0:1+events08:38:270
1440399893891,2cyclictest13748-21turbostat11:14:553
1438699893890,3cyclictest9080-21turbostat08:19:560
1439899892890,2cyclictest31392-21turbostat07:54:552
1438699889888,1cyclictest11499-21kworker/0:1+events08:30:310
1438699887886,1cyclictest29301-21kworker/0:1+events09:47:390
1440399883881,2cyclictest15586-21turbostat08:34:563
1440399870868,2cyclictest32174-21turbostat11:34:553
1439899856855,1cyclictest1012-21turbostat07:59:552
1439899853851,2cyclictest20813-21turbostat07:29:552
1439399693691,1cyclictest19610-21turbostat08:44:551
1439899667666,1cyclictest3953-21turbostat09:54:552
1438699639635,3cyclictest16337-21turbostat07:19:540
1440399634631,3cyclictest20813-21turbostat07:29:553
1438699632630,2cyclictest23112-21turbostat10:49:560
1438699629626,3cyclictest13308-21turbostat09:29:550
1440399613612,1cyclictest22769-21turbostat11:59:563
1440399576574,2cyclictest27706-21turbostat12:39:553
1440399568567,1cyclictest32297-21turbostat10:59:553
1438699560558,2cyclictest18490-21turbostat12:29:540
1438699550549,1cyclictest9159-21turbostat11:09:560
1439399548544,4cyclictest5061-21turbostat08:09:551
1440399533530,3cyclictest13578-21turbostat10:04:553
1439899525523,2cyclictest27419-21turbostat12:04:552
1439399523521,2cyclictest32440-21turbostat10:24:551
1439399520517,3cyclictest32174-21turbostat11:34:551
1439399487484,3cyclictest32040-21turbostat12:09:541
14403992221,1cyclictest67750irq/123-lan08:25:173
14403991918,1cyclictest67750irq/123-lan12:20:163
14403991918,1cyclictest67750irq/123-lan09:45:223
14403991918,1cyclictest67750irq/123-lan09:10:143
14403991918,1cyclictest67750irq/123-lan08:40:153
14403991818,0cyclictest67750irq/123-lan11:20:533
14403991817,1cyclictest67750irq/123-lan12:17:463
14403991817,1cyclictest67750irq/123-lan11:28:443
14403991817,1cyclictest67750irq/123-lan09:40:343
14403991817,1cyclictest67750irq/123-lan09:24:253
14403991817,1cyclictest67750irq/123-lan09:15:173
14403991817,1cyclictest67750irq/123-lan08:35:203
14403991817,1cyclictest67750irq/123-lan08:10:203
14403991717,0cyclictest67750irq/123-lan11:40:133
14403991716,1cyclictest67750irq/123-lan12:31:263
14403991716,1cyclictest67750irq/123-lan11:45:173
14403991716,1cyclictest67750irq/123-lan10:52:243
14403991716,1cyclictest67750irq/123-lan10:25:113
14403991716,1cyclictest67750irq/123-lan09:55:113
14403991716,1cyclictest67750irq/123-lan08:55:183
14403991715,1cyclictest67750irq/123-lan10:35:063
14403991616,0cyclictest67750irq/123-lan09:35:453
14403991615,1cyclictest67750irq/123-lan11:35:153
14403991615,1cyclictest67750irq/123-lan10:40:173
14403991615,1cyclictest67750irq/123-lan09:30:203
14403991515,0cyclictest67750irq/123-lan11:52:023
14403991515,0cyclictest67750irq/123-lan09:04:253
14403991515,0cyclictest67750irq/123-lan07:20:183
14403991514,1cyclictest67750irq/123-lan10:30:193
14403991514,1cyclictest67750irq/123-lan09:05:223
14403991514,1cyclictest67750irq/123-lan08:45:113
14403991211,1cyclictest67750irq/123-lan08:50:033
14403991211,1cyclictest67750irq/123-lan08:20:133
14403991111,0cyclictest67750irq/123-lan08:18:143
143869998,1cyclictest784-21snmpd08:47:010
143869998,1cyclictest784-21snmpd08:13:480
143869998,1cyclictest782-21lldpd12:22:070
143869998,1cyclictest782-21lldpd09:59:480
143869998,1cyclictest782-21lldpd09:20:230
143989988,0cyclictest784-21snmpd09:43:512
143989988,0cyclictest784-21snmpd08:11:492
143989988,0cyclictest782-21lldpd12:31:012
143989988,0cyclictest782-21lldpd10:28:472
143989988,0cyclictest782-21lldpd08:30:242
143989987,1cyclictest784-21snmpd11:25:002
143989987,1cyclictest784-21snmpd08:27:392
143989987,1cyclictest784-21snmpd08:24:392
143989987,1cyclictest782-21lldpd09:01:552
143989987,1cyclictest507-21systemd-resolve11:20:192
143989986,2cyclictest784-21snmpd10:55:562
143989986,2cyclictest784-21snmpd08:09:292
143989986,1cyclictest784-21snmpd11:50:372
143939988,0cyclictest784-21snmpd10:51:471
143939988,0cyclictest784-21snmpd07:19:351
143939987,1cyclictest784-21snmpd07:28:071
143939986,2cyclictest784-21snmpd11:26:041
143939986,2cyclictest784-21snmpd07:51:231
143869988,0cyclictest784-21snmpd10:27:170
143869988,0cyclictest784-21snmpd08:56:130
143869988,0cyclictest784-21snmpd07:59:000
143869987,1cyclictest507-21systemd-resolve10:34:060
143869987,1cyclictest245-20systemd-journal09:13:150
143869986,2cyclictest784-21snmpd12:13:380
143869986,2cyclictest784-21snmpd12:05:050
143869986,2cyclictest784-21snmpd10:05:050
143869986,2cyclictest784-21snmpd07:35:470
143869986,2cyclictest784-21snmpd07:13:340
143989977,0cyclictest784-21snmpd11:34:162
143989977,0cyclictest782-21lldpd09:25:322
143989977,0cyclictest782-21lldpd07:36:512
143989976,1cyclictest784-21snmpd12:28:172
143989976,1cyclictest784-21snmpd08:18:422
143989975,2cyclictest784-21snmpd11:17:362
143989975,2cyclictest784-21snmpd10:37:142
143989973,3cyclictest21682-21ssh09:33:412
143989973,3cyclictest1769-21ssh09:11:212
143989973,3cyclictest15026-21smartctl11:10:162
143939977,0cyclictest784-21snmpd09:41:271
143939977,0cyclictest782-21lldpd12:01:231
143939977,0cyclictest782-21lldpd08:25:421
143939977,0cyclictest782-21lldpd07:55:331
143939976,1cyclictest526-21avahi-daemon09:39:021
143939976,1cyclictest507-21systemd-resolve09:52:381
143939976,1cyclictest14331-21gltestperf10:00:121
143939975,2cyclictest784-21snmpd10:09:231
143939975,2cyclictest784-21snmpd07:30:461
143939973,3cyclictest11179-21ssh09:55:591
143869977,0cyclictest784-21snmpd11:36:580
143869977,0cyclictest782-21lldpd12:16:190
143869977,0cyclictest782-21lldpd08:04:500
143869976,1cyclictest784-21snmpd10:21:310
143869976,1cyclictest526-21avahi-daemon11:53:340
143869975,2cyclictest784-21snmpd12:04:390
143869975,2cyclictest784-21snmpd07:52:080
143869975,2cyclictest784-21snmpd07:49:290
143869973,3cyclictest5497-21mailstats09:15:200
143989966,0cyclictest784-21snmpd11:35:112
143989966,0cyclictest784-21snmpd08:58:072
143989966,0cyclictest762-21ntpd09:45:522
143989965,1cyclictest784-21snmpd10:24:192
143989965,1cyclictest784-21snmpd07:21:372
143989965,1cyclictest4998-21cpuspeed_turbos11:00:142
143989965,1cyclictest3962-21needreboot08:00:152
143989964,2cyclictest784-21snmpd10:52:292
143989963,2cyclictest574-21rs:main2
143989963,2cyclictest17322-21ssh12:23:032
143989963,2cyclictest15887-21ssh11:46:102
143989962,3cyclictest9943-21memory11:40:162
143989962,3cyclictest23953-21ntp_states07:30:212
143989961,5cyclictest547-21NetworkManager08:38:482
143939966,0cyclictest784-21snmpd11:15:121
143939966,0cyclictest784-21snmpd11:04:071
143939965,1cyclictest32369-21fschecks_time09:45:141
143939965,1cyclictest15812-21/usr/sbin/munin08:30:171
143939964,2cyclictest784-21snmpd11:20:221
143939963,3cyclictest547-21NetworkManager09:34:291
143939962,3cyclictest25433-21dpkg07:35:011
143939962,3cyclictest24241-21ld12:30:151
143869966,0cyclictest782-21lldpd11:41:260
143869966,0cyclictest782-21lldpd09:08:170
143869965,1cyclictest762-21ntpd10:40:170
143869963,2cyclictest31884-21ssh10:19:060
143869963,2cyclictest15769-21ssh10:35:310
143989955,0cyclictest784-21snmpd10:17:012
143989955,0cyclictest782-21lldpd08:52:102
143989955,0cyclictest782-21lldpd07:42:182
143989955,0cyclictest782-21lldpd07:11:572
143989954,1cyclictest526-21avahi-daemon09:23:302
143989954,1cyclictest245-20systemd-journal10:14:512
143989953,2cyclictest547-21NetworkManager10:31:072
143989952,3cyclictest30037-21ls07:45:132
143989952,2cyclictest5342-21gcc12:10:152
143989952,2cyclictest27233-21ssh10:49:162
143989952,2cyclictest24528-21ssh09:35:412
143989952,2cyclictest23454-21tail11:55:142
143989952,2cyclictest20531-21ssh10:40:442
143989951,3cyclictest6360-21ssh09:16:122
143989951,3cyclictest19474-21cat10:05:172
143989950,4cyclictest30273-21fw_forwarded_lo09:05:132
143939955,0cyclictest784-21snmpd07:40:461
143939955,0cyclictest782-21lldpd08:00:361
143939953,2cyclictest762-21ntpd10:10:491
143939953,2cyclictest19442-21sed07:20:141
143939952,2cyclictest29409-21wc10:15:221
143939952,2cyclictest12567-21ld08:20:181
143939952,2cyclictest10843-21sed11:05:201
143939951,3cyclictest14682-21memory11:10:131
143869955,0cyclictest782-21lldpd11:57:330
143869955,0cyclictest526-21avahi-daemon07:40:050
143869952,2cyclictest6742-21ssh11:00:590
143869951,3cyclictest24517-21grep10:10:250
143989941,2cyclictest9424-21cut12:15:122
143939941,2cyclictest762-21ntpd08:35:171
143939941,2cyclictest15252-21strings07:10:151
143939940,4cyclictest30443-21ntpq07:45:161
143869941,2cyclictest24795-21smartctl08:50:170
143869940,3cyclictest13940-21ssh11:45:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional