You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-24 - 00:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Wed Sep 23, 2026 12:43:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
11999910551051,2cyclictest27031-21kworker/3:2+events10:54:383
11999910521050,2cyclictest5128-21kworker/3:0+events07:28:143
11999910521050,2cyclictest5128-21kworker/3:0+events07:20:453
11999910521050,2cyclictest15358-21kworker/3:0+events09:47:313
11999910521050,2cyclictest15358-21kworker/3:0+events09:05:423
11999910521050,2cyclictest15246-21kworker/3:2+events09:54:533
11999910521050,2cyclictest11836-21kworker/3:1+events08:56:263
11999910511049,2cyclictest9568-21kworker/3:2+events08:53:403
11999910511049,2cyclictest29109-21kworker/3:1+events12:33:393
11999910511049,2cyclictest27031-21kworker/3:2+events10:43:063
11999910511049,2cyclictest18462-21kworker/3:1+events12:16:283
11999910511049,2cyclictest15358-21kworker/3:0+events09:12:233
11999910511049,2cyclictest13261-21kworker/3:1+events07:42:083
11999910511049,2cyclictest10836-21kworker/3:2+events07:33:583
11999910501048,2cyclictest28824-21kworker/3:2+events12:01:153
11999910501048,2cyclictest24679-21kworker/3:3+events12:27:123
11999910501048,2cyclictest11836-21kworker/3:1+events09:02:533
11999910501047,3cyclictest28824-21kworker/3:2+events11:52:503
11999910491048,1cyclictest28824-21kworker/3:2+events11:56:273
11999910491047,2cyclictest24679-21kworker/3:3+events12:23:183
11999910481047,1cyclictest15358-21kworker/3:0+events09:38:163
11909910481047,1cyclictest7286-21kworker/1:2+events08:47:441
11999910471046,1cyclictest14287-21kworker/3:0+events10:20:443
11999910471045,2cyclictest20291-21kworker/3:1+events10:36:373
11999910401038,2cyclictest14287-21kworker/3:0+events10:26:503
11849910401037,3cyclictest7993-21turbostat10:54:560
11999910381036,2cyclictest6881-21kworker/3:1+events09:41:523
11999910371035,2cyclictest27031-21kworker/3:2+events10:46:503
11999910361034,2cyclictest20291-21kworker/3:1+events09:58:043
11999910351033,2cyclictest20291-21kworker/3:1+events10:33:183
11909910341033,1cyclictest18983-21kworker/1:0+events08:35:201
11909910301028,2cyclictest9332-21kworker/1:0+events09:25:441
11849910301028,2cyclictest11484-21kworker/0:1+events08:00:220
11999910281024,2cyclictest14221-21turbostat12:14:553
11959910271024,3cyclictest11216-21turbostat07:39:552
11959910261024,2cyclictest961-21turbostat10:09:552
11909910261025,1cyclictest31687-21kworker/1:2+events08:33:241
11959910231021,2cyclictest905-21kworker/2:0+events10:17:022
11849910231022,1cyclictest22390-21kworker/0:1+events09:19:130
11959910191018,1cyclictest905-21kworker/2:0+events11:07:362
11959910191016,3cyclictest905-21kworker/2:0+events10:12:522
11909910191018,1cyclictest9332-21kworker/1:0+events09:30:231
11909910191018,1cyclictest5210-21kworker/1:1+events08:43:061
11909910171016,1cyclictest26674-21kworker/1:1+events09:24:241
11959910151014,1cyclictest20516-21kworker/2:1+events11:43:162
11849910141012,2cyclictest30406-21kworker/0:1+events11:26:290
11959910101008,2cyclictest20516-21kworker/2:1+events11:45:442
11909910101009,1cyclictest13062-21kworker/1:0+events10:57:111
11909910081007,1cyclictest3815-21kworker/1:2+events12:38:301
11909910041003,1cyclictest21038-21kworker/1:1+events07:58:091
11999910031001,2cyclictest17032-21turbostat11:04:563
11909910031002,1cyclictest29137-21kworker/1:2+events07:52:121
118499998997,1cyclictest10216-21kworker/0:0+events11:36:290
119099997996,1cyclictest13062-21kworker/1:0+events11:00:221
118499997996,1cyclictest30406-21kworker/0:1+events11:16:290
118499996994,2cyclictest24552-21turbostat11:49:550
118499995993,1cyclictest22390-21kworker/0:1+events10:00:220
119099994993,1cyclictest15579-21kworker/1:1+events07:48:011
118499991990,1cyclictest10216-21kworker/0:0+events12:09:460
119599987985,2cyclictest7186-21turbostat07:29:542
118499986983,3cyclictest10032-21turbostat10:19:550
118499983982,1cyclictest11484-21kworker/0:1+events08:15:230
119099975974,1cyclictest5105-21turbostat12:04:551
119599972970,2cyclictest2608-21turbostat09:34:552
119999969967,2cyclictest31461-21turbostat08:29:553
118499969968,0cyclictest5742-21kworker/0:2+events11:24:240
119999968966,2cyclictest12729-21turbostat10:59:553
118499967965,2cyclictest11484-21kworker/0:1+events08:10:220
118499965964,1cyclictest22089-21kworker/0:0+events08:05:220
118499963963,0cyclictest30406-21kworker/0:1+events12:12:220
118499962961,1cyclictest30406-21kworker/0:1+events11:31:290
118499962961,1cyclictest16696-21kworker/0:2+events07:39:420
118499961960,1cyclictest29122-21kworker/0:0+events08:23:200
118499961960,1cyclictest16696-21kworker/0:2+events07:10:220
118499959958,1cyclictest16696-21kworker/0:2+events07:16:300
118499958957,1cyclictest29122-21kworker/0:0+events08:28:290
118499949946,3cyclictest13611-21turbostat09:04:550
118499948948,0cyclictest28929-21kworker/0:2+events11:10:230
119099947945,2cyclictest16068-21turbostat09:49:541
119099924921,3cyclictest15407-21turbostat11:39:551
119999907905,2cyclictest19341-21turbostat07:59:553
119099899896,3cyclictest21463-21turbostat11:09:561
118499883881,2cyclictest3881-21turbostat12:39:550
118499870868,2cyclictest27382-21turbostat10:39:560
119099692690,2cyclictest31527-21turbostat12:34:541
118499684681,3cyclictest5105-21turbostat12:04:550
119999669665,4cyclictest24552-21turbostat11:49:553
119099658657,1cyclictest13294-21turbostat07:44:551
119999656655,1cyclictest10032-21turbostat10:19:553
119099654652,2cyclictest31461-21turbostat08:29:551
118499644643,1cyclictest24677-21turbostat09:59:550
119999638635,3cyclictest11216-21turbostat07:39:553
118499638635,3cyclictest17672-21turbostat09:14:540
119999615612,3cyclictest3130-21turbostat07:19:553
118499611609,2cyclictest9196-21turbostat07:34:540
118499605603,2cyclictest21463-21turbostat11:09:560
119999595593,2cyclictest7576-21turbostat08:49:553
119999583581,2cyclictest2608-21turbostat09:34:553
119999555552,3cyclictest3881-21turbostat12:39:553
119599554553,1cyclictest28983-21turbostat10:04:542
118499543542,1cyclictest19341-21turbostat07:59:550
119099524523,1cyclictest22030-21turbostat09:19:551
119099514512,2cyclictest7993-21turbostat10:54:561
119599502499,3cyclictest17032-21turbostat11:04:562
119599405404,1cyclictest15407-21turbostat11:39:562
1195993800,379cyclictest17104-21ssh12:12:212
1190992019,1cyclictest67750irq/123-lan11:15:171
1190992019,1cyclictest26674-21kworker/1:1+events_freezable10:22:081
1190991919,0cyclictest784-21snmpd08:22:331
1190991919,0cyclictest67750irq/123-lan09:38:431
1190991918,1cyclictest67750irq/123-lan10:36:581
1190991918,1cyclictest67750irq/123-lan10:16:071
1190991918,1cyclictest67750irq/123-lan10:14:411
1190991818,0cyclictest67750irq/123-lan12:29:571
1190991818,0cyclictest67750irq/123-lan12:05:161
1190991818,0cyclictest67750irq/123-lan11:32:471
1190991818,0cyclictest67750irq/123-lan10:46:031
1190991818,0cyclictest67750irq/123-lan10:25:031
1190991818,0cyclictest67750irq/123-lan09:00:541
1190991818,0cyclictest67750irq/123-lan08:59:491
1190991818,0cyclictest67750irq/123-lan08:15:211
1190991818,0cyclictest67750irq/123-lan07:21:111
1190991817,1cyclictest67750irq/123-lan12:20:121
1190991817,1cyclictest67750irq/123-lan12:15:221
1190991817,1cyclictest67750irq/123-lan11:28:021
1190991817,1cyclictest67750irq/123-lan10:42:291
1190991817,1cyclictest67750irq/123-lan09:40:181
1190991817,1cyclictest67750irq/123-lan08:13:081
1190991717,0cyclictest67750irq/123-lan11:24:531
1190991717,0cyclictest67750irq/123-lan10:04:131
1190991717,0cyclictest67750irq/123-lan09:57:111
1190991717,0cyclictest67750irq/123-lan07:12:491
1190991716,1cyclictest67750irq/123-lan12:12:261
1190991716,1cyclictest67750irq/123-lan11:49:281
1190991616,0cyclictest67750irq/123-lan11:55:131
1190991616,0cyclictest67750irq/123-lan11:43:041
1190991616,0cyclictest67750irq/123-lan11:12:261
1190991616,0cyclictest67750irq/123-lan10:30:041
1190991616,0cyclictest67750irq/123-lan10:06:531
1190991616,0cyclictest67750irq/123-lan09:52:121
1190991616,0cyclictest67750irq/123-lan09:13:581
1190991616,0cyclictest67750irq/123-lan09:05:131
1190991615,1cyclictest67750irq/123-lan08:50:121
1190991615,1cyclictest67750irq/123-lan07:30:181
1190991515,0cyclictest67750irq/123-lan11:51:031
1190991515,0cyclictest67750irq/123-lan07:25:111
1190991514,1cyclictest67750irq/123-lan08:07:261
1190991414,0cyclictest67750irq/123-lan07:15:121
1190991313,0cyclictest67750irq/123-lan08:00:121
1190991313,0cyclictest29137-21kworker/1:2+events_freezable07:35:171
1184991311,2cyclictest5163-21turbostat07:24:550
119599118,3cyclictest784-21snmpd10:50:542
119599118,3cyclictest784-21snmpd07:22:352
119599118,3cyclictest572-21in:imuxsock11:12:422
119599106,4cyclictest784-21snmpd12:17:492
119599102,2cyclictest245-20systemd-journal08:59:082
119599100,2cyclictest3419-21ssh10:44:352
11999998,1cyclictest961-21turbostat10:09:553
11999998,1cyclictest784-21snmpd11:23:023
11999998,1cyclictest784-21snmpd08:43:113
11999998,1cyclictest784-21snmpd08:05:583
11999998,1cyclictest784-21snmpd07:53:393
11959997,2cyclictest784-21snmpd10:45:302
11959996,3cyclictest784-21snmpd11:32:232
11959996,3cyclictest784-21snmpd08:40:022
11959996,3cyclictest32154-21cat08:25:142
11959992,6cyclictest17754-21diskstats07:50:122
11959992,3cyclictest16793-21sed07:45:202
11959990,3cyclictest15389-21taskset09:02:012
11849998,1cyclictest507-21systemd-resolve09:54:010
11999988,0cyclictest782-21lldpd10:01:033
11999988,0cyclictest782-21lldpd08:10:463
11999987,1cyclictest784-21snmpd08:33:073
11999987,1cyclictest526-21avahi-daemon12:08:453
11999987,1cyclictest507-21systemd-resolve09:20:483
11959987,1cyclictest784-21snmpd10:26:002
11959987,1cyclictest507-21systemd-resolve09:41:212
11959986,2cyclictest784-21snmpd12:39:162
11959986,2cyclictest784-21snmpd11:22:352
11959986,2cyclictest784-21snmpd07:31:502
11959985,3cyclictest526-21avahi-daemon09:06:512
11959985,3cyclictest25733-21cpuspeed_turbos08:10:132
11959984,4cyclictest784-21snmpd10:59:342
11959984,4cyclictest245-20systemd-journal09:21:342
11959983,5cyclictest784-21snmpd08:05:282
11959983,5cyclictest18488-21cat09:10:142
11959982,3cyclictest547-21NetworkManager09:50:272
11959981,3cyclictest2395-21cat11:55:202
11959981,3cyclictest11778-21sed08:54:592
11959980,4cyclictest30097-21ntp_kernel_pll_11:50:212
11959980,3cyclictest28808-21/usr/sbin/munin08:15:202
11959980,3cyclictest25904-21ntpq09:55:182
11849988,0cyclictest784-21snmpd12:17:550
11849988,0cyclictest784-21snmpd09:38:190
11849988,0cyclictest782-21lldpd11:44:160
11849988,0cyclictest782-21lldpd10:13:100
11849987,1cyclictest784-21snmpd10:56:520
11849987,1cyclictest784-21snmpd09:08:060
11849987,1cyclictest19942-21ntp_offset10:25:170
11849986,2cyclictest784-21snmpd07:45:320
11849984,3cyclictest2423-21grep10:05:130
11999976,1cyclictest507-21systemd-resolve11:32:523
11999975,2cyclictest784-21snmpd10:14:263
11999975,2cyclictest784-21snmpd08:15:453
11999975,2cyclictest784-21snmpd08:00:413
11959977,0cyclictest784-21snmpd09:28:242
11959977,0cyclictest782-21lldpd07:42:232
11959975,2cyclictest784-21snmpd12:00:032
11959974,3cyclictest526-21avahi-daemon10:36:562
11959972,2cyclictest7748-21cat09:35:152
11959971,3cyclictest29972-21perl08:20:122
11959971,3cyclictest245-20systemd-journal10:30:052
11959971,2cyclictest24419-21ssh09:16:272
11959970,4cyclictest31346-21ntpq11:15:182
11959970,4cyclictest28320-21ntpq12:25:162
11959970,4cyclictest1668-21perl07:10:122
11959970,3cyclictest7821-21/usr/sbin/munin08:45:112
11959970,3cyclictest7280-21cut11:25:142
11959970,3cyclictest4623-21ntp_states08:35:172
11959970,3cyclictest23393-21diskstats12:20:122
11959970,3cyclictest2217-21ntpq08:30:192
11959970,3cyclictest20379-21/usr/sbin/munin07:55:172
11959970,3cyclictest16515-21ssh10:20:552
11959970,3cyclictest16503-21grep09:45:102
11959970,3cyclictest11028-21ntpq12:05:152
11959970,2cyclictest928-21sed12:30:232
11959970,2cyclictest22078-21interrupts08:00:112
11849977,0cyclictest784-21snmpd07:42:090
11849977,0cyclictest782-21lldpd10:31:320
11849977,0cyclictest782-21lldpd09:31:510
11849977,0cyclictest782-21lldpd09:22:530
11849976,1cyclictest6028-21cut08:40:120
11849976,1cyclictest526-21avahi-daemon12:28:330
11849976,1cyclictest526-21avahi-daemon10:49:020
11849976,1cyclictest526-21avahi-daemon10:43:430
11849975,2cyclictest784-21snmpd12:23:550
11849975,2cyclictest784-21snmpd11:56:170
11849975,2cyclictest784-21snmpd08:48:440
11849974,2cyclictest8255-21ntpq07:25:160
11999966,0cyclictest782-21lldpd09:26:523
11999963,2cyclictest29430-21ssh11:13:503
11999963,2cyclictest22591-21ntpq11:05:163
11999962,3cyclictest9283-21ssh11:27:283
11959960,2cyclictest3547-21df07:15:122
11849966,0cyclictest784-21snmpd08:52:170
11849966,0cyclictest782-21lldpd08:35:580
11849965,1cyclictest526-21avahi-daemon09:48:120
11849964,2cyclictest784-21snmpd07:53:360
11849963,2cyclictest245-20systemd-journal11:03:490
11849963,0cyclictest32321-21ssh11:53:250
11849961,4cyclictest547-21NetworkManager10:21:570
11999954,1cyclictest245-20systemd-journal11:37:303
11999952,2cyclictest31550-21mailstats11:15:213
11999952,2cyclictest245-20systemd-journal11:41:363
11999952,2cyclictest245-20systemd-journal07:45:173
11999951,3cyclictest4085-21fw_packets08:35:133
11999951,3cyclictest24261-21ssh09:16:103
11849952,3cyclictest32172-21sed09:25:200
11849952,2cyclictest32256-21kworker/u16:0+nfsiod12:31:080
11849952,2cyclictest14649-21sh09:42:240
11849950,4cyclictest2177-21ntpq08:30:190
11999941,2cyclictest30022-21expr08:20:133
11999940,3cyclictest1544-21df_inode07:10:133
11849941,2cyclictest11891-21cpu08:55:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional