You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-31 - 10:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Mon Aug 31, 2026 00:43:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29569910491048,1cyclictest8638-21kworker/2:1+events23:56:032
29609910481046,2cyclictest28964-21kworker/3:1+events21:51:253
29609910471045,2cyclictest28964-21kworker/3:1+events21:55:213
29609910461044,2cyclictest6776-21kworker/3:0+events00:07:593
29569910441043,1cyclictest8638-21kworker/2:1+events23:42:342
29569910441042,2cyclictest9299-21turbostat20:49:552
29609910431041,2cyclictest9182-21kworker/3:2+events22:53:523
29569910431041,2cyclictest21828-21kworker/2:2+events23:47:552
29519910421040,2cyclictest3055-21turbostat21:29:541
29569910411040,1cyclictest8638-21kworker/2:1+events23:53:272
29609910401039,1cyclictest30969-21kworker/3:0+events20:22:143
29609910401037,3cyclictest20220-21kworker/3:1+events22:19:453
29609910391036,3cyclictest9449-21turbostat23:39:553
29609910381036,2cyclictest21897-21kworker/3:2+events00:29:553
29609910381035,3cyclictest9182-21kworker/3:2+events22:43:163
29609910371035,2cyclictest29686-21kworker/3:1+events23:26:063
29439910371033,4cyclictest12989-21turbostat19:39:550
29609910341032,2cyclictest4872-21kworker/3:1+events19:19:253
29609910311029,2cyclictest9182-21kworker/3:2+events22:22:353
29609910311028,3cyclictest9182-21kworker/3:2+events22:14:473
29609910311028,3cyclictest30969-21kworker/3:0+events20:54:283
29609910301027,3cyclictest6776-21kworker/3:0+events00:03:553
29609910301027,3cyclictest5248-21turbostat20:39:553
29609910291027,2cyclictest20556-21kworker/3:0+events23:20:223
29609910291027,2cyclictest12886-21kworker/3:0+events19:41:293
29609910291026,3cyclictest29686-21kworker/3:1+events23:18:163
29609910291026,3cyclictest20694-21kworker/3:0+events20:00:233
29609910291025,4cyclictest27485-21kworker/3:1+events00:23:373
29609910281026,2cyclictest20694-21kworker/3:0+events20:11:383
29609910281025,3cyclictest6453-21kworker/3:1+events00:37:373
29609910271025,2cyclictest6776-21kworker/3:0+events00:16:073
29609910271024,3cyclictest2187-21kworker/3:0+events22:03:393
29609910261024,2cyclictest9182-21kworker/3:2+events22:30:233
29609910261023,3cyclictest7173-21kworker/3:2+events19:57:143
29609910251022,3cyclictest29686-21kworker/3:1+events23:31:283
29439910251024,1cyclictest19017-21kworker/0:1+events21:40:200
29609910241022,2cyclictest459-21kworker/3:0+events00:31:513
29609910241022,2cyclictest20694-21kworker/3:0+events19:52:213
29609910231020,3cyclictest7173-21kworker/3:2+events19:23:313
29439910231022,1cyclictest13363-21kworker/0:0+events21:37:100
29609910221020,2cyclictest3133-21kworker/3:1+events20:34:233
29609910221020,2cyclictest17289-21kworker/3:1+events19:48:173
29609910221019,3cyclictest24824-21kworker/3:1+events20:06:153
29609910211020,1cyclictest31700-21kworker/3:0+events22:25:223
29609910211019,2cyclictest31700-21kworker/3:0+events22:35:223
29609910211018,3cyclictest28919-21kworker/3:1+events20:16:063
29439910211020,1cyclictest21661-21kworker/0:2+events21:16:480
29609910201017,3cyclictest9182-21kworker/3:2+events22:09:243
29609910191017,2cyclictest9182-21kworker/3:2+events22:46:263
29609910191017,2cyclictest6776-21kworker/3:0+events00:11:053
29609910191016,3cyclictest30969-21kworker/3:0+events20:56:353
29609910161014,2cyclictest30969-21kworker/3:0+events20:28:253
29519910111010,1cyclictest15834-21kworker/1:0+events19:11:361
29609910081005,3cyclictest2980-21kworker/3:2+events21:27:553
294399999998,1cyclictest19017-21kworker/0:1+events21:20:220
294399995993,2cyclictest20643-21kworker/0:2+events22:58:130
296099994992,2cyclictest23851-21turbostat21:49:543
294399989988,1cyclictest20643-21kworker/0:2+events21:46:060
295199973970,3cyclictest30234-21turbostat21:24:541
295199968967,1cyclictest15834-21kworker/1:0+events20:47:321
296099963960,3cyclictest8248-21turbostat21:34:543
295699959957,2cyclictest7549-21kworker/2:2+events20:43:402
294399957956,1cyclictest19017-21kworker/0:1+events21:10:220
294399955954,1cyclictest9371-21kworker/0:0+events19:30:230
294399952951,1cyclictest20643-21kworker/0:2+events23:07:440
294399951950,1cyclictest1522-21kworker/0:3+events19:25:220
294399945944,1cyclictest20643-21kworker/0:2+events23:04:450
294399945944,1cyclictest16006-21kworker/0:2+events21:05:220
296099943941,2cyclictest7269-21turbostat20:44:553
294399936935,1cyclictest32558-21kworker/0:0+events23:11:270
294399935934,1cyclictest14649-21kworker/0:0+events21:00:220
295699923921,2cyclictest3042-21turbostat00:04:552
295699918915,1cyclictest7549-21kworker/2:2+events21:31:482
296099907905,2cyclictest19397-21turbostat21:14:553
294399887885,2cyclictest8699-21turbostat00:09:550
296099884881,3cyclictest10236-21turbostat23:09:553
294399677674,3cyclictest13337-21turbostat20:59:550
296099674673,1cyclictest12989-21turbostat19:39:553
295699672669,3cyclictest5248-21turbostat20:39:552
296099636633,3cyclictest2833-21turbostat19:14:553
294399630628,2cyclictest8248-21turbostat21:34:540
295199620617,3cyclictest7269-21turbostat20:44:551
296099605604,1cyclictest30234-21turbostat21:24:543
296099591590,1cyclictest30235-21turbostat23:59:553
295699562560,2cyclictest3055-21turbostat21:29:542
294399551549,2cyclictest6930-21turbostat19:24:550
294399533530,3cyclictest8309-21turbostat00:39:550
296099526525,1cyclictest15429-21turbostat23:14:563
296099512509,3cyclictest9299-21turbostat20:49:553
294399502500,2cyclictest27026-21turbostat22:54:550
295699479478,1cyclictest9449-21turbostat23:39:552
2951993520,351cyclictest11386-21sh19:30:121
2960992018,2cyclictest67750irq/123-lan21:15:133
2960991919,0cyclictest67750irq/123-lan19:25:133
2960991818,0cyclictest67750irq/123-lan21:06:153
2960991817,1cyclictest67750irq/123-lan23:50:123
2960991817,1cyclictest67750irq/123-lan23:47:403
2960991817,1cyclictest67750irq/123-lan21:35:173
2960991816,2cyclictest67750irq/123-lan23:02:073
2960991816,2cyclictest67750irq/123-lan21:43:213
2960991815,3cyclictest67750irq/123-lan22:55:143
2960991717,0cyclictest67750irq/123-lan19:30:183
2960991716,1cyclictest67750irq/123-lan23:43:023
29609998,1cyclictest67750irq/123-lan21:00:153
29569998,1cyclictest507-21systemd-resolve22:03:002
29519998,1cyclictest782-21lldpd22:05:031
29439998,1cyclictest784-21snmpd20:42:520
29439998,1cyclictest782-21lldpd23:25:500
29569988,0cyclictest784-21snmpd19:24:392
29569988,0cyclictest782-21lldpd22:47:182
29569988,0cyclictest782-21lldpd21:57:232
29569988,0cyclictest782-21lldpd21:14:042
29569988,0cyclictest782-21lldpd19:41:432
29569988,0cyclictest782-21lldpd00:34:512
29569988,0cyclictest782-21lldpd00:28:102
29569987,1cyclictest784-21snmpd21:09:082
29569987,1cyclictest784-21snmpd20:57:352
29569987,1cyclictest782-21lldpd21:50:492
29569986,2cyclictest784-21snmpd21:18:152
29569986,2cyclictest784-21snmpd20:24:012
29569986,2cyclictest784-21snmpd19:36:302
29519988,0cyclictest784-21snmpd23:39:151
29519988,0cyclictest784-21snmpd00:07:341
29519988,0cyclictest782-21lldpd23:31:131
29519988,0cyclictest782-21lldpd23:15:561
29519988,0cyclictest782-21lldpd23:12:541
29519988,0cyclictest782-21lldpd22:28:091
29519987,1cyclictest784-21snmpd21:04:421
29519987,1cyclictest784-21snmpd19:58:491
29519987,1cyclictest782-21lldpd23:06:201
29519987,1cyclictest7743-21gltestperf22:00:121
29519987,1cyclictest526-21avahi-daemon21:14:251
29519987,1cyclictest526-21avahi-daemon00:13:131
29519987,1cyclictest507-21systemd-resolve22:52:511
29519987,1cyclictest507-21systemd-resolve00:24:541
29519987,1cyclictest27027-21munin-plugin-st22:50:001
29519986,2cyclictest784-21snmpd22:19:021
29519986,2cyclictest784-21snmpd19:15:481
29519986,2cyclictest784-21snmpd00:37:351
29439988,0cyclictest784-21snmpd23:24:150
29439988,0cyclictest784-21snmpd19:46:370
29439988,0cyclictest782-21lldpd20:35:200
29439988,0cyclictest782-21lldpd20:11:070
29439988,0cyclictest782-21lldpd19:56:430
29439987,1cyclictest784-21snmpd19:42:420
29569977,0cyclictest784-21snmpd23:23:542
29569977,0cyclictest784-21snmpd20:50:432
29569977,0cyclictest784-21snmpd20:00:242
29569977,0cyclictest784-21snmpd00:24:042
29569977,0cyclictest782-21lldpd21:49:172
29569977,0cyclictest782-21lldpd20:06:412
29569977,0cyclictest782-21lldpd19:15:192
29569976,1cyclictest526-21avahi-daemon23:06:212
29569976,1cyclictest526-21avahi-daemon19:47:342
29569976,1cyclictest31077-21gltestperf21:20:172
29569976,1cyclictest245-20systemd-journal23:16:312
29569975,2cyclictest784-21snmpd22:51:332
29569975,2cyclictest784-21snmpd22:20:132
29569975,2cyclictest784-21snmpd20:10:452
29569975,2cyclictest784-21snmpd19:58:222
29569973,3cyclictest21548-21ssh21:41:482
29519977,0cyclictest784-21snmpd20:21:061
29519977,0cyclictest784-21snmpd19:54:221
29519977,0cyclictest784-21snmpd00:30:531
29519977,0cyclictest782-21lldpd23:44:301
29519977,0cyclictest782-21lldpd21:52:331
29519977,0cyclictest782-21lldpd20:56:081
29519977,0cyclictest782-21lldpd20:09:111
29519977,0cyclictest782-21lldpd19:39:071
29519977,0cyclictest782-21lldpd19:21:071
29519976,1cyclictest526-21avahi-daemon20:50:211
29519976,1cyclictest16608-21sh22:40:011
29519975,2cyclictest784-21snmpd22:32:251
29519975,2cyclictest784-21snmpd22:20:191
29519975,2cyclictest784-21snmpd22:13:371
29519973,3cyclictest1836-21ssh23:26:391
29439977,0cyclictest784-21snmpd23:58:430
29439977,0cyclictest784-21snmpd23:37:450
29439977,0cyclictest784-21snmpd22:14:100
29439977,0cyclictest784-21snmpd20:05:060
29439977,0cyclictest784-21snmpd00:02:400
29439977,0cyclictest782-21lldpd21:58:540
29439977,0cyclictest782-21lldpd20:23:540
29439977,0cyclictest782-21lldpd20:01:570
29439977,0cyclictest782-21lldpd00:19:050
29439976,1cyclictest526-21avahi-daemon23:49:560
29439975,2cyclictest784-21snmpd22:35:590
29439975,2cyclictest784-21snmpd22:24:580
29439975,2cyclictest784-21snmpd19:53:580
29439973,3cyclictest5766-21ssh00:31:290
29569966,0cyclictest784-21snmpd22:17:412
29569966,0cyclictest782-21lldpd20:27:522
29569966,0cyclictest574-21rs:main2
29569965,1cyclictest8753-21cpuspeed_turbos00:35:102
29569965,1cyclictest526-21avahi-daemon22:55:452
29569965,1cyclictest526-21avahi-daemon00:11:432
29569965,1cyclictest10025-21ntpq00:05:192
29569964,2cyclictest547-21NetworkManager22:38:472
29569963,3cyclictest6915-21ssh23:31:302
29569963,3cyclictest3434-21fw_packets19:10:132
29569963,2cyclictest18320-21ssh23:11:502
29569961,5cyclictest547-21NetworkManager20:16:262
29519966,0cyclictest784-21snmpd21:45:061
29519966,0cyclictest784-21snmpd20:36:371
29519965,1cyclictest784-21snmpd23:45:031
29519965,1cyclictest784-21snmpd23:20:181
29519965,1cyclictest784-21snmpd21:34:301
29519965,1cyclictest507-21systemd-resolve21:38:011
29519965,1cyclictest29461-21cpuspeed_turbos20:15:111
29519964,2cyclictest784-21snmpd00:25:441
29519963,3cyclictest547-21NetworkManager20:27:271
29519963,2cyclictest5063-21ssh21:57:161
29519963,2cyclictest2936-21ssh22:57:071
29519962,3cyclictest20417-21cat00:15:161
29439966,0cyclictest784-21snmpd19:12:470
29439966,0cyclictest782-21lldpd23:53:480
29439966,0cyclictest782-21lldpd22:00:150
29439966,0cyclictest782-21lldpd00:21:090
29439965,1cyclictest784-21snmpd22:31:040
29439965,1cyclictest784-21snmpd20:50:580
29439965,1cyclictest507-21systemd-resolve23:33:360
29439963,2cyclictest574-21rs:main0
29439962,4cyclictest547-21NetworkManager00:10:130
29439962,3cyclictest2573-21sed20:25:200
29439962,3cyclictest23801-21ssh22:45:310
29439962,3cyclictest17977-21ssh23:42:230
29439960,1cyclictest30787-21unixbench_multi21:50:220
29569955,0cyclictest9831-21needreboot19:25:152
29569955,0cyclictest526-21avahi-daemon19:50:142
29569954,1cyclictest784-21snmpd19:30:122
29569953,2cyclictest784-21snmpd22:41:472
29569953,2cyclictest784-21snmpd22:13:492
29569952,2cyclictest5330-21ssh23:00:062
29569952,2cyclictest4401-21ssh22:27:272
29569952,2cyclictest23070-21rm00:17:462
29569952,2cyclictest15595-21ssh22:07:272
29569951,3cyclictest7786-21grep22:30:222
29569951,3cyclictest4111-21ssh23:29:492
29519955,0cyclictest507-21systemd-resolve19:46:341
29519954,1cyclictest31383-21ntpq23:55:161
29519953,2cyclictest784-21snmpd19:44:221
29519952,2cyclictest6109-21ssh00:01:251
29519952,2cyclictest27225-21ssh21:15:501
29519952,2cyclictest25313-21ssh23:50:051
29519952,2cyclictest23420-21cpuspeed_turbos20:00:121
29519952,2cyclictest18325-21netstat21:05:161
29519952,2cyclictest17937-21ntpq22:40:161
29519951,3cyclictest5279-21sh23:00:011
29519951,3cyclictest10148-21pmu-power19:25:181
29439955,0cyclictest526-21avahi-daemon20:45:510
29439953,2cyclictest784-21snmpd23:15:130
29439953,2cyclictest784-21snmpd22:25:040
29439953,2cyclictest784-21snmpd20:34:460
29439952,3cyclictest762-21ntpd00:26:400
29439952,2cyclictest29386-21apt20:15:110
29439952,2cyclictest22835-21ssh22:15:010
29439952,2cyclictest18716-21ssh22:40:390
29439952,2cyclictest14024-21sed22:05:220
29569942,2cyclictest784-21snmpd21:00:032
29569942,2cyclictest4124-21tr20:30:162
29519941,2cyclictest3569-21cpuspeed_turbos20:30:131
29519941,2cyclictest19213-21ssh21:40:121
29519940,4cyclictest28102-21ntpq20:10:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional