You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-27 - 06:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack8slot6s.osadl.org (updated Thu Aug 27, 2026 00:43:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
315629910521050,2cyclictest5213-21kworker/1:2+events19:23:071
315629910521050,2cyclictest11362-21kworker/1:2+events20:41:031
315629910511049,2cyclictest8139-21kworker/1:0+events22:55:301
315629910511049,2cyclictest8139-21kworker/1:0+events22:47:101
315629910511049,2cyclictest25285-21kworker/1:0+events19:18:511
315629910511049,2cyclictest25285-21kworker/1:0+events19:14:421
315629910511049,2cyclictest2081-21kworker/1:1+events19:35:431
315629910501048,2cyclictest7262-21kworker/1:0+events19:31:061
315629910501048,2cyclictest7262-21kworker/1:0+events19:28:331
315629910501048,2cyclictest2320-21kworker/1:1+events22:30:431
315629910501048,2cyclictest21026-21kworker/1:1+events00:16:461
315629910481046,2cyclictest8139-21kworker/1:0+events22:40:221
315629910461044,2cyclictest2320-21kworker/1:1+events22:36:501
315579910461044,2cyclictest16824-21kworker/0:2+events21:28:560
315659910351034,1cyclictest25825-21kworker/2:0+events00:34:112
315629910351033,2cyclictest21026-21kworker/1:1+events00:22:541
315629910341032,2cyclictest8139-21kworker/1:0+events22:51:081
315629910341031,3cyclictest22428-21turbostat23:24:551
315709910331032,1cyclictest5464-21kworker/3:0+events20:54:473
315709910261024,2cyclictest8289-21kworker/3:1+events22:11:063
315579910241023,1cyclictest16824-21kworker/0:2+events21:31:280
315579910231022,1cyclictest5176-21kworker/0:2+events23:37:000
315659910211018,3cyclictest31628-21turbostat21:29:542
315579910201019,1cyclictest13227-21kworker/0:1+events23:44:130
315659910181017,1cyclictest6227-21kworker/2:2+events00:36:492
315659910181017,1cyclictest1421-21kworker/2:1+events00:01:052
315659910161013,3cyclictest3167-21turbostat22:04:552
315709910111009,2cyclictest21237-21turbostat23:54:553
315579910041003,1cyclictest12770-21kworker/0:1+events21:44:200
315579910031002,1cyclictest8393-21kworker/0:0+events23:10:450
315579910031002,1cyclictest16824-21kworker/0:2+events21:14:540
315629910021000,2cyclictest14786-21turbostat21:44:541
315579910021002,0cyclictest16824-21kworker/0:2+events21:35:220
315579910011000,1cyclictest780-21kworker/0:0+events23:32:550
315579910011000,1cyclictest1332-21kworker/0:0+events21:23:400
31570991000999,1cyclictest27313-21kworker/3:0+events22:22:553
3156599998997,1cyclictest11586-21turbostat23:14:562
3155799997996,1cyclictest5176-21kworker/0:2+events22:02:330
3155799996995,1cyclictest8393-21kworker/0:0+events23:08:180
3155799996995,1cyclictest16824-21kworker/0:2+events21:16:280
3155799995994,1cyclictest17650-21kworker/0:1+events20:00:220
3157099994991,2cyclictest27313-21kworker/3:0+events22:18:193
3155799994993,1cyclictest29605-21kworker/0:2+events20:30:210
3155799994993,1cyclictest25492-21kworker/0:1+events20:13:190
3155799993992,1cyclictest29605-21kworker/0:2+events20:27:560
3155799993992,1cyclictest21352-21kworker/0:0+events20:05:230
3157099992989,1cyclictest27313-21kworker/3:0+events23:21:303
3155799992991,1cyclictest13227-21kworker/0:1+events00:07:440
3157099991989,2cyclictest10447-21kworker/3:2+events00:10:453
3155799991990,1cyclictest30707-21kworker/0:0+events21:58:410
3157099989986,1cyclictest8289-21kworker/3:1+events22:07:113
3155799989988,1cyclictest25492-21kworker/0:1+events20:19:100
3155799989988,1cyclictest1552-21kworker/0:1+events23:01:260
3157099988986,2cyclictest27313-21kworker/3:0+events22:28:483
3155799985984,1cyclictest21352-21kworker/0:0+events20:20:230
3155799985983,2cyclictest29583-21turbostat22:29:550
3157099984983,1cyclictest27313-21kworker/3:0+events23:49:443
3157099978977,1cyclictest20546-21kworker/3:2+events21:54:373
3157099978975,3cyclictest15664-21turbostat19:54:553
3155799977974,1cyclictest17650-21kworker/0:1+events19:58:420
3157099976975,1cyclictest27637-21turbostat23:29:563
3157099975974,1cyclictest5464-21kworker/3:0+events21:02:103
3157099974973,1cyclictest25042-21kworker/3:2+events20:39:403
3157099974973,1cyclictest13609-21kworker/3:1+events21:06:283
3157099970966,3cyclictest28387-21turbostat22:59:553
3156599970969,1cyclictest25825-21kworker/2:0+events00:29:272
3156599970969,1cyclictest11869-21kworker/2:0+events19:43:402
3157099969968,1cyclictest16719-21kworker/3:2+events23:18:393
3157099965964,1cyclictest13609-21kworker/3:1+events21:47:353
3157099963961,2cyclictest7732-21kworker/3:2+events20:57:213
3157099961958,1cyclictest29170-21kworker/3:1+events20:45:203
3156599960958,2cyclictest11869-21kworker/2:0+events19:53:282
3156599959956,3cyclictest3449-21turbostat20:44:542
3156599955954,1cyclictest11869-21kworker/2:0+events23:55:222
3156599955954,1cyclictest11869-21kworker/2:0+events19:47:342
3157099937936,1cyclictest5584-21turbostat19:29:553
3155799935934,1cyclictest19993-21turbostat21:49:540
3156599930928,2cyclictest10848-21turbostat23:44:552
3156599917915,2cyclictest31826-21turbostat20:34:542
3156599905902,3cyclictest4482-21turbostat00:09:562
3157099894892,2cyclictest21230-21turbostat21:19:553
3156299889887,2cyclictest14005-21turbostat21:09:541
3157099883881,2cyclictest20076-21turbostat00:24:563
3155799877874,3cyclictest2390-21turbostat22:34:560
3157099864862,2cyclictest9611-21turbostat19:39:563
3157099688685,3cyclictest10848-21turbostat23:44:553
3157099673672,1cyclictest31826-21turbostat20:34:543
3157099662659,3cyclictest4482-21turbostat00:09:563
3157099659656,3cyclictest14786-21turbostat21:44:543
3156599633632,1cyclictest20076-21turbostat00:24:562
3156299619617,2cyclictest29583-21turbostat22:29:551
3156599617615,2cyclictest9611-21turbostat19:39:562
3156299602598,3cyclictest1436-21turbostat20:39:551
3157099572569,3cyclictest3167-21turbostat22:04:553
3155799565562,3cyclictest14005-21turbostat21:09:540
3156599563562,1cyclictest21237-21turbostat23:54:562
3157099556554,2cyclictest11586-21turbostat23:14:563
3155799550548,2cyclictest25183-21turbostat21:54:540
3155799549548,1cyclictest15664-21turbostat19:54:550
3155799549547,2cyclictest27637-21turbostat23:29:550
3155799538537,1cyclictest28387-21turbostat22:59:560
3156299530527,3cyclictest9663-21turbostat00:14:561
3157099513512,1cyclictest3449-21turbostat20:44:543
3155799417412,4cyclictest31653-21turbostat00:04:550
31570991918,1cyclictest67750irq/123-lan23:35:123
31570991918,1cyclictest67750irq/123-lan22:35:103
31570991918,1cyclictest67750irq/123-lan21:57:233
31570991918,1cyclictest67750irq/123-lan21:35:003
31570991818,0cyclictest67750irq/123-lan21:27:493
31570991818,0cyclictest67750irq/123-lan21:10:403
31570991817,1cyclictest67750irq/123-lan23:58:553
31570991817,1cyclictest67750irq/123-lan23:30:133
31570991817,1cyclictest67750irq/123-lan21:38:413
31570991817,1cyclictest67750irq/123-lan20:15:173
31570991817,1cyclictest67750irq/123-lan19:30:213
31570991817,1cyclictest67750irq/123-lan00:34:263
31570991817,1cyclictest67750irq/123-lan00:00:163
31570991717,0cyclictest67750irq/123-lan22:40:113
31570991717,0cyclictest67750irq/123-lan20:25:123
31570991716,1cyclictest67750irq/123-lan23:08:493
31570991716,1cyclictest67750irq/123-lan22:30:183
31570991716,1cyclictest67750irq/123-lan19:55:163
31570991716,1cyclictest67750irq/123-lan00:17:243
31570991616,0cyclictest67750irq/123-lan23:00:113
31570991616,0cyclictest67750irq/123-lan22:48:133
31570991616,0cyclictest67750irq/123-lan19:15:083
31570991616,0cyclictest67750irq/123-lan00:25:473
31570991615,1cyclictest67750irq/123-lan00:36:043
31570991515,0cyclictest67750irq/123-lan19:45:193
31570991515,0cyclictest67750irq/123-lan19:40:273
31570991514,1cyclictest67750irq/123-lan22:50:183
31570991514,1cyclictest67750irq/123-lan21:23:123
31570991414,0cyclictest67750irq/123-lan20:20:183
31570991413,1cyclictest67750irq/123-lan20:00:153
31565991413,1cyclictest784-21snmpd22:41:242
31557991414,0cyclictest784-21snmpd00:33:500
31570991313,0cyclictest67750irq/123-lan19:10:113
3156299130,13cyclictest9620-21latency_hist20:54:591
31570991111,0cyclictest67750irq/123-lan20:10:153
31570991110,1cyclictest67750irq/123-lan20:05:173
31557991010,0cyclictest2231-21cat19:15:130
315659999,0cyclictest782-21lldpd22:47:202
315629998,1cyclictest784-21snmpd21:46:561
315629998,1cyclictest782-21lldpd23:46:481
315629998,1cyclictest782-21lldpd19:48:361
315579998,1cyclictest784-21snmpd20:38:480
315709987,1cyclictest67750irq/123-lan19:20:103
315659988,0cyclictest784-21snmpd22:13:332
315659988,0cyclictest784-21snmpd21:01:452
315659988,0cyclictest784-21snmpd20:11:402
315659986,2cyclictest784-21snmpd23:27:512
315659983,4cyclictest5787-21cat19:25:002
315659983,4cyclictest30423-21cat21:55:012
315629988,0cyclictest784-21snmpd20:20:021
315629988,0cyclictest784-21snmpd20:05:041
315629988,0cyclictest782-21lldpd23:02:171
315629987,1cyclictest526-21avahi-daemon23:42:581
315629987,1cyclictest507-21systemd-resolve23:27:331
315629987,1cyclictest31692-21cut21:55:181
315629986,2cyclictest784-21snmpd21:16:161
315629986,2cyclictest784-21snmpd21:01:271
315629986,2cyclictest784-21snmpd19:52:401
315629986,2cyclictest784-21snmpd19:42:251
315629986,2cyclictest22104-21cat23:50:151
315579988,0cyclictest784-21snmpd23:58:270
315579988,0cyclictest784-21snmpd23:53:130
315579988,0cyclictest784-21snmpd22:42:150
315579988,0cyclictest784-21snmpd20:55:250
315579988,0cyclictest784-21snmpd20:50:240
315579988,0cyclictest784-21snmpd19:23:190
315579988,0cyclictest782-21lldpd22:48:230
315579987,1cyclictest784-21snmpd19:45:160
315579987,1cyclictest782-21lldpd19:31:000
315579986,2cyclictest784-21snmpd22:20:060
315579986,2cyclictest762-21ntpd22:52:510
315659977,0cyclictest784-21snmpd19:29:212
315659977,0cyclictest784-21snmpd19:18:242
315659977,0cyclictest782-21lldpd23:15:352
315659977,0cyclictest782-21lldpd19:58:492
315659976,1cyclictest9058-21gltestperf22:05:122
315659976,1cyclictest526-21avahi-daemon22:57:502
315659975,2cyclictest784-21snmpd23:00:342
315659975,2cyclictest784-21snmpd21:11:392
315659975,2cyclictest784-21snmpd20:22:262
315659975,2cyclictest10433-21cat21:35:132
315659974,2cyclictest9547-21ssh22:35:312
315659974,2cyclictest24743-21ssh21:49:252
315659973,3cyclictest547-21NetworkManager20:05:012
315659973,3cyclictest31918-21ssh22:26:002
315659973,3cyclictest23670-21ssh22:19:032
315659972,4cyclictest5709-21cat23:35:012
315629977,0cyclictest784-21snmpd00:34:201
315629977,0cyclictest782-21lldpd22:23:501
315629977,0cyclictest782-21lldpd22:16:111
315629977,0cyclictest782-21lldpd20:49:491
315629977,0cyclictest782-21lldpd20:01:461
315629977,0cyclictest782-21lldpd00:29:131
315629976,1cyclictest784-21snmpd23:18:201
315629976,1cyclictest784-21snmpd22:01:531
315629976,1cyclictest784-21snmpd00:00:061
315629976,1cyclictest782-21lldpd21:22:351
315629976,1cyclictest507-21systemd-resolve00:06:271
315629976,1cyclictest21227-21dump-pmu-power21:14:591
315629975,2cyclictest784-21snmpd20:58:251
315629973,3cyclictest29133-21ssh21:53:181
315629972,2cyclictest10909-21cat23:40:001
315629970,6cyclictest16116-21ssh23:14:061
315579977,0cyclictest784-21snmpd22:36:520
315579977,0cyclictest784-21snmpd00:39:260
315579977,0cyclictest10488-21cat00:10:140
315579976,1cyclictest784-21snmpd21:00:360
315579976,1cyclictest762-21ntpd22:12:500
315579976,1cyclictest526-21avahi-daemon23:20:540
315579976,1cyclictest507-21systemd-resolve00:26:460
315579975,2cyclictest784-21snmpd20:47:380
315579975,2cyclictest762-21ntpd19:10:020
315579975,2cyclictest762-21ntpd00:20:250
315579975,2cyclictest762-21ntpd00:18:390
315579973,3cyclictest10733-21ssh22:06:050
315659966,0cyclictest782-21lldpd22:33:232
315659966,0cyclictest782-21lldpd20:53:492
315659966,0cyclictest15361-21cpuspeed_turbos00:15:102
315659965,1cyclictest784-21snmpd20:45:022
315659964,2cyclictest784-21snmpd20:18:362
315659963,2cyclictest9576-21ssh23:07:132
315659963,2cyclictest29603-21awk22:25:002
315659963,2cyclictest23327-21ssh21:15:422
315659963,2cyclictest19729-21ssh21:44:392
315659963,2cyclictest18473-21ssh23:46:102
315659963,2cyclictest1504-21ssh21:57:432
315659962,3cyclictest6329-21fw_forwarded_lo23:35:122
315659962,3cyclictest30380-21ssh21:23:172
315659962,3cyclictest26217-21ssh23:23:032
315659962,3cyclictest12139-21cat21:00:002
315629966,0cyclictest784-21snmpd21:39:171
315629966,0cyclictest782-21lldpd20:33:211
315629966,0cyclictest526-21avahi-daemon20:29:241
315629963,2cyclictest9318-21ssh23:06:511
315629963,2cyclictest5552-21ssh23:34:551
315629963,2cyclictest17165-21ssh22:12:501
315629962,3cyclictest4674-21grep00:35:141
315579963,3cyclictest547-21NetworkManager23:16:320
315659955,0cyclictest782-21lldpd20:35:582
315659955,0cyclictest782-21lldpd19:31:062
315659953,2cyclictest784-21snmpd20:27:172
315659952,3cyclictest547-21NetworkManager20:00:012
315659952,2cyclictest4874-21grep21:30:132
315659952,2cyclictest25834-21ssh22:51:282
315659952,2cyclictest12806-21ssh00:12:092
315659951,3cyclictest14644-21sh21:05:142
315629954,1cyclictest526-21avahi-daemon22:05:491
315629953,2cyclictest784-21snmpd20:15:051
315629952,2cyclictest5785-21cut21:30:211
315629952,2cyclictest507-21systemd-resolve20:12:271
315629951,3cyclictest1071-21users21:25:211
315629950,4cyclictest18225-21fw_forwarded_lo19:55:141
315579954,1cyclictest784-21snmpd20:41:380
315579954,1cyclictest526-21avahi-daemon19:40:130
315579953,2cyclictest762-21ntpd19:28:300
315579952,2cyclictest20185-21grep22:15:160
315579950,4cyclictest17313-21ntpq23:45:170
315659940,4cyclictest32451-21ntpq19:10:142
315629940,3cyclictest27001-21diskstats23:55:121
315579941,2cyclictest10144-21grep19:35:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional