You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-07 - 21:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Mon Sep 07, 2026 12:43:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
101779910541050,4cyclictest11515-21kworker/3:0+events10:05:433
101779910521050,2cyclictest6497-21kworker/3:2+events09:42:313
101779910521049,3cyclictest25629-21kworker/3:0+events11:37:133
101779910511049,2cyclictest6497-21kworker/3:2+events09:39:243
101779910511049,2cyclictest25629-21kworker/3:0+events11:29:213
101779910511049,2cyclictest21149-21kworker/3:1+events10:00:223
101779910511049,2cyclictest15627-21kworker/3:2+events10:45:453
101779910511049,2cyclictest15627-21kworker/3:2+events10:44:473
101599910511049,2cyclictest9346-21kworker/0:1+events11:53:150
101779910501048,2cyclictest27546-21kworker/3:0+events09:22:563
101779910501048,2cyclictest20442-21kworker/3:1+events11:06:273
101779910501048,2cyclictest20442-21kworker/3:1+events10:51:053
101779910501048,2cyclictest12714-21kworker/3:2+events08:38:293
101649910501049,1cyclictest4583-21kworker/1:2+events08:12:201
101779910491048,1cyclictest25629-21kworker/3:0+events11:30:203
101779910491047,2cyclictest6586-21kworker/3:2+events11:41:283
101779910491047,2cyclictest21149-21kworker/3:1+events09:48:163
101779910491046,3cyclictest4154-21kworker/3:0+events10:38:273
101779910491046,2cyclictest30693-21kworker/3:0+events11:03:393
101729910491045,3cyclictest12629-21kworker/2:1+events12:03:222
101779910481046,2cyclictest6497-21kworker/3:2+events09:33:293
101779910481046,2cyclictest17692-21kworker/3:1+events10:13:303
101779910471045,2cyclictest6586-21kworker/3:2+events11:58:523
101779910471045,2cyclictest312-21kworker/3:1+events09:16:293
101779910471045,2cyclictest20442-21kworker/3:1+events11:15:223
101779910471045,2cyclictest15627-21kworker/3:2+events10:55:433
101779910471044,3cyclictest27546-21kworker/3:0+events09:10:453
101729910471045,2cyclictest11427-21kworker/2:0+events12:22:412
101729910471044,2cyclictest6448-21kworker/2:1+events12:16:282
101779910461044,2cyclictest8541-21kworker/3:2+events08:20:543
101779910461044,2cyclictest20442-21kworker/3:1+events11:21:283
101729910461043,2cyclictest23395-21kworker/2:2+events12:13:372
101779910441043,1cyclictest24838-21kworker/3:2+events07:42:533
101729910441042,2cyclictest22766-21turbostat12:39:552
101729910441042,2cyclictest12629-21kworker/2:1+events12:05:422
101779910431041,2cyclictest21149-21kworker/3:1+events09:55:453
101779910431041,2cyclictest10589-21kworker/3:0+events08:30:313
101779910421040,2cyclictest4154-21kworker/3:0+events10:34:463
101779910421040,2cyclictest26875-21kworker/3:1+events08:18:363
101779910401038,2cyclictest11056-21kworker/3:2+events11:13:213
101779910371036,1cyclictest11969-21kworker/3:1+events07:17:453
101779910361034,2cyclictest16508-21kworker/3:0+events07:33:403
101599910361034,2cyclictest4502-21turbostat09:24:550
101779910351033,2cyclictest7391-21kworker/3:0+events07:11:263
101779910311030,1cyclictest12714-21kworker/3:2+events08:42:013
101599910291028,1cyclictest21059-21kworker/0:0+events10:17:340
101649910281026,2cyclictest32027-21turbostat10:29:561
101599910241022,2cyclictest21059-21kworker/0:0+events12:28:420
101599910201017,3cyclictest22955-21turbostat11:29:550
101779910141011,3cyclictest8893-21turbostat12:24:553
101729910111009,2cyclictest22796-21turbostat10:19:552
101599910111010,1cyclictest20124-21kworker/0:1+events12:33:380
101779910041003,1cyclictest24838-21kworker/3:2+events07:48:503
1015999996995,1cyclictest23443-21kworker/0:1+events10:24:410
1015999989988,1cyclictest21059-21kworker/0:0+events09:04:220
1017799986985,1cyclictest21149-21kworker/3:1+events09:54:133
1015999985983,1cyclictest20124-21kworker/0:1+events12:36:490
1015999984983,1cyclictest21059-21kworker/0:0+events08:57:120
1015999982981,1cyclictest21059-21kworker/0:0+events11:47:310
1015999981980,1cyclictest21059-21kworker/0:0+events09:08:080
1017799972970,2cyclictest11969-21kworker/3:1+events07:27:533
1015999968966,2cyclictest21040-21turbostat07:39:550
1015999948946,2cyclictest6872-21turbostat08:24:550
1017299947945,2cyclictest25130-21turbostat09:09:552
1016499940939,1cyclictest16089-21kworker/1:0+events07:59:031
1017799939937,2cyclictest12714-21kworker/3:2+events08:50:433
1016499937936,1cyclictest27395-21kworker/1:2+events08:06:141
1016499933932,1cyclictest16089-21kworker/1:0+events07:51:051
1016499927926,1cyclictest27395-21kworker/1:2+events08:01:071
1015999924922,2cyclictest8792-21turbostat10:39:560
1015999904901,3cyclictest13702-21turbostat11:19:550
1017799901898,3cyclictest27384-21turbostat10:24:563
1017799898897,1cyclictest27546-21kworker/3:0+events09:26:523
1017299897894,3cyclictest16589-21turbostat07:29:542
1016499895891,4cyclictest10927-21turbostat08:34:551
1017799885884,1cyclictest16508-21kworker/3:0+events07:36:513
1017799881881,0cyclictest26875-21kworker/3:1+events08:27:213
1017799881880,1cyclictest11969-21kworker/3:1+events07:20:433
1017799878878,0cyclictest26875-21kworker/3:1+events08:48:143
1015999662660,2cyclictest8893-21turbostat12:24:550
1017799621619,2cyclictest22766-21turbostat12:39:553
1016499609606,3cyclictest25154-21turbostat07:49:541
1015999591590,1cyclictest18193-21turbostat10:14:550
1017799555554,1cyclictest2835-21turbostat08:14:553
1017799550546,4cyclictest13617-21turbostat11:54:553
1017299542539,3cyclictest18218-21turbostat11:59:552
1015999533530,3cyclictest19079-21turbostat08:54:550
1017799530528,2cyclictest32027-21turbostat10:29:563
1017799526523,3cyclictest25130-21turbostat09:09:553
1016499506505,1cyclictest27384-21turbostat10:24:561
1015999436434,2cyclictest4348-21turbostat11:44:550
10177991918,1cyclictest67750irq/123-lan12:30:163
10177991817,1cyclictest67750irq/123-lan12:17:303
10177991817,1cyclictest67750irq/123-lan12:09:593
10177991817,1cyclictest67750irq/123-lan12:00:273
10177991817,1cyclictest67750irq/123-lan07:52:143
10177991717,0cyclictest67750irq/123-lan12:10:183
10177991716,1cyclictest67750irq/123-lan09:00:123
10177991616,0cyclictest67750irq/123-lan10:16:293
10177991615,1cyclictest67750irq/123-lan12:25:133
10177991615,1cyclictest67750irq/123-lan11:46:513
10177991414,0cyclictest67750irq/123-lan08:55:103
10177991413,1cyclictest67750irq/123-lan08:05:123
10177991313,0cyclictest67750irq/123-lan07:57:143
10177991110,1cyclictest67750irq/123-lan08:00:143
101649996,3cyclictest784-21snmpd11:03:351
101729988,0cyclictest784-21snmpd10:32:432
101729988,0cyclictest784-21snmpd09:23:582
101729988,0cyclictest784-21snmpd08:05:342
101729988,0cyclictest782-21lldpd08:17:372
101729986,2cyclictest784-21snmpd11:12:412
101729986,2cyclictest784-21snmpd09:50:532
101729986,2cyclictest784-21snmpd09:33:322
101729986,2cyclictest784-21snmpd08:37:302
101729986,2cyclictest21786-21cat08:55:132
101729986,2cyclictest19169-21cat09:35:152
101649988,0cyclictest782-21lldpd12:23:461
101649988,0cyclictest782-21lldpd12:17:021
101649988,0cyclictest782-21lldpd11:52:341
101649988,0cyclictest782-21lldpd11:31:111
101649988,0cyclictest782-21lldpd10:52:371
101649988,0cyclictest782-21lldpd10:03:101
101649988,0cyclictest782-21lldpd08:44:271
101649988,0cyclictest782-21lldpd07:26:131
101649987,1cyclictest784-21snmpd10:35:101
101649987,1cyclictest526-21avahi-daemon09:32:491
101649987,1cyclictest526-21avahi-daemon07:41:151
101649987,1cyclictest507-21systemd-resolve10:47:141
101649986,2cyclictest784-21snmpd11:37:091
101649986,2cyclictest784-21snmpd09:57:561
101649986,2cyclictest784-21snmpd08:38:241
101649986,2cyclictest784-21snmpd08:23:051
101649985,2cyclictest784-21snmpd09:45:111
101599988,0cyclictest784-21snmpd10:05:060
101599988,0cyclictest784-21snmpd08:29:140
101599988,0cyclictest784-21snmpd08:11:440
101599988,0cyclictest784-21snmpd07:56:280
101599988,0cyclictest782-21lldpd11:55:240
101599988,0cyclictest782-21lldpd10:31:250
101599988,0cyclictest782-21lldpd09:53:180
101599988,0cyclictest782-21lldpd09:31:340
101599988,0cyclictest782-21lldpd07:42:480
101599987,1cyclictest784-21snmpd11:37:300
101599987,1cyclictest784-21snmpd10:50:520
101599987,1cyclictest784-21snmpd09:28:360
101599987,1cyclictest784-21snmpd08:08:340
101599987,1cyclictest782-21lldpd11:06:130
101599986,2cyclictest784-21snmpd12:18:250
101599986,2cyclictest784-21snmpd11:24:590
101599986,2cyclictest784-21snmpd08:49:350
101599986,2cyclictest784-21snmpd07:15:420
101729977,0cyclictest784-21snmpd11:04:082
101729977,0cyclictest784-21snmpd10:46:282
101729977,0cyclictest784-21snmpd07:53:252
101729977,0cyclictest784-21snmpd07:21:152
101729977,0cyclictest784-21snmpd07:18:512
101729977,0cyclictest784-21snmpd07:14:152
101729977,0cyclictest782-21lldpd11:19:552
101729976,1cyclictest9475-21gltestperf10:35:142
101729976,1cyclictest784-21snmpd11:22:292
101729976,1cyclictest507-21systemd-resolve11:31:282
101729976,1cyclictest10155-21cyclictest09:20:002
101729975,2cyclictest784-21snmpd12:27:082
101729975,2cyclictest784-21snmpd08:43:422
101729975,2cyclictest784-21snmpd07:35:452
101649977,0cyclictest784-21snmpd11:45:301
101649977,0cyclictest784-21snmpd09:52:411
101649977,0cyclictest782-21lldpd09:17:181
101649977,0cyclictest782-21lldpd08:29:141
101649977,0cyclictest782-21lldpd07:24:241
101649976,1cyclictest526-21avahi-daemon07:32:091
101649976,1cyclictest507-21systemd-resolve10:32:521
101649975,2cyclictest784-21snmpd08:56:571
101649973,3cyclictest30957-21ssh10:57:501
101649973,3cyclictest29286-21users12:05:221
101649973,3cyclictest15393-21ssh11:15:211
101599977,0cyclictest782-21lldpd12:14:520
101599977,0cyclictest782-21lldpd09:16:340
101599976,1cyclictest784-21snmpd12:05:370
101599976,1cyclictest526-21avahi-daemon09:47:500
101599976,1cyclictest526-21avahi-daemon07:45:180
101599975,2cyclictest784-21snmpd10:42:400
101599975,2cyclictest784-21snmpd07:12:290
101599975,2cyclictest762-21ntpd10:03:290
101599972,4cyclictest29294-21cat07:55:000
101729966,0cyclictest782-21lldpd10:54:492
101729966,0cyclictest782-21lldpd07:43:562
101729966,0cyclictest526-21avahi-daemon08:00:202
101729965,1cyclictest784-21snmpd10:21:052
101729965,1cyclictest526-21avahi-daemon11:52:312
101729965,1cyclictest526-21avahi-daemon09:04:042
101729964,2cyclictest19740-21cat07:30:142
101729963,2cyclictest3426-21head08:10:132
101729963,2cyclictest31193-21ssh10:58:142
101729963,2cyclictest27008-21ssh11:29:072
101729963,2cyclictest17485-21ssh10:08:502
101729962,4cyclictest547-21NetworkManager11:42:192
101729962,4cyclictest547-21NetworkManager08:49:442
101729962,3cyclictest2387-21ssh11:36:432
101729962,3cyclictest11552-21ssh09:26:202
101649966,0cyclictest784-21snmpd10:16:331
101649966,0cyclictest782-21lldpd12:25:561
101649965,1cyclictest784-21snmpd09:06:421
101649965,1cyclictest18810-21fschecks_time12:30:121
101649964,2cyclictest784-21snmpd11:11:321
101649963,3cyclictest547-21NetworkManager12:03:451
101649963,3cyclictest547-21NetworkManager11:43:211
101649962,3cyclictest5411-21fw_forwarded_lo08:15:131
101649962,3cyclictest20226-21ssh11:55:411
101599966,0cyclictest784-21snmpd07:27:360
101599966,0cyclictest782-21lldpd09:44:100
101599966,0cyclictest526-21avahi-daemon11:11:400
101599966,0cyclictest526-21avahi-daemon10:46:300
101599965,1cyclictest784-21snmpd10:55:130
101599965,1cyclictest762-21ntpd09:55:350
101599965,1cyclictest762-21ntpd08:40:230
101599964,2cyclictest784-21snmpd09:35:080
101599964,2cyclictest784-21snmpd08:17:500
101599964,2cyclictest762-21ntpd08:00:150
101599963,3cyclictest547-21NetworkManager11:00:020
101599963,3cyclictest507-21systemd-resolve07:20:270
101599963,2cyclictest985-21awk10:25:190
101599963,0cyclictest31992-21ssh09:14:360
101599962,3cyclictest12468-21grep08:30:200
101599960,6cyclictest574-21rs:main0
101729955,0cyclictest784-21snmpd07:59:582
101729955,0cyclictest782-21lldpd10:43:112
101729955,0cyclictest25675-21fschecks_time07:45:122
101729954,1cyclictest18705-21fschecks_count12:30:122
101729953,2cyclictest5328-21cat11:05:152
101729952,3cyclictest28105-21df_inode09:10:122
101729952,2cyclictest7549-21sh08:20:142
101729952,2cyclictest574-21rs:main2
101729952,2cyclictest547-21NetworkManager09:56:132
101729952,2cyclictest30344-21ssh09:46:552
101729952,2cyclictest25625-21sh09:41:432
101729952,2cyclictest20520-21sed08:50:232
101729952,2cyclictest1534-21ssh10:25:302
101729952,2cyclictest11201-21ssh10:00:572
101649954,1cyclictest526-21avahi-daemon09:27:551
101649954,1cyclictest331ktimers/109:10:021
101649954,1cyclictest23447-21cpuspeed_turbos09:00:121
101649954,1cyclictest13160-21ntpq07:15:171
101649953,2cyclictest784-21snmpd12:35:261
101649953,2cyclictest784-21snmpd08:50:091
101649953,2cyclictest547-21NetworkManager11:07:281
101649953,2cyclictest547-21NetworkManager09:20:191
101649952,2cyclictest574-21rs:main1
101649952,2cyclictest2720-21ssh12:12:241
101649952,2cyclictest24852-21sshd09:40:251
101649952,2cyclictest20775-21ssh09:36:231
101649952,2cyclictest18193-21grep08:45:161
101649952,2cyclictest14888-21find10:40:201
101649951,3cyclictest18864-21diskstats11:20:111
101649950,4cyclictest14880-21ntpq10:05:171
101599955,0cyclictest782-21lldpd11:33:050
101599955,0cyclictest526-21avahi-daemon07:30:350
101729941,2cyclictest9893-21uname11:45:162
101729940,3cyclictest11354-21diskstats08:30:122
101729940,3cyclictest10005-21ntp_states08:25:172
101649940,4cyclictest11119-21ntpq07:10:171
101649940,3cyclictest22149-21ld07:35:161
101649940,3cyclictest18741-21diskstats10:10:141
101599941,2cyclictest13339-21grep08:35:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional