You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-14 - 02:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Sun Sep 13, 2026 12:43:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
189069910531050,2cyclictest5651-21kworker/3:0+events08:00:223
189069910531050,2cyclictest11175-21kworker/3:1+events10:45:223
189069910521050,2cyclictest26598-21kworker/3:1+events07:15:213
189069910521050,2cyclictest11016-21kworker/3:2+events08:31:383
189069910521049,3cyclictest5651-21kworker/3:0+events08:14:303
189069910521049,2cyclictest16831-21kworker/3:1+events11:15:243
189069910511049,2cyclictest8429-21kworker/3:0+events10:06:303
189069910511049,2cyclictest8429-21kworker/3:0+events10:03:213
189069910511049,2cyclictest25467-21kworker/3:0+events12:30:553
189069910511049,2cyclictest16831-21kworker/3:1+events11:10:453
189069910511049,2cyclictest14623-21kworker/3:2+events07:52:083
189069910511049,2cyclictest11175-21kworker/3:1+events10:52:583
189069910511049,2cyclictest11016-21kworker/3:2+events08:27:393
189069910511048,3cyclictest1991-21kworker/3:2+events12:06:523
189069910511048,3cyclictest19520-21kworker/3:2+events12:29:473
189069910511047,3cyclictest6062-21kworker/3:2+events10:28:083
189069910501049,1cyclictest25596-21kworker/3:0+events09:08:123
189069910501048,2cyclictest26598-21kworker/3:1+events07:11:143
189069910501048,2cyclictest11016-21kworker/3:2+events08:17:333
189069910501047,3cyclictest8429-21kworker/3:0+events09:56:283
189069910491048,1cyclictest3058-21kworker/3:1+events07:55:223
189069910491047,2cyclictest8429-21kworker/3:0+events10:10:223
189069910491047,2cyclictest5808-21kworker/3:2+events09:16:263
189069910491047,2cyclictest27645-21kworker/3:1+events09:01:133
189069910491047,2cyclictest19520-21kworker/3:2+events12:38:293
189069910491047,2cyclictest14623-21kworker/3:2+events07:34:323
189069910481046,2cyclictest5808-21kworker/3:2+events09:14:053
189069910481046,2cyclictest27645-21kworker/3:1+events08:49:243
189069910481046,2cyclictest25596-21kworker/3:0+events08:52:293
189069910481046,2cyclictest11016-21kworker/3:2+events08:40:223
189069910471045,2cyclictest8429-21kworker/3:0+events09:50:323
189069910471045,2cyclictest14623-21kworker/3:2+events07:29:053
189069910461044,2cyclictest1189-21kworker/3:1+events12:21:353
189069910451042,3cyclictest24977-21kworker/3:0+events07:38:593
189069910441042,2cyclictest8429-21kworker/3:0+events10:18:413
189069910441042,2cyclictest24250-21kworker/3:1+events10:21:143
189069910441041,3cyclictest3058-21kworker/3:1+events08:05:213
189069910441041,3cyclictest11016-21kworker/3:2+events08:36:263
189069910421040,2cyclictest14623-21kworker/3:2+events07:23:373
189069910421040,2cyclictest11016-21kworker/3:2+events08:21:273
188909910411039,2cyclictest1427-21turbostat09:09:550
189069910401037,3cyclictest27645-21kworker/3:1+events08:57:323
188909910371034,3cyclictest24692-21turbostat11:59:550
189069910361034,2cyclictest27645-21kworker/3:1+events09:22:123
188979910301028,2cyclictest30162-21turbostat10:54:551
188979910291027,2cyclictest2161-21kworker/1:1+events09:45:211
188979910271026,1cyclictest24523-21kworker/1:0+events09:29:451
188979910261025,1cyclictest29686-21kworker/1:0+events09:39:281
188909910251024,1cyclictest29366-21turbostat07:39:550
188909910211020,1cyclictest12579-21kworker/0:1+events10:32:380
188979910201019,1cyclictest25004-21kworker/1:2+events09:44:501
188979910201018,1cyclictest29536-21kworker/1:2+events12:10:211
188979910171015,1cyclictest14172-21kworker/1:1+events12:00:221
189069910151013,2cyclictest12266-21turbostat10:34:553
188979910141013,1cyclictest18733-21kworker/1:1+events09:34:001
188979910141013,1cyclictest14172-21kworker/1:1+events11:07:001
188909910071007,0cyclictest30232-21kworker/0:1+events11:41:040
189069910051002,3cyclictest25416-21turbostat11:24:553
189029910051002,3cyclictest22867-21turbostat07:24:542
188909910051003,2cyclictest30232-21kworker/0:1+events11:33:000
188979910031003,0cyclictest2580-21kworker/1:2+events10:57:561
188979910031002,1cyclictest25004-21kworker/1:2+events10:35:221
188909910031002,1cyclictest18459-21kworker/0:2+events11:52:330
188909910021001,1cyclictest8982-21kworker/0:0+events11:37:430
188909910021001,1cyclictest10087-21kworker/0:0+events12:19:120
188979910011000,1cyclictest17254-21kworker/1:0+events10:41:591
18890991000999,1cyclictest16534-21kworker/0:2+events11:26:480
18890991000999,1cyclictest16534-21kworker/0:2+events07:48:260
1889099999998,1cyclictest18459-21kworker/0:2+events11:49:260
1889799997996,1cyclictest2580-21kworker/1:2+events11:01:061
1889799997996,1cyclictest2580-21kworker/1:2+events11:01:061
1889099992991,1cyclictest31761-21kworker/0:1+events07:42:120
1890299987984,3cyclictest8499-21turbostat09:54:562
1890299958956,2cyclictest21194-21turbostat10:44:562
1890299957956,1cyclictest25328-21kworker/2:1+events11:20:432
1889099956954,2cyclictest20839-21turbostat07:19:540
1890299954952,2cyclictest5703-21turbostat12:14:552
1890299953950,3cyclictest4002-21turbostat09:49:562
1890299934931,3cyclictest10145-21turbostat12:19:542
1889099929926,3cyclictest27969-21turbostat12:39:540
1890299927924,3cyclictest17382-21turbostat10:04:562
1889099920918,2cyclictest5105-21turbostat07:59:550
1889799888887,1cyclictest20938-21turbostat11:19:561
1889099866862,3cyclictest27338-21turbostat07:34:540
1890699702700,2cyclictest1078-21turbostat07:49:543
1890699680678,2cyclictest29135-21turbostat12:04:553
1890299638637,1cyclictest27969-21turbostat12:39:542
1889099629626,3cyclictest7363-21turbostat10:29:560
1889799615612,3cyclictest1251-21turbostat12:09:551
1890699614611,3cyclictest21194-21turbostat10:44:563
1890699606605,1cyclictest4002-21turbostat09:49:563
1889799606604,2cyclictest13390-21turbostat09:24:551
1889799604603,1cyclictest24692-21turbostat11:59:541
1890699591590,1cyclictest12028-21turbostat11:09:563
1890699587586,1cyclictest10145-21turbostat12:19:543
1890299587584,3cyclictest20938-21turbostat11:19:562
1889099553550,3cyclictest5703-21turbostat12:14:550
1889799527524,3cyclictest12266-21turbostat10:34:551
1889099501499,2cyclictest25416-21turbostat11:24:550
18906991918,1cyclictest67750irq/123-lan12:10:173
18906991818,0cyclictest67750irq/123-lan11:40:183
18906991818,0cyclictest67750irq/123-lan11:33:103
18906991818,0cyclictest67750irq/123-lan11:28:123
18906991817,1cyclictest67750irq/123-lan11:55:193
18906991817,1cyclictest67750irq/123-lan11:50:203
18906991717,0cyclictest67750irq/123-lan09:36:223
18906991717,0cyclictest67750irq/123-lan09:31:413
18906991717,0cyclictest67750irq/123-lan09:29:303
18906991716,1cyclictest67750irq/123-lan11:35:143
18906991716,1cyclictest67750irq/123-lan10:56:573
18906991616,0cyclictest67750irq/123-lan10:35:163
18906991616,0cyclictest67750irq/123-lan09:40:073
18906991616,0cyclictest67750irq/123-lan07:43:363
18906991515,0cyclictest67750irq/123-lan11:49:473
18906991514,1cyclictest67750irq/123-lan11:03:113
18906991514,1cyclictest67750irq/123-lan11:03:103
18890991515,0cyclictest784-21snmpd08:42:300
18902991412,2cyclictest762-21ntpd10:27:112
18890991212,0cyclictest784-21snmpd07:23:250
18890991111,0cyclictest784-21snmpd08:05:410
18890991111,0cyclictest784-21snmpd07:51:370
18890991111,0cyclictest13772-21cat09:55:150
18890991010,0cyclictest8351-21cat11:00:140
18890991010,0cyclictest8351-21cat11:00:140
18890991010,0cyclictest784-21snmpd11:06:190
18890991010,0cyclictest784-21snmpd10:46:280
18890991010,0cyclictest784-21snmpd10:43:550
18890991010,0cyclictest784-21snmpd10:01:410
18890991010,0cyclictest784-21snmpd09:47:110
18890991010,0cyclictest784-21snmpd09:02:540
18890991010,0cyclictest28449-21cat08:50:140
18890991010,0cyclictest21745-21cat11:15:150
189029998,1cyclictest784-21snmpd08:26:442
188979997,0cyclictest191rcu_preempt11:51:231
188979997,0cyclictest191rcu_preempt09:15:471
188979992,4cyclictest20699-21wc12:25:211
188909999,0cyclictest784-21snmpd10:55:360
188909999,0cyclictest784-21snmpd10:16:200
188909999,0cyclictest784-21snmpd10:11:590
188909999,0cyclictest784-21snmpd10:05:090
188909999,0cyclictest784-21snmpd08:12:170
188909999,0cyclictest784-21snmpd07:10:480
188909999,0cyclictest32694-21cat09:40:130
189029988,0cyclictest784-21snmpd08:50:162
189029988,0cyclictest784-21snmpd07:19:132
189029988,0cyclictest782-21lldpd11:51:482
189029988,0cyclictest782-21lldpd11:48:062
189029987,1cyclictest784-21snmpd12:29:492
189029987,1cyclictest784-21snmpd11:12:022
189029987,1cyclictest784-21snmpd10:48:422
189029987,1cyclictest784-21snmpd10:09:452
189029987,1cyclictest526-21avahi-daemon10:31:332
189029986,2cyclictest784-21snmpd12:03:332
189029986,2cyclictest784-21snmpd08:42:392
189029986,2cyclictest784-21snmpd08:16:212
189029984,3cyclictest26931-21diskstats10:10:162
188979988,0cyclictest784-21snmpd08:21:231
188979988,0cyclictest784-21snmpd07:38:471
188979988,0cyclictest782-21lldpd07:22:121
188979987,1cyclictest782-21lldpd11:31:421
188979987,1cyclictest526-21avahi-daemon10:27:521
188979987,0cyclictest191rcu_preempt10:15:411
188979986,2cyclictest784-21snmpd12:38:071
188979986,2cyclictest784-21snmpd08:09:081
188979985,0cyclictest191rcu_preempt11:38:271
188979984,2cyclictest34-21ksoftirqd/110:04:291
188979983,4cyclictest526-21avahi-daemon11:21:251
188909988,0cyclictest784-21snmpd09:18:040
188909988,0cyclictest782-21lldpd10:51:220
188909988,0cyclictest782-21lldpd09:21:010
188909987,1cyclictest784-21snmpd12:29:250
188909986,2cyclictest784-21snmpd08:33:120
188909986,2cyclictest762-21ntpd08:23:270
189029977,0cyclictest784-21snmpd09:55:292
189029977,0cyclictest784-21snmpd07:42:432
189029977,0cyclictest25937-21gltestperf07:25:132
189029976,1cyclictest9483-21cpuspeed_turbos08:05:132
189029976,1cyclictest784-21snmpd10:59:382
189029976,1cyclictest526-21avahi-daemon12:20:472
189029976,1cyclictest526-21avahi-daemon09:38:182
189029976,1cyclictest526-21avahi-daemon09:14:482
189029976,1cyclictest526-21avahi-daemon08:36:372
189029976,1cyclictest1594-21fschecks_time07:45:132
189029976,1cyclictest12459-21cpuspeed_turbos11:05:112
189029975,2cyclictest784-21snmpd09:03:512
189029975,2cyclictest784-21snmpd08:31:002
189029975,2cyclictest784-21snmpd07:11:422
189029972,5cyclictest547-21NetworkManager07:31:462
188979977,0cyclictest784-21snmpd12:18:511
188979977,0cyclictest782-21lldpd07:50:151
188979976,0cyclictest191rcu_preempt12:21:371
188979976,0cyclictest191rcu_preempt11:49:091
188979976,0cyclictest191rcu_preempt11:10:191
188979976,0cyclictest191rcu_preempt10:08:551
188979976,0cyclictest191rcu_preempt07:30:141
188979975,2cyclictest762-21ntpd09:00:091
188979975,2cyclictest762-21ntpd08:51:501
188979975,0cyclictest191rcu_preempt10:13:331
188979975,0cyclictest191rcu_preempt08:34:031
188979975,0cyclictest191rcu_preempt08:25:201
188979974,1cyclictest34-21ksoftirqd/108:10:141
188979973,2cyclictest0-21swapper/107:15:201
188979973,0cyclictest191rcu_preempt11:40:151
188979972,3cyclictest25269-21unixbench_multi12:30:241
188979970,5cyclictest22733-21ntpq08:35:181
188909977,0cyclictest784-21snmpd10:35:300
188909977,0cyclictest784-21snmpd09:53:540
188909976,1cyclictest784-21snmpd12:07:130
188909976,1cyclictest784-21snmpd09:28:590
188909975,2cyclictest784-21snmpd12:03:180
188909975,2cyclictest762-21ntpd08:17:020
188909975,2cyclictest23748-21cat09:30:190
189029966,0cyclictest784-21snmpd10:22:512
189029966,0cyclictest784-21snmpd08:12:032
189029966,0cyclictest507-21systemd-resolve08:59:112
189029965,1cyclictest784-21snmpd11:01:442
189029965,1cyclictest784-21snmpd11:01:432
189029965,1cyclictest782-21lldpd08:49:152
189029965,1cyclictest526-21avahi-daemon09:05:362
189029964,2cyclictest784-21snmpd09:24:552
189029963,3cyclictest7557-21df_inode08:00:162
189029963,2cyclictest574-21rs:main2
189029963,2cyclictest27526-21ssh11:57:092
189029963,2cyclictest245-20systemd-journal12:30:392
189029962,3cyclictest6361-21ssh11:34:552
189029962,3cyclictest4851-21ssh12:08:322
189029962,3cyclictest2078-21ssh10:54:012
189029962,3cyclictest2056-21ssh09:41:372
189029961,5cyclictest547-21NetworkManager10:38:322
188979966,0cyclictest784-21snmpd08:41:181
188979966,0cyclictest782-21lldpd07:25:331
188979966,0cyclictest321rcuc/108:15:191
188979965,1cyclictest507-21systemd-resolve09:55:281
188979965,0cyclictest191rcu_preempt09:50:111
188979965,0cyclictest191rcu_preempt08:00:401
188979964,2cyclictest4228-21ssh09:10:121
188979964,2cyclictest18885-21cyclictest11:25:061
188979963,2cyclictest0-21swapper/110:23:261
188979962,3cyclictest26006-21apt-get08:45:101
188979962,1cyclictest191rcu_preempt07:40:161
188979961,3cyclictest34-21ksoftirqd/107:45:121
188979961,3cyclictest34-21ksoftirqd/107:10:171
188979961,3cyclictest0-21swapper/107:55:151
188979960,4cyclictest34-21ksoftirqd/110:45:161
188909966,0cyclictest784-21snmpd08:02:010
188909966,0cyclictest526-21avahi-daemon07:25:100
188909966,0cyclictest4535-21sh10:20:210
188909965,1cyclictest784-21snmpd08:55:550
188909964,2cyclictest547-21NetworkManager09:13:180
188909963,2cyclictest245-20systemd-journal09:36:040
188909963,2cyclictest19418-21ssh11:12:200
189029954,1cyclictest4152-21ntpq07:50:172
189029953,2cyclictest762-21ntpd11:25:282
189029953,2cyclictest762-21ntpd09:15:092
189029953,2cyclictest547-21NetworkManager10:16:222
189029952,3cyclictest547-21NetworkManager08:24:312
189029952,2cyclictest7768-21wc11:35:152
189029952,2cyclictest24110-21ntpq09:30:222
189029951,4cyclictest5471-21cut07:55:102
189029951,3cyclictest18725-21awk09:25:132
188979954,0cyclictest191rcu_preempt08:55:141
188979950,1cyclictest0-21swapper/109:05:171
188909955,0cyclictest784-21snmpd08:35:150
188909952,2cyclictest24137-21tune2fs12:30:160
188909952,2cyclictest16453-21ssh12:20:230
189029940,3cyclictest29789-21diskstats07:35:142
188909941,2cyclictest26312-21grep08:45:120
188909940,3cyclictest17657-21diskstats08:25:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional