You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-30 - 02:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3s.osadl.org (updated Thu Jul 30, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
319782242164,25sleep20-21swapper/219:07:572
318322239164,24sleep10-21swapper/119:06:031
318692232163,56sleep30-21swapper/319:06:343
320902202164,25sleep00-21swapper/019:09:250
1385521720,4sleep33223699cyclictest22:05:173
16802430,5sleep31668-21gltestperf00:05:203
3221699393,12cyclictest28150-21awk22:40:010
264772390,2sleep30-21swapper/322:35:143
3221699371,11cyclictest13942-21/usr/sbin/munin19:40:150
3221699369,24cyclictest25326-21munin-run21:20:010
3221699351,5cyclictest0-21swapper/021:40:230
32216993512,4cyclictest24569-21grep22:30:160
71022340,6sleep23222999cyclictest20:35:262
3221699347,13cyclictest29117-21idleruntime-cro20:15:010
32216993413,3cyclictest11178-21munin-run23:15:010
101052340,4sleep03221699cyclictest19:30:150
3222999331,10cyclictest0-21swapper/223:17:582
3222499331,24cyclictest0-21swapper/123:42:561
3222499331,10cyclictest0-21swapper/120:48:541
3222499330,14cyclictest0-21swapper/123:56:551
3222499321,10cyclictest0-21swapper/119:38:401
3222499320,24cyclictest0-21swapper/100:32:391
3222499320,23cyclictest0-21swapper/122:42:561
3221699328,3cyclictest21161-21/usr/sbin/munin21:10:190
3221699328,3cyclictest21161-21/usr/sbin/munin21:10:190
3221699322,8cyclictest0-21swapper/023:35:130
3221699322,8cyclictest0-21swapper/023:35:130
3221699321,11cyclictest21772-21/usr/sbin/munin22:25:120
204202320,4sleep2311ktimersoftd/223:35:132
204202320,4sleep2311ktimersoftd/223:35:132
3222999310,13cyclictest0-21swapper/220:01:342
3222499311,9cyclictest0-21swapper/119:26:281
3222499311,10cyclictest0-21swapper/120:01:101
3222499310,12cyclictest0-21swapper/122:51:241
3222499310,12cyclictest0-21swapper/121:00:461
3221699316,3cyclictest18245-1kworker/u9:200:07:070
3221699313,13cyclictest4743-21grep00:10:260
32216993112,2cyclictest0-21swapper/022:55:270
3222499301,21cyclictest0-21swapper/121:47:031
3222499301,11cyclictest0-21swapper/123:35:521
3222499301,11cyclictest0-21swapper/123:35:521
3222499301,11cyclictest0-21swapper/120:26:301
3222499301,11cyclictest0-21swapper/119:44:361
3222499301,11cyclictest0-21swapper/119:32:241
3222499301,11cyclictest0-21swapper/119:20:091
3222499301,11cyclictest0-21swapper/119:18:251
3222499301,10cyclictest0-21swapper/120:38:301
3222499301,10cyclictest0-21swapper/119:58:251
3222499300,23cyclictest0-21swapper/100:14:231
3222499300,11cyclictest0-21swapper/122:30:521
3221699302,11cyclictest9839-21/usr/sbin/munin00:25:170
3221699301,14cyclictest12136-21munin-run20:50:000
3221699301,10cyclictest10775-21/usr/sbin/munin22:00:240
32216993010,3cyclictest87550irq/128-enp1s0-23:02:460
3222499291,9cyclictest0-21swapper/120:31:101
3222499291,12cyclictest19592-21sendmail_mailtr19:50:251
3222499291,12cyclictest0-21swapper/123:13:041
3222499291,11cyclictest0-21swapper/122:18:401
3222499291,11cyclictest0-21swapper/120:08:221
3222499291,10cyclictest0-21swapper/120:21:341
3222499290,9cyclictest0-21swapper/119:11:091
3222499290,11cyclictest0-21swapper/121:06:051
3221699291,13cyclictest21564-21perf22:25:010
3222499281,9cyclictest0-21swapper/121:27:121
3222499281,11cyclictest0-21swapper/119:46:051
3222499281,10cyclictest0-21swapper/122:45:561
3222499280,9cyclictest0-21swapper/120:42:331
3222499280,13cyclictest0-21swapper/123:07:001
3222499280,12cyclictest0-21swapper/123:26:001
3222499280,12cyclictest0-21swapper/123:15:041
3222499280,12cyclictest0-21swapper/121:23:441
3222499280,11cyclictest0-21swapper/122:04:201
3222499280,11cyclictest0-21swapper/121:38:471
3222499280,11cyclictest0-21swapper/120:16:221
3222499280,10cyclictest0-21swapper/120:50:411
3221699281,10cyclictest31640-21kworker/0:121:35:530
3222499271,9cyclictest0-21swapper/121:50:441
3222499271,9cyclictest0-21swapper/100:05:431
3222499271,17cyclictest0-21swapper/100:25:431
3222499271,12cyclictest0-21swapper/122:06:401
3222499271,11cyclictest0-21swapper/123:01:021
3222499271,10cyclictest0-21swapper/120:56:301
3222499271,10cyclictest0-21swapper/120:11:421
3222499270,9cyclictest0-21swapper/121:59:001
3222499270,10cyclictest0-21swapper/123:54:431
3221699271,17cyclictest15514-21kworker/0:123:33:170
3221699271,12cyclictest9320-21munin-run19:30:020
3223699260,10cyclictest0-21swapper/321:48:043
3222999261,6cyclictest0-21swapper/221:56:072
3222499261,9cyclictest0-21swapper/122:36:241
3222499261,9cyclictest0-21swapper/100:17:231
3222499261,12cyclictest0-21swapper/122:23:561
3222499261,10cyclictest0-21swapper/122:57:241
3222499260,12cyclictest0-21swapper/121:42:431
3222499260,10cyclictest0-21swapper/122:25:441
3222499260,10cyclictest0-21swapper/100:22:531
3221699268,15cyclictest4930-21munin-run19:20:020
3221699261,5cyclictest0-21swapper/021:50:100
3223699251,13cyclictest0-21swapper/320:51:113
3223699251,11cyclictest0-21swapper/319:21:333
3222499257,9cyclictest0-21swapper/123:46:231
3222499251,11cyclictest0-21swapper/123:20:481
3222499250,9cyclictest0-21swapper/121:18:371
3222499250,9cyclictest0-21swapper/100:35:561
3222499250,11cyclictest0-21swapper/123:34:521
3221699251,10cyclictest23085-21kworker/0:121:22:520
3221699250,12cyclictest3350-21awk20:30:020
3223699242,18cyclictest0-21swapper/320:35:243
32236992410,2cyclictest0-21swapper/320:01:353
3222499241,9cyclictest0-21swapper/100:00:551
3222499240,11cyclictest0-21swapper/121:10:481
3222499240,11cyclictest0-21swapper/121:10:481
3222499240,10cyclictest0-21swapper/122:11:441
3222499240,10cyclictest0-21swapper/121:31:281
3221699248,3cyclictest13931-21kworker/0:219:45:240
3223699234,3cyclictest10100-21unixbench_multi21:55:253
3223699233,16cyclictest21898-40dnf23:36:513
3223699233,16cyclictest21898-40dnf23:36:513
3223699231,15cyclictest0-21swapper/319:10:473
3222999237,2cyclictest0-21swapper/220:17:252
3222999231,7cyclictest25676-21sshd22:31:472
3221699231,6cyclictest0-21swapper/000:36:280
32216992311,2cyclictest29421-21kworker/0:221:32:160
3223699225,2cyclictest0-21swapper/319:35:173
3223699221,6cyclictest0-21swapper/323:04:323
3223699221,6cyclictest0-21swapper/322:48:243
3223699221,6cyclictest0-21swapper/322:40:203
3223699221,11cyclictest0-21swapper/321:35:203
32236992210,3cyclictest0-21swapper/322:22:513
3222999221,8cyclictest0-21swapper/200:35:232
3222999220,10cyclictest0-21swapper/219:27:092
3221699227,4cyclictest20184-21kworker/0:023:41:330
3221699222,10cyclictest9859-21kworker/0:121:01:310
3221699221,12cyclictest3899-21kworker/0:121:49:500
3223699219,6cyclictest0-21swapper/322:16:143
3223699219,3cyclictest0-21swapper/323:48:483
3223699217,7cyclictest21496-21df21:10:153
3223699217,7cyclictest21496-21df21:10:153
3223699217,2cyclictest0-21swapper/319:45:213
3223699216,12cyclictest23384-21netstat23:40:173
3223699215,3cyclictest14586-21strings23:20:203
3223699213,3cyclictest0-21swapper/322:52:303
32236992114,2cyclictest0-21swapper/300:35:173
3223699211,17cyclictest16140-21diskstats23:25:133
3223699211,13cyclictest24416-21ntp_states21:15:203
3223699211,12cyclictest31082-21chrt20:17:253
3223699210,11cyclictest0-21swapper/319:25:113
3222999213,15cyclictest25713-21tr21:20:132
3222999211,7cyclictest12036-21/usr/sbin/munin00:30:192
32229992112,6cyclictest26583-21kworker/u8:020:50:042
3221699217,3cyclictest7049-21kworker/0:000:18:140
3221699216,3cyclictest25391-21kworker/0:019:22:460
3221699215,5cyclictest18432-21kworker/0:221:05:470
3221699211,4cyclictest0-21swapper/022:16:450
3221699211,12cyclictest32422-21/usr/sbin/munin19:10:250
3223699206,3cyclictest87350irq/126-enp1s0-22:10:203
32236992011,3cyclictest25831-21sshd22:33:473
3223699201,13cyclictest0-21swapper/323:09:223
3223699201,10cyclictest0-21swapper/319:55:493
32236992010,3cyclictest13744-21sshd00:31:493
32236992010,2cyclictest0-21swapper/300:23:303
3223699200,6cyclictest0-21swapper/300:26:013
3222999209,8cyclictest5293-21pmu-power21:45:222
3222999202,4cyclictest31177-21/usr/sbin/munin00:00:282
3222999201,6cyclictest942-21cut22:50:162
3222999201,6cyclictest20026-21ntp_states21:05:232
3222999201,5cyclictest29335-21users21:25:262
3222999201,5cyclictest19574-21/usr/sbin/munin22:20:192
3222999201,5cyclictest0-21swapper/223:11:582
3222999201,5cyclictest0-21swapper/221:41:462
3222999201,4cyclictest0-21swapper/222:25:122
32229992014,3cyclictest27450-21gltestperf23:50:162
3221699209,3cyclictest20532-21/usr/sbin/munin19:55:260
3221699208,2cyclictest3899-21kworker/0:122:10:300
3221699208,11cyclictest0-21swapper/020:45:010
3221699207,3cyclictest8660-21kworker/0:023:26:330
3221699206,3cyclictest13173-21kworker/0:020:56:060
3221699205,4cyclictest9859-21kworker/0:120:51:260
3221699205,4cyclictest26418-21kworker/0:123:54:170
3221699201,9cyclictest3899-21kworker/0:122:06:090
3221699201,6cyclictest0-21swapper/021:58:010
3221699201,6cyclictest0-21swapper/021:26:370
3221699201,4cyclictest7739-21munin-run20:40:020
32216992010,2cyclictest13940-21df_inode23:20:130
3223699199,3cyclictest0-21swapper/321:21:193
3223699199,2cyclictest0-21swapper/300:18:443
3223699199,2cyclictest0-21swapper/300:12:373
3223699194,6cyclictest0-21swapper/321:40:233
3223699191,9cyclictest790-21in:imjournal23:10:203
3223699191,6cyclictest26782-21/usr/sbin/munin23:50:123
3223699191,6cyclictest0-21swapper/321:34:303
3223699191,17cyclictest0-21swapper/322:02:273
3223699191,15cyclictest15951-21sed20:55:243
32236991911,2cyclictest11987-21grep23:15:133
3223699191,10cyclictest694-21sshd00:04:483
3222999195,3cyclictest87650irq/129-enp1s0-00:25:222
3222999195,3cyclictest5630-21cstates00:15:142
3222999193,4cyclictest11746-21/usr/sbin/munin19:35:122
3222999191,4cyclictest0-21swapper/222:41:322
3222999191,2cyclictest0-21swapper/200:10:252
3221699199,6cyclictest0-21swapper/022:54:570
3221699197,2cyclictest24159-21kworker/0:223:46:290
3221699195,2cyclictest9299-21kworker/0:200:24:540
3221699194,2cyclictest87350irq/126-enp1s0-22:45:140
3221699192,3cyclictest0-21swapper/020:16:480
32216991913,3cyclictest24470-21kworker/0:120:08:120
3221699191,2cyclictest87650irq/129-enp1s0-20:25:020
3221699191,10cyclictest28983-21/usr/sbin/munin23:55:160
3221699190,9cyclictest0-21swapper/023:05:130
95552181,2sleep30-21swapper/320:40:293
43462181,2sleep30-21swapper/322:57:183
3223699189,3cyclictest0-21swapper/323:56:053
3223699185,6cyclictest41-21ksoftirqd/320:28:413
32236991814,3cyclictest0-21swapper/321:00:123
32236991813,3cyclictest22582-21irqstats22:25:173
3223699181,15cyclictest0-21swapper/320:12:213
32236991811,4cyclictest26220-21ls20:05:243
32236991811,2cyclictest0-21swapper/321:50:233
3223699181,11cyclictest0-21swapper/320:45:283
3223699181,10cyclictest2960-21/usr/sbin/munin19:15:163
3223699181,10cyclictest27748-21/usr/sbin/munin21:25:163
3223699180,6cyclictest0-21swapper/323:33:453
3222999185,2cyclictest87550irq/128-enp1s0-20:05:262
3222999182,4cyclictest5198-21sshd20:30:482
3222999182,3cyclictest0-21swapper/219:52:212
3222999182,3cyclictest0-21swapper/219:45:072
3222999182,15cyclictest0-21swapper/200:08:232
3222999181,7cyclictest7371-21needreboot21:50:162
3222999181,5cyclictest0-21swapper/223:41:102
3222999181,5cyclictest0-21swapper/223:05:102
3222999181,5cyclictest0-21swapper/222:03:452
3222999181,4cyclictest32140-21/usr/sbin/munin21:35:212
3222999181,4cyclictest25169-21NetworkManager21:18:112
3222999181,4cyclictest0-21swapper/223:00:082
3222999181,4cyclictest0-21swapper/222:55:362
32229991812,4cyclictest31649-21tr20:20:112
32229991811,4cyclictest909-21sssd_nss20:10:142
3221699187,3cyclictest11397-21/usr/sbin/munin23:15:220
3221699185,4cyclictest29077-21kworker/0:020:32:470
3221699185,4cyclictest19193-21kworker/0:022:41:020
3221699181,8cyclictest0-21swapper/000:01:100
3221699181,5cyclictest18245-1kworker/u9:200:31:240
152012181,3sleep20-21swapper/223:20:292
109242180,1sleep20-21swapper/220:45:192
3223699171,13cyclictest19329-21grep19:50:213
3223699171,10cyclictest0-21swapper/321:05:173
3222999171,9cyclictest0-21swapper/223:46:002
3222999171,9cyclictest0-21swapper/221:10:352
3222999171,9cyclictest0-21swapper/221:10:352
3222999171,7cyclictest0-21swapper/220:25:172
3222999171,5cyclictest16707-21taskset23:25:182
3222999171,5cyclictest0-21swapper/222:18:422
3222999171,5cyclictest0-21swapper/219:23:122
3222999171,4cyclictest0-21swapper/221:32:192
3222999171,2cyclictest0-21swapper/200:20:532
32229991712,3cyclictest3855-21cat19:15:202
32229991712,3cyclictest1-21systemd19:10:212
3222999171,15cyclictest0-21swapper/220:55:142
3222999171,14cyclictest0-21swapper/219:58:252
32229991710,4cyclictest31538-21needreboot22:45:202
3221699173,3cyclictest19999-21kworker/0:020:03:400
3221699171,10cyclictest0-21swapper/019:39:550
3223699162,11cyclictest4560-21cut20:30:193
3223699161,3cyclictest32517-21grep20:20:193
32236991610,3cyclictest13942-21/usr/sbin/munin19:40:223
3222999169,4cyclictest15310-21/usr/sbin/munin19:40:232
3222999161,8cyclictest0-21swapper/219:30:222
3222999161,5cyclictest17898-21head21:00:222
3222999161,5cyclictest0-21swapper/222:38:042
32229991612,3cyclictest0-21swapper/223:34:392
32229991612,3cyclictest0-21swapper/222:05:012
32229991611,3cyclictest28984-21acpi23:55:152
32229991611,2cyclictest15787-21gltestperf22:10:142
3221699161,4cyclictest19999-21kworker/0:019:51:270
3223699159,3cyclictest10218-21smartctl19:30:163
32229991510,3cyclictest8253-21cat20:40:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional