You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-12 - 14:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Sat Sep 12, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
179542245170,23sleep20-21swapper/207:07:322
179652236164,24sleep30-21swapper/307:07:413
181372233164,57sleep00-21swapper/007:09:560
181032203166,24sleep10-21swapper/107:09:291
18234999895,2cyclictest0-21swapper/211:28:152
18234997774,2cyclictest0-21swapper/212:34:112
98832610,4sleep07-21ksoftirqd/009:15:160
113342590,4sleep211338-21fschecks_count08:05:142
36992550,4sleep33701-21unixbench_singl07:45:253
18234995553,1cyclictest0-21swapper/207:32:312
180472510,1sleep30-21swapper/309:35:013
242852440,1sleep00-21swapper/007:20:180
18218993810,13cyclictest3163-21munin-run10:15:000
1822999341,23cyclictest0-21swapper/108:40:401
1823499330,14cyclictest0-21swapper/212:05:142
1821899338,12cyclictest12573-21awk11:50:010
1821899338,12cyclictest12573-21awk11:50:010
18234993221,3cyclictest0-21swapper/209:20:372
1822999321,9cyclictest0-21swapper/107:41:491
1821899324,13cyclictest20406-21munin-run07:15:000
1821899323,14cyclictest22329-21munin-run09:45:020
1822999315,18cyclictest2245-21sshd08:56:261
1823499301,15cyclictest0-21swapper/210:53:512
1823499301,14cyclictest0-21swapper/212:20:142
1823499301,11cyclictest18440-21mailstats12:00:262
1823499300,14cyclictest0-21swapper/208:35:182
1822999301,13cyclictest0-21swapper/108:51:511
1822999301,10cyclictest22282-21df_inode08:30:161
1821899309,10cyclictest5983-21awk11:35:010
1823999291,20cyclictest0-21swapper/310:06:483
1823999291,20cyclictest0-21swapper/309:51:113
1823999291,20cyclictest0-21swapper/309:49:203
1823999291,19cyclictest0-21swapper/311:32:573
1822999291,8cyclictest0-21swapper/108:05:081
1822999291,12cyclictest0-21swapper/110:36:451
1822999291,12cyclictest0-21swapper/110:36:451
1822999291,10cyclictest0-21swapper/109:13:151
1823999283,9cyclictest0-21swapper/312:29:053
18239992813,9cyclictest0-21swapper/310:32:003
1823999281,19cyclictest0-21swapper/310:38:303
1823999281,19cyclictest0-21swapper/310:38:303
1823999281,19cyclictest0-21swapper/309:13:173
1823999280,19cyclictest0-21swapper/311:44:313
1823499280,11cyclictest0-21swapper/211:20:152
1823499280,11cyclictest0-21swapper/207:54:092
1822999281,6cyclictest0-21swapper/110:23:261
18229992813,12cyclictest27633-21systemd-journal12:20:581
1822999281,10cyclictest0-21swapper/111:41:021
1823999271,19cyclictest0-21swapper/311:46:423
1823999271,19cyclictest0-21swapper/311:46:423
1823999271,19cyclictest0-21swapper/310:49:023
1823999271,18cyclictest31559-21kworker/u8:111:29:383
1823999270,18cyclictest0-21swapper/311:21:003
18234992712,3cyclictest6888-21diskstats07:55:132
1823499271,12cyclictest0-21swapper/210:56:392
1822999271,9cyclictest0-21swapper/109:59:281
1822999271,16cyclictest0-21swapper/107:49:401
1822999271,12cyclictest0-21swapper/112:30:181
1822999271,12cyclictest0-21swapper/112:06:421
1822999271,12cyclictest0-21swapper/111:22:341
1822999271,11cyclictest0-21swapper/107:25:311
1822999271,11cyclictest0-21swapper/107:10:361
1822999271,10cyclictest0-21swapper/109:09:111
1822999271,10cyclictest0-21swapper/108:49:401
1822999271,10cyclictest0-21swapper/107:56:561
1822999270,12cyclictest0-21swapper/111:57:161
1822999270,11cyclictest0-21swapper/112:27:021
37312260,4sleep121-21rcuc/109:00:191
1823999261,7cyclictest23369-21cat07:20:023
18239992614,3cyclictest0-21swapper/309:27:143
18239992614,3cyclictest0-21swapper/309:27:143
1823999261,19cyclictest0-21swapper/311:16:323
1823999261,18cyclictest0-21swapper/309:59:313
1823999261,17cyclictest0-21swapper/312:32:383
1823999261,17cyclictest0-21swapper/311:03:083
1823999261,17cyclictest0-21swapper/310:54:103
1823999261,17cyclictest0-21swapper/310:10:033
1823999261,16cyclictest0-21swapper/311:55:443
1823999260,18cyclictest0-21swapper/311:08:063
1823999260,17cyclictest0-21swapper/310:04:243
1823499269,2cyclictest0-21swapper/209:31:332
18234992611,3cyclictest27272-21sed07:25:252
1823499261,13cyclictest0-21swapper/210:44:382
1822999261,9cyclictest0-21swapper/111:01:571
1822999261,9cyclictest0-21swapper/109:47:321
1822999261,8cyclictest0-21swapper/111:06:131
1822999261,12cyclictest0-21swapper/107:23:191
1822999261,11cyclictest0-21swapper/109:50:581
1822999261,11cyclictest0-21swapper/109:28:541
1822999261,11cyclictest0-21swapper/109:28:531
1822999261,11cyclictest0-21swapper/108:13:121
1822999260,9cyclictest0-21swapper/109:32:161
1823999253,16cyclictest87350irq/126-enp1s0-12:15:153
18239992514,3cyclictest0-21swapper/310:55:353
18239992514,3cyclictest0-21swapper/310:15:353
1823999251,18cyclictest0-21swapper/310:28:443
1823999251,16cyclictest0-21swapper/310:24:373
1823999251,15cyclictest0-21swapper/309:15:093
1823499257,3cyclictest0-21swapper/207:37:572
1822999258,4cyclictest28635-21ntp_states11:10:221
1822999251,9cyclictest0-21swapper/108:36:261
1822999251,9cyclictest0-21swapper/108:25:321
1822999251,9cyclictest0-21swapper/108:24:241
1822999251,11cyclictest0-21swapper/112:15:201
1822999251,10cyclictest0-21swapper/111:53:441
1822999251,10cyclictest0-21swapper/109:19:091
1821899252,11cyclictest21596-21/usr/sbin/munin12:10:140
1821899250,12cyclictest8510-21sadc08:00:010
229142240,2sleep00-21swapper/009:45:150
18239992415,4cyclictest0-21swapper/309:35:213
1823999241,4cyclictest0-21swapper/309:40:103
1823999241,17cyclictest0-21swapper/312:11:313
1823999241,16cyclictest87350irq/126-enp1s0-08:35:253
1823999241,16cyclictest0-21swapper/310:40:343
1823999241,15cyclictest0-21swapper/311:10:283
1823999240,17cyclictest0-21swapper/312:03:543
1823999240,16cyclictest0-21swapper/309:20:233
1823499247,2cyclictest0-21swapper/209:10:022
1823499241,9cyclictest0-21swapper/207:11:162
1823499241,17cyclictest0-21swapper/208:15:042
1823499241,17cyclictest0-21swapper/208:15:042
1823499241,11cyclictest0-21swapper/211:48:482
1823499241,11cyclictest0-21swapper/211:48:472
1822999241,9cyclictest0-21swapper/110:17:181
1822999241,9cyclictest0-21swapper/110:08:561
1822999241,9cyclictest0-21swapper/109:22:171
1822999241,9cyclictest0-21swapper/108:18:301
1822999241,9cyclictest0-21swapper/108:18:301
1822999241,8cyclictest0-21swapper/109:40:241
1822999241,8cyclictest0-21swapper/107:53:241
1822999241,15cyclictest0-21swapper/107:15:211
1822999241,14cyclictest0-21swapper/111:28:581
1822999241,10cyclictest0-21swapper/108:02:501
1821899246,15cyclictest29902-21perf07:35:020
1821899242,11cyclictest7780-21/usr/sbin/munin10:25:250
1821899242,11cyclictest25716-21/usr/sbin/munin07:25:190
1823999231,16cyclictest87350irq/126-enp1s0-07:43:313
1823999230,17cyclictest0-21swapper/312:35:253
1823999230,14cyclictest0-21swapper/312:20:143
1823499236,9cyclictest0-21swapper/208:30:142
1822999236,7cyclictest0-21swapper/112:04:441
1822999231,9cyclictest0-21swapper/111:48:181
1822999231,9cyclictest0-21swapper/111:48:181
1822999231,18cyclictest0-21swapper/111:33:151
1822999230,10cyclictest0-21swapper/110:49:591
1821899236,15cyclictest11962-21perf10:35:000
1821899231,11cyclictest31354-21/usr/sbin/munin10:05:160
1821899231,11cyclictest18155-21/usr/sbin/munin09:35:160
1821899231,11cyclictest14371-21/usr/sbin/munin10:40:200
18218992310,2cyclictest3441-21sshd11:25:530
1823999221,8cyclictest579-21systemd-journal08:25:003
1823999221,16cyclictest0-21swapper/311:51:363
1823999221,12cyclictest0-21swapper/312:05:173
1823499229,3cyclictest0-21swapper/211:59:182
1823499222,2cyclictest0-21swapper/209:40:132
18234992212,4cyclictest87550irq/128-enp1s0-08:50:182
1823499221,14cyclictest24136-21users09:45:252
18234992211,3cyclictest9765-21awk10:30:012
1823499221,13cyclictest0-21swapper/208:41:582
1822999224,5cyclictest0-21swapper/109:39:591
1822999221,9cyclictest0-21swapper/110:11:061
1822999221,9cyclictest0-21swapper/107:39:231
1822999221,5cyclictest0-21swapper/110:43:141
18229992214,3cyclictest0-21swapper/110:56:431
1822999221,10cyclictest0-21swapper/107:33:271
1822999220,10cyclictest0-21swapper/112:14:161
1821899228,3cyclictest4965-21/usr/sbin/munin09:05:180
1821899226,4cyclictest21536-21fschecks_time10:55:160
1821899221,19cyclictest0-21swapper/008:30:270
1821899221,10cyclictest28706-21sshd09:57:280
318692211,2sleep30-21swapper/307:37:283
1823999218,2cyclictest6948-21awk09:10:013
1823999211,11cyclictest7113-21memory11:35:183
1823499219,3cyclictest0-21swapper/211:34:032
1823499219,3cyclictest0-21swapper/209:56:382
1823499211,4cyclictest8077-21memory09:10:192
1823499211,14cyclictest0-21swapper/212:15:282
1823499211,13cyclictest0-21swapper/211:05:272
1823499211,11cyclictest0-21swapper/211:11:172
1823499210,12cyclictest0-21swapper/211:16:112
1822999216,3cyclictest29166-21egrep10:00:141
1822999213,3cyclictest0-21swapper/112:39:321
1822999211,15cyclictest0-21swapper/110:50:211
1822999211,10cyclictest0-21swapper/111:16:141
1822999211,10cyclictest0-21swapper/110:27:221
1821899216,4cyclictest6933-21diskmemload12:35:480
18218992114,4cyclictest29697-21grep12:25:250
1821899211,11cyclictest0-21swapper/008:00:240
1821899211,10cyclictest0-21swapper/007:40:150
1821899210,19cyclictest0-21swapper/012:00:140
1823999203,2cyclictest88950irq/132-enp2s0-08:57:263
1823999202,9cyclictest27288-21awk07:25:263
1823999201,7cyclictest30066-21unixbench_singl08:45:273
1823999201,6cyclictest29886-21idleruntime-cro07:35:023
1823999201,14cyclictest87350irq/126-enp1s0-08:16:533
1823999201,14cyclictest87350irq/126-enp1s0-08:16:533
1823999201,10cyclictest2781-21egrep09:00:133
1823499201,6cyclictest0-21swapper/210:13:092
18234992011,4cyclictest16904-21df10:45:142
1823499201,12cyclictest0-21swapper/208:45:202
1823499201,12cyclictest0-21swapper/207:19:442
1823499201,11cyclictest0-21swapper/207:20:412
1823499200,13cyclictest0-21swapper/211:00:292
1823499200,13cyclictest0-21swapper/207:41:192
1822999205,4cyclictest755-21rngd11:36:061
1822999202,4cyclictest10893-21memory10:30:211
1821899207,3cyclictest87550irq/128-enp1s0-11:18:300
1821899206,3cyclictest0-21swapper/008:07:460
1821899205,3cyclictest87550irq/128-enp1s0-07:45:210
1821899203,15cyclictest87350irq/126-enp1s0-07:15:030
1821899201,8cyclictest0-21swapper/009:34:460
18218992012,5cyclictest30261-21cpuspeed_turbos07:35:140
1821899200,19cyclictest0-21swapper/010:45:160
1821899200,10cyclictest0-21swapper/011:08:180
1823999195,3cyclictest10944-21/usr/sbin/munin08:05:183
1823999193,8cyclictest87350irq/126-enp1s0-07:50:213
1823999193,4cyclictest9598-21sort08:00:183
1823999193,4cyclictest32702-21perf08:55:023
1823999192,9cyclictest14023-21grep08:10:183
1823999191,4cyclictest20400-21sh07:15:003
1823999191,10cyclictest808-21snmpd08:42:533
1823499195,3cyclictest0-21swapper/209:54:142
1823499191,7cyclictest0-21swapper/210:05:592
1823499191,3cyclictest0-21swapper/212:35:122
18234991913,3cyclictest0-21swapper/212:25:222
18234991912,4cyclictest30177-21ntpq10:00:222
1823499191,11cyclictest0-21swapper/208:27:422
1821899195,3cyclictest87550irq/128-enp1s0-07:50:250
1821899193,5cyclictest89250irq/135-enp2s0-11:54:140
1821899192,5cyclictest827-21runrttasks12:31:020
1821899191,9cyclictest0-21swapper/009:54:260
1821899191,7cyclictest13317-21sshd09:22:270
1821899191,7cyclictest0-21swapper/012:06:400
1821899191,6cyclictest0-21swapper/010:24:060
1821899191,16cyclictest26117-21awk08:40:010
1821899191,15cyclictest23790-21/usr/sbin/munin12:15:190
1821899191,11cyclictest0-21swapper/008:50:190
1823999181,5cyclictest22784-21latency08:30:193
1823999181,5cyclictest20552-21irqstats08:25:183
1823999181,14cyclictest7455-21memory07:55:173
1823499181,7cyclictest14148-21munin-run10:40:012
1823499181,7cyclictest14148-21munin-run10:40:012
1823499181,7cyclictest0-21swapper/211:37:392
1823499181,6cyclictest0-21swapper/209:03:502
1823499181,3cyclictest17532-21/usr/sbin/munin08:20:182
18234991813,2cyclictest0-21swapper/207:45:212
1823499181,2cyclictest0-21swapper/209:26:502
1823499181,2cyclictest0-21swapper/209:26:502
1823499181,10cyclictest0-21swapper/211:43:472
1823499180,12cyclictest0-21swapper/208:55:302
1821899188,7cyclictest1142-21gltestperf08:55:160
1821899188,1cyclictest0-21swapper/008:40:110
1821899187,3cyclictest10079-21sshd11:41:280
1821899186,3cyclictest87550irq/128-enp1s0-09:10:210
1821899185,3cyclictest87550irq/128-enp1s0-11:55:120
1821899182,4cyclictest12609-21cut10:35:160
1821899182,4cyclictest12609-21cut10:35:160
1821899181,5cyclictest8564-1kworker/u9:110:16:090
1821899181,5cyclictest32305-21df_inode11:20:140
1821899181,4cyclictest28528-21/usr/sbin/munin08:45:200
1821899181,15cyclictest0-21swapper/008:10:380
18218991811,4cyclictest15337-21acpi08:15:130
18218991811,4cyclictest15337-21acpi08:15:120
1821899181,11cyclictest0-21swapper/010:01:000
279092170,4sleep011-21rcuc/011:10:140
1823999171,3cyclictest23519-21/usr/sbin/munin07:20:173
1823499177,3cyclictest0-21swapper/210:34:342
1823499173,4cyclictest21944-21diskstats12:10:162
1823499171,6cyclictest8885-21grep08:00:122
1823499171,6cyclictest5456-21latency_hist10:20:012
1823499171,3cyclictest0-21swapper/211:53:162
18234991712,3cyclictest5582-21/usr/sbin/munin10:20:222
18234991712,3cyclictest18155-21/usr/sbin/munin09:35:252
1823499171,12cyclictest0-21swapper/209:15:162
18234991710,4cyclictest14217-21sh08:10:222
1821899174,3cyclictest87550irq/128-enp1s0-08:20:160
1821899171,6cyclictest0-21swapper/010:52:190
1821899171,3cyclictest23167-21/usr/sbin/munin11:00:270
18218991712,3cyclictest20803-21ntp_states08:25:220
1821899171,10cyclictest0-21swapper/011:35:140
1821899161,7cyclictest0-21swapper/009:04:040
18218991610,3cyclictest14308-21grep09:25:150
18218991610,3cyclictest14308-21grep09:25:150
18218991510,3cyclictest26267-21ls12:20:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional