You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-07 - 15:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot3s.osadl.org (updated Tue Jul 07, 2026 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
113332239163,24sleep20-21swapper/207:06:342
113322203166,25sleep10-21swapper/107:06:331
112622203166,24sleep30-21swapper/307:05:383
114442202163,26sleep00-21swapper/007:08:010
282502580,4sleep10-21swapper/107:45:141
11696995754,2cyclictest0-21swapper/208:43:002
68512540,2sleep30-21swapper/310:35:243
1168999401,17cyclictest20636-21unixbench-2d07:25:251
1170099391,13cyclictest22412-21ntp_kernel_pll_07:30:233
11696993936,2cyclictest0-21swapper/208:27:262
11696993936,2cyclictest0-21swapper/208:27:262
291712360,2sleep339-21rcuc/309:00:163
1168999351,10cyclictest0-21swapper/108:22:211
11684993510,12cyclictest21144-21munin-run07:30:020
1168999341,24cyclictest0-21swapper/108:19:541
14042330,4sleep11408-21timerwakeupswit07:55:251
1168999331,14cyclictest0-21swapper/108:39:171
1168999331,11cyclictest0-21swapper/110:40:091
11684993212,17cyclictest28978-21munin-run10:15:010
1170099311,17cyclictest0-21swapper/309:30:153
1168999311,14cyclictest0-21swapper/111:38:161
1168999311,14cyclictest0-21swapper/111:03:391
1168999311,14cyclictest0-21swapper/109:52:101
1168999311,12cyclictest7661-21timerwakeupswit11:50:261
1168999311,12cyclictest0-21swapper/112:14:561
1168999311,11cyclictest0-21swapper/111:11:551
301642300,3sleep230-21rcuc/209:00:272
1168999306,4cyclictest0-21swapper/108:56:131
1168999301,15cyclictest0-21swapper/110:22:171
1168999301,14cyclictest0-21swapper/111:41:121
1168999301,14cyclictest0-21swapper/111:16:241
1168999301,14cyclictest0-21swapper/109:44:541
1168999300,12cyclictest0-21swapper/108:25:451
1168999300,12cyclictest0-21swapper/108:25:451
1168499301,15cyclictest9810-21munin-run10:45:010
1170099290,13cyclictest0-21swapper/309:47:133
1168999296,3cyclictest0-21swapper/108:43:491
1168999295,3cyclictest0-21swapper/107:14:101
1168999291,14cyclictest0-21swapper/109:25:501
1168999291,14cyclictest0-21swapper/109:11:141
1168499296,11cyclictest17038-21perf12:15:010
1168999281,15cyclictest0-21swapper/110:01:061
1168999281,15cyclictest0-21swapper/109:45:101
1168999281,15cyclictest0-21swapper/107:40:261
1168999281,15cyclictest0-21swapper/107:21:291
1168999281,14cyclictest0-21swapper/107:17:291
1168999281,12cyclictest24735-21ps07:35:241
1168999281,12cyclictest10678-21smartctl09:30:261
1168999281,12cyclictest0-21swapper/112:01:161
1168999281,11cyclictest0-21swapper/110:51:151
1168499281,14cyclictest18964-21idleruntime-cro07:25:010
11700992714,7cyclictest8928-21open_inodes10:40:223
11700992712,2cyclictest0-21swapper/310:00:203
1170099270,12cyclictest0-21swapper/310:14:143
1168999279,3cyclictest0-21swapper/109:55:201
1168999271,2cyclictest87350irq/126-enp1s0-11:25:261
1168999271,18cyclictest0-21swapper/109:16:141
1168999271,15cyclictest0-21swapper/112:37:201
1168999271,15cyclictest0-21swapper/112:32:081
1168999271,14cyclictest0-21swapper/111:05:231
1168999271,14cyclictest0-21swapper/108:46:011
1168999271,13cyclictest0-21swapper/112:15:041
1168999271,13cyclictest0-21swapper/109:08:581
1168999271,10cyclictest0-21swapper/108:31:451
1168499271,13cyclictest993-21munin-run10:25:000
1168499271,13cyclictest25818-21munin-run12:35:010
1170099261,8cyclictest20800-21turbostat.cron11:10:023
1170099261,13cyclictest0-21swapper/308:22:223
1168999266,3cyclictest0-21swapper/110:45:031
1168999266,3cyclictest0-21swapper/110:45:031
1168999261,9cyclictest2053-21irqstats10:25:171
1170099257,3cyclictest0-21swapper/311:53:053
1170099252,10cyclictest18207-21/usr/sbin/munin09:50:123
11700992512,2cyclictest9199-21awk09:30:003
1170099251,13cyclictest0-21swapper/308:41:063
1170099251,13cyclictest0-21swapper/308:38:223
1170099251,13cyclictest0-21swapper/308:17:383
11700992511,3cyclictest0-21swapper/307:15:243
1168999252,19cyclictest6367-21munin-run08:10:011
1168999251,15cyclictest0-21swapper/112:07:281
1168999251,12cyclictest0-21swapper/108:10:301
1168999251,11cyclictest0-21swapper/110:18:171
1170099241,13cyclictest0-21swapper/311:30:593
1170099241,12cyclictest0-21swapper/310:51:523
1170099241,12cyclictest0-21swapper/307:12:183
1170099241,11cyclictest8448-21/usr/sbin/munin11:55:143
1168999246,3cyclictest0-21swapper/111:31:001
1168999241,11cyclictest0-21swapper/110:12:571
1168999241,10cyclictest0-21swapper/109:36:221
1168999241,10cyclictest0-21swapper/109:36:221
1168999241,10cyclictest0-21swapper/109:24:141
1168999241,10cyclictest0-21swapper/108:53:571
1170099238,3cyclictest24827-21cpu10:05:133
1170099237,2cyclictest0-21swapper/307:50:113
11700992316,4cyclictest28221-21tail11:25:153
1170099231,3cyclictest26517-21kworker/u8:110:45:133
1170099231,3cyclictest26517-21kworker/u8:110:45:133
1170099231,12cyclictest0-21swapper/309:16:373
1170099231,11cyclictest808-21snmpd12:02:553
1170099231,11cyclictest15393-21/usr/sbin/munin08:30:213
1170099231,11cyclictest0-21swapper/308:25:063
1170099231,11cyclictest0-21swapper/308:25:063
1170099231,11cyclictest0-21swapper/308:10:383
1170099230,13cyclictest0-21swapper/308:57:303
1170099230,12cyclictest0-21swapper/307:23:253
1169699230,6cyclictest6265-21cpu11:50:132
1168999239,2cyclictest0-21swapper/107:30:161
1168999231,12cyclictest0-21swapper/111:46:401
1168499237,14cyclictest0-21swapper/009:05:020
1168499231,16cyclictest0-21swapper/012:04:090
1168499231,16cyclictest0-21swapper/010:56:490
1168499230,11cyclictest15177-21perf08:30:000
1168499230,11cyclictest15177-21perf08:30:000
1170099228,3cyclictest2192-21/usr/sbin/munin08:00:183
1170099228,3cyclictest13811-21/usr/sbin/munin09:40:123
1170099221,13cyclictest0-21swapper/311:39:593
1170099221,12cyclictest0-21swapper/312:15:523
1170099221,12cyclictest0-21swapper/312:07:313
1170099221,12cyclictest0-21swapper/311:48:193
1170099221,12cyclictest0-21swapper/311:14:403
1170099221,11cyclictest0-21swapper/311:40:313
1170099221,11cyclictest0-21swapper/310:20:103
1170099221,11cyclictest0-21swapper/309:55:203
1170099221,10cyclictest0-21swapper/311:00:383
1170099220,1cyclictest0-21swapper/312:30:273
1169699221,6cyclictest0-21swapper/212:13:332
1168999222,11cyclictest26786-21munin-run10:10:011
1168999221,10cyclictest0-21swapper/110:37:191
11684992210,3cyclictest231782grep12:25:250
1170099217,3cyclictest27961-21/usr/sbin/munin07:45:183
1170099216,3cyclictest0-21swapper/312:20:223
1170099214,15cyclictest87350irq/126-enp1s0-09:10:153
1170099211,11cyclictest0-21swapper/312:37:243
1170099210,19cyclictest0-21swapper/311:20:163
1170099210,19cyclictest0-21swapper/310:30:243
1169699211,5cyclictest0-21swapper/212:31:502
11696992111,2cyclictest28361-21awk09:00:012
1168999218,3cyclictest0-21swapper/110:55:271
1168999217,3cyclictest0-21swapper/107:54:021
1168999216,3cyclictest0-21swapper/111:59:521
1168999216,3cyclictest0-21swapper/110:31:571
11689992116,3cyclictest2463-21expr08:00:131
1168999211,11cyclictest30638-21cat09:05:021
1168499217,4cyclictest7591-21needreboot08:10:180
1168499216,6cyclictest0-21swapper/011:48:280
1168499215,2cyclictest87350irq/126-enp1s0-10:16:240
1168499212,9cyclictest0-21swapper/009:23:560
1168499211,9cyclictest4632-21kworker/0:109:27:400
1168499211,6cyclictest0-21swapper/012:08:410
1168499211,17cyclictest17984-21munin-run09:50:010
1170099201,14cyclictest24541-21ls11:15:263
1170099200,7cyclictest87650irq/129-enp1s0-07:27:153
1170099200,3cyclictest0-21swapper/308:49:303
1169699202,9cyclictest892250kworker/u8:209:51:212
1169699201,5cyclictest0-21swapper/209:33:432
1168999205,8cyclictest87550irq/128-enp1s0-11:20:261
1168999201,8cyclictest87350irq/126-enp1s0-12:27:241
11689992014,3cyclictest19792-21df_inode12:20:151
1168499209,3cyclictest23570-21/usr/sbin/munin07:35:160
1168499209,3cyclictest12246-21sed09:35:150
1168499209,3cyclictest12246-21sed09:35:150
1168499206,5cyclictest0-21swapper/012:18:000
1168499205,9cyclictest0-21swapper/011:12:350
1168499205,6cyclictest0-21swapper/011:33:570
1168499203,15cyclictest10434-21perf12:00:000
1168499202,10cyclictest26038-21/usr/sbin/munin12:35:150
1168499202,10cyclictest22604-21/usr/sbin/munin10:00:180
1168499201,5cyclictest0-21swapper/009:15:360
1168499201,16cyclictest0-21swapper/009:30:280
1168499201,12cyclictest0-21swapper/010:35:210
1168499201,11cyclictest26551-21pmu-power11:20:230
1170099197,3cyclictest87350irq/126-enp1s0-07:55:253
1170099194,3cyclictest87350irq/126-enp1s0-09:35:193
1170099194,3cyclictest87350irq/126-enp1s0-09:35:193
1170099192,5cyclictest16034-21needreboot12:10:183
1170099192,14cyclictest6368-21basename08:10:003
11700991913,3cyclictest27298-21sed07:40:253
1170099191,2cyclictest0-21swapper/310:25:123
1170099191,15cyclictest5936-21missed_timers09:20:193
1170099190,2cyclictest0-21swapper/309:06:183
1169699193,4cyclictest13018-21expr12:05:122
1169699192,10cyclictest0-21swapper/209:19:352
1169699191,8cyclictest0-21swapper/211:46:142
1169699191,7cyclictest827-21runrttasks09:58:332
1169699191,6cyclictest0-21swapper/209:37:042
1169699191,6cyclictest0-21swapper/209:37:042
1169699191,6cyclictest0-21swapper/209:25:432
1168499198,3cyclictest25769-21/usr/sbin/munin07:40:210
1168499198,2cyclictest19470-21kworker/u8:109:07:550
1168499193,10cyclictest0-21swapper/011:50:150
11684991911,5cyclictest827-21runrttasks11:05:280
1168499191,12cyclictest0-21swapper/011:35:110
1168499191,11cyclictest0-21swapper/010:46:430
1168499191,11cyclictest0-21swapper/010:46:430
1168499191,11cyclictest0-21swapper/010:09:020
1168499191,10cyclictest20154-21sshd12:20:170
1168499191,10cyclictest11892-21/usr/sbin/munin07:10:120
1170099182,3cyclictest16251-21sshd10:58:183
1170099181,4cyclictest23623-21perf12:30:023
1170099181,3cyclictest87350irq/126-enp1s0-10:18:483
1169699185,4cyclictest6646-21chrt09:20:492
1169699185,3cyclictest19424-21cat12:20:022
1169699182,3cyclictest0-21swapper/211:31:232
1169699181,8cyclictest22717-21ntp_states12:25:222
1169699181,8cyclictest15627-21grep07:15:182
1169699181,7cyclictest2765-21memory11:40:192
1169699181,7cyclictest0-21swapper/212:20:142
1169699181,7cyclictest0-21swapper/210:30:172
1169699181,5cyclictest0-21swapper/211:09:512
1169699181,5cyclictest0-21swapper/210:15:152
11696991814,3cyclictest0-21swapper/208:12:112
11696991814,3cyclictest0-21swapper/207:49:152
11696991814,3cyclictest0-21swapper/207:25:262
1169699181,11cyclictest0-21swapper/211:57:592
11696991810,5cyclictest8955-21pmu-power10:40:212
1168499189,6cyclictest8922-21kworker/u8:209:50:240
1168499188,3cyclictest4159-21sshd10:30:170
1168499186,3cyclictest87550irq/128-enp1s0-08:50:200
1168499186,3cyclictest0-21swapper/009:41:380
1168499185,3cyclictest2177-21grep09:10:260
1168499182,13cyclictest32350-21/usr/sbin/munin07:55:200
11684991812,3cyclictest24014-21wc08:50:020
11684991811,4cyclictest0-21swapper/011:15:440
1168499181,11cyclictest0-21swapper/011:25:290
123542171,2sleep20-21swapper/212:01:362
1170099171,9cyclictest0-21swapper/308:54:103
11700991711,3cyclictest24237-21gltestperf07:35:163
1169699176,3cyclictest87350irq/126-enp1s0-10:00:172
1169699176,3cyclictest32757-21diskmemload09:47:302
1169699172,5cyclictest0-21swapper/209:40:412
1169699171,9cyclictest0-21swapper/208:00:572
1169699171,7cyclictest21031-21/usr/sbin/munin11:10:202
1169699171,7cyclictest17584-21/usr/sbin/munin08:35:152
1169699171,7cyclictest0-21swapper/210:05:122
1169699171,6cyclictest0-21swapper/210:52:322
1169699171,6cyclictest0-21swapper/210:24:392
1169699171,6cyclictest0-21swapper/209:10:022
1169699171,5cyclictest0-21swapper/211:25:192
1169699171,5cyclictest0-21swapper/207:40:132
1169699171,4cyclictest11492-21unixbench-2d10:45:252
1169699171,4cyclictest11492-21unixbench-2d10:45:252
1169699171,4cyclictest0-21swapper/207:30:192
11696991713,3cyclictest0-21swapper/211:38:202
11696991713,3cyclictest0-21swapper/208:34:312
11696991713,3cyclictest0-21swapper/208:21:482
1169699171,14cyclictest0-21swapper/208:18:422
11696991711,3cyclictest23393-21cut11:15:122
1168499179,7cyclictest7-21ksoftirqd/008:55:080
1168499176,3cyclictest0-21swapper/010:52:480
1168499175,5cyclictest15759-21fschecks_count08:30:170
1168499175,4cyclictest0-21swapper/011:00:240
1168499172,5cyclictest3031-40dnf11:40:230
11684991712,3cyclictest21376-21/usr/sbin/munin07:30:200
11684991711,4cyclictest0-21swapper/009:55:400
1168499171,14cyclictest0-21swapper/008:06:140
1168499171,10cyclictest0-21swapper/010:25:010
1168499171,10cyclictest0-21swapper/007:15:190
1169699169,4cyclictest30250-21tr07:50:112
1169699169,4cyclictest27115-21cpuspeed_turbos10:10:142
1169699162,5cyclictest1967-21munin-run08:00:022
1169699162,3cyclictest13542-21sshd07:11:062
1169699161,6cyclictest22900-21/usr/sbin/munin08:45:192
1169699161,6cyclictest0-21swapper/210:59:502
1169699161,3cyclictest87350irq/126-enp1s0-12:35:402
1169699161,3cyclictest0-21swapper/211:02:302
11696991611,3cyclictest6694-21ntpq10:35:222
11696991611,3cyclictest26160-21if_enp1s011:20:172
11696991611,3cyclictest1294-21cat10:25:122
11696991610,3cyclictest16976-21/usr/sbin/munin07:20:162
1168499164,3cyclictest87550irq/128-enp1s0-07:45:170
11684991612,3cyclictest0-21swapper/008:41:190
11684991612,3cyclictest0-21swapper/008:35:060
1168499161,14cyclictest0-21swapper/007:54:260
1168499161,12cyclictest10991-21/usr/sbin/munin08:20:210
1169699159,3cyclictest30932-21expr09:05:122
1169699159,3cyclictest25254-21ntp_states08:50:222
1169699158,4cyclictest5002-21gltestperf08:05:162
11696991510,3cyclictest23826-21expr07:35:132
1168499156,5cyclictest909-21sssd_nss08:00:190
11684991510,3cyclictest8792-21/usr/sbin/munin08:15:200
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional