You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-28 - 06:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot3s.osadl.org (updated Sun Jun 28, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
252492239164,24sleep20-21swapper/219:07:572
252072212180,21sleep30-21swapper/319:07:263
232172204170,23sleep00-21swapper/019:05:010
252832202164,25sleep10-21swapper/119:08:241
2682821700,2sleep32550699cyclictest19:10:233
2520721410,5sleep22550099cyclictest21:35:022
2998821380,5sleep32550699cyclictest20:30:173
2744621280,4sleep12549699cyclictest19:12:211
25500998380,2cyclictest0-21swapper/220:27:272
25500996663,2cyclictest0-21swapper/220:32:162
188342640,2sleep20-21swapper/223:45:202
119682620,3sleep232-21ksoftirqd/222:15:222
68272570,11sleep00-21swapper/020:50:200
135922380,3sleep213597-21cpuspeed_turbos23:35:122
135922380,3sleep213597-21cpuspeed_turbos23:35:122
2550699371,27cyclictest0-21swapper/321:26:353
25500993728,8cyclictest0-21swapper/200:00:452
25500993525,3cyclictest0-21swapper/200:06:042
2549199359,12cyclictest19842-21awk23:50:010
2550699341,13cyclictest0-21swapper/321:49:253
2550099341,13cyclictest0-21swapper/223:18:452
2549199340,13cyclictest18253-21kworker/0:221:25:230
2550099331,13cyclictest0-21swapper/219:25:492
2549699331,9cyclictest495-21jbd2/dm-0-823:57:321
2549199334,13cyclictest14197-21diskmemload22:25:010
25506993224,3cyclictest0-21swapper/300:31:203
2550699321,12cyclictest0-21swapper/300:21:583
2550699321,12cyclictest0-21swapper/300:16:183
2550699321,11cyclictest0-21swapper/322:34:173
25500993219,3cyclictest0-21swapper/200:31:192
2550099321,18cyclictest0-21swapper/223:20:412
2550099321,16cyclictest0-21swapper/220:21:332
2550099321,15cyclictest0-21swapper/221:41:142
2550099321,15cyclictest0-21swapper/219:14:572
2550099321,14cyclictest0-21swapper/221:46:342
2550099321,14cyclictest0-21swapper/220:00:052
2550099321,13cyclictest0-21swapper/221:07:222
2549199325,12cyclictest5076-21idleruntime-cro00:30:000
2550099318,3cyclictest0-21swapper/219:31:452
25500993117,3cyclictest0-21swapper/220:48:432
2550099311,16cyclictest0-21swapper/219:47:292
2550099311,14cyclictest0-21swapper/223:40:052
2550099311,14cyclictest0-21swapper/219:15:572
2550099311,13cyclictest0-21swapper/222:53:372
2550099311,13cyclictest0-21swapper/221:55:382
2550099311,13cyclictest0-21swapper/221:38:142
2550099310,13cyclictest0-21swapper/222:46:332
2549199314,13cyclictest3823-21munin-run22:00:000
2549199311,14cyclictest28013-21munin-run22:55:020
25506993019,3cyclictest0-21swapper/300:03:473
25506993013,6cyclictest0-21swapper/319:55:123
25506993012,7cyclictest0-21swapper/300:36:043
2550699301,15cyclictest0-21swapper/321:04:453
2550699301,13cyclictest0-21swapper/321:22:493
2550699301,13cyclictest0-21swapper/319:15:453
2550099302,17cyclictest22987-21munin-run21:30:022
25500993011,8cyclictest0-21swapper/220:55:052
2550099301,14cyclictest0-21swapper/223:06:532
2550099301,14cyclictest0-21swapper/221:17:102
2550099301,14cyclictest0-21swapper/220:54:492
2550099301,14cyclictest0-21swapper/219:23:502
2550099301,13cyclictest0-21swapper/222:57:412
2550099301,13cyclictest0-21swapper/222:02:302
2550099301,13cyclictest0-21swapper/220:17:132
2550099301,13cyclictest0-21swapper/220:05:132
2550099301,13cyclictest0-21swapper/220:05:132
2550099301,13cyclictest0-21swapper/219:42:092
2550099301,12cyclictest0-21swapper/219:57:092
25500993010,7cyclictest0-21swapper/222:43:292
25506992918,3cyclictest0-21swapper/323:48:313
2550699291,12cyclictest0-21swapper/322:35:053
2550699291,12cyclictest0-21swapper/320:02:293
2550099297,3cyclictest0-21swapper/223:01:012
2550099291,15cyclictest0-21swapper/222:23:052
2550099291,15cyclictest0-21swapper/222:10:572
2550099291,14cyclictest0-21swapper/223:30:092
2550099291,14cyclictest0-21swapper/200:21:172
2550099291,13cyclictest0-21swapper/200:36:532
2550099291,12cyclictest19500-21smartctl21:20:182
2550099291,12cyclictest0-21swapper/200:11:382
2550099291,11cyclictest0-21swapper/222:35:492
2550099290,14cyclictest0-21swapper/220:13:012
2549699292,15cyclictest26121-40dnf00:01:041
2549699290,27cyclictest0-21swapper/121:40:231
2549199296,11cyclictest11389-21basename19:50:000
2549199292,10cyclictest10385-21kworker/0:122:19:430
2549199291,19cyclictest18253-21kworker/0:221:30:230
2549199291,13cyclictest16414-21idleruntime-cro21:15:010
2550699289,3cyclictest0-21swapper/323:20:333
2550699288,3cyclictest0-21swapper/320:23:213
25506992815,3cyclictest0-21swapper/320:05:493
25506992815,3cyclictest0-21swapper/320:05:493
2550699281,21cyclictest0-21swapper/322:16:543
2550699281,12cyclictest0-21swapper/323:42:533
25506992810,3cyclictest0-21swapper/321:06:253
2550099286,4cyclictest0-21swapper/219:35:212
2550099283,3cyclictest0-21swapper/223:25:052
2550099281,25cyclictest0-21swapper/222:30:032
2550099281,13cyclictest0-21swapper/221:11:262
2550099281,12cyclictest0-21swapper/223:58:182
2550099281,12cyclictest0-21swapper/221:51:222
2550099281,12cyclictest0-21swapper/200:18:542
2550099281,11cyclictest0-21swapper/219:53:292
2550099280,12cyclictest0-21swapper/200:26:572
2549199282,10cyclictest18960-21kworker/0:200:20:590
2549199281,12cyclictest5850-21kworker/0:222:05:070
2550699273,14cyclictest401ktimersoftd/319:43:343
25506992716,3cyclictest0-21swapper/322:46:313
2550699271,20cyclictest0-21swapper/319:24:253
2550699271,14cyclictest0-21swapper/319:31:463
2550699271,12cyclictest0-21swapper/322:28:143
2550699271,12cyclictest0-21swapper/321:38:053
2550699271,10cyclictest0-21swapper/323:38:293
2550699271,10cyclictest0-21swapper/323:38:293
25506992710,3cyclictest0-21swapper/300:12:133
2550099271,13cyclictest0-21swapper/221:04:412
2550099271,10cyclictest0-21swapper/223:54:502
2550099271,10cyclictest0-21swapper/223:10:312
2549699270,8cyclictest0-21swapper/121:20:011
2549199272,15cyclictest21142-21kworker/0:122:39:200
2549199272,13cyclictest15774-21munin-run20:00:010
2549199272,10cyclictest30260-21kworker/0:019:17:150
2549199271,9cyclictest27376-21kworker/0:021:53:350
2549199271,15cyclictest27841-21kworker/0:023:03:480
2549199271,11cyclictest8571-21kworker/0:023:23:150
2549199271,10cyclictest18960-21kworker/0:200:15:190
25506992613,8cyclictest0-21swapper/322:24:323
2550699261,16cyclictest0-21swapper/319:29:343
2550699261,13cyclictest0-21swapper/323:25:253
2550699261,12cyclictest0-21swapper/321:56:053
2550699261,10cyclictest0-21swapper/323:13:213
2550099267,3cyclictest0-21swapper/220:43:332
2550099261,15cyclictest0-21swapper/220:36:292
2549699261,17cyclictest579-21systemd-journal19:35:011
2549199261,10cyclictest8114-21kworker/0:022:11:590
2549199261,10cyclictest18960-21kworker/0:200:11:470
2549199261,10cyclictest18907-21kworker/0:222:32:200
2549199261,10cyclictest12996-21kworker/0:323:34:550
2549199261,10cyclictest10385-21kworker/0:122:27:440
2550699259,3cyclictest28348-21sshd00:06:243
2550699258,3cyclictest0-21swapper/323:56:293
2550699257,3cyclictest0-21swapper/322:55:133
2550699257,3cyclictest0-21swapper/322:06:303
2550699251,3cyclictest14774-21diskstats21:10:133
2550699251,15cyclictest0-21swapper/321:52:533
2550699251,14cyclictest0-21swapper/322:00:423
2550699251,13cyclictest0-21swapper/320:58:093
2550699251,13cyclictest0-21swapper/320:43:333
2550699251,13cyclictest0-21swapper/320:18:533
2550699251,13cyclictest0-21swapper/319:50:263
2550699251,12cyclictest0-21swapper/320:25:053
2550699251,11cyclictest6075-21ntp_states19:35:223
2550699251,11cyclictest0-21swapper/322:51:333
25506992510,3cyclictest0-21swapper/323:52:233
25506992510,3cyclictest0-21swapper/323:33:373
2549699253,4cyclictest0-21swapper/120:50:241
2549699251,3cyclictest593-21perf00:20:001
2549199252,8cyclictest18960-21kworker/0:223:53:230
2549199251,14cyclictest27376-21kworker/0:022:00:430
2550699248,3cyclictest0-21swapper/322:10:543
2550699247,3cyclictest0-21swapper/322:41:573
2550699241,14cyclictest0-21swapper/300:27:543
2550699241,13cyclictest0-21swapper/323:01:013
2550699241,13cyclictest0-21swapper/320:14:133
2550699241,11cyclictest31392-21/usr/sbin/munin20:35:153
2550099241,15cyclictest0-21swapper/222:09:212
2550099241,13cyclictest0-21swapper/222:25:452
2549699246,10cyclictest20066-21basename23:50:131
25496992418,3cyclictest25414-21/usr/sbin/munin21:35:201
2549699241,3cyclictest5088-21munin-run00:30:001
25496992413,2cyclictest0-21swapper/123:05:121
2549699241,16cyclictest5155-21pmu-power22:00:231
25496992411,11cyclictest5187-21kworker/u8:322:16:541
2549699240,17cyclictest0-21swapper/120:02:251
2549199242,8cyclictest9628-21kworker/0:021:09:350
2549199241,9cyclictest17762-21kworker/0:220:15:280
25491992412,3cyclictest26435-21kworker/0:120:00:190
25491992411,3cyclictest6317-21kworker/0:123:27:230
2549199241,11cyclictest0-21swapper/020:33:250
25506992311,8cyclictest0-21swapper/321:41:313
2550699231,11cyclictest0-21swapper/323:07:533
2549699239,4cyclictest0-21swapper/100:30:161
2549699235,13cyclictest0-21swapper/123:40:231
2549199238,14cyclictest0-21swapper/020:15:020
2549199232,7cyclictest18960-21kworker/0:223:56:430
2549199231,9cyclictest18253-21kworker/0:221:40:030
2549199231,8cyclictest18960-21kworker/0:200:04:590
2549199231,7cyclictest26269-21kworker/0:100:07:230
2549199231,12cyclictest25826-21perf22:50:020
2549199231,10cyclictest27841-21kworker/0:023:10:520
2550699225,4cyclictest7399-21sshd20:52:383
2550699221,14cyclictest0-21swapper/321:18:013
2550699221,11cyclictest6136-21sshd23:15:243
2550699221,11cyclictest0-21swapper/321:32:373
2550699221,11cyclictest0-21swapper/319:45:263
2549699229,5cyclictest9707-21grep23:25:161
2549699222,15cyclictest0-21swapper/119:53:041
2549699221,8cyclictest0-21swapper/122:49:211
2549699221,8cyclictest0-21swapper/100:24:181
2549699221,8cyclictest0-21swapper/100:07:361
25496992214,3cyclictest0-21swapper/121:25:511
25496992214,1cyclictest0-21swapper/119:25:011
2549699221,18cyclictest29546-21diskstats20:30:141
2549699221,15cyclictest14197-21diskmemload22:08:201
2549699221,11cyclictest9791-21turbostat21:00:011
2549699221,11cyclictest23609-21munin-run22:45:011
2549199228,2cyclictest4768-21kworker/0:100:36:070
2549199227,9cyclictest27841-21kworker/0:022:59:320
2549199222,9cyclictest18253-21kworker/0:221:17:270
2549199222,7cyclictest17762-21kworker/0:220:22:560
2549199222,11cyclictest18253-21kworker/0:221:36:390
2549199220,10cyclictest17762-21kworker/0:220:25:200
25506992110,3cyclictest0-21swapper/320:48:233
2549699213,10cyclictest1161-21cut19:25:161
2549699211,4cyclictest14426-21/usr/sbin/munin21:10:111
25496992114,2cyclictest0-21swapper/123:15:311
2549699211,3cyclictest10789-21kworker/u8:023:32:131
25496992112,3cyclictest0-21swapper/100:36:361
2549699211,18cyclictest32014-21uptime23:00:261
2549699211,15cyclictest0-21swapper/122:20:531
2549699211,15cyclictest0-21swapper/121:56:141
2549699211,15cyclictest0-21swapper/121:05:511
2549699211,11cyclictest30790-21needreboot21:45:191
2549699211,11cyclictest27143-21grep20:25:121
2549699211,11cyclictest26783-21awk20:25:011
2549699211,11cyclictest24578-21perf20:20:011
25496992110,3cyclictest31740-21diskstats20:35:131
2549699210,5cyclictest0-21swapper/119:55:141
2549699210,4cyclictest14767-21kworker/u8:222:32:351
2549199217,3cyclictest7367-21kworker/0:120:55:190
2549199217,3cyclictest3159-21kworker/0:221:00:150
2549199216,2cyclictest26435-21kworker/0:119:32:110
2549199215,2cyclictest6890-21kworker/0:319:41:230
2549199212,11cyclictest4768-21kworker/0:100:30:030
2549199211,8cyclictest4238-21kworker/0:219:11:080
2549199211,8cyclictest26435-21kworker/0:119:51:270
2549199211,6cyclictest930-21kworker/0:020:41:360
2549199211,5cyclictest30599-21if_err_enp2s021:45:170
2549199211,12cyclictest6317-21kworker/0:123:42:470
2549199211,12cyclictest27841-21kworker/0:023:15:200
2549199211,11cyclictest3159-21kworker/0:220:45:070
2549199211,11cyclictest21142-21kworker/0:122:42:440
2549199211,11cyclictest18253-21kworker/0:221:22:310
2549699209,2cyclictest0-21swapper/119:15:031
2549699203,4cyclictest0-21swapper/123:10:351
2549699202,2cyclictest29940-21kworker/u8:022:59:481
2549699202,15cyclictest16482-21wc22:25:251
2549699201,6cyclictest0-21swapper/122:51:351
2549699201,6cyclictest0-21swapper/121:24:201
25496992016,3cyclictest0-21swapper/122:37:511
25496992015,3cyclictest0-21swapper/119:49:391
25496992013,2cyclictest0-21swapper/120:40:041
2549699201,14cyclictest0-21swapper/123:22:341
2549699201,14cyclictest0-21swapper/122:14:431
2549699201,14cyclictest0-21swapper/121:31:411
2549699201,14cyclictest0-21swapper/119:39:521
2549699201,13cyclictest0-21swapper/123:36:441
2549699201,13cyclictest0-21swapper/123:36:441
25496992011,3cyclictest0-21swapper/121:50:361
2549699201,11cyclictest19270-21ntp_states20:05:211
2549699201,11cyclictest19270-21ntp_states20:05:211
2549699201,11cyclictest11997-21latency_hist21:05:021
2549699200,18cyclictest0-21swapper/120:45:171
2549199209,3cyclictest13462-21/usr/sbin/munin23:35:260
2549199209,3cyclictest13462-21/usr/sbin/munin23:35:260
2549199202,9cyclictest17762-21kworker/0:220:05:320
2549199202,9cyclictest17762-21kworker/0:220:05:320
2549199202,10cyclictest32335-21kworker/0:123:05:120
2549199201,11cyclictest521-21/usr/sbin/munin19:25:160
2549199201,11cyclictest26435-21kworker/0:119:35:030
2549699191,14cyclictest0-21swapper/100:13:151
2549699191,10cyclictest8236-21ntp_kernel_err19:40:201
25496991910,2cyclictest87650irq/129-enp1s0-20:10:451
2549699190,14cyclictest0-21swapper/123:45:511
2549199195,4cyclictest26435-21kworker/0:119:22:310
2549199191,8cyclictest17762-21kworker/0:220:35:360
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional