You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-18 - 01:08
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Thu Sep 17, 2026 00:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1229421910,22sleep0161ktimers/019:09:060
173121160,28sleep21734-21fschecks_count20:14:282
123939910371,11cyclictest181rcu_preempt19:34:200
12393999642,29cyclictest14273-21ssh23:40:090
12393999642,29cyclictest14273-21ssh23:40:090
12393999633,31cyclictest15-21ksoftirqd/000:04:070
12393999558,30cyclictest0-21swapper/020:24:370
12393999470,18cyclictest6013-21kernelversion21:39:310
12393999455,9cyclictest181rcu_preempt00:04:110
12393999455,9cyclictest181rcu_preempt00:04:110
12401999245,12cyclictest181rcu_preempt21:41:242
12393999169,14cyclictest343-21runrttasks23:53:190
12393999169,14cyclictest343-21runrttasks23:53:190
12393999163,19cyclictest4518-21latency_hist23:24:070
12401999061,20cyclictest1622-21sh23:19:192
1239399903,43cyclictest15-21ksoftirqd/021:33:480
113112909,56sleep211388-21cat19:05:102
12401998934,12cyclictest181rcu_preempt22:49:362
12393998841,12cyclictest181rcu_preempt22:24:200
12393998841,12cyclictest181rcu_preempt22:24:200
123062886,52sleep112330-21ls19:09:081
12398998767,14cyclictest16510-21sh00:35:411
12398998760,20cyclictest9336-21sh20:39:031
12398998760,20cyclictest9336-21sh20:39:031
1239399878,38cyclictest15-21ksoftirqd/022:37:520
1239399878,38cyclictest15-21ksoftirqd/022:37:520
12393998646,14cyclictest181rcu_preempt20:50:130
12393998559,8cyclictest181rcu_preempt22:48:270
12398998464,14cyclictest10454-21sh00:25:381
12398998425,24cyclictest0-21swapper/123:50:131
12398998425,24cyclictest0-21swapper/123:50:131
12393998455,21cyclictest4711-21latency_hist20:24:070
12393998448,11cyclictest181rcu_preempt20:59:260
12393998426,29cyclictest0-21swapper/000:28:140
12401998362,7cyclictest181rcu_preempt22:32:522
12401998361,15cyclictest200-21systemd-journal23:47:262
12401998345,8cyclictest181rcu_preempt19:19:312
12401998343,11cyclictest181rcu_preempt21:56:002
1240199831,38cyclictest39-21ksoftirqd/200:14:582
12393998367,11cyclictest12382-21cyclictest21:34:400
12393998367,11cyclictest12382-21cyclictest21:34:400
12393998360,8cyclictest181rcu_preempt22:58:140
12393998355,8cyclictest181rcu_preempt23:05:370
12393998328,33cyclictest0-21swapper/022:19:390
12401998241,13cyclictest181rcu_preempt23:43:252
12401998241,13cyclictest181rcu_preempt23:43:252
12401998236,17cyclictest181rcu_preempt23:58:362
12401998236,17cyclictest181rcu_preempt23:58:362
12398998266,10cyclictest25086-21ssh23:04:281
12398998262,14cyclictest12382-21cyclictest20:59:041
12398998228,26cyclictest0-21swapper/122:49:371
12393998257,20cyclictest12382-21cyclictest22:14:310
12401998152,8cyclictest181rcu_preempt20:21:062
12401998151,7cyclictest181rcu_preempt20:55:512
12401998141,8cyclictest181rcu_preempt20:40:022
12398998152,22cyclictest0-21swapper/100:04:391
12398998152,22cyclictest0-21swapper/100:04:391
12393998152,8cyclictest181rcu_preempt23:10:410
12393998152,8cyclictest181rcu_preempt23:10:410
12393998150,11cyclictest181rcu_preempt21:05:200
12393998141,28cyclictest17949-21cut21:59:250
12393998138,32cyclictest3052-21ssh00:14:200
1239399813,50cyclictest15-21ksoftirqd/019:49:290
12401998044,9cyclictest181rcu_preempt19:43:092
12393998057,8cyclictest181rcu_preempt21:17:250
12393998051,16cyclictest181rcu_preempt23:44:320
12393998038,13cyclictest181rcu_preempt21:51:510
12393998036,8cyclictest181rcu_preempt19:14:380
12393998036,8cyclictest181rcu_preempt19:14:380
12401997958,13cyclictest12382-21cyclictest21:18:402
12401997940,10cyclictest181rcu_preempt19:51:222
12401997935,12cyclictest181rcu_preempt21:21:462
12401997933,8cyclictest181rcu_preempt20:04:142
12398997960,13cyclictest5246-21netstat20:24:311
1239899793,67cyclictest31786-21apt-get00:09:181
1239899793,67cyclictest31786-21apt-get00:09:181
12393997944,17cyclictest181rcu_preempt22:13:570
12393997936,10cyclictest181rcu_preempt22:44:000
1239399793,40cyclictest15-21ksoftirqd/021:54:360
1239399792,34cyclictest15-21ksoftirqd/022:04:240
12401997843,16cyclictest181rcu_preempt00:36:492
12401997837,13cyclictest181rcu_preempt20:35:462
12401997837,13cyclictest181rcu_preempt20:35:462
1240199783,42cyclictest39-21ksoftirqd/220:49:212
12398997833,33cyclictest0-21swapper/119:29:061
12393997857,14cyclictest200-21systemd-journal21:29:040
12393997836,15cyclictest181rcu_preempt23:18:120
12393997833,36cyclictest24049-21grep23:54:400
12393997833,36cyclictest24049-21grep23:54:400
12393997832,8cyclictest181rcu_preempt20:04:190
12393997831,24cyclictest15-21ksoftirqd/022:59:370
12393997831,24cyclictest15-21ksoftirqd/022:59:370
73072770,15sleep0161ktimers/023:29:030
12401997733,15cyclictest181rcu_preempt21:09:082
12401997733,15cyclictest181rcu_preempt21:09:082
12398997762,10cyclictest12382-21cyclictest20:49:291
12398997756,15cyclictest1-21systemd21:38:191
12398997756,15cyclictest1-21systemd21:38:191
12398997755,16cyclictest662-21cron22:44:031
12398997749,22cyclictest7062-21ntp_kernel_err22:34:321
12398997749,22cyclictest7062-21ntp_kernel_err22:34:321
12393997742,27cyclictest12736-21latency_hist20:49:070
12393997732,7cyclictest181rcu_preempt21:46:340
12393997732,7cyclictest181rcu_preempt21:46:340
12401997652,15cyclictest10067-21sh22:39:292
12401997651,18cyclictest12382-21cyclictest22:02:172
12401997638,10cyclictest181rcu_preempt20:24:512
12401997636,10cyclictest181rcu_preempt23:11:592
12401997636,10cyclictest181rcu_preempt23:11:592
12401997632,16cyclictest181rcu_preempt23:00:142
12401997632,16cyclictest181rcu_preempt23:00:142
12401997616,25cyclictest0-21swapper/221:05:202
12398997641,30cyclictest0-21swapper/122:14:421
12398997623,31cyclictest0-21swapper/123:13:381
12398997623,31cyclictest0-21swapper/123:13:381
12393997654,8cyclictest181rcu_preempt00:22:510
12393997649,10cyclictest181rcu_preempt19:39:300
12393997632,12cyclictest181rcu_preempt20:44:060
12393997631,14cyclictest181rcu_preempt20:10:130
12401997550,8cyclictest181rcu_preempt22:14:312
12401997549,11cyclictest181rcu_preempt23:15:582
12401997545,8cyclictest181rcu_preempt21:47:352
12401997545,8cyclictest181rcu_preempt21:47:352
12401997541,12cyclictest14161-21munin-node21:31:112
12401997538,9cyclictest181rcu_preempt22:56:132
12401997537,12cyclictest181rcu_preempt23:06:402
12401997534,15cyclictest181rcu_preempt20:02:092
12401997534,15cyclictest181rcu_preempt20:02:092
12398997561,9cyclictest22018-21unixbench_singl19:34:401
12398997559,9cyclictest0-21swapper/123:54:131
12398997559,9cyclictest0-21swapper/123:54:131
12398997554,14cyclictest343-21runrttasks23:45:071
1239899752,66cyclictest25880-21kworker/u19:0+flush-179:023:19:521
1239899752,65cyclictest13995-21ssh23:39:231
1239899752,65cyclictest13995-21ssh23:39:231
12393997541,8cyclictest181rcu_preempt23:35:050
12393997541,8cyclictest181rcu_preempt23:35:050
12393997538,17cyclictest181rcu_preempt20:29:200
12393997535,8cyclictest181rcu_preempt19:54:290
1239399753,41cyclictest15-21ksoftirqd/000:10:320
1239399753,41cyclictest15-21ksoftirqd/000:10:320
12393997513,8cyclictest161ktimers/019:29:390
103892752,7sleep00-21swapper/023:34:060
12401997435,10cyclictest181rcu_preempt22:06:512
12401997413,36cyclictest39-21ksoftirqd/222:26:062
12401997413,36cyclictest39-21ksoftirqd/222:26:062
12401997412,27cyclictest39-21ksoftirqd/221:34:262
12401997412,27cyclictest39-21ksoftirqd/221:34:262
12398997456,13cyclictest12382-21cyclictest22:04:251
12398997452,16cyclictest29900-21threads19:59:381
12398997452,16cyclictest29900-21threads19:59:381
12398997442,25cyclictest0-21swapper/122:29:251
12398997441,24cyclictest13896-21modprobe19:14:041
12393997449,15cyclictest16966-21ssh00:36:550
12393997444,11cyclictest181rcu_preempt22:52:440
12393997442,12cyclictest181rcu_preempt20:14:240
12393997418,33cyclictest15-21ksoftirqd/019:19:410
1240699737,49cyclictest0-21swapper/300:09:263
1240699737,49cyclictest0-21swapper/300:09:263
12401997337,9cyclictest181rcu_preempt22:44:472
12401997336,9cyclictest181rcu_preempt22:36:402
12401997336,9cyclictest181rcu_preempt22:36:402
12401997336,11cyclictest181rcu_preempt23:25:442
12401997332,32cyclictest19658-21cut19:29:262
12401997325,21cyclictest39-21ksoftirqd/200:00:152
12398997354,13cyclictest343-21runrttasks20:19:201
12393997353,8cyclictest181rcu_preempt22:29:270
12393997350,8cyclictest181rcu_preempt19:44:210
12393997343,10cyclictest181rcu_preempt20:03:240
12393997343,10cyclictest181rcu_preempt20:03:240
12393997341,10cyclictest181rcu_preempt19:24:350
1240699723,54cyclictest10749-21chrt23:34:203
1240699723,54cyclictest10749-21chrt23:34:203
12401997244,8cyclictest181rcu_preempt00:19:162
1240199724,39cyclictest39-21ksoftirqd/200:24:252
12401997239,10cyclictest11792-21ssh21:49:212
12401997233,12cyclictest181rcu_preempt00:30:242
12401997233,12cyclictest181rcu_preempt00:30:242
12401997233,11cyclictest181rcu_preempt19:47:302
12401997233,10cyclictest26101-21ssh22:12:592
12401997224,27cyclictest0-21swapper/219:55:012
12398997252,14cyclictest19272-21diskmemload21:44:281
12398997252,14cyclictest19272-21diskmemload21:44:281
12398997236,23cyclictest0-21swapper/123:29:171
12398997222,24cyclictest27496-21/usr/sbin/munin19:54:201
12398997218,24cyclictest5382-21sh00:18:251
12393997248,9cyclictest21211-21kworker/u19:1+flush-179:021:19:140
12393997241,10cyclictest181rcu_preempt19:11:210
1239399722,36cyclictest15-21ksoftirqd/020:54:230
12401997140,8cyclictest181rcu_preempt20:09:262
12401997137,8cyclictest181rcu_preempt20:44:272
12401997136,9cyclictest181rcu_preempt00:12:352
12401997136,9cyclictest181rcu_preempt00:12:352
12401997133,9cyclictest181rcu_preempt21:24:592
12393997141,7cyclictest181rcu_preempt00:30:460
12393997141,7cyclictest181rcu_preempt00:30:460
12393997125,12cyclictest181rcu_preempt20:38:030
12393997125,12cyclictest181rcu_preempt20:38:030
1240699704,52cyclictest26816-21taskset21:19:393
1240699702,55cyclictest24315-21chrt19:44:203
1240699702,53cyclictest23007-21chrt23:54:143
1240699702,53cyclictest23007-21chrt23:54:143
12401997047,7cyclictest181rcu_preempt22:19:282
12401997038,10cyclictest181rcu_preempt23:34:462
12401997038,10cyclictest181rcu_preempt23:34:462
12401997035,11cyclictest181rcu_preempt19:14:092
12401997035,11cyclictest181rcu_preempt19:14:092
12401997033,11cyclictest181rcu_preempt19:34:322
12401997033,10cyclictest181rcu_preempt23:49:152
12401997033,10cyclictest181rcu_preempt23:49:152
12401997028,9cyclictest371rcuc/200:04:562
12401997028,9cyclictest371rcuc/200:04:562
12398997039,16cyclictest23985-21kworker/u17:1-rpciod22:20:141
12398997033,26cyclictest0-21swapper/122:54:341
12398997019,23cyclictest0-21swapper/123:24:191
1240699694,50cyclictest0-21swapper/321:49:183
1240699693,52cyclictest8437-21chrt20:34:303
1240699693,52cyclictest8437-21chrt20:34:303
1240699693,51cyclictest6525-21taskset00:19:313
1240699693,51cyclictest16790-21taskset21:57:323
1240699693,50cyclictest10739-21chrt20:41:183
1240699692,54cyclictest12671-21chrt00:29:363
1240699692,54cyclictest12671-21chrt00:29:363
12401996932,8cyclictest181rcu_preempt20:29:152
1240199692,41cyclictest39-21ksoftirqd/223:29:182
12398996944,18cyclictest32255-21fschecks_time21:29:251
12398996927,24cyclictest0-21swapper/100:31:501
12398996927,24cyclictest0-21swapper/100:31:501
1239399693,25cyclictest15-21ksoftirqd/021:11:520
1239399693,25cyclictest15-21ksoftirqd/021:11:520
1240699683,53cyclictest28256-21taskset23:09:323
1240699683,53cyclictest28256-21taskset23:09:323
1240699683,52cyclictest16844-21taskset22:50:033
1240699683,51cyclictest3715-21chrt20:19:363
1240699683,50cyclictest26170-21taskset23:05:393
1240699682,54cyclictest29818-21taskset00:04:403
1240699682,54cyclictest29818-21taskset00:04:403
12401996841,12cyclictest181rcu_preempt19:09:412
12401996839,9cyclictest181rcu_preempt21:03:102
12398996848,15cyclictest12382-21cyclictest21:44:061
12398996838,23cyclictest0-21swapper/123:14:391
12398996835,23cyclictest0-21swapper/122:59:501
12398996835,23cyclictest0-21swapper/122:59:501
1240699673,52cyclictest0-21swapper/322:44:153
1240699673,50cyclictest28309-21chrt00:02:493
1240699673,50cyclictest20329-21taskset22:03:523
12401996732,8cyclictest181rcu_preempt19:25:362
12398996749,13cyclictest0-21swapper/121:54:261
12398996733,23cyclictest0-21swapper/121:05:371
12398996726,18cyclictest0-21swapper/121:22:321
12398996721,20cyclictest0-21swapper/123:35:521
12398996721,20cyclictest0-21swapper/123:35:521
1240699664,50cyclictest15834-21chrt20:59:043
1240699664,49cyclictest29620-21chrt19:59:313
1240699664,49cyclictest29620-21chrt19:59:313
1240699664,49cyclictest14318-21chrt23:40:163
1240699664,49cyclictest14318-21chrt23:40:163
1240699663,51cyclictest14955-21chrt19:14:323
1240699663,51cyclictest14955-21chrt19:14:323
1240699663,50cyclictest7133-21chrt22:34:363
1240699663,50cyclictest7133-21chrt22:34:363
1240699663,50cyclictest25493-21taskset22:11:183
1240699663,50cyclictest16251-21chrt20:59:243
1240699663,49cyclictest3947-21taskset21:35:323
1240699663,49cyclictest3947-21taskset21:35:323
1240699662,52cyclictest5720-21chrt22:32:493
1240699662,52cyclictest28278-21chrt22:15:393
1240699662,51cyclictest31861-21chrt23:14:433
1240699662,51cyclictest15564-21chrt00:34:303
1240699662,49cyclictest748-21taskset21:30:043
12398996646,13cyclictest21524-21sh21:11:541
12398996646,13cyclictest21524-21sh21:11:541
1239899663,43cyclictest24506-21sort19:44:261
12398996623,29cyclictest0-21swapper/122:47:581
12398996621,16cyclictest19272-21diskmemload21:49:591
12398996620,38cyclictest32173-21latency_hist20:09:061
1104726618,26sleep30-21swapper/319:04:583
1240699654,49cyclictest2537-21chrt23:20:353
1240699653,50cyclictest9839-21chrt23:32:423
1240699653,50cyclictest32281-21chrt20:09:073
1240699653,49cyclictest9734-21chrt22:39:223
1240699653,49cyclictest23022-21chrt22:07:573
1240699652,50cyclictest24919-21chrt21:17:153
1240699652,49cyclictest6262-21chrt23:26:103
1240699652,49cyclictest21764-21taskset21:12:333
1240699652,49cyclictest21764-21taskset21:12:333
12398996547,10cyclictest0-21swapper/121:24:341
1239899653,47cyclictest24110-21fschecks_time22:09:261
1239899653,41cyclictest26565-21awk19:49:381
12398996531,26cyclictest0-21swapper/122:04:051
12398996522,33cyclictest26710-21awk23:59:361
1240699644,47cyclictest0-21swapper/321:24:153
1240699643,49cyclictest21377-21chrt23:50:293
1240699643,49cyclictest21377-21chrt23:50:293
1240699643,48cyclictest23233-21taskset23:00:533
1240699643,48cyclictest23233-21taskset23:00:533
1240699643,47cyclictest3421-21taskset00:14:293
1240699643,47cyclictest27156-21chrt19:53:033
1240699642,51cyclictest1825-21taskset22:25:243
1240699642,51cyclictest1825-21taskset22:25:243
1240699642,50cyclictest22068-21chrt19:34:533
1240699642,50cyclictest10065-21chrt21:45:413
1240699642,50cyclictest10065-21chrt21:45:413
1240699642,49cyclictest7428-21chrt21:41:443
1240699642,49cyclictest19069-21chrt21:06:333
1240699642,48cyclictest12452-21taskset20:47:383
12398996449,9cyclictest359-21snmpd00:22:311
12398996447,10cyclictest0-21swapper/119:29:211
1239899644,39cyclictest17427-21latency_hist21:04:071
1239899643,42cyclictest24239-21grep19:44:071
1239899643,41cyclictest12575-21latency_hist20:49:071
12398996430,26cyclictest0-21swapper/120:14:261
12398996427,26cyclictest0-21swapper/119:23:321
1240699635,45cyclictest0-21swapper/320:49:173
1240699633,50cyclictest32683-21chrt20:09:273
1240699633,48cyclictest28045-21chrt19:54:333
1240699633,48cyclictest1680-21chrt20:14:253
1240699633,47cyclictest11180-21chrt00:27:373
1240699633,47cyclictest0-21swapper/322:19:143
1240699632,50cyclictest18642-21chrt22:54:173
1240699632,49cyclictest5223-21chrt20:24:303
1240699632,49cyclictest20188-21chrt19:29:383
1240699632,49cyclictest18215-21chrt23:45:063
12398996344,12cyclictest0-21swapper/120:09:251
1240699624,46cyclictest17529-21chrt19:23:313
1240699624,45cyclictest23778-21chrt19:40:523
1240699623,47cyclictest17994-21chrt19:24:203
1240699622,47cyclictest7380-21chrt20:29:483
1240699622,47cyclictest13569-21taskset19:09:503
12398996248,8cyclictest0-21swapper/119:19:061
12398996248,8cyclictest0-21swapper/119:19:061
12398996241,15cyclictest6553-21fschecks_count20:29:251
1239899623,39cyclictest11147-21latency_hist20:44:071
1239899613,44cyclictest423-21/usr/sbin/munin22:24:411
1239899613,44cyclictest423-21/usr/sbin/munin22:24:411
12398996130,23cyclictest0-21swapper/121:15:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional