You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-14 - 03:09
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Mon Sep 14, 2026 00:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30077216415,104sleep029594-21/usr/sbin/munin19:04:520
31098999950,42cyclictest24432-21fschecks_time21:39:392
31098999950,42cyclictest24432-21fschecks_time21:39:392
31098999844,46cyclictest0-21swapper/223:59:422
31098999838,53cyclictest20330-21sh23:19:522
31098999830,59cyclictest0-21swapper/223:41:072
31098999830,59cyclictest0-21swapper/223:41:072
31098999811,56cyclictest7658-21apt-get22:59:292
31098999529,11cyclictest0-21swapper/222:04:542
31098999529,11cyclictest0-21swapper/222:04:542
3109899946,38cyclictest5812-21cat19:29:192
3109899943,36cyclictest27380-21sort21:44:382
31098999433,55cyclictest0-21swapper/220:09:182
3109899934,38cyclictest21954-21/usr/sbin/munin22:30:132
3109899934,38cyclictest21954-21/usr/sbin/munin22:30:132
3109899934,34cyclictest343-21runrttasks19:14:222
3109899934,34cyclictest343-21runrttasks19:14:222
31098999341,43cyclictest0-21swapper/221:09:432
31098999337,12cyclictest326-21systemd-logind00:34:162
3109899932,42cyclictest6646-21switchtime19:29:512
31098999240,44cyclictest0-21swapper/220:59:412
3109899923,38cyclictest27694-21latency_hist22:39:182
3109899923,34cyclictest1963-21/usr/sbin/munin22:49:422
3109899923,34cyclictest1963-21/usr/sbin/munin22:49:422
3109899922,30cyclictest9083-21latency_hist19:39:192
31094999271,15cyclictest23598-21sh23:24:541
31094999266,20cyclictest24651-21sh21:39:441
31094999266,20cyclictest24651-21sh21:39:441
3109899913,33cyclictest4395-21/usr/sbin/munin22:54:342
3109899913,33cyclictest10045-21sort22:09:382
3109899903,34cyclictest10161-21grep19:39:542
3109899903,34cyclictest10161-21grep19:39:542
3109899903,32cyclictest19718-21sort20:09:532
3109899903,28cyclictest21843-21apt-key22:29:212
3109899903,26cyclictest25997-21/usr/sbin/munin23:29:432
3109899902,43cyclictest1312-21tr20:54:412
3109899902,43cyclictest1312-21tr20:54:412
31094999070,15cyclictest31118-21fschecks_count22:44:381
3109899893,41cyclictest10894-21cstates19:44:342
3109899893,35cyclictest9451-21grep21:14:462
3109899893,34cyclictest2210-21fschecks_count23:44:372
3109899893,34cyclictest13737-21unixbench_multi22:14:542
3109899893,31cyclictest15624-21/usr/sbin/munin19:59:342
3109899893,31cyclictest15624-21/usr/sbin/munin19:59:342
31098998933,11cyclictest0-21swapper/223:24:332
31098998929,11cyclictest0-21swapper/220:54:192
3109899892,37cyclictest26576-21cut00:25:012
3109899892,33cyclictest16233-21munin-plugin-st23:14:152
31098998923,13cyclictest0-21swapper/200:09:162
31098998923,13cyclictest0-21swapper/200:09:162
31098998834,46cyclictest0-21swapper/219:54:372
3109899883,37cyclictest26943-21cpu20:34:312
3109899883,37cyclictest23784-21ssh00:19:522
3109899883,36cyclictest2735-21apt-get19:19:392
3109899883,34cyclictest30777-21latency_hist22:44:202
3109899883,34cyclictest30777-21latency_hist22:44:202
3109899883,34cyclictest28665-21cstates20:39:332
3109899883,31cyclictest25310-21/usr/sbin/munin20:29:422
3109899883,31cyclictest25310-21/usr/sbin/munin20:29:422
3109899883,31cyclictest21940-21latency_hist20:19:192
3109899883,30cyclictest23848-21ssh21:39:172
31098998830,8cyclictest0-21swapper/223:09:172
31098998830,8cyclictest0-21swapper/223:09:172
31098998827,9cyclictest0-21swapper/223:34:412
31098998827,12cyclictest0-21swapper/200:09:362
31098998827,12cyclictest0-21swapper/200:09:362
3109899882,35cyclictest8607-21find23:54:462
3109899874,31cyclictest32650-21uname00:34:512
3109899873,34cyclictest20797-21/usr/sbin/munin21:34:172
3109899873,34cyclictest20797-21/usr/sbin/munin21:34:172
3109899873,32cyclictest13289-21unixbench_multi19:49:532
3109899873,32cyclictest12131-21/usr/sbin/munin21:20:012
3109899873,31cyclictest979-21fschecks_time21:54:392
3109899873,30cyclictest31107-21tune2fs22:44:372
3109899873,26cyclictest19749-21sed22:24:532
3109899872,34cyclictest4191-21find21:04:262
3109899872,27cyclictest23818-21cstates20:24:332
3109899872,27cyclictest23818-21cstates20:24:332
31094998766,15cyclictest17118-21ntp_kernel_pll_23:14:441
31082998772,9cyclictest8936-21sh22:08:010
31082998772,9cyclictest8936-21sh22:08:010
301132876,56sleep130144-21/usr/sbin/munin19:04:531
3109899863,37cyclictest31072-21cyclictest19:09:202
3109899863,35cyclictest19793-21/usr/sbin/munin00:15:132
3109899863,33cyclictest22901-21cat20:20:132
3109899862,32cyclictest4637-21timerandwakeup21:59:542
3109899862,30cyclictest31508-21fschecks_time19:09:412
3109899862,30cyclictest31508-21fschecks_time19:09:412
3109499864,57cyclictest31-21ksoftirqd/119:44:281
67082850,10sleep30-21swapper/322:04:293
67082850,10sleep30-21swapper/322:04:293
3109899853,35cyclictest15717-21/usr/sbin/munin22:19:402
3109899853,34cyclictest17966-21ssh23:15:492
3109899853,32cyclictest15107-21/usr/sbin/munin21:24:532
3109899852,28cyclictest7704-21latency_hist23:54:182
31094998566,10cyclictest0-21swapper/120:39:511
31094998554,24cyclictest8054-21kworker/u19:0+flush-179:021:19:421
31082998566,9cyclictest0-21swapper/019:24:470
3109899843,28cyclictest30415-21fschecks_time20:44:392
3109899842,26cyclictest30405-21fschecks_time21:49:402
31094998355,9cyclictest181rcu_preempt20:09:401
31094998264,9cyclictest0-21swapper/121:49:411
31094998263,9cyclictest0-21swapper/120:44:521
31094998263,10cyclictest0-21swapper/119:19:191
31094998263,10cyclictest0-21swapper/119:19:191
31082998262,8cyclictest181rcu_preempt20:09:520
31082998260,16cyclictest4268-21ntp_kernel_err21:59:450
31094998163,8cyclictest0-21swapper/122:59:161
31082998162,9cyclictest0-21swapper/022:39:150
31082998162,13cyclictest15534-21latency_hist19:59:190
31082998152,10cyclictest22082-21ssh23:23:590
31094998063,11cyclictest5552-21diskmemload00:04:251
31094998063,11cyclictest5552-21diskmemload00:04:251
31082998046,28cyclictest0-21swapper/022:44:420
31094997963,11cyclictest343-21runrttasks21:24:471
31094997959,7cyclictest181rcu_preempt00:15:131
31094997959,15cyclictest7002-21fschecks_time22:04:411
31094997959,15cyclictest7002-21fschecks_time22:04:411
31094997958,8cyclictest181rcu_preempt23:19:261
31094997951,8cyclictest181rcu_preempt21:14:161
31082997964,9cyclictest24142-21rm21:39:290
31082997964,9cyclictest24142-21rm21:39:290
31094997864,7cyclictest0-21swapper/123:39:421
31094997864,7cyclictest0-21swapper/123:39:421
31094997861,10cyclictest0-21swapper/100:29:531
31094997860,13cyclictest0-21swapper/120:24:501
31094997860,13cyclictest0-21swapper/120:24:501
31094997850,7cyclictest181rcu_preempt20:34:421
31082997863,8cyclictest0-21swapper/000:29:490
31082997846,8cyclictest181rcu_preempt21:24:430
31082997846,7cyclictest181rcu_preempt23:52:500
31094997761,9cyclictest0-21swapper/123:59:541
31094997759,9cyclictest0-21swapper/100:34:441
31094997758,9cyclictest0-21swapper/119:59:201
31094997756,10cyclictest0-21swapper/119:54:191
31082997764,9cyclictest8040-21kworker/u19:0+flush-179:000:24:350
31082997761,7cyclictest0-21swapper/021:49:430
31082997760,12cyclictest31072-21cyclictest22:52:150
31082997760,12cyclictest31072-21cyclictest22:52:150
31082997758,9cyclictest0-21swapper/000:34:500
31082997757,13cyclictest4016-21latency_hist21:04:180
31082997745,7cyclictest181rcu_preempt22:28:380
31082997745,7cyclictest181rcu_preempt22:28:380
31082997745,10cyclictest22521-21ssh21:35:450
31082997730,21cyclictest181rcu_preempt21:14:530
31094997661,8cyclictest0-21swapper/120:54:181
31094997661,10cyclictest19191-21unixbench_singl21:29:541
31094997661,10cyclictest19191-21unixbench_singl21:29:541
31094997658,13cyclictest5552-21diskmemload23:54:021
31094997657,13cyclictest26652-21sh00:25:031
31094997655,8cyclictest181rcu_preempt19:29:391
31094997643,10cyclictest181rcu_preempt20:29:511
31094997643,10cyclictest181rcu_preempt20:29:511
31082997659,8cyclictest0-21swapper/020:04:190
31082997659,8cyclictest0-21swapper/020:04:190
31082997658,8cyclictest0-21swapper/023:59:580
31082997657,9cyclictest0-21swapper/000:09:510
31082997657,9cyclictest0-21swapper/000:09:510
31094997546,8cyclictest181rcu_preempt23:04:521
31094997546,8cyclictest181rcu_preempt23:04:521
31082997560,7cyclictest0-21swapper/020:24:160
31082997558,11cyclictest7831-21ssh22:59:580
31082997557,12cyclictest13289-21unixbench_multi19:49:530
31082997547,12cyclictest2378-21ssh23:44:410
31082997539,8cyclictest181rcu_preempt20:44:460
31082997539,13cyclictest0-21swapper/023:10:140
31082997537,11cyclictest181rcu_preempt19:19:450
31094997460,9cyclictest25487-21apt-get00:24:161
31094997458,11cyclictest10617-21ssh23:59:161
31094997458,10cyclictest26190-21sh23:29:501
31094997457,12cyclictest5552-21diskmemload21:35:561
31094997456,9cyclictest0-21swapper/119:09:411
31094997456,9cyclictest0-21swapper/119:09:411
31094997456,12cyclictest25773-21switchtime22:34:531
31094997454,14cyclictest1961-21sh22:49:431
31094997454,14cyclictest1961-21sh22:49:431
31094997450,9cyclictest181rcu_preempt22:24:501
31094997450,9cyclictest181rcu_preempt22:24:501
31094997447,8cyclictest181rcu_preempt23:34:241
31082997459,8cyclictest0-21swapper/000:15:070
31082997457,8cyclictest0-21swapper/023:04:270
31082997457,8cyclictest0-21swapper/023:04:270
31082997456,9cyclictest0-21swapper/023:34:330
31082997456,8cyclictest0-21swapper/000:07:330
31082997456,8cyclictest0-21swapper/000:07:330
31094997360,7cyclictest0-21swapper/120:04:391
31094997356,8cyclictest0-21swapper/120:54:511
31094997356,8cyclictest0-21swapper/120:54:511
31094997354,10cyclictest0-21swapper/120:59:381
31094997346,7cyclictest181rcu_preempt00:09:561
31094997346,7cyclictest181rcu_preempt00:09:561
31082997356,8cyclictest0-21swapper/019:09:400
31082997356,8cyclictest0-21swapper/019:09:400
31082997351,15cyclictest526-21rs:main0
31082997350,16cyclictest31072-21cyclictest21:23:370
31082997333,12cyclictest181rcu_preempt23:55:420
31094997252,15cyclictest662-21cron21:09:161
31094997251,8cyclictest181rcu_preempt22:59:471
31094997247,18cyclictest5552-21diskmemload22:34:191
31094997247,18cyclictest5552-21diskmemload22:34:191
31082997258,10cyclictest31072-21cyclictest23:32:580
31082997258,10cyclictest31072-21cyclictest00:19:470
31082997252,9cyclictest0-21swapper/021:54:330
31082997238,9cyclictest181rcu_preempt19:44:450
31082997231,9cyclictest181rcu_preempt19:14:330
31082997231,9cyclictest181rcu_preempt19:14:330
31094997157,7cyclictest0-21swapper/122:39:561
31094997157,7cyclictest0-21swapper/122:39:561
31082997155,10cyclictest17132-21ntp_offset23:14:440
31082997153,9cyclictest0-21swapper/019:34:490
31082997153,8cyclictest0-21swapper/021:33:000
31082997153,8cyclictest0-21swapper/021:33:000
31082997152,7cyclictest181rcu_preempt22:43:090
31082997152,7cyclictest181rcu_preempt22:43:090
31094997052,11cyclictest343-21runrttasks19:34:421
31094997050,10cyclictest0-21swapper/123:44:441
31094997050,10cyclictest0-21swapper/119:24:191
31082997055,10cyclictest0-21swapper/022:20:460
31082997053,12cyclictest5552-21diskmemload21:47:050
31082997048,15cyclictest25066-21cut20:29:170
31082997048,15cyclictest25066-21cut20:29:170
31082997039,11cyclictest181rcu_preempt21:13:030
31094996954,9cyclictest10202-21sh22:09:411
31094996943,18cyclictest11071-21sh21:17:561
31094996939,7cyclictest181rcu_preempt22:14:331
31082996954,8cyclictest0-21swapper/020:49:540
31082996937,10cyclictest181rcu_preempt23:26:150
31082996915,40cyclictest7515-21apt-key22:59:200
31094996853,8cyclictest0-21swapper/122:04:181
31094996852,9cyclictest0-21swapper/123:14:171
3108299686,35cyclictest15-21ksoftirqd/022:14:310
31082996847,15cyclictest662-21cron22:34:160
31082996847,15cyclictest662-21cron22:34:160
31082996843,20cyclictest0-21swapper/019:29:280
3108299683,36cyclictest10037-21/usr/sbin/munin22:09:380
31082996831,9cyclictest181rcu_preempt19:39:390
31082996831,9cyclictest181rcu_preempt19:39:390
31094996744,10cyclictest181rcu_preempt20:19:351
31094996717,12cyclictest181rcu_preempt21:59:161
31082996752,9cyclictest0-21swapper/020:04:510
31094996651,8cyclictest0-21swapper/119:39:541
31094996651,8cyclictest0-21swapper/119:39:541
31094996647,14cyclictest5552-21diskmemload22:19:561
31094996645,11cyclictest0-21swapper/120:04:201
31094996645,11cyclictest0-21swapper/120:04:201
31094996637,6cyclictest181rcu_preempt19:24:331
31082996643,8cyclictest181rcu_preempt20:29:460
31082996643,8cyclictest181rcu_preempt20:29:460
31094996546,10cyclictest0-21swapper/120:14:541
31094996527,19cyclictest31072-21cyclictest21:48:061
31082996549,11cyclictest0-21swapper/020:59:160
31082996549,11cyclictest0-21swapper/020:59:160
31082996530,29cyclictest0-21swapper/023:39:390
31082996530,29cyclictest0-21swapper/023:39:390
3108299643,45cyclictest29537-21grep20:39:520
31082996340,7cyclictest181rcu_preempt20:15:480
15512630,9sleep30-21swapper/321:54:523
3108299624,42cyclictest4434-21fschecks_time21:04:370
3110499594,34cyclictest7607-21chrt22:59:233
3110499593,40cyclictest521-21taskset21:54:193
3110499585,42cyclictest21875-21taskset21:34:503
3110499583,43cyclictest0-21swapper/323:59:193
3110499564,40cyclictest3509-21taskset22:52:383
3110499564,40cyclictest3509-21taskset22:52:383
3110499563,43cyclictest16358-21taskset19:59:523
3110499563,43cyclictest16358-21taskset19:59:523
3110499563,42cyclictest7130-21taskset21:10:193
31104995633,16cyclictest451rcuc/321:43:383
31104995633,16cyclictest451rcuc/321:43:383
3110499562,40cyclictest13239-21taskset23:09:133
3110499562,40cyclictest13239-21taskset23:09:133
3110499554,36cyclictest31944-21taskset20:49:383
3110499553,37cyclictest2886-21chrt23:44:523
3110499553,35cyclictest27837-21chrt22:39:203
3110499553,35cyclictest27837-21chrt22:39:203
3110499552,42cyclictest1056-21taskset00:36:083
3110499544,37cyclictest1637-21chrt23:44:153
3110499544,37cyclictest1637-21chrt23:44:153
3110499543,40cyclictest11264-21taskset23:59:443
3110499543,34cyclictest25366-21taskset23:29:193
3110499542,38cyclictest30178-21taskset00:30:083
3110499534,34cyclictest17894-21chrt21:29:183
3110499533,39cyclictest17760-21taskset22:22:293
3110499533,37cyclictest6678-21chrt19:29:513
3110499533,32cyclictest5811-21chrt19:29:193
3110499526,30cyclictest0-21swapper/321:19:383
3110499524,38cyclictest24513-21chrt22:34:053
3110499524,38cyclictest24513-21chrt22:34:053
3110499523,37cyclictest7949-21chrt19:34:413
3110499523,37cyclictest10759-21chrt21:17:043
3110499523,36cyclictest26846-21chrt20:34:193
3110499523,36cyclictest26846-21chrt20:34:193
3110499523,34cyclictest26445-21chrt22:36:083
3110499522,39cyclictest15361-21taskset22:19:123
3110499522,38cyclictest3946-21taskset21:04:153
3110499522,37cyclictest14142-21chrt00:04:383
3110499522,37cyclictest14142-21chrt00:04:383
3110499516,34cyclictest7625-21taskset23:54:163
3110499513,39cyclictest9880-21taskset19:39:523
3110499513,39cyclictest9880-21taskset19:39:523
3110499513,39cyclictest5627-21taskset22:02:103
3110499513,38cyclictest14782-21taskset19:54:503
3110499513,37cyclictest7311-21taskset22:59:183
3110499513,35cyclictest17461-21taskset20:04:383
3110499513,32cyclictest17254-21chrt23:14:483
3110499512,37cyclictest21253-21chrt22:28:533
3110499512,37cyclictest21253-21chrt22:28:533
3110499504,34cyclictest32513-21chrt22:46:253
3110499503,35cyclictest28032-21taskset00:27:293
3110499502,36cyclictest14834-21chrt23:10:183
3110499502,34cyclictest20348-21taskset21:33:033
3110499502,34cyclictest20348-21taskset21:33:033
3110499494,32cyclictest11145-21chrt19:44:423
3110499493,35cyclictest19364-21chrt00:13:543
3110499493,35cyclictest19364-21chrt00:13:543
3110499493,34cyclictest29044-21taskset21:47:103
3110499493,31cyclictest20273-21chrt20:14:183
3110499493,30cyclictest2843-21chrt19:19:513
3110499492,33cyclictest12953-21taskset19:49:483
3110499483,33cyclictest21658-21chrt23:22:473
3110499483,31cyclictest27958-21chrt23:33:493
3110499483,31cyclictest10853-21chrt22:10:003
3110499482,32cyclictest28773-21chrt23:34:383
3110499473,31cyclictest23670-21taskset00:19:503
3110499472,32cyclictest21532-21chrt00:16:383
3110499464,31cyclictest24519-21taskset20:24:513
3110499464,31cyclictest24519-21taskset20:24:513
3110499463,34cyclictest30891-21taskset20:44:523
3110499443,32cyclictest23293-21taskset20:22:243
3110499443,29cyclictest30078-21chrt20:44:193
31104994429,9cyclictest451rcuc/319:14:213
31104994429,9cyclictest451rcuc/319:14:213
3110499433,30cyclictest20814-21taskset20:14:433
3110499433,28cyclictest4173-21taskset21:04:203
302042439,20sleep30-21swapper/319:04:593
3110499423,30cyclictest32115-21chrt19:09:553
3110499423,30cyclictest32115-21chrt19:09:553
3110499423,26cyclictest924-21taskset20:54:233
3110499423,26cyclictest924-21taskset20:54:233
3110499413,28cyclictest28004-21taskset20:35:573
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional