You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-13 - 10:28
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Mon Apr 13, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2373021578,127sleep123784-21tr18:59:021
2373021578,127sleep123784-21tr18:59:021
24620991093,44cyclictest15284-21fschecks_time23:18:392
246209910037,56cyclictest3486-21needreboot19:34:562
23325210011,58sleep223393-21fschecks_time18:58:462
23325210011,58sleep223393-21fschecks_time18:58:462
24620999951,41cyclictest13394-21kworker/u17:2-xprtiod23:28:452
24620999836,54cyclictest0-21swapper/223:54:502
24620999713,34cyclictest22399-21apt-config20:33:222
24620999641,7cyclictest0-21swapper/221:58:512
24620999635,54cyclictest352-21runrttasks21:58:212
2462099963,37cyclictest13475-21cat20:03:532
2462099963,37cyclictest13475-21cat20:03:532
24616999676,11cyclictest21169-21if_err_eth000:23:421
24620999536,11cyclictest0-21swapper/221:43:472
2462099953,39cyclictest10746-21latency_hist19:58:202
2462099946,40cyclictest6132-21taskset21:13:482
2462099943,35cyclictest12147-21fschecks_count21:23:402
24620999417,30cyclictest11692-21systemd-run00:08:182
24613999419,46cyclictest15-21ksoftirqd/023:33:320
24613999419,46cyclictest15-21ksoftirqd/023:33:320
24620999336,11cyclictest0-21swapper/200:08:502
2462099933,34cyclictest22862-21fschecks_time20:34:312
24620999326,11cyclictest0-21swapper/222:28:192
24620999240,46cyclictest0-21swapper/221:48:402
2462099923,35cyclictest989-21grep23:48:532
2462099923,35cyclictest989-21grep23:48:532
2462099923,35cyclictest16010-21df20:13:362
2462099923,34cyclictest29974-21latency_hist19:18:212
2462099922,37cyclictest30810-21fschecks_count20:58:392
2462099922,33cyclictest26884-21kworker/u17:1+xprtiod00:23:432
24616999270,15cyclictest24600-21cyclictest19:43:211
2462099913,32cyclictest1549-21grep19:29:342
2462099912,39cyclictest3540-21ps21:08:522
2462099912,39cyclictest3540-21ps21:08:522
2462099912,36cyclictest29702-21cat22:48:212
2462099912,36cyclictest29702-21cat22:48:212
24620999034,13cyclictest0-21swapper/222:58:422
2462099893,36cyclictest27005-21fschecks_count19:08:412
2462099893,32cyclictest5828-21latency_hist22:08:202
2462099893,32cyclictest5828-21latency_hist22:08:202
24620998929,11cyclictest0-21swapper/223:08:382
2462099892,40cyclictest14990-21timerandwakeup20:08:532
2462699882,37cyclictest23897-21taskset23:33:353
2462699882,37cyclictest23897-21taskset23:33:353
2462099883,35cyclictest29298-21latency_hist23:43:192
2462099883,35cyclictest18809-21latency_hist20:23:202
2462099883,35cyclictest12016-21/usr/sbin/munin23:13:512
2462099883,34cyclictest352-21runrttasks20:25:492
24616998862,8cyclictest31-21ksoftirqd/123:12:311
24620998735,11cyclictest0-21swapper/222:23:182
2462099873,35cyclictest6687-21ssh23:58:462
2462099873,29cyclictest7030-21timerwakeupswit19:43:552
2462099872,32cyclictest6688-21/usr/sbin/munin22:08:462
2462099872,32cyclictest6688-21/usr/sbin/munin22:08:462
2462099863,41cyclictest30582-21/usr/sbin/munin22:48:522
2462099863,36cyclictest29195-21fschecks_count20:53:392
24620998631,10cyclictest0-21swapper/200:13:412
2461399855,35cyclictest15-21ksoftirqd/019:03:310
2461399855,35cyclictest15-21ksoftirqd/019:03:310
267822840,15sleep0161ktimers/021:48:350
24616998451,10cyclictest181rcu_preempt22:43:211
2462099833,33cyclictest8749-21latency_hist23:08:202
24620998262,13cyclictest31978-21diskmemload21:38:362
24620998262,13cyclictest31978-21diskmemload21:38:362
24620998259,14cyclictest2952-21systemd-journal19:06:362
24620998259,14cyclictest2952-21systemd-journal19:06:362
2462099823,39cyclictest392-21/usr/sbin/munin21:03:462
24620998212,33cyclictest30701-21awk23:43:562
24620998212,33cyclictest30701-21awk23:43:562
24616998267,8cyclictest0-21swapper/123:03:211
24613998267,10cyclictest24600-21cyclictest23:30:210
40482810,4sleep1301ktimers/122:03:531
40482810,4sleep1301ktimers/122:03:531
24613998162,13cyclictest9240-21cstates21:19:020
24613998160,15cyclictest4139-21timerwakeupswit22:03:540
24613998160,15cyclictest4139-21timerwakeupswit22:03:540
2462099803,39cyclictest352-21runrttasks23:34:222
2462099803,39cyclictest352-21runrttasks23:34:222
24620998032,9cyclictest0-21swapper/200:18:482
24620998032,9cyclictest0-21swapper/200:18:482
24616998047,9cyclictest181rcu_preempt22:24:551
24613998055,18cyclictest13624-21ssh21:25:410
24613998040,12cyclictest181rcu_preempt21:13:330
2462099793,35cyclictest21544-21memory22:33:452
24616997959,14cyclictest14718-21latency_hist00:13:201
24616997954,7cyclictest181rcu_preempt23:56:341
24616997941,7cyclictest181rcu_preempt19:08:431
2462099783,45cyclictest26554-21modprobe22:43:172
2462099783,45cyclictest18346-21cut22:28:392
24616997857,15cyclictest18055-21/usr/sbin/munin22:28:451
24613997857,15cyclictest14668-21latency_hist22:23:190
24613997848,21cyclictest20413-21cron20:28:180
24620997751,9cyclictest181rcu_preempt00:30:142
24616997758,13cyclictest24500-21sh00:28:531
24616997757,15cyclictest11698-21sh00:08:201
24616997757,14cyclictest31849-21fschecks_count19:23:381
24616997654,16cyclictest27608-21fschecks_time20:48:401
24616997654,16cyclictest27608-21fschecks_time20:48:401
24616997634,31cyclictest14642-21modprobe23:18:171
24613997659,13cyclictest13100-21kworker/u18:0+events_unbound21:48:170
24613997659,10cyclictest7851-21kworker/u17:1+rpciod22:13:510
24613997658,12cyclictest13069-21rm00:08:560
301662753,7sleep00-21swapper/021:54:320
24616997560,9cyclictest16272-21ssh00:14:361
24616997553,15cyclictest15660-21sh21:28:541
24616997553,15cyclictest15660-21sh21:28:541
24616997547,7cyclictest181rcu_preempt21:54:511
24613997556,11cyclictest30858-21ssh23:44:190
24613997556,11cyclictest30858-21ssh23:44:190
2461399754,40cyclictest32439-21cstates21:03:350
24613997541,25cyclictest31409-21sh19:23:180
24613997458,10cyclictest31978-21diskmemload22:38:390
24613997457,12cyclictest19106-21kworker/u18:1-rpciod00:18:430
24613997457,12cyclictest19106-21kworker/u18:1-rpciod00:18:430
24613997454,15cyclictest25639-21kworker/u19:0+events_unbound19:18:180
2462099733,35cyclictest31978-21diskmemload23:23:272
24616997352,15cyclictest32236-21sh23:48:171
24616997352,15cyclictest32236-21sh23:48:171
24616997348,8cyclictest31-21ksoftirqd/119:54:441
24616997346,19cyclictest29266-21modprobe21:53:171
24616997335,9cyclictest181rcu_preempt22:43:471
24616997335,9cyclictest181rcu_preempt22:43:471
24613997357,9cyclictest14681-21apt-get22:23:320
24613997356,11cyclictest311-21ssh22:53:340
24613997355,12cyclictest277-21dbus-daemon22:13:190
24613997355,12cyclictest277-21dbus-daemon22:13:190
24613997351,15cyclictest25509-21idleruntime-cro20:43:180
2461399734,38cyclictest21261-21fschecks_count22:33:400
24620997251,15cyclictest15542-21processes21:28:522
24620997251,15cyclictest15542-21processes21:28:522
24620997248,17cyclictest2952-21systemd-journal20:53:192
24620997248,17cyclictest2952-21systemd-journal20:53:192
2462099722,33cyclictest31978-21diskmemload22:53:402
24616997252,14cyclictest18406-21ntp_kernel_pll_21:33:461
24616997250,16cyclictest16671-21sh23:20:421
24616997224,11cyclictest0-21swapper/121:38:391
24616997224,11cyclictest0-21swapper/121:38:391
24613997258,9cyclictest30119-21kworker/u17:0+rpciod21:58:390
24613997255,11cyclictest0-21swapper/020:53:210
24613997255,11cyclictest0-21swapper/020:53:210
24613997255,10cyclictest2415-21latency_hist19:33:220
24613997251,15cyclictest25767-21acpi20:43:300
24613997220,22cyclictest13102-21grep23:13:560
24620997152,14cyclictest9447-21kernelversion22:13:442
24616997152,13cyclictest8932-21latency_hist22:13:211
24616997152,13cyclictest8932-21latency_hist22:13:211
24616997152,13cyclictest32102-21latency_hist21:03:201
24616997150,15cyclictest2759-40mandb23:53:181
24616997150,15cyclictest2759-40mandb23:53:181
24613997153,13cyclictest4614-21sh22:59:460
24613997153,12cyclictest298-21in:imuxsock21:33:450
24613997144,8cyclictest181rcu_preempt22:45:250
24613997144,8cyclictest181rcu_preempt22:45:250
24616997055,9cyclictest2814-21ssh21:08:291
24616997055,9cyclictest2814-21ssh21:08:291
24616997055,8cyclictest0-21swapper/122:58:561
24616997050,14cyclictest32727-21/usr/sbin/munin22:53:371
24616997050,14cyclictest13376-21threads20:03:531
24616997050,14cyclictest13376-21threads20:03:531
24616997024,10cyclictest12175-21fschecks_count21:23:401
24613997053,11cyclictest30307-21ssh22:48:440
24613997050,14cyclictest32023-21ssh21:03:200
2461399704,52cyclictest24473-21latency_hist19:03:220
2461399704,52cyclictest24473-21latency_hist19:03:220
24620996945,14cyclictest12504-21latency_hist20:03:202
24620996945,14cyclictest12504-21latency_hist20:03:202
24616996952,10cyclictest2989-21latency_hist22:03:201
24616996947,9cyclictest31-21ksoftirqd/121:17:091
2461399696,37cyclictest15-21ksoftirqd/023:48:320
2461399696,37cyclictest15-21ksoftirqd/023:48:320
24613996955,8cyclictest18049-21rm23:23:350
24613996953,10cyclictest23688-21ssh00:28:280
24613996952,11cyclictest6439-21ssh23:58:410
24613996947,15cyclictest29497-21latency_hist23:43:200
92432680,6sleep30-21swapper/323:08:383
24620996855,9cyclictest24600-21cyclictest21:34:452
24620996828,8cyclictest181rcu_preempt21:19:192
24616996853,9cyclictest18096-21sh23:23:391
24616996850,13cyclictest12789-21fschecks_time22:19:521
24616996847,15cyclictest1-21systemd20:33:201
24616996844,9cyclictest181rcu_preempt22:50:501
24616996840,16cyclictest181rcu_preempt22:33:511
24613996847,14cyclictest300-21rs:main0
24613996841,20cyclictest28576-21kworker/u17:0+nfsiod21:09:390
24613996841,20cyclictest28576-21kworker/u17:0+nfsiod21:09:390
24613996828,10cyclictest181rcu_preempt20:38:210
233942686,14sleep30-21swapper/318:58:473
233942686,14sleep30-21swapper/318:58:473
2462099672,36cyclictest32434-21timerandwakeup19:23:522
24616996746,15cyclictest2557-21kworker/u19:2-rpciod23:28:561
24616996737,10cyclictest0-21swapper/119:07:541
24616996737,10cyclictest0-21swapper/119:07:541
24613996753,9cyclictest58-21kcompactd020:17:390
24613996749,8cyclictest0-21swapper/019:48:540
24613996749,12cyclictest181rcu_preempt20:28:470
2462699663,36cyclictest19830-21chrt21:36:343
24620996635,10cyclictest7985-21fschecks_time19:48:392
2462099663,44cyclictest352-21runrttasks20:44:132
2462099662,40cyclictest24672-21ntp_kernel_pll_20:38:462
24616996649,8cyclictest0-21swapper/119:43:461
24616996633,9cyclictest181rcu_preempt22:16:191
24613996650,8cyclictest0-21swapper/020:08:540
24613996650,11cyclictest16940-21ssh21:31:450
24613996650,11cyclictest16940-21ssh21:31:450
2461399664,47cyclictest352-21runrttasks19:13:200
24613996642,10cyclictest181rcu_preempt21:38:250
24613996642,10cyclictest181rcu_preempt21:38:250
24613996639,20cyclictest32240-21kworker/u17:2+nfsiod23:05:190
24613996636,21cyclictest11965-21kworker/u17:2+nfsiod00:15:520
2461399663,33cyclictest21753-21sed00:23:550
24620996529,12cyclictest181rcu_preempt19:18:342
24616996552,9cyclictest24600-21cyclictest20:24:261
24616996548,11cyclictest2952-21systemd-journal19:32:351
24616996540,14cyclictest0-21swapper/119:33:321
24616996538,9cyclictest181rcu_preempt20:38:201
2461699652,26cyclictest30520-21needreboot19:18:441
24613996552,9cyclictest58-21kcompactd019:46:130
24613996551,10cyclictest24600-21cyclictest23:19:550
24613996547,13cyclictest0-21swapper/019:58:210
2461399654,44cyclictest4397-21latency_hist19:38:200
2461399654,39cyclictest18297-21fschecks_count22:28:390
2462099642,40cyclictest5974-21cat19:43:202
24616996446,12cyclictest1-21systemd23:38:181
24616996446,12cyclictest1-21systemd23:38:181
24616996445,9cyclictest0-21swapper/120:13:351
24616996445,13cyclictest26381-21munin-node20:48:211
24616996436,8cyclictest181rcu_preempt21:44:591
24616996432,22cyclictest9128-21tr19:53:181
24616996431,7cyclictest181rcu_preempt19:13:531
2461399644,27cyclictest12819-21grep20:03:390
2461399644,27cyclictest12819-21grep20:03:390
24613996427,9cyclictest0-21swapper/023:08:430
2461699638,13cyclictest31978-21diskmemload00:18:551
2461699638,13cyclictest31978-21diskmemload00:18:551
24616996339,18cyclictest544-21cron20:38:211
24613996348,10cyclictest58-21kcompactd019:59:170
24613996348,10cyclictest58-21kcompactd019:59:170
24613996347,11cyclictest58-21kcompactd000:03:500
24613996341,15cyclictest559-21latency_hist19:28:220
24616996244,11cyclictest0-21swapper/121:03:341
24616996243,13cyclictest2952-21systemd-journal00:00:451
24616996243,13cyclictest10158-21unixbench_singl21:19:221
24613996241,14cyclictest0-21swapper/020:53:400
24613996227,27cyclictest17876-21latency20:18:430
24616996146,10cyclictest11620-21awk19:58:501
24616996146,10cyclictest11620-21awk19:58:501
24616996145,8cyclictest0-21swapper/120:18:541
24616996145,12cyclictest24600-21cyclictest23:42:111
24616996142,9cyclictest0-21swapper/120:58:211
2462699603,35cyclictest917-21kworker/3:1+events22:03:153
2462699602,37cyclictest917-21kworker/3:1+events21:52:363
24616996039,8cyclictest181rcu_preempt20:12:421
2461399603,42cyclictest4653-21cstates19:38:370
2462699582,32cyclictest917-21kworker/3:1+events00:18:383
2462699582,32cyclictest917-21kworker/3:1+events00:18:383
2462699572,39cyclictest917-21kworker/3:1+events22:43:423
2462699572,39cyclictest917-21kworker/3:1+events22:43:423
2462699572,36cyclictest22566-21chrt20:33:363
2462699553,33cyclictest19866-21chrt22:30:553
2462699553,32cyclictest0-21swapper/323:53:203
2462699553,32cyclictest0-21swapper/323:53:203
2462699543,36cyclictest30086-21chrt23:43:453
2462699543,36cyclictest30086-21chrt23:43:453
2462699543,35cyclictest2354-21taskset22:56:423
2462699543,35cyclictest17478-21taskset21:33:173
2462699543,35cyclictest17478-21taskset21:33:173
2462699543,35cyclictest14588-21taskset22:23:053
2462699543,33cyclictest22588-21taskset00:25:583
2462699543,33cyclictest17793-21taskset20:18:413
2462699542,39cyclictest3569-21taskset21:08:523
2462699542,39cyclictest3569-21taskset21:08:523
2462699542,37cyclictest7675-21chrt00:00:163
2462699542,36cyclictest917-21kworker/3:1+events23:41:223
2462699542,35cyclictest23503-21chrt22:37:573
2462699532,36cyclictest13190-21taskset21:24:253
2462699532,34cyclictest24270-21chrt00:28:463
2462699532,34cyclictest15955-21chrt20:13:353
2462699524,31cyclictest4272-21taskset22:03:553
2462699524,31cyclictest4272-21taskset22:03:553
2462699523,36cyclictest22910-21taskset23:31:493
2462699523,33cyclictest7591-21chrt21:15:173
2462699523,31cyclictest10874-21chrt22:16:073
2462699522,37cyclictest19163-21chrt23:24:273
2462699522,32cyclictest30899-21chrt21:55:073
2462699514,34cyclictest20180-21taskset20:25:263
2462699513,32cyclictest917-21kworker/3:1+events20:03:363
2462699513,32cyclictest917-21kworker/3:1+events20:03:363
2462699512,33cyclictest24210-21chrt20:38:343
24626995034,8cyclictest0-21swapper/323:54:393
2462699502,32cyclictest2320-21taskset21:07:473
2462699493,33cyclictest15307-21chrt22:23:543
2462699492,31cyclictest30727-21taskset20:58:353
2462699483,29cyclictest17737-21chrt00:18:203
2462699483,29cyclictest13974-21chrt00:11:313
2462699482,32cyclictest9264-21chrt21:19:023
2462699482,32cyclictest21536-21chrt20:28:543
2462699482,30cyclictest6388-21chrt23:03:393
2462699482,30cyclictest26318-21chrt19:08:213
2462699482,30cyclictest26318-21chrt19:08:213
2462699475,30cyclictest27386-21chrt20:48:303
2462699475,30cyclictest27386-21chrt20:48:303
2462699472,30cyclictest10812-21taskset19:58:203
2462699463,31cyclictest12827-21chrt23:13:533
2462699463,29cyclictest24142-21taskset22:38:383
2462699454,29cyclictest20937-21taskset21:38:363
2462699454,29cyclictest20937-21taskset21:38:363
2462699453,31cyclictest6759-21taskset22:08:503
2462699453,31cyclictest6759-21taskset22:08:503
2462699452,31cyclictest28235-21taskset19:13:193
2462699452,29cyclictest917-21kworker/3:1-events00:08:123
2462699443,31cyclictest8862-21chrt19:50:223
2462699443,30cyclictest31179-21chrt22:49:453
2462699442,30cyclictest23715-21chrt21:43:313
2462699433,29cyclictest15260-21taskset23:18:393
2462699433,27cyclictest2987-21taskset19:33:583
2462699432,29cyclictest4582-21taskset19:38:363
2462699432,20cyclictest0-21swapper/319:28:343
2462699423,28cyclictest6166-21chrt19:43:343
2462699422,29cyclictest11327-21chrt19:58:413
2462699422,29cyclictest11327-21chrt19:58:413
2462699413,26cyclictest887-21chrt19:28:213
2462699413,26cyclictest28472-21taskset19:13:313
2462699413,26cyclictest25939-21chrt20:43:403
2462699412,29cyclictest5270-21chrt23:01:403
2462699403,27cyclictest31446-21chrt19:23:193
2462699402,26cyclictest15081-21chrt20:08:533
2462699382,26cyclictest29172-21taskset20:53:373
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional