You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-14 - 14:16
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Tue Jul 14, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31306213513,84sleep031339-21munin-run18:57:050
31306213513,84sleep031339-21munin-run18:57:050
314729911444,13cyclictest0-21swapper/120:52:261
305412999,62sleep230559-21pluginstate18:52:472
305412999,62sleep230559-21pluginstate18:52:472
272942980,7sleep10-21swapper/123:19:451
3147299978,64cyclictest8229-21apt-get23:42:211
31472999327,25cyclictest0-21swapper/120:22:071
31467999370,17cyclictest277-21dbus-daemon20:57:070
31467999278,8cyclictest0-21swapper/023:42:070
31467999278,8cyclictest0-21swapper/023:42:070
3146799913,54cyclictest15-21ksoftirqd/023:32:240
31472999078,8cyclictest0-21swapper/122:56:281
31472999078,8cyclictest0-21swapper/122:56:281
31467998870,12cyclictest277-21dbus-daemon21:22:050
31479998772,9cyclictest16153-21sh23:02:032
31479998663,11cyclictest0-21swapper/221:12:272
31479998657,8cyclictest181rcu_preempt23:13:312
31467998672,7cyclictest0-21swapper/021:47:240
31467998667,9cyclictest0-21swapper/022:22:080
31479998568,11cyclictest11085-21ntpd22:33:222
31479998564,9cyclictest0-21swapper/222:02:082
31479998557,7cyclictest181rcu_preempt00:03:052
31467998572,7cyclictest0-21swapper/019:22:220
31479998465,9cyclictest0-21swapper/222:17:422
31479998464,10cyclictest0-21swapper/223:08:412
31479998464,10cyclictest0-21swapper/223:08:412
31479998464,10cyclictest0-21swapper/220:52:062
31479998452,10cyclictest181rcu_preempt20:59:562
31467998469,9cyclictest14082-21vmstat22:02:450
31467998465,9cyclictest0-21swapper/019:27:280
31479998368,9cyclictest23812-21kworker/u17:3-xprtiod21:52:072
31479998363,10cyclictest0-21swapper/220:22:082
31472998335,13cyclictest181rcu_preempt21:52:131
31472998335,13cyclictest181rcu_preempt21:52:131
31467998359,17cyclictest31454-21cyclictest21:52:470
31467998359,17cyclictest31454-21cyclictest21:52:470
3146799834,44cyclictest15-21ksoftirqd/022:58:130
31479998266,9cyclictest0-21swapper/219:27:062
31479998265,10cyclictest27735-21kworker/u17:0-xprtiod23:42:052
31479998265,10cyclictest27735-21kworker/u17:0-xprtiod23:42:052
31479998262,12cyclictest298-21in:imuxsock19:48:042
31479998261,10cyclictest0-21swapper/219:02:072
31479998261,10cyclictest0-21swapper/219:02:072
31472998255,15cyclictest0-21swapper/123:47:451
31472998255,15cyclictest0-21swapper/123:47:451
31472998254,19cyclictest8447-21xargs19:22:401
31479998163,9cyclictest0-21swapper/220:27:072
31479998161,9cyclictest0-21swapper/221:28:202
3147999812,48cyclictest39-21ksoftirqd/222:49:312
31472998164,11cyclictest23812-21kworker/u17:3+events_unbound21:42:041
31472998164,11cyclictest23812-21kworker/u17:3+events_unbound21:42:041
31472998157,19cyclictest6865-21threads19:18:021
31467998162,13cyclictest2374-21systemd-journal21:10:080
31467998162,13cyclictest2374-21systemd-journal21:10:080
99462808,8sleep10-21swapper/122:51:271
31479998069,7cyclictest0-21swapper/220:07:242
31479998063,11cyclictest5928-21diskmemload21:46:132
31479998060,10cyclictest0-21swapper/220:52:382
31472998057,10cyclictest31-21ksoftirqd/121:16:561
31472998046,10cyclictest181rcu_preempt22:26:311
31472998045,13cyclictest5928-21diskmemload21:07:311
31472998045,13cyclictest5928-21diskmemload21:07:311
31467998061,9cyclictest0-21swapper/019:32:380
31467998027,22cyclictest15-21ksoftirqd/019:12:260
31479997965,8cyclictest0-21swapper/219:07:332
31479997961,9cyclictest0-21swapper/219:57:072
31479997959,9cyclictest0-21swapper/222:52:262
31479997959,9cyclictest0-21swapper/222:52:262
3147999793,48cyclictest39-21ksoftirqd/220:43:462
31472997956,16cyclictest15591-21sh23:53:261
31472997945,20cyclictest352-21runrttasks23:23:331
3147299792,40cyclictest31-21ksoftirqd/122:31:341
31467997962,8cyclictest0-21swapper/021:43:310
31467997960,9cyclictest0-21swapper/022:22:330
31467997958,15cyclictest19828-21threads22:12:410
31467997958,10cyclictest0-21swapper/021:27:380
31467997947,12cyclictest181rcu_preempt22:42:520
31467997946,10cyclictest6483-21ssh20:57:250
3146799792,50cyclictest15-21ksoftirqd/023:49:590
3146799792,50cyclictest15-21ksoftirqd/023:49:590
31479997865,7cyclictest0-21swapper/220:32:072
31479997864,8cyclictest0-21swapper/221:37:342
31479997864,8cyclictest0-21swapper/221:37:342
31479997863,8cyclictest0-21swapper/223:52:452
31479997855,16cyclictest27696-21latency21:32:312
31479997851,8cyclictest181rcu_preempt23:02:412
31479997848,10cyclictest181rcu_preempt00:07:202
31472997858,14cyclictest297-21systemd-logind20:37:051
31472997858,14cyclictest297-21systemd-logind20:37:051
31472997844,18cyclictest31-21ksoftirqd/119:02:081
31472997839,8cyclictest181rcu_preempt00:17:061
3146799787,41cyclictest15-21ksoftirqd/021:57:410
31467997860,11cyclictest16928-21apt-get23:57:040
31467997857,16cyclictest27938-21/usr/sbin/munin22:27:340
31467997855,17cyclictest1730-21cut22:37:260
3146799784,44cyclictest15-21ksoftirqd/020:25:560
31467997844,10cyclictest31924-21fw_packets18:57:270
31467997844,10cyclictest31924-21fw_packets18:57:270
3146799783,43cyclictest15-21ksoftirqd/019:09:150
3146799782,54cyclictest15-21ksoftirqd/000:12:590
31479997762,8cyclictest0-21swapper/223:17:402
31479997762,11cyclictest31454-21cyclictest00:17:032
31479997761,12cyclictest31454-21cyclictest21:02:172
31479997758,9cyclictest0-21swapper/222:22:262
31479997758,13cyclictest2374-21systemd-journal23:37:042
31467997763,8cyclictest0-21swapper/023:57:590
31467997762,8cyclictest0-21swapper/000:02:140
31467997758,9cyclictest0-21swapper/021:12:300
3146799775,45cyclictest15-21ksoftirqd/020:02:420
3146799775,45cyclictest15-21ksoftirqd/020:02:420
31467997746,21cyclictest277-21dbus-daemon19:52:050
3146799773,47cyclictest15-21ksoftirqd/023:24:200
3146799771,49cyclictest15-21ksoftirqd/000:09:050
31479997662,10cyclictest31454-21cyclictest23:23:472
31479997658,12cyclictest2588-21sendmail19:07:042
31479997657,13cyclictest13693-21mailstats22:02:392
31479997656,14cyclictest7392-21latency_hist22:47:082
31479997655,15cyclictest2374-21systemd-journal19:45:162
31479997655,15cyclictest2374-21systemd-journal19:45:162
31472997659,10cyclictest24644-21kworker/u19:2-nfsiod21:24:441
31467997660,9cyclictest0-21swapper/000:17:260
31467997660,10cyclictest9813-21rm21:02:360
31467997659,12cyclictest12532-21ssh22:55:190
31467997659,12cyclictest12532-21ssh22:55:190
31467997655,15cyclictest31454-21cyclictest21:32:340
31479997544,8cyclictest181rcu_preempt21:23:382
31472997553,13cyclictest18400-21/usr/sbin/munin23:58:111
3147299753,28cyclictest13480-21ssh22:02:321
31467997554,11cyclictest0-21swapper/022:07:150
31467997553,13cyclictest0-21swapper/020:32:410
31467997553,13cyclictest0-21swapper/020:32:410
31467997553,12cyclictest0-21swapper/022:47:420
3146799753,38cyclictest6086-21apt-get19:17:180
3146799752,41cyclictest15-21ksoftirqd/020:47:340
31467997510,39cyclictest15-21ksoftirqd/021:23:590
31479997463,6cyclictest0-21swapper/220:37:352
31479997463,6cyclictest0-21swapper/220:37:352
31479997460,7cyclictest0-21swapper/219:32:402
31479997455,10cyclictest0-21swapper/200:02:082
31479997444,8cyclictest181rcu_preempt00:25:162
31479997444,8cyclictest181rcu_preempt00:25:162
31479997438,9cyclictest181rcu_preempt20:12:312
31479997434,15cyclictest181rcu_preempt20:35:252
31479997434,15cyclictest181rcu_preempt20:35:252
31479997430,8cyclictest181rcu_preempt21:55:522
31479997430,8cyclictest181rcu_preempt21:55:522
3147299743,42cyclictest0-21swapper/123:33:251
31472997427,12cyclictest0-21swapper/119:09:551
3147299742,19cyclictest1021-21ssh00:22:401
3147299742,19cyclictest1021-21ssh00:22:401
31467997455,9cyclictest0-21swapper/020:27:250
31467997441,26cyclictest0-21swapper/023:27:300
3146799743,45cyclictest15-21ksoftirqd/023:16:420
3146799743,41cyclictest15-21ksoftirqd/020:13:490
3146799743,26cyclictest396-21/usr/sbin/munin00:22:260
3146799743,26cyclictest396-21/usr/sbin/munin00:22:260
3146799742,39cyclictest15-21ksoftirqd/020:42:300
31467997413,32cyclictest15-21ksoftirqd/023:07:120
31467997413,32cyclictest15-21ksoftirqd/023:07:120
31479997357,8cyclictest0-21swapper/222:12:342
31479997356,10cyclictest0-21swapper/222:07:312
31479997345,8cyclictest181rcu_preempt21:07:172
31479997345,8cyclictest181rcu_preempt21:07:172
31479997332,8cyclictest181rcu_preempt23:42:262
31479997323,8cyclictest0-21swapper/223:47:462
31479997323,8cyclictest0-21swapper/223:47:462
31472997359,9cyclictest16794-21kworker/u18:0+rpciod21:44:551
31467997359,7cyclictest0-21swapper/022:32:250
31467997356,10cyclictest642-21apt-key21:42:070
31467997356,10cyclictest642-21apt-key21:42:070
3146799733,47cyclictest15-21ksoftirqd/019:37:160
3146799733,36cyclictest15-21ksoftirqd/023:19:540
3146799732,42cyclictest15-21ksoftirqd/019:57:070
3146799732,41cyclictest15-21ksoftirqd/023:46:330
31479997257,8cyclictest0-21swapper/219:17:082
31479997241,8cyclictest181rcu_preempt21:22:082
31479997222,22cyclictest6652-21ntpq19:17:582
31472997255,11cyclictest352-21runrttasks00:04:171
31472997239,25cyclictest31-21ksoftirqd/121:32:431
31472997237,16cyclictest181rcu_preempt20:44:471
3147299722,44cyclictest5550-21sed22:42:431
3147299722,43cyclictest352-21runrttasks22:02:071
31467997258,10cyclictest31454-21cyclictest19:06:570
31467997253,13cyclictest27167-21latency_hist20:22:070
3146799722,42cyclictest15-21ksoftirqd/020:07:150
278442720,13sleep1291rcuc/120:22:321
31467997158,7cyclictest0-21swapper/019:42:170
31467997158,7cyclictest0-21swapper/019:42:170
3146799713,42cyclictest15-21ksoftirqd/023:03:140
3146799713,42cyclictest15-21ksoftirqd/020:37:400
3146799713,42cyclictest15-21ksoftirqd/020:37:400
301962718,35sleep130252-21/usr/sbin/munin18:52:331
301962718,35sleep130252-21/usr/sbin/munin18:52:331
31479997052,12cyclictest2129-21open_files22:37:372
31479997050,9cyclictest0-21swapper/220:02:272
31479997050,9cyclictest0-21swapper/220:02:272
31472997050,13cyclictest11687-21threads19:32:401
31472997039,10cyclictest181rcu_preempt20:14:531
3147299702,31cyclictest32098-21netstat18:57:311
3147299702,31cyclictest32098-21netstat18:57:311
31467997018,29cyclictest15-21ksoftirqd/019:57:270
31479996950,9cyclictest0-21swapper/219:57:292
31479996949,10cyclictest0-21swapper/219:37:392
3147999694,27cyclictest29837-21ssh00:17:232
31472996955,10cyclictest31454-21cyclictest22:57:291
31472996950,12cyclictest1898-21latency_hist23:32:081
31472996948,15cyclictest5928-21diskmemload00:12:051
3147299693,33cyclictest24184-21ssh21:27:101
3147299693,33cyclictest21683-21ssh22:17:061
31472996930,9cyclictest181rcu_preempt22:37:541
31472996930,7cyclictest181rcu_preempt20:57:411
3147299692,44cyclictest29889-21grep20:27:421
31479996839,9cyclictest181rcu_preempt23:27:592
31472996840,20cyclictest23253-21threads20:07:441
31472996829,8cyclictest181rcu_preempt20:47:191
3147299682,29cyclictest22271-21wc20:07:071
3147299682,29cyclictest22271-21wc20:07:071
31479996751,9cyclictest0-21swapper/222:27:422
31479996747,9cyclictest0-21swapper/219:27:422
31472996750,12cyclictest31454-21cyclictest19:57:401
31472996746,15cyclictest29626-21/usr/sbin/munin00:17:321
31472996740,8cyclictest181rcu_preempt23:12:241
31472996737,14cyclictest23219-21kworker/u18:1+rpciod22:21:171
3147299673,42cyclictest16986-21head19:47:481
31472996717,16cyclictest181rcu_preempt19:27:391
3147299664,48cyclictest7781-21apt-key23:42:061
3147299664,48cyclictest7781-21apt-key23:42:061
31472996642,11cyclictest181rcu_preempt20:37:161
31472996642,11cyclictest181rcu_preempt20:37:161
31472996637,8cyclictest181rcu_preempt19:52:321
31472996628,12cyclictest181rcu_preempt19:43:191
31472996628,12cyclictest181rcu_preempt19:43:191
31472996628,10cyclictest181rcu_preempt22:10:061
31472996620,7cyclictest181rcu_preempt21:07:071
31472996552,9cyclictest31-21ksoftirqd/121:49:351
31472996545,14cyclictest5928-21diskmemload21:17:251
3147299645,47cyclictest19497-21ssh23:07:161
3147299645,47cyclictest19497-21ssh23:07:161
31472996337,12cyclictest181rcu_preempt19:14:331
3147299632,31cyclictest5928-21diskmemload22:32:431
31472996240,16cyclictest0-21swapper/123:03:161
3147299623,32cyclictest12525-21/usr/sbin/munin19:37:391
3028726224,22sleep30-21swapper/318:52:353
3028726224,22sleep30-21swapper/318:52:353
31486995920,29cyclictest4265-21chrt21:47:263
3148699544,39cyclictest18339-21taskset19:52:413
3148699544,37cyclictest23389-21taskset00:07:073
3148699532,41cyclictest9183-21taskset21:55:453
3148699532,41cyclictest9183-21taskset21:55:453
3148699532,41cyclictest3501-21taskset21:46:413
3148699532,41cyclictest14321-21taskset21:10:243
3148699532,41cyclictest14321-21taskset21:10:243
3148699532,39cyclictest9886-21taskset19:27:373
3148699523,39cyclictest17277-21chrt21:15:183
3148699523,39cyclictest10654-21chrt22:52:223
3148699523,39cyclictest10654-21chrt22:52:223
3148699514,37cyclictest23877-21taskset23:13:303
3148699513,37cyclictest1865-21chrt23:32:073
3148699513,36cyclictest28915-21chrt20:27:083
3148699513,35cyclictest32564-21chrt00:22:083
3148699503,34cyclictest8136-21chrt23:42:093
3148699502,38cyclictest3438-21taskset22:40:053
3148699502,37cyclictest20349-21chrt22:13:293
3148699502,37cyclictest17893-21chrt23:57:593
3148699494,35cyclictest29496-21taskset20:27:373
3148699493,37cyclictest20235-21chrt21:20:133
3148699493,36cyclictest8001-21taskset22:47:313
3148699493,36cyclictest23059-21taskset21:24:473
3148699493,35cyclictest16532-21chrt19:47:393
3148699493,35cyclictest16067-21taskset22:07:213
3148699492,37cyclictest27324-21chrt21:32:213
3148699492,35cyclictest25896-21chrt21:29:173
3148699492,35cyclictest16990-21chrt23:02:323
3148699492,34cyclictest4673-21taskset22:42:243
3148699483,36cyclictest14824-21taskset22:04:503
3148699483,35cyclictest25170-21taskset22:22:253
3148699483,35cyclictest10299-21chrt21:57:283
3148699483,34cyclictest7714-21chrt23:42:053
3148699483,34cyclictest7714-21chrt23:42:053
3148699483,34cyclictest30730-21chrt20:32:273
3148699483,34cyclictest30730-21chrt20:32:273
3148699483,34cyclictest22480-21chrt20:07:153
3148699483,33cyclictest14690-21taskset19:43:253
3148699483,33cyclictest14690-21taskset19:43:253
3148699483,33cyclictest1233-21taskset20:42:073
3148699483,33cyclictest1233-21taskset20:42:073
3148699483,31cyclictest8564-21taskset21:01:073
3148699482,36cyclictest15494-21taskset23:53:083
3148699482,35cyclictest8132-21chrt19:22:313
3148699482,35cyclictest11003-21chrt21:04:323
3148699482,34cyclictest3841-21chrt23:34:433
3148699482,34cyclictest23139-21chrt00:07:033
3148699474,34cyclictest26010-21chrt23:17:323
3148699473,33cyclictest28847-21chrt00:15:433
3148699473,33cyclictest24574-21chrt22:21:573
3148699473,32cyclictest6024-21chrt19:17:083
3148699473,31cyclictest15962-21chrt23:01:343
3148699472,33cyclictest12701-21chrt19:37:263
3148699463,34cyclictest30315-21taskset22:31:133
3148699463,32cyclictest31015-21chrt23:26:483
3148699463,32cyclictest3014-21taskset00:27:073
3148699463,32cyclictest3014-21taskset00:27:073
3148699463,29cyclictest4493-21chrt20:52:083
3148699453,30cyclictest19637-21chrt23:07:223
3148699453,30cyclictest19637-21chrt23:07:223
31486994525,15cyclictest208612sleep320:02:223
31486994525,15cyclictest208612sleep320:02:223
3148699452,32cyclictest32461-21taskset22:33:493
3148699433,29cyclictest32558-21taskset21:41:133
3148699433,29cyclictest32558-21taskset21:41:133
3148699423,29cyclictest13215-21taskset23:50:083
3148699423,29cyclictest13215-21taskset23:50:083
3148699413,27cyclictest25036-21taskset20:12:423
3148699413,25cyclictest2772-21taskset19:07:083
3148699412,30cyclictest6094-21chrt19:17:203
31486994122,12cyclictest53942chrt20:52:403
3148699403,27cyclictest11189-21chrt19:32:273
3148699402,27cyclictest3319-21chrt19:07:323
3148699393,27cyclictest26244-21taskset20:17:363
3148699393,26cyclictest31974-21taskset18:57:273
3148699393,26cyclictest31974-21taskset18:57:273
3148699392,28cyclictest19399-21taskset19:57:233
3148699392,26cyclictest2078-21chrt20:42:403
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional