You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-27 - 16:31
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Mon Jul 27, 2026 12:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2318521390,5sleep017558-1kworker/0:2H+kblockd06:56:140
23139991246,111cyclictest10371-21apt-get11:56:250
231399912036,74cyclictest0-21swapper/010:06:130
23139991195,109cyclictest0-21swapper/011:11:300
23139991195,109cyclictest0-21swapper/011:11:300
231399911632,78cyclictest0-21swapper/009:12:130
231399911632,78cyclictest0-21swapper/009:12:130
2158921127,17sleep10-21swapper/106:52:001
231399911121,52cyclictest5553-21rm10:55:170
231399911121,52cyclictest5553-21rm10:55:170
231399910854,48cyclictest30601-21latency_hist08:56:120
23139991073,95cyclictest0-21swapper/009:46:220
231399910655,45cyclictest31334-21sendmail_mailqu11:36:450
231399910655,45cyclictest31334-21sendmail_mailqu11:36:450
231499910553,14cyclictest1389-21cut09:01:111
23139991053,63cyclictest26797-21cron07:06:090
231399910438,50cyclictest4775-21ssh09:06:260
23139991043,84cyclictest23428-21apt-get06:56:450
23139991032,66cyclictest8955-21rm11:00:470
231399910317,69cyclictest0-21swapper/011:06:480
231399910317,69cyclictest0-21swapper/011:06:480
23139991024,86cyclictest27627-21apt-get11:31:220
231399910237,57cyclictest0-21swapper/007:31:090
231399910220,52cyclictest0-21swapper/009:32:260
231399910218,55cyclictest12048-21sh11:06:030
23139991013,67cyclictest10119-21apt-config11:56:120
23139991009,77cyclictest4187-21ssh11:46:240
231399910040,55cyclictest23116-21cyclictest10:31:310
231399910017,66cyclictest4172-21ssh09:05:360
2313999984,61cyclictest22747-21grep08:31:340
23139999835,54cyclictest0-21swapper/012:01:410
23139999835,54cyclictest0-21swapper/012:01:410
2313999982,59cyclictest23805-21taskset12:17:450
23139999734,53cyclictest0-21swapper/008:41:360
23139999631,55cyclictest0-21swapper/010:01:120
23139999625,63cyclictest0-21swapper/010:21:470
23159999545,43cyclictest0-21swapper/209:26:402
2313999953,86cyclictest0-21swapper/011:26:230
23139999537,50cyclictest0-21swapper/009:16:360
23139999534,54cyclictest0-21swapper/010:41:110
2313999952,65cyclictest10232-21ssh10:07:580
2313999952,65cyclictest10232-21ssh10:07:580
2313999952,59cyclictest0-21swapper/007:16:290
2313999952,59cyclictest0-21swapper/007:16:290
23159999428,9cyclictest0-21swapper/207:46:202
2313999943,55cyclictest0-21swapper/007:36:350
23139999435,51cyclictest0-21swapper/009:51:470
23139999435,51cyclictest0-21swapper/009:51:470
23139999327,56cyclictest0-21swapper/008:01:130
2285329313,55sleep222869-21cron06:56:092
2315999924,37cyclictest24676-21ssh11:26:262
2315999923,33cyclictest7627-21latency_hist07:46:132
23159999225,25cyclictest12993-21seq09:19:272
23159999219,27cyclictest277-21dbus-daemon11:16:102
23159999219,27cyclictest277-21dbus-daemon11:16:102
23139999263,20cyclictest28644-21latency_hist07:11:120
23139999263,20cyclictest28644-21latency_hist07:11:120
23139999239,46cyclictest300-21rs:main0
2313999923,52cyclictest0-21swapper/010:41:390
23139999212,52cyclictest19972-21ntp_states12:11:380
23159999147,37cyclictest0-21swapper/210:06:092
2315999913,33cyclictest9178-21latency_hist11:01:112
23159999130,53cyclictest0-21swapper/211:26:122
23159999130,53cyclictest0-21swapper/211:26:122
2313999913,61cyclictest20918-21expr10:26:310
2313999913,61cyclictest20918-21expr10:26:310
23159999039,44cyclictest0-21swapper/210:41:112
2315999903,42cyclictest10390-21ssh11:56:242
23159999027,55cyclictest0-21swapper/207:56:402
23139999031,49cyclictest0-21swapper/007:31:380
2315999894,34cyclictest19691-21fschecks_time12:11:322
23159998941,40cyclictest0-21swapper/210:06:252
23159998941,40cyclictest0-21swapper/210:06:252
2315999893,38cyclictest1987-21switchtime11:41:472
2313999893,77cyclictest0-21swapper/011:16:210
23139998931,52cyclictest0-21swapper/010:21:120
23139998925,58cyclictest0-21swapper/009:42:080
2315999884,40cyclictest28692-21/usr/sbin/munin07:11:282
2315999884,40cyclictest28692-21/usr/sbin/munin07:11:282
23159998840,41cyclictest0-21swapper/208:21:442
23159998840,41cyclictest0-21swapper/208:21:442
23159998835,43cyclictest0-21swapper/210:56:122
23159998835,43cyclictest0-21swapper/210:56:122
23159998830,7cyclictest0-21swapper/209:21:422
23159998826,12cyclictest0-21swapper/208:31:352
2315999882,28cyclictest30517-21find10:41:462
2313999883,77cyclictest15671-21ssh09:23:350
2313999882,58cyclictest0-21swapper/008:21:200
2313999882,58cyclictest0-21swapper/008:21:200
2313999882,55cyclictest32325-21latency07:21:370
23159998772,8cyclictest0-21swapper/208:57:422
2315999874,34cyclictest24756-21seq09:38:092
23159998738,42cyclictest0-21swapper/209:01:362
23159998734,46cyclictest0-21swapper/207:06:462
23159998734,46cyclictest0-21swapper/207:06:462
2315999873,38cyclictest14410-21grep12:01:482
2315999873,38cyclictest14410-21grep12:01:482
2315999873,35cyclictest3506-21wc07:31:482
2315999873,33cyclictest3096-21latency_hist10:51:132
23159998727,8cyclictest0-21swapper/208:17:292
23159998727,53cyclictest0-21swapper/209:31:322
2315999872,34cyclictest29142-21/usr/sbin/munin09:46:292
2315999872,32cyclictest27901-21fschecks_time11:31:312
23139998737,43cyclictest0-21swapper/010:11:400
2313999873,55cyclictest16872-21cat12:06:350
23139998728,52cyclictest0-21swapper/008:16:120
2313999872,67cyclictest30606-21diskmemload09:27:280
23139998717,55cyclictest1950-21/usr/sbin/munin11:41:470
2315999864,34cyclictest23103-21runrttasks10:30:302
2315999864,34cyclictest23103-21runrttasks10:30:302
2315999863,31cyclictest26081-21ps07:01:452
2315999863,27cyclictest31277-21head07:16:472
2315999863,27cyclictest31277-21head07:16:472
23159998630,9cyclictest0-21swapper/212:06:232
2313999864,74cyclictest0-21swapper/010:46:200
2313999862,57cyclictest30606-21diskmemload09:00:160
2313999862,55cyclictest15410-21taskset08:08:570
2313999862,53cyclictest25363-21ssh12:21:230
2315999854,32cyclictest32400-21fschecks_count09:51:312
2315999854,32cyclictest32400-21fschecks_count09:51:312
2315999854,31cyclictest18775-21cstates11:16:272
23159998534,44cyclictest0-21swapper/210:21:252
2315999853,35cyclictest9820-21latency_hist11:56:122
2315999853,35cyclictest352-21runrttasks09:42:142
2315999853,26cyclictest352-21runrttasks09:57:422
2315999853,26cyclictest12867-21cut08:01:302
2315999853,26cyclictest12867-21cut08:01:302
2315999853,25cyclictest5039-21timerandwakeup11:46:442
23159998532,46cyclictest30606-21diskmemload09:06:342
2315999852,37cyclictest277-21dbus-daemon11:41:102
2315999852,37cyclictest277-21dbus-daemon11:41:102
2315999852,36cyclictest7925-21/usr/sbin/munin09:12:212
2315999852,36cyclictest7925-21/usr/sbin/munin09:12:212
2315999852,24cyclictest12040-21fschecks_time10:11:342
2313999854,51cyclictest23459-21fschecks_time09:36:300
23139998514,65cyclictest5235-21kworker/u17:3+xprtiod11:21:220
23139998514,65cyclictest5235-21kworker/u17:3+xprtiod11:21:220
2315999844,32cyclictest23661-21fschecks_count06:56:512
2315999843,29cyclictest12090-21modprobe11:06:082
23159998426,7cyclictest0-21swapper/207:26:122
2315999842,30cyclictest29532-21fschecks_time08:51:332
2315999842,30cyclictest28161-21latency_hist12:26:112
2315999842,28cyclictest29160-21sort08:51:122
2315999842,28cyclictest14488-21fschecks_time08:06:312
2313999842,72cyclictest9378-21/usr/sbin/munin07:51:460
23139998418,59cyclictest24685-21fschecks_time08:37:010
23139998418,59cyclictest24685-21fschecks_time08:37:010
2315999833,39cyclictest22699-21taskset12:16:322
2315999833,36cyclictest352-21runrttasks07:40:172
2315999833,35cyclictest24385-21/usr/sbin/munin08:36:202
2315999833,35cyclictest24385-21/usr/sbin/munin08:36:202
2315999833,31cyclictest26440-21latency_hist10:36:112
2315999833,31cyclictest25995-21/usr/sbin/munin08:41:312
2315999833,31cyclictest25023-21munin-node07:51:202
23159998330,8cyclictest0-21swapper/208:14:072
23139998331,46cyclictest0-21swapper/007:16:090
23139998331,46cyclictest0-21swapper/007:16:090
23165998214,34cyclictest0-21swapper/311:11:253
23165998214,34cyclictest0-21swapper/311:11:253
2315999824,27cyclictest1078-21/usr/sbin/munin07:26:262
23159998224,13cyclictest0-21swapper/211:06:382
23159998224,13cyclictest0-21swapper/211:06:382
23149998259,17cyclictest20526-21sh09:31:351
2313999823,49cyclictest352-21runrttasks08:46:410
2313999822,55cyclictest17702-21apt-get08:16:260
2315999813,31cyclictest17059-21sh10:20:232
2315999813,30cyclictest21147-21fschecks_count08:26:322
2315999813,30cyclictest21147-21fschecks_count08:26:322
2313999803,68cyclictest7628-21apt-key07:46:130
23139998027,46cyclictest0-21swapper/008:01:400
23139998027,46cyclictest0-21swapper/008:01:400
23149997846,8cyclictest31-21ksoftirqd/110:12:251
2316599773,52cyclictest17395-21chrt09:26:573
23149997755,16cyclictest8903-21sh10:06:291
23149997755,16cyclictest8903-21sh10:06:291
2314999773,59cyclictest6585-21ssh10:56:471
23139997718,54cyclictest0-21swapper/008:26:350
23139997718,54cyclictest0-21swapper/008:26:350
23149997644,9cyclictest24212-21ssh11:25:561
23149997644,9cyclictest24212-21ssh11:25:561
23165997533,30cyclictest10949-21taskset09:16:223
23149997537,31cyclictest31213-21kworker/u18:3+nfsiod09:38:101
23149997454,14cyclictest22139-21latency_hist12:16:111
23149997453,15cyclictest6177-21apt-key10:56:121
23149997453,15cyclictest6177-21apt-key10:56:121
23149997451,17cyclictest14229-21sh11:08:431
23149997451,17cyclictest14229-21sh11:08:431
2314999743,46cyclictest28479-21timerwakeupswit08:46:471
23149997415,47cyclictest5235-21kworker/u17:3+ext4-rsv-conversion11:56:381
23149997351,15cyclictest23134-21sh12:16:431
23149997349,14cyclictest0-21swapper/109:27:291
2314999733,29cyclictest10297-21timerandwakeup11:01:471
23149997325,35cyclictest17559-21date08:16:131
23149997251,14cyclictest11763-21threads07:56:461
2316599717,43cyclictest0-21swapper/311:56:263
23149997155,10cyclictest297-21systemd-logind11:51:091
23149997149,12cyclictest0-21swapper/110:06:101
23149997140,9cyclictest181rcu_preempt09:12:161
23149997140,9cyclictest181rcu_preempt09:12:161
2314999713,37cyclictest18056-21fschecks_time08:17:351
23149997049,11cyclictest0-21swapper/112:21:321
2314999704,46cyclictest8406-21ntp_states07:47:391
23149997030,9cyclictest181rcu_preempt11:41:441
148362700,6sleep30-21swapper/308:06:413
23149996855,9cyclictest23116-21cyclictest07:31:341
23149996845,14cyclictest0-21swapper/109:41:431
23149996843,8cyclictest181rcu_preempt07:41:241
23149996814,28cyclictest15668-21apt-get11:11:351
23149996814,28cyclictest15668-21apt-get11:11:351
179322680,5sleep3451rcuc/308:17:313
23149996747,13cyclictest0-21swapper/110:00:471
2314999674,41cyclictest352-21runrttasks09:21:541
2314999674,39cyclictest7544-21ssh09:11:121
23149996735,9cyclictest181rcu_preempt09:46:251
23149996733,9cyclictest181rcu_preempt11:40:251
23149996733,9cyclictest181rcu_preempt11:40:251
2314999673,37cyclictest28643-21unixbench_multi11:31:461
23149996731,10cyclictest0-21swapper/111:21:111
23149996652,8cyclictest0-21swapper/108:51:381
23149996645,15cyclictest9851-21apt-key11:56:101
2314999664,34cyclictest352-21runrttasks12:07:181
23149996642,15cyclictest58-21kcompactd009:16:321
23149996642,12cyclictest0-21swapper/107:11:261
23149996642,12cyclictest0-21swapper/107:11:261
23149996641,15cyclictest0-21swapper/109:06:021
2314999663,45cyclictest352-21runrttasks07:29:371
2314999663,43cyclictest21291-21ntp_states10:26:401
2314999663,43cyclictest21291-21ntp_states10:26:401
2314999663,39cyclictest12590-21/usr/sbin/munin08:01:421
2314999663,39cyclictest12590-21/usr/sbin/munin08:01:421
23149996626,15cyclictest181rcu_preempt12:06:121
23149996626,15cyclictest181rcu_preempt12:06:121
23149996625,9cyclictest2494-21ssh09:56:121
23149996625,9cyclictest2494-21ssh09:56:121
2314999662,45cyclictest23914-21needreboot06:56:561
2168626612,35sleep30-21swapper/306:52:153
2314999654,30cyclictest14234-21/usr/sbin/munin08:06:351
23149996541,16cyclictest0-21swapper/107:21:121
23149996541,16cyclictest0-21swapper/107:21:121
2314999653,45cyclictest20730-21latency_hist08:26:121
2314999653,45cyclictest20730-21latency_hist08:26:121
2314999653,41cyclictest352-21runrttasks10:33:221
2314999653,41cyclictest26416-21unixbench_singl07:01:491
2314999653,41cyclictest23454-21unixbench_multi08:31:501
2314999652,43cyclictest352-21runrttasks11:30:191
2314999644,22cyclictest9695-21idleruntime07:51:331
23149996442,16cyclictest29522-21sh10:41:121
23149996440,11cyclictest0-21swapper/108:27:021
23149996440,11cyclictest0-21swapper/108:27:021
23149996436,19cyclictest26025-21cpu08:41:271
2314999643,37cyclictest17709-21cpu10:21:281
23165996325,26cyclictest6292-21chrt10:01:393
23149996341,14cyclictest24404-21apt-get08:36:381
23149996341,14cyclictest24404-21apt-get08:36:381
2314999633,41cyclictest1136-21timerwakeupswit10:46:451
2314999633,31cyclictest27086-21/usr/sbin/munin07:06:421
2314999633,31cyclictest27086-21/usr/sbin/munin07:06:421
2314999632,43cyclictest29638-21/usr/sbin/munin10:41:441
23165996223,27cyclictest3585-21taskset11:45:313
2314999623,34cyclictest5765-21latency_hist07:41:121
23165996124,26cyclictest15349-21chrt12:04:163
23165996124,26cyclictest15349-21chrt12:04:163
2314999613,23cyclictest14790-21awk10:16:261
2316599603,41cyclictest23913-21taskset10:31:313
23165996026,22cyclictest10270-21chrt09:15:133
23165996026,22cyclictest10270-21chrt09:15:133
23165996024,23cyclictest28088-21taskset07:06:473
23165996024,23cyclictest28088-21taskset07:06:473
23165996024,23cyclictest27415-21chrt08:46:103
23165996021,26cyclictest20930-21taskset12:13:043
2314999604,38cyclictest17460-21sshd07:21:281
23165995924,23cyclictest2842-21chrt09:56:283
23165995919,25cyclictest19458-21taskset08:21:273
23165995919,25cyclictest19458-21taskset08:21:273
2316599584,37cyclictest6668-21taskset09:09:083
23165995826,22cyclictest22490-21chrt11:21:453
23165995826,22cyclictest22490-21chrt11:21:453
23165995823,20cyclictest16239-21chrt08:11:353
23165995822,23cyclictest23642-21chrt08:32:583
23165995822,23cyclictest13068-21chrt11:06:413
23165995822,23cyclictest13068-21chrt11:06:413
23165995821,23cyclictest31710-21taskset09:50:493
23165995723,21cyclictest982-21taskset10:46:433
23165995723,21cyclictest31179-21chrt08:56:233
23165995722,23cyclictest17698-21chrt10:21:263
23165995722,21cyclictest28422-21chrt09:45:053
23165995722,21cyclictest28116-21chrt08:46:393
23165995722,21cyclictest25631-21chrt08:39:503
23165995722,21cyclictest25631-21chrt08:39:503
23165995722,21cyclictest11669-21taskset11:04:593
224832570,8sleep30-21swapper/309:35:123
2316599563,38cyclictest6432-21chrt07:41:383
23165995625,20cyclictest26824-21chrt07:06:103
23165995623,20cyclictest24336-21chrt09:36:583
23165995622,22cyclictest30252-21chrt08:51:493
23165995622,21cyclictest2104-21chrt09:01:343
23165995621,24cyclictest7409-21taskset10:58:263
23165995621,22cyclictest27198-21chrt11:30:533
23165995620,24cyclictest7820-21chrt07:46:443
23165995620,24cyclictest22657-21taskset10:29:163
23165995620,24cyclictest22657-21taskset10:29:163
23165995620,22cyclictest2903-21taskset07:31:333
2316599552,38cyclictest3397-21chrt10:51:293
2316599552,38cyclictest3397-21chrt10:51:293
23165995523,21cyclictest7928-21chrt11:51:443
23165995523,21cyclictest21970-21chrt08:27:113
23165995523,21cyclictest21970-21chrt08:27:113
23165995522,22cyclictest30637-21chrt11:36:243
23165995522,22cyclictest30637-21chrt11:36:243
23165995522,21cyclictest26669-21chrt10:36:213
23165995521,22cyclictest6373-21chrt11:49:533
23165995521,22cyclictest18654-21chrt11:16:223
23165995521,22cyclictest17257-21taskset10:20:573
23165995521,21cyclictest5187-21chrt07:36:443
23165995519,24cyclictest29674-21taskset10:41:243
23165995519,24cyclictest13770-21taskset08:03:543
23165995519,24cyclictest13770-21taskset08:03:543
23165995519,23cyclictest12369-21taskset08:01:103
23165995518,25cyclictest32197-21taskset09:51:253
23165995518,25cyclictest32197-21taskset09:51:253
23165995518,24cyclictest11681-21taskset10:11:203
23165995517,25cyclictest18675-21taskset12:10:083
2316599543,39cyclictest10503-21taskset10:08:453
2316599543,39cyclictest10503-21taskset10:08:453
2316599542,36cyclictest9556-21taskset07:51:303
23165995421,21cyclictest710-21chrt07:25:093
23165995420,22cyclictest22354-21chrt12:16:203
23165995420,21cyclictest23465-21chrt06:56:423
23165995420,19cyclictest25461-21chrt12:21:273
23165995419,23cyclictest28970-21taskset11:32:273
23165995321,20cyclictest29836-21chrt07:13:313
23165995321,20cyclictest29836-21chrt07:13:313
23165995321,20cyclictest14832-21chrt09:21:453
23165995320,21cyclictest2107-21chrt07:27:003
23165995319,20cyclictest30840-21chrt07:16:393
23165995319,20cyclictest30840-21chrt07:16:393
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional