You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-24 - 09:32
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Fri Apr 24, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
234759911847,44cyclictest18436-21ntp_kernel_err20:23:000
22471211411,71sleep022512-21awk18:58:070
23475991133,92cyclictest18802-21/usr/sbin/munin22:29:280
23475991133,79cyclictest21512-21latency_hist23:27:370
23475991123,59cyclictest0-21swapper/023:40:210
23475991113,66cyclictest0-21swapper/023:22:330
23475991103,68cyclictest19716-21df_inode00:17:530
234909910848,49cyclictest0-21swapper/122:58:101
234759910814,55cyclictest29323-21kernelversion21:53:140
234759910814,55cyclictest29323-21kernelversion21:53:140
234909910642,10cyclictest0-21swapper/100:12:581
23475991063,66cyclictest7317-21find23:03:090
23475991063,66cyclictest7317-21find23:03:090
234759910517,52cyclictest0-21swapper/023:08:080
234759910510,57cyclictest0-21swapper/021:27:560
234759910510,57cyclictest0-21swapper/021:27:560
23475991043,87cyclictest0-21swapper/021:07:450
2256121049,67sleep122567-21/usr/sbin/munin18:58:121
23475991034,57cyclictest0-21swapper/021:38:120
23475991023,70cyclictest30729-21ssh23:42:460
23475991023,64cyclictest0-21swapper/021:47:350
23475991023,64cyclictest0-21swapper/021:47:350
23475991022,69cyclictest27601-21ssh00:30:590
23475991022,65cyclictest7639-21ssh22:11:280
23475991022,65cyclictest7639-21ssh22:11:280
23495991019,82cyclictest21725-21ssh23:27:492
234909910135,13cyclictest0-21swapper/121:38:011
234759910148,47cyclictest3981-40find23:52:340
23475991013,71cyclictest25532-21switchtime23:33:070
23475991013,71cyclictest25532-21switchtime23:33:070
234759910128,65cyclictest0-21swapper/022:22:360
23475991003,65cyclictest17892-21latency_hist20:22:370
23475991002,64cyclictest3941-21rm22:04:320
23475991002,62cyclictest6172-21unin-run23:02:340
23490999940,13cyclictest0-21swapper/123:33:091
23490999940,13cyclictest0-21swapper/123:33:091
23490999937,12cyclictest0-21swapper/120:12:561
2347599995,78cyclictest0-21swapper/020:27:460
23475999939,54cyclictest365-21ntpd21:23:040
23475999938,52cyclictest30613-21diskmemload22:38:030
23475999938,52cyclictest30613-21diskmemload22:38:030
2347599992,63cyclictest2960-21runrttasks22:57:060
2347599992,62cyclictest1000-21runrttasks21:07:080
2347599992,62cyclictest1000-21runrttasks21:07:080
23490999844,8cyclictest0-21swapper/123:27:441
23490999837,10cyclictest0-21swapper/119:57:361
2349099983,37cyclictest23448-21timerwakeupswit00:23:101
2349099983,37cyclictest23448-21timerwakeupswit00:23:101
23490999832,12cyclictest0-21swapper/122:43:031
2347599984,62cyclictest0-21swapper/020:37:270
2347599983,63cyclictest32362-21ntpq21:58:000
2347599983,60cyclictest22102-21ssh23:28:290
23475999828,54cyclictest18140-21kworker/u17:3+xprtiod00:05:020
23490999735,11cyclictest0-21swapper/120:27:581
2347599974,83cyclictest29906-21ssh22:47:500
2347599974,78cyclictest13100-21apt-get20:07:450
23475999729,47cyclictest0-21swapper/019:23:060
2347599972,63cyclictest1-21systemd00:17:340
23490999642,9cyclictest0-21swapper/122:37:561
23490999642,9cyclictest0-21swapper/122:37:561
2347599965,52cyclictest0-21swapper/000:10:030
2347599965,52cyclictest0-21swapper/000:10:030
23475999641,47cyclictest0-21swapper/022:13:070
2347599962,55cyclictest0-21swapper/023:58:590
23475999613,53cyclictest10086-21diskmemload21:22:170
2349099954,35cyclictest19440-21latency_hist21:37:371
23490999536,9cyclictest0-21swapper/121:57:361
23490999536,9cyclictest0-21swapper/121:57:361
2347599958,52cyclictest0-21swapper/021:12:460
2347599956,58cyclictest0-21swapper/022:25:290
2347599956,58cyclictest0-21swapper/022:25:290
23475999538,48cyclictest0-21swapper/021:52:330
2347599953,51cyclictest0-21swapper/000:25:570
2347599953,51cyclictest0-21swapper/000:25:570
2347599952,60cyclictest30072-21seq19:19:520
2347599952,60cyclictest23931-21uniq19:02:550
2347599952,60cyclictest23931-21uniq19:02:550
2347599952,58cyclictest23892-21seq20:39:260
23490999436,11cyclictest0-21swapper/121:12:431
2347599943,58cyclictest9418-21seq19:55:030
2347599942,59cyclictest28757-21seq20:54:370
2347599942,59cyclictest28757-21seq20:54:370
23495999368,10cyclictest181rcu_preempt22:37:372
2349099934,33cyclictest352-21runrttasks20:07:591
23490999326,9cyclictest0-21swapper/119:27:351
2347599936,54cyclictest0-21swapper/020:02:460
2347599936,54cyclictest0-21swapper/020:02:460
2347599933,83cyclictest21715-21switchtime22:33:080
2347599932,55cyclictest11120-21seq20:01:190
2349099924,38cyclictest352-21runrttasks22:30:541
2349099924,37cyclictest6023-21ssh23:54:571
2349099924,36cyclictest17763-21latency_hist20:22:361
23490999243,41cyclictest0-21swapper/100:32:351
2349099923,42cyclictest13588-21cpuspeed00:07:481
2349099923,42cyclictest13588-21cpuspeed00:07:481
2349099923,36cyclictest30915-21/usr/sbin/munin21:03:031
2349099923,36cyclictest30915-21/usr/sbin/munin21:03:031
2349099923,34cyclictest27509-21latency_hist20:52:361
2349099923,34cyclictest27509-21latency_hist20:52:361
23490999225,17cyclictest0-21swapper/122:48:021
2347599924,75cyclictest26021-21apt-get20:47:480
2347599924,75cyclictest26021-21apt-get20:47:480
23475999237,47cyclictest0-21swapper/020:12:560
2347599922,63cyclictest19676-21snmpd23:56:540
2347599922,56cyclictest30283-21seq20:58:330
2349099915,31cyclictest25112-21latency_hist19:07:371
2349099915,31cyclictest25112-21latency_hist19:07:371
2349099913,42cyclictest12946-21latency23:12:581
2349099913,42cyclictest12946-21latency23:12:581
2349099913,38cyclictest28011-21kernelversion20:52:591
2349099913,38cyclictest28011-21kernelversion20:52:591
2347599913,75cyclictest12658-21df_inode23:12:510
2347599913,75cyclictest12658-21df_inode23:12:510
2347599913,55cyclictest4200-21seq19:38:270
2347599912,65cyclictest352-21runrttasks22:47:310
2347599912,58cyclictest2661-21seq19:34:180
2347599912,54cyclictest28421-21seq19:14:170
2349099904,34cyclictest10182-21latency_hist00:02:351
2347599902,56cyclictest7912-21seq19:51:470
2347599902,53cyclictest5950-21seq19:45:190
2349099893,39cyclictest32345-21fschecks_count19:27:551
2349099893,36cyclictest4572-21latency_hist19:42:361
2349099893,36cyclictest30976-21fschecks_count23:42:551
2349099893,35cyclictest22579-21latency_hist20:37:351
2349099892,39cyclictest2024-21cron22:02:321
2347599892,56cyclictest25737-21seq20:47:320
2347599892,55cyclictest32145-21seq19:27:410
2347599892,55cyclictest26871-21seq19:10:200
23495998862,17cyclictest28544-21apt-key21:52:372
2349099884,36cyclictest26537-21timerwakeupswit19:08:111
2349099883,37cyclictest23749-21latency_hist22:37:361
2347599882,53cyclictest17027-21runrttasks21:33:340
2349099874,36cyclictest6581-21cstates23:02:511
2349099874,36cyclictest6581-21cstates23:02:511
2349099873,37cyclictest9252-21threads22:13:091
2349099873,37cyclictest352-21runrttasks21:00:441
2349099864,36cyclictest10478-21/usr/sbin/munin00:03:061
2349099864,34cyclictest13014-21latency_hist20:07:361
2349099864,34cyclictest13014-21latency_hist20:07:361
2349099863,38cyclictest2293-21switchtime21:08:091
2229628626,38sleep20-21swapper/218:58:012
23495998560,9cyclictest181rcu_preempt23:02:512
23495998560,9cyclictest181rcu_preempt23:02:512
23490998535,9cyclictest0-21swapper/121:18:101
2349099853,42cyclictest28732-21ssh21:52:381
2349099853,36cyclictest16191-21latency_hist21:32:431
2349099853,36cyclictest16191-21latency_hist21:32:431
2349099853,36cyclictest10585-21cat19:58:071
2349599847,33cyclictest30613-21diskmemload23:58:272
23495998416,40cyclictest39-21ksoftirqd/222:47:312
2349099843,36cyclictest352-21runrttasks21:45:131
2349099843,36cyclictest352-21runrttasks21:45:131
2349099843,30cyclictest11595-21/usr/sbin/munin22:18:591
23495998326,15cyclictest0-21swapper/219:02:502
23495998326,15cyclictest0-21swapper/219:02:502
23495998250,13cyclictest181rcu_preempt20:18:362
23495998249,8cyclictest181rcu_preempt00:21:262
23495998246,12cyclictest181rcu_preempt21:08:082
23495998162,14cyclictest31313-21threads19:23:072
23495998158,8cyclictest181rcu_preempt19:33:532
2349099819,34cyclictest3184-21cat22:57:351
23495998063,10cyclictest19990-21latency21:37:592
23495998044,27cyclictest9620-21modprobe19:57:342
2349099803,31cyclictest27928-21processes19:13:031
23495997960,13cyclictest6137-21latency_hist19:47:372
23495997958,12cyclictest0-21swapper/200:08:042
23495997958,12cyclictest0-21swapper/200:08:042
2349599792,46cyclictest39-21ksoftirqd/220:04:022
2349599792,46cyclictest39-21ksoftirqd/220:04:022
23495997911,57cyclictest17853-21apt-get22:27:502
23490997957,14cyclictest2374-21systemd-journal23:26:401
23490997956,16cyclictest30613-21diskmemload23:48:121
2349099794,33cyclictest30424-21latency_hist19:22:361
2349099793,35cyclictest3015-21latency_hist19:37:361
23495997862,11cyclictest23466-21cyclictest23:19:212
23495997845,8cyclictest181rcu_preempt20:42:512
2349599784,46cyclictest39-21ksoftirqd/223:47:322
2350299773,38cyclictest0-21swapper/323:58:223
23495997759,13cyclictest30613-21diskmemload00:15:262
23495997753,8cyclictest181rcu_preempt23:42:342
23495997751,9cyclictest181rcu_preempt23:12:432
23495997751,9cyclictest181rcu_preempt23:12:432
23495997744,13cyclictest181rcu_preempt21:43:102
23495997744,13cyclictest181rcu_preempt21:43:102
23495997738,16cyclictest181rcu_preempt22:40:482
23495997738,16cyclictest181rcu_preempt22:40:482
23495997732,8cyclictest181rcu_preempt21:19:252
23495997725,18cyclictest0-21swapper/223:33:442
23495997725,18cyclictest0-21swapper/223:33:442
23495997652,16cyclictest2374-21systemd-journal23:00:592
23495997629,15cyclictest181rcu_preempt23:48:522
2349599762,37cyclictest39-21ksoftirqd/222:17:372
2349599762,28cyclictest39-21ksoftirqd/222:09:422
2349599762,28cyclictest39-21ksoftirqd/222:09:422
2349099763,37cyclictest8000-21ls19:52:331
2349599757,56cyclictest28853-21ssh21:52:522
2349599757,56cyclictest28853-21ssh21:52:522
23495997554,15cyclictest10067-21fschecks_count19:57:542
23495997542,11cyclictest181rcu_preempt20:29:322
23495997540,18cyclictest181rcu_preempt21:24:442
23495997538,13cyclictest181rcu_preempt19:48:562
23495997534,9cyclictest0-21swapper/221:13:012
23495997532,17cyclictest181rcu_preempt23:54:452
23495997528,17cyclictest0-21swapper/222:07:352
23495997515,31cyclictest39-21ksoftirqd/221:34:202
2349599746,54cyclictest11920-21ssh22:18:352
23495997452,16cyclictest12151-21idleruntime-cro23:12:332
23495997449,19cyclictest2374-21systemd-journal21:28:402
23495997449,19cyclictest2374-21systemd-journal21:28:402
23495997444,7cyclictest181rcu_preempt20:17:372
2349599737,20cyclictest32744-21ssh21:58:082
2349599736,31cyclictest39-21ksoftirqd/200:22:552
2349599736,31cyclictest39-21ksoftirqd/200:22:552
23495997351,8cyclictest181rcu_preempt22:22:452
23495997351,8cyclictest181rcu_preempt22:22:452
23495997336,17cyclictest181rcu_preempt20:42:342
2349599732,37cyclictest39-21ksoftirqd/220:58:062
23495997252,12cyclictest0-21swapper/200:03:092
23495997250,14cyclictest23466-21cyclictest20:24:542
2349599723,51cyclictest22100-21grep20:33:082
23495997226,29cyclictest0-21swapper/222:48:092
23490997259,9cyclictest23466-21cyclictest23:09:411
23495997152,12cyclictest23466-21cyclictest19:16:402
23495997151,13cyclictest298-21in:imuxsock22:57:332
23495997149,8cyclictest181rcu_preempt00:28:062
23495997147,18cyclictest1196-21apt-key19:32:362
23495997137,12cyclictest181rcu_preempt19:07:572
23495997134,12cyclictest181rcu_preempt19:39:572
23495997132,18cyclictest181rcu_preempt21:04:312
23495997132,18cyclictest181rcu_preempt21:04:312
23490997154,11cyclictest297-21systemd-logind22:07:341
23490997151,11cyclictest0-21swapper/119:47:371
23490997144,11cyclictest58-21kcompactd021:23:111
2350299703,45cyclictest0-21swapper/321:52:573
2350299703,45cyclictest0-21swapper/321:52:573
23495997036,12cyclictest181rcu_preempt19:17:552
23495997032,13cyclictest181rcu_preempt23:26:392
23495997031,12cyclictest181rcu_preempt20:48:062
23495997031,12cyclictest181rcu_preempt20:48:062
23490997052,11cyclictest300-21rs:main1
2350299693,57cyclictest21804-21chrt23:28:033
23490996944,17cyclictest24154-21munin-plugin-st20:42:351
23490996848,13cyclictest24512-21cstates20:42:501
2349099684,43cyclictest19955-21kernelversion00:17:591
23495996744,10cyclictest181rcu_preempt20:52:552
23495996744,10cyclictest181rcu_preempt20:52:552
2349099673,44cyclictest27921-21fschecks_count23:37:521
2349099663,45cyclictest18450-21ntp_kernel_pll_20:23:001
2349599652,49cyclictest14520-21cat20:12:352
23490996544,15cyclictest17750-21apt-key22:27:371
23490996544,15cyclictest17750-21apt-key22:27:371
2350299648,31cyclictest0-21swapper/322:27:543
23502996411,30cyclictest0-21swapper/322:18:353
23490996342,15cyclictest8124-21latency_hist22:12:361
23490996342,15cyclictest8124-21latency_hist22:12:361
23502996230,22cyclictest10785-21chrt21:22:563
23502995947,8cyclictest451rcuc/320:22:003
23502995844,9cyclictest451rcuc/320:17:063
2350299582,42cyclictest17996-21taskset23:22:113
124942580,5sleep30-21swapper/323:12:473
124942580,5sleep30-21swapper/323:12:473
23502995743,9cyclictest451rcuc/323:01:373
23502995743,9cyclictest451rcuc/319:34:583
2350299573,34cyclictest7413-21chrt21:17:373
23502995643,9cyclictest451rcuc/320:53:353
23502995643,9cyclictest451rcuc/320:53:353
23502995642,9cyclictest451rcuc/320:43:233
23502995641,9cyclictest451rcuc/321:21:073
23502995641,9cyclictest451rcuc/320:40:523
23502995543,8cyclictest451rcuc/319:22:233
23502995542,9cyclictest451rcuc/320:26:543
23502995542,8cyclictest451rcuc/323:55:273
23502995542,8cyclictest451rcuc/321:39:193
23502995541,9cyclictest451rcuc/323:09:073
23502995541,10cyclictest451rcuc/319:15:473
23502995441,9cyclictest451rcuc/320:34:243
23502995441,9cyclictest451rcuc/320:11:523
23502995441,9cyclictest451rcuc/319:11:543
23502995436,10cyclictest451rcuc/319:55:523
23502995435,13cyclictest451rcuc/319:49:403
2350299543,36cyclictest7930-21chrt22:12:173
2350299543,36cyclictest7930-21chrt22:12:173
23502995341,8cyclictest451rcuc/323:24:393
23502995341,8cyclictest451rcuc/300:20:563
23502995339,9cyclictest451rcuc/322:49:013
23502995339,9cyclictest451rcuc/321:32:143
23502995339,9cyclictest451rcuc/321:32:143
23502995339,9cyclictest451rcuc/321:06:073
23502995339,9cyclictest451rcuc/321:06:073
2350299533,38cyclictest3240-21taskset23:50:403
2350299533,36cyclictest25720-21taskset23:33:093
2350299533,36cyclictest25720-21taskset23:33:093
2350299533,35cyclictest10542-21chrt19:58:063
2350299532,41cyclictest826-21chrt21:59:123
23502995239,9cyclictest451rcuc/322:42:363
23502995239,9cyclictest451rcuc/322:42:363
23502995239,9cyclictest451rcuc/319:31:023
23502995238,9cyclictest451rcuc/320:59:183
23502995234,11cyclictest451rcuc/320:04:023
23502995234,11cyclictest451rcuc/320:04:023
23502995234,11cyclictest451rcuc/319:40:293
2350299523,37cyclictest15930-21taskset00:11:513
2350299523,37cyclictest15930-21taskset00:11:513
2350299523,35cyclictest26956-21chrt22:42:483
2350299522,41cyclictest15713-21chrt22:23:113
2350299522,41cyclictest15713-21chrt22:23:113
2232525211,25sleep30-21swapper/318:58:023
2350299515,36cyclictest31118-21taskset23:42:583
23502995139,8cyclictest451rcuc/322:55:343
23502995138,8cyclictest451rcuc/319:45:443
23502995137,9cyclictest451rcuc/320:52:013
23502995137,9cyclictest451rcuc/320:52:013
23502995137,9cyclictest451rcuc/319:23:163
2350299513,37cyclictest17404-21chrt00:13:093
2350299513,34cyclictest2352-21chrt22:02:443
2350299512,39cyclictest28271-21chrt23:37:593
2350299512,33cyclictest21081-21chrt20:32:363
23502995038,8cyclictest451rcuc/322:14:243
2350299502,36cyclictest17919-21taskset21:33:593
2350299502,35cyclictest10681-21taskset00:02:513
23502994936,9cyclictest451rcuc/319:05:513
23502994936,9cyclictest451rcuc/319:05:513
2350299492,36cyclictest8011-21chrt23:04:283
2350299492,36cyclictest8011-21chrt23:04:283
2350299492,36cyclictest23766-21chrt00:23:343
2350299492,36cyclictest23766-21chrt00:23:343
23502994836,8cyclictest451rcuc/321:47:343
23502994836,8cyclictest451rcuc/321:47:343
2350299482,34cyclictest26276-21chrt21:48:093
2350299473,30cyclictest26787-21chrt00:28:413
23502994432,8cyclictest451rcuc/322:36:003
2350299412,29cyclictest3414-21taskset21:10:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional