You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-09 - 08:43
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Thu Apr 09, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23263211312,73sleep223292-21/usr/sbin/munin18:59:022
24463991123,92cyclictest3532-21apt-get22:03:551
23523210612,65sleep123536-21/usr/sbin/munin18:59:131
1765821040,7sleep131572-21diskmemload23:19:091
2998621030,8sleep20-21swapper/222:46:322
24463991033,31cyclictest14803-21sh22:21:421
24471991029,80cyclictest19858-21apt-get23:23:522
244639910249,35cyclictest24450-21cyclictest21:44:111
24471991014,85cyclictest12884-21taskset22:18:582
2446399993,81cyclictest12216-21apt-get20:03:491
2446399993,81cyclictest12216-21apt-get20:03:491
24463999811,78cyclictest28038-21apt-get22:43:491
2446399972,27cyclictest352-21runrttasks23:55:271
2446399972,27cyclictest352-21runrttasks23:55:271
24459999754,28cyclictest31572-21diskmemload22:03:520
2446399962,33cyclictest29201-21latency_hist00:33:361
24471999564,22cyclictest3327-21latency_hist22:03:372
24471999564,22cyclictest3327-21latency_hist22:03:372
24463999554,34cyclictest352-21runrttasks23:59:331
2446399952,30cyclictest16624-21sh00:12:001
2447899948,52cyclictest0-21swapper/323:23:523
24463999436,51cyclictest0-21swapper/119:59:091
2446399942,31cyclictest21396-21sh23:25:241
2445999949,49cyclictest15-21ksoftirqd/020:38:370
2446399933,28cyclictest31954-21sh21:57:141
24463999315,50cyclictest15904-21seq21:30:111
24459999371,16cyclictest27811-21apt-key22:43:380
24463999245,40cyclictest25865-21unixbench_multi22:39:151
2446399923,29cyclictest1673-21sh21:59:561
2446399923,29cyclictest1673-21sh21:59:561
2445999924,55cyclictest170-21jbd2/mmcblk2p2-819:03:370
2380529116,46sleep30-21swapper/318:59:193
24471999062,8cyclictest181rcu_preempt20:18:462
2446399904,74cyclictest4048-21apt-get19:38:461
2446399904,74cyclictest4048-21apt-get19:38:461
24471998948,10cyclictest181rcu_preempt20:43:472
24471998948,10cyclictest181rcu_preempt20:43:472
2446399894,73cyclictest0-21swapper/121:13:471
2446399894,73cyclictest0-21swapper/121:13:471
2446399893,6cyclictest0-21swapper/120:23:481
24459998916,57cyclictest5210-21ssh23:53:340
335028814,11sleep03466-21tail21:09:120
24463998871,11cyclictest0-21swapper/122:59:011
24463998871,11cyclictest0-21swapper/122:59:011
2446399883,7cyclictest0-21swapper/123:48:471
3132128718,10sleep031251-21latency_hist19:23:370
2446399873,38cyclictest22457-21/usr/sbin/munin22:34:051
2446399872,37cyclictest25965-21/usr/sbin/munin23:33:481
24459998764,18cyclictest16623-21kworker/u19:1+events_unbound00:23:330
24459998764,18cyclictest16623-21kworker/u19:1+events_unbound00:23:330
24459998746,8cyclictest181rcu_preempt22:09:230
24459998745,33cyclictest300-21rs:main0
24459998745,33cyclictest300-21rs:main0
24471998667,12cyclictest23237-21systemd-run00:23:352
24471998667,12cyclictest23237-21systemd-run00:23:352
24471998665,15cyclictest12978-21timerandwakeup20:04:082
24471998665,15cyclictest12978-21timerandwakeup20:04:082
24471998662,17cyclictest24450-21cyclictest19:39:062
2446399863,36cyclictest7695-21/usr/sbin/munin19:48:571
24459998667,13cyclictest21482-21sendmail22:33:340
2445999865,57cyclictest15-21ksoftirqd/019:23:590
24471998538,8cyclictest181rcu_preempt20:13:352
2447199853,30cyclictest2480-21ls22:54:112
2446399853,8cyclictest0-21swapper/120:08:461
2446399852,37cyclictest19676-21snmpd22:23:451
2446399852,31cyclictest8730-21grep21:18:561
24471998464,14cyclictest12534-21apt-key22:18:382
24471998445,15cyclictest24450-21cyclictest00:15:262
24463998461,16cyclictest26647-21latency_hist20:48:361
24463998461,16cyclictest26647-21latency_hist20:48:361
2446399843,6cyclictest0-21swapper/100:18:461
2446399843,6cyclictest0-21swapper/100:18:461
2445999843,46cyclictest15-21ksoftirqd/020:32:290
24459998414,44cyclictest15-21ksoftirqd/000:13:380
2446399833,33cyclictest24209-21taskset23:29:511
2446399832,36cyclictest29720-21latency_hist20:58:371
2446399832,35cyclictest13521-21ssh21:26:521
2446399832,35cyclictest13521-21ssh21:26:521
2446399832,35cyclictest11676-21/usr/sbin/munin00:03:501
2446399832,34cyclictest23718-21fschecks_count20:38:541
2446399832,34cyclictest23718-21fschecks_count20:38:541
24459998358,17cyclictest2345-21latency_hist23:48:370
24471998261,14cyclictest9464-21sh21:19:112
24471998244,12cyclictest181rcu_preempt23:52:552
2446399824,69cyclictest30220-21apt-get20:59:011
2446399823,6cyclictest0-21swapper/119:13:491
2446399823,29cyclictest562-21cat19:28:371
2446399822,8cyclictest0-21swapper/119:08:521
24471998139,9cyclictest181rcu_preempt20:24:042
2446399813,36cyclictest17561-21/usr/sbin/munin20:19:021
2446399813,34cyclictest19553-21mailstats22:29:081
2446399812,38cyclictest2127-21idleruntime-cro23:48:341
2446399812,38cyclictest2127-21idleruntime-cro23:48:341
2446399812,35cyclictest29555-21latency_hist21:53:381
2446399812,35cyclictest29555-21latency_hist21:53:381
2446399812,29cyclictest1404-21latency_hist22:53:361
2446399812,29cyclictest1404-21latency_hist22:53:361
24459998162,13cyclictest3222-21kworker/u17:2+rpciod21:14:090
24459998162,13cyclictest3222-21kworker/u17:2+rpciod21:14:090
2445999814,46cyclictest15-21ksoftirqd/000:23:590
2445999814,44cyclictest15-21ksoftirqd/021:59:120
2445999814,44cyclictest15-21ksoftirqd/021:59:120
2447899807,45cyclictest0-21swapper/322:43:553
2447899802,35cyclictest13425-21chrt23:13:363
2447899802,35cyclictest13425-21chrt23:13:363
24471998051,10cyclictest181rcu_preempt23:08:432
24471998051,10cyclictest181rcu_preempt23:08:432
24471998048,15cyclictest181rcu_preempt21:30:522
24471998040,14cyclictest23586-21awk23:29:062
2446399803,36cyclictest17489-21ssh21:33:461
2446399803,32cyclictest17779-21fschecks_count00:13:541
24463998029,43cyclictest21540-21unixbench_singl21:39:111
24463998029,43cyclictest21540-21unixbench_singl21:39:111
2446399802,39cyclictest8213-21ssh23:04:041
2446399802,37cyclictest6634-21/usr/sbin/munin19:44:081
2446399802,35cyclictest3127-21ps21:09:081
2446399802,35cyclictest30196-21/usr/sbin/munin19:18:571
2446399802,35cyclictest23530-21sh00:23:461
2446399802,31cyclictest13065-21sh23:12:591
2446399802,31cyclictest13065-21sh23:12:591
2446399802,30cyclictest9164-21/usr/sbin/munin19:53:541
2446399802,30cyclictest829-21taskset19:28:541
2445999804,47cyclictest15-21ksoftirqd/019:03:520
2445999804,42cyclictest15-21ksoftirqd/023:21:510
2447899795,29cyclictest5661-21chrt22:59:103
2447899795,29cyclictest5661-21chrt22:59:103
2447899792,33cyclictest25316-21chrt22:39:043
24471997962,12cyclictest24450-21cyclictest00:03:542
24471997959,13cyclictest297-21systemd-logind23:08:342
2446399793,35cyclictest15939-21/usr/sbin/munin20:14:011
2446399793,29cyclictest13807-21/usr/sbin/munin23:13:521
2446399792,31cyclictest4925-21cat19:39:101
2446399792,31cyclictest3575-21runrttasks22:56:361
2446399792,28cyclictest32156-21fschecks_count21:03:551
2446399792,28cyclictest32156-21fschecks_count21:03:551
2445999793,43cyclictest15-21ksoftirqd/023:38:440
24459997928,28cyclictest15-21ksoftirqd/023:03:590
24459997922,34cyclictest15-21ksoftirqd/022:34:260
24471997863,9cyclictest12223-21kworker/u17:3-xprtiod23:19:412
24471997851,17cyclictest29436-21latency_hist00:33:372
24471997845,9cyclictest20176-21ssh22:30:002
2446399783,6cyclictest0-21swapper/120:33:451
2446399783,34cyclictest29075-21cstates23:38:511
2446399783,29cyclictest24987-21/usr/sbin/munin19:03:581
2446399782,37cyclictest9579-21ssh22:13:451
2446399782,29cyclictest7175-21/usr/sbin/munin22:09:051
24459997848,21cyclictest544-21cron23:58:330
24459997848,21cyclictest544-21cron23:58:330
24459997836,10cyclictest181rcu_preempt23:59:010
24459997831,12cyclictest181rcu_preempt20:18:560
24471997755,15cyclictest26223-21kworker/u17:3-nfsiod23:47:492
24471997755,15cyclictest26223-21kworker/u17:3-nfsiod23:47:492
2446399772,31cyclictest28196-21cat20:53:361
24459997755,14cyclictest24450-21cyclictest20:08:530
24459997743,22cyclictest0-21swapper/021:34:080
24459997741,29cyclictest23126-21fschecks_count23:28:550
2445999772,46cyclictest15-21ksoftirqd/021:06:310
2445999772,46cyclictest15-21ksoftirqd/021:06:310
24459997718,41cyclictest30220-21apt-get20:58:490
2447899762,34cyclictest27186-21taskset20:49:003
24471997651,9cyclictest181rcu_preempt22:33:452
2446399762,32cyclictest20727-21/usr/sbin/munin20:28:591
2445999766,47cyclictest15-21ksoftirqd/022:23:440
24459997660,10cyclictest2041-21sh22:54:010
24459997648,11cyclictest181rcu_preempt23:36:450
2445999763,47cyclictest15-21ksoftirqd/023:43:500
2447899754,32cyclictest5552-21taskset19:43:363
24471997559,12cyclictest24450-21cyclictest22:04:252
24471997558,10cyclictest6945-21kworker/u17:1-xprtiod21:38:372
24471997548,21cyclictest28121-21latency_hist19:13:362
2445999759,37cyclictest15-21ksoftirqd/023:09:250
2445999759,37cyclictest15-21ksoftirqd/023:09:250
2445999753,32cyclictest17011-21sed20:18:370
24478997412,34cyclictest0-21swapper/322:18:493
24471997453,16cyclictest29396-21kworker/u17:2-ext4-rsv-conversion20:58:372
24471997453,16cyclictest24450-21cyclictest19:48:552
24471997441,13cyclictest24450-21cyclictest21:08:142
24471997441,13cyclictest24450-21cyclictest21:08:142
24471997431,11cyclictest24450-21cyclictest20:59:072
2447199742,61cyclictest30039-21cron21:54:502
24459997454,14cyclictest1-21systemd21:33:350
24459997431,9cyclictest181rcu_preempt20:08:100
24459997431,9cyclictest181rcu_preempt20:08:100
2447899733,27cyclictest18862-21taskset22:28:503
2447899732,34cyclictest11346-21chrt21:23:373
24471997352,14cyclictest2589-21ssh21:08:522
24471997349,15cyclictest0-21swapper/222:43:362
24471997327,23cyclictest26149-21fschecks_count23:33:532
2445999734,39cyclictest15-21ksoftirqd/019:44:050
2445999733,35cyclictest15-21ksoftirqd/022:19:370
24459997328,15cyclictest181rcu_preempt20:53:590
2445999732,46cyclictest15-21ksoftirqd/019:09:090
2447199724,44cyclictest31777-21fschecks_time19:23:552
24471997235,11cyclictest30567-21rm23:40:462
24471997232,11cyclictest181rcu_preempt22:49:032
24471997232,11cyclictest181rcu_preempt22:49:032
24471997223,12cyclictest181rcu_preempt00:08:542
2445999727,29cyclictest15-21ksoftirqd/019:58:370
24459997258,10cyclictest24450-21cyclictest22:17:400
24459997257,11cyclictest24450-21cyclictest21:44:100
24459997244,19cyclictest0-21swapper/019:33:380
2445999723,30cyclictest15-21ksoftirqd/023:14:090
24459997230,17cyclictest181rcu_preempt19:35:000
24459997230,17cyclictest181rcu_preempt19:35:000
24471997155,9cyclictest0-21swapper/221:14:122
24471997155,9cyclictest0-21swapper/221:14:122
24471997139,9cyclictest181rcu_preempt23:13:552
24471997137,17cyclictest0-21swapper/222:58:562
24471997137,17cyclictest0-21swapper/222:58:562
24471997129,12cyclictest181rcu_preempt21:39:072
24471997129,12cyclictest181rcu_preempt21:39:072
2447199712,29cyclictest20454-21grep20:28:542
2445999716,29cyclictest15-21ksoftirqd/021:48:500
2445999716,29cyclictest15-21ksoftirqd/021:48:500
24459997149,12cyclictest0-21swapper/021:39:090
24459997149,12cyclictest0-21swapper/021:39:090
24459997140,24cyclictest0-21swapper/021:53:480
24459997140,22cyclictest26473-21apt20:48:330
24459997140,22cyclictest26473-21apt20:48:330
2445999713,47cyclictest15-21ksoftirqd/000:28:550
2445999713,38cyclictest5495-21munin-plugin-st19:43:330
2447899705,50cyclictest8825-21chrt23:04:323
2447899703,30cyclictest3536-21taskset22:03:453
2447899703,27cyclictest16770-21chrt00:12:243
2447899703,21cyclictest24848-21chrt21:44:523
2447899702,30cyclictest515-21chrt22:51:323
2447899702,30cyclictest515-21chrt22:51:323
24471997056,10cyclictest24450-21cyclictest20:38:582
24471997056,10cyclictest24450-21cyclictest20:38:582
24471997055,10cyclictest365-21ntpd22:24:042
24471997043,9cyclictest181rcu_preempt20:34:022
24471997029,19cyclictest0-21swapper/220:48:572
2445999708,33cyclictest15-21ksoftirqd/021:23:450
2445999708,33cyclictest15-21ksoftirqd/021:23:450
24459997047,8cyclictest181rcu_preempt20:39:130
24459997047,8cyclictest181rcu_preempt20:39:130
2445999704,42cyclictest15-21ksoftirqd/020:01:180
24459997038,12cyclictest181rcu_preempt22:59:290
24459997038,12cyclictest181rcu_preempt22:59:290
24459997026,22cyclictest15-21ksoftirqd/022:44:550
24459997019,20cyclictest0-21swapper/000:09:360
24459997017,21cyclictest15-21ksoftirqd/000:03:540
24459997014,32cyclictest15-21ksoftirqd/020:27:310
2447899692,25cyclictest1967-21chrt22:53:593
24471996948,13cyclictest0-21swapper/221:53:382
24471996948,13cyclictest0-21swapper/221:53:382
2445999693,36cyclictest11141-21latency_hist21:23:360
2445999693,30cyclictest20380-21fschecks_time23:24:310
2447899683,29cyclictest11655-21chrt21:23:533
2447899683,29cyclictest11655-21chrt21:23:533
2447899682,28cyclictest24775-21chrt19:03:513
2447899682,24cyclictest9836-21chrt22:13:543
24471996851,11cyclictest170-21jbd2/mmcblk2p2-823:56:582
24471996851,11cyclictest170-21jbd2/mmcblk2p2-823:56:582
24471996831,17cyclictest181rcu_preempt19:44:022
24471996824,13cyclictest181rcu_preempt19:58:552
24471996823,12cyclictest9758-21chrt19:54:082
24471996820,9cyclictest26389-21latency_hist21:48:372
24471996813,16cyclictest0-21swapper/221:23:552
24471996813,16cyclictest0-21swapper/221:23:552
2445999683,45cyclictest28567-21sort19:13:560
2447899673,23cyclictest9877-21chrt19:54:093
2447899672,29cyclictest29177-21taskset00:33:313
2447199674,46cyclictest25579-21grep19:04:092
2447199673,44cyclictest29208-21unixbench_multi19:14:092
2447199673,42cyclictest875-21fschecks_count19:28:562
24459996743,10cyclictest181rcu_preempt19:48:510
2445999673,45cyclictest15-21ksoftirqd/020:48:570
2447899663,24cyclictest4955-21chrt21:13:073
2447899663,23cyclictest31524-21chrt19:23:473
2447899663,22cyclictest29918-21chrt21:54:123
2447899663,22cyclictest12013-21chrt20:03:363
2447899662,27cyclictest6684-21taskset21:14:293
2447899662,27cyclictest6684-21taskset21:14:293
2447199664,42cyclictest2820-21fschecks_count19:34:532
2447199664,42cyclictest2820-21fschecks_count19:34:532
2447899654,20cyclictest16835-21chrt20:18:353
2447899653,26cyclictest22880-21chrt00:22:423
2447899653,26cyclictest22880-21chrt00:22:423
2447899653,25cyclictest22742-21taskset21:42:253
2447899653,25cyclictest22742-21taskset21:42:253
2447899653,23cyclictest8967-21chrt23:58:593
2447899653,23cyclictest26522-21chrt21:48:493
2447899653,23cyclictest26522-21chrt21:48:493
2447899652,24cyclictest25723-21chrt00:27:153
2447899652,24cyclictest2121-21chrt23:48:333
2447899652,24cyclictest2121-21chrt23:48:333
2447899652,23cyclictest29801-21taskset23:39:093
24471996540,18cyclictest31350-21latency_hist19:23:362
2447199652,43cyclictest16034-21proc_pri20:14:052
2447899643,40cyclictest917-21kworker/3:1+events23:29:053
2447899643,24cyclictest13777-21chrt00:07:263
2447899643,23cyclictest7120-21chrt19:48:343
2447899643,23cyclictest1254-21taskset21:06:103
2447899643,23cyclictest1254-21taskset21:06:103
2447899643,22cyclictest16355-21chrt23:18:353
2447899642,24cyclictest19870-21taskset21:37:473
2447899642,24cyclictest15108-21taskset21:29:043
24471996446,13cyclictest0-21swapper/222:08:432
2447899637,23cyclictest8491-21chrt19:51:013
2447899633,23cyclictest18092-21chrt22:27:323
2447899633,23cyclictest17415-21chrt20:18:593
2447899633,22cyclictest2893-21chrt22:03:203
2447899633,22cyclictest2893-21chrt22:03:203
2447899633,22cyclictest17788-21chrt00:13:543
2447899632,24cyclictest6856-21taskset23:54:503
2447899632,24cyclictest6856-21taskset23:54:503
2447899632,24cyclictest2551-21chrt19:33:523
2447899632,24cyclictest2551-21chrt19:33:523
2447899632,24cyclictest24652-21taskset20:40:543
2447899632,24cyclictest24652-21taskset20:40:543
2447899632,23cyclictest29921-21chrt19:18:493
2447899632,22cyclictest27321-21chrt23:35:043
2447199633,33cyclictest23745-21fschecks_count00:23:552
2447899627,20cyclictest0-21swapper/322:13:363
2447899623,21cyclictest5293-21chrt23:53:363
2447899616,15cyclictest0-21swapper/320:24:093
2447899614,21cyclictest24384-21chrt22:38:183
2447899613,24cyclictest31422-21taskset21:01:523
2447899613,22cyclictest29561-21chrt20:56:423
2447899613,20cyclictest1857-21chrt19:31:453
2447899612,24cyclictest20288-21taskset20:28:473
2447899602,23cyclictest27705-21taskset19:09:503
2447199602,30cyclictest8490-21/usr/sbin/munin23:58:592
2447899578,14cyclictest0-21swapper/320:03:583
2447899578,14cyclictest0-21swapper/320:03:583
2447899562,38cyclictest19246-21chrt23:23:253
24478995612,12cyclictest0-21swapper/320:33:423
2447899552,34cyclictest13955-21chrt20:08:513
2447899543,34cyclictest28749-21chrt19:13:593
2447899492,34cyclictest26316-21chrt20:46:413
2447899492,34cyclictest26316-21chrt20:46:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional