You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-31 - 13:47
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Wed Dec 31, 2025 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1766991138,77cyclictest15559-21apt-get23:59:421
1766991048,65cyclictest2200-21taskset22:45:131
1766991048,65cyclictest2200-21taskset22:45:131
1766999826,14cyclictest0-21swapper/119:19:391
176699974,43cyclictest11928-21latency_hist23:54:281
1766999739,8cyclictest0-21swapper/119:35:011
63929626,43sleep30-21swapper/319:04:583
1766999668,21cyclictest382-21rs:main1
1766999668,21cyclictest382-21rs:main1
1766999611,67cyclictest20387-21apt-get20:04:491
177399956,48cyclictest39-21ksoftirqd/220:44:482
176699953,34cyclictest15156-21unixbench_multi23:05:031
159292950,7sleep115920-21ssh22:14:291
1766999439,49cyclictest0-21swapper/120:24:491
1766999438,48cyclictest0-21swapper/123:29:471
176699933,30cyclictest22753-21ssh00:10:031
176699933,30cyclictest22753-21ssh00:10:031
176699933,30cyclictest18242-21/usr/sbin/munin21:25:021
1766999329,12cyclictest0-21swapper/123:20:161
176699926,65cyclictest20399-21apt-get23:14:431
176699926,65cyclictest20399-21apt-get23:14:431
1766999234,11cyclictest0-21swapper/123:45:011
176699922,39cyclictest27535-21grep00:19:271
176699922,35cyclictest15031-40mandb23:59:291
176699922,34cyclictest13413-21modprobe19:44:241
57929114,52sleep1617-21if_eth019:04:561
1773999174,10cyclictest1465-21apt-key22:44:262
1773999154,14cyclictest1749-21cyclictest20:56:302
1773999154,14cyclictest1749-21cyclictest20:56:302
1766999135,48cyclictest0-21swapper/123:09:581
1766999135,48cyclictest0-21swapper/123:09:581
176699913,34cyclictest12037-21ssh21:14:281
176699913,34cyclictest12037-21ssh21:14:281
176699913,32cyclictest9696-21basename22:04:251
176699912,30cyclictest30580-21latency_hist00:24:271
176699912,30cyclictest30580-21latency_hist00:24:271
1766999058,21cyclictest31756-21timerwakeupswit00:25:031
1766999037,46cyclictest0-21swapper/122:50:001
176699903,30cyclictest408-21fschecks_time23:34:481
176699902,35cyclictest10884-21latency_hist22:59:281
176699902,35cyclictest10884-21latency_hist22:59:281
176699902,33cyclictest29962-21latency_hist20:34:281
1773998947,9cyclictest181rcu_preempt23:14:392
1773998947,9cyclictest181rcu_preempt23:14:392
176699893,38cyclictest2128-21fschecks_count19:09:471
176699893,34cyclictest19240-21switchtime20:00:001
176699893,33cyclictest4445-21/usr/sbin/munin00:34:481
176699893,33cyclictest4445-21/usr/sbin/munin00:34:481
176699893,33cyclictest346-21runrttasks19:19:271
176699893,32cyclictest15221-21/usr/sbin/munin21:19:551
176699893,29cyclictest27467-21taskset21:39:431
176699893,29cyclictest27467-21taskset21:39:431
1766998932,13cyclictest12889-21ssh22:09:281
176699892,35cyclictest6637-21cut21:00:001
176699892,32cyclictest31589-21latency_hist20:39:271
176699883,37cyclictest537-21grep20:41:021
176699883,36cyclictest22079-21/usr/sbin/munin22:24:451
176699883,36cyclictest16151-21cpuspeed22:14:431
176699883,35cyclictest5734-21latency_hist20:59:261
176699883,35cyclictest5734-21latency_hist20:59:261
176699883,35cyclictest13774-21ls19:44:441
176699883,33cyclictest23876-21fschecks_time20:14:491
176699883,33cyclictest23876-21fschecks_time20:14:491
176699883,33cyclictest17374-21seq19:54:551
176699883,31cyclictest28697-21ntp_kernel_pll_22:34:531
176699883,31cyclictest25563-21latency22:29:501
1766998829,12cyclictest0-21swapper/123:24:531
176699882,39cyclictest25230-21/usr/sbin/munin20:19:571
176699882,38cyclictest19846-21mailstats22:19:591
176699882,30cyclictest840-21/usr/sbin/munin21:50:011
176699882,28cyclictest16714-21latency_hist19:54:271
176699873,33cyclictest12470-21fschecks_time21:14:471
176699873,28cyclictest20590-21ssh00:07:261
176699872,34cyclictest9666-21wc19:30:021
176699872,33cyclictest3693-21/usr/sbin/munin23:39:531
176699872,33cyclictest3693-21/usr/sbin/munin23:39:531
176699872,32cyclictest8908-21latency_hist21:09:281
176699872,30cyclictest4219-21latency_hist20:54:281
148928715,35sleep00-21swapper/019:07:530
176699863,41cyclictest31232-21cat21:45:041
176699862,30cyclictest8675-21latency_hist19:29:281
176699854,33cyclictest1740-21mailstats20:45:071
176699853,33cyclictest2152-21sendmail_mailqu00:30:021
176699853,33cyclictest11944-21timerandwakeup23:00:011
176699853,29cyclictest22326-21if_eth020:09:491
1766998511,28cyclictest24713-21kernelversion21:34:521
1762998568,11cyclictest24560-21fschecks_time21:34:480
1773998464,14cyclictest8918-21diskmemload00:34:592
1773998464,14cyclictest8918-21diskmemload00:34:592
176699843,29cyclictest31159-21/usr/sbin/munin22:39:361
1762998463,14cyclictest913-21latency_hist20:44:270
1762998453,9cyclictest181rcu_preempt22:49:540
1773998362,13cyclictest25792-21processes22:29:572
1773998342,10cyclictest181rcu_preempt20:09:302
1773998339,11cyclictest181rcu_preempt19:49:582
1773998334,17cyclictest181rcu_preempt23:33:592
1762998341,21cyclictest181rcu_preempt22:39:270
1773998243,10cyclictest1749-21cyclictest19:25:002
1773998148,8cyclictest181rcu_preempt22:48:452
1773998148,8cyclictest181rcu_preempt22:48:452
1773998145,11cyclictest181rcu_preempt21:09:362
1773998145,11cyclictest181rcu_preempt21:09:362
176699813,25cyclictest22344-21sed21:30:041
176699813,25cyclictest22344-21sed21:30:041
1762998155,14cyclictest181rcu_preempt21:54:180
1773998064,10cyclictest8918-21diskmemload00:24:582
1773998063,10cyclictest28189-21ssh22:34:402
1773998061,14cyclictest392-21cron21:49:242
1762998061,15cyclictest17754-21kworker/u19:1+events_unbound19:44:250
176299803,55cyclictest15559-21apt-get23:59:450
1762997960,13cyclictest8918-21diskmemload21:39:480
1762997960,13cyclictest8918-21diskmemload21:39:480
176299792,62cyclictest2407-21taskset22:45:240
176299792,62cyclictest2407-21taskset22:45:240
1773997856,7cyclictest181rcu_preempt21:39:472
1773997856,7cyclictest181rcu_preempt21:39:472
1773997843,15cyclictest181rcu_preempt00:15:272
1773997839,11cyclictest181rcu_preempt00:00:292
177399783,38cyclictest39-21ksoftirqd/220:29:382
1762997860,11cyclictest346-21runrttasks22:10:030
1773997740,11cyclictest181rcu_preempt19:45:012
1773997739,8cyclictest181rcu_preempt21:04:572
1762997752,15cyclictest363-21snmpd21:14:280
1762997752,15cyclictest363-21snmpd21:14:280
1762997750,9cyclictest181rcu_preempt23:50:030
1762997750,18cyclictest6046-21latency_hist23:44:280
1762997750,18cyclictest6046-21latency_hist23:44:280
1773997661,9cyclictest2399-21kworker/u17:3-xprtiod23:44:282
1773997661,9cyclictest2399-21kworker/u17:3-xprtiod23:44:282
1773997661,10cyclictest27118-21ntp_offset23:24:552
1773997659,11cyclictest15107-40gzip23:59:272
1773997658,12cyclictest320-21dbus-daemon23:24:262
1773997656,14cyclictest31409-21apt-get20:39:292
1773997645,22cyclictest26579-21modprobe20:24:242
1773997642,11cyclictest18148-21kworker/u18:1-writeback23:39:282
1773997638,10cyclictest181rcu_preempt22:19:582
1773997627,14cyclictest181rcu_preempt22:08:492
1762997658,11cyclictest1-21systemd19:59:250
1762997640,23cyclictest24904-21kworker/u17:1+rpciod23:59:120
176299763,59cyclictest20387-21apt-get20:04:400
1773997559,10cyclictest4891-21unixbench_multi19:15:032
1773997542,10cyclictest181rcu_preempt21:54:222
1773997537,7cyclictest181rcu_preempt20:04:562
1773997535,13cyclictest181rcu_preempt19:29:432
1773997533,15cyclictest181rcu_preempt19:20:042
1762997558,10cyclictest8918-21diskmemload22:24:360
1762997550,8cyclictest181rcu_preempt00:11:360
1762997550,8cyclictest181rcu_preempt00:11:360
162827512,40sleep21630-21munin-run19:09:262
1773997445,12cyclictest181rcu_preempt00:10:002
1773997445,12cyclictest181rcu_preempt00:10:002
1773997440,8cyclictest181rcu_preempt00:30:042
1773997439,12cyclictest181rcu_preempt20:59:422
1773997430,16cyclictest181rcu_preempt23:08:472
1762997447,8cyclictest181rcu_preempt20:55:000
1762997447,8cyclictest181rcu_preempt20:55:000
1762997442,8cyclictest181rcu_preempt19:50:040
1762997437,10cyclictest181rcu_preempt22:00:400
1773997346,9cyclictest181rcu_preempt22:09:432
1773997342,10cyclictest181rcu_preempt23:01:332
1773997340,9cyclictest181rcu_preempt21:24:282
1773997337,16cyclictest181rcu_preempt22:24:452
177399733,36cyclictest10807-21wc22:59:272
177399733,36cyclictest10807-21wc22:59:272
1762997360,9cyclictest1749-21cyclictest00:36:450
1762997360,9cyclictest1749-21cyclictest00:36:450
1762997358,10cyclictest29345-21kworker/u19:0-rpciod00:33:200
1762997321,10cyclictest211rcuc/020:44:430
1773997255,11cyclictest9822-21sed22:04:282
1773997252,13cyclictest28344-21/usr/sbin/munin00:19:542
1773997252,13cyclictest28344-21/usr/sbin/munin00:19:542
1773997242,8cyclictest23981-21ssh21:34:262
1773997242,8cyclictest23981-21ssh21:34:262
1773997241,8cyclictest181rcu_preempt23:51:412
1773997239,10cyclictest181rcu_preempt22:16:512
1773997223,10cyclictest181rcu_preempt22:49:492
1762997256,10cyclictest11980-21ssh22:07:280
1762997254,12cyclictest17579-21ssh23:09:500
1762997254,12cyclictest17579-21ssh23:09:500
1762997251,15cyclictest1422-21cut22:44:270
1762997242,22cyclictest24544-21grep20:15:040
1762997242,22cyclictest24544-21grep20:15:040
1762997237,26cyclictest0-21swapper/000:24:280
1762997237,26cyclictest0-21swapper/000:24:280
1762997235,8cyclictest181rcu_preempt19:35:010
177399713,31cyclictest6623-21tr21:59:252
177399713,31cyclictest6623-21tr21:59:252
1773997132,17cyclictest181rcu_preempt23:09:292
1773997132,17cyclictest181rcu_preempt23:09:292
177399712,53cyclictest19305-21unixbench_singl21:25:032
1762997156,10cyclictest2161-21ssh23:37:440
1762997155,10cyclictest1-21systemd22:19:480
1762997147,8cyclictest181rcu_preempt00:04:520
1762997145,20cyclictest5030-21ssh21:55:080
1762997145,20cyclictest5030-21ssh21:55:080
1762997144,9cyclictest181rcu_preempt23:44:360
1762997142,20cyclictest27796-21modprobe22:34:250
1762997139,10cyclictest181rcu_preempt00:17:570
1762997137,23cyclictest0-21swapper/022:59:280
1762997137,23cyclictest0-21swapper/022:59:280
1773997054,11cyclictest1749-21cyclictest20:39:292
1773997033,18cyclictest181rcu_preempt23:44:532
1762997055,9cyclictest24119-21latency_hist21:34:280
1762997055,9cyclictest24119-21latency_hist21:34:280
1762997052,13cyclictest22100-21cstates20:09:430
1762997049,14cyclictest30617-21fschecks_time21:44:500
1762997042,8cyclictest181rcu_preempt21:19:420
1773996942,8cyclictest181rcu_preempt21:14:542
1773996927,13cyclictest181rcu_preempt19:54:392
1762996951,9cyclictest0-21swapper/020:49:490
1762996946,15cyclictest25586-21ssh23:22:470
176299693,38cyclictest2933-21vmstat19:10:040
176299693,31cyclictest26484-21/usr/sbin/munin23:24:490
1762996932,10cyclictest19869-21sh21:26:260
1762996923,23cyclictest28985-21ntpq20:29:560
1773996838,9cyclictest181rcu_preempt19:39:272
1773996835,8cyclictest181rcu_preempt19:09:592
1773996831,13cyclictest181rcu_preempt19:59:462
1762996854,10cyclictest24844-21kworker/u19:3-flush-0:3223:29:570
1762996850,9cyclictest0-21swapper/020:24:420
1762996847,7cyclictest181rcu_preempt23:01:170
1762996843,8cyclictest181rcu_preempt22:17:050
1762996840,9cyclictest181rcu_preempt23:18:490
1762996840,9cyclictest181rcu_preempt23:18:490
1762996830,9cyclictest181rcu_preempt21:04:590
1773996735,8cyclictest181rcu_preempt20:51:162
1762996739,20cyclictest12473-21fschecks_time21:14:470
1762996738,9cyclictest181rcu_preempt20:00:010
1762996735,8cyclictest181rcu_preempt19:44:500
176299673,27cyclictest31403-21modprobe20:39:250
1773996649,12cyclictest0-21swapper/220:19:282
1773996649,12cyclictest0-21swapper/220:19:282
1773996639,9cyclictest181rcu_preempt20:25:012
1762996652,8cyclictest1180-21diskmemload00:29:260
1773996549,9cyclictest381ktimers/219:39:502
1762996550,8cyclictest0-21swapper/021:04:270
1762996534,12cyclictest0-21swapper/020:23:170
1762996531,9cyclictest181rcu_preempt23:05:550
1762996447,8cyclictest0-21swapper/019:24:480
1762996432,10cyclictest181rcu_preempt19:20:010
177899633,43cyclictest7180-21chrt21:59:483
177399633,33cyclictest19089-21sed00:04:532
1773996332,13cyclictest181rcu_preempt21:39:032
1762996341,7cyclictest181rcu_preempt19:29:510
177899624,39cyclictest12130-21chrt23:54:283
1762996137,8cyclictest181rcu_preempt19:14:300
177899604,37cyclictest27246-21chrt21:39:303
177899604,37cyclictest27246-21chrt21:39:303
177899583,40cyclictest21153-21taskset21:29:283
1778995811,28cyclictest0-21swapper/320:40:423
177899573,42cyclictest4334-21chrt00:34:283
177899572,42cyclictest17394-21taskset22:15:443
177899562,43cyclictest26583-21taskset00:17:093
177899554,39cyclictest1900-21taskset22:44:293
177899554,37cyclictest32507-21chrt21:48:053
1778995541,9cyclictest451rcuc/322:04:433
177899553,38cyclictest24423-21chrt21:34:433
177899552,44cyclictest13721-21taskset23:56:023
177899552,40cyclictest21116-21taskset00:08:513
177899544,37cyclictest12557-21taskset19:39:553
177899544,33cyclictest23489-21chrt23:19:353
177899543,40cyclictest27050-21taskset20:24:473
177899543,38cyclictest32640-21chrt23:34:433
177899542,41cyclictest29844-21taskset23:29:483
177899542,38cyclictest31536-21chrt00:25:003
1778995410,28cyclictest0-21swapper/300:00:113
177899533,41cyclictest22719-21taskset23:18:173
177899533,41cyclictest22719-21taskset23:18:173
177899533,37cyclictest17830-21taskset21:24:093
177899532,38cyclictest1044-21chrt21:49:433
177899532,35cyclictest11364-21chrt19:35:013
177899524,36cyclictest19875-21chrt23:14:243
177899524,36cyclictest19875-21chrt23:14:243
177899524,34cyclictest13616-21chrt22:09:573
177899523,37cyclictest2314-21chrt22:45:183
177899523,37cyclictest2314-21chrt22:45:183
177899523,36cyclictest28007-21taskset22:34:283
177899514,36cyclictest14979-21chrt21:19:273
177899514,34cyclictest28618-21chrt00:20:013
177899514,34cyclictest28618-21chrt00:20:013
177899513,38cyclictest21823-21taskset20:09:283
177899513,37cyclictest3165-21chrt20:49:543
177899513,35cyclictest14294-21chrt23:04:443
1778995130,14cyclictest451rcuc/323:39:543
1778995130,14cyclictest451rcuc/323:39:543
177899512,38cyclictest6685-21chrt21:59:273
177899512,38cyclictest6685-21chrt21:59:273
177899512,36cyclictest28905-21taskset20:29:533
177899504,35cyclictest27508-21chrt23:25:023
177899503,34cyclictest21454-21chrt21:29:453
177899503,34cyclictest21454-21chrt21:29:453
177899503,33cyclictest6764-21taskset00:38:173
177899503,33cyclictest6764-21taskset00:38:173
177899493,35cyclictest10514-21chrt22:58:523
177899493,35cyclictest10514-21chrt22:58:523
177899493,34cyclictest31791-21taskset20:39:293
177899493,33cyclictest2452-21chrt20:49:253
177899493,32cyclictest5723-21chrt20:59:253
177899493,32cyclictest5723-21chrt20:59:253
177899493,32cyclictest30939-21taskset22:39:273
177899484,31cyclictest13963-21chrt23:04:283
177899483,34cyclictest22434-21chrt22:24:483
177899483,34cyclictest20435-21chrt22:20:433
177899483,33cyclictest7455-21chrt21:04:283
177899482,33cyclictest1954-21chrt19:09:423
1778994734,8cyclictest451rcuc/320:01:353
177899473,35cyclictest9480-21chrt19:29:593
177899473,33cyclictest9893-21chrt21:09:593
177899473,33cyclictest9893-21chrt21:09:593
177899473,33cyclictest25107-21taskset20:19:283
177899473,33cyclictest25107-21taskset20:19:283
177899472,32cyclictest15088-21chrt19:49:273
1778994633,9cyclictest451rcuc/319:54:223
177899463,34cyclictest8024-21chrt23:47:063
177899463,33cyclictest22511-21chrt00:10:003
177899463,33cyclictest22511-21chrt00:10:003
177899463,31cyclictest7231-21chrt19:24:363
177899452,33cyclictest6455-21chrt22:51:023
177899444,30cyclictest4379-21chrt19:14:533
177899443,32cyclictest25838-21taskset20:19:573
177899443,31cyclictest22282-21chrt20:09:483
177899423,22cyclictest7853-21chrt21:04:483
1778994224,13cyclictest451rcuc/319:56:293
177899412,29cyclictest6204-21taskset19:19:593
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional