You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-26 - 05:56
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Fri Dec 26, 2025 00:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17139219711,85sleep117177-21switchtime19:06:241
17139219711,85sleep117177-21switchtime19:06:241
1788599126109,9cyclictest0-21swapper/123:50:071
178859910737,12cyclictest0-21swapper/122:45:121
178859910733,16cyclictest0-21swapper/123:40:071
17885991053,35cyclictest20859-21modprobe20:54:451
178859910435,14cyclictest0-21swapper/121:25:041
17885991043,39cyclictest11879-21grep20:25:221
178859910348,15cyclictest16669-21timerandwakeup23:36:241
178729910384,13cyclictest30214-40install23:59:440
178989910118,57cyclictest0-21swapper/323:50:053
178939910017,48cyclictest39-21ksoftirqd/223:28:452
1789899993,70cyclictest12846-21chrt20:30:013
1789899993,70cyclictest12846-21chrt20:30:013
1788599994,40cyclictest9832-21latency_hist00:19:471
1788599993,36cyclictest13353-21ntp_kernel_err20:30:411
1788599993,36cyclictest13353-21ntp_kernel_err20:30:411
1788599988,43cyclictest14682-21apt-get20:34:581
17872999820,27cyclictest3026-21apt-get22:20:010
1789399953,60cyclictest26793-21apt-get19:35:022
17893999418,33cyclictest39-21ksoftirqd/223:55:072
173262948,59sleep016380-21/usr/sbin/munin19:06:270
173262948,59sleep016380-21/usr/sbin/munin19:06:270
17893999318,30cyclictest39-21ksoftirqd/222:49:462
17893999219,38cyclictest39-21ksoftirqd/222:35:072
17893999123,37cyclictest39-21ksoftirqd/222:10:182
17872999110,36cyclictest0-21swapper/021:39:590
1787299906,61cyclictest12780-21/usr/sbin/munin23:30:140
17893998967,15cyclictest11698-21ssh22:34:472
17893998967,15cyclictest11698-21ssh22:34:472
1789399892,57cyclictest39-21ksoftirqd/223:12:192
1787299890,60cyclictest0-21swapper/000:29:590
17893998849,17cyclictest181rcu_preempt22:16:222
17893998849,17cyclictest181rcu_preempt22:16:222
17893998819,36cyclictest39-21ksoftirqd/200:13:292
17885998860,19cyclictest25729-21diskmemload22:00:131
17872998862,19cyclictest24037-21apt-get23:49:490
17893998643,8cyclictest181rcu_preempt19:43:372
1787299863,67cyclictest363-21snmpd23:35:060
1788599854,38cyclictest346-21runrttasks21:59:441
1788599854,38cyclictest346-21runrttasks21:59:441
1787299858,68cyclictest23887-21ssh22:00:520
1789399843,46cyclictest39-21ksoftirqd/219:26:252
1789399843,46cyclictest39-21ksoftirqd/219:26:252
1789399832,43cyclictest39-21ksoftirqd/200:18:342
1787299838,33cyclictest24605-21apt-get23:50:090
17872998354,21cyclictest17558-21sh21:50:070
1789399828,42cyclictest39-21ksoftirqd/200:22:362
1789399828,40cyclictest39-21ksoftirqd/220:44:322
17893998262,14cyclictest25729-21diskmemload22:28:092
17893998257,18cyclictest29341-21fschecks_time21:15:102
17893998251,22cyclictest19196-21kworker/u19:1+nfsiod22:54:482
17893998251,22cyclictest19196-21kworker/u19:1+nfsiod22:54:482
1789399822,48cyclictest39-21ksoftirqd/220:19:282
1789399822,48cyclictest39-21ksoftirqd/220:19:282
17872998261,15cyclictest27394-21netstat23:00:140
17872998261,15cyclictest27394-21netstat23:00:140
17893998160,16cyclictest17863-21cyclictest21:40:502
17893998142,13cyclictest181rcu_preempt21:35:202
17893998142,13cyclictest181rcu_preempt21:35:202
17893998138,22cyclictest13515-21ntpq00:25:132
17893998138,22cyclictest13515-21ntpq00:25:132
17885998157,16cyclictest15247-21apt-config23:34:481
17885998147,10cyclictest181rcu_preempt23:44:581
17872998166,10cyclictest17863-21cyclictest20:05:220
17872998144,26cyclictest4019-21cron20:04:440
17872998144,26cyclictest4019-21cron20:04:440
17872998124,30cyclictest0-21swapper/023:20:040
17872998122,41cyclictest0-21swapper/023:29:460
17872998116,24cyclictest25708-21cron21:09:440
17893998058,16cyclictest7576-21latency_hist20:14:472
17893998047,9cyclictest181rcu_preempt22:03:512
17893998012,41cyclictest39-21ksoftirqd/220:19:572
17893998012,41cyclictest39-21ksoftirqd/220:19:572
17885998065,9cyclictest27248-21sendmail_mailqu22:05:201
17885998017,43cyclictest4526-21apt-get20:04:591
17872998056,16cyclictest171-21jbd2/mmcblk2p2-800:14:500
1787299803,47cyclictest12678-21apt-config20:29:480
17872998019,30cyclictest201-21systemd-journal23:44:440
17893997947,14cyclictest181rcu_preempt21:00:172
17893997942,8cyclictest181rcu_preempt22:21:542
17893997926,13cyclictest181rcu_preempt20:29:132
17893997919,35cyclictest39-21ksoftirqd/222:55:112
1787299797,28cyclictest14178-21apt-get20:34:580
1787299796,30cyclictest3492-21/usr/sbin/munin23:15:030
1787299796,30cyclictest3492-21/usr/sbin/munin23:15:030
17872997955,16cyclictest20622-21ntp_offset19:15:160
1640327911,41sleep215895-21apt-get19:04:582
1640327911,41sleep215895-21apt-get19:04:582
1789899784,49cyclictest0-21swapper/323:36:103
17893997863,11cyclictest17863-21cyclictest20:00:102
17893997863,11cyclictest17863-21cyclictest20:00:102
17893997832,12cyclictest181rcu_preempt23:01:402
17893997832,12cyclictest181rcu_preempt23:01:402
17893997828,12cyclictest381ktimers/221:45:212
17893997821,34cyclictest39-21ksoftirqd/221:50:222
17893997812,28cyclictest39-21ksoftirqd/221:28:312
17885997851,17cyclictest24032-21kworker/u17:0-xprtiod00:20:141
17872997857,16cyclictest17863-21cyclictest00:37:490
17872997856,15cyclictest17863-21cyclictest21:27:540
1789399774,46cyclictest39-21ksoftirqd/222:08:112
17893997733,21cyclictest39-21ksoftirqd/200:06:532
1789399771,45cyclictest39-21ksoftirqd/221:21:302
1789399771,45cyclictest39-21ksoftirqd/221:21:302
17893997710,43cyclictest39-21ksoftirqd/219:23:132
17885997762,10cyclictest17863-21cyclictest19:56:081
17885997762,10cyclictest17863-21cyclictest19:56:081
17885997746,11cyclictest0-21swapper/120:04:471
17885997746,11cyclictest0-21swapper/120:04:471
1788599774,43cyclictest30373-21latency_hist23:59:481
1787299775,24cyclictest30963-21/usr/sbin/munin00:00:100
17872997751,17cyclictest320-21dbus-daemon21:34:450
17872997735,7cyclictest181rcu_preempt19:26:330
17872997735,7cyclictest181rcu_preempt19:26:330
17872997719,16cyclictest0-21swapper/022:15:110
17872997719,16cyclictest0-21swapper/022:15:110
17872997718,33cyclictest16296-21/usr/sbin/munin20:40:090
17893997659,11cyclictest24395-21cpu21:05:012
1789399763,57cyclictest12835-21apt-get20:29:592
1789399763,57cyclictest12835-21apt-get20:29:592
1789399763,38cyclictest39-21ksoftirqd/223:16:242
1789399763,38cyclictest39-21ksoftirqd/223:16:242
17893997631,17cyclictest181rcu_preempt20:48:252
17893997630,16cyclictest181rcu_preempt21:30:402
17893997628,11cyclictest181rcu_preempt20:37:032
1789399762,45cyclictest39-21ksoftirqd/200:01:342
1789399762,38cyclictest39-21ksoftirqd/220:57:152
1789399762,38cyclictest39-21ksoftirqd/220:57:152
17893997610,34cyclictest39-21ksoftirqd/223:45:512
17885997640,13cyclictest0-21swapper/100:00:211
17872997638,20cyclictest26854-21kworker/u19:1+flush-179:820:30:120
17872997638,20cyclictest26854-21kworker/u19:1+flush-179:820:30:120
1787299763,38cyclictest30271-21latency23:05:110
17872997633,27cyclictest26818-21seq19:35:140
17872997620,31cyclictest0-21swapper/022:56:570
17893997560,11cyclictest17863-21cyclictest19:15:042
17893997552,17cyclictest18642-21latency_hist00:34:462
17872997553,17cyclictest17863-21cyclictest00:10:180
17872997528,34cyclictest577-21fschecks_count23:10:090
17872997528,13cyclictest17158-21latency_hist21:49:470
17872997511,27cyclictest18131-21kworker/u17:1+rpciod22:50:130
17872997511,27cyclictest18131-21kworker/u17:1+rpciod22:50:130
17872997510,45cyclictest1045-21modprobe19:55:170
17872997510,45cyclictest1045-21modprobe19:55:170
1789399746,57cyclictest15433-21apt-get23:35:262
17893997443,11cyclictest181rcu_preempt20:08:022
1789399743,37cyclictest39-21ksoftirqd/223:20:422
1789399743,37cyclictest39-21ksoftirqd/223:06:192
1789399743,34cyclictest39-21ksoftirqd/219:30:382
1789399743,20cyclictest21534-21modprobe00:39:442
1789399742,23cyclictest39-21ksoftirqd/220:50:212
1788599744,29cyclictest16374-21sort20:40:021
1788599743,41cyclictest15109-21ssh22:40:011
1788599743,41cyclictest15109-21ssh22:40:011
17885997434,11cyclictest181rcu_preempt23:14:471
17885997417,14cyclictest17863-21cyclictest20:20:461
17885997417,14cyclictest17863-21cyclictest20:20:461
17885997411,13cyclictest181rcu_preempt00:39:431
1787299743,23cyclictest12960-21/usr/sbin/munin00:24:560
1787299743,23cyclictest12960-21/usr/sbin/munin00:24:560
17893997350,12cyclictest18417-21/usr/sbin/munin23:40:212
17893997330,8cyclictest181rcu_preempt22:44:442
17893997330,8cyclictest181rcu_preempt22:44:442
17893997330,11cyclictest181rcu_preempt19:13:122
1788599738,31cyclictest19781-21cpuspeed19:14:521
17885997354,10cyclictest5836-21ssh21:30:081
17885997333,18cyclictest181rcu_preempt22:29:391
1787299738,26cyclictest28606-21ssh21:14:300
1787299738,26cyclictest28606-21ssh21:14:300
1787299735,27cyclictest20317-21/usr/sbin/munin21:54:560
1787299735,27cyclictest20317-21/usr/sbin/munin21:54:560
17893997238,15cyclictest181rcu_preempt21:13:292
17893997238,15cyclictest181rcu_preempt21:13:292
17893997228,13cyclictest181rcu_preempt19:57:562
17893997228,13cyclictest181rcu_preempt19:57:562
1788599723,49cyclictest11918-21latency_hist23:29:471
1787299726,26cyclictest29689-21taskset19:43:540
1787299724,56cyclictest26294-21idleruntime-cro19:34:440
1787299723,35cyclictest32548-21netstat21:20:130
1787299723,35cyclictest32548-21netstat21:20:130
1789399719,28cyclictest25154-21latency23:50:262
17893997122,20cyclictest39-21ksoftirqd/221:55:182
17893997122,20cyclictest39-21ksoftirqd/221:55:182
17893997119,8cyclictest381ktimers/219:45:142
17893997119,8cyclictest381ktimers/219:45:142
17893997119,36cyclictest15307-21date23:34:492
17885997151,14cyclictest9256-21switchtime21:35:191
17885997151,14cyclictest9256-21switchtime21:35:191
17885997148,14cyclictest31-21ksoftirqd/119:44:481
17885997134,9cyclictest181rcu_preempt22:35:221
1787299719,26cyclictest11692-21ssh00:22:040
1787299717,26cyclictest346-21runrttasks21:00:060
1787299717,25cyclictest31540-21ssh22:13:050
1787299715,30cyclictest30692-21sendmail_mailtr19:45:260
1787299715,30cyclictest30692-21sendmail_mailtr19:45:260
1787299714,49cyclictest9323-21latency_hist20:19:480
1787299714,49cyclictest9323-21latency_hist20:19:480
1787299714,28cyclictest23156-21wc19:24:470
1787299713,29cyclictest566-21apt-config19:54:450
17872997120,25cyclictest20676-21chrt20:52:250
17872997113,23cyclictest0-21swapper/022:25:110
17872997113,22cyclictest29268-21ssh22:09:470
17872997112,26cyclictest346-21runrttasks22:29:580
17872997112,26cyclictest346-21runrttasks22:29:580
17872997110,23cyclictest0-21swapper/022:44:580
17893997018,26cyclictest39-21ksoftirqd/219:50:112
1788599705,47cyclictest7474-21latency_hist20:14:451
17885997049,14cyclictest24497-21kworker/u19:2-rpciod22:55:461
17885997046,15cyclictest0-21swapper/121:50:141
1787299709,23cyclictest1711-21ntp_states00:05:180
1787299705,27cyclictest9892-21ntpq20:20:130
1787299705,27cyclictest9892-21ntpq20:20:130
1787299704,27cyclictest9008-21/usr/sbin/munin21:35:120
1787299704,27cyclictest9008-21/usr/sbin/munin21:35:120
17872997038,22cyclictest12671-21systemctl22:35:190
17872997018,20cyclictest29517-21latency21:15:140
17885996950,13cyclictest31477-21latency_hist19:49:481
17885996950,13cyclictest31477-21latency_hist19:49:481
17885996946,14cyclictest0-21swapper/119:25:211
17885996946,14cyclictest0-21swapper/119:25:211
1788599694,44cyclictest32230-21latency_hist22:14:471
17885996933,12cyclictest181rcu_preempt00:10:151
1788599693,22cyclictest31-21ksoftirqd/121:14:151
1788599693,22cyclictest31-21ksoftirqd/121:14:151
17885996930,16cyclictest181rcu_preempt00:25:191
17885996930,16cyclictest181rcu_preempt00:25:191
17885996923,14cyclictest8097-21uname20:15:131
17885996923,14cyclictest8097-21uname20:15:131
17885996919,32cyclictest27361-21ntp_kernel_pll_19:36:031
17885996916,10cyclictest181rcu_preempt21:40:081
17872996947,15cyclictest6363-21fschecks_time20:10:080
1787299693,43cyclictest22669-21latency_hist20:59:480
1787299693,43cyclictest22669-21latency_hist20:59:480
17885996852,9cyclictest0-21swapper/121:15:251
17885996845,16cyclictest16523-21sh00:30:151
17885996835,15cyclictest21653-21ntpq20:55:151
17885996835,15cyclictest21653-21ntpq20:55:151
1788599683,27cyclictest32422-21ssh23:09:391
17885996831,10cyclictest181rcu_preempt23:20:071
17885996817,16cyclictest181rcu_preempt22:50:041
17885996817,16cyclictest181rcu_preempt22:50:041
1787299684,26cyclictest18750-21/usr/sbin/munin19:10:180
17885996748,14cyclictest16481-21kworker/u19:2+events_unbound19:14:441
17885996747,10cyclictest0-21swapper/122:29:561
17885996747,10cyclictest0-21swapper/122:29:561
17885996736,13cyclictest23199-21latency_hist19:24:481
1788599673,14cyclictest17932-21apt-get20:45:011
17872996720,13cyclictest15785-21cat22:40:180
17872996720,13cyclictest15785-21cat22:40:180
17872996717,18cyclictest18027-21seq20:45:050
111912670,7sleep30-21swapper/320:25:073
17885996644,12cyclictest0-21swapper/122:15:231
17885996644,12cyclictest0-21swapper/122:15:231
17885996643,7cyclictest0-21swapper/121:05:191
20962650,9sleep30-21swapper/323:12:143
1788599653,35cyclictest15488-21users21:45:241
1788599652,26cyclictest3327-21ssh22:20:081
17885996448,10cyclictest3256-21ssh00:08:281
17885996447,11cyclictest201-21systemd-journal21:04:441
17885996446,13cyclictest0-21swapper/121:20:111
17885996446,13cyclictest0-21swapper/121:20:111
17885996441,17cyclictest4243-21ssh23:15:201
17885996441,17cyclictest4243-21ssh23:15:201
17885996421,32cyclictest32154-21ntp_kernel_pll_19:50:231
1788599632,40cyclictest26273-21cron19:34:441
1788599613,14cyclictest26850-21sh22:59:571
1788599613,14cyclictest26850-21sh22:59:571
1789899593,43cyclictest14211-21taskset23:32:523
1789899552,42cyclictest20864-21taskset21:55:133
1789899552,42cyclictest20864-21taskset21:55:133
1789899543,37cyclictest11567-21taskset22:34:373
1789899543,37cyclictest11567-21taskset22:34:373
1789899533,37cyclictest24548-21taskset22:01:133
1789899533,35cyclictest25447-21chrt19:30:123
1789899523,38cyclictest751-21chrt21:20:363
1789899523,38cyclictest751-21chrt21:20:363
1789899523,32cyclictest7249-21chrt21:32:153
17898995214,27cyclictest0-21swapper/300:04:453
1789899513,35cyclictest29394-21chrt23:04:283
1789899513,35cyclictest29394-21chrt23:04:283
1789899513,34cyclictest28335-21taskset22:07:253
1789899512,38cyclictest24297-21chrt21:04:473
1789899512,34cyclictest21266-21chrt23:44:473
17898995111,26cyclictest0-21swapper/319:35:023
1789899508,30cyclictest0-21swapper/320:05:043
1789899503,33cyclictest24136-21chrt23:49:463
1789899503,33cyclictest13561-21taskset21:43:173
1789899503,32cyclictest29719-21chrt23:58:253
1789899502,37cyclictest18498-21chrt22:45:153
193022493,6sleep30-21swapper/300:35:123
1789899493,36cyclictest31169-21taskset21:18:213
1789899493,34cyclictest8617-21chrt23:23:493
1789899493,34cyclictest32195-21chrt22:14:473
1789899493,26cyclictest0-21swapper/319:55:313
1789899493,26cyclictest0-21swapper/319:55:313
1789899492,38cyclictest2737-21chrt22:19:453
1789899492,38cyclictest2737-21chrt22:19:453
1789899492,36cyclictest8461-21chrt20:15:233
1789899492,36cyclictest8461-21chrt20:15:233
1789899492,34cyclictest22685-21chrt22:52:243
1789899492,34cyclictest22685-21chrt22:52:243
1789899492,31cyclictest14828-21chrt22:39:483
1789899492,31cyclictest11143-21chrt21:39:453
1789899492,31cyclictest11143-21chrt21:39:453
1789899492,30cyclictest18772-21chrt00:34:473
1789899487,29cyclictest16353-21chrt21:47:513
1789899486,32cyclictest4546-21taskset22:21:363
1789899483,34cyclictest17236-21chrt22:43:453
1789899483,34cyclictest17236-21chrt22:43:453
1789899483,32cyclictest4277-21taskset00:10:083
1789899483,32cyclictest29684-21chrt19:43:513
1789899482,32cyclictest9993-21taskset00:19:563
1789899482,32cyclictest26397-21taskset21:10:103
1789899482,32cyclictest26397-21taskset21:10:103
1789899482,31cyclictest9224-21chrt23:24:503
1789899473,35cyclictest8719-21taskset22:29:463
1789899473,35cyclictest4040-21chrt23:15:183
1789899473,35cyclictest4040-21chrt23:15:183
1789899473,31cyclictest16063-21taskset20:39:453
1789899472,34cyclictest3941-21chrt00:09:473
1789899472,31cyclictest6376-21chrt20:10:083
1789899472,30cyclictest24876-21chrt22:55:243
1789899463,33cyclictest18448-21taskset21:50:233
1789899462,33cyclictest20960-21chrt20:54:473
1789899462,33cyclictest18949-21chrt19:10:203
1789899462,30cyclictest23722-21chrt19:25:103
1789899462,30cyclictest23722-21chrt19:25:103
1789899462,30cyclictest17305-21chrt20:40:263
1789899462,29cyclictest927-21chrt19:54:493
1789899453,33cyclictest22648-21chrt19:20:203
1789899453,30cyclictest4013-21chrt21:26:273
1789899453,30cyclictest10775-21chrt20:24:463
1789899453,30cyclictest10775-21chrt20:24:463
1789899452,34cyclictest15328-21chrt00:29:003
1789899452,34cyclictest15328-21chrt00:29:003
164142458,20sleep30-21swapper/319:05:073
164142458,20sleep30-21swapper/319:05:073
1789899443,32cyclictest22118-21taskset20:55:243
1789899443,32cyclictest22118-21taskset20:55:243
1789899442,29cyclictest31465-21chrt23:06:553
1789899442,27cyclictest19920-21chrt19:14:543
1789899434,29cyclictest17930-21taskset20:44:593
1789899432,32cyclictest4068-21chrt20:04:453
1789899432,32cyclictest4068-21chrt20:04:453
1789899423,25cyclictest7096-21chrt00:15:023
1789899413,28cyclictest25774-21chrt21:09:453
1789899383,24cyclictest30038-21taskset19:44:573
1789899383,24cyclictest30038-21taskset19:44:573
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional