You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-29 - 06:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Mon Dec 29, 2025 00:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
263689911237,22cyclictest9503-21cut19:54:560
26368991123,44cyclictest7736-21netstat22:15:010
26368991123,44cyclictest7736-21netstat22:15:010
26368991103,22cyclictest24489-21/usr/sbin/munin00:34:460
263779910835,35cyclictest5826-21sh19:44:321
263779910835,35cyclictest5826-21sh19:44:321
26368991083,40cyclictest12718-21latency_hist23:19:370
25415210610,71sleep225447-21ntp_offset19:05:082
26368991052,93cyclictest27383-21modprobe20:49:550
26368991052,93cyclictest27383-21modprobe20:49:550
263689910467,29cyclictest7636-21latency_hist19:49:360
263689910467,29cyclictest7636-21latency_hist19:49:360
26368991009,82cyclictest16252-21apt-get22:29:560
26368999972,19cyclictest19511-21unixbench_singl00:25:100
26368999972,19cyclictest19511-21unixbench_singl00:25:100
26368999868,21cyclictest28026-21latency_hist19:14:360
26368999562,20cyclictest13379-21fschecks_time22:24:550
26368999541,15cyclictest26359-21cyclictest22:19:580
26388999374,13cyclictest18859-21/usr/sbin/munin23:29:522
26377999368,18cyclictest32333-21kworker/u17:3-xprtiod23:29:521
26388999273,9cyclictest0-21swapper/220:10:012
26377999270,15cyclictest2491-21ssh21:10:121
26377999270,15cyclictest2491-21ssh21:10:121
2639699907,54cyclictest0-21swapper/300:34:533
26368999057,11cyclictest181rcu_preempt21:34:490
250622908,54sleep125065-21df_inode19:04:551
26368998856,12cyclictest181rcu_preempt20:04:460
26388998770,11cyclictest5914-21kworker/u17:3-events_unbound00:04:492
26368998721,24cyclictest30480-21apt-get19:19:480
26388998565,10cyclictest0-21swapper/219:15:112
26377998568,10cyclictest0-21swapper/120:04:351
26368998554,10cyclictest181rcu_preempt21:54:560
26388998469,9cyclictest0-21swapper/222:30:532
26388998465,13cyclictest18852-21latency_hist21:39:362
254542849,53sleep024894-21/usr/sbin/munin19:05:100
26388998362,14cyclictest7692-21latency22:15:002
26388998362,14cyclictest7692-21latency22:15:002
26368998358,19cyclictest13395-21switchtime00:15:070
26388998263,13cyclictest25247-21kworker/u18:0-rpciod22:24:352
26368998229,27cyclictest181rcu_preempt20:25:050
26368998229,27cyclictest181rcu_preempt20:25:050
2623328212,44sleep30-21swapper/319:09:343
26388998158,15cyclictest22006-21sh22:39:362
26388998061,11cyclictest0-21swapper/219:44:552
26388998061,11cyclictest0-21swapper/219:44:552
26388998058,17cyclictest25708-21unixbench_multi23:40:102
2637799807,60cyclictest10463-21mailstats00:10:081
2637799807,60cyclictest10463-21mailstats00:10:081
26377998060,10cyclictest0-21swapper/122:59:531
2636899801,52cyclictest31536-21grep23:50:120
2636899801,52cyclictest31536-21grep23:50:120
26388997961,11cyclictest24329-21latency_hist00:34:362
26388997959,13cyclictest7053-21systemd-run21:19:342
26388997957,16cyclictest15835-21apt-key22:29:352
26388997957,15cyclictest26359-21cyclictest22:46:342
26388997956,12cyclictest0-21swapper/221:59:562
26388997956,12cyclictest0-21swapper/221:59:562
26377997957,12cyclictest0-21swapper/123:49:571
26377997957,12cyclictest0-21swapper/123:49:571
26377997955,17cyclictest392-21cron21:59:331
2637799793,61cyclictest30480-21apt-get19:19:521
26377997911,57cyclictest16252-21apt-get22:29:481
26368997955,17cyclictest4506-21ssh23:04:550
26368997948,11cyclictest181rcu_preempt22:49:420
2636899794,64cyclictest15581-21rm00:19:500
2636899794,64cyclictest15581-21rm00:19:500
2636899793,38cyclictest13999-21sed21:30:080
26368997932,29cyclictest181rcu_preempt21:53:310
26368997932,29cyclictest181rcu_preempt21:53:310
26388997864,7cyclictest0-21swapper/219:40:092
26388997864,7cyclictest0-21swapper/219:40:092
26377997861,9cyclictest0-21swapper/119:34:571
26377997851,19cyclictest16017-21cut20:14:571
26377997851,19cyclictest16017-21cut20:14:571
26368997855,16cyclictest26359-21cyclictest20:00:580
26368997836,16cyclictest14789-21ssh23:22:040
26388997754,15cyclictest0-21swapper/222:40:082
26388997749,17cyclictest0-21swapper/223:24:352
26377997757,10cyclictest0-21swapper/123:40:061
26377997756,11cyclictest0-21swapper/123:14:321
26377997755,15cyclictest1135-21diskmemload00:00:001
26368997749,11cyclictest60-21kcompactd000:09:510
26368997749,11cyclictest60-21kcompactd000:09:510
2638899766,61cyclictest355-21switchtime19:25:072
26388997659,7cyclictest0-21swapper/223:29:342
26388997654,12cyclictest0-21swapper/221:50:022
26388997654,12cyclictest0-21swapper/221:50:022
26388997653,13cyclictest23048-21timerandwakeup20:35:092
26388997623,11cyclictest371rcuc/221:04:532
26377997657,13cyclictest320-21dbus-daemon20:44:341
26377997656,10cyclictest0-21swapper/121:14:511
26368997659,11cyclictest25724-21rm23:40:100
2639699752,38cyclictest26422-21chrt22:45:563
26388997556,12cyclictest0-21swapper/220:04:562
26388997553,12cyclictest0-21swapper/220:39:542
26388997551,13cyclictest0-21swapper/220:59:342
26388997542,24cyclictest24615-21modprobe21:49:332
26388997542,24cyclictest24615-21modprobe21:49:332
2638899753,51cyclictest24702-21diskstats00:34:572
2637799756,60cyclictest22712-21cat23:35:081
26377997559,9cyclictest0-21swapper/122:14:531
26377997559,9cyclictest0-21swapper/122:14:531
26377997555,13cyclictest0-21swapper/119:29:361
26377997554,11cyclictest0-21swapper/120:29:541
26368997555,12cyclictest26359-21cyclictest22:39:380
26368997553,15cyclictest26359-21cyclictest21:15:550
2636899754,54cyclictest24523-21ssh23:39:360
26368997537,28cyclictest0-21swapper/021:13:070
26368997537,28cyclictest0-21swapper/021:13:070
26368997535,33cyclictest0-21swapper/021:20:020
2636899753,36cyclictest4095-21latency_hist23:04:350
26388997459,8cyclictest0-21swapper/221:10:032
26388997459,8cyclictest0-21swapper/221:10:032
26388997455,12cyclictest0-21swapper/221:30:062
26388997453,16cyclictest26359-21cyclictest20:29:532
26388997452,15cyclictest13099-21latency_hist21:29:362
26388997452,10cyclictest0-21swapper/221:55:092
26377997455,9cyclictest0-21swapper/120:39:351
26377997450,13cyclictest0-21swapper/121:50:071
26377997450,13cyclictest0-21swapper/121:50:071
2637799743,50cyclictest13234-21ssh22:24:511
26377997425,29cyclictest31-21ksoftirqd/121:44:251
26368997455,12cyclictest11694-21kworker/u17:2+xprtiod23:46:140
26368997448,17cyclictest10481-21ssh21:24:560
2636899744,34cyclictest31735-21switchtime22:55:120
2636899744,32cyclictest7397-21ntp_offset00:05:010
26368997443,12cyclictest181rcu_preempt23:09:450
2639699738,51cyclictest0-21swapper/322:29:493
2639699733,50cyclictest14335-21chrt21:30:193
26388997357,9cyclictest0-21swapper/200:09:582
26388997357,9cyclictest0-21swapper/200:09:582
26388997357,11cyclictest4278-21/usr/sbin/munin23:05:132
26388997354,11cyclictest0-21swapper/220:59:572
26388997351,15cyclictest0-21swapper/223:09:592
26388997317,20cyclictest0-21swapper/221:44:322
26377997359,8cyclictest0-21swapper/121:24:041
2637799732,53cyclictest24698-21ssh00:34:561
26368997349,17cyclictest24401-21latency_hist00:34:360
2639699723,56cyclictest1897-21chrt21:10:013
2639699723,56cyclictest1897-21chrt21:10:013
2639699723,55cyclictest12564-21taskset20:04:493
2639699723,54cyclictest4572-21taskset22:09:573
26388997252,9cyclictest0-21swapper/223:50:102
26388997252,9cyclictest0-21swapper/223:50:102
26377997258,9cyclictest346-21runrttasks00:15:291
26377997258,8cyclictest0-21swapper/122:04:351
26377997258,8cyclictest0-21swapper/122:04:351
26377997257,10cyclictest1135-21diskmemload22:10:121
26377997255,8cyclictest0-21swapper/100:19:551
26377997255,8cyclictest0-21swapper/100:19:551
26377997253,9cyclictest0-21swapper/120:54:351
26377997253,9cyclictest0-21swapper/120:54:351
26377997249,13cyclictest0-21swapper/119:54:351
26377997249,13cyclictest0-21swapper/119:54:351
26377997247,17cyclictest0-21swapper/123:24:551
2636899725,52cyclictest25043-21/usr/sbin/munin22:45:130
2639699713,54cyclictest22461-21taskset00:30:123
2639699712,52cyclictest13667-21chrt23:20:033
2639699712,34cyclictest5546-21taskset00:01:483
26388997153,12cyclictest3979-21latency_hist23:04:342
26388997150,15cyclictest18244-21latency_hist00:24:352
26388997150,15cyclictest18244-21latency_hist00:24:352
26388997150,14cyclictest0-21swapper/222:49:512
26388997148,12cyclictest0-21swapper/220:19:592
26388997148,12cyclictest0-21swapper/220:19:592
2637799717,19cyclictest10359-21cpuspeed21:24:511
26377997157,10cyclictest26359-21cyclictest22:34:521
26377997155,8cyclictest0-21swapper/123:15:101
26377997123,29cyclictest31-21ksoftirqd/121:47:421
26377997123,29cyclictest31-21ksoftirqd/121:47:421
26377997116,20cyclictest30121-21latency_hist19:19:361
26368997147,16cyclictest22055-21ssh22:39:370
2636899714,46cyclictest16065-21fschecks_time20:14:580
2636899714,46cyclictest16065-21fschecks_time20:14:580
2636899713,39cyclictest1127-21munin-run21:09:330
2636899712,43cyclictest21707-21munin-run21:44:330
26368997113,24cyclictest20921-21sshd23:33:030
2639699704,53cyclictest24814-21taskset20:40:083
2639699703,55cyclictest13666-21taskset00:15:093
2639699703,54cyclictest24571-21chrt22:44:053
2639699703,53cyclictest7607-21chrt23:09:583
2639699703,52cyclictest11289-21taskset23:15:253
2639699703,36cyclictest982-21chrt23:54:473
2639699703,36cyclictest982-21chrt23:54:473
2639699703,35cyclictest30616-21chrt20:59:483
2639699702,55cyclictest24692-21chrt23:39:443
2639699702,37cyclictest24076-21taskset23:38:473
26388997051,9cyclictest0-21swapper/223:54:562
26388997051,9cyclictest0-21swapper/223:54:562
26388997050,10cyclictest0-21swapper/223:45:122
26388997049,10cyclictest0-21swapper/222:09:592
26388997036,25cyclictest2944-21cstates19:34:492
26388997023,35cyclictest32658-21diskmemload22:57:072
26377997052,11cyclictest26359-21cyclictest20:59:361
26368997053,10cyclictest6880-21idleruntime-cro22:14:320
2636899704,53cyclictest30850-21ssh21:59:450
2636899704,53cyclictest30850-21ssh21:59:450
2636899703,34cyclictest1743-21sensors_temp23:55:070
2636899703,34cyclictest1743-21sensors_temp23:55:070
2636899702,26cyclictest5872-21grep19:44:340
2636899702,26cyclictest5872-21grep19:44:340
26368997015,16cyclictest15-21ksoftirqd/019:34:460
2639699693,54cyclictest5705-21taskset23:06:053
2639699693,53cyclictest27438-21taskset21:54:133
2639699693,53cyclictest27438-21taskset21:54:133
2639699692,54cyclictest29524-21chrt22:51:273
2639699692,36cyclictest4249-21chrt19:39:343
26388996948,16cyclictest19050-21sh00:25:032
26388996948,16cyclictest19050-21sh00:25:032
26377996954,9cyclictest346-21runrttasks19:55:101
26377996949,12cyclictest1135-21diskmemload23:54:501
26377996949,12cyclictest1135-21diskmemload23:54:501
26377996948,12cyclictest0-21swapper/119:10:081
26377996945,16cyclictest0-21swapper/100:29:031
26377996945,16cyclictest0-21swapper/100:29:031
26368996946,14cyclictest201-21systemd-journal00:04:030
26368996939,21cyclictest22529-21cut20:34:560
26368996937,11cyclictest181rcu_preempt22:05:010
2639699682,53cyclictest16590-21chrt00:20:123
2639699682,53cyclictest16590-21chrt00:20:123
26388996852,9cyclictest0-21swapper/200:01:372
2638899684,44cyclictest346-21runrttasks21:24:192
26388996825,12cyclictest181rcu_preempt20:45:022
26377996852,10cyclictest6932-21systemd-run23:09:341
26377996850,8cyclictest0-21swapper/120:49:361
26377996850,14cyclictest32333-21kworker/u17:3+events_unbound23:24:321
26377996849,10cyclictest0-21swapper/123:44:531
26377996848,12cyclictest0-21swapper/121:35:011
26377996843,16cyclictest26359-21cyclictest22:54:361
2637799682,55cyclictest15019-21timerandwakeup20:10:111
26368996851,11cyclictest2630-21systemd19:34:350
2639699673,52cyclictest14365-21taskset22:25:413
2639699672,34cyclictest27805-21chrt19:13:253
26388996749,9cyclictest0-21swapper/219:24:362
26388996748,7cyclictest181rcu_preempt23:37:102
26388996747,12cyclictest0-21swapper/223:15:112
26388996735,23cyclictest0-21swapper/219:09:582
26377996752,9cyclictest24770-21systemd-run22:44:331
26377996750,12cyclictest1135-21diskmemload22:57:521
2637799674,43cyclictest12788-21fschecks_time20:04:551
26377996741,19cyclictest9570-21kworker/u19:2+events_unbound21:34:331
26377996725,31cyclictest22236-21timerandwakeup00:30:091
2637799672,43cyclictest19009-21latency_hist20:24:361
2637799672,43cyclictest19009-21latency_hist20:24:361
2637799672,27cyclictest31-21ksoftirqd/100:05:001
2637799671,29cyclictest31-21ksoftirqd/122:24:301
2636899674,40cyclictest7922-21fschecks_time19:49:550
2636899674,40cyclictest7922-21fschecks_time19:49:550
26368996743,17cyclictest20749-21cpuspeed20:29:490
2639699664,47cyclictest15081-21chrt20:10:113
2639699663,50cyclictest2234-21chrt19:30:103
2639699663,49cyclictest6542-21chrt19:45:003
2639699663,49cyclictest6542-21chrt19:45:003
2639699663,33cyclictest32188-21taskset23:51:553
2639699663,33cyclictest32188-21taskset23:51:553
2639699662,50cyclictest9022-21taskset19:54:283
2639699662,50cyclictest9022-21taskset19:54:283
2639699662,34cyclictest5920-21taskset21:16:293
2639699662,33cyclictest20189-21taskset20:27:003
2639699662,33cyclictest20189-21taskset20:27:003
2639699662,32cyclictest405-21chrt22:57:433
2638899669,40cyclictest16551-21switchtime20:15:102
2638899669,40cyclictest16551-21switchtime20:15:102
26388996651,8cyclictest0-21swapper/222:04:582
2637799663,45cyclictest1236-21/usr/sbin/munin19:29:591
2637799663,36cyclictest27649-21ssh22:49:291
2636899663,44cyclictest30314-21sh20:59:320
2636899663,44cyclictest18709-21latency_hist23:29:350
26368996624,29cyclictest18787-21modprobe20:24:320
26368996624,29cyclictest18787-21modprobe20:24:320
2636899662,34cyclictest14506-21ls20:09:570
2639699653,49cyclictest29678-21chrt21:57:133
2639699653,32cyclictest7901-21chrt21:20:023
2639699653,32cyclictest16935-21chrt20:16:473
2639699653,32cyclictest16935-21chrt20:16:473
2639699653,32cyclictest12044-21chrt20:01:333
2639699652,51cyclictest32636-21chrt19:25:043
2639699652,50cyclictest9566-21chrt19:54:573
2639699652,50cyclictest16350-21chrt23:24:573
2639699652,34cyclictest30506-21taskset19:19:583
2639699652,32cyclictest21419-21taskset22:38:303
2638899654,39cyclictest8092-21latency19:49:582
2638899654,39cyclictest8092-21latency19:49:582
2638899653,42cyclictest346-21runrttasks20:24:452
2638899653,42cyclictest346-21runrttasks20:24:452
2638899653,39cyclictest27614-21fschecks_count20:50:502
2638899653,39cyclictest27614-21fschecks_count20:50:502
26388996519,37cyclictest11657-21switchtime20:00:082
2636899653,23cyclictest32294-21fschecks_count19:24:540
26368996524,13cyclictest181rcu_preempt21:00:090
26368996514,13cyclictest181rcu_preempt19:14:410
2639699643,50cyclictest12020-21taskset00:13:473
2639699643,50cyclictest12020-21taskset00:13:473
2639699643,31cyclictest3406-21chrt22:08:033
2639699642,49cyclictest20681-21taskset21:41:373
2639699642,29cyclictest28080-21chrt20:51:013
2639699642,29cyclictest28080-21chrt20:51:013
2639699642,28cyclictest21489-21chrt20:30:083
26388996451,7cyclictest0-21swapper/200:15:052
26388996444,12cyclictest0-21swapper/219:30:092
26377996449,9cyclictest0-21swapper/119:45:011
26377996449,9cyclictest0-21swapper/119:45:011
26377996449,10cyclictest2890-21ssh22:06:311
2637799642,42cyclictest31132-21ntpq21:00:041
2636899643,32cyclictest24637-21idleruntime-cro21:49:330
2636899643,32cyclictest24637-21idleruntime-cro21:49:330
2639699632,32cyclictest30159-21chrt20:57:463
2639699632,30cyclictest32002-21chrt22:00:213
2639699632,30cyclictest32002-21chrt22:00:213
2639699632,29cyclictest7440-21chrt00:05:043
2639699632,29cyclictest32368-21chrt21:04:543
2639699632,29cyclictest23329-21chrt20:36:033
2639699632,26cyclictest2695-21taskset23:00:463
2637799634,32cyclictest19285-21fschecks_time20:24:551
2637799634,32cyclictest19285-21fschecks_time20:24:551
2636899634,39cyclictest346-21runrttasks20:44:250
2639699623,28cyclictest16123-21chrt21:34:453
2639699623,26cyclictest29665-21chrt23:48:033
2639699622,49cyclictest9406-21chrt22:18:243
2639699622,49cyclictest9406-21chrt22:18:243
2639699622,31cyclictest26786-21chrt20:48:463
2639699622,29cyclictest19144-21chrt00:25:063
2639699622,29cyclictest19144-21chrt00:25:063
2639699622,28cyclictest12911-21chrt21:29:343
2639699622,27cyclictest29681-21chrt19:16:213
26388996247,8cyclictest0-21swapper/219:59:362
26368996240,11cyclictest0-21swapper/020:45:110
2639699613,29cyclictest18603-21chrt20:22:143
2639699613,29cyclictest18603-21chrt20:22:143
2639699613,27cyclictest24438-21chrt21:49:033
2639699613,27cyclictest24438-21chrt21:49:033
2639699612,28cyclictest10770-21chrt22:20:023
26377996146,7cyclictest0-21swapper/121:04:471
2639699603,25cyclictest5675-21chrt19:42:463
2639699603,25cyclictest5675-21chrt19:42:463
2639699592,26cyclictest20078-21chrt23:30:323
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional