You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-12 - 00:16
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Mon May 11, 2026 12:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17811991233,29cyclictest16683-21mailstats08:31:570
17811991183,27cyclictest10981-21runrttasks09:30:470
17811991183,27cyclictest10981-21runrttasks09:30:470
17811991123,30cyclictest10801-21ssh12:12:020
178119911227,50cyclictest0-21swapper/009:16:490
178359911168,36cyclictest5838-21apt10:16:232
178359911168,36cyclictest5838-21apt10:16:232
178359911146,35cyclictest6227-21sh09:21:572
178359910972,30cyclictest17655-21apt-key08:36:272
17811991092,26cyclictest23638-21ssh11:40:310
17811991073,33cyclictest22589-21latency_hist08:51:260
17811991073,26cyclictest10872-21sh11:18:060
17811991073,24cyclictest7313-21ssh12:06:530
17811991073,24cyclictest22410-21sh10:42:250
17811991063,29cyclictest21605-21grep09:47:010
17811991063,29cyclictest21605-21grep09:47:010
17811991063,27cyclictest352-21runrttasks12:04:270
17811991063,25cyclictest1140-21ssh09:13:170
17811991063,25cyclictest1140-21ssh09:13:170
1757221066,15sleep10-21swapper/107:00:041
178359910453,42cyclictest23654-21latency_hist09:51:282
178359910453,42cyclictest23654-21latency_hist09:51:282
17835991042,27cyclictest28555-21seq09:06:032
17811991043,22cyclictest3424-21idleruntime-cro12:01:220
17811991043,22cyclictest3424-21idleruntime-cro12:01:220
17811991042,26cyclictest20295-21rm12:27:580
17811991042,26cyclictest20295-21rm12:27:580
17811991034,24cyclictest7226-21awk10:17:240
17811991033,30cyclictest12883-21ntp_states10:26:550
17811991033,20cyclictest13377-21processes12:16:560
17811991032,23cyclictest16852-21/usr/sbin/munin12:22:340
178359910239,37cyclictest9818-21ntp_kernel_err08:11:512
17811991025,27cyclictest753-21fschecks_time11:01:460
17811991022,28cyclictest28613-21fschecks_time07:31:460
17811991022,28cyclictest28613-21fschecks_time07:31:460
17835991012,29cyclictest12336-21cat11:21:332
17835991012,29cyclictest12336-21cat11:21:332
17835991002,29cyclictest20749-21chrt10:41:082
17835991002,26cyclictest28043-21seq07:30:572
17811991004,31cyclictest5840-21modprobe08:01:230
178119910041,52cyclictest0-21swapper/009:41:530
17811991003,26cyclictest27181-21cut09:56:490
17811991003,26cyclictest27181-21cut09:56:490
17811991002,24cyclictest28122-21ssh09:04:490
178119910016,27cyclictest31924-21rm11:53:380
178119910011,28cyclictest11763-21/usr/sbin/munin09:31:440
1783599992,27cyclictest2152-21runrttasks09:16:082
1783599992,27cyclictest2152-21runrttasks09:16:082
1781199993,11cyclictest30074-21apt-get10:56:350
1781199993,11cyclictest30074-21apt-get10:56:350
1781199992,25cyclictest26548-21sed09:56:250
17835999859,34cyclictest17801-21cyclictest09:52:232
1783599982,31cyclictest12041-21taskset12:15:302
1781199985,13cyclictest125402sleep008:21:350
1781199984,9cyclictest0-21swapper/011:48:520
1781199984,22cyclictest2689-21latency_hist07:51:250
1781199984,10cyclictest0-21swapper/010:06:350
1781199983,36cyclictest14373-21ssh11:24:350
1781199983,36cyclictest14373-21ssh11:24:350
1781199983,28cyclictest3507-21mailstats07:51:570
1783599973,33cyclictest17091-21runrttasks09:41:052
1783599973,33cyclictest17091-21runrttasks09:41:052
1783599973,29cyclictest6206-21seq12:05:562
1781199974,29cyclictest7869-21ssh09:25:260
1781199973,31cyclictest14510-21/usr/sbin/munin09:36:560
1781199973,31cyclictest14510-21/usr/sbin/munin09:36:560
1781199973,27cyclictest15357-21ssh11:26:380
1781199973,24cyclictest5026-21sh10:14:050
1781199973,24cyclictest5026-21sh10:14:050
1783599962,27cyclictest19944-21ssh09:45:332
1783599962,25cyclictest10551-21runrttasks09:29:342
1783599962,25cyclictest10551-21runrttasks09:29:342
1781199965,25cyclictest30230-21apt-get07:36:490
1781199964,27cyclictest15919-21latency_hist08:31:260
1781199963,32cyclictest11085-21ntpd11:08:020
1781199963,21cyclictest23361-21sort07:16:400
1781199963,21cyclictest23361-21sort07:16:400
1781199962,29cyclictest7894-21fschecks_count08:06:430
17811999611,31cyclictest277-21dbus-daemon07:46:240
17835999556,33cyclictest0-21swapper/209:07:012
1783599952,32cyclictest21074-21seq07:07:282
1781199953,31cyclictest9537-21grep10:21:450
1781199953,30cyclictest24804-21ps10:46:570
1781199953,24cyclictest6951-21awk11:11:510
1781199953,24cyclictest6951-21awk11:11:510
1781199952,27cyclictest15568-21fschecks_time10:31:450
17835999471,10cyclictest39-21ksoftirqd/211:45:232
17835999471,10cyclictest39-21ksoftirqd/211:45:232
17835999450,36cyclictest0-21swapper/209:31:492
1783599943,27cyclictest23064-21chrt11:38:532
1783599942,26cyclictest25617-21runrttasks10:48:072
1783599942,24cyclictest5590-21seq07:58:302
1781199943,28cyclictest17696-21latency_hist07:01:270
17835999353,32cyclictest0-21swapper/208:06:442
1783599933,22cyclictest2687-21latency_hist07:51:252
1781199933,6cyclictest0-21swapper/008:41:330
1781199933,32cyclictest25688-21modprobe09:01:220
1781199933,30cyclictest19033-21switchtime10:36:580
1781199933,27cyclictest11674-21cat08:16:580
1781199933,26cyclictest19349-21latency_hist07:06:260
1781199933,26cyclictest19349-21latency_hist07:06:260
1781199933,23cyclictest31695-21latency_hist09:11:250
1781199932,32cyclictest28811-21taskset10:53:420
1781199932,24cyclictest20051-21apt-get07:06:370
17835999250,35cyclictest29980-21latency_hist07:36:262
17835999250,35cyclictest29980-21latency_hist07:36:262
1783599924,61cyclictest0-21swapper/212:01:262
1783599924,61cyclictest0-21swapper/212:01:262
17835999240,44cyclictest0-21swapper/210:02:012
1783599923,28cyclictest2071-21taskset10:09:092
1783599923,26cyclictest352-21runrttasks08:43:372
1783599922,27cyclictest25923-21cat09:01:262
17824999261,10cyclictest181rcu_preempt10:22:271
1782499923,59cyclictest15891-21apt-get12:21:431
1781199925,8cyclictest0-21swapper/007:26:350
1781199923,31cyclictest19414-21latency_hist08:41:260
1781199923,28cyclictest24597-21fschecks_time11:41:450
1781199923,28cyclictest24597-21fschecks_time11:41:450
1781199923,28cyclictest18747-21/usr/sbin/munin11:31:490
1781199923,28cyclictest18747-21/usr/sbin/munin11:31:490
1781199923,14cyclictest19676-21snmpd10:01:350
17835999145,39cyclictest25693-21diskmemload12:17:032
1783599913,8cyclictest0-21swapper/208:16:382
1781199916,22cyclictest24637-21apt-get07:21:380
1781199916,22cyclictest24637-21apt-get07:21:380
1781199914,32cyclictest22797-21cpuspeed08:51:390
1781199913,27cyclictest6074-21/usr/sbin/munin08:01:510
1781199913,27cyclictest6074-21/usr/sbin/munin08:01:510
1783599902,27cyclictest16726-21sh10:33:062
1781199903,31cyclictest21397-21latency_hist07:11:270
17835998966,14cyclictest427-21kworker/u19:111:56:232
17835998947,34cyclictest0-21swapper/208:46:342
1783599893,39cyclictest7064-21/usr/sbin/munin08:02:012
1783599893,39cyclictest7064-21/usr/sbin/munin08:02:012
17824998954,16cyclictest181rcu_preempt09:04:391
17824998952,19cyclictest181rcu_preempt10:59:431
17824998952,19cyclictest181rcu_preempt10:59:431
17824998952,12cyclictest181rcu_preempt08:31:541
1781199894,28cyclictest9320-21/usr/sbin/munin08:11:420
17835998862,14cyclictest0-21swapper/207:16:272
1783599884,75cyclictest18877-21runrttasks12:26:362
1783599884,75cyclictest18877-21runrttasks12:26:362
17835998738,41cyclictest0-21swapper/212:06:482
1783599873,36cyclictest23982-21/usr/sbin/munin07:16:572
1783599873,36cyclictest23982-21/usr/sbin/munin07:16:572
1783599872,34cyclictest18636-21sendmail_mailst07:01:562
1783599872,34cyclictest18636-21sendmail_mailst07:01:562
1783599872,29cyclictest31518-21runrttasks07:40:522
17824998763,8cyclictest181rcu_preempt10:11:401
17824998763,8cyclictest181rcu_preempt10:11:401
17824998749,13cyclictest181rcu_preempt10:27:211
17835998650,29cyclictest0-21swapper/210:26:562
17835998647,32cyclictest0-21swapper/210:57:202
17835998647,32cyclictest0-21swapper/210:57:202
17835998647,30cyclictest21617-21cat10:41:492
1783599863,38cyclictest25704-21/usr/sbin/munin07:22:072
1783599863,38cyclictest25704-21/usr/sbin/munin07:22:072
1783599863,36cyclictest1147-21/usr/sbin/munin11:01:572
1783599863,30cyclictest352-21runrttasks11:33:232
1783599863,30cyclictest352-21runrttasks11:33:232
1783599862,40cyclictest27871-21ssh11:46:552
1782499867,55cyclictest23578-21latency_hist09:51:271
1782499867,55cyclictest23578-21latency_hist09:51:271
17824998663,17cyclictest22458-21systemctl08:51:231
17824998658,9cyclictest181rcu_preempt12:11:561
17835998562,15cyclictest17801-21cyclictest12:23:052
17835998562,14cyclictest1-21systemd10:53:012
1783599853,37cyclictest6412-21sh11:11:352
1783599853,37cyclictest6412-21sh11:11:352
1783599853,34cyclictest4168-21mailstats11:06:592
1783599852,8cyclictest0-21swapper/211:26:392
17824998560,19cyclictest170-21jbd2/mmcblk2p2-809:21:491
17824998557,19cyclictest20771-21sh12:29:191
17824998557,19cyclictest20771-21sh12:29:191
17824998541,9cyclictest181rcu_preempt11:56:441
17824998541,9cyclictest181rcu_preempt11:56:441
17824998538,8cyclictest181rcu_preempt09:10:251
17824998510,52cyclictest25138-21apt-get07:21:371
17824998510,52cyclictest25138-21apt-get07:21:371
17824998448,13cyclictest181rcu_preempt08:46:261
17824998446,16cyclictest181rcu_preempt09:27:191
17824998446,16cyclictest181rcu_preempt09:27:191
17824998429,14cyclictest0-21swapper/109:16:351
17835998362,12cyclictest13905-21munin-plugin-st08:26:232
1783599833,6cyclictest0-21swapper/207:51:362
1783599833,28cyclictest648-21runrttasks07:45:382
1783599832,26cyclictest22715-21/usr/sbin/munin08:51:492
17824998351,10cyclictest181rcu_preempt09:42:361
17824998344,8cyclictest181rcu_preempt07:06:481
17824998338,12cyclictest17801-21cyclictest08:27:571
1783599823,32cyclictest3210-21sh09:16:562
1783599822,36cyclictest6386-21apt-get10:16:422
1783599822,33cyclictest18958-21/usr/sbin/munin08:38:072
17835998217,55cyclictest0-21swapper/211:19:452
17824998248,12cyclictest181rcu_preempt11:36:551
17824998247,13cyclictest181rcu_preempt08:01:461
17824998247,13cyclictest181rcu_preempt08:01:461
17824998245,18cyclictest181rcu_preempt12:01:531
17824998241,20cyclictest181rcu_preempt09:33:111
17824998241,12cyclictest181rcu_preempt11:42:431
17824998241,12cyclictest181rcu_preempt11:42:431
17824998238,34cyclictest4227-21grep07:56:231
1783599813,30cyclictest17696-21latency_hist07:01:282
1783599813,27cyclictest14542-21/usr/sbin/munin08:27:312
17824998149,16cyclictest181rcu_preempt07:36:431
17824998143,17cyclictest181rcu_preempt10:46:411
17824998142,17cyclictest181rcu_preempt07:31:401
17824998142,17cyclictest181rcu_preempt07:31:401
1783599803,30cyclictest27781-21unixbench_multi09:57:012
1783599803,30cyclictest27781-21unixbench_multi09:57:012
17824998060,14cyclictest7773-21unixbench_singl12:07:011
17824998059,14cyclictest9921-21processes08:11:551
17824998019,42cyclictest23868-21ssh09:51:461
1784299793,47cyclictest0-21swapper/312:21:453
17824997953,17cyclictest14186-21latency_hist08:26:261
17824997947,23cyclictest14734-21kworker/u17:3+events_unbound11:16:221
17824997947,23cyclictest14734-21kworker/u17:3+events_unbound11:16:221
17824997947,10cyclictest181rcu_preempt09:00:521
17824997944,12cyclictest181rcu_preempt10:40:171
1782499793,46cyclictest1226-21apt-get07:46:371
1782499792,39cyclictest19908-21ssh11:33:201
1782499792,39cyclictest19908-21ssh11:33:201
1783599783,31cyclictest11948-21munin-plugin-st10:26:232
17824997845,11cyclictest181rcu_preempt11:24:341
17824997845,11cyclictest181rcu_preempt11:24:341
17824997840,27cyclictest8985-21modprobe10:21:221
17824997840,22cyclictest32131-21ssh11:54:141
17824997839,9cyclictest181rcu_preempt11:28:511
17824997754,8cyclictest181rcu_preempt09:16:261
17824997754,8cyclictest181rcu_preempt09:16:261
17824997748,10cyclictest181rcu_preempt11:19:291
17824997743,16cyclictest181rcu_preempt11:50:151
17824997738,8cyclictest181rcu_preempt07:26:451
17824997736,18cyclictest181rcu_preempt08:41:261
17824997732,10cyclictest0-21swapper/108:52:461
17824997653,9cyclictest181rcu_preempt11:10:401
17824997652,8cyclictest181rcu_preempt10:06:261
17824997645,12cyclictest181rcu_preempt10:42:321
17824997636,7cyclictest181rcu_preempt10:31:331
17824997636,14cyclictest181rcu_preempt12:19:501
17824997634,13cyclictest181rcu_preempt09:41:271
17824997634,13cyclictest181rcu_preempt09:41:271
17824997627,10cyclictest0-21swapper/107:06:271
17824997627,10cyclictest0-21swapper/107:06:271
17824997553,9cyclictest181rcu_preempt09:57:231
17824997553,9cyclictest181rcu_preempt09:57:231
17824997530,12cyclictest181rcu_preempt07:42:301
17824997439,12cyclictest181rcu_preempt07:16:561
17824997439,12cyclictest181rcu_preempt07:16:561
17824997428,8cyclictest181rcu_preempt10:53:261
1782499737,31cyclictest406-21/usr/sbin/munin11:01:501
17824997340,12cyclictest181rcu_preempt08:06:431
17824997337,8cyclictest181rcu_preempt10:06:501
17824997332,15cyclictest181rcu_preempt08:16:591
17824997329,14cyclictest181rcu_preempt07:11:481
17824997235,14cyclictest181rcu_preempt07:58:301
1784299633,44cyclictest0-21swapper/310:16:583
1784299615,34cyclictest0-21swapper/307:21:473
1784299615,34cyclictest0-21swapper/307:21:473
1784299614,37cyclictest0-21swapper/307:47:213
1784299597,32cyclictest23779-21chrt09:51:313
1784299565,34cyclictest15185-21taskset11:26:263
1784299565,34cyclictest15185-21taskset11:26:263
1784299564,41cyclictest22484-21taskset08:51:233
1784299563,35cyclictest21189-21taskset10:41:373
1784299554,37cyclictest19604-21chrt08:41:393
1784299553,41cyclictest9675-21chrt09:27:063
1784299553,41cyclictest9675-21chrt09:27:063
1784299553,40cyclictest10689-21chrt08:16:233
1784299553,38cyclictest2199-21chrt09:16:153
1784299553,38cyclictest2199-21chrt09:16:153
1784299553,38cyclictest15692-21taskset10:31:483
1784299544,39cyclictest18068-21taskset07:01:403
1784299544,39cyclictest18068-21taskset07:01:403
1784299543,40cyclictest880-21taskset11:01:493
1784299542,41cyclictest14798-21chrt12:19:533
1784299534,36cyclictest8529-21chrt11:14:553
1784299534,36cyclictest8529-21chrt11:14:553
1784299534,36cyclictest31463-21taskset10:03:513
1784299534,35cyclictest30231-21taskset10:57:073
1784299534,35cyclictest30231-21taskset10:57:073
1784299533,39cyclictest20116-21chrt12:27:283
1784299533,39cyclictest20116-21chrt12:27:283
1784299533,39cyclictest18835-21chrt10:36:553
1784299533,39cyclictest11803-21chrt10:26:003
1784299533,37cyclictest6286-21chrt10:16:283
1784299533,37cyclictest6286-21chrt10:16:283
1784299533,37cyclictest26762-21taskset10:51:223
1784299533,36cyclictest13336-21chrt09:33:553
1784299532,41cyclictest6322-21chrt12:06:153
1784299532,36cyclictest31950-21chrt11:53:423
1784299524,37cyclictest26781-21chrt09:56:373
1784299524,37cyclictest26781-21chrt09:56:373
1784299523,39cyclictest11925-21chrt12:15:113
1784299523,36cyclictest18638-21chrt09:42:023
1784299523,36cyclictest18086-21chrt11:31:253
1784299523,35cyclictest22027-21chrt07:11:493
1784299523,35cyclictest15943-21chrt08:31:273
1684125210,25sleep30-21swapper/306:57:003
1784299513,34cyclictest2970-21taskset09:16:483
1784299513,33cyclictest5592-21chrt09:21:393
1784299513,33cyclictest4949-21taskset07:56:513
1784299512,36cyclictest6260-21chrt08:01:453
1784299512,36cyclictest6260-21chrt08:01:453
1784299512,35cyclictest24699-21chrt07:21:243
1784299512,35cyclictest24699-21chrt07:21:243
1784299512,35cyclictest22293-21chrt11:37:003
1784299512,33cyclictest30189-21chrt07:36:273
1784299504,32cyclictest28582-21taskset07:31:453
1784299503,37cyclictest32723-21taskset10:06:393
1784299503,35cyclictest28448-21taskset10:52:403
1784299503,32cyclictest28301-21chrt11:47:013
1784299502,35cyclictest28863-21chrt09:06:263
1784299494,37cyclictest30893-21taskset09:09:103
1784299494,34cyclictest1111-21chrt11:56:463
1784299494,34cyclictest1111-21chrt11:56:463
1784299493,37cyclictest15121-21chrt09:36:513
1784299493,37cyclictest15121-21chrt09:36:513
1784299493,36cyclictest13644-21chrt08:23:153
1784299493,35cyclictest25993-21chrt11:43:453
1784299493,35cyclictest25993-21chrt11:43:453
1784299493,34cyclictest28175-21chrt07:31:253
1784299493,31cyclictest4360-21chrt07:56:263
1784299492,37cyclictest8230-21chrt12:08:143
1784299492,33cyclictest11454-21taskset08:16:523
1784299492,33cyclictest10158-21chrt11:16:573
1784299492,29cyclictest12238-21taskset10:26:333
1784299483,33cyclictest4743-21taskset11:07:413
1784299483,31cyclictest7889-21taskset08:06:433
1784299472,32cyclictest19063-21chrt11:31:573
1784299472,32cyclictest19063-21chrt11:31:573
1784299463,31cyclictest19237-21chrt08:41:233
1784299452,30cyclictest32468-21chrt07:41:563
1784299452,30cyclictest21228-21chrt09:46:563
1784299452,30cyclictest21228-21chrt09:46:563
1784299452,30cyclictest17433-21chrt08:36:253
1784299443,32cyclictest20409-21chrt07:06:503
1784299432,26cyclictest30772-21chrt07:37:433
1784299423,27cyclictest25735-21chrt09:01:243
1784299393,26cyclictest23263-21chrt08:51:523
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional