You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-11 - 15:41
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Sat Apr 11, 2026 12:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
272409911793,16cyclictest0-21swapper/007:33:280
27240991124,41cyclictest352-21runrttasks09:43:370
272409910937,64cyclictest0-21swapper/009:09:010
27240991082,34cyclictest7564-21grep07:38:570
272409910741,58cyclictest0-21swapper/009:58:250
27240991073,48cyclictest17737-21cat10:23:260
272409910622,8cyclictest0-21swapper/009:58:400
27240991053,26cyclictest24745-21/usr/sbin/munin10:33:550
27256991045,27cyclictest32708-21seq08:58:042
272409910437,60cyclictest27325-21/usr/sbin/munin10:39:290
27240991043,39cyclictest6651-21switchtime10:03:580
27240991043,38cyclictest15753-21/usr/sbin/munin11:14:010
27240991033,44cyclictest3591-21runrttasks10:53:250
27240991033,37cyclictest2086-21ssh09:03:270
27240991033,37cyclictest2086-21ssh09:03:270
27240991033,35cyclictest13405-21/usr/sbin/munin07:58:350
27240991033,33cyclictest14208-21kworker/u17:4+xprtiod10:53:390
27240991033,27cyclictest1109-21apt-get11:43:320
272409910322,30cyclictest3312-21latency_hist07:28:270
272409910322,18cyclictest3297-21unixbench_singl09:04:000
27240991023,44cyclictest14017-21latency_hist09:23:260
27240991023,31cyclictest32129-21ls08:53:580
27240991023,27cyclictest8799-21/usr/sbin/munin10:08:460
27240991023,27cyclictest28277-21latency_hist12:28:270
27240991023,25cyclictest13540-21acpi12:03:340
27240991013,41cyclictest7924-21grep10:59:010
27240991013,37cyclictest11936-21df07:53:440
27240991013,37cyclictest11936-21df07:53:440
27240991013,34cyclictest20272-21/usr/sbin/munin09:33:530
27240991013,32cyclictest31007-21cstates11:38:400
27240991013,25cyclictest32483-21cat07:18:260
27240991012,29cyclictest14576-21fschecks_time09:23:460
27240991012,27cyclictest5297-21cut11:48:560
27240991004,35cyclictest15260-21fschecks_time08:03:460
272409910038,56cyclictest0-21swapper/011:13:230
27240991003,27cyclictest10821-21unixbench_multi11:03:590
27240991003,27cyclictest10821-21unixbench_multi11:03:590
27240991002,29cyclictest19217-21unixbench_singl08:14:020
27240991002,29cyclictest19217-21unixbench_singl08:14:020
2725699993,25cyclictest6957-21ssh11:52:322
27248999942,33cyclictest1911-21diskmemload10:09:161
27240999938,54cyclictest0-21swapper/012:14:010
2724099993,34cyclictest20823-21unixbench_singl08:18:590
2724099993,29cyclictest28229-21grep11:33:450
2724099993,29cyclictest28229-21grep11:33:450
2724099993,21cyclictest21070-21/usr/sbin/munin10:28:340
27240999833,7cyclictest0-21swapper/009:43:240
27240999833,7cyclictest0-21swapper/009:43:240
2724099983,31cyclictest22197-21fschecks_count11:23:470
2724099983,31cyclictest22197-21fschecks_count11:23:470
2724099983,30cyclictest32035-21ls09:53:260
2724099973,38cyclictest8536-21/usr/sbin/munin07:43:470
2724099973,36cyclictest30616-21fschecks_time10:43:470
2724099973,36cyclictest30616-21fschecks_time10:43:470
2724099973,36cyclictest18157-21latency_hist08:13:270
2724099973,36cyclictest18157-21latency_hist08:13:270
2724099973,29cyclictest11832-21/usr/sbin/munin10:13:450
2724099973,29cyclictest11832-21/usr/sbin/munin10:13:450
2725699962,68cyclictest27347-21apt-get10:38:472
2724099965,37cyclictest18083-21apt-get10:24:050
2724099965,37cyclictest18083-21apt-get10:24:050
2724099964,35cyclictest5759-21chrt07:34:410
2724099964,29cyclictest27227-21cyclictest07:03:270
2724099963,35cyclictest8562-21fschecks_time09:13:470
2724099963,35cyclictest28440-21fschecks_time08:43:470
2724099963,35cyclictest28440-21fschecks_time08:43:470
2724099963,34cyclictest30039-21ps07:08:570
2724099963,34cyclictest10388-21latency_hist11:58:260
2724099963,34cyclictest10388-21latency_hist11:58:260
2724099963,33cyclictest25688-21ssh11:28:570
2724099963,24cyclictest22669-21taskset12:18:440
2724099962,32cyclictest24753-21cpu08:33:400
2724099962,32cyclictest24753-21cpu08:33:400
27240999622,11cyclictest0-21swapper/011:58:560
2725699952,40cyclictest23784-21load09:38:512
2725699952,40cyclictest23784-21load09:38:512
27240999538,9cyclictest18660-21rm09:29:510
27240999528,9cyclictest0-21swapper/008:23:460
27240999521,20cyclictest23845-21ls08:29:000
2725699942,66cyclictest1109-21apt-get11:43:372
2725699942,59cyclictest18979-21apt-get11:18:392
2724099943,37cyclictest19682-21if_err_eth011:19:440
2724099943,35cyclictest16498-21/usr/sbin/munin12:08:520
2724099943,31cyclictest31337-21latency_hist08:53:270
27240999427,8cyclictest0-21swapper/007:08:270
2724099942,30cyclictest27576-21unixbench_multi08:40:300
2724099942,30cyclictest27576-21unixbench_multi08:40:300
2724099934,33cyclictest32688-21/usr/sbin/munin07:18:460
2724099932,30cyclictest11738-21latency_hist07:53:280
27256999271,13cyclictest19943-21modprobe09:33:232
27256999253,30cyclictest29024-21sh12:28:532
27256999245,40cyclictest2520-21latency_hist09:58:272
2725699913,30cyclictest22504-21wc12:18:392
2724099913,29cyclictest29508-21awk12:29:010
198562910,6sleep2371rcuc/208:18:332
2725699906,13cyclictest3884-21apt-get10:53:392
27248999056,25cyclictest6676-21sh07:38:241
27256998961,21cyclictest521-21cron10:48:242
27256998961,21cyclictest521-21cron10:48:242
2725699895,38cyclictest1302-21latency10:48:492
27256998945,37cyclictest298-21in:imuxsock11:54:132
27256998945,37cyclictest298-21in:imuxsock11:54:132
2725699883,7cyclictest0-21swapper/212:08:362
2725699882,35cyclictest21646-21cut08:23:462
27248998861,20cyclictest544-21cron07:28:241
27248998846,8cyclictest181rcu_preempt11:38:551
2725699873,8cyclictest0-21swapper/207:34:272
2725699872,10cyclictest0-21swapper/211:38:382
27248998770,10cyclictest30641-21sshd12:32:131
27248998752,11cyclictest181rcu_preempt11:19:491
27256998642,36cyclictest0-21swapper/209:48:592
2725699863,35cyclictest21947-21/usr/sbin/munin11:23:592
2725699863,35cyclictest21947-21/usr/sbin/munin11:23:592
2725699863,34cyclictest7667-21switchtime10:58:592
2725699863,34cyclictest352-21runrttasks12:27:242
2725699863,28cyclictest352-21runrttasks07:28:282
2725699862,41cyclictest10884-21grep11:03:592
2725699862,41cyclictest10884-21grep11:03:592
2725699862,32cyclictest18871-21runrttasks10:24:222
2725699862,32cyclictest18871-21runrttasks10:24:222
27248998658,19cyclictest15410-21sh10:18:541
27248998658,12cyclictest181rcu_preempt08:08:261
2725699853,6cyclictest0-21swapper/208:48:392
2725699853,35cyclictest30723-21latency_hist11:38:262
2725699853,35cyclictest30723-21latency_hist11:38:262
27256998533,46cyclictest0-21swapper/210:19:062
2725699853,33cyclictest8302-21idleruntime-cro07:43:242
2725699853,33cyclictest24599-21latency_hist08:33:262
2725699853,31cyclictest2107-21tune2fs07:23:462
27248998538,15cyclictest181rcu_preempt11:43:321
2725699843,35cyclictest26391-21cstates09:43:422
2725699843,35cyclictest22234-21latency_hist12:18:252
2725699843,34cyclictest26551-21apt-get08:39:342
2725699843,34cyclictest26551-21apt-get08:39:342
27248998462,8cyclictest181rcu_preempt11:49:141
27248998459,18cyclictest19404-21latency_hist12:13:271
27248998454,21cyclictest1911-21diskmemload12:13:571
2725699833,6cyclictest0-21swapper/207:48:372
2725699833,37cyclictest18438-21munin-run11:18:232
2725699833,33cyclictest374-21sshd10:18:132
2725699833,33cyclictest374-21sshd10:18:132
2725699833,33cyclictest11871-21/usr/sbin/munin07:53:422
2725699833,33cyclictest11871-21/usr/sbin/munin07:53:422
2725699832,8cyclictest0-21swapper/207:43:392
2725699832,32cyclictest3401-21diskmemload09:58:512
2725699832,32cyclictest14220-21cat07:59:002
2725699832,22cyclictest17063-21latency_hist09:28:262
27248998355,11cyclictest181rcu_preempt09:30:141
27248998350,13cyclictest181rcu_preempt09:37:421
27248998342,30cyclictest24662-21grep11:28:261
27248998342,30cyclictest24662-21grep11:28:261
27248998339,15cyclictest181rcu_preempt08:38:421
27248998339,15cyclictest181rcu_preempt08:38:421
2725699823,34cyclictest25200-21taskset10:34:012
2725699823,33cyclictest28966-21latency_hist07:08:272
2725699822,37cyclictest22738-21sh10:30:342
2725699822,36cyclictest12969-21ssh11:08:432
2725699822,33cyclictest17316-21switchtime08:08:592
2725699822,33cyclictest17316-21switchtime08:08:592
2725699822,28cyclictest29949-21ntpq07:08:532
27248998260,15cyclictest567-21sh10:48:241
27248998260,15cyclictest567-21sh10:48:241
27248998245,15cyclictest181rcu_preempt10:13:491
27248998245,15cyclictest181rcu_preempt10:13:491
2726399812,34cyclictest0-21swapper/311:43:493
2725699814,23cyclictest721-21/usr/sbin/munin08:58:452
2725699814,23cyclictest721-21/usr/sbin/munin08:58:452
2725699813,37cyclictest8251-21/usr/sbin/munin09:13:582
2725699813,33cyclictest13881-21/usr/sbin/munin12:03:472
2725699813,30cyclictest10973-21sh10:11:562
2725699812,33cyclictest2841-21sh09:03:532
27248998163,10cyclictest21536-21runrttasks08:23:411
27248998162,12cyclictest24475-21systemd-run08:33:251
2725699804,67cyclictest24882-21cut08:33:452
2725699804,67cyclictest24882-21cut08:33:452
2725699803,33cyclictest352-21runrttasks10:04:012
2725699803,32cyclictest29498-21sh08:48:232
2725699803,32cyclictest29498-21sh08:48:232
2725699803,27cyclictest10968-21if_eth011:58:482
2725699802,34cyclictest25774-21/usr/sbin/munin11:28:582
2725699802,33cyclictest11970-21/usr/sbin/munin09:18:572
27248998061,13cyclictest16925-21cron09:28:231
27248998052,8cyclictest181rcu_preempt09:46:151
27248998047,12cyclictest181rcu_preempt07:09:231
27248998045,12cyclictest181rcu_preempt10:50:331
27248998044,14cyclictest181rcu_preempt08:08:351
27248998044,14cyclictest181rcu_preempt08:08:351
27248998043,11cyclictest181rcu_preempt09:55:401
27248998034,13cyclictest181rcu_preempt07:48:411
2725699794,30cyclictest31080-21/usr/sbin/munin07:13:562
27256997933,38cyclictest0-21swapper/207:23:262
2725699793,27cyclictest15229-21fschecks_count08:03:462
2725699792,32cyclictest5263-21ssh09:08:402
27248997949,10cyclictest181rcu_preempt10:05:281
27248997945,11cyclictest181rcu_preempt09:42:241
27248997945,11cyclictest181rcu_preempt09:42:241
27248997941,13cyclictest181rcu_preempt08:43:451
27248997941,13cyclictest181rcu_preempt08:43:451
27248997936,22cyclictest181rcu_preempt11:33:241
27248997936,13cyclictest181rcu_preempt08:34:061
27248997936,13cyclictest181rcu_preempt08:34:061
270402796,57sleep20-21swapper/207:02:492
2725699783,31cyclictest20702-21ssh09:33:502
27248997852,20cyclictest13847-21cut12:03:461
27248997852,11cyclictest181rcu_preempt11:03:551
27248997852,11cyclictest181rcu_preempt11:03:551
27248997843,16cyclictest27037-21cp10:38:271
27248997838,11cyclictest181rcu_preempt08:58:461
27248997838,11cyclictest181rcu_preempt08:58:461
2724899783,33cyclictest31-21ksoftirqd/110:53:411
27248997814,55cyclictest18083-21apt-get10:23:401
27248997814,55cyclictest18083-21apt-get10:23:401
2725699773,22cyclictest19612-21idleruntime-cro08:18:232
2725699773,22cyclictest19612-21idleruntime-cro08:18:232
27248997760,12cyclictest10842-21sh09:18:101
27248997748,9cyclictest181rcu_preempt09:20:171
27248997748,13cyclictest181rcu_preempt07:32:381
27248997743,14cyclictest181rcu_preempt12:28:171
27248997741,12cyclictest181rcu_preempt07:39:011
27248997740,16cyclictest181rcu_preempt07:58:521
27248997739,18cyclictest181rcu_preempt11:54:571
27248997739,18cyclictest181rcu_preempt11:54:571
27248997736,13cyclictest181rcu_preempt08:53:341
27248997732,19cyclictest181rcu_preempt10:59:361
27248997550,11cyclictest181rcu_preempt11:08:441
27248997549,10cyclictest181rcu_preempt09:10:401
27248997546,20cyclictest5020-21latency_hist09:08:261
27248997539,15cyclictest181rcu_preempt08:14:001
27248997539,15cyclictest181rcu_preempt08:14:001
27248997534,13cyclictest181rcu_preempt12:00:471
27248997459,10cyclictest27227-21cyclictest10:31:331
27248997443,13cyclictest181rcu_preempt07:48:271
27248997438,9cyclictest181rcu_preempt11:17:411
27248997434,17cyclictest3361-21/usr/sbin/munin09:58:501
27248997432,11cyclictest181rcu_preempt07:13:581
27248997350,10cyclictest181rcu_preempt10:40:501
27248997344,12cyclictest181rcu_preempt08:48:521
27248997335,13cyclictest181rcu_preempt09:49:131
27248997330,12cyclictest181rcu_preempt12:22:091
2638527311,35sleep10-21swapper/106:59:051
27248997246,9cyclictest181rcu_preempt07:07:081
27248997144,9cyclictest181rcu_preempt11:33:481
27248997144,9cyclictest181rcu_preempt11:33:481
27248997134,13cyclictest181rcu_preempt08:18:391
27248997032,7cyclictest181rcu_preempt07:53:461
27248997032,7cyclictest181rcu_preempt07:53:461
27248996947,8cyclictest181rcu_preempt07:21:371
2726399679,33cyclictest0-21swapper/311:18:383
2726399673,30cyclictest0-21swapper/310:39:213
27263996730,24cyclictest23283-21chrt09:38:363
27263996730,24cyclictest23283-21chrt09:38:363
27263996729,26cyclictest29505-21chrt08:48:233
27263996729,26cyclictest29505-21chrt08:48:233
27263996729,25cyclictest17603-21taskset11:16:053
27263996729,23cyclictest22665-21chrt10:30:213
27263996728,26cyclictest10861-21taskset10:11:363
27263996633,20cyclictest8787-21chrt09:13:513
27263996631,23cyclictest8472-21chrt10:08:203
27263996631,23cyclictest22493-21chrt12:18:393
27263996629,25cyclictest6544-21taskset07:36:513
27263996628,26cyclictest3897-21chrt10:53:403
27263996628,26cyclictest27107-21taskset11:32:063
2726399653,51cyclictest462-21chrt10:48:133
2726399653,51cyclictest462-21chrt10:48:133
27263996533,19cyclictest13355-21chrt11:08:523
27263996532,21cyclictest2167-21chrt09:57:253
27263996531,22cyclictest18571-21chrt12:11:423
27263996531,21cyclictest28096-21chrt11:33:403
27263996531,21cyclictest28096-21chrt11:33:403
27263996530,23cyclictest14385-21chrt07:59:023
27263996528,25cyclictest23818-21chrt11:26:113
27263996528,25cyclictest23818-21chrt11:26:113
27263996432,21cyclictest12447-21chrt11:08:233
27263996432,21cyclictest12447-21chrt11:08:233
27263996431,21cyclictest1031-21chrt10:48:423
27263996431,20cyclictest30370-21chrt08:48:563
27263996431,20cyclictest25214-21chrt10:34:023
27263996431,20cyclictest16804-21chrt08:08:463
27263996431,20cyclictest16804-21chrt08:08:463
27263996430,22cyclictest2846-21chrt09:58:353
27263996429,24cyclictest28448-21chrt12:28:393
27263996429,23cyclictest15064-21chrt10:18:463
27263996429,21cyclictest12329-21chrt07:53:563
27263996429,21cyclictest12329-21chrt07:53:563
27263996428,25cyclictest12836-21taskset12:02:173
27263996427,25cyclictest26925-21taskset12:25:073
27263996427,24cyclictest15778-21taskset09:25:103
27263996426,24cyclictest5740-21taskset11:49:053
27263996333,20cyclictest18097-21chrt10:23:363
27263996333,20cyclictest18097-21chrt10:23:363
27263996332,19cyclictest27953-21chrt09:45:383
27263996331,20cyclictest8076-21chrt10:59:243
27263996330,21cyclictest30730-21chrt07:12:373
27263996330,21cyclictest27427-21chrt07:03:393
27263996329,24cyclictest9905-21taskset11:57:233
27263996329,24cyclictest9905-21taskset11:57:233
27263996329,22cyclictest19872-21chrt08:18:383
27263996328,24cyclictest17593-21taskset09:28:473
27263996328,23cyclictest25054-21chrt08:33:493
27263996328,23cyclictest25054-21chrt08:33:493
27263996325,25cyclictest27929-21taskset08:43:253
27263996325,25cyclictest27929-21taskset08:43:253
2708326315,30sleep30-21swapper/307:03:233
27263996229,21cyclictest31519-21taskset09:52:143
27263996229,20cyclictest11769-21chrt09:18:503
27263996228,22cyclictest5200-21chrt09:08:323
27263996132,19cyclictest8747-21chrt07:43:473
27263996131,19cyclictest32488-21chrt11:40:253
27263996130,20cyclictest10502-21chrt07:48:493
27263996129,19cyclictest1599-21chrt08:59:373
27263996129,19cyclictest1599-21chrt08:59:373
27263996128,22cyclictest21962-21chrt08:23:523
27263996128,21cyclictest15026-21chrt12:05:053
27263996127,23cyclictest20309-21taskset09:33:383
27263996127,22cyclictest4418-21chrt09:07:023
27263996127,22cyclictest19374-21chrt08:15:383
27263996127,22cyclictest19374-21chrt08:15:383
27263996029,20cyclictest2915-21chrt07:24:523
27263996028,20cyclictest32465-21chrt08:54:513
27263996028,20cyclictest23055-21chrt08:28:343
27263996027,21cyclictest7573-21chrt07:38:573
27263996025,25cyclictest20870-21taskset12:14:413
27263995928,20cyclictest32198-21chrt07:15:253
27263995927,21cyclictest3471-21chrt07:28:373
27263995927,21cyclictest16320-21chrt08:08:133
27263995926,21cyclictest1325-21chrt07:20:063
27263995826,21cyclictest13442-21chrt10:15:213
27263995826,21cyclictest13442-21chrt10:15:213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional