You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-06 - 16:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Mon Jul 06, 2026 12:44:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6612991212,32cyclictest6613-21latency_hist06:57:400
66129911545,62cyclictest0-21swapper/010:57:460
66129911442,9cyclictest0-21swapper/009:53:530
66129911442,9cyclictest0-21swapper/009:53:530
66129911333,12cyclictest0-21swapper/010:03:090
66129911036,12cyclictest0-21swapper/008:42:350
66129910834,13cyclictest0-21swapper/012:07:510
6612991083,29cyclictest24030-21fschecks_time09:14:040
6612991083,29cyclictest24030-21fschecks_time09:14:040
6612991065,45cyclictest7171-21/usr/sbin/munin12:23:080
6612991063,39cyclictest25403-21latency_hist07:52:380
6612991063,22cyclictest13050-21grep10:43:120
6612991063,22cyclictest13050-21grep10:43:120
6630991053,81cyclictest24232-21chrt11:57:553
66129910534,12cyclictest0-21swapper/009:43:120
66129910534,12cyclictest0-21swapper/009:43:120
66129910528,12cyclictest0-21swapper/011:38:070
66129910523,19cyclictest0-21swapper/008:22:350
6612991044,36cyclictest7060-21grep08:33:590
6612991043,26cyclictest26742-21apt-get10:12:480
66129910428,11cyclictest0-21swapper/011:21:350
6612991042,34cyclictest32751-21cstates10:22:520
6612991035,35cyclictest9404-21fschecks_count11:32:560
6612991035,31cyclictest27058-21/usr/sbin/munin07:58:110
6612991034,45cyclictest4042-21ssh12:18:050
66129910337,9cyclictest0-21swapper/007:18:020
66129910337,9cyclictest0-21swapper/007:18:020
6612991033,33cyclictest24081-21fschecks_time10:07:580
6612991033,33cyclictest24081-21fschecks_time10:07:580
6612991032,37cyclictest7269-21grep11:28:120
639121036,15sleep10-21swapper/106:56:371
6612991023,40cyclictest19743-21wc07:33:110
6612991023,39cyclictest8374-21grep07:02:380
6612991023,38cyclictest554-21fschecks_time09:28:510
6612991023,38cyclictest23795-21latency_hist07:47:380
6612991023,38cyclictest23795-21latency_hist07:47:380
6612991022,26cyclictest26644-21ssh11:07:380
66199910153,40cyclictest0-21swapper/107:03:451
66199910153,40cyclictest0-21swapper/107:03:451
6612991015,31cyclictest352-21runrttasks10:42:200
66129910145,9cyclictest536-21ssh12:12:450
6612991014,32cyclictest10541-21latency_hist07:07:380
6612991014,32cyclictest10541-21latency_hist07:07:380
6612991014,28cyclictest18609-21open_inodes10:53:070
6612991014,28cyclictest18609-21open_inodes10:53:070
6612991012,42cyclictest18476-21ntpq09:58:070
6612991004,36cyclictest6378-21fschecks_time10:32:570
6612991004,30cyclictest23639-21idleruntime-cro11:57:350
6612991004,30cyclictest23639-21idleruntime-cro11:57:350
6612991003,36cyclictest30932-21processes08:08:090
6612991003,36cyclictest30932-21processes08:08:090
6612991003,36cyclictest11660-21mailstats08:48:080
6612991003,36cyclictest11660-21mailstats08:48:080
6612991003,35cyclictest9340-21/usr/sbin/munin08:43:050
6612991003,35cyclictest21014-21find09:08:090
6612991002,38cyclictest26974-21latency_hist07:57:380
66129910017,33cyclictest352-21runrttasks11:13:570
66129910017,33cyclictest352-21runrttasks11:13:570
661299994,36cyclictest3465-21fschecks_count11:22:540
661299994,36cyclictest3465-21fschecks_count11:22:540
661299993,34cyclictest16096-21grep07:23:300
661299993,32cyclictest5875-21/usr/sbin/munin09:37:580
661299992,38cyclictest3778-21uname10:28:060
661299992,35cyclictest30561-21timerandwakeup10:18:110
661299992,35cyclictest30561-21timerandwakeup10:18:110
661299992,34cyclictest27227-21cstates12:02:530
661299992,32cyclictest18327-21grep11:48:000
661299986,31cyclictest32098-21mktemp09:27:380
661299986,31cyclictest32098-21mktemp09:27:380
6612999838,7cyclictest0-21swapper/009:07:370
6612999836,11cyclictest297-21systemd-logind07:17:360
661299983,31cyclictest31908-21/usr/sbin/munin08:13:040
661299983,31cyclictest31908-21/usr/sbin/munin08:13:040
6612999828,12cyclictest0-21swapper/007:37:520
661299982,35cyclictest3822-21grep09:33:110
661299974,32cyclictest5996-21latency_hist08:32:390
661299974,32cyclictest5996-21latency_hist08:32:390
661299974,30cyclictest352-21runrttasks10:49:090
6612999732,12cyclictest0-21swapper/011:09:050
6612999732,10cyclictest11950irq/279-mmc208:57:340
661299972,26cyclictest15246-21expr11:42:550
661299972,26cyclictest15246-21expr11:42:550
232772970,5sleep123279-21latency_hist09:12:381
661299964,35cyclictest3386-21grep08:23:090
661299962,36cyclictest27804-21ssh09:19:230
661999954,78cyclictest6192-21sshd08:33:271
661299954,34cyclictest352-21runrttasks09:49:060
661299954,34cyclictest17120-21latency_hist09:02:380
661299954,34cyclictest17120-21latency_hist09:02:380
661299953,33cyclictest29539-21cat08:03:120
661999943,28cyclictest352-21runrttasks10:38:551
661299934,32cyclictest25184-21find11:59:050
6619999252,34cyclictest11260-21ssh09:46:481
6619999252,34cyclictest11260-21ssh09:46:481
661999923,63cyclictest23634-21apt-get11:57:531
646429212,52sleep22374-21systemd-journal06:57:352
662399912,69cyclictest5948-21kworker/u17:4+ext4-rsv-conversion11:57:552
661999912,53cyclictest27077-21apt-get07:57:461
661299914,31cyclictest352-21runrttasks07:08:260
6619999046,36cyclictest0-21swapper/111:17:361
6619999046,36cyclictest0-21swapper/111:17:361
661299893,33cyclictest17386-21/usr/sbin/munin07:28:090
6619998867,15cyclictest5636-21latency_hist09:37:371
661999882,29cyclictest4831-21ssh11:24:371
661999882,29cyclictest4831-21ssh11:24:371
661999868,58cyclictest32463-21apt-get09:27:511
6619998644,37cyclictest32522-21kworker/u17:4+events_unbound09:52:341
6623998566,12cyclictest2374-21systemd-journal10:57:352
6623998566,12cyclictest2374-21systemd-journal10:57:352
6619998565,14cyclictest17754-21true11:47:351
6619998565,14cyclictest17754-21true11:47:351
661999852,38cyclictest8751-21latency_hist10:37:381
6623998463,14cyclictest6603-21cyclictest09:02:522
662399844,56cyclictest32498-21ssh09:27:512
6619998437,41cyclictest0-21swapper/111:17:501
661999843,32cyclictest18051-21/usr/sbin/munin09:03:091
661999842,34cyclictest32263-21date09:27:391
661999842,34cyclictest32263-21date09:27:391
6623998346,8cyclictest181rcu_preempt09:08:022
6619998364,13cyclictest12082-21acpi11:37:431
661999833,33cyclictest15420-21/usr/sbin/munin09:53:521
661999833,33cyclictest15420-21/usr/sbin/munin09:53:521
661999832,32cyclictest27493-21/usr/sbin/munin10:13:111
661999823,72cyclictest0-21swapper/110:02:471
661999823,37cyclictest7756-21sshd08:37:491
661999823,34cyclictest23873-21ssh11:02:471
661999823,34cyclictest12519-21fschecks_count07:12:561
661999822,32cyclictest7799-21rm12:23:521
661999822,32cyclictest22902-21diskmemload11:00:451
661999822,32cyclictest18360-21head07:28:121
6623998160,12cyclictest27677-21timerwakeupswit10:13:132
6623998157,18cyclictest4180-21sh11:23:092
6623998157,18cyclictest4180-21sh11:23:092
661999813,7cyclictest0-21swapper/107:17:491
661999813,7cyclictest0-21swapper/107:17:491
661999813,36cyclictest11875-21cat11:37:371
661999813,35cyclictest2684-21apt-get08:22:471
661999813,35cyclictest19462-21/usr/sbin/munin07:33:021
661999813,34cyclictest4464-21/usr/sbin/munin08:27:531
661999813,34cyclictest4464-21/usr/sbin/munin08:27:531
661999813,34cyclictest24373-21ntpq07:48:021
661999813,31cyclictest17576-21sh10:52:251
661999812,7cyclictest0-21swapper/108:57:481
661999812,7cyclictest0-21swapper/108:57:481
661999812,65cyclictest1628-21runrttasks12:13:131
661999812,35cyclictest11400-21/usr/sbin/munin08:48:001
661999812,35cyclictest11400-21/usr/sbin/munin08:48:001
661999812,32cyclictest8394-21taskset09:42:081
661999812,30cyclictest17841-21apt09:57:451
661999804,70cyclictest0-21swapper/111:27:501
6619998039,33cyclictest0-21swapper/109:12:421
6619998039,33cyclictest0-21swapper/109:12:421
661999803,7cyclictest0-21swapper/110:07:501
661999803,7cyclictest0-21swapper/110:07:501
661999803,72cyclictest0-21swapper/111:52:521
661999803,72cyclictest0-21swapper/111:52:521
661999803,6cyclictest0-21swapper/110:17:501
661999803,6cyclictest0-21swapper/110:17:501
661999802,8cyclictest0-21swapper/111:47:481
661999802,35cyclictest14658-21latency_hist10:47:381
661999802,35cyclictest14658-21latency_hist10:47:381
661999802,33cyclictest87-21kswapd007:22:511
661999802,32cyclictest4625-21sh12:18:171
661999793,8cyclictest0-21swapper/108:42:471
661999793,35cyclictest545-21cat10:22:581
661999793,33cyclictest18593-21sh10:53:061
661999793,33cyclictest18593-21sh10:53:061
661999793,30cyclictest26280-21apt09:17:451
661999792,37cyclictest30258-21seq12:07:541
661999792,35cyclictest13943-21latency_hist08:57:381
661999792,33cyclictest3168-21ssh10:27:481
661999792,32cyclictest29732-21latency_hist12:07:391
6623997848,19cyclictest23626-21latency_hist11:57:382
6623997848,19cyclictest23626-21latency_hist11:57:382
662399783,56cyclictest6193-21sshd08:33:312
6619997839,33cyclictest30560-21fschecks_time08:07:581
6619997839,33cyclictest30560-21fschecks_time08:07:581
661999783,35cyclictest23013-21/usr/sbin/munin07:43:121
661999783,35cyclictest23013-21/usr/sbin/munin07:43:121
661999783,27cyclictest7662-21sort06:58:131
661999782,36cyclictest26097-21mailstats07:53:081
6623997757,12cyclictest2374-21systemd-journal10:02:352
6623997755,16cyclictest9402-21fschecks_time10:37:582
661999772,33cyclictest11371-21/usr/sbin/munin07:08:071
661999772,30cyclictest21179-21/usr/sbin/munin07:38:081
6623997648,19cyclictest12543-21cut07:12:572
6623997647,18cyclictest20095-21kworker/u17:3+events_unbound07:42:352
661999763,32cyclictest26858-21ssh11:07:461
661999763,31cyclictest1089-21cpu08:17:511
661999763,30cyclictest813-21latency_hist08:17:351
661999763,30cyclictest813-21latency_hist08:17:351
661999762,34cyclictest29473-21/usr/sbin/munin08:03:121
6623997554,11cyclictest0-21swapper/210:22:572
6623997457,12cyclictest1328-21sh12:13:102
6623997455,13cyclictest5656-21latency_hist08:32:382
6623997455,13cyclictest5656-21latency_hist08:32:382
6623997354,13cyclictest25559-21sh09:16:272
6623997354,13cyclictest25559-21sh09:16:272
6623997352,7cyclictest181rcu_preempt09:33:262
6623997343,19cyclictest0-21swapper/208:08:102
6623997343,19cyclictest0-21swapper/208:08:102
662399732,49cyclictest10808-21latency_hist08:47:382
6623997257,10cyclictest6603-21cyclictest08:38:052
662399723,43cyclictest28606-21latency_hist08:02:382
171462720,9sleep30-21swapper/309:02:393
171462720,9sleep30-21swapper/309:02:393
6623997153,11cyclictest6603-21cyclictest10:04:072
6623997152,14cyclictest352-21runrttasks07:23:332
6623997150,15cyclictest18265-21ssh11:47:592
6623997149,16cyclictest352-21runrttasks12:20:262
662399713,51cyclictest975-21wc08:17:382
662399713,51cyclictest975-21wc08:17:382
550127112,33sleep30-21swapper/306:53:083
6623997052,12cyclictest6853-21diskmemload11:28:062
6623996953,8cyclictest0-21swapper/211:03:102
6623996953,10cyclictest14628-21apt-key09:52:362
6623996949,10cyclictest0-21swapper/210:08:132
6623996949,10cyclictest0-21swapper/210:08:132
662399693,35cyclictest6983-21fschecks_count06:57:582
6623996851,11cyclictest23539-21kworker/u17:007:47:362
6623996851,11cyclictest23539-21kworker/u17:007:47:362
6623996847,7cyclictest181rcu_preempt11:18:042
6623996846,8cyclictest181rcu_preempt10:28:302
6623996846,13cyclictest0-21swapper/208:58:112
6623996846,13cyclictest0-21swapper/208:58:112
662399682,45cyclictest9075-21apt-get07:02:522
662399682,45cyclictest9075-21apt-get07:02:522
662399675,42cyclictest27868-21unixbench_multi11:08:122
662399673,45cyclictest352-21runrttasks12:11:172
6623996726,31cyclictest12820-21switchtime10:43:102
6623996726,31cyclictest12820-21switchtime10:43:102
6623996722,32cyclictest6461-21basename10:32:582
663099663,41cyclictest5595-21taskset09:37:353
6623996649,12cyclictest6603-21cyclictest12:22:452
6623996648,12cyclictest14796-21sh11:42:372
6623996648,12cyclictest10184-21unixbench_multi11:33:112
6623996640,16cyclictest0-21swapper/209:43:102
6623996640,16cyclictest0-21swapper/209:43:102
662399663,48cyclictest11194-21fschecks_count08:47:562
662399663,48cyclictest11194-21fschecks_count08:47:562
662399662,33cyclictest27386-21fschecks_time12:02:572
6623996618,12cyclictest381ktimers/210:57:582
663099652,40cyclictest917-21kworker/3:1+events11:17:463
6623996546,13cyclictest15264-21latency_hist07:22:392
6623996546,13cyclictest15264-21latency_hist07:22:392
662399654,39cyclictest15631-21/usr/sbin/munin09:53:572
662399654,39cyclictest15631-21/usr/sbin/munin09:53:572
662399653,44cyclictest352-21runrttasks08:03:272
662399652,46cyclictest32445-21apt11:17:352
662399652,46cyclictest32445-21apt11:17:352
6623996445,11cyclictest0-21swapper/210:48:082
6623996444,7cyclictest181rcu_preempt07:07:522
662399643,43cyclictest29216-21/usr/sbin/munin09:22:552
662399643,43cyclictest29216-21/usr/sbin/munin09:22:552
662399642,42cyclictest25296-21latency_hist07:52:372
6623996343,8cyclictest181rcu_preempt07:29:392
662399633,42cyclictest3214-21ntp_states08:23:052
662399633,39cyclictest1176-21cstates08:17:522
662399633,36cyclictest13943-21latency_hist08:57:392
6623996244,13cyclictest26694-21sh09:18:002
6623996241,13cyclictest0-21swapper/211:42:592
6623996241,13cyclictest0-21swapper/211:42:592
662399622,42cyclictest352-21runrttasks07:34:532
662399622,37cyclictest29976-21fschecks_time10:17:572
662399622,37cyclictest29976-21fschecks_time10:17:572
6623996139,9cyclictest181rcu_preempt09:40:272
662399613,42cyclictest25688-21fschecks_count07:52:572
663099603,40cyclictest28973-21taskset12:05:263
6630996024,24cyclictest28011-21taskset07:58:113
6630996022,24cyclictest9481-21chrt08:42:533
663099593,39cyclictest20317-21taskset09:07:503
663099593,36cyclictest6298-21chrt11:27:513
6630995923,24cyclictest13991-21chrt08:57:363
6630995922,24cyclictest31012-21chrt09:25:063
6630995922,24cyclictest31012-21chrt09:25:063
6630995822,23cyclictest7083-21taskset12:23:053
6630995821,25cyclictest13486-21taskset07:15:303
663099573,38cyclictest25319-21taskset09:15:453
663099573,38cyclictest25319-21taskset09:15:453
663099572,39cyclictest28764-21chrt11:10:323
6630995722,22cyclictest13934-21chrt07:17:493
6630995722,22cyclictest13934-21chrt07:17:493
6630995720,24cyclictest2644-21taskset09:32:363
6630995720,23cyclictest571-21taskset12:12:473
6630995720,23cyclictest30246-21taskset12:07:533
6630995720,23cyclictest12066-21taskset09:47:533
6630995622,21cyclictest15146-21chrt09:52:563
6630995622,21cyclictest15146-21chrt09:52:563
6630995621,22cyclictest19597-21taskset07:33:083
6630995620,23cyclictest17977-21chrt10:52:483
6630995620,23cyclictest17977-21chrt10:52:483
6630995620,22cyclictest1761-21taskset08:18:083
663099554,34cyclictest9531-21chrt11:32:583
663099552,36cyclictest8281-21chrt07:02:383
6630995523,20cyclictest26563-21chrt10:12:383
6630995523,20cyclictest26563-21chrt10:12:383
6630995522,20cyclictest2317-21taskset10:26:123
6630995521,21cyclictest25796-21chrt11:05:333
6630995521,21cyclictest20550-21chrt11:52:163
6630995516,25cyclictest6116-21taskset12:22:323
663099543,37cyclictest7651-21chrt10:34:273
6630995422,19cyclictest16781-21chrt07:23:453
6630995421,21cyclictest9567-21chrt07:03:353
6630995421,21cyclictest9567-21chrt07:03:353
6630995421,21cyclictest28041-21taskset09:20:033
6630995420,22cyclictest7950-21chrt09:40:513
6630995420,22cyclictest5302-21taskset08:28:133
6630995420,22cyclictest5302-21taskset08:28:133
6630995420,22cyclictest4131-21chrt11:23:093
6630995420,22cyclictest4131-21chrt11:23:093
6630995420,20cyclictest17850-21chrt09:57:463
6630995420,20cyclictest10242-21chrt09:43:563
6630995420,20cyclictest10242-21chrt09:43:563
6630995419,22cyclictest17284-21taskset10:51:343
663099534,38cyclictest3229-21chrt10:27:503
663099534,30cyclictest917-21kworker/3:1+events07:07:503
663099533,33cyclictest23637-21chrt11:57:353
663099533,33cyclictest23637-21chrt11:57:353
663099532,32cyclictest11874-21chrt10:42:383
6630995321,20cyclictest699-21chrt08:16:223
6630995321,20cyclictest699-21chrt08:16:223
6630995321,19cyclictest22227-21chrt07:42:483
6630995321,19cyclictest22227-21chrt07:42:483
6630995320,21cyclictest23288-21chrt10:07:113
6630995319,23cyclictest21602-21chrt07:38:153
6630995319,22cyclictest18764-21chrt07:32:353
663099522,36cyclictest17107-21chrt11:45:553
663099522,36cyclictest17107-21chrt11:45:553
6630995222,19cyclictest31788-21chrt08:12:373
6630995222,19cyclictest31788-21chrt08:12:373
6630995220,21cyclictest29994-21chrt11:12:553
6630995220,21cyclictest29994-21chrt11:12:553
6630995220,21cyclictest29705-21chrt10:17:493
6630995220,21cyclictest29705-21chrt10:17:493
6630995219,22cyclictest18261-21chrt09:03:113
6630995219,20cyclictest8773-21chrt08:38:403
663099512,36cyclictest13696-21chrt11:39:343
6630995120,19cyclictest25237-21chrt07:52:363
6630995119,21cyclictest6136-21chrt08:32:423
6630995118,21cyclictest20925-21chrt10:57:473
6630995118,21cyclictest12135-21chrt08:50:063
6630995118,21cyclictest12135-21chrt08:50:063
6630995118,20cyclictest29742-21chrt08:04:073
6630995118,20cyclictest13900-21chrt10:45:263
6630995118,20cyclictest13900-21chrt10:45:263
6630995114,24cyclictest27895-21taskset10:13:233
6630995018,20cyclictest4085-21chrt08:27:363
663099482,34cyclictest25690-21chrt07:52:573
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional