You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 15:08
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Sun May 10, 2026 12:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2316421290,2sleep1291rcuc/111:05:521
6112991135,32cyclictest6118-21latency_hist07:01:310
6112991135,32cyclictest6118-21latency_hist07:01:310
6112991083,33cyclictest32287-21/usr/sbin/munin10:26:540
6112991083,29cyclictest352-21runrttasks11:01:370
61129910655,45cyclictest27291-21ntp_offset11:11:540
6112991065,25cyclictest21643-21kernelversion11:56:510
61129910651,49cyclictest20274-21timerandwakeup09:12:040
61129910651,49cyclictest20274-21timerandwakeup09:12:040
61129910633,8cyclictest0-21swapper/009:01:470
6112991063,33cyclictest11417-21cron08:56:270
61129910632,7cyclictest0-21swapper/010:42:120
6121991057,76cyclictest25800-21sh09:22:061
61219910510,79cyclictest14242-21apt-get09:56:471
6112991052,29cyclictest6684-21latency_hist12:26:310
6126991044,48cyclictest29347-21ssh10:21:402
6112991043,45cyclictest12062-21fschecks_count10:46:510
6112991033,31cyclictest2820-21/usr/sbin/munin10:31:530
6112991033,31cyclictest2820-21/usr/sbin/munin10:31:530
6112991032,29cyclictest1980-21latency_hist09:36:310
6112991025,25cyclictest7340-21fschecks_time12:27:270
6112991023,41cyclictest3555-21kernelversion11:26:530
6112991023,41cyclictest3555-21kernelversion11:26:530
6112991023,34cyclictest20140-21/usr/sbin/munin10:06:540
6112991023,27cyclictest17007-21runrttasks10:01:300
6018210216,60sleep26035-21systemd-run07:01:292
6018210216,60sleep26035-21systemd-run07:01:292
6126991015,52cyclictest352-21runrttasks10:41:402
6112991013,37cyclictest4228-21sendmail_mailqu08:32:000
6112991013,35cyclictest14518-21kworker/u17:0+rpciod11:46:370
6112991013,27cyclictest8893-21cat09:47:040
6112991012,34cyclictest11318-21fschecks_time09:51:500
61129910034,8cyclictest0-21swapper/008:11:580
6112991003,35cyclictest32752-21latency_hist08:21:310
6112991003,34cyclictest23823-21cstates11:06:440
6112991003,26cyclictest32624-21sort11:21:380
6112991003,25cyclictest8407-21cpuspeed07:06:460
61129910031,8cyclictest0-21swapper/012:02:000
61129910031,11cyclictest0-21swapper/009:21:310
61129910026,11cyclictest0-21swapper/010:06:270
61129910026,11cyclictest0-21swapper/010:06:270
6112991002,27cyclictest29322-21mailstats09:27:030
6126999951,41cyclictest1-21systemd10:32:132
6126999951,41cyclictest1-21systemd10:32:132
6126999950,43cyclictest2507-21/usr/sbin/munin10:31:292
611299993,42cyclictest13668-21fschecks_time07:21:490
611299993,35cyclictest20590-21latency_hist11:01:300
611299993,35cyclictest20590-21latency_hist11:01:300
611299993,30cyclictest9307-21/usr/sbin/munin11:36:510
611299993,30cyclictest7076-21ls07:02:020
611299993,30cyclictest7076-21ls07:02:020
6112999931,14cyclictest25599-21sh09:21:370
611299992,39cyclictest19899-21apt-get07:41:420
611299992,36cyclictest17077-21sendmail_mailqu09:07:000
611299992,32cyclictest18209-21/usr/sbin/munin11:51:520
611299992,32cyclictest18209-21/usr/sbin/munin11:51:520
611299985,27cyclictest31698-21taskset12:13:240
611299985,27cyclictest31698-21taskset12:13:240
6112999834,8cyclictest0-21swapper/009:46:300
611299983,38cyclictest352-21runrttasks11:36:030
611299983,38cyclictest352-21runrttasks11:36:030
611299983,38cyclictest21479-21/usr/sbin/munin07:46:500
611299983,38cyclictest21479-21/usr/sbin/munin07:46:500
611299983,32cyclictest9962-21/usr/sbin/munin07:11:480
611299983,32cyclictest9962-21/usr/sbin/munin07:11:480
611299983,28cyclictest13033-21diskmemload10:51:480
520029813,59sleep15206-21open_files06:57:081
520029813,59sleep15206-21open_files06:57:081
611299974,33cyclictest8679-21fschecks_time08:46:510
611299973,36cyclictest6618-21cat10:37:010
611299973,34cyclictest19784-21latency_hist07:41:300
611299973,32cyclictest27540-21/usr/sbin/munin12:06:520
611299973,30cyclictest2929-21grep08:27:050
611299973,30cyclictest2929-21grep08:27:050
6112999731,60cyclictest0-21swapper/007:17:110
612699966,54cyclictest0-21swapper/211:06:382
6121999670,17cyclictest13664-21sh09:01:491
611299964,24cyclictest603-21/usr/sbin/munin12:16:550
611299963,35cyclictest23280-21awk10:11:430
611299963,35cyclictest23280-21awk10:11:430
611299963,34cyclictest29025-21apt10:21:270
611299963,32cyclictest25368-21find07:56:590
611299963,32cyclictest15565-21ntp_kernel_pll_07:26:570
611299963,31cyclictest16692-21cpu07:31:430
611299954,34cyclictest6812-21sort08:41:360
611299953,30cyclictest11951-21fschecks_time08:56:500
6112999531,8cyclictest0-21swapper/010:21:440
611299952,24cyclictest1032-21mailstats08:22:000
6126999445,39cyclictest6793-21apt-get12:27:162
611299944,34cyclictest23112-21/usr/sbin/munin07:52:040
611299944,32cyclictest12972-21processes11:42:000
611299943,34cyclictest29795-21cstates11:16:450
611299943,34cyclictest29795-21cstates11:16:450
611299943,32cyclictest5632-21wc08:36:530
611299943,27cyclictest28237-21fschecks_time08:06:510
611299942,32cyclictest26634-21fschecks_time08:01:500
6126999342,45cyclictest2374-21systemd-journal09:15:362
6126999342,45cyclictest2374-21systemd-journal09:15:362
611299932,10cyclictest3146-21grep09:37:040
612699923,38cyclictest3273-21cron12:21:262
6126999042,41cyclictest12319-21sh11:41:422
6126998960,20cyclictest6513-21fschecks_count07:01:482
6126998960,20cyclictest6513-21fschecks_count07:01:482
612699882,39cyclictest14781-21latency_hist07:26:302
6121998813,53cyclictest28576-21ssh09:26:381
6126998666,12cyclictest0-21swapper/209:37:032
6126998664,16cyclictest29427-21latency_hist08:11:302
612699864,8cyclictest0-21swapper/208:46:432
612699863,59cyclictest11828-21apt-get07:16:412
612699863,28cyclictest24191-21sh12:01:402
612699862,74cyclictest13683-21apt-get09:56:422
612699862,36cyclictest23434-21grep09:17:042
612699862,28cyclictest352-21runrttasks08:01:402
6132998510,61cyclictest6936-21chrt12:27:023
6126998546,32cyclictest0-21swapper/210:11:432
6126998546,32cyclictest0-21swapper/210:11:432
612699854,34cyclictest17979-21/usr/sbin/munin10:02:022
612699854,34cyclictest17979-21/usr/sbin/munin10:02:022
612699853,38cyclictest3270-21cstates11:26:452
612699853,38cyclictest3270-21cstates11:26:452
612699853,35cyclictest353-21/usr/sbin/munin08:21:382
612699853,34cyclictest16858-21fschecks_count07:31:482
612699852,37cyclictest6297-21apt-config12:26:272
6121998554,22cyclictest14955-21latency_hist07:26:301
6126998467,11cyclictest23424-21ssh11:06:282
6126998462,16cyclictest18361-21sh10:56:562
6126998462,16cyclictest18361-21sh10:56:562
6126998462,16cyclictest11623-21apt-key07:16:312
6126998444,33cyclictest0-21swapper/209:07:002
6126998444,32cyclictest0-21swapper/207:56:302
612699843,34cyclictest21215-21rm11:56:372
6126998433,43cyclictest6120-21fschecks_count10:36:492
6121998465,12cyclictest19859-21idleruntime-cro10:06:271
6121998465,12cyclictest19859-21idleruntime-cro10:06:271
612699835,7cyclictest0-21swapper/210:46:442
612699833,38cyclictest25825-21apt-get10:16:392
612699833,36cyclictest19091-21/usr/sbin/munin11:52:002
612699833,36cyclictest19091-21/usr/sbin/munin11:52:002
612199833,71cyclictest6248-21apt-get12:26:411
612699823,35cyclictest6160-21unixbench_singl08:37:042
612699823,33cyclictest29263-21sshd09:27:022
612699823,29cyclictest11429-21cut08:56:272
612699823,27cyclictest25737-21/usr/sbin/munin07:57:052
612699822,34cyclictest11647-21ssh09:51:592
612199822,55cyclictest11324-21apt-get07:16:421
612699813,33cyclictest12178-21ntp_kernel_err08:56:552
6126998129,47cyclictest0-21swapper/211:17:012
6126998129,47cyclictest0-21swapper/211:17:012
612699812,35cyclictest5270-21fschecks_count09:41:472
612699812,35cyclictest2171-21cut08:26:502
612699812,35cyclictest2171-21cut08:26:502
6126998110,30cyclictest17763-21diskmemload11:51:052
612699804,25cyclictest30026-21ls08:11:532
612699803,37cyclictest14813-21ssh10:51:422
612699803,34cyclictest8633-21ntp_states09:46:582
612699803,33cyclictest15656-21processes07:27:012
612699803,32cyclictest18551-21cut07:36:492
612699803,24cyclictest6340-21/usr/sbin/munin11:32:472
612699803,24cyclictest6340-21/usr/sbin/munin11:32:472
612699802,32cyclictest21668-21/usr/sbin/munin07:46:482
612699802,32cyclictest21668-21/usr/sbin/munin07:46:482
612699802,20cyclictest13345-21apt-get09:01:372
6121998057,15cyclictest25221-21ssh09:21:291
6121998055,17cyclictest13033-21diskmemload09:53:471
612699793,36cyclictest11325-21idleruntime-cro07:16:272
612699793,34cyclictest6975-21/usr/sbin/munin08:41:442
612699793,32cyclictest2192-21taskset11:24:422
612699793,22cyclictest31562-21apt-get09:31:382
6121997959,15cyclictest6104-21cyclictest10:17:341
612699784,34cyclictest12012-21ssh11:41:282
612699783,34cyclictest26871-21ssh11:11:442
612699783,34cyclictest20281-21cstates10:06:432
612699783,32cyclictest25620-21apt09:21:382
612699783,31cyclictest4253-21mailstats08:32:002
612699783,30cyclictest7917-21grep07:06:332
6121997828,35cyclictest31-21ksoftirqd/111:16:521
6121997828,35cyclictest31-21ksoftirqd/111:16:521
612699773,20cyclictest30335-21/usr/sbin/munin12:11:472
612699773,20cyclictest30335-21/usr/sbin/munin12:11:472
612699772,28cyclictest31505-21idleruntime08:16:512
612699763,26cyclictest20664-21timerandwakeup07:42:052
612699762,33cyclictest28644-21taskset12:08:142
6121997658,12cyclictest2374-21systemd-journal07:16:281
6121997658,12cyclictest2374-21systemd-journal07:16:281
6121997652,15cyclictest9675-21cron07:11:271
612199764,40cyclictest31-21ksoftirqd/108:17:481
6121997630,11cyclictest0-21swapper/111:51:481
6121997630,11cyclictest0-21swapper/111:51:481
612199762,41cyclictest31-21ksoftirqd/108:16:021
612199755,37cyclictest31-21ksoftirqd/111:46:331
6121997552,16cyclictest277-21dbus-daemon12:26:281
6121997552,14cyclictest6104-21cyclictest10:07:061
6121997525,20cyclictest31-21ksoftirqd/112:00:531
612199752,51cyclictest31-21ksoftirqd/107:31:031
6121997524,27cyclictest31-21ksoftirqd/111:32:421
6121997524,27cyclictest31-21ksoftirqd/111:32:421
612199752,24cyclictest31-21ksoftirqd/109:16:151
612199752,24cyclictest31-21ksoftirqd/109:16:151
6121997416,40cyclictest18871-21grep07:36:591
6121997347,19cyclictest3188-21kworker/u17:3+events_unbound08:31:261
6121997347,19cyclictest3188-21kworker/u17:3+events_unbound08:31:261
612199731,30cyclictest31-21ksoftirqd/110:23:151
612199725,40cyclictest31-21ksoftirqd/107:46:131
6121997250,16cyclictest11994-21sh10:46:481
612199723,43cyclictest24621-21latency_hist07:56:301
6121997145,7cyclictest181rcu_preempt11:42:511
6121997145,16cyclictest6149-21latency_hist11:31:311
6121997145,16cyclictest6149-21latency_hist11:31:311
612199712,51cyclictest26085-21latency_hist08:01:271
6121997053,11cyclictest20423-21awk11:01:271
6121997053,11cyclictest20423-21awk11:01:271
612199704,43cyclictest31-21ksoftirqd/110:13:171
612199704,43cyclictest31-21ksoftirqd/110:13:171
6121997044,17cyclictest0-21swapper/109:36:501
612199703,25cyclictest31-21ksoftirqd/110:36:441
612199697,44cyclictest22842-21modprobe07:51:271
612199697,44cyclictest22842-21modprobe07:51:271
6121996953,9cyclictest0-21swapper/108:36:431
6121996950,7cyclictest181rcu_preempt12:20:161
6121996946,12cyclictest0-21swapper/110:26:511
6121996940,20cyclictest0-21swapper/111:06:491
612199693,47cyclictest5273-21fschecks_count09:41:471
612199683,44cyclictest9090-21grep09:47:061
612199682,34cyclictest31-21ksoftirqd/107:32:201
612199674,41cyclictest352-21runrttasks08:22:291
612199673,40cyclictest31-21ksoftirqd/107:05:471
612199673,40cyclictest31-21ksoftirqd/107:05:471
612199673,35cyclictest27662-21timerwakeupswit11:12:021
612199673,35cyclictest11381-21runrttasks10:46:131
6121996717,13cyclictest0-21swapper/110:36:061
6121996717,13cyclictest0-21swapper/110:36:061
612199667,26cyclictest31-21ksoftirqd/112:03:481
6121996647,7cyclictest181rcu_preempt08:48:481
6121996647,13cyclictest11085-21ntpd08:56:561
612199659,20cyclictest31-21ksoftirqd/112:14:181
612199659,20cyclictest31-21ksoftirqd/112:14:181
612199655,35cyclictest31-21ksoftirqd/109:09:571
6121996544,7cyclictest181rcu_preempt08:05:071
612199653,34cyclictest10344-21fschecks_time08:51:511
6121996524,24cyclictest31-21ksoftirqd/110:51:551
612199643,40cyclictest352-21runrttasks08:10:141
6121996417,11cyclictest291rcuc/112:06:591
6121996338,8cyclictest181rcu_preempt09:35:341
6121996310,27cyclictest31-21ksoftirqd/111:22:431
613299622,44cyclictest917-21kworker/3:1+events09:56:393
612199623,44cyclictest7097-21fschecks_time08:41:481
612199622,38cyclictest31-21ksoftirqd/111:37:091
524126214,32sleep30-21swapper/306:57:103
524126214,32sleep30-21swapper/306:57:103
613299603,40cyclictest26174-21taskset09:22:453
613299602,40cyclictest917-21kworker/3:1+events08:21:493
612199605,33cyclictest31-21ksoftirqd/108:35:181
613299592,41cyclictest31954-21taskset08:17:023
613299583,37cyclictest7073-21chrt07:02:023
613299583,37cyclictest7073-21chrt07:02:023
613299582,40cyclictest16723-21chrt11:48:063
613299582,39cyclictest9072-21chrt07:07:043
613299575,32cyclictest0-21swapper/311:31:463
613299575,32cyclictest0-21swapper/311:31:463
613299573,41cyclictest1381-21taskset09:35:263
613299573,37cyclictest18488-21taskset07:36:493
613299573,36cyclictest29727-21taskset10:21:523
613299572,37cyclictest31017-21chrt08:16:303
613299563,42cyclictest20876-21chrt09:13:083
613299563,42cyclictest20876-21chrt09:13:083
613299562,37cyclictest1048-21chrt12:16:523
613299554,39cyclictest8624-21chrt09:46:573
613299554,38cyclictest2996-21taskset10:31:453
613299554,38cyclictest2996-21taskset10:31:453
613299553,38cyclictest8209-21chrt10:40:373
613299552,38cyclictest29327-21taskset09:27:033
613299544,37cyclictest2277-21taskset09:36:483
613299544,34cyclictest917-21kworker/3:1+events11:29:473
613299544,34cyclictest917-21kworker/3:1+events11:29:473
613299543,38cyclictest355-21taskset11:21:473
613299543,38cyclictest19246-21chrt09:11:303
613299543,33cyclictest16853-21chrt10:54:473
613299542,38cyclictest11997-21taskset09:52:043
613299533,37cyclictest31364-21chrt12:12:273
613299533,37cyclictest31364-21chrt12:12:273
613299533,33cyclictest10758-21chrt11:38:013
613299532,39cyclictest32228-21chrt11:21:143
613299532,39cyclictest32228-21chrt11:21:143
613299532,39cyclictest23039-21taskset07:51:313
613299532,39cyclictest23039-21taskset07:51:313
613299524,36cyclictest27721-21chrt11:12:023
613299523,38cyclictest13920-21chrt07:21:543
613299523,37cyclictest19744-21taskset10:59:343
613299523,37cyclictest19744-21taskset10:59:343
613299523,36cyclictest26777-21chrt12:06:223
613299523,36cyclictest12632-21taskset11:41:503
613299523,35cyclictest29348-21taskset08:11:273
613299523,35cyclictest21535-21chrt11:02:013
613299523,32cyclictest22123-21taskset11:57:023
613299522,36cyclictest20374-21taskset10:06:483
613299522,35cyclictest7605-21chrt08:42:013
613299522,33cyclictest13268-21taskset10:48:073
613299514,35cyclictest19926-21chrt11:53:423
613299514,35cyclictest19926-21chrt11:53:423
613299513,37cyclictest12004-21chrt08:56:513
613299513,36cyclictest24370-21chrt11:07:003
613299513,33cyclictest25525-21chrt07:57:023
613299512,37cyclictest9522-21taskset10:41:583
613299512,37cyclictest11787-21chrt07:16:323
613299512,36cyclictest17651-21chrt07:32:033
613299504,32cyclictest4585-21taskset12:22:033
613299503,36cyclictest18953-21taskset10:03:563
613299503,36cyclictest18953-21taskset10:03:563
613299503,36cyclictest13733-21taskset09:01:493
613299503,35cyclictest5154-21chrt08:36:303
613299502,37cyclictest10774-21taskset07:12:023
613299502,30cyclictest23803-21chrt10:11:593
613299502,30cyclictest23803-21chrt10:11:593
613299493,33cyclictest28277-21taskset12:07:113
6132994918,22cyclictest0-21swapper/310:16:533
6132994916,21cyclictest0-21swapper/307:46:293
613299485,32cyclictest7097-21chrt09:44:463
613299484,33cyclictest23652-21chrt09:17:303
613299483,35cyclictest23189-21chrt07:51:433
613299483,34cyclictest6006-21chrt08:37:033
6132994514,18cyclictest0-21swapper/308:56:273
613299433,27cyclictest26513-21taskset08:01:473
613299433,27cyclictest15060-21taskset07:26:443
6132994226,9cyclictest0-21swapper/307:16:453
613299422,28cyclictest1222-21taskset10:27:523
6132994212,17cyclictest0-21swapper/308:46:453
613299412,28cyclictest3375-21chrt08:31:283
613299412,28cyclictest3375-21chrt08:31:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional