You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-23 - 00:44
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Wed Jul 22, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25237991053,28cyclictest23661-21runrttasks12:17:082
252289910255,42cyclictest25206-21cyclictest08:42:001
252289910255,42cyclictest25206-21cyclictest08:42:001
25228999736,11cyclictest0-21swapper/112:02:071
2522899973,38cyclictest11366-21ssh11:02:091
25228999731,13cyclictest0-21swapper/109:01:331
2523799963,27cyclictest9855-21runrttasks11:55:112
2523799962,27cyclictest6417-21taskset09:06:402
25228999679,11cyclictest297-21systemd-logind07:01:311
25228999650,14cyclictest181rcu_preempt11:06:431
2522899964,39cyclictest9240-21latency_hist09:11:331
25228999622,18cyclictest0-21swapper/111:12:061
2523799952,28cyclictest14879-21chrt10:14:142
25228999538,10cyclictest0-21swapper/109:21:341
25219999571,16cyclictest22831-21latency_hist08:26:330
2522899943,38cyclictest7878-21sh10:56:581
2522899943,38cyclictest7878-21sh10:56:581
25219999425,36cyclictest0-21swapper/012:11:440
25228999367,18cyclictest22368-21sh12:16:331
2522899934,38cyclictest24813-21fschecks_time08:31:531
2522899933,40cyclictest6714-21fschecks_count07:36:521
2522899933,37cyclictest27910-21netstat07:01:591
25228999331,9cyclictest0-21swapper/108:16:321
2523799923,43cyclictest10642-21apt-get11:56:402
2523799923,29cyclictest6473-21runrttasks11:49:002
25228999234,12cyclictest0-21swapper/110:18:031
2522899923,37cyclictest13494-21latency_hist12:01:321
2522899923,36cyclictest352-21runrttasks09:55:041
2522899923,36cyclictest352-21runrttasks09:55:041
25228999231,10cyclictest0-21swapper/109:02:531
25228999231,10cyclictest0-21swapper/107:11:321
25228999231,10cyclictest0-21swapper/107:11:321
2523799912,73cyclictest7153-21modprobe10:01:512
25228999173,12cyclictest170-21jbd2/mmcblk2p2-811:27:091
2522899915,34cyclictest13012-21/usr/sbin/munin10:12:041
2522899913,36cyclictest31187-21fschecks_time07:11:531
2522899913,36cyclictest31187-21fschecks_time07:11:531
2522899913,36cyclictest21251-21/usr/sbin/munin09:32:061
2522899913,34cyclictest21498-21fschecks_count08:21:521
2522899913,30cyclictest10041-21/usr/sbin/munin10:06:561
2521999917,60cyclictest7137-21ssh10:01:480
2523799903,37cyclictest6528-21apt-get07:36:422
2522899904,35cyclictest4678-21mailstats09:57:031
2522899904,32cyclictest16606-21cstates08:07:011
2522899904,32cyclictest16606-21cstates08:07:011
2522899903,44cyclictest31906-21unixbench_multi08:52:081
2522899903,36cyclictest352-21runrttasks08:46:481
2522899903,32cyclictest893-21timerwakeupswit07:17:041
2522899902,36cyclictest24487-21latency_hist08:31:331
2522899902,27cyclictest7662-21/usr/sbin/munin11:51:571
2521999902,78cyclictest28789-21apt-get07:06:440
2521999902,78cyclictest28789-21apt-get07:06:440
248702906,65sleep10-21swapper/106:53:571
2523799892,31cyclictest12975-21/usr/sbin/munin07:56:572
2522899897,22cyclictest277-21dbus-daemon09:46:311
2522899897,22cyclictest277-21dbus-daemon09:46:311
2522899893,43cyclictest4556-21latency_hist11:46:331
2522899893,43cyclictest4556-21latency_hist11:46:331
2522899893,37cyclictest28386-21fschecks_time11:31:511
2522899893,37cyclictest28386-21fschecks_time11:31:511
2522899893,36cyclictest5448-21ssh10:52:091
2522899893,35cyclictest9397-21/usr/sbin/munin09:11:541
2523799883,39cyclictest30126-21/usr/sbin/munin08:47:052
2523799883,37cyclictest27864-21apt-get10:36:412
2522899884,35cyclictest16601-21/usr/sbin/munin12:07:011
2522899883,36cyclictest9483-21latency_hist07:46:331
2522899883,36cyclictest9483-21latency_hist07:46:331
2522899883,34cyclictest18734-21kernelversion09:26:541
2522899883,34cyclictest18734-21kernelversion09:26:541
2522899883,33cyclictest23492-21wc12:17:061
2522899883,32cyclictest26951-21timerandwakeup08:37:041
2522899883,32cyclictest11644-21grep07:51:531
25228998824,20cyclictest16023-21systemctl08:06:311
2522899882,40cyclictest1400-21wc10:46:461
25228998811,66cyclictest31312-21apt-get11:36:461
2523799873,36cyclictest31312-21apt-get11:36:422
2522899874,34cyclictest10532-21timerandwakeup07:47:061
2522899874,33cyclictest19231-21fschecks_time10:21:521
2522899873,39cyclictest19215-21cpu11:16:451
2522899873,39cyclictest19215-21cpu11:16:451
2522899873,35cyclictest15633-21fschecks_time09:21:531
25219998768,13cyclictest15048-21ssh09:21:320
25219998764,15cyclictest5198-21sh10:52:060
25237998640,39cyclictest0-21swapper/210:51:462
2523799863,46cyclictest18292-21apt-get09:26:402
2523799863,46cyclictest18292-21apt-get09:26:402
2523799863,39cyclictest14263-21/usr/sbin/munin12:02:022
2523799863,36cyclictest26043-21sh12:21:562
2523799862,32cyclictest1247-21grep08:57:052
2522899864,34cyclictest352-21runrttasks10:03:501
2522899863,32cyclictest24727-21latency_hist10:31:321
25219998663,16cyclictest2374-21systemd-journal11:20:040
25219998663,16cyclictest2374-21systemd-journal11:20:040
2510028614,43sleep225098-21munin-run06:56:302
2523799853,5cyclictest0-21swapper/211:31:422
2523799853,5cyclictest0-21swapper/211:31:422
2523799853,33cyclictest14015-21tr11:07:022
2523799852,38cyclictest27880-21/usr/sbin/munin07:01:582
2523799852,38cyclictest24487-21sh09:36:432
2523799852,36cyclictest4087-21fschecks_time09:02:502
2522899854,31cyclictest14550-21latency_hist08:01:331
2522899853,33cyclictest30507-21latency_hist10:41:331
25219998556,21cyclictest2374-21systemd-journal08:06:300
2523799843,6cyclictest0-21swapper/207:41:482
2523799843,6cyclictest0-21swapper/207:41:482
2523799843,36cyclictest881-21modprobe09:51:332
2523799843,36cyclictest881-21modprobe09:51:332
2522899843,34cyclictest23920-21apt-get09:36:391
2521999844,38cyclictest12791-21latency_hist10:11:310
2521999842,46cyclictest15-21ksoftirqd/007:26:090
25237998342,35cyclictest352-21runrttasks10:30:092
25237998340,36cyclictest0-21swapper/207:26:292
2523799833,68cyclictest16321-21grep10:16:522
2523799833,38cyclictest19077-21sed11:16:322
2523799833,37cyclictest18978-21rm10:21:442
2523799832,30cyclictest5264-21/usr/sbin/munin07:31:552
2522899834,37cyclictest21039-21munin-run08:21:291
2522899833,36cyclictest25505-21/usr/sbin/munin12:21:571
2521999833,38cyclictest32261-21latency_hist07:16:290
2521999833,38cyclictest32261-21latency_hist07:16:290
2523799824,37cyclictest3535-21/usr/sbin/munin07:26:522
2523799823,37cyclictest385-21/usr/sbin/munin07:16:522
2523799823,37cyclictest25308-21ssh10:31:542
2523799823,33cyclictest13418-21unixbench_singl09:17:082
2523799823,31cyclictest22308-21/usr/sbin/munin11:21:462
2523799823,31cyclictest21851-21/usr/sbin/munin09:31:582
2523799823,30cyclictest12059-21basename09:16:292
2523799823,29cyclictest31173-21grep07:11:532
2523799823,29cyclictest31173-21grep07:11:532
2523799822,38cyclictest25329-21taskset11:26:482
2523799822,35cyclictest17971-21latency_hist08:11:332
2523799822,35cyclictest17971-21latency_hist08:11:332
2523799822,32cyclictest22949-21/usr/sbin/munin08:26:452
2523799822,28cyclictest9828-21/usr/sbin/munin07:46:482
2522899823,37cyclictest30417-21taskset09:46:411
25219998253,21cyclictest297-21systemd-logind07:36:310
2523799813,37cyclictest2348-21open_files11:42:032
2523799813,37cyclictest2348-21open_files11:42:032
2523799813,35cyclictest4619-21ssh09:57:032
2523799813,30cyclictest15351-21/usr/sbin/munin09:21:472
2523799812,31cyclictest27048-21/usr/sbin/munin08:37:052
2523799812,30cyclictest27384-21rm09:41:352
2523799812,30cyclictest27384-21rm09:41:352
25228998163,13cyclictest31-21ksoftirqd/107:32:291
25219998158,16cyclictest30127-21sh09:46:310
25219998158,16cyclictest30127-21sh09:46:310
25219998156,19cyclictest27782-21cpu08:41:480
25219998156,19cyclictest27782-21cpu08:41:480
25219998155,17cyclictest24806-21ssh09:37:380
25219998153,20cyclictest25321-21timerandwakeup08:32:050
25237998037,34cyclictest0-21swapper/208:41:572
25237998037,34cyclictest0-21swapper/208:41:572
2523799803,6cyclictest0-21swapper/207:51:452
2523799803,35cyclictest12237-21taskset10:10:102
2523799803,31cyclictest20390-21/usr/sbin/munin12:12:042
2523799803,30cyclictest29291-21/usr/sbin/munin07:06:402
2523799803,30cyclictest29291-21/usr/sbin/munin07:06:402
2523799803,30cyclictest1446-21cpuspeed10:46:472
2523799803,28cyclictest31897-21seq09:48:092
2523799802,36cyclictest8408-21vmstat10:57:092
2523799802,36cyclictest8408-21vmstat10:57:092
2523799802,30cyclictest31178-21runrttasks08:51:512
2523799802,30cyclictest31176-21/usr/sbin/munin10:41:542
2523799802,30cyclictest19434-21grep11:16:502
2523799802,30cyclictest19434-21grep11:16:502
25219998060,13cyclictest32367-21diskmemload09:30:540
25219998060,13cyclictest32367-21diskmemload09:30:540
25219998039,9cyclictest0-21swapper/011:41:540
25219998039,9cyclictest0-21swapper/011:41:540
2521999803,48cyclictest25452-21netstat10:31:570
2523799794,33cyclictest25177-21/usr/sbin/munin08:32:032
2523799794,32cyclictest20795-21taskset08:18:302
2523799793,31cyclictest10237-21/usr/sbin/munin11:02:042
2523799793,30cyclictest16745-21ssh12:06:462
2523799793,30cyclictest14799-21fschecks_count08:01:522
2523799792,33cyclictest25547-21/usr/sbin/munin06:56:482
2523799792,29cyclictest21386-21cat08:21:472
2521999795,51cyclictest15-21ksoftirqd/011:32:150
2521999795,51cyclictest15-21ksoftirqd/011:32:150
2521999787,50cyclictest30760-21apt-get11:36:440
25219997858,13cyclictest0-21swapper/009:56:040
25219997858,13cyclictest0-21swapper/009:56:040
25219997843,24cyclictest17810-21cron08:11:300
25219997843,24cyclictest17810-21cron08:11:300
25219997842,28cyclictest0-21swapper/011:22:040
2521999783,53cyclictest15-21ksoftirqd/011:58:090
2521999783,28cyclictest7563-21fschecks_count10:56:520
2521999783,28cyclictest7563-21fschecks_count10:56:520
2523799772,30cyclictest18158-21sort08:11:482
25219997759,11cyclictest25206-21cyclictest12:22:020
25219997745,21cyclictest29154-21latency_hist07:06:330
2521999773,44cyclictest15-21ksoftirqd/007:49:030
2521999773,38cyclictest23905-21cron09:36:290
2521999773,33cyclictest18818-21ssh10:21:330
2521999773,33cyclictest15-21ksoftirqd/011:13:450
2521999772,47cyclictest23267-1kworker/0:2H+mmc_complete10:41:320
25219997716,32cyclictest5505-21sh09:04:510
25228997655,14cyclictest27453-21sh10:36:161
25219997649,20cyclictest2914-21diskmemload09:01:300
2521999762,53cyclictest15298-21/usr/sbin/munin09:22:040
2522899752,36cyclictest22131-21apt-get11:21:391
25219997560,10cyclictest10182-21kworker/u17:1+rpciod12:01:540
2521999754,35cyclictest15-21ksoftirqd/012:21:120
2521999752,35cyclictest352-21runrttasks10:44:070
2521999743,36cyclictest25107-21/usr/sbin/munin11:26:570
2521999743,36cyclictest24725-21cat10:31:320
25219997410,41cyclictest15-21ksoftirqd/008:41:060
25219997355,12cyclictest7191-21ssh11:51:030
2521999733,45cyclictest15-21ksoftirqd/012:07:390
2521999733,32cyclictest2095-21mailstats10:47:040
25228997253,14cyclictest6269-21sh11:48:271
25219997254,11cyclictest526-21ssh09:51:160
25219997250,15cyclictest6982-21ssh09:06:590
2521999723,34cyclictest1411-21apt07:21:290
2382727220,35sleep30-21swapper/306:51:513
25228997144,19cyclictest3541-21fschecks_time07:26:521
2521999713,35cyclictest12223-21latency_hist09:16:330
188492710,6sleep00-21swapper/008:12:060
25243997033,20cyclictest0-21swapper/311:36:543
2521999705,34cyclictest15-21ksoftirqd/008:53:540
2521999703,44cyclictest4831-21latency_hist07:31:330
2521999703,33cyclictest21683-21latency_hist10:26:310
25219996942,20cyclictest352-21runrttasks11:05:000
2521999693,44cyclictest25459-21sort06:56:460
25219996928,17cyclictest15-21ksoftirqd/010:15:050
2521999692,32cyclictest12721-21sh07:56:290
25228996845,15cyclictest25206-21cyclictest07:24:551
2521999687,27cyclictest13377-21cstates11:06:460
25219996843,15cyclictest0-21swapper/011:56:290
2521999676,25cyclictest15-21ksoftirqd/008:17:410
2521999673,43cyclictest23298-21kernelversion08:26:550
2524399663,42cyclictest917-21kworker/3:1+events08:31:463
25243996610,45cyclictest0-21swapper/309:37:043
2521999663,42cyclictest14382-21latency_hist08:01:320
2521999663,27cyclictest15-21ksoftirqd/010:01:180
2433526610,36sleep024382-21awk06:52:120
2521999654,41cyclictest30368-21unixbench_singl08:47:070
2524399643,37cyclictest917-21kworker/3:1+events07:46:303
2524399643,37cyclictest917-21kworker/3:1+events07:46:303
25228996444,13cyclictest0-21swapper/110:46:291
2521999643,49cyclictest8116-21/usr/sbin/munin07:41:500
2521999643,49cyclictest8116-21/usr/sbin/munin07:41:500
2524399637,30cyclictest0-21swapper/309:01:503
2524399633,45cyclictest15966-21chrt08:06:303
2524399622,41cyclictest917-21kworker/3:1+events09:11:533
2524399622,39cyclictest917-21kworker/3:1+events07:32:003
2524399613,43cyclictest16484-21chrt08:06:423
2524399613,43cyclictest16484-21chrt08:06:423
2524399612,41cyclictest917-21kworker/3:1+events07:27:023
25243996112,32cyclictest29325-21chrt07:06:483
25243996112,32cyclictest29325-21chrt07:06:483
2521999612,34cyclictest15-21ksoftirqd/007:36:420
2524399603,43cyclictest6764-21chrt07:36:533
2524399603,41cyclictest16205-21chrt10:16:513
2524399594,41cyclictest22531-21taskset12:16:383
2524399593,37cyclictest13915-21taskset07:57:063
25243995910,16cyclictest0-21swapper/310:01:443
2524399584,38cyclictest917-21kworker/3:1+events09:06:423
2524399584,36cyclictest27745-21chrt08:41:383
2524399584,36cyclictest27745-21chrt08:41:383
2524399583,36cyclictest21315-21chrt09:31:463
25243995819,22cyclictest0-21swapper/309:31:303
25243995819,22cyclictest0-21swapper/309:31:303
2524399574,39cyclictest31733-21taskset07:12:063
2524399574,39cyclictest31733-21taskset07:12:063
2524399574,36cyclictest25133-21chrt10:31:513
2524399573,38cyclictest22843-21chrt11:22:013
2524399573,38cyclictest16533-21taskset11:11:523
2524399573,37cyclictest26223-21taskset12:22:023
2524399572,38cyclictest26723-21taskset11:28:303
2524399563,38cyclictest16058-21chrt11:11:323
2524399563,37cyclictest11768-21chrt11:03:173
2524399563,31cyclictest917-21kworker/3:1+events12:15:343
2524399562,37cyclictest7124-21taskset11:50:523
2524399554,36cyclictest29176-21chrt11:32:063
2524399554,36cyclictest29176-21chrt11:32:063
2524399554,32cyclictest23160-21taskset08:26:513
2524399553,39cyclictest27626-21chrt07:01:533
2524399553,38cyclictest17243-21taskset09:24:293
2524399553,37cyclictest21232-21chrt08:21:333
2524399552,38cyclictest3144-21chrt07:26:323
2524399543,37cyclictest12065-21chrt07:52:043
2524399543,35cyclictest18986-21chrt10:21:453
2524399543,33cyclictest31828-21chrt08:52:073
25243995411,31cyclictest0-21swapper/310:56:293
2524399533,35cyclictest917-21kworker/3:1+events10:14:043
2524399533,31cyclictest27382-21taskset09:41:353
2524399533,31cyclictest27382-21taskset09:41:353
2524399532,33cyclictest5749-21chrt09:59:153
2524399523,37cyclictest15784-21taskset12:05:073
2524399523,36cyclictest2862-21taskset11:42:163
2524399523,36cyclictest2862-21taskset11:42:163
2524399523,36cyclictest18268-21taskset08:11:533
2524399523,34cyclictest30998-21chrt10:41:483
2524399522,35cyclictest26501-21chrt08:36:523
2524399514,33cyclictest9736-21chrt11:00:583
2524399514,33cyclictest9736-21chrt11:00:583
2524399513,37cyclictest16677-21taskset12:06:453
2524399513,29cyclictest917-21kworker/3:1+events07:46:553
2524399512,33cyclictest19153-21taskset11:16:383
2524399512,33cyclictest19153-21taskset11:16:383
2524399503,33cyclictest9874-21chrt11:55:143
2524399503,32cyclictest14106-21chrt09:18:573
2524399503,30cyclictest917-21kworker/3:1+events11:56:533
2524399502,36cyclictest4034-21taskset10:51:293
2524399502,33cyclictest2948-21chrt09:01:313
2524399493,33cyclictest31798-21chrt09:47:523
2524399492,37cyclictest29528-21chrt08:46:483
2524399492,34cyclictest26886-21chrt07:01:333
2524399492,34cyclictest1284-21chrt09:51:533
2524399492,34cyclictest1284-21chrt09:51:533
2524399492,32cyclictest927-21chrt07:17:043
2524399482,29cyclictest10095-21chrt10:06:453
2524399472,31cyclictest28387-21chrt10:36:583
2524399453,31cyclictest21758-21chrt08:21:573
2524399433,29cyclictest22063-21taskset10:26:483
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional