You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-19 - 10:26
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Tue May 19, 2026 00:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2024222012,169sleep12031-21apt-get18:56:031
35029912232,60cyclictest0-21swapper/022:16:230
35029912232,60cyclictest0-21swapper/022:16:230
3502991214,82cyclictest352-21runrttasks23:47:280
3502991202,88cyclictest26320-21runrttasks21:26:150
3502991172,78cyclictest16044-21grep19:36:270
3502991163,82cyclictest23679-21apt-config23:10:560
3502991127,61cyclictest0-21swapper/023:16:070
3502991104,69cyclictest0-21swapper/022:00:530
3502991103,70cyclictest2731-21idleruntime21:41:150
3502991103,67cyclictest20570-21latency_hist00:00:560
3502991097,90cyclictest0-21swapper/000:06:080
3502991093,69cyclictest32693-21/usr/sbin/munin23:26:310
3502991093,67cyclictest26553-21grep22:21:160
3502991093,67cyclictest26553-21grep22:21:160
3502991073,75cyclictest29915-21grep22:26:270
3502991072,74cyclictest20721-21/usr/sbin/munin00:01:270
3502991072,74cyclictest20721-21/usr/sbin/munin00:01:270
3502991072,74cyclictest17924-21/usr/sbin/munin23:01:040
3502991062,78cyclictest8233-21latency_hist21:50:560
35029910523,52cyclictest0-21swapper/022:36:230
35159910484,13cyclictest14007-21latency_hist22:00:552
35029910443,54cyclictest0-21swapper/023:12:180
3502991043,64cyclictest0-21swapper/019:31:120
35029910431,44cyclictest0-21swapper/000:15:550
3502991042,71cyclictest0-21swapper/000:27:360
3502991023,64cyclictest3907-21/usr/sbin/munin23:31:250
35029910140,54cyclictest0-21swapper/022:05:560
3502991005,66cyclictest0-21swapper/021:20:560
3502991003,89cyclictest8831-21cut23:40:550
3502991002,67cyclictest18148-21xargs21:11:510
3515999980,13cyclictest352-21runrttasks20:36:112
3510999941,18cyclictest181rcu_preempt19:06:121
3510999941,18cyclictest181rcu_preempt19:06:121
350299993,64cyclictest11660-21kworker/u17:3+rpciod23:51:330
3502999910,56cyclictest0-21swapper/020:36:200
156242990,6sleep30-21swapper/322:56:283
156242990,6sleep30-21swapper/322:56:283
350299989,53cyclictest0-21swapper/022:07:160
350299983,85cyclictest23960-21apt-get20:01:260
350299983,81cyclictest6379-21tune2fs22:42:000
350299983,81cyclictest6379-21tune2fs22:42:000
350299982,65cyclictest32440-21ssh21:36:230
350299982,64cyclictest1328-21ssh00:22:170
350299982,64cyclictest1328-21ssh00:22:170
350299982,63cyclictest32275-21cut00:20:560
350299974,63cyclictest10912-21ssh23:43:460
3502999722,47cyclictest0-21swapper/022:46:240
3502999722,47cyclictest0-21swapper/022:46:240
3502999637,49cyclictest0-21swapper/019:11:290
3502999637,49cyclictest0-21swapper/019:11:290
350299962,86cyclictest12045-21/usr/sbin/munin21:01:310
3502999620,48cyclictest389-21kworker/u18:1+events_unbound19:15:570
3502999620,48cyclictest389-21kworker/u18:1+events_unbound19:15:570
350299952,63cyclictest9556-21ssh21:51:520
350299952,61cyclictest29323-21basename21:31:160
350299952,61cyclictest29323-21basename21:31:160
350299943,80cyclictest5731-21apt-get19:06:120
350299943,80cyclictest5731-21apt-get19:06:120
350299943,80cyclictest3382-21latency_hist19:00:580
3502999428,48cyclictest0-21swapper/022:51:520
350299942,83cyclictest20632-21fschecks_time22:11:140
3502999414,54cyclictest0-21swapper/019:41:100
350299934,76cyclictest28818-21apt-get20:16:050
3502999336,49cyclictest300-21rs:main0
350299932,60cyclictest30117-21/usr/sbin/munin23:21:150
350299932,60cyclictest30117-21/usr/sbin/munin23:21:150
3515999255,11cyclictest181rcu_preempt21:51:362
350299923,81cyclictest23776-21latency_hist20:00:570
350299923,78cyclictest13950-21ssh21:06:080
350299922,76cyclictest27209-21apt-get20:11:020
350299922,64cyclictest0-21swapper/022:31:290
350299922,64cyclictest0-21swapper/022:31:290
3510999148,13cyclictest23872-21rm00:06:151
350299913,76cyclictest7673-21apt-get20:51:060
351599902,49cyclictest21162-21grep21:16:302
3510999062,21cyclictest12559-21sh22:51:431
350299904,79cyclictest15809-21grep22:56:290
350299904,79cyclictest15809-21grep22:56:290
350299903,78cyclictest18625-21/usr/sbin/munin19:46:310
350299903,78cyclictest18625-21/usr/sbin/munin19:46:310
3502999030,54cyclictest352-21runrttasks19:51:100
3502999030,54cyclictest352-21runrttasks19:51:100
350299893,73cyclictest4689-21grep20:41:150
350299883,61cyclictest23524-21wc21:21:180
3515998767,12cyclictest6133-21runrttasks23:36:072
3515998762,18cyclictest32487-21fschecks_time22:31:142
3515998762,18cyclictest32487-21fschecks_time22:31:142
3510998723,31cyclictest0-21swapper/123:27:021
350299879,71cyclictest0-21swapper/020:50:550
350299879,71cyclictest0-21swapper/020:50:550
350299874,73cyclictest0-21swapper/020:06:080
350299874,73cyclictest0-21swapper/020:06:080
350299872,54cyclictest11710-21seq19:23:180
3515998665,8cyclictest181rcu_preempt20:50:552
3515998665,8cyclictest181rcu_preempt20:50:552
3510998642,8cyclictest181rcu_preempt22:26:091
351099863,60cyclictest18005-21ssh23:01:141
350299863,75cyclictest912-21modprobe20:30:530
350299863,74cyclictest10637-21modprobe21:00:530
350299863,74cyclictest10637-21modprobe21:00:530
350299863,70cyclictest31968-21latency_hist20:25:560
3510998559,18cyclictest15017-21sh22:01:281
3510998554,11cyclictest181rcu_preempt23:46:051
3510998546,13cyclictest0-21swapper/119:31:161
351599843,38cyclictest1946-21switchtime20:31:562
3510998466,13cyclictest17704-21kernelversion21:11:411
3510998458,18cyclictest10635-21diskmemload21:04:421
350299843,71cyclictest3906-21tune2fs19:01:140
350299843,70cyclictest0-21swapper/019:26:120
3515998361,16cyclictest17888-21timerwakeupswit19:41:292
3515998339,12cyclictest181rcu_preempt22:41:192
3515998339,12cyclictest181rcu_preempt22:41:192
3510998363,13cyclictest10464-21kworker/u17:0-nfsiod21:16:091
3510998354,19cyclictest5996-21latency_hist23:35:561
3510998336,8cyclictest181rcu_preempt23:20:171
3515998258,10cyclictest39-21ksoftirqd/221:29:492
3515998248,9cyclictest181rcu_preempt22:21:012
3515998248,9cyclictest181rcu_preempt22:21:012
3510998258,18cyclictest23832-21latency_hist23:10:571
3510998243,9cyclictest26990-21ssh00:11:181
3510998238,12cyclictest181rcu_preempt19:16:141
3510998238,12cyclictest181rcu_preempt19:16:141
3515998154,13cyclictest181rcu_preempt22:16:252
3515998154,13cyclictest181rcu_preempt22:16:252
351099813,57cyclictest5906-21ssh22:41:191
351099813,57cyclictest5906-21ssh22:41:191
3510998119,24cyclictest0-21swapper/121:27:451
3515998060,14cyclictest5058-21systemd-run19:05:542
3515998056,9cyclictest181rcu_preempt19:11:032
3515998056,9cyclictest181rcu_preempt19:11:032
3510998040,8cyclictest181rcu_preempt21:31:151
3510998040,8cyclictest181rcu_preempt21:31:151
3510998037,10cyclictest181rcu_preempt21:36:321
3510998026,31cyclictest0-21swapper/123:52:481
3515997953,20cyclictest10515-21latency_hist19:20:562
3515997953,20cyclictest10515-21latency_hist19:20:562
3515997949,11cyclictest181rcu_preempt23:46:112
3515997930,8cyclictest181rcu_preempt23:51:052
3510997953,9cyclictest181rcu_preempt22:56:281
3510997953,9cyclictest181rcu_preempt22:56:281
3510997941,17cyclictest3493-21cyclictest20:01:481
3510997939,11cyclictest181rcu_preempt21:24:151
3510997932,12cyclictest181rcu_preempt21:08:391
3515997853,17cyclictest3493-21cyclictest22:15:552
3515997850,14cyclictest181rcu_preempt21:45:562
3515997843,11cyclictest181rcu_preempt22:56:272
3515997843,11cyclictest181rcu_preempt22:56:272
3510997836,14cyclictest181rcu_preempt22:15:541
3510997813,24cyclictest0-21swapper/122:46:181
3510997813,24cyclictest0-21swapper/122:46:181
3515997755,17cyclictest26615-21kworker/u18:2+events_unbound20:25:532
3515997748,8cyclictest181rcu_preempt20:11:132
3510997737,12cyclictest181rcu_preempt00:16:281
3510997736,20cyclictest0-21swapper/120:31:571
3510997735,10cyclictest181rcu_preempt19:26:091
292727710,39sleep30-21swapper/318:56:393
3515997655,13cyclictest10464-21kworker/u17:0-xprtiod21:40:462
3515997644,24cyclictest5802-21modprobe23:35:532
351599763,62cyclictest17436-21ssh22:06:092
3510997648,21cyclictest26615-21kworker/u18:2+nfsiod21:57:461
3510997641,9cyclictest181rcu_preempt20:46:281
3510997641,9cyclictest181rcu_preempt20:46:281
3510997632,8cyclictest181rcu_preempt20:21:261
3510997631,32cyclictest16762-21modprobe19:40:531
3510997626,12cyclictest181rcu_preempt21:46:201
3515997554,13cyclictest0-21swapper/223:02:202
3515997552,17cyclictest10635-21diskmemload21:01:262
3515997544,12cyclictest181rcu_preempt22:50:562
3515997544,12cyclictest181rcu_preempt22:50:562
3515997541,8cyclictest181rcu_preempt00:01:222
3515997541,8cyclictest181rcu_preempt00:01:222
3510997540,9cyclictest181rcu_preempt22:40:551
3522997429,31cyclictest6378-21chrt22:42:003
3522997429,31cyclictest6378-21chrt22:42:003
3522997427,33cyclictest1436-21chrt22:33:163
3522997427,33cyclictest1436-21chrt22:33:163
3515997451,12cyclictest0-21swapper/222:40:542
351599744,38cyclictest26582-21apt23:15:522
3515997443,11cyclictest181rcu_preempt21:11:202
3515997435,13cyclictest0-21swapper/200:22:472
3515997435,13cyclictest0-21swapper/200:22:472
3515997425,18cyclictest0-21swapper/200:00:392
3515997420,9cyclictest0-21swapper/220:51:102
3510997454,14cyclictest32404-21idleruntime-cro23:25:531
3510997454,14cyclictest32404-21idleruntime-cro23:25:531
3510997448,16cyclictest0-21swapper/123:56:101
3510997436,10cyclictest181rcu_preempt19:21:161
3515997354,12cyclictest3493-21cyclictest23:41:532
3515997347,11cyclictest181rcu_preempt19:36:212
3515997345,10cyclictest181rcu_preempt00:20:122
3515997342,9cyclictest181rcu_preempt22:26:162
3510997344,21cyclictest8720-21sh21:51:181
3510997343,9cyclictest9024-21/usr/sbin/munin23:41:181
3510997328,8cyclictest181rcu_preempt20:11:071
261927311,29sleep20-21swapper/218:56:332
3515997252,14cyclictest297-21systemd-logind19:50:532
3515997252,14cyclictest297-21systemd-logind19:50:532
3515997245,12cyclictest181rcu_preempt23:06:112
3515997244,9cyclictest181rcu_preempt21:47:342
3510997255,11cyclictest18414-21ssh22:07:201
3510997253,12cyclictest3493-21cyclictest19:50:561
3510997227,12cyclictest181rcu_preempt20:16:271
3515997154,12cyclictest3493-21cyclictest22:51:462
3515997152,13cyclictest23457-21latency_hist20:00:572
3515997151,8cyclictest39-21ksoftirqd/223:21:472
3515997151,8cyclictest39-21ksoftirqd/223:21:472
3515997141,8cyclictest28784-21ssh00:14:422
3515997138,9cyclictest32521-21ntpq20:26:222
3515997138,11cyclictest181rcu_preempt23:20:522
3515997134,12cyclictest181rcu_preempt21:31:142
3515997134,12cyclictest181rcu_preempt21:31:142
351599713,37cyclictest0-21swapper/219:51:242
351599713,37cyclictest0-21swapper/219:51:242
3510997142,8cyclictest181rcu_preempt00:23:521
3510997142,8cyclictest181rcu_preempt00:23:521
3510997135,9cyclictest23525-21cut00:05:561
3510997135,9cyclictest23525-21cut00:05:561
3510997133,9cyclictest181rcu_preempt19:11:311
3510997133,9cyclictest181rcu_preempt19:11:311
3510997131,13cyclictest181rcu_preempt22:20:421
3510997131,13cyclictest181rcu_preempt22:20:421
3510997119,35cyclictest5260-21timerwakeupswit20:41:281
351599705,29cyclictest16594-21ssh21:10:542
3515997050,12cyclictest0-21swapper/219:06:212
3515997050,12cyclictest0-21swapper/219:06:212
3515997043,12cyclictest181rcu_preempt19:35:232
3515997034,15cyclictest181rcu_preempt21:22:492
351599703,25cyclictest30147-21cron20:20:532
351599702,49cyclictest23458-21apt-get20:01:082
3510997043,8cyclictest181rcu_preempt19:55:561
3510997043,8cyclictest181rcu_preempt19:55:561
3510997039,9cyclictest181rcu_preempt22:31:091
3510997039,9cyclictest181rcu_preempt22:31:091
3510997039,8cyclictest181rcu_preempt19:56:501
3510997036,8cyclictest181rcu_preempt20:56:221
3510997036,8cyclictest181rcu_preempt20:56:221
3522996935,23cyclictest12008-21chrt22:51:253
3515996948,9cyclictest181rcu_preempt00:06:402
3515996943,16cyclictest0-21swapper/200:26:142
3515996940,8cyclictest181rcu_preempt20:08:212
3515996940,8cyclictest181rcu_preempt20:08:212
3510996951,11cyclictest3493-21cyclictest20:55:561
3510996934,8cyclictest181rcu_preempt20:26:251
3510996925,12cyclictest181rcu_preempt00:27:161
3522996832,22cyclictest28175-21chrt20:11:273
3522996831,25cyclictest20187-21taskset21:16:093
3510996844,12cyclictest181rcu_preempt23:40:561
3510996837,8cyclictest181rcu_preempt22:21:111
3510996837,8cyclictest181rcu_preempt22:21:111
351099682,45cyclictest20132-21latency_hist19:50:551
3522996733,22cyclictest6248-21chrt19:06:243
3522996733,22cyclictest6248-21chrt19:06:243
3522996732,22cyclictest23074-21chrt21:21:083
3522996731,23cyclictest25191-21taskset22:18:453
3522996731,23cyclictest25191-21taskset22:18:453
3522996731,23cyclictest16806-21chrt19:40:553
3515996746,11cyclictest0-21swapper/220:56:042
3515996746,11cyclictest0-21swapper/220:56:042
3510996738,8cyclictest181rcu_preempt23:12:231
3510996737,7cyclictest181rcu_preempt19:01:051
3510996725,20cyclictest31258-21kworker/u17:1-events_unbound21:41:091
270802670,8sleep30-21swapper/323:16:113
3522996633,21cyclictest28822-21chrt20:16:053
3522996633,21cyclictest19162-21chrt21:14:063
3522996633,20cyclictest16624-21chrt22:05:153
3522996631,23cyclictest24011-21chrt23:11:033
3522996631,21cyclictest15862-21chrt21:08:513
3522996630,24cyclictest8060-21taskset23:38:573
3522996630,23cyclictest9510-21taskset21:51:453
3522996630,23cyclictest8928-21chrt22:46:143
3522996630,23cyclictest8928-21chrt22:46:143
3522996629,24cyclictest20680-21taskset22:11:153
3522996629,24cyclictest18831-21taskset22:07:303
3515996644,12cyclictest0-21swapper/223:30:522
3515996644,12cyclictest0-21swapper/220:41:182
3515996626,12cyclictest0-21swapper/219:27:192
3510996634,8cyclictest181rcu_preempt20:06:101
3510996634,8cyclictest181rcu_preempt20:06:101
3522996541,16cyclictest9287-21chrt20:56:063
3522996541,16cyclictest9287-21chrt20:56:063
3522996532,20cyclictest4854-21taskset23:33:063
3522996531,22cyclictest13462-21chrt21:05:313
3522996531,21cyclictest30447-21chrt23:21:263
3522996531,21cyclictest30447-21chrt23:21:263
3522996531,21cyclictest29097-21taskset21:31:093
3522996531,21cyclictest29097-21taskset21:31:093
3522996531,21cyclictest10595-21chrt23:42:513
3522996530,23cyclictest4185-21taskset00:27:063
3522996530,23cyclictest20131-21chrt23:04:433
3522996530,21cyclictest3919-21chrt20:37:523
3522996529,23cyclictest27441-21chrt22:21:453
3522996529,23cyclictest27441-21chrt22:21:453
3522996528,25cyclictest12196-21taskset19:26:093
3522996528,23cyclictest26630-21taskset00:11:083
3515996535,21cyclictest14679-21grep22:01:192
351099653,42cyclictest17272-21fschecks_time19:41:151
3522996434,20cyclictest23938-21chrt20:01:023
3522996432,20cyclictest14624-21chrt23:50:533
3522996431,22cyclictest17864-21chrt23:56:083
3522996431,20cyclictest410-21chrt21:36:323
3522996431,20cyclictest22696-21chrt23:08:403
3522996431,19cyclictest31043-21chrt22:28:383
3522996429,22cyclictest21513-21chrt19:54:403
3522996429,22cyclictest21513-21chrt19:54:403
3522996426,25cyclictest1925-21taskset00:24:023
3522996426,25cyclictest1925-21taskset00:24:023
3522996331,21cyclictest614-21chrt20:27:113
3522996331,20cyclictest7336-21chrt20:50:363
3522996331,20cyclictest7336-21chrt20:50:363
3522996330,20cyclictest16200-21chrt23:52:033
3522996328,22cyclictest2589-21taskset23:30:063
3522996328,22cyclictest13830-21chrt19:31:073
3522996327,23cyclictest3885-21taskset19:01:143
3522996327,23cyclictest31729-21taskset20:25:373
3522996325,24cyclictest1466-21taskset20:31:203
3515996314,13cyclictest181rcu_preempt19:25:552
351099633,41cyclictest2799-21/usr/sbin/munin20:36:041
3522996232,20cyclictest22150-21chrt19:56:183
3522996231,19cyclictest11258-21chrt21:56:023
3522996230,21cyclictest18073-21chrt19:42:093
3522996230,20cyclictest5256-21chrt22:40:463
3522996230,19cyclictest10587-21chrt19:21:063
3522996229,22cyclictest5321-21chrt21:46:013
3522996228,20cyclictest2419-21taskset21:41:063
3522996227,24cyclictest21941-21taskset00:01:483
3522996227,24cyclictest21941-21taskset00:01:483
3522996130,20cyclictest4438-21chrt20:41:083
3522996129,21cyclictest9972-21chrt19:16:373
3522996129,21cyclictest9972-21chrt19:16:373
3522996129,21cyclictest26609-21chrt20:07:103
3522996129,21cyclictest26609-21chrt20:07:103
3522996129,21cyclictest19815-21chrt19:48:533
3522996129,21cyclictest19815-21chrt19:48:533
3522996129,20cyclictest27084-21chrt21:26:313
3522996128,21cyclictest29622-21chrt00:16:073
3522996127,23cyclictest25165-21chrt00:07:443
3522996027,21cyclictest8057-21chrt19:11:273
3522996027,21cyclictest8057-21chrt19:11:273
3522996027,21cyclictest7868-21chrt20:51:163
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional