You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-17 - 14:39
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Mon Aug 17, 2026 00:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24573212314,77sleep124579-21/usr/sbin/munin19:05:331
24573212314,77sleep124579-21/usr/sbin/munin19:05:331
254959911539,11cyclictest0-21swapper/021:45:140
25495991133,39cyclictest17202-21ssh00:10:300
254959911246,61cyclictest0-21swapper/021:55:050
25495991113,41cyclictest30798-21cat23:40:180
25495991113,41cyclictest30798-21cat23:40:180
254959911044,13cyclictest31660-21apt.systemd.dai00:33:200
25495991103,26cyclictest19399-21/usr/sbin/munin21:40:240
254959910941,8cyclictest0-21swapper/019:50:250
254959910938,12cyclictest0-21swapper/020:59:530
25495991093,30cyclictest32000-21/usr/sbin/munin22:00:180
25495991093,30cyclictest32000-21/usr/sbin/munin22:00:180
254959910841,11cyclictest22447-21ssh22:36:210
254959910841,11cyclictest22447-21ssh22:36:210
254959910745,55cyclictest24736-21sendmail_mailst23:30:280
254959910745,55cyclictest24736-21sendmail_mailst23:30:280
255159910652,29cyclictest24155-21threads00:20:292
255159910652,29cyclictest24155-21threads00:20:292
255159910651,48cyclictest1-21systemd22:22:402
255159910651,48cyclictest1-21systemd22:22:402
25495991065,27cyclictest24000-21awk00:20:270
25495991065,27cyclictest24000-21awk00:20:270
254959910636,14cyclictest32628-21diskmemload00:25:280
254959910611,34cyclictest1187-21ssh23:45:110
25495991054,27cyclictest13985-21cat21:30:290
25495991053,53cyclictest6491-21apt21:19:520
254959910511,34cyclictest20208-21fschecks_count20:30:580
25495991043,27cyclictest5923-21cpu22:10:090
25495991043,27cyclictest5923-21cpu22:10:090
25495991042,42cyclictest8974-21/usr/sbin/munin22:15:260
25495991042,27cyclictest25758-21cpu21:50:090
255079910381,15cyclictest32628-21diskmemload22:36:051
255079910381,15cyclictest32628-21diskmemload22:36:051
25495991033,37cyclictest25485-21cyclictest19:09:560
25495991033,37cyclictest25485-21cyclictest19:09:560
25495991033,30cyclictest27374-21fschecks_count23:35:150
25495991023,31cyclictest353-21runrttasks00:00:580
25495991013,28cyclictest377-21snmpd19:25:130
25495991003,25cyclictest353-21runrttasks23:52:290
2549599994,33cyclictest24526-21cron20:44:510
2549599994,33cyclictest24526-21cron20:44:510
2549599993,37cyclictest353-21runrttasks22:05:010
2549599993,24cyclictest15570-21/usr/sbin/munin23:15:310
2549599985,35cyclictest14981-21idleruntime-cro22:24:520
2549599985,35cyclictest14981-21idleruntime-cro22:24:520
25515999747,43cyclictest0-21swapper/221:00:302
25515999746,43cyclictest0-21swapper/200:30:172
25515999740,48cyclictest0-21swapper/219:19:532
2549599974,35cyclictest30659-21awk21:00:320
25515999646,43cyclictest31689-21kernelversion22:50:182
25515999646,43cyclictest31689-21kernelversion22:50:182
2549599964,32cyclictest28353-21mailstats19:15:280
2549599963,38cyclictest18452-21rm22:30:050
2549599963,38cyclictest18452-21rm22:30:050
2549599962,36cyclictest27797-21latency_hist22:44:540
2551599953,36cyclictest20396-21cut00:15:142
25515999443,44cyclictest27218-21switchtime00:25:282
25515999441,46cyclictest17436-21sh23:19:542
25515999435,9cyclictest0-21swapper/223:24:522
25515999435,9cyclictest0-21swapper/223:24:522
25515999433,8cyclictest0-21swapper/221:20:202
2551599943,32cyclictest3168-21latency_hist19:39:542
2549599944,34cyclictest353-21runrttasks20:12:570
2551599934,40cyclictest21258-21apt-get22:35:172
2551599934,40cyclictest21258-21apt-get22:35:172
2551599923,36cyclictest12056-21kernelversion20:05:582
2551599923,35cyclictest9671-21latency_hist19:59:552
2551599923,35cyclictest9217-21ssh23:05:272
2551599923,34cyclictest21938-21unixbench_singl23:25:332
25515999231,8cyclictest0-21swapper/222:56:302
2551599922,34cyclictest4044-21cmp23:49:542
2551599922,33cyclictest6292-21cron19:49:522
25515999140,45cyclictest3348-21sh22:05:242
25515999135,9cyclictest0-21swapper/223:54:542
2551599913,34cyclictest8095-21latency_hist19:54:562
2551599913,31cyclictest23774-21ps20:40:252
2551599913,31cyclictest23774-21ps20:40:252
25515999130,12cyclictest0-21swapper/220:59:322
25515999039,8cyclictest0-21swapper/221:35:172
25515999039,8cyclictest0-21swapper/221:35:172
25515999036,8cyclictest0-21swapper/221:55:082
25515999036,7cyclictest0-21swapper/220:35:252
2551599903,32cyclictest13354-21fschecks_count21:30:162
2551599903,31cyclictest20302-21fschecks_time20:31:012
2551599903,30cyclictest699-21grep19:30:242
2551599903,30cyclictest699-21grep19:30:242
2551599903,23cyclictest256632/usr/sbin/munin19:10:172
25515999023,13cyclictest0-21swapper/200:05:192
25515998973,8cyclictest0-21swapper/223:59:562
25515998966,16cyclictest27934-21latency_hist20:54:552
25515998962,19cyclictest32628-21diskmemload21:29:402
25515998962,19cyclictest32628-21diskmemload21:29:402
2551599894,27cyclictest20432-21/usr/sbin/munin21:40:282
2551599893,37cyclictest5102-21taskset23:00:032
2551599893,37cyclictest5102-21taskset23:00:032
2551599893,34cyclictest27797-21latency_hist22:44:542
2551599893,34cyclictest12442-21grep23:10:312
2551599893,33cyclictest28370-21fschecks_count22:45:142
2551599893,32cyclictest24944-21timerwakeupswit23:30:312
2551599893,32cyclictest24944-21timerwakeupswit23:30:312
25515998928,12cyclictest0-21swapper/221:05:272
2551599892,32cyclictest18609-21ntpq20:25:212
25495998922,22cyclictest32628-21diskmemload23:59:530
2551599883,34cyclictest32000-21/usr/sbin/munin22:00:322
2551599883,34cyclictest32000-21/usr/sbin/munin22:00:322
2551599883,34cyclictest15276-21latency20:15:182
2551599883,34cyclictest15276-21latency20:15:182
2551599883,33cyclictest32385-21latency_hist19:29:552
2551599883,30cyclictest26797-21grep21:50:302
25515998830,8cyclictest0-21swapper/221:45:032
2551599882,35cyclictest31116-21ssh23:40:272
2551599882,35cyclictest31116-21ssh23:40:272
2551599882,33cyclictest9390-21if_eth022:15:172
2551599874,27cyclictest18710-21fschecks_count22:30:152
2551599874,27cyclictest18710-21fschecks_count22:30:152
2551599873,32cyclictest5573-21ssh21:17:332
2551599873,32cyclictest25788-21grep20:45:302
2551599873,32cyclictest25788-21grep20:45:302
2551599873,30cyclictest6620-21/usr/sbin/munin22:10:262
2551599873,30cyclictest6620-21/usr/sbin/munin22:10:262
2551599872,31cyclictest16738-21taskset20:20:152
2551599872,31cyclictest16738-21taskset20:20:152
25507998769,12cyclictest297-21in:imuxsock20:19:521
25507998769,12cyclictest297-21in:imuxsock20:19:521
2549599873,37cyclictest353-21runrttasks00:09:170
2551599864,31cyclictest1206-21/usr/sbin/munin00:35:302
2551599863,35cyclictest29772-21latency19:20:202
2551599863,33cyclictest11439-21apt-key20:04:552
25515998629,13cyclictest0-21swapper/221:10:112
25515998627,12cyclictest0-21swapper/220:10:172
2551599862,35cyclictest10629-21cpu00:00:092
2551599862,32cyclictest4261-21grep19:40:292
2551599862,31cyclictest30085-21latency_hist23:39:542
25507998665,15cyclictest25485-21cyclictest22:50:161
25507998665,15cyclictest25485-21cyclictest22:50:161
2550799863,64cyclictest19457-21apt-get20:30:061
25507998553,23cyclictest11733-21fschecks_time23:10:171
25515998433,8cyclictest19650-21sh00:14:292
2551599842,29cyclictest15896-21ntpq22:25:232
2552299833,32cyclictest32035-21chrt21:05:283
25507998361,14cyclictest7100-21ssh21:20:141
25507998256,19cyclictest446-21cron20:44:521
25507998256,19cyclictest446-21cron20:44:521
25507998255,18cyclictest353-21runrttasks00:00:191
2552299813,37cyclictest20909-21chrt23:25:123
2550799814,40cyclictest10385-21ssh23:59:551
240332815,53sleep30-21swapper/319:05:103
240332815,53sleep30-21swapper/319:05:103
2552299803,29cyclictest18262-21chrt22:29:553
2551599802,31cyclictest25631-21latency_hist19:09:582
2552299793,30cyclictest3130-21chrt22:05:183
2552299792,29cyclictest29296-21taskset21:55:173
25507997961,13cyclictest17990-21fschecks_time23:20:131
25507997961,13cyclictest17990-21fschecks_time23:20:131
25495997957,16cyclictest23061-21latency_hist00:19:540
25507997860,9cyclictest0-21swapper/121:09:561
25495997856,16cyclictest11163-21latency_hist20:04:550
2552299773,33cyclictest11914-21chrt00:01:033
2550799779,58cyclictest4828-21cpu23:51:101
2549599773,32cyclictest17848-21cron20:24:520
2549599773,32cyclictest17848-21cron20:24:520
25507997610,23cyclictest31-21ksoftirqd/121:34:541
25495997657,12cyclictest297-21in:imuxsock19:44:530
25495997648,20cyclictest25485-21cyclictest23:25:310
25495997643,14cyclictest18408-21ssh21:38:100
25495997643,14cyclictest18408-21ssh21:38:100
25507997527,9cyclictest301ktimers/100:10:061
2549599754,37cyclictest3326-21latency_hist21:14:560
25495997537,29cyclictest15627-21fschecks_time22:25:180
25495997528,10cyclictest181rcu_preempt22:50:070
25495997528,10cyclictest181rcu_preempt22:50:070
2552299743,30cyclictest8430-21chrt21:21:453
25507997457,11cyclictest299-21rs:main1
25507997450,17cyclictest6990-21sh23:02:271
25507997450,17cyclictest6990-21sh23:02:271
25507997450,17cyclictest11894-21latency_hist22:19:541
25507997444,22cyclictest15083-21sh23:15:221
25495997452,14cyclictest25485-21cyclictest23:07:570
2549599744,51cyclictest13474-1kworker/0:2H+mmc_complete20:05:050
25495997427,11cyclictest181rcu_preempt19:45:080
25507997353,13cyclictest25485-21cyclictest22:07:341
2550799733,50cyclictest3894-21ssh21:15:141
2550799733,47cyclictest29344-21/usr/sbin/munin19:20:111
25507997329,12cyclictest0-21swapper/122:22:191
25507997329,12cyclictest0-21swapper/122:22:191
25495997336,27cyclictest0-21swapper/021:21:400
2552299723,31cyclictest27571-21chrt22:44:183
25507997233,24cyclictest31-21ksoftirqd/121:46:111
25507997217,32cyclictest31-21ksoftirqd/119:51:131
25507997149,15cyclictest0-21swapper/120:55:301
25507997147,14cyclictest1974-21ssh22:55:071
25507997144,17cyclictest30897-21kworker/u17:1-rpciod21:25:181
25507997144,17cyclictest30897-21kworker/u17:1-rpciod21:25:181
2549599712,37cyclictest3072-21munin-plugin-st19:39:520
2550799705,54cyclictest15322-21ssh22:25:071
25507997040,22cyclictest1959-21sh21:11:221
25507997035,19cyclictest31-21ksoftirqd/123:33:051
25507997035,19cyclictest31-21ksoftirqd/123:33:051
2550799701,38cyclictest31-21ksoftirqd/120:28:001
2549599704,55cyclictest17686-21/usr/sbin/munin23:20:180
2549599704,55cyclictest17686-21/usr/sbin/munin23:20:180
25522996948,13cyclictest202812chrt20:31:013
25507996945,14cyclictest0-21swapper/100:09:531
25507996943,18cyclictest0-21swapper/123:40:291
25507996943,18cyclictest0-21swapper/123:40:291
2550799693,44cyclictest30337-21switchtime00:30:301
2550799693,38cyclictest19073-21modprobe21:39:521
2550799693,38cyclictest19073-21modprobe21:39:521
2550799692,46cyclictest353-21runrttasks19:15:111
2549599695,43cyclictest31484-21fschecks_count21:05:140
2549599693,51cyclictest353-21runrttasks22:48:360
2549599693,33cyclictest2141-21/usr/sbin/munin22:55:120
2550799683,26cyclictest13028-21unin-run20:09:531
25507996822,8cyclictest301ktimers/122:45:111
25495996854,10cyclictest25485-21cyclictest19:20:200
25495996843,18cyclictest0-21swapper/023:00:190
25495996843,18cyclictest0-21swapper/023:00:190
25507996752,10cyclictest26360-21/usr/sbin/munin00:25:261
25507996750,11cyclictest7599-21kworker/u19:1+flush-179:022:40:371
25507996745,12cyclictest0-21swapper/123:05:261
25507996738,21cyclictest446-21cron00:24:521
25507996738,21cyclictest446-21cron00:24:521
2550799673,49cyclictest5863-21runrttasks22:10:031
2550799673,49cyclictest5863-21runrttasks22:10:031
25507996728,22cyclictest31-21ksoftirqd/123:35:431
25507996720,7cyclictest301ktimers/120:35:151
2549599673,53cyclictest9962-21/usr/sbin/munin21:25:280
2549599673,53cyclictest9962-21/usr/sbin/munin21:25:280
2549599673,50cyclictest16223-21munin-run20:19:520
2549599673,50cyclictest16223-21munin-run20:19:520
2549599673,46cyclictest26381-21latency_hist20:49:550
2549599673,37cyclictest11841-21cut23:10:190
2549599672,30cyclictest1149-21chrt00:35:300
25495996721,36cyclictest26457-21cpu20:50:080
229772670,8sleep30-21swapper/300:19:523
25507996647,13cyclictest32628-21diskmemload23:45:251
25507996646,14cyclictest22965-21idleruntime-cro00:19:511
2550799664,22cyclictest31-21ksoftirqd/119:45:121
2550799663,42cyclictest29651-21/usr/sbin/munin21:00:201
25507996622,35cyclictest1038-21wc00:35:291
2549599663,53cyclictest204-21systemd-journal19:14:520
2550799654,44cyclictest353-21runrttasks19:25:091
2550799654,39cyclictest21674-21sort22:34:561
2550799654,39cyclictest21674-21sort22:34:561
25507996541,16cyclictest0-21swapper/121:40:201
2550799653,40cyclictest13843-21wc20:10:251
25507996446,7cyclictest181rcu_preempt20:49:121
25507996446,7cyclictest181rcu_preempt20:49:121
25507996443,9cyclictest0-21swapper/119:10:291
25507996439,17cyclictest0-21swapper/120:24:541
25507996439,17cyclictest0-21swapper/120:24:541
2550799643,41cyclictest32316-21tune2fs22:00:151
2550799643,41cyclictest32316-21tune2fs22:00:151
25507996415,12cyclictest32628-21diskmemload21:50:041
25495996436,20cyclictest18733-21grep20:25:260
2549599642,30cyclictest26142-21cron20:49:520
25507996344,7cyclictest181rcu_preempt19:39:161
2550799634,37cyclictest785-21mailstats19:30:251
2550799634,37cyclictest785-21mailstats19:30:251
25507996336,19cyclictest21179-21fschecks_time23:25:181
2550799633,38cyclictest353-21runrttasks19:40:231
2550799633,36cyclictest26557-21cstates20:50:101
2549599634,48cyclictest22949-21grep20:39:530
241832639,33sleep224006-21/usr/sbin/munin19:05:172
25507996245,11cyclictest10018-21fschecks_count20:00:161
2549599622,29cyclictest8863-21sensors_temp19:55:270
2552299612,40cyclictest30224-21taskset21:00:253
2550799614,41cyclictest9689-21latency_hist19:59:551
25495996144,9cyclictest0-21swapper/019:30:290
25495996144,9cyclictest0-21swapper/019:30:290
2552299604,39cyclictest3733-21taskset23:49:443
25522995915,34cyclictest0-21swapper/323:59:533
323472580,13sleep30-21swapper/322:50:303
323472580,13sleep30-21swapper/322:50:303
2552299573,40cyclictest24627-21chrt22:39:523
2552299573,40cyclictest24627-21chrt22:39:523
2552299573,36cyclictest9854-21chrt23:06:433
25522995732,13cyclictest171062chrt00:10:153
2552299572,36cyclictest28538-21chrt21:54:523
2552299563,36cyclictest12211-21chrt23:10:283
2552299554,35cyclictest18511-21chrt23:20:253
2552299554,35cyclictest18511-21chrt23:20:253
2552299553,36cyclictest25763-21taskset19:10:103
2552299553,34cyclictest16060-21chrt23:16:323
2552299543,38cyclictest13153-21chrt21:30:083
2552299543,37cyclictest3542-21chrt21:14:563
2552299543,37cyclictest13874-21chrt20:10:263
2552299543,36cyclictest31745-21taskset19:25:293
2552299543,36cyclictest30193-21chrt23:39:553
2552299543,33cyclictest4548-21chrt23:50:333
2552299543,33cyclictest31114-21taskset00:31:523
2552299543,33cyclictest1180-21chrt19:31:203
2552299543,33cyclictest1180-21chrt19:31:203
2552299534,31cyclictest8213-21chrt19:55:103
2552299534,30cyclictest16586-21chrt20:20:103
2552299534,30cyclictest16586-21chrt20:20:103
2552299533,37cyclictest9820-21taskset22:15:283
2552299533,34cyclictest32336-21chrt22:00:153
2552299533,34cyclictest32336-21chrt22:00:153
2552299533,34cyclictest21359-21chrt21:42:363
2552299533,34cyclictest18912-21chrt22:30:193
2552299533,34cyclictest18912-21chrt22:30:193
2552299532,36cyclictest28394-21chrt20:55:193
2552299532,33cyclictest908-21kworker/3:1+events20:50:143
2552299532,32cyclictest29136-21chrt19:19:543
2552299523,37cyclictest10001-21chrt20:00:163
2552299523,36cyclictest1061-21taskset00:35:293
2552299523,33cyclictest10759-21chrt21:25:303
2552299523,33cyclictest10759-21chrt21:25:303
2552299523,32cyclictest30325-21taskset19:20:313
2552299523,30cyclictest3254-21taskset19:39:553
2552299514,30cyclictest22275-21chrt20:35:273
2552299513,37cyclictest16738-21chrt00:09:553
2552299513,31cyclictest12579-21chrt22:20:183
2552299513,31cyclictest12579-21chrt22:20:183
2552299512,37cyclictest4296-21taskset22:58:413
2552299512,33cyclictest24403-21chrt23:30:183
2552299512,33cyclictest24403-21chrt23:30:183
2552299503,35cyclictest23750-21chrt21:45:403
2552299502,34cyclictest7975-21taskset23:04:533
2552299502,34cyclictest7975-21taskset23:04:533
2552299502,34cyclictest15337-21chrt20:15:203
2552299502,34cyclictest15337-21chrt20:15:203
2552299493,30cyclictest6482-21taskset19:49:553
2552299492,35cyclictest31644-21taskset23:41:053
2552299492,35cyclictest31644-21taskset23:41:053
25522994915,21cyclictest24153-21chrt00:20:293
25522994915,21cyclictest24153-21chrt00:20:293
2552299463,32cyclictest29858-21chrt22:47:053
2552299462,33cyclictest5126-21chrt21:16:263
2552299443,28cyclictest6888-21chrt22:10:293
2552299443,28cyclictest6888-21chrt22:10:293
2552299443,28cyclictest11640-21taskset20:04:583
2552299442,29cyclictest27006-21taskset00:25:233
2552299422,29cyclictest17489-21taskset21:35:473
2552299422,29cyclictest17489-21taskset21:35:473
2552299413,26cyclictest18107-21taskset20:25:043
2552299412,29cyclictest24910-21taskset20:45:113
2552299412,29cyclictest24910-21taskset20:45:113
2552299393,27cyclictest3509-21taskset19:40:113
2552299393,27cyclictest23204-21taskset20:40:043
2552299393,27cyclictest23204-21taskset20:40:043
2552299392,26cyclictest6727-21taskset19:50:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional