You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-19 - 05:08
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Sun Jul 19, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1027722220,19sleep30-21swapper/318:52:253
9689214010,22sleep00-21swapper/018:51:580
11150991113,67cyclictest6606-21runrttasks23:17:120
2863921080,5sleep228638-21ssh21:12:132
2863921080,5sleep228638-21ssh21:12:132
111509910843,58cyclictest16171-21grep23:32:220
111509910543,54cyclictest11857-21apt-key20:36:470
111639910474,20cyclictest2374-21systemd-journal22:22:171
111639910474,20cyclictest2374-21systemd-journal22:22:171
11150991032,72cyclictest13308-21seq21:41:380
11150991032,62cyclictest13025-21seq22:34:140
111639910219,54cyclictest15567-21apt-get20:47:031
111509910233,47cyclictest10550-21ssh23:23:230
111719910158,8cyclictest181rcu_preempt20:47:502
11171991013,59cyclictest26267-21ssh22:56:542
11171991013,59cyclictest26267-21ssh22:56:542
111509910139,53cyclictest0-21swapper/022:57:160
111509910139,53cyclictest0-21swapper/022:57:160
111509910050,44cyclictest28714-21/usr/sbin/munin22:07:210
111509910050,44cyclictest28714-21/usr/sbin/munin22:07:210
111509910038,54cyclictest0-21swapper/000:16:530
111509910038,54cyclictest0-21swapper/000:16:530
11150991003,63cyclictest3747-21grep23:12:170
11150999946,45cyclictest0-21swapper/019:22:100
1115099992,59cyclictest22138-21seq22:49:300
1115099992,23cyclictest6411-21ssh21:29:080
1115099992,23cyclictest6411-21ssh21:29:080
1115099974,52cyclictest0-21swapper/019:58:220
11150999742,49cyclictest14465-21ssh22:37:060
11150999742,49cyclictest14465-21ssh22:37:060
11150999740,52cyclictest0-21swapper/019:07:110
1115099973,62cyclictest12089-21apt-get20:37:040
11150999735,52cyclictest0-21swapper/022:22:090
11150999735,52cyclictest0-21swapper/022:22:090
230552960,6sleep00-21swapper/021:02:390
11150999640,49cyclictest0-21swapper/000:01:430
1115099963,88cyclictest0-21swapper/022:01:560
1115099953,58cyclictest3406-21runrttasks20:10:290
11150999534,52cyclictest21533-21modprobe21:01:440
1115099951,55cyclictest0-21swapper/021:20:580
1115099943,62cyclictest13917-21ssh23:29:190
1115099942,6cyclictest0-21swapper/022:11:540
1115099942,59cyclictest22643-21seq19:29:470
11150999415,55cyclictest0-21swapper/021:25:080
11150999340,45cyclictest30001-21fschecks_time19:52:060
1115099933,77cyclictest8629-21apt-get20:26:560
1115099933,77cyclictest8629-21apt-get20:26:560
1115099932,6cyclictest0-21swapper/019:41:550
1117199923,24cyclictest352-21runrttasks20:38:202
11150999241,45cyclictest3372-21sh22:17:260
11150999236,48cyclictest5561-21fschecks_count20:17:050
11150999232,53cyclictest0-21swapper/023:06:560
11150999231,53cyclictest27991-21sh21:11:490
1117199914,39cyclictest352-21runrttasks00:01:592
11150999142,44cyclictest0-21swapper/021:32:220
11150999142,43cyclictest23672-21ssh23:46:160
11150999138,47cyclictest352-21runrttasks22:52:180
1115099913,77cyclictest0-21swapper/020:51:560
1115099912,62cyclictest24588-21kworker/u17:1+rpciod23:02:110
997029011,43sleep20-21swapper/218:52:122
11171999044,39cyclictest0-21swapper/222:22:212
11171999044,39cyclictest0-21swapper/222:22:212
1117199903,40cyclictest15160-21timerandwakeup22:37:202
1117199903,40cyclictest15160-21timerandwakeup22:37:202
1115099903,7cyclictest0-21swapper/000:21:560
1115099903,77cyclictest6608-21ssh00:11:330
11150999037,46cyclictest31115-21ssh21:17:060
11150999037,46cyclictest31115-21ssh21:17:060
1115099903,61cyclictest7587-21awk00:12:130
1115099903,61cyclictest7587-21awk00:12:130
11163998917,23cyclictest0-21swapper/123:40:271
11150998932,49cyclictest0-21swapper/023:51:460
11150998932,49cyclictest0-21swapper/023:51:460
11150998931,49cyclictest23173-21cstates19:32:020
11150998931,49cyclictest23173-21cstates19:32:020
11171998845,35cyclictest0-21swapper/200:17:092
11171998845,35cyclictest0-21swapper/200:17:092
11171998835,46cyclictest25033-21sort19:37:372
11171998835,46cyclictest25033-21sort19:37:372
11171998817,31cyclictest29347-21diskmemload23:55:152
1117199873,36cyclictest23055-21/usr/sbin/munin19:32:192
1117199873,36cyclictest23055-21/usr/sbin/munin19:32:192
11163998731,49cyclictest0-21swapper/119:26:591
1115099873,6cyclictest0-21swapper/020:01:540
1115099873,5cyclictest0-21swapper/021:46:560
1115099872,70cyclictest16625-21apt-get19:12:020
1115099872,63cyclictest22470-21latency_hist21:56:450
1117199864,40cyclictest29044-21fschecks_time22:07:082
1117199864,40cyclictest29044-21fschecks_time22:07:082
1117199864,40cyclictest10651-21/usr/sbin/munin21:37:082
11171998616,31cyclictest1-21systemd00:01:442
11171998614,27cyclictest29234-21seq19:48:222
1115099863,76cyclictest13672-21apt-get21:41:590
1115099863,74cyclictest18173-21apt-get23:37:010
11150998637,43cyclictest0-21swapper/020:46:450
11150998633,46cyclictest0-21swapper/022:27:080
1115099862,76cyclictest12272-21/usr/sbin/munin18:57:200
1115099862,75cyclictest28810-21/usr/sbin/munin19:47:170
1115099862,59cyclictest2419-21ssh00:03:550
11150998616,63cyclictest23218-21/usr/sbin/munin21:57:100
1117199854,41cyclictest23794-21/usr/sbin/munin21:57:212
1117199854,39cyclictest352-21runrttasks23:10:052
1117199853,40cyclictest20329-21ssh21:52:152
1117199853,39cyclictest24844-21ssh23:47:162
1117199853,39cyclictest24844-21ssh23:47:162
1117199853,32cyclictest12968-21/usr/sbin/munin00:22:042
1115099853,6cyclictest0-21swapper/020:47:510
11171998445,33cyclictest18536-21diskmemload23:26:552
1117199843,32cyclictest25656-21/usr/sbin/munin22:02:092
1117199843,30cyclictest5842-21latency_hist23:16:462
1117199843,30cyclictest14997-21/usr/sbin/munin19:07:012
1115099844,70cyclictest0-21swapper/020:22:000
1115099844,70cyclictest0-21swapper/020:22:000
1115099843,75cyclictest14791-21latency_hist19:06:460
1115099843,73cyclictest27223-21ssh23:51:570
1117199834,36cyclictest8404-21latency_hist20:26:452
1117199834,36cyclictest8404-21latency_hist20:26:452
1117199833,40cyclictest2450-21fschecks_time20:07:072
1117199833,40cyclictest20941-21ssh23:41:462
1117199833,39cyclictest24338-21grep22:52:212
1117199833,39cyclictest17452-21fschecks_time20:52:062
1117199833,29cyclictest12930-21munin-node19:01:542
11171998330,46cyclictest0-21swapper/218:57:172
1117199832,42cyclictest18196-21cat22:42:202
11171998317,28cyclictest277-21dbus-daemon19:17:182
11171998317,28cyclictest277-21dbus-daemon19:17:182
1115099833,75cyclictest0-21swapper/022:41:570
1115099833,75cyclictest0-21swapper/019:16:580
1115099833,75cyclictest0-21swapper/019:16:580
1115099832,7cyclictest0-21swapper/019:37:360
1115099832,7cyclictest0-21swapper/019:37:360
1117199824,31cyclictest352-21runrttasks21:05:222
1117199823,39cyclictest31873-21awk21:17:282
1117199823,38cyclictest1905-21cron22:16:432
1117199823,31cyclictest8099-21ssh00:12:222
1117199823,31cyclictest8099-21ssh00:12:222
1117199823,26cyclictest9162-21/usr/sbin/munin23:22:022
1117199822,33cyclictest23916-40mandb23:46:472
1117199822,30cyclictest11986-21cmp20:36:472
11163998218,28cyclictest11714-21seq00:19:031
11163998218,28cyclictest11714-21seq00:19:031
1117199814,39cyclictest18536-21diskmemload21:22:122
1117199813,35cyclictest18985-21cstates20:57:022
1117199813,29cyclictest19568-21sort21:51:472
1117199813,28cyclictest2948-21wc22:17:172
1117199813,28cyclictest17610-21ssh23:36:062
1117199813,24cyclictest32731-21modprobe20:01:442
11171998129,43cyclictest0-21swapper/222:32:172
1117199812,39cyclictest352-21runrttasks19:21:582
1117199812,30cyclictest6012-21mailstats20:17:162
11163998152,20cyclictest20544-21switchtime19:22:171
1117199804,32cyclictest10148-21latency_hist20:31:472
1117199804,32cyclictest10148-21latency_hist20:31:472
11171998036,37cyclictest0-21swapper/223:02:122
1117199803,35cyclictest14595-21grep20:42:192
1117199803,34cyclictest4008-21fschecks_time20:12:052
11171998033,40cyclictest0-21swapper/222:47:192
1117199803,27cyclictest721-21/usr/sbin/munin20:02:042
1117199803,23cyclictest14455-21/usr/sbin/munin21:42:182
1117199802,29cyclictest29969-21grep19:52:052
1117199802,24cyclictest8432-21ssh22:27:022
1116399806,59cyclictest24418-21apt-get19:36:571
1116399806,59cyclictest24418-21apt-get19:36:571
11163998050,21cyclictest29307-21sh23:02:021
11163998019,38cyclictest0-21swapper/121:29:221
11163998019,38cyclictest0-21swapper/121:29:221
11163998017,24cyclictest8438-21date20:26:461
11163998017,24cyclictest8438-21date20:26:461
1115099804,68cyclictest3783-21apt-get20:11:560
1117699793,35cyclictest10403-21taskset23:22:593
1117699793,34cyclictest31591-21chrt19:57:083
1117199794,32cyclictest24547-21apt-get21:06:492
1117199793,29cyclictest8137-21wc21:32:122
1117199793,28cyclictest6733-21open_files23:17:172
1117199793,27cyclictest5537-21sort21:27:172
1117199793,27cyclictest5537-21sort21:27:172
1117199792,23cyclictest27086-21/usr/sbin/munin19:42:162
11163997958,14cyclictest2374-21systemd-journal21:01:451
11163997922,26cyclictest2262-21seq00:03:301
11163997916,24cyclictest6465-21ssh00:11:081
1117199783,31cyclictest21666-21fschecks_count19:27:052
1117199782,31cyclictest4855-21latency00:08:112
11171997814,24cyclictest18037-21date19:16:472
11163997856,15cyclictest21142-21sh23:41:541
11163997819,24cyclictest16329-21seq19:11:361
1117699773,27cyclictest8164-21chrt21:32:133
11163997719,24cyclictest25734-21seq23:48:301
11163997719,24cyclictest25734-21seq23:48:301
1117699753,35cyclictest920-21chrt23:08:033
11176997526,40cyclictest151342chrt20:46:453
1116399752,51cyclictest8149-21kworker/u17:3+flush-179:020:37:051
11163997518,21cyclictest27709-21seq19:44:541
11163997517,24cyclictest8642-21seq00:13:521
11163997517,24cyclictest8642-21seq00:13:521
11163997512,28cyclictest1719-21ssh23:08:531
11163997428,40cyclictest0-21swapper/121:07:121
11163997426,18cyclictest4208-21seq22:19:471
11163997416,28cyclictest15374-21seq00:25:561
11163997416,28cyclictest12533-21seq19:00:021
11163997411,24cyclictest0-21swapper/122:32:441
11163997329,33cyclictest0-21swapper/122:37:001
11163997329,33cyclictest0-21swapper/122:37:001
11163997327,23cyclictest18536-21diskmemload00:01:431
11163997316,19cyclictest18536-21diskmemload23:15:301
11163997313,27cyclictest0-21swapper/121:51:221
1117699724,25cyclictest13852-21chrt00:22:193
1117699723,28cyclictest16594-21chrt20:49:083
1116399727,28cyclictest0-21swapper/121:17:231
11163997256,11cyclictest297-21systemd-logind22:46:441
11163997233,26cyclictest9097-21ssh23:21:591
11163997224,37cyclictest0-21swapper/121:41:591
11163997211,24cyclictest0-21swapper/121:25:031
11163997150,14cyclictest2374-21systemd-journal20:19:171
11163997149,14cyclictest297-21systemd-logind19:56:451
11163997118,18cyclictest1756-21seq20:05:191
11163997116,18cyclictest14842-21seq20:44:241
1117699703,27cyclictest20237-21chrt21:52:103
1116399707,24cyclictest0-21swapper/122:12:301
11163997040,22cyclictest31393-21/usr/sbin/munin22:11:461
11163997040,22cyclictest31393-21/usr/sbin/munin22:11:461
11163997018,22cyclictest25642-21seq22:55:541
11163997010,21cyclictest0-21swapper/121:59:121
1117699694,33cyclictest26188-21chrt22:02:073
1117699694,26cyclictest23456-21chrt21:57:173
1117699693,27cyclictest1311-21taskset21:21:433
1117699692,24cyclictest18116-21taskset23:36:493
11163996944,15cyclictest18521-21kworker/u19:0-rpciod23:17:181
11163996915,22cyclictest9583-21sh22:27:591
11163996912,22cyclictest0-21swapper/120:33:581
11163996911,27cyclictest21410-21runrttasks22:47:271
1117699685,28cyclictest14363-21chrt23:30:333
1117699683,31cyclictest12157-21taskset21:38:253
1117699683,29cyclictest19750-21chrt22:46:163
1117699683,25cyclictest23469-21taskset22:52:043
1117699682,29cyclictest5564-21chrt20:17:053
1117699682,27cyclictest9439-21taskset00:16:063
1117699682,27cyclictest9439-21taskset00:16:063
1117699682,25cyclictest26710-21chrt19:42:053
1117699682,23cyclictest27108-21chrt21:09:513
11163996831,23cyclictest23074-21apt-get19:32:011
11163996831,23cyclictest23074-21apt-get19:32:011
11163996815,24cyclictest352-21runrttasks20:12:281
1117699673,27cyclictest21566-21chrt23:42:083
1117699672,25cyclictest2784-21chrt20:07:163
11163996746,15cyclictest17166-21/usr/sbin/munin20:52:051
11163996744,17cyclictest23153-21ssh21:02:551
11163996743,18cyclictest15321-21sh23:32:061
11163996723,40cyclictest0-21swapper/122:01:541
11163996717,22cyclictest30152-21ssh21:15:001
11163996717,22cyclictest30152-21ssh21:15:001
11163996717,22cyclictest13821-21sh23:29:031
11163996712,24cyclictest0-21swapper/121:56:171
1117699663,26cyclictest27988-21chrt23:52:183
1117699662,27cyclictest22483-21chrt21:02:173
1117699662,25cyclictest12824-21chrt22:33:403
11163996645,16cyclictest9038-21latency20:27:101
11163996645,16cyclictest9038-21latency20:27:101
11163996644,15cyclictest30040-21latency_hist23:56:461
11163996644,12cyclictest0-21swapper/121:32:231
1116399662,41cyclictest18536-21diskmemload22:57:081
1116399662,41cyclictest18536-21diskmemload22:57:081
11163996612,23cyclictest0-21swapper/119:21:161
11163996612,23cyclictest0-21swapper/119:21:161
1117699653,26cyclictest22009-21taskset19:27:153
1117699653,26cyclictest1025-21taskset22:14:153
1117699652,28cyclictest32669-21chrt00:01:103
11163996537,18cyclictest0-21swapper/121:37:121
11163996511,22cyclictest0-21swapper/119:14:281
1117699643,27cyclictest1009-21chrt20:02:103
1117699643,25cyclictest6513-21chrt22:22:503
1117699643,25cyclictest6513-21chrt22:22:503
1117699643,24cyclictest11290-21chrt20:33:093
1117699632,27cyclictest3694-21chrt22:18:203
1116399633,37cyclictest32355-21head19:57:241
11163996324,28cyclictest0-21swapper/119:01:581
96642629,32sleep19683-21apt-get18:51:551
1117699623,29cyclictest28932-21chrt22:07:063
1117699623,29cyclictest28932-21chrt22:07:063
1117699623,29cyclictest16739-21chrt21:47:003
1117699622,25cyclictest6383-21taskset00:10:553
1116399629,26cyclictest0-21swapper/120:11:481
1117699613,25cyclictest1134-21chrt00:02:073
11176996126,7cyclictest0-21swapper/320:37:233
1117699612,25cyclictest24893-21taskset19:36:513
1117699612,25cyclictest24893-21taskset19:36:513
1117699603,41cyclictest21246-21taskset22:47:223
11176996012,11cyclictest0-21swapper/320:56:483
11176996010,15cyclictest0-21swapper/321:22:203
11163996036,14cyclictest0-21swapper/119:47:201
1117699593,25cyclictest14671-21taskset19:06:173
1117699582,40cyclictest15266-21chrt19:07:063
11176995813,12cyclictest0-21swapper/321:27:133
11176995813,12cyclictest0-21swapper/321:27:133
1117699572,39cyclictest16810-21chrt22:41:263
1117699572,39cyclictest16810-21chrt22:41:263
1117699564,39cyclictest10189-21taskset00:17:033
1117699564,39cyclictest10189-21taskset00:17:033
1117699564,36cyclictest19096-21chrt20:57:063
1117699563,38cyclictest32293-21chrt23:06:473
1117699555,37cyclictest25375-21chrt23:47:323
1117699555,37cyclictest25375-21chrt23:47:323
1117699554,31cyclictest917-21kworker/3:1+events21:41:583
1117699553,36cyclictest4066-21chrt23:12:213
1117699552,37cyclictest20547-21chrt19:22:173
1117699543,32cyclictest3772-21chrt20:11:543
1117699542,34cyclictest9907-21chrt22:28:543
1117699533,37cyclictest28685-21chrt21:12:153
1117699533,37cyclictest28685-21chrt21:12:153
1117699533,35cyclictest17003-21chrt19:12:123
1117699532,38cyclictest28718-21chrt19:47:163
1117699512,36cyclictest10172-21taskset20:31:473
1117699512,36cyclictest10172-21taskset20:31:473
1117699503,31cyclictest8095-21chrt23:19:553
1117699494,31cyclictest11540-21taskset18:57:053
1117699493,34cyclictest8316-21taskset20:26:403
1117699493,34cyclictest8316-21taskset20:26:403
1117699493,34cyclictest17176-21taskset23:34:553
1117699493,31cyclictest31342-21chrt19:56:473
1117699483,33cyclictest23566-21taskset19:32:133
1117699483,33cyclictest23566-21taskset19:32:133
1117699483,31cyclictest18621-21taskset19:17:103
1117699483,31cyclictest18621-21taskset19:17:103
1117699453,29cyclictest26225-21chrt22:56:503
1117699453,29cyclictest26225-21chrt22:56:503
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional