You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-20 - 10:14
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Mon Apr 20, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24987213511,89sleep225077-21grep18:58:372
256219910864,37cyclictest27393-21ssh00:32:481
256179910842,27cyclictest25626-21latency_hist19:02:540
256179910755,43cyclictest12358-21ssh21:23:020
256219910443,9cyclictest0-21swapper/123:23:231
256219910438,12cyclictest0-21swapper/120:28:161
256219910438,12cyclictest0-21swapper/120:28:161
256179910445,22cyclictest20357-21sh00:19:260
256179910342,24cyclictest21722-21tune2fs20:28:130
256179910342,24cyclictest21722-21tune2fs20:28:130
256179910338,22cyclictest24335-21sh00:27:270
256219910144,8cyclictest0-21swapper/123:53:251
256219910144,8cyclictest0-21swapper/123:53:251
256219910132,13cyclictest0-21swapper/120:58:231
256179910147,50cyclictest0-21swapper/020:23:020
256179910141,29cyclictest24479-21latency_hist23:32:520
256179910141,29cyclictest24479-21latency_hist23:32:520
256179910140,37cyclictest882-21latency21:03:170
256179910034,23cyclictest0-21swapper/000:13:160
25621999936,12cyclictest0-21swapper/119:33:021
2562199993,48cyclictest6534-21sshd19:42:521
2562199993,39cyclictest24323-21cron23:32:481
2562199993,39cyclictest24323-21cron23:32:481
25617999944,44cyclictest9457-21apt-get22:12:490
25617999943,47cyclictest32530-21diskmemload22:05:460
25617999940,24cyclictest7073-21cpuspeed23:58:070
25617999939,22cyclictest8759-21memory19:48:180
25617999939,22cyclictest8759-21memory19:48:180
2562199985,36cyclictest3507-21idleruntime-cro22:02:491
25617999839,33cyclictest32530-21diskmemload23:03:020
2562199975,34cyclictest10217-21fschecks_time19:53:131
2562199974,39cyclictest15095-21cpu20:08:571
25621999742,13cyclictest9676-21munin-plugin-st23:07:511
25621999733,15cyclictest0-21swapper/119:28:221
25617999742,23cyclictest27749-21grep21:48:170
25617999739,23cyclictest311-21ssh23:46:420
25617999739,23cyclictest311-21ssh23:46:420
25617999738,23cyclictest29518-21ssh23:40:460
2562199964,42cyclictest28110-21fschecks_time19:08:151
2562199964,42cyclictest28110-21fschecks_time19:08:151
25621999637,13cyclictest0-21swapper/120:22:531
25617999640,17cyclictest26687-21grep20:43:150
25617999638,23cyclictest3837-21irqstats19:33:160
2562199953,28cyclictest12939-21/usr/sbin/munin22:18:131
2562199953,28cyclictest12939-21/usr/sbin/munin22:18:131
25617999550,38cyclictest1162-21ls21:58:120
25617999542,22cyclictest10060-21runrttasks22:13:050
25617999538,34cyclictest15677-21latency_hist22:22:520
25617999538,34cyclictest15677-21latency_hist22:22:520
2562199943,36cyclictest1589-21latency_hist19:27:531
2562199933,43cyclictest30911-21sh20:57:491
2562199933,37cyclictest9076-21grep19:48:271
2562199933,37cyclictest9076-21grep19:48:271
25617999342,42cyclictest1331-21ssh22:52:590
25617999341,12cyclictest15010-21apt-get20:08:020
25617999338,22cyclictest25955-21ssh23:33:560
25617999338,19cyclictest275372cut22:42:520
25617999336,24cyclictest19447-21ntp_states23:23:190
25617999332,24cyclictest25867-21unixbench_singl00:28:280
25632999270,15cyclictest25608-21cyclictest00:07:522
2562199925,38cyclictest506-21munin-run21:57:501
2562199925,38cyclictest506-21munin-run21:57:501
2562199925,35cyclictest15631-21latency_hist23:17:511
25621999231,12cyclictest0-21swapper/119:02:551
25617999254,30cyclictest18377-21runrttasks21:33:040
25617999247,36cyclictest0-21swapper/021:08:040
25617999245,31cyclictest16479-21/usr/sbin/munin22:23:190
25617999242,22cyclictest32530-21diskmemload21:32:520
25617999240,19cyclictest10484-21uname19:53:210
25617999236,19cyclictest6462-21systemd-run19:42:500
25621999122,26cyclictest29358-21latency_hist20:52:511
25617999138,21cyclictest13332-21ssh00:08:130
25617999137,20cyclictest29808-21ssh22:46:310
25632999054,12cyclictest181rcu_preempt23:46:102
25632999054,12cyclictest181rcu_preempt23:46:102
2562199905,39cyclictest352-21runrttasks22:55:511
2562199904,41cyclictest13244-21cut00:08:121
2562199904,33cyclictest24658-21/usr/sbin/munin23:33:181
25621999035,15cyclictest25608-21cyclictest23:47:241
25621999035,15cyclictest25608-21cyclictest23:47:241
25617999036,23cyclictest11487-21rm23:09:360
25617999036,23cyclictest11487-21rm23:09:360
25617999034,21cyclictest30913-21sh20:57:490
2562199893,31cyclictest13515-21grep20:03:151
25617998945,37cyclictest21727-21/usr/sbin/munin21:38:150
25617998945,16cyclictest32576-21ssh21:02:510
25617998941,15cyclictest2701-21ssh23:50:040
25617998940,42cyclictest14930-21seq23:15:570
25617998939,21cyclictest19676-21snmpd21:47:370
25617998937,23cyclictest18316-21ssh23:22:150
25617998936,19cyclictest8479-21rm21:16:130
25617998936,19cyclictest8479-21rm21:16:130
25617998934,19cyclictest0-21swapper/020:37:520
25617998931,28cyclictest6668-21idleruntime-cro23:57:490
25617998931,28cyclictest6668-21idleruntime-cro23:57:490
2562199884,37cyclictest21088-21latency_hist21:37:521
25617998853,30cyclictest0-21swapper/022:58:020
25617998853,30cyclictest0-21swapper/022:58:020
25617998841,12cyclictest32639-21wc19:22:530
25617998838,40cyclictest23340-21ssh22:34:430
25617998838,40cyclictest23340-21ssh22:34:430
25617998836,18cyclictest10694-21sh21:19:040
25617998832,16cyclictest2375-21ntpq19:28:170
2562199873,38cyclictest9960-21/usr/sbin/munin23:08:221
2562199873,38cyclictest9960-21/usr/sbin/munin23:08:221
2562199873,38cyclictest352-21runrttasks21:38:041
2562199873,35cyclictest352-21runrttasks19:15:051
25617998756,19cyclictest6704-21apt-get19:43:010
25617998755,20cyclictest32703-21apt-get19:23:010
25617998741,21cyclictest31165-21timerandwakeup21:53:270
25617998741,21cyclictest31165-21timerandwakeup21:53:270
25617998735,20cyclictest28900-21runrttasks19:09:210
25617998735,20cyclictest28900-21runrttasks19:09:210
25617998734,25cyclictest0-21swapper/020:38:230
25617998732,24cyclictest0-21swapper/000:07:520
25621998669,11cyclictest19752-21ssh22:29:191
25617998643,30cyclictest31367-21mailstats22:48:240
25617998635,18cyclictest21325-21sh22:32:280
25617998633,21cyclictest11925-21cut19:58:150
25621998569,11cyclictest25608-21cyclictest21:28:281
25617998556,19cyclictest27934-21apt-get20:48:020
25617998551,24cyclictest0-21swapper/019:03:040
25617998547,29cyclictest18822-21/usr/sbin/munin20:18:230
2563299843,22cyclictest13067-21latency_hist20:02:522
25617998442,35cyclictest13146-21/usr/sbin/munin20:03:170
25632998360,17cyclictest21451-21apt-get23:27:542
25621998357,17cyclictest5072-21ssh22:58:261
25621998357,17cyclictest5072-21ssh22:58:261
25617998334,17cyclictest17111-21grep20:13:190
2562199826,32cyclictest0-21swapper/122:13:251
2562199813,41cyclictest352-21runrttasks00:03:011
25621998066,10cyclictest25608-21cyclictest21:08:241
2562199803,29cyclictest12463-21/usr/sbin/munin21:23:071
25632997950,23cyclictest352-21runrttasks19:12:532
25632997950,23cyclictest352-21runrttasks19:12:532
25621997962,11cyclictest3098-21systemd-run21:07:511
25617997941,25cyclictest29738-21grep19:13:120
25632997855,15cyclictest31112-21latency_hist20:57:532
25621997853,16cyclictest1139-21apt-key22:52:541
25632997746,12cyclictest181rcu_preempt20:23:272
25621997641,12cyclictest181rcu_preempt22:03:281
25632997547,20cyclictest25608-21cyclictest23:53:262
25632997547,20cyclictest25608-21cyclictest23:53:262
25632997547,20cyclictest0-21swapper/221:38:172
25632997544,22cyclictest32530-21diskmemload21:03:122
25632997533,35cyclictest22283-21timerandwakeup20:28:262
25632997533,35cyclictest22283-21timerandwakeup20:28:262
25632997530,32cyclictest11409-21cat19:57:522
25632997458,9cyclictest30755-21diskmemload22:48:072
25621997457,12cyclictest18546-21latency_hist22:27:531
25621997457,11cyclictest18608-21latency_hist23:22:521
25621997449,17cyclictest0-21swapper/100:18:201
25632997354,14cyclictest19617-21kernelversion22:29:162
25632997349,16cyclictest27754-21apt-key00:32:542
25632997312,19cyclictest10865-21wc23:08:252
25632997312,19cyclictest10865-21wc23:08:252
2562199736,38cyclictest7055-21awk21:13:231
2562199736,38cyclictest7055-21awk21:13:231
25621997323,13cyclictest181rcu_preempt23:38:271
25632997255,11cyclictest32530-21diskmemload21:28:182
25632997219,13cyclictest0-21swapper/200:13:072
25632997214,42cyclictest6652-21latency_hist22:07:532
25621997249,17cyclictest10028-21apt-key00:02:531
25621997232,11cyclictest25528-21ssh21:43:481
25632997140,22cyclictest29531-21latency_hist20:52:532
2563299713,22cyclictest7470-21wc19:43:252
25632997115,17cyclictest18837-21apt-config22:27:522
25621997151,11cyclictest1596-21ntp_kernel_err23:48:181
25621997144,18cyclictest0-21swapper/121:48:171
25621997135,20cyclictest373-21lldpd22:08:161
2562199713,40cyclictest25330-21ntpq22:38:221
25632997052,12cyclictest4362-21timerwakeupswit21:08:262
2563299704,34cyclictest14039-21unixbench_singl22:18:282
2563299704,34cyclictest14039-21unixbench_singl22:18:282
25632997034,24cyclictest0-21swapper/200:23:292
2563299703,37cyclictest18871-21awk20:18:232
25632997014,33cyclictest0-21swapper/219:18:082
25621997046,16cyclictest26499-21fschecks_count20:43:121
25621997038,15cyclictest181rcu_preempt00:23:201
2562199703,33cyclictest352-21runrttasks21:20:511
2563999693,47cyclictest777-21taskset22:52:493
2563299696,24cyclictest9641-21latency_hist22:12:522
25632996949,14cyclictest1042-21forks21:58:102
25632996946,14cyclictest9869-21apt-key00:02:522
25632996927,31cyclictest31310-21grep21:53:282
25632996927,31cyclictest31310-21grep21:53:282
2563299692,45cyclictest2117-21fschecks_time19:28:112
2563299692,35cyclictest8227-21ssh23:03:382
2562199696,28cyclictest0-21swapper/119:03:041
2562199694,34cyclictest11820-21fschecks_time19:58:121
25621996941,12cyclictest0-21swapper/120:33:131
25632996855,9cyclictest25608-21cyclictest20:14:242
25632996847,11cyclictest0-21swapper/222:33:202
25632996847,11cyclictest0-21swapper/222:33:202
25632996846,12cyclictest0-21swapper/222:14:112
2563299683,30cyclictest25061-21mailstats21:43:242
2563299683,29cyclictest30445-21latency_hist22:47:522
2563299682,45cyclictest21207-21latency_hist21:37:532
2563299682,33cyclictest18646-21latency_hist23:22:522
2563299682,25cyclictest4928-21latency_hist19:37:522
25621996850,12cyclictest21787-21/usr/sbin/munin22:33:261
25621996850,12cyclictest21787-21/usr/sbin/munin22:33:261
25621996825,33cyclictest20622-21find20:23:251
2554626825,27sleep30-21swapper/319:02:543
25632996746,15cyclictest24965-21cut22:38:112
25632996745,16cyclictest32530-21diskmemload00:09:262
25632996740,19cyclictest28207-21sort21:48:272
25632996735,12cyclictest181rcu_preempt19:38:142
2562199674,22cyclictest6903-21fschecks_count19:43:121
25621996722,35cyclictest30395-21latency_hist22:47:521
25632996651,9cyclictest9894-21load21:18:172
25632996648,13cyclictest23577-21netstat20:33:172
2563299663,32cyclictest39-21ksoftirqd/223:48:562
25632996621,8cyclictest0-21swapper/220:03:042
25621996638,20cyclictest16907-21grep20:13:141
2562199663,44cyclictest25557-21timerwakeupswit20:38:261
2563999659,29cyclictest0-21swapper/322:13:253
2563299654,40cyclictest9308-21latency_hist21:17:532
25632996543,14cyclictest25608-21cyclictest19:17:532
2563299653,39cyclictest1875-21fschecks_time22:54:202
25632996449,8cyclictest0-21swapper/221:13:202
25632996448,11cyclictest2374-21systemd-journal20:11:522
25632996446,10cyclictest0-21swapper/222:58:232
25632996446,10cyclictest0-21swapper/222:58:232
25632996443,15cyclictest32721-21sh21:02:542
2563299643,42cyclictest25980-21fschecks_count19:03:112
25632996423,8cyclictest181rcu_preempt20:38:262
25632996414,17cyclictest27757-21expr23:38:072
2562199643,39cyclictest16771-21sh00:13:241
25621996411,21cyclictest31919-21wc19:18:261
25632996348,9cyclictest352-21runrttasks00:18:292
25632996337,18cyclictest0-21swapper/221:23:282
2563999623,31cyclictest0-21swapper/300:03:013
25632996241,12cyclictest15724-21latency_hist23:17:522
25632996238,12cyclictest0-21swapper/219:23:272
2563999613,35cyclictest917-21kworker/3:1+events00:01:183
25632996142,14cyclictest23053-21unixbench_multi23:29:242
25632996142,14cyclictest23053-21unixbench_multi23:29:242
2563299612,49cyclictest9663-21sendmail19:52:492
2563299612,49cyclictest9663-21sendmail19:52:492
2563299612,40cyclictest25284-21ntpq23:33:182
2563999602,39cyclictest917-21kworker/3:1+events22:53:563
2563999593,38cyclictest7312-21chrt19:43:233
2563999593,35cyclictest917-21kworker/3:1+events21:30:443
2563299592,21cyclictest27830-21latency_hist20:47:522
2563999584,39cyclictest13880-21chrt21:24:393
2563999584,36cyclictest31097-21chrt19:18:033
2563999584,36cyclictest13605-21chrt22:18:223
2563999584,36cyclictest13605-21chrt22:18:223
2563999583,39cyclictest975-21taskset19:23:263
2563999582,38cyclictest917-21kworker/3:1+events23:30:253
2563999582,38cyclictest917-21kworker/3:1+events23:30:253
2563999573,41cyclictest20218-21taskset00:19:023
2563999573,40cyclictest16117-21chrt23:18:113
2563999573,39cyclictest25457-21taskset22:38:263
2563999573,37cyclictest3814-21taskset19:33:143
2563999572,38cyclictest13810-21taskset20:03:233
2563999563,39cyclictest5657-21chrt19:38:213
2563999563,39cyclictest11863-21taskset19:58:133
2563999563,38cyclictest24960-21chrt23:33:113
2563999563,36cyclictest18167-21chrt22:26:453
2563999563,35cyclictest31448-21chrt20:58:133
2563999562,36cyclictest2224-21chrt21:05:303
2563999553,39cyclictest8467-21taskset22:10:043
2563999553,37cyclictest31075-21chrt23:43:173
2563999553,37cyclictest31075-21chrt23:43:173
2563999553,36cyclictest19794-21chrt20:22:533
2563999544,33cyclictest9955-21taskset19:53:073
2563999544,32cyclictest11324-21chrt21:20:503
2563999543,38cyclictest29368-21taskset19:12:533
2563999543,38cyclictest29368-21taskset19:12:533
2563999543,33cyclictest7005-21chrt21:13:203
2563999543,33cyclictest7005-21chrt21:13:203
2563999534,30cyclictest21639-21chrt21:38:133
2563999533,36cyclictest4958-21chrt22:58:253
2563999533,36cyclictest4958-21chrt22:58:253
2563999533,35cyclictest355-21taskset21:57:273
2563999533,35cyclictest355-21taskset21:57:273
2563999533,35cyclictest27682-21taskset00:32:523
2563999533,33cyclictest1504-21chrt23:48:153
2563999532,37cyclictest20217-21taskset21:35:383
2563999532,35cyclictest4334-21chrt23:53:143
2563999532,35cyclictest4334-21chrt23:53:143
2563999532,35cyclictest31025-21chrt20:57:523
2563999523,36cyclictest21779-21taskset22:32:593
2563999523,36cyclictest21779-21taskset22:32:593
2563999523,36cyclictest20826-21chrt22:31:013
2563999523,35cyclictest3226-21chrt19:32:503
2563999522,38cyclictest28204-21taskset23:38:183
2563999522,36cyclictest27787-21chrt20:47:523
2563999522,35cyclictest9223-21chrt23:06:323
2563999513,33cyclictest17081-21chrt20:13:173
2563999513,32cyclictest24040-21chrt20:33:263
2563999512,34cyclictest15957-21taskset20:09:163
2563999512,33cyclictest20962-21chrt23:26:263
2563999512,32cyclictest8394-21chrt19:48:093
2563999512,32cyclictest8394-21chrt19:48:093
25639995114,19cyclictest0-21swapper/323:08:043
25639995114,19cyclictest0-21swapper/323:08:043
2563999504,32cyclictest18738-21chrt00:17:523
2563999503,35cyclictest14080-21chrt23:13:293
2563999503,34cyclictest23216-21taskset00:24:133
2563999503,34cyclictest1943-21chrt21:58:283
2563999502,36cyclictest30355-21taskset19:13:263
2563999502,34cyclictest30077-21chrt22:47:183
2563999492,36cyclictest13143-21taskset00:08:073
2563999483,32cyclictest26625-21chrt21:46:533
2563999483,30cyclictest28130-21chrt20:48:123
2563999483,30cyclictest21546-21chrt20:28:083
2563999483,30cyclictest21546-21chrt20:28:083
2563999482,34cyclictest5246-21taskset22:04:243
2563999475,31cyclictest26812-21chrt19:03:383
2563999473,31cyclictest3343-21chrt21:07:573
2563999473,30cyclictest27676-21chrt21:48:153
2563999443,28cyclictest24939-21taskset20:38:123
25639993912,15cyclictest0-21swapper/320:23:263
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional