You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-17 - 18:26
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Fri Apr 17, 2026 12:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
217409912154,60cyclictest30342-21systemctl10:53:000
217409912154,60cyclictest30342-21systemctl10:53:000
217409912057,57cyclictest19861-21cpu11:28:160
217409911939,10cyclictest0-21swapper/011:53:180
217409911372,26cyclictest30892-21ssh10:53:180
21740991102,41cyclictest18454-21latency_hist10:33:020
21740991102,41cyclictest18454-21latency_hist10:33:020
21740991102,30cyclictest21169-21ssh12:23:340
21740991094,39cyclictest352-21runrttasks12:17:000
217409910816,41cyclictest8731-21kworker/u17:2+rpciod09:23:330
21740991073,53cyclictest1307-21apt-config07:38:040
21740991073,53cyclictest1307-21apt-config07:38:040
21740991072,27cyclictest4652-21cpu11:03:150
21740991072,27cyclictest4652-21cpu11:03:150
21740991063,42cyclictest23502-21munin-node07:08:120
217409910539,11cyclictest0-21swapper/007:28:160
21740991043,44cyclictest5477-21ls11:58:150
21740991042,44cyclictest13595-21ssh09:30:000
21740991034,34cyclictest22399-21ntp_kernel_err07:03:270
21740991033,37cyclictest22952-21grep08:44:060
21740991033,37cyclictest22952-21grep08:44:060
21740991014,29cyclictest17876-21ls09:38:030
21740991014,29cyclictest17876-21ls09:38:030
217409910119,28cyclictest4488-21latency_hist07:48:020
21572210114,61sleep221587-21modprobe07:03:002
2175399993,80cyclictest10902-21chrt11:13:173
2175399993,80cyclictest10902-21chrt11:13:173
2174799993,82cyclictest1432-21apt-get07:38:172
2174799993,82cyclictest1432-21apt-get07:38:172
2174099994,38cyclictest9243-21latency_hist08:03:020
21740999657,32cyclictest11472-21kernelversion11:13:590
21740999657,32cyclictest11472-21kernelversion11:13:590
2174799953,69cyclictest10281-21apt-get11:13:162
2174799953,69cyclictest10281-21apt-get11:13:162
2174599913,29cyclictest21891-21/usr/sbin/munin07:03:311
2174099913,67cyclictest1072-21apt-get10:58:140
2174099913,36cyclictest3217-21ssh09:13:450
21747999060,22cyclictest24334-21fschecks_time09:48:242
21747999060,22cyclictest24334-21fschecks_time09:48:242
21747998943,18cyclictest181rcu_preempt11:38:262
21747998943,18cyclictest181rcu_preempt11:38:262
21745998947,35cyclictest0-21swapper/108:53:361
21740998954,26cyclictest18187-21ssh09:38:180
21747998863,8cyclictest181rcu_preempt09:20:582
2174799883,48cyclictest39-21ksoftirqd/210:52:302
2174799883,48cyclictest39-21ksoftirqd/210:52:302
21745998851,32cyclictest21731-21cyclictest10:27:561
21745998844,36cyclictest2374-21systemd-journal09:29:121
21745998835,42cyclictest0-21swapper/111:18:251
2174599882,39cyclictest1509-21apt-get10:58:041
21740998817,56cyclictest26691-21idleruntime-cro09:52:590
21740998817,56cyclictest26691-21idleruntime-cro09:52:590
21745998744,35cyclictest0-21swapper/111:08:001
21745998744,35cyclictest0-21swapper/111:08:001
2174099873,43cyclictest28860-21ssh09:55:380
21747998662,15cyclictest21731-21cyclictest10:17:362
21747998662,15cyclictest21731-21cyclictest10:17:362
21747998633,31cyclictest39-21ksoftirqd/211:03:342
21747998633,31cyclictest39-21ksoftirqd/211:03:342
2174599863,40cyclictest17396-21ssh12:18:111
2174599863,36cyclictest2490-21/usr/sbin/munin11:53:141
2174599863,32cyclictest26162-21proc_pri07:13:311
2174599863,32cyclictest26162-21proc_pri07:13:311
2174599863,28cyclictest10561-21grep10:18:351
2174599862,35cyclictest25008-21sh12:30:051
21740998645,31cyclictest0-21swapper/010:38:190
2174799857,65cyclictest2061-21ssh10:59:012
21747998559,17cyclictest0-21swapper/208:43:592
21747998559,17cyclictest0-21swapper/208:43:592
21747998534,8cyclictest181rcu_preempt08:27:182
21747998511,65cyclictest19676-21snmpd09:13:492
21745998523,55cyclictest23918-21grep11:33:361
2174599852,30cyclictest10052-21/usr/sbin/munin09:23:371
21740998562,14cyclictest26467-21latency_hist12:33:030
2174799844,44cyclictest39-21ksoftirqd/211:25:082
2174799842,30cyclictest39-21ksoftirqd/212:08:002
21745998466,12cyclictest9833-21kworker/u19:0-flush-179:007:23:201
2174599845,67cyclictest27025-21ssh09:53:101
21745998437,39cyclictest0-21swapper/108:44:051
21745998437,39cyclictest0-21swapper/108:44:051
2174599843,6cyclictest0-21swapper/109:38:151
2174599843,35cyclictest25112-21/usr/sbin/munin09:48:391
2174599843,35cyclictest25112-21/usr/sbin/munin09:48:391
2174599843,31cyclictest20575-21ssh12:23:191
2174599842,37cyclictest19710-21/usr/sbin/munin10:33:371
21747998336,32cyclictest0-21swapper/212:13:072
21745998339,37cyclictest0-21swapper/111:58:251
2174599833,39cyclictest9310-21modprobe10:18:001
2174599833,39cyclictest9310-21modprobe10:18:001
2174599833,32cyclictest21868-21/usr/sbin/munin09:43:341
2174599832,40cyclictest582-21ssh10:03:111
2174599832,40cyclictest582-21ssh10:03:111
2174599832,36cyclictest28671-21latency_hist11:43:011
2174599832,36cyclictest28671-21latency_hist11:43:011
2174599832,35cyclictest30576-21netstat09:58:271
2174599832,33cyclictest5128-21rm09:16:261
2174799827,35cyclictest39-21ksoftirqd/207:29:032
21747998251,9cyclictest181rcu_preempt12:18:302
21745998264,12cyclictest30963-21kworker/u17:0-xprtiod12:10:131
21745998240,35cyclictest0-21swapper/109:18:261
21745998237,38cyclictest15365-21sh09:33:221
21745998237,38cyclictest15365-21sh09:33:221
21745998234,10cyclictest0-21swapper/110:08:361
2174599823,36cyclictest8302-21modprobe12:03:021
2174599823,29cyclictest12049-21grep08:08:361
2174599823,29cyclictest12049-21grep08:08:361
2174599822,34cyclictest30636-21fschecks_time07:28:211
2174599822,32cyclictest32478-21/usr/sbin/munin11:48:301
21740998243,12cyclictest181rcu_preempt09:47:590
21747998164,11cyclictest23974-21ntp_kernel_pll_12:28:292
2174599814,68cyclictest32500-21tune2fs09:08:221
2174599813,68cyclictest1612-21apt-get10:58:211
2174599813,38cyclictest27207-21runrttasks07:18:161
2174599813,38cyclictest27207-21runrttasks07:18:161
2174599813,36cyclictest17312-21latency_hist08:28:021
2174599813,31cyclictest14844-21/usr/sbin/munin08:18:271
2174599813,28cyclictest29173-21/usr/sbin/munin09:03:211
2174599812,39cyclictest21605-21sshd10:38:111
21747998057,17cyclictest544-21cron12:27:592
2174799804,53cyclictest39-21ksoftirqd/209:07:192
21747998040,17cyclictest181rcu_preempt10:48:032
2174799803,35cyclictest39-21ksoftirqd/208:58:342
2174799802,48cyclictest39-21ksoftirqd/207:08:202
2174599803,36cyclictest14933-21/usr/sbin/munin12:13:261
2174599803,34cyclictest954-21runrttasks07:38:001
2174599803,34cyclictest954-21runrttasks07:38:001
2174599803,33cyclictest18942-21modprobe08:33:001
2174599803,32cyclictest19300-21/usr/sbin/munin08:33:161
2174599803,31cyclictest1854-21/usr/sbin/munin07:39:191
2174599803,31cyclictest1854-21/usr/sbin/munin07:39:191
2174599803,30cyclictest10404-21grep08:03:361
2174599802,34cyclictest11139-21seq11:13:351
2174599802,34cyclictest11139-21seq11:13:351
2174599802,33cyclictest9003-21ssh11:09:271
2174599802,30cyclictest24283-21/usr/sbin/munin07:08:261
21747997959,14cyclictest12507-21sh09:28:262
21747997956,7cyclictest181rcu_preempt07:03:352
21747997938,17cyclictest181rcu_preempt08:29:302
21747997938,13cyclictest0-21swapper/210:37:342
2174599794,29cyclictest21803-21latency_hist07:03:041
2174599793,9cyclictest0-21swapper/111:28:141
2174599793,33cyclictest25103-21/usr/sbin/munin08:48:371
2174599793,31cyclictest13040-21/usr/sbin/munin08:13:231
2174599793,30cyclictest22195-21apt-key08:43:011
2174599792,33cyclictest17399-21ntpq11:23:281
2174599792,30cyclictest17570-21seq10:31:041
2174599792,30cyclictest17570-21seq10:31:041
21747997848,12cyclictest181rcu_preempt07:50:152
21747997847,8cyclictest181rcu_preempt10:18:322
2174799784,41cyclictest39-21ksoftirqd/210:54:162
21747997840,10cyclictest28784-21/usr/sbin/munin07:23:242
21747997828,11cyclictest0-21swapper/209:57:332
2174599783,32cyclictest5374-21/usr/sbin/munin07:48:351
2174599783,29cyclictest352-21runrttasks11:44:411
2174599782,32cyclictest3983-21grep07:43:361
2174599782,32cyclictest30728-21latency_hist10:53:051
2174599782,32cyclictest30728-21latency_hist10:53:051
2174599782,32cyclictest27782-21grep08:58:211
2174599782,31cyclictest8150-21fschecks_time07:58:211
21740997862,10cyclictest28874-21diskmemload12:04:210
21747997753,10cyclictest181rcu_preempt07:13:272
21747997753,10cyclictest181rcu_preempt07:13:272
21747997750,13cyclictest181rcu_preempt12:10:262
2174599773,34cyclictest6766-21/usr/sbin/munin07:53:271
2174599773,30cyclictest25360-21/usr/sbin/munin10:43:341
21747997655,7cyclictest181rcu_preempt11:46:492
21747997647,7cyclictest181rcu_preempt10:02:022
21747997647,12cyclictest181rcu_preempt09:36:522
21747997647,12cyclictest181rcu_preempt09:36:522
21747997645,8cyclictest181rcu_preempt11:11:202
21747997644,8cyclictest181rcu_preempt09:39:222
21740997657,12cyclictest21731-21cyclictest10:09:550
2175399753,38cyclictest0-21swapper/310:53:563
21747997551,18cyclictest1378-21latency_hist07:38:042
21747997538,15cyclictest181rcu_preempt09:44:382
21747997537,16cyclictest181rcu_preempt08:20:222
21740997562,8cyclictest15-21ksoftirqd/010:33:220
21740997557,12cyclictest28874-21diskmemload09:08:170
21740997551,15cyclictest9916-21latency08:03:250
215702756,52sleep00-21swapper/007:02:570
21747997451,8cyclictest181rcu_preempt10:03:332
21747997451,8cyclictest181rcu_preempt10:03:332
21747997444,8cyclictest181rcu_preempt11:49:412
21747997438,16cyclictest181rcu_preempt10:28:522
21747997438,16cyclictest181rcu_preempt10:28:522
21747997433,14cyclictest181rcu_preempt07:59:412
21740997447,9cyclictest181rcu_preempt10:17:590
21740997447,9cyclictest181rcu_preempt10:17:590
21740997447,10cyclictest181rcu_preempt10:01:080
21747997348,9cyclictest181rcu_preempt11:35:542
21747997339,8cyclictest181rcu_preempt10:41:582
21747997338,16cyclictest181rcu_preempt09:09:062
21747997336,16cyclictest181rcu_preempt11:57:182
21747997331,10cyclictest0-21swapper/207:46:592
2174099734,50cyclictest27557-21kernelversion07:18:260
2174099734,50cyclictest27557-21kernelversion07:18:260
2174099734,32cyclictest27422-21latency_hist08:58:030
21747997250,16cyclictest940-21cut07:38:022
21747997249,10cyclictest181rcu_preempt12:00:362
21747997246,20cyclictest22284-21apt-key08:43:022
21747997239,11cyclictest181rcu_preempt07:22:132
21747997239,11cyclictest181rcu_preempt07:22:132
21747997237,13cyclictest181rcu_preempt08:05:512
21747997235,11cyclictest181rcu_preempt07:53:442
21747997229,16cyclictest39-21ksoftirqd/211:28:222
21740997257,9cyclictest22217-21systemctl08:43:010
21740997247,12cyclictest21731-21cyclictest11:22:380
21740997236,11cyclictest181rcu_preempt09:03:170
21740997232,30cyclictest28874-21diskmemload11:53:020
21747997153,12cyclictest28874-21diskmemload09:23:232
21747997146,10cyclictest181rcu_preempt08:17:002
21747997140,21cyclictest0-21swapper/208:48:272
21747997138,15cyclictest181rcu_preempt10:23:552
21747997138,14cyclictest181rcu_preempt11:19:592
21747997135,16cyclictest181rcu_preempt08:56:072
2174799713,32cyclictest39-21ksoftirqd/208:08:312
2174799713,32cyclictest39-21ksoftirqd/208:08:312
21740997158,8cyclictest15-21ksoftirqd/010:43:360
21740997143,21cyclictest0-21swapper/010:23:280
2174099713,53cyclictest352-21runrttasks10:18:360
21747997038,11cyclictest181rcu_preempt10:08:072
21740997056,9cyclictest28874-21diskmemload09:23:000
21740997055,9cyclictest15314-21unixbench_singl08:18:360
316982690,6sleep30-21swapper/311:48:023
2174099696,50cyclictest25808-21/usr/sbin/munin11:38:310
2174099696,50cyclictest25808-21/usr/sbin/munin11:38:310
2174099693,37cyclictest5393-21switchtime07:48:350
2174099693,32cyclictest23895-21/usr/sbin/munin11:33:360
21740996913,17cyclictest10763-21cat11:13:040
226002680,8sleep30-21swapper/312:26:443
2175399684,53cyclictest14452sleep307:38:153
2175399684,53cyclictest14452sleep307:38:153
21740996833,13cyclictest181rcu_preempt07:13:210
21740996833,13cyclictest181rcu_preempt07:13:210
21740996736,7cyclictest181rcu_preempt07:23:200
2174099673,32cyclictest849-21fschecks_time10:03:210
2174099673,32cyclictest849-21fschecks_time10:03:210
2174099673,30cyclictest28952-21cat11:43:160
2174099673,29cyclictest352-21runrttasks08:32:320
2174099672,33cyclictest20055-21modprobe12:22:590
21740996650,10cyclictest11988-21ntp_states12:08:290
21747996510,11cyclictest19410-21grep08:33:212
2174099657,31cyclictest6370-21cpuspeed07:53:160
21740996549,10cyclictest28874-21diskmemload11:23:240
21753996418,27cyclictest0-21swapper/310:59:023
21740996441,12cyclictest0-21swapper/007:39:230
21740996441,12cyclictest0-21swapper/007:39:230
2080326425,21sleep30-21swapper/306:58:413
2174099633,40cyclictest28104-21ps08:58:310
2174099624,44cyclictest20136-21tail08:33:360
21740996240,13cyclictest0-21swapper/008:48:340
2174099623,28cyclictest16119-21df08:23:180
2174099614,38cyclictest13134-21kernelversion08:13:250
2174099592,25cyclictest11331-21tune2fs08:08:210
2174099592,25cyclictest11331-21tune2fs08:08:210
2175399582,43cyclictest31228-21taskset09:06:123
2175399582,41cyclictest22737-21chrt11:33:033
2175399582,38cyclictest14415-21taskset11:18:283
21753995811,26cyclictest0-21swapper/308:43:573
21753995811,26cyclictest0-21swapper/308:43:573
21753995610,24cyclictest0-21swapper/309:13:493
2175399553,40cyclictest9032-21taskset11:09:333
2175399553,35cyclictest31016-21chrt07:28:313
2175399542,38cyclictest22535-21chrt08:43:043
2175399534,36cyclictest26130-21chrt12:33:003
2175399534,35cyclictest24582-21chrt10:43:103
2175399533,39cyclictest32523-21taskset11:48:313
2175399533,39cyclictest10865-21chrt10:19:033
2175399533,36cyclictest30783-21chrt09:58:333
2175399532,38cyclictest18645-21taskset12:19:023
2175399532,35cyclictest16608-21chrt09:34:583
2175399532,35cyclictest16608-21chrt09:34:583
2175399523,38cyclictest12664-21taskset08:13:023
2175399523,38cyclictest12664-21taskset08:13:023
2175399523,36cyclictest10621-21chrt12:06:503
2175399523,35cyclictest30254-21taskset10:52:593
2175399523,35cyclictest30254-21taskset10:52:593
2175399523,35cyclictest14216-21chrt12:13:023
2175399522,38cyclictest9189-21taskset10:17:453
2175399522,38cyclictest9189-21taskset10:17:453
2175399522,36cyclictest26582-21taskset08:53:333
2175399522,33cyclictest30326-21chrt07:28:033
2175399514,35cyclictest25789-21taskset07:13:203
2175399514,35cyclictest25789-21taskset07:13:203
2175399513,36cyclictest4649-21taskset07:48:123
2175399513,33cyclictest4399-21taskset07:47:593
2175399512,37cyclictest18882-21chrt09:38:353
2175399512,33cyclictest8205-21chrt12:03:013
2175399512,33cyclictest23615-21taskset07:08:063
2175399504,33cyclictest22107-21taskset10:38:243
2175399503,34cyclictest24410-21chrt08:48:183
2175399503,34cyclictest17347-21chrt10:30:263
2175399503,34cyclictest17347-21chrt10:30:263
2175399503,33cyclictest16323-21chrt08:23:273
2175399503,32cyclictest28903-21chrt09:03:003
2175399503,30cyclictest15892-21chrt08:23:023
2175399502,36cyclictest1593-21taskset10:03:353
2175399502,36cyclictest1593-21taskset10:03:353
2175399502,33cyclictest2591-21chrt09:13:013
2175399493,33cyclictest22894-21chrt11:33:153
2175399493,33cyclictest18878-21chrt11:26:143
2175399492,33cyclictest6993-21taskset11:07:143
2175399492,33cyclictest6993-21taskset11:07:143
2175399492,32cyclictest9357-21chrt09:23:233
2175399483,35cyclictest20230-21chrt10:35:043
2175399483,34cyclictest3038-21chrt11:53:293
2175399483,33cyclictest26757-21taskset11:38:343
2175399483,33cyclictest26757-21taskset11:38:343
2175399483,33cyclictest26272-21taskset09:51:523
2175399483,33cyclictest26272-21taskset09:51:523
2175399483,32cyclictest6033-21chrt10:12:233
2175399483,31cyclictest13565-21chrt08:13:353
2175399482,34cyclictest13038-21chrt10:23:253
2175399474,34cyclictest21804-21taskset09:43:343
2175399473,32cyclictest29004-21taskset09:56:023
2175399473,31cyclictest23306-21taskset07:08:023
2175399472,31cyclictest8034-21chrt09:21:123
2175399464,29cyclictest32454-21chrt07:33:253
2175399464,29cyclictest32454-21chrt07:33:253
2175399463,33cyclictest19822-21taskset08:33:323
2175399463,29cyclictest15456-21chrt12:13:353
2175399443,31cyclictest9515-21taskset08:03:143
2175399442,29cyclictest27424-21chrt07:18:233
2175399442,29cyclictest27424-21chrt07:18:233
2175399413,29cyclictest8071-21chrt07:58:203
2175399413,27cyclictest7646-21taskset07:58:003
2175399413,27cyclictest19164-21chrt08:33:033
2175399413,26cyclictest11944-21taskset09:28:063
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional