You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-08 - 10:41
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Tue Sep 08, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31946217811,50sleep132002-21/usr/sbin/munin19:05:231
31290215812,105sleep231291-21/usr/sbin/munin19:04:512
2816821160,6sleep10-21swapper/123:25:171
3049911283,18cyclictest343-21runrttasks20:44:441
3237221106,16sleep00-21swapper/019:06:160
309991065,81cyclictest10411-21apt-get23:50:012
304991052,44cyclictest23684-21apt-get21:34:551
309999630,11cyclictest0-21swapper/223:40:182
30499967,73cyclictest10448-21ssh23:50:011
304999636,54cyclictest0-21swapper/123:09:421
304999629,9cyclictest0-21swapper/123:00:111
309999543,47cyclictest32761-21cyclictest23:44:572
309999541,11cyclictest524-21in:imuxsock23:59:402
309999433,12cyclictest0-21swapper/222:05:182
304999475,13cyclictest2113-21munin-node21:24:431
304999475,13cyclictest2113-21munin-node21:24:431
309999270,15cyclictest3478-21latency_hist21:54:432
30999923,43cyclictest22724-21/usr/sbin/munin00:10:072
30999923,43cyclictest22724-21/usr/sbin/munin00:10:072
32099914,73cyclictest10507-21taskset23:50:033
309999134,9cyclictest0-21swapper/222:45:182
309999134,8cyclictest0-21swapper/223:35:152
30499912,43cyclictest23490-21dpkg21:34:441
309999074,10cyclictest14506-21modprobe23:04:402
30999903,35cyclictest24713-21grep20:20:162
30999903,35cyclictest24713-21grep20:20:162
30999903,32cyclictest25422-21cpu20:24:572
309999029,8cyclictest0-21swapper/221:45:172
309999029,8cyclictest0-21swapper/221:45:172
30999902,31cyclictest2541-21/usr/sbin/munin19:15:152
30999902,31cyclictest20556-21timerandwakeup00:05:142
30999902,30cyclictest24110-21fschecks_time21:35:262
304999040,42cyclictest0-21swapper/122:20:021
304999033,7cyclictest0-21swapper/120:14:411
304999031,9cyclictest0-21swapper/100:24:571
309998937,8cyclictest0-21swapper/223:34:402
30999893,40cyclictest18956-21apt-get20:04:542
309998933,9cyclictest0-21swapper/200:35:072
309998933,9cyclictest0-21swapper/200:35:072
30999892,33cyclictest26487-21unixbench_singl22:30:182
304998970,12cyclictest3787-21/usr/sbin/munin21:55:541
304998968,15cyclictest343-21runrttasks21:10:181
304998940,42cyclictest0-21swapper/119:24:441
304998924,23cyclictest6500-21ls22:50:201
309998873,8cyclictest0-21swapper/222:35:032
30999883,28cyclictest31958-21/usr/sbin/munin00:25:012
30999883,27cyclictest10869-21/usr/sbin/munin19:39:492
309998832,17cyclictest0-21swapper/220:10:582
304998824,11cyclictest0-21swapper/122:00:101
309998735,7cyclictest0-21swapper/223:10:042
30999873,32cyclictest5468-21latency_hist00:34:422
30999873,28cyclictest5708-21apt-key19:24:432
309998732,8cyclictest0-21swapper/222:10:172
309998724,12cyclictest0-21swapper/220:19:072
309998724,12cyclictest0-21swapper/220:19:072
304998737,42cyclictest0-21swapper/122:05:071
304998735,45cyclictest0-21swapper/120:05:171
304998735,44cyclictest0-21swapper/120:59:431
304998735,44cyclictest0-21swapper/120:59:431
30499873,52cyclictest31833-21apt-get22:40:001
304998733,46cyclictest0-21swapper/123:30:121
304998725,8cyclictest0-21swapper/122:15:041
30499872,38cyclictest7649-21latency_hist21:09:441
30499872,31cyclictest18461-21cat21:26:121
30499872,31cyclictest18461-21cat21:26:121
30499872,30cyclictest9459-21fschecks_count19:35:031
309998631,15cyclictest30462-21fschecks_count20:40:022
304998637,42cyclictest0-21swapper/121:15:021
304998637,42cyclictest0-21swapper/119:55:031
304998637,42cyclictest0-21swapper/119:55:031
30499863,26cyclictest25252-21latency_hist22:29:431
30499863,26cyclictest25252-21latency_hist22:29:431
30499862,38cyclictest11833-21head19:40:151
30499862,35cyclictest28737-21fschecks_count22:35:021
30499862,31cyclictest9835-21latency_hist23:49:441
304998613,32cyclictest18142-21timerandwakeup20:00:151
309998554,15cyclictest4279-21apt-key20:59:442
309998554,15cyclictest4279-21apt-key20:59:442
30999853,31cyclictest8432-21sh22:01:212
309998533,11cyclictest0-21swapper/220:34:582
309998531,12cyclictest16480-21fschecks_time22:15:022
309998530,12cyclictest26730-21timerwakeupswit00:15:162
309998521,11cyclictest0-21swapper/200:00:032
304998564,15cyclictest14757-21mailstats19:50:151
304998535,42cyclictest0-21swapper/123:55:151
30499853,31cyclictest18215-21fschecks_time23:10:061
30499853,25cyclictest4314-21cut23:40:031
30499852,29cyclictest748-21fschecks_count21:50:011
30499852,29cyclictest6284-21tune2fs19:25:191
30499852,28cyclictest2707-21fschecks_count22:45:031
309998434,8cyclictest0-21swapper/219:34:592
30999843,33cyclictest23957-21latency_hist23:19:432
309998431,9cyclictest0-21swapper/221:19:312
30999842,34cyclictest31812-21/usr/sbin/munin22:41:172
30999842,27cyclictest28007-21vmstat20:30:182
30499845,31cyclictest3979-21apt-get20:59:541
304998441,37cyclictest1830-21timerandwakeup20:50:141
304998441,37cyclictest1830-21timerandwakeup20:50:141
304998440,37cyclictest7483-21diskmemload00:06:411
304998437,40cyclictest7483-21diskmemload22:11:181
30499843,32cyclictest26318-21unixbench_multi20:25:171
304998426,12cyclictest0-21swapper/120:19:571
304998426,12cyclictest0-21swapper/120:19:571
304998425,8cyclictest0-21swapper/121:44:561
304998425,8cyclictest0-21swapper/121:44:561
30499842,34cyclictest725-21fschecks_count19:10:031
30499842,34cyclictest725-21fschecks_count19:10:031
30499842,32cyclictest3924-21latency_hist23:39:441
30499842,29cyclictest28344-21ssh00:19:071
30499842,24cyclictest27716-21wc20:30:141
309998329,9cyclictest0-21swapper/200:21:072
309998329,9cyclictest0-21swapper/200:21:072
309998324,9cyclictest0-21swapper/221:56:002
304998360,14cyclictest19334-21modprobe00:04:391
30499833,32cyclictest343-21runrttasks23:20:251
304998330,45cyclictest0-21swapper/100:20:011
304998330,45cyclictest0-21swapper/100:20:011
304998327,8cyclictest0-21swapper/100:39:431
304998327,8cyclictest0-21swapper/100:39:431
30499832,34cyclictest343-21runrttasks00:33:581
30499832,32cyclictest25627-21latency_hist00:14:431
30499832,32cyclictest25627-21latency_hist00:14:431
30499832,30cyclictest9111-21latency22:55:061
30499832,29cyclictest26403-21grep22:30:171
30499832,29cyclictest23554-21grep20:19:411
30499832,29cyclictest23554-21grep20:19:411
30499832,26cyclictest21234-21fschecks_count23:15:021
300998364,13cyclictest1943-21latency_hist19:14:420
300998364,13cyclictest1943-21latency_hist19:14:420
309998257,15cyclictest0-21swapper/219:50:042
30999823,30cyclictest17685-21ssh21:25:532
30999823,30cyclictest17685-21ssh21:25:532
304998262,13cyclictest2110-21systemd19:14:441
304998230,9cyclictest0-21swapper/121:39:521
300998250,26cyclictest16278-40mandb23:59:410
32099812,66cyclictest6065-21taskset19:25:033
32099812,31cyclictest28916-21chrt00:19:533
32099812,31cyclictest28916-21chrt00:19:533
309998156,15cyclictest20914-21sh21:30:112
30499813,25cyclictest32122-21fschecks_time20:45:021
304998117,13cyclictest0-21swapper/119:29:581
304998117,13cyclictest0-21swapper/119:29:581
300998166,9cyclictest23738-21rm21:34:590
30499802,31cyclictest28884-21tune2fs20:35:041
30499793,25cyclictest13894-21latency_hist19:49:431
309997836,10cyclictest16346-21kworker/u17:1-nfsiod23:08:412
309997758,13cyclictest526-21rs:main2
300997757,13cyclictest31719-21latency_hist20:44:430
300997659,11cyclictest7483-21diskmemload23:19:570
309997439,26cyclictest7563-21date21:09:432
300997460,10cyclictest32761-21cyclictest22:15:030
300997460,10cyclictest22902-21kworker/u19:0-rpciod22:50:500
300997459,9cyclictest17411-21ssh00:00:160
300997458,9cyclictest31702-21kworker/u17:2+xprtiod00:32:120
300997458,10cyclictest30876-21ssh00:22:370
300997458,10cyclictest30876-21ssh00:22:370
30099743,39cyclictest23585-21latency_hist20:19:430
30099743,39cyclictest23585-21latency_hist20:19:430
3186527318,36sleep30-21swapper/319:05:213
309997356,11cyclictest8542-21mailstats21:10:152
30999732,31cyclictest16725-21sh21:24:392
30999732,31cyclictest16725-21sh21:24:392
300997354,13cyclictest170-21jbd2/mmcblk2p2-819:14:570
30999723,27cyclictest24252-21cstates23:19:572
300997258,10cyclictest2-21kthreadd21:04:420
300997257,10cyclictest32761-21cyclictest21:18:090
300997251,14cyclictest28351-21latency_hist22:34:430
32099713,30cyclictest5984-21chrt21:58:073
309997154,12cyclictest0-21swapper/223:25:022
309997139,7cyclictest181rcu_preempt22:25:222
309997139,7cyclictest181rcu_preempt22:25:222
30999713,50cyclictest4510-21apt-get20:59:542
300997151,13cyclictest23984-21fschecks_count20:20:010
300997151,13cyclictest23984-21fschecks_count20:20:010
300997151,12cyclictest1243-21ssh23:35:030
32099703,25cyclictest1219-21chrt21:50:133
309997049,12cyclictest0-21swapper/220:50:112
309997049,12cyclictest0-21swapper/220:50:112
300997053,13cyclictest32761-21cyclictest21:24:000
300997053,13cyclictest32761-21cyclictest21:24:000
300997045,8cyclictest0-21swapper/021:45:030
300997045,8cyclictest0-21swapper/021:45:030
30099704,36cyclictest31682-21apt-config22:39:450
300997039,23cyclictest2367-21modprobe20:54:390
300997039,23cyclictest2367-21modprobe20:54:390
242172703,7sleep00-21swapper/000:11:200
242172703,7sleep00-21swapper/000:11:200
32099693,28cyclictest13509-21chrt23:02:013
32099692,24cyclictest2070-21chrt20:50:573
32099692,24cyclictest2070-21chrt20:50:573
300996955,10cyclictest32761-21cyclictest23:32:300
300996954,9cyclictest3542-21unixbench_multi20:55:150
300996954,9cyclictest3542-21unixbench_multi20:55:150
300996953,10cyclictest7483-21diskmemload21:10:150
300996952,11cyclictest26661-21ssh00:15:160
300996952,10cyclictest20862-21ssh23:14:430
300996951,11cyclictest523-21unixbench_multi00:25:170
300996943,19cyclictest31965-21kworker/u17:3+nfsiod23:47:440
300996940,21cyclictest26417-21kworker/u17:0+nfsiod23:28:150
30099693,35cyclictest32741-21netstat22:41:060
32099683,25cyclictest8148-21chrt21:10:043
32099682,26cyclictest6883-21chrt00:35:333
32099682,26cyclictest6883-21chrt00:35:333
32099682,25cyclictest11995-21chrt21:15:193
309996855,9cyclictest32761-21cyclictest21:44:432
309996850,9cyclictest0-21swapper/222:24:432
300996855,9cyclictest2766-21kworker/u17:0+xprtiod23:50:320
300996853,9cyclictest8435-21kworker/u17:000:39:400
300996853,9cyclictest8435-21kworker/u17:000:39:400
300996852,11cyclictest32761-21cyclictest22:23:450
300996851,11cyclictest317-21dbus-daemon20:29:410
300996841,20cyclictest16346-21kworker/u17:1+nfsiod23:19:210
300996840,21cyclictest31965-21kworker/u17:3+nfsiod23:44:230
30099683,35cyclictest27268-21fschecks_time20:30:020
30099682,32cyclictest10601-21munin-plugin-st19:39:400
32099673,26cyclictest2563-21chrt19:14:553
32099673,25cyclictest12379-21chrt19:44:433
32099672,25cyclictest8500-21taskset19:30:183
32099672,25cyclictest8500-21taskset19:30:183
30099674,52cyclictest8618-21/usr/sbin/munin22:55:130
300996745,15cyclictest3265-21idleruntime-cro21:54:400
30099674,33cyclictest3787-21/usr/sbin/munin21:56:180
300996743,16cyclictest15731-21ssh23:05:170
30099673,34cyclictest12295-21seq23:00:080
32099669,33cyclictest0-21swapper/322:40:153
32099663,26cyclictest31997-21chrt20:44:573
32099663,26cyclictest17038-21chrt21:24:443
32099663,26cyclictest17038-21chrt21:24:443
32099663,25cyclictest25861-21chrt21:38:103
32099663,24cyclictest19190-21chrt21:27:233
32099663,24cyclictest19190-21chrt21:27:233
32099663,23cyclictest10415-21chrt19:37:303
32099662,24cyclictest27725-21chrt22:33:293
309996642,15cyclictest16615-21unixbench_multi19:55:172
309996642,15cyclictest16615-21unixbench_multi19:55:172
300996650,10cyclictest21998-21latency_hist20:14:430
300996648,12cyclictest343-21runrttasks22:01:260
300996648,11cyclictest15475-21systemd-run19:54:420
300996644,15cyclictest526-21rs:main0
30099664,33cyclictest20062-21munin-plugin-st21:29:400
30099664,33cyclictest20062-21munin-plugin-st21:29:400
300996637,22cyclictest6589-21kworker/u17:0+nfsiod21:30:460
32099653,24cyclictest16603-21chrt23:59:553
32099653,23cyclictest7154-21taskset22:51:363
32099653,22cyclictest31984-21chrt00:24:513
32099653,22cyclictest30734-21taskset23:30:053
32099653,22cyclictest29388-21chrt23:28:153
32099653,21cyclictest26556-21chrt20:27:203
32099652,28cyclictest17938-21taskset23:09:553
32099652,26cyclictest21193-21chrt22:22:273
32099652,25cyclictest28525-21chrt00:19:373
32099652,24cyclictest17316-21chrt23:08:583
32099652,23cyclictest12946-21chrt19:45:083
32099652,22cyclictest7293-21chrt22:00:023
32099652,22cyclictest18689-21chrt22:19:023
300996550,10cyclictest32761-21cyclictest22:09:120
30099654,35cyclictest343-21runrttasks20:09:310
300996541,17cyclictest877-21sendmail-mta00:05:250
300996538,20cyclictest26201-21kworker/u17:0+nfsiod21:40:340
32099643,23cyclictest27561-21chrt20:30:113
32099643,23cyclictest24005-21chrt00:10:453
32099643,23cyclictest24005-21chrt00:10:453
32099643,23cyclictest23382-21chrt23:18:433
32099643,23cyclictest11884-21chrt22:07:193
32099642,26cyclictest3258-21taskset00:30:143
32099642,23cyclictest971-21taskset19:10:093
32099642,23cyclictest971-21taskset19:10:093
32099642,23cyclictest4269-21chrt22:47:063
309996446,13cyclictest7387-21latency_hist19:29:432
309996435,20cyclictest9486-21ls22:55:152
30099644,31cyclictest31998-21cstates20:44:570
300996434,21cyclictest7010-21grep19:25:340
30099643,29cyclictest13416-21tune2fs22:10:050
300996425,26cyclictest343-21runrttasks22:25:380
300996425,26cyclictest343-21runrttasks22:25:380
30099642,34cyclictest343-21runrttasks21:06:430
32099633,23cyclictest7260-21chrt21:07:093
32099633,23cyclictest3955-21taskset23:39:463
32099633,23cyclictest28526-21chrt21:41:593
32099633,23cyclictest24812-21chrt23:20:133
32099633,22cyclictest31178-21chrt21:45:523
32099633,22cyclictest31178-21chrt21:45:523
32099633,22cyclictest30659-21chrt22:38:073
32099633,22cyclictest20144-21chrt20:07:563
32099633,21cyclictest29637-21chrt20:35:343
32099633,21cyclictest21352-21chrt00:06:523
32099632,25cyclictest2644-21chrt20:54:533
32099632,25cyclictest2644-21chrt20:54:533
32099632,23cyclictest24995-21chrt20:22:543
32099632,23cyclictest24995-21chrt20:22:543
30999633,33cyclictest5561-21/usr/sbin/munin22:50:012
30999632,38cyclictest925-21latency_hist20:49:442
300996326,26cyclictest0-21swapper/019:24:440
300996310,43cyclictest2524-21cpu22:44:570
32099623,24cyclictest5280-21chrt19:21:313
32099623,23cyclictest5834-21taskset21:04:293
32099623,23cyclictest17001-21chrt19:59:173
32099623,23cyclictest17001-21chrt19:59:173
32099623,23cyclictest14215-21chrt23:55:133
32099623,22cyclictest22135-21chrt20:14:443
32099623,22cyclictest22135-21chrt20:14:443
32099623,22cyclictest20327-21taskset21:29:473
32099623,21cyclictest8441-21chrt23:45:583
32099623,21cyclictest22376-21chrt22:24:543
32099623,21cyclictest22376-21chrt22:24:543
32099622,24cyclictest10429-21taskset22:57:073
32099622,23cyclictest30265-21chrt20:39:553
309996246,9cyclictest0-21swapper/219:45:172
32099613,21cyclictest21637-21chrt20:11:283
32099612,22cyclictest17344-21chrt19:59:543
32099612,21cyclictest15169-21chrt19:51:193
300996144,10cyclictest0-21swapper/019:55:040
300996144,10cyclictest0-21swapper/019:55:040
30099613,48cyclictest13257-21timerandwakeup19:45:160
30099613,43cyclictest12447-21latency_hist19:44:440
32099603,21cyclictest14872-21chrt22:12:053
32099602,22cyclictest2939-21taskset23:37:473
30099603,46cyclictest317-21dbus-daemon19:34:420
30099603,46cyclictest317-21dbus-daemon19:34:420
309995940,13cyclictest7917-21fschecks_time19:30:042
309995940,13cyclictest7917-21fschecks_time19:30:042
300995927,23cyclictest28723-21ls20:34:590
309995633,13cyclictest0-21swapper/219:09:582
309995633,13cyclictest0-21swapper/219:09:582
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional