You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-08 - 00:38
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Sun Dec 07, 2025 12:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19785218510,40sleep019789-21systemd-run07:08:240
198929910043,50cyclictest0-21swapper/211:58:242
19892999818,33cyclictest3630-21fschecks_count09:23:452
19892999722,25cyclictest25454-21seq11:00:432
19877999773,16cyclictest10031-21latency_hist12:28:250
1989299964,40cyclictest11138-21unixbench_multi10:34:432
19892999532,9cyclictest0-21swapper/210:03:422
19892999532,10cyclictest0-21swapper/210:18:352
19892999526,11cyclictest0-21swapper/208:34:012
1989299952,38cyclictest975-21latency_hist07:48:242
1988199953,62cyclictest23812-21apt-get10:58:371
226022940,7sleep10-21swapper/109:58:331
226022940,7sleep10-21swapper/109:58:331
1989299943,42cyclictest22492-21latency_hist08:53:242
19892999432,9cyclictest0-21swapper/211:43:092
1989299934,38cyclictest4181-21/usr/sbin/munin10:23:512
19892999335,9cyclictest0-21swapper/210:53:462
19892999335,9cyclictest0-21swapper/210:53:462
19892999334,51cyclictest0-21swapper/212:13:432
19892999322,12cyclictest0-21swapper/209:39:282
1989299924,34cyclictest22364-21cat09:58:242
1989299924,32cyclictest355-21runrttasks11:28:392
1989299923,39cyclictest25291-21/usr/sbin/munin11:59:112
1989299923,39cyclictest25291-21/usr/sbin/munin11:59:112
1989299923,33cyclictest6963-21grep08:03:582
1989299923,33cyclictest6963-21grep08:03:582
54292910,4sleep30-21swapper/312:19:063
1989299914,38cyclictest355-21runrttasks12:18:522
1989299914,36cyclictest3359-21open_files07:53:542
1989299914,36cyclictest3359-21open_files07:53:542
1989299914,36cyclictest30987-21/usr/sbin/munin10:13:482
1989299914,36cyclictest30987-21/usr/sbin/munin10:13:482
1989299914,33cyclictest11098-21latency_hist08:18:252
1989299914,33cyclictest11098-21latency_hist08:18:252
1989299913,35cyclictest17821-21modprobe10:48:212
1989299913,33cyclictest20105-21cut08:44:002
1989299913,33cyclictest13333-21ps08:23:572
1989299913,32cyclictest22886-21fschecks_count09:58:452
1989299913,32cyclictest22886-21fschecks_count09:58:452
19892999126,58cyclictest0-21swapper/211:08:522
19892999126,12cyclictest0-21swapper/208:29:592
1989299912,41cyclictest11160-21sshd08:18:392
1989299912,41cyclictest11160-21sshd08:18:392
19877999165,17cyclictest7973-21latency_hist11:28:250
19877999110,73cyclictest23321-21kworker/u18:1+ext4-rsv-conversion10:34:120
1989299904,36cyclictest25693-21/usr/sbin/munin07:23:462
1989299904,32cyclictest6100-21runrttasks11:23:582
1989299903,40cyclictest1734-21unixbench_multi09:19:012
1989299903,37cyclictest8833-21/usr/sbin/munin09:33:472
1989299903,33cyclictest9499-21ls10:33:242
1989299903,32cyclictest27544-21wc07:28:552
1989299903,32cyclictest25819-21latency_hist09:03:252
1989299903,32cyclictest19756-21latency_hist07:08:272
1989299903,31cyclictest17496-21grep11:43:592
19892999029,12cyclictest0-21swapper/212:23:592
19892999027,13cyclictest0-21swapper/207:58:562
19892999021,23cyclictest21916-21switchtime07:13:332
1989299895,36cyclictest6121-21/usr/sbin/munin09:28:462
1989299895,36cyclictest6121-21/usr/sbin/munin09:28:462
1989299895,32cyclictest4618-21ssh11:22:072
1989299894,34cyclictest25097-21ls07:23:222
1989299893,39cyclictest23149-21ntp_kernel_err08:53:492
1989299893,39cyclictest23149-21ntp_kernel_err08:53:492
1989299893,35cyclictest19647-21fschecks_time11:48:472
1989299893,35cyclictest19647-21fschecks_time11:48:472
1989299893,33cyclictest17837-21apt-get08:38:352
19892998932,8cyclictest0-21swapper/212:28:452
19892998932,8cyclictest0-21swapper/207:33:592
19892998928,9cyclictest181rcu_preempt09:08:552
19892998928,9cyclictest181rcu_preempt09:08:552
19892998927,8cyclictest0-21swapper/209:48:552
19892998927,8cyclictest0-21swapper/209:48:552
19892998925,10cyclictest0-21swapper/210:08:482
1989299892,40cyclictest26894-21unixbench_singl09:04:212
1989299892,30cyclictest28423-21fschecks_count12:03:442
19881998960,20cyclictest24946-21apt-key11:58:251
19881998959,19cyclictest0-21swapper/110:33:411
1989299884,34cyclictest1188-21latency_hist12:13:242
1989299883,33cyclictest13682-21grep10:38:592
1989299883,33cyclictest13682-21grep10:38:592
1989299883,30cyclictest16880-21latency_hist09:48:252
1989299883,30cyclictest16880-21latency_hist09:48:252
1989299883,27cyclictest11221-21idleruntime11:33:492
1989299882,34cyclictest31986-21latency_hist07:43:262
1989299874,35cyclictest7393-21apt-get08:08:342
1989299874,29cyclictest32420-21fschecks_count11:13:442
1989299874,29cyclictest32420-21fschecks_count11:13:442
1989299873,31cyclictest1439-21fschecks_time07:48:442
1989299873,31cyclictest1439-21fschecks_time07:48:442
1989299873,29cyclictest13105-21ssh12:33:342
1989299872,36cyclictest26804-21fschecks_count11:03:452
1989299872,36cyclictest26804-21fschecks_count11:03:452
19892998719,13cyclictest0-21swapper/207:08:452
1988199876,64cyclictest7899-21apt-get08:08:521
19881998736,26cyclictest0-21swapper/111:33:571
1989299864,34cyclictest355-21runrttasks09:14:012
1989299863,32cyclictest20681-21latency_hist10:53:242
19881998668,12cyclictest207-21systemd-journal09:53:211
19881998668,12cyclictest207-21systemd-journal09:53:211
19881998664,15cyclictest16799-21fschecks_time11:43:451
1988199863,72cyclictest355-21runrttasks07:44:041
19877998562,16cyclictest22092-21latency_hist11:53:250
19877998562,16cyclictest22092-21latency_hist11:53:250
19877998559,17cyclictest27268-21diskmemload10:53:450
19877998559,17cyclictest27268-21diskmemload10:53:450
1987799842,73cyclictest24682-21/usr/sbin/munin10:59:480
19881998268,10cyclictest19868-21cyclictest10:13:531
19881998268,10cyclictest19868-21cyclictest10:13:531
1987799827,65cyclictest25384-21sshd11:58:470
1987799827,65cyclictest25384-21sshd11:58:470
19877998260,16cyclictest25118-21sh10:03:220
19877998260,16cyclictest25118-21sh10:03:220
19877998232,40cyclictest5989-21latency_hist08:03:250
19877998157,15cyclictest3074-21fschecks_time07:53:450
19877998157,15cyclictest3074-21fschecks_time07:53:450
19881998060,15cyclictest5933-21latency_hist09:28:241
19881998028,18cyclictest0-21swapper/110:20:571
19877997946,23cyclictest0-21swapper/010:49:000
1987799794,39cyclictest15-21ksoftirqd/007:08:430
19881997860,13cyclictest26241-21threads11:59:151
19881997860,13cyclictest26241-21threads11:59:151
1988199772,50cyclictest17837-21apt-get08:38:591
19881997721,20cyclictest0-21swapper/110:27:591
19877997757,13cyclictest12493-21idleruntime-cro08:23:210
19877997757,13cyclictest12493-21idleruntime-cro08:23:210
19877997756,14cyclictest25097-21latency_hist11:58:250
19877997734,25cyclictest18789-21ssh11:47:410
1987799773,38cyclictest10426-21uniq12:28:450
19881997657,13cyclictest1-21systemd09:38:211
19881997655,15cyclictest2237-21latency_hist11:18:251
19881997655,15cyclictest2237-21latency_hist11:18:251
19881997653,15cyclictest27716-21threads07:28:571
19877997657,13cyclictest32095-21cpu07:44:090
19877997655,14cyclictest19868-21cyclictest09:15:400
19877997655,12cyclictest28111-21ssh10:08:250
1987799765,37cyclictest15-21ksoftirqd/012:18:090
19877997649,22cyclictest13710-21ssh11:38:380
19877997645,19cyclictest6065-21latency_hist09:28:250
1987799764,49cyclictest9682-21apt-key10:33:260
19881997518,40cyclictest6712-21cat08:03:551
19881997518,40cyclictest6712-21cat08:03:551
19881997511,54cyclictest30564-21ssh09:13:441
1987799755,36cyclictest15-21ksoftirqd/010:43:580
19877997551,16cyclictest26666-21ssh10:04:290
19877997551,15cyclictest2776-21ssh11:18:440
19877997549,18cyclictest9687-21rm09:33:570
19877997547,19cyclictest355-21runrttasks12:33:500
1987799753,36cyclictest6121-21/usr/sbin/munin09:28:480
1987799753,36cyclictest6121-21/usr/sbin/munin09:28:480
1987799753,35cyclictest12175-21awk09:38:480
19881997454,14cyclictest16267-21idleruntime08:33:491
19881997448,19cyclictest30967-21latency_hist12:08:251
1979627411,37sleep119803-21munin-run07:08:251
184682749,40sleep30-21swapper/307:03:423
19881997353,13cyclictest0-21swapper/109:23:241
19881997351,13cyclictest0-21swapper/110:33:251
19881997345,19cyclictest355-21runrttasks10:49:021
19881997342,11cyclictest19742-21kworker/u19:1+events_unbound07:43:231
19881997340,9cyclictest181rcu_preempt12:38:241
19877997355,12cyclictest27268-21diskmemload11:28:400
19877997353,14cyclictest22296-21idleruntime-cro09:58:220
19877997350,14cyclictest1679-21ssh10:18:460
1987799734,45cyclictest15-21ksoftirqd/010:26:130
19877997324,11cyclictest161ktimers/011:18:240
19877997324,11cyclictest161ktimers/011:18:240
19881997257,10cyclictest19868-21cyclictest07:53:591
19881997257,10cyclictest19868-21cyclictest07:53:591
19881997250,16cyclictest309-21systemd-logind07:38:231
19881997248,16cyclictest316-21rs:main1
19881997242,20cyclictest1705-21chrt07:48:501
19881997242,20cyclictest1705-21chrt07:48:501
19881997240,23cyclictest11366-21fschecks_count08:18:481
19881997240,23cyclictest11366-21fschecks_count08:18:481
19877997250,16cyclictest316-21rs:main0
1987799723,35cyclictest19721-21latency_hist09:53:250
1987799723,35cyclictest19721-21latency_hist09:53:250
1987799722,35cyclictest3163-21sh09:23:240
1987799722,34cyclictest22868-21fschecks_count08:53:430
1987799722,34cyclictest22868-21fschecks_count08:53:430
1987799715,32cyclictest15-21ksoftirqd/009:05:380
19877997151,14cyclictest1-21systemd12:23:220
19899997018,31cyclictest0-21swapper/310:58:453
19881997046,19cyclictest12813-21kworker/u19:0+events_unbound10:08:211
1987799703,34cyclictest15-21ksoftirqd/012:09:020
1987799703,27cyclictest15-21ksoftirqd/008:33:310
1987799702,38cyclictest15-21ksoftirqd/012:06:170
19881996955,8cyclictest13553-21timerandwakeup10:38:581
19881996955,8cyclictest13553-21timerandwakeup10:38:581
19881996951,12cyclictest309-21systemd-logind11:23:221
19881996948,15cyclictest20262-21fschecks_count07:08:441
1988199694,39cyclictest8059-21/usr/sbin/munin11:29:001
19881996934,8cyclictest181rcu_preempt09:29:001
19881996934,8cyclictest181rcu_preempt09:29:001
19881996927,33cyclictest25641-21modprobe09:03:211
1987799695,39cyclictest15-21ksoftirqd/010:18:230
1987799695,39cyclictest15-21ksoftirqd/010:18:230
19877996950,12cyclictest355-21runrttasks11:08:540
1987799694,35cyclictest23499-21sendmail-msp07:18:220
19877996921,38cyclictest31542-21/usr/sbin/munin07:43:240
19881996850,13cyclictest29061-21threads10:08:571
19881996848,13cyclictest21706-21timerwakeupswit07:13:291
19881996846,14cyclictest27268-21diskmemload10:47:101
19881996842,19cyclictest30219-21latency_hist09:13:251
19881996842,19cyclictest30219-21latency_hist09:13:251
1987799684,38cyclictest15-21ksoftirqd/008:13:520
1987799684,38cyclictest15-21ksoftirqd/008:13:520
1987799681,21cyclictest13255-21ntpq08:23:530
19881996746,14cyclictest0-21swapper/111:03:391
19881996746,14cyclictest0-21swapper/111:03:391
19881996745,12cyclictest0-21swapper/108:53:581
19881996745,12cyclictest0-21swapper/108:53:581
1988199673,42cyclictest20954-21latency_hist08:48:251
1987799679,49cyclictest6466-21latency08:03:480
1987799679,49cyclictest6466-21latency08:03:480
19877996743,16cyclictest0-21swapper/008:33:240
19877996742,15cyclictest0-21swapper/007:53:220
19877996742,15cyclictest0-21swapper/007:53:220
19877996736,25cyclictest28303-21mailstats09:08:570
19877996736,25cyclictest28303-21mailstats09:08:570
19877996712,35cyclictest15-21ksoftirqd/011:37:550
19881996646,13cyclictest9772-21fschecks_time08:13:451
19881996646,13cyclictest9772-21fschecks_time08:13:451
1988199663,51cyclictest15295-21unixbench_singl09:44:011
1988199663,51cyclictest15295-21unixbench_singl09:44:011
19881996633,23cyclictest0-21swapper/112:18:261
1987799664,39cyclictest15-21ksoftirqd/008:11:200
1987799664,37cyclictest15-21ksoftirqd/011:07:020
1987799664,37cyclictest15-21ksoftirqd/011:07:020
1987799664,35cyclictest15-21ksoftirqd/007:28:550
1987799663,51cyclictest14215-21/usr/sbin/munin09:43:460
1987799663,51cyclictest14215-21/usr/sbin/munin09:43:460
1987799663,43cyclictest28951-21cat07:33:480
1987799663,31cyclictest24915-21find08:58:540
19877996620,22cyclictest211rcuc/007:23:380
19877996610,43cyclictest21954-21grep08:48:590
19881996545,14cyclictest22283-21latency_hist09:58:251
19881996544,15cyclictest10123-21ssh12:28:321
1988199653,45cyclictest12739-21/usr/sbin/munin08:23:581
1988199653,37cyclictest21587-21mailstats10:53:581
1988199653,37cyclictest21587-21mailstats10:53:581
62872640,7sleep30-21swapper/310:27:153
19881996446,12cyclictest0-21swapper/112:08:561
19881996441,17cyclictest30073-21kworker/u17:3-rpciod09:39:181
19881996419,34cyclictest5089-21sensors_temp12:18:561
1987799643,39cyclictest24000-21fschecks_time07:18:460
1987799642,38cyclictest15-21ksoftirqd/010:38:550
1987799642,38cyclictest15-21ksoftirqd/010:38:550
19881996343,11cyclictest0-21swapper/111:09:071
1988199633,44cyclictest26316-21latency09:03:471
1988199633,43cyclictest7502-21fschecks_time12:23:451
1988199632,45cyclictest26281-21unixbench_multi07:23:581
1987799632,37cyclictest15-21ksoftirqd/008:43:500
19881996242,12cyclictest0-21swapper/111:23:341
19881996239,15cyclictest27268-21diskmemload11:38:461
19881996236,18cyclictest0-21swapper/108:49:001
1988199622,40cyclictest4656-21fschecks_count07:58:441
19881996140,13cyclictest0-21swapper/108:28:441
19899996011,31cyclictest0-21swapper/310:33:413
1989999593,37cyclictest5134-21taskset07:58:563
1989999592,39cyclictest21290-21taskset09:55:153
19881995941,10cyclictest0-21swapper/111:49:011
19881995941,10cyclictest0-21swapper/111:49:011
1989999573,36cyclictest10008-21chrt08:13:513
1989999573,36cyclictest10008-21chrt08:13:513
1989999563,40cyclictest16360-21taskset11:43:243
19899995627,19cyclictest20613-21chrt08:46:113
1989999553,36cyclictest20812-21taskset10:53:293
1989999553,36cyclictest20812-21taskset10:53:293
1989999552,38cyclictest26905-21chrt09:04:213
1989999544,38cyclictest14021-21taskset12:33:573
1989999543,40cyclictest20711-21taskset10:53:253
1989999543,38cyclictest25418-21taskset11:58:523
1989999543,38cyclictest25418-21taskset11:58:523
1989999543,37cyclictest31020-21taskset10:13:343
1989999543,37cyclictest31020-21taskset10:13:343
1989999543,35cyclictest895-21kworker/3:1+events07:38:533
1989999543,33cyclictest10501-21taskset12:28:473
1989999542,37cyclictest14182-21taskset10:40:203
1989999542,37cyclictest14182-21taskset10:40:203
1989999534,33cyclictest19869-21chrt11:48:513
1989999534,33cyclictest19869-21chrt11:48:513
1989999533,38cyclictest9404-21taskset10:33:233
1989999532,40cyclictest23287-21taskset09:58:553
1989999532,40cyclictest23287-21taskset09:58:553
1989999532,36cyclictest17387-21chrt09:48:463
1989999532,36cyclictest17387-21chrt09:48:463
1989999532,34cyclictest32166-21chrt11:13:303
1989999532,34cyclictest32166-21chrt11:13:303
1989999523,36cyclictest7544-21chrt08:08:243
1989999523,36cyclictest7544-21chrt08:08:243
1989999523,33cyclictest8884-21chrt11:28:583
1989999513,37cyclictest24583-21chrt07:18:593
1989999513,36cyclictest2342-21chrt09:20:593
1989999513,35cyclictest16314-21chrt09:47:083
1989999513,35cyclictest16314-21chrt09:47:083
1989999512,37cyclictest32079-21taskset07:43:583
1989999512,37cyclictest26533-21chrt11:03:263
1989999512,37cyclictest26533-21chrt11:03:263
1989999512,35cyclictest30949-21chrt10:13:263
1989999503,37cyclictest4214-21taskset12:18:253
1989999503,34cyclictest31473-21chrt09:14:063
1989999502,37cyclictest28840-21taskset12:03:553
1989999502,36cyclictest30135-21taskset09:13:253
1989999502,36cyclictest30135-21taskset09:13:253
1989999502,33cyclictest1405-21chrt10:18:323
1989999493,35cyclictest7411-21taskset12:23:443
1989999493,35cyclictest7080-21chrt09:29:013
1989999493,35cyclictest7080-21chrt09:29:013
1989999493,31cyclictest8671-21chrt08:09:303
1989999492,36cyclictest25898-21chrt07:23:533
1989999492,35cyclictest20675-21taskset07:08:553
1989999492,34cyclictest22797-21chrt11:53:503
1989999492,34cyclictest11572-21chrt09:38:253
1989999492,33cyclictest16121-21chrt10:43:573
1989999486,29cyclictest23752-21chrt08:55:143
1989999486,29cyclictest23752-21chrt08:55:143
1989999483,34cyclictest3556-21chrt09:23:423
1989999483,34cyclictest30792-21chrt11:10:023
1989999483,33cyclictest28449-21chrt07:33:253
1989999483,33cyclictest21889-21chrt07:13:323
1989999483,33cyclictest1437-21chrt07:48:443
1989999483,33cyclictest1437-21chrt07:48:443
1989999482,35cyclictest11784-21taskset09:38:373
1989999482,32cyclictest16730-21chrt08:34:043
1989999474,33cyclictest16671-21taskset11:43:413
19899994736,6cyclictest0-21swapper/308:39:133
1989999473,35cyclictest21046-21chrt08:48:343
1989999473,33cyclictest4928-21chrt11:23:073
1989999473,33cyclictest31411-21chrt12:08:453
1989999473,32cyclictest6800-21chrt11:25:133
1989999473,32cyclictest15266-21chrt08:28:583
1989999473,31cyclictest12081-21chrt11:34:253
1989999473,30cyclictest26924-21taskset10:05:193
1989999463,31cyclictest4246-21chrt07:58:243
1989999463,31cyclictest4246-21chrt07:58:243
1989999452,27cyclictest24358-21chrt08:58:383
1989999443,29cyclictest14267-21chrt08:28:253
1989999402,26cyclictest28542-21taskset07:33:293
1989999392,26cyclictest11163-21taskset08:18:413
1989999392,26cyclictest11163-21taskset08:18:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional