You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-17 - 07:27
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackcslot0.osadl.org (updated Wed Sep 17, 2025 00:43:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2727121730,6sleep22814999cyclictest22:20:238
27323212792,30sleep40-21swapper/419:07:0510
27571212289,28sleep30-21swapper/319:10:179
27386211893,20sleep100-21swapper/1019:08:002
27567211492,16sleep10-21swapper/119:10:151
27566211092,14sleep00-21swapper/019:10:140
525321080,0sleep100-21swapper/1023:24:432
27685210690,9sleep110-21swapper/1119:11:133
2267921060,6sleep102815899cyclictest23:26:512
2755221050,4sleep72815499cyclictest00:28:5113
2084021030,2sleep100-21swapper/1023:18:332
1575221030,2sleep215753-2110-uname23:17:538
1665321010,0sleep90-21swapper/923:38:0415
91342990,0sleep30-21swapper/323:41:109
291852980,1sleep150-21swapper/1521:18:177
291852980,1sleep150-21swapper/1521:18:177
273032980,0sleep40-21swapper/422:24:1010
196142970,0sleep120-21swapper/1221:21:104
196142970,0sleep120-21swapper/1221:21:104
59582960,7sleep125-21ksoftirqd/121:54:091
2747929675,17sleep50-21swapper/519:08:5911
193052960,0sleep100-21swapper/1023:42:062
131852960,1sleep50-21swapper/522:26:3511
8692930,5sleep122816299cyclictest21:53:324
2767629274,13sleep20-21swapper/219:11:048
33462890,1sleep90-21swapper/923:56:3115
33462890,1sleep90-21swapper/923:56:3115
2760228976,9sleep150-21swapper/1519:10:457
2748028972,4sleep60-21swapper/619:09:0012
8082880,2sleep5809-21sshd23:40:0811
306392880,6sleep112815999cyclictest22:40:203
2751728876,8sleep70-21swapper/719:09:3313
155842880,1sleep120-21swapper/1221:47:234
284212870,0sleep110-21swapper/1123:03:203
296942860,1sleep150-21swapper/1521:37:327
296942860,1sleep150-21swapper/1521:37:327
2760128676,6sleep140-21swapper/1419:10:446
230682860,0sleep40-21swapper/400:24:1510
2759928575,6sleep120-21swapper/1219:10:424
130352850,0sleep60-21swapper/623:13:4112
2754228472,8sleep130-21swapper/1319:09:555
63162830,1sleep76320-21cpuspeed_turbos22:01:4513
210452830,1sleep150-21swapper/1523:30:577
52862820,0sleep00-21swapper/023:20:480
2744528275,3sleep90-21swapper/919:08:3115
12212820,0sleep120-21swapper/1223:08:104
2738228172,4sleep80-21swapper/819:07:5814
259122810,0sleep2311rcuc/221:41:048
259122810,0sleep2311rcuc/221:41:048
18582810,0sleep70-21swapper/721:34:2813
18582810,0sleep70-21swapper/721:34:2813
326402790,0sleep150-21swapper/1523:43:477
245582720,0sleep80-21swapper/823:19:0414
142612710,1sleep50-21swapper/521:13:0211
130352600,2sleep122816299cyclictest23:41:364
16132580,0sleep110-21swapper/1123:12:093
30582550,5sleep558-21ksoftirqd/522:25:1511
239722510,6sleep12814599cyclictest22:59:091
1139099510,0rtkit-daemon0-21swapper/1421:34:2410
1139099510,0rtkit-daemon0-21swapper/1421:34:2410
2816499500,47cyclictest0-21swapper/1422:21:556
2816499500,43cyclictest525-21systemd-journal22:49:206
2816499491,39cyclictest9178-21systemd-cgroups23:09:186
2816499490,41cyclictest0-21swapper/1421:28:436
2816499490,41cyclictest0-21swapper/1421:28:436
2816499480,45cyclictest16216-21kill23:54:066
2816499480,45cyclictest16216-21kill23:54:066
2816499480,41cyclictest0-21swapper/1422:27:266
2816499480,40cyclictest0-21swapper/1422:13:536
43942470,1sleep10-21swapper/100:18:191
2816499471,43cyclictest19477-21bash23:30:446
2816499471,37cyclictest0-21swapper/1423:03:036
2816499471,35cyclictest26992-21sshd22:08:386
2816499471,35cyclictest0-21swapper/1422:03:416
2816499470,42cyclictest0-21swapper/1423:47:076
2816499470,41cyclictest1-21systemd22:35:006
2816499470,40cyclictest0-21swapper/1400:32:176
2816499470,37cyclictest0-21swapper/1400:41:026
2816499470,36cyclictest0-21swapper/1421:53:396
2816499470,36cyclictest0-21swapper/1400:27:516
2816499460,43cyclictest0-21swapper/1421:21:566
2816499460,43cyclictest0-21swapper/1421:21:566
2816499460,40cyclictest22207-21kworker/u34:022:01:226
2816499460,40cyclictest0-21swapper/1421:13:016
2816499460,38cyclictest525-21systemd-journal23:11:496
2816499460,35cyclictest190291unixbench_multi21:17:036
2816499460,35cyclictest190291unixbench_multi21:17:036
2816499460,35cyclictest0-21swapper/1400:23:036
2816499460,35cyclictest0-21swapper/1400:20:276
2816499452,41cyclictest525-21systemd-journal21:50:156
2816499451,37cyclictest0-21swapper/1400:13:326
2816499450,38cyclictest1349-21nfsd23:22:346
2816499450,37cyclictest26469-21kill22:39:516
2816499450,36cyclictest6640-21systemd-cgroups22:45:106
2816499450,34cyclictest0-21swapper/1421:43:566
238862450,1sleep31-21systemd23:18:589
1139099450,0rtkit-daemon0-21swapper/420:31:1610
2816499441,36cyclictest13113-21env23:17:336
2816499441,36cyclictest1-21systemd22:59:206
2816499440,42cyclictest0-21swapper/1421:34:336
2816499440,42cyclictest0-21swapper/1421:34:336
2816499440,37cyclictest525-21systemd-journal23:59:526
2816499440,37cyclictest525-21systemd-journal23:59:526
2816499440,37cyclictest5251systemd-journal21:37:366
2816499440,37cyclictest5251systemd-journal21:37:366
2816499440,35cyclictest525-21systemd-journal23:39:106
2816499440,32cyclictest0-21swapper/1423:41:436
2816499431,40cyclictest0-21swapper/1422:19:396
2816499431,39cyclictest477-21jbd2/md124p1-823:32:576
2816499430,37cyclictest525-21systemd-journal00:03:366
2816499430,37cyclictest525-21systemd-journal00:03:366
2816499430,35cyclictest0-21swapper/1400:07:076
2816499430,35cyclictest0-21swapper/1400:07:076
2816499430,33cyclictest0-21swapper/1422:53:266
2816499430,31cyclictest0-21swapper/1421:07:156
2816499420,39cyclictest30772-21cat20:31:446
2816399421,3cyclictest0-21swapper/1321:27:415
2816399421,3cyclictest0-21swapper/1321:27:415
1139099420,1rtkit-daemon12307-21systemctl22:46:0010
2816499410,39cyclictest29874-21cstates19:51:446
28163994115,2cyclictest0-21swapper/1323:57:495
28163994115,2cyclictest0-21swapper/1323:57:495
276372410,1sleep100-21swapper/1022:36:182
174322410,0sleep70-21swapper/723:58:0513
174322410,0sleep70-21swapper/723:58:0513
2816499401,36cyclictest0-21swapper/1420:16:526
2816499400,38cyclictest375-21md125_raid120:51:446
2816499400,37cyclictest3444-21timerandwakeup19:57:046
2816499400,31cyclictest7011cstates19:16:466
2816399400,4cyclictest2152-21sh22:56:375
2815299400,37cyclictest0-21swapper/523:34:5311
248972400,1sleep120-21swapper/1221:21:434
248972400,1sleep120-21swapper/1221:21:434
1139099400,0rtkit-daemon0-21swapper/1200:11:044
1139099400,0rtkit-daemon0-21swapper/1200:11:044
2816499391,36cyclictest0-21swapper/1420:21:546
2816499390,37cyclictest9508-21grep19:26:446
2816499390,37cyclictest20765-21switchtime20:57:016
2816499390,36cyclictest27718-21missed_timers20:26:536
2816499390,36cyclictest22061-21fschecks_time19:41:466
2816499390,36cyclictest1779-21/usr/sbin/munin20:36:536
2816499390,36cyclictest17666-21latency_hist20:16:326
2815299390,37cyclictest0-21swapper/521:21:5611
2815299390,37cyclictest0-21swapper/521:21:5611
247402390,1sleep60-21swapper/623:46:5012
186732390,1sleep111041rcuc/1100:27:353
2816499380,36cyclictest0-21swapper/1419:36:526
2816499380,36cyclictest0-21swapper/1419:21:516
2816499380,35cyclictest30604-21nfsd419:11:536
2816499380,35cyclictest10247-21grep20:06:506
2815299380,31cyclictest0-21swapper/521:32:4811
2815299380,31cyclictest0-21swapper/521:32:4811
176452380,4sleep62815399cyclictest22:46:4012
1139099380,1rtkit-daemon1894-21sendmail-mta23:13:074
2816499371,33cyclictest4907-21ffmpeg20:01:326
2816499370,35cyclictest26180-21grep19:46:466
2816499370,35cyclictest24307-21sed21:01:546
2816499370,34cyclictest0-21swapper/1420:41:456
2816499370,34cyclictest0-21swapper/1419:31:476
28162993715,3cyclictest16378-21run-parts22:22:404
2815299370,35cyclictest0-21swapper/520:21:4211
2815299370,32cyclictest0-21swapper/522:01:3211
2815299370,31cyclictest0-21swapper/523:45:3511
2815299370,28cyclictest0-21swapper/521:21:0711
2815299370,28cyclictest0-21swapper/521:21:0711
2815299370,27cyclictest0-21swapper/522:45:4511
2815299370,27cyclictest0-21swapper/522:12:4311
2816499362,32cyclictest0-21swapper/1420:46:496
2815299360,34cyclictest0-21swapper/523:49:5211
2815299360,34cyclictest0-21swapper/521:38:0811
2815299360,34cyclictest0-21swapper/521:38:0811
2815299360,31cyclictest0-21swapper/521:59:1111
2815299360,30cyclictest0-21swapper/523:56:2111
2815299360,30cyclictest0-21swapper/523:56:2111
2815299360,30cyclictest0-21swapper/500:32:0711
1139099360,3rtkit-daemon11389-21rtkit-daemon21:42:524
2815299350,33cyclictest0-21swapper/523:58:0911
2815299350,33cyclictest0-21swapper/523:58:0911
2815299350,33cyclictest0-21swapper/522:56:1811
2815299350,33cyclictest0-21swapper/521:47:4011
2815299350,33cyclictest0-21swapper/500:22:4011
2815299350,31cyclictest0-21swapper/523:21:2311
2815299350,30cyclictest0-21swapper/521:29:0211
2815299350,30cyclictest0-21swapper/521:29:0211
2815299350,28cyclictest0-21swapper/523:23:4011
2815299350,28cyclictest0-21swapper/522:18:0911
2815299350,27cyclictest0-21swapper/500:40:0511
2815299350,26cyclictest0-21swapper/500:07:5211
2815299350,26cyclictest0-21swapper/500:07:5211
272122350,1sleep40-21swapper/400:28:4810
226352350,1sleep90-21swapper/923:18:4815
1139099350,3rtkit-daemon11389-21rtkit-daemon22:03:484
1139099350,1rtkit-daemon1351-21nfsd21:20:4312
1139099350,1rtkit-daemon1351-21nfsd21:20:4312
2816399340,17cyclictest24899-21sshd23:15:165
2816399340,17cyclictest0-21swapper/1321:56:495
2816399340,17cyclictest0-21swapper/1300:08:405
2816399340,17cyclictest0-21swapper/1300:08:405
2815299340,32cyclictest0-21swapper/523:06:4911
2815299340,32cyclictest0-21swapper/522:33:4811
2815299340,29cyclictest0-21swapper/522:40:2311
2815299340,28cyclictest0-21swapper/500:30:3111
2815299340,28cyclictest0-21swapper/500:11:4111
2815299340,27cyclictest0-21swapper/523:12:1611
2815299340,27cyclictest0-21swapper/500:03:1111
2815299340,27cyclictest0-21swapper/500:03:1111
2815299340,25cyclictest31186-21systemd-cgroups21:41:3911
2815299340,25cyclictest0-21swapper/519:58:5711
98932330,1sleep30-21swapper/322:34:029
28152993315,16cyclictest0-21swapper/523:28:2511
2815299330,28cyclictest0-21swapper/520:26:5311
2815299330,27cyclictest0-21swapper/523:06:0311
2815299330,27cyclictest0-21swapper/522:58:3711
2815299330,26cyclictest0-21swapper/500:20:3811
28152993220,10cyclictest22050irq/30-ahci[00020:31:3711
28152993220,10cyclictest1350-21nfsd19:51:5611
2815299320,28cyclictest0-21swapper/520:41:4011
2815299320,27cyclictest0-21swapper/522:46:5611
2815299320,26cyclictest0-21swapper/522:07:3311
2815299320,26cyclictest0-21swapper/519:12:2911
2815299320,25cyclictest0-21swapper/521:55:3711
115192320,2sleep011501-21sshd22:02:100
1139099320,0rtkit-daemon0-21swapper/1421:02:3912
1139099320,0rtkit-daemon0-21swapper/1200:04:364
1139099320,0rtkit-daemon0-21swapper/1200:04:364
316332310,1sleep2321ktimersoftd/222:40:278
28282310,1sleep30-21swapper/321:22:399
28282310,1sleep30-21swapper/321:22:399
2815299310,30cyclictest0-21swapper/519:16:4811
2815299310,28cyclictest0-21swapper/521:06:5311
2815299310,25cyclictest0-21swapper/521:00:3911
2815299310,25cyclictest0-21swapper/519:46:5911
26062310,27sleep40-21swapper/421:38:1010
26062310,27sleep40-21swapper/421:38:1010
222152310,0sleep60-21swapper/600:32:0312
1139099310,0rtkit-daemon0-21swapper/520:05:375
1139099310,0rtkit-daemon0-21swapper/421:44:2210
113412310,1sleep00-21swapper/023:53:260
113412310,1sleep00-21swapper/023:53:260
325692300,1sleep20-21swapper/223:47:528
306592300,1sleep40-21swapper/400:07:2510
306592300,1sleep40-21swapper/400:07:2510
303262300,26sleep230421-21sshd23:36:068
28152993012,17cyclictest0-21swapper/520:16:3111
2815299300,26cyclictest0-21swapper/520:41:3111
2815299300,24cyclictest0-21swapper/520:01:3911
2815299300,23cyclictest0-21swapper/520:46:5111
2815299300,23cyclictest0-21swapper/519:31:4611
221512300,1sleep70-21swapper/723:54:5313
221512300,1sleep70-21swapper/723:54:5313
305382290,1sleep30-21swapper/300:07:239
305382290,1sleep30-21swapper/300:07:239
2816399290,2cyclictest0-21swapper/1322:08:305
2815299290,27cyclictest0-21swapper/520:54:1111
2815299290,27cyclictest0-21swapper/520:09:3911
2815299290,27cyclictest0-21swapper/519:38:1311
2815299290,23cyclictest0-21swapper/519:42:2511
2815299290,23cyclictest0-21swapper/519:26:4411
2815299290,21cyclictest0-21swapper/519:21:3611
2814999291,3cyclictest0-21swapper/221:12:468
270882290,25sleep10-21swapper/119:46:531
244362290,1sleep120-21swapper/1221:14:144
211672290,25sleep21068-21dbus-daemon00:10:368
211672290,25sleep21068-21dbus-daemon00:10:368
208972290,2sleep30-21swapper/321:13:499
143522290,1sleep2321ktimersoftd/220:11:458
1139099290,1rtkit-daemon11389-21rtkit-daemon23:16:2010
1139099290,0rtkit-daemon0-21swapper/422:48:1310
1139099290,0rtkit-daemon0-21swapper/419:58:2010
63922280,24sleep100-21swapper/1023:52:492
63922280,24sleep100-21swapper/1023:52:492
37822280,2sleep150-21swapper/1522:29:277
2815299280,26cyclictest0-21swapper/520:16:3711
2815299280,23cyclictest0-21swapper/521:04:3711
28151992821,4cyclictest27861-21sd-pam22:16:4310
279152280,1sleep100-21swapper/1023:47:122
240772281,3sleep31233-21gdbus22:03:509
239222280,1sleep110-21swapper/1123:51:133
201172280,1sleep80-21swapper/800:06:2514
201172280,1sleep80-21swapper/800:06:2514
185012280,3sleep1018787-21systemd22:11:422
153932280,2sleep30-21swapper/323:10:099
146882280,1sleep100-21swapper/1023:06:002
1139099280,0rtkit-daemon0-21swapper/621:08:4712
1139099280,0rtkit-daemon0-21swapper/400:19:3210
1139099280,0rtkit-daemon0-21swapper/1219:17:574
52742270,6sleep125-21ksoftirqd/122:37:131
255132270,1sleep80-21swapper/823:51:2614
204592270,0sleep20-21swapper/221:21:168
204592270,0sleep20-21swapper/221:21:168
166982270,1sleep60-21swapper/621:13:2012
134172270,2sleep90-21swapper/923:13:4515
110612270,23sleep110-21swapper/1122:18:253
75572260,1sleep150-21swapper/1523:33:057
65152260,2sleep100-21swapper/1023:08:572
302382260,21sleep80-21swapper/822:40:1814
283092260,2sleep81297-21sshd21:14:4214
28163992615,3cyclictest0-21swapper/1321:37:135
28163992615,3cyclictest0-21swapper/1321:37:135
2815999260,4cyclictest8770-21sshd22:41:333
229242260,1sleep30-21swapper/300:24:159
216602260,2sleep30-21swapper/321:44:189
189392261,21sleep60-21swapper/623:02:1112
137042260,1sleep130-21swapper/1323:45:385
1139099260,8rtkit-daemon0-21swapper/1223:25:224
38022250,20sleep100-21swapper/1021:22:462
38022250,20sleep100-21swapper/1021:22:462
24962250,21sleep20-21swapper/219:16:578
167682250,0sleep40-21swapper/422:31:0710
147742250,21sleep20-21swapper/222:50:218
121532250,22sleep40-21swapper/423:33:4310
1139099250,3rtkit-daemon30081-21bash21:49:2010
1139099250,0rtkit-daemon0-21swapper/423:21:3410
111972250,2sleep20-21swapper/200:01:308
111972250,2sleep20-21swapper/200:01:308
105612250,1sleep10-21swapper/122:21:591
103522250,22sleep30-21swapper/321:51:009
80292240,1sleep60-21swapper/600:04:4712
80292240,1sleep60-21swapper/600:04:4712
75172240,1sleep90-21swapper/920:01:5915
64162240,2sleep80-21swapper/800:38:1514
57732240,4sleep32815099cyclictest21:19:259
57732240,4sleep32815099cyclictest21:19:259
40032240,22sleep90-21swapper/919:58:4715
38292240,1sleep130-21swapper/1323:24:305
2814599241,20cyclictest1745-21sshd21:38:041
2814599241,20cyclictest1745-21sshd21:38:041
28072240,1sleep80-21swapper/800:08:0114
28072240,1sleep80-21swapper/800:08:0114
264322240,1sleep80-21swapper/800:32:3814
124892240,20sleep80-21swapper/822:14:5314
123572240,1sleep100-21swapper/1020:08:092
113952240,2sleep20-21swapper/220:06:578
1139099240,0rtkit-daemon0-21swapper/1219:28:174
87952230,1sleep80-21swapper/822:57:0914
78902230,1sleep10-21swapper/123:25:071
71672230,1sleep10-21swapper/122:05:561
7042230,7sleep62815399cyclictest23:28:0712
46042230,19sleep20-21swapper/223:04:328
327172230,2sleep00-21swapper/022:24:550
28163992316,4cyclictest11120-21systemd23:29:365
2816399230,2cyclictest0-21swapper/1322:51:515
2815899231,20cyclictest0-21swapper/1020:41:442
28157992318,3cyclictest891ktimersoftd/922:18:0315
28157992312,7cyclictest90-21ksoftirqd/923:34:4215
2815799231,20cyclictest0-21swapper/900:37:0415
28156992318,2cyclictest13273-21sshd23:21:4414
2815199230,20cyclictest0-21swapper/419:36:4610
245902230,2sleep31073-21systemd-logind23:15:159
216192230,1sleep90-21swapper/922:11:5715
17062230,1sleep00-21swapper/023:08:150
1139099230,0rtkit-daemon0-21swapper/420:33:2210
28163992218,2cyclictest12745-21kworker/13:523:04:355
2816399220,2cyclictest0-21swapper/1322:21:415
2815999221,19cyclictest16965-21kworker/11:421:17:223
2815999221,19cyclictest16965-21kworker/11:421:17:223
28158992218,2cyclictest0-21swapper/1022:21:432
2815799224,15cyclictest90-21ksoftirqd/922:44:1915
28154992216,4cyclictest21147-2110-uname00:40:0713
28151992215,5cyclictest8451-21sh21:23:2210
28151992215,5cyclictest8451-21sh21:23:2210
2815199221,2cyclictest0-21swapper/400:01:4410
2815199221,2cyclictest0-21swapper/400:01:4410
2815199221,19cyclictest0-21swapper/422:52:4710
2815199220,20cyclictest2429-21sshd00:37:4210
270802220,1sleep130-21swapper/1321:52:465
219602220,0sleep80-21swapper/822:31:4314
210732220,1sleep110-21swapper/1122:03:263
197452220,1sleep130-21swapper/1321:47:575
182322220,0sleep2321ktimersoftd/221:51:468
168772220,18sleep10-21swapper/123:50:151
33872210,19sleep130-21swapper/1319:17:565
28165992116,3cyclictest8272-21sh21:42:317
28165992116,3cyclictest26110-21run-parts21:52:387
28163992114,4cyclictest15963-21sh23:34:135
2816399211,18cyclictest8476-21sshd00:18:485
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional