You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 06:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackcslot0.osadl.org (updated Tue Dec 02, 2025 00:43:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3033021760,7sleep121333999cyclictest00:28:054
3033021760,7sleep121333999cyclictest00:28:054
12567212695,25sleep90-21swapper/919:08:4415
128012120103,10sleep100-21swapper/1019:11:422
533321150,1sleep60-21swapper/600:04:5912
2473721150,6sleep51333199cyclictest22:33:4611
12699211393,15sleep20-21swapper/219:10:138
12692211295,13sleep120-21swapper/1219:10:074
2015421110,2sleep20-21swapper/221:27:078
2289021050,2sleep0121rcuc/021:49:170
848521030,0sleep120-21swapper/1222:32:174
12891210291,7sleep130-21swapper/1319:12:175
313421000,1sleep130-21swapper/1320:17:465
1208421000,4sleep150-21swapper/1523:17:327
233932990,1sleep131201rcuc/1323:37:335
1263529985,10sleep140-21swapper/1419:09:376
55602980,1sleep141281rcuc/1422:13:036
55602980,1sleep141281rcuc/1422:13:036
295132980,2sleep30-21swapper/322:08:359
295132980,2sleep30-21swapper/322:08:359
41842970,1sleep50-21swapper/500:25:0211
314962970,1sleep40-21swapper/423:42:1310
212812970,1sleep20-21swapper/200:16:098
152052970,1sleep80-21swapper/822:43:5914
117192970,2sleep50-21swapper/521:40:3211
1139099960,84rtkit-daemon0-21swapper/419:11:4110
42202940,0sleep120-21swapper/1222:57:414
286262930,1sleep10-21swapper/121:46:211
1290829381,8sleep110-21swapper/1119:12:313
56962920,0sleep40-21swapper/423:31:3110
1270029284,4sleep30-21swapper/319:10:149
326462910,1sleep121068-21dbus-daemon22:23:434
306082910,1sleep10-21swapper/123:26:551
28412900,0sleep70-21swapper/722:49:5113
131452900,1sleep50-21swapper/500:15:0811
94562890,6sleep1098-21ksoftirqd/1023:02:122
48122890,1sleep60-21swapper/623:16:3512
149722880,0sleep90-21swapper/922:47:4415
189422850,5sleep21332599cyclictest22:44:258
1286228572,7sleep10-21swapper/119:11:491
1266828572,8sleep60-21swapper/619:10:0112
1258628565,15sleep70-21swapper/719:08:5813
1286928472,8sleep80-21swapper/819:11:5614
1286128375,4sleep00-21swapper/019:11:480
1276828375,4sleep150-21swapper/1519:11:157
1266528375,4sleep50-21swapper/519:10:0011
181302820,0sleep40-21swapper/400:26:5310
319452800,0sleep130-21swapper/1322:57:105
315642800,0sleep40-21swapper/423:04:3010
29532800,6sleep151334499cyclictest22:42:457
247752800,7sleep21332599cyclictest21:27:398
17252780,0sleep90-21swapper/923:08:2015
144802780,0sleep70-21swapper/723:09:5413
65472750,5sleep111333899cyclictest23:42:523
28802730,0sleep140-21swapper/1400:39:546
28802730,0sleep140-21swapper/1400:39:546
50282650,1sleep90-21swapper/900:36:3615
50282650,1sleep90-21swapper/900:36:3615
241442630,1sleep10-21swapper/122:33:421
1139099590,30rtkit-daemon0-21swapper/1219:43:444
1139099590,30rtkit-daemon0-21swapper/1219:43:444
308242580,0sleep40-21swapper/400:04:0910
1139099510,18rtkit-daemon1894-21sendmail-mta00:19:5410
190142490,0sleep30-21swapper/300:15:549
1332299480,5cyclictest0-21swapper/022:20:440
1332299480,5cyclictest0-21swapper/022:20:440
1332299480,45cyclictest19190-21sshd23:40:360
1139099480,19rtkit-daemon0-21swapper/423:53:5310
1332299467,36cyclictest26615-21sshd00:00:150
1332299460,43cyclictest26131-21sshd23:33:460
1332299460,3cyclictest1068-21dbus-daemon22:46:300
1139099460,6rtkit-daemon0-21swapper/1223:41:124
7602450,0sleep50-21swapper/500:04:2611
1332299451,38cyclictest1073-21systemd-logind23:07:200
1332299440,41cyclictest31596-21sshd00:04:150
1332299438,32cyclictest1068-21dbus-daemon22:15:410
1332299438,32cyclictest1068-21dbus-daemon22:15:410
1332299436,34cyclictest1233-21gdbus21:12:390
1332299434,4cyclictest1233-21gdbus00:41:230
1332299434,4cyclictest1233-21gdbus00:41:230
227692420,23sleep10-21swapper/121:38:041
1332299429,30cyclictest0-21swapper/021:44:220
1332299428,30cyclictest7025-2110-uname23:27:470
1332299426,4cyclictest9402-21gdbus22:50:270
1332299421,4cyclictest30707-21systemd23:53:210
1332299421,38cyclictest10475-21sshd22:54:240
1332299420,4cyclictest29220-21systemd23:00:340
1332299420,4cyclictest2659-21gdbus00:19:540
1332299420,4cyclictest11542-21sshd23:24:380
1332299420,40cyclictest0-21swapper/021:21:320
1332299420,39cyclictest32501-21sshd00:08:060
124992420,1sleep80-21swapper/821:48:0514
1334499412,3cyclictest3000-21kworker/15:522:49:537
1332299418,29cyclictest4075-21uname21:57:550
1332299418,29cyclictest4075-21uname21:57:550
1332299411,37cyclictest3790-21uname21:54:160
13322994111,28cyclictest0-21swapper/000:24:430
13322994110,29cyclictest0-21swapper/000:17:060
13322994110,28cyclictest525-21systemd-journal22:11:270
13322994110,28cyclictest525-21systemd-journal22:11:270
1332299410,3cyclictest5467-21gdbus22:32:580
1332299410,3cyclictest1068-21dbus-daemon22:23:500
172592400,1sleep110-21swapper/1121:26:443
1334499401,19cyclictest11113-21diskstats19:47:477
1332299408,29cyclictest21968-21sshd21:41:460
1332299407,30cyclictest1233-21gdbus21:30:570
1332299401,3cyclictest1238-21gdbus23:15:280
1332299401,3cyclictest1068-21dbus-daemon00:31:350
1332299401,3cyclictest1068-21dbus-daemon00:31:350
1332299400,4cyclictest17699-21gdbus22:37:420
1332299399,27cyclictest0-21swapper/021:36:310
1332299396,3cyclictest7266-21uname23:42:540
1332299394,3cyclictest25478-21systemctl00:35:150
1332299394,3cyclictest25478-21systemctl00:35:150
1332299391,4cyclictest21211-21sshd21:23:100
1332299390,37cyclictest0-21swapper/022:27:530
1139099390,0rtkit-daemon0-21swapper/1221:22:124
309972380,3sleep131334099cyclictest21:57:315
1332299381,4cyclictest20645-21systemd23:48:040
1332299381,3cyclictest1068-21dbus-daemon23:11:170
193332370,6sleep1098-21ksoftirqd/1022:07:382
193332370,6sleep1098-21ksoftirqd/1022:07:382
1332299371,4cyclictest5467-21gdbus22:03:310
1332299371,4cyclictest5467-21gdbus22:03:310
1332299371,4cyclictest2344-21kill23:19:580
60732360,1sleep70-21swapper/721:29:0413
1332299363,3cyclictest19724-21nfsd419:17:530
1332299362,3cyclictest0-21swapper/020:37:440
1139099360,4rtkit-daemon0-21swapper/420:33:3810
1332299355,3cyclictest11813-21df_abs20:27:480
124852350,2sleep151334499cyclictest23:24:457
280202340,6sleep81333499cyclictest22:27:0814
13344993415,2cyclictest0-21swapper/1520:27:507
1332299344,3cyclictest7510-21kernelversion19:42:530
1332299344,3cyclictest7510-21kernelversion19:42:530
1332299344,3cyclictest4686-21grep20:17:550
1332299341,4cyclictest0-21swapper/019:22:450
317462330,1sleep40-21swapper/422:08:5210
317462330,1sleep40-21swapper/422:08:5210
1332299333,3cyclictest25267-21expr20:42:540
119502330,4sleep131334099cyclictest22:39:475
58612320,28sleep21-21systemd00:25:168
322862320,28sleep10-21swapper/122:46:011
274522320,1sleep120-21swapper/1222:12:134
274522320,1sleep120-21swapper/1222:12:134
262532320,1sleep150-21swapper/1522:33:577
1333299320,23cyclictest0-21swapper/622:56:2912
1332299322,4cyclictest0-21swapper/020:52:450
1332299321,4cyclictest25614-21timerwakeupswit20:03:040
1332299321,3cyclictest0-21swapper/020:22:560
1332299320,4cyclictest13838-21head21:07:540
1332299320,2cyclictest0-21swapper/021:02:350
129002320,1sleep110-21swapper/1122:25:183
1139099320,6rtkit-daemon0-21swapper/1222:20:354
1139099320,6rtkit-daemon0-21swapper/1222:20:354
1333299314,6cyclictest0-21swapper/621:30:0012
1332299314,3cyclictest15407-21grep20:32:470
1332299312,3cyclictest27648-21grep20:47:470
1139099310,0rtkit-daemon0-21swapper/722:29:0713
1139099310,0rtkit-daemon0-21swapper/1220:20:134
32982301,3sleep95464-21packagekitd23:16:2215
307472300,1sleep20-21swapper/223:26:568
229642300,1sleep40-21swapper/422:04:0410
229642300,1sleep40-21swapper/422:04:0410
1333799308,19cyclictest0-21swapper/1000:00:592
1332299305,3cyclictest13149-21unixbench_singl19:48:020
1332299304,3cyclictest1361-21runrttasks19:58:380
1332299303,3cyclictest27071-21cpuspeed_turbos20:07:500
1332299300,3cyclictest2384-21grep19:37:460
1332299300,3cyclictest2384-21grep19:37:460
1139099300,1rtkit-daemon0-21swapper/420:17:3210
70452290,0sleep120-21swapper/1200:40:244
70452290,0sleep120-21swapper/1200:40:244
4562290,1sleep40-21swapper/423:19:4610
32562290,1sleep449-21ksoftirqd/422:20:2210
32562290,1sleep449-21ksoftirqd/422:20:2210
261532290,2sleep70-21swapper/723:29:5913
233632290,2sleep110-21swapper/1100:03:133
217712290,0sleep130-21swapper/1321:34:315
198542290,25sleep150-21swapper/1522:55:377
13343992923,2cyclictest131rcu_preempt21:44:446
1333799290,21cyclictest1069-21NetworkManager23:09:042
1332599290,3cyclictest1-21systemd22:05:518
1332599290,3cyclictest1-21systemd22:05:518
1332299295,3cyclictest5703-21switchtime20:58:040
48692280,1sleep110-21swapper/1122:53:403
311382280,1sleep341-21ksoftirqd/323:45:419
262722280,1sleep150-21swapper/1522:30:277
158942280,1sleep20-21swapper/223:43:488
1333999288,19cyclictest0-21swapper/1221:32:564
1333999280,4cyclictest0-21swapper/1200:17:454
1333799286,12cyclictest0-21swapper/1023:38:052
13337992821,6cyclictest971ktimersoftd/1021:37:452
1333299287,13cyclictest16148-21sshd22:25:4212
1333299286,16cyclictest66-21ksoftirqd/623:39:1212
13332992818,4cyclictest131rcu_preempt21:27:0512
13332992816,7cyclictest66-21ksoftirqd/623:58:5612
13332992810,10cyclictest144-21kswork22:58:5312
1332299281,3cyclictest647-21ntp_states20:12:560
129682280,1sleep30-21swapper/323:24:499
88492272,2sleep51068-21dbus-daemon21:18:2111
58862270,1sleep130-21swapper/1300:09:085
48332270,1sleep110-21swapper/1123:57:523
308952270,1sleep10-21swapper/120:12:471
295542270,2sleep929558-2110-uname00:03:5815
256722270,2sleep425671-21bash21:23:4010
207672270,2sleep140-21swapper/1421:12:536
192982270,1sleep80-21swapper/822:40:4714
1334399270,22cyclictest0-21swapper/1400:15:216
1333799277,10cyclictest0-21swapper/1022:22:292
1333799277,10cyclictest0-21swapper/1022:22:292
1333799274,12cyclictest971ktimersoftd/1021:21:502
13337992711,6cyclictest0-21swapper/1023:07:022
13337992710,13cyclictest1238-21gdbus21:47:242
13333992713,7cyclictest0-21swapper/722:13:3113
13333992713,7cyclictest0-21swapper/722:13:3113
1333299276,12cyclictest66-21ksoftirqd/621:45:5712
13332992721,4cyclictest0-21swapper/600:09:0012
13323992715,7cyclictest131rcu_preempt21:25:441
1332299271,4cyclictest15196-21df_abs19:52:470
91082260,24sleep150-21swapper/1519:43:207
91082260,24sleep150-21swapper/1519:43:207
293682260,2sleep20-21swapper/222:08:358
293682260,2sleep20-21swapper/222:08:358
28312260,1sleep40-21swapper/400:01:1410
254222260,23sleep140-21swapper/1422:48:386
234232260,0sleep90-21swapper/921:13:0315
226362260,6sleep449-21ksoftirqd/423:14:4010
218642260,1sleep80-21swapper/823:40:5814
13343992617,4cyclictest130-21ksoftirqd/1423:52:546
13343992610,6cyclictest130-21ksoftirqd/1400:03:536
1333999261,23cyclictest0-21swapper/1223:52:444
1333799268,10cyclictest98-21ksoftirqd/1023:18:072
13337992610,12cyclictest9402-21gdbus22:55:192
1333299269,9cyclictest66-21ksoftirqd/621:38:1512
1333299267,9cyclictest0-21swapper/600:33:4712
1333299267,9cyclictest0-21swapper/600:33:4712
1333299267,10cyclictest66-21ksoftirqd/623:50:5112
1333299266,11cyclictest66-21ksoftirqd/622:21:0012
1333299266,11cyclictest66-21ksoftirqd/622:21:0012
1333299266,10cyclictest0-21swapper/600:22:4412
1333299265,15cyclictest66-21ksoftirqd/621:53:3212
1333299265,13cyclictest0-21swapper/622:17:3212
1333299265,13cyclictest0-21swapper/622:17:3212
1333299265,12cyclictest0-21swapper/600:41:0512
1333299265,12cyclictest0-21swapper/600:41:0512
1333299265,11cyclictest0-21swapper/622:02:4512
1333299265,11cyclictest0-21swapper/622:02:4512
1333299265,11cyclictest0-21swapper/621:50:1312
1333299264,20cyclictest641rcuc/621:35:3112
1333299264,16cyclictest13824-21systemd-cgroups21:18:5612
1333299264,12cyclictest66-21ksoftirqd/623:53:2312
13332992622,2cyclictest641rcuc/621:17:2912
13332992612,9cyclictest131rcu_preempt23:31:2612
1332299264,2cyclictest1420-21latency_hist19:37:320
1332299264,2cyclictest1420-21latency_hist19:37:320
1332299260,4cyclictest26369-21tr19:27:470
1332299260,4cyclictest26369-21tr19:27:470
129262260,1sleep110-21swapper/1121:51:533
114702260,1sleep70-21swapper/721:33:2213
1139099260,0rtkit-daemon0-21swapper/420:24:2910
1139099260,0rtkit-daemon0-21swapper/1223:22:344
112522260,2sleep811345-21sshd00:29:5514
112522260,2sleep811345-21sshd00:29:5514
92612250,22sleep30-21swapper/319:45:169
92612250,22sleep30-21swapper/319:45:169
87952250,1sleep30-21swapper/322:06:289
87952250,1sleep30-21swapper/322:06:289
8092250,21sleep30-21swapper/300:28:299
8092250,21sleep30-21swapper/300:28:299
80892250,1sleep30-21swapper/322:17:099
80892250,1sleep30-21swapper/322:17:099
321982250,1sleep150-21swapper/1523:19:407
316702250,21sleep10-21swapper/100:24:271
301512250,18sleep90-21swapper/922:23:2515
300152250,2sleep430163-21sshd21:13:4710
171802250,1sleep70-21swapper/722:36:4413
166542250,1sleep150-21swapper/1523:28:477
136382250,1sleep140-21swapper/1400:10:456
1334399258,7cyclictest130-21ksoftirqd/1421:28:086
1333799259,13cyclictest1068-21dbus-daemon21:23:012
1333799257,6cyclictest4209-21sshd22:16:402
1333799257,6cyclictest4209-21sshd22:16:402
1333799256,9cyclictest98-21ksoftirqd/1022:03:512
1333799256,9cyclictest98-21ksoftirqd/1022:03:512
1333799255,11cyclictest1-21systemd23:33:432
1333799255,11cyclictest0-21swapper/1022:33:352
1333799255,10cyclictest0-21swapper/1022:46:002
13337992517,6cyclictest0-21swapper/1021:58:412
13337992517,6cyclictest0-21swapper/1021:58:412
1333799251,5cyclictest0-21swapper/1022:37:472
13337992514,5cyclictest0-21swapper/1000:32:252
13337992514,5cyclictest0-21swapper/1000:32:252
13333992516,4cyclictest141rcu_sched23:55:0813
1333299258,10cyclictest66-21ksoftirqd/623:25:3712
1333299256,9cyclictest19337-21systemd23:10:3212
1333299256,12cyclictest26191-21sshd22:12:0412
1333299256,12cyclictest26191-21sshd22:12:0412
1333299256,11cyclictest0-21swapper/600:19:1212
1333299256,10cyclictest66-21ksoftirqd/623:36:3012
1333299256,10cyclictest66-21ksoftirqd/622:42:4612
1333299256,10cyclictest0-21swapper/622:31:0812
1333299255,9cyclictest0-21swapper/622:37:5312
1333299255,10cyclictest66-21ksoftirqd/623:20:4612
1333299252,12cyclictest66-21ksoftirqd/623:04:0112
1139099250,0rtkit-daemon0-21swapper/419:14:4410
1139099250,0rtkit-daemon0-21swapper/1220:01:264
111632250,1sleep140-21swapper/1422:25:056
96322240,20sleep90-21swapper/923:24:2415
79602240,1sleep80-21swapper/821:14:5614
75452240,1sleep150-21swapper/1522:17:057
75452240,1sleep150-21swapper/1522:17:057
60032240,0sleep150-21swapper/1521:39:547
41642240,1sleep70-21swapper/700:08:4613
249262240,21sleep150-21swapper/1522:18:587
249262240,21sleep150-21swapper/1522:18:587
190722240,1sleep9881rcuc/923:32:5615
168872240,1sleep30-21swapper/319:14:479
160612240,1sleep110-21swapper/1122:10:533
160612240,1sleep110-21swapper/1122:10:533
13343992417,5cyclictest0-21swapper/1423:12:166
1333799247,9cyclictest0-21swapper/1022:24:502
1333799247,9cyclictest0-21swapper/1000:37:302
1333799247,9cyclictest0-21swapper/1000:37:302
1333799247,9cyclictest0-21swapper/1000:02:422
1333799247,14cyclictest1073-21systemd-logind00:13:352
1333799247,10cyclictest98-21ksoftirqd/1000:08:082
1333799245,9cyclictest31612-21kill23:30:412
1333799245,6cyclictest1068-21dbus-daemon23:53:072
1333799244,10cyclictest0-21swapper/1021:56:052
13337992418,2cyclictest0-21swapper/1023:26:082
13337992416,5cyclictest15792-21systemd-cgroups21:48:272
1333399249,7cyclictest74-21ksoftirqd/700:01:4313
1333299246,9cyclictest66-21ksoftirqd/623:46:2512
1333299246,9cyclictest0-21swapper/622:49:2112
1333299246,6cyclictest17698-21sshd00:15:4512
1333299246,10cyclictest66-21ksoftirqd/600:30:1612
1333299246,10cyclictest66-21ksoftirqd/600:30:1612
1333299244,19cyclictest641rcuc/622:34:0112
1333299244,10cyclictest66-21ksoftirqd/621:57:3812
1333299244,10cyclictest66-21ksoftirqd/621:57:3812
13329992419,3cyclictest525-21systemd-journal22:53:0310
1332699241,11cyclictest0-21swapper/321:57:469
1332699241,11cyclictest0-21swapper/321:57:469
1139099240,1rtkit-daemon1894-21sendmail-mta19:27:1510
105932240,2sleep80-21swapper/822:58:0914
65422230,1sleep30-21swapper/321:36:319
59212230,1sleep155917-21sshd00:36:427
59212230,1sleep155917-21sshd00:36:427
48982230,2sleep50-21swapper/521:32:4711
311692230,20sleep50-21swapper/521:46:4211
290582230,20sleep150-21swapper/1519:28:057
290582230,20sleep150-21swapper/1519:28:057
210522230,0sleep50-21swapper/521:23:0911
184322230,19sleep140-21swapper/1422:11:126
184322230,19sleep140-21swapper/1422:11:126
170952230,1sleep20-21swapper/223:28:498
157562230,1sleep90-21swapper/922:10:5115
157562230,1sleep90-21swapper/922:10:5115
136002230,1sleep110-21swapper/1100:15:143
1334399239,5cyclictest131rcu_preempt21:18:286
1334399238,9cyclictest1032-17auditd23:15:176
13343992319,2cyclictest0-21swapper/1421:35:316
13343992313,5cyclictest131rcu_preempt23:20:056
13343992312,5cyclictest130-21ksoftirqd/1423:58:246
1333799237,8cyclictest0-21swapper/1022:48:122
1333799236,8cyclictest98-21ksoftirqd/1023:14:292
1333799234,12cyclictest98-21ksoftirqd/1023:48:362
1333799234,10cyclictest0-21swapper/1021:34:282
1333799233,14cyclictest29119-21sshd21:13:412
13337992316,4cyclictest0-21swapper/1021:28:092
1333799231,13cyclictest98-21ksoftirqd/1000:42:142
1333799231,13cyclictest98-21ksoftirqd/1000:42:142
1333799230,20cyclictest16874-2110-uname00:19:012
1333799230,19cyclictest16021-21sshd00:22:392
1333599231,21cyclictest1351-21nfsd23:05:5715
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional