You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 08:58
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackcslot0.osadl.org (updated Sat Nov 29, 2025 00:43:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2138221410,7sleep03067899cyclictest23:52:340
301822118102,11sleep100-21swapper/1019:12:132
576421130,1sleep150-21swapper/1521:49:157
794221080,1sleep10-21swapper/100:20:241
610021080,1sleep70-21swapper/721:27:1113
3236821080,0sleep120-21swapper/1223:12:234
30160210592,9sleep90-21swapper/919:11:5615
29963210492,8sleep120-21swapper/1219:09:514
29894210482,17sleep10-21swapper/119:09:081
1104321040,2sleep311041-21cp22:58:049
687821030,2sleep15138-21ksoftirqd/1521:53:167
2853921020,0sleep100-21swapper/1023:38:022
2193721020,7sleep53068599cyclictest21:43:2811
2996529975,19sleep140-21swapper/1419:09:536
256762980,1sleep20-21swapper/221:37:008
183762980,1sleep120-21swapper/1221:39:344
16522970,1sleep100-21swapper/1000:09:032
127212950,1sleep130-21swapper/1322:32:015
127212950,1sleep130-21swapper/1322:32:015
81842940,1sleep120-21swapper/1222:12:284
299382940,1sleep8801rcuc/821:59:3414
2996229381,8sleep110-21swapper/1119:09:503
268672930,1sleep666-21ksoftirqd/622:10:4612
54992920,0sleep30-21swapper/322:00:339
242952920,6sleep113069499cyclictest23:18:473
180312920,1sleep100-21swapper/1021:43:012
126442920,0sleep00-21swapper/023:06:180
131502910,1sleep100-21swapper/1023:32:412
45612900,2sleep04560-21sshd22:19:140
3019328981,4sleep30-21swapper/319:12:229
3015928972,4sleep80-21swapper/819:11:5514
88532880,0sleep00-21swapper/023:32:150
2989328772,11sleep00-21swapper/019:09:070
83992860,2sleep51068-21dbus-daemon21:49:3711
54462860,1sleep90-21swapper/922:31:0915
54462860,1sleep90-21swapper/922:31:0915
49632860,1sleep121121rcuc/1200:02:084
323002860,1sleep50-21swapper/523:00:4611
131562850,1sleep1113159-2110-uname21:35:333
3009928475,4sleep150-21swapper/1519:11:467
3008628472,8sleep20-21swapper/219:11:338
285612840,1sleep100-21swapper/1000:27:472
12962840,0sleep40-21swapper/423:20:0010
3006828375,4sleep40-21swapper/419:11:1910
3001428375,4sleep50-21swapper/519:10:3211
2999728375,4sleep70-21swapper/719:10:1813
2996428372,7sleep130-21swapper/1319:09:525
2993728375,4sleep60-21swapper/619:09:2912
95872820,0sleep80-21swapper/822:08:4014
62492810,1sleep40-21swapper/422:04:1910
81252780,1sleep110-21swapper/1123:32:103
52752760,19sleep396150irq/39-eth0-TxR21:41:509
51042750,6sleep93069199cyclictest23:35:2015
25002680,5sleep113069499cyclictest22:00:093
29652630,1sleep80-21swapper/823:42:4914
155202620,6sleep73068999cyclictest23:29:0913
1139099510,3rtkit-daemon11389-21rtkit-daemon00:13:5711
1139099470,0rtkit-daemon0-21swapper/722:19:0113
75632460,1sleep137553-21sshd22:42:435
1139099460,0rtkit-daemon0-21swapper/1520:03:437
1139099460,0rtkit-daemon0-21swapper/1520:03:437
315652450,1sleep8801rcuc/800:28:0114
30685994512,26cyclictest1069-21NetworkManager00:36:5111
3068599447,24cyclictest0-21swapper/523:45:3211
30685994416,17cyclictest1068-21dbus-daemon21:56:5311
30685994414,22cyclictest5467-21gdbus22:02:5511
30685994411,25cyclictest19537-21systemd21:28:3311
30685994410,27cyclictest1238-21gdbus23:32:4111
1139099440,22rtkit-daemon0-21swapper/723:26:1713
55592430,1sleep40-21swapper/423:39:1610
30685994313,21cyclictest10236-21sshd21:13:1211
30685994311,24cyclictest6672-21sshd21:38:1311
275662430,26sleep796450irq/42-eth0-TxR21:15:0913
145592430,0sleep20-21swapper/223:36:338
1139099430,1rtkit-daemon7421-21sshd22:12:237
3068599428,24cyclictest0-21swapper/522:33:0311
3068599428,24cyclictest0-21swapper/522:33:0311
3068599427,33cyclictest0-21swapper/523:12:4811
30685994215,18cyclictest10115-21systemd-cgroups22:54:1011
30685994214,21cyclictest1068-21dbus-daemon23:04:1811
30685994212,23cyclictest0-21swapper/522:10:3411
3068599419,24cyclictest0-21swapper/523:09:4711
3068599419,20cyclictest28553-21systemd-cgroups00:23:5411
3068599415,28cyclictest1073-21systemd-logind21:22:3511
30685994116,17cyclictest1068-21dbus-daemon23:32:2211
30685994112,22cyclictest1068-21dbus-daemon23:40:4511
30685994110,24cyclictest1233-21gdbus21:57:4711
228972410,1sleep20-21swapper/223:48:528
102052410,1sleep20-21swapper/221:13:128
3068599405,28cyclictest0-21swapper/500:30:2811
3068599405,27cyclictest7243-21sshd22:46:3311
30685994010,20cyclictest1233-21gdbus22:41:1711
30685994010,20cyclictest1233-21gdbus22:41:1711
198362400,1sleep00-21swapper/022:47:510
142792400,2sleep100-21swapper/1023:59:102
72812390,2sleep23068199cyclictest21:42:058
3068599398,24cyclictest0-21swapper/500:08:2511
3068599398,22cyclictest1233-21gdbus23:50:5911
3068599398,21cyclictest0-21swapper/522:21:3511
3068599396,26cyclictest0-21swapper/522:13:4611
3068599395,32cyclictest0-21swapper/523:26:5011
3068599394,26cyclictest0-21swapper/523:19:2911
3068599393,30cyclictest0-21swapper/523:56:4411
30685993913,19cyclictest1-21systemd22:28:4011
30685993913,19cyclictest1-21systemd22:28:4011
30685993910,22cyclictest1-21systemd21:18:4211
30685993910,21cyclictest10985-21sshd00:06:4411
273242390,4sleep233-21ksoftirqd/223:19:108
3069199389,23cyclictest0-21swapper/922:46:1915
3069199388,27cyclictest0-21swapper/923:58:2215
3068599389,27cyclictest0-21swapper/500:19:2611
30685993821,15cyclictest0-21swapper/521:34:1011
1139099380,3rtkit-daemon2748-21ngircd22:29:3113
1139099380,3rtkit-daemon2748-21ngircd22:29:3113
1139099380,0rtkit-daemon0-21swapper/1523:29:357
3068599377,25cyclictest0-21swapper/522:47:4711
3068599371,29cyclictest0-21swapper/500:38:2511
30685993711,19cyclictest0-21swapper/500:01:5811
3069199368,26cyclictest0-21swapper/900:15:3615
3069199366,22cyclictest5874-21kill00:20:0115
30691993615,18cyclictest3507-21sshd21:41:3815
3069199360,34cyclictest0-21swapper/923:26:5015
30685993614,19cyclictest30962-21systemd-cgroups22:26:2711
30685993614,19cyclictest30962-21systemd-cgroups22:26:2711
323622350,2sleep0111ktimersoftd/023:46:250
3069199356,26cyclictest21599-21sshd21:17:5315
3069199356,20cyclictest0-21swapper/922:09:3715
30691993512,21cyclictest96050irq/38-eth0-TxR23:12:4815
30685993517,16cyclictest0-21swapper/520:27:5411
30685993515,17cyclictest0-21swapper/520:42:5211
30685993511,22cyclictest0-21swapper/520:52:4511
3069199346,25cyclictest0-21swapper/923:29:4415
3069199345,26cyclictest32672-21sshd22:18:3915
3069199343,29cyclictest0-21swapper/922:23:5815
3069199343,29cyclictest0-21swapper/922:23:5815
3069199341,25cyclictest0-21swapper/921:35:4815
30685993416,16cyclictest14410-21awk19:32:3011
30685993415,18cyclictest0-21swapper/519:12:4311
30685993414,18cyclictest3494-21cstates19:17:4211
30685993412,20cyclictest0-21swapper/521:04:5811
118362340,2sleep14130-21ksoftirqd/1400:15:006
3069199338,22cyclictest10475-21cups-browsed21:25:4315
3069199336,22cyclictest0-21swapper/921:56:3315
3069199336,19cyclictest8607-21kill21:31:2915
3069199335,21cyclictest0-21swapper/921:48:1315
3069199334,26cyclictest4289-21sshd21:12:4315
3069199333,28cyclictest0-21swapper/923:11:5315
3069199333,28cyclictest0-21swapper/923:00:3015
30691993327,3cyclictest18399-21env23:40:5215
3069199332,29cyclictest0-21swapper/923:57:0915
3069199332,29cyclictest0-21swapper/923:49:4415
30691993314,17cyclictest0-21swapper/921:47:1115
3069199331,29cyclictest1-21systemd00:05:1215
30691993310,20cyclictest12380-21sshd00:33:1015
3068599339,22cyclictest0-21swapper/520:07:5611
3068599339,22cyclictest0-21swapper/520:07:5611
30685993315,16cyclictest0-21swapper/519:52:5311
30685993315,16cyclictest0-21swapper/519:52:5311
30685993315,15cyclictest0-21swapper/520:22:5611
30685993314,17cyclictest0-21swapper/520:01:3711
30685993314,17cyclictest0-21swapper/520:01:3711
30685993313,18cyclictest26524-21chrt19:44:4411
30685993312,19cyclictest0-21swapper/520:34:0911
30685993312,19cyclictest0-21swapper/520:17:4511
30685993312,19cyclictest0-21swapper/520:13:2411
160632330,2sleep1116043-21sshd21:46:473
126182330,2sleep140-21swapper/1400:33:136
3069199329,20cyclictest1098-21in:imuxsock22:02:4815
3069199329,20cyclictest0-21swapper/900:27:5615
3069199327,23cyclictest0-21swapper/922:13:1215
3069199326,24cyclictest26549-21systemd23:03:4815
3069199326,20cyclictest16675-21kill22:47:4115
3069199323,23cyclictest0-21swapper/922:40:0415
3069199323,23cyclictest0-21swapper/922:40:0415
30691993217,13cyclictest19919-21run-parts00:37:5315
30691993214,15cyclictest15376-21systemd-cgroups21:57:5215
3069199321,28cyclictest0-21swapper/922:56:2315
30691993212,17cyclictest0-21swapper/900:22:5415
30685993215,15cyclictest7121-21grep19:22:4311
30685993213,17cyclictest0-21swapper/520:37:3311
30685993213,17cyclictest0-21swapper/519:38:0211
3068599321,29cyclictest0-21swapper/519:47:4211
30685993212,19cyclictest0-21swapper/520:02:3311
30685993212,19cyclictest0-21swapper/520:02:3311
30685993210,20cyclictest0-21swapper/520:58:0211
30685993210,20cyclictest0-21swapper/520:47:4211
30685993210,20cyclictest0-21swapper/519:32:4811
262892320,0sleep30-21swapper/321:43:589
137932320,0sleep70-21swapper/722:47:2613
8402310,27sleep20-21swapper/222:18:448
3069199318,19cyclictest5251systemd-journal23:21:4815
3069199317,21cyclictest1-21systemd22:32:3315
3069199317,21cyclictest1-21systemd22:32:3315
3069199316,23cyclictest0-21swapper/920:42:5515
3069199310,28cyclictest0-21swapper/900:07:5315
30685993113,16cyclictest0-21swapper/521:07:4211
269442311,26sleep71-21systemd23:42:0013
1139099310,0rtkit-daemon0-21swapper/721:27:5513
9162300,26sleep70-21swapper/723:46:3313
80452300,1sleep70-21swapper/723:58:1913
43032300,1sleep60-21swapper/600:28:3112
3069199308,17cyclictest4484-21systemd23:42:5415
289062300,1sleep20-21swapper/223:11:588
261742300,22sleep105464-21packagekitd21:25:382
230032300,27sleep141073-21systemd-logind23:30:056
209382300,25sleep110-21swapper/1122:06:193
171762300,0sleep110-21swapper/1121:50:473
153112300,2sleep30-21swapper/300:03:049
80372290,25sleep10-21swapper/121:57:201
73832290,26sleep100-21swapper/1020:38:032
319382290,25sleep60-21swapper/623:04:3312
3069199299,18cyclictest0-21swapper/919:47:4915
3069199298,19cyclictest0-21swapper/921:07:5615
3069199298,19cyclictest0-21swapper/920:47:4915
3069199296,21cyclictest0-21swapper/921:02:4815
3069199295,22cyclictest0-21swapper/920:32:5615
30691992911,16cyclictest0-21swapper/920:04:3015
30691992911,16cyclictest0-21swapper/920:04:3015
288932290,2sleep150-21swapper/1500:04:537
186162290,25sleep150-21swapper/1523:18:047
181312290,25sleep00-21swapper/021:54:380
152122290,25sleep20-21swapper/200:03:038
148792290,1sleep41068-21dbus-daemon22:35:5310
148792290,1sleep41068-21dbus-daemon22:35:5310
1139099290,3rtkit-daemon1068-21dbus-daemon21:58:4513
62822280,2sleep80-21swapper/821:38:1114
54142280,2sleep145415-2110-uname22:49:576
49922280,24sleep00-21swapper/000:05:580
3069199289,17cyclictest26239-21ntpq20:22:5415
3069199289,17cyclictest0-21swapper/919:38:0215
3069199287,19cyclictest0-21swapper/919:22:0615
3069199285,21cyclictest0-21swapper/919:32:5115
3069199283,23cyclictest0-21swapper/919:52:5515
3069199283,23cyclictest0-21swapper/919:52:5515
3069199281,25cyclictest0-21swapper/920:07:4815
3069199281,25cyclictest0-21swapper/920:07:4815
30691992810,16cyclictest0-21swapper/919:23:4915
277942280,23sleep00-21swapper/000:08:220
234512280,24sleep00-21swapper/022:06:400
221292280,3sleep133069699cyclictest21:17:565
180242280,24sleep20-21swapper/222:13:248
1139099280,0rtkit-daemon0-21swapper/723:01:5813
1139099280,0rtkit-daemon0-21swapper/720:33:5713
75852270,25sleep70-21swapper/720:40:2413
71942270,23sleep40-21swapper/422:19:3510
47802270,23sleep00-21swapper/023:08:590
320292270,2sleep60-21swapper/623:38:3112
3069199279,16cyclictest0-21swapper/920:58:4715
3069199279,16cyclictest0-21swapper/920:17:5415
3069199279,16cyclictest0-21swapper/919:12:5815
3069199278,17cyclictest0-21swapper/920:12:4415
3069199277,18cyclictest0-21swapper/920:52:4715
3069199277,18cyclictest0-21swapper/919:57:5215
3069199277,18cyclictest0-21swapper/919:57:5215
3069199276,16cyclictest13563-21ls19:27:5715
3069199273,18cyclictest0-21swapper/919:42:5115
30352270,23sleep10-21swapper/123:20:151
283432270,2sleep20-21swapper/200:18:028
253162270,23sleep100-21swapper/1023:45:322
251622270,23sleep30-21swapper/322:25:439
251622270,23sleep30-21swapper/322:25:439
162012270,23sleep150-21swapper/1523:52:027
147362270,0sleep150-21swapper/1522:20:337
14592270,1sleep00-21swapper/000:01:400
138562270,1sleep1013860-21run-parts22:50:572
1139099270,1rtkit-daemon10591-21sshd21:57:327
1139099270,0rtkit-daemon0-21swapper/722:05:2513
107732270,1sleep120-21swapper/1223:47:424
75902260,1sleep140-21swapper/1420:02:336
75902260,1sleep140-21swapper/1420:02:336
5532261,2sleep70-21swapper/700:24:3213
48142260,23sleep00-21swapper/021:41:480
44942260,1sleep130-21swapper/1321:34:325
317042260,4sleep031799-21sshd23:00:420
3069199267,17cyclictest0-21swapper/920:37:3315
3069199267,17cyclictest0-21swapper/920:27:3515
171412260,2sleep110-21swapper/1123:40:433
145012260,1sleep10-21swapper/121:39:081
137802260,3sleep101-21systemd22:09:152
64642250,1sleep40-21swapper/400:25:1710
62002250,1sleep00-21swapper/020:37:530
323392250,0sleep40-21swapper/422:57:0010
317862250,2sleep70-21swapper/723:53:3013
30688992514,6cyclictest66-21ksoftirqd/622:37:3412
30688992514,6cyclictest66-21ksoftirqd/622:37:3412
30688992512,7cyclictest66-21ksoftirqd/623:43:1512
258152250,0sleep110-21swapper/1121:14:573
249212250,2sleep1424920-21sshd21:55:236
208042250,1sleep150-21swapper/1521:36:257
172712250,1sleep130-21swapper/1300:37:435
1139099250,3rtkit-daemon19845-21systemd21:43:1513
1139099250,1rtkit-daemon0-21swapper/723:54:177
108902250,21sleep30-21swapper/300:33:029
104462250,0sleep110-21swapper/1100:06:403
49722240,21sleep30-21swapper/322:53:329
36092240,1sleep60-21swapper/622:15:3512
325452240,2sleep30-21swapper/323:31:149
30696992418,3cyclictest5464-21packagekitd22:20:225
3069599240,20cyclictest3931-21sshd00:13:194
3069099241,21cyclictest0-21swapper/823:37:4314
30688992411,7cyclictest131rcu_preempt23:16:1112
30688992411,6cyclictest66-21ksoftirqd/621:16:2812
272632240,2sleep135467-21gdbus23:07:515
234162240,1sleep10-21swapper/123:30:071
232442240,0sleep130-21swapper/1300:34:315
22612240,1sleep80-21swapper/819:53:0614
22612240,1sleep80-21swapper/819:53:0614
209342240,2sleep140-21swapper/1421:39:546
199792240,2sleep70-21swapper/700:37:5413
17352240,22sleep60-21swapper/619:13:2912
149252240,1sleep131211ktimersoftd/1322:13:015
1139099240,1rtkit-daemon0-21swapper/723:34:397
1139099240,1rtkit-daemon0-21swapper/722:24:087
1139099240,1rtkit-daemon0-21swapper/722:24:087
113552240,1sleep150-21swapper/1523:02:177
102682240,1sleep80-21swapper/800:10:0614
78592230,0sleep70-21swapper/723:09:2213
314842230,1sleep13122-21ksoftirqd/1322:56:535
30699992314,6cyclictest9760-21sshd23:13:136
3069999230,20cyclictest18612-21systemd22:28:426
3069999230,20cyclictest18612-21systemd22:28:426
30695992318,3cyclictest1068-21dbus-daemon21:47:414
3069599231,20cyclictest0-21swapper/1200:27:434
3069599231,19cyclictest10475-21cups-browsed21:36:084
3068899238,9cyclictest131rcu_preempt22:23:0112
3068899238,9cyclictest131rcu_preempt22:23:0112
3068899233,18cyclictest651ktimersoftd/622:54:5112
30688992316,4cyclictest131rcu_preempt22:34:5412
30688992316,4cyclictest131rcu_preempt22:34:5412
30688992312,7cyclictest141rcu_sched22:50:2512
30688992311,7cyclictest131rcu_preempt22:44:3612
30688992310,7cyclictest141rcu_sched22:30:0812
30688992310,7cyclictest141rcu_sched22:30:0812
289382230,1sleep10971ktimersoftd/1022:22:202
278782230,0sleep150-21swapper/1523:27:097
264492230,1sleep10-21swapper/121:29:301
243762230,1sleep00-21swapper/021:01:090
170502230,19sleep60-21swapper/621:32:3212
141682230,0sleep100-21swapper/1022:58:272
128752230,1sleep120-21swapper/1220:47:434
1139099230,3rtkit-daemon1078-21ptp4l00:35:5810
1139099230,1rtkit-daemon1068-21dbus-daemon23:06:277
1139099230,1rtkit-daemon0-21swapper/723:33:5213
1139099230,1rtkit-daemon0-21swapper/700:11:447
104432230,1sleep141291ktimersoftd/1422:38:596
104432230,1sleep141291ktimersoftd/1422:38:596
63962220,2sleep150-21swapper/1519:58:347
63962220,2sleep150-21swapper/1519:58:347
315492220,20sleep40-21swapper/420:32:0410
30699992217,3cyclictest5464-21packagekitd23:33:566
3069999220,20cyclictest0-21swapper/1422:33:186
3069999220,20cyclictest0-21swapper/1422:33:186
30695992217,3cyclictest27909-21sshd23:34:134
3069599221,18cyclictest5467-21gdbus23:40:284
3069599221,18cyclictest0-21swapper/1223:12:374
3069599220,19cyclictest8648-21sshd22:31:324
3069599220,19cyclictest8648-21sshd22:31:324
3069599220,19cyclictest5467-21gdbus23:47:304
3069599220,19cyclictest0-21swapper/1221:23:044
3069499220,20cyclictest3966-21kworker/11:423:35:273
3068999221,18cyclictest0-21swapper/700:18:0213
3068899229,7cyclictest131rcu_preempt21:18:3812
3068899228,7cyclictest131rcu_preempt00:04:1412
3068899227,7cyclictest66-21ksoftirqd/623:17:4712
3068899227,7cyclictest131rcu_preempt23:59:3512
3068899224,15cyclictest0-21swapper/600:37:4412
30688992216,4cyclictest13364-21systemd21:24:0812
3068899221,4cyclictest66-21ksoftirqd/621:37:4712
3068899221,3cyclictest1-21systemd21:51:2612
30688992211,6cyclictest131rcu_preempt22:06:5012
30688992210,7cyclictest66-21ksoftirqd/622:00:5812
30688992210,7cyclictest66-21ksoftirqd/600:20:3812
30688992210,6cyclictest66-21ksoftirqd/623:28:2312
30688992210,6cyclictest66-21ksoftirqd/622:18:4112
30688992210,6cyclictest131rcu_preempt23:11:0512
30688992210,6cyclictest131rcu_preempt00:12:5412
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional