You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-21 - 22:09
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackcslot0.osadl.org (updated Thu Aug 21, 2025 00:43:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3059321250,3sleep131509899cyclictest00:03:455
135622124104,13sleep120-21swapper/1219:06:374
14441212392,26sleep130-21swapper/1319:09:355
2040921120,1sleep150-21swapper/1521:44:267
14347211075,30sleep140-21swapper/1419:08:166
521121090,6sleep121509699cyclictest21:42:304
2769421080,1sleep90-21swapper/922:46:2315
2943921070,1sleep40-21swapper/423:29:0510
1148321070,8sleep11508099cyclictest21:28:411
1564621040,1sleep20-21swapper/221:25:138
1947721030,1sleep70-21swapper/721:44:1813
14248210392,7sleep60-21swapper/619:06:4712
82202990,5sleep151510199cyclictest00:05:077
197612980,0sleep30-21swapper/323:51:009
1424629877,16sleep50-21swapper/519:06:4611
130582980,17sleep90-21swapper/900:15:1115
301842970,1sleep40-21swapper/423:32:5010
301842970,1sleep40-21swapper/423:32:5010
189082970,1sleep30-21swapper/322:14:439
325172950,1sleep50-21swapper/522:42:5011
209702950,1sleep140-21swapper/1421:25:576
171022940,5sleep51508699cyclictest23:58:0011
274762920,2sleep127481-21run-parts23:01:291
1447229172,4sleep80-21swapper/819:10:0214
254772900,0sleep100-21swapper/1023:59:092
1437129072,13sleep00-21swapper/019:08:340
54452880,1sleep00-21swapper/023:52:500
317192880,1sleep120-21swapper/1223:59:594
1437228872,11sleep10-21swapper/119:08:351
168532870,1sleep140-21swapper/1420:31:246
1424428772,4sleep30-21swapper/319:06:449
84942860,1sleep90-21swapper/923:26:3015
81862860,1sleep30-21swapper/319:41:299
1435928572,8sleep90-21swapper/919:08:2715
1431928577,4sleep40-21swapper/419:07:5010
198452840,1sleep60-21swapper/621:33:1912
154162840,1sleep140-21swapper/1422:56:216
1436028472,8sleep100-21swapper/1019:08:282
1424928468,11sleep70-21swapper/719:06:4813
1458828375,4sleep150-21swapper/1519:10:577
1438228375,4sleep110-21swapper/1119:08:453
122422830,1sleep121131ktimersoftd/1223:46:204
177092810,1sleep917706-21sshd21:11:1015
177092810,1sleep917706-21sshd21:11:1015
1431728173,4sleep20-21swapper/219:07:488
106362810,1sleep80-21swapper/823:34:3014
106362810,1sleep80-21swapper/823:34:3014
145022800,0sleep120-21swapper/1222:25:344
271992720,5sleep31508399cyclictest21:19:059
1139099560,1rtkit-daemon0-21swapper/422:36:514
139582550,0sleep130-21swapper/1323:03:505
150052530,0sleep70-21swapper/722:17:4413
208512500,1sleep150-21swapper/1500:16:057
101992490,2sleep610205-2110-uname23:30:4412
15100994816,30cyclictest12739-21bash22:29:176
15100994715,16cyclictest24560-21sshd23:05:186
1139099470,1rtkit-daemon1-21systemd23:45:214
1139099470,1rtkit-daemon1-21systemd23:45:214
15089994613,30cyclictest0-21swapper/722:35:1413
15089994413,28cyclictest16371-21systemd21:55:3513
15100994316,24cyclictest1068-21dbus-daemon22:36:586
1508999431,39cyclictest0-21swapper/721:59:4613
15089994310,4cyclictest2488-21env21:12:4613
15089994310,4cyclictest2488-21env21:12:4613
1508999430,9cyclictest16534-21sshd00:06:1113
1139099430,20rtkit-daemon8933-21sshd22:47:564
1139099430,0rtkit-daemon0-21swapper/400:16:034
1510099422,21cyclictest0-21swapper/1423:56:226
15100994214,25cyclictest0-21swapper/1421:34:306
15100994213,22cyclictest2465-21systemd-cgroups21:27:276
15100994212,28cyclictest2776-21/usr/sbin/munin22:31:456
15100994211,29cyclictest32326-21systemctl23:29:276
1508999421,3cyclictest4857-21chrt21:46:2613
1139099420,1rtkit-daemon1073-21systemd-logind22:06:504
1510099419,29cyclictest0-21swapper/1421:45:426
1510099418,30cyclictest16761-21systemd-cgroups23:50:406
15100994125,14cyclictest1069-21NetworkManager21:50:016
15100994123,16cyclictest12444-21systemd23:42:066
15100994123,16cyclictest12444-21systemd23:42:066
15100994113,25cyclictest0-21swapper/1421:37:466
15100994111,27cyclictest15708-21uname00:06:086
15100994111,24cyclictest12583-21sshd00:09:346
15100994111,24cyclictest0-21swapper/1400:30:416
15100994111,18cyclictest1234-21polkitd21:56:336
1508999411,38cyclictest20940-21cp21:25:5713
15089994110,4cyclictest21313-21sh23:31:4613
15089994110,4cyclictest21313-21sh23:31:4613
15089994110,3cyclictest30229-21sshd22:08:2913
1508999410,9cyclictest23748-21sshd22:22:3013
1508999410,39cyclictest0-21swapper/721:36:4513
1510099407,22cyclictest0-21swapper/1421:14:106
1510099407,22cyclictest0-21swapper/1421:14:106
1510099404,33cyclictest0-21swapper/1422:01:406
15100994012,22cyclictest0-21swapper/1422:14:376
15100994011,23cyclictest525-21systemd-journal23:37:476
15100994011,23cyclictest525-21systemd-journal23:37:476
15100994011,23cyclictest0-21swapper/1423:09:526
15100994011,22cyclictest20276-21sshd22:22:036
15100994010,23cyclictest0-21swapper/1400:36:486
1508999406,3cyclictest624-21gdbus21:33:5213
15089994014,23cyclictest525-21systemd-journal22:44:5613
15089994012,25cyclictest13326-21sshd23:53:4813
15089994011,3cyclictest3362-21sshd00:25:2913
15100993914,16cyclictest32729-21kill22:54:376
15100993913,24cyclictest0-21swapper/1421:16:166
15100993912,23cyclictest0-21swapper/1419:51:336
15100993911,21cyclictest0-21swapper/1422:06:126
15100993911,19cyclictest3192-21systemd-cgroups23:17:596
1508999396,4cyclictest2822-21run-parts21:20:0513
1508999390,13cyclictest6089-21systemd22:05:4213
312772380,0sleep80-21swapper/821:23:1114
1510099388,28cyclictest32389-21systemd-cgroups22:16:166
15100993820,16cyclictest4856-21systemd22:47:236
15100993816,20cyclictest16375-21sshd00:15:336
15100993816,20cyclictest0-21swapper/1423:55:076
15100993815,21cyclictest2035-21sshd23:33:216
15100993815,21cyclictest2035-21sshd23:33:216
15100993815,20cyclictest8375-21systemd-cgroups22:43:576
15100993812,24cyclictest0-21swapper/1423:12:396
15100993812,23cyclictest1068-21dbus-daemon00:22:166
15100993811,25cyclictest0-21swapper/1400:31:356
15100993810,25cyclictest1068-21dbus-daemon23:22:346
1508999388,3cyclictest31539-21sshd23:17:2913
15089993813,22cyclictest0-21swapper/720:46:2113
15089993811,25cyclictest20796-21sshd22:56:4413
15089993810,3cyclictest6717-21switchtime23:56:4013
15100993714,20cyclictest25127-21sshd00:20:136
1510099371,34cyclictest0-21swapper/1421:54:526
15100993711,24cyclictest0-21swapper/1420:11:256
1508999378,3cyclictest1069-21NetworkManager22:46:1313
1508999378,2cyclictest0-21swapper/723:46:4913
1508999377,3cyclictest10583-21sshd22:40:2713
1508999375,3cyclictest0-21swapper/723:11:2113
1508999371,9cyclictest32713-21systemd23:02:0213
15089993711,2cyclictest0-21swapper/700:11:2113
1508999370,10cyclictest400-21sshd22:27:4013
1508999370,10cyclictest21990-21sshd23:39:3113
1508999370,10cyclictest21990-21sshd23:39:3113
15100993613,21cyclictest0-21swapper/1420:06:366
15100993611,22cyclictest0-21swapper/1421:01:106
15100993611,22cyclictest0-21swapper/1421:01:106
1508999369,4cyclictest2619-21sshd21:27:2913
1508999368,3cyclictest14215-21run-parts23:27:0513
1508999367,4cyclictest21409-21cp00:27:2813
1508999367,3cyclictest27254-2110-uname00:32:0313
1508999366,3cyclictest6817-21needreboot20:56:3113
1508999366,3cyclictest6817-21needreboot20:56:3113
15089993610,1cyclictest0-21swapper/700:16:1213
1508999360,11cyclictest7624-21sshd00:05:0113
1508999360,10cyclictest15703-21sshd22:14:2113
15100993515,17cyclictest32108-21users20:46:406
15100993513,20cyclictest0-21swapper/1419:36:226
15100993513,19cyclictest20935-21awk20:36:226
15100993512,21cyclictest25933-21df_abs20:41:246
1508999357,3cyclictest4112-21systemd23:41:2113
1508999357,3cyclictest4112-21systemd23:41:2113
1508999355,6cyclictest0-21swapper/723:09:0113
1508999352,4cyclictest0-21swapper/719:51:2313
15100993416,17cyclictest0-21swapper/1419:21:276
15100993415,18cyclictest0-21swapper/1420:57:076
15100993415,18cyclictest0-21swapper/1420:57:076
15100993415,17cyclictest0-21swapper/1420:24:046
15100993415,16cyclictest0-21swapper/1420:51:356
15100993414,18cyclictest0-21swapper/1420:01:286
15100993414,18cyclictest0-21swapper/1419:31:516
15100993413,19cyclictest0-21swapper/1419:44:236
15100993413,18cyclictest0-21swapper/1421:06:276
15100993413,18cyclictest0-21swapper/1421:06:276
15100993412,20cyclictest0-21swapper/1420:16:266
15100993411,17cyclictest0-21swapper/1419:46:356
1508999349,3cyclictest9938-21sh22:55:4513
1508999347,4cyclictest1233-21gdbus23:22:1513
15089993410,1cyclictest0-21swapper/700:37:3213
292142330,1sleep129220-2110-uname21:49:321
1510099339,22cyclictest0-21swapper/1419:21:096
1510099333,24cyclictest0-21swapper/1419:26:286
15100993316,15cyclictest0-21swapper/1420:00:146
15100993315,16cyclictest0-21swapper/1420:26:296
15100993313,18cyclictest0-21swapper/1419:11:226
76242320,2sleep00-21swapper/000:14:130
1508999325,3cyclictest8856-21cpuspeed_turbos20:21:1913
261192310,2sleep010-21ksoftirqd/022:30:590
176912310,1sleep20-21swapper/223:46:438
154412310,27sleep50-21swapper/523:27:1511
1508999310,10cyclictest19873-21sshd20:33:3013
1139099310,23rtkit-daemon11389-21rtkit-daemon19:52:3010
74162300,1sleep60-21swapper/622:55:2512
313202300,1sleep60-21swapper/622:35:2312
166122300,1sleep20-21swapper/222:21:398
1508999301,4cyclictest0-21swapper/720:26:2213
124812300,3sleep612794-21uname23:11:2912
1139099300,2rtkit-daemon8793-21munin-node00:30:514
108762300,0sleep00-21swapper/023:11:220
30102290,1sleep90-21swapper/921:53:5715
236982290,2sleep51073-21systemd-logind21:48:4911
136202290,1sleep50-21swapper/522:10:2911
63972280,0sleep100-21swapper/1022:32:132
308682280,1sleep20-21swapper/221:45:498
267102280,1sleep00-21swapper/023:09:300
24922280,1sleep20-21swapper/200:41:018
214082284,20sleep91238-21gdbus00:31:3115
206072280,1sleep110-21swapper/1122:49:223
188462280,1sleep20-21swapper/223:54:298
150742280,2sleep015078-21systemd-cgroups21:14:130
150742280,2sleep015078-21systemd-cgroups21:14:130
124842280,2sleep50-21swapper/500:15:0811
1139099280,3rtkit-daemon11760-21systemd22:13:5110
75382270,2sleep100-21swapper/1000:37:252
23182270,22sleep80-21swapper/823:25:5814
195622270,1sleep150-21swapper/1522:30:107
1139099270,3rtkit-daemon13062-21ngircd22:23:5610
63252260,0sleep100-21swapper/1022:51:292
40972260,1sleep10-21swapper/123:41:221
40972260,1sleep10-21swapper/123:41:221
305202260,1sleep90-21swapper/923:51:5515
269122260,2sleep30-21swapper/322:08:039
232232260,1sleep50-21swapper/523:43:3411
232232260,1sleep50-21swapper/523:43:3411
219612260,23sleep80-21swapper/800:06:3714
184832260,3sleep018601-21sshd22:14:410
1508999265,3cyclictest20362-21cpuspeed_turbos19:56:2213
1508999260,9cyclictest1282-21snmpd19:36:3913
1508499264,12cyclictest49-21ksoftirqd/421:21:4310
15084992622,2cyclictest471rcuc/422:37:2710
15084992620,4cyclictest471rcuc/400:40:4810
15084992618,6cyclictest49-21ksoftirqd/400:28:1910
147652260,22sleep50-21swapper/523:54:0011
146652260,1sleep10-21swapper/121:32:411
13162260,0sleep10-21swapper/100:29:131
1139099260,0rtkit-daemon0-21swapper/420:24:2910
66032250,1sleep10-21swapper/100:13:581
55952250,0sleep80-21swapper/822:51:2714
54442250,7sleep111509499cyclictest23:37:303
54442250,7sleep111509499cyclictest23:37:303
215562250,1sleep90-21swapper/923:08:5115
213912250,2sleep120-21swapper/1223:24:094
20112250,1sleep10-21swapper/100:32:591
159752250,0sleep30-21swapper/322:29:429
156752250,1sleep120-21swapper/1222:17:504
1509899250,23cyclictest8991-21kworker/13:021:17:175
15094992520,3cyclictest5464-21packagekitd23:31:183
15094992520,3cyclictest5464-21packagekitd23:31:183
1508499259,9cyclictest31904-21systemd-cgroups22:54:3010
1508499257,9cyclictest49-21ksoftirqd/421:29:2010
1508499256,14cyclictest49-21ksoftirqd/421:11:3510
1508499256,14cyclictest49-21ksoftirqd/421:11:3510
1508499255,14cyclictest49-21ksoftirqd/421:31:5110
1508499251,14cyclictest49-21ksoftirqd/422:41:2210
1508499250,5cyclictest0-21swapper/423:56:2210
1139099250,2rtkit-daemon11389-21rtkit-daemon22:32:4210
1139099250,1rtkit-daemon18235-21systemd-cgroups22:18:0810
102632250,1sleep0121rcuc/023:03:210
79132240,1sleep20-21swapper/222:20:528
315632240,2sleep15832-21jbd2/md125-823:25:337
294932240,1sleep029800-21sshd22:46:310
242622240,1sleep20-21swapper/222:30:468
236082240,1sleep60-21swapper/622:15:1912
223792240,6sleep121509699cyclictest23:31:544
223792240,6sleep121509699cyclictest23:31:544
218072240,0sleep30-21swapper/321:37:009
165272240,2sleep80-21swapper/800:22:5614
1508999246,3cyclictest16342-21df_abs19:11:2313
1508499246,9cyclictest49-21ksoftirqd/422:59:3110
1508499246,12cyclictest49-21ksoftirqd/423:43:4310
1508499246,12cyclictest49-21ksoftirqd/423:43:4310
1508499246,11cyclictest22283-21sshd21:56:1110
1508499246,10cyclictest0-21swapper/422:50:1210
1508499245,8cyclictest49-21ksoftirqd/422:28:5710
1508499244,9cyclictest0-21swapper/421:49:2710
1508499244,15cyclictest49-21ksoftirqd/421:44:1810
1508499243,20cyclictest49-21ksoftirqd/421:53:3510
1508499243,19cyclictest481ktimersoftd/400:16:2110
1508399244,18cyclictest0-21swapper/322:51:259
1139099240,2rtkit-daemon11389-21rtkit-daemon20:41:394
73702230,2sleep21351-21nfsd21:46:388
43492230,2sleep20-21swapper/223:41:238
43492230,2sleep20-21swapper/223:41:238
31692230,0sleep120-21swapper/1223:14:154
273562230,1sleep150-21swapper/1522:22:567
191482230,1sleep30-21swapper/323:16:159
187272230,1sleep10-21swapper/123:58:131
181542230,20sleep50-21swapper/500:10:3511
159472230,21sleep120-21swapper/1220:30:514
15101992318,3cyclictest1233-21gdbus22:16:217
15093992319,2cyclictest95850irq/36-eth0-TxR23:27:132
15093992316,5cyclictest971ktimersoftd/1022:11:062
15093992314,5cyclictest98-21ksoftirqd/1021:51:472
15093992313,5cyclictest131rcu_preempt22:18:542
15093992312,6cyclictest98-21ksoftirqd/1023:01:152
1509399231,17cyclictest98-21ksoftirqd/1022:49:222
1508699230,4cyclictest0-21swapper/523:36:2111
1508699230,4cyclictest0-21swapper/523:36:2111
1508499238,8cyclictest49-21ksoftirqd/423:01:5910
1508499237,8cyclictest49-21ksoftirqd/423:06:3910
1508499237,8cyclictest49-21ksoftirqd/400:33:1010
1508499236,10cyclictest49-21ksoftirqd/421:36:2810
1508499235,11cyclictest49-21ksoftirqd/422:04:5710
1508499235,11cyclictest49-21ksoftirqd/400:05:4710
1508499234,9cyclictest49-21ksoftirqd/423:13:3810
1508499234,14cyclictest49-21ksoftirqd/421:17:2110
1139099230,2rtkit-daemon20219-21sshd00:06:294
108742230,21sleep00-21swapper/019:42:470
71962220,2sleep150-21swapper/1521:28:087
304602220,1sleep90-21swapper/923:36:4115
304602220,1sleep90-21swapper/923:36:4115
300422220,1sleep60-21swapper/622:19:3612
29702229,6sleep6141rcu_sched23:02:2412
284592220,19sleep10-21swapper/121:22:491
269292220,1sleep20-21swapper/219:25:278
25532220,1sleep100-21swapper/1019:32:002
231322220,2sleep50-21swapper/500:39:2711
156072220,1sleep120-21swapper/1222:33:204
1510199220,20cyclictest0-21swapper/1523:16:277
15098992216,4cyclictest18392-21sshd21:51:525
1509699229,10cyclictest30822-21sshd22:16:104
15094992216,3cyclictest525-21systemd-journal21:21:213
1509399229,9cyclictest141rcu_sched21:57:242
1509399228,6cyclictest131rcu_preempt23:00:152
1509399227,7cyclictest141rcu_sched23:34:122
1509399227,7cyclictest141rcu_sched23:34:122
1509399226,8cyclictest0-21swapper/1022:36:552
15093992215,5cyclictest31966-2110-uname00:07:522
15093992212,5cyclictest131rcu_preempt22:45:102
1509399221,19cyclictest32234-21kworker/10:100:31:232
15093992211,6cyclictest131rcu_preempt22:14:232
15093992210,7cyclictest141rcu_sched00:16:572
1509399220,17cyclictest0-21swapper/1021:37:242
15089992212,8cyclictest0-21swapper/720:41:2213
1508899221,18cyclictest27749-21systemd23:59:2712
1508699221,3cyclictest1068-21dbus-daemon22:15:4511
1508699221,19cyclictest1-21systemd23:14:3711
1508499229,7cyclictest141rcu_sched00:07:2610
1508499226,9cyclictest49-21ksoftirqd/422:06:2410
1508499226,8cyclictest49-21ksoftirqd/423:46:5210
1508499226,8cyclictest49-21ksoftirqd/423:25:5710
1508499226,8cyclictest1073-21systemd-logind00:22:1610
1508499225,8cyclictest49-21ksoftirqd/423:56:0010
15084992219,2cyclictest471rcuc/423:36:1610
15084992219,2cyclictest471rcuc/423:36:1610
1508399220,19cyclictest14753-21systemd00:15:239
126352220,1sleep80-21swapper/821:02:0414
126352220,1sleep80-21swapper/821:02:0414
70592210,17sleep60-21swapper/600:37:2112
326102210,1sleep00-21swapper/000:40:430
309782210,19sleep50-21swapper/519:29:4611
264312210,1sleep150-21swapper/1522:57:287
225352210,17sleep00-21swapper/022:07:280
203712210,1sleep90-21swapper/921:36:5115
182952210,1sleep60-21swapper/621:14:3612
182952210,1sleep60-21swapper/621:14:3612
161612210,17sleep150-21swapper/1500:31:047
1509399219,8cyclictest98-21ksoftirqd/1023:17:322
1509399218,7cyclictest141rcu_sched21:47:072
1509399214,15cyclictest0-21swapper/1000:29:422
15093992117,2cyclictest961rcuc/1021:13:222
15093992117,2cyclictest961rcuc/1021:13:222
15093992117,2cyclictest95850irq/36-eth0-TxR00:21:302
15093992117,2cyclictest95850irq/36-eth0-TxR00:01:422
15093992117,2cyclictest0-21swapper/1021:34:202
15093992115,4cyclictest95850irq/36-eth0-TxR23:06:362
15093992111,5cyclictest98-21ksoftirqd/1023:41:442
15093992111,5cyclictest98-21ksoftirqd/1023:41:442
15093992111,5cyclictest131rcu_preempt23:55:562
15093992111,5cyclictest131rcu_preempt22:27:042
1509199216,13cyclictest22452-21kill23:43:2615
1509199216,13cyclictest22452-21kill23:43:2615
1508899212,16cyclictest1233-21gdbus21:27:5512
15088992117,2cyclictest0-21swapper/600:33:0612
15088992115,3cyclictest19983-21systemd-cgroups00:23:2212
15088992114,4cyclictest7330-21sshd21:24:1012
15088992114,4cyclictest13348-21sshd21:55:1312
1508899211,3cyclictest525-21systemd-journal22:56:2812
15086992116,3cyclictest6116-21run-parts22:32:1111
1508699211,4cyclictest32622-21run-parts23:25:4111
15086992114,2cyclictest0-21swapper/500:17:1411
1508699211,3cyclictest624-21gdbus22:48:1611
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional