You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 06:07
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackcslot0.osadl.org (updated Sun Nov 23, 2025 00:43:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1712721410,7sleep1572799cyclictest00:33:377
2006521230,12sleep1496350irq/41-eth0-TxR23:42:466
2190221170,0sleep120-21swapper/1200:37:504
436621150,1sleep90-21swapper/921:33:4615
32635211191,16sleep130-21swapper/1319:11:525
32633211190,16sleep110-21swapper/1119:11:503
1765221070,2sleep100-21swapper/1023:15:582
32634210692,10sleep120-21swapper/1219:11:514
3134921030,0sleep150-21swapper/1522:17:287
2514421030,0sleep130-21swapper/1323:12:485
32338210281,16sleep100-21swapper/1019:08:362
2943921010,1sleep90-21swapper/922:47:1915
1758621010,1sleep00-21swapper/023:23:330
2319921000,1sleep00-21swapper/022:20:100
308122990,4sleep1272499cyclictest00:16:104
215122980,5sleep170699cyclictest00:30:421
299822970,7sleep871899cyclictest22:06:2314
86982960,1sleep20-21swapper/223:18:308
1139099960,87rtkit-daemon0-21swapper/619:12:2312
269902950,1sleep100-21swapper/1022:24:202
131182950,1sleep130-21swapper/1320:02:555
108992940,1sleep20-21swapper/222:15:078
252412930,1sleep10-21swapper/122:05:441
239822920,1sleep40-21swapper/421:50:4810
86572910,1sleep40-21swapper/423:30:1810
5302910,0sleep00-21swapper/022:28:590
311842910,5sleep1472699cyclictest22:17:276
3244829072,4sleep90-21swapper/919:09:5615
281612900,1sleep150-21swapper/1521:32:487
226472900,0sleep80-21swapper/821:32:2514
98882890,3sleep871899cyclictest22:41:0914
3238728981,4sleep30-21swapper/319:09:179
3236428970,14sleep20-21swapper/219:09:008
3232528970,14sleep10-21swapper/119:08:271
100732890,1sleep110-21swapper/1100:40:303
82242880,1sleep20-21swapper/223:41:308
185432880,1sleep80-21swapper/800:01:4014
67642870,0sleep140-21swapper/1421:41:216
321602870,1sleep70-21swapper/722:47:3313
52362860,0sleep100-21swapper/1021:55:592
156262860,0sleep60-21swapper/621:31:3512
3241628572,9sleep140-21swapper/1419:09:286
3236928572,8sleep70-21swapper/719:09:0513
157192850,0sleep90-21swapper/900:05:0915
3245528472,8sleep150-21swapper/1519:10:027
83712830,0sleep150-21swapper/1523:14:427
3248428375,4sleep50-21swapper/519:10:2411
3244328375,4sleep40-21swapper/419:09:5010
2921528375,4sleep80-21swapper/819:07:3014
146162820,1sleep914615-21systemd-cgroups00:37:1515
3236228072,4sleep00-21swapper/019:08:580
319152800,6sleep1472699cyclictest23:59:056
207732790,1sleep720780-21sh00:13:5713
314672780,0sleep20-21swapper/221:15:128
175192750,1sleep40-21swapper/400:41:2710
158962740,0sleep20-21swapper/223:23:198
319712720,0sleep50-21swapper/521:55:2211
305692670,1sleep130-21swapper/1322:36:095
219122600,1sleep50-21swapper/521:42:4811
20672600,1sleep102063-21sshd21:33:312
161422570,0sleep30-21swapper/300:08:489
1139099560,8rtkit-daemon31175-21sshd22:13:457
132272480,18sleep120-21swapper/1222:34:004
1139099480,0rtkit-daemon0-21swapper/1321:10:0511
72099467,6cyclictest4942-21sh00:17:372
1139099460,2rtkit-daemon11389-21rtkit-daemon19:23:1912
71699459,34cyclictest3165-21sshd23:06:5913
71699459,34cyclictest3165-21sshd23:06:5913
1139099450,1rtkit-daemon11389-21rtkit-daemon21:48:027
72099440,1cyclictest0-21swapper/1022:27:312
1139099440,6rtkit-daemon0-21swapper/1321:17:3011
96802420,6sleep1172199cyclictest23:22:403
71699427,33cyclictest16313-2110-uname23:31:1613
716994213,27cyclictest21687-21sh21:39:1913
721994113,3cyclictest20902-21systemd21:20:593
72099419,2cyclictest145-21kauditd23:28:512
72099411,9cyclictest1-21systemd22:33:542
71699417,31cyclictest26388-21sshd21:25:0313
721994013,4cyclictest5580-21sshd23:18:063
72099408,3cyclictest19194-21sh23:53:492
72099408,3cyclictest1-21systemd23:06:472
72099408,3cyclictest1-21systemd23:06:472
720994014,24cyclictest23216-21sshd00:23:262
72099401,3cyclictest0-21swapper/1022:55:022
72099401,3cyclictest0-21swapper/1022:55:022
720994012,25cyclictest1190-21systemd22:21:242
72099400,37cyclictest0-21swapper/1021:24:312
72099400,1cyclictest0-21swapper/1023:40:302
72099400,10cyclictest10586-21sshd00:36:472
71699408,29cyclictest1-21systemd21:34:1313
71699406,2cyclictest7530-21systemd-cgroups00:40:0813
716994011,26cyclictest6377-21sshd00:25:2213
716994010,28cyclictest949-21systemd21:55:3113
716994010,27cyclictest0-21swapper/723:24:3013
71699400,38cyclictest26011-21sshd22:24:1313
71699400,37cyclictest31709-21systemd00:31:5513
43962400,0sleep110-21swapper/1122:29:263
721993914,4cyclictest13289-21sshd23:49:313
72099398,3cyclictest525-21systemd-journal22:58:042
72099398,3cyclictest525-21systemd-journal22:58:042
720993912,25cyclictest0-21swapper/1021:50:262
720993910,6cyclictest29507-21idleruntime23:47:442
720993910,3cyclictest0-21swapper/1021:38:082
720993910,2cyclictest3206-21sshd22:47:462
72099390,11cyclictest0-21swapper/1023:09:082
72099390,11cyclictest0-21swapper/1023:09:082
71699399,28cyclictest2650-21sh22:36:4013
71699399,28cyclictest11134-21sshd00:08:1413
71699398,28cyclictest28862-21sh23:13:1313
71699397,5cyclictest27970-21systemctl22:54:1613
71699397,5cyclictest27970-21systemctl22:54:1613
71699397,30cyclictest10014-21systemd22:41:1013
71699395,5cyclictest1042-17auditd21:49:4513
71699395,3cyclictest1068-21dbus-daemon00:35:4013
71699395,32cyclictest28432-21kworker/7:121:14:0413
71699394,5cyclictest0-21swapper/723:54:3413
716993910,26cyclictest1297-21sshd23:21:3613
71699390,36cyclictest1529-21systemd00:21:0813
225242390,5sleep270899cyclictest21:54:158
72199388,3cyclictest7915-21sshd22:55:473
72199388,3cyclictest7915-21sshd22:55:473
721993813,4cyclictest31516-21sshd22:39:493
721993812,3cyclictest1297-21sshd21:49:223
721993811,3cyclictest1297-21sshd21:28:243
72199380,13cyclictest23671-21sshd23:05:243
72199380,13cyclictest23671-21sshd23:05:243
72099389,4cyclictest5464-21packagekitd21:57:382
72099388,4cyclictest5467-21gdbus00:03:302
72099388,3cyclictest1068-21dbus-daemon23:25:012
72099387,4cyclictest14228-21run-parts22:04:172
72099386,3cyclictest5467-21gdbus23:17:382
72099381,3cyclictest0-21swapper/1022:12:372
71699389,23cyclictest23951-21systemd-cgroups21:47:0913
71699388,27cyclictest1471-21sshd23:40:3513
71699387,3cyclictest1-21systemd22:04:2813
71699387,3cyclictest1-21systemd00:06:5313
71699387,3cyclictest11322-21ystemctl22:15:0913
71699387,29cyclictest4239-21systemd22:44:0413
71699386,3cyclictest6732-21systemd21:59:4413
71699386,3cyclictest22651-21timerandwakeup23:57:5813
71699385,3cyclictest30210-21sh21:18:2213
71699385,3cyclictest26984-21uname23:01:5913
71699385,3cyclictest26984-21uname23:01:5913
71699385,3cyclictest17262-21systemd22:12:2013
71699384,3cyclictest10463-21systemd-cgroups21:30:5613
71699381,35cyclictest525-21systemd-journal23:52:0313
71699381,35cyclictest10260-21run-parts23:33:5813
71699380,8cyclictest26009-21sshd23:43:2813
72199370,13cyclictest17323-21sshd00:22:493
72099378,4cyclictest1068-21dbus-daemon00:09:182
72099378,3cyclictest14878-21sshd22:12:032
72099372,7cyclictest525-21systemd-journal21:28:482
720993710,2cyclictest0-21swapper/1022:46:302
716993711,24cyclictest10724-21sshd22:18:3313
250292370,0sleep140-21swapper/1422:31:546
112792370,8sleep671599cyclictest00:18:1312
71699366,2cyclictest0-21swapper/719:57:4613
716993612,21cyclictest0-21swapper/723:07:3813
716993612,21cyclictest0-21swapper/723:07:3813
95682350,1sleep30-21swapper/323:52:459
72199350,14cyclictest2493-21sshd00:17:083
72199350,13cyclictest8477-21systemd22:44:363
72099358,3cyclictest1297-21sshd23:35:372
72099358,3cyclictest11658-21sh21:16:352
72099356,4cyclictest5467-21gdbus23:44:302
72099355,4cyclictest30442-21sh22:39:392
72099355,3cyclictest20155-21sshd00:01:522
720993511,3cyclictest15824-2110-uname00:41:162
71699357,2cyclictest30586-21systemd22:28:4213
136652350,31sleep35467-21gdbus22:11:569
1139099350,0rtkit-daemon0-21swapper/423:32:2810
72199349,4cyclictest26578-21sshd00:10:033
72199349,4cyclictest17873-21sshd23:16:003
72099348,3cyclictest27572-21sshd21:21:452
72099341,30cyclictest0-21swapper/1000:30:272
191252340,24sleep1172199cyclictest23:42:433
113672340,6sleep270899cyclictest00:36:528
72199339,4cyclictest11703-21run-parts21:12:583
721993310,3cyclictest30514-21sshd23:55:133
721993310,3cyclictest1068-21dbus-daemon21:43:213
72099336,3cyclictest353-21cpuspeed_turbos21:07:382
72099330,10cyclictest1544-21sshd00:16:522
227392330,1sleep40-21swapper/423:12:3910
1139099330,1rtkit-daemon0-21swapper/1523:09:187
1139099330,1rtkit-daemon0-21swapper/1523:09:187
72199329,3cyclictest16418-21updatedb.mlocat00:17:333
721993210,3cyclictest2563-21systemd22:07:023
72099326,3cyclictest1297-21sshd21:42:342
71699321,4cyclictest0-21swapper/719:27:3813
716993212,18cyclictest0-21swapper/719:52:3913
36542320,1sleep90-21swapper/923:48:1815
167682320,1sleep116772-2110-uname23:53:311
1139099320,16rtkit-daemon0-21swapper/1521:41:337
72199310,13cyclictest17021-21sshd22:49:263
71699318,19cyclictest20258-21cpuspeed_turbos20:52:3813
71699316,3cyclictest29423-21nfsd420:22:4813
71699316,3cyclictest29423-21nfsd420:22:4813
71699310,7cyclictest29157-21sshd19:43:0613
71699310,28cyclictest1282-21snmpd19:37:5113
1139099310,1rtkit-daemon0-21swapper/423:52:1510
1139099310,13rtkit-daemon0-21swapper/1419:23:516
721993010,4cyclictest32333-21sshd00:07:183
721993010,3cyclictest1068-21dbus-daemon21:41:093
72199300,13cyclictest17476-21sshd23:38:293
72099302,5cyclictest0-21swapper/1020:57:372
71699307,3cyclictest29904-21irqstats21:02:4413
71699300,4cyclictest135150nfsd20:37:3413
71699300,4cyclictest135150nfsd20:37:3413
49702300,2sleep90-21swapper/922:10:5815
46122300,0sleep150-21swapper/1500:39:467
3652300,1sleep151371ktimersoftd/1522:28:587
307242300,2sleep30-21swapper/300:35:189
214392300,2sleep221555-21sshd22:19:578
320922290,1sleep20-21swapper/223:47:568
304762290,26sleep120-21swapper/1220:22:584
304762290,26sleep120-21swapper/1220:22:584
252682290,1sleep10-21swapper/123:16:531
136232290,2sleep120-21swapper/1222:52:434
136232290,2sleep120-21swapper/1222:52:434
72199288,4cyclictest5550-21run-parts21:26:253
72099280,1cyclictest0-21swapper/1020:52:482
71699284,2cyclictest9892-21cstates19:22:3813
71699280,7cyclictest0-21swapper/719:32:5213
71699280,6cyclictest0-21swapper/719:49:3713
37332280,2sleep130-21swapper/1322:21:445
274882280,0sleep40-21swapper/423:43:4110
271842280,1sleep60-21swapper/623:32:3012
256862280,1sleep140-21swapper/1400:15:086
233262280,24sleep10-21swapper/123:28:081
178352280,1sleep30-21swapper/321:24:029
8392271,2sleep1973-21sshd23:21:431
7592270,23sleep8925-21systemd22:21:2314
72099276,3cyclictest30411-21nfsd421:02:472
71699274,2cyclictest11215-21cstates20:02:4013
71699273,3cyclictest3784-21sshd21:12:2513
71699273,3cyclictest15314-21cstates20:07:4013
71699273,2cyclictest1347-21nfsd20:12:3713
70499279,14cyclictest1068-21dbus-daemon22:58:470
70499279,14cyclictest1068-21dbus-daemon22:58:470
704992718,6cyclictest1238-21gdbus21:50:520
70499271,23cyclictest0-21swapper/000:10:220
53132270,2sleep35315-21sshd22:33:039
40612270,23sleep120-21swapper/1223:48:224
304592270,1sleep10-21swapper/123:36:251
254482270,2sleep60-21swapper/622:09:3512
250702270,2sleep20-21swapper/200:09:528
218062270,2sleep40-21swapper/423:57:5210
185552270,2sleep60-21swapper/623:50:0912
136122271,3sleep91238-21gdbus21:13:1215
1139099270,5rtkit-daemon0-21swapper/1523:19:597
72599260,7cyclictest131rcu_preempt21:56:125
721992610,3cyclictest20029-2110-uname23:35:083
72099267,3cyclictest25784-21timerwakeupswit20:17:562
72099267,3cyclictest25784-21timerwakeupswit20:17:562
71699264,2cyclictest3738-21grep19:12:5613
71699263,3cyclictest23104-21cpuspeed20:17:4013
71699263,3cyclictest23104-21cpuspeed20:17:4013
706992621,3cyclictest22432-21gdbus22:37:381
70499260,2cyclictest0-21swapper/020:32:360
70499260,2cyclictest0-21swapper/020:32:360
66672260,0sleep50-21swapper/500:32:3811
247342260,24sleep130-21swapper/1319:38:365
231452260,1sleep90-21swapper/921:24:4215
23052260,22sleep80-21swapper/822:51:3614
175702260,1sleep00-21swapper/021:57:250
1139099260,0rtkit-daemon0-21swapper/1320:10:395
110912260,1sleep120-21swapper/1223:52:554
88902250,2sleep30-21swapper/321:30:459
725992514,6cyclictest122-21ksoftirqd/1322:44:145
72199259,3cyclictest19404-21sshd23:08:343
72199259,3cyclictest19404-21sshd23:08:343
71699256,2cyclictest13641-21missed_timers20:42:4613
715992520,3cyclictest17509-21sshd00:30:1612
704992511,11cyclictest5467-21gdbus22:23:440
704992510,13cyclictest0-21swapper/000:06:120
309132250,2sleep10-21swapper/122:28:451
293922251,21sleep629509-21systemd23:09:4512
293922251,21sleep629509-21systemd23:09:4512
211002250,21sleep50-21swapper/523:27:5611
202192250,1sleep00-21swapper/022:38:210
184182250,1sleep50-21swapper/523:50:0811
123182250,21sleep60-21swapper/621:53:0012
1139099250,0rtkit-daemon0-21swapper/423:52:5210
1139099250,0rtkit-daemon0-21swapper/1200:06:194
109772250,1sleep130-21swapper/1320:40:545
109772250,1sleep130-21swapper/1320:40:545
88892240,1sleep10-21swapper/100:07:591
77812240,2sleep30-21swapper/321:34:139
72599240,21cyclictest15249-21sshd21:27:385
72499241,21cyclictest0-21swapper/1220:02:394
721992410,4cyclictest19095-21sshd23:57:413
72199240,13cyclictest7105-21sshd00:36:213
719992419,3cyclictest90-21ksoftirqd/923:46:0215
71699245,2cyclictest832-21jbd2/md125-820:27:4013
71699245,2cyclictest832-21jbd2/md125-820:27:4013
715992411,8cyclictest66-21ksoftirqd/621:21:5212
715992411,6cyclictest141rcu_sched21:16:4012
715992410,8cyclictest66-21ksoftirqd/600:22:3712
71499240,22cyclictest0-21swapper/520:17:4711
71499240,22cyclictest0-21swapper/520:17:4711
70499248,14cyclictest0-21swapper/022:16:190
70499248,13cyclictest1068-21dbus-daemon21:23:360
298612240,20sleep20-21swapper/222:43:138
221232240,2sleep922247-21sshd00:42:0515
176882240,20sleep90-21swapper/922:31:0115
146662240,0sleep20-21swapper/200:12:488
81652230,2sleep48293-21sshd22:48:2010
725992313,5cyclictest131rcu_preempt00:32:095
72599230,13cyclictest122-21ksoftirqd/1300:22:515
724992317,3cyclictest1238-21gdbus00:29:074
72099236,3cyclictest12394-21cstates20:42:392
719992318,3cyclictest90-21ksoftirqd/923:36:5015
71899230,21cyclictest0-21swapper/820:07:2414
71599232,2cyclictest66-21ksoftirqd/621:33:3412
70899238,12cyclictest1068-21dbus-daemon00:02:428
70499239,11cyclictest6858-21systemd23:52:370
70499236,9cyclictest10-21ksoftirqd/000:34:140
70499233,17cyclictest0-21swapper/023:37:550
704992316,5cyclictest111ktimersoftd/023:21:170
70499231,19cyclictest25701-21systemd22:09:360
704992311,9cyclictest1073-21systemd-logind22:33:200
704992310,10cyclictest1245-21gdbus21:20:290
70499230,21cyclictest0-21swapper/021:43:350
58172230,2sleep140-21swapper/1400:25:196
292052230,20sleep20-21swapper/222:50:588
271172230,2sleep60-21swapper/622:35:4712
198902230,19sleep30-21swapper/321:13:549
17552230,19sleep11891-21sshd22:54:581
17552230,19sleep11891-21sshd22:54:581
172202230,6sleep871899cyclictest20:07:5514
170612230,19sleep50-21swapper/521:35:1911
140262230,2sleep60-21swapper/622:41:4012
1139099230,1rtkit-daemon9728-21sshd21:16:215
1139099230,0rtkit-daemon0-21swapper/722:43:407
1139099230,0rtkit-daemon0-21swapper/619:47:556
74342220,1sleep80-21swapper/821:45:0014
72799221,5cyclictest0-21swapper/1500:21:427
72799220,19cyclictest980-21systemctl23:48:017
72599229,8cyclictest131rcu_preempt23:22:255
72599226,9cyclictest122-21ksoftirqd/1323:46:275
72599223,10cyclictest122-21ksoftirqd/1321:22:475
725992214,4cyclictest131rcu_preempt22:51:315
72599221,18cyclictest8515-21grep21:37:505
725992211,6cyclictest122-21ksoftirqd/1323:28:545
725992210,7cyclictest131rcu_preempt00:10:315
724992217,3cyclictest23871-21env23:39:174
724992213,3cyclictest16741-21sshd23:34:434
71999220,2cyclictest0-21swapper/921:27:3815
718992216,4cyclictest32363-21uname22:25:0414
71699223,3cyclictest3460-21cpuspeed_turbos20:32:3713
71699223,3cyclictest3460-21cpuspeed_turbos20:32:3713
71699223,3cyclictest18354-21swap20:47:5413
71599229,8cyclictest141rcu_sched00:07:3012
71599228,8cyclictest141rcu_sched23:01:3112
71599228,8cyclictest141rcu_sched23:01:3112
71599227,8cyclictest131rcu_preempt23:17:4612
715992213,2cyclictest131rcu_preempt23:57:5812
71499220,20cyclictest0-21swapper/523:07:5311
71499220,20cyclictest0-21swapper/523:07:5311
71499220,20cyclictest0-21swapper/521:27:4711
70499229,11cyclictest0-21swapper/022:56:590
70499229,11cyclictest0-21swapper/022:56:590
70499226,8cyclictest0-21swapper/000:27:590
70499225,11cyclictest0-21swapper/021:57:550
70499223,17cyclictest0-21swapper/023:30:000
70499223,17cyclictest0-21swapper/020:07:320
70499223,16cyclictest1-21systemd21:33:570
70499222,17cyclictest16660-21uname00:13:030
704992218,2cyclictest12957-21systemd-cgroups22:48:560
704992218,2cyclictest111ktimersoftd/023:44:300
704992215,5cyclictest111ktimersoftd/023:09:570
704992215,5cyclictest111ktimersoftd/023:09:570
704992215,5cyclictest0-21swapper/022:06:550
704992210,10cyclictest26594-21sshd21:28:460
70499220,19cyclictest2745-21systemd-cgroups21:15:350
70499220,19cyclictest0-21swapper/000:18:390
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional