You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-28 - 22:50
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackcslot0.osadl.org (updated Sat Jun 28, 2025 00:43:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
80442135120,10sleep60-21swapper/619:09:4712
78172130104,21sleep140-21swapper/1419:07:156
11390991250,112rtkit-daemon0-21swapper/1319:08:115
80842120104,11sleep70-21swapper/719:10:2013
609821160,2sleep882-21ksoftirqd/819:05:4714
7850211588,23sleep110-21swapper/1119:07:443
2946021140,5sleep774-21ksoftirqd/721:44:2113
1435821100,6sleep3855699cyclictest22:26:169
2202321080,1sleep30-21swapper/319:25:469
7968210689,13sleep00-21swapper/019:09:250
567821040,1sleep100-21swapper/1021:59:382
124821040,1sleep150-21swapper/1522:28:377
2917921020,1sleep40-21swapper/421:36:4810
1985921010,0sleep130-21swapper/1300:25:275
2163521000,1sleep90-21swapper/922:19:5715
216612990,0sleep80-21swapper/822:15:5514
263152960,2sleep30-21swapper/323:59:379
43672950,1sleep20-21swapper/222:40:108
125282950,2sleep70-21swapper/723:09:5613
312592930,1sleep140-21swapper/1421:48:126
106962910,1sleep5561rcuc/500:01:1511
115982900,0sleep20-21swapper/200:35:428
787728972,13sleep10-21swapper/119:08:061
215472890,2sleep121073-21systemd-logind21:32:404
100082880,1sleep150-21swapper/1523:20:467
100082880,1sleep150-21swapper/1523:20:467
99662860,2sleep30-21swapper/323:05:519
248092860,1sleep110-21swapper/1120:45:513
248092860,1sleep110-21swapper/1120:45:513
771428567,14sleep50-21swapper/519:06:0111
18762840,1sleep21874-21sshd22:21:108
145182840,0sleep20-21swapper/223:17:368
145182840,0sleep20-21swapper/223:17:368
117822840,1sleep00-21swapper/023:46:430
790428375,4sleep90-21swapper/919:08:3015
784228375,4sleep30-21swapper/319:07:369
779628375,4sleep150-21swapper/1519:07:007
778128375,4sleep20-21swapper/219:06:478
772228375,4sleep100-21swapper/1019:06:072
279802830,1sleep10-21swapper/121:54:571
125022830,0sleep110-21swapper/1122:40:513
785128272,6sleep120-21swapper/1219:07:454
66812820,0sleep80-21swapper/821:23:3014
183152820,0sleep00-21swapper/021:53:510
107882820,1sleep100-21swapper/1022:51:162
56062810,1sleep5855899cyclictest23:31:0211
56062810,1sleep5855899cyclictest23:31:0211
204902810,1sleep120-21swapper/1223:58:584
780228072,4sleep40-21swapper/419:07:0510
162942760,0sleep150-21swapper/1521:14:157
152292760,5sleep7856199cyclictest23:17:4113
152292760,5sleep7856199cyclictest23:17:4113
154182720,1sleep1015421-21nfsd419:55:502
75132710,2sleep111-21systemd21:49:143
46922700,0sleep110-21swapper/1100:35:033
216072600,1sleep100-21swapper/1022:09:002
78312590,1sleep140-21swapper/1422:32:526
308672590,2sleep51100-21rs:main11
308672590,2sleep51100-21rs:main11
252282570,1sleep70-21swapper/700:33:3913
191422560,0sleep70-21swapper/723:43:5413
1139099520,2rtkit-daemon11389-21rtkit-daemon22:58:164
1139099520,0rtkit-daemon0-21swapper/1319:28:015
1139099510,0rtkit-daemon0-21swapper/1320:34:0810
855799509,32cyclictest19506-21systemd23:36:2510
1139099490,0rtkit-daemon0-21swapper/1319:37:5610
86942480,0sleep20-21swapper/222:18:148
4872480,6sleep0855099cyclictest23:08:290
855799461,42cyclictest22189-21sshd22:05:3110
855799461,42cyclictest18456-21sshd23:03:1910
855799460,4cyclictest1245-21gdbus22:14:0210
8550994610,34cyclictest1068-21dbus-daemon23:59:030
8561994416,26cyclictest22597-21sshd00:36:4513
855799449,32cyclictest7196-2110-uname22:43:5710
855799444,4cyclictest1545-21rm21:26:4710
8557994410,26cyclictest18100-21sshd22:59:4610
855099445,3cyclictest7142-21sshd22:32:480
855099442,39cyclictest525-21systemd-journal21:35:160
855099442,39cyclictest1-21systemd23:17:420
855099442,39cyclictest1-21systemd23:17:420
8550994410,32cyclictest525-21systemd-journal22:03:140
1139099440,7rtkit-daemon1894-21sendmail-mta23:06:494
856199436,35cyclictest0-21swapper/721:49:1313
855799438,7cyclictest6051-21uname22:47:1410
855099430,40cyclictest31560-21sshd21:26:280
1139099430,0rtkit-daemon0-21swapper/1200:29:024
857399429,31cyclictest6579-21sshd22:10:467
857399428,31cyclictest1073-21systemd-logind21:38:177
8557994211,28cyclictest9795-21systemd21:13:3210
855099429,30cyclictest1297-21sshd23:04:370
855099421,9cyclictest1-21systemd00:03:520
8550994210,30cyclictest1297-21sshd21:43:030
855099420,40cyclictest0-21swapper/000:33:020
1139099420,7rtkit-daemon11389-21rtkit-daemon22:10:264
8573994119,19cyclictest0-21swapper/1522:50:587
8573994113,23cyclictest0-21swapper/1522:38:437
8573994111,23cyclictest0-21swapper/1521:59:507
8573994110,29cyclictest5238-21sshd23:57:067
8573994110,29cyclictest10482-21systemd-cgroups21:31:207
8573994110,29cyclictest0-21swapper/1523:32:107
8573994110,29cyclictest0-21swapper/1523:32:107
855799418,31cyclictest9652-21ystemctl00:13:3910
855799415,3cyclictest1-21systemd00:29:0910
855799410,39cyclictest5497-21sshd22:50:4810
855799410,38cyclictest1-21systemd21:25:0610
855099416,7cyclictest145-17kauditd00:18:370
855099415,3cyclictest5467-21gdbus00:14:300
855099415,3cyclictest1234-21polkitd00:08:090
8550994112,26cyclictest10683-21sh21:16:530
8550994111,27cyclictest21160-21apt-get20:05:300
8550994111,27cyclictest11906-21sshd21:24:110
8550994110,28cyclictest0-21swapper/021:10:550
855099410,39cyclictest24908-21uname22:52:530
855099410,38cyclictest17053-21sshd00:36:050
1139099410,6rtkit-daemon0-21swapper/1222:41:374
857399408,30cyclictest1-21systemd00:08:347
8573994011,27cyclictest11743-21systemd22:44:307
8573994011,23cyclictest1068-21dbus-daemon22:02:037
856199408,24cyclictest169461tr21:35:4213
855799409,29cyclictest18150-21systemd22:37:4010
855799407,3cyclictest20617-21sshd00:02:2510
855799406,3cyclictest12846-21run-parts21:49:5310
855799400,4cyclictest2449-21systemd23:41:5410
855799400,38cyclictest1-21systemd22:25:0810
855099409,29cyclictest525-21systemd-journal21:38:140
855099409,28cyclictest6327-21run-parts23:23:550
855099409,28cyclictest6327-21run-parts23:23:550
855099406,4cyclictest27641-21sshd23:52:120
855099406,3cyclictest1297-21sshd00:30:170
855099406,2cyclictest5092-21systemd-cgroups22:47:080
855099405,4cyclictest12778-21systemd22:29:580
855099405,3cyclictest8241-2110-uname00:24:050
857399397,30cyclictest20981-21sshd23:40:307
857399396,31cyclictest0-21swapper/1521:49:137
857399394,33cyclictest0-21swapper/1522:46:037
8573993911,26cyclictest1-21systemd21:22:417
8561993911,26cyclictest1350-21nfsd22:34:4213
856199390,37cyclictest0-21swapper/722:46:0313
855799396,3cyclictest29359-21rm23:19:2210
855799396,3cyclictest29359-21rm23:19:2210
855799395,3cyclictest11180-21kill22:29:4610
8557993911,26cyclictest25831-21sshd23:55:4710
8557993911,25cyclictest0-21swapper/421:34:3610
855799390,3cyclictest15395-21sshd23:25:0010
855799390,3cyclictest15395-21sshd23:25:0010
855099399,4cyclictest8927-21sh22:36:350
855099398,4cyclictest1068-21dbus-daemon23:37:470
855099397,4cyclictest1297-21sshd21:45:550
855099395,3cyclictest25646-21basename22:05:470
855099394,5cyclictest0-21swapper/023:29:470
855099394,5cyclictest0-21swapper/023:29:470
855099392,5cyclictest25253-21systemd23:15:110
855099392,5cyclictest25253-21systemd23:15:110
855099391,9cyclictest11858-21sshd23:43:010
855099391,17cyclictest1068-21dbus-daemon23:31:070
855099391,17cyclictest1068-21dbus-daemon23:31:070
855099390,9cyclictest16628-21sshd22:15:360
1139099390,0rtkit-daemon0-21swapper/422:12:494
1139099390,0rtkit-daemon0-21swapper/419:38:594
857399389,26cyclictest0-21swapper/1500:20:247
857399388,24cyclictest0-21swapper/1500:00:427
8573993819,16cyclictest0-21swapper/1500:31:067
8573993813,23cyclictest30519-21sshd23:30:277
8573993813,23cyclictest30519-21sshd23:30:277
8573993813,23cyclictest0-21swapper/1521:41:147
856199387,29cyclictest525-21systemd-journal22:40:2613
855799389,27cyclictest1068-21dbus-daemon21:16:2210
855799387,28cyclictest19686-21sh00:21:4110
855799386,4cyclictest1245-21gdbus22:04:0010
855799385,3cyclictest1068-21dbus-daemon22:34:3510
855799380,35cyclictest17638-21sshd21:42:4610
855799380,35cyclictest17454-21env21:53:4610
855799380,17cyclictest0-21swapper/420:35:5810
855099387,5cyclictest19586-21systemd21:57:270
855099385,3cyclictest32516-21sh22:24:580
1139099380,3rtkit-daemon0-21swapper/1222:23:244
857399379,22cyclictest0-21swapper/1500:30:177
857399378,27cyclictest1-21systemd21:18:207
8573993713,22cyclictest29044-21sshd23:19:207
8573993713,22cyclictest29044-21sshd23:19:207
8573993712,24cyclictest0-21swapper/1522:31:347
8573993712,23cyclictest17512-21sshd23:06:367
8570993712,18cyclictest0-21swapper/1322:44:155
856199378,15cyclictest0-21swapper/723:39:3013
856199374,30cyclictest1073-21systemd-logind21:26:0613
856199373,31cyclictest525-21systemd-journal00:06:1013
855799377,3cyclictest1297-21sshd23:07:5910
855799376,3cyclictest26072-21cp00:19:0510
855799374,31cyclictest0-21swapper/423:34:5510
855799374,31cyclictest0-21swapper/423:34:5510
8557993711,24cyclictest10412-21sshd00:35:3610
855799370,4cyclictest27922-21systemd23:52:1410
855799370,34cyclictest0-21swapper/400:32:5910
855099377,3cyclictest14620-21systemd22:41:020
855099375,3cyclictest389-21md124_raid122:10:590
855099375,3cyclictest1068-21dbus-daemon22:59:120
857399368,26cyclictest9281-21env00:24:137
857399365,25cyclictest0-21swapper/1500:40:267
857399362,32cyclictest0-21swapper/1522:07:147
8573993619,10cyclictest322031sshd21:55:267
8573993617,17cyclictest0-21swapper/1522:56:497
8573993616,18cyclictest25044-21sshd22:16:177
8573993614,20cyclictest5147-21tr23:45:547
8573993613,21cyclictest9149-21rm23:13:087
8573993613,21cyclictest9149-21rm23:13:087
8573993613,21cyclictest0-21swapper/1500:10:377
8573993610,24cyclictest11457-21systemd22:22:247
8573993610,23cyclictest1068-21dbus-daemon21:26:067
856199366,28cyclictest0-21swapper/722:26:5913
856199365,25cyclictest5467-21gdbus21:31:5513
8561993616,18cyclictest1068-21dbus-daemon22:50:5813
855799366,3cyclictest16400-21run-parts21:57:0110
855799365,4cyclictest3970-2110-uname00:07:5710
855799364,4cyclictest16958-21sshd23:47:2510
855799363,4cyclictest19834-21sshd22:19:4210
855099366,3cyclictest32306-21sed19:35:590
855099364,2cyclictest0-21swapper/019:15:550
66072360,1sleep100-21swapper/1022:36:172
51472360,0sleep80-21swapper/821:45:2614
319642360,0sleep7731ktimersoftd/723:26:3113
319642360,0sleep7731ktimersoftd/723:26:3113
857399359,19cyclictest323861cat20:15:487
8573993515,18cyclictest9594-21systemd23:35:317
8573993515,18cyclictest8640-21expr20:25:537
8573993513,20cyclictest23953-21systemd23:03:567
8573993513,20cyclictest0-21swapper/1523:54:577
8573993510,21cyclictest213521cpuspeed_turbos19:25:457
856199357,26cyclictest31308-21systemd-cgroups00:26:3013
856199357,26cyclictest1-21systemd22:44:3013
856199355,27cyclictest0-21swapper/721:10:5013
856199355,26cyclictest38950md124_raid122:17:2313
8561993515,18cyclictest11778-21systemd-cgroups22:04:1213
855799355,4cyclictest30414-21sshd23:11:4110
855799355,4cyclictest30414-21sshd23:11:4110
855799355,3cyclictest4992-21grep21:00:4910
855799355,3cyclictest4992-21grep21:00:4910
855799354,4cyclictest13702-21sshd23:28:2310
855799354,4cyclictest13702-21sshd23:28:2310
855799354,3cyclictest1692-21latency_hist20:20:2910
855099351,4cyclictest0-21swapper/019:55:400
39332350,5sleep882-21ksoftirqd/822:57:5514
1139099350,6rtkit-daemon11389-21rtkit-daemon23:32:544
1139099350,6rtkit-daemon11389-21rtkit-daemon23:32:544
8573993414,13cyclictest0-21swapper/1519:42:357
8573993413,20cyclictest0-21swapper/1520:10:297
8573993413,19cyclictest0-21swapper/1519:18:087
8573993410,22cyclictest11812-21cat19:10:577
856199349,23cyclictest1348-21nfsd00:16:3013
856199349,23cyclictest0-21swapper/723:30:4613
856199349,23cyclictest0-21swapper/723:30:4613
856199346,26cyclictest30260-21sshd23:52:3213
856199345,27cyclictest9754-21systemd22:58:3913
856199344,28cyclictest0-21swapper/723:50:0213
8561993410,20cyclictest1-21systemd22:10:3913
8573993317,9cyclictest0-21swapper/1520:55:557
8573993317,9cyclictest0-21swapper/1520:55:557
8573993315,17cyclictest0-21swapper/1520:00:387
8573993314,18cyclictest0-21swapper/1519:56:167
8573993313,18cyclictest0-21swapper/1519:50:187
8573993313,18cyclictest0-21swapper/1519:36:247
8570993310,6cyclictest0-21swapper/1321:18:145
856199335,26cyclictest0-21swapper/721:21:4313
856199334,27cyclictest0-21swapper/723:14:4913
856199334,27cyclictest0-21swapper/723:14:4913
856199334,26cyclictest0-21swapper/722:21:0213
8561993320,11cyclictest30318-21sshd21:15:4213
8561993312,19cyclictest7065-21systemd00:00:5113
855099335,3cyclictest24635-21missed_timers20:45:500
855099335,3cyclictest24635-21missed_timers20:45:500
258862330,3sleep6651ktimersoftd/623:37:1212
258672330,1sleep140-21swapper/1421:36:266
1139099330,1rtkit-daemon8548-21cyclictest20:41:2110
1139099330,1rtkit-daemon8548-21cyclictest20:41:2110
8573993214,17cyclictest0-21swapper/1521:01:317
8573993214,17cyclictest0-21swapper/1521:01:317
8573993214,17cyclictest0-21swapper/1520:51:077
8573993214,17cyclictest0-21swapper/1520:51:077
8573993214,16cyclictest11480-21grep19:50:527
8573993214,16cyclictest0-21swapper/1520:35:477
8573993214,16cyclictest0-21swapper/1520:09:537
8573993213,17cyclictest3233-21expr20:20:427
8573993212,19cyclictest0-21swapper/1519:32:397
8573993211,20cyclictest0-21swapper/1520:41:307
8573993211,20cyclictest0-21swapper/1520:41:307
8573993210,20cyclictest1351-21nfsd19:20:387
8573993210,20cyclictest11363-21cstates20:30:457
857099325,9cyclictest0-21swapper/1323:07:015
857099320,26cyclictest1068-21dbus-daemon22:50:215
856199328,22cyclictest1-21systemd00:11:4413
856199327,23cyclictest11655-21systemd23:20:5213
856199327,23cyclictest11655-21systemd23:20:5213
856199327,23cyclictest0-21swapper/700:23:4913
856199327,19cyclictest27902-21kill23:00:4913
856199326,23cyclictest20816-21ls21:50:4513
8561993216,13cyclictest16235-21systemctl23:58:2813
855799321,3cyclictest0-21swapper/420:00:4410
855099321,4cyclictest0-21swapper/019:40:380
49052320,1sleep110-21swapper/1100:27:183
238982320,2sleep30-21swapper/322:56:149
163472320,1sleep60-21swapper/621:24:4912
1139099320,0rtkit-daemon0-21swapper/420:51:314
1139099320,0rtkit-daemon0-21swapper/420:51:314
8573993112,17cyclictest0-21swapper/1520:45:497
8573993112,17cyclictest0-21swapper/1520:45:497
857099311,24cyclictest1068-21dbus-daemon22:51:075
857099311,24cyclictest0-21swapper/1323:17:145
857099311,24cyclictest0-21swapper/1323:17:145
857099311,24cyclictest0-21swapper/1322:28:515
857099310,26cyclictest0-21swapper/1323:02:555
857099310,24cyclictest0-21swapper/1323:42:165
856199316,24cyclictest0-21swapper/720:14:1013
856199316,22cyclictest0-21swapper/719:10:5713
8561993114,15cyclictest0-21swapper/722:07:0213
8561993112,17cyclictest0-21swapper/721:56:4613
855799315,3cyclictest24023-21fschecks_count20:45:4510
855799315,3cyclictest24023-21fschecks_count20:45:4510
855799314,3cyclictest1428-21cpuspeed_turbos19:40:3910
34592310,2sleep80-21swapper/823:45:4814
266352310,1sleep90-21swapper/900:11:1315
1139099310,2rtkit-daemon5467-21gdbus21:15:534
857399304,23cyclictest0-21swapper/1521:05:447
857099307,12cyclictest1245-21gdbus21:47:285
857099303,21cyclictest0-21swapper/1322:55:575
8570993022,3cyclictest23149-21systemd-cgroups23:22:075
8570993022,3cyclictest23149-21systemd-cgroups23:22:075
856199309,19cyclictest0-21swapper/720:40:4813
856199309,19cyclictest0-21swapper/720:40:4813
856199307,21cyclictest0-21swapper/720:56:0713
856199307,21cyclictest0-21swapper/720:56:0713
856199307,21cyclictest0-21swapper/720:00:2913
856199307,21cyclictest0-21swapper/719:20:4613
856199306,23cyclictest0-21swapper/720:35:4413
856199302,26cyclictest0-21swapper/720:25:5413
8561993010,18cyclictest12072-21sh20:30:5013
855799304,3cyclictest8548-21cyclictest20:05:4610
78272300,26sleep90-21swapper/923:27:3815
78272300,26sleep90-21swapper/923:27:3815
70372300,1sleep50-21swapper/523:12:5111
70372300,1sleep50-21swapper/523:12:5111
309562300,1sleep140-21swapper/1420:15:416
263322300,0sleep90-21swapper/900:06:4015
232392300,2sleep30-21swapper/300:06:199
857099296,13cyclictest0-21swapper/1323:52:465
8570992923,4cyclictest1201rcuc/1300:30:165
857099292,19cyclictest15427-21systemd23:39:545
857099291,21cyclictest25666-21sshd21:51:035
8570992912,13cyclictest0-21swapper/1323:57:255
857099290,23cyclictest25956-21systemd21:29:435
857099290,22cyclictest0-21swapper/1321:57:525
857099290,21cyclictest0-21swapper/1323:15:075
857099290,21cyclictest0-21swapper/1323:15:075
856199299,18cyclictest0-21swapper/720:45:4613
856199299,18cyclictest0-21swapper/720:45:4613
856199297,21cyclictest0-21swapper/721:06:0213
856199297,20cyclictest0-21swapper/720:20:4013
856199296,21cyclictest0-21swapper/720:50:4713
856199296,21cyclictest0-21swapper/720:50:4713
8561992915,12cyclictest5230-21/usr/sbin/munin19:45:5413
8561992911,16cyclictest0-21swapper/719:40:5513
8561992911,16cyclictest0-21swapper/719:26:1013
8561992911,16cyclictest0-21swapper/719:15:4213
8561992910,17cyclictest0-21swapper/719:30:4813
855799294,3cyclictest28601-21mailstats20:10:5210
855799293,3cyclictest5195-21cat20:20:5810
855099295,2cyclictest17073-21/usr/sbin/munin19:20:520
66652290,25sleep100-21swapper/1023:57:202
276452290,1sleep50-21swapper/521:11:5711
117082290,1sleep80-21swapper/822:29:5014
1139099290,19rtkit-daemon0-21swapper/1220:21:354
857099289,10cyclictest0-21swapper/1323:45:305
857099282,21cyclictest5251systemd-journal00:08:515
857099282,17cyclictest1-21systemd21:22:515
857099281,21cyclictest1-21systemd22:37:515
8570992812,13cyclictest5467-21gdbus21:13:445
856199288,18cyclictest0-21swapper/720:06:0213
856199287,19cyclictest0-21swapper/720:15:5113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional