You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-10-10 - 18:11
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackcslot0.osadl.org (updated Thu Oct 10, 2024 00:43:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
982121450,7sleep2727699cyclictest00:29:028
181221370,0sleep80-21swapper/823:54:5414
181221370,0sleep80-21swapper/823:54:5414
2114021360,3sleep2727699cyclictest22:59:108
1768721240,2sleep11998-21dbus-daemon23:09:293
65622123105,13sleep30-21swapper/319:11:219
2971521230,1sleep70-21swapper/700:05:0313
2971521230,1sleep70-21swapper/700:05:0313
3393211693,16sleep70-21swapper/719:09:4913
6536211391,18sleep130-21swapper/1319:10:595
335721120,1sleep93007-1kworker/9:1H00:39:1915
3377210982,22sleep140-21swapper/1419:09:406
6669210791,11sleep00-21swapper/019:12:540
2232721060,0sleep110-21swapper/1122:26:573
1337821050,6sleep14729299cyclictest23:30:246
6480210379,19sleep120-21swapper/1219:10:104
1981321010,2sleep8811ktimersoftd/823:02:4514
1662621010,2sleep80-21swapper/821:25:2314
1662621010,2sleep80-21swapper/821:25:2314
867021000,4sleep11728899cyclictest23:48:163
867021000,4sleep11728899cyclictest23:48:163
753721000,5sleep4727899cyclictest21:42:2610
2683121000,1sleep60-21swapper/623:43:0612
1112621000,2sleep4509-21systemd-journal22:32:4310
243762990,8sleep7728299cyclictest23:13:4213
88862980,1sleep00-21swapper/021:46:070
119742980,0sleep20-21swapper/221:28:468
119742980,0sleep20-21swapper/221:28:468
116392950,6sleep4727899cyclictest23:52:1010
116392950,6sleep4727899cyclictest23:52:1010
3132940,1sleep00-21swapper/021:20:100
3132940,1sleep00-21swapper/021:20:100
302122940,5sleep7728299cyclictest00:27:3113
244272940,1sleep140-21swapper/1422:16:186
180232940,1sleep120-21swapper/1223:38:394
16432940,5sleep2727699cyclictest22:38:508
278422920,0sleep110-21swapper/1122:09:503
52892900,0sleep60-21swapper/600:39:3212
682328972,4sleep90-21swapper/919:14:0815
650628772,11sleep10-21swapper/119:10:311
193712870,1sleep8998-21dbus-daemon00:26:1214
684328674,8sleep100-21swapper/1019:14:252
663028672,9sleep150-21swapper/1519:12:217
302772860,2sleep830258-21sshd22:45:3714
189892860,2sleep218990-21sshd23:13:048
101882860,0sleep80-21swapper/822:14:5214
236292850,2sleep00-21swapper/022:44:570
157512850,0sleep120-21swapper/1223:23:564
113772850,0sleep20-21swapper/223:08:488
110232850,1sleep70-21swapper/723:41:1513
101552850,1sleep40-21swapper/421:28:3210
101552850,1sleep40-21swapper/421:28:3210
667328472,7sleep40-21swapper/419:12:5810
654828472,8sleep60-21swapper/619:11:0812
651728375,4sleep110-21swapper/1119:10:413
650728372,6sleep20-21swapper/219:10:328
338528375,4sleep50-21swapper/519:09:4711
264192830,1sleep20-21swapper/223:57:458
264192830,1sleep20-21swapper/223:57:458
223582830,0sleep30-21swapper/322:05:259
164632830,6sleep5727999cyclictest00:11:1011
662328274,4sleep80-21swapper/819:12:1414
176032820,1sleep20-21swapper/200:07:228
176032820,1sleep20-21swapper/200:07:228
172282820,1sleep150-21swapper/1523:52:537
172282820,1sleep150-21swapper/1523:52:537
215242800,8sleep4727899cyclictest22:12:4010
78312780,1sleep4481ktimersoftd/421:56:5310
254732760,0sleep20-21swapper/221:51:468
251822710,0sleep10-21swapper/100:38:071
142482670,2sleep1514249-21sshd00:22:027
97392640,0sleep50-21swapper/522:21:4611
249902550,1sleep80-21swapper/822:30:3914
727799511,4cyclictest998-21dbus-daemon22:43:599
104582500,2sleep610455-21sshd21:14:3912
104582500,2sleep610455-21sshd21:14:3912
727799463,4cyclictest1030-21polkitd23:47:219
727799463,4cyclictest1030-21polkitd23:47:219
727799459,33cyclictest145-21kauditd22:45:019
727799443,3cyclictest31904-21gdbus00:37:399
727799437,33cyclictest18070-21sshd21:33:119
727799437,33cyclictest18070-21sshd21:33:119
727799435,3cyclictest31904-21gdbus22:22:249
727799434,3cyclictest998-21dbus-daemon00:16:399
7277994311,29cyclictest26519-2110-uname23:10:109
40792430,2sleep0998-21dbus-daemon22:10:430
727799423,4cyclictest3646-21sshd22:17:399
727799421,4cyclictest11213-21systemd22:50:329
727799421,38cyclictest16032-2110-uname00:07:119
727799421,38cyclictest16032-2110-uname00:07:119
727799421,37cyclictest11496-21systemd-cgroups23:41:189
7277994211,28cyclictest3703-21sshd00:02:229
7277994210,30cyclictest14391-21systemd23:52:319
7277994210,30cyclictest14391-21systemd23:52:319
727799420,8cyclictest17850-21sshd21:58:009
727799419,29cyclictest27712-21sshd23:57:559
727799419,29cyclictest27712-21sshd23:57:559
727799411,4cyclictest30928-21systemd22:10:069
727799411,3cyclictest9582-21systemd-cgroups00:39:529
7277994111,27cyclictest22962-21sshd22:37:279
727799410,1cyclictest0-21swapper/323:27:109
727799409,28cyclictest1455-21sshd21:20:189
727799409,28cyclictest1455-21sshd21:20:189
727799407,3cyclictest1-21systemd23:08:449
727799401,5cyclictest0-21swapper/322:32:589
727799401,4cyclictest21340-21sshd22:26:509
7277994011,27cyclictest0-21swapper/300:34:129
7277994010,28cyclictest18069-21systemd21:47:169
727799395,3cyclictest4947-21sshd23:37:049
727799395,3cyclictest18863-21systemd21:51:039
727799393,3cyclictest12234-21gdbus21:36:299
727799391,6cyclictest1-21systemd21:41:239
7277993910,27cyclictest0-21swapper/321:16:369
7277993910,27cyclictest0-21swapper/321:16:369
7277993910,26cyclictest28929-21sshd21:26:499
7277993910,26cyclictest28929-21sshd21:26:499
727799390,36cyclictest21154-21sshd23:24:339
727799390,36cyclictest15088-21sshd23:02:109
727799387,3cyclictest8664-21missed_timers23:29:579
727799384,3cyclictest998-21dbus-daemon23:15:309
727799383,3cyclictest998-21dbus-daemon00:23:369
727799383,3cyclictest1005-21NetworkManager00:25:189
727799382,3cyclictest0-21swapper/300:11:329
235652380,7sleep7728299cyclictest00:16:0113
81272370,1sleep2509-21systemd-journal23:19:308
727799374,3cyclictest12234-21gdbus21:59:509
727799361,6cyclictest21718-21systemd22:55:109
728999350,2cyclictest92750irq/35-eth0-TxR21:54:494
727799356,3cyclictest31551-21latency_hist19:44:379
93732340,2sleep09368-21sshd00:32:450
7290993428,4cyclictest0-21swapper/1321:53:545
7290993421,11cyclictest509-21systemd-journal21:46:335
727799340,32cyclictest0-21swapper/320:04:399
64572340,29sleep100-21swapper/1000:32:242
314582340,1sleep100-21swapper/1023:18:182
253182340,6sleep12728999cyclictest21:48:114
7290993328,3cyclictest6699-21systemd-cgroups21:31:505
7290993328,3cyclictest6699-21systemd-cgroups21:31:505
727799334,3cyclictest31584-21nfsd420:59:589
727799334,3cyclictest13434-21fschecks_time19:59:529
727799332,4cyclictest0-21swapper/320:34:489
287402330,0sleep120-21swapper/1221:37:454
242652330,5sleep558-21ksoftirqd/500:01:0011
7290993226,4cyclictest0-21swapper/1321:19:155
7290993226,4cyclictest0-21swapper/1321:19:155
7290993225,5cyclictest93150irq/39-eth0-TxR22:01:065
291242320,2sleep990-21ksoftirqd/922:52:3615
277112320,20sleep100-21swapper/1023:50:072
277112320,20sleep100-21swapper/1023:50:072
224462320,1sleep60-21swapper/623:24:3912
7290993127,3cyclictest0-21swapper/1321:44:055
7290993126,3cyclictest0-21swapper/1321:54:505
7290993125,4cyclictest0-21swapper/1321:21:085
7290993125,4cyclictest0-21swapper/1321:21:085
7290993124,5cyclictest0-21swapper/1321:24:525
7290993124,5cyclictest0-21swapper/1321:24:525
727799315,3cyclictest25368-21idleruntime19:34:559
727799311,3cyclictest5842-21fschecks_time20:29:519
37002310,8sleep7728299cyclictest23:47:4013
37002310,8sleep7728299cyclictest23:47:4013
23142310,2sleep00-21swapper/021:52:490
166892310,1sleep016692-21timerandwakeup00:00:070
160692310,1sleep150-21swapper/1522:19:007
7290993026,3cyclictest0-21swapper/1320:14:015
7290993026,2cyclictest1201rcuc/1300:06:055
7290993026,2cyclictest1201rcuc/1300:06:055
7290993025,3cyclictest1-21systemd21:38:385
729099300,28cyclictest1201rcuc/1323:47:465
729099300,28cyclictest1201rcuc/1323:47:465
7278993015,8cyclictest0-21swapper/423:12:3810
727799301,5cyclictest504150nfsd19:19:489
727799301,4cyclictest0-21swapper/320:19:499
218062300,2sleep100-21swapper/1023:35:092
205782300,1sleep100-21swapper/1021:58:202
174522300,0sleep100-21swapper/1022:29:532
164952300,2sleep60-21swapper/600:18:5712
14272300,5sleep15729399cyclictest00:05:277
14272300,5sleep15729399cyclictest00:05:277
105032300,0sleep90-21swapper/922:14:5315
98752290,7sleep0727199cyclictest22:29:110
7290992924,3cyclictest0-21swapper/1320:34:525
7290992923,3cyclictest31804-21ntpq20:59:585
727799295,3cyclictest2921-21wc20:25:009
301472290,1sleep5561rcuc/521:26:5611
301472290,1sleep5561rcuc/521:26:5611
269662290,1sleep50-21swapper/520:15:0011
163462290,1sleep70-21swapper/723:05:4113
7290992825,2cyclictest0-21swapper/1320:52:015
7290992823,4cyclictest0-21swapper/1319:51:005
7290992814,13cyclictest1201rcuc/1322:58:205
7278992810,8cyclictest131rcu_preempt22:03:5810
727799281,3cyclictest18683-21latency20:44:579
56942280,24sleep125041-21nfsd00:05:574
56942280,24sleep125041-21nfsd00:05:574
295512280,1sleep130-21swapper/1322:27:475
142862280,0sleep140-21swapper/1422:40:066
134722280,1sleep110-21swapper/1100:40:093
133512280,2sleep60-21swapper/622:39:5912
99572270,23sleep100-21swapper/1022:07:582
7290992724,2cyclictest0-21swapper/1319:45:255
7290992723,3cyclictest0-21swapper/1320:25:415
727799276,3cyclictest21004-21grep20:09:499
727799272,3cyclictest25107-21gpgconf20:54:389
727799271,3cyclictest28514-21grep20:55:069
60652270,0sleep150-21swapper/1522:57:157
269572270,1sleep10-21swapper/121:16:101
269572270,1sleep10-21swapper/121:16:101
107562270,1sleep50-21swapper/523:41:1411
102292270,1sleep910226-21systemd-cgroups21:53:4515
98152263,19sleep21030-21polkitd00:21:348
729099268,8cyclictest122-21ksoftirqd/1323:54:375
729099268,8cyclictest122-21ksoftirqd/1323:54:375
7290992617,5cyclictest122-21ksoftirqd/1300:16:285
727799264,3cyclictest16346-21latency_hist20:44:379
727799261,5cyclictest0-21swapper/320:15:009
300072260,22sleep430301-21cut23:39:5210
277982260,3sleep727918-21systemd-cgroups22:49:0613
252102260,1sleep80-21swapper/800:04:4914
252102260,1sleep80-21swapper/800:04:4914
239402260,22sleep00-21swapper/022:30:320
203982260,1sleep90-21swapper/900:22:4415
19452260,2sleep140-21swapper/1421:52:476
190582260,22sleep60-21swapper/600:11:4412
172632260,2sleep00-21swapper/023:56:390
172632260,2sleep00-21swapper/023:56:390
171962260,0sleep70-21swapper/722:36:4713
146002260,1sleep20-21swapper/223:02:078
88952250,21sleep110-21swapper/1121:57:013
729299250,23cyclictest0-21swapper/1420:39:496
729099256,9cyclictest0-21swapper/1322:54:025
729099255,12cyclictest122-21ksoftirqd/1322:35:055
7290992521,3cyclictest0-21swapper/1321:10:145
7290992521,3cyclictest0-21swapper/1321:10:145
7290992521,3cyclictest0-21swapper/1320:45:185
7290992521,3cyclictest0-21swapper/1320:40:445
7290992520,3cyclictest0-21swapper/1320:34:065
7287992521,2cyclictest93050irq/38-eth0-TxR21:29:402
7287992521,2cyclictest93050irq/38-eth0-TxR21:29:402
727799255,3cyclictest17534-21missed_timers19:24:559
727799253,2cyclictest5039-21nfsd19:29:579
727799252,5cyclictest503550nfsd19:50:529
56162250,1sleep6641rcuc/623:59:0912
56162250,1sleep6641rcuc/623:59:0912
315382250,21sleep90-21swapper/923:43:4215
284542250,2sleep628591-21systemd22:38:0712
28442250,1sleep20-21swapper/222:49:488
217752250,2sleep20-21swapper/221:29:508
217752250,2sleep20-21swapper/221:29:508
199762250,22sleep70-21swapper/722:40:4913
180912250,2sleep111051ktimersoftd/1122:51:183
174152250,21sleep10-21swapper/123:56:401
174152250,21sleep10-21swapper/123:56:401
117362250,0sleep140-21swapper/1421:32:266
117362250,0sleep140-21swapper/1421:32:266
110892250,5sleep4727899cyclictest21:39:3010
7293992419,3cyclictest6168-21run-parts23:33:317
729099246,11cyclictest122-21ksoftirqd/1323:20:065
729099244,12cyclictest0-21swapper/1323:42:325
7290992420,3cyclictest0-21swapper/1320:22:545
7290992420,2cyclictest1201rcuc/1323:08:445
7290992417,5cyclictest0-21swapper/1322:16:305
728199246,8cyclictest0-21swapper/600:24:2012
7279992417,4cyclictest9158-21systemd23:15:2511
727899249,10cyclictest131rcu_preempt23:16:3810
727899247,7cyclictest131rcu_preempt00:04:4410
727899247,7cyclictest131rcu_preempt00:04:4410
727799245,3cyclictest1962-21cpuspeed21:04:489
727799243,3cyclictest32599-21cpuspeed_turbos19:44:529
727799240,2cyclictest0-21swapper/319:14:569
72132240,21sleep110-21swapper/1123:19:243
48972240,2sleep00-21swapper/023:11:270
43642240,1sleep110-21swapper/1123:51:133
43642240,1sleep110-21swapper/1123:51:133
325512240,1sleep7721rcuc/722:17:1513
256772240,2sleep00-21swapper/022:52:120
203742240,1sleep10-21swapper/121:51:141
189072240,1sleep120-21swapper/1222:05:034
18282240,1sleep20-21swapper/219:45:018
106082240,1sleep40-21swapper/423:01:3710
90002230,1sleep10-21swapper/122:00:191
78542230,1sleep80-21swapper/821:46:0014
729099237,9cyclictest122-21ksoftirqd/1300:32:365
729099237,6cyclictest8541-21sshd00:36:065
729099236,10cyclictest122-21ksoftirqd/1322:21:125
729099235,7cyclictest0-21swapper/1322:34:295
729099235,17cyclictest1211ktimersoftd/1300:25:455
729099234,11cyclictest122-21ksoftirqd/1322:45:115
7290992319,2cyclictest93150irq/39-eth0-TxR23:29:375
728199237,9cyclictest66-21ksoftirqd/621:57:5612
728199236,7cyclictest0-21swapper/621:53:5212
728199235,16cyclictest66-21ksoftirqd/622:48:1612
7281992319,3cyclictest0-21swapper/623:12:3812
7281992315,2cyclictest66-21ksoftirqd/622:09:2512
7281992313,5cyclictest141rcu_sched23:02:0012
7281992312,6cyclictest66-21ksoftirqd/623:24:0712
7281992312,6cyclictest131rcu_preempt21:45:2812
728199230,4cyclictest0-21swapper/600:39:4212
727899235,10cyclictest49-21ksoftirqd/400:25:4710
7278992316,2cyclictest131rcu_preempt00:01:5410
7278992314,5cyclictest131rcu_preempt22:56:4910
727899230,4cyclictest1055-21in:imuxsock23:55:5810
727899230,4cyclictest1055-21in:imuxsock23:55:5810
727799231,4cyclictest0-21swapper/319:54:489
68882230,1sleep20-21swapper/200:17:498
61172230,0sleep00-21swapper/020:29:540
312632236,13sleep231371-21sshd22:31:228
270822230,21sleep60-21swapper/619:35:1212
223192230,0sleep150-21swapper/1521:26:027
223192230,0sleep150-21swapper/1521:26:027
130052230,20sleep110-21swapper/1122:18:413
128722230,1sleep110-21swapper/1120:39:433
118032230,0sleep20-21swapper/200:40:008
111152230,19sleep150-21swapper/1522:21:577
104252230,19sleep80-21swapper/800:32:5414
729099228,8cyclictest122-21ksoftirqd/1323:36:415
729099227,8cyclictest122-21ksoftirqd/1322:04:455
729099225,9cyclictest122-21ksoftirqd/1323:09:495
729099224,9cyclictest0-21swapper/1323:29:555
729099223,10cyclictest0-21swapper/1322:40:535
7290992219,2cyclictest0-21swapper/1320:57:285
7290992218,3cyclictest0-21swapper/1321:04:395
7290992218,2cyclictest93150irq/39-eth0-TxR23:56:365
7290992218,2cyclictest93150irq/39-eth0-TxR23:56:365
7290992218,2cyclictest1201rcuc/1300:01:235
7290992216,5cyclictest0-21swapper/1323:18:455
728799220,19cyclictest9042-21sshd21:21:102
728799220,19cyclictest9042-21sshd21:21:102
728399220,20cyclictest12232-21cups-browsed22:58:5814
728399220,20cyclictest0-21swapper/822:52:4114
728199225,9cyclictest509-21systemd-journal22:03:1112
728199225,2cyclictest6675-21systemd-cgroups23:08:1612
728199225,10cyclictest66-21ksoftirqd/623:45:4612
728199225,10cyclictest66-21ksoftirqd/623:45:4612
728199224,9cyclictest2491-21systemd-cgroups22:31:4512
728199224,11cyclictest66-21ksoftirqd/621:21:0012
728199224,11cyclictest66-21ksoftirqd/621:21:0012
7281992218,2cyclictest651ktimersoftd/621:39:3112
7281992215,4cyclictest30187-21run-parts22:09:5912
7281992212,5cyclictest131rcu_preempt22:58:0112
7281992211,6cyclictest141rcu_sched23:53:0312
7281992211,6cyclictest141rcu_sched23:53:0312
7281992211,6cyclictest131rcu_preempt23:30:2912
728199220,4cyclictest26578-21kworker/6:023:17:0812
728199220,19cyclictest5039-21nfsd22:15:1012
727999220,20cyclictest0-21swapper/520:39:5611
727999220,19cyclictest0-21swapper/521:54:5911
7278992212,6cyclictest131rcu_preempt23:22:4410
7278992212,5cyclictest49-21ksoftirqd/400:22:4510
7278992212,5cyclictest131rcu_preempt21:30:5510
7278992212,5cyclictest131rcu_preempt21:30:5510
7278992212,3cyclictest49-21ksoftirqd/421:18:3010
7278992212,3cyclictest49-21ksoftirqd/421:18:3010
7278992211,7cyclictest131rcu_preempt21:45:2910
7276992216,4cyclictest12230-21systemd22:11:408
727699220,20cyclictest23635-21kworker/2:122:06:198
7271992215,4cyclictest10793-21run-parts23:48:290
7271992215,4cyclictest10793-21run-parts23:48:290
71492220,2sleep110-21swapper/1123:26:213
49782220,0sleep40-21swapper/422:07:1910
292732220,18sleep21-21systemd00:34:568
289872220,1sleep00-21swapper/023:28:500
281482220,5sleep12114-21ksoftirqd/1221:19:504
281482220,5sleep12114-21ksoftirqd/1221:19:504
192322220,2sleep90-21swapper/923:31:0815
113792220,1sleep1511382-21env00:03:197
110432221,18sleep911178-21systemd-cgroups23:37:4915
73782210,1sleep60-21swapper/622:53:5412
729399210,19cyclictest6984-21sshd22:32:157
729299211,3cyclictest1-21systemd23:08:116
7292992113,3cyclictest12460-21sshd21:35:536
729099218,8cyclictest122-21ksoftirqd/1323:00:465
729099216,8cyclictest122-21ksoftirqd/1322:10:195
729099214,9cyclictest0-21swapper/1300:09:545
729099213,4cyclictest1211ktimersoftd/1300:19:415
7290992118,2cyclictest0-21swapper/1320:14:595
7290992117,3cyclictest0-21swapper/1319:44:275
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional