You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-15 - 14:35
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackcslot0.osadl.org (updated Tue Jul 15, 2025 00:43:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1160421740,6sleep52888499cyclictest22:52:2911
1160421740,6sleep52888499cyclictest22:52:2911
415521540,5sleep142889799cyclictest22:40:526
1333021510,6sleep02887799cyclictest22:26:590
1821621360,0sleep20-21swapper/222:34:478
2980821310,1sleep30-21swapper/321:59:459
3075121290,1sleep50-21swapper/521:34:5511
283162123103,16sleep10-21swapper/119:09:401
283172121104,11sleep20-21swapper/219:09:418
2580621190,0sleep120-21swapper/1223:12:074
2580621190,0sleep120-21swapper/1223:12:074
28326211693,19sleep90-21swapper/919:09:4815
282732116104,8sleep130-21swapper/1319:09:045
28271211396,12sleep110-21swapper/1119:09:023
28078211191,16sleep00-21swapper/019:06:270
28320210993,8sleep30-21swapper/319:09:429
28083210790,13sleep50-21swapper/519:06:3211
2952221060,1sleep142889799cyclictest00:12:206
28176210694,8sleep120-21swapper/1219:07:434
2169921060,1sleep150-21swapper/1523:11:387
2169921060,1sleep150-21swapper/1523:11:387
666721040,1sleep150-21swapper/1522:44:537
3025921010,4sleep12114-21ksoftirqd/1223:42:214
2826721010,2sleep1428264-21systemd-cgroups00:35:366
12272980,0sleep140-21swapper/1421:52:596
2832129777,16sleep40-21swapper/419:09:4310
279852970,1sleep130-21swapper/1323:08:565
279852970,1sleep130-21swapper/1323:08:565
134072960,2sleep113375-21sshd21:39:561
46772950,1sleep40-21swapper/400:13:1410
48002940,1sleep90-21swapper/920:36:0915
48002940,1sleep90-21swapper/920:36:0915
22412930,1sleep1241ktimersoftd/123:57:241
213102930,6sleep132889699cyclictest22:10:055
286992910,4sleep12887899cyclictest23:30:561
254062910,2sleep1325385-21sshd23:48:475
2820929072,4sleep60-21swapper/619:08:0912
228512900,2sleep125-21ksoftirqd/122:53:451
228512900,2sleep125-21ksoftirqd/122:53:451
157892900,1sleep111-21systemd22:45:543
100362900,1sleep30-21swapper/322:59:569
100362900,1sleep30-21swapper/322:59:569
16712890,1sleep110-21swapper/1121:10:573
2827428872,12sleep140-21swapper/1419:09:056
71662870,1sleep120-21swapper/1219:21:044
169772870,1sleep10-21swapper/123:44:341
2832728667,14sleep100-21swapper/1019:09:492
2840228472,8sleep150-21swapper/1519:10:107
2826728375,4sleep80-21swapper/819:08:5914
219592830,1sleep120-21swapper/1221:19:474
215202830,1sleep80-21swapper/822:13:2914
2497228072,4sleep70-21swapper/719:05:4613
179502790,0sleep90-21swapper/900:23:1315
261702770,1sleep125-21ksoftirqd/123:26:551
261722760,0sleep100-21swapper/1022:54:102
261722760,0sleep100-21swapper/1022:54:102
307202730,2sleep92888999cyclictest23:16:2115
194602720,0sleep10-21swapper/100:15:521
204032630,1sleep30-21swapper/321:51:289
307402610,2sleep125-21ksoftirqd/123:20:301
275962610,1sleep131211ktimersoftd/1322:39:475
190812590,0sleep110-21swapper/1100:15:463
205892560,6sleep82888899cyclictest21:37:0614
52662520,0sleep20-21swapper/200:21:468
2888299500,3cyclictest1068-21dbus-daemon21:45:279
247592500,17sleep696450irq/42-eth0-TxR22:10:3012
2887799491,45cyclictest525-21systemd-journal22:41:500
2887799490,1cyclictest0-21swapper/021:36:170
1139099490,2rtkit-daemon11389-21rtkit-daemon22:44:304
2887799482,43cyclictest1-21systemd21:55:130
2887799480,2cyclictest0-21swapper/023:25:190
2887799480,1cyclictest0-21swapper/023:52:110
2887799473,36cyclictest1245-21gdbus21:30:450
2887799471,2cyclictest5467-21gdbus22:59:120
2887799471,2cyclictest5467-21gdbus22:59:120
2887799471,2cyclictest1245-21gdbus22:31:340
2887799470,44cyclictest12339-21sshd22:23:220
2887799470,1cyclictest0-21swapper/000:01:110
1139099470,33rtkit-daemon525-21systemd-journal22:38:3010
2888299461,3cyclictest1068-21dbus-daemon22:12:039
2888299460,4cyclictest20680-21uname22:38:519
2888299460,2cyclictest0-21swapper/322:16:279
2887799462,41cyclictest525-21systemd-journal00:34:150
2887799460,43cyclictest20770-21sshd00:06:260
2887799460,3cyclictest1-21systemd00:10:550
2887799460,2cyclictest1351-21nfsd00:25:430
2887799460,2cyclictest0-21swapper/023:07:510
2887799460,2cyclictest0-21swapper/023:07:510
2887799460,2cyclictest0-21swapper/022:04:390
2887799460,2cyclictest0-21swapper/021:16:550
2887799460,1cyclictest0-21swapper/022:13:590
2887799460,1cyclictest0-21swapper/021:26:560
2888299450,3cyclictest25335-21sshd22:28:239
2888299450,3cyclictest1069-21NetworkManager22:33:359
2887799452,40cyclictest1238-21gdbus21:41:430
2887799450,3cyclictest1073-21systemd-logind23:35:510
2887799450,2cyclictest1068-21dbus-daemon21:22:550
2887799450,2cyclictest0-21swapper/023:43:350
2887799450,2cyclictest0-21swapper/023:19:590
2887799450,2cyclictest0-21swapper/022:35:500
2887799450,2cyclictest0-21swapper/021:13:360
2887799450,1cyclictest0-21swapper/023:46:430
2887799450,1cyclictest0-21swapper/021:46:270
2887799450,1cyclictest0-21swapper/019:16:510
2888299448,33cyclictest1245-21gdbus22:45:449
2888299446,5cyclictest13814-21gdbus22:03:299
2888299440,3cyclictest14407-21uname23:22:039
2887799443,37cyclictest13814-21gdbus22:16:110
2887799440,2cyclictest0-21swapper/023:14:220
2887799440,2cyclictest0-21swapper/023:14:220
2887799440,2cyclictest0-21swapper/023:01:110
2887799440,2cyclictest0-21swapper/023:01:110
2887799440,2cyclictest0-21swapper/022:47:230
2887799440,2cyclictest0-21swapper/000:24:230
2887799440,2cyclictest0-21swapper/000:19:580
2887799440,1cyclictest0-21swapper/023:35:070
2887799440,1cyclictest0-21swapper/022:51:500
2887799440,1cyclictest0-21swapper/022:51:500
2887799440,1cyclictest0-21swapper/022:07:190
2887799440,1cyclictest0-21swapper/021:58:270
2887799440,1cyclictest0-21swapper/021:01:150
2888299435,9cyclictest0-21swapper/322:07:369
2888299430,4cyclictest17192-21systemd22:23:599
2888299430,3cyclictest5467-21gdbus21:19:119
2888299430,3cyclictest3878-21sshd21:11:039
2887799430,4cyclictest24847-21uname23:56:110
2887799430,2cyclictest0-21swapper/023:26:510
1139099430,1rtkit-daemon0-21swapper/421:22:1910
2888299420,4cyclictest26837-21sshd22:50:439
2888299420,4cyclictest26837-21sshd22:50:439
2888299420,3cyclictest525-21systemd-journal00:35:109
2888299420,3cyclictest23361-21systemd-cgroups00:27:119
2888299420,3cyclictest1068-21dbus-daemon23:19:519
2888299420,3cyclictest1068-21dbus-daemon21:40:039
2888299420,39cyclictest6671-21systemd00:05:119
2887799422,38cyclictest31951-21processes19:51:080
2887799420,1cyclictest0-21swapper/020:01:190
2887799420,1cyclictest0-21swapper/019:35:470
2887799420,1cyclictest0-21swapper/019:25:230
2888299418,30cyclictest26151-21run-parts00:20:439
2888299418,30cyclictest1-21systemd23:53:019
2888299413,3cyclictest1068-21dbus-daemon23:02:279
2888299413,3cyclictest1068-21dbus-daemon23:02:279
2888299410,3cyclictest14456-21grep19:30:599
2888299410,3cyclictest1068-21dbus-daemon21:25:319
2888299410,38cyclictest11881-21systemd00:37:069
2888299410,2cyclictest0-21swapper/323:30:159
2887799410,2cyclictest0-21swapper/020:40:560
2887799410,2cyclictest0-21swapper/020:40:560
2887799410,2cyclictest0-21swapper/000:37:270
2887799410,1cyclictest0-21swapper/020:16:470
2887799410,1cyclictest0-21swapper/019:58:070
258052410,2sleep7721rcuc/721:30:5613
55682400,14sleep40-21swapper/400:21:4810
2888299409,28cyclictest1-21systemd23:32:119
2888299408,30cyclictest19612-21sshd23:08:019
2888299408,30cyclictest19612-21sshd23:08:019
2888299407,30cyclictest1068-21dbus-daemon23:14:299
2888299407,30cyclictest1068-21dbus-daemon23:14:299
2888299405,3cyclictest13665-21sshd00:19:249
2888299403,3cyclictest1068-21dbus-daemon00:10:459
2888299402,36cyclictest7853-21systemd23:40:059
2888299400,4cyclictest1068-21dbus-daemon23:55:439
2888299400,3cyclictest5467-21gdbus23:45:359
2888299400,3cyclictest1068-21dbus-daemon21:25:559
2887799400,1cyclictest0-21swapper/021:06:070
2887799400,1cyclictest0-21swapper/020:52:510
2887799400,1cyclictest0-21swapper/020:31:310
2887799400,1cyclictest0-21swapper/020:31:310
2887799400,1cyclictest0-21swapper/020:22:470
2887799400,1cyclictest0-21swapper/020:22:470
2887799400,1cyclictest0-21swapper/020:06:110
2887799400,1cyclictest0-21swapper/019:45:590
2887799400,1cyclictest0-21swapper/019:40:430
2887799400,1cyclictest0-21swapper/019:31:070
1139099400,16rtkit-daemon0-21swapper/1223:04:284
1139099400,16rtkit-daemon0-21swapper/1223:04:284
28888993913,4cyclictest11243-21sh23:54:4814
2888299395,5cyclictest5122-21uname22:40:559
2888299394,3cyclictest1090-21sshd21:31:359
2887799390,1cyclictest0-21swapper/020:37:190
2887799390,1cyclictest0-21swapper/020:37:190
2887799390,1cyclictest0-21swapper/020:26:030
2887799390,1cyclictest0-21swapper/020:26:030
2887799390,1cyclictest0-21swapper/019:25:590
2887799390,1cyclictest0-21swapper/019:13:590
2888299388,27cyclictest1-21systemd21:48:399
2888299383,3cyclictest1068-21dbus-daemon00:09:159
2888299380,3cyclictest1068-21dbus-daemon23:47:479
2887799380,1cyclictest0-21swapper/020:56:070
2887799380,1cyclictest0-21swapper/020:46:110
145152380,1sleep141291ktimersoftd/1421:22:146
302032370,1sleep530205-21run-parts00:04:1011
215582370,1sleep921555-21run-parts21:19:4415
1139099370,2rtkit-daemon2445-21sshd22:58:5910
1139099370,2rtkit-daemon2445-21sshd22:58:5910
1139099370,0rtkit-daemon0-21swapper/400:38:3010
302572360,1sleep70-21swapper/723:31:0213
2888299361,3cyclictest0-21swapper/320:46:029
2887799360,1cyclictest0-21swapper/020:11:110
249152360,1sleep70-21swapper/722:10:3113
2888299352,3cyclictest1348-21nfsd19:20:559
2888299351,3cyclictest0-21swapper/319:51:149
2888299351,3cyclictest0-21swapper/319:26:059
2887899356,23cyclictest0-21swapper/121:19:221
2887899350,28cyclictest0-21swapper/123:50:221
33842340,2sleep93383-21bash21:28:3415
2888899341,15cyclictest1068-21dbus-daemon21:51:0514
2888299343,3cyclictest2031-21fschecks_count19:15:599
2888299340,31cyclictest32405-21sshd19:12:429
2887899346,19cyclictest5467-21gdbus00:00:531
2887899344,23cyclictest1069-21NetworkManager23:53:221
229422340,2sleep11106-21ksoftirqd/1123:45:163
165372340,1sleep50-21swapper/522:27:2011
94482330,1sleep40-21swapper/422:33:4410
2888299335,3cyclictest2819-21cpuspeed_turbos20:35:579
2888299335,3cyclictest2819-21cpuspeed_turbos20:35:579
2888299331,3cyclictest0-21swapper/321:06:039
2888299331,2cyclictest0-21swapper/321:01:059
2887899332,25cyclictest0-21swapper/121:45:211
2888299326,23cyclictest1282-21snmpd20:56:179
2888299323,3cyclictest19977-21timerandwakeup19:36:119
2888299321,3cyclictest0-21swapper/319:47:479
2888299321,2cyclictest0-21swapper/320:15:409
2888299320,3cyclictest7447-21cstates20:40:579
2888299320,3cyclictest7447-21cstates20:40:579
2887899327,18cyclictest1068-21dbus-daemon22:23:221
2887899325,20cyclictest0-21swapper/122:37:391
2887899322,28cyclictest0-21swapper/123:14:331
2887899322,28cyclictest0-21swapper/123:14:331
2887899322,23cyclictest0-21swapper/121:28:531
2887899321,28cyclictest0-21swapper/121:24:031
26352320,0sleep40-21swapper/421:28:2910
166332320,0sleep80-21swapper/800:37:3914
1139099320,1rtkit-daemon1-21systemd21:45:3410
1139099320,0rtkit-daemon0-21swapper/423:15:0110
1139099320,0rtkit-daemon0-21swapper/423:15:0110
78662310,27sleep77963-21sshd22:04:5513
324152310,7sleep72888699cyclictest22:11:0313
30632310,0sleep50-21swapper/522:07:4911
2888299313,3cyclictest31199-21cstates20:30:559
2888299313,3cyclictest31199-21cstates20:30:559
2888299311,3cyclictest15730-21df_inode20:50:589
2888299310,4cyclictest10948-21tune2fs20:05:569
2888299310,3cyclictest22329-21expr19:40:569
2887899313,18cyclictest0-21swapper/123:02:391
2887899313,18cyclictest0-21swapper/123:02:391
2887899312,20cyclictest1804-21systemd-cgroups23:39:191
2887899310,6cyclictest0-21swapper/122:32:531
2887899310,23cyclictest0-21swapper/121:53:581
253992310,0sleep140-21swapper/1423:01:276
253992310,0sleep140-21swapper/1423:01:276
153112310,1sleep60-21swapper/600:22:5412
1139099310,2rtkit-daemon3127-21munin-node20:32:404
1139099310,2rtkit-daemon3127-21munin-node20:32:404
107802310,1sleep60-21swapper/621:11:4012
76542300,1sleep97652-21systemd-cgroups23:03:1515
76542300,1sleep97652-21systemd-cgroups23:03:1515
2888299301,3cyclictest0-21swapper/320:25:589
2888299301,3cyclictest0-21swapper/320:25:589
2888299301,3cyclictest0-21swapper/320:00:579
2888299301,2cyclictest0-21swapper/320:15:579
2887899304,20cyclictest0-21swapper/100:38:071
2887899303,18cyclictest0-21swapper/121:32:301
2887899302,22cyclictest13814-21gdbus23:08:321
2887899302,22cyclictest13814-21gdbus23:08:321
2887899302,22cyclictest0-21swapper/122:43:251
2887899301,18cyclictest5251systemd-journal22:46:121
2887899300,22cyclictest0-21swapper/123:00:351
2887899300,22cyclictest0-21swapper/123:00:351
2887899300,19cyclictest0-21swapper/121:56:591
27962300,1sleep150-21swapper/1521:46:037
1139099300,3rtkit-daemon3127-21munin-node21:14:074
2888299291,3cyclictest23729-21perl20:20:579
2888299291,3cyclictest23729-21perl20:20:579
2888299291,3cyclictest1282-21snmpd19:56:059
2887899293,24cyclictest0-21swapper/121:48:131
2887899291,20cyclictest13814-21gdbus23:22:461
2887899291,19cyclictest0-21swapper/100:31:521
2887899291,17cyclictest1068-21dbus-daemon22:28:391
28878992911,15cyclictest1069-21NetworkManager21:12:181
2887899290,21cyclictest27043-21sshd00:24:141
28422291,2sleep20-21swapper/223:53:428
178692290,1sleep80-21swapper/823:59:2514
143642290,25sleep140-21swapper/1423:40:476
4922280,5sleep92888999cyclictest21:52:5415
312222280,1sleep70-21swapper/723:27:3413
2889499285,21cyclictest1051ktimersoftd/1121:49:183
2887899282,24cyclictest0-21swapper/122:01:481
2887899281,19cyclictest1073-21systemd-logind00:06:581
2887899280,20cyclictest8621-21systemd-cgroups00:25:461
2887899280,20cyclictest0-21swapper/122:11:311
271192280,1sleep40-21swapper/423:45:4210
194472280,1sleep11106-21ksoftirqd/1123:30:013
155732280,0sleep140-21swapper/1423:29:326
1139099280,1rtkit-daemon3842-21systemd22:11:2510
1139099280,1rtkit-daemon1-21systemd00:29:2310
67702270,1sleep120-21swapper/1222:18:554
2889499276,14cyclictest256891sd-pam21:59:163
2889499276,11cyclictest106-21ksoftirqd/1122:44:223
2889499275,10cyclictest0-21swapper/1122:14:473
28894992722,3cyclictest1041rcuc/1123:17:333
28894992720,5cyclictest0-21swapper/1121:28:593
2888499277,14cyclictest131rcu_preempt23:55:4311
2887899273,18cyclictest265431sd-pam00:11:561
2887899271,19cyclictest1068-21dbus-daemon22:06:401
166312270,2sleep13122-21ksoftirqd/1321:51:065
111322270,2sleep110-21swapper/1100:33:253
93532260,2sleep29348-21cp21:29:158
61822260,1sleep80-21swapper/800:21:5314
2889699260,24cyclictest0-21swapper/1300:15:535
28878992610,13cyclictest1068-21dbus-daemon22:17:221
163702260,2sleep100-21swapper/1022:42:092
154382260,2sleep151361rcuc/1500:02:277
133142260,1sleep40-21swapper/423:55:0310
126132260,2sleep120-21swapper/1221:57:474
87612250,0sleep140-21swapper/1421:29:116
77182250,2sleep150-21swapper/1500:32:587
64112250,1sleep140-21swapper/1423:35:586
34202250,2sleep101-21systemd22:48:022
324312250,1sleep60-21swapper/621:31:2712
2889499256,10cyclictest5464-21packagekitd23:37:013
2889499255,9cyclictest0-21swapper/1123:59:173
2889499254,11cyclictest106-21ksoftirqd/1100:06:503
28894992521,2cyclictest1041rcuc/1123:35:243
28894992516,7cyclictest96050irq/38-eth0-TxR22:36:293
2888899251,3cyclictest0-21swapper/800:32:4514
2888499253,19cyclictest58-21ksoftirqd/521:49:1811
2888499250,8cyclictest0-21swapper/522:05:3911
2888499250,16cyclictest0-21swapper/521:53:0711
2887899258,15cyclictest0-21swapper/121:01:061
2887899256,17cyclictest0-21swapper/120:23:561
2887899256,17cyclictest0-21swapper/120:23:561
203252250,1sleep130-21swapper/1320:56:005
176442250,1sleep120-21swapper/1223:18:494
10252250,3sleep50-21swapper/523:24:1911
91672240,0sleep40-21swapper/422:08:3610
7362240,2sleep140-21swapper/1422:36:206
306512240,1sleep120-21swapper/1221:10:114
301452240,1sleep50-21swapper/523:16:1811
291102240,1sleep20-21swapper/223:30:578
2889499249,9cyclictest131rcu_preempt00:40:063
2889499248,14cyclictest1041rcuc/1100:12:533
2889499246,9cyclictest0-21swapper/1122:25:483
2889499246,9cyclictest0-21swapper/1100:26:023
2889499246,11cyclictest106-21ksoftirqd/1123:05:423
2889499246,11cyclictest106-21ksoftirqd/1123:05:423
2889499245,8cyclictest0-21swapper/1122:23:023
2889499245,18cyclictest106-21ksoftirqd/1121:53:383
2889499244,11cyclictest0-21swapper/1123:00:063
2889499244,11cyclictest0-21swapper/1123:00:063
2889499243,12cyclictest0-21swapper/1121:18:373
28894992421,2cyclictest1041rcuc/1122:18:173
28894992419,4cyclictest106-21ksoftirqd/1121:40:053
28894992412,6cyclictest131rcu_preempt21:31:173
2889499240,18cyclictest106-21ksoftirqd/1122:09:093
2888499246,10cyclictest58-21ksoftirqd/523:08:2911
2888499246,10cyclictest58-21ksoftirqd/523:08:2911
28884992420,2cyclictest561rcuc/522:41:0311
28884992414,5cyclictest131rcu_preempt23:42:3811
28884992411,9cyclictest131rcu_preempt22:34:0011
28884992411,7cyclictest58-21ksoftirqd/521:19:0611
2888499240,19cyclictest58-21ksoftirqd/522:59:0311
2888499240,19cyclictest58-21ksoftirqd/522:59:0311
2887899247,15cyclictest0-21swapper/119:19:531
2887899246,16cyclictest0-21swapper/120:00:471
2887899245,17cyclictest0-21swapper/120:51:041
2887899245,17cyclictest0-21swapper/119:11:061
2887899243,19cyclictest0-21swapper/120:40:421
2887899243,19cyclictest0-21swapper/120:40:421
2887899243,19cyclictest0-21swapper/120:25:591
2887899243,19cyclictest0-21swapper/120:25:591
267802240,0sleep80-21swapper/821:34:2614
237782240,20sleep923885-21systemd22:50:2515
227922240,1sleep222790-21sshd21:19:538
195512240,2sleep110-21swapper/1123:15:203
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional