You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-12 - 15:14
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackcslot0.osadl.org (updated Thu Jul 11, 2024 00:43:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
581521530,6sleep131472099cyclictest21:52:335
141112130103,23sleep50-21swapper/519:11:4311
141622124103,17sleep150-21swapper/1519:12:257
2192921170,0sleep150-21swapper/1500:35:447
2192921170,0sleep150-21swapper/1500:35:447
13959211791,21sleep00-21swapper/019:09:460
411921160,2sleep50-21swapper/500:18:2711
411921160,2sleep50-21swapper/500:18:2711
13944211694,17sleep20-21swapper/219:09:328
14079211492,17sleep80-21swapper/819:11:1414
13962211493,16sleep30-21swapper/319:09:499
14116211291,16sleep100-21swapper/1019:11:482
14177211193,14sleep120-21swapper/1219:12:384
2696421100,2sleep50-21swapper/500:24:5211
2696421100,2sleep50-21swapper/500:24:5211
14189210492,7sleep70-21swapper/719:12:4913
545121030,6sleep121471899cyclictest21:22:344
2994721010,6sleep141472199cyclictest23:42:146
14110210180,16sleep40-21swapper/419:11:4210
7392980,1sleep140-21swapper/1423:30:436
283212970,1sleep00-21swapper/023:41:590
299202950,1sleep1029918-21systemd-cgroups21:58:392
196892950,1sleep100-21swapper/1021:35:232
79472940,2sleep10998-21dbus-daemon21:19:042
67432940,0sleep20-21swapper/223:35:048
154682940,6sleep81471199cyclictest22:19:1314
10382940,1sleep30-21swapper/323:07:489
196462930,0sleep100-21swapper/1023:40:492
314202920,0sleep120-21swapper/1223:34:104
238092920,1sleep80-21swapper/800:11:4914
238092920,1sleep80-21swapper/800:11:4914
257002900,1sleep10-21swapper/122:32:281
1401429082,4sleep10-21swapper/119:10:191
81292890,2sleep80-21swapper/800:06:1214
239252890,7sleep21470299cyclictest23:25:598
1392028976,9sleep130-21swapper/1319:09:105
164382880,0sleep140-21swapper/1422:34:496
1419628876,8sleep140-21swapper/1419:12:566
1426128772,10sleep90-21swapper/919:13:0715
204512860,2sleep1120450-21sshd23:52:203
86792850,2sleep40-21swapper/422:45:3710
17802850,0sleep50-21swapper/521:14:5811
1397528472,8sleep110-21swapper/1119:09:573
68692830,0sleep30-21swapper/323:35:059
252582820,0sleep20-21swapper/223:13:568
34272810,0sleep20-21swapper/200:37:408
34272810,0sleep20-21swapper/200:37:408
1411228072,4sleep60-21swapper/619:11:4412
132992800,1sleep90-21swapper/922:34:2815
90492790,1sleep130-21swapper/1323:24:015
261322790,1sleep00-21swapper/023:18:110
91012780,0sleep150-21swapper/1521:26:227
280722760,6sleep91471399cyclictest00:28:4115
280722760,6sleep91471399cyclictest00:28:4115
152452750,7sleep11470199cyclictest00:42:591
177952660,6sleep31470399cyclictest22:04:419
15182640,0sleep100-21swapper/1023:27:112
1472099471,3cyclictest1005-21NetworkManager00:07:425
1472099470,3cyclictest3203-21systemd22:07:025
1472099470,3cyclictest1-21systemd23:04:425
1472099470,3cyclictest1076-21gdbus00:36:545
1472099470,3cyclictest1076-21gdbus00:36:545
1472099470,2cyclictest0-21swapper/1323:17:545
312852460,0sleep110-21swapper/1121:47:583
1472099461,3cyclictest8037-21systemd22:33:545
1472099460,3cyclictest16924-21/usr/sbin/munin00:23:425
1472099460,3cyclictest16924-21/usr/sbin/munin00:23:425
1472099460,3cyclictest0-21swapper/1300:29:265
1472099460,3cyclictest0-21swapper/1300:29:265
1472099457,35cyclictest7349-2110-uname22:22:295
1472099451,41cyclictest509-21systemd-journal21:31:535
1472099450,4cyclictest27695-21systemd21:21:225
1472099450,3cyclictest1433-21systemd21:55:305
1472099450,3cyclictest10961-21sshd22:57:065
1472099450,3cyclictest1030-21polkitd23:44:505
1472099450,2cyclictest0-21swapper/1300:42:145
158172440,1sleep50-21swapper/500:43:0311
1472099443,38cyclictest1-21systemd23:50:535
1472099440,4cyclictest30094-21systemd22:59:065
1472099440,3cyclictest29651-21systemd-cgroups00:08:385
1472099440,3cyclictest29651-21systemd-cgroups00:08:385
1472099440,3cyclictest18220-21systemctl21:46:145
1472099440,3cyclictest0-21swapper/1300:17:185
1472099440,3cyclictest0-21swapper/1300:17:185
1472099440,2cyclictest0-21swapper/1322:01:065
1472099437,33cyclictest28449-21sshd22:09:505
1472099435,35cyclictest1076-21gdbus23:54:545
1472099430,3cyclictest6778-21systemd-cgroups21:33:545
1472099430,3cyclictest25083-21systemd00:00:465
1472099430,3cyclictest1-21systemd22:32:345
1472099430,3cyclictest1076-21gdbus23:19:145
1472099430,2cyclictest0-21swapper/1322:48:465
1472099430,2cyclictest0-21swapper/1322:25:545
1472099429,30cyclictest25782-21sshd23:10:255
1472099427,33cyclictest741-21sshd23:38:305
1472099420,4cyclictest998-21dbus-daemon00:22:505
1472099420,4cyclictest998-21dbus-daemon00:22:505
1472099420,4cyclictest29864-21sshd23:30:185
1472099420,3cyclictest509-21systemd-journal21:17:305
1472099420,3cyclictest18695-21sshd22:42:545
1472099420,3cyclictest12352-21sshd22:15:225
1472099420,3cyclictest1-21systemd22:44:425
1472099410,39cyclictest5041-21nfsd23:34:305
1472099410,38cyclictest21704-21sshd21:24:045
1472099400,38cyclictest32352-21sshd21:40:205
80982390,0sleep00-21swapper/022:37:480
106272390,1sleep6998-21dbus-daemon00:38:3212
18362380,1sleep150-21swapper/1500:29:197
18362380,1sleep150-21swapper/1500:29:197
43602370,0sleep140-21swapper/1421:48:336
1472099371,3cyclictest25721-21nfsd419:23:465
1472099371,3cyclictest0-21swapper/1319:48:435
1472099371,3cyclictest0-21swapper/1319:48:435
1472099370,2cyclictest0-21swapper/1320:33:305
297752360,6sleep121471899cyclictest00:21:314
297752360,6sleep121471899cyclictest00:21:314
237412360,4sleep91471399cyclictest21:43:0715
225862360,0sleep00-21swapper/021:54:120
1472099365,3cyclictest17033-21fschecks_time19:53:365
1472099365,3cyclictest17033-21fschecks_time19:53:365
1472099361,4cyclictest22624-21nfsd420:38:455
1472099361,3cyclictest25823-21snmpd20:52:305
1472099361,3cyclictest0-21swapper/1320:18:375
1472099361,3cyclictest0-21swapper/1319:43:465
1472099361,3cyclictest0-21swapper/1319:43:465
1472099361,3cyclictest0-21swapper/1319:28:355
14720993610,23cyclictest15886-21expr19:13:325
197652350,0sleep90-21swapper/923:13:3115
155492350,0sleep60-21swapper/622:57:4212
1472099351,3cyclictest3734-21chrt20:54:475
1472099351,3cyclictest19806-21grep19:18:345
1472099351,3cyclictest0-21swapper/1320:28:355
1472099341,3cyclictest0-21swapper/1320:03:205
1472099341,3cyclictest0-21swapper/1320:03:205
1472099340,3cyclictest780-21cstates20:13:325
1472099340,32cyclictest0-21swapper/1319:40:005
14714993430,3cyclictest0-21swapper/1019:58:252
14714993430,3cyclictest0-21swapper/1019:58:252
230872330,2sleep20-21swapper/222:20:118
1472099331,3cyclictest7629-21cat20:58:595
1472099331,3cyclictest15442-21ps21:08:495
1472099331,3cyclictest0-21swapper/1320:43:365
1472099330,3cyclictest8700-21cut20:23:325
1472099330,3cyclictest29579-21fschecks_time20:08:405
72472320,1sleep100-21swapper/1022:37:422
318532320,1sleep3998-21dbus-daemon23:30:329
300962320,28sleep60-21swapper/623:30:2012
1472099321,3cyclictest25048-21cstates20:03:325
1472099321,3cyclictest25048-21cstates20:03:325
1472099320,3cyclictest32198-21grep19:33:335
14714993228,3cyclictest0-21swapper/1019:36:002
1471199320,2cyclictest0-21swapper/823:08:3114
117192320,0sleep10-21swapper/122:23:061
79862310,2sleep71-21systemd00:06:1213
79342310,1sleep150-21swapper/1522:37:477
37722310,1sleep150-21swapper/1522:52:307
36842310,2sleep20-21swapper/222:26:058
187872310,1sleep80-21swapper/821:38:4914
1472099311,3cyclictest0-21swapper/1321:03:525
14714993127,3cyclictest0-21swapper/1020:08:262
119332310,1sleep00-21swapper/021:45:220
97042300,1sleep5561rcuc/519:43:4711
97042300,1sleep5561rcuc/519:43:4711
73952300,1sleep60-21swapper/621:33:5912
61242300,27sleep50-21swapper/523:04:2311
291812300,1sleep30-21swapper/300:01:189
249442300,0sleep150-21swapper/1521:39:287
190302300,1sleep40-21swapper/422:42:5810
164352300,1sleep140-21swapper/1421:11:236
157352300,2sleep50-21swapper/500:15:4811
157352300,2sleep50-21swapper/500:15:4811
14714993024,4cyclictest0-21swapper/1019:43:532
14714993024,4cyclictest0-21swapper/1019:43:532
86142290,1sleep3401ktimersoftd/322:53:069
293302290,25sleep00-21swapper/021:28:480
283232290,2sleep50-21swapper/521:51:2211
281552290,0sleep120-21swapper/1223:18:234
25922290,1sleep2321ktimersoftd/200:05:348
244772290,1sleep00-21swapper/023:37:280
235362290,3sleep623657-21systemd00:11:4812
235362290,3sleep623657-21systemd00:11:4812
197272290,2sleep111030-21polkitd22:28:063
187142290,2sleep30-21swapper/323:44:289
14714992925,3cyclictest0-21swapper/1019:48:402
14714992925,3cyclictest0-21swapper/1019:48:402
1470199297,20cyclictest0-21swapper/122:58:501
81142280,25sleep70-21swapper/722:33:5513
61732280,25sleep50-21swapper/523:31:2411
47452280,24sleep150-21swapper/1522:18:167
46002280,1sleep144597-21systemd22:18:146
27552280,2sleep112868-21sshd21:15:053
267062280,2sleep120-21swapper/1221:58:254
210232280,1sleep40-21swapper/421:39:0110
178492280,1sleep120-21swapper/1223:32:514
14714992824,3cyclictest0-21swapper/1020:03:252
14714992824,3cyclictest0-21swapper/1020:03:252
11782280,0sleep150-21swapper/1522:06:457
91412270,1sleep101471499cyclictest00:09:592
91412270,1sleep101471499cyclictest00:09:592
58522270,0sleep150-21swapper/1522:30:027
53992270,2sleep00-21swapper/022:26:180
262762270,1sleep90-21swapper/921:58:2215
221792270,2sleep30-21swapper/322:12:599
218492270,1sleep10-21swapper/119:18:521
216502270,23sleep90-21swapper/922:09:0515
1471499275,6cyclictest0-21swapper/1023:56:392
14714992721,5cyclictest0-21swapper/1019:38:482
102602270,1sleep00-21swapper/022:00:060
84512260,4sleep91471399cyclictest22:26:4315
304972260,6sleep430479-21sshd21:47:5110
241112260,0sleep00-21swapper/000:24:310
241112260,0sleep00-21swapper/000:24:310
1471499265,11cyclictest98-21ksoftirqd/1021:44:312
1471499260,20cyclictest5038-21nfsd23:10:342
14707992619,4cyclictest1014-21systemd-logind23:07:5210
1469999260,24cyclictest0-21swapper/020:03:290
1469999260,24cyclictest0-21swapper/020:03:290
11972260,1sleep40-21swapper/400:25:4010
11972260,1sleep40-21swapper/400:25:4010
58022250,1sleep90-21swapper/923:54:1215
45542250,1sleep50-21swapper/522:37:2211
290652250,22sleep70-21swapper/722:17:2013
275092250,2sleep30-21swapper/321:54:489
266292250,3sleep1126768-21sshd21:39:413
230782252,2sleep1523208-21sshd22:47:267
205202250,21sleep20-21swapper/221:17:048
1472299254,19cyclictest138-21ksoftirqd/1523:59:187
1471499258,7cyclictest0-21swapper/1000:22:022
1471499258,7cyclictest0-21swapper/1000:22:022
1471499258,15cyclictest971ktimersoftd/1023:03:192
1471499258,15cyclictest16031-21sshd00:43:052
1471499256,17cyclictest0-21swapper/1023:58:522
1471499254,9cyclictest0-21swapper/1023:07:282
14714992521,3cyclictest0-21swapper/1019:53:372
14714992521,3cyclictest0-21swapper/1019:53:372
14714992521,2cyclictest961rcuc/1021:13:492
14707992512,6cyclictest141rcu_sched23:51:5410
140222250,1sleep60-21swapper/600:06:5812
129352250,4sleep774-21ksoftirqd/723:28:2813
110972250,0sleep70-21swapper/723:16:1013
84752240,2sleep10-21swapper/121:56:221
5292240,1sleep20-21swapper/222:17:468
315252240,1sleep140-21swapper/1421:21:496
311212240,2sleep110-21swapper/1120:48:533
282252240,0sleep30-21swapper/323:53:179
275462240,20sleep00-21swapper/022:05:590
20922240,0sleep70-21swapper/720:13:4613
208612240,1sleep120-21swapper/1200:03:524
150672240,20sleep120-21swapper/1222:04:194
1471499249,13cyclictest20635-2110-uname00:24:012
1471499249,13cyclictest20635-2110-uname00:24:012
1471499247,10cyclictest0-21swapper/1022:12:542
1471499244,9cyclictest0-21swapper/1000:14:552
1471499244,9cyclictest0-21swapper/1000:14:552
1471499244,10cyclictest98-21ksoftirqd/1021:25:452
1471499242,12cyclictest98-21ksoftirqd/1023:31:402
1471499242,10cyclictest0-21swapper/1021:31:172
14714992417,5cyclictest961rcuc/1022:45:072
14714992411,11cyclictest0-21swapper/1022:18:452
14707992413,6cyclictest131rcu_preempt23:10:3310
14707992411,6cyclictest131rcu_preempt22:21:2810
14703992418,4cyclictest32019-21env22:29:159
105182240,2sleep60-21swapper/600:26:4712
105182240,2sleep60-21swapper/600:26:4712
92902230,1sleep50-21swapper/523:19:5311
90582230,1sleep20-21swapper/221:03:268
64242230,1sleep50-21swapper/500:09:3811
64242230,1sleep50-21swapper/500:09:3811
326532230,0sleep00-21swapper/023:57:460
29142230,2sleep40-21swapper/420:53:5110
279902230,1sleep150-21swapper/1520:45:447
279712230,1sleep90-21swapper/922:43:5115
268252230,1sleep60-21swapper/619:25:5412
26642230,21sleep60-21swapper/619:36:0412
23482230,20sleep60-21swapper/622:14:0712
190352230,2sleep30-21swapper/322:46:589
185062230,2sleep111471599cyclictest21:31:393
169562230,0sleep40-21swapper/420:33:3010
15092230,3sleep61-21systemd23:07:5212
1472199231,3cyclictest0-21swapper/1423:08:316
1471899231,20cyclictest0-21swapper/1222:18:304
1471899231,19cyclictest998-21dbus-daemon00:42:294
1471499238,13cyclictest93250irq/40-eth0-TxR19:13:382
1471499237,10cyclictest28528-21sshd23:18:262
1471499237,10cyclictest0-21swapper/1022:05:042
1471499236,6cyclictest961rcuc/1023:47:302
1471499236,11cyclictest98-21ksoftirqd/1022:55:362
14714992317,4cyclictest93250irq/40-eth0-TxR23:38:182
14714992316,5cyclictest10816-21bash23:16:082
1471499231,2cyclictest0-21swapper/1021:03:302
1471499231,20cyclictest1-21systemd22:38:572
1471499231,11cyclictest98-21ksoftirqd/1022:15:192
14714992310,11cyclictest998-21dbus-daemon22:30:112
1470799236,7cyclictest49-21ksoftirqd/422:54:0610
1470799234,17cyclictest481ktimersoftd/423:43:1810
1470799232,10cyclictest49-21ksoftirqd/400:00:3410
14707992313,6cyclictest141rcu_sched21:58:5110
14707992310,7cyclictest49-21ksoftirqd/423:28:4610
1470799230,2cyclictest0-21swapper/422:52:0310
1470799230,17cyclictest22884-21kworker/4:323:35:0110
14699992318,3cyclictest1005-21NetworkManager22:43:040
67792220,2sleep20-21swapper/222:41:208
319632220,1sleep110-21swapper/1123:03:413
220782220,1sleep150-21swapper/1522:24:097
213952220,19sleep20-21swapper/223:52:288
211342220,1sleep20-21swapper/200:00:138
161122220,1sleep00-21swapper/022:46:390
14982221,3sleep12028-21df23:38:331
1472199221,19cyclictest998-21dbus-daemon00:20:016
1472199221,19cyclictest998-21dbus-daemon00:20:016
14718992214,6cyclictest11320-21sshd22:49:254
1471899221,18cyclictest509-21systemd-journal00:23:224
1471899221,18cyclictest509-21systemd-journal00:23:224
1471899221,18cyclictest23624-21env21:58:074
1471899220,20cyclictest0-21swapper/1223:38:454
14715992216,3cyclictest11104-21sshd21:56:433
14715992215,4cyclictest9288-21sshd23:24:033
14715992213,6cyclictest998-21dbus-daemon22:44:333
1471499227,8cyclictest0-21swapper/1023:50:122
1471499226,9cyclictest0-21swapper/1000:35:032
1471499226,9cyclictest0-21swapper/1000:35:032
1471499226,9cyclictest0-21swapper/1000:30:202
1471499226,9cyclictest0-21swapper/1000:30:202
1471499226,9cyclictest0-21swapper/1000:08:072
1471499226,14cyclictest0-21swapper/1021:53:402
1471499225,10cyclictest0-21swapper/1021:42:232
1471499223,10cyclictest98-21ksoftirqd/1022:49:352
14714992212,5cyclictest141rcu_sched22:26:052
14711992215,4cyclictest24864-21sshd23:02:5614
1471199221,3cyclictest27833-21sshd22:13:3514
1471099221,18cyclictest998-21dbus-daemon00:13:3913
1471099221,18cyclictest998-21dbus-daemon00:13:3913
14708992217,3cyclictest1014-21systemd-logind23:58:3311
1470799229,5cyclictest131rcu_preempt21:19:5010
1470799225,7cyclictest49-21ksoftirqd/423:46:1810
1470799224,2cyclictest481ktimersoftd/400:12:0810
1470799224,2cyclictest481ktimersoftd/400:12:0810
1470799223,7cyclictest49-21ksoftirqd/423:14:0310
1470799223,11cyclictest49-21ksoftirqd/400:32:0210
1470799223,11cyclictest49-21ksoftirqd/400:32:0210
1470799221,9cyclictest49-21ksoftirqd/423:18:2810
14707992219,2cyclictest481ktimersoftd/400:04:4610
14707992215,4cyclictest15718-21sshd22:31:1610
1470799220,16cyclictest49-21ksoftirqd/422:12:3210
1470799220,16cyclictest49-21ksoftirqd/421:30:4710
70862210,1sleep70-21swapper/700:42:0213
249482210,1sleep150-21swapper/1521:36:017
215732210,1sleep80-21swapper/800:31:5314
215732210,1sleep80-21swapper/800:31:5314
18392210,2sleep40-21swapper/422:14:0510
14722992115,3cyclictest0-21swapper/1523:50:587
1472299211,18cyclictest998-21dbus-daemon22:58:437
1472299211,17cyclictest0-21swapper/1521:18:367
1472299210,18cyclictest0-21swapper/1500:07:527
1472199211,4cyclictest11898-21bash22:49:286
1472199211,17cyclictest5410-21systemd00:05:546
1472199210,16cyclictest0-21swapper/1423:05:246
14718992117,2cyclictest0-21swapper/1200:38:204
14718992117,2cyclictest0-21swapper/1200:38:204
14718992115,3cyclictest0-21swapper/1222:10:444
1471899211,3cyclictest19287-21sshd22:38:464
14718992113,5cyclictest28799-21sshd23:07:144
14718992113,5cyclictest19485-21sshd23:44:324
1471899210,19cyclictest0-21swapper/1222:33:574
14715992114,4cyclictest9629-21sh23:58:463
1471499216,9cyclictest98-21ksoftirqd/1021:51:242
1471499216,13cyclictest0-21swapper/1021:13:202
14713992112,4cyclictest3333-2110-uname21:33:3615
14711992114,4cyclictest31955-21systemd00:21:4614
14711992114,4cyclictest31955-21systemd00:21:4614
14711992114,4cyclictest2621-21run-parts23:27:1814
14711992114,4cyclictest12795-21sshd00:15:1614
14711992114,4cyclictest12795-21sshd00:15:1614
1471199211,18cyclictest0-21swapper/820:03:3014
1471199211,18cyclictest0-21swapper/820:03:3014
1471099210,2cyclictest0-21swapper/721:40:3113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional