You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-15 - 19:48
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Mon Apr 15, 2024 12:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
915621200,1sleep125-21ksoftirqd/109:45:431
93822860,0sleep40-21swapper/411:02:584
22162860,0sleep0111rcuc/012:37:450
168312760,0sleep30-21swapper/312:23:223
290582700,0sleep50-21swapper/509:41:225
290582700,0sleep50-21swapper/509:41:225
100642630,1sleep510061-21grepconf.sh09:58:285
2867625227,9sleep70-21swapper/707:05:587
236552490,0sleep20-21swapper/211:49:022
177482490,0sleep10-21swapper/111:38:591
104852480,2sleep72911899cyclictest11:16:077
2855024734,9sleep30-21swapper/307:04:503
75232460,0sleep20-21swapper/212:12:562
173962460,0sleep60-21swapper/612:20:056
289422450,0sleep60-21swapper/610:06:306
2886824533,8sleep50-21swapper/507:08:375
2860024532,9sleep60-21swapper/607:05:336
226552450,0sleep70-21swapper/711:39:217
202072450,0sleep50-21swapper/509:46:515
133452450,0sleep10-21swapper/111:16:251
151972440,0sleep22909299cyclictest11:29:152
6052430,0sleep40-21swapper/409:25:444
18632430,0sleep00-21swapper/012:21:510
314082420,0sleep30-21swapper/310:58:413
2885424230,9sleep40-21swapper/407:08:284
2879424129,9sleep20-21swapper/207:07:382
171342410,0sleep30-21swapper/311:06:583
141092410,0sleep50-21swapper/509:24:015
2860424029,8sleep10-21swapper/107:05:361
2854724029,8sleep00-21swapper/007:04:470
10322400,0sleep10-21swapper/110:22:591
326032390,0sleep50-21swapper/510:19:275
58312310,2sleep42910199cyclictest11:05:474
116162280,0sleep70-21swapper/710:27:137
161632190,0sleep50-21swapper/511:13:335
29105991813,1cyclictest14981-21sshd11:38:425
293442170,1sleep2698-21dbus-daemon12:14:562
29111991713,3cyclictest65-21ksoftirqd/611:57:316
29111991713,3cyclictest65-21ksoftirqd/611:57:316
29105991613,2cyclictest10173-21/usr/sbin/munin11:44:215
29085991615,1cyclictest20188-21kworker/1:412:04:071
263742160,0sleep50-21swapper/511:24:065
178732160,1sleep449-21ksoftirqd/410:14:454
29085991514,1cyclictest29065-21kworker/1:511:48:311
76742140,0sleep60-21swapper/612:28:466
321102140,1sleep132113-21idleruntime09:54:161
2910599148,3cyclictest965-21systemd09:32:155
2910599148,3cyclictest965-21systemd09:32:155
29105991411,2cyclictest1-21systemd11:52:395
2908599141,1cyclictest25-21ksoftirqd/111:11:271
2908599141,1cyclictest131rcu_sched09:14:171
29079991413,1cyclictest0-21swapper/010:58:030
281492140,0sleep40-21swapper/409:47:384
81842130,0sleep70-21swapper/712:35:047
2911199130,2cyclictest3130-21systemd09:38:476
2910599132,10cyclictest1-21systemd09:27:575
2910199130,12cyclictest20888-21gdbus10:56:044
29094991311,1cyclictest0-21swapper/309:16:053
2908599139,1cyclictest25-21ksoftirqd/112:18:541
29085991312,1cyclictest5631-21kworker/1:309:50:391
29085991312,1cyclictest14111-21kworker/1:312:21:271
29085991311,2cyclictest25-21ksoftirqd/112:03:071
29085991311,2cyclictest25-21ksoftirqd/112:03:071
29085991311,1cyclictest25-21ksoftirqd/111:27:451
29085991310,3cyclictest25-21ksoftirqd/110:10:081
2907999132,5cyclictest879-21polkitd11:13:160
262522130,0sleep20-21swapper/209:50:322
175192130,0sleep40-21swapper/411:19:564
157392130,0sleep10-21swapper/110:37:331
132762130,0sleep0111rcuc/010:59:580
2911899122,4cyclictest30011-21sshd10:35:337
2911199129,1cyclictest65-21ksoftirqd/612:36:126
2911199128,1cyclictest65-21ksoftirqd/610:36:436
29111991210,1cyclictest65-21ksoftirqd/611:18:416
2911199120,6cyclictest6775-21systemd12:09:266
2910599129,2cyclictest1-21systemd09:10:195
29105991210,1cyclictest1-21systemd12:07:255
29101991210,2cyclictest4149-21kworker/u16:312:24:104
2909499120,11cyclictest12610-21id12:03:503
2909499120,11cyclictest12610-21id12:03:503
2908599128,1cyclictest25-21ksoftirqd/110:26:141
2908599127,3cyclictest25-21ksoftirqd/110:54:101
2908599127,3cyclictest25-21ksoftirqd/109:09:131
29085991212,0cyclictest5728-21kworker/1:509:20:411
29085991211,1cyclictest25182-21kworker/1:207:38:351
29085991210,2cyclictest25-21ksoftirqd/110:04:171
29085991210,1cyclictest488-21systemd-journal09:27:211
29085991210,1cyclictest4835-21kworker/1:010:30:211
29085991210,1cyclictest25-21ksoftirqd/112:30:551
29085991210,1cyclictest25-21ksoftirqd/111:49:121
29085991210,1cyclictest25-21ksoftirqd/111:31:221
29085991210,1cyclictest25-21ksoftirqd/110:49:131
29085991210,1cyclictest14816-21kworker/1:410:16:191
2907999123,4cyclictest8599-21sshd11:31:460
2907999122,4cyclictest766-21gdbus11:41:280
2907999122,4cyclictest7637-21sshd09:54:570
29079991210,1cyclictest10059-21sshd11:34:570
2907999120,12cyclictest25988-21kworker/0:212:28:080
28232120,0sleep60-21swapper/612:05:556
2911899119,1cyclictest698-21dbus-daemon12:04:457
2911899115,3cyclictest17405-21sshd10:57:157
29118991110,1cyclictest0-21swapper/709:22:417
2911199119,2cyclictest65-21ksoftirqd/611:00:106
2911199119,1cyclictest65-21ksoftirqd/611:25:256
2911199119,1cyclictest65-21ksoftirqd/610:58:586
2911199119,1cyclictest65-21ksoftirqd/610:43:006
2911199119,1cyclictest65-21ksoftirqd/610:26:456
2911199119,1cyclictest65-21ksoftirqd/610:15:046
2911199119,1cyclictest65-21ksoftirqd/609:52:566
2911199119,1cyclictest12355-21systemd10:53:316
2911199118,3cyclictest65-21ksoftirqd/611:34:466
2911199115,4cyclictest0-21swapper/609:42:056
2911199115,4cyclictest0-21swapper/609:42:056
2911199113,3cyclictest698-21dbus-daemon10:10:026
29111991110,1cyclictest65-21ksoftirqd/609:10:336
29111991110,1cyclictest0-21swapper/609:55:586
2911199110,11cyclictest29010-21kworker/6:210:46:326
2911199110,10cyclictest12917-21systemd09:33:266
2911199110,10cyclictest12917-21systemd09:33:266
2910599119,2cyclictest8110-21cp12:09:355
2910599116,3cyclictest13080-21sshd09:17:315
2910599114,4cyclictest32432-21sshd12:02:355
2910599114,4cyclictest32432-21sshd12:02:355
2910599110,10cyclictest18429-21sshd11:16:565
2910199114,3cyclictest698-21dbus-daemon11:35:474
2910199110,11cyclictest0-21swapper/409:40:334
2910199110,11cyclictest0-21swapper/409:40:334
2909499119,1cyclictest41-21ksoftirqd/309:30:133
2909499119,1cyclictest41-21ksoftirqd/309:30:133
2909499119,1cyclictest19204-21systemd-cgroups09:27:363
2909499119,1cyclictest0-21swapper/310:47:513
2909499114,4cyclictest14336-21sshd10:17:453
2909499112,5cyclictest11528-21sshd11:03:123
2909499110,10cyclictest698-21dbus-daemon11:12:073
2909499110,10cyclictest31787-21kworker/3:010:24:193
2909499110,10cyclictest0-21swapper/310:08:263
2909499110,1cyclictest0-21swapper/309:35:463
2909299119,1cyclictest10429-21systemctl10:49:532
2909299119,1cyclictest1-21systemd12:00:592
2909299119,1cyclictest1-21systemd12:00:592
2909299115,4cyclictest19315-21kworker/2:411:05:452
2909299114,4cyclictest10453-21sshd11:34:592
2909299112,3cyclictest698-21dbus-daemon11:55:342
2909299112,3cyclictest698-21dbus-daemon11:55:342
2909299110,11cyclictest22025-21kworker/2:309:56:132
2909299110,10cyclictest699-21avahi-daemon12:33:562
2908599119,1cyclictest6428-21kworker/1:009:39:451
2908599119,1cyclictest6428-21kworker/1:009:39:451
2908599119,1cyclictest25-21ksoftirqd/111:58:341
2908599119,1cyclictest25-21ksoftirqd/111:58:341
2908599119,1cyclictest25-21ksoftirqd/111:40:531
2908599119,1cyclictest25-21ksoftirqd/111:20:161
2908599119,1cyclictest25-21ksoftirqd/110:42:321
2908599119,1cyclictest25-21ksoftirqd/109:29:291
2908599119,1cyclictest25-21ksoftirqd/109:29:291
2908599119,1cyclictest0-21swapper/111:06:011
2908599118,2cyclictest25-21ksoftirqd/110:03:101
2908599117,1cyclictest25-21ksoftirqd/112:35:261
2908599117,1cyclictest25-21ksoftirqd/112:11:571
2908599117,1cyclictest25-21ksoftirqd/111:01:021
2908599117,1cyclictest25-21ksoftirqd/110:44:191
2908599116,3cyclictest25-21ksoftirqd/109:38:221
29085991111,0cyclictest25182-21kworker/1:209:00:031
29085991110,1cyclictest25-21ksoftirqd/112:24:241
2907999112,5cyclictest717-21systemd-logind11:14:520
2907999112,4cyclictest26180-21sshd09:12:260
2907999112,4cyclictest1-21systemd11:08:510
2907999111,4cyclictest8566-21cp10:20:200
2907999111,4cyclictest17493-21sed10:34:170
2907999111,4cyclictest1-21systemd10:27:150
2907999111,4cyclictest0-21swapper/011:20:060
2907999110,5cyclictest596-21bash09:44:500
2907999110,11cyclictest31074-21kworker/0:009:17:170
2907999110,1cyclictest0-21swapper/009:27:530
251752110,0sleep50-21swapper/510:18:555
151672110,0sleep40-21swapper/411:51:204
83162100,0sleep70-21swapper/712:03:247
83162100,0sleep70-21swapper/712:03:247
2911899107,3cyclictest73-21ksoftirqd/709:41:317
2911899107,3cyclictest73-21ksoftirqd/709:41:317
2911899105,4cyclictest717-21systemd-logind10:16:277
2911899104,3cyclictest717-21systemd-logind10:03:307
2911899104,3cyclictest698-21dbus-daemon10:22:477
2911899104,3cyclictest698-21dbus-daemon09:29:477
2911899104,3cyclictest698-21dbus-daemon09:29:477
2911899104,3cyclictest0-21swapper/709:54:247
2911899102,5cyclictest4685-21sshd11:12:187
2911199109,1cyclictest65-21ksoftirqd/611:33:306
2911199109,1cyclictest65-21ksoftirqd/611:04:126
2911199109,1cyclictest65-21ksoftirqd/609:14:066
2911199108,1cyclictest65-21ksoftirqd/612:17:226
2911199108,1cyclictest65-21ksoftirqd/611:46:376
2911199108,1cyclictest65-21ksoftirqd/610:01:276
2911199108,1cyclictest65-21ksoftirqd/609:21:366
2911199108,1cyclictest3773-21systemd11:50:076
2911199104,4cyclictest25346-21systemd09:24:566
2911199100,7cyclictest28606-21kworker/6:410:19:276
2911199100,7cyclictest10369-21kworker/6:312:30:206
2911199100,10cyclictest0-21swapper/610:32:336
2910599109,1cyclictest0-21swapper/511:55:055
2910599109,1cyclictest0-21swapper/511:55:055
2910599109,1cyclictest0-21swapper/509:38:155
2910599108,1cyclictest57-21ksoftirqd/511:42:555
2910599104,3cyclictest881-21gdbus10:46:505
2910599104,3cyclictest0-21swapper/510:43:345
2910599103,4cyclictest488-21systemd-journal10:09:345
2910599100,7cyclictest0-21swapper/510:50:015
2910599100,1cyclictest488-21systemd-journal10:02:035
2910199108,1cyclictest698-21dbus-daemon11:32:574
2910199107,3cyclictest49-21ksoftirqd/412:18:304
2910199106,3cyclictest698-21dbus-daemon12:19:374
2910199102,5cyclictest9832-21sshd10:53:154
2910199102,5cyclictest698-21dbus-daemon11:26:514
2910199100,9cyclictest0-21swapper/407:34:074
2910199100,10cyclictest0-21swapper/410:39:504
2910199100,1cyclictest0-21swapper/412:07:054
2909499106,1cyclictest41-21ksoftirqd/312:17:173
2909499105,2cyclictest14490-21sshd12:35:433
2909499104,3cyclictest9366-21sshd12:28:563
2909499104,3cyclictest698-21dbus-daemon09:51:443
2909499104,3cyclictest31287-21systemd-cgroups10:35:413
2909499104,3cyclictest13122-21grepconf.sh11:41:363
2909499104,3cyclictest1-21systemd10:10:093
2909499104,3cyclictest0-21swapper/312:07:583
2909499104,3cyclictest0-21swapper/311:47:383
2909499103,4cyclictest720-21in:imjournal09:57:353
2909499102,4cyclictest0-21swapper/311:39:043
2909499100,9cyclictest22244-21kworker/3:511:49:593
2909499100,10cyclictest10667-21sshd10:53:223
2909499100,10cyclictest0-21swapper/309:21:463
2909299109,1cyclictest33-21ksoftirqd/212:34:442
2909299109,1cyclictest0-21swapper/210:10:382
2909299107,2cyclictest15885-21bash09:39:532
2909299107,2cyclictest15885-21bash09:39:532
2909299104,5cyclictest23526-21grepconf.sh12:24:032
2909299104,3cyclictest19442-21munin-run07:49:062
2909299103,5cyclictest1-21systemd10:34:522
2909299103,4cyclictest20886-21packagekitd10:01:222
2909299102,6cyclictest18304-21bash10:24:322
2909299101,4cyclictest698-21dbus-daemon09:27:082
2909299100,4cyclictest1-21systemd10:23:392
2909299100,10cyclictest0-21swapper/211:23:092
2909299100,1cyclictest576-21bash11:27:532
2908599109,1cyclictest25182-21kworker/1:208:23:391
2908599109,1cyclictest25182-21kworker/1:207:24:531
29085991010,0cyclictest25182-21kworker/1:208:13:351
2907999105,2cyclictest28649-21httpd12:30:030
2907999103,6cyclictest7168-21sshd09:39:100
2907999103,6cyclictest7168-21sshd09:39:100
2907999103,4cyclictest0-21swapper/008:09:070
2907999103,1cyclictest879-21polkitd10:09:480
2907999103,1cyclictest20888-21gdbus10:01:120
2907999102,4cyclictest717-21systemd-logind10:40:510
2907999102,4cyclictest1608-21sshd10:45:480
2907999101,4cyclictest2176-21sshd11:28:020
2907999101,4cyclictest13101-21sshd12:00:380
2907999101,4cyclictest13101-21sshd12:00:380
2907999101,4cyclictest0-21swapper/012:04:470
2907999100,4cyclictest0-21swapper/009:20:410
2907999100,10cyclictest21647-21kworker/0:411:52:510
166042100,0sleep5551rcuc/510:24:235
291189999,0cyclictest0-21swapper/709:11:207
291189998,1cyclictest1889-21nfsd11:08:377
291189993,3cyclictest698-21dbus-daemon12:14:237
291189993,3cyclictest21836-21grep11:36:097
291189993,3cyclictest15472-21sshd11:48:117
291189992,4cyclictest18721-21bash09:49:447
291189992,3cyclictest698-21dbus-daemon12:25:077
291189992,3cyclictest23153-21systemd-cgroups12:11:137
291189992,3cyclictest16870-21grepconf.sh10:40:517
291189990,9cyclictest6689-21kworker/7:011:22:337
291119997,1cyclictest65-21ksoftirqd/611:59:156
291119997,1cyclictest65-21ksoftirqd/611:59:156
291119997,1cyclictest65-21ksoftirqd/611:19:186
291119997,1cyclictest65-21ksoftirqd/611:09:406
291119997,1cyclictest65-21ksoftirqd/609:45:426
291119992,3cyclictest1-21systemd11:39:106
291059999,0cyclictest0-21swapper/511:07:485
291059999,0cyclictest0-21swapper/510:08:315
291059999,0cyclictest0-21swapper/509:49:365
291059994,2cyclictest483-21sshd12:21:435
291059994,0cyclictest0-21swapper/508:53:175
291059992,4cyclictest28257-21grepconf.sh11:30:285
291059992,4cyclictest16417-21systemd-cgroups11:26:125
291059990,9cyclictest0-21swapper/512:37:025
291059990,9cyclictest0-21swapper/511:03:565
291059990,3cyclictest0-21swapper/509:03:035
291059990,0cyclictest0-21swapper/509:07:255
291059990,0cyclictest0-21swapper/508:54:165
291019999,0cyclictest0-21swapper/410:29:354
291019999,0cyclictest0-21swapper/410:24:424
291019999,0cyclictest0-21swapper/410:11:184
291019999,0cyclictest0-21swapper/409:24:004
291019998,1cyclictest49-21ksoftirqd/412:29:414
291019998,1cyclictest49-21ksoftirqd/409:59:504
291019998,1cyclictest0-21swapper/411:42:444
291019998,1cyclictest0-21swapper/409:35:024
291019997,1cyclictest7618-21systemd-cgroups11:12:374
291019997,1cyclictest49-21ksoftirqd/409:55:174
291019997,1cyclictest0-21swapper/410:04:314
291019993,3cyclictest23868-21grep12:11:174
291019993,3cyclictest14032-21httpd10:39:044
291019993,3cyclictest13386-21systemd10:20:534
291019992,4cyclictest16525-21bash10:47:224
291019992,3cyclictest10331-21systemd09:52:124
291019992,3cyclictest0-21swapper/411:44:084
291019990,8cyclictest25739-21kworker/4:209:16:314
291019990,0cyclictest0-21swapper/411:14:214
290949999,0cyclictest0-21swapper/312:31:543
290949999,0cyclictest0-21swapper/310:32:143
290949998,1cyclictest0-21swapper/310:42:293
290949994,3cyclictest13948-21sshd10:20:563
290949994,3cyclictest0-21swapper/309:40:473
290949994,3cyclictest0-21swapper/309:40:473
290949993,3cyclictest717-21systemd-logind11:15:313
290949993,3cyclictest698-21dbus-daemon11:56:003
290949993,3cyclictest698-21dbus-daemon11:56:003
290949993,3cyclictest698-21dbus-daemon11:28:373
290949992,4cyclictest879-21polkitd09:13:193
290949991,5cyclictest717-21systemd-logind09:47:273
290949990,9cyclictest0-21swapper/308:39:313
290949990,6cyclictest30257-21bash10:00:173
290949990,0cyclictest0-21swapper/311:30:443
290929994,4cyclictest698-21dbus-daemon09:14:102
290929994,3cyclictest26420-21systemd-cgroups11:01:242
290929993,2cyclictest698-21dbus-daemon09:10:042
290929992,4cyclictest698-21dbus-daemon10:29:582
290929992,4cyclictest13650irq/24-ahci[00010:08:172
290929992,4cyclictest0-21swapper/210:17:262
290929992,3cyclictest0-21swapper/211:40:512
290929990,9cyclictest5942-21kworker/2:211:50:322
290929990,9cyclictest0-21swapper/210:44:322
290929990,6cyclictest23578-21grepconf.sh10:57:532
290929990,3cyclictest698-21dbus-daemon11:14:142
290929990,3cyclictest698-21dbus-daemon11:09:092
290929990,2cyclictest0-21swapper/210:39:112
290859999,0cyclictest25182-21kworker/1:208:18:171
290859998,1cyclictest25182-21kworker/1:208:46:211
290859998,1cyclictest25182-21kworker/1:208:04:451
290859998,1cyclictest25182-21kworker/1:207:17:571
290859997,1cyclictest25-21ksoftirqd/108:24:131
290799993,1cyclictest15556-21bash11:57:490
290799993,1cyclictest15556-21bash11:57:490
290799992,6cyclictest494-21sshd10:29:180
290799990,9cyclictest0-21swapper/010:53:010
290799990,9cyclictest0-21swapper/010:15:010
290799990,8cyclictest28022-21kworker/0:310:04:470
290799990,7cyclictest0-21swapper/009:31:330
290799990,7cyclictest0-21swapper/009:31:330
290799990,4cyclictest22718-21sshd12:14:190
290799990,1cyclictest17907-21sshd09:49:380
290799990,0cyclictest0-21swapper/012:12:000
291189985,2cyclictest881-21gdbus10:46:187
291189985,2cyclictest698-21dbus-daemon11:24:477
291189985,2cyclictest698-21dbus-daemon11:00:487
291189985,2cyclictest698-21dbus-daemon09:36:497
291189985,2cyclictest389-21sshd09:48:077
291189983,3cyclictest1-21systemd10:04:337
291189983,1cyclictest26801-21cp11:30:197
291189982,4cyclictest19388-21bash11:58:117
291189982,4cyclictest19388-21bash11:58:117
291189982,4cyclictest12131-21systemd09:14:187
291189982,3cyclictest9945-21grepconf.sh10:30:147
291189982,3cyclictest698-21dbus-daemon12:20:327
291189982,3cyclictest29410-21sshd09:25:217
291189982,3cyclictest21516-21sshd12:29:587
291189982,3cyclictest19433-21sd-pam10:50:527
291189981,3cyclictest30299-21sshd10:09:387
291189980,4cyclictest717-21systemd-logind11:50:247
291119987,1cyclictest65-21ksoftirqd/607:19:076
291119983,2cyclictest0-21swapper/607:59:186
291059983,2cyclictest0-21swapper/508:39:375
291059983,0cyclictest0-21swapper/508:13:015
291059982,3cyclictest7539-21systemd12:16:035
291059982,3cyclictest1-21systemd12:24:285
291059980,8cyclictest669-21kworker/5:210:54:555
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional