You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-28 - 09:26
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Sun Apr 28, 2024 00:43:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
193152920,2sleep52069399cyclictest23:50:305
52462700,0sleep30-21swapper/300:24:003
166142530,0sleep70-21swapper/700:38:217
2035125126,9sleep00-21swapper/019:06:130
2034325127,9sleep10-21swapper/119:06:061
2041225026,9sleep30-21swapper/319:07:043
2020725026,8sleep70-21swapper/719:04:207
137402490,0sleep30-21swapper/300:34:333
105582490,0sleep20-21swapper/222:01:112
248282480,1sleep4879-21polkitd22:58:234
2050024825,8sleep60-21swapper/619:08:196
2032924826,8sleep50-21swapper/519:05:545
237652470,0sleep10-21swapper/122:27:131
131652470,0sleep40-21swapper/423:43:484
90622460,0sleep6641ktimersoftd/623:49:276
74782460,0sleep10-21swapper/123:46:181
74302460,0sleep60-21swapper/600:01:526
316462460,0sleep20-21swapper/221:35:242
300322460,0sleep40-21swapper/422:24:364
287712460,1sleep428774-21unixbench_multi19:19:054
148162460,0sleep60-21swapper/621:55:226
260502450,1sleep6784-21runrttasks23:32:376
126642450,2sleep62070299cyclictest21:52:106
149772440,1sleep014975-21dump-pmu-power23:03:400
85022430,0sleep50-21swapper/500:27:385
286382420,0sleep00-21swapper/022:33:530
2030024128,9sleep20-21swapper/219:05:352
84522400,0sleep70-21swapper/719:43:597
79302400,0sleep60-21swapper/621:36:166
279692400,0sleep30-21swapper/300:10:143
270612400,0sleep10-21swapper/122:08:551
2032824028,9sleep40-21swapper/419:05:534
301992390,0sleep00-21swapper/023:48:330
22162390,0sleep30-21swapper/323:14:403
108072360,2sleep12066099cyclictest22:16:401
107752190,0sleep60-21swapper/623:18:436
173602180,0sleep60-21swapper/623:41:026
116862180,1sleep011673-21bash22:29:040
35782170,0sleep60-21swapper/622:56:176
2068499172,2cyclictest0-21swapper/423:58:314
2068499170,4cyclictest698-21dbus-daemon23:38:404
186062170,0sleep50-21swapper/522:14:115
272772160,0sleep6641ktimersoftd/621:44:096
2068499161,6cyclictest698-21dbus-daemon22:02:124
2068499160,5cyclictest30277-21sshd00:20:054
2068499160,2cyclictest2247-21sshd22:18:554
2068499160,2cyclictest2247-21sshd22:18:554
2067699165,4cyclictest41-21ksoftirqd/321:14:203
40792150,1sleep6698-21dbus-daemon00:37:006
2068499151,3cyclictest17780-21grep22:14:064
2068499143,2cyclictest1-21systemd22:51:154
2068499142,1cyclictest5627-21systemd-cgroups21:42:154
2068499141,7cyclictest1824-21sshd21:51:064
2068499141,5cyclictest6798-21sshd21:11:034
2068499140,7cyclictest881-21gdbus21:25:414
2068499140,7cyclictest16442-21id00:09:034
2068499140,6cyclictest698-21dbus-daemon23:06:284
2068499140,3cyclictest1-21systemd23:40:544
2068499140,2cyclictest2685-21sshd23:21:014
2067699146,3cyclictest41-21ksoftirqd/300:14:393
2067699145,1cyclictest41-21ksoftirqd/323:51:433
2067699144,2cyclictest41-21ksoftirqd/323:25:443
2067699141,2cyclictest41-21ksoftirqd/323:43:353
2067699141,1cyclictest0-21swapper/321:38:063
2067699140,4cyclictest41-21ksoftirqd/323:29:193
2067699140,2cyclictest0-21swapper/322:30:203
2066899149,1cyclictest717-21systemd-logind21:08:452
2066899145,1cyclictest33-21ksoftirqd/222:11:462
20668991410,4cyclictest33-21ksoftirqd/223:58:132
20668991410,1cyclictest33-21ksoftirqd/222:22:202
20668991410,1cyclictest33-21ksoftirqd/222:22:202
163922140,0sleep20-21swapper/200:21:582
88162130,0sleep01-21systemd23:30:550
81512130,1sleep08150-21bash23:21:340
21162130,0sleep70-21swapper/722:25:037
2069399132,5cyclictest879-21polkitd00:36:485
2069399132,5cyclictest7483-21sshd23:43:175
2068499131,7cyclictest13202-21sshd21:33:444
2068499131,5cyclictest16844-21systemd23:25:274
2068499131,4cyclictest1392-21grepconf.sh23:30:104
2068499131,3cyclictest1-21systemd21:57:194
2068499131,1cyclictest9734-21sshd23:03:094
2068499131,1cyclictest49-21ksoftirqd/422:31:244
2068499131,1cyclictest0-21swapper/423:09:334
2068499130,5cyclictest698-21dbus-daemon22:07:014
2068499130,5cyclictest16742-21systemd23:59:304
2068499130,4cyclictest26416-21systemd-cgroups21:44:044
2067699134,1cyclictest41-21ksoftirqd/321:54:053
2067699132,1cyclictest41-21ksoftirqd/321:28:523
2067699131,3cyclictest41-21ksoftirqd/321:41:593
2067699131,3cyclictest0-21swapper/322:42:593
2067699131,2cyclictest0-21swapper/322:56:353
2067699131,2cyclictest0-21swapper/300:06:123
2067699130,3cyclictest0-21swapper/322:23:193
2067699130,3cyclictest0-21swapper/322:23:193
2067699130,12cyclictest24713-21sshd21:19:133
2066899138,4cyclictest33-21ksoftirqd/221:46:572
2066899137,4cyclictest33-21ksoftirqd/223:06:372
20668991310,3cyclictest33-21ksoftirqd/221:54:112
20668991310,2cyclictest717-21systemd-logind22:44:382
2066899131,1cyclictest33-21ksoftirqd/200:28:222
2066899130,12cyclictest7049-21kworker/2:022:34:002
2066899130,1cyclictest33-21ksoftirqd/221:49:272
131462130,0sleep40-21swapper/400:15:124
295832120,0sleep60-21swapper/621:19:426
27362120,0sleep10-21swapper/121:32:431
262612120,0sleep2488-21systemd-journal00:32:402
225342120,0sleep20-21swapper/223:19:442
2069399123,8cyclictest11991-21kill23:46:455
2069399122,4cyclictest879-21polkitd23:37:085
2069399122,4cyclictest0-21swapper/522:36:235
2069399120,11cyclictest22594-21sshd21:37:435
2068499122,1cyclictest18372-21systemd-cgroups21:31:044
2068499122,1cyclictest0-21swapper/422:44:144
2068499121,7cyclictest1-21systemd23:13:454
2068499120,6cyclictest698-21dbus-daemon21:20:134
2068499120,5cyclictest5585-21sshd21:14:204
2068499120,5cyclictest3150-21grepconf.sh00:04:314
2068499120,5cyclictest28383-21sshd23:51:244
2068499120,5cyclictest26852-21sd-pam22:39:494
2068499120,5cyclictest23704-21sshd00:25:534
2067699124,2cyclictest41-21ksoftirqd/322:53:253
2067699124,1cyclictest41-21ksoftirqd/321:45:163
2067699124,1cyclictest41-21ksoftirqd/321:24:093
20676991211,1cyclictest391rcuc/322:16:523
2067699121,3cyclictest692-21nscd21:49:003
2067699121,3cyclictest41-21ksoftirqd/323:19:123
2067699121,3cyclictest0-21swapper/322:59:113
2067699121,2cyclictest41-21ksoftirqd/322:34:433
2067699121,2cyclictest26302-21sshd22:08:523
2067699121,2cyclictest0-21swapper/321:09:063
2067699120,3cyclictest0-21swapper/322:06:253
2067699120,2cyclictest0-21swapper/323:54:173
2067699120,1cyclictest0-21swapper/323:08:453
2067699120,1cyclictest0-21swapper/323:07:023
2067699120,1cyclictest0-21swapper/322:44:023
2067699120,1cyclictest0-21swapper/321:59:423
2067699120,1cyclictest0-21swapper/300:32:153
2066899129,3cyclictest33-21ksoftirqd/222:14:482
2066899129,2cyclictest33-21ksoftirqd/200:09:432
2066899128,1cyclictest33-21ksoftirqd/221:43:352
20668991210,1cyclictest33-21ksoftirqd/222:57:392
20668991210,1cyclictest33-21ksoftirqd/221:28:442
20668991210,1cyclictest33-21ksoftirqd/200:33:442
2066899120,2cyclictest131rcu_sched23:01:112
2066899120,11cyclictest879-21polkitd23:39:012
2066099129,2cyclictest32688-21sshd22:03:221
2066099126,3cyclictest1-21systemd21:42:251
2065299129,1cyclictest9-21ksoftirqd/023:34:160
2065299120,11cyclictest4438-21systemd23:27:270
2065299120,11cyclictest16773-21bash22:20:180
2065299120,11cyclictest16773-21bash22:20:180
130992120,0sleep00-21swapper/000:37:580
90102110,0sleep60-21swapper/621:39:216
249132110,1sleep1241ktimersoftd/121:34:431
208742110,0sleep7711rcuc/722:45:297
2070999114,4cyclictest16298-21sshd21:40:057
2070999113,4cyclictest13587-21/usr/sbin/munin21:49:037
2070999110,10cyclictest0-21swapper/723:15:077
2070999110,10cyclictest0-21swapper/723:00:087
2070299113,4cyclictest0-21swapper/623:01:246
2069399119,1cyclictest14390-21systemd22:23:175
2069399119,1cyclictest14390-21systemd22:23:175
2069399110,11cyclictest14465-21kworker/5:322:02:545
2068499111,2cyclictest17866-21kill22:11:084
2068499111,1cyclictest0-21swapper/422:34:254
2068499110,4cyclictest12293-21bash00:34:234
2068499110,3cyclictest754-21NetworkManager00:29:424
2067699115,3cyclictest41-21ksoftirqd/323:58:563
2067699111,4cyclictest41-21ksoftirqd/322:24:033
2067699111,2cyclictest41-21ksoftirqd/300:19:523
2067699110,4cyclictest0-21swapper/323:36:433
2067699110,0cyclictest0-21swapper/323:45:443
2066899119,1cyclictest33-21ksoftirqd/223:53:402
2066899119,1cyclictest33-21ksoftirqd/223:13:592
2066899119,1cyclictest33-21ksoftirqd/222:39:532
2066899119,1cyclictest33-21ksoftirqd/222:28:492
2066899119,1cyclictest33-21ksoftirqd/221:26:282
2066899119,1cyclictest33-21ksoftirqd/221:18:522
2066899119,1cyclictest33-21ksoftirqd/221:16:122
2066899118,1cyclictest33-21ksoftirqd/223:09:172
20668991110,1cyclictest33-21ksoftirqd/222:50:072
2066899111,6cyclictest33-21ksoftirqd/223:32:032
2066899111,1cyclictest33-21ksoftirqd/200:15:502
2066899110,2cyclictest121rcu_preempt23:33:542
2066899110,10cyclictest717-21systemd-logind22:24:172
2066899110,10cyclictest26970-21kworker/2:322:07:382
2066099119,1cyclictest28537-21systemd-cgroups21:56:441
2066099118,1cyclictest25-21ksoftirqd/122:40:251
2065299119,1cyclictest0-21swapper/023:55:310
2065299119,1cyclictest0-21swapper/021:40:060
20652991110,1cyclictest0-21swapper/021:54:070
2065299111,4cyclictest698-21dbus-daemon22:40:200
17402110,0sleep50-21swapper/523:11:385
105732110,1sleep010568-21sshd00:31:010
313242100,0sleep40-21swapper/419:24:024
292572100,0sleep10-21swapper/100:29:341
286092100,0sleep50-21swapper/500:29:305
2070999106,3cyclictest73-21ksoftirqd/722:12:137
2070999106,1cyclictest73-21ksoftirqd/723:49:457
2070999104,5cyclictest698-21dbus-daemon22:55:587
2070999104,3cyclictest15292-21sshd21:24:307
2070999103,4cyclictest717-21systemd-logind21:18:337
2070999103,4cyclictest5640-21systemd23:43:067
2070999103,4cyclictest553-21bash00:04:157
2070999103,4cyclictest31969-21sshd22:43:337
2070999103,4cyclictest17036-21systemd21:34:017
2070999102,4cyclictest717-21systemd-logind21:20:017
2070999101,4cyclictest698-21dbus-daemon21:56:417
2070999100,9cyclictest27442-21systemd-cgroups21:09:217
2070999100,10cyclictest0-21swapper/723:07:147
2070999100,10cyclictest0-21swapper/722:06:427
2070299105,1cyclictest0-21swapper/622:35:206
2070299105,1cyclictest0-21swapper/621:24:436
2070299103,4cyclictest754-21NetworkManager21:29:306
2070299103,4cyclictest11504-21sshd00:31:076
2070299102,5cyclictest0-21swapper/600:25:596
2070299102,3cyclictest86350irq/25-em1-rx-021:13:056
2070299101,6cyclictest0-21swapper/622:13:056
2070299101,4cyclictest0-21swapper/622:45:176
2070299100,4cyclictest6636-21systemd-cgroups00:20:596
2070299100,10cyclictest0-21swapper/622:49:476
2070299100,0cyclictest0-21swapper/623:36:176
2069399109,1cyclictest0-21swapper/521:24:385
2069399103,6cyclictest450-21systemd-cgroups21:20:025
2069399103,1cyclictest14212-21systemd-cgroups23:06:375
2069399103,1cyclictest0-21swapper/522:32:475
2069399101,4cyclictest6443-21sshd22:44:035
2069399100,9cyclictest24375-21systemd00:13:125
2069399100,4cyclictest27445-21systemd-cgroups21:09:215
2069399100,4cyclictest24147-21sshd22:11:455
2069399100,4cyclictest1-21systemd21:48:005
2069399100,4cyclictest0-21swapper/521:42:425
2069399100,10cyclictest20184-21sshd21:31:155
2069399100,10cyclictest0-21swapper/522:43:275
2067699103,1cyclictest41-21ksoftirqd/320:33:533
2066899108,1cyclictest33-21ksoftirqd/223:43:532
2066899108,1cyclictest33-21ksoftirqd/200:05:162
2066899107,2cyclictest33-21ksoftirqd/223:25:452
2066899100,10cyclictest2353-21kworker/2:223:59:042
2066099109,1cyclictest19726-21sshd21:49:391
2066099109,1cyclictest0-21swapper/123:28:391
2066099109,1cyclictest0-21swapper/122:56:581
2066099107,2cyclictest25-21ksoftirqd/100:17:441
2066099104,3cyclictest16329-21systemd-cgroups23:37:541
2066099104,3cyclictest0-21swapper/122:35:501
2066099100,1cyclictest0-21swapper/123:19:061
2065299109,1cyclictest9-21ksoftirqd/023:05:560
2065299109,1cyclictest9-21ksoftirqd/022:54:470
2065299109,1cyclictest9-21ksoftirqd/021:32:250
2065299108,1cyclictest9-21ksoftirqd/023:51:480
2065299108,1cyclictest9-21ksoftirqd/022:49:460
2065299108,1cyclictest9-21ksoftirqd/021:22:020
2065299106,1cyclictest9-21ksoftirqd/023:12:430
2065299104,3cyclictest698-21dbus-daemon21:48:400
2065299104,3cyclictest30151-21sshd21:13:450
2065299103,6cyclictest881-21gdbus00:04:340
2065299103,6cyclictest698-21dbus-daemon22:03:530
2065299103,4cyclictest5032-21grepconf.sh21:45:120
2065299102,5cyclictest488-21systemd-journal21:09:410
2065299100,4cyclictest0-21swapper/022:11:340
2065299100,10cyclictest0-21swapper/022:13:500
2065299100,0cyclictest0-21swapper/000:28:190
207099995,1cyclictest73-21ksoftirqd/723:34:177
207099994,4cyclictest717-21systemd-logind00:26:187
207099994,4cyclictest488-21systemd-journal22:22:037
207099994,4cyclictest488-21systemd-journal22:22:037
207099994,4cyclictest0-21swapper/700:21:417
207099993,5cyclictest21785-21sshd23:29:017
207099993,4cyclictest717-21systemd-logind22:15:087
207099993,3cyclictest879-21polkitd21:47:507
207099993,3cyclictest717-21systemd-logind22:34:247
207099993,3cyclictest717-21systemd-logind00:14:537
207099993,3cyclictest23932-21smart_sda00:29:037
207099992,4cyclictest21483-21systemd22:48:447
207099992,4cyclictest19321-21systemd22:29:487
207099990,9cyclictest13659-21kworker/7:023:48:127
207099990,9cyclictest0-21swapper/723:59:347
207099990,9cyclictest0-21swapper/700:09:287
207099990,0cyclictest0-21swapper/721:33:357
207029999,0cyclictest0-21swapper/623:44:196
207029999,0cyclictest0-21swapper/600:14:546
207029997,1cyclictest0-21swapper/623:54:156
207029997,1cyclictest0-21swapper/623:27:566
207029994,4cyclictest23141-21id22:14:386
207029994,3cyclictest717-21systemd-logind23:16:426
207029994,2cyclictest20888-21gdbus22:00:286
207029993,5cyclictest6452-21bash23:05:516
207029993,2cyclictest717-21systemd-logind22:18:536
207029993,2cyclictest717-21systemd-logind22:18:536
207029993,1cyclictest0-21swapper/623:09:046
207029992,3cyclictest698-21dbus-daemon00:10:126
207029990,9cyclictest0-21swapper/620:14:016
207029990,5cyclictest1-21systemd22:39:256
207029990,4cyclictest0-21swapper/622:29:076
207029990,0cyclictest0-21swapper/622:04:136
207029990,0cyclictest0-21swapper/621:16:176
207029990,0cyclictest0-21swapper/600:04:306
206939997,1cyclictest1-21systemd00:07:295
206939995,3cyclictest23721-21sshd23:13:475
206939994,4cyclictest754-21NetworkManager00:16:525
206939993,3cyclictest0-21swapper/522:25:415
206939993,1cyclictest9585-21sshd22:56:525
206939992,6cyclictest86450irq/26-em1-tx-022:49:345
206939991,2cyclictest717-21systemd-logind21:49:115
206939991,2cyclictest488-21systemd-journal23:57:365
206939990,9cyclictest0-21swapper/521:58:135
206939990,4cyclictest29961-21sshd00:23:225
206939990,0cyclictest0-21swapper/522:04:415
206939990,0cyclictest0-21swapper/521:16:175
206769990,3cyclictest10683-21strings20:49:043
206609998,1cyclictest25-21ksoftirqd/100:35:231
206609996,1cyclictest25-21ksoftirqd/123:08:511
206609995,3cyclictest22792-21kill23:04:121
206609995,1cyclictest5833-21sshd23:30:371
206609995,1cyclictest25-21ksoftirqd/123:42:591
206609994,3cyclictest717-21systemd-logind21:23:481
206609994,2cyclictest31532-21sshd22:46:331
206609993,4cyclictest15441-21httpd23:15:211
206609993,3cyclictest4174-21sshd00:04:371
206609993,3cyclictest3556-21grep22:18:591
206609993,3cyclictest3556-21grep22:18:591
206609993,3cyclictest20886-21packagekitd23:01:031
206609993,3cyclictest15671-21sshd22:29:251
206609993,3cyclictest13123-21sshd00:02:271
206609991,3cyclictest31468-21sshd22:49:331
206609990,9cyclictest0-21swapper/121:14:101
206609990,5cyclictest3154-21grepconf.sh00:23:521
206609990,4cyclictest20886-21packagekitd00:21:271
206609990,2cyclictest1-21systemd23:51:121
206609990,1cyclictest0-21swapper/121:10:131
206609990,0cyclictest0-21swapper/123:56:141
206529998,1cyclictest9-21ksoftirqd/023:58:560
206529994,3cyclictest16371-21sshd21:58:420
206529993,4cyclictest25558-21bash23:38:470
206529993,3cyclictest27968-21sshd23:17:150
206529993,3cyclictest24908-21sshd00:09:550
206529993,3cyclictest13510-21sshd22:47:580
206529992,3cyclictest1-21systemd22:25:200
206529991,5cyclictest21265-21sshd21:37:350
206529990,9cyclictest0-21swapper/021:27:100
207099988,0cyclictest0-21swapper/719:33:567
207099988,0cyclictest0-21swapper/719:18:577
207099985,2cyclictest30574-21sshd23:54:377
207099983,4cyclictest2628-21sshd23:24:037
207099983,4cyclictest20483-21sshd23:19:317
207099983,2cyclictest0-21swapper/720:20:407
207099980,8cyclictest19027-21kworker/7:023:09:197
207099980,4cyclictest0-21swapper/721:59:107
207029983,2cyclictest21297-21sshd22:23:526
207029982,3cyclictest18468-21munin-run20:08:406
206939985,2cyclictest7132-21bash00:01:505
206939985,2cyclictest698-21dbus-daemon22:59:165
206939984,3cyclictest29821-21rm23:23:415
206939981,4cyclictest0-21swapper/523:21:485
206939980,8cyclictest0-21swapper/520:18:535
206939980,2cyclictest879-21polkitd23:29:315
206939980,0cyclictest0-21swapper/519:24:025
206849981,1cyclictest0-21swapper/420:40:134
206849981,1cyclictest0-21swapper/420:25:464
206849981,1cyclictest0-21swapper/420:14:054
206849980,1cyclictest0-21swapper/419:42:364
206769984,4cyclictest41-21ksoftirqd/320:47:173
206769983,3cyclictest41-21ksoftirqd/320:03:503
206769981,3cyclictest41-21ksoftirqd/319:44:223
206769981,3cyclictest27933-21strings19:18:593
206769981,3cyclictest1889-21nfsd19:15:113
206769981,3cyclictest0-21swapper/321:03:563
206769981,1cyclictest0-21swapper/319:23:513
206769980,3cyclictest0-21swapper/320:15:103
206689986,1cyclictest33-21ksoftirqd/219:43:482
206689983,2cyclictest0-21swapper/220:56:232
206689983,2cyclictest0-21swapper/220:04:432
206689980,0cyclictest0-21swapper/219:13:512
206609984,3cyclictest25-21ksoftirqd/121:18:421
206609980,5cyclictest0-21swapper/100:08:531
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional