You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-26 - 00:06
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Thu Apr 25, 2024 12:43:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
228812970,0sleep10-21swapper/111:19:171
32482940,2sleep5333799cyclictest10:08:415
192632890,2sleep2331599cyclictest12:26:372
74642830,1sleep47462-21ls10:58:464
296125127,8sleep70-21swapper/707:06:267
295525127,8sleep30-21swapper/307:06:223
283525126,8sleep40-21swapper/407:04:394
190412490,0sleep40-21swapper/411:57:164
223952470,0sleep40-21swapper/412:17:104
291672460,0sleep30-21swapper/311:23:113
320722440,0sleep50-21swapper/510:41:175
302712440,0sleep20-21swapper/211:58:262
50032430,0sleep00-21swapper/012:12:100
305024230,8sleep00-21swapper/007:07:400
299824230,8sleep60-21swapper/607:06:576
114932420,0sleep10-21swapper/110:02:531
57472410,0sleep00-21swapper/012:31:480
41432410,0sleep10-21swapper/111:11:241
50492400,0sleep00-21swapper/011:23:560
287224028,9sleep50-21swapper/507:05:125
68022390,0sleep70-21swapper/712:31:557
303223928,8sleep20-21swapper/207:07:262
171112390,2sleep7335199cyclictest10:52:567
285923825,9sleep10-21swapper/107:05:001
76382360,0sleep50-21swapper/509:52:155
76382360,0sleep50-21swapper/509:52:155
284012340,1sleep4488-21systemd-journal11:42:024
111492230,1sleep449-21ksoftirqd/410:42:284
45262190,1sleep10-21swapper/110:22:151
24152180,0sleep30-21swapper/311:33:043
119752180,1sleep111974-21grepconf.sh12:06:181
47292160,0sleep20-21swapper/211:52:332
332399162,1cyclictest0-21swapper/310:39:123
278522160,0sleep7721ktimersoftd/709:34:157
167062160,1sleep316702-21sshd12:13:253
3300991514,0cyclictest0-21swapper/010:49:420
3300991513,2cyclictest0-21swapper/008:50:260
198792150,0sleep70-21swapper/709:10:217
198792150,0sleep70-21swapper/709:10:217
75822140,1sleep77574-21sshd12:09:047
3344991410,3cyclictest65-21ksoftirqd/610:08:356
332399141,10cyclictest0-21swapper/310:01:413
3315991411,1cyclictest33-21ksoftirqd/210:19:022
330099142,9cyclictest919-21kworker/0:412:05:250
298922140,0sleep70-21swapper/711:07:387
232282140,0sleep10-21swapper/109:27:291
334499136,2cyclictest65-21ksoftirqd/609:44:336
334499130,12cyclictest21316-21systemd12:20:136
333799132,5cyclictest488-21systemd-journal10:54:095
3315991311,1cyclictest33-21ksoftirqd/209:43:552
330099139,1cyclictest9-21ksoftirqd/012:24:320
3300991312,1cyclictest9-21ksoftirqd/009:45:230
3300991311,1cyclictest9-21ksoftirqd/011:01:500
330099130,12cyclictest1-21systemd11:57:300
156132130,0sleep1241ktimersoftd/110:13:261
5972120,0sleep01-21systemd11:20:200
3351991210,1cyclictest9160-21systemd-cgroups12:25:337
335199120,11cyclictest0-21swapper/709:54:127
334499129,1cyclictest65-21ksoftirqd/612:28:416
3344991210,1cyclictest65-21ksoftirqd/610:33:146
333799124,7cyclictest879-21polkitd12:26:055
333799122,4cyclictest8609-21sshd11:18:055
333799122,4cyclictest14418-21systemd09:36:175
333799121,5cyclictest754-21NetworkManager10:11:555
333799120,1cyclictest879-21polkitd11:00:215
333199129,2cyclictest1300-21bash11:48:584
333199123,4cyclictest0-21swapper/409:27:054
333199121,5cyclictest488-21systemd-journal11:10:354
333199120,11cyclictest10722-21sshd10:06:014
3323991210,1cyclictest0-21swapper/311:56:363
3315991210,2cyclictest33-21ksoftirqd/209:12:592
3315991210,2cyclictest33-21ksoftirqd/209:12:592
331599120,11cyclictest653-21sshd09:18:492
330899120,11cyclictest0-21swapper/109:21:301
330099129,2cyclictest9-21ksoftirqd/009:49:060
330099129,2cyclictest9-21ksoftirqd/009:49:060
330099129,1cyclictest9-21ksoftirqd/011:29:340
330099129,1cyclictest9-21ksoftirqd/010:39:580
330099129,1cyclictest488-21systemd-journal11:50:320
330099127,1cyclictest9-21ksoftirqd/009:39:300
3300991210,2cyclictest698-21dbus-daemon10:05:550
3300991210,1cyclictest9-21ksoftirqd/012:37:370
3300991210,1cyclictest9-21ksoftirqd/011:39:000
3300991210,1cyclictest9-21ksoftirqd/009:56:090
3300991210,1cyclictest9-21ksoftirqd/009:35:360
330099121,10cyclictest0-21swapper/009:25:250
330099120,12cyclictest0-21swapper/010:25:360
330099120,11cyclictest5268-21systemd-cgroups11:14:300
330099120,11cyclictest5197-21kworker/0:111:59:000
158122120,0sleep20-21swapper/209:49:402
158122120,0sleep20-21swapper/209:49:402
109912120,0sleep20-21swapper/207:19:102
109912120,0sleep20-21swapper/207:19:102
335199115,3cyclictest3482-21sshd10:08:437
335199114,4cyclictest6059-21sshd10:35:197
335199110,1cyclictest10264-21systemd-cgroups11:02:287
334499119,1cyclictest65-21ksoftirqd/611:43:306
334499119,1cyclictest65-21ksoftirqd/610:25:256
334499119,1cyclictest65-21ksoftirqd/610:18:566
334499119,1cyclictest3793-21sshd10:48:186
334499118,2cyclictest29601-21grepconf.sh11:10:426
334499117,1cyclictest65-21ksoftirqd/611:48:396
334499117,1cyclictest65-21ksoftirqd/609:58:396
334499114,4cyclictest65-21ksoftirqd/611:16:326
334499112,1cyclictest65-21ksoftirqd/611:49:126
3344991110,1cyclictest65-21ksoftirqd/611:23:296
334499111,10cyclictest3372-21kworker/6:410:50:226
334499111,1cyclictest131rcu_sched10:34:006
334499111,1cyclictest131rcu_sched09:19:096
334499110,8cyclictest793-21kworker/6:112:14:056
333799114,6cyclictest698-21dbus-daemon09:17:435
333799114,5cyclictest488-21systemd-journal11:45:275
333799112,4cyclictest27986-21bash09:54:135
333799112,4cyclictest25730-21sshd11:41:465
333799112,4cyclictest0-21swapper/509:46:375
333799112,4cyclictest0-21swapper/509:28:365
333799111,7cyclictest0-21swapper/511:35:395
333799111,5cyclictest1-21systemd12:02:595
333799111,4cyclictest28352-21grep10:01:135
333799110,11cyclictest0-21swapper/509:30:295
333799110,10cyclictest0-21swapper/510:49:095
333799110,1cyclictest720-21in:imjournal09:23:425
333799110,1cyclictest11935-21bash11:12:105
333799110,1cyclictest0-21swapper/512:38:155
333199119,1cyclictest0-21swapper/412:31:414
333199111,4cyclictest22113-21sshd10:50:034
333199110,11cyclictest0-21swapper/411:18:164
333199110,10cyclictest698-21dbus-daemon10:24:094
333199110,10cyclictest0-21swapper/409:49:284
333199110,10cyclictest0-21swapper/409:49:284
332399115,4cyclictest698-21dbus-daemon12:22:183
332399115,3cyclictest15197-21sshd10:19:563
332399110,10cyclictest24893-21systemd-cgroups09:11:043
332399110,10cyclictest24893-21systemd-cgroups09:11:043
332399110,10cyclictest24386-21bash11:13:253
331599119,2cyclictest33-21ksoftirqd/210:57:532
331599119,2cyclictest27926-21sshd11:45:072
331599119,1cyclictest33-21ksoftirqd/212:17:012
331599119,1cyclictest33-21ksoftirqd/211:04:332
331599119,1cyclictest33-21ksoftirqd/210:10:152
331599118,2cyclictest33-21ksoftirqd/212:38:272
331599118,2cyclictest33-21ksoftirqd/210:43:122
331599116,3cyclictest33-21ksoftirqd/211:35:412
331599116,3cyclictest16849-21grep11:31:152
331599115,4cyclictest0-21swapper/210:52:392
331599110,11cyclictest5342-21kworker/2:310:03:512
331599110,11cyclictest29816-21kworker/2:409:31:362
330899119,2cyclictest0-21swapper/110:41:351
330899110,10cyclictest0-21swapper/110:53:341
330099119,1cyclictest9-21ksoftirqd/011:44:010
330099119,1cyclictest9-21ksoftirqd/010:38:270
330099119,1cyclictest9-21ksoftirqd/010:10:520
330099119,1cyclictest9-21ksoftirqd/010:02:560
330099119,1cyclictest9-21ksoftirqd/009:32:100
330099119,1cyclictest9-21ksoftirqd/009:19:500
330099119,1cyclictest31689-21sshd09:12:000
330099119,1cyclictest31689-21sshd09:12:000
330099118,1cyclictest9-21ksoftirqd/012:20:140
3300991110,1cyclictest9-21ksoftirqd/010:54:560
330099111,1cyclictest3153-21kworker/0:410:22:180
330099110,2cyclictest131rcu_sched12:18:300
330099110,11cyclictest0-21swapper/010:44:100
330099110,10cyclictest1370-21kworker/0:011:05:560
178102110,0sleep20-21swapper/212:00:182
335199109,1cyclictest0-21swapper/710:20:327
335199109,1cyclictest0-21swapper/709:31:497
335199109,1cyclictest0-21swapper/709:24:537
335199104,3cyclictest4515-21systemd-cgroups11:46:037
335199104,3cyclictest12064-21sshd11:49:577
335199100,10cyclictest0-21swapper/711:43:287
335199100,10cyclictest0-21swapper/710:29:507
335199100,0cyclictest0-21swapper/710:02:347
335199100,0cyclictest0-21swapper/709:39:047
334499109,1cyclictest65-21ksoftirqd/611:05:406
334499109,1cyclictest65-21ksoftirqd/609:53:406
334499109,1cyclictest65-21ksoftirqd/609:53:406
334499109,1cyclictest65-21ksoftirqd/609:24:026
334499109,1cyclictest65-21ksoftirqd/609:14:326
334499109,1cyclictest65-21ksoftirqd/609:12:526
334499109,1cyclictest65-21ksoftirqd/609:12:526
334499109,1cyclictest20888-21gdbus09:33:096
334499108,2cyclictest65-21ksoftirqd/611:54:486
334499108,1cyclictest65-21ksoftirqd/612:29:076
334499108,1cyclictest65-21ksoftirqd/612:10:296
334499108,1cyclictest65-21ksoftirqd/612:04:346
334499108,1cyclictest65-21ksoftirqd/611:33:596
334499108,1cyclictest65-21ksoftirqd/611:30:386
334499108,1cyclictest65-21ksoftirqd/611:24:096
334499108,1cyclictest65-21ksoftirqd/609:40:056
334499107,2cyclictest1-21systemd09:37:226
334499107,1cyclictest65-21ksoftirqd/611:59:006
334499107,1cyclictest65-21ksoftirqd/610:16:066
334499100,10cyclictest0-21swapper/612:35:216
334499100,10cyclictest0-21swapper/610:59:306
333799109,1cyclictest0-21swapper/512:06:325
333799103,6cyclictest1-21systemd11:49:295
333799101,4cyclictest0-21swapper/512:14:025
333799100,5cyclictest0-21swapper/512:32:415
333799100,4cyclictest25025-21grepconf.sh12:20:375
333799100,4cyclictest1565-21sshd10:44:385
333799100,10cyclictest8802-21kworker/5:410:26:545
333799100,10cyclictest8149-21kworker/5:309:12:595
333799100,10cyclictest8149-21kworker/5:309:12:595
333199108,1cyclictest717-21systemd-logind09:17:454
333199103,6cyclictest24008-21sshd10:33:574
333199103,1cyclictest18712-21sshd11:34:314
333199102,7cyclictest4314-21bash09:48:424
333199100,4cyclictest879-21polkitd09:28:534
332399109,1cyclictest41-21ksoftirqd/311:27:393
332399109,1cyclictest0-21swapper/310:28:383
332399109,1cyclictest0-21swapper/309:53:103
332399109,1cyclictest0-21swapper/309:53:103
332399109,1cyclictest0-21swapper/309:31:063
332399109,1cyclictest0-21swapper/308:08:573
332399105,3cyclictest24287-21bash10:47:053
331599109,1cyclictest33-21ksoftirqd/212:22:522
331599109,1cyclictest33-21ksoftirqd/211:41:532
331599109,1cyclictest33-21ksoftirqd/211:15:482
331599109,1cyclictest33-21ksoftirqd/211:09:152
331599109,1cyclictest33-21ksoftirqd/210:16:512
331599108,2cyclictest33-21ksoftirqd/210:28:492
331599108,1cyclictest33-21ksoftirqd/212:12:102
331599108,1cyclictest33-21ksoftirqd/212:07:052
331599108,1cyclictest33-21ksoftirqd/210:38:332
331599107,1cyclictest0-21swapper/212:31:152
331599103,1cyclictest698-21dbus-daemon11:20:592
331599100,10cyclictest5342-21kworker/2:310:48:212
330899109,1cyclictest25-21ksoftirqd/110:47:341
330899109,1cyclictest0-21swapper/109:14:341
330899104,3cyclictest10807-21sshd11:49:481
330899103,4cyclictest22807-21sshd10:07:191
330899100,1cyclictest0-21swapper/111:05:141
330099109,1cyclictest9-21ksoftirqd/011:34:050
330099109,1cyclictest0-21swapper/007:58:160
330099108,2cyclictest1-21systemd09:14:320
330099108,1cyclictest9-21ksoftirqd/011:10:240
330099108,1cyclictest9-21ksoftirqd/010:29:300
330099108,1cyclictest9-21ksoftirqd/010:14:110
293882100,0sleep50-21swapper/512:11:185
143922100,1sleep3879-21polkitd11:15:263
33519999,0cyclictest0-21swapper/711:28:387
33519998,1cyclictest0-21swapper/710:40:247
33519997,1cyclictest21145-21df07:38:567
33519994,2cyclictest18372-21sshd11:15:517
33519993,4cyclictest28187-21kworker/u16:209:18:577
33519993,3cyclictest698-21dbus-daemon10:24:377
33519993,3cyclictest17605-21sshd10:16:557
33519992,5cyclictest1152-21kworker/u16:311:11:497
33519992,4cyclictest1086-21systemd11:20:237
33519992,3cyclictest31670-21systemd-cgroups09:51:217
33519992,3cyclictest31670-21systemd-cgroups09:51:217
33519991,4cyclictest15084-21bash09:16:597
33519990,9cyclictest23676-21kworker/7:011:29:247
33519990,9cyclictest0-21swapper/710:11:237
33519990,0cyclictest0-21swapper/711:57:587
33449998,1cyclictest1-21systemd10:09:126
33449997,1cyclictest65-21ksoftirqd/610:54:536
33449997,1cyclictest65-21ksoftirqd/610:38:556
33449990,9cyclictest0-21swapper/608:29:156
33449990,4cyclictest0-21swapper/610:00:186
33379994,1cyclictest698-21dbus-daemon11:23:185
33379993,3cyclictest1-21systemd10:34:275
33379993,3cyclictest0-21swapper/511:24:395
33379992,6cyclictest32744-21systemd11:04:345
33379992,3cyclictest879-21polkitd10:20:565
33379991,4cyclictest879-21polkitd10:30:115
33379990,9cyclictest0-21swapper/508:43:595
33379990,4cyclictest488-21systemd-journal10:14:425
33379990,4cyclictest0-21swapper/511:29:055
33379990,0cyclictest0-21swapper/511:54:445
33319999,0cyclictest0-21swapper/411:19:574
33319999,0cyclictest0-21swapper/409:20:464
33319994,4cyclictest879-21polkitd10:19:084
33319993,5cyclictest18496-21systemctl11:06:294
33319992,6cyclictest879-21polkitd10:47:084
33319992,3cyclictest879-21polkitd12:02:044
33319992,1cyclictest879-21polkitd10:09:594
33319991,7cyclictest12103-21systemd12:06:194
33319990,9cyclictest0-21swapper/409:41:384
33319990,8cyclictest13665-21kworker/4:511:45:254
33319990,0cyclictest0-21swapper/410:33:274
33319990,0cyclictest0-21swapper/409:34:394
33239999,0cyclictest0-21swapper/311:41:573
33239997,1cyclictest41-21ksoftirqd/312:18:113
33239997,1cyclictest41-21ksoftirqd/310:05:593
33239997,1cyclictest0-21swapper/309:45:263
33239995,2cyclictest24197-21sshd09:17:553
33239995,1cyclictest3601-21cp10:54:403
33239994,3cyclictest4071-21grepconf.sh09:38:343
33239992,4cyclictest13361-21sd-pam11:05:583
33239992,3cyclictest28693-21systemd-cgroups11:01:013
33239990,8cyclictest28256-21kworker/3:212:04:063
33239990,8cyclictest28256-21kworker/3:211:50:163
33239990,0cyclictest0-21swapper/310:50:143
33239990,0cyclictest0-21swapper/310:10:403
33159997,2cyclictest33-21ksoftirqd/210:24:162
33159997,2cyclictest33-21ksoftirqd/210:00:172
33159997,2cyclictest33-21ksoftirqd/209:58:032
33159997,1cyclictest33-21ksoftirqd/209:33:572
33159997,1cyclictest33-21ksoftirqd/209:27:572
33159993,4cyclictest8376-21sshd11:02:172
33159992,6cyclictest0-21swapper/211:28:172
33089999,0cyclictest0-21swapper/112:21:131
33089998,1cyclictest25-21ksoftirqd/111:16:221
33089998,1cyclictest0-21swapper/111:31:371
33089997,1cyclictest25-21ksoftirqd/109:40:501
33089995,2cyclictest22896-21kworker/1:211:35:391
33089994,4cyclictest20290-21sshd12:33:191
33089994,3cyclictest18184-21sshd09:56:471
33089994,3cyclictest1133-21grepconf.sh09:38:171
33089994,3cyclictest11265-21sshd10:59:041
33089994,2cyclictest5465-21kill12:02:251
33089993,3cyclictest745-21rs:main1
33089992,3cyclictest22208-21sshd10:33:501
33089990,9cyclictest12393-21kworker/1:210:32:261
33089990,8cyclictest17065-21kworker/1:212:17:361
33089990,8cyclictest17065-21kworker/1:212:11:311
33009998,0cyclictest0-21swapper/007:48:020
33009997,2cyclictest1888-21nfsd07:39:220
33009994,0cyclictest0-21swapper/008:47:080
33519986,1cyclictest0-21swapper/709:03:457
33519985,2cyclictest1110-21grepconf.sh12:34:287
33519984,3cyclictest15518-21sshd10:55:587
33519983,3cyclictest20701-21sshd12:13:487
33519983,3cyclictest19508-21systemd11:34:377
33519983,1cyclictest0-21swapper/707:46:467
33519982,3cyclictest30804-21sshd12:21:147
33519982,3cyclictest30757-21sshd10:44:167
33519982,3cyclictest0-21swapper/709:44:467
33519980,8cyclictest0-21swapper/712:04:587
33519980,7cyclictest23284-21kworker/7:311:58:487
33379982,6cyclictest0-21swapper/509:39:495
33319988,0cyclictest0-21swapper/410:14:514
33319985,2cyclictest698-21dbus-daemon12:33:574
33319985,2cyclictest698-21dbus-daemon11:01:074
33319985,2cyclictest17737-21grepconf.sh12:10:044
33319984,2cyclictest10329-21missed_timers09:09:024
33319984,2cyclictest10329-21missed_timers09:09:024
33319984,1cyclictest25656-21httpd08:58:364
33319983,3cyclictest698-21dbus-daemon11:24:484
33319980,8cyclictest26439-21kworker/4:109:53:534
33319980,8cyclictest18549-21kworker/4:112:19:454
33319980,4cyclictest26428-21systemd-cgroups12:24:024
33319980,2cyclictest879-21polkitd09:59:194
33239988,0cyclictest0-21swapper/312:25:083
33239988,0cyclictest0-21swapper/308:29:083
33239988,0cyclictest0-21swapper/307:09:033
33239985,1cyclictest41-21ksoftirqd/310:14:343
33239982,4cyclictest30606-21sshd09:54:313
33239982,3cyclictest25775-21systemd12:33:513
33239982,3cyclictest23240-21bash09:20:593
33239982,3cyclictest21571-21grepconf.sh11:34:493
33239982,3cyclictest20060-21bash10:30:153
33239982,3cyclictest1-21systemd12:30:573
33239982,2cyclictest7879-21systemd11:59:153
33239980,8cyclictest7586-21kworker/3:110:36:483
33239980,8cyclictest21035-21kworker/3:111:44:243
33239980,6cyclictest0-21swapper/309:24:133
33159987,1cyclictest33-21ksoftirqd/209:39:002
33159987,1cyclictest33-21ksoftirqd/209:15:532
33089987,1cyclictest0-21swapper/107:48:571
33089985,2cyclictest881-21gdbus10:24:031
33089985,2cyclictest17064-21bash10:56:081
33089985,2cyclictest16158-21systemd11:44:011
33089982,5cyclictest17770-21sd-pam09:46:441
33089982,3cyclictest698-21dbus-daemon11:24:491
33089982,3cyclictest20888-21gdbus10:15:481
33089982,3cyclictest0-21swapper/111:55:001
33089981,3cyclictest717-21systemd-logind09:10:121
33089981,3cyclictest717-21systemd-logind09:10:121
33089980,8cyclictest0-21swapper/109:51:371
33089980,8cyclictest0-21swapper/109:51:371
33089980,7cyclictest14672-21kworker/1:012:34:111
33089980,7cyclictest14672-21kworker/1:012:25:031
33009987,0cyclictest0-21swapper/007:20:080
33009987,0cyclictest0-21swapper/007:20:080
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional