You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-18 - 17:43
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack2slot0.osadl.org (updated Thu Sep 18, 2025 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1513821040,0sleep015117-21sshd12:03:510
798521010,0sleep37982-21sshd11:00:493
174872980,1sleep217488-21bash09:55:122
59302720,0sleep10-21swapper/109:57:151
313072680,0sleep00-21swapper/011:46:440
14082660,0sleep00-21swapper/010:03:270
27172580,0sleep20-21swapper/212:31:042
205622580,0sleep30-21swapper/310:46:323
79832570,0sleep10-21swapper/110:26:471
16852570,0sleep00-21swapper/012:05:350
286682550,0sleep30-21swapper/310:12:553
145342530,0sleep30-21swapper/308:15:013
23222510,0sleep00-21swapper/012:37:390
215532510,0sleep10-21swapper/111:36:271
99482500,0sleep10-21swapper/110:54:551
60102500,0sleep30-21swapper/309:43:593
51712500,0sleep25175-21bash10:48:112
293952500,0sleep30-21swapper/311:06:023
26932500,0sleep10-21swapper/109:20:021
269282500,0sleep20-21swapper/212:27:032
221302500,0sleep30-21swapper/309:25:243
128192490,0sleep00-21swapper/009:57:580
94182470,0sleep10-21swapper/111:47:461
75052470,0sleep00-21swapper/009:17:040
127322470,0sleep00-21swapper/010:21:000
29232460,2sleep21973799cyclictest09:16:352
251802460,1sleep125-21ksoftirqd/109:42:361
195922450,0sleep30-21swapper/311:30:093
122032410,1sleep3370-21jbd2/sda1-812:06:383
252452400,0sleep00-21swapper/009:25:460
116872400,0sleep20-21swapper/211:07:292
292852390,2sleep01972499cyclictest12:20:490
1958923631,3sleep10-21swapper/107:09:341
266082350,0sleep30-21swapper/310:05:583
212762320,14sleep375550irq/24-eno1-rx-09:15:123
194822298,7sleep00-21swapper/007:08:130
1942322311,7sleep30-21swapper/307:07:283
1784222210,7sleep20-21swapper/207:05:102
235002200,0sleep20-21swapper/212:04:402
122382160,0sleep10-21swapper/111:29:271
112872160,0sleep3391rcuc/311:16:483
223112150,2sleep01972499cyclictest12:16:590
1973099152,4cyclictest5886-21grepconf.sh12:18:351
19724991513,1cyclictest1-21systemd11:38:350
40862140,0sleep20-21swapper/211:59:402
225542140,0sleep20-21swapper/210:40:302
200382140,0sleep220041-21bash10:24:562
19737991412,1cyclictest15214-21lspci09:21:172
1973099149,4cyclictest27738-21sshd11:18:251
1973099142,4cyclictest2390-21bash10:38:401
1973099140,6cyclictest30243-21sshd09:49:481
1973099140,6cyclictest12262-21bash10:20:581
194342140,0sleep30-21swapper/310:58:523
158742140,0sleep10-21swapper/111:04:481
102432140,0sleep00-21swapper/011:10:230
217072130,0sleep10-21swapper/112:20:101
1974199130,12cyclictest19026-21grep12:36:003
1973099131,6cyclictest28500-21tty12:08:141
1973099130,6cyclictest646-21dbus-daemon10:58:311
1973099130,6cyclictest14563-21grep11:20:141
1973099130,10cyclictest0-21swapper/110:00:351
1973099130,0cyclictest0-21swapper/112:01:061
1973099130,0cyclictest0-21swapper/111:10:261
19724991311,1cyclictest7166-21bash12:34:550
19724991311,1cyclictest15179-21grep10:39:550
1972499130,12cyclictest0-21swapper/010:53:450
1972499130,11cyclictest6215-21grep09:30:340
1972499130,0cyclictest0-21swapper/010:34:250
175462130,0sleep00-21swapper/011:54:440
169602130,0sleep00-21swapper/010:40:040
138002130,0sleep30-21swapper/311:20:113
19741991210,1cyclictest7068-21bash10:42:103
19741991210,1cyclictest649-21avahi-daemon11:47:203
19741991210,1cyclictest18907-21sshd12:10:203
1974199120,11cyclictest6057-21smart_sda09:20:203
1974199120,11cyclictest19981-21sshd11:55:003
1974199120,11cyclictest16786-21id11:39:103
1974199120,11cyclictest0-21swapper/310:21:203
19737991211,0cyclictest17299-21snmpd11:46:022
19737991210,1cyclictest13836-21bash10:39:462
19737991210,1cyclictest0-21swapper/211:40:372
1973799120,11cyclictest0-21swapper/211:23:072
1973099122,7cyclictest23825-21sshd11:42:581
19730991210,1cyclictest13066-21grep10:11:151
1973099120,8cyclictest893-21sshd11:34:351
1973099120,8cyclictest13254-21sshd12:28:581
1973099120,6cyclictest9386-21sshd09:50:521
1973099120,6cyclictest646-21dbus-daemon10:49:371
1973099120,6cyclictest18210-21lspci10:15:061
1973099120,5cyclictest16248-21sshd10:43:031
1973099120,1cyclictest4903-21irqstats07:50:141
1973099120,1cyclictest18144-21sshd09:31:531
1973099120,1cyclictest0-21swapper/111:08:561
1973099120,11cyclictest11790-21bash11:57:161
1973099120,11cyclictest1011-21sshd12:38:161
1973099120,11cyclictest0-21swapper/109:08:151
1972499129,2cyclictest26765-21sshd10:28:370
1972499120,1cyclictest23730-21sshd11:21:050
1972499120,11cyclictest6914-21sshd10:10:340
1972499120,11cyclictest16288-21bash09:24:540
107572120,0sleep30-21swapper/311:10:273
1974199110,10cyclictest75550irq/24-eno1-rx-10:31:103
1974199110,10cyclictest0-21swapper/309:10:303
1974199110,10cyclictest0-21swapper/309:10:303
1974199110,0cyclictest0-21swapper/310:27:203
1973799119,1cyclictest30358-21sshd09:12:362
1973799119,1cyclictest30358-21sshd09:12:362
1973799119,1cyclictest29817-21sshd09:53:052
1973799119,1cyclictest18661-21lspci09:28:342
19737991110,0cyclictest0-21swapper/212:15:372
19737991110,0cyclictest0-21swapper/211:52:372
19737991110,0cyclictest0-21swapper/210:17:462
1973799110,10cyclictest0-21swapper/211:11:372
1973799110,0cyclictest0-21swapper/210:04:162
1973099115,5cyclictest20213-21sshd09:15:041
1973099112,5cyclictest31084-21cat07:35:141
1973099112,5cyclictest29698-21sshd12:11:251
1973099110,8cyclictest27564-21tty12:30:191
1973099110,8cyclictest12690-21grepconf.sh09:10:081
1973099110,8cyclictest12690-21grepconf.sh09:10:081
1973099110,5cyclictest5222-21tty10:07:081
1973099110,5cyclictest32600-21sshd09:36:441
1973099110,5cyclictest14118-21grep10:30:221
1973099110,1cyclictest11682-21sshd09:27:521
1973099110,1cyclictest0-21swapper/111:52:361
1972499110,10cyclictest0-21swapper/011:03:350
1972499110,0cyclictest0-21swapper/011:09:350
1972499110,0cyclictest0-21swapper/010:06:030
25762100,0sleep20-21swapper/212:37:412
1974199109,0cyclictest0-21swapper/312:19:403
1974199109,0cyclictest0-21swapper/309:40:003
1974199108,1cyclictest20029-21seq08:25:183
1973799109,0cyclictest7256-21lspci11:25:442
1973799109,0cyclictest2116-21lspci11:34:412
1973799109,0cyclictest0-21swapper/210:51:562
1973799109,0cyclictest0-21swapper/209:37:362
1973799109,0cyclictest0-21swapper/207:20:162
1973099107,2cyclictest0-21swapper/107:55:451
1973099100,6cyclictest29462-21ntp_states07:30:141
1972499109,0cyclictest0-21swapper/011:58:150
1972499109,0cyclictest0-21swapper/011:44:350
1972499100,0cyclictest0-21swapper/008:06:230
27326290,0sleep227323-21lspci10:34:522
27245290,1sleep3391rcuc/312:20:363
197419998,0cyclictest0-21swapper/311:41:203
197419998,0cyclictest0-21swapper/311:25:303
197419998,0cyclictest0-21swapper/309:32:503
197419997,1cyclictest4510-21grepconf.sh10:16:493
197419996,2cyclictest9160-21sshd09:50:503
197379998,0cyclictest0-21swapper/212:24:572
197379998,0cyclictest0-21swapper/212:11:472
197379998,0cyclictest0-21swapper/210:59:362
197379990,0cyclictest0-21swapper/208:09:562
197309993,5cyclictest17299-21snmpd08:18:091
197309993,5cyclictest17299-21snmpd08:12:071
197309993,5cyclictest17299-21snmpd07:40:331
197309993,5cyclictest17299-21snmpd07:17:111
197309991,4cyclictest0-21swapper/108:05:181
197309990,1cyclictest4147-21sadc07:50:001
197309990,1cyclictest25395-21ntpq07:20:161
197309990,1cyclictest0-21swapper/108:25:171
197309990,0cyclictest0-21swapper/108:50:201
197249998,0cyclictest0-21swapper/010:48:440
197249998,0cyclictest0-21swapper/010:15:040
197249998,0cyclictest0-21swapper/008:47:240
197249998,0cyclictest0-21swapper/008:17:240
197249997,1cyclictest31072-21sshd09:43:130
197249990,0cyclictest0-21swapper/007:27:230
197419985,2cyclictest649-21avahi-daemon10:00:473
197419985,2cyclictest11712-21sshd10:36:253
197419984,3cyclictest649-21avahi-daemon12:04:323
197419984,2cyclictest700-21gdbus12:34:343
197379987,0cyclictest11194-21lspci09:44:322
197379986,1cyclictest33-21ksoftirqd/211:15:582
197379985,2cyclictest649-21avahi-daemon12:05:232
197379985,2cyclictest649-21avahi-daemon11:38:542
197379985,2cyclictest649-21avahi-daemon11:00:242
197309983,4cyclictest17299-21snmpd08:58:131
197309983,4cyclictest17299-21snmpd07:11:341
197309982,5cyclictest8935-21cat08:00:141
197309982,5cyclictest28546-21sadc08:50:001
197309982,5cyclictest27851-21grep07:25:231
197309982,5cyclictest17299-21snmpd08:21:041
197309981,6cyclictest394-21/usr/sbin/munin09:00:121
197309980,1cyclictest23946-21sed08:35:201
197249987,0cyclictest0-21swapper/011:28:440
197249987,0cyclictest0-21swapper/008:30:230
197249986,1cyclictest31461-21lspci11:15:350
197249986,1cyclictest18053-21sshd09:10:440
197249986,1cyclictest18053-21sshd09:10:440
19033280,0sleep20-21swapper/210:11:542
197419976,0cyclictest0-21swapper/309:09:203
197419975,1cyclictest646-21dbus-daemon09:48:083
197419975,1cyclictest28059-21sshd11:55:403
197419975,1cyclictest27641-21lspci10:50:163
197419973,3cyclictest649-21avahi-daemon09:57:483
197379974,2cyclictest649-21avahi-daemon09:32:072
197379974,2cyclictest648-21systemd-logind10:26:162
197379974,1cyclictest20653-21sshd10:05:212
197379974,1cyclictest14059-21sshd09:48:082
197309971,5cyclictest20720-21cat08:30:111
197309970,0cyclictest0-21swapper/108:40:151
197249976,0cyclictest0-21swapper/011:32:350
197249976,0cyclictest0-21swapper/007:11:240
197249975,1cyclictest700-21gdbus12:26:400
197249975,1cyclictest648-21systemd-logind09:46:090
197249975,1cyclictest14748-21sshd10:58:260
197249974,2cyclictest649-21avahi-daemon12:12:490
197249974,2cyclictest23772-21sshd09:35:480
197249970,0cyclictest0-21swapper/008:50:340
197419963,2cyclictest649-21avahi-daemon12:25:053
197419962,2cyclictest19910-21sshd07:10:073
197419960,5cyclictest0-21swapper/308:42:193
197419960,5cyclictest0-21swapper/308:06:493
197419960,1cyclictest26888-21tr08:45:133
197379965,0cyclictest5184-21lspci09:07:162
197379964,1cyclictest31104-21mii-tool07:35:132
197379962,1cyclictest0-21swapper/207:17:202
197379960,5cyclictest0-21swapper/208:40:452
197249965,0cyclictest0-21swapper/007:45:440
197249964,1cyclictest649-21avahi-daemon09:08:250
197249963,2cyclictest646-21dbus-daemon09:50:040
197249963,2cyclictest17299-21snmpd08:25:070
197249960,5cyclictest0-21swapper/008:55:530
197249960,5cyclictest0-21swapper/008:11:540
197419954,0cyclictest0-21swapper/308:30:193
197419954,0cyclictest0-21swapper/307:35:283
197419953,1cyclictest8899-21cat08:00:133
197419953,1cyclictest682-21lsmd08:55:193
197419953,1cyclictest2799-21cat07:45:143
197419953,1cyclictest23771-21cat08:35:203
197419953,1cyclictest17299-21snmpd09:00:053
197419953,1cyclictest17299-21snmpd08:20:013
197419953,1cyclictest17299-21snmpd08:11:223
197419953,1cyclictest17299-21snmpd07:56:573
197419953,1cyclictest17299-21snmpd07:51:033
197419953,1cyclictest17299-21snmpd07:41:003
197419953,1cyclictest17299-21snmpd07:30:413
197419953,1cyclictest17299-21snmpd07:25:233
197419950,4cyclictest28780-21sshd08:50:083
197379953,1cyclictest682-21lsmd07:27:452
197379953,1cyclictest620-13audispd07:40:202
197379953,1cyclictest19001-21hwlatdetect08:25:112
197379953,1cyclictest17299-21snmpd08:52:262
197379953,1cyclictest17299-21snmpd08:00:042
197379953,1cyclictest17299-21snmpd07:55:122
197379953,1cyclictest17299-21snmpd07:52:002
197379953,1cyclictest17299-21snmpd07:49:182
197379953,1cyclictest17299-21snmpd07:13:042
197379953,1cyclictest1053-21lldpd09:00:202
197379952,2cyclictest23301-21/usr/sbin/munin08:35:152
197379950,4cyclictest28596-21/usr/sbin/munin07:30:092
197379950,2cyclictest12684-21/usr/sbin/munin08:10:122
197379950,0cyclictest0-21swapper/208:47:462
197249954,0cyclictest0-21swapper/007:35:540
197249953,1cyclictest682-21lsmd07:34:450
197249953,1cyclictest22992-21cat07:15:130
197249953,1cyclictest17299-21snmpd08:40:110
197249953,1cyclictest17299-21snmpd08:00:340
197249953,1cyclictest17299-21snmpd07:58:000
197249953,1cyclictest17299-21snmpd07:50:170
197249953,1cyclictest17299-21snmpd07:41:030
197249953,1cyclictest17299-21snmpd07:20:350
197249953,1cyclictest1058-21sendmail08:35:160
197249953,1cyclictest1053-21lldpd08:20:350
197419942,1cyclictest22134-21if_err_eno107:15:013
197419940,3cyclictest24688-21cstates07:20:113
197379942,1cyclictest75650irq/25-eno1-tx-08:55:122
197379942,1cyclictest649-21avahi-daemon08:20:082
197379942,1cyclictest20910-21expr08:30:112
197379942,1cyclictest0-21swapper/208:15:102
197249942,1cyclictest566-21cstates09:00:100
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional