You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-13 - 03:11
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackaslot8.osadl.org (updated Tue May 13, 2025 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
74082322290,20sleep20-21swapper/219:06:372
75842281214,22sleep30-21swapper/319:08:553
74892275242,21sleep00-21swapper/019:07:390
74952273237,22sleep10-21swapper/119:07:441
239062600,0sleep20-21swapper/222:59:462
245572590,0sleep00-21swapper/022:14:270
284642580,0sleep30-21swapper/322:40:013
185412210,1sleep30-21swapper/300:31:213
199252200,0sleep10-21swapper/122:55:571
270092190,1sleep00-21swapper/021:53:330
57032180,0sleep10-21swapper/121:15:211
312922180,0sleep30-21swapper/323:26:343
297942180,0sleep00-21swapper/023:03:410
778699122,3cyclictest6916-21munin-plugin-st22:25:013
778699111,3cyclictest4184-21ls21:15:013
777899119,1cyclictest19701-21apt19:35:011
778699101,6cyclictest8548-21wc21:40:173
778699100,7cyclictest12275-21ssh22:50:223
777899105,3cyclictest19982-21systemd-journal00:39:401
77869997,1cyclictest31718-21apt-get23:05:153
77869995,2cyclictest25951-21ssh00:10:143
77869994,3cyclictest19982-21systemd-journal20:09:193
77869993,4cyclictest19982-21systemd-journal23:44:113
77869990,6cyclictest31573-21ssh00:15:063
77869990,6cyclictest26201-21ssh23:23:313
77869990,6cyclictest23327-21ssh22:13:203
77869990,6cyclictest13850-21sh22:06:093
77869990,6cyclictest12869-21ssh23:36:033
77789994,3cyclictest19982-21systemd-journal22:40:461
77789994,3cyclictest19982-21systemd-journal20:42:161
77789990,8cyclictest16402-21sed21:45:231
6483290,0sleep00-21swapper/022:24:360
23802290,1sleep20-21swapper/220:55:162
22035290,1chrt12-21ksoftirqd/023:42:570
77869986,0cyclictest6350-21apt-get23:55:123
77869985,3cyclictest0-21swapper/323:45:133
77869984,3cyclictest9376-21apt20:24:593
77869984,3cyclictest508-21dbus-daemon21:02:183
77869984,3cyclictest508-21dbus-daemon19:25:003
77869984,3cyclictest14139-21head23:15:193
77869983,4cyclictest21851-21apt00:35:143
77869983,4cyclictest0-21swapper/321:05:143
77869982,5cyclictest0-21swapper/320:45:203
77869981,3cyclictest29090-21ssh21:55:163
77869981,1cyclictest34-21ksoftirqd/322:45:213
77869980,6cyclictest9518-21ssh22:04:053
77869980,5cyclictest4510-21ssh21:37:273
77869980,5cyclictest32558-21ssh21:34:543
77869980,5cyclictest2390-21ssh22:44:323
77869980,4cyclictest9302-21sed19:10:213
77869980,4cyclictest25099-21ssh23:00:173
77869980,4cyclictest0-21swapper/321:54:253
77869980,3cyclictest17755-21apt-get19:30:143
77829985,2cyclictest25894-21memory21:00:162
77829984,3cyclictest508-21dbus-daemon22:45:002
77829984,3cyclictest508-21dbus-daemon19:35:572
77829984,2cyclictest19982-21systemd-journal00:25:002
77829981,4cyclictest18399-21dump-pmu-power21:25:002
77789986,1cyclictest28128-21ssh21:30:491
77789983,3cyclictest19982-21systemd-journal20:47:421
77789983,3cyclictest19982-21systemd-journal20:23:051
77789983,3cyclictest19982-21systemd-journal19:52:371
77789982,4cyclictest13790-21grep20:35:001
77789982,3cyclictest26332-21munin-run19:50:011
77789981,4cyclictest14036-21munin-run22:30:011
77769985,2cyclictest508-21dbus-daemon21:35:010
77769984,3cyclictest19982-21systemd-journal20:58:580
77769984,3cyclictest11736-21in:imuxsock23:27:040
77769980,5cyclictest28245-21grep22:40:010
25073280,0sleep10-21swapper/121:00:111
16900280,0sleep10-21swapper/123:16:551
77869977,0cyclictest18837-21apt-get21:25:123
77869975,1cyclictest22864-21apt-get20:55:113
77869974,2cyclictest9640-21cron23:35:013
77869974,2cyclictest29741-11brltty20:00:083
77869974,2cyclictest26331-21awk19:50:013
77869974,2cyclictest18214-21munin-run20:45:003
77869974,2cyclictest16873-21timerandwakeup19:25:193
77869974,2cyclictest16350-21ssh22:30:303
77869973,3cyclictest7153-21rm22:25:023
77869973,3cyclictest7151-21latency_hist20:20:003
77869973,3cyclictest6357-21apt23:10:153
77869973,3cyclictest5430-21hddtemp_smartct00:20:163
77869973,3cyclictest4870-21rm21:15:163
77869973,3cyclictest466-21ps23:50:193
77869973,3cyclictest28841-21sh22:16:483
77869973,3cyclictest28768-21/usr/sbin/munin19:55:113
77869973,3cyclictest24333-21latency_hist19:45:013
77869973,3cyclictest21108-21fschecks_count20:50:173
77869973,3cyclictest18197-21tail22:55:143
77869973,3cyclictest14648-21hddtemp_smartct20:35:183
77869973,3cyclictest1-21systemd20:30:363
77869973,3cyclictest10431-21cp19:15:013
77869973,3cyclictest10092-21fschecks_time20:25:163
77869973,3cyclictest0-21swapper/320:10:153
77869973,2cyclictest12793-40gzip00:00:003
77869972,4cyclictest20138-21cpuspeed_turbos19:35:143
77869972,4cyclictest10767-21/usr/sbin/munin00:25:223
77869971,4cyclictest0-21swapper/321:46:433
77869971,1cyclictest34-21ksoftirqd/321:20:023
77869971,1cyclictest27343-21wc19:50:193
77869970,3cyclictest19724-21taskset00:05:123
77829976,1cyclictest28-21ksoftirqd/223:20:012
77829974,2cyclictest508-21dbus-daemon23:44:112
77829974,2cyclictest19982-21systemd-journal23:10:232
77829974,2cyclictest11736-21in:imuxsock19:40:322
77829974,2cyclictest11736-21in:imuxsock00:06:172
77829973,3cyclictest19982-21systemd-journal00:27:092
77829973,3cyclictest0-21swapper/222:39:442
77829973,1cyclictest0-21swapper/223:39:272
77829973,1cyclictest0-21swapper/222:00:512
77829972,3cyclictest19982-21systemd-journal19:31:362
77829970,6cyclictest7215-21systemctl21:40:022
77789974,2cyclictest668-21runrttasks23:49:131
77789974,2cyclictest30727-21cron19:59:591
77789973,3cyclictest508-21dbus-daemon20:20:001
77789973,3cyclictest30654-21snmpd20:12:241
77789973,3cyclictest23825-21sshd19:41:221
77789973,3cyclictest17068-21cron23:40:001
77789973,3cyclictest0-21swapper/123:52:411
77789973,3cyclictest0-21swapper/122:20:261
77789973,1cyclictest0-21swapper/122:12:401
77789972,3cyclictest19982-21systemd-journal22:04:531
77789972,3cyclictest19982-21systemd-journal21:13:091
77789971,3cyclictest9940-21turbostat19:15:011
77789970,4cyclictest28236-21ssh22:40:011
77789970,3cyclictest13040-21turbostat23:15:011
77769974,2cyclictest508-21dbus-daemon22:34:310
77769974,2cyclictest19982-21systemd-journal21:29:220
77769973,3cyclictest7950-21awk23:55:210
77769973,3cyclictest508-21dbus-daemon23:40:010
77769973,3cyclictest29241-21diskmemload21:42:280
77769973,3cyclictest19982-21systemd-journal23:50:000
77769973,3cyclictest19982-21systemd-journal19:36:210
77769973,3cyclictest19982-21systemd-journal00:17:580
77769973,3cyclictest0-21swapper/000:12:330
77769973,1cyclictest0-21swapper/023:11:180
77769973,1cyclictest0-21swapper/022:29:550
77769973,1cyclictest0-21swapper/021:48:070
77769973,1cyclictest0-21swapper/000:27:480
77769972,4cyclictest30654-21snmpd22:01:240
77769972,3cyclictest19982-21systemd-journal23:33:200
77769972,3cyclictest19982-21systemd-journal23:05:230
77769972,3cyclictest19982-21systemd-journal21:36:050
77769972,3cyclictest19982-21systemd-journal20:50:110
77769972,2cyclictest19982-21systemd-journal21:58:120
77769970,5cyclictest19982-21systemd-journal22:43:160
77769970,4cyclictest26343-21basename19:50:010
77769970,1cyclictest16183-21basename19:25:150
5348270,0sleep00-21swapper/023:54:300
4919270,0sleep10-21swapper/123:09:101
24590270,0sleep10-21swapper/123:22:031
16579270,1sleep20-21swapper/221:45:302
77829966,0cyclictest0-21swapper/223:45:332
77829966,0cyclictest0-21swapper/220:25:232
77829965,1cyclictest28-21ksoftirqd/222:20:012
77829963,2cyclictest913-21awk23:50:202
77829963,2cyclictest8889-21wc22:25:222
77829963,2cyclictest8093-21df_inode21:40:152
77829963,2cyclictest7143-21sh20:19:592
77829963,2cyclictest28-21ksoftirqd/219:15:012
77829963,2cyclictest10858-21cat22:50:142
77829962,3cyclictest5164-21ntpd22:20:182
77829962,3cyclictest508-21dbus-daemon20:37:412
77829962,3cyclictest31871-21expr23:05:162
77829962,3cyclictest30654-21snmpd23:55:062
77829962,3cyclictest30654-21snmpd23:26:222
77829962,3cyclictest30654-21snmpd20:51:072
77829962,3cyclictest30654-21snmpd20:48:372
77829962,3cyclictest30654-21snmpd20:40:252
77829962,3cyclictest30654-21snmpd20:33:282
77829962,3cyclictest30654-21snmpd20:00:482
77829962,3cyclictest30654-21snmpd19:52:032
77829962,3cyclictest30654-21snmpd19:45:122
77829962,3cyclictest30654-21snmpd19:25:172
77829962,3cyclictest30654-21snmpd19:20:472
77829962,3cyclictest30654-21snmpd00:01:192
77829962,3cyclictest29459-21cat19:55:152
77829962,3cyclictest27974-21cat21:05:172
77829962,3cyclictest26306-21ls21:30:142
77829962,3cyclictest22341-21cat23:20:222
77829962,3cyclictest22184-21cat00:35:152
77829962,3cyclictest16811-21cat00:30:182
77829962,3cyclictest1-21systemd22:11:092
77829962,1cyclictest0-21swapper/223:32:542
77829962,1cyclictest0-21swapper/223:01:332
77829962,1cyclictest0-21swapper/222:30:262
77829962,1cyclictest0-21swapper/222:05:272
77829962,1cyclictest0-21swapper/221:55:182
77829962,1cyclictest0-21swapper/221:51:132
77829962,1cyclictest0-21swapper/221:10:072
77829962,1cyclictest0-21swapper/200:19:162
77829962,1cyclictest0-21swapper/200:13:232
77829961,3cyclictest19982-21systemd-journal19:10:542
77829960,6cyclictest0-21swapper/222:45:242
77829960,3cyclictest18894-21awk21:25:112
77829960,2cyclictest19982-21systemd-journal21:15:212
77789966,0cyclictest0-21swapper/100:05:011
77789965,1cyclictest29241-21diskmemload00:15:151
77789964,1cyclictest17056-21needreboot00:30:191
77789963,3cyclictest28166-21kworker/1:221:05:151
77789963,2cyclictest3299-21seq21:36:141
77789963,2cyclictest28843-21sh21:55:041
77789963,2cyclictest27984-21rm23:01:551
77789963,2cyclictest27853-21sh00:10:451
77789963,2cyclictest19116-21gdbus19:30:011
77789963,2cyclictest10608-21gdbus19:15:021
77789962,3cyclictest7984-21expr21:40:151
77789962,3cyclictest6981-21expr23:55:151
77789962,3cyclictest3515-21wc22:45:141
77789962,3cyclictest30654-21snmpd23:26:041
77789962,3cyclictest30654-21snmpd20:25:491
77789962,3cyclictest30654-21snmpd19:37:141
77789962,3cyclictest30654-21snmpd00:00:041
77789962,3cyclictest22933-21cat20:55:101
77789962,3cyclictest19982-21systemd-journal21:20:011
77789962,3cyclictest1713-21ntp_kernel_err20:05:181
77789962,3cyclictest13711-21ls19:20:171
77789962,3cyclictest1-21systemd20:40:001
77789962,3cyclictest11500-21cat00:25:181
77789962,3cyclictest11014-21fschecks_count22:50:151
77789962,3cyclictest0-21swapper/123:30:111
77789962,1cyclictest0-21swapper/123:40:041
77789962,1cyclictest0-21swapper/122:15:031
77789962,1cyclictest0-21swapper/122:05:101
77789961,4cyclictest4060-21apt-get00:20:011
77789961,1cyclictest0-21swapper/122:30:161
77789960,2cyclictest23239-21date21:50:191
77789960,0cyclictest0-21swapper/121:27:051
77769963,2cyclictest8261-21cat20:20:180
77769963,2cyclictest6486-21awk20:15:180
77769963,2cyclictest4566-21processes22:45:200
77769963,2cyclictest30654-21snmpd00:36:140
77769963,2cyclictest21687-21hddtemp_smartct23:20:160
77769963,2cyclictest19982-21systemd-journal20:04:440
77769963,2cyclictest18706-21ssh23:18:350
77769963,2cyclictest18266-21hddtemp_smartct22:55:160
77769963,2cyclictest18195-21sh20:44:590
77769963,2cyclictest14290-21ssh21:21:100
77769963,2cyclictest1379-21head20:05:150
77769963,2cyclictest12442-21hddtemp_smartct20:30:170
77769963,2cyclictest1-21systemd19:53:030
77769963,2cyclictest12093-21ntpq19:15:210
77769963,1cyclictest0-21swapper/000:32:550
77769962,3cyclictest9494-21ssh00:23:580
77769962,3cyclictest508-21dbus-daemon19:56:230
77769962,3cyclictest30654-21snmpd22:50:390
77769962,3cyclictest30654-21snmpd22:15:260
77769962,3cyclictest30654-21snmpd20:25:160
77769962,3cyclictest30654-21snmpd20:10:120
77769962,3cyclictest30654-21snmpd19:10:140
77769962,3cyclictest30654-21snmpd00:05:050
77769962,3cyclictest23590-21cat19:40:200
77769962,3cyclictest14380-21expr20:35:160
77769962,3cyclictest12567-21mandb00:00:000
77769962,3cyclictest0-21swapper/022:05:140
77769961,3cyclictest19982-21systemd-journal21:16:290
77769960,0cyclictest0-21swapper/021:10:520
77829952,2cyclictest849-21cpuspeed20:05:132
77829952,2cyclictest8393-21sh20:20:192
77829952,2cyclictest3671-21cat20:10:172
77789952,2cyclictest31023-21apt-get20:00:131
77789952,2cyclictest20752-21/usr/sbin/munin20:50:151
77769953,1cyclictest502-21systemd-logind19:20:070
77769952,2cyclictest25606-21grep21:00:140
77769952,2cyclictest18937-21grep20:45:160
77769951,3cyclictest30654-21snmpd19:30:050
77769950,3cyclictest19982-21systemd-journal21:07:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional