You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 09:41
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackdslot0.osadl.org (updated Sun Nov 23, 2025 00:45:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1164499234,13cyclictest14781-21sed00:00:010
1164499234,11cyclictest25757-21http19:35:010
1164499230,5cyclictest0-21swapper/021:21:510
1164999222,17cyclictest5537-21nvmesmart_nvme023:40:182
1164599213,11cyclictest14377-21python319:15:021
11649991915,3cyclictest29641-21cpuspeed_turbos19:40:162
1164999190,16cyclictest11789-21fschecks_time20:05:162
1164599190,1cyclictest0-21swapper/120:30:011
1164599190,18cyclictest28876-21grep20:35:201
11644991918,1cyclictest9-21ksoftirqd/000:10:140
11644991917,1cyclictest0-21swapper/020:38:500
11644991917,1cyclictest0-21swapper/000:02:500
1164499190,18cyclictest3257-21sendmail-mta20:47:580
1164999182,15cyclictest5889-21cstates19:55:162
11649991817,1cyclictest28-21ksoftirqd/222:06:012
11649991817,1cyclictest28-21ksoftirqd/222:06:002
11649991814,3cyclictest19953-21sed22:15:012
1164999181,16cyclictest28288-21smtpd22:27:582
11644991816,1cyclictest0-21swapper/023:41:590
11644991816,1cyclictest0-21swapper/019:24:340
11644991814,3cyclictest3308-21cron23:39:010
1164499181,16cyclictest24980-21cat23:20:110
1164499181,16cyclictest10522-21chrt20:03:050
1164999172,1cyclictest4854-21if_eno120:50:172
1164999172,1cyclictest271rcuc/220:15:162
11649991716,1cyclictest28-21ksoftirqd/200:37:342
11649991715,1cyclictest28-21ksoftirqd/223:37:432
11649991715,1cyclictest28-21ksoftirqd/222:40:172
11649991715,1cyclictest271rcuc/221:50:142
11649991715,1cyclictest271rcuc/221:35:232
11649991715,1cyclictest0-21swapper/222:20:142
11649991715,1cyclictest0-21swapper/220:04:082
11649991714,2cyclictest9325-21cut22:50:192
11649991714,2cyclictest7663-21latency_hist22:50:002
11649991714,2cyclictest6987-21cpuspeed_turbos20:55:152
11649991714,2cyclictest4785-21sed21:45:232
11649991714,2cyclictest2707-21lxd20:38:572
11649991714,2cyclictest25299-21ntpq00:15:182
11649991714,2cyclictest18555-21uniq19:20:162
11649991714,2cyclictest17853-21cron21:15:012
11649991714,2cyclictest16498-21sed00:00:202
11649991714,2cyclictest14989-21munin-run00:00:002
11649991714,2cyclictest11935-21http21:05:002
11649991713,3cyclictest1299-21lxd23:18:572
11649991713,3cyclictest11362-21dpkg22:00:012
1164999171,15cyclictest30772-21cat22:30:222
1164999171,15cyclictest28373-21switchtime00:20:242
1164999171,15cyclictest271rcuc/221:30:142
1164999171,15cyclictest271rcuc/221:20:152
1164999171,15cyclictest271rcuc/220:40:152
1164999171,15cyclictest271rcuc/220:20:162
1164999171,15cyclictest271rcuc/220:10:162
1164999171,15cyclictest24987-21awk20:30:142
1164999171,15cyclictest10754-21strings23:50:172
1164999171,15cyclictest10754-21strings23:50:172
1164999170,2cyclictest9211-21latency_hist23:50:002
1164999170,2cyclictest4581-21H212:26:332
1164999170,2cyclictest31555-21taskset00:27:472
1164999170,2cyclictest20541-21sed22:15:022
1164999170,2cyclictest13963-21nfsd421:05:182
1164999170,16cyclictest414-20systemd-journal21:18:082
1164999170,16cyclictest26208-21nfsd423:20:182
1164999170,16cyclictest24867-21processes19:30:212
1164999170,16cyclictest15026-21pzem_power23:00:202
1164999170,16cyclictest12179-21grep19:10:162
1164999170,16cyclictest0-21swapper/220:25:212
1164599179,4cyclictest0-21swapper/100:01:541
1164599172,1cyclictest21-21ksoftirqd/119:31:471
1164599172,1cyclictest0-21swapper/121:50:271
11645991715,1cyclictest817-21missed_timers19:45:181
11645991715,1cyclictest0-21swapper/121:14:541
11645991715,1cyclictest0-21swapper/119:51:101
11645991714,2cyclictest9782-21tr23:50:131
11645991714,2cyclictest9782-21tr23:50:131
11645991714,2cyclictest5457-21ntpq20:50:191
11645991714,2cyclictest3764-21grep22:40:181
11645991714,2cyclictest31791-21nfsd420:40:191
11645991714,2cyclictest30736-21nfsd400:25:181
11645991714,2cyclictest20189-21https22:15:011
11645991714,2cyclictest18250-21/usr/sbin/munin22:10:161
11645991714,2cyclictest13019-21perl23:55:181
11645991714,2cyclictest11743-21taskset21:03:461
11645991713,3cyclictest22335-21nfsd421:20:191
1164599171,15cyclictest283952chrt22:29:151
1164599171,15cyclictest12919-21expr21:05:151
1164599170,2cyclictest2440-21sendmail_mailqu23:35:201
1164599170,2cyclictest12351-21grep20:05:191
1164599170,1cyclictest4534-21H212:26:331
1164599170,1cyclictest0-21swapper/122:05:131
1164599170,1cyclictest0-21swapper/122:05:121
1164599170,1cyclictest0-21swapper/100:05:051
1164599170,16cyclictest7406-21ls19:55:241
1164599170,16cyclictest4304-21grep00:35:201
1164599170,16cyclictest30007-21snapd23:16:491
1164599170,16cyclictest21572-21sshd20:22:271
1164599170,16cyclictest12835-21chrt22:57:261
1164599170,16cyclictest10693-21turbostat20:05:011
1164599170,16cyclictest0-21swapper/123:45:211
1164599170,16cyclictest0-21swapper/119:15:221
11644991715,1cyclictest0-21swapper/021:35:010
11644991715,1cyclictest0-21swapper/020:33:140
11644991714,2cyclictest9036-21chrt20:59:280
11644991714,2cyclictest6964-21awk21:50:180
11644991714,2cyclictest30564-21/usr/sbin/munin20:40:200
11644991714,2cyclictest2254-21latency_hist21:45:010
11644991714,2cyclictest14223-21sed23:00:140
1164499171,15cyclictest0-21swapper/023:50:000
1164499170,2cyclictest7034-21smartctl19:55:230
1164499170,2cyclictest31031-21plymouth00:25:210
1164499170,2cyclictest30264-21latency21:35:190
1164499170,2cyclictest30105-21nfsd422:30:170
1164499170,2cyclictest15089-21ls22:05:130
1164499170,2cyclictest15089-21ls22:05:130
1164499170,2cyclictest11077-21/usr/sbin/munin20:05:180
1164499170,16cyclictest881-21avahi-daemon19:45:220
1164499170,16cyclictest3722-21awk19:50:180
1164499170,16cyclictest3692-21master00:05:120
1164499170,16cyclictest3441-21gsd-color23:05:550
1164499170,16cyclictest29573-21tr19:40:150
1164499170,16cyclictest28284-21sshd22:27:520
1164499170,16cyclictest2350-21munin-plugin-st00:35:000
1164499170,16cyclictest19519-21smtp22:11:140
1164499170,16cyclictest17825-21sed20:15:200
1164499170,16cyclictest15504-21cat21:10:010
1164499170,16cyclictest14273-21apt19:15:010
1164499170,16cyclictest12267-21apt-config21:05:010
1164499170,16cyclictest12062-21cut22:00:140
1164499170,16cyclictest0-21swapper/023:12:330
1164499170,16cyclictest0-21swapper/000:35:230
11649991615,1cyclictest28-21ksoftirqd/200:30:122
11649991615,1cyclictest28-21ksoftirqd/200:10:132
11649991615,1cyclictest271rcuc/219:25:162
11649991615,0cyclictest271rcuc/220:45:152
11649991614,1cyclictest29235-21cut23:25:212
11649991614,1cyclictest271rcuc/223:05:012
11649991614,1cyclictest271rcuc/222:35:192
11649991614,1cyclictest14822-21apt-key19:15:012
11649991614,1cyclictest0-21swapper/221:25:202
11649991614,1cyclictest0-21swapper/219:50:142
11649991613,2cyclictest10847-21cut22:55:142
1164999160,2cyclictest1086-21snmpd21:40:202
1164999160,15cyclictest19768-21cstates23:10:142
1164999160,15cyclictest11862-21dpkg22:00:022
1164999160,15cyclictest0-21swapper/223:30:132
1164999160,15cyclictest0-21swapper/219:45:152
1164999160,15cyclictest0-21swapper/200:05:122
11645991614,1cyclictest874-21/usr/sbin/munin20:45:121
11645991614,1cyclictest24907-21/usr/sbin/munin20:30:131
11645991614,1cyclictest21198-21/usr/sbin/munin00:10:141
11645991614,1cyclictest13972-21/usr/sbin/munin20:10:131
11645991614,1cyclictest13465-21taskset23:00:071
11645991614,1cyclictest0-21swapper/123:40:141
11645991614,1cyclictest0-21swapper/123:25:161
11645991614,1cyclictest0-21swapper/123:20:111
11645991614,1cyclictest0-21swapper/123:08:321
11645991614,1cyclictest0-21swapper/122:50:151
11645991614,1cyclictest0-21swapper/122:00:201
11645991614,1cyclictest0-21swapper/121:45:141
11645991614,1cyclictest0-21swapper/121:35:171
11645991614,1cyclictest0-21swapper/119:35:121
11645991614,1cyclictest0-21swapper/119:20:141
11645991613,2cyclictest8589-21dpkg21:55:021
11645991613,2cyclictest28271-21timerandwakeup21:30:201
11645991613,2cyclictest24653-21http_loadtime00:15:151
1164599161,1cyclictest1413-21mailstats00:30:211
1164599160,2cyclictest24443-21fschecks_time21:25:171
1164599160,2cyclictest0-21swapper/120:55:191
1164599160,15cyclictest32131-21/usr/sbin/munin21:40:221
1164599160,15cyclictest29799-21kernelversion22:30:171
1164599160,15cyclictest26841-21grep00:20:161
1164599160,15cyclictest23414-21/usr/sbin/munin22:20:211
1164599160,15cyclictest0-21swapper/123:10:171
1164599160,15cyclictest0-21swapper/122:45:151
1164599160,15cyclictest0-21swapper/122:35:111
1164599160,15cyclictest0-21swapper/119:40:161
1164599160,15cyclictest0-21swapper/119:25:141
11644991616,0cyclictest0-21swapper/023:15:220
11644991614,1cyclictest5457-21cpuspeed_turbos22:45:140
11644991614,1cyclictest24643-21/usr/sbin/munin00:15:150
11644991614,1cyclictest242222chrt20:26:290
11644991614,1cyclictest0-21swapper/023:50:170
11644991614,1cyclictest0-21swapper/023:50:160
11644991614,1cyclictest0-21swapper/023:25:200
11644991614,1cyclictest0-21swapper/022:50:370
11644991614,1cyclictest0-21swapper/021:55:220
11644991613,2cyclictest32422-21cat22:35:170
11644991613,2cyclictest27772-21nfsd419:35:270
11644991613,2cyclictest23846-21expr22:20:150
11644991613,2cyclictest15858-21cat21:10:140
11644991613,2cyclictest11179-21expr22:55:140
1164499160,2cyclictest2585-21apache_processe21:45:140
1164499160,2cyclictest20614-21find19:25:120
1164499160,1cyclictest0-21swapper/022:15:020
1164499160,1cyclictest0-21swapper/021:15:170
1164499160,16cyclictest26099-21perf21:30:010
1164499160,15cyclictest3992-21apt-get20:50:140
1164499160,15cyclictest2955-21fschecks_time22:40:150
1164499160,15cyclictest26902-21grep00:20:150
1164499160,15cyclictest14907-21apt-config19:15:020
1164499160,15cyclictest14000-21apt-get20:10:140
1164499160,15cyclictest0-21swapper/020:20:170
1164599150,14cyclictest0-21swapper/123:30:121
1164499150,15cyclictest0-21swapper/023:30:200
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional