You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-09 - 06:02
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackdslot0.osadl.org (updated Fri May 09, 2025 00:46:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
16671993322,10cyclictest11262-21cat20:55:002
1666999274,18cyclictest0-21swapper/019:35:000
1666999225,14cyclictest0-21swapper/000:27:520
1667099210,20cyclictest823-21systemd-network20:40:291
1667199196,12cyclictest0-21swapper/200:12:512
1667199193,15cyclictest0-21swapper/221:19:002
16671991915,3cyclictest1603-21lxd21:00:552
16670991917,1cyclictest0-21swapper/120:04:341
1667099190,18cyclictest0-21swapper/122:58:541
1667099190,18cyclictest0-21swapper/121:10:161
1667099190,18cyclictest0-21swapper/120:27:011
16669991917,1cyclictest0-21swapper/022:00:170
16669991914,4cyclictest0-21swapper/019:48:000
1666999190,3cyclictest0-21swapper/021:30:190
1666999190,17cyclictest27377-21nfsd400:15:210
16671991816,1cyclictest0-21swapper/220:40:012
16671991816,1cyclictest0-21swapper/220:40:012
16671991813,4cyclictest0-21swapper/222:20:002
1667199181,16cyclictest8012-21nfsd423:40:182
1667199181,16cyclictest21109-21expr19:15:202
1667199180,17cyclictest11355-21cpuspeed_turbos22:50:142
1667199180,17cyclictest11355-21cpuspeed_turbos22:50:142
1667099182,15cyclictest7081-21tr19:50:141
1667099182,15cyclictest22409-21mailstats22:10:211
1667099182,15cyclictest0-21swapper/120:18:281
16670991816,1cyclictest0-21swapper/120:45:241
16670991813,4cyclictest0-21swapper/122:39:251
1667099181,2cyclictest27681-21smartctl22:20:181
16669991816,1cyclictest9-21ksoftirqd/022:45:180
16669991816,1cyclictest0-21swapper/022:26:180
16669991815,2cyclictest5109-21sessionclean22:38:590
16669991814,3cyclictest0-21swapper/023:15:130
16669991814,3cyclictest0-21swapper/021:10:580
1666999181,16cyclictest1555-21lxd22:05:190
1666999180,3cyclictest31977-21turbostat20:35:000
1666999180,16cyclictest17948-21cstates00:00:160
1667199172,1cyclictest28-21ksoftirqd/219:33:102
16671991715,1cyclictest24962-21/usr/sbin/munin23:15:202
16671991715,1cyclictest242922chrt23:11:252
16671991715,1cyclictest0-21swapper/222:21:472
16671991715,1cyclictest0-21swapper/221:43:492
16671991714,2cyclictest3080-21/usr/sbin/munin20:40:152
16671991714,2cyclictest27116-21expr20:25:162
16671991714,2cyclictest25496-21expr19:25:162
16671991714,2cyclictest25058-21nfsd420:20:162
16671991714,2cyclictest20874-21processes23:05:212
16671991714,2cyclictest20826-21apache_volume22:10:132
16671991714,2cyclictest19959-21systemctl20:10:222
16671991714,2cyclictest19959-21systemctl20:10:222
16671991714,2cyclictest17963-21grep22:05:152
16671991714,2cyclictest17774-21cat00:00:162
16671991714,2cyclictest11015-21ntp_kernel_pll_19:55:182
1667199171,1cyclictest310812chrt22:27:282
1667199170,2cyclictest3278-21chrt21:36:432
1667199170,2cyclictest30940-21cstates21:30:142
1667199170,2cyclictest29639-21if_eno100:20:172
1667199170,2cyclictest26802-21sed21:20:222
1667199170,2cyclictest23839-21nvmesmart_nvme019:20:162
1667199170,2cyclictest14971-21wc22:00:002
1667199170,2cyclictest12527-21missed_timers20:55:202
1667199170,2cyclictest1132-21snapd00:31:122
1667199170,1cyclictest0-21swapper/222:30:442
1667199170,16cyclictest8877-21vmstat19:50:222
1667199170,16cyclictest6221-21sessionclean23:39:012
1667199170,16cyclictest3904-21munin-run19:45:012
1667199170,16cyclictest20266-21expr21:10:152
1667199170,16cyclictest1798-21systemd21:50:072
1667199170,16cyclictest16221-21smartctl20:05:172
1667199170,16cyclictest0-21swapper/219:37:302
1667099179,4cyclictest0-21swapper/121:21:471
16670991715,1cyclictest0-21swapper/123:08:071
16670991714,2cyclictest881-21avahi-daemon23:45:211
16670991714,2cyclictest7058-21apache_volume21:45:131
16670991714,2cyclictest6502-21hddtemp_smartct22:40:161
16670991714,2cyclictest4904-21fschecks_time19:45:161
16670991714,2cyclictest29864-21cat00:20:171
16670991714,2cyclictest19872-21expr19:15:151
16670991714,2cyclictest19542-21latency_hist21:10:021
16670991714,2cyclictest17698-21smartctl19:10:151
16670991714,2cyclictest1708-21cat19:40:131
16670991714,2cyclictest15334-21cat21:00:191
16670991714,2cyclictest1354-21lxd20:52:551
1667099171,15cyclictest206482chrt20:14:401
1667099171,15cyclictest206482chrt20:14:401
1667099171,15cyclictest0-21swapper/119:55:191
1667099170,2cyclictest31706-21apt-key22:30:011
1667099170,17cyclictest0-21swapper/121:57:431
1667099170,16cyclictest9882-21ntp_kernel_pll_22:45:191
1667099170,16cyclictest881-21avahi-daemon23:50:201
1667099170,16cyclictest31825-21ntpq23:25:221
1667099170,16cyclictest28341-21expr19:30:151
1667099170,16cyclictest24895-21processes00:10:211
1667099170,16cyclictest23483-21nfsd423:10:181
1667099170,16cyclictest16838-21mailstats22:00:231
1667099170,16cyclictest1555-21lxd00:28:551
1667099170,16cyclictest1532-21fschecks_time23:30:171
1667099170,16cyclictest1405-21lxd20:06:541
1667099170,16cyclictest13004-21pzem_power20:55:231
1667099170,16cyclictest12019-21if_eno122:50:171
1667099170,16cyclictest12019-21if_eno122:50:171
16669991715,1cyclictest0-21swapper/023:35:220
16669991715,1cyclictest0-21swapper/020:48:140
16669991715,1cyclictest0-21swapper/020:10:160
16669991715,1cyclictest0-21swapper/020:10:160
16669991714,2cyclictest9383-21cat19:55:010
16669991714,2cyclictest8151-21latency_hist22:45:010
16669991714,2cyclictest414-20systemd-journal21:54:550
16669991714,2cyclictest3638-21chrt19:44:030
16669991714,2cyclictest22841-21cstates23:10:140
16669991714,2cyclictest2049-21systemd-journal21:39:000
16669991714,2cyclictest17791-21sed23:00:170
16669991714,2cyclictest16467-21turbostat.cron19:14:550
16669991714,2cyclictest15403-21grep22:55:200
16669991714,2cyclictest13688-21cat20:00:160
16669991713,3cyclictest0-21swapper/019:55:220
1666999170,2cyclictest881-21avahi-daemon00:35:270
1666999170,2cyclictest3076-21rs:main0
1666999170,2cyclictest23624-21cpuspeed_turbos22:15:150
1666999170,2cyclictest11803-21cron23:50:000
1666999170,1cyclictest0-21swapper/000:05:510
1666999170,16cyclictest967-21thermald21:05:220
1666999170,16cyclictest32102-21apache_volume22:30:110
1666999170,16cyclictest30378-21latency_hist23:25:020
1666999170,16cyclictest25599-21expr19:25:150
1666999170,16cyclictest23179-21cron20:17:010
1666999170,16cyclictest19841-21expr23:05:150
1666999170,16cyclictest19793-21ls19:15:150
1666999170,16cyclictest14813-21tr22:00:000
1666999170,16cyclictest1356-21lxd21:02:540
1666999170,16cyclictest13538-21grep23:50:170
1666999170,16cyclictest0-21swapper/023:40:180
1666999170,15cyclictest0-21swapper/020:35:120
1666999170,15cyclictest0-21swapper/020:35:120
16671991614,1cyclictest29226-21awk21:25:242
16671991614,1cyclictest20389-21/usr/sbin/munin00:05:152
16671991614,1cyclictest10194-21fschecks_time23:45:172
16671991614,1cyclictest0-21swapper/223:30:192
16671991614,1cyclictest0-21swapper/223:00:142
16671991614,1cyclictest0-21swapper/222:40:182
16671991614,1cyclictest0-21swapper/221:05:162
16671991614,1cyclictest0-21swapper/220:30:192
16671991614,1cyclictest0-21swapper/220:15:152
16671991614,1cyclictest0-21swapper/220:00:142
16671991614,1cyclictest0-21swapper/219:10:182
16671991614,1cyclictest0-21swapper/200:35:282
16671991613,2cyclictest26206-21cpuspeed_turbos00:15:162
1667199160,1cyclictest0-21swapper/223:55:122
1667199160,1cyclictest0-21swapper/222:45:162
1667199160,1cyclictest0-21swapper/200:25:142
1667199160,15cyclictest13338-21sed23:50:162
1667199160,15cyclictest0-21swapper/223:25:172
1667199160,15cyclictest0-21swapper/223:23:232
1667199160,15cyclictest0-21swapper/222:55:162
1667199160,15cyclictest0-21swapper/222:35:162
1667199160,15cyclictest0-21swapper/222:00:142
1667199160,15cyclictest0-21swapper/221:45:122
16670991614,1cyclictest27742-21apt-get23:20:171
16670991614,1cyclictest20620-21cpuspeed_turbos00:05:141
16670991614,1cyclictest0-21swapper/123:35:181
16670991614,1cyclictest0-21swapper/123:15:131
16670991614,1cyclictest0-21swapper/123:00:131
16670991614,1cyclictest0-21swapper/121:40:011
16670991614,1cyclictest0-21swapper/121:30:231
16670991614,1cyclictest0-21swapper/121:25:241
16670991614,1cyclictest0-21swapper/119:25:131
16670991614,1cyclictest0-21swapper/100:35:021
1667099161,1cyclictest24317-21if_eno122:15:181
1667099160,2cyclictest29726-21ls22:25:171
1667099160,2cyclictest26230-21tr00:15:161
1667099160,2cyclictest25751-21sed20:20:191
1667099160,2cyclictest2510-21seq20:39:211
1667099160,2cyclictest2510-21seq20:39:211
1667099160,1cyclictest0-21swapper/123:55:161
1667099160,1cyclictest0-21swapper/121:20:011
1667099160,1cyclictest0-21swapper/120:30:151
1667099160,1cyclictest0-21swapper/119:20:151
1667099160,15cyclictest3849-21grep00:30:211
1667099160,15cyclictest19292-21systemctl00:00:231
1667099160,15cyclictest18427-21expr22:05:151
1667099160,15cyclictest0-21swapper/123:40:141
1667099160,15cyclictest0-21swapper/121:50:201
1667099160,15cyclictest0-21swapper/121:40:191
1667099160,15cyclictest0-21swapper/119:35:161
16669991614,1cyclictest20923-21cpuspeed_turbos22:10:130
16669991614,1cyclictest14954-21apt-get23:55:140
16669991614,1cyclictest0-21swapper/023:25:240
16669991614,1cyclictest0-21swapper/022:20:170
16669991614,1cyclictest0-21swapper/020:43:420
16669991614,1cyclictest0-21swapper/020:05:150
16669991614,1cyclictest0-21swapper/000:30:140
16669991614,1cyclictest0-21swapper/000:10:180
1666999160,2cyclictest23851-21apache_processe20:20:100
1666999160,2cyclictest22967-21expr19:20:110
1666999160,15cyclictest9335-21df20:50:120
1666999160,15cyclictest30760-21expr19:35:160
1666999160,15cyclictest22981-21expr21:15:140
1666999160,15cyclictest11529-21cpuspeed_turbos20:55:160
1666999160,15cyclictest11232-21dpkg22:50:130
1666999160,15cyclictest11232-21dpkg22:50:130
1666999160,15cyclictest1034-21cstates23:30:160
1666999160,15cyclictest0-21swapper/021:45:170
1666999160,15cyclictest0-21swapper/021:40:180
1666999160,15cyclictest0-21swapper/021:20:150
1666999160,15cyclictest0-21swapper/020:25:150
1666999160,15cyclictest0-21swapper/000:20:160
16671991513,1cyclictest0-21swapper/220:45:172
1667199150,14cyclictest0-21swapper/219:45:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional