You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 10:20
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackdslot0.osadl.org (updated Fri Nov 28, 2025 00:45:59)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
378993120,10cyclictest28-21ksoftirqd/219:47:042
376992421,2cyclictest19708-21awk21:35:201
37699230,22cyclictest0-21swapper/100:37:131
375992316,6cyclictest12276-21cat19:30:010
37599230,22cyclictest0-21swapper/023:01:090
37599220,21cyclictest823-21systemd-network19:18:390
376992110,10cyclictest3441-21gsd-color19:50:181
378992018,1cyclictest0-21swapper/222:20:012
376992017,2cyclictest454-21plymouthd21:19:151
376992014,5cyclictest6682-21latency_hist19:20:001
376992014,3cyclictest0-21swapper/120:40:151
37599203,15cyclictest14436-21apt-key20:30:020
375992016,3cyclictest13788-21nfsd419:30:190
37599201,18cyclictest4705-21grep00:00:210
37599200,19cyclictest20054-21needreboot23:30:170
37899193,1cyclictest7764-21expr21:15:132
378991917,1cyclictest0-21swapper/219:40:172
378991914,3cyclictest0-21swapper/221:47:582
378991914,2cyclictest4530-21H212:26:332
37899190,18cyclictest0-21swapper/219:24:292
37699194,1cyclictest0-21swapper/122:23:041
376991917,1cyclictest0-21swapper/123:45:001
376991916,2cyclictest4581-21H212:26:331
376991914,3cyclictest3579-21munin-plugin-st19:15:011
37699190,18cyclictest0-21swapper/121:05:001
37599191,2cyclictest930-21cpuspeed_turbos19:10:150
37599190,1cyclictest0-21swapper/022:04:460
37599190,18cyclictest0-21swapper/021:41:230
37599190,16cyclictest30261-21nvmesmart_nvme022:50:190
37899185,12cyclictest0-21swapper/223:10:232
37899182,15cyclictest28-21ksoftirqd/200:29:342
378991816,1cyclictest0-21swapper/222:00:132
378991815,2cyclictest19261-21vmstat20:35:232
378991815,2cyclictest18708-21cpuspeed_turbos21:35:162
378991814,2cyclictest27131-21https00:40:002
37899180,3cyclictest31917-21https20:05:022
37899180,3cyclictest12452-21munin-run00:15:012
37899180,2cyclictest24003-21cat00:35:012
37899180,17cyclictest24349-21date23:40:012
37899180,17cyclictest16707-21nvmesmart_nvme019:35:212
37899180,16cyclictest0-21swapper/200:08:372
376991816,1cyclictest10852-21cut22:15:181
376991816,1cyclictest0-21swapper/119:45:031
376991814,3cyclictest22597-21nfsd421:40:171
376991814,3cyclictest11491-21ls19:25:221
37699181,2cyclictest31840-21munin-run20:05:011
37699181,16cyclictest12893-21chrt00:15:061
37699180,2cyclictest4523-21H212:26:331
37699180,17cyclictest1072-21fschecks_time22:00:141
37699180,17cyclictest0-21swapper/121:30:231
37699180,15cyclictest14878-21latency_hist19:35:011
37699180,15cyclictest0-21swapper/120:10:021
375991816,1cyclictest9-21ksoftirqd/000:27:300
375991816,1cyclictest25895-21switchtime00:35:210
375991814,3cyclictest25673-21master19:51:140
37599180,17cyclictest0-21swapper/020:05:140
37599180,16cyclictest26987-21nfsd420:50:200
378991715,1cyclictest1314-21cat19:10:152
378991715,1cyclictest0-21swapper/221:28:122
378991714,2cyclictest6073-21taskset00:01:222
378991714,2cyclictest5958-21cron22:10:042
378991714,2cyclictest5773-21nvmesmart_nvme019:15:202
378991714,2cyclictest4776-21lxd21:08:562
378991714,2cyclictest2245-21timerandwakeup23:55:212
378991714,1cyclictest0-21swapper/222:45:232
37899171,15cyclictest13188-21threads20:25:222
37899171,15cyclictest11321-21uname19:25:202
37899170,2cyclictest6057-21python322:10:022
37899170,2cyclictest4532-21H212:26:332
37899170,2cyclictest2341-21cron20:09:002
37899170,2cyclictest15658-21chrt23:23:482
37899170,2cyclictest10877-21grep23:15:132
37899170,1cyclictest0-21swapper/222:59:012
37899170,1cyclictest0-21swapper/222:59:012
37899170,16cyclictest9306-21df20:20:162
37899170,16cyclictest9306-21df20:20:152
37899170,16cyclictest7869-21date23:10:012
37899170,16cyclictest4001-21if_err_eno120:10:172
37899170,16cyclictest22599-21cut21:40:172
37899170,16cyclictest20907-21fschecks_time20:40:152
37899170,16cyclictest20238-21nfsd423:30:182
37899170,16cyclictest17983-21cpuspeed_turbos22:30:142
37899170,16cyclictest16319-21cpuspeed_turbos23:25:142
37899170,16cyclictest15578-21cat22:25:162
37899170,16cyclictest1555-21lxd19:54:562
37899170,16cyclictest13700-21nvmesmart_nvme022:20:192
37899170,16cyclictest12580-21apache_volume19:30:122
37899170,16cyclictest0-21swapper/220:15:212
37699178,0cyclictest0-21swapper/122:05:181
376991715,1cyclictest0-21swapper/120:12:171
376991715,1cyclictest0-21swapper/100:05:191
376991714,2cyclictest6281-21expr20:15:141
376991714,2cyclictest5974-21apt-get22:10:031
376991714,2cyclictest4536-21H212:26:331
376991714,2cyclictest28577-21tr20:55:151
376991714,2cyclictest28054-21sed19:55:211
376991714,2cyclictest27892-21fschecks_count23:45:151
376991714,2cyclictest15476-21sed21:30:001
376991714,2cyclictest13863-21python320:30:011
376991714,2cyclictest11214-21munin-plugin-st20:25:021
376991714,2cyclictest11214-21munin-plugin-st20:25:011
376991714,2cyclictest10904-21awk21:20:141
37699171,15cyclictest7122-21latency_hist21:15:011
37699170,2cyclictest3355-21qmgr21:48:581
37699170,2cyclictest27234-21/usr/sbin/munin21:50:201
37699170,2cyclictest22034-21systemctl22:35:221
37699170,2cyclictest18477-21wget22:30:171
37699170,2cyclictest16274-21awk22:25:191
37699170,2cyclictest11406-21/usr/sbin/munin23:15:161
37699170,16cyclictest7504-21cat19:20:171
37699170,16cyclictest4526-21H212:26:331
37699170,16cyclictest4523-21H212:26:331
37699170,16cyclictest3908-21cstates00:00:151
37699170,16cyclictest3212-21munin-run00:00:021
37699170,16cyclictest30463-21ls23:50:161
37699170,16cyclictest27057-21ntp_kernel_pll_20:50:201
37699170,16cyclictest23532-21cpuspeed_turbos22:40:141
37699170,16cyclictest2049-21systemd-journal23:00:041
37699170,16cyclictest18910-21apache_volume23:30:111
37699170,16cyclictest18487-21expr19:40:131
37699170,16cyclictest17972-21latency_hist19:40:011
37699170,16cyclictest1590-21apache_accesses21:05:131
37699170,16cyclictest1555-21lxd22:48:561
375991715,1cyclictest0-21swapper/022:45:220
375991715,1cyclictest0-21swapper/020:55:150
375991715,1cyclictest0-21swapper/020:10:130
375991714,2cyclictest2281-21grep21:05:160
375991714,2cyclictest20489-21ls20:40:130
375991714,2cyclictest19954-21cut20:40:000
375991714,2cyclictest19854-21runrttasks22:32:340
375991714,2cyclictest1411-21lxd23:24:570
375991714,2cyclictest13789-21perf20:30:020
375991714,2cyclictest11165-21expr23:15:140
37599171,2cyclictest22308-21tr19:45:170
37599171,15cyclictest0-21swapper/022:40:160
37599170,2cyclictest8751-21chrt22:10:340
37599170,2cyclictest6758-21/usr/sbin/munin00:05:140
37599170,2cyclictest4526-21H212:26:330
37599170,2cyclictest29323-21expr20:00:130
37599170,2cyclictest28746-21munin-run20:00:010
37599170,2cyclictest19350-21if_lxdbr021:35:190
37599170,2cyclictest14269-21chrt22:20:440
37599170,2cyclictest12258-21chrt21:20:290
37599170,1cyclictest4534-21H212:26:330
37599170,1cyclictest4534-21H212:26:330
37599170,1cyclictest0-21swapper/000:10:130
37599170,16cyclictest8247-21hddtemp_smartct21:15:150
37599170,16cyclictest6050-21python322:10:000
37599170,16cyclictest4593-21H212:26:330
37599170,16cyclictest31558-21/usr/sbin/munin22:55:220
37599170,16cyclictest31558-21/usr/sbin/munin22:55:220
37599170,16cyclictest1411-21lxd19:24:560
37599170,16cyclictest1304-21lxd21:30:560
37599170,16cyclictest10565-21date23:15:000
37599170,16cyclictest0-21swapper/022:15:190
378991614,1cyclictest0-21swapper/222:50:182
378991614,1cyclictest0-21swapper/221:50:152
378991614,1cyclictest0-21swapper/221:20:132
378991614,1cyclictest0-21swapper/219:55:092
378991613,2cyclictest28782-21expr20:55:142
37899160,2cyclictest30703-21/usr/sbin/munin21:55:152
37899160,2cyclictest26897-21nfsd420:50:212
37899160,1cyclictest30435-21expr23:50:172
37899160,1cyclictest0-21swapper/222:40:142
37899160,1cyclictest0-21swapper/220:30:022
37899160,15cyclictest32494-21awk21:00:232
37899160,15cyclictest2796-21sed23:00:152
37899160,15cyclictest0-21swapper/223:45:192
37899160,15cyclictest0-21swapper/223:40:192
37899160,15cyclictest0-21swapper/222:35:072
37899160,15cyclictest0-21swapper/221:30:142
37899160,15cyclictest0-21swapper/221:10:152
37899160,15cyclictest0-21swapper/220:45:152
37899160,15cyclictest0-21swapper/220:45:152
376991614,1cyclictest229602sleep120:45:131
376991614,1cyclictest229602sleep120:45:131
376991614,1cyclictest0-21swapper/123:25:031
376991614,1cyclictest0-21swapper/120:30:011
376991613,2cyclictest5785-21cut23:05:141
37699160,15cyclictest29104-21ls22:50:151
37699160,15cyclictest21638-21apache_volume23:35:121
37699160,15cyclictest18223-21irqstats20:35:171
37699160,15cyclictest0-21swapper/123:10:141
37699160,15cyclictest0-21swapper/122:55:151
37699160,15cyclictest0-21swapper/122:55:141
37699160,15cyclictest0-21swapper/121:55:151
375991614,1cyclictest0-21swapper/023:40:240
375991614,1cyclictest0-21swapper/021:50:220
375991614,1cyclictest0-21swapper/021:00:200
375991614,1cyclictest0-21swapper/020:45:190
375991614,1cyclictest0-21swapper/020:45:180
375991614,1cyclictest0-21swapper/019:44:370
375991614,1cyclictest0-21swapper/000:15:180
37599161,1cyclictest0-21swapper/022:35:120
37599160,15cyclictest5249-21tr23:05:140
37599160,15cyclictest4530-21H212:26:330
37599160,15cyclictest30272-21cat21:55:140
37599160,15cyclictest16160-21/usr/sbin/munin23:25:140
37599160,15cyclictest1079-21gdbus23:50:230
37599160,15cyclictest0-21swapper/023:55:210
37599160,15cyclictest0-21swapper/023:45:160
37599160,15cyclictest0-21swapper/022:25:150
37599160,15cyclictest0-21swapper/021:25:140
37599160,15cyclictest0-21swapper/020:15:160
375991513,1cyclictest0-21swapper/019:35:160
37599150,1cyclictest0-21swapper/023:35:200
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional