You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-15 - 19:06
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot2.osadl.org (updated Sat Nov 15, 2025 12:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
177942202186,11sleep10-21swapper/107:08:121
157132195178,12sleep00-21swapper/007:05:060
179152194178,11sleep30-21swapper/307:09:473
178922175158,12sleep20-21swapper/207:09:302
213162450,0sleep00-21swapper/009:29:540
321992430,0sleep10-21swapper/111:28:171
250582400,0sleep30-21swapper/307:20:183
6552360,0sleep10-21swapper/107:35:211
1812799111,0cyclictest0-21swapper/211:10:132
1812799100,1cyclictest581-21apt-get10:40:142
181279981,1cyclictest4818-21idleruntime-cro10:10:002
181279981,1cyclictest18515-21fschecks_time12:10:162
181279981,1cyclictest0-21swapper/211:25:142
181279981,0cyclictest0-21swapper/209:50:142
181279980,2cyclictest621-21ssh11:45:122
181279980,1cyclictest5910-21munin-run08:55:002
181279980,1cyclictest3424-21apt08:50:002
181279980,1cyclictest28669-21munin-run07:30:002
181279980,1cyclictest23510-21diskstats11:40:142
181279980,1cyclictest22734-21ntp_states07:15:192
181279980,1cyclictest2137-21perl07:40:162
181279980,1cyclictest13544-21sh09:10:022
181279980,1cyclictest11974-21ssh09:25:142
181279980,1cyclictest11170-21munin-run12:40:002
181279972,1cyclictest30800-21cpuspeed10:55:152
181279971,1cyclictest714-21snmpd07:48:112
181279971,1cyclictest16959-21cut08:10:152
181279971,1cyclictest11477-21open_files09:40:212
181279971,1cyclictest11371-21expr10:45:162
181279970,7cyclictest25931-21ssh10:52:282
181279970,2cyclictest24712-21expr07:20:172
181279970,1cyclictest9387-21cut09:00:162
181279970,1cyclictest8609-21perl12:05:132
181279970,1cyclictest85472sleep207:54:282
181279970,1cyclictest85252sleep211:32:322
181279970,1cyclictest6137-21df11:15:142
181279970,1cyclictest5022-21aten2_r5power_e12:20:132
181279970,1cyclictest4440-21ntp_states10:25:212
181279970,1cyclictest3332-21ssh09:20:202
181279970,1cyclictest32178-21/usr/sbin/munin12:00:212
181279970,1cyclictest30876-21cron08:40:012
181279970,1cyclictest28389-21perl12:15:162
181279970,1cyclictest27686-21ssh09:16:152
181279970,1cyclictest27489-21users10:20:232
181279970,1cyclictest25958-21sleep08:30:002
181279970,1cyclictest23721-21perl10:35:152
181279970,1cyclictest22242-21wc11:55:212
181279970,1cyclictest22053-21ssh09:30:142
181279970,1cyclictest20745-21perl09:45:152
181279970,1cyclictest20632-21chrt08:15:412
181279970,1cyclictest19881-21users10:00:252
181279970,1cyclictest16738-21grep12:25:202
181279970,1cyclictest15263-21ls09:10:182
181279970,1cyclictest15263-21ls09:10:172
181279970,1cyclictest15087-21ssh11:03:292
181279970,1cyclictest15066-21ntp_states08:05:192
181279970,1cyclictest14795-21munin-run10:15:002
181279970,1cyclictest13811-21perl11:35:172
181279970,1cyclictest13792-21perl10:30:162
181279970,1cyclictest13569-21apt08:05:002
181279970,1cyclictest1171-21cat07:40:002
181279970,1cyclictest1-21systemd12:30:212
181279970,0cyclictest28420-21apt08:35:002
181369966,0cyclictest0-21swapper/309:20:143
181369960,3cyclictest10614-21/usr/sbin/munin11:50:173
181279961,1cyclictest7678-21ssh09:55:122
181279960,1cyclictest8840-21/usr/sbin/munin07:55:122
181279960,1cyclictest6706-21/usr/sbin/munin08:55:132
181279960,1cyclictest32326-21tr09:35:132
181279960,1cyclictest31342-21expr08:40:142
181279960,1cyclictest29182-21/usr/sbin/munin07:30:142
181279960,1cyclictest21597-21expr08:20:142
181279960,1cyclictest18742-21df07:10:132
181279960,1cyclictest17894-21ssh11:05:062
181279960,1cyclictest15326-21rm11:20:052
181279960,1cyclictest15181-21ssh10:15:072
181279960,1cyclictest10500-21ssh11:50:102
181369955,0cyclictest15729-21kworker/3:112:02:313
181369952,2cyclictest8449-21df_inode12:05:133
181369950,5cyclictest1738-21rm12:18:173
181369950,5cyclictest0-21swapper/311:33:013
181369950,5cyclictest0-21swapper/309:35:123
181369950,5cyclictest0-21swapper/309:16:153
181369950,4cyclictest11345-21perl10:45:173
181369950,4cyclictest0-21swapper/311:55:113
181249952,2cyclictest0-21swapper/107:45:131
181249951,3cyclictest0-21swapper/108:35:121
181249951,2cyclictest0-21swapper/110:25:141
181249951,2cyclictest0-21swapper/109:00:131
181249951,2cyclictest0-21swapper/107:25:131
181249950,3cyclictest0-21swapper/109:45:121
181189955,0cyclictest0-21swapper/010:00:420
181189954,1cyclictest5048-21munin-run11:15:010
181189950,5cyclictest0-21swapper/012:27:190
181189950,0cyclictest0-21swapper/011:23:030
181369944,0cyclictest0-21swapper/311:11:213
181369944,0cyclictest0-21swapper/310:38:363
181369944,0cyclictest0-21swapper/310:10:043
181369943,1cyclictest23710-21/usr/sbin/munin08:25:213
181369943,1cyclictest22934-21gltestperf09:30:163
181369942,1cyclictest14251-21ssh09:25:293
181369941,3cyclictest55950irq/130-eth012:35:193
181369941,3cyclictest30762-21fschecks_time09:50:163
181369941,3cyclictest20533-21ssh09:13:063
181369941,3cyclictest20533-21ssh09:13:063
181369940,4cyclictest61-21kswapd010:25:143
181369940,4cyclictest3666-21apt-get08:50:113
181369940,4cyclictest10242-21ssh11:00:263
181369940,4cyclictest0-21swapper/312:10:153
181369940,4cyclictest0-21swapper/310:55:133
181369940,4cyclictest0-21swapper/310:50:193
181369940,4cyclictest0-21swapper/310:20:153
181369940,4cyclictest0-21swapper/310:09:213
181369940,4cyclictest0-21swapper/309:55:233
181369940,4cyclictest0-21swapper/309:47:153
181369940,4cyclictest0-21swapper/309:41:353
181369940,4cyclictest0-21swapper/307:30:143
181369940,3cyclictest811-21sh11:45:143
181369940,3cyclictest7602-21sh10:43:313
181369940,3cyclictest5721-21tune2fs12:20:143
181369940,3cyclictest30378-21ssh11:43:473
181369940,3cyclictest21817-21ssh10:01:373
181369940,3cyclictest19745-21python07:10:203
181369940,3cyclictest19270-21df_inode08:15:143
181369940,3cyclictest19063-21sh12:26:473
181369940,3cyclictest16797-21diskstats08:10:133
181369940,2cyclictest10390-21ssh11:17:073
181369940,1cyclictest16530-21memory10:15:183
181369940,1cyclictest13016-21/usr/sbin/munin11:35:203
181249944,0cyclictest0-21swapper/112:21:531
181249944,0cyclictest0-21swapper/111:55:241
181249944,0cyclictest0-21swapper/109:16:481
181249942,1cyclictest0-21swapper/109:20:131
181249941,2cyclictest0-21swapper/112:05:111
181249941,2cyclictest0-21swapper/111:35:151
181249941,2cyclictest0-21swapper/111:30:171
181249941,2cyclictest0-21swapper/111:15:121
181249941,2cyclictest0-21swapper/110:30:141
181249941,2cyclictest0-21swapper/110:20:141
181249941,2cyclictest0-21swapper/110:15:121
181249941,2cyclictest0-21swapper/109:55:131
181249941,2cyclictest0-21swapper/108:55:121
181249941,2cyclictest0-21swapper/108:50:111
181249941,2cyclictest0-21swapper/108:40:131
181249941,2cyclictest0-21swapper/108:15:131
181249941,2cyclictest0-21swapper/108:10:121
181249941,2cyclictest0-21swapper/107:15:121
181249940,4cyclictest0-21swapper/110:55:001
181249940,4cyclictest0-21swapper/109:40:321
181249940,3cyclictest0-21swapper/109:50:131
181249940,2cyclictest0-21swapper/112:30:151
181249940,0cyclictest0-21swapper/110:42:301
181189944,0cyclictest0-21swapper/012:02:200
181189944,0cyclictest0-21swapper/011:00:380
181189944,0cyclictest0-21swapper/009:53:160
181189944,0cyclictest0-21swapper/009:45:080
181189944,0cyclictest0-21swapper/009:36:150
181189943,1cyclictest26502-21tr08:30:140
181189942,1cyclictest26012-21ssh10:20:170
181189942,1cyclictest16341-21ssh12:09:270
181189941,2cyclictest0-21swapper/011:35:150
181189940,4cyclictest18042-21ssh11:05:130
181189940,4cyclictest13529-21diskmemload12:22:400
181189940,4cyclictest0-21swapper/012:30:150
181189940,4cyclictest0-21swapper/012:18:590
181189940,4cyclictest0-21swapper/011:40:130
181189940,4cyclictest0-21swapper/010:55:130
181189940,4cyclictest0-21swapper/010:45:140
181189940,4cyclictest0-21swapper/010:35:130
181189940,4cyclictest0-21swapper/010:25:140
181189940,4cyclictest0-21swapper/010:10:130
181189940,4cyclictest0-21swapper/010:05:440
181189940,4cyclictest0-21swapper/007:45:130
181189940,3cyclictest6029-21cat08:55:010
181189940,3cyclictest3838-21sed07:45:000
181189940,3cyclictest32573-21rm11:45:010
181189940,3cyclictest3142-21ssh11:30:160
181189940,3cyclictest3037-21ssh09:20:190
181189940,3cyclictest26103-21df11:25:160
181189940,3cyclictest15174-21cut09:10:170
181189940,3cyclictest15174-21cut09:10:170
181189940,3cyclictest0-21swapper/009:05:140
10192240,0sleep10-21swapper/107:55:201
181369930,3cyclictest9352-21df_inode09:00:153
181369930,3cyclictest26953-21df_inode07:25:143
181369930,3cyclictest25523-21apt-get11:25:133
181369930,3cyclictest21041-21munin-node07:15:003
181369930,3cyclictest17751-21rm11:05:013
181369930,3cyclictest13800-21apt-get08:05:133
181369930,3cyclictest0-21swapper/312:30:083
181369930,3cyclictest0-21swapper/311:20:053
181369930,3cyclictest0-21swapper/310:30:073
181369930,3cyclictest0-21swapper/309:05:133
181369930,3cyclictest0-21swapper/308:55:013
181369930,3cyclictest0-21swapper/308:45:133
181369930,3cyclictest0-21swapper/308:40:133
181369930,3cyclictest0-21swapper/308:30:133
181369930,3cyclictest0-21swapper/308:20:103
181369930,3cyclictest0-21swapper/308:00:113
181369930,3cyclictest0-21swapper/307:55:123
181369930,3cyclictest0-21swapper/307:50:093
181369930,3cyclictest0-21swapper/307:45:133
181369930,3cyclictest0-21swapper/307:40:143
181369930,3cyclictest0-21swapper/307:35:023
181369930,1cyclictest28417-21latency_hist08:35:003
181249931,2cyclictest0-21swapper/110:00:161
181249931,1cyclictest0-21swapper/112:15:131
181249931,1cyclictest0-21swapper/112:15:001
181249931,1cyclictest0-21swapper/111:50:141
181249931,1cyclictest0-21swapper/111:45:011
181249931,1cyclictest0-21swapper/111:40:121
181249931,1cyclictest0-21swapper/111:20:111
181249931,1cyclictest0-21swapper/111:10:131
181249931,1cyclictest0-21swapper/111:00:161
181249931,1cyclictest0-21swapper/110:50:141
181249931,1cyclictest0-21swapper/109:35:131
181249931,1cyclictest0-21swapper/109:30:141
181249931,1cyclictest0-21swapper/109:10:131
181249931,1cyclictest0-21swapper/109:10:131
181249931,1cyclictest0-21swapper/109:05:131
181249931,1cyclictest0-21swapper/108:45:141
181249931,1cyclictest0-21swapper/108:00:111
181249931,1cyclictest0-21swapper/107:40:131
181249930,3cyclictest0-21swapper/108:33:161
181249930,3cyclictest0-21swapper/107:13:591
181249930,2cyclictest0-21swapper/110:45:131
181249930,2cyclictest0-21swapper/110:10:121
181249930,2cyclictest0-21swapper/107:50:131
181189932,1cyclictest31882-21df07:35:150
181189932,1cyclictest2546-21expr12:35:180
181189931,2cyclictest471-21sh10:40:120
181189931,2cyclictest20682-21kworker/0:207:15:020
181189931,1cyclictest21107-21ssh10:50:150
181189931,1cyclictest0-21swapper/008:45:130
181189930,3cyclictest29415-21df07:30:150
181189930,3cyclictest258762sleep009:15:300
181189930,3cyclictest17813-21apt-get12:10:150
181189930,3cyclictest13529-21diskmemload09:40:120
181189930,3cyclictest11330-21apt-get08:00:110
181189930,3cyclictest0-21swapper/011:55:080
181189930,3cyclictest0-21swapper/011:50:060
181189930,3cyclictest0-21swapper/011:15:130
181189930,3cyclictest0-21swapper/010:30:130
181189930,3cyclictest0-21swapper/010:15:130
181189930,3cyclictest0-21swapper/009:55:040
181189930,3cyclictest0-21swapper/009:30:130
181189930,3cyclictest0-21swapper/009:00:120
181189930,3cyclictest0-21swapper/008:55:130
181189930,3cyclictest0-21swapper/008:40:020
181189930,3cyclictest0-21swapper/008:35:120
181189930,3cyclictest0-21swapper/008:25:110
181189930,3cyclictest0-21swapper/008:20:140
181189930,3cyclictest0-21swapper/008:15:120
181189930,3cyclictest0-21swapper/008:10:120
181189930,3cyclictest0-21swapper/008:05:120
181189930,3cyclictest0-21swapper/007:55:130
181189930,3cyclictest0-21swapper/007:50:130
181189930,3cyclictest0-21swapper/007:25:120
181189930,3cyclictest0-21swapper/007:20:130
181189930,3cyclictest0-21swapper/007:10:110
181249921,1cyclictest0-21swapper/112:35:161
181249921,1cyclictest0-21swapper/108:05:121
181249921,0cyclictest0-21swapper/110:05:141
181249920,2cyclictest0-21swapper/112:00:141
181249920,1cyclictest0-21swapper/112:25:131
181249920,1cyclictest0-21swapper/111:05:131
181249920,1cyclictest0-21swapper/110:35:141
181249920,1cyclictest0-21swapper/109:25:131
181249920,1cyclictest0-21swapper/108:25:001
181249920,1cyclictest0-21swapper/108:20:141
181249920,1cyclictest0-21swapper/107:30:141
181249920,1cyclictest0-21swapper/107:25:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional