You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-27 - 21:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Thu Nov 27, 2025 12:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
70242203186,12sleep10-21swapper/107:07:071
69812196179,12sleep30-21swapper/307:06:333
72042185167,13sleep00-21swapper/007:09:270
71962120103,12sleep20-21swapper/207:09:212
108072510,0sleep20-21swapper/209:30:002
275112500,1sleep327514-21cpuspeed12:20:113
65202470,0sleep22845-21diskmemload11:20:172
26792460,0sleep00-21swapper/010:45:400
48152450,0sleep10-21swapper/109:59:021
18122450,0sleep10-21swapper/110:45:211
146002440,0sleep00-21swapper/011:24:520
124412430,0sleep00-21swapper/012:12:080
319552420,0sleep00-21swapper/011:16:390
298972410,0sleep20-21swapper/207:50:232
239422400,0sleep20-21swapper/211:12:532
199262160,1sleep10-21swapper/112:32:311
179162130,0sleep10-21swapper/111:42:141
742599111,9cyclictest0-21swapper/010:50:110
314202100,0sleep20-21swapper/209:55:222
74259991,4cyclictest4127-21basename10:15:010
74259985,3cyclictest0-21swapper/010:28:020
74259982,5cyclictest19161-21cat07:30:150
74259981,0cyclictest0-21swapper/012:15:120
74259980,1cyclictest7050-21munin-run12:10:000
74259980,1cyclictest29572-21munin-run11:00:010
74259980,1cyclictest20392-21munin-run08:40:000
74259980,1cyclictest17534-21users08:30:200
74259971,1cyclictest5200-21taskset08:08:170
74259971,1cyclictest4211-21netstat11:35:160
74259971,1cyclictest28130-21sed12:20:130
74259971,1cyclictest24030-21basename07:40:160
74259971,1cyclictest12266-21grep09:30:170
74259971,1cyclictest11780-21cat08:20:140
74259971,1cyclictest0-21swapper/008:00:090
74259970,1cyclictest9823-21diskstats09:45:130
74259970,1cyclictest9352-21ssh12:27:010
74259970,1cyclictest5227-21df_inode10:15:150
74259970,1cyclictest3937-21diskstats09:10:140
74259970,1cyclictest31442-21ssh09:40:120
74259970,1cyclictest29197-21memory07:50:190
74259970,1cyclictest2842-21munin-run09:10:010
74259970,1cyclictest27826-21munin-run08:55:000
74259970,1cyclictest26900-21ntp_states07:45:170
74259970,1cyclictest26156-21cat12:35:190
74259970,1cyclictest24439-21perl09:20:150
74259970,1cyclictest23390-21df11:45:140
74259970,1cyclictest22297-21munin-run11:45:000
74259970,1cyclictest2229-21apt-get10:30:110
74259970,1cyclictest22260-21apt-get10:40:120
74259970,1cyclictest22167-21tail09:35:120
74259970,1cyclictest21756-21latency_hist10:40:010
74259970,1cyclictest21196-21df_inode07:35:120
74259970,1cyclictest21196-21df_inode07:35:120
74259970,1cyclictest20649-21apt08:40:110
74259970,1cyclictest20173-21apt-get12:00:090
74259970,1cyclictest17351-21munin-run11:10:010
74259970,1cyclictest14871-21ntp_states09:15:170
74259970,1cyclictest13964-21perl08:25:130
74259970,1cyclictest13017-21latency_hist07:20:010
74259970,1cyclictest10854-21df11:55:130
74259970,1cyclictest10513-21munin-run08:20:000
74259970,1cyclictest0-21swapper/009:00:100
74259970,0cyclictest23127-21apt-get08:45:140
74309960,5cyclictest29123-21chrt12:37:071
74309960,5cyclictest130782sleep111:07:191
74259961,5cyclictest10034-21kworker/0:211:10:010
74259960,1cyclictest8064-21df_inode07:10:140
74259960,1cyclictest7510-21df_inode10:00:130
74259960,1cyclictest6300-21df08:10:120
74259960,1cyclictest32612-21ssh11:50:050
74259960,1cyclictest30869-21rm11:00:140
74259960,1cyclictest30569-21apt07:55:110
74259960,1cyclictest29865-21df09:55:130
74259960,1cyclictest28059-21apt08:55:110
74259960,1cyclictest25312-21ssh11:30:120
74259960,1cyclictest18988-21ssh09:50:030
74259960,1cyclictest1868-21df_inode09:25:140
74259960,1cyclictest17561-21df10:05:150
74259960,1cyclictest16252-21df07:25:110
74259960,1cyclictest15963-21df11:25:150
74259960,1cyclictest15245-21df_inode10:20:130
74259960,1cyclictest14725-21tr12:30:120
74259960,1cyclictest13798-21diskstats07:20:140
74399955,0cyclictest28353-21kworker/3:012:19:553
74399955,0cyclictest28353-21kworker/3:011:57:313
74399955,0cyclictest28353-21kworker/3:009:54:213
74399955,0cyclictest28353-21kworker/3:008:35:593
74399955,0cyclictest28353-21kworker/3:008:24:463
74399955,0cyclictest28353-21kworker/3:008:02:233
74399955,0cyclictest28353-21kworker/3:007:17:363
74399950,5cyclictest0-21swapper/311:52:573
74399950,5cyclictest0-21swapper/310:51:253
74399950,4cyclictest5960-21df11:20:153
74399950,4cyclictest582-21/usr/sbin/munin09:05:143
74399950,4cyclictest28105-21/usr/sbin/munin07:50:233
74399950,4cyclictest0-21swapper/307:40:133
74349955,0cyclictest0-21swapper/211:34:412
74349953,2cyclictest9133-21kworker/2:111:03:492
74349953,1cyclictest21177-21ssh10:06:412
74349950,5cyclictest24158-21ssh10:56:372
74349950,5cyclictest0-21swapper/212:08:312
74349950,5cyclictest0-21swapper/211:16:352
74349950,5cyclictest0-21swapper/209:39:012
74349950,0cyclictest0-21swapper/210:26:352
74309953,1cyclictest0-21swapper/112:15:121
74309950,5cyclictest0-21swapper/111:53:561
74309950,5cyclictest0-21swapper/107:25:021
74309950,4cyclictest9077-21chrt10:16:511
74309950,4cyclictest90372chrt09:45:041
74309950,4cyclictest8824-21chrt11:21:131
74309950,4cyclictest7481-21chrt12:10:091
74309950,4cyclictest6308-21chrt09:10:541
74309950,4cyclictest594-21chrt11:17:201
74309950,4cyclictest5696-21chrt10:31:121
74309950,4cyclictest4645-21chrt12:25:101
74309950,4cyclictest321-21chrt09:24:471
74309950,4cyclictest31934-21chrt11:33:141
74309950,4cyclictest30568-21chrt07:55:111
74309950,4cyclictest299832sleep107:50:381
74309950,4cyclictest280032chrt10:26:081
74309950,4cyclictest27664-21chrt08:53:071
74309950,4cyclictest2743-21chrt08:03:271
74309950,4cyclictest27229-21chrt12:20:091
74309950,4cyclictest2613-21chrt11:35:021
74309950,4cyclictest24471-21chrt10:40:251
74309950,4cyclictest24075-21chrt09:35:441
74309950,4cyclictest23941-21taskset12:01:241
74309950,4cyclictest19290-21chrt10:05:301
74309950,4cyclictest19115-21chrt09:17:341
74309950,4cyclictest17789-21chrt10:20:591
74309950,4cyclictest17646-21chrt11:25:291
74309950,4cyclictest170722sleep111:58:201
74309950,4cyclictest1690-21chrt10:13:311
74309950,4cyclictest16668-21chrt10:36:491
74309950,4cyclictest1234-21chrt09:25:111
74309950,4cyclictest0-21swapper/110:55:111
74309950,4cyclictest0-21swapper/107:20:121
7374250,0sleep20-21swapper/209:44:172
74399944,0cyclictest0-21swapper/311:44:483
74399944,0cyclictest0-21swapper/311:38:513
74399944,0cyclictest0-21swapper/310:43:593
74399942,1cyclictest3109-21/usr/sbin/munin08:05:183
74399942,1cyclictest30073-21/usr/sbin/munin12:05:183
74399942,1cyclictest15196-21ssh12:13:543
74399942,1cyclictest13439-21/usr/sbin/munin09:15:193
74399941,3cyclictest20673-21expr12:00:113
74399941,1cyclictest0-21swapper/311:30:113
74399940,4cyclictest6493-21sed12:25:183
74399940,4cyclictest1230-21irqstats10:45:183
74399940,4cyclictest0-21swapper/312:35:183
74399940,4cyclictest0-21swapper/311:00:053
74399940,4cyclictest0-21swapper/310:35:073
74399940,4cyclictest0-21swapper/307:10:123
74399940,3cyclictest6922-21munin-node10:00:113
74399940,3cyclictest4647-21/usr/sbin/munin10:15:223
74399940,3cyclictest26332-21perl07:45:133
74399940,3cyclictest21950-21cat10:55:203
74399940,3cyclictest21823-21sh10:07:053
74399940,3cyclictest0-21swapper/308:45:153
74399940,1cyclictest30545-21/usr/sbin/munin09:00:163
74399940,1cyclictest22824-21/usr/sbin/munin11:45:153
74399940,1cyclictest21306-21/usr/sbin/munin09:35:143
74399940,1cyclictest20647-21/usr/sbin/munin08:40:123
74399940,1cyclictest14487-21/usr/sbin/munin12:30:133
74399940,1cyclictest13219-21/usr/sbin/munin08:25:133
74399940,1cyclictest11399-21/usr/sbin/munin09:30:223
74399940,1cyclictest10936-21sensors09:45:193
74399940,0cyclictest0-21swapper/310:11:333
74349944,0cyclictest0-21swapper/211:41:232
74349944,0cyclictest0-21swapper/208:20:172
74349943,1cyclictest9133-21kworker/2:108:26:532
74349943,1cyclictest3371-21ssh10:46:062
74349943,1cyclictest2769-21ssh11:35:082
74349943,1cyclictest25461-21ssh11:45:422
74349943,1cyclictest11301-21ssh10:01:472
74349942,2cyclictest714-21snmpd10:18:032
74349942,1cyclictest714-21snmpd07:33:222
74349941,3cyclictest23508-21kworker/2:012:15:122
74349940,4cyclictest7875-21ssh11:05:112
74349940,4cyclictest2787-21df10:30:132
74349940,4cyclictest13266-21apt-get07:20:122
74349940,4cyclictest12386-21ssh12:12:062
74349940,4cyclictest0-21swapper/212:30:352
74349940,4cyclictest0-21swapper/212:21:262
74349940,4cyclictest0-21swapper/212:03:292
74349940,4cyclictest0-21swapper/211:55:122
74349940,4cyclictest0-21swapper/210:50:182
74349940,4cyclictest0-21swapper/210:10:462
74349940,4cyclictest0-21swapper/209:50:212
74349940,4cyclictest0-21swapper/209:15:112
74349940,4cyclictest0-21swapper/208:55:112
74349940,4cyclictest0-21swapper/207:55:152
74349940,3cyclictest32757-21/usr/sbin/munin11:50:142
74349940,3cyclictest324902sleep209:01:212
74349940,3cyclictest3102-21ssh09:10:022
74349940,3cyclictest2837-21latency_hist09:10:012
74349940,3cyclictest26944-21tr08:50:202
74349940,3cyclictest25946-21ssh10:41:202
74349940,3cyclictest25083-21perl12:35:132
74349940,3cyclictest24705-21if_eth409:20:172
74349940,3cyclictest18122-21sed07:30:012
74349940,3cyclictest17433-21/usr/sbin/munin08:30:192
74349940,3cyclictest15970-21df_inode11:25:162
74349940,3cyclictest14668-21/usr/sbin/munin10:20:122
74349940,3cyclictest13178-21rm12:29:302
74349940,3cyclictest0-21swapper/209:30:142
74349940,0cyclictest0-21swapper/209:48:462
74309942,1cyclictest0-21swapper/111:45:121
74309942,1cyclictest0-21swapper/110:50:111
74309942,1cyclictest0-21swapper/107:10:121
74309941,3cyclictest5742chrt09:05:051
74309941,2cyclictest0-21swapper/111:10:111
74309941,2cyclictest0-21swapper/108:35:091
74309941,2cyclictest0-21swapper/108:10:011
74309941,2cyclictest0-21swapper/107:30:111
74309940,4cyclictest32534-21taskset09:01:561
74309940,4cyclictest29810-21chrt12:05:001
74309940,4cyclictest1151-21taskset11:01:071
74309940,4cyclictest112092sleep109:30:071
74309940,4cyclictest0-21swapper/110:01:021
74309940,4cyclictest0-21swapper/109:51:171
74309940,4cyclictest0-21swapper/109:40:121
74309940,4cyclictest0-21swapper/108:15:201
74309940,4cyclictest0-21swapper/107:40:131
74309940,3cyclictest20656-21chrt07:35:011
74309940,3cyclictest20656-21chrt07:35:011
74309940,3cyclictest107442sleep108:20:051
5373240,0sleep30-21swapper/309:43:033
4674240,0sleep30-21swapper/311:19:553
74399933,0cyclictest0-21swapper/310:25:153
74399933,0cyclictest0-21swapper/307:30:013
74399933,0cyclictest0-21swapper/307:20:083
74399932,1cyclictest29404-21grep08:55:173
74399932,1cyclictest15139-21ssh11:25:043
74399931,1cyclictest25594-21apt-get08:50:123
74399931,1cyclictest2244-21aten2_r5power_c10:30:113
74399931,1cyclictest17815-21apt-get11:10:113
74399931,1cyclictest0-21swapper/307:55:113
74399930,3cyclictest7816-21ssh11:05:093
74399930,3cyclictest15691-21apt-get08:30:113
74399930,3cyclictest0-21swapper/309:25:153
74399930,3cyclictest0-21swapper/309:20:103
74399930,3cyclictest0-21swapper/308:15:113
74399930,3cyclictest0-21swapper/308:10:103
74399930,3cyclictest0-21swapper/307:35:113
74399930,3cyclictest0-21swapper/307:35:113
74399930,3cyclictest0-21swapper/307:25:103
74399930,2cyclictest3111-21sh09:10:033
74399930,1cyclictest29296-21apt09:55:113
74399930,1cyclictest0-21swapper/310:20:103
74349932,1cyclictest714-21snmpd08:45:022
74349932,1cyclictest714-21snmpd08:00:052
74349932,1cyclictest714-21snmpd07:45:082
74349931,1cyclictest0-21swapper/207:15:122
74349930,3cyclictest8192-21grep07:10:142
74349930,3cyclictest21648-21sshd07:35:152
74349930,3cyclictest21648-21sshd07:35:152
74349930,3cyclictest12019-21ssh10:35:032
74349930,3cyclictest0-21swapper/208:40:112
74349930,3cyclictest0-21swapper/208:35:012
74349930,3cyclictest0-21swapper/208:15:012
74349930,3cyclictest0-21swapper/208:10:012
74349930,3cyclictest0-21swapper/208:05:092
74349930,3cyclictest0-21swapper/207:40:132
74309930,3cyclictest0-21swapper/108:55:041
74309930,3cyclictest0-21swapper/108:45:061
74309930,3cyclictest0-21swapper/108:40:061
74309930,3cyclictest0-21swapper/108:30:011
74309930,3cyclictest0-21swapper/108:25:011
74309930,3cyclictest0-21swapper/108:05:031
74309930,3cyclictest0-21swapper/107:45:021
74309930,3cyclictest0-21swapper/107:15:031
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional