You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 21:12
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot1.osadl.org (updated Wed Dec 03, 2025 12:44:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2217039890,0EcMasterDemoSyn2218029tAtEmLog_009:37:395
2294239840,0EcMasterDemoSyn2295229tAtEmLog_010:33:255
2964239830,0EcMasterDemoSyn2965229tAtEmLog_007:10:355
1048039830,0EcMasterDemoSyn1049129tAtEmLog_010:23:175
448039820,0EcMasterDemoSyn449029tAtEmLog_010:18:135
3112839820,0EcMasterDemoSyn3113829tAtEmLog_010:13:085
2909239820,0EcMasterDemoSyn2910229tAtEmLog_008:16:305
1962739820,0EcMasterDemoSyn1963729tAtEmLog_007:56:145
1801739820,0EcMasterDemoSyn1802729tAtEmLog_009:02:095
1328239820,0EcMasterDemoSyn1329229tAtEmLog_008:52:005
1092639820,0EcMasterDemoSyn1093629tAtEmLog_008:46:565
444339810,0EcMasterDemoSyn445329tAtEmLog_009:22:255
2513939810,0EcMasterDemoSyn2514929tAtEmLog_010:08:045
227439810,0EcMasterDemoSyn228429tAtEmLog_007:20:435
2037339810,0EcMasterDemoSyn2038329tAtEmLog_009:07:125
2921239800,0EcMasterDemoSyn2922229tAtEmLog_010:38:305
104039800,0EcMasterDemoSyn105129tAtEmLog_009:47:465
2781139540,0EcMasterDemoSyn27819-21grep09:42:434
1962739480,0EcMasterDemoSyn0-21swapper/307:56:143
111339330,4EcMasterDemoSyn1118-21runintdemo5min11:34:161
728839300,0EcMasterDemoSyn7297-21runintdemo5min07:30:521
984139220,0EcMasterDemoSyn9848-21ps10:48:383
2965229200,0tAtEmLog_00-21swapper/507:15:395
2875399180,10tDemoTimingTask29428-21chrt12:25:025
2769699186,9cyclictest728-21gdbus08:30:262
2769499185,10cyclictest12465-21sh09:56:430
2769499180,14cyclictest7156-21sh10:45:470
2769499180,10cyclictest4899-21sh11:36:270
2769499180,10cyclictest24042-21sh11:25:430
201729180,0tAtEmLog_00-21swapper/511:13:585
112499180,15tDemoTimingTask5113-21chrt11:36:425
2832339170,9EcMasterDemoSyn29676-21taskset11:05:005
2769799175,9cyclictest781-21postgres10:56:133
2769699174,9cyclictest18283-21latency_hist10:30:012
27695991714,2cyclictest23741-21kworker/1:1+events11:08:541
27695991713,3cyclictest704-21dbus-daemon12:20:001
2769499170,9cyclictest29178-21sh11:57:480
2769499170,9cyclictest28197-21sh11:30:010
2769499170,9cyclictest1740-21sh10:41:500
2769499170,13cyclictest21958-21sh10:58:500
2463399170,7tDemoTimingTask27239-21chrt09:15:095
2463399170,7tDemoTimingTask25887-21chrt09:13:455
2769899160,9cyclictest19253-21switchtime07:55:264
2769699169,6cyclictest25429-21sed09:40:182
2769699162,10cyclictest20604-21grep10:30:252
27694991612,3cyclictest20495-21cpuspeed_turbos08:00:160
2769499160,9cyclictest805-21runrttasks10:35:380
2769499160,8cyclictest20630-21diskmemload11:53:590
2769499160,7cyclictest7601-21sh12:06:590
2769499160,7cyclictest7601-21sh12:06:590
2769499160,11cyclictest1636-21taskset08:29:050
143299160,10tDemoTimingTask1760-21chrt08:30:005
112499160,14tDemoTimingTask27894-21turbostat11:34:555
746229150,6tAtEmLog_012870-21chrt11:43:315
27698991511,3cyclictest21754-21cpuspeed_turbos10:05:144
2769899150,9cyclictest805-21runrttasks12:39:204
2769899150,9cyclictest805-21runrttasks08:09:234
2769799153,8cyclictest334-21systemd-journal07:51:483
2769799152,9cyclictest12535-21runrttasks07:40:513
2769799151,11cyclictest16-1pr/legacy08:36:463
2769799151,10cyclictest25990-21/usr/sbin/munin10:35:213
2769799151,10cyclictest17706-21runrttasks10:29:363
2769799150,9cyclictest5591-21/usr/sbin/munin07:30:143
2769799150,10cyclictest15807-21ssh10:00:093
2769699159,2cyclictest0-21swapper/211:20:002
2769699153,9cyclictest738-21snmpd09:07:472
2769699153,9cyclictest12970-21seq12:10:542
2769699152,9cyclictest24401-21runintdemo5min08:06:252
2769699151,8cyclictest2389550kworker/u24:1+flush-9:009:57:232
2769699150,9cyclictest24569-21sh11:26:182
2769599151,10cyclictest16846-21ssh10:54:461
2769599151,10cyclictest13051-21ssh12:37:261
2769599150,9cyclictest3499-21sh12:04:061
2769599150,8cyclictest23482-21timerandwakeup11:25:281
2769599150,12cyclictest22744-21wget08:05:131
2769599150,11cyclictest27639-21diskstats10:10:171
2769499157,6cyclictest11361-21perl07:40:230
2769499156,7cyclictest5610-21sh09:51:120
2769499154,8cyclictest27277-21tr10:10:130
2769499151,10cyclictest18262-21rm10:01:260
2769499150,9cyclictest32531-21sh11:06:420
2769499150,9cyclictest29004-21diskmemload11:04:290
2769499150,8cyclictest805-21runrttasks10:51:440
2769499150,8cyclictest23886-21sh10:06:280
2769499150,6cyclictest4537-21/usr/sbin/munin12:30:200
2769499150,11cyclictest6856-21smartctl10:20:140
1046339150,2EcMasterDemoSyn651irq_work/509:27:305
2769899144,5cyclictest10297-21head09:55:194
2769899142,9cyclictest22088-21taskset08:01:544
2769899141,9cyclictest6114-21/usr/sbin/munin10:20:124
2769899141,9cyclictest24667-21sh10:35:044
2769899141,9cyclictest20835-21cat09:09:594
2769899141,9cyclictest1262150latency_hist09:30:004
2769899141,9cyclictest11502-21/usr/sbin/munin08:50:144
27698991410,3cyclictest829-21runintdemo5min12:30:054
27698991410,3cyclictest4360-21cron12:05:004
27698991410,3cyclictest24546-21needreboot11:00:184
2769899140,11cyclictest15616-21aten2_r5power_p07:50:124
2769899140,10cyclictest31046-21ssh11:05:194
2769899140,10cyclictest28980-21unixbench_multi09:15:254
2769899140,10cyclictest21997-21/usr/sbin/munin11:25:254
2769899140,10cyclictest21139-21sh10:31:314
2769899140,10cyclictest21129-21chrt11:24:154
2769899140,10cyclictest14540-21idleruntime08:55:244
2769899140,10cyclictest11387-21grep11:15:284
2769799143,8cyclictest22134-21cut10:05:163
2769799143,6cyclictest20194-21sensors_temp10:30:213
2769799141,9cyclictest29047-21runrttasks08:16:043
2769799140,9cyclictest3314-21ssh11:10:053
2769799140,9cyclictest29468-21ssh10:10:583
2769799140,11cyclictest16-1pr/legacy08:26:373
2769799140,10cyclictest977-21/usr/sbin/munin09:20:253
2769799140,10cyclictest9625-21/usr/sbin/munin09:55:203
2769799140,10cyclictest9389-21ssh10:22:083
2769799140,10cyclictest805-21runrttasks12:24:093
2769799140,10cyclictest805-21runrttasks08:56:063
2769799140,10cyclictest805-21runrttasks08:40:073
2769799140,10cyclictest738-21snmpd11:16:403
2769799140,10cyclictest738-21snmpd07:49:223
2769799140,10cyclictest5632-21ssh10:20:003
2769799140,10cyclictest5002-21chrt07:26:163
2769799140,10cyclictest4614-21cat09:50:243
2769799140,10cyclictest3606-21ssh12:30:143
2769799140,10cyclictest32412-21tr10:40:203
2769799140,10cyclictest32077-21rm11:32:393
2769799140,10cyclictest31735-21runrttasks11:05:463
2769799140,10cyclictest31428-21sh12:25:313
2769799140,10cyclictest31265-21sh09:17:583
2769799140,10cyclictest28100-21ssh11:56:073
2769799140,10cyclictest27563-21runrttasks09:42:093
2769799140,10cyclictest25365-21ssh11:00:323
2769799140,10cyclictest2478-21cut11:35:133
2769799140,10cyclictest24362-21grep08:06:223
2769799140,10cyclictest22681-21cut11:25:203
2769799140,10cyclictest22437-21sh11:51:333
2769799140,10cyclictest21744-21ssh09:36:423
2769799140,10cyclictest21120-21/usr/sbin/munin09:10:153
2769799140,10cyclictest1960-21sh12:02:243
2769799140,10cyclictest18244-21rm09:34:463
2769799140,10cyclictest17775-21sh11:20:373
2769799140,10cyclictest17251-21ssh12:15:103
2769799140,10cyclictest14199-21sh10:51:213
2769799140,10cyclictest13530-21chrt08:54:513
2769799140,10cyclictest1320-21/usr/sbin/munin07:20:173
2769799140,10cyclictest1236-21ssh09:48:043
2769799140,10cyclictest12111-21sh11:42:363
2769799140,10cyclictest11770-21sh12:35:513
2769799140,10cyclictest10858-21ssh12:10:153
2769799140,10cyclictest10139-21ssh09:27:003
2769699145,7cyclictest2639-21munin-plugin-st07:25:002
2769699141,9cyclictest26476-21unixbench_multi10:35:262
2769699141,10cyclictest32000-21/usr/sbin/munin12:00:182
2769699141,10cyclictest26102-21ssh10:09:062
2769699141,10cyclictest10116-21sed08:45:192
2769699140,9cyclictest32582-21ssh09:19:342
2769699140,10cyclictest9741-21apache207:52:442
2769699140,10cyclictest8246-21/usr/sbin/munin07:35:182
2769699140,10cyclictest805-21runrttasks11:58:302
2769699140,10cyclictest805-21runrttasks11:20:572
2769699140,10cyclictest805-21runrttasks08:40:092
2769699140,10cyclictest7653-21hddtemp_smartct09:25:152
2769699140,10cyclictest7227-21sh12:33:002
2769699140,10cyclictest6153-21aten2_r5power_c10:20:122
2769699140,10cyclictest5579-21chrt07:30:032
2769699140,10cyclictest5090-21chrt07:27:272
2769699140,10cyclictest5044-21runrttasks10:45:092
2769699140,10cyclictest32446-21forks08:25:152
2769699140,10cyclictest31242-21runrttasks10:40:082
2769699140,10cyclictest3122-21/usr/sbin/munin09:50:162
2769699140,10cyclictest30618-21/usr/sbin/munin07:15:182
2769699140,10cyclictest2992-21runrttasks10:16:012
2769699140,10cyclictest29733-21sh09:45:022
2769699140,10cyclictest28377-21ntpq10:10:212
2769699140,10cyclictest27054-21idleruntime-cro08:15:002
2769699140,10cyclictest25683-21sh12:21:242
2769699140,10cyclictest25603-21ssh11:00:482
2769699140,10cyclictest25584-21ssh09:13:242
2769699140,10cyclictest2302-21aten2_r5power_v11:35:122
2769699140,10cyclictest21842-21sh09:36:492
2769699140,10cyclictest20367-21wget08:00:162
2769699140,10cyclictest17902-21/usr/sbin/munin07:55:152
2769699140,10cyclictest17900-21sh11:47:262
2769699140,10cyclictest17185-21sh12:15:062
2769699140,10cyclictest16507-21cpuspeed_turbos09:00:142
2769699140,10cyclictest1591-21fschecks_time09:20:152
2769699140,10cyclictest15862-21/usr/sbin/munin10:00:182
2769699140,10cyclictest15500-21date07:50:002
2769699140,10cyclictest15319-21ssh09:30:522
2769699140,10cyclictest13100-21unixbench_singl08:50:242
2769699140,10cyclictest12219-21rm12:36:232
2769699140,10cyclictest10736-21sh11:40:512
2769699140,10cyclictest1045-21ssh11:07:502
2769599142,10cyclictest29679-21grep07:10:391
2769599141,9cyclictest31542-21runrttasks08:22:001
2769599141,9cyclictest11577-21sh09:28:481
27695991410,3cyclictest24015-21idleruntime-cro09:40:001
2769599140,10cyclictest977-21/usr/sbin/munin09:20:241
2769599140,10cyclictest9625-21/usr/sbin/munin09:55:211
2769599140,10cyclictest9106-21tune2fs11:40:161
2769599140,10cyclictest8637-21ssh10:21:131
2769599140,10cyclictest8479-21ssh11:14:121
2769599140,10cyclictest805-21runrttasks07:21:371
2769599140,10cyclictest8007-21ssh11:39:561
2769599140,10cyclictest7113-21ssh10:45:421
2769599140,10cyclictest5043-21sh12:30:291
2769599140,10cyclictest32615-21ssh10:40:221
2769599140,10cyclictest32280-21sh09:46:161
2769599140,10cyclictest3165-21cmp07:25:031
2769599140,10cyclictest30744-21ntpq12:25:221
2769599140,10cyclictest3056-21ssh10:16:061
2769599140,10cyclictest27578-21rm11:03:031
2769599140,10cyclictest26703-21sh10:35:391
2769599140,10cyclictest25247-21runrttasks10:08:101
2769599140,10cyclictest25134-21df_inode09:40:161
2769599140,10cyclictest23722-21ssh09:11:201
2769599140,10cyclictest22546-21sh10:59:331
2769599140,10cyclictest2085-21wget08:30:181
2769599140,10cyclictest19842-21grep11:50:121
2769599140,10cyclictest18847-21/usr/sbin/munin10:30:201
2769599140,10cyclictest18538-21sh11:21:261
2769599140,10cyclictest15499-21memory11:45:211
2769599140,10cyclictest12793-21ssh11:17:201
2769599140,10cyclictest10746-21/usr/sbin/munin12:10:201
2769499146,6cyclictest16724-21chrt11:45:560
2769499143,8cyclictest754-21lldpd07:16:250
2769499142,9cyclictest6791-21sh11:12:200
2769499141,9cyclictest15694-21runrttasks08:57:200
2769499141,9cyclictest13911-21idleruntime07:45:180
2769499141,7cyclictest18275-21latency_hist09:04:590
2769499141,10cyclictest24508-21runrttasks08:07:470
2769499140,8cyclictest146-1kworker/0:1H+kblockd09:40:320
2769499140,7cyclictest8922-21ssh09:25:370
2769499140,7cyclictest15691-21sh11:20:080
2769499140,7cyclictest12526-21tr10:25:130
2769499140,12cyclictest4819-21sh09:22:590
2769499140,12cyclictest13490-21ssh11:44:230
2769499140,10cyclictest805-21runrttasks12:15:030
2769499140,10cyclictest805-21runrttasks08:11:270
2769499140,10cyclictest5138-21chrt07:28:060
2769499140,10cyclictest31582-21sh09:18:200
2769499140,10cyclictest31397-21chrt08:20:410
2769499140,10cyclictest3130-21ssh12:03:400
2769499140,10cyclictest28539-21mailstats08:15:200
2769499140,10cyclictest24478-21sh09:12:090
2769499140,10cyclictest22270-21ssh09:37:430
2769499140,10cyclictest21870-21ssh10:32:210
2769499140,10cyclictest14790-21ssh12:12:510
2769499140,10cyclictest14258-21/usr/sbin/munin09:30:200
2769499140,10cyclictest10291-21hddtemp_smartct12:35:170
2769499140,10cyclictest10159-21fschecks_time11:15:190
2189029140,6tAtEmLog_026925-21taskset11:02:185
1925099140,11tDemoTimingTask21956-21chrt11:50:565
2769899131,9cyclictest15869-21apt-get10:00:134
2769899130,9cyclictest8529-21/usr/sbin/munin11:40:164
2769899130,9cyclictest8494-21sh12:08:074
2769899130,9cyclictest8494-21sh12:08:074
2769899130,9cyclictest829-21runintdemo5min07:41:034
2769899130,9cyclictest805-21runrttasks12:25:254
2769899130,9cyclictest805-21runrttasks09:35:064
2769899130,9cyclictest805-21runrttasks08:20:024
2769899130,9cyclictest805-21runrttasks07:25:054
2769899130,9cyclictest738-21snmpd09:04:314
2769899130,9cyclictest738-21snmpd08:14:294
2769899130,9cyclictest6736-21runrttasks08:40:064
2769899130,9cyclictest666-21ssh10:15:084
2769899130,9cyclictest5591-21/usr/sbin/munin07:30:264
2769899130,9cyclictest3426-21/usr/sbin/munin11:10:124
2769899130,9cyclictest31185-21ssh10:40:034
2769899130,9cyclictest30315-21gpgv07:15:064
2769899130,9cyclictest29890-21aten2_r5power_e09:45:124
2769899130,9cyclictest28287-21sh11:30:074
2769899130,9cyclictest27740-21fschecks_count08:15:154
2769899130,9cyclictest27230-21/usr/sbin/munin10:10:224
2769899130,9cyclictest26589-21idleruntime11:55:184
2769899130,9cyclictest23005-21/usr/sbin/munin12:20:194
2769899130,9cyclictest21527-21ssh09:10:164
2769899130,9cyclictest17468-21/usr/sbin/munin10:55:174
2769899130,9cyclictest17300-21ssh09:33:284
2769899130,9cyclictest13811-21rm12:11:494
2769899130,9cyclictest1034-21aten2_r5power_e09:20:124
2769899130,9cyclictest10147-21taskset07:37:244
2769899130,8cyclictest16-1pr/legacy08:31:424
2769899130,10cyclictest5492-21fschecks_count10:45:164
2769899130,10cyclictest535-21/usr/sbin/munin07:20:164
2769899130,10cyclictest3802-21sed09:50:154
2769899130,10cyclictest29739-21sshd07:11:244
2769899130,10cyclictest2526-21cpuspeed_turbos11:35:134
2769899130,10cyclictest22335-21runrttasks11:51:254
2769899130,10cyclictest17262-21ssh12:15:114
2769899130,10cyclictest14922-21/usr/sbin/munin11:45:164
2769899130,10cyclictest14828-21unixbench_singl07:45:254
2769899130,10cyclictest12872-21ssh10:25:144
2769899130,10cyclictest11772-21/usr/sbin/munin10:50:174
2769799131,9cyclictest18565-21runrttasks09:05:023
2769799130,9cyclictest8338-21aten2_r5power_p07:35:163
2769799130,9cyclictest4885-21sh12:05:143
2769799130,9cyclictest4885-21sh12:05:143
2769799130,9cyclictest30618-21/usr/sbin/munin07:15:233
2769799130,9cyclictest30562-21latency08:20:193
2769799130,9cyclictest27775-21latency_hist07:09:593
2769799130,9cyclictest24982-21/usr/sbin/munin08:10:163
2769799130,9cyclictest2144-21cpu08:30:193
2769799130,9cyclictest20279-21/usr/sbin/munin08:00:223
2769799130,9cyclictest16224-21/usr/sbin/munin09:00:123
2769799130,7cyclictest14630-21aten2_r5power_v11:45:143
2769799130,10cyclictest28361-21tune2fs07:10:153
2769799130,10cyclictest10803-21vmstat08:45:263
2769699131,9cyclictest3377-21ssh11:10:082
2769699130,9cyclictest5607-21mailstats08:35:202
2769699130,9cyclictest4807-21ssh12:05:092
2769699130,9cyclictest4807-21ssh12:05:092
2769699130,9cyclictest29724-21/usr/sbin/munin08:20:152
2769699130,9cyclictest28208-21sh11:30:022
2769699130,9cyclictest27514-21cpuspeed08:15:132
2769699130,9cyclictest11024-21/usr/sbin/munin07:40:222
2769699130,10cyclictest805-21runrttasks12:25:502
2769699130,10cyclictest805-21runrttasks10:50:142
2769699130,10cyclictest29798-21chrt07:12:122
2769699130,10cyclictest20340-21perl11:50:142
2769699130,10cyclictest17468-21/usr/sbin/munin10:55:132
2769699130,10cyclictest15755-21runrttasks08:58:102
2769599131,9cyclictest12118-21sh10:25:051
2769599130,9cyclictest9174-21aten2_r5power_p08:45:131
2769599130,9cyclictest8246-21/usr/sbin/munin07:35:221
2769599130,9cyclictest805-21runrttasks09:50:161
2769599130,9cyclictest571-21needreboot08:25:181
2769599130,9cyclictest30604-21seq09:17:151
2769599130,9cyclictest29025-21runrttasks08:15:451
2769599130,9cyclictest27878-21taskset11:55:471
2769599130,9cyclictest25317-21cstates08:10:131
2769599130,9cyclictest20431-21taskset09:07:281
2769599130,9cyclictest20430-21cpu08:00:161
2769599130,9cyclictest17912-21runrttasks09:00:521
2769599130,9cyclictest17902-21/usr/sbin/munin07:55:211
2769599130,9cyclictest16682-21ssh10:00:181
2769599130,9cyclictest16622-21ntp_offset07:50:191
2769599130,9cyclictest13202-21/usr/sbin/munin07:45:151
2769599130,9cyclictest12943-21ssh09:30:031
2769599130,9cyclictest11502-21/usr/sbin/munin08:50:171
2769599130,9cyclictest10850-21/usr/sbin/munin07:40:281
2769599130,10cyclictest5965-21/usr/sbin/munin12:05:231
2769599130,10cyclictest5965-21/usr/sbin/munin12:05:231
2769599130,10cyclictest5081-21if_enp5s008:35:161
2769599130,10cyclictest32484-21chrt07:17:351
2769599130,10cyclictest22878-21sh12:20:031
2769599130,10cyclictest15030-21awk08:55:271
2769499135,6cyclictest5410-21ntp_kernel_err08:35:200
2769499131,9cyclictest29534-21sh12:25:090
2769499131,10cyclictest23011-21apt-get12:20:120
2769499130,9cyclictest8601-21runrttasks08:42:310
2769499130,9cyclictest829-21runintdemo5min07:51:110
2769499130,9cyclictest8146-21find07:35:010
2769499130,9cyclictest31080-21sendmail_mailst09:45:200
2769499130,9cyclictest2005-21chrt08:30:020
2769499130,9cyclictest19629-21ntpq09:05:180
2769499130,9cyclictest18459-21fschecks_time07:55:160
2769499130,7cyclictest12317-21sed08:50:180
2769499130,10cyclictest6124-21fschecks_time07:30:150
2769499130,10cyclictest28224-21cut07:10:140
2769499130,10cyclictest1943-21awk07:20:250
2769499130,10cyclictest1597-21sed10:15:220
2728099130,11tDemoTimingTask27330-21chrt07:05:495
1351799130,10tDemoTimingTask15921-21taskset11:45:255
446129120,10tAtEmLog_05248-21taskset07:29:375
3148129120,9tAtEmLog_0316642sleep508:23:435
303239120,0EcMasterDemoSyn0-21swapper/512:30:035
2769899120,9cyclictest9450-21df_inode08:45:144
2769899120,9cyclictest4370-21/usr/sbin/munin08:35:214
2769899120,9cyclictest32277-21cpuspeed_turbos08:25:134
2769599120,8cyclictest6839-21aten2_r5power_v08:40:121
2769499120,7cyclictest11006-21chrt08:47:280
2239039120,5EcMasterDemoSyn25367-21taskset12:21:015
1953529120,2tAtEmLog_0200262sleep510:03:435
1591039120,0EcMasterDemoSyn0-21swapper/510:53:425
1255839120,0EcMasterDemoSyn0-21swapper/507:41:015
984139110,0EcMasterDemoSyn0-21swapper/510:48:385
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional