You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 08:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot1.osadl.org (updated Fri Nov 28, 2025 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29494391100,0EcMasterDemoSyn29499-21runintdemo5min22:02:350
2696039970,1EcMasterDemoSyn301rcuc/123:49:031
54322920,0sleep00-21swapper/021:46:470
2397639830,0EcMasterDemoSyn2398629tAtEmLog_023:03:255
2330439830,0EcMasterDemoSyn2331429tAtEmLog_019:15:145
190562830,0sleep00-21swapper/021:35:090
1686439830,0EcMasterDemoSyn1687429tAtEmLog_021:11:515
651239820,0EcMasterDemoSyn652229tAtEmLog_019:45:395
651239820,0EcMasterDemoSyn652229tAtEmLog_019:45:395
2342439820,0EcMasterDemoSyn2343429tAtEmLog_020:21:095
2196339820,0EcMasterDemoSyn2197329tAtEmLog_000:34:415
1022339820,0EcMasterDemoSyn1023329tAtEmLog_021:01:445
2305439810,0EcMasterDemoSyn2306429tAtEmLog_021:37:125
20464993229,2cyclictest9150-21kworker/1:2+events23:43:581
1530739260,1EcMasterDemoSyn15313-21ps21:32:083
2579029180,7tAtEmLog_025842-21chrt20:26:315
216502180,0sleep30-21swapper/322:18:383
20474991814,3cyclictest32312-21sh22:05:003
2045999180,14cyclictest8795-21sh23:15:020
2045999180,14cyclictest8795-21sh23:15:010
476329170,14tAtEmLog_05928-21taskset22:08:385
3183499170,10tDemoTimingTask31838-21taskset19:30:275
3050929170,7tAtEmLog_030735-21chrt20:39:045
2331429170,10tAtEmLog_025439-21chrt19:20:015
2046899179,6cyclictest21422-21sed19:10:162
2045999170,9cyclictest805-21runrttasks23:51:390
2045999170,9cyclictest21687-21sh00:34:190
2045999170,9cyclictest1855-21taskset19:36:050
2045999170,7cyclictest10724-21sh22:10:540
2045999170,14cyclictest13125-21ssh21:51:460
527799160,9tDemoTimingTask6472-21chrt23:34:565
2105499160,9tDemoTimingTask21058-21taskset20:16:055
2047899161,9cyclictest16-1pr/legacy20:05:554
2047499169,3cyclictest1286839EcMasterDemoSyn22:12:423
2047499165,8cyclictest1429-21sed19:35:253
2047499161,13cyclictest31195-21grep00:15:203
2047499161,11cyclictest4575-21ssh23:11:393
2047499160,9cyclictest18901-21sh23:00:093
2047499160,10cyclictest8981-21ssh23:15:113
2047499160,10cyclictest8981-21ssh23:15:113
2046499160,9cyclictest1719-21proc_pri22:05:221
2045999168,6cyclictest31032-21cat20:40:000
20459991613,2cyclictest0-21swapper/021:20:000
2045999160,9cyclictest805-21runrttasks19:58:390
2045999160,9cyclictest805-21runrttasks19:20:490
1649999160,9tDemoTimingTask18332-21taskset22:59:555
1530739160,7EcMasterDemoSyn18519-21taskset21:34:555
887899150,9tDemoTimingTask9220-21taskset19:54:555
2172099150,9tDemoTimingTask16931-21sleep21:59:555
2047899154,8cyclictest781-21postgres23:40:574
20478991511,3cyclictest26105-21cron20:30:004
20478991511,3cyclictest11152-21cpuspeed_turbos00:25:164
2047899151,10cyclictest27811-21/usr/sbin/munin19:25:124
2047899150,9cyclictest24916-21taskset23:25:164
2047899150,12cyclictest16-1pr/legacy21:27:044
2047499153,9cyclictest781-21postgres19:54:493
2047499153,9cyclictest10996-21processes19:55:193
2047499151,11cyclictest11649-21ssh22:33:193
2047499151,10cyclictest4745-21sh22:28:493
2047499151,10cyclictest25400-21ssh22:42:093
2047499151,10cyclictest17504-21runrttasks22:59:133
2047499151,10cyclictest16262-21ssh22:35:433
2047499150,9cyclictest20252-21sh21:56:153
2047499150,11cyclictest19165-21/usr/sbin/munin21:35:213
2047499150,10cyclictest31964-21tune2fs20:40:153
2047499150,10cyclictest22246-21sh23:45:193
2046899153,8cyclictest281-21jbd2/md0-822:05:022
2046899152,9cyclictest14736-21cstates23:40:142
2046899151,10cyclictest30650-21runrttasks20:37:522
2046899151,10cyclictest26718-21diskmemload21:18:552
2046899151,10cyclictest22105-21ssh23:23:402
2046899150,10cyclictest26677-21fschecks_time23:05:152
2046499151,10cyclictest9720-21/usr/sbin/munin21:50:151
2046499151,10cyclictest30887-21runrttasks23:50:591
2046499151,10cyclictest30155-21ssh23:28:551
2046499151,10cyclictest26836-21ssh21:19:021
2046499151,10cyclictest20943-21sh22:39:531
2046499151,10cyclictest13972-21ssh00:27:071
2046499150,9cyclictest805-21runrttasks22:04:171
2046499150,8cyclictest15008-21ssh00:03:431
2046499150,11cyclictest16-1pr/legacy20:26:121
2045999156,7cyclictest9972-21sh23:36:220
2045999152,9cyclictest31301-21sh23:08:190
2045999152,9cyclictest28227-21latency_hist23:50:000
20459991511,3cyclictest717-21in:imuxsock23:35:000
2045999150,6cyclictest5631-21timerandwakeup22:50:250
2045999150,13cyclictest17889-21sh22:37:090
2045999150,11cyclictest21766-21rm00:08:370
2045999150,11cyclictest15070-21sh21:31:570
2047899144,7cyclictest313-21latency23:30:164
2047899141,9cyclictest4365-21sh22:28:304
2047899141,9cyclictest3445-21sh21:45:214
2047899141,9cyclictest2256-21diskmemload00:18:204
2047899141,10cyclictest23091-21ssh21:58:414
20478991410,3cyclictest6617-21fschecks_time20:55:154
20478991410,3cyclictest22932-21cpuspeed_turbos00:35:134
20478991410,3cyclictest10703-21sh22:55:004
2047899140,8cyclictest20733-21sh23:22:284
2047899140,12cyclictest29641-21missed_timers20:35:184
2047899140,10cyclictest9027-21ssh22:10:164
2047899140,10cyclictest7910-21sh23:14:314
2047899140,10cyclictest5161-21/usr/sbin/munin19:45:124
2047899140,10cyclictest5161-21/usr/sbin/munin19:45:124
2047899140,10cyclictest334-21systemd-journal21:42:494
2047899140,10cyclictest29873-21sh23:07:034
2047899140,10cyclictest28476-21sh22:01:414
2047899140,10cyclictest26860-21ssh22:21:324
2047899140,10cyclictest23667-21grep19:15:154
2047899140,10cyclictest22259-21ssh23:01:484
2047899140,10cyclictest16886-21rm00:30:084
2047899140,10cyclictest15865-21taskset21:54:064
2047499145,7cyclictest22239-21perl20:20:173
2047499144,8cyclictest22135-21cut21:15:193
2047499141,9cyclictest25237-21timerandwakeup00:10:273
2047499141,10cyclictest281-21jbd2/md0-820:09:353
2047499140,9cyclictest30322-21cpu19:30:123
2047499140,12cyclictest1915-21rm22:05:263
2047499140,10cyclictest9720-21/usr/sbin/munin21:50:223
2047499140,10cyclictest8465-21/usr/sbin/munin21:00:253
2047499140,10cyclictest805-21runrttasks22:21:273
2047499140,10cyclictest7712-21diskmemload21:48:433
2047499140,10cyclictest5592-21sh23:56:103
2047499140,10cyclictest4897-21fschecks_time00:20:173
2047499140,10cyclictest4026-21/usr/sbin/munin22:50:173
2047499140,10cyclictest400-21ssh21:44:023
2047499140,10cyclictest334-21systemd-journal19:17:003
2047499140,10cyclictest30214-21sed21:20:253
2047499140,10cyclictest29108-21/usr/sbin/munin22:45:203
2047499140,10cyclictest28756-21runrttasks20:35:053
2047499140,10cyclictest28275-21taskset20:32:363
2047499140,10cyclictest27805-21/usr/sbin/munin19:25:143
2047499140,10cyclictest26034-21sh23:05:043
2047499140,10cyclictest24149-21tr20:25:143
2047499140,10cyclictest23976-21ssh23:25:023
2047499140,10cyclictest22689-21/usr/sbin/munin00:35:143
2047499140,10cyclictest21371-21taskset20:19:583
2047499140,10cyclictest19506-21ssh00:06:293
2047499140,10cyclictest17141-21cpuspeed20:10:133
2047499140,10cyclictest16957-21/usr/sbin/munin00:30:193
2047499140,10cyclictest14360-21/usr/sbin/munin23:40:253
2047499140,10cyclictest13699-21ssh23:39:523
2047499140,10cyclictest13638-21ssh00:26:423
2046899146,6cyclictest11100-21tr21:05:152
2046899145,7cyclictest17476-21aten2_r5power_c21:55:132
2046899141,9cyclictest29996-21sh21:41:342
2046899141,9cyclictest24967-21ssh21:38:532
2046899141,9cyclictest24517-21ssh00:35:282
2046899141,9cyclictest19867-21ssh00:06:512
2046899141,9cyclictest17754-21kernelversion20:10:182
2046899141,9cyclictest17191-21ssh21:12:082
2046899141,9cyclictest12908-21ssh23:39:032
2046899141,9cyclictest11911-21runrttasks19:59:432
2046899141,7cyclictest30510-21/usr/sbin/munin19:30:132
2046899140,8cyclictest31286-21ssh00:15:212
2046899140,8cyclictest12667-21ssh00:01:102
2046899140,12cyclictest4339-21sensors_fan23:55:212
2046899140,10cyclictest9720-21/usr/sbin/munin21:50:162
2046899140,10cyclictest9678-21sensors_temp22:10:202
2046899140,10cyclictest805-21runrttasks22:00:022
2046899140,10cyclictest805-21runrttasks20:52:562
2046899140,10cyclictest805-21runrttasks20:04:422
2046899140,10cyclictest7484-21ssh21:48:312
2046899140,10cyclictest738-21snmpd20:18:012
2046899140,10cyclictest7184-21/usr/sbin/munin19:50:202
2046899140,10cyclictest5579-21irqstats19:45:152
2046899140,10cyclictest5579-21irqstats19:45:152
2046899140,10cyclictest4704-21sh23:11:462
2046899140,10cyclictest4565-21grep22:50:152
2046899140,10cyclictest3706-21ssh21:25:122
2046899140,10cyclictest3572-21taskset22:27:502
2046899140,10cyclictest32513-21/usr/sbin/munin19:35:202
2046899140,10cyclictest30040-21threads21:20:242
2046899140,10cyclictest29474-21cat23:50:202
2046899140,10cyclictest27805-21/usr/sbin/munin19:25:202
2046899140,10cyclictest2697-21sh23:31:342
2046899140,10cyclictest25850-21sh22:20:402
2046899140,10cyclictest24888-21head23:25:162
2046899140,10cyclictest2475-21/usr/sbin/munin19:40:192
2046899140,10cyclictest2475-21/usr/sbin/munin19:40:192
2046899140,10cyclictest24173-21sh22:41:042
2046899140,10cyclictest23608-21ssh00:10:072
2046899140,10cyclictest23064-21runrttasks19:15:012
2046899140,10cyclictest21178-21ssh23:00:472
2046899140,10cyclictest19111-21runrttasks00:30:592
2046899140,10cyclictest16559-21runrttasks20:08:382
2046899140,10cyclictest16132-21aten2_r5power_p22:15:132
2046899140,10cyclictest14378-21ssh21:31:212
2046899140,10cyclictest14064-21ssh22:35:102
2046899140,10cyclictest12586-21ssh23:16:552
2046899140,10cyclictest12586-21ssh23:16:542
2046899140,10cyclictest12043-21awk00:25:212
2046899140,10cyclictest10304-21chrt21:02:172
2046499144,6cyclictest1581-21grep23:10:141
2046499144,6cyclictest11620-21cut22:55:141
2046499143,8cyclictest738-21snmpd22:17:071
2046499143,8cyclictest18236-21swap20:10:241
2046499142,9cyclictest19273-21ssh00:06:171
2046499141,9cyclictest4639-21tr00:20:161
2046499141,9cyclictest27506-21latency_hist19:25:001
2046499141,9cyclictest26286-21sh22:42:561
2046499141,9cyclictest15792-21postgres21:10:511
2046499141,9cyclictest12266-21/usr/sbin/munin20:00:191
20464991410,3cyclictest27018-21cpuspeed_turbos21:40:131
2046499140,9cyclictest2478-21apt19:40:111
2046499140,9cyclictest2478-21apt19:40:111
2046499140,10cyclictest9091-21grep21:00:201
2046499140,10cyclictest8734-21unixbench_multi19:50:271
2046499140,10cyclictest8558-21ssh22:52:581
2046499140,10cyclictest8406-21aten2_r5power_p22:10:121
2046499140,10cyclictest805-21runrttasks20:42:391
2046499140,10cyclictest805-21runrttasks19:46:481
2046499140,10cyclictest805-21runrttasks19:46:481
2046499140,10cyclictest805-21runrttasks00:17:041
2046499140,10cyclictest6014-21ssh21:47:181
2046499140,10cyclictest5741-21ssh21:25:331
2046499140,10cyclictest32237-21cut23:30:131
2046499140,10cyclictest31108-21cat20:40:001
2046499140,10cyclictest28185-21chrt20:31:221
2046499140,10cyclictest28175-21rm23:05:351
2046499140,10cyclictest26739-21ssh00:11:491
2046499140,10cyclictest25895-21sh00:37:101
2046499140,10cyclictest25179-21ssh23:04:321
2046499140,10cyclictest24431-21sed19:15:241
2046499140,10cyclictest23687-21runrttasks22:20:101
2046499140,10cyclictest20768-21ssh21:56:411
2046499140,10cyclictest2054-21hddtemp_smartct20:45:161
2046499140,10cyclictest18537-21sh23:20:251
2046499140,10cyclictest1795-21ssh22:26:181
2046499140,10cyclictest11771-21chrt23:37:581
2045999146,6cyclictest805-21runrttasks19:32:090
2045999146,6cyclictest21290-21ssh23:00:530
2045999146,6cyclictest17960-21head20:10:180
2045999146,6cyclictest16832-21ps21:11:500
2045999145,7cyclictest10376-21seq21:03:150
2045999145,3cyclictest5564-21unixbench_singl21:25:280
2045999143,8cyclictest10438-21wc00:00:160
2045999142,9cyclictest18916-21switchtime21:55:240
2045999142,8cyclictest28549-21interrupts19:25:170
2045999141,9cyclictest26732-21sh00:11:470
2045999141,9cyclictest25770-21diskmemload00:37:020
2045999141,8cyclictest5505-21sh22:08:160
2045999141,7cyclictest21387-21latency_hist20:20:010
2045999141,10cyclictest6719-21runrttasks19:47:560
2045999141,10cyclictest6719-21runrttasks19:47:550
2045999141,10cyclictest32570-21sh22:46:530
2045999141,10cyclictest31891-21sh21:21:470
2045999140,7cyclictest3191-21cpuspeed_turbos23:55:140
2045999140,7cyclictest11882-21tr22:55:140
2045999140,4cyclictest146-1kworker/0:1H+kblockd22:25:540
2045999140,12cyclictest30470-21cstates00:15:150
2045999140,12cyclictest28003-21sh22:22:320
2045999140,12cyclictest24453-21ssh22:41:200
2045999140,12cyclictest17469-21sh22:15:220
2045999140,12cyclictest14779-21cat23:40:160
2045999140,11cyclictest805-21runrttasks20:53:530
2045999140,10cyclictest9081-21ssh22:31:040
2045999140,10cyclictest805-21runrttasks20:09:570
2045999140,10cyclictest7332-21sensors20:55:200
2045999140,10cyclictest4229-21ssh00:20:080
2045999140,10cyclictest27454-21ntp_kernel_err20:30:190
2045999140,10cyclictest27154-21df21:40:130
2045999140,10cyclictest26138-21ssh23:25:300
2045999140,10cyclictest22569-21sh23:24:020
2045999140,10cyclictest13998-21ssh00:27:100
2045999140,10cyclictest1281-21chrt23:10:100
2950429130,3tAtEmLog_029821-21chrt22:02:495
2697029130,10tAtEmLog_0673-21taskset23:53:175
2215339130,0EcMasterDemoSyn0-21swapper/523:23:425
2047899132,8cyclictest21809-21cpuspeed22:40:134
2047899132,8cyclictest21799-21load21:15:174
2047899131,8cyclictest26410-21/usr/sbin/munin20:30:254
2047899131,8cyclictest15828-21ssh22:15:004
2047899130,9cyclictest9834-21rm00:00:014
2047899130,9cyclictest9210-21cpuspeed23:15:134
2047899130,9cyclictest9210-21cpuspeed23:15:134
2047899130,9cyclictest847-21sh22:05:184
2047899130,9cyclictest762-21sshd19:58:164
2047899130,9cyclictest738-21snmpd19:50:154
2047899130,9cyclictest6902-21ssh23:35:034
2047899130,9cyclictest6515-21ssh22:30:094
2047899130,9cyclictest3656-21/usr/sbin/munin19:40:194
2047899130,9cyclictest3656-21/usr/sbin/munin19:40:194
2047899130,9cyclictest31439-21/usr/sbin/munin20:40:174
2047899130,9cyclictest30156-21/usr/sbin/munin19:30:194
2047899130,9cyclictest29874-21ssh23:50:244
2047899130,9cyclictest29023-21sh22:45:094
2047899130,9cyclictest28331-21ssh21:20:054
2047899130,9cyclictest25457-21/usr/sbin/munin19:20:164
2047899130,9cyclictest2349-21latency_hist19:40:004
2047899130,9cyclictest22854-21ssh23:45:234
2047899130,9cyclictest21989-21sh21:36:174
2047899130,9cyclictest21849-21grep19:10:204
2047899130,9cyclictest21709-21/usr/sbin/munin20:20:154
2047899130,9cyclictest19321-21/usr/sbin/munin20:15:254
2047899130,9cyclictest17187-21/usr/sbin/munin20:10:144
2047899130,9cyclictest16621-21/usr/sbin/munin00:05:184
2047899130,9cyclictest1644-21cut20:45:134
2047899130,9cyclictest15999-21diskmemload22:35:274
2047899130,9cyclictest13444-21/usr/sbin/munin21:10:184
2047899130,9cyclictest12870-21processes22:55:214
2047899130,9cyclictest12266-21/usr/sbin/munin20:00:144
2047899130,9cyclictest11405-21/usr/sbin/munin21:30:144
2047899130,9cyclictest11108-21cut21:05:154
2047899130,9cyclictest10068-21/usr/sbin/munin00:00:234
2047899130,10cyclictest8732-21expr21:00:184
2047899130,10cyclictest805-21runrttasks20:53:324
2047899130,10cyclictest6023-21grep00:20:264
2047899130,10cyclictest26437-21ssh00:11:304
2047499131,9cyclictest27294-21taskset19:22:173
2047499130,9cyclictest829-21runintdemo5min19:10:113
2047499130,9cyclictest805-21runrttasks21:08:133
2047499130,9cyclictest738-21snmpd20:45:373
2047499130,9cyclictest6822-21taskset20:55:173
2047499130,9cyclictest3737-21apt20:50:143
2047499130,9cyclictest2423-21sh23:31:213
2047499130,9cyclictest17781-21sh23:20:203
2047499130,9cyclictest13373-21ssh21:10:083
2047499130,9cyclictest12260-21taskset20:00:063
2047499130,10cyclictest4928-21aten2_r5power_v19:45:113
2047499130,10cyclictest4928-21aten2_r5power_v19:45:113
2047499130,10cyclictest4549-21basename21:25:203
2047499130,10cyclictest28393-21ssh23:50:113
2047499130,10cyclictest10112-21grep00:00:163
2046899130,9cyclictest805-21runrttasks20:30:342
2046899130,9cyclictest805-21runrttasks00:20:002
2046899130,9cyclictest7556-21timerwakeupswit20:55:252
2046899130,9cyclictest30513-21sh22:45:212
2046899130,9cyclictest2601-21sendmail_mailtr20:45:222
2046899130,9cyclictest25629-21cpuspeed19:20:162
2046899130,9cyclictest24854-21grep20:25:182
2046899130,9cyclictest21709-21/usr/sbin/munin20:20:152
2046899130,9cyclictest12207-21/usr/sbin/munin22:55:172
2046899130,10cyclictest6412-21ssh22:30:042
2046899130,10cyclictest31330-21apt-config20:40:002
2046899130,10cyclictest23200-21unixbench_multi23:45:272
2046499134,6cyclictest11596-21if_enp5s021:05:191
2046499130,9cyclictest8516-21ssh22:30:311
2046499130,9cyclictest6095-21atemtest20:55:121
2046499130,9cyclictest3749-21aten2_r5power_c20:50:141
2046499130,9cyclictest32498-21runrttasks19:35:011
2046499130,9cyclictest30156-21/usr/sbin/munin19:30:171
2046499130,9cyclictest29670-21runrttasks19:27:371
2046499130,9cyclictest29459-21cut22:45:141
2046499130,9cyclictest2807-21sh23:55:021
2046499130,9cyclictest21713-21ssh21:36:021
2046499130,9cyclictest21695-21taskset20:20:021
2046499130,9cyclictest20987-21cpuspeed_turbos19:10:141
2046499130,9cyclictest19321-21/usr/sbin/munin20:15:251
2046499130,9cyclictest16777-21sh00:30:011
2046499130,9cyclictest16372-21taskset20:06:051
2046499130,9cyclictest12189-21gltestperf21:30:161
2046499130,10cyclictest8862-21rm23:15:061
2046499130,10cyclictest8862-21rm23:15:051
2046499130,10cyclictest32562-21ssh21:22:201
2046499130,10cyclictest11012-21sendmail_mailqu19:55:201
2045999135,7cyclictest25308-21/usr/sbin/munin20:25:230
2045999134,8cyclictest32480-21/usr/sbin/munin20:40:190
2045999131,7cyclictest7861-21/usr/sbin/munin19:50:190
2045999130,9cyclictest11922-21taskset21:05:210
2045999130,7cyclictest20642-21cat19:10:000
2045999130,7cyclictest146-1kworker/0:1H+kblockd20:45:260
2045999130,7cyclictest146-1kworker/0:1H+kblockd19:40:140
2045999130,7cyclictest146-1kworker/0:1H+kblockd19:40:140
2045999130,4cyclictest16-1pr/legacy19:15:130
2045999130,11cyclictest805-21runrttasks20:02:340
2045999130,10cyclictest738-21snmpd19:12:290
1986999134,7tDemoTimingTask23647-21taskset23:45:465
1837299131,10tDemoTimingTask18896-21chrt19:05:165
327739120,0EcMasterDemoSyn0-21swapper/500:19:295
3215999120,10tDemoTimingTask3702-21chrt21:25:125
3214839120,0EcMasterDemoSyn0-21swapper/521:22:005
313899120,10tDemoTimingTask5423-21chrt20:50:545
313899120,10tDemoTimingTask3356-21chrt20:49:065
2905529120,2tAtEmLog_023212-21turbostat00:14:545
2835739120,0EcMasterDemoSyn0-21swapper/522:22:525
2815299120,10tDemoTimingTask28263-21chrt20:32:265
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional