You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-31 - 02:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot1.osadl.org (updated Tue Mar 31, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
864521020,2sleep132-21ksoftirqd/123:37:471
156739930,0EcMasterDemoSyn157729tAtEmLog_023:15:505
3076039820,0EcMasterDemoSyn3077029tAtEmLog_020:08:135
2956139820,0EcMasterDemoSyn2957129tAtEmLog_022:55:335
202939810,0EcMasterDemoSyn203929tAtEmLog_019:07:215
1967939810,0EcMasterDemoSyn1968929tAtEmLog_000:36:585
15812790,0sleep20-21swapper/222:23:162
113842580,1sleep311385-21ssh21:53:353
46622210,0sleep30-21swapper/321:15:023
2341992118,2cyclictest13628-21turbostat21:24:554
232899201,7cyclictest8908-21latency_hist21:35:001
190492200,0sleep20-21swapper/221:40:132
68199190,8tDemoTimingTask3011-21chrt20:17:355
232899190,9cyclictest24742-21ssh00:04:431
232899190,9cyclictest2429-21ssh23:16:241
232899190,11cyclictest1198-21runrttasks21:47:491
232899190,10cyclictest4639-21latency_hist19:15:001
232899190,10cyclictest4095-21/usr/sbin/munin23:00:181
232899190,10cyclictest10979-21ssh21:35:221
441799180,8tDemoTimingTask6602-21chrt19:15:295
2436339180,0EcMasterDemoSyn0-21swapper/221:04:002
234199180,11cyclictest16-1pr/legacy20:23:244
234199180,11cyclictest16-1pr/legacy20:23:244
232899180,9cyclictest29412-21sh22:03:171
232899180,10cyclictest3464-21sh22:06:111
232899180,10cyclictest3464-21sh22:06:111
232899180,10cyclictest3165-21chrt20:19:011
232899180,10cyclictest12023-21/usr/sbin/munin19:30:161
232699180,10cyclictest19085-21sh00:19:240
232699180,10cyclictest1714-21sh23:15:550
884429170,7tAtEmLog_016638-21taskset22:13:595
884429170,7tAtEmLog_016638-21taskset22:13:595
801999170,7tDemoTimingTask9917-21chrt21:35:145
441799170,7tDemoTimingTask4630-21chrt19:14:595
371029170,6tAtEmLog_06114-21taskset21:50:195
233799170,9cyclictest8182-21ssh23:01:523
233299179,6cyclictest30257-21tail00:25:152
233299171,12cyclictest16-1pr/legacy21:34:242
232899170,9cyclictest23441-21needreboot23:10:191
232899170,10cyclictest13212-21/usr/sbin/munin23:05:141
232699174,9cyclictest13098-21sh21:19:390
232699171,12cyclictest16-1pr/legacy23:36:060
232699171,12cyclictest16-1pr/legacy20:53:500
232699171,12cyclictest16-1pr/legacy00:11:360
232699171,10cyclictest20745-21/usr/sbin/munin20:55:160
232699170,9cyclictest8790-21sh21:34:560
232699170,9cyclictest780-21runrttasks22:59:050
232699170,9cyclictest3081-21sh00:28:130
232699170,9cyclictest30474-21sh21:45:380
232699170,9cyclictest20832-21sh23:44:580
232699170,9cyclictest19160-21sh00:36:370
232699170,9cyclictest13582-21sh00:34:030
232699170,9cyclictest11919-21sh23:57:200
232699170,14cyclictest12583-21sh21:54:260
232699170,14cyclictest1189-21sh22:40:220
232699170,13cyclictest24757-21sh00:04:430
140799170,10tDemoTimingTask6950-21chrt23:55:005
677599160,9tDemoTimingTask7303-21taskset19:20:005
638299160,9tDemoTimingTask6391-21taskset23:00:395
3113529160,9tAtEmLog_022891-21sleep21:29:555
233799165,8cyclictest326-21systemd-journal20:12:593
233799164,8cyclictest1770-21which22:05:203
233799164,8cyclictest1770-21which22:05:203
232899162,11cyclictest16-1pr/legacy00:21:441
232899160,9cyclictest780-21runrttasks19:50:321
232899160,10cyclictest20811-21sh22:15:311
232699160,9cyclictest780-21runrttasks19:49:240
232699160,8cyclictest17778-21ssh21:56:370
232699160,8cyclictest16306-21sh22:13:460
232699160,8cyclictest16306-21sh22:13:460
232699160,10cyclictest21997-21cut23:45:160
456299150,9tDemoTimingTask13102-21taskset22:29:565
234199155,8cyclictest2385-21/usr/sbin/munin00:10:124
234199155,8cyclictest22019-21/usr/sbin/munin00:20:234
2341991512,2cyclictest15437-21runintdemo5min23:41:114
233799158,6cyclictest29367-21grep21:45:183
233799154,8cyclictest17306-21wc20:48:503
233799154,8cyclictest10139-21sed23:20:213
233799153,8cyclictest11989-21latency_hist19:30:003
233799150,9cyclictest23228-21rm00:20:583
233799150,9cyclictest15701-21ssh00:17:023
233299153,8cyclictest27483-21kernelversion23:30:172
233299151,10cyclictest17052-21/usr/sbin/munin19:40:172
232899152,10cyclictest16-1pr/legacy20:13:161
232899151,10cyclictest3471-21ssh22:59:581
232899150,8cyclictest3078-21ssh23:52:291
232899150,10cyclictest24608-21ssh21:42:581
232699154,8cyclictest21396-21runrttasks22:15:560
232699153,8cyclictest26440-21fw_packets00:05:180
232699153,8cyclictest26440-21fw_packets00:05:180
232699150,9cyclictest780-21runrttasks22:34:570
232699150,7cyclictest174-1kworker/0:1H+kblockd22:09:420
232699150,7cyclictest174-1kworker/0:1H+kblockd22:09:410
232699150,7cyclictest174-1kworker/0:1H+kblockd20:15:270
232699150,7cyclictest174-1kworker/0:1H+kblockd19:40:290
1808539150,4EcMasterDemoSyn230952sleep522:17:075
801999140,7tDemoTimingTask32126-21sleep21:34:555
2438899141,5tDemoTimingTask29000-21turbostat00:25:005
234199149,4cyclictest20894-21unixbench_singl21:40:254
234199145,8cyclictest5137-21/usr/sbin/munin23:00:174
234199144,5cyclictest9661-21timerandwakeup20:30:224
234199143,8cyclictest755-21postgres20:49:364
234199141,9cyclictest8797-21sh22:26:564
234199141,9cyclictest15735-40mandb00:00:024
234199141,9cyclictest15500-21/usr/sbin/munin00:35:254
234199140,8cyclictest28290-21ssh22:37:464
234199140,7cyclictest3974-21sh23:35:164
234199140,11cyclictest11969-21ssh23:57:214
234199140,10cyclictest9400-21ssh21:17:084
234199140,10cyclictest780-21runrttasks22:55:594
234199140,10cyclictest6715-21cut00:30:124
234199140,10cyclictest31393-21sh22:04:404
234199140,10cyclictest30240-21rm00:07:124
234199140,10cyclictest30240-21rm00:07:114
234199140,10cyclictest28748-21ssh23:13:334
234199140,10cyclictest28683-21rm21:27:414
234199140,10cyclictest25696-21sh23:29:474
234199140,10cyclictest25011-21sh23:46:324
234199140,10cyclictest2136-21ssh00:27:344
234199140,10cyclictest17535-21rm23:07:034
234199140,10cyclictest17466-21ssh00:18:154
234199140,10cyclictest1110-21sh22:05:154
234199140,10cyclictest1110-21sh22:05:154
233799147,6cyclictest648-21dbus-daemon00:25:213
233799144,8cyclictest22748-21/usr/sbin/munin23:10:163
233799143,8cyclictest716-21snmpd00:07:343
233799143,8cyclictest716-21snmpd00:07:343
233799141,9cyclictest780-21runrttasks21:35:283
233799141,9cyclictest30321-21sh23:50:023
233799141,9cyclictest25090-21diskmemload22:35:333
233799141,9cyclictest23265-21apt-get22:00:123
233799141,9cyclictest16894-21sh23:06:353
233799141,9cyclictest15499-21rm23:41:133
233799140,10cyclictest8418-21grep23:55:213
233799140,10cyclictest6555-21sed20:25:163
233799140,10cyclictest6342-21ssh22:43:393
233799140,10cyclictest5656-21ssh23:35:423
233799140,10cyclictest4121-21grep00:10:243
233799140,10cyclictest3566-21sh23:17:133
233799140,10cyclictest30168-21cat23:50:003
233799140,10cyclictest28512-21chrt20:04:483
233799140,10cyclictest28216-21interrupts22:55:173
233799140,10cyclictest28150-21df_inode21:10:143
233799140,10cyclictest26672-21ssh22:19:363
233799140,10cyclictest26026-21latency_hist23:30:003
233799140,10cyclictest25854-21ssh21:25:443
233799140,10cyclictest23809-21latency_hist19:55:003
233799140,10cyclictest22311-21latency_hist21:00:013
233799140,10cyclictest20115-21ssh21:58:153
233799140,10cyclictest18493-21grep00:00:243
233799140,10cyclictest17513-21ssh22:14:353
233799140,10cyclictest17513-21ssh22:14:353
233799140,10cyclictest17052-21/usr/sbin/munin19:40:133
233799140,10cyclictest16258-21sh22:30:473
233799140,10cyclictest15965-21sh22:49:023
233799140,10cyclictest14164-21sh21:20:093
233799140,10cyclictest13090-21ssh22:29:563
233799140,10cyclictest12269-21tr19:30:123
233299143,8cyclictest1-21systemd19:10:192
233299142,8cyclictest27306-21latency20:00:172
233299141,9cyclictest8914-21sh00:30:462
233299141,9cyclictest31883-21ssh22:40:022
233299141,9cyclictest13058-21netstat19:30:202
233299141,10cyclictest20522-21/usr/sbin/munin00:20:122
233299140,8cyclictest22854-21cut23:10:162
233299140,8cyclictest17867-21sh00:35:422
233299140,11cyclictest6888-21ssh21:50:282
233299140,11cyclictest3055-21ssh22:05:542
233299140,11cyclictest3055-21ssh22:05:542
233299140,10cyclictest780-21runrttasks21:20:522
233299140,10cyclictest7787-21chrt23:01:352
233299140,10cyclictest7316-21ssh22:25:542
233299140,10cyclictest447-21runrttasks20:10:382
233299140,10cyclictest32556-21ssh21:47:062
233299140,10cyclictest3039-21grep00:10:162
233299140,10cyclictest30122-21ssh00:07:062
233299140,10cyclictest30122-21ssh00:07:062
233299140,10cyclictest28890-21/usr/sbin/munin20:05:202
233299140,10cyclictest26526-21chrt21:06:352
233299140,10cyclictest26086-21sh23:47:172
233299140,10cyclictest25407-21sh22:35:462
233299140,10cyclictest25238-21ssh22:18:362
233299140,10cyclictest24383-21sh23:28:512
233299140,10cyclictest23054-21ssh22:00:022
233299140,10cyclictest21471-21ssh22:34:232
233299140,10cyclictest20712-21ssh21:58:402
233299140,10cyclictest19695-21runrttasks20:54:172
233299140,10cyclictest19162-21rm23:08:112
233299140,10cyclictest16220-21/usr/sbin/munin20:45:162
233299140,10cyclictest16157-21taskset19:37:062
233299140,10cyclictest15671-21ssh22:48:492
233299140,10cyclictest11055-21sed19:25:242
233299140,10cyclictest10919-21ssh21:18:112
233299140,10cyclictest10128-21ssh23:56:062
232899145,8cyclictest18348-21/usr/sbin/munin22:50:141
232899141,9cyclictest9701-21diskmemload00:31:201
232899141,9cyclictest780-21runrttasks20:20:041
232899141,9cyclictest780-21runrttasks20:20:041
232899141,9cyclictest4940-21/usr/sbin/munin19:15:171
232899141,9cyclictest23155-21sh23:27:591
232899141,9cyclictest21462-21/usr/sbin/munin23:45:221
232899141,9cyclictest17120-21sh23:42:201
232899141,10cyclictest16-1pr/legacy22:45:231
232899141,10cyclictest16-1pr/legacy21:09:031
232899141,10cyclictest16-1pr/legacy00:06:311
232899141,10cyclictest16-1pr/legacy00:06:311
232899141,10cyclictest15273-21sensors_fan22:30:211
232899141,10cyclictest13731-21ssh23:22:451
232899140,11cyclictest10175-21perl22:10:171
232899140,11cyclictest10175-21perl22:10:171
232899140,10cyclictest9973-21sh00:14:241
232899140,10cyclictest7005-21sh21:15:291
232899140,10cyclictest5923-21/usr/sbin/munin20:25:201
232899140,10cyclictest29139-21ssh23:30:481
232899140,10cyclictest27100-21head20:00:151
232899140,10cyclictest24403-21expr19:55:141
232899140,10cyclictest2410-21ssh00:27:441
232899140,10cyclictest23438-21/usr/sbin/munin21:25:231
232899140,10cyclictest22654-21runrttasks22:35:011
232899140,10cyclictest20803-21rm00:37:451
232899140,10cyclictest19720-21tr19:45:141
232899140,10cyclictest17704-21/usr/sbin/munin20:50:131
232899140,10cyclictest16893-21ssh21:20:551
232899140,10cyclictest13731-21sh23:58:361
232899140,10cyclictest13117-21aten2_r5power_v20:40:131
232899140,10cyclictest10905-21grep20:35:131
232699146,6cyclictest13324-21ssh23:05:130
232699145,7cyclictest29258-21sh21:10:220
232699144,6cyclictest15879-21timerandwakeup21:20:240
232699143,8cyclictest3663-21awk19:10:190
232699141,9cyclictest744-21postgres20:39:360
232699141,9cyclictest25455-21diskmemload00:22:310
232699141,9cyclictest20332-21diskmemload22:50:420
232699141,11cyclictest19745-21hddtemp_smartct21:40:160
232699140,7cyclictest23480-21aten2_r5power_c21:25:120
232699140,7cyclictest18216-21head23:25:160
232699140,7cyclictest12446-21ssh22:46:340
232699140,12cyclictest5109-21expr23:00:170
232699140,12cyclictest32438-21sh22:22:110
232699140,12cyclictest23271-21ssh22:00:110
232699140,10cyclictest780-21runrttasks20:45:520
232699140,10cyclictest7172-21sensors_volt20:25:210
232699140,10cyclictest28720-21sh23:30:320
232699140,10cyclictest27829-21sh23:12:550
232699140,10cyclictest16007-21sed19:35:240
232699140,10cyclictest15155-21ssh21:38:000
232699140,10cyclictest1421-21ssh23:51:200
232699140,10cyclictest11684-21sh23:21:190
2767799130,11tDemoTimingTask3813-21chrt22:24:505
2461299130,11tDemoTimingTask27654-21chrt23:48:235
2341991310,2cyclictest490-21cpuspeed_turbos21:30:134
2341991310,2cyclictest31672-21cpuspeed_turbos23:15:134
234199130,9cyclictest9974-21fschecks_count21:35:154
234199130,9cyclictest9336-21ssh22:10:014
234199130,9cyclictest9336-21ssh22:10:014
234199130,9cyclictest780-21runrttasks21:52:204
234199130,9cyclictest780-21runrttasks19:55:124
234199130,9cyclictest7183-21runrttasks20:25:244
234199130,9cyclictest716-21snmpd19:53:094
234199130,9cyclictest31136-21latency_hist20:10:004
234199130,9cyclictest30387-21rm23:50:064
234199130,9cyclictest27946-21rm21:45:024
234199130,9cyclictest27730-21/usr/sbin/munin21:10:184
234199130,9cyclictest27326-21gltestperf23:30:154
234199130,9cyclictest27013-21grep21:10:004
234199130,9cyclictest23372-21sh22:52:494
234199130,9cyclictest21205-21ssh22:15:474
234199130,9cyclictest20058-21/usr/sbin/munin20:55:134
234199130,9cyclictest19480-21hostname19:45:134
234199130,9cyclictest17090-21aten2_r5power_e19:40:124
234199130,9cyclictest16161-21sh22:30:414
234199130,9cyclictest16086-21sh21:55:274
234199130,9cyclictest1532-21df20:15:134
234199130,9cyclictest12891-21runrttasks23:22:094
234199130,9cyclictest11592-21runrttasks19:28:244
234199130,9cyclictest1065-21latency_hist20:15:004
234199130,11cyclictest17943-21cpuspeed_turbos20:50:134
234199130,10cyclictest9215-21ssh22:45:144
234199130,10cyclictest6837-21chrt19:17:594
234199130,10cyclictest28943-21memory22:20:164
234199130,10cyclictest2703-21sh22:41:064
234199130,10cyclictest22930-21perl21:00:144
234199130,10cyclictest16-1pr/legacy20:38:374
234199130,10cyclictest13427-21grep20:40:154
233799135,7cyclictest18577-21/usr/sbin/munin22:50:153
233799132,6cyclictest18364-21sed20:50:153
233799131,9cyclictest6621-21runrttasks19:15:443
233799131,9cyclictest12608-21seq20:39:103
233799130,9cyclictest780-21runrttasks19:58:093
233799130,9cyclictest7667-21df19:20:133
233799130,9cyclictest6022-21apt-config00:30:003
233799130,9cyclictest3568-21/usr/sbin/munin20:20:143
233799130,9cyclictest3568-21/usr/sbin/munin20:20:143
233799130,9cyclictest32408-21rm21:30:003
233799130,9cyclictest28890-21/usr/sbin/munin20:05:143
233799130,9cyclictest27687-21ssh22:20:043
233799130,9cyclictest26575-21/usr/sbin/munin23:30:203
233799130,9cyclictest2588-21/usr/sbin/munin19:10:153
233799130,9cyclictest24824-21tr21:05:143
233799130,9cyclictest14376-21/usr/sbin/munin19:35:173
233799130,10cyclictest8711-21grep20:30:143
233799130,10cyclictest3124-21chrt20:18:273
233799130,10cyclictest22667-21cut21:00:133
233799130,10cyclictest21221-21ssh21:40:373
233799130,10cyclictest15477-21diskmemload00:35:113
233799130,10cyclictest13428-21cut20:40:153
233299131,9cyclictest30619-21diskmemload23:50:112
233299131,9cyclictest24882-21kernelversion19:55:182
233299131,7cyclictest8275-21/usr/sbin/munin20:30:122
233299131,7cyclictest3309-21/usr/sbin/munin23:35:252
233299131,7cyclictest11205-21/usr/sbin/munin20:35:152
233299130,9cyclictest9472-21/usr/sbin/munin21:35:172
233299130,9cyclictest9461-21sh22:10:072
233299130,9cyclictest9461-21sh22:10:072
233299130,9cyclictest6985-21cron19:19:592
233299130,9cyclictest5923-21/usr/sbin/munin20:25:262
233299130,9cyclictest2397-21latency_hist19:10:002
233299130,9cyclictest17688-21diskmemload22:50:012
233299130,9cyclictest13559-21sh00:15:342
233299130,9cyclictest13003-21/usr/sbin/munin20:40:172
233299130,10cyclictest9831-21ssh23:20:182
233299130,10cyclictest29591-21runrttasks22:55:352
233299130,10cyclictest27504-21ssh21:10:022
233299130,10cyclictest23987-21fschecks_time21:25:142
233299130,10cyclictest2269-21ssh23:16:182
233299130,10cyclictest20062-21dpkg20:55:112
233299130,10cyclictest19408-21/usr/sbin/munin19:45:212
233299130,10cyclictest17987-21ssh00:00:202
233299130,10cyclictest12480-21sh23:40:122
232899130,9cyclictest9664-21taskset19:25:071
232899130,9cyclictest9092-21runrttasks19:22:011
232899130,9cyclictest8275-21/usr/sbin/munin20:30:121
232899130,9cyclictest780-21runrttasks20:55:061
232899130,9cyclictest5403-21fschecks_time21:50:141
232899130,9cyclictest4456-21sh22:25:041
232899130,9cyclictest27892-21aten2_r5power_c22:20:111
232899130,9cyclictest27730-21/usr/sbin/munin21:10:201
232899130,9cyclictest23113-21basename21:00:161
232899130,9cyclictest18876-21runrttasks19:42:391
232899130,9cyclictest15354-21/usr/sbin/munin20:45:161
232899130,9cyclictest14747-21forks19:35:141
232899130,9cyclictest13412-21ssh00:15:271
232899130,7cyclictest14406-21cut21:55:141
232899130,10cyclictest28952-21wget20:05:121
232899130,10cyclictest1142-21cp22:40:201
232699131,9cyclictest9147-21cut20:30:170
232699131,9cyclictest23397-21needreboot21:00:180
232699131,9cyclictest22290-21fschecks_time19:50:140
232699131,6cyclictest26947-21fschecks_time20:00:140
232699131,6cyclictest174-1kworker/0:1H+kblockd20:20:060
232699131,6cyclictest174-1kworker/0:1H+kblockd20:20:060
232699130,9cyclictest8999-21runrttasks19:20:450
232699130,9cyclictest780-21runrttasks19:15:000
232699130,9cyclictest4627-21runintdemo5min22:25:100
232699130,9cyclictest13449-21sed19:30:250
232699130,7cyclictest25381-21sh22:35:450
232699130,7cyclictest174-1kworker/0:1H+kblockd20:10:020
232699130,7cyclictest174-1kworker/0:1H+kblockd19:55:400
232699130,6cyclictest780-21runrttasks21:05:120
232699130,10cyclictest29841-21sed20:05:180
232699130,10cyclictest174-1kworker/0:1H+kblockd20:41:380
232699130,10cyclictest10019-21df19:25:130
2199499130,10tDemoTimingTask24340-21taskset21:03:495
1999229130,3tAtEmLog_023045-21chrt22:52:365
1077199130,10tDemoTimingTask14636-21chrt22:48:055
883439120,1EcMasterDemoSyn671rcuc/522:09:555
883439120,1EcMasterDemoSyn671rcuc/522:09:555
623439120,0EcMasterDemoSyn0-21swapper/523:36:075
308899120,10tDemoTimingTask5333-21chrt20:21:535
308899120,10tDemoTimingTask5333-21chrt20:21:535
2598399120,10tDemoTimingTask28319-21chrt20:02:505
234199124,6cyclictest3358-21kernelversion19:10:174
234199120,9cyclictest8575-21python319:20:214
234199120,9cyclictest26483-21/usr/sbin/munin20:00:204
234199120,9cyclictest14994-21gltestperf19:35:154
234199120,9cyclictest12186-21grep19:30:124
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional