You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-03 - 01:59
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot1.osadl.org (updated Fri Apr 03, 2026 00:44:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
133039990,1EcMasterDemoSyn1337-21grep21:45:182
2678139930,1EcMasterDemoSyn41-21ksoftirqd/200:22:282
562639910,1EcMasterDemoSyn5633-21grep22:41:034
3193339910,0EcMasterDemoSyn3194329tAtEmLog_000:07:165
2777139820,0EcMasterDemoSyn2778129tAtEmLog_023:46:585
233792820,0sleep40-21swapper/422:50:454
2032939820,0EcMasterDemoSyn2033929tAtEmLog_021:55:245
1275639820,0EcMasterDemoSyn1276629tAtEmLog_019:18:155
1020339820,0EcMasterDemoSyn1021329tAtEmLog_023:01:205
101639820,0EcMasterDemoSyn102629tAtEmLog_022:56:165
55939810,0EcMasterDemoSyn56929tAtEmLog_021:09:475
2694739810,0EcMasterDemoSyn2695729tAtEmLog_019:48:395
2090639810,0EcMasterDemoSyn2091629tAtEmLog_020:44:275
1115039570,0EcMasterDemoSyn1116029tAtEmLog_020:29:125
309342370,0sleep50-21swapper/522:37:265
155792360,0sleep015577-21ssh23:58:290
247482340,0sleep40-21swapper/423:45:184
38252210,0sleep00-21swapper/021:46:360
821599200,7cyclictest203061sh21:55:231
821599200,7cyclictest17711latency_hist20:05:011
821599190,7cyclictest24270-21ssh23:09:531
821599190,6cyclictest253981timerwakeupswit23:45:261
821599190,11cyclictest27375-21latency_hist19:50:011
2837499190,8tDemoTimingTask4499-21chrt21:29:505
821599184,9cyclictest7316-21cut20:15:141
821599180,7cyclictest646950latency_hist20:15:001
821599180,7cyclictest14120-21grep00:15:181
821599180,6cyclictest9717-21runintdemo5min21:14:551
821099186,9cyclictest1174-21sh22:20:370
821099185,9cyclictest22155-21sh22:32:360
821099180,14cyclictest8653-21sh00:12:220
821099180,14cyclictest29497-21sh00:05:310
821099180,14cyclictest17728-21sh21:54:560
821099180,14cyclictest12180-21sh22:09:380
821099180,10cyclictest18252-21sh21:36:420
821099180,10cyclictest14637-21sh23:21:400
821099180,10cyclictest14637-21sh23:21:400
3166339180,15EcMasterDemoSyn1401-21chrt20:00:505
3166339180,0EcMasterDemoSyn0-21swapper/519:58:495
2956529180,8tAtEmLog_030224-21chrt22:00:555
1766129180,0tAtEmLog_00-21swapper/500:22:275
1252299180,7tDemoTimingTask16271-21chrt00:35:045
859639170,7EcMasterDemoSyn11265-21taskset00:14:125
821899179,6cyclictest2249-21tr23:15:132
821599175,7cyclictest24483-21sh22:15:351
821599171,12cyclictest29007-21sh21:42:541
821599170,7cyclictest98881fschecks_time23:55:131
821599170,7cyclictest799-21ssh00:25:401
821599170,7cyclictest32636-21sh21:26:511
821599170,7cyclictest29403-21ssh00:05:271
821599170,7cyclictest24836-21cat19:45:001
821599170,7cyclictest14751-21chrt23:04:321
821599170,6cyclictest26046-21ssh21:23:361
821599170,6cyclictest25842-21sh22:52:281
821599170,16cyclictest18634-21switchtime22:30:241
821599170,15cyclictest21670-21ssh22:14:581
821599170,13cyclictest545-21ssh22:55:581
821099175,9cyclictest716-21snmpd19:18:130
821099171,8cyclictest23756-21cpuspeed_turbos21:40:130
821099170,9cyclictest8020-21sh21:13:480
821099170,9cyclictest780-21runrttasks19:36:430
821099170,9cyclictest23520-21sh00:20:240
821099170,9cyclictest21838-21sh23:08:100
821099170,9cyclictest12940-21sh22:27:290
821099170,9cyclictest12635-21sh23:03:020
821099170,12cyclictest15930-21sh22:46:510
821099170,11cyclictest174-1kworker/0:1H+kblockd23:47:170
2931999170,10tDemoTimingTask29435-21chrt19:54:565
1548739170,0EcMasterDemoSyn0-21swapper/522:10:375
1362199170,7tDemoTimingTask20541-21chrt00:00:345
1362199170,7tDemoTimingTask15134-21chrt23:58:105
822399165,8cyclictest716-21snmpd21:19:403
822399160,9cyclictest29422-21sh22:36:213
821899163,9cyclictest16431-21cut20:35:002
821599165,10cyclictest32367-21cat22:20:181
821599164,8cyclictest117672wc20:25:141
821599164,10cyclictest8339-21grep22:25:141
821599162,6cyclictest27558-21seq20:55:211
821599161,8cyclictest4947-21cat22:05:171
821599161,8cyclictest10873-21rm22:44:411
821599160,9cyclictest11463-21cat20:24:591
821599160,8cyclictest30283-21ssh00:24:561
821599160,8cyclictest28970-21tr21:00:111
821599160,8cyclictest27414-21ssh22:00:081
821599160,8cyclictest25238-21ssh23:10:111
821599160,8cyclictest2436-21sh21:45:381
821599160,8cyclictest23741-21date20:50:001
821599160,8cyclictest20453-21ssh23:43:081
821599160,8cyclictest18784-21sed20:40:001
821599160,8cyclictest18702-21rm00:35:371
821599160,8cyclictest16736-21ssh21:35:381
821599160,8cyclictest15733-21unixbench-2d20:30:251
821599160,8cyclictest15704-21aten2_r5power_v19:25:121
821599160,8cyclictest15704-21aten2_r5power_v19:25:121
821599160,8cyclictest1555-21ssh22:39:341
821599160,8cyclictest14429-21ssh22:45:501
821599160,8cyclictest1422-21ssh23:32:271
821599160,8cyclictest14212-21needreboot19:20:151
821599160,8cyclictest13733-21ssh21:16:231
821599160,8cyclictest10923-21ssh23:20:121
821599160,8cyclictest10923-21ssh23:20:121
821599160,8cyclictest10869-21ssh00:13:561
821599160,8cyclictest10603-21pluginstate21:50:211
821599160,8cyclictest10132-21rm00:30:551
821599160,7cyclictest3755-21ssh23:51:331
821599160,7cyclictest296021latency_hist19:55:011
821599160,7cyclictest21680-21unixbench-2d23:25:241
821599160,7cyclictest20445-21ssh00:00:291
821599160,6cyclictest12600-21ssh21:34:061
821599160,4cyclictest5722-21ssh23:16:431
821599160,14cyclictest31505-21cut21:05:161
821599160,13cyclictest5448-21ssh23:35:061
821099165,8cyclictest780-21runrttasks20:26:380
821099160,8cyclictest780-21runrttasks20:50:580
821099160,7cyclictest15083-21grep23:40:140
821099160,13cyclictest3235-21fschecks_count22:40:140
2460029160,9tAtEmLog_024605-21chrt19:43:365
1116199160,9tDemoTimingTask11245-21chrt20:24:565
860629150,6tAtEmLog_017383-21taskset00:17:135
822799154,8cyclictest1134-21wc22:56:194
822799150,9cyclictest23378-21sh21:21:444
822399154,8cyclictest648-21dbus-daemon20:35:193
8223991511,3cyclictest4768-21turbostat.cron21:30:003
822399151,10cyclictest21673-21sh00:37:433
822399150,9cyclictest632-21diskmemload00:11:403
822399150,9cyclictest20306-21smartctl21:20:163
822399150,8cyclictest9475-21runrttasks23:36:473
822399150,11cyclictest8337-21ssh22:42:553
822399150,11cyclictest30625-21ssh00:06:203
822399150,11cyclictest12327-21diskstats22:45:143
821899152,9cyclictest21156-21latency_hist20:45:002
821899151,10cyclictest21798-21sh23:08:082
821899151,10cyclictest20426-21ssh21:38:142
821899151,10cyclictest16043-21ssh23:58:492
821899151,10cyclictest12401-21runrttasks21:51:122
821899150,5cyclictest24751-21idleruntime-cro19:45:002
821899150,11cyclictest8368-21if_err_enp6s000:30:192
821599151,7cyclictest29749-21/usr/sbin/munin19:55:131
821599150,8cyclictest8562-21aten2_r5power_p19:10:121
821599150,7cyclictest8226-21latency_hist19:09:591
821599150,7cyclictest2508-21tune2fs20:05:141
821599150,7cyclictest24210-21grep20:50:111
821599150,7cyclictest20552-21tr19:35:121
821599150,7cyclictest19314-21cat20:40:131
821599150,7cyclictest18882-21cat19:30:161
821599150,7cyclictest10897-21aten2_r5power_c19:15:121
821099156,7cyclictest14193-21/usr/sbin/munin00:15:190
821099153,9cyclictest780-21runrttasks19:33:040
821099151,9cyclictest13095-21sh23:39:190
821099151,5cyclictest13313-21cut22:10:120
821099151,10cyclictest780-21runrttasks19:10:300
821099150,9cyclictest8848-21diskmemload23:19:000
821099150,9cyclictest22690-21ssh21:57:020
821099150,9cyclictest22143-21open_inodes22:50:160
821099150,8cyclictest25768-21sh22:16:280
25930991514,0tDemoTimingTask0-21swapper/520:59:385
2222399150,8tDemoTimingTask22382-21turbostat19:40:005
134029150,13tAtEmLog_05090-21taskset21:47:295
822799146,6cyclictest648-21dbus-daemon21:15:004
822799145,8cyclictest8205-21cyclictest19:46:574
822799141,9cyclictest31639-21sh23:30:574
822799141,9cyclictest27556-21sshd23:10:404
822799141,9cyclictest23961-21ssh21:57:564
822799141,9cyclictest1962-21diskmemload22:21:094
822799141,9cyclictest16520-21munin-run22:30:004
822799141,9cyclictest11887-21sh21:33:364
822799141,7cyclictest11405-21taskset23:38:084
822799141,10cyclictest9758-21mailstats19:10:184
822799141,10cyclictest28539-21runrttasks21:25:074
822799141,10cyclictest16-1pr/legacy22:05:324
822799140,9cyclictest14750-21sed19:20:224
822799140,11cyclictest811-21runintdemo5min22:36:024
822799140,10cyclictest9123-21sh21:50:064
822799140,10cyclictest617-21cron21:10:004
822799140,10cyclictest5664-21sh23:16:414
822799140,10cyclictest5024-21chrt23:52:274
822799140,10cyclictest3782-21runrttasks20:06:164
822799140,10cyclictest32119-21/usr/sbin/munin20:00:174
822799140,10cyclictest30665-21ssh22:01:144
822799140,10cyclictest17517-21ssh21:19:024
822799140,10cyclictest15823-21ssh23:05:054
822799140,10cyclictest15352-21ssh00:15:464
822399147,5cyclictest32302-21grep22:20:183
822399143,6cyclictest4660-21expr20:10:123
822399142,9cyclictest17495-21sed19:33:253
822399142,8cyclictest9103-21switchtime00:30:263
822399141,9cyclictest9321-21/usr/sbin/munin21:50:143
822399141,9cyclictest780-21runrttasks21:05:363
822399141,9cyclictest24479-21runrttasks22:15:353
822399141,9cyclictest10988-21sh21:32:583
8223991411,2cyclictest19382-21sh22:30:403
822399141,10cyclictest15298-21ssh00:15:433
822399141,10cyclictest13316-21wget19:20:113
822399140,9cyclictest15607-21/usr/sbin/munin19:25:123
822399140,9cyclictest15607-21/usr/sbin/munin19:25:123
822399140,10cyclictest921-21which20:00:183
822399140,10cyclictest9153-21/usr/sbin/munin20:20:153
822399140,10cyclictest780-21runrttasks21:40:333
822399140,10cyclictest780-21runrttasks19:58:133
822399140,10cyclictest7793-21ssh23:54:243
822399140,10cyclictest6881-21/usr/sbin/munin23:00:243
822399140,10cyclictest570-21sh21:45:133
822399140,10cyclictest3994-21sh22:05:053
822399140,10cyclictest3910-21ssh23:15:273
822399140,10cyclictest31708-21sh00:25:163
822399140,10cyclictest31295-21sh22:55:183
822399140,10cyclictest27951-21rm23:10:563
822399140,10cyclictest27483-21/usr/sbin/munin22:00:153
822399140,10cyclictest25931-21sh22:52:313
822399140,10cyclictest25080-21sensors_fan23:45:213
822399140,10cyclictest23684-21ssh00:02:473
822399140,10cyclictest21918-21ssh00:20:093
822399140,10cyclictest19871-21/usr/sbin/munin23:25:123
822399140,10cyclictest18639-21sh22:12:493
822399140,10cyclictest18359-21/usr/sbin/munin21:55:153
822399140,10cyclictest17630-21ssh21:36:163
822399140,10cyclictest16880-21runrttasks23:40:383
822399140,10cyclictest15963-21/usr/sbin/munin23:05:123
822399140,10cyclictest14595-21ssh23:57:473
822399140,10cyclictest13865-21sh23:21:093
822399140,10cyclictest13865-21sh23:21:093
822399140,10cyclictest10880-21/usr/sbin/munin19:15:193
822399140,10cyclictest10167-21sh22:25:333
821899148,5cyclictest25133-21tr19:45:112
821899145,7cyclictest31217-21sed22:55:172
821899145,7cyclictest1052-21/usr/sbin/munin23:50:152
821899144,9cyclictest32461-21/usr/sbin/munin20:00:122
821899142,8cyclictest326-21systemd-journal19:10:402
821899141,9cyclictest29163-21ssh22:36:102
821899141,9cyclictest27693-21diskmemload23:10:452
821899141,9cyclictest14858-21runintdemo5min22:46:072
821899141,9cyclictest10448-21/usr/sbin/munin21:15:172
8218991411,2cyclictest22563-21grep22:15:132
8218991411,2cyclictest19092-21kworker/2:2+events22:25:492
821899141,10cyclictest16-1pr/legacy20:29:132
821899140,7cyclictest32383-21sensors_fan00:25:202
821899140,12cyclictest12811-21ssh23:39:072
821899140,11cyclictest1358-21ssh00:08:322
821899140,10cyclictest8762-21runrttasks20:19:122
821899140,10cyclictest8199-21ssh00:12:032
821899140,10cyclictest780-21runrttasks21:32:112
821899140,10cyclictest716-21snmpd19:35:112
821899140,10cyclictest5837-21ssh21:12:172
821899140,10cyclictest582-21chrt21:09:492
821899140,10cyclictest5375-21ssh22:23:342
821899140,10cyclictest32745-21sh23:31:432
821899140,10cyclictest326-21systemd-journal22:05:402
821899140,10cyclictest31038-21sh21:25:432
821899140,10cyclictest30506-21sh22:01:082
821899140,10cyclictest29738-21runrttasks19:55:012
821899140,10cyclictest28189-21ssh21:42:202
821899140,10cyclictest26547-21ssh23:46:062
821899140,10cyclictest22789-21ssh21:57:062
821899140,10cyclictest22508-21sendmail_mailtr20:45:192
821899140,10cyclictest20147-21latency_hist19:35:002
821899140,10cyclictest18517-21cut22:30:212
821899140,10cyclictest18381-21users00:35:242
821899140,10cyclictest18352-21/usr/sbin/munin00:00:212
821899140,10cyclictest17657-21ssh23:41:122
821899140,10cyclictest14018-21ssh23:04:012
821899140,10cyclictest12921-21/usr/sbin/munin00:15:162
821899140,10cyclictest12550-21rm23:20:252
821899140,10cyclictest12550-21rm23:20:242
821899140,10cyclictest11237-21/usr/sbin/munin19:15:132
821099146,6cyclictest29705-21missed_timers21:25:180
821099145,8cyclictest5999-21/usr/sbin/munin21:30:150
821099142,8cyclictest5795-21timerandwakeup20:10:240
821099141,9cyclictest780-21runrttasks20:07:350
821099141,9cyclictest6987-21cpuspeed_turbos20:15:120
821099141,9cyclictest29328-21sh22:36:170
821099141,9cyclictest26752-21sh23:10:210
821099141,9cyclictest22672-21ssh00:02:040
8210991410,3cyclictest25619-21gltestperf19:45:140
821099140,7cyclictest553-21/usr/sbin/munin23:50:120
821099140,7cyclictest3406-21ssh00:27:320
821099140,7cyclictest31798-21sh22:02:020
821099140,7cyclictest1258-21sh22:56:250
821099140,7cyclictest11908-21python321:15:220
821099140,12cyclictest21049-21sh00:37:180
821099140,10cyclictest9840-21ssh00:30:420
821099140,10cyclictest780-21runrttasks20:04:380
821099140,10cyclictest30802-21sed23:30:240
821099140,10cyclictest26985-21chrt20:55:150
821099140,10cyclictest25980-21ssh23:28:190
821099140,10cyclictest22953-21runrttasks20:45:260
821099140,10cyclictest22432-21rm21:21:040
821099140,10cyclictest14013-21grep19:20:150
1252299140,2tDemoTimingTask154082sleep500:34:425
822799135,6cyclictest22098-21cut00:20:124
822799132,8cyclictest21092-21tr23:25:194
822799131,9cyclictest30609-21latency19:55:164
822799131,7cyclictest14722-21cut23:40:124
822799130,9cyclictest9528-21ssh00:30:304
822799130,9cyclictest8069-21ssh23:00:174
822799130,9cyclictest780-21runrttasks19:50:244
822799130,9cyclictest32345-21sh21:45:024
822799130,9cyclictest30927-21/usr/sbin/munin00:25:114
822799130,9cyclictest27357-21sh00:05:104
822799130,9cyclictest24543-21chrt19:43:054
822799130,9cyclictest23491-21/usr/sbin/munin21:40:134
822799130,9cyclictest21787-21/usr/sbin/munin19:35:234
822799130,9cyclictest19228-21ssh22:30:344
822799130,9cyclictest17874-21ssh00:35:204
822799130,9cyclictest16453-21cat20:35:004
822799130,9cyclictest15607-21/usr/sbin/munin19:25:134
822799130,9cyclictest15607-21/usr/sbin/munin19:25:134
822799130,9cyclictest14426-21/usr/sbin/munin21:35:124
822799130,9cyclictest13166-21sh22:10:084
822799130,9cyclictest11901-21/usr/sbin/munin22:45:154
822799130,10cyclictest9561-21df_inode20:20:124
822799130,10cyclictest9137-21ssh23:55:084
822799130,10cyclictest7722-21cat20:15:174
822799130,10cyclictest5582-21sh00:10:244
822799130,10cyclictest4417-21/usr/sbin/munin20:10:204
822799130,10cyclictest30340-21runrttasks21:00:334
822799130,10cyclictest26544-21grep20:55:114
822799130,10cyclictest24012-21aten2_r5power_p20:50:104
822799130,10cyclictest22330-21ssh22:15:124
822799130,10cyclictest20700-21taskset20:41:474
822799130,10cyclictest18352-21/usr/sbin/munin00:00:194
822799130,10cyclictest16893-21cpuspeed_turbos20:35:124
822799130,10cyclictest13323-21sh23:20:464
822799130,10cyclictest13323-21sh23:20:464
822399131,7cyclictest9096-21latency_hist20:20:003
822399130,9cyclictest8162-21latency_hist19:10:003
822399130,9cyclictest780-21runrttasks21:10:053
822399130,9cyclictest30772-21unixbench_singl23:30:233
822399130,9cyclictest26299-21/usr/sbin/munin20:55:113
822399130,9cyclictest25317-21timerandwakeup20:50:223
822399130,9cyclictest18942-21/usr/sbin/munin20:40:163
822399130,9cyclictest14214-21/usr/sbin/munin20:30:113
822399130,9cyclictest12235-21hddtemp_smartct20:25:173
822399130,10cyclictest8550-21/usr/sbin/munin19:10:113
822399130,10cyclictest731-21sshd20:49:393
822399130,10cyclictest27631-21expr19:50:133
822399130,10cyclictest25915-21/usr/sbin/munin19:45:163
822399130,10cyclictest24658-21chrt19:43:583
822399130,10cyclictest20317-21/usr/sbin/munin19:35:183
821899131,9cyclictest27424-21processes20:55:182
821899130,9cyclictest9147-21runrttasks20:20:062
821899130,9cyclictest780-21runrttasks20:12:262
821899130,9cyclictest2772-21/usr/sbin/munin22:40:172
821899130,9cyclictest27495-21aten2_r5power_v19:50:132
821899130,9cyclictest23947-21/usr/sbin/munin20:50:152
821899130,9cyclictest23036-21sh22:50:302
821899130,9cyclictest22114-21sh21:20:512
821899130,9cyclictest2067-21/usr/sbin/munin20:05:242
821899130,9cyclictest17333-21/usr/sbin/munin20:35:152
821899130,9cyclictest15622-21sh22:10:412
821899130,10cyclictest20002-21sh23:25:132
821899130,10cyclictest16421-21sed19:25:162
821899130,10cyclictest16421-21sed19:25:162
821099135,6cyclictest9863-21gltestperf20:20:140
821099131,7cyclictest29411-21/usr/sbin/munin21:00:150
821099130,9cyclictest32040-21grep21:05:190
821099130,9cyclictest21266-21latency_hist20:45:000
821099130,9cyclictest18494-21taskset20:39:050
821099130,9cyclictest14203-21runrttasks20:30:020
821099130,10cyclictest30418-21hddtemp_smartct19:55:150
821099130,10cyclictest27691-21expr19:50:130
821099130,10cyclictest24494-21chrt19:42:240
821099130,10cyclictest17332-21runrttasks19:26:230
821099130,10cyclictest17332-21runrttasks19:26:220
822799120,9cyclictest21388-21/usr/sbin/munin20:45:124
822799120,9cyclictest18312-21df19:30:124
822799120,9cyclictest12435-21memory20:25:184
822799120,9cyclictest11458-21gltestperf19:15:154
822399120,9cyclictest28672-21/usr/sbin/munin21:00:113
822399120,9cyclictest2070-21apt-get20:05:123
821899123,7cyclictest13347-21/usr/sbin/munin19:20:102
821899120,9cyclictest29985-21/usr/sbin/munin21:00:222
798199120,10tDemoTimingTask8468-21taskset19:10:055
562599120,10tDemoTimingTask7789-21chrt19:05:475
345999120,7tDemoTimingTask3463-21taskset00:27:335
2400639120,0EcMasterDemoSyn0-21swapper/522:51:135
1974999120,10tDemoTimingTask16524-21turbostat22:34:555
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional