You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-19 - 06:14
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot1.osadl.org (updated Fri Dec 19, 2025 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
632339910,1EcMasterDemoSyn6328-21runintdemo5min22:57:053
1578439910,0EcMasterDemoSyn1579429tAtEmLog_022:26:395
65632890,1sleep2391rcuc/222:40:002
502339830,0EcMasterDemoSyn503329tAtEmLog_000:08:045
1760639830,0EcMasterDemoSyn1761629tAtEmLog_020:35:065
1419939830,0EcMasterDemoSyn1420929tAtEmLog_019:19:025
3216639820,0EcMasterDemoSyn3217829tAtEmLog_000:23:175
2277339810,0EcMasterDemoSyn2278329tAtEmLog_020:45:145
1657939810,0EcMasterDemoSyn1658929tAtEmLog_019:24:055
1184139810,0EcMasterDemoSyn1185129tAtEmLog_019:13:585
946739800,0EcMasterDemoSyn947729tAtEmLog_019:08:545
248452700,0sleep10-21swapper/122:14:291
1657939670,2EcMasterDemoSyn16588-21wc19:24:060
258112530,0sleep10-21swapper/120:50:211
28082310,0sleep50-21swapper/523:30:255
9663992316,6cyclictest9037-21sed00:10:183
964899235,10cyclictest9795-21sh21:30:420
964899230,9cyclictest20916-21sh22:47:180
964899228,6cyclictest19162-21grep00:15:260
964899225,10cyclictest25982-21sh21:22:210
964899224,10cyclictest10633-21tail23:35:150
964899224,10cyclictest10633-21tail23:35:150
964899220,9cyclictest904-21sh22:00:430
964899220,9cyclictest14534-21diskmemload21:50:300
964899220,9cyclictest14534-21diskmemload21:50:300
964899220,10cyclictest11953-21diskmemload23:18:180
964899210,9cyclictest6844-21sh21:46:340
964899210,9cyclictest30219-21rm21:25:010
964899210,9cyclictest26261-21ssh22:32:330
964899210,9cyclictest24148-21diskmemload22:14:000
964899210,9cyclictest14207-21sh22:25:340
964899210,7cyclictest831-21runrttasks23:04:320
964899210,12cyclictest30128-21ssh22:16:480
964899205,7cyclictest863-21latency_hist20:00:000
964899201,9cyclictest8404-21sh21:13:090
964899201,9cyclictest32056-21sh21:42:570
964899201,7cyclictest29568-21awk23:45:200
964899201,11cyclictest7764-21latency_hist00:10:000
964899200,9cyclictest9125-21sh22:23:220
964899200,9cyclictest7623-21fschecks_count22:40:150
964899200,9cyclictest5151-21diskmemload00:25:450
964899200,9cyclictest5149-21sh23:50:140
964899200,9cyclictest27360-21rm23:09:050
964899200,9cyclictest26585-21cpuspeed_turbos00:20:150
964899200,9cyclictest25400-21ssh21:39:580
964899200,9cyclictest21450-21sh23:59:310
964899200,9cyclictest20501-21sh00:35:120
964899200,9cyclictest16377-21sh21:17:140
964899200,8cyclictest20133-21runintdemo5min23:22:270
964899200,10cyclictest12461-21ssh00:12:000
178529200,0tAtEmLog_00-21swapper/522:06:215
964899191,7cyclictest1764-21latency_hist21:10:000
964899190,9cyclictest864-21runintdemo5min20:40:130
964899190,9cyclictest6143-21diskmemload23:32:430
964899190,9cyclictest425-21sh22:35:510
964899190,9cyclictest32099-21latency_hist21:05:000
964899190,9cyclictest23111-21mailstats21:55:190
964899190,9cyclictest18475-21/usr/sbin/munin20:35:180
964899190,9cyclictest18107-21diskmemload00:33:390
964899190,9cyclictest13268-21grep19:15:180
964899190,7cyclictest831-21runrttasks20:00:010
964899190,7cyclictest175-1kworker/0:1H+kblockd22:53:460
964899190,12cyclictest15332-21ssh22:09:180
964899190,10cyclictest23603-21runrttasks23:42:250
964899190,10cyclictest175-1kworker/0:1H+kblockd19:42:020
964899185,6cyclictest5558-21latency_hist20:10:000
964899185,6cyclictest5558-21latency_hist20:09:590
964899185,6cyclictest1-21systemd23:25:050
964899184,6cyclictest21861-21aten2_r5power_v19:35:130
964899184,6cyclictest10977-21ntpq19:10:190
964899181,9cyclictest9877-21runrttasks20:15:280
964899180,9cyclictest831-21runrttasks19:51:500
964899180,9cyclictest831-21runrttasks19:30:330
964899180,9cyclictest28194-21runrttasks19:46:140
964899180,9cyclictest25207-21fschecks_time20:50:170
964899180,9cyclictest24076-21users20:45:250
964899180,9cyclictest1276-21diskmemload23:12:110
964899180,8cyclictest5864-21apt-get20:10:120
964899180,7cyclictest13307-21diskstats20:25:140
964899180,10cyclictest5518-21ssh22:56:310
964899180,10cyclictest25792-21sh00:01:090
966399179,6cyclictest18410-21head00:15:193
966399170,13cyclictest17557-21cut20:35:013
965799179,6cyclictest26884-21sed22:15:152
965799170,10cyclictest18593-21ssh23:21:242
964899170,6cyclictest19319-21cat19:30:000
460329170,7tAtEmLog_04894-21taskset21:10:465
314499170,10tDemoTimingTask3148-21chrt20:04:415
2920499170,7tDemoTimingTask30031-21chrt21:41:355
2005299170,10tDemoTimingTask20056-21chrt20:40:115
1948399171,6tDemoTimingTask19807-21chrt21:36:095
966799162,10cyclictest24813-21sh00:37:054
966799162,10cyclictest24178-21runrttasks23:06:514
966799161,8cyclictest4909-21ssh22:56:064
966799161,8cyclictest18438-21sh22:10:214
966799160,9cyclictest1131-21sh00:24:154
966399169,6cyclictest26161-21rm23:25:253
966399164,9cyclictest8096-21latency_hist20:15:013
9663991612,3cyclictest19733-21gltestperf23:40:153
965799160,10cyclictest31531-21ssh21:42:362
9653991612,3cyclictest30341-21cron22:00:001
965399160,13cyclictest11234-21ssh22:24:501
964899160,8cyclictest11175-21/usr/sbin/munin20:20:180
2315399160,9tDemoTimingTask23269-21taskset00:18:175
2135239160,7EcMasterDemoSyn23467-21taskset19:36:125
1549729160,6tAtEmLog_019541-21chrt21:53:545
1549729160,6tAtEmLog_019541-21chrt21:53:545
966799155,6cyclictest30761-21ssh23:28:344
966799153,8cyclictest28160-21ssh23:45:124
966799152,8cyclictest10981-21sort20:20:174
966799150,9cyclictest5131-21ssh00:25:454
966399155,7cyclictest27381-21kworker/u24:3+nfsiod21:20:153
966399154,8cyclictest23438-21cut00:00:173
966399153,9cyclictest744-21snmpd00:37:133
966399150,11cyclictest8199-21ssh23:15:423
965799156,7cyclictest8618-21rm22:05:182
965799154,8cyclictest17490-21which22:45:202
965799150,9cyclictest32568-21diskmemload22:54:062
965799150,9cyclictest10025-21sh00:29:162
965799150,8cyclictest287-21jbd2/md0-819:16:232
965799150,11cyclictest27157-21sh21:23:092
965799150,10cyclictest4175-21ssh23:14:122
965399159,5cyclictest27011-21cut00:20:161
965399153,9cyclictest744-21snmpd23:53:131
965399153,9cyclictest744-21snmpd22:17:231
965399152,7cyclictest15359-21kworker/u24:0+nfsiod22:44:451
9653991511,3cyclictest27381-21kworker/u24:3-xprtiod21:35:151
965399151,10cyclictest831-21runrttasks21:54:131
965399151,10cyclictest831-21runrttasks21:54:131
965399151,10cyclictest8212-21/usr/sbin/munin20:15:131
965399150,9cyclictest831-21runrttasks21:04:151
965399150,9cyclictest16052-21sed19:20:221
964899154,6cyclictest16762-21timerwakeupswit20:30:270
966799147,4cyclictest26468-21grep22:15:144
966799146,4cyclictest29112-21sh00:03:304
966799145,8cyclictest0-21swapper/421:51:104
966799145,8cyclictest0-21swapper/421:51:104
966799143,8cyclictest7648-21cpuspeed_turbos21:30:144
966799143,8cyclictest21828-21fschecks_time22:30:164
966799142,8cyclictest8046-21cat20:15:014
966799142,8cyclictest4551-21ssh23:31:374
966799142,8cyclictest25502-21cat19:40:244
966799142,8cyclictest23776-21ssh22:49:184
966799142,8cyclictest19270-21sh21:19:094
966799142,8cyclictest18008-21sh00:15:164
966799142,8cyclictest14369-21ssh22:44:074
966799142,7cyclictest24702-21rm21:21:294
966799141,9cyclictest5804-21ssh00:08:364
966799141,9cyclictest17125-21sh23:56:294
966799140,12cyclictest28205-21sed21:40:264
966799140,10cyclictest831-21runrttasks23:18:584
966799140,10cyclictest8265-21fschecks_count22:05:154
966799140,10cyclictest6832-21diskmemload21:46:334
966799140,10cyclictest25665-21sh23:43:504
966799140,10cyclictest24403-21rm21:55:564
966799140,10cyclictest2106-21sh21:10:024
966799140,10cyclictest16279-21ssh22:26:594
966799140,10cyclictest15398-21aten2_r5power_v23:20:144
966799140,10cyclictest13987-21sh23:01:054
966799140,10cyclictest1000-21latency_hist20:00:004
966399144,8cyclictest3610-21/usr/sbin/munin00:25:183
966399141,9cyclictest831-21runrttasks22:16:383
966399141,9cyclictest831-21runrttasks19:56:313
966399141,9cyclictest6514-21sh23:32:593
966399141,9cyclictest4744-21/usr/sbin/munin23:50:143
966399141,10cyclictest10684-21kernelversion19:10:173
9663991410,3cyclictest0-21swapper/321:51:103
9663991410,3cyclictest0-21swapper/321:51:103
966399140,9cyclictest22572-21sh23:05:443
966399140,9cyclictest1154-21/usr/sbin/munin20:00:253
966399140,10cyclictest8713-21sh21:30:213
966399140,10cyclictest831-21runrttasks23:22:023
966399140,10cyclictest7746-21/usr/sbin/munin22:05:143
966399140,10cyclictest7214-21ssh21:46:483
966399140,10cyclictest578-21ssh21:25:493
966399140,10cyclictest3704-21ssh23:13:523
966399140,10cyclictest32122-21/usr/sbin/munin21:05:183
966399140,10cyclictest31344-21cat22:00:143
966399140,10cyclictest30111-21ssh00:21:483
966399140,10cyclictest30078-21runrttasks22:35:013
966399140,10cyclictest2945-21sh00:06:353
966399140,10cyclictest29242-21runrttasks20:57:273
966399140,10cyclictest29080-21expr23:45:183
966399140,10cyclictest28121-21users19:45:253
966399140,10cyclictest27856-21python321:40:223
966399140,10cyclictest2731-21cut21:10:143
966399140,10cyclictest25163-21netstat19:40:183
966399140,10cyclictest25069-21grep20:50:153
966399140,10cyclictest23205-21chrt21:55:223
966399140,10cyclictest22385-21/usr/sbin/munin20:45:253
966399140,10cyclictest20839-21irqstats20:40:183
966399140,10cyclictest18727-21ssh22:45:483
966399140,10cyclictest17247-21sh22:27:393
966399140,10cyclictest16913-21/usr/sbin/munin22:10:173
966399140,10cyclictest14952-21latency_hist20:30:013
966399140,10cyclictest14781-21ssh23:01:383
966399140,10cyclictest14216-21ssh21:15:453
966399140,10cyclictest13700-21/usr/sbin/munin23:55:233
966399140,10cyclictest12466-21ssh23:35:513
966399140,10cyclictest12466-21ssh23:35:503
966399140,10cyclictest11636-21/usr/sbin/munin00:30:193
966399140,10cyclictest10604-21ssh22:41:333
966399140,10cyclictest10599-21/usr/sbin/munin20:20:173
966399140,10cyclictest1058-21ssh22:54:443
965799143,8cyclictest10025-21wget23:35:142
965799143,8cyclictest10025-21wget23:35:132
965799141,9cyclictest29245-21sshd20:57:282
965799141,9cyclictest22842-21sh23:05:552
965799141,9cyclictest20912-21df_inode00:35:142
965799141,9cyclictest14792-21diskmemload21:50:422
965799141,9cyclictest14792-21diskmemload21:50:412
9657991412,2cyclictest31820-21turbostat21:09:552
965799141,10cyclictest19404-21runrttasks21:35:542
965799141,10cyclictest17655-21/usr/sbin/munin20:35:142
965799140,9cyclictest8255-21ssh23:15:442
965799140,9cyclictest4721-21which20:05:212
965799140,9cyclictest4721-21which20:05:212
965799140,12cyclictest6805-21ssh23:50:302
965799140,11cyclictest6283-21ssh21:46:092
965799140,10cyclictest831-21runrttasks23:33:212
965799140,10cyclictest831-21runrttasks19:56:282
965799140,10cyclictest831-21runrttasks19:47:492
965799140,10cyclictest808-21postgres20:18:012
965799140,10cyclictest744-21snmpd19:29:462
965799140,10cyclictest7413-21/usr/sbin/munin21:30:222
965799140,10cyclictest6976-21ssh22:40:082
965799140,10cyclictest6709-21ssh21:12:012
965799140,10cyclictest6614-21uname20:10:182
965799140,10cyclictest4253-21ssh21:28:152
965799140,10cyclictest3783-21sh22:02:402
965799140,10cyclictest3393-21ps22:55:212
965799140,10cyclictest3233-21cpuspeed_turbos22:20:132
965799140,10cyclictest30249-21rm21:59:562
965799140,10cyclictest2918-21ssh00:06:342
965799140,10cyclictest28758-21perl23:45:142
965799140,10cyclictest27890-21awk00:20:252
965799140,10cyclictest27366-21ssh00:02:172
965799140,10cyclictest24575-21ssh23:43:032
965799140,10cyclictest21418-21sh00:17:022
965799140,10cyclictest21269-21/usr/sbin/munin22:30:142
965799140,10cyclictest17344-21ssh22:10:142
965799140,10cyclictest16688-21ssh23:56:112
965799140,10cyclictest16566-21swap20:30:262
965799140,10cyclictest15373-21ssh22:26:232
965799140,10cyclictest15316-21/usr/sbin/munin19:20:142
965799140,10cyclictest12996-21sed23:00:252
965799140,10cyclictest12637-21ssh00:12:092
965399142,9cyclictest19294-21sed20:35:261
965399142,9cyclictest1473-21ssh23:48:051
965399141,9cyclictest6094-21ssh00:26:261
965399141,9cyclictest16542-21sh22:27:101
965399141,9cyclictest15257-21/usr/sbin/munin23:20:281
965399141,9cyclictest12267-21/usr/sbin/munin19:15:151
965399141,9cyclictest1149-21runrttasks20:00:081
965399141,10cyclictest7130-21sh21:46:451
965399141,10cyclictest4489-21ssh22:38:361
965399141,10cyclictest18703-21ssh23:57:361
965399140,9cyclictest30413-21sh21:25:101
965399140,9cyclictest28834-21/usr/sbin/munin19:50:191
965399140,10cyclictest831-21runrttasks21:44:481
965399140,10cyclictest831-21runrttasks20:43:411
965399140,10cyclictest831-21runrttasks19:42:001
965399140,10cyclictest8174-21sh23:15:401
965399140,10cyclictest7436-21ssh00:09:471
965399140,10cyclictest722-21sh22:00:361
965399140,10cyclictest6968-21ssh21:12:111
965399140,10cyclictest3616-21sh23:30:561
965399140,10cyclictest28007-21gltestperf20:55:171
965399140,10cyclictest26571-21sh00:01:431
965399140,10cyclictest25342-21/usr/sbin/munin22:50:191
965399140,10cyclictest23907-21sh21:20:571
965399140,10cyclictest20503-21/usr/sbin/munin00:35:251
965399140,10cyclictest20418-21runrttasks23:05:141
965399140,10cyclictest18282-21missed_timers00:15:191
965399140,10cyclictest16064-21ssh22:09:481
965399140,10cyclictest15957-21ssh22:45:021
965399140,10cyclictest12242-21cut21:15:141
965399140,10cyclictest1165-21sh23:12:071
965399140,10cyclictest10948-21rm23:00:051
965399140,10cyclictest10912-21cpuspeed_turbos20:20:161
965399140,10cyclictest10506-21ssh21:31:101
2920499140,6tDemoTimingTask5675-21taskset21:45:475
1503199148,0tDemoTimingTask0-21swapper/523:38:335
1503199148,0tDemoTimingTask0-21swapper/523:38:325
966799138,2cyclictest171rcu_preempt00:30:044
966799133,6cyclictest1502-21df20:00:134
966799132,7cyclictest26867-21fschecks_count19:45:134
966799132,7cyclictest24856-21wget20:50:154
966799132,7cyclictest15730-21fschecks_count20:30:154
966799131,8cyclictest8964-21irqstats20:15:174
966799131,8cyclictest60-21ksoftirqd/400:10:444
966799131,8cyclictest5161-21/usr/sbin/munin23:50:144
966799131,8cyclictest32688-21fschecks_time21:05:164
966799131,8cyclictest31485-21ssh21:25:184
966799131,8cyclictest30759-21expr21:00:184
966799131,8cyclictest27286-21touch20:55:004
966799131,8cyclictest26362-21memory22:50:184
966799131,8cyclictest21761-21taskset20:40:464
966799131,8cyclictest19656-21expr19:30:124
966799131,8cyclictest16858-21grep21:35:144
966799131,8cyclictest13112-21cat19:15:174
966799131,6cyclictest744-21snmpd19:20:084
966799131,10cyclictest30906-21/usr/sbin/munin22:00:124
966799130,9cyclictest9917-21/usr/sbin/munin19:10:204
966799130,9cyclictest29266-21/usr/sbin/munin23:10:154
966799130,9cyclictest28834-21/usr/sbin/munin19:50:204
966799130,9cyclictest22385-21/usr/sbin/munin20:45:134
966799130,9cyclictest21757-21/usr/sbin/munin19:35:134
966799130,9cyclictest16997-21/usr/sbin/munin19:25:134
966799130,9cyclictest12946-21/usr/sbin/munin20:25:174
966799130,10cyclictest9693-21chrt23:35:014
966799130,10cyclictest9693-21chrt23:35:014
966799130,10cyclictest3562-21aten2_r5power_e20:05:134
966799130,10cyclictest3562-21aten2_r5power_e20:05:124
966799130,10cyclictest30352-21rm22:35:124
966799130,10cyclictest2681-21rm22:20:024
966399130,9cyclictest9007-21latency20:15:183
966399130,9cyclictest4762-21python320:05:213
966399130,9cyclictest4762-21python320:05:213
966399130,9cyclictest29766-21/usr/sbin/munin21:00:123
966399130,9cyclictest29556-21cat19:50:183
966399130,9cyclictest19830-21fschecks_count19:30:123
966399130,9cyclictest19397-21chrt20:35:563
966399130,9cyclictest16800-21aten2_r5power_p21:35:143
966399130,9cyclictest12385-21aten2_r5power_v19:15:123
966399130,10cyclictest4455-21/usr/sbin/munin22:20:243
966399130,10cyclictest24304-21ssh22:31:123
966399130,10cyclictest22367-21expr19:35:153
966399130,10cyclictest17345-21df19:25:143
966399130,10cyclictest15021-21grep19:20:123
965799135,6cyclictest29409-21fw_packets19:50:172
965799135,6cyclictest20400-21grep20:40:142
965799130,9cyclictest831-21runrttasks20:45:352
965799130,9cyclictest30529-21uname21:00:172
965799130,9cyclictest25099-21diskmemload23:25:172
965799130,9cyclictest24129-21/usr/sbin/munin19:40:172
965799130,9cyclictest12946-21/usr/sbin/munin20:25:142
965799130,9cyclictest1284-21cpu20:00:132
965799130,9cyclictest12649-21/usr/sbin/munin21:15:172
965799130,9cyclictest11820-21mailstats20:20:232
965799130,9cyclictest10274-21df_inode19:10:142
965799130,10cyclictest831-21runrttasks19:35:302
965799130,10cyclictest26570-21chrt20:52:102
965799130,10cyclictest19754-21df19:30:122
965799130,10cyclictest11593-21ssh00:30:102
965399130,9cyclictest9939-21/usr/sbin/munin23:35:251
965399130,9cyclictest9939-21/usr/sbin/munin23:35:251
965399130,9cyclictest9671-21latency_hist19:10:001
965399130,9cyclictest831-21runrttasks19:35:081
965399130,9cyclictest8147-21taskset00:10:061
965399130,9cyclictest5137-21rm22:56:151
965399130,9cyclictest31189-21/usr/sbin/munin19:55:201
965399130,9cyclictest26542-21aten2_r5power_e19:45:121
965399130,9cyclictest25785-21python323:25:211
965399130,9cyclictest21815-21fschecks_time22:30:161
965399130,9cyclictest19402-21/usr/sbin/munin19:30:121
965399130,9cyclictest18805-21ssh23:40:041
965399130,9cyclictest17000-21apt-get19:25:121
965399130,9cyclictest15300-21/usr/sbin/munin20:30:151
965399130,9cyclictest13115-21cpuspeed20:25:131
965399130,9cyclictest11711-21aten2_r5power_e00:30:131
965399130,9cyclictest10192-21grep19:10:131
965399130,10cyclictest5921-21aten2_r5power_e20:10:131
965399130,10cyclictest4087-21gltestperf20:05:151
965399130,10cyclictest4087-21gltestperf20:05:151
965399130,10cyclictest327-21systemd-journal20:48:231
964899132,8cyclictest28428-21nfs4_client20:55:200
550029130,1tAtEmLog_00-21swapper/520:12:015
1775099130,11tDemoTimingTask25287-21chrt00:37:255
966799120,8cyclictest60-21ksoftirqd/420:35:194
966399120,9cyclictest9665-21latency_hist19:09:593
965399124,6cyclictest32466-21cstates21:05:151
56099120,10tDemoTimingTask1256-21taskset21:26:135
2957599120,10tDemoTimingTask4764-21chrt22:55:595
2609429120,10tAtEmLog_028304-21taskset19:47:445
2485629120,10tAtEmLog_0577-21taskset22:00:315
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional