You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-31 - 05:04
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot1.osadl.org (updated Wed Dec 31, 2025 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
676139850,0EcMasterDemoSyn677229tAtEmLog_000:19:305
912139830,0EcMasterDemoSyn913129tAtEmLog_020:31:195
2581239830,0EcMasterDemoSyn2582229tAtEmLog_023:54:095
487839820,0EcMasterDemoSyn488829tAtEmLog_021:52:265
2700639820,0EcMasterDemoSyn2701629tAtEmLog_020:00:535
1621639820,0EcMasterDemoSyn1622629tAtEmLog_020:46:315
1169439820,0EcMasterDemoSyn1170429tAtEmLog_000:04:175
82939810,0EcMasterDemoSyn84029tAtEmLog_023:03:255
440739810,0EcMasterDemoSyn441729tAtEmLog_020:21:095
3207339810,0EcMasterDemoSyn3208329tAtEmLog_020:11:015
3176039810,0EcMasterDemoSyn3177029tAtEmLog_019:05:055
3176039810,0EcMasterDemoSyn3177029tAtEmLog_019:05:055
2111039810,0EcMasterDemoSyn2112029tAtEmLog_000:09:215
2286339800,0EcMasterDemoSyn2287329tAtEmLog_022:02:355
7072780,0sleep3481rcuc/322:08:233
1857439690,0EcMasterDemoSyn0-21swapper/320:51:353
1191339570,0EcMasterDemoSyn1192329tAtEmLog_022:38:035
172639410,0EcMasterDemoSyn1735-21runintdemo5min19:10:094
187102260,1sleep30-21swapper/323:13:173
1473992316,5cyclictest17203-21cron21:59:594
1048699230,18tDemoTimingTask14453-21chrt21:40:015
146899201,13cyclictest16-1pr/legacy19:40:353
69322190,0sleep40-21swapper/421:53:504
27122190,0sleep20-21swapper/223:59:212
146899198,4cyclictest16-1pr/legacy20:26:133
146899190,8cyclictest23598-21ssh22:58:043
146899181,13cyclictest16-1pr/legacy21:52:253
146899180,10cyclictest20789-21ssh22:20:083
146499182,11cyclictest6221-21ssh21:35:132
145099180,14cyclictest29143-21sh21:48:020
145099180,14cyclictest16379-21sh23:48:470
145099180,10cyclictest15131-21sh23:29:440
699099170,10tDemoTimingTask7009-21chrt21:16:575
2614199170,10tDemoTimingTask26165-21chrt22:22:525
2568029170,6tAtEmLog_025957-21turbostat21:10:015
1756599170,10tDemoTimingTask17915-21chrt19:44:565
1513499170,7tDemoTimingTask17822-21chrt22:55:055
1513499170,7tDemoTimingTask15413-21chrt22:53:285
147399175,8cyclictest9440-21cron20:35:004
146499179,6cyclictest2767-21grep19:10:182
145099170,9cyclictest17133-21sh21:22:370
145099170,9cyclictest13990-21sh22:52:290
145099170,9cyclictest1354-21sh00:15:330
145099170,9cyclictest11407-21sh00:21:240
145099170,14cyclictest11308-21sh22:14:480
147399161,11cyclictest16-1pr/legacy21:37:124
147399160,11cyclictest11436-21ssh22:50:404
146899162,10cyclictest10008-21sed21:55:203
146499164,8cyclictest16406-21ssh23:48:492
146499163,9cyclictest19159-21ssh21:24:112
146499163,8cyclictest831-21runrttasks00:00:022
146499163,8cyclictest31621-21runrttasks20:10:212
146499162,9cyclictest26588-21cut23:00:002
146499162,10cyclictest5756-21diskmemload23:24:222
146499162,10cyclictest15879-21/usr/sbin/munin19:40:262
145099163,9cyclictest29687-21sh22:25:090
145099161,9cyclictest13113-21sh22:33:490
145099160,9cyclictest29773-21sh21:11:200
145099160,9cyclictest19897-21sh22:37:370
145099160,14cyclictest27487-21ssh21:28:530
145099160,13cyclictest831-21runrttasks19:49:120
145099160,13cyclictest831-21runrttasks19:32:060
1048699160,14tDemoTimingTask15969-21chrt21:40:195
1026199160,9tDemoTimingTask3136-21sleep23:09:565
147399150,9cyclictest32722-21/usr/sbin/munin22:45:164
147399150,11cyclictest16-1pr/legacy00:14:244
146899151,9cyclictest10743-21sh22:14:203
1468991511,3cyclictest717-21in:imuxsock23:39:593
146899151,11cyclictest16-1pr/legacy23:08:293
146899151,10cyclictest2982-21ssh23:22:263
146899151,10cyclictest10731-21diskmemload00:39:573
146899150,9cyclictest29854-21ssh21:48:323
146899150,2cyclictest0-21swapper/321:40:003
146899150,10cyclictest20636-21ssh00:09:013
146499155,8cyclictest2714-21ssh21:50:502
146499155,8cyclictest24700-21sh00:10:302
146499154,8cyclictest30264-21cut22:25:142
146499154,6cyclictest8899-21/usr/sbin/munin19:25:132
146499153,8cyclictest9026-21sed23:25:242
146499153,8cyclictest29472-21ssh21:30:122
146499153,8cyclictest26698-21taskset23:17:362
146499153,8cyclictest20128-21runrttasks19:48:032
146499153,8cyclictest20006-21ssh22:00:362
146499153,8cyclictest14772-21/usr/sbin/munin21:40:202
146499152,9cyclictest6950-21ssh22:48:452
146499152,9cyclictest4594-21fschecks_time23:05:172
146499152,9cyclictest27851-21ssh23:00:172
146499152,9cyclictest21243-21df22:20:132
146499152,9cyclictest21177-21fschecks_count21:25:152
146499152,9cyclictest16501-21latency_hist00:25:002
146499152,9cyclictest13138-21sh21:57:022
146499152,8cyclictest3832-21memory21:15:182
146499152,8cyclictest14643-21sh22:15:482
146499151,9cyclictest31115-21ssh00:32:502
146499151,10cyclictest3718-21processes00:35:202
146499151,10cyclictest20812-21cpuspeed_turbos19:50:142
146499150,8cyclictest533-21sh22:08:162
145799159,5cyclictest21870-21tr21:00:131
145799157,6cyclictest32476-21tail23:20:231
145799152,9cyclictest1890-21ssh23:04:081
1457991512,2cyclictest15420-21sh23:30:001
145799151,10cyclictest831-21runrttasks00:37:391
145799151,10cyclictest22005-21runrttasks21:44:211
145099157,6cyclictest5268-21tr20:25:130
145099156,7cyclictest9970-21ssh23:08:180
145099156,7cyclictest25335-21munin-run00:30:000
145099153,9cyclictest2652-21sh22:09:490
145099151,10cyclictest10911-21sh21:37:290
145099150,9cyclictest14858-21sh22:15:580
145099150,8cyclictest28070-21sh23:18:350
145099150,8cyclictest25210-21gltestperf23:35:160
135199150,8tDemoTimingTask1794-21turbostat00:35:015
147399147,5cyclictest10129-21sort20:35:144
147399145,8cyclictest4601-21/usr/sbin/munin19:15:134
147399143,8cyclictest24661-21sh22:40:174
147399141,9cyclictest8679-21diskmemload22:30:404
147399141,9cyclictest7550-21ssh21:17:184
147399141,9cyclictest31810-21ssh00:33:214
147399141,9cyclictest31641-21sh00:15:134
147399141,9cyclictest30109-21latency_hist23:20:004
147399141,9cyclictest23978-21sh23:34:574
147399141,9cyclictest16212-21ssh21:40:204
147399140,12cyclictest26520-21ssh23:35:294
147399140,11cyclictest17829-21grep20:50:194
147399140,10cyclictest831-21runrttasks00:08:274
147399140,10cyclictest8297-21chrt19:22:274
147399140,10cyclictest4513-21ssh21:34:154
147399140,10cyclictest2687-21/usr/sbin/munin20:20:184
147399140,10cyclictest23845-21df21:45:164
147399140,10cyclictest23263-21ssh23:52:154
147399140,10cyclictest2147-21ssh22:09:274
147399140,10cyclictest19445-21sendmail_mailtr19:45:214
147399140,10cyclictest19427-21sh22:37:114
147399140,10cyclictest17774-21sh23:12:274
147399140,10cyclictest15060-21ssh22:16:064
146899147,6cyclictest1067-21apache220:23:563
146899143,8cyclictest4598-21cut19:15:133
146899143,8cyclictest12153-21grep19:30:193
146899143,7cyclictest30961-21kworker/u24:4+nfsiod21:15:023
146899142,9cyclictest807-21postgres00:15:483
146899141,9cyclictest7908-21sh00:01:363
146899141,9cyclictest744-21snmpd21:29:233
146899141,9cyclictest28492-21/usr/sbin/munin23:00:243
146899141,9cyclictest21147-21sh23:50:463
146899141,9cyclictest19898-21sh22:00:323
146899141,10cyclictest3936-21runrttasks23:41:143
146899141,10cyclictest23474-21runrttasks21:03:243
146899141,10cyclictest16-1pr/legacy20:31:173
146899140,9cyclictest27959-21/usr/sbin/munin20:05:143
146899140,9cyclictest14300-21latency19:35:173
146899140,8cyclictest29877-21diskmemload23:55:373
146899140,7cyclictest456-21grep22:45:133
146899140,12cyclictest25372-21ssh00:10:593
146899140,10cyclictest9449-21ssh23:25:393
146899140,10cyclictest831-21runrttasks19:21:023
146899140,10cyclictest6760-21df_inode22:30:133
146899140,10cyclictest32763-21ssh21:31:193
146899140,10cyclictest32729-21ssh22:26:083
146899140,10cyclictest28226-21sh22:42:073
146899140,10cyclictest26947-21sh23:17:463
146899140,10cyclictest25762-21runrttasks00:30:053
146899140,10cyclictest20276-21cron19:49:593
146899140,10cyclictest20085-21ssh22:37:443
146899140,10cyclictest19497-21ssh23:31:403
146899140,10cyclictest17438-21rm21:40:563
146899140,10cyclictest15177-21runrttasks23:47:573
146899140,10cyclictest13204-21ntp_kernel_pll_21:20:193
146899140,10cyclictest12229-21ssh22:15:133
146899140,10cyclictest10208-21cat22:50:193
146499145,6cyclictest30961-21kworker/u24:4+xprtiod21:46:102
146499144,8cyclictest9286-21chrt20:32:582
146499144,8cyclictest17026-21sh00:25:112
146499143,7cyclictest15855-21sed20:45:272
146499143,6cyclictest1907-21ssh00:15:562
146499142,9cyclictest26684-21ssh23:35:362
146499142,8cyclictest24646-21ssh23:53:192
146499142,8cyclictest24049-21grep22:40:132
146499142,8cyclictest14761-21python319:35:222
146499142,8cyclictest13361-21df23:10:122
146499142,8cyclictest12409-21python319:30:222
146499141,9cyclictest287-21jbd2/md0-820:40:012
146499141,9cyclictest25154-21python321:05:212
146499141,9cyclictest20855-21runrttasks20:55:422
146499141,9cyclictest20855-21runrttasks20:55:422
146499141,9cyclictest15122-21wget22:35:122
146499141,9cyclictest11538-21runrttasks22:14:592
146499141,10cyclictest27950-21chrt20:05:032
146499140,7cyclictest29597-21ssh21:11:132
146499140,10cyclictest9583-21df22:50:152
146499140,10cyclictest8550-21ssh22:30:352
146499140,10cyclictest7210-21irqstats19:20:182
146499140,10cyclictest4844-21interrupts19:15:152
146499140,10cyclictest19710-21ssh23:31:502
146499140,10cyclictest1671-21ssh23:40:162
146499140,10cyclictest13649-21/usr/sbin/munin00:05:132
145799148,3cyclictest16497-21gltestperf19:40:181
145799145,7cyclictest918-21/usr/sbin/munin22:45:151
145799144,6cyclictest27518-21apt-config20:05:001
145799142,8cyclictest442-21cpuspeed20:15:141
145799141,9cyclictest4534-21runrttasks20:22:211
145799141,9cyclictest31282-21ssh23:39:001
145799141,9cyclictest29588-21ssh22:25:051
145799141,9cyclictest15294-21latency20:45:201
145799141,9cyclictest12028-21ssh22:15:071
1457991411,2cyclictest717-21in:imuxsock19:20:001
145799141,10cyclictest18101-21ssh23:12:511
145799140,8cyclictest20266-21runrttasks19:49:551
145799140,11cyclictest28583-21ssh22:05:251
145799140,10cyclictest9085-21ssh23:07:411
145799140,10cyclictest8884-21rm22:13:011
145799140,10cyclictest8586-21grep00:20:171
145799140,10cyclictest854-21/usr/sbin/munin23:40:181
145799140,10cyclictest8373-21sh22:30:271
145799140,10cyclictest7948-21ssh00:01:361
145799140,10cyclictest7053-21chrt20:29:381
145799140,10cyclictest32356-21runrttasks20:14:221
145799140,10cyclictest32329-21ssh21:31:001
145799140,10cyclictest2989-21grep21:15:121
145799140,10cyclictest29852-21runrttasks20:07:591
145799140,10cyclictest28901-21ssh00:31:101
145799140,10cyclictest28306-21ssh22:42:101
145799140,10cyclictest26686-21ssh21:46:211
145799140,10cyclictest24848-21ssh23:16:191
145799140,10cyclictest23563-21ssh23:34:411
145799140,10cyclictest22631-21/usr/sbin/munin00:10:131
145799140,10cyclictest20984-21sh23:50:381
145799140,10cyclictest20899-21taskset22:01:131
145799140,10cyclictest20755-21sh22:20:081
145799140,10cyclictest17533-21ssh00:06:481
145799140,10cyclictest15532-21ssh21:58:511
145799140,10cyclictest14898-21ssh22:35:041
145799140,10cyclictest14621-21sh22:52:571
145799140,10cyclictest10565-21awk23:45:181
145099146,7cyclictest0-21swapper/019:50:140
145099145,7cyclictest7511-21ssh00:01:180
145099143,8cyclictest21358-21latency_hist21:00:000
145099143,8cyclictest21358-21latency_hist21:00:000
145099141,9cyclictest831-21runrttasks20:53:140
145099141,9cyclictest27650-21sh22:41:420
145099141,7cyclictest3769-21latency_hist19:15:000
145099141,10cyclictest28188-21sh22:24:190
145099140,8cyclictest16-1pr/legacy21:57:290
145099140,7cyclictest831-21runrttasks23:40:140
145099140,7cyclictest831-21runrttasks21:09:430
145099140,12cyclictest31603-21ssh21:30:300
145099140,12cyclictest2331-21sh22:45:290
145099140,10cyclictest831-21runrttasks23:21:360
145099140,10cyclictest831-21runrttasks23:03:430
145099140,10cyclictest831-21runrttasks21:52:250
145099140,10cyclictest831-21runrttasks19:56:460
145099140,10cyclictest831-21runrttasks19:25:280
145099140,10cyclictest4882-21ssh21:15:290
145099140,10cyclictest29705-21ssh00:31:490
145099140,10cyclictest26869-21ssh00:12:030
145099140,10cyclictest2206-21sh23:59:020
145099140,10cyclictest21898-21sh23:51:170
145099140,10cyclictest20710-21sh23:14:420
145099140,10cyclictest18264-21ssh22:55:140
145099140,10cyclictest17909-21diskmemload00:07:040
145099140,10cyclictest16231-21sh21:40:220
145099140,10cyclictest15765-21ssh23:30:040
1734699130,11tDemoTimingTask17446-21taskset22:17:525
149099130,11tDemoTimingTask3590-21chrt21:33:355
147399131,9cyclictest4748-21sh00:00:164
147399131,9cyclictest4040-21df22:10:174
147399131,9cyclictest29206-21sensors_temp20:05:204
147399131,9cyclictest26969-21/usr/sbin/munin23:00:184
147399131,7cyclictest19705-21ssh00:25:564
147399130,9cyclictest8796-21/usr/sbin/munin19:25:144
147399130,9cyclictest831-21runrttasks23:45:214
147399130,9cyclictest831-21runrttasks23:25:334
147399130,9cyclictest808-21postgres21:06:394
147399130,9cyclictest7097-21munin-run20:30:004
147399130,9cyclictest3769-21ssh23:05:034
147399130,9cyclictest3271-21ssh23:40:454
147399130,9cyclictest30442-21ssh23:20:024
147399130,9cyclictest30041-21sh23:55:444
147399130,9cyclictest29464-21ssh22:25:014
147399130,9cyclictest29344-21ssh21:11:024
147399130,9cyclictest27257-21runrttasks20:03:454
147399130,9cyclictest23192-21users21:00:254
147399130,9cyclictest23077-21sh22:20:374
147399130,9cyclictest22935-21/usr/sbin/munin19:55:154
147399130,9cyclictest2259-21sh00:35:074
147399130,9cyclictest20661-21aten2_r5power_p19:50:134
147399130,9cyclictest20528-21ssh21:25:064
147399130,9cyclictest18609-21sh22:00:184
147399130,9cyclictest18418-21cstates22:55:144
147399130,9cyclictest14475-21/usr/sbin/munin20:45:164
147399130,9cyclictest13866-21diskstats19:35:144
147399130,9cyclictest11993-21ssh21:20:114
147399130,10cyclictest9313-21netstat00:20:204
147399130,10cyclictest1662-21sed20:15:264
147399130,10cyclictest11869-21if_enp5s019:30:174
146899135,6cyclictest18385-21awk00:25:223
146899130,9cyclictest8836-21aten2_r5power_e19:25:123
146899130,9cyclictest831-21runrttasks20:15:143
146899130,9cyclictest7945-21ssh00:20:053
146899130,9cyclictest30708-21cstates20:10:133
146899130,9cyclictest28462-21sed21:10:253
146899130,9cyclictest25678-21runintdemo5min21:06:483
146899130,9cyclictest25364-21aten2_r5power_e20:00:143
146899130,9cyclictest21796-21mailstats19:50:213
146899130,9cyclictest19940-21kernelversion20:55:193
146899130,9cyclictest19940-21kernelversion20:55:193
146899130,9cyclictest1758-21/usr/sbin/munin19:10:193
146899130,9cyclictest14960-21tune2fs20:45:183
146899130,9cyclictest12601-21fschecks_time20:40:163
146899130,9cyclictest10325-21gltestperf20:35:153
146899130,10cyclictest23154-21ls19:55:153
146499133,7cyclictest743-21tune2fs20:15:162
146499132,6cyclictest9909-21/usr/sbin/munin20:35:132
146499130,9cyclictest831-21runrttasks21:03:262
146499130,9cyclictest831-21runrttasks20:50:152
146499130,9cyclictest6811-21runrttasks20:26:232
146499130,9cyclictest3046-21df_inode20:20:132
146499130,9cyclictest26145-21sed20:00:172
146499130,9cyclictest22939-21apt-get19:55:142
145799132,8cyclictest17197-21forks20:50:161
145799131,9cyclictest19175-21ssh00:25:331
145799131,9cyclictest11825-21runrttasks19:30:171
145799130,9cyclictest8982-21unixbench_singl20:30:261
145799130,9cyclictest8792-21taskset19:25:091
145799130,9cyclictest8149-21taskset19:20:271
145799130,9cyclictest319-21ssh21:50:121
145799130,9cyclictest27690-21/usr/sbin/munin23:55:211
145799130,9cyclictest26684-21cpu21:10:131
145799130,9cyclictest23969-21aten2_r5power_p21:05:121
145799130,9cyclictest20669-21/usr/sbin/munin21:25:151
145799130,9cyclictest20584-21/usr/sbin/munin19:50:161
145799130,9cyclictest19174-21runrttasks20:55:101
145799130,9cyclictest19174-21runrttasks20:55:101
145799130,9cyclictest1706-21taskset19:10:011
145799130,9cyclictest15232-21taskset19:35:441
145799130,9cyclictest12110-21aten2_r5power_v21:20:131
145799130,12cyclictest30961-21kworker/u24:4+rpciod22:55:221
145799130,10cyclictest831-21runrttasks21:35:081
145799130,10cyclictest2897-21ssh00:16:391
145799130,10cyclictest23189-21cpuspeed_turbos19:55:151
145799130,10cyclictest13965-21runrttasks20:42:201
145799130,10cyclictest10505-21kernelversion20:35:171
145099132,8cyclictest3588-21missed_timers20:20:170
145099132,8cyclictest327-21systemd-journal20:42:480
145099131,9cyclictest744-21snmpd19:35:320
145099130,9cyclictest4382-21expr19:15:120
145099130,9cyclictest29655-21grep20:05:560
145099130,9cyclictest26862-21unixbench_multi20:00:240
145099130,9cyclictest2203-21sh00:35:060
145099130,9cyclictest17563-21ssh22:00:020
145099130,9cyclictest15915-21aten2_r5power_c19:40:150
145099130,7cyclictest831-21runrttasks20:32:170
145099130,7cyclictest831-21runrttasks19:22:370
145099130,7cyclictest31239-21memory20:10:180
145099130,7cyclictest175-1kworker/0:1H+kblockd21:02:280
145099130,11cyclictest10321-21/usr/sbin/munin20:35:160
145099130,10cyclictest831-21runrttasks20:49:440
145099130,10cyclictest831-21runrttasks20:19:090
1026199130,11tDemoTimingTask17589-21taskset23:12:195
466329120,10tAtEmLog_05787-21chrt19:15:395
2816939120,1EcMasterDemoSyn651irq_work/523:18:385
2568199120,10tDemoTimingTask28627-21chrt21:10:325
2331439120,0EcMasterDemoSyn0-21swapper/521:01:435
2252999120,10tDemoTimingTask30970-21taskset23:38:495
2093239120,0EcMasterDemoSyn0-21swapper/520:56:395
2093239120,0EcMasterDemoSyn0-21swapper/520:56:395
1857439120,1EcMasterDemoSyn0-21swapper/520:51:355
172639120,0EcMasterDemoSyn0-21swapper/519:10:095
147399120,9cyclictest30366-21/usr/sbin/munin20:10:144
147399120,9cyclictest20870-21runrttasks20:55:564
147399120,9cyclictest20870-21runrttasks20:55:554
147399120,9cyclictest16349-21uniq19:40:174
147399120,9cyclictest12176-21aten2_r5power_e20:40:134
146899124,6cyclictest1649-21cut19:10:003
146899124,6cyclictest1649-21cut19:10:003
145799121,7cyclictest140450latency_hist19:10:001
145799121,7cyclictest140450latency_hist19:10:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional