You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-28 - 06:48
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot1.osadl.org (updated Sat Mar 28, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1874439910,0EcMasterDemoSyn1875429tAtEmLog_000:22:065
1637139910,0EcMasterDemoSyn1638129tAtEmLog_022:15:205
286902890,0sleep20-21swapper/223:34:082
991539830,0EcMasterDemoSyn992529tAtEmLog_019:27:595
1463339830,0EcMasterDemoSyn1464329tAtEmLog_019:38:075
1039539830,0EcMasterDemoSyn1040529tAtEmLog_021:19:335
2907739820,0EcMasterDemoSyn2908829tAtEmLog_022:05:125
2513539820,0EcMasterDemoSyn2514529tAtEmLog_021:09:245
2277639820,0EcMasterDemoSyn2278529tAtEmLog_021:04:205
2105139820,0EcMasterDemoSyn2106129tAtEmLog_022:35:375
1332839820,0EcMasterDemoSyn1333829tAtEmLog_020:44:035
2476039810,0EcMasterDemoSyn2477029tAtEmLog_023:31:245
2438739810,0EcMasterDemoSyn2439629tAtEmLog_019:58:255
1360239810,0EcMasterDemoSyn1361229tAtEmLog_000:37:205
1360239810,0EcMasterDemoSyn1361229tAtEmLog_000:37:195
121112590,0sleep00-21swapper/023:42:320
40092350,4sleep420115-21kworker/u24:2+flush-9:022:26:234
2881739240,1EcMasterDemoSyn28826-21wc21:29:423
203992210,3sleep50-21swapper/523:47:035
607991815,2cyclictest9226-21turbostat22:30:010
60799180,14cyclictest4904-21sh23:38:530
60799180,10cyclictest9625-21sh21:54:290
60799180,10cyclictest22836-21taskset00:25:000
277429180,10tAtEmLog_03262-21chrt19:15:015
1483399180,10tDemoTimingTask19462-21taskset00:05:015
620991714,2cyclictest24900-21kworker/4:1+events23:51:394
61999171,12cyclictest16-1pr/legacy23:11:063
60799174,9cyclictest31406-21ssh00:29:420
60799170,9cyclictest32477-21sh23:18:180
60799170,9cyclictest29620-21sh23:52:120
60799170,9cyclictest25750-21sh23:32:030
60799170,9cyclictest18969-21sh22:52:480
60799170,14cyclictest32019-21sh21:13:250
60799170,14cyclictest3157-21sh22:44:280
60799170,14cyclictest3157-21sh22:44:270
60799170,14cyclictest20939-21sh23:11:310
60799170,12cyclictest25066-21sh00:07:430
2886899170,7tDemoTimingTask30284-21chrt23:52:395
2886899170,6tDemoTimingTask4452-21chrt23:55:585
1702329170,10tAtEmLog_017028-21taskset19:43:125
61999163,9cyclictest11662-21sensors_temp19:30:213
61999160,12cyclictest16-1pr/legacy21:55:013
61699163,9cyclictest720-21ntpd22:00:192
61199167,7cyclictest16279-21grep21:40:121
61199165,8cyclictest19520-21sh23:46:271
611991611,4cyclictest6544-21turbostat23:40:001
60799160,8cyclictest17073-21sh21:22:500
60799160,8cyclictest11914-21sh22:13:180
3021139160,0EcMasterDemoSyn0-21swapper/522:40:425
3021139160,0EcMasterDemoSyn0-21swapper/522:40:415
1938229160,9tAtEmLog_019530-21taskset19:49:555
62099152,12cyclictest10455-21ssh21:19:344
62099151,9cyclictest5460-21diskmemload00:32:544
620991511,2cyclictest6201-21tr22:10:134
62099150,9cyclictest21921-21ssh22:18:584
61999155,8cyclictest9151-21/usr/sbin/munin00:35:143
61999155,8cyclictest9151-21/usr/sbin/munin00:35:133
61999151,12cyclictest20781-21ssh23:29:583
61999151,10cyclictest1698-21/usr/sbin/munin23:55:183
61999150,11cyclictest16-1pr/legacy23:00:583
61699154,6cyclictest6260-21tune2fs19:20:152
61699153,8cyclictest11399-21latency_hist21:20:012
616991512,2cyclictest695-21rs:main2
61699151,10cyclictest14240-21sh21:20:502
61699150,9cyclictest8841-21sh23:22:522
61699150,9cyclictest780-21runrttasks20:57:102
61699150,9cyclictest21388-21fschecks_time21:25:142
61699150,9cyclictest17910-21diskmemload00:21:322
61699150,9cyclictest13126-21runrttasks22:31:232
61699150,12cyclictest24907-1python300:25:232
61199157,6cyclictest25505-21cut20:00:141
61199155,8cyclictest15063-21/usr/sbin/munin22:50:211
61199154,6cyclictest9627-21grep20:35:151
61199153,8cyclictest23095-21latency_hist00:25:011
61199151,10cyclictest8850-21ssh21:53:571
61199151,10cyclictest5891-21sendmail_mailst22:45:191
61199151,10cyclictest5891-21sendmail_mailst22:45:191
61199150,9cyclictest7249-21sh21:17:201
61199150,13cyclictest21527-21aten2_r5power_p23:30:131
60799157,6cyclictest23768-21tail21:05:130
60799156,7cyclictest5754-21sh00:15:120
60799151,13cyclictest6110-21cat21:34:590
60799150,8cyclictest16180-21fschecks_count21:40:120
60799150,6cyclictest25233-21diskmemload00:34:200
60799150,6cyclictest17115-21memory23:45:170
60799150,14cyclictest29794-21ssh22:05:150
60799150,12cyclictest8519-21runrttasks20:33:060
60799150,11cyclictest9772-21ssh23:23:330
1191399150,9tDemoTimingTask9229-21sleep22:34:565
62099144,5cyclictest25032-21missed_timers22:20:184
62099143,8cyclictest7922-21interrupts23:40:154
62099141,9cyclictest24831-21/usr/sbin/munin21:45:134
62099141,9cyclictest10914-21/usr/sbin/munin00:00:194
62099141,10cyclictest3285-21sensors_temp21:50:224
62099141,10cyclictest12390-21runrttasks19:34:074
62099140,11cyclictest5420-21wget00:15:104
62099140,10cyclictest9997-21runrttasks23:59:504
62099140,10cyclictest882-21seq21:31:264
62099140,10cyclictest776-21sh00:12:084
62099140,10cyclictest3309-21wget19:15:124
62099140,10cyclictest3084-21dpkg23:20:014
62099140,10cyclictest30837-21chrt20:10:224
62099140,10cyclictest29394-21ssh22:58:474
62099140,10cyclictest2395-21sh23:37:054
62099140,10cyclictest21974-21ssh00:05:324
62099140,10cyclictest21534-21rm23:11:574
62099140,10cyclictest20284-21ssh00:23:104
62099140,10cyclictest16975-21sh21:40:174
62099140,10cyclictest16785-21users19:40:244
62099140,10cyclictest1673-21rm23:19:274
62099140,10cyclictest12193-21awk21:55:184
62099140,10cyclictest12175-21ssh00:36:184
62099140,10cyclictest12175-21ssh00:36:184
61999147,6cyclictest600-21cyclictest22:58:073
61999143,8cyclictest9810-21tr22:30:133
61999143,8cyclictest716-21snmpd00:11:433
61999141,9cyclictest9833-21runrttasks19:27:013
61999141,9cyclictest8270-21sh22:46:393
61999141,9cyclictest8270-21sh22:46:383
61999141,9cyclictest22492-21sh21:43:463
61999141,9cyclictest21918-21/usr/sbin/munin21:00:193
61999141,10cyclictest16-1pr/legacy20:13:363
61999141,10cyclictest16-1pr/legacy19:22:543
61999140,9cyclictest5733-21diskmemload00:15:113
61999140,10cyclictest945-21ssh23:18:563
61999140,10cyclictest7012-21ssh00:34:013
61999140,10cyclictest6784-21sh21:52:303
61999140,10cyclictest5842-21ssh21:34:513
61999140,10cyclictest5773-21ssh21:16:183
61999140,10cyclictest4482-21sh23:38:343
61999140,10cyclictest31403-21ssh22:05:383
61999140,10cyclictest27547-21ssh21:45:563
61999140,10cyclictest25114-21/usr/sbin/munin23:50:183
61999140,10cyclictest24991-21sh22:02:313
61999140,10cyclictest23977-21/usr/sbin/munin22:20:143
61999140,10cyclictest21094-21sh22:18:233
61999140,10cyclictest20414-21threads22:35:223
61999140,10cyclictest19325-21memory20:55:163
61999140,10cyclictest17642-21rm00:03:483
61999140,10cyclictest1574-21grep22:25:163
61999140,10cyclictest14426-21/usr/sbin/munin00:20:213
61999140,10cyclictest12511-21rm22:13:423
61999140,10cyclictest12160-21sh23:42:363
61699141,9cyclictest572-21/usr/sbin/munin22:25:202
61699141,9cyclictest19498-21chrt19:49:282
61699141,10cyclictest2275-21ssh00:30:392
61699140,9cyclictest811-21runintdemo5min20:44:052
61699140,9cyclictest27171-21ssh23:50:292
61699140,8cyclictest8218-21diskmemload21:53:302
61699140,10cyclictest918-21/usr/sbin/munin19:10:192
61699140,10cyclictest7886-21ssh22:10:272
61699140,10cyclictest780-21runrttasks21:42:182
61699140,10cyclictest5343-21sh23:03:172
61699140,10cyclictest5112-21runrttasks19:17:482
61699140,10cyclictest30775-21ssh21:12:342
61699140,10cyclictest29042-21ssh22:05:092
61699140,10cyclictest27774-21ssh00:09:392
61699140,10cyclictest26465-21sh22:20:452
61699140,10cyclictest24927-21taskset21:06:422
61699140,10cyclictest2454-21sh00:13:192
61699140,10cyclictest23923-21ssh23:13:392
61699140,10cyclictest22076-21python321:00:202
61699140,10cyclictest2107-21sh22:43:432
61699140,10cyclictest2107-21sh22:43:422
61699140,10cyclictest16119-21/usr/sbin/munin23:45:112
61699140,10cyclictest15922-21ssh23:26:312
61699140,10cyclictest14710-21ssh22:15:042
61699140,10cyclictest14483-21runrttasks19:36:152
61699140,10cyclictest14430-21ssh21:56:302
61699140,10cyclictest14313-21sh23:08:132
61699140,10cyclictest12053-21ssh23:42:302
61699140,10cyclictest11119-21ssh00:35:332
61699140,10cyclictest11119-21ssh00:35:322
61699140,10cyclictest10386-21chrt21:36:422
61199146,6cyclictest1432-21if_enp6s022:25:161
61199144,7cyclictest728-21lldpd21:38:131
61199141,9cyclictest8795-21ssh23:59:001
61199141,9cyclictest8170-21sh00:16:001
61199141,9cyclictest780-21runrttasks20:26:151
61199141,9cyclictest29056-21grep22:05:091
61199141,9cyclictest27047-21sh22:57:071
61199141,9cyclictest14659-21sh22:15:011
61199141,10cyclictest15451-21runrttasks20:46:041
611991410,3cyclictest29270-21fschecks_count00:10:121
611991410,3cyclictest20131-21cpuspeed_turbos00:05:141
61199140,9cyclictest27448-21sh23:50:401
61199140,11cyclictest25471-21sh22:20:221
61199140,10cyclictest9189-21ssh23:40:301
61199140,10cyclictest780-21runrttasks20:05:131
61199140,10cyclictest780-21runrttasks00:27:241
61199140,10cyclictest32385-21/usr/sbin/munin00:30:201
61199140,10cyclictest30995-21ssh22:41:141
61199140,10cyclictest30995-21ssh22:41:131
61199140,10cyclictest30946-21ssh23:17:131
61199140,10cyclictest29733-21ssh21:11:501
61199140,10cyclictest27447-21ssh21:45:511
61199140,10cyclictest20823-21/usr/sbin/munin21:00:111
61199140,10cyclictest19646-21diskmemload23:10:391
61199140,10cyclictest19317-21fschecks_time22:35:131
61199140,10cyclictest17495-21/usr/sbin/munin19:45:171
61199140,10cyclictest16841-21idleruntime-cro23:10:001
61199140,10cyclictest12981-21sh22:31:161
61199140,10cyclictest11374-21ssh00:35:441
61199140,10cyclictest11374-21ssh00:35:431
61199140,10cyclictest1107-21ssh21:31:361
61199140,10cyclictest10846-21ssh21:55:061
61199140,10cyclictest10363-21ssh23:23:571
60799146,6cyclictest6917-21ssh21:17:070
60799146,6cyclictest326-21systemd-journal21:25:160
60799146,6cyclictest32043-21cut20:15:120
60799146,6cyclictest23285-21fschecks_time22:55:170
60799145,7cyclictest6308-21cat22:45:240
60799145,7cyclictest6308-21cat22:45:230
60799143,8cyclictest326-21systemd-journal00:35:000
60799143,8cyclictest326-21systemd-journal00:35:000
60799141,9cyclictest780-21runrttasks19:20:200
60799141,9cyclictest18308-21cat20:55:000
60799140,7cyclictest21167-21sh22:18:250
60799140,7cyclictest13149-21ssh23:07:230
60799140,6cyclictest10417-21apt-get19:30:130
60799140,12cyclictest30392-21sh21:47:570
60799140,12cyclictest29148-21/usr/sbin/munin00:10:110
60799140,12cyclictest10365-21perl22:30:150
60799140,11cyclictest15336-21sh23:26:070
60799140,10cyclictest6849-21/usr/sbin/munin21:35:120
60799140,10cyclictest29573-21munin-node20:10:100
60799140,10cyclictest21530-21ssh22:35:550
60799140,10cyclictest20905-21cat22:00:170
60799140,10cyclictest1600-21idleruntime19:10:150
60799140,10cyclictest15952-21ssh21:57:340
60799140,10cyclictest11053-21aten2_r5power_v00:00:130
62099139,3cyclictest23749-21grep00:25:144
62099132,8cyclictest28193-21cstates22:40:114
62099132,8cyclictest28193-21cstates22:40:104
62099131,9cyclictest3610-21taskset20:21:074
62099131,9cyclictest21505-21taskset22:35:544
62099131,8cyclictest21021-21cpuspeed_turbos21:25:134
62099130,9cyclictest9575-21sh22:30:044
62099130,9cyclictest9391-21forks20:35:144
62099130,9cyclictest780-21runrttasks23:05:054
62099130,9cyclictest780-21runrttasks19:10:024
62099130,9cyclictest7414-21runrttasks19:21:064
62099130,9cyclictest716-21snmpd20:05:084
62099130,9cyclictest6666-21/usr/sbin/munin20:30:174
62099130,9cyclictest4295-21sh22:45:044
62099130,9cyclictest4295-21sh22:45:034
62099130,9cyclictest29267-21sh22:05:144
62099130,9cyclictest26911-21taskset20:04:574
62099130,9cyclictest25551-21sh21:10:024
62099130,9cyclictest23292-21/usr/sbin/munin21:05:114
62099130,9cyclictest22064-21ssh22:00:274
62099130,9cyclictest21186-21rm23:30:014
62099130,9cyclictest20146-21/usr/sbin/munin19:50:134
62099130,9cyclictest19646-21mailstats20:55:214
62099130,9cyclictest17503-21cat19:45:124
62099130,9cyclictest16174-21aten2_r5power_p20:50:124
62099130,9cyclictest15882-21chrt23:45:014
62099130,9cyclictest13749-21/usr/sbin/munin20:45:224
62099130,9cyclictest1304-21ssh23:00:284
62099130,9cyclictest12432-21idleruntime21:20:144
62099130,9cyclictest1200-21runrttasks20:15:404
62099130,8cyclictest21498-21sshd21:00:154
62099130,10cyclictest14296-21sh23:25:244
62099130,10cyclictest14273-21sh22:50:164
62099130,10cyclictest12893-21cpu19:35:134
62099130,10cyclictest11207-21rm21:37:164
61999139,3cyclictest17190-21cron19:45:003
61999131,9cyclictest24980-21taskset21:07:253
61999131,9cyclictest10364-21vmstat23:05:273
61999130,9cyclictest8579-21ssh21:35:253
61999130,9cyclictest780-21runrttasks20:39:383
61999130,9cyclictest780-21runrttasks20:21:213
61999130,9cyclictest4419-21processes19:15:203
61999130,9cyclictest32458-21fschecks_time20:15:143
61999130,9cyclictest25482-21gltestperf20:00:143
61999130,9cyclictest23411-21/usr/sbin/munin00:25:143
61999130,9cyclictest19873-21/usr/sbin/munin00:05:123
61999130,9cyclictest17872-21fschecks_count19:45:143
61999130,9cyclictest16761-21runrttasks21:22:363
61999130,9cyclictest16119-21/usr/sbin/munin23:45:163
61999130,9cyclictest14132-21fschecks_count20:45:173
61999130,9cyclictest12769-21/usr/sbin/munin19:35:123
61999130,9cyclictest11965-21fw_packets20:40:123
61999130,10cyclictest8265-21unixbench_singl20:30:253
61999130,10cyclictest5643-21taskset23:20:383
61999130,10cyclictest5311-21ntp_kernel_pll_20:25:193
61999130,10cyclictest29050-21ssh22:40:163
61999130,10cyclictest29050-21ssh22:40:163
61999130,10cyclictest28634-21sh21:11:053
61999130,10cyclictest21144-21sh23:30:003
61999130,10cyclictest20116-21rm19:50:003
61999130,10cyclictest1591-21/usr/sbin/munin19:10:153
61999130,10cyclictest14059-21awk22:50:153
61699135,6cyclictest23779-21switchtime19:55:242
61699131,9cyclictest8084-21aten2_r5power_c19:25:112
61699131,9cyclictest30032-21ssh23:16:342
61699130,9cyclictest9363-21cstates20:35:132
61699130,9cyclictest900-21latency_hist19:10:012
61699130,9cyclictest780-21runrttasks20:26:462
61699130,9cyclictest6666-21/usr/sbin/munin20:30:262
61699130,9cyclictest5300-21ssh00:15:072
61699130,9cyclictest4361-21ssh22:45:062
61699130,9cyclictest4361-21ssh22:45:052
61699130,9cyclictest31924-21chrt20:15:092
61699130,9cyclictest29882-21/usr/sbin/munin21:30:142
61699130,9cyclictest29562-21runrttasks20:10:012
61699130,9cyclictest27225-21/usr/sbin/munin20:05:242
61699130,9cyclictest24872-21/usr/sbin/munin20:00:232
61699130,9cyclictest2306-21fschecks_count20:20:142
61699130,9cyclictest17183-21open_files20:50:182
61699130,9cyclictest1676-21sh23:55:102
61699130,9cyclictest14247-21perl20:45:162
61699130,9cyclictest13612-21/usr/sbin/munin22:50:252
61699130,9cyclictest10914-21/usr/sbin/munin00:00:192
61699130,9cyclictest10414-21/usr/sbin/munin19:30:152
61699130,8cyclictest23154-21kworker/u24:1+rpciod22:55:012
61699130,10cyclictest24720-21ssh21:45:052
61699130,10cyclictest18762-21ssh22:35:092
61699130,10cyclictest16876-21runrttasks19:41:242
61199135,6cyclictest13418-21expr19:35:151
61199131,9cyclictest8273-21sh22:10:441
61199130,9cyclictest8774-21if_enp6s019:25:151
61199130,9cyclictest780-21runrttasks23:00:061
61199130,9cyclictest780-21runrttasks20:20:081
61199130,9cyclictest780-21runrttasks19:55:581
61199130,9cyclictest779-21latency_hist19:10:011
61199130,9cyclictest6666-21/usr/sbin/munin20:30:131
61199130,9cyclictest6401-21basename19:20:171
61199130,9cyclictest32171-21cpuspeed_turbos20:15:121
61199130,9cyclictest29940-21entropy20:10:131
61199130,9cyclictest23520-21ssh21:25:581
61199130,9cyclictest20138-21chrt20:55:521
61199130,9cyclictest20035-21/usr/sbin/munin22:00:131
61199130,9cyclictest16598-21taskset20:50:141
61199130,9cyclictest15344-21cpuspeed19:40:131
61199130,9cyclictest13869-21sh00:01:081
61199130,9cyclictest12158-21sh23:25:011
61199130,9cyclictest11464-21ssh21:20:021
61199130,9cyclictest11175-21kernelversion19:30:181
61199130,10cyclictest3399-21cpu19:15:131
61199130,10cyclictest24121-21missed_timers21:05:161
60799135,6cyclictest20211-21taskset19:50:120
60799131,9cyclictest765-21latency_hist19:10:010
60799130,9cyclictest9324-21/usr/sbin/munin19:25:220
60799130,9cyclictest716-21snmpd21:04:580
60799130,9cyclictest716-21snmpd19:40:490
60799130,9cyclictest26422-21unixbench_multi20:00:230
60799130,9cyclictest23826-21switchtime19:55:240
60799130,9cyclictest11888-21perl20:40:130
60799130,7cyclictest174-1kworker/0:1H+kblockd20:08:580
60799130,6cyclictest32106-21/usr/sbin/munin23:00:130
60799130,10cyclictest9088-21basename20:35:120
60799130,10cyclictest3577-21chrt20:20:380
60799130,10cyclictest326-21systemd-journal20:29:520
60799130,10cyclictest23774-21sh22:20:020
60799130,10cyclictest2085-21df_inode23:55:120
60799130,10cyclictest19518-21ntp_offset20:55:180
60799130,10cyclictest18730-21grep19:45:200
60799130,10cyclictest15525-21taskset20:47:030
60799130,10cyclictest13888-21wc19:35:200
48299130,11tDemoTimingTask8751-21chrt00:16:265
3045399131,10tDemoTimingTask32743-21taskset19:07:055
1070999130,11tDemoTimingTask16725-21taskset21:58:065
992529120,9tAtEmLog_010410-21chrt19:30:105
623039120,0EcMasterDemoSyn0-21swapper/520:28:505
62099124,6cyclictest4537-21cpuspeed_turbos20:25:134
62099120,9cyclictest8059-21/usr/sbin/munin19:25:124
62099120,9cyclictest23147-21gltestperf19:55:164
62099120,9cyclictest12840-21sort20:40:214
61999127,4cyclictest27877-21sed20:05:163
61999121,8cyclictest23170-21hddtemp_smartct19:55:163
61999120,7cyclictest16089-21taskset20:50:033
61699120,9cyclictest21801-21vmstat19:50:272
61199124,6cyclictest1478-21fschecks_time19:10:141
61199120,8cyclictest12303-21missed_timers20:40:151
61199120,10cyclictest20646-21perl19:50:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional