You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-16 - 06:32
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot1.osadl.org (updated Mon Mar 16, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
10172930,1sleep00-21swapper/022:10:530
2980839920,0EcMasterDemoSyn2981829tAtEmLog_020:26:345
1772139920,0EcMasterDemoSyn1773129tAtEmLog_023:49:245
3198139910,0EcMasterDemoSyn3199129tAtEmLog_022:28:165
2205539840,0EcMasterDemoSyn2206529tAtEmLog_000:09:415
2705039830,0EcMasterDemoSyn2706029tAtEmLog_021:32:305
1768339830,0EcMasterDemoSyn1769329tAtEmLog_021:27:255
715839820,0EcMasterDemoSyn716829tAtEmLog_020:46:515
314439820,0EcMasterDemoSyn315429tAtEmLog_019:30:475
2317039820,0EcMasterDemoSyn2318129tAtEmLog_022:58:415
2274239820,0EcMasterDemoSyn2275229tAtEmLog_022:23:125
1353139820,0EcMasterDemoSyn1354129tAtEmLog_023:29:075
2608239810,0EcMasterDemoSyn2609229tAtEmLog_000:29:585
1292239810,0EcMasterDemoSyn1293229tAtEmLog_000:04:375
1187139810,0EcMasterDemoSyn1188129tAtEmLog_020:56:595
19722240,3sleep50-21swapper/523:40:165
23869991911,6cyclictest13562-21grep22:00:183
143932190,0sleep40-21swapper/422:18:464
2385699185,9cyclictest18684-21sh22:03:230
2385699180,14cyclictest14168-21sh23:46:540
2730729170,7tAtEmLog_027479-21taskset22:43:345
2387399171,12cyclictest16-1pr/legacy23:13:534
2387399171,12cyclictest16-1pr/legacy23:13:534
2386999170,9cyclictest26203-21sh21:50:013
2386499174,9cyclictest26557-21processes20:20:202
2386499170,9cyclictest27631-21ssh23:54:552
2385699175,9cyclictest25201-21runrttasks23:17:180
2385699170,9cyclictest9463-21hddtemp_smartct00:20:150
2385699170,9cyclictest29087-21sh22:26:160
2385699170,9cyclictest28067-21sh23:00:430
2385699170,9cyclictest23942-21sh00:10:160
2385699170,9cyclictest23209-21sh23:34:290
2385699170,9cyclictest17981-21sh21:45:150
2385699170,9cyclictest11761-21sh23:27:530
2385699170,7cyclictest8203-21cat00:19:590
2385699170,7cyclictest26544-21sh23:54:120
2385699170,14cyclictest21725-21sh00:09:280
2385699170,14cyclictest1493-21sh21:18:570
2254899170,10tDemoTimingTask22556-21chrt21:47:425
1322799170,13tDemoTimingTask13336-21chrt21:42:425
1024799170,10tDemoTimingTask10561-21chrt19:49:555
80432160,0sleep40-21swapper/421:57:254
2609229160,9tAtEmLog_02594-21chrt00:34:555
23869991612,3cyclictest15820-21needreboot21:05:163
2386499164,9cyclictest755-21postgres22:50:052
2386499164,8cyclictest716-21snmpd00:06:042
2386199168,6cyclictest3690-21tr00:35:131
23861991612,3cyclictest13525-21ssh22:35:211
2385699162,10cyclictest8134-21sh22:32:530
23856991612,3cyclictest27154-21runintdemo5min20:21:290
2385699160,8cyclictest780-21runrttasks22:45:040
2385699160,8cyclictest29355-21sh21:34:030
2385699160,8cyclictest18175-21sh22:38:290
2385699160,7cyclictest22730-21/usr/sbin/munin22:40:230
2385699160,6cyclictest174-1kworker/0:1H+kblockd22:08:510
2385699160,13cyclictest780-21runrttasks19:52:410
3107799150,9tDemoTimingTask31087-21taskset00:14:455
2387399152,9cyclictest17325-21latency_hist00:24:594
2387399151,10cyclictest7495-21sh23:43:294
2387399151,10cyclictest6294-21runrttasks00:18:454
2387399150,9cyclictest4705-21taskset20:40:394
2387399150,10cyclictest22688-21cat22:40:224
2386999153,8cyclictest3094-21grep00:35:033
2386999153,8cyclictest16980-21switchtime23:30:243
23869991513,2cyclictest13870-21kworker/3:2+events22:43:273
23869991511,3cyclictest20780-21cpuspeed_turbos23:15:113
2386999151,10cyclictest9952-21cat20:55:003
2386999150,9cyclictest780-21runrttasks19:50:273
2386999150,9cyclictest24293-21sh23:35:023
2386999150,8cyclictest29585-21ssh23:01:483
2386499153,9cyclictest657-21unixbench_singl19:25:252
2386499153,8cyclictest13604-21unixbench_singl22:35:222
2386499152,9cyclictest858-21ssh23:57:342
2386499152,9cyclictest27338-21ssh22:43:302
2386499152,8cyclictest16490-21kworker/u24:2+xprtiod21:46:142
23864991511,3cyclictest5851-21cron19:40:002
2386499151,10cyclictest780-21runrttasks19:55:312
2386499151,10cyclictest7523-21/usr/sbin/munin22:15:142
2386499151,10cyclictest6061-21runrttasks00:18:352
2386499150,8cyclictest27385-21sh23:18:512
2386499150,10cyclictest15170-21ssh21:43:582
2386199154,6cyclictest30512-21cut20:30:141
2386199154,6cyclictest10264-21seq21:23:381
2386199153,9cyclictest1297-21wget23:40:121
2386199151,9cyclictest3762-21rm23:23:381
2386199150,9cyclictest5495-21ssh21:55:381
2385699157,6cyclictest15278-21cron19:59:590
2385699156,7cyclictest7818-21ssh00:01:010
2385699156,7cyclictest4685-21sh21:38:040
2385699156,7cyclictest20180-21sh22:56:390
2385699154,8cyclictest9327-21latency19:45:170
2385699151,11cyclictest174-1kworker/0:1H+kblockd19:32:090
2385699151,10cyclictest9503-21ssh00:38:230
2385699150,13cyclictest590-21sendmail23:40:000
298362140,10sleep030053-21diskstats23:20:130
2387399141,9cyclictest16258-21sh22:37:114
2387399141,10cyclictest27232-21sh22:07:574
2387399141,10cyclictest20273-21runrttasks00:08:264
23873991410,3cyclictest15409-21gltestperf21:05:134
2387399140,9cyclictest8360-21ssh23:25:284
2387399140,9cyclictest23052-21/usr/sbin/munin00:10:174
2387399140,11cyclictest10776-21cut23:45:154
2387399140,10cyclictest9977-21users21:40:244
2387399140,10cyclictest9015-21ssh21:22:454
2387399140,10cyclictest780-21runrttasks20:22:304
2387399140,10cyclictest347-21switchtime19:25:244
2387399140,10cyclictest32761-21sh23:21:144
2387399140,10cyclictest32756-21ssh22:10:254
2387399140,10cyclictest27139-21fschecks_count21:15:134
2387399140,10cyclictest24471-21sh21:13:514
2387399140,10cyclictest21859-21ssh21:30:034
2387399140,10cyclictest21156-21ssh23:50:254
2387399140,10cyclictest20631-21sh21:46:254
2387399140,10cyclictest19808-21sh22:04:094
2387399140,10cyclictest15742-21df23:30:124
2387399140,10cyclictest13623-21idleruntime21:25:164
2386999140,8cyclictest21629-21/usr/sbin/munin20:10:233
2386999140,10cyclictest8656-21fschecks_time21:40:143
2386999140,10cyclictest716-21snmpd20:15:303
2386999140,10cyclictest6660-21fschecks_time19:40:143
2386999140,10cyclictest6528-21/usr/sbin/munin00:00:233
2386999140,10cyclictest569-21sh23:21:253
2386999140,10cyclictest4421-21sh22:48:343
2386999140,10cyclictest4288-21sh21:37:473
2386999140,10cyclictest3229-21/usr/sbin/munin21:55:143
2386999140,10cyclictest30267-21ssh23:55:273
2386999140,10cyclictest2893-21sshd00:16:233
2386999140,10cyclictest28827-21ssh21:33:403
2386999140,10cyclictest28199-21/usr/sbin/munin21:15:213
2386999140,10cyclictest28059-21apt-get20:25:123
2386999140,10cyclictest26428-21ssh22:07:243
2386999140,10cyclictest26138-21ssh00:11:123
2386999140,10cyclictest25402-21chrt20:20:113
2386999140,10cyclictest24014-21sh23:52:253
2386999140,10cyclictest21567-21ssh00:26:473
2386999140,10cyclictest2156-21rm23:05:033
2386999140,10cyclictest20562-21sh21:46:213
2386999140,10cyclictest19027-21sh21:28:203
2386999140,10cyclictest1785-21ssh22:29:463
2386999140,10cyclictest14876-21taskset19:55:243
2386999140,10cyclictest11698-21ssh22:16:573
2386999140,10cyclictest11558-21ssh22:35:023
2386999140,10cyclictest11552-21sh23:45:213
2386999140,10cyclictest11125-21sh21:24:123
2386999140,10cyclictest10850-21sh00:20:333
2386999140,10cyclictest10339-21sh23:26:513
2386499146,7cyclictest23851-21cyclictest00:00:122
2386499145,8cyclictest23851-21cyclictest22:10:442
2386499141,9cyclictest29787-21runrttasks20:26:252
2386499141,9cyclictest24120-21/usr/sbin/munin19:10:242
2386499141,9cyclictest22714-21sh22:23:112
2386499141,9cyclictest17676-21diskmemload22:55:192
2386499141,9cyclictest14749-21ssh23:11:232
2386499141,9cyclictest14749-21ssh23:11:222
2386499141,9cyclictest10193-21runrttasks19:45:342
2386499141,10cyclictest25025-21runrttasks20:19:232
2386499141,10cyclictest16677-21chrt21:07:302
2386499141,10cyclictest16-1pr/legacy20:46:502
2386499140,11cyclictest18495-21basename00:25:162
2386499140,10cyclictest9423-21runrttasks20:50:502
2386499140,10cyclictest780-21runrttasks21:51:282
2386499140,10cyclictest780-21runrttasks19:50:102
2386499140,10cyclictest780-21runrttasks19:21:492
2386499140,10cyclictest7331-21ssh00:36:502
2386499140,10cyclictest7078-21ssh21:21:272
2386499140,10cyclictest6073-21ssh23:25:002
2386499140,10cyclictest5410-21runrttasks21:38:342
2386499140,10cyclictest3496-21ssh23:23:272
2386499140,10cyclictest30723-21tr20:30:142
2386499140,10cyclictest30180-21sh22:45:032
2386499140,10cyclictest29447-21sh00:30:582
2386499140,10cyclictest29356-21ssh21:15:522
2386499140,10cyclictest29184-21sh23:01:312
2386499140,10cyclictest26093-21ssh22:25:112
2386499140,10cyclictest2595-21ssh22:30:092
2386499140,10cyclictest25396-21sh00:10:402
2386499140,10cyclictest2372-21/usr/sbin/munin23:40:172
2386499140,10cyclictest22481-21diskstats21:30:132
2386499140,10cyclictest21898-21runrttasks23:33:362
2386499140,10cyclictest21612-21/usr/sbin/munin22:05:112
2386499140,10cyclictest20166-21ssh21:10:582
2386499140,10cyclictest19720-21sleep20:06:202
2386499140,10cyclictest16186-21gltestperf20:00:152
2386499140,10cyclictest15870-21chrt21:26:102
2386499140,10cyclictest12459-21/usr/sbin/munin21:00:222
2386499140,10cyclictest12426-21ssh22:00:122
2386499140,10cyclictest12320-21ssh23:45:362
2386499140,10cyclictest11275-21ssh00:20:522
2386499140,10cyclictest10219-21chrt21:58:522
2386199146,6cyclictest8087-21sh20:50:131
2386199141,9cyclictest5051-21/usr/sbin/munin00:00:231
2386199141,9cyclictest3030-21/usr/sbin/munin20:40:171
2386199141,9cyclictest25403-21/usr/sbin/munin20:20:151
2386199141,9cyclictest23002-21runrttasks00:10:081
2386199141,9cyclictest17296-21if_enp6s022:55:161
2386199141,9cyclictest16435-21ssh00:05:411
23861991411,2cyclictest16532-21head23:30:181
2386199140,9cyclictest29697-21ssh21:51:191
2386199140,9cyclictest29148-21/usr/sbin/munin19:20:171
2386199140,9cyclictest10175-21/usr/sbin/munin23:45:121
2386199140,10cyclictest969-21wc00:15:201
2386199140,10cyclictest9070-21ssh23:25:571
2386199140,10cyclictest780-21runrttasks20:55:271
2386199140,10cyclictest6393-21ssh22:14:351
2386199140,10cyclictest5619-21taskset19:36:561
2386199140,10cyclictest32762-21/usr/sbin/munin20:35:131
2386199140,10cyclictest31998-21ssh00:32:461
2386199140,10cyclictest3170-21ssh22:47:421
2386199140,10cyclictest31622-21grep19:25:121
2386199140,10cyclictest29442-21ssh22:26:291
2386199140,10cyclictest28507-21ssh21:33:271
2386199140,10cyclictest24843-21sh23:17:031
2386199140,10cyclictest23782-21ssh22:05:331
2386199140,10cyclictest23112-21ssh00:27:531
2386199140,10cyclictest23000-21ssh21:12:501
2386199140,10cyclictest21816-21irqstats22:40:161
2386199140,10cyclictest20443-21ssh23:50:181
2386199140,10cyclictest20348-21rm22:21:351
2386199140,10cyclictest20312-21/usr/sbin/munin20:10:151
2386199140,10cyclictest17112-21ssh21:45:061
2386199140,10cyclictest16827-21ssh22:02:081
2386199140,10cyclictest16246-21ssh21:26:261
2386199140,10cyclictest14277-21sh21:43:201
2386199140,10cyclictest13229-21/usr/sbin/munin19:55:101
2386199140,10cyclictest13208-21sh22:53:201
2386199140,10cyclictest11410-21grep22:35:011
2386199140,10cyclictest1122-21ssh23:04:331
2386199140,10cyclictest11121-21sh00:20:451
2386199140,10cyclictest10878-21/usr/sbin/munin19:50:201
2385699146,6cyclictest2758-21df23:05:150
2385699145,7cyclictest13432-21sh21:42:450
2385699143,8cyclictest30809-21runrttasks19:20:280
2385699141,9cyclictest29943-21runrttasks20:27:540
2385699141,11cyclictest32766-21apt-get20:35:110
2385699141,10cyclictest19982-21sh00:25:390
2385699140,9cyclictest16512-21sh22:20:010
2385699140,7cyclictest17118-21wc21:10:010
2385699140,6cyclictest174-1kworker/0:1H+kblockd20:51:100
2385699140,12cyclictest26949-21ssh00:30:120
2385699140,12cyclictest18140-21grep21:10:150
2385699140,12cyclictest12882-21chrt22:53:080
2385699140,12cyclictest10441-21sh21:23:450
2385699140,10cyclictest6890-21sh21:56:360
2385699140,10cyclictest30561-21sh23:55:390
2385699140,10cyclictest17432-21sh23:13:170
2385699140,10cyclictest17432-21sh23:13:160
2385699140,10cyclictest15600-21ssh22:19:380
2385699140,10cyclictest14913-21sh21:25:300
1798729140,6tAtEmLog_025373-21taskset22:06:405
3238929130,3tAtEmLog_08770-21taskset23:08:335
2387399135,6cyclictest9257-21if_err_enp6s019:45:154
2387399135,6cyclictest17025-21tune2fs22:55:144
2387399135,6cyclictest16316-21cat20:00:174
2387399133,8cyclictest12468-21/usr/sbin/munin21:00:104
2387399130,9cyclictest828-21fschecks_time20:35:154
2387399130,9cyclictest780-21runrttasks19:55:554
2387399130,9cyclictest7737-21/usr/sbin/munin20:50:184
2387399130,9cyclictest744-21postgres19:39:584
2387399130,9cyclictest4893-21ssh00:00:064
2387399130,9cyclictest3863-21ssh22:30:194
2387399130,9cyclictest32296-21vmstat22:45:234
2387399130,9cyclictest29611-21ssh22:26:374
2387399130,9cyclictest29426-21sort20:25:244
2387399130,9cyclictest29262-21aten2_r5power_v19:20:104
2387399130,9cyclictest28137-21chrt23:55:054
2387399130,9cyclictest26772-21/usr/sbin/munin19:15:124
2387399130,9cyclictest26617-21sh00:30:074
2387399130,9cyclictest26459-21ssh23:00:164
2387399130,9cyclictest26403-21sed19:15:004
2387399130,9cyclictest26196-21sh21:50:014
2387399130,9cyclictest25322-21sh23:35:184
2387399130,9cyclictest24772-21runintdemo5min20:16:254
2387399130,9cyclictest2356-21/usr/sbin/munin23:05:204
2387399130,9cyclictest21953-21/usr/sbin/munin20:10:244
2387399130,9cyclictest20482-21/usr/sbin/munin23:15:224
2387399130,9cyclictest20234-21sh00:25:504
2387399130,9cyclictest17960-21/usr/sbin/munin20:05:204
2387399130,9cyclictest16520-21ssh22:20:034
2387399130,9cyclictest10951-21aten2_r5power_p19:50:124
2387399130,9cyclictest10086-21/usr/sbin/munin20:55:214
2387399130,10cyclictest8303-21ntpq22:50:194
2387399130,10cyclictest780-21runrttasks19:42:104
2387399130,10cyclictest6510-21sh00:36:144
2387399130,10cyclictest5918-21fschecks_time20:45:154
2387399130,10cyclictest32165-21/usr/sbin/munin21:35:164
2387399130,10cyclictest31484-21open_files20:30:204
2387399130,10cyclictest1818-21cat19:30:134
2386999139,3cyclictest27728-21missed_timers00:30:173
2386999135,6cyclictest14756-21grep21:05:013
2386999131,9cyclictest14267-21cpuspeed00:05:123
2386999130,9cyclictest8526-21/usr/sbin/munin19:45:163
2386999130,9cyclictest780-21runrttasks19:17:153
2386999130,9cyclictest780-21runrttasks19:10:213
2386999130,9cyclictest5386-21/usr/sbin/munin20:45:203
2386999130,9cyclictest31501-21/usr/sbin/munin19:25:143
2386999130,9cyclictest30777-21/usr/sbin/munin22:10:243
2386999130,9cyclictest23965-21latency_hist19:10:003
2386999130,9cyclictest2013-21taskset20:35:263
2386999130,9cyclictest16480-21ssh22:20:013
2386999130,9cyclictest15605-21runrttasks20:00:093
2386999130,9cyclictest1449-21/usr/sbin/munin19:30:253
2386999130,9cyclictest11406-21/usr/sbin/munin23:10:203
2386999130,9cyclictest11406-21/usr/sbin/munin23:10:203
2386999130,9cyclictest10148-21grep20:55:103
2386999130,7cyclictest2845-21sort20:40:003
2386999130,10cyclictest7160-21basename22:50:123
2386999130,10cyclictest3528-21interrupts22:30:173
2386999130,10cyclictest19774-21ssh21:10:423
2386999130,10cyclictest18488-21ssh22:55:273
2386999130,10cyclictest18431-21fschecks_time20:05:153
2386999130,10cyclictest1131-21ssh23:40:083
2386499134,7cyclictest10447-21/usr/sbin/munin20:55:122
2386499130,9cyclictest7559-21timerandwakeup19:40:222
2386499130,9cyclictest3112-21taskset19:30:302
2386499130,9cyclictest3030-21/usr/sbin/munin20:40:112
2386499130,9cyclictest24366-21ssh23:35:052
2386499130,9cyclictest2341-21taskset23:05:112
2386499130,9cyclictest22128-21runintdemo5min20:11:232
2386499130,10cyclictest28710-21runrttasks19:18:322
2386499130,10cyclictest2172-21taskset20:36:542
2386199130,9cyclictest8567-21aten2_r5power_e19:45:111
2386199130,9cyclictest780-21runrttasks19:40:491
2386199130,9cyclictest7495-21ssh22:15:101
2386199130,9cyclictest5387-21/usr/sbin/munin20:45:121
2386199130,9cyclictest28273-21/usr/sbin/munin23:55:171
2386199130,9cyclictest27925-21sed20:25:001
2386199130,9cyclictest27652-21latency21:15:151
2386199130,9cyclictest26984-21cpuspeed_turbos19:15:121
2386199130,9cyclictest24541-21/usr/sbin/munin23:35:251
2386199130,9cyclictest2209-21kernelversion19:30:161
2386199130,9cyclictest2123-21sh23:05:031
2386199130,9cyclictest18548-21grep20:05:151
2386199130,9cyclictest15927-21cpuspeed_turbos20:00:151
2386199130,9cyclictest14820-21apt-get21:05:101
2386199130,9cyclictest1380-21ssh21:35:481
2386199130,9cyclictest12579-21cpu21:00:101
2386199130,9cyclictest11245-21ssh23:10:041
2386199130,9cyclictest11245-21ssh23:10:031
2386199130,10cyclictest24920-21runrttasks20:17:581
2385699132,8cyclictest20451-21cpu20:10:130
2385699130,9cyclictest780-21runrttasks20:40:330
2385699130,9cyclictest780-21runrttasks20:01:220
2385699130,9cyclictest6179-21apt-get19:40:100
2385699130,9cyclictest27390-21gltestperf19:15:140
2385699130,9cyclictest23879-21cat19:10:000
2385699130,9cyclictest23377-21df20:15:140
2385699130,9cyclictest19779-21taskset20:06:590
2385699130,9cyclictest11153-21/usr/sbin/munin20:55:170
2385699130,7cyclictest174-1kworker/0:1H+kblockd21:01:450
2385699130,7cyclictest174-1kworker/0:1H+kblockd20:46:040
2385699130,7cyclictest174-1kworker/0:1H+kblockd19:10:010
2385699130,11cyclictest1705-21fschecks_time23:40:140
2385699130,10cyclictest31492-21taskset19:25:030
2385699130,10cyclictest31237-21latency20:30:180
2385699130,10cyclictest26256-21sh21:50:030
2385699130,10cyclictest174-1kworker/0:1H+kblockd19:35:110
2318299130,11tDemoTimingTask31206-21chrt23:02:575
2207029130,3tAtEmLog_024830-21taskset21:14:045
917039120,2EcMasterDemoSyn15135-21chrt23:11:405
917039120,2EcMasterDemoSyn15135-21chrt23:11:405
77542120,4sleep50-21swapper/520:50:135
3179639125,0EcMasterDemoSyn0-21swapper/521:57:495
3107799120,10tDemoTimingTask5072-21taskset00:17:535
2730899120,10tDemoTimingTask3590-21taskset22:48:005
2386999122,7cyclictest30406-21smartctl19:20:213
2386999120,9cyclictest5027-21mailstats19:35:193
2386999120,8cyclictest286-21jbd2/md0-820:30:023
2386199120,8cyclictest24118-21runrttasks19:10:101
2343199120,10tDemoTimingTask23491-21taskset19:05:525
2211199120,10tDemoTimingTask24702-21taskset20:15:415
1662199120,10tDemoTimingTask16701-21chrt21:07:515
836139110,0EcMasterDemoSyn0-21swapper/521:22:205
77139110,0EcMasterDemoSyn0-21swapper/519:25:435
550239110,0EcMasterDemoSyn671rcuc/519:35:515
437039110,0EcMasterDemoSyn0-21swapper/523:24:035
415429110,1tAtEmLog_00-21swapper/522:17:295
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional