You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-11 - 11:02
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot1.osadl.org (updated Wed Mar 11, 2026 00:44:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2111139890,1EcMasterDemoSyn21120-21runintdemo5min00:14:233
288539840,0EcMasterDemoSyn2894-21runintdemo5min19:45:380
429739830,0EcMasterDemoSyn430729tAtEmLog_022:53:165
3171539830,0EcMasterDemoSyn3172529tAtEmLog_023:08:285
1859739830,0EcMasterDemoSyn1860729tAtEmLog_022:43:075
1343439830,0EcMasterDemoSyn1344429tAtEmLog_022:58:205
957539820,0EcMasterDemoSyn958529tAtEmLog_021:27:045
945039820,0EcMasterDemoSyn946029tAtEmLog_022:38:045
2828939820,0EcMasterDemoSyn2829929tAtEmLog_021:37:125
2539439820,0EcMasterDemoSyn2540429tAtEmLog_019:25:215
2377739820,0EcMasterDemoSyn2378729tAtEmLog_021:52:245
2210239820,0EcMasterDemoSyn2211229tAtEmLog_020:26:135
1413039820,0EcMasterDemoSyn1413929tAtEmLog_021:11:505
2252239810,0EcMasterDemoSyn2253229tAtEmLog_019:20:185
2111139810,0EcMasterDemoSyn2112129tAtEmLog_000:14:245
53532610,1sleep20-21swapper/221:42:142
2112129290,0tAtEmLog_00-21swapper/500:19:275
1649699225,12cyclictest326-21systemd-journal20:34:543
270339190,1EcMasterDemoSyn2710-21grep00:39:452
1649699194,8cyclictest3195-21sh22:35:003
1649699190,7cyclictest26831-21sh21:54:333
1649699190,7cyclictest25047-21awk23:40:183
1649699190,7cyclictest11431-21sh00:08:553
1649699180,7cyclictest27078-21sh21:36:233
1649699180,7cyclictest24443-21ssh22:28:323
1649699180,7cyclictest18992-21sh22:07:273
1647999180,14cyclictest14073-21sh23:52:150
761299170,7tDemoTimingTask7632-21chrt19:55:495
2351629170,7tAtEmLog_029220-21taskset22:30:335
1649699174,8cyclictest481-21sh23:09:193
1649699173,9cyclictest19643-21sed23:20:013
1649699172,11cyclictest9975-21taskset20:00:513
1649699171,7cyclictest18899-21sh00:30:163
1649699171,14cyclictest22345-21taskset20:29:003
1649699171,10cyclictest27236-21cat00:00:003
1649699170,8cyclictest21828-21cat23:20:243
1649699170,7cyclictest9536-21diskmemload23:14:163
1649699170,7cyclictest29826-21grep21:55:243
1649699170,7cyclictest2219-21runrttasks00:20:593
1649699170,7cyclictest14294-21ssh23:35:003
1649699170,6cyclictest5711-21ssh22:35:253
1649699170,6cyclictest2910-21runintdemo5min19:45:413
1649699170,4cyclictest20382-21ssh23:01:513
1649699170,14cyclictest20017-21head21:15:163
1648599171,12cyclictest16-1pr/legacy22:48:111
1647999171,6cyclictest26543-21awk00:35:000
1647999171,14cyclictest2938-21/usr/sbin/munin23:10:160
1647999170,9cyclictest9163-21/usr/sbin/munin00:25:130
1647999170,9cyclictest31109-21ssh21:20:540
1647999170,9cyclictest15650-21atemtest20:15:120
1647999170,9cyclictest1369-21fschecks_time21:40:140
1647999170,14cyclictest22519-21sh21:34:420
1647999170,13cyclictest780-21runrttasks20:53:190
3216499160,9tDemoTimingTask32744-21chrt20:50:015
1737529160,9tAtEmLog_017380-21taskset20:16:065
1650199160,9cyclictest17169-21ssh21:13:564
1649699167,7cyclictest11691-21tail21:45:213
1649699165,9cyclictest6038-21sed22:00:163
1649699161,7cyclictest9719-21ssh22:20:163
1649699161,7cyclictest9538-21latency_hist21:10:013
1649699161,7cyclictest9538-21latency_hist21:10:003
1649699161,7cyclictest780-21runrttasks23:25:033
1649699161,7cyclictest32187-21ssh21:21:383
1649699161,7cyclictest31053-21ssh00:36:513
1649699161,7cyclictest30934-21ssh00:19:443
1649699161,7cyclictest22432-21fschecks_time22:45:133
1649699161,7cyclictest13757-21ssh00:27:343
1649699161,7cyclictest1275-21fschecks_count21:40:153
1649699160,8cyclictest9345-21ssh22:55:273
1649699160,8cyclictest7522-21sed19:55:233
1649699160,8cyclictest4501-21ps20:56:363
1649699160,8cyclictest3817-21ssh22:52:553
1649699160,8cyclictest326-21systemd-journal21:11:193
1649699160,8cyclictest2872-21ssh22:16:103
1649699160,8cyclictest28640-21wc20:40:193
1649699160,8cyclictest27915-21sort00:00:133
1649699160,8cyclictest27064-21grep19:30:143
1649699160,8cyclictest2674-21latency_hist20:55:013
1649699160,8cyclictest20199-21ssh21:33:043
1649699160,8cyclictest17677-21cat19:10:183
1649699160,8cyclictest17586-21chrt23:35:593
1649699160,8cyclictest13183-21sh22:40:123
1649699160,8cyclictest10245-21head23:50:143
1649699160,7cyclictest29232-21sh22:13:163
1649699160,7cyclictest17202-21grep20:15:243
1649699160,7cyclictest14610-21sort20:10:243
1649699160,5cyclictest9536-21diskmemload23:47:323
1649699160,3cyclictest18580-21taskset20:20:143
1649299167,4cyclictest3541-21ssh21:24:112
1649299165,8cyclictest716-21snmpd19:23:112
16492991614,2cyclictest9040-21kworker/2:2+events21:06:462
16492991614,2cyclictest9040-21kworker/2:2+events21:06:452
16492991612,3cyclictest7956-21cron20:00:002
1649299160,9cyclictest19090-21/usr/sbin/munin19:15:192
1648599167,8cyclictest16470-21cyclictest22:02:431
1648599164,7cyclictest648-21dbus-daemon19:55:001
1648599160,8cyclictest286-21jbd2/md0-819:55:071
1648599160,10cyclictest31579-21chrt23:44:281
1647999165,8cyclictest6951-21grep22:00:240
1647999164,8cyclictest18555-21seq21:14:550
1647999163,9cyclictest29626-21sh22:13:330
1647999163,10cyclictest22040-21sh23:20:250
1647999160,9cyclictest4389-21sh23:29:230
1647999160,9cyclictest13902-21sh22:58:390
1647999160,8cyclictest17299-21sh23:35:470
1259499160,9tDemoTimingTask5245-21sleep23:34:565
925139150,3EcMasterDemoSyn651irq_work/521:06:475
925139150,3EcMasterDemoSyn651irq_work/521:06:465
1650199159,5cyclictest24590-21grep00:15:254
16501991511,3cyclictest648-21dbus-daemon21:05:004
16501991511,3cyclictest645-21avahi-daemon20:30:004
1650199150,9cyclictest21337-21ssh23:56:004
1650199150,9cyclictest20399-21smartctl19:15:214
1649699151,7cyclictest31534-21cpuspeed_turbos19:40:123
1649699150,7cyclictest5249-21cpu21:00:103
1649699150,7cyclictest5118-21sh21:25:053
1649699150,7cyclictest3899-21cstates19:50:133
1649699150,7cyclictest30256-21cmp20:45:003
1649699150,7cyclictest29838-21needreboot19:35:183
1649699150,7cyclictest25092-21dpkg20:35:023
1649699150,7cyclictest24228-21basename19:25:113
1649699150,7cyclictest21758-21taskset19:20:033
1649699150,7cyclictest19345-21grep19:15:123
1649699150,7cyclictest16503-21cat19:10:013
1649699150,7cyclictest11147-21fschecks_time20:05:143
1649299155,8cyclictest32335-21/usr/sbin/munin00:20:152
1649299154,8cyclictest326-21systemd-journal19:31:112
1649299154,7cyclictest41-21ksoftirqd/221:45:112
1649299154,7cyclictest31040-21latency_hist19:40:002
1649299153,8cyclictest19127-21uname22:25:172
1649299153,7cyclictest344-21aten2_r5power_e20:50:132
16492991512,2cyclictest9040-21kworker/2:2+events22:48:112
1649299151,10cyclictest20972-21kernelversion23:20:162
1649299150,9cyclictest716-21snmpd22:17:562
1649299150,8cyclictest41-21ksoftirqd/223:12:342
1648599159,5cyclictest30892-21sh19:40:001
1648599157,6cyclictest23358-21cut20:30:151
1648599153,9cyclictest648-21dbus-daemon19:31:451
1648599151,10cyclictest350-21cut20:50:131
1648599151,10cyclictest29792-21sh21:38:151
1648599151,10cyclictest18432-21diskmemload22:07:041
1648599151,10cyclictest15225-21runrttasks20:13:341
1648599150,11cyclictest27801-21ssh23:05:431
1648599150,11cyclictest12778-21/usr/sbin/munin22:40:141
1648599150,10cyclictest28952-21ntp_offset00:00:201
1647999156,7cyclictest32459-21sh22:32:510
1647999156,7cyclictest28015-21cat21:20:000
1647999156,7cyclictest27123-21chrt23:41:200
1647999153,9cyclictest21574-21sh23:02:410
1647999150,9cyclictest780-21runrttasks21:57:170
1647999150,12cyclictest174-1kworker/0:1H+kblockd20:10:350
1008139152,11EcMasterDemoSyn10805-21chrt22:03:025
2172229140,2tAtEmLog_0237792sleep523:40:095
1664729141,11tAtEmLog_019247-21chrt00:30:175
1650199148,3cyclictest2836239EcMasterDemoSyn22:12:414
1650199144,7cyclictest3380-21wc23:10:204
1650199143,8cyclictest1853-21cat19:45:164
1650199142,9cyclictest4047-21sh00:22:174
1650199142,9cyclictest26147-21sh21:18:484
1650199142,8cyclictest22637-21if_err_enp6s022:45:154
1650199141,9cyclictest32651-21ssh23:45:004
1650199141,9cyclictest15874-21sh00:10:404
1650199141,9cyclictest14714-21sh23:52:434
16501991410,3cyclictest5499-21cpuspeed_turbos21:25:124
1650199140,9cyclictest8838-21sh23:31:114
1650199140,11cyclictest2438-21ssh22:52:004
1650199140,10cyclictest8650-21ssh22:01:334
1650199140,10cyclictest780-21runrttasks19:10:214
1650199140,10cyclictest780-21runrttasks00:33:214
1650199140,10cyclictest5896-21runrttasks19:55:034
1650199140,10cyclictest3645-21ssh22:35:074
1650199140,10cyclictest30809-21ssh21:55:594
1650199140,10cyclictest29556-21sh22:30:484
1650199140,10cyclictest26466-21ssh23:40:534
1650199140,10cyclictest23521-21chrt19:21:014
1650199140,10cyclictest23397-21hddtemp_smartct20:30:164
1650199140,10cyclictest23341-21latency_hist21:35:014
1650199140,10cyclictest17922-21taskset22:25:064
1650199140,10cyclictest14381-21ssh00:28:004
1649299149,2cyclictest171rcu_preempt22:36:002
1649299149,2cyclictest171rcu_preempt00:13:232
1649299147,6cyclictest16470-21cyclictest21:18:582
1649299146,5cyclictest391rcuc/222:56:112
1649299146,4cyclictest716-21snmpd20:19:332
1649299145,3cyclictest11399-21uname20:05:172
1649299144,6cyclictest41-21ksoftirqd/223:51:432
1649299144,6cyclictest26422-21cat23:05:182
1649299144,6cyclictest16042-21rm23:17:332
1649299144,4cyclictest28897-21sh23:25:022
1649299143,8cyclictest18753-21cpu23:55:122
1649299143,7cyclictest12953-21ssh21:11:032
1649299141,9cyclictest7693-21ssh21:25:462
1649299141,9cyclictest7390-21ssh00:06:032
1649299141,9cyclictest3553-21/usr/sbin/munin19:50:162
1649299141,9cyclictest32061-21ssh00:02:092
16492991410,3cyclictest19210-21needreboot00:30:172
16492991410,3cyclictest17527-21cut23:00:172
1649299140,9cyclictest41-21ksoftirqd/220:10:482
1649299140,7cyclictest23484-21irqstats20:30:172
1649299140,10cyclictest9926-21users20:00:262
1649299140,10cyclictest780-21runrttasks21:03:432
1649299140,10cyclictest703-21taskset19:43:322
1649299140,10cyclictest4272-21runrttasks20:55:242
1649299140,10cyclictest30316-21ssh21:38:372
1649299140,10cyclictest30137-21sh21:55:312
1649299140,10cyclictest24136-21ssh21:52:392
1649299140,10cyclictest23865-21load00:15:182
1649299140,10cyclictest23547-21expr22:10:122
1649299140,10cyclictest20446-21ssh22:08:282
1649299140,10cyclictest19582-21sh21:32:382
1649299140,10cyclictest16901-21ssh00:29:472
1649299140,10cyclictest15724-21uname23:35:172
1649299140,10cyclictest13488-21ssh22:40:142
1649299140,10cyclictest12926-21ssh23:34:022
1649299140,10cyclictest11126-21ssh22:20:342
1648599141,9cyclictest32697-21ssh23:45:031
1648599141,9cyclictest1958-21runrttasks22:15:331
1648599141,6cyclictest2632850cat19:30:001
1648599141,10cyclictest7555-21sh23:12:561
1648599140,7cyclictest16996-21sh21:30:511
1648599140,5cyclictest1530750kworker/u24:0+flush-9:021:58:301
1648599140,11cyclictest29313-21ssh00:35:371
1648599140,10cyclictest9834-21/usr/sbin/munin21:45:231
1648599140,10cyclictest9680-21timerandwakeup20:00:241
1648599140,10cyclictest9444-21ssh23:31:381
1648599140,10cyclictest9199-21runrttasks21:06:121
1648599140,10cyclictest9199-21runrttasks21:06:121
1648599140,10cyclictest8940-21sh22:20:091
1648599140,10cyclictest780-21runrttasks19:45:091
1648599140,10cyclictest7191-21ssh22:55:051
1648599140,10cyclictest6399-21ssh22:35:531
1648599140,10cyclictest5441-21sh21:42:171
1648599140,10cyclictest4488-21seq20:56:261
1648599140,10cyclictest31735-21sh00:20:041
1648599140,10cyclictest31605-21ssh23:25:471
1648599140,10cyclictest31204-21/usr/sbin/munin19:40:121
1648599140,10cyclictest30768-21/usr/sbin/munin22:50:221
1648599140,10cyclictest28422-21/usr/sbin/munin21:20:121
1648599140,10cyclictest26163-21ssh00:16:241
1648599140,10cyclictest22942-21runrttasks22:10:001
1648599140,10cyclictest21909-21ssh22:26:471
1648599140,10cyclictest21401-21ssh23:56:021
1648599140,10cyclictest21297-21tr23:20:181
1648599140,10cyclictest21087-21ssh21:50:331
1648599140,10cyclictest20773-21ssh00:30:571
1648599140,10cyclictest20376-21/usr/sbin/munin20:25:161
1648599140,10cyclictest19913-21perl21:15:151
1648599140,10cyclictest19090-21/usr/sbin/munin19:15:241
1648599140,10cyclictest18025-21/usr/sbin/munin20:20:171
1648599140,10cyclictest17871-21ssh23:00:201
1648599140,10cyclictest17155-21ssh21:13:561
1648599140,10cyclictest16480-21ssh00:29:311
1648599140,10cyclictest15641-21/usr/sbin/munin20:15:241
1648599140,10cyclictest14847-21/usr/sbin/munin23:35:161
1648599140,10cyclictest13530-21ssh00:10:101
1648599140,10cyclictest13069-21ssh23:51:321
1648599140,10cyclictest11093-21/usr/sbin/munin23:15:111
1648599140,10cyclictest10635-21/usr/sbin/munin20:05:191
1648599140,10cyclictest10613-21sh00:08:201
1647999146,6cyclictest4994-21ssh22:17:390
1647999146,6cyclictest326-21systemd-journal20:32:180
1647999146,6cyclictest22672-21ssh21:51:390
1647999146,6cyclictest10504-21grep21:45:140
1647999145,7cyclictest23116-21sh22:27:390
1647999141,9cyclictest7848-21sh23:30:290
1647999141,9cyclictest780-21runrttasks21:08:470
1647999141,9cyclictest780-21runrttasks21:08:460
1647999141,9cyclictest4650-21runrttasks20:58:000
1647999141,11cyclictest13739-21/usr/sbin/munin22:40:160
1647999141,10cyclictest7198-21sh21:25:240
1647999140,7cyclictest7159-21python319:55:210
1647999140,7cyclictest174-1kworker/0:1H+kblockd20:49:140
1647999140,7cyclictest14830-21cut22:05:130
1647999140,6cyclictest174-1kworker/0:1H+kblockd19:28:320
1647999140,13cyclictest28023-21ssh23:05:530
1647999140,12cyclictest28127-21sh00:00:140
1647999140,12cyclictest174-1kworker/0:1H+kblockd19:13:130
1647999140,11cyclictest29891-21/usr/sbin/munin19:35:190
1647999140,10cyclictest5906-21sh00:23:360
1647999140,10cyclictest32161-21ssh22:50:230
1647999140,10cyclictest28303-21taskset19:32:000
1647999140,10cyclictest27867-21sed00:35:160
1647999140,10cyclictest26141-21sh22:47:040
1647999140,10cyclictest25939-21diskmemload21:35:340
1647999140,10cyclictest25780-21sh00:16:070
1647999140,10cyclictest21066-21taskset19:19:030
1647999140,10cyclictest20365-21grep23:55:250
1647999140,10cyclictest19526-21sh00:13:160
1647999140,10cyclictest13603-21rm23:15:490
1647999140,10cyclictest12103-21sh00:09:220
1414199140,12tDemoTimingTask23353-21chrt21:16:545
761299130,5tDemoTimingTask9948-21taskset20:00:335
2815729130,2tAtEmLog_028531-21turbostat19:35:005
1650199139,3cyclictest26174-21gltestperf23:05:154
1650199135,6cyclictest7975-21tune2fs21:05:154
1650199135,6cyclictest7975-21tune2fs21:05:144
1650199135,6cyclictest17171-21basename23:00:154
1650199132,8cyclictest7399-21aten2_r5power_c22:55:144
1650199131,9cyclictest29520-21sh23:25:144
1650199131,7cyclictest27144-21/usr/sbin/munin19:30:144
1650199130,9cyclictest9826-21irqstats22:20:174
1650199130,9cyclictest848-21cron23:17:004
1650199130,9cyclictest608-21runrttasks19:42:294
1650199130,9cyclictest3546-21taskset19:50:054
1650199130,9cyclictest31442-21ssh21:21:074
1650199130,9cyclictest30405-21/usr/sbin/munin20:45:194
1650199130,9cyclictest2988-21cpuspeed_turbos20:55:134
1650199130,9cyclictest28821-21runrttasks19:35:024
1650199130,9cyclictest27761-21/usr/sbin/munin20:40:174
1650199130,9cyclictest27584-21ssh21:36:434
1650199130,9cyclictest27077-21/usr/sbin/munin00:35:214
1650199130,9cyclictest25358-21rm20:35:014
1650199130,9cyclictest20509-21df_inode23:20:124
1650199130,9cyclictest1951-21/usr/sbin/munin21:40:194
1650199130,9cyclictest1866-21ssh22:15:294
1650199130,9cyclictest18050-21aten2_r5power_c20:20:124
1650199130,9cyclictest18031-21sh22:06:474
1650199130,9cyclictest15641-21/usr/sbin/munin20:15:154
1650199130,9cyclictest15481-21cut20:15:004
1650199130,9cyclictest14634-21sh23:35:034
1650199130,9cyclictest12617-21ssh22:40:024
1650199130,9cyclictest12469-21ssh21:45:514
1650199130,9cyclictest11867-21grep20:05:204
1650199130,10cyclictest9634-21sed20:00:244
1650199130,10cyclictest4303-21ssh00:05:014
1650199130,10cyclictest27695-21ssh00:00:114
1650199130,10cyclictest22301-21ssh21:51:244
1649299135,4cyclictest25408-21/usr/sbin/munin19:25:212
1649299134,7cyclictest24385-21fschecks_count23:40:142
1649299132,8cyclictest28654-21missed_timers20:40:182
1649299131,8cyclictest41-21ksoftirqd/219:14:092
1649299131,8cyclictest31248-21grep22:50:152
1649299130,9cyclictest780-21runrttasks20:35:082
1649299130,9cyclictest3188-21chrt19:49:162
1649299130,9cyclictest18700-21runrttasks20:20:162
1649299130,10cyclictest4835-21ssh22:00:022
1649299130,10cyclictest405-21/usr/sbin/munin23:45:162
1649299130,10cyclictest31070-21chrt20:45:142
1649299130,10cyclictest27742-21expr22:30:142
1649299130,10cyclictest21942-21grep20:25:222
1648599130,9cyclictest780-21runrttasks19:22:151
1648599130,9cyclictest695-21rs:main1
1648599130,9cyclictest5238-21/usr/sbin/munin21:25:141
1648599130,9cyclictest31065-21cut20:45:151
1648599130,9cyclictest29316-21ssh22:30:381
1648599130,9cyclictest25590-21cpuspeed_turbos20:35:141
1648599130,10cyclictest5463-21ls21:00:121
1648599130,10cyclictest29445-21runrttasks20:40:481
1647999131,9cyclictest5199-21runrttasks22:35:190
1647999130,9cyclictest8655-21forks20:00:140
1647999130,9cyclictest780-21runrttasks20:26:490
1647999130,9cyclictest6429-21sed21:00:220
1647999130,9cyclictest5411-21taskset19:52:280
1647999130,9cyclictest31562-21df_inode19:40:140
1647999130,9cyclictest27978-21/usr/sbin/munin20:40:120
1647999130,9cyclictest2654-21ssh23:45:320
1647999130,9cyclictest11299-21hddtemp_smartct20:05:160
1647999130,7cyclictest174-1kworker/0:1H+kblockd19:22:300
1647999130,11cyclictest10705-21timerwakeupswit22:20:240
1647999130,10cyclictest25266-21gpgv20:35:020
1647999130,10cyclictest19240-21awk20:20:200
1439299130,11tDemoTimingTask16199-21chrt19:06:595
525299120,10tDemoTimingTask5334-21taskset19:51:255
2447499120,10tDemoTimingTask24675-21taskset20:33:395
1673899121,9tDemoTimingTask22010-21chrt23:56:295
1650199120,9cyclictest24830-21idleruntime19:25:154
1650199120,7cyclictest836-21tune2fs20:50:164
1200039120,0EcMasterDemoSyn0-21swapper/500:09:205
760739110,0EcMasterDemoSyn0-21swapper/523:49:025
722239110,0EcMasterDemoSyn0-21swapper/500:24:335
540139110,0EcMasterDemoSyn0-21swapper/521:42:175
47539110,0EcMasterDemoSyn0-21swapper/521:57:305
3050639110,0EcMasterDemoSyn0-21swapper/519:35:315
288539110,0EcMasterDemoSyn0-21swapper/519:45:395
278698110,1EcMasterJobTask270339EcMasterDemoSyn00:39:495
2667939110,0EcMasterDemoSyn0-21swapper/523:23:415
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional