You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-12 - 07:05
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot1.osadl.org (updated Mon Jan 12, 2026 00:44:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2259039930,0EcMasterDemoSyn2260029tAtEmLog_019:44:085
2944139910,0EcMasterDemoSyn2945129tAtEmLog_021:25:335
1240239910,0EcMasterDemoSyn1241229tAtEmLog_020:29:475
2983639860,0EcMasterDemoSyn2984629tAtEmLog_022:36:335
1053639860,0EcMasterDemoSyn1054629tAtEmLog_021:15:255
1073739840,0EcMasterDemoSyn1074729tAtEmLog_021:50:555
1313739830,0EcMasterDemoSyn1314729tAtEmLog_019:23:525
602739820,0EcMasterDemoSyn603729tAtEmLog_019:08:395
264239820,0EcMasterDemoSyn265229tAtEmLog_020:09:295
1991139820,0EcMasterDemoSyn1992129tAtEmLog_023:06:585
1080839820,0EcMasterDemoSyn1081829tAtEmLog_023:01:545
840039810,0EcMasterDemoSyn841029tAtEmLog_019:13:425
2795639810,0EcMasterDemoSyn2796629tAtEmLog_019:54:165
1147139810,0EcMasterDemoSyn1148129tAtEmLog_022:26:245
659539580,0EcMasterDemoSyn660529tAtEmLog_022:46:405
2729639390,1EcMasterDemoSyn27302-21ps21:00:124
1053639290,1EcMasterDemoSyn10527-21ssh21:15:251
38862270,1sleep00-21swapper/023:15:450
80022200,1sleep0221rcuc/022:07:070
623399191,13cyclictest16-1pr/legacy22:41:361
622999195,10cyclictest32056-21sh22:02:380
284562190,0sleep20-21swapper/200:23:472
623899181,13cyclictest16-1pr/legacy23:12:014
622999180,10cyclictest31455-21sh21:26:550
737829170,14tAtEmLog_09559-21taskset20:22:165
660699170,7tDemoTimingTask6939-21chrt22:41:495
6238991713,3cyclictest13905-21sendmail23:40:004
623899170,10cyclictest26588-21apt-config22:00:014
6237991710,6cyclictest16075-21tr23:05:133
623699170,9cyclictest5993-21taskset00:29:302
623399179,6cyclictest30574-21cron22:20:001
6229991710,3cyclictest23388-21needreboot21:40:170
622999170,9cyclictest29040-21sh23:12:010
622999170,9cyclictest12011-21sh00:32:260
622999170,13cyclictest831-21runrttasks21:06:010
2487299170,10tDemoTimingTask24896-21chrt22:51:465
153499170,10tDemoTimingTask1553-21chrt21:45:515
1522139170,14EcMasterDemoSyn23502-21chrt23:26:405
1477199170,7tDemoTimingTask16808-21chrt20:35:505
737999160,9tDemoTimingTask7454-21turbostat20:20:005
623899160,9cyclictest14914-21sh23:21:594
623899160,9cyclictest14914-21sh23:21:594
623799168,6cyclictest27323-21wc19:50:233
623799160,9cyclictest20916-21sh21:20:583
623799160,9cyclictest20916-21sh21:20:583
623799160,9cyclictest20305-21rm00:19:243
623799160,12cyclictest16-1pr/legacy19:39:023
623799160,12cyclictest16-1pr/legacy19:23:513
623699168,6cyclictest7115-21tr19:10:142
623699164,8cyclictest807-21postgres19:25:142
623699162,10cyclictest4142-21ssh23:51:542
623399164,9cyclictest327-21systemd-journal23:27:591
623399164,9cyclictest15623-21tune2fs20:35:161
623399160,12cyclictest13584-21rm23:21:011
623399160,12cyclictest13584-21rm23:21:011
622999160,9cyclictest831-21runrttasks21:02:500
622999160,8cyclictest12032-21sh21:16:110
622999160,10cyclictest18952-21ssh21:55:220
1548429160,14tAtEmLog_024345-21chrt21:40:325
623899157,7cyclictest6223-21cyclictest00:11:584
623899156,7cyclictest32550-21ssh22:20:184
623899153,9cyclictest327-21systemd-journal20:22:004
623899150,9cyclictest3133850latency_hist21:10:014
623899150,9cyclictest23122-21sh21:57:594
623799153,8cyclictest22105-21latency_hist20:50:003
623799151,11cyclictest16-1pr/legacy23:01:533
623799150,12cyclictest16-1pr/legacy21:55:583
623699159,4cyclictest8039-21tail00:30:162
623699157,6cyclictest13426-21cut20:30:152
623699154,8cyclictest760-21lldpd22:10:102
623699154,8cyclictest32104-21fschecks_time22:20:152
623699152,9cyclictest2499-21chrt20:07:422
623699152,9cyclictest10207-21sh23:01:292
623699150,10cyclictest508050latency_hist20:15:002
623399153,8cyclictest10357-21proc_pri23:55:201
623399152,9cyclictest32191-21if_enp5s021:45:161
623399151,9cyclictest1860-21sh22:56:581
6233991511,3cyclictest30839-21idleruntime-cro21:45:001
623399151,11cyclictest16-1pr/legacy21:35:411
623399151,10cyclictest831-21runrttasks21:58:361
623399150,13cyclictest23515-21sed00:20:241
623399150,11cyclictest8809-21sh23:19:141
622999156,7cyclictest8123-21grep23:00:190
622999156,7cyclictest5165-21ssh22:40:370
622999156,7cyclictest31404-21ssh23:30:510
622999156,7cyclictest18530-21df_inode21:20:140
622999156,7cyclictest18530-21df_inode21:20:140
622999155,8cyclictest31589-21ssh00:07:130
622999153,9cyclictest24935-21ssh22:51:480
622999151,10cyclictest16805-21sh23:59:310
622999150,9cyclictest7421-21ssh21:14:330
622999150,9cyclictest15072-21sh22:46:120
622999150,6cyclictest7764-21cat21:50:010
622999150,11cyclictest22926-21cut22:15:180
622999150,11cyclictest18326-21sh23:41:460
622999150,10cyclictest15055-21rm21:53:550
623899144,8cyclictest24573-21/usr/sbin/munin20:55:134
623899143,8cyclictest31122-21sed22:55:164
623899143,8cyclictest26572-21cut19:50:144
623899143,8cyclictest23657-21df_inode19:45:134
623899141,9cyclictest30121-21runrttasks19:56:534
623899141,9cyclictest29955-21missed_timers23:30:184
623899141,9cyclictest20630-21runrttasks23:43:224
623899141,9cyclictest18790-21sh00:18:194
623899141,9cyclictest16831-21sh22:11:454
623899141,9cyclictest11415-21runrttasks21:51:224
623899141,10cyclictest16-1pr/legacy21:25:324
623899140,9cyclictest22201-21/usr/sbin/munin20:50:114
623899140,10cyclictest9008-21ssh23:00:384
623899140,10cyclictest8134-21/usr/sbin/munin22:25:194
623899140,10cyclictest8112-21rm22:42:394
623899140,10cyclictest3747-21ssh21:30:014
623899140,10cyclictest3339-21ssh00:27:374
623899140,10cyclictest31387-21ssh21:45:104
623899140,10cyclictest3094-21ssh23:51:104
623899140,10cyclictest30733-21/usr/sbin/munin20:00:124
623899140,10cyclictest3031-21/usr/sbin/munin20:10:164
623899140,10cyclictest28940-21ssh21:43:414
623899140,10cyclictest27108-21/usr/sbin/munin00:05:194
623899140,10cyclictest26334-21/usr/sbin/munin22:35:154
623899140,10cyclictest25296-21ssh23:27:524
623899140,10cyclictest23736-21ssh00:20:284
623899140,10cyclictest23219-21date23:45:014
623899140,10cyclictest22610-21grep22:15:164
623899140,10cyclictest21191-21ssh22:31:504
623899140,10cyclictest21183-21ssh21:21:094
623899140,10cyclictest21183-21ssh21:21:084
623899140,10cyclictest19846-21timerandwakeup00:00:254
623899140,10cyclictest18493-21diskmemload23:05:574
623899140,10cyclictest17500-21/usr/sbin/munin20:40:224
623899140,10cyclictest16693-21grep20:35:264
623899140,10cyclictest16171-21/usr/sbin/munin00:35:164
623899140,10cyclictest1614-21sed20:05:184
623899140,10cyclictest13284-21ssh21:17:034
623899140,10cyclictest13051-21ssh21:35:094
623899140,10cyclictest10418-21/usr/sbin/munin20:25:254
623799144,8cyclictest13139-21/usr/sbin/munin21:35:123
623799141,9cyclictest8263-21fschecks_time20:20:143
623799141,9cyclictest7784-21sh22:06:593
623799141,9cyclictest5605-21sort20:15:113
623799141,9cyclictest3517-21diskmemload00:10:033
623799141,9cyclictest31267-21ssh00:06:583
623799141,9cyclictest31144-21runrttasks21:07:323
623799141,9cyclictest26581-21sh23:28:463
623799141,9cyclictest25573-21ssh00:04:183
623799141,10cyclictest8878-21grep22:25:163
6237991410,3cyclictest17511-21idleruntime-cro00:00:013
623799140,10cyclictest9173-21grep00:30:263
623799140,10cyclictest8509-21ssh23:36:133
623799140,10cyclictest831-21runrttasks20:06:043
623799140,10cyclictest5503-21runrttasks00:29:093
623799140,10cyclictest521-21sh22:56:053
623799140,10cyclictest4786-21ssh23:16:243
623799140,10cyclictest420-21ssh22:03:133
623799140,10cyclictest3625-21sh23:51:333
623799140,10cyclictest32604-21sh23:31:423
623799140,10cyclictest29361-21needreboot19:55:183
623799140,10cyclictest28594-21ssh00:23:533
623799140,10cyclictest27824-21fschecks_time21:25:143
623799140,10cyclictest2703-21runrttasks22:40:023
623799140,10cyclictest25762-21/usr/sbin/munin23:10:163
623799140,10cyclictest24136-21ssh22:15:323
623799140,10cyclictest24005-21chrt00:39:333
623799140,10cyclictest22536-21rm22:50:223
623799140,10cyclictest21838-21sh22:32:183
623799140,10cyclictest20126-21sh23:43:013
623799140,10cyclictest17828-21chrt19:33:483
623799140,10cyclictest17503-21apt-get20:40:143
623799140,10cyclictest16944-21ssh23:23:233
623799140,10cyclictest16944-21ssh23:23:233
623799140,10cyclictest1669-21ssh21:10:323
623799140,10cyclictest16170-21ntp_kernel_err20:35:203
623799140,10cyclictest14476-21chrt20:31:043
623799140,10cyclictest14392-21sh22:45:453
623799140,10cyclictest11160-21ssh21:34:003
623799140,10cyclictest10554-21sh21:50:483
623699147,6cyclictest1067-21apache219:55:482
623699143,8cyclictest760-21lldpd22:28:252
623699143,6cyclictest4226-21grep22:05:142
623699141,9cyclictest7216-21sh23:18:052
623699141,9cyclictest6335-21diskmemload21:30:382
623699141,9cyclictest27134-21sh22:17:342
623699141,9cyclictest15924-21/usr/sbin/munin19:30:202
623699141,10cyclictest16-1pr/legacy22:56:492
623699140,11cyclictest16218-21awk23:05:152
623699140,10cyclictest8857-21/usr/sbin/munin19:15:372
623699140,10cyclictest831-21runrttasks21:17:292
623699140,10cyclictest7665-21chrt00:11:512
623699140,10cyclictest6897-21ssh21:14:102
623699140,10cyclictest6032-21sed20:15:142
623699140,10cyclictest31750-21ssh23:49:522
623699140,10cyclictest31142-21sh21:45:012
623699140,10cyclictest29635-21df21:05:132
623699140,10cyclictest29573-21sh00:05:462
623699140,10cyclictest29159-21ssh23:12:052
623699140,10cyclictest28103-21ssh23:29:522
623699140,10cyclictest26927-21/usr/sbin/munin22:00:252
623699140,10cyclictest24741-21chrt21:40:472
623699140,10cyclictest23301-21/usr/sbin/munin19:45:122
623699140,10cyclictest22311-21cut20:50:132
623699140,10cyclictest21362-21ssh21:56:442
623699140,10cyclictest21171-21wget22:50:122
623699140,10cyclictest19848-21/usr/sbin/munin20:45:172
623699140,10cyclictest19657-21sh21:38:362
623699140,10cyclictest18207-21wget21:20:132
623699140,10cyclictest18207-21wget21:20:122
623699140,10cyclictest17202-21apt22:30:122
623699140,10cyclictest16491-21ssh00:16:452
623699140,10cyclictest14450-21/usr/sbin/munin23:40:202
623699140,10cyclictest14407-21sh22:45:452
623699140,10cyclictest13952-21ssh21:53:092
623699140,10cyclictest12297-21/usr/sbin/munin23:20:232
623699140,10cyclictest12297-21/usr/sbin/munin23:20:232
623699140,10cyclictest11263-21taskset23:55:362
623699140,10cyclictest10720-21ssh23:37:452
623699140,10cyclictest10366-21ssh22:44:152
623399141,9cyclictest31041-21runrttasks21:26:391
623399141,9cyclictest30411-21swap23:30:241
623399141,9cyclictest19062-21rm00:36:021
623399141,11cyclictest16-1pr/legacy22:46:401
623399141,10cyclictest30087-21runrttasks22:01:151
623399141,10cyclictest28540-21taskset23:11:401
623399141,10cyclictest26929-21ssh00:05:031
623399141,10cyclictest16-1pr/legacy21:00:111
623399140,7cyclictest16492-21ssh00:16:451
623399140,10cyclictest8722-21sh23:36:211
623399140,10cyclictest8711-21grep21:50:131
623399140,10cyclictest8312-21ssh00:30:191
623399140,10cyclictest744-21snmpd23:50:151
623399140,10cyclictest744-21snmpd20:05:571
623399140,10cyclictest7355-21ssh23:00:131
623399140,10cyclictest7133-21ssh22:24:411
623399140,10cyclictest5600-21ssh22:05:291
623399140,10cyclictest435-21sed21:10:161
623399140,10cyclictest3400-21entropy20:10:161
623399140,10cyclictest327-21systemd-journal20:28:171
623399140,10cyclictest29284-21/usr/sbin/munin21:05:151
623399140,10cyclictest27572-21ntp_offset22:35:221
623399140,10cyclictest27405-21wc19:50:251
623399140,10cyclictest25171-21runrttasks19:48:181
623399140,10cyclictest23272-21sh23:45:041
623399140,10cyclictest22756-21cat22:50:241
623399140,10cyclictest22186-21sh23:44:291
623399140,10cyclictest20922-21sh00:01:001
623399140,10cyclictest19861-21grep19:35:251
623399140,10cyclictest19030-21unixbench-2d22:30:251
623399140,10cyclictest18845-21ssh23:06:121
623399140,10cyclictest1799-21rm00:26:311
623399140,10cyclictest17955-21taskset21:20:051
623399140,10cyclictest17955-21taskset21:20:041
623399140,10cyclictest17898-21chrt19:34:041
623399140,10cyclictest17500-21/usr/sbin/munin20:40:131
623399140,10cyclictest14584-21sh22:28:351
623399140,10cyclictest13558-21runrttasks19:25:011
623399140,10cyclictest13547-21fschecks_time22:10:141
623399140,10cyclictest11220-21/usr/sbin/munin19:20:141
623399140,10cyclictest10182-21ssh21:33:211
622999147,4cyclictest0-21swapper/023:22:100
622999147,4cyclictest0-21swapper/023:22:090
622999146,6cyclictest831-21runrttasks19:39:030
622999146,6cyclictest23390-21ssh23:45:080
622999145,3cyclictest831-21runrttasks20:19:530
622999141,9cyclictest831-21runrttasks20:51:450
622999141,9cyclictest4940-21sh00:28:440
622999141,9cyclictest32475-21sh22:55:430
622999141,10cyclictest7762-21runrttasks20:20:080
622999141,10cyclictest10825-21sh00:14:050
622999140,9cyclictest831-21runrttasks20:57:300
622999140,8cyclictest175-1kworker/0:1H+kblockd19:16:270
622999140,7cyclictest31878-21diskstats22:20:130
622999140,7cyclictest27393-21sh23:29:220
622999140,12cyclictest17184-21python319:30:210
622999140,12cyclictest16174-21/usr/sbin/munin23:05:140
622999140,10cyclictest831-21runrttasks21:37:500
622999140,10cyclictest8170-21runrttasks19:10:440
622999140,10cyclictest7912-21sh22:25:030
622999140,10cyclictest7086-21threads23:35:240
622999140,10cyclictest707-21grep23:50:140
622999140,10cyclictest7001-21rm21:31:070
622999140,10cyclictest28699-21sh22:35:450
622999140,10cyclictest25484-21sh00:21:430
622999140,10cyclictest23921-21gltestperf19:45:130
622999140,10cyclictest20806-21sh00:37:170
622999140,10cyclictest20635-21runrttasks00:19:380
622999140,10cyclictest19609-21sh00:00:210
622999140,10cyclictest18593-21grep22:30:210
622999140,10cyclictest16253-21ssh22:11:210
622999140,10cyclictest12222-21runrttasks20:27:260
2019039140,6EcMasterDemoSyn21648-21taskset21:21:295
2019039140,6EcMasterDemoSyn21648-21taskset21:21:285
623899135,6cyclictest6954-21tune2fs19:10:134
623899134,8cyclictest5755-21/usr/sbin/munin20:15:144
623899134,7cyclictest22001-21/usr/sbin/munin19:40:224
623899130,9cyclictest9076-21/usr/sbin/munin23:55:144
623899130,9cyclictest8896-21cut19:15:254
623899130,9cyclictest831-21runrttasks22:45:384
623899130,9cyclictest3402-21sh22:05:054
623899130,9cyclictest31839-21/usr/sbin/munin21:10:104
623899130,9cyclictest21021-21/usr/sbin/munin22:50:254
623899130,9cyclictest20426-21gltestperf20:45:134
623899130,9cyclictest18277-21/usr/sbin/munin19:35:114
623899130,9cyclictest15924-21/usr/sbin/munin19:30:124
623899130,9cyclictest13570-21/usr/sbin/munin19:25:134
623899130,9cyclictest12771-21/usr/sbin/munin20:30:114
623899130,9cyclictest1225-21ssh23:15:024
623899130,9cyclictest11220-21/usr/sbin/munin19:20:164
623899130,9cyclictest10251-21ssh00:31:104
623799131,9cyclictest9006-21wget21:15:123
623799131,9cyclictest29075-21latency_hist21:05:003
623799131,9cyclictest24830-21sh21:40:513
623799131,8cyclictest26280-21sh22:35:113
623799130,9cyclictest831-21runrttasks19:15:143
623799130,9cyclictest6494-21/usr/sbin/munin19:10:223
623799130,9cyclictest6180-21latency_hist19:10:013
623799130,9cyclictest566-21timerandwakeup21:45:243
623799130,9cyclictest31857-21ssh22:20:133
623799130,9cyclictest22201-21/usr/sbin/munin20:50:163
623799130,9cyclictest20647-21/usr/sbin/munin19:40:153
623799130,9cyclictest15142-21diskmemload22:10:343
623799130,9cyclictest10418-21/usr/sbin/munin20:25:153
623799130,8cyclictest14002-21fschecks_count19:25:133
623799130,7cyclictest24555-21apt20:55:113
623799130,10cyclictest3592-21fschecks_time20:10:173
623799130,10cyclictest31495-21kernelversion20:00:183
623799130,10cyclictest24987-21chrt19:45:503
623799130,10cyclictest23337-21ssh23:45:063
623699130,9cyclictest7968-21cpuspeed_turbos20:20:122
623699130,9cyclictest31471-21cat20:00:162
623699130,9cyclictest28667-21ssh23:30:042
623699130,9cyclictest26966-21cut21:00:112
623699130,9cyclictest26004-21/usr/sbin/munin19:50:142
623699130,9cyclictest24490-21runrttasks20:55:002
623699130,9cyclictest18277-21/usr/sbin/munin19:35:202
623699130,9cyclictest15126-21/usr/sbin/munin20:35:192
623699130,9cyclictest10418-21/usr/sbin/munin20:25:132
623699130,10cyclictest28886-21switchtime21:25:242
623699130,10cyclictest26177-21sh22:35:052
623699130,10cyclictest21686-21/usr/sbin/munin19:40:172
623699130,10cyclictest19027-21rm00:00:172
623699130,10cyclictest17775-21sh00:35:242
623699130,10cyclictest17500-21/usr/sbin/munin20:40:142
623399131,10cyclictest16-1pr/legacy20:55:071
623399130,9cyclictest9453-21runrttasks20:20:531
623399130,9cyclictest8857-21/usr/sbin/munin19:15:341
623399130,9cyclictest831-21runrttasks20:46:061
623399130,9cyclictest5379-21/usr/sbin/munin20:15:121
623399130,9cyclictest4893-21grep00:10:181
623399130,9cyclictest31264-21fschecks_time20:00:141
623399130,9cyclictest28358-21/usr/sbin/munin19:55:121
623399130,9cyclictest23158-21needreboot20:50:181
623399130,9cyclictest13111-21seq20:30:131
623399130,10cyclictest6853-21df19:10:131
622999130,9cyclictest831-21runrttasks20:07:080
622999130,9cyclictest21854-21latency_hist20:50:000
622999130,9cyclictest11766-21fschecks_time19:20:150
622999130,7cyclictest4717-21chrt20:10:510
622999130,7cyclictest28429-21cut19:55:120
622999130,7cyclictest175-1kworker/0:1H+kblockd19:41:290
622999130,11cyclictest15971-21cat20:35:190
622999130,10cyclictest808-21postgres20:30:170
622999130,10cyclictest31096-21diskstats20:00:140
622999130,10cyclictest18490-21head20:40:190
622999130,10cyclictest13887-21grep19:25:130
1477199130,4tDemoTimingTask14833-21turbostat20:34:595
1305899130,10tDemoTimingTask17043-21chrt00:35:165
1075839130,2EcMasterDemoSyn651irq_work/519:18:475
623699124,6cyclictest6426-21date19:10:002
623699120,9cyclictest11891-21hddtemp_smartct19:20:162
623399120,9cyclictest21882-21grep19:40:191
622999120,9cyclictest6344-21latency_hist19:10:000
622999120,7cyclictest25967-21cat19:50:010
3032599120,10tDemoTimingTask32580-21chrt20:03:025
2844539120,2EcMasterDemoSyn4873-21chrt23:52:265
2530039120,0EcMasterDemoSyn0-21swapper/519:49:135
2435029124,0tAtEmLog_00-21swapper/523:32:005
230639120,0EcMasterDemoSyn0-21swapper/522:21:205
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional