You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-11 - 23:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Mon May 11, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
875023925,0sleep00-21swapper/006:54:070
860823525,0sleep20-21swapper/206:52:252
304450350,0irq/25-eth10-21swapper/306:51:073
882223329,0sleep10-21swapper/106:54:561
89809960,0cyclictest0-21swapper/211:44:192
89809960,0cyclictest0-21swapper/207:15:592
89799966,0cyclictest7159-21diskmemload09:01:011
89789966,0cyclictest0-21swapper/012:11:200
35160,0ktimersoftd/30-21swapper/308:51:153
339960,0migration/34409-21sh11:11:183
339960,0migration/314646-21sh09:17:553
29715060,0irq/26-eth0-rx-29053-21grep11:06:193
29715060,0irq/26-eth0-rx-22496-21python07:26:193
29715060,0irq/26-eth0-rx-2167-21tr11:31:133
149960,0migration/06561-21sh09:33:040
149960,0migration/06473-21sh10:13:520
149960,0migration/022469-21munin-node07:26:180
89809955,0cyclictest0-21swapper/211:57:242
89809955,0cyclictest0-21swapper/211:26:302
89809955,0cyclictest0-21swapper/210:36:032
89809955,0cyclictest0-21swapper/208:56:532
89809955,0cyclictest0-21swapper/207:05:252
89809950,0cyclictest0-21swapper/210:03:442
89809950,0cyclictest0-21swapper/209:07:172
89809950,0cyclictest0-21swapper/208:06:072
89799955,0cyclictest7159-21diskmemload12:24:231
89799955,0cyclictest7159-21diskmemload09:29:141
89799955,0cyclictest0-21swapper/109:59:491
89799955,0cyclictest0-21swapper/109:38:321
89799954,0cyclictest20737-21irqcore08:41:131
89799950,0cyclictest0-21swapper/112:02:481
89799950,0cyclictest0-21swapper/111:56:581
89799950,0cyclictest0-21swapper/110:10:041
8215050,0irq/24-0000:00:30053-21cat07:35:572
339950,0migration/3927-21kthreadcore09:51:133
339950,0migration/38251-21fschecks_time09:56:103
339950,0migration/37859-21runrttasks09:36:073
339950,0migration/35405-21munin-node09:11:173
339950,0migration/331587-21irqrtprio12:11:123
339950,0migration/330554-21taskset07:57:083
339950,0migration/329860-21munin-node09:46:203
339950,0migration/323162-21latency_hist10:45:563
339950,0migration/322217-21sshd11:04:323
339950,0migration/320417-21taskset12:21:483
339950,0migration/317695-21munin-node08:11:173
339950,0migration/316728-21munin-node07:01:123
339950,0migration/316648-21latency_hist07:25:573
339950,0migration/315944-21kthreadcore08:36:123
339950,0migration/315769-21cpuspeed_turbos10:21:083
339950,0migration/315217-21latency_hist10:40:563
339950,0migration/312061-21munin-node07:46:123
339950,0migration/31201-21kthreadcore09:31:123
339950,0migration/310856-21taskset07:45:343
339950,0migration/310245-21kthreadcore11:36:123
29725050,0irq/27-eth0-tx-5532-21ssh11:34:022
29725050,0irq/27-eth0-tx-30125-21munin-node07:36:110
29725050,0irq/27-eth0-tx-28691-21load10:46:142
29725050,0irq/27-eth0-tx-21871-40yum-updatesd-he10:42:402
29725050,0irq/27-eth0-tx-21366-21ssh09:01:182
29725050,0irq/27-eth0-tx-20322-21ssh11:21:172
29725050,0irq/27-eth0-tx-15878-21munin-node09:41:172
29725050,0irq/27-eth0-tx-15121-21ssh09:39:332
29725050,0irq/27-eth0-tx-15007-21munin-node11:01:102
29725050,0irq/27-eth0-tx-13810-21sed09:56:202
29725050,0irq/27-eth0-tx-13055-21munin-node08:36:090
29725050,0irq/27-eth0-tx-0-21swapper/212:14:092
29725050,0irq/27-eth0-tx-0-21swapper/211:50:542
29725050,0irq/27-eth0-tx-0-21swapper/210:53:022
29725050,0irq/27-eth0-tx-0-21swapper/210:27:492
29725050,0irq/27-eth0-tx-0-21swapper/210:14:582
29725050,0irq/27-eth0-tx-0-21swapper/007:49:480
29725050,0irq/27-eth0-tx-0-21swapper/007:21:160
29725050,0irq/27-eth0-tx-0-21swapper/007:01:070
29725050,0irq/27-eth0-tx-0-21swapper/006:56:160
29715050,0irq/26-eth0-rx-4411-21munin-node07:16:123
29715050,0irq/26-eth0-rx-28531-21smartctl11:26:183
29715050,0irq/26-eth0-rx-21313-21missed_timers09:41:153
29715050,0irq/26-eth0-rx-20963-21kthreadcore09:21:133
29715050,0irq/26-eth0-rx-18825-21cut09:01:133
29715050,0irq/26-eth0-rx-13557-21ssh10:57:353
29715050,0irq/26-eth0-rx-12152-21grep10:36:143
29715050,0irq/26-eth0-rx-11089-21cut11:56:143
29715050,0irq/26-eth0-rx-0-21swapper/312:06:253
29715050,0irq/26-eth0-rx-0-21swapper/311:51:083
29715050,0irq/26-eth0-rx-0-21swapper/311:49:093
29715050,0irq/26-eth0-rx-0-21swapper/311:18:263
29715050,0irq/26-eth0-rx-0-21swapper/310:51:423
29715050,0irq/26-eth0-rx-0-21swapper/310:29:303
29715050,0irq/26-eth0-rx-0-21swapper/310:01:093
29715050,0irq/26-eth0-rx-0-21swapper/309:28:473
29715050,0irq/26-eth0-rx-0-21swapper/308:06:153
259950,0migration/29448-21kthreadcore08:31:142
259950,0migration/25835-21taskset10:32:432
249950,0watchdog/26559-21runrttasks09:11:542
249950,0watchdog/226458-21cut12:06:142
249950,0watchdog/20-21swapper/208:16:152
19150,0ktimersoftd/10-21swapper/110:45:131
179950,0migration/131527-21sh09:10:201
179950,0migration/131117-21kthreadcore07:11:131
179950,0migration/123820-21latency_hist09:45:561
179950,0migration/122762-21latency_hist11:05:561
179950,0migration/12244-21kthreadcore09:51:131
179950,0migration/11888-21kthreadcore11:51:131
169950,0watchdog/14744-21ssh11:31:461
169950,0watchdog/129093-21munin-node10:46:181
169950,0watchdog/116668-21if_err_eth007:01:111
149950,0migration/09614-21taskset07:17:460
149950,0migration/08641-21munin-node09:56:120
149950,0migration/08334-21df_abs09:16:080
149950,0migration/08109-21kthreadcore08:06:130
149950,0migration/07758-21kthreadcore07:41:140
149950,0migration/06862-21sh10:35:380
149950,0migration/03931-21kthreadcore08:01:120
149950,0migration/029620-21sh10:46:470
149950,0migration/029392-21kthreadcore07:56:150
149950,0migration/028886-21kthreadcore08:46:130
149950,0migration/027715-21kthreadcore10:06:140
149950,0migration/024359-21fschecks_count09:06:090
149950,0migration/024133-21kthreadcore11:26:120
149950,0migration/023772-21cpuspeed_turbos07:31:090
149950,0migration/022251-21kthreadcore08:41:140
149950,0migration/020055-21latency11:21:140
149950,0migration/016344-21kthreadcore08:11:140
149950,0migration/016194-21sh09:21:070
149950,0migration/015818-21kthreadcore12:21:130
149950,0migration/014148-21sh11:20:060
149950,0migration/012923-21needreboot08:56:150
89809944,0cyclictest0-21swapper/209:50:582
89809943,0cyclictest3556-21lldpd12:22:512
89809943,0cyclictest3556-21lldpd11:12:202
89809943,0cyclictest3556-21lldpd09:26:192
89809943,0cyclictest3556-21lldpd07:45:472
89809943,0cyclictest3556-21lldpd07:21:472
89809943,0cyclictest3556-21lldpd07:11:172
89809943,0cyclictest3556-21lldpd07:06:462
89809943,0cyclictest26265-21awk08:21:102
89809943,0cyclictest1477-21awk08:51:112
89809943,0cyclictest12954-21awk08:11:112
89809943,0cyclictest10045-21awk06:56:122
89809943,0cyclictest0-21swapper/209:46:142
89799944,0cyclictest0-21swapper/109:24:021
89799943,0cyclictest549-21awk08:26:121
89799943,0cyclictest3241-21irqbalance11:27:341
89799943,0cyclictest3241-21irqbalance10:52:041
89799943,0cyclictest3241-21irqbalance09:32:551
89799943,0cyclictest18639-21awk07:51:111
89799943,0cyclictest11997-21awk07:46:111
89789943,0cyclictest8460-21awk10:16:120
89789943,0cyclictest3514-21snmpd12:17:180
89789943,0cyclictest3514-21snmpd10:23:480
89789943,0cyclictest3241-21irqbalance10:51:040
89789943,0cyclictest3241-21irqbalance09:39:450
89789943,0cyclictest3241-21irqbalance09:26:050
89789943,0cyclictest3241-21irqbalance07:06:150
89789943,0cyclictest31638-21awk11:31:100
89789943,0cyclictest17964-21kworker/0:211:59:190
89789943,0cyclictest17964-21kworker/0:209:51:140
89789943,0cyclictest16781-21awk09:41:120
89789940,0cyclictest0-21swapper/012:10:140
89789940,0cyclictest0-21swapper/012:02:300
89789940,0cyclictest0-21swapper/011:48:570
89789940,0cyclictest0-21swapper/011:42:400
89789940,0cyclictest0-21swapper/011:38:420
89789940,0cyclictest0-21swapper/009:05:370
4140,0ktimersoftd/00-21swapper/010:41:360
339940,0migration/3767-21irqcore08:26:123
339940,0migration/327089-21kthreadcore10:06:143
339940,0migration/326537-21kthreadcore07:31:143
339940,0migration/315321-21fschecks_time11:21:103
339940,0migration/313199-21sensors_fan08:56:183
339940,0migration/312810-21dump-pmu-power08:35:523
29725040,0irq/27-eth0-tx-15364-21ssh10:20:562
29725040,0irq/27-eth0-tx-14356-21ssh12:01:022
29725040,0irq/27-eth0-tx-0-21swapper/211:51:012
29725040,0irq/27-eth0-tx-0-21swapper/211:15:582
29725040,0irq/27-eth0-tx-0-21swapper/210:56:072
29725040,0irq/27-eth0-tx-0-21swapper/210:05:572
29725040,0irq/27-eth0-tx-0-21swapper/209:31:542
29725040,0irq/27-eth0-tx-0-21swapper/209:21:062
29725040,0irq/27-eth0-tx-0-21swapper/209:16:002
29715040,0irq/26-eth0-rx-0-21swapper/312:16:023
29715040,0irq/26-eth0-rx-0-21swapper/312:01:073
29715040,0irq/26-eth0-rx-0-21swapper/311:42:453
29715040,0irq/26-eth0-rx-0-21swapper/310:32:253
29715040,0irq/26-eth0-rx-0-21swapper/310:17:023
29715040,0irq/26-eth0-rx-0-21swapper/310:12:533
29715040,0irq/26-eth0-rx-0-21swapper/309:06:093
29715040,0irq/26-eth0-rx-0-21swapper/308:18:373
29715040,0irq/26-eth0-rx-0-21swapper/307:06:093
259940,0migration/28321-21kthreadcore11:36:122
259940,0migration/227397-21kthreadcore11:06:132
259940,0migration/225900-21kthreadcore07:56:132
259940,0migration/224300-21idleruntime07:31:112
259940,0migration/220993-21kthreadcore07:26:152
259940,0migration/219094-21runrttasks08:36:292
259940,0migration/211264-21kthreadcore12:16:152
249940,0watchdog/232199-21sed08:01:112
249940,0watchdog/232058-21taskset08:26:072
249940,0watchdog/20-21swapper/210:16:142
179940,0migration/19556-21kthreadcore08:06:131
179940,0migration/17648-21cpuspeed_turbos08:56:091
179940,0migration/16995-21fschecks_time08:31:101
179940,0migration/14415-21runrttasks07:40:581
179940,0migration/132743-21fschecks_time09:11:101
179940,0migration/131676-21kernelversion12:11:131
179940,0migration/125884-21fschecks_count08:21:091
179940,0migration/12588-21kthreadcore08:51:131
179940,0migration/121216-21sh10:01:141
179940,0migration/119043-21munin-node08:36:181
179940,0migration/116248-21runrttasks10:21:101
179940,0migration/115061-21df_abs11:21:091
179940,0migration/112395-21switchtime11:36:181
179940,0migration/11089-21kthreadcore11:11:131
169940,0watchdog/19308-21timerwakeupswit07:16:191
169940,0watchdog/132198-21cut08:01:111
169940,0watchdog/119307-21perl08:16:111
169940,0watchdog/116532-21cut09:41:101
169940,0watchdog/113708-21ssh10:58:021
169940,0watchdog/112186-21kthreadcore10:16:141
169940,0watchdog/111579-21ssh10:36:141
169940,0watchdog/10-21swapper/112:16:141
169940,0watchdog/10-21swapper/111:06:421
149940,0migration/030757-21kthreadcore07:11:130
149940,0migration/029596-21kthreadcore08:21:130
149940,0migration/027273-21kthreadcore09:46:140
149940,0migration/023252-21ntp_states07:51:160
89809932,0cyclictest3556-21lldpd07:55:482
89809932,0cyclictest22006-21cut08:41:142
89809932,0cyclictest16781-21sensors07:46:182
89799933,0cyclictest7581-21awk11:16:101
89799932,0cyclictest32506-21awk10:31:121
89799932,0cyclictest25061-21kthreadcore12:06:131
89799932,0cyclictest17694-21sendmail_mailqu08:11:171
89799932,0cyclictest10355-21grep09:16:131
89799930,0cyclictest27324-21grep07:31:141
89799930,0cyclictest26943-21perl08:46:091
89789933,0cyclictest17964-21kworker/0:211:06:030
89789933,0cyclictest17964-21kworker/0:210:36:070
89789933,0cyclictest17964-21kworker/0:209:11:020
89789933,0cyclictest17964-21kworker/0:208:16:500
89789932,0cyclictest8340-21tr10:56:130
89789932,0cyclictest461-21awk08:26:110
89789932,0cyclictest31883-21awk11:11:100
89789932,0cyclictest24408-21awk10:26:100
89789932,0cyclictest15758-21aten_r1power_cu10:01:060
89789932,0cyclictest15012-21aten_r1power_cu11:01:080
89789930,0cyclictest31350-21perl11:51:090
89789930,0cyclictest0-21swapper/008:31:130
29725030,0irq/27-eth0-tx-0-21swapper/008:51:080
29715030,0irq/26-eth0-rx-0-21swapper/308:21:173
29715030,0irq/26-eth0-rx-0-21swapper/308:03:033
29715030,0irq/26-eth0-rx-0-21swapper/307:51:243
29715030,0irq/26-eth0-rx-0-21swapper/307:36:173
29715030,0irq/26-eth0-rx-0-21swapper/307:15:183
29715030,0irq/26-eth0-rx-0-21swapper/306:56:183
249930,0watchdog/20-21swapper/208:46:072
179930,0migration/13884-21runrttasks07:37:211
179930,0migration/110148-21if_eth1.206:56:121
169930,0watchdog/16559-21runrttasks10:26:061
169930,0watchdog/10-21swapper/111:46:101
169930,0watchdog/10-21swapper/111:41:101
169930,0watchdog/10-21swapper/110:11:141
169930,0watchdog/10-21swapper/107:26:191
169930,0watchdog/10-21swapper/107:06:151
89799921,0cyclictest24397-21latency_hist07:55:571
89799920,0cyclictest0-21swapper/107:21:001
29715020,0irq/26-eth0-rx-0-21swapper/308:46:093
29715020,0irq/26-eth0-rx-0-21swapper/308:41:103
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional