You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-01 - 15:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Mon Jun 01, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450350,0irq/25-eth10-21swapper/306:49:263
1289723523,0sleep00-21swapper/006:51:060
1276323526,0sleep10-21swapper/106:49:351
1287723425,0sleep20-21swapper/206:50:562
339960,0migration/39810-21sh11:32:153
339960,0migration/32381-21sh10:48:123
339960,0migration/32292-21sh09:25:413
339960,0migration/319720-21sh08:59:033
339960,0migration/317769-21sh11:36:593
339960,0migration/316723-21sh12:15:053
339960,0migration/31565-21sh11:26:483
29725060,0irq/27-eth0-tx-29640-21munin-node08:19:200
29715060,0irq/26-eth0-rx-8822-21cut09:49:143
29715060,0irq/26-eth0-rx-6312-21kthreadcore10:29:133
29715060,0irq/26-eth0-rx-15449-21iostat_ios07:19:133
29715060,0irq/26-eth0-rx-13860-21grep11:54:123
29715060,0irq/26-eth0-rx-11950-21cpu09:54:053
149960,0migration/027629-21sh09:43:530
149960,0migration/024840-21sh12:20:200
149960,0migration/017901-21timerandwakeup09:14:190
149960,0migration/017212-21sh11:14:510
149960,0migration/01691-21sh11:47:510
149960,0migration/016435-21rt-features11:34:160
132119960,0cyclictest0-21swapper/211:56:352
132109966,0cyclictest0-21swapper/111:23:061
132109960,0cyclictest0-21swapper/110:39:351
132089960,0cyclictest0-21swapper/010:54:550
339950,0migration/39628-21runrttasks11:11:053
339950,0migration/37240-21kthreadcore10:49:133
339950,0migration/35642-21kthreadcore08:24:133
339950,0migration/35642-21kthreadcore08:24:133
339950,0migration/329911-21kthreadcore11:04:153
339950,0migration/329678-21kthreadcore09:04:123
339950,0migration/328519-21kthreadcore11:44:133
339950,0migration/32800-21latency_hist07:58:573
339950,0migration/326831-21munin-node12:04:083
339950,0migration/326227-21kthreadcore07:24:153
339950,0migration/326061-21runrttasks09:59:493
339950,0migration/322519-21kthreadcore10:19:153
339950,0migration/32063-21munin-node07:09:133
339950,0migration/319200-21fschecks_count11:39:103
339950,0migration/312174-21cpuspeed_turbos09:34:103
339950,0migration/311441-21cpuspeed_turbos10:34:083
339950,0migration/310003-21taskset10:10:303
29725050,0irq/27-eth0-tx-4313-21sshd08:44:350
29725050,0irq/27-eth0-tx-3141-21munin-node10:29:192
29725050,0irq/27-eth0-tx-2363-21munin-node11:49:142
29725050,0irq/27-eth0-tx-0-21swapper/212:04:432
29725050,0irq/27-eth0-tx-0-21swapper/211:14:222
29725050,0irq/27-eth0-tx-0-21swapper/211:11:302
29725050,0irq/27-eth0-tx-0-21swapper/211:00:002
29725050,0irq/27-eth0-tx-0-21swapper/209:50:392
29725050,0irq/27-eth0-tx-0-21swapper/209:12:232
29725050,0irq/27-eth0-tx-0-21swapper/209:06:142
29725050,0irq/27-eth0-tx-0-21swapper/008:54:130
29725050,0irq/27-eth0-tx-0-21swapper/008:11:340
29725050,0irq/27-eth0-tx-0-21swapper/007:49:150
29715050,0irq/26-eth0-rx-6559-21runrttasks12:02:553
29715050,0irq/26-eth0-rx-6559-21runrttasks09:29:523
29715050,0irq/26-eth0-rx-32500-21kthreadcore10:24:133
29715050,0irq/26-eth0-rx-27999-21kthreadcore08:39:153
29715050,0irq/26-eth0-rx-27935-21munin-node10:04:083
29715050,0irq/26-eth0-rx-25916-21unixbench-2d09:39:223
29715050,0irq/26-eth0-rx-20877-21kthreadcore08:09:153
29715050,0irq/26-eth0-rx-16296-21kthreadcore06:54:143
29715050,0irq/26-eth0-rx-14789-21kthreadcore08:54:133
29715050,0irq/26-eth0-rx-0-21swapper/311:49:143
29715050,0irq/26-eth0-rx-0-21swapper/311:01:173
29715050,0irq/26-eth0-rx-0-21swapper/310:43:533
29715050,0irq/26-eth0-rx-0-21swapper/309:48:273
29715050,0irq/26-eth0-rx-0-21swapper/309:20:093
29715050,0irq/26-eth0-rx-0-21swapper/309:15:383
29715050,0irq/26-eth0-rx-0-21swapper/309:09:153
29715050,0irq/26-eth0-rx-0-21swapper/308:33:593
29715050,0irq/26-eth0-rx-0-21swapper/308:19:153
27150,0ktimersoftd/20-21swapper/210:19:342
259950,0migration/232673-21latency10:44:152
259950,0migration/229564-21latency_hist08:18:572
259950,0migration/228354-21fschecks_time07:29:112
259950,0migration/227991-21unixbench_multi07:49:232
259950,0migration/227279-21sh10:03:192
259950,0migration/226897-21sh09:21:172
259950,0migration/226591-21taskset09:40:582
259950,0migration/22654-21sh09:46:592
259950,0migration/225825-21sh11:42:072
259950,0migration/225742-21sh12:02:162
259950,0migration/222137-21kthreadcore08:59:132
259950,0migration/221937-21latency_hist07:48:572
259950,0migration/220419-21kthreadcore12:19:122
259950,0migration/217739-21sh10:35:252
259950,0migration/211129-21fschecks_count11:34:082
249950,0watchdog/227575-21diskstats11:04:102
19150,0ktimersoftd/10-21swapper/111:03:091
19150,0ktimersoftd/10-21swapper/110:19:341
149950,0migration/09301-21sh11:51:180
149950,0migration/08815-21latency10:09:150
149950,0migration/06242-21kthreadcore11:29:120
149950,0migration/05348-21kthreadcore10:49:130
149950,0migration/05260-21kthreadcore09:09:140
149950,0migration/05071-21kthreadcore07:59:140
149950,0migration/031065-21kthreadcore07:04:130
149950,0migration/030658-21kthreadcore10:04:140
149950,0migration/028109-21cpuspeed_turbos09:04:080
149950,0migration/027755-21fschecks_time11:04:110
149950,0migration/027138-21kthreadcore08:14:140
149950,0migration/025361-21unixbench_singl10:39:200
149950,0migration/025344-21sh08:59:170
149950,0migration/025216-21sendmail_mailqu10:19:190
149950,0migration/024361-21munin-node11:39:180
149950,0migration/022053-21if_eth1.207:24:120
149950,0migration/020195-21timerandwakeup07:19:190
149950,0migration/019671-21latency_hist09:58:560
149950,0migration/019289-21timerwakeupswit06:54:200
149950,0migration/01559-21unixbench-2d09:44:190
149950,0migration/015041-21kthreadcore09:34:150
149950,0migration/01295-21taskset10:45:030
149950,0migration/012947-21kthreadcore08:04:140
149950,0migration/010096-21latency_hist12:13:560
132119955,0cyclictest0-21swapper/211:26:192
132119950,0cyclictest0-21swapper/208:54:302
132109955,0cyclictest0-21swapper/110:44:481
132109955,0cyclictest0-21swapper/110:27:081
132109950,0cyclictest0-21swapper/112:02:031
132109950,0cyclictest0-21swapper/111:41:521
132109950,0cyclictest0-21swapper/109:12:341
132089950,0cyclictest0-21swapper/009:21:160
132089950,0cyclictest0-21swapper/007:11:590
339940,0migration/312620-21kthreadcore08:04:143
339940,0migration/312496-21kthreadcore07:39:143
29725040,0irq/27-eth0-tx-771-21ssh11:45:162
29725040,0irq/27-eth0-tx-7060-21kthreadcore11:29:132
29725040,0irq/27-eth0-tx-18832-21ssh11:18:562
29725040,0irq/27-eth0-tx-10278-21munin-node12:14:172
29725040,0irq/27-eth0-tx-0-21swapper/210:54:012
29725040,0irq/27-eth0-tx-0-21swapper/210:49:212
29725040,0irq/27-eth0-tx-0-21swapper/210:24:272
29725040,0irq/27-eth0-tx-0-21swapper/210:04:102
29715040,0irq/26-eth0-rx-0-21swapper/312:19:163
29715040,0irq/26-eth0-rx-0-21swapper/312:09:143
29715040,0irq/26-eth0-rx-0-21swapper/311:19:023
29715040,0irq/26-eth0-rx-0-21swapper/311:17:473
29715040,0irq/26-eth0-rx-0-21swapper/310:54:463
29715040,0irq/26-eth0-rx-0-21swapper/310:14:113
29715040,0irq/26-eth0-rx-0-21swapper/308:48:453
29715040,0irq/26-eth0-rx-0-21swapper/307:04:133
259940,0migration/28605-21idleruntime07:14:112
259940,0migration/25051-21runrttasks08:24:122
259940,0migration/25051-21runrttasks08:24:122
259940,0migration/24443-21kthreadcore07:59:142
259940,0migration/230828-21kthreadcore07:04:132
259940,0migration/222014-21kthreadcore06:59:152
259940,0migration/22035-21aten_r1power_cu07:34:082
259940,0migration/219187-21kthreadcore08:09:142
259940,0migration/216101-21kthreadcore09:34:152
259940,0migration/213141-21irqcore09:14:122
259940,0migration/211028-21kthreadcore07:39:132
249940,0watchdog/23413-21ssh10:09:032
249940,0watchdog/229517-21idleruntime07:54:112
249940,0watchdog/215171-21cut09:54:112
249940,0watchdog/20-21swapper/212:09:142
249940,0watchdog/20-21swapper/210:14:152
249940,0watchdog/20-21swapper/209:29:072
249940,0watchdog/20-21swapper/206:56:112
169940,0watchdog/19739-21timerandwakeup08:24:191
169940,0watchdog/19739-21timerandwakeup08:24:191
169940,0watchdog/16876-21cut12:09:141
169940,0watchdog/127747-21if_eth1.307:04:111
169940,0watchdog/115172-21tr09:54:111
169940,0watchdog/10-21swapper/111:09:141
169940,0watchdog/10-21swapper/109:39:151
169940,0watchdog/10-21swapper/107:29:151
149940,0migration/09325-21kthreadcore07:14:140
149940,0migration/06666-21kthreadcore10:29:130
149940,0migration/029766-21irqcore07:54:130
149940,0migration/025946-21timerwakeupswit06:59:210
149940,0migration/018805-21kthreadcore07:44:130
149940,0migration/015996-21missed_timers08:29:170
149940,0migration/014595-21kthreadcore12:14:130
132119944,0cyclictest0-21swapper/209:25:142
132119943,0cyclictest3514-21snmpd07:24:062
132119943,0cyclictest3514-21snmpd07:20:062
132119943,0cyclictest31580-21awk08:44:122
132119943,0cyclictest20011-21grep10:39:112
132119943,0cyclictest16128-21turbostat08:08:572
132119943,0cyclictest11468-21awk08:29:132
132109944,0cyclictest0-21swapper/107:49:001
132109944,0cyclictest0-21swapper/107:39:231
132109943,0cyclictest8696-21awk07:14:121
132109943,0cyclictest4572-21awk09:49:111
132109943,0cyclictest3899-21awk07:59:131
132109943,0cyclictest3556-21lldpd12:19:051
132109943,0cyclictest3556-21lldpd11:32:351
132109943,0cyclictest3556-21lldpd10:34:021
132109943,0cyclictest3556-21lldpd09:45:001
132109943,0cyclictest3514-21snmpd11:14:361
132109943,0cyclictest3514-21snmpd11:04:211
132109943,0cyclictest3514-21snmpd10:33:211
132109943,0cyclictest3514-21snmpd10:14:061
132109943,0cyclictest3514-21snmpd10:13:061
132109943,0cyclictest3514-21snmpd09:31:361
132109943,0cyclictest3514-21snmpd09:19:061
132109943,0cyclictest3514-21snmpd09:14:361
132109943,0cyclictest3514-21snmpd09:06:361
132109943,0cyclictest3514-21snmpd08:51:061
132109943,0cyclictest3514-21snmpd08:19:061
132109943,0cyclictest3514-21snmpd08:09:061
132109943,0cyclictest3514-21snmpd07:09:061
132109943,0cyclictest3241-21irqbalance11:25:141
132109943,0cyclictest3241-21irqbalance09:34:151
132109943,0cyclictest3241-21irqbalance08:59:351
132109943,0cyclictest3241-21irqbalance08:56:251
132109943,0cyclictest3241-21irqbalance08:14:151
132109943,0cyclictest3241-21irqbalance08:04:151
132109943,0cyclictest2934-21awk07:34:121
132109943,0cyclictest29084-21awk09:24:111
132109943,0cyclictest21085-21awk06:59:141
132109943,0cyclictest16263-21awk07:44:111
132109943,0cyclictest11686-21awk11:34:111
132109943,0cyclictest11571-21awk08:29:131
132109943,0cyclictest11384-21awk11:54:101
132109943,0cyclictest11283-21awk12:14:111
132109943,0cyclictest10217-21sadc11:53:561
132089944,0cyclictest0-21swapper/009:56:060
132089943,0cyclictest9613-21awk07:39:110
132089943,0cyclictest5800-21awk08:49:110
132089943,0cyclictest4646-21awk09:49:110
132089943,0cyclictest3644-21hwlatdetect11:09:110
132089943,0cyclictest3514-21snmpd07:33:360
132089943,0cyclictest3241-21irqbalance11:24:340
132089943,0cyclictest3241-21irqbalance11:03:450
132089943,0cyclictest3241-21irqbalance10:37:550
132089943,0cyclictest3241-21irqbalance10:14:150
132089943,0cyclictest28992-21awk09:24:110
132089943,0cyclictest11453-21awk11:54:100
132089942,0cyclictest12544-21kworker/0:211:21:050
29725030,0irq/27-eth0-tx-0-21swapper/008:39:090
29715030,0irq/26-eth0-rx-0-21swapper/308:53:423
29715030,0irq/26-eth0-rx-0-21swapper/308:29:153
29715030,0irq/26-eth0-rx-0-21swapper/308:14:083
29715030,0irq/26-eth0-rx-0-21swapper/308:00:513
29715030,0irq/26-eth0-rx-0-21swapper/307:49:303
29715030,0irq/26-eth0-rx-0-21swapper/307:36:443
29715030,0irq/26-eth0-rx-0-21swapper/307:31:013
29715030,0irq/26-eth0-rx-0-21swapper/306:59:093
259930,0migration/223942-21if_eth1.208:14:122
179930,0migration/16434-21kthreadcore10:49:131
169930,0watchdog/129518-21idleruntime07:54:111
169930,0watchdog/126329-21ssh10:00:351
169930,0watchdog/10-21swapper/110:53:581
169930,0watchdog/10-21swapper/107:24:151
132119933,0cyclictest3241-21irqbalance07:12:452
132119932,0cyclictest6856-21cut08:49:132
132119932,0cyclictest3514-21snmpd08:39:062
132119932,0cyclictest3241-21irqbalance08:37:052
132109933,0cyclictest3556-21lldpd12:04:051
132109933,0cyclictest3556-21lldpd07:21:301
132109932,0cyclictest31453-21awk08:44:111
132109932,0cyclictest29258-21ls08:39:161
132109932,0cyclictest28654-21awk10:04:111
132109932,0cyclictest14279-21awk06:54:121
132109930,0cyclictest27767-21iostat_ios11:44:121
132109930,0cyclictest0-21swapper/108:34:171
132089933,0cyclictest15596-21kworker/0:208:34:140
132089933,0cyclictest12642-21kworker/0:110:24:040
132089933,0cyclictest11289-21kworker/0:112:04:130
132089933,0cyclictest11289-21kworker/0:111:59:110
132089932,0cyclictest3776-21munin-node09:29:080
132089932,0cyclictest3514-21snmpd08:24:060
132089932,0cyclictest3514-21snmpd08:24:060
132089932,0cyclictest3052-21irqcore07:34:130
29715020,0irq/26-eth0-rx-0-21swapper/307:44:083
29715020,0irq/26-eth0-rx-0-21swapper/307:14:093
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional