You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-12 - 05:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Tue May 12, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2560824132,0sleep10-21swapper/118:52:441
304450350,0irq/25-eth10-21swapper/318:51:143
2581723026,0sleep20-21swapper/218:55:132
2577523026,0sleep00-21swapper/018:54:390
29715070,0irq/26-eth0-rx-14269-21ssh00:09:493
4160,0ktimersoftd/00-21swapper/022:26:280
35160,0ktimersoftd/30-21swapper/321:30:303
339960,0migration/332119-21munin-node00:01:073
339960,0migration/331475-21sh20:58:553
339960,0migration/324293-21sh21:55:553
339960,0migration/323786-21sh22:56:043
339960,0migration/321587-21sh23:52:003
29725060,0irq/27-eth0-tx-10107-21munin-node20:21:070
29715060,0irq/26-eth0-rx-24570-21if_eth1.323:16:083
259529965,0cyclictest17474-21ls20:50:531
259519960,0cyclictest14639-21sh23:08:570
259519960,0cyclictest0-21swapper/021:07:240
259519960,0cyclictest0-21swapper/000:09:480
149960,0migration/02568-21kthreadcore23:01:100
4150,0ktimersoftd/00-21swapper/022:41:310
4150,0ktimersoftd/00-21swapper/022:02:250
339950,0migration/34192-21hddtemp_smartct20:16:073
339950,0migration/33640-21irqrtprio19:51:103
339950,0migration/33009-21kthreadcore19:01:103
339950,0migration/323719-21taskset21:34:153
339950,0migration/322493-21sh21:51:223
339950,0migration/321943-21latency21:11:123
339950,0migration/321898-21kthreadcore19:16:093
339950,0migration/318702-21kthreadcore23:31:113
339950,0migration/315996-21cpuspeed_turbos22:51:053
339950,0migration/31504-21munin-node19:26:053
339950,0migration/314380-21fschecks_count19:11:073
339950,0migration/314223-21munin-node21:46:193
339950,0migration/313584-21kthreadcore20:46:113
339950,0migration/31326-21latency_hist19:25:533
339950,0migration/31260-21irqcore21:41:093
339950,0migration/311773-21kthreadcore19:56:113
339950,0migration/310115-21kthreadcore21:06:113
29725050,0irq/27-eth0-tx-26014-21kthreadcore23:16:102
29725050,0irq/27-eth0-tx-24349-21munin-node21:56:172
29725050,0irq/27-eth0-tx-21233-21ssh00:11:512
29725050,0irq/27-eth0-tx-14834-21munin-node19:36:180
29725050,0irq/27-eth0-tx-12260-21latency00:06:112
29725050,0irq/27-eth0-tx-0-21swapper/223:42:342
29725050,0irq/27-eth0-tx-0-21swapper/223:39:062
29725050,0irq/27-eth0-tx-0-21swapper/222:48:512
29725050,0irq/27-eth0-tx-0-21swapper/221:17:072
29715050,0irq/26-eth0-rx-9812-21grep23:46:103
29715050,0irq/26-eth0-rx-8041-21crond21:05:533
29715050,0irq/26-eth0-rx-32531-21aten_r1power_vo22:01:043
29715050,0irq/26-eth0-rx-28173-21kthreadcore21:16:103
29715050,0irq/26-eth0-rx-24780-21fschecks_time22:16:073
29715050,0irq/26-eth0-rx-23539-21cut00:16:073
29715050,0irq/26-eth0-rx-23522-21ssh22:35:213
29715050,0irq/26-eth0-rx-1963-21grep22:41:103
29715050,0irq/26-eth0-rx-14975-21ssh22:28:403
29715050,0irq/26-eth0-rx-138192sleep323:27:113
29715050,0irq/26-eth0-rx-0-21swapper/323:56:583
29715050,0irq/26-eth0-rx-0-21swapper/323:43:033
29715050,0irq/26-eth0-rx-0-21swapper/323:23:143
29715050,0irq/26-eth0-rx-0-21swapper/323:06:113
29715050,0irq/26-eth0-rx-0-21swapper/322:36:283
29715050,0irq/26-eth0-rx-0-21swapper/322:22:003
29715050,0irq/26-eth0-rx-0-21swapper/322:06:343
29715050,0irq/26-eth0-rx-0-21swapper/321:25:043
29715050,0irq/26-eth0-rx-0-21swapper/300:22:413
259539955,0cyclictest0-21swapper/223:29:142
259539950,0cyclictest0-21swapper/223:10:572
259539950,0cyclictest0-21swapper/222:42:382
259539950,0cyclictest0-21swapper/221:50:512
259539950,0cyclictest0-21swapper/221:39:112
259529953,0cyclictest30889-21awk20:36:091
259529950,0cyclictest0-21swapper/123:00:141
259519955,0cyclictest0-21swapper/022:23:140
259519955,0cyclictest0-21swapper/021:17:570
259519952,0cyclictest11990-21awk20:46:090
259519950,0cyclictest0-21swapper/023:28:200
259950,0migration/28178-21fschecks_time23:06:052
259950,0migration/25666-21kthreadcore20:16:102
259950,0migration/232149-21kthreadcore20:11:112
259950,0migration/231201-21sh00:00:572
259950,0migration/230105-21munin-node20:56:182
259950,0migration/225730-21latency19:16:112
259950,0migration/222649-21unixbench-2d20:26:162
259950,0migration/217804-21kthreadcore19:36:112
259950,0migration/217433-21irqrtprio22:11:122
259950,0migration/216867-21irqcore20:01:112
259950,0migration/216840-21idleruntime22:31:072
259950,0migration/216181-21munin-node22:51:062
259950,0migration/215628-21cpuspeed_turbos23:31:062
249950,0watchdog/221294-21kthreadcore21:31:122
249950,0watchdog/20-21swapper/221:13:122
179950,0migration/19594-21kthreadcore19:06:101
179950,0migration/15041-21missed_timers22:41:111
179950,0migration/130929-21taskset21:36:541
179950,0migration/128714-21munin-node23:56:111
179950,0migration/113660-21latency21:46:141
179950,0migration/112993-21unixbench_singl00:06:181
169950,0watchdog/115333-21diskstats00:11:071
149950,0migration/05974-21sh00:04:030
149950,0migration/03823-21kthreadcore21:01:120
149950,0migration/03093-21fschecks_time19:51:070
149950,0migration/030645-21sh22:37:450
149950,0migration/029905-21unixbench_singl22:56:170
149950,0migration/027266-21kthreadcore22:16:110
149950,0migration/023267-21sh21:12:420
149950,0migration/013909-21switchtime19:31:170
149950,0migration/012137-21kthreadcore22:46:110
8215040,0irq/24-0000:00:0-21swapper/218:55:542
339940,0migration/38288-21aten_r1power_fr19:31:063
339940,0migration/329404-21unixbench-2d23:36:163
339940,0migration/325948-21kthreadcore20:31:113
339940,0migration/324351-21kthreadcore20:06:113
339940,0migration/322916-21runrttasks20:27:553
339940,0migration/310874-21kthreadcore19:06:103
29725040,0irq/27-eth0-tx-32705-21munin-node21:41:052
29725040,0irq/27-eth0-tx-31627-21ssh23:21:002
29725040,0irq/27-eth0-tx-2861-21kthreadcore00:21:102
29725040,0irq/27-eth0-tx-16625-21cpuspeed_turbos21:51:052
29725040,0irq/27-eth0-tx-0-21swapper/223:56:032
29725040,0irq/27-eth0-tx-0-21swapper/223:50:562
29725040,0irq/27-eth0-tx-0-21swapper/223:45:592
29725040,0irq/27-eth0-tx-0-21swapper/222:36:142
29725040,0irq/27-eth0-tx-0-21swapper/222:21:042
29725040,0irq/27-eth0-tx-0-21swapper/222:16:072
29725040,0irq/27-eth0-tx-0-21swapper/222:06:092
29725040,0irq/27-eth0-tx-0-21swapper/221:25:572
29725040,0irq/27-eth0-tx-0-21swapper/221:01:002
29725040,0irq/27-eth0-tx-0-21swapper/020:16:040
29725040,0irq/27-eth0-tx-0-21swapper/019:41:110
29725040,0irq/27-eth0-tx-0-21swapper/019:11:050
29725040,0irq/27-eth0-tx-0-21swapper/019:01:070
29715040,0irq/26-eth0-rx-23304-21grep19:41:113
29715040,0irq/26-eth0-rx-21578-21timerandwakeup20:01:183
29715040,0irq/26-eth0-rx-0-21swapper/323:11:573
29715040,0irq/26-eth0-rx-0-21swapper/323:02:133
29715040,0irq/26-eth0-rx-0-21swapper/322:46:393
29715040,0irq/26-eth0-rx-0-21swapper/322:11:123
29715040,0irq/26-eth0-rx-0-21swapper/321:36:233
29715040,0irq/26-eth0-rx-0-21swapper/320:36:123
29715040,0irq/26-eth0-rx-0-21swapper/320:21:123
29715040,0irq/26-eth0-rx-0-21swapper/300:11:573
259539943,0cyclictest9108-21awk19:31:092
259539943,0cyclictest3514-21snmpd20:43:042
259539943,0cyclictest3514-21snmpd19:51:342
259539943,0cyclictest3514-21snmpd19:41:042
259539943,0cyclictest3514-21snmpd19:11:042
259539943,0cyclictest3514-21snmpd19:01:342
259539943,0cyclictest3241-21irqbalance00:16:012
259539943,0cyclictest1666-21cut22:01:092
259539941,0cyclictest0-21swapper/222:56:052
259529944,0cyclictest0-21swapper/123:20:531
259529944,0cyclictest0-21swapper/122:40:221
259529943,0cyclictest9486-21awk21:26:101
259529943,0cyclictest9001-21awk22:06:071
259529943,0cyclictest873-21awk22:01:071
259529943,0cyclictest8537-21awk23:06:071
259529943,0cyclictest3556-21lldpd23:44:561
259529943,0cyclictest3556-21lldpd19:55:551
259529943,0cyclictest3556-21lldpd19:31:241
259529943,0cyclictest3556-21lldpd00:01:261
259529943,0cyclictest3514-21snmpd22:51:041
259529943,0cyclictest3514-21snmpd22:21:041
259529943,0cyclictest3514-21snmpd19:28:341
259529943,0cyclictest3241-21irqbalance22:12:121
259529943,0cyclictest3241-21irqbalance21:06:321
259529943,0cyclictest31891-21awk00:21:071
259529943,0cyclictest30007-21awk20:11:091
259529943,0cyclictest28030-21awk19:21:081
259529943,0cyclictest24295-21awk20:31:091
259529943,0cyclictest17304-21awk21:51:081
259529943,0cyclictest15669-21awk19:36:091
259529940,0cyclictest0-21swapper/121:42:301
259519944,0cyclictest0-21swapper/023:21:150
259519944,0cyclictest0-21swapper/022:51:270
259519944,0cyclictest0-21swapper/021:48:320
259519943,0cyclictest8985-21awk22:06:060
259519943,0cyclictest8042-21awk23:46:080
259519943,0cyclictest3556-21lldpd19:05:550
259519943,0cyclictest3514-21snmpd23:40:490
259519943,0cyclictest3514-21snmpd21:51:490
259519943,0cyclictest3514-21snmpd21:39:490
259519943,0cyclictest3514-21snmpd19:26:040
259519943,0cyclictest24393-21awk20:31:090
259519943,0cyclictest24137-21awk23:56:070
259519943,0cyclictest18555-21awk20:51:090
259519943,0cyclictest16964-21awk22:31:080
259519943,0cyclictest16388-21awk23:31:090
259519940,0cyclictest16738-21munin-node20:26:060
259940,0migration/2576-21aten_r1power_vo21:21:062
259940,0migration/23761-21kthreadcore19:26:102
259940,0migration/228402-21cpuspeed_turbos19:46:052
259940,0migration/223882-21kthreadcore20:06:102
259940,0migration/215636-21proc_pri20:21:152
259940,0migration/213009-21kthreadcore22:26:112
259940,0migration/210996-21latency_hist20:45:532
259940,0migration/21046-21kthreadcore20:36:112
249940,0watchdog/29903-21sh19:56:082
249940,0watchdog/212025-21cut21:06:122
249940,0watchdog/20-21swapper/223:01:032
249940,0watchdog/20-21swapper/219:21:082
179940,0migration/19030-21irqcore22:26:091
179940,0migration/18933-21kthreadcore20:41:111
179940,0migration/16749-21taskset21:21:241
179940,0migration/126122-21kthreadcore21:56:111
179940,0migration/124352-21aten2.4-expect22:16:051
179940,0migration/119231-21kthreadcore22:31:101
179940,0migration/117892-21kthreadcore23:31:111
179940,0migration/117793-21irqcore21:11:101
179940,0migration/117291-21munin-node21:31:081
179940,0migration/116549-21gltestperf20:01:091
179940,0migration/114215-21diskmemload23:28:211
169940,0watchdog/18430-21gltestperf22:46:071
169940,0watchdog/131818-21cut23:01:031
169940,0watchdog/13161-21kthreadcore21:01:121
169940,0watchdog/120395-21kthreadcore23:11:111
169940,0watchdog/10-21swapper/123:51:111
169940,0watchdog/10-21swapper/123:16:111
169940,0watchdog/10-21swapper/120:21:121
149940,0migration/0673-21fschecks_count21:41:060
149940,0migration/05894-21kthreadcore20:41:100
149940,0migration/04660-21kthreadcore21:21:110
149940,0migration/032447-21kthreadcore20:11:110
149940,0migration/031798-21kthreadcore20:36:110
149940,0migration/026560-21kthreadcore20:06:110
149940,0migration/025133-21hddtemp_smartct21:56:080
339930,0migration/32863-21taskset20:12:483
29715030,0irq/26-eth0-rx-0-21swapper/320:41:493
29715030,0irq/26-eth0-rx-0-21swapper/319:46:053
29715030,0irq/26-eth0-rx-0-21swapper/318:56:043
259539932,0cyclictest3514-21snmpd19:06:042
259539932,0cyclictest29040-21sensors20:31:152
259529932,0cyclictest6168-21sensors19:01:151
259529932,0cyclictest28149-21sensors20:06:151
259529932,0cyclictest19505-21sensors19:11:151
259529932,0cyclictest18658-21awk20:51:091
259529932,0cyclictest1394-21sensors19:46:151
259529931,0cyclictest16981-21cpuspeed_turbos20:26:061
259529930,0cyclictest3052-21perl19:51:071
259519932,0cyclictest9998-21awk19:56:090
259519932,0cyclictest9010-21entropy21:26:080
259519932,0cyclictest3556-21lldpd19:17:550
259519932,0cyclictest3241-21irqbalance23:52:510
259519932,0cyclictest3241-21irqbalance22:12:020
259519932,0cyclictest31572-21aten_r1power_vo23:41:050
259519932,0cyclictest29027-21awk19:46:080
259519932,0cyclictest25247-21awk20:56:090
259519932,0cyclictest24110-21diskmemload00:21:150
259519932,0cyclictest23878-21awk00:16:090
259519932,0cyclictest17549-21awk21:31:090
259519932,0cyclictest16675-21awk20:01:100
259519932,0cyclictest15691-21cut00:11:080
259519932,0cyclictest0-21swapper/023:19:020
259519930,0cyclictest0-21swapper/019:21:060
259519930,0cyclictest0-21swapper/018:59:510
249930,0watchdog/222840-21kthreadcore20:51:122
179930,0migration/15501-21kthreadcore20:16:101
179930,0migration/130600-21sh20:56:431
179930,0migration/121356-21if_err_eth119:16:071
169930,0watchdog/121484-21munin-node19:41:161
169930,0watchdog/10-21swapper/123:46:111
169930,0watchdog/10-21swapper/123:36:031
169930,0watchdog/10-21swapper/121:16:441
169930,0watchdog/10-21swapper/118:56:081
169930,0watchdog/10-21swapper/100:16:111
35120,0ktimersoftd/316883-21kthreadcore19:36:113
29715020,0irq/26-eth0-rx-0-21swapper/320:51:053
259519921,0cyclictest15659-21aten2.4-expect23:11:050
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional