You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-04 - 03:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sat Jul 04, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3078125240,0sleep20-21swapper/218:49:512
3078424231,0sleep00-21swapper/018:49:530
304450370,0irq/25-eth10-21swapper/318:46:073
3054323525,0sleep10-21swapper/118:47:061
309389976,0cyclictest0-21swapper/119:51:071
35160,0ktimersoftd/30-21swapper/300:07:283
339960,0migration/36862-21munin-node19:21:043
339960,0migration/328904-21sh23:30:563
339960,0migration/327731-21sh23:28:143
339960,0migration/322216-21irqcore22:46:063
339960,0migration/311003-21sh00:17:313
309429960,0cyclictest0-21swapper/223:02:202
29725060,0irq/27-eth0-tx-5341-21df_abs22:36:002
29725060,0irq/27-eth0-tx-1269-21grep21:11:072
29715060,0irq/26-eth0-rx-5166-21diskstats23:16:013
179960,0migration/116353-21kthreadcore22:41:051
149960,0migration/06231-21kthreadcore00:16:050
149960,0migration/027901-21sh21:47:130
149960,0migration/022196-21munin-node22:46:060
149960,0migration/019782-21sh22:02:130
149960,0migration/019627-21sh23:02:370
149960,0migration/011155-21switchtime21:36:130
8215050,0irq/24-0000:00:4786-21cat22:35:502
8215050,0irq/24-0000:00:29200-21cat20:50:502
8215050,0irq/24-0000:00:21371-21munin-run21:25:502
339950,0migration/3858-21kthreadcore23:11:063
339950,0migration/36623-21kthreadcore20:56:063
339950,0migration/36139-21irqcore22:16:063
339950,0migration/34941-21sh22:56:013
339950,0migration/332694-21kthreadcore21:31:073
339950,0migration/331092-21latency_hist18:50:513
339950,0migration/330133-21fschecks_time21:51:033
339950,0migration/329530-21cpuspeed_turbos22:31:013
339950,0migration/328629-21taskset21:08:523
339950,0migration/328376-21runrttasks20:26:013
339950,0migration/327807-21taskset21:46:573
339950,0migration/326853-21switchtime23:06:113
339950,0migration/321459-21ksmtuned20:20:353
339950,0migration/320217-21sh21:03:133
339950,0migration/319026-21timerandwakeup22:41:113
339950,0migration/314294-21irqcore22:01:053
339950,0migration/313779-21gltestperf23:01:033
339950,0migration/313556-21munin-node23:41:033
339950,0migration/313295-21sh22:21:033
339950,0migration/312286-21taskset21:17:563
309429955,0cyclictest31426-21unixbench_singl19:11:122
309429955,0cyclictest0-21swapper/221:29:382
309429955,0cyclictest0-21swapper/220:36:012
309429950,0cyclictest0-21swapper/223:15:152
309429950,0cyclictest0-21swapper/222:53:022
309429950,0cyclictest0-21swapper/200:19:352
309389955,0cyclictest0-21swapper/121:01:261
309389950,0cyclictest0-21swapper/121:42:261
309369950,0cyclictest0-21swapper/023:33:540
309369950,0cyclictest0-21swapper/022:21:040
309369950,0cyclictest0-21swapper/021:21:570
29725050,0irq/27-eth0-tx-6445-21munin-node19:21:050
29725050,0irq/27-eth0-tx-6201-21irqcore21:56:032
29725050,0irq/27-eth0-tx-28930-21munin-node23:31:022
29725050,0irq/27-eth0-tx-26603-21open_files23:06:082
29725050,0irq/27-eth0-tx-21774-21if_err_eth123:26:042
29725050,0irq/27-eth0-tx-14386-21iostat21:41:052
29725050,0irq/27-eth0-tx-12117-21munin-node19:01:030
29725050,0irq/27-eth0-tx-0-21swapper/222:11:132
29725050,0irq/27-eth0-tx-0-21swapper/222:06:002
29725050,0irq/27-eth0-tx-0-21swapper/221:36:082
29725050,0irq/27-eth0-tx-0-21swapper/200:06:282
29725050,0irq/27-eth0-tx-0-21swapper/019:36:010
29725050,0irq/27-eth0-tx-0-21swapper/019:30:100
29715050,0irq/26-eth0-rx-4393-21latency_hist23:35:503
29715050,0irq/26-eth0-rx-22597-21kthreadcore19:06:073
29715050,0irq/26-eth0-rx-19231-21ssh23:21:523
29715050,0irq/26-eth0-rx-18316-21rt-features00:01:093
29715050,0irq/26-eth0-rx-15896-21kthreadcore19:51:073
29715050,0irq/26-eth0-rx-0-21swapper/323:56:443
29715050,0irq/26-eth0-rx-0-21swapper/323:52:543
29715050,0irq/26-eth0-rx-0-21swapper/323:47:433
29715050,0irq/26-eth0-rx-0-21swapper/322:54:293
29715050,0irq/26-eth0-rx-0-21swapper/322:35:523
29715050,0irq/26-eth0-rx-0-21swapper/321:26:053
29715050,0irq/26-eth0-rx-0-21swapper/321:22:323
29715050,0irq/26-eth0-rx-0-21swapper/321:12:093
29715050,0irq/26-eth0-rx-0-21swapper/300:11:023
259950,0migration/28466-21munin-node19:46:052
259950,0migration/27428-21kthreadcore21:16:082
259950,0migration/228631-21sh22:49:592
259950,0migration/221837-21fschecks_time22:26:022
259950,0migration/218568-21kthreadcore20:41:072
259950,0migration/217115-21kthreadcore19:26:082
259950,0migration/211008-21sh23:16:292
179950,0migration/17763-21kthreadcore22:16:071
179950,0migration/12323-21taskset20:30:301
179950,0migration/113744-21fschecks_time22:21:041
179950,0migration/112038-21sh00:00:081
169950,0watchdog/128318-21if_eth120:01:051
169950,0watchdog/126573-21grep22:46:081
149950,0migration/08764-21kthreadcore23:56:070
149950,0migration/08748-21kernelversion19:46:060
149950,0migration/06497-21kthreadcore22:56:070
149950,0migration/06373-21irqrtprio21:56:030
149950,0migration/05408-21fschecks_count20:56:030
149950,0migration/028773-21latency_hist23:50:500
149950,0migration/028763-21aten_r1power_po00:10:590
149950,0migration/026224-21kthreadcore21:26:060
149950,0migration/025789-21kthreadcore23:06:060
149950,0migration/02546-21kthreadcore18:51:080
149950,0migration/024446-21kthreadcore20:21:080
149950,0migration/023948-21kthreadcore21:06:070
149950,0migration/023120-21kthreadcore22:06:060
149950,0migration/022392-21kthreadcore23:46:060
149950,0migration/020637-21dump-pmu-power23:25:500
149950,0migration/016339-21irqrtprio20:16:060
149950,0migration/01563-21kthreadcore21:51:070
149950,0migration/014993-21kthreadcore22:41:040
149950,0migration/014747-21taskset20:14:520
149950,0migration/014337-21kthreadcore23:41:050
149950,0migration/012715-21aten_r1power_vo00:01:000
149950,0migration/011640-21taskset18:58:070
339940,0migration/37537-21kthreadcore20:31:093
339940,0migration/35905-21gltestperf21:56:013
339940,0migration/32878-21kthreadcore19:16:073
339940,0migration/324069-21kthreadcore19:56:073
309429944,0cyclictest0-21swapper/223:45:512
309429943,0cyclictest7347-21awk19:21:062
309429943,0cyclictest384-21sadc19:40:502
309429943,0cyclictest3564-21awk20:31:062
309429943,0cyclictest3556-21lldpd19:53:552
309429943,0cyclictest3241-21irqbalance20:11:192
309429943,0cyclictest3241-21irqbalance20:01:092
309429943,0cyclictest3241-21irqbalance19:56:192
309429943,0cyclictest3241-21irqbalance19:16:392
309429943,0cyclictest2629-21awk20:06:062
309429943,0cyclictest19748-21awk19:06:052
309429943,0cyclictest13049-21awk19:01:052
309429940,0cyclictest11894-21grep19:00:512
309389944,0cyclictest0-21swapper/121:06:101
309389943,0cyclictest9308-21awk20:11:041
309389943,0cyclictest7372-21awk19:21:061
309389943,0cyclictest617-21awk19:16:051
309389943,0cyclictest3556-21lldpd23:25:541
309389943,0cyclictest3556-21lldpd23:05:551
309389943,0cyclictest3556-21lldpd22:25:551
309389943,0cyclictest3556-21lldpd22:11:551
309389943,0cyclictest3556-21lldpd21:46:551
309389943,0cyclictest3556-21lldpd21:25:551
309389943,0cyclictest3556-21lldpd21:13:251
309389943,0cyclictest3556-21lldpd20:21:551
309389943,0cyclictest3556-21lldpd19:40:551
309389943,0cyclictest3556-21lldpd19:05:541
309389943,0cyclictest3556-21lldpd00:01:241
309389943,0cyclictest3241-21irqbalance23:51:081
309389943,0cyclictest30300-21awk21:51:041
309389943,0cyclictest30232-21awk20:51:071
309389943,0cyclictest29963-21awk23:11:031
309389943,0cyclictest29646-21awk00:11:031
309389943,0cyclictest23545-21awk20:46:051
309389943,0cyclictest21523-21awk00:06:041
309369944,0cyclictest15837-21kworker/0:219:09:140
309369944,0cyclictest0-21swapper/022:18:330
309369943,0cyclictest6208-21sadc19:20:500
309369943,0cyclictest3556-21lldpd23:37:240
309369943,0cyclictest3556-21lldpd23:11:550
309369943,0cyclictest3514-21snmpd21:00:570
309369943,0cyclictest3514-21snmpd20:45:570
309369943,0cyclictest3514-21snmpd20:38:420
309369943,0cyclictest3514-21snmpd19:31:120
309369943,0cyclictest3241-21irqbalance23:16:080
309369943,0cyclictest3241-21irqbalance22:30:580
309369943,0cyclictest3241-21irqbalance22:26:280
309369943,0cyclictest3241-21irqbalance21:31:590
309369943,0cyclictest3241-21irqbalance21:16:290
309369943,0cyclictest3241-21irqbalance21:11:090
309369943,0cyclictest3241-21irqbalance00:07:480
309369943,0cyclictest21760-21awk23:26:040
309369943,0cyclictest14342-21awk21:41:040
29725040,0irq/27-eth0-tx-29433-21ssh21:50:582
29725040,0irq/27-eth0-tx-21502-21ssh21:05:572
29725040,0irq/27-eth0-tx-0-21swapper/223:58:282
29725040,0irq/27-eth0-tx-0-21swapper/223:50:552
29725040,0irq/27-eth0-tx-0-21swapper/223:42:052
29725040,0irq/27-eth0-tx-0-21swapper/223:36:082
29725040,0irq/27-eth0-tx-0-21swapper/222:56:022
29725040,0irq/27-eth0-tx-0-21swapper/222:41:052
29725040,0irq/27-eth0-tx-0-21swapper/222:17:472
29725040,0irq/27-eth0-tx-0-21swapper/222:01:022
29725040,0irq/27-eth0-tx-0-21swapper/221:01:212
29725040,0irq/27-eth0-tx-0-21swapper/220:56:122
29725040,0irq/27-eth0-tx-0-21swapper/200:01:062
29725040,0irq/27-eth0-tx-0-21swapper/020:45:220
29725040,0irq/27-eth0-tx-0-21swapper/019:41:120
29715040,0irq/26-eth0-rx-10564-21kthreadcore20:11:063
29715040,0irq/26-eth0-rx-0-21swapper/322:26:003
29715040,0irq/26-eth0-rx-0-21swapper/322:11:003
29715040,0irq/26-eth0-rx-0-21swapper/322:06:103
29715040,0irq/26-eth0-rx-0-21swapper/321:43:473
29715040,0irq/26-eth0-rx-0-21swapper/321:36:113
29715040,0irq/26-eth0-rx-0-21swapper/320:53:153
29715040,0irq/26-eth0-rx-0-21swapper/318:55:513
259940,0migration/2965-21kthreadcore00:11:052
259940,0migration/25148-21kthreadcore19:41:072
259940,0migration/231970-21kthreadcore21:31:062
259940,0migration/231728-21fschecks_time18:51:042
259940,0migration/229759-21kthreadcore20:26:072
259940,0migration/225019-21kthreadcore20:21:082
259940,0migration/222168-21gltestperf21:46:022
259940,0migration/221338-21kthreadcore19:31:062
259940,0migration/216781-21kthreadcore20:16:062
259940,0migration/215969-21kthreadcore23:21:072
259940,0migration/213582-21munin-node22:21:042
259940,0migration/21151-21kthreadcore20:51:102
179940,0migration/1950-21kthreadcore18:51:081
179940,0migration/16712-21kthreadcore20:56:061
179940,0migration/1446-21kthreadcore21:31:071
179940,0migration/126295-21missed_timers23:46:081
179940,0migration/119184-21sh23:01:201
179940,0migration/117507-21kthreadcore19:26:081
179940,0migration/117355-21kthreadcore23:21:071
179940,0migration/116282-21kthreadcore22:01:061
179940,0migration/113538-21hwlatdetect23:41:031
179940,0migration/111204-21vmstat22:36:111
169940,0watchdog/18951-21kthreadcore21:36:081
169940,0watchdog/13854-21ssh22:33:201
169940,0watchdog/130109-21munin-node22:51:041
169940,0watchdog/125594-21aten_r1power_vo19:11:011
169940,0watchdog/120733-21runrttasks21:24:041
169940,0watchdog/10-21swapper/123:36:121
169940,0watchdog/10-21swapper/123:32:001
169940,0watchdog/10-21swapper/122:56:081
169940,0watchdog/10-21swapper/121:16:081
169940,0watchdog/10-21swapper/120:31:091
149940,0migration/07727-21kthreadcore22:36:040
149940,0migration/04271-21kthreadcore20:31:080
149940,0migration/032145-21kthreadcore20:51:090
149940,0migration/01624-21latency_hist20:05:500
149940,0migration/015871-21kthreadcore19:51:070
35130,0ktimersoftd/30-21swapper/319:34:343
35130,0ktimersoftd/30-21swapper/319:11:513
339930,0migration/326442-21runrttasks19:35:583
339930,0migration/315643-21runrttasks20:39:303
309389932,0cyclictest8419-21awk19:46:051
309389932,0cyclictest8004-21kthreadcore18:56:071
309389932,0cyclictest3556-21lldpd23:15:551
309389932,0cyclictest3556-21lldpd22:05:551
309389932,0cyclictest3556-21lldpd20:15:551
309389932,0cyclictest3556-21lldpd19:57:251
309389932,0cyclictest3556-21lldpd19:31:251
309389932,0cyclictest16941-21awk20:41:051
309389930,0cyclictest5557-21diskstats21:56:001
309389930,0cyclictest4364-21munin-node00:16:081
309369933,0cyclictest15837-21kworker/0:222:50:570
309369933,0cyclictest15837-21kworker/0:222:11:310
309369933,0cyclictest15837-21kworker/0:220:25:520
309369933,0cyclictest15837-21kworker/0:220:00:520
309369933,0cyclictest15837-21kworker/0:219:10:520
309369932,0cyclictest23891-21cut19:56:070
29715030,0irq/26-eth0-rx-0-21swapper/320:21:463
29715030,0irq/26-eth0-rx-0-21swapper/320:07:283
29715030,0irq/26-eth0-rx-0-21swapper/320:01:063
29715030,0irq/26-eth0-rx-0-21swapper/319:46:003
29715030,0irq/26-eth0-rx-0-21swapper/319:41:133
29715030,0irq/26-eth0-rx-0-21swapper/319:26:023
29715030,0irq/26-eth0-rx-0-21swapper/319:02:433
169930,0watchdog/10-21swapper/120:36:091
169930,0watchdog/10-21swapper/120:06:051
169930,0watchdog/10-21swapper/119:38:331
169930,0watchdog/10-21swapper/119:01:011
35120,0ktimersoftd/323606-21if_eth120:46:053
29715020,0irq/26-eth0-rx-0-21swapper/320:41:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional