You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-22 - 23:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Wed Oct 22, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
291350910,0irq/25-eth10-21swapper/306:36:073
1503427271,0sleep00-21swapper/006:40:400
1485524334,0sleep20-21swapper/206:38:342
1494523324,0sleep10-21swapper/106:39:371
15113991515,0cyclictest11656-21sensors11:26:102
15113991212,0cyclictest24027-21sensors07:56:062
28415070,0irq/27-eth0-tx-0-21swapper/007:07:090
151139970,0cyclictest0-21swapper/208:40:592
151129970,0cyclictest0-21swapper/109:41:011
339960,0migration/33760-21sh11:42:543
339960,0migration/316773-21kthreadcore11:31:013
339960,0migration/312315-21sh09:00:403
339960,0migration/311327-21sh09:45:063
28415060,0irq/27-eth0-tx-7531-21ssh10:44:282
28415060,0irq/27-eth0-tx-15332-21munin-node06:41:050
28415060,0irq/27-eth0-tx-0-21swapper/212:07:062
28415060,0irq/27-eth0-tx-0-21swapper/212:02:362
28415060,0irq/27-eth0-tx-0-21swapper/211:51:012
28415060,0irq/27-eth0-tx-0-21swapper/211:45:472
28415060,0irq/27-eth0-tx-0-21swapper/211:36:152
28415060,0irq/27-eth0-tx-0-21swapper/211:12:242
28415060,0irq/27-eth0-tx-0-21swapper/210:38:512
28415060,0irq/27-eth0-tx-0-21swapper/210:06:112
28415060,0irq/27-eth0-tx-0-21swapper/209:55:592
28415060,0irq/27-eth0-tx-0-21swapper/209:50:382
28415060,0irq/27-eth0-tx-0-21swapper/209:39:272
28415060,0irq/27-eth0-tx-0-21swapper/209:30:082
28415060,0irq/27-eth0-tx-0-21swapper/209:01:032
28415060,0irq/27-eth0-tx-0-21swapper/208:58:332
28415060,0irq/27-eth0-tx-0-21swapper/208:46:252
28405060,0irq/26-eth0-rx-3334-21ssh09:15:333
28405060,0irq/26-eth0-rx-27162-21df09:30:593
28405060,0irq/26-eth0-rx-1982-21cut08:51:053
28405060,0irq/26-eth0-rx-17353-21ntp_offset09:46:053
249960,0watchdog/210702-21cut08:36:032
249960,0watchdog/20-21swapper/211:42:232
249960,0watchdog/20-21swapper/211:16:592
249960,0watchdog/20-21swapper/210:17:312
151139966,0cyclictest13510-21diskmemload10:51:412
151129965,0cyclictest29280-21cat08:50:441
149960,0migration/026099-21aten_r1power_cu10:15:560
149960,0migration/025417-21switchtime09:51:100
149960,0migration/018508-21sh09:24:480
149960,0migration/013246-21sh11:30:170
8225050,0irq/24-0000:00:0-21swapper/209:06:302
339950,0migration/39185-21taskset10:02:053
339950,0migration/38949-21munin-node06:56:103
339950,0migration/35490-21kthreadcore07:21:013
339950,0migration/35372-21df_abs07:46:043
339950,0migration/33438-21kthreadcore10:21:033
339950,0migration/331351-21runrttasks11:21:023
339950,0migration/330693-21sh10:58:373
339950,0migration/32921-21kthreadcore10:41:013
339950,0migration/328846-21kthreadcore07:36:033
339950,0migration/328212-21taskset06:48:403
339950,0migration/327579-21kthreadcore08:01:023
339950,0migration/326109-21diskmemload09:29:213
339950,0migration/325285-21timerwakeupswit08:21:063
339950,0migration/324695-21taskset10:35:323
339950,0migration/324638-21kthreadcore07:11:013
339950,0migration/323967-21munin-node10:11:073
339950,0migration/32370-21taskset09:36:433
339950,0migration/320349-21taskset09:05:513
339950,0migration/317793-21taskset07:51:323
339950,0migration/316651-21unixbench_singl10:06:093
339950,0migration/31540-21runrttasks09:57:523
339950,0migration/315089-21munin-node07:01:053
339950,0migration/311963-21taskset11:48:053
339950,0migration/311724-21switchtime11:26:113
339950,0migration/310969-21taskset09:20:083
28415050,0irq/27-eth0-tx-8875-21vmstat10:01:102
28415050,0irq/27-eth0-tx-4005-21ssh08:54:052
28415050,0irq/27-eth0-tx-3931-21munin-node07:21:050
28415050,0irq/27-eth0-tx-26121-21munin-node08:26:110
28415050,0irq/27-eth0-tx-13640-21munin-node11:30:582
28415050,0irq/27-eth0-tx-10626-21munin-node07:26:060
28415050,0irq/27-eth0-tx-0-21swapper/211:56:122
28415050,0irq/27-eth0-tx-0-21swapper/210:56:212
28415050,0irq/27-eth0-tx-0-21swapper/209:51:452
28415050,0irq/27-eth0-tx-0-21swapper/008:42:030
28415050,0irq/27-eth0-tx-0-21swapper/008:06:050
28415050,0irq/27-eth0-tx-0-21swapper/006:46:060
28405050,0irq/26-eth0-rx-28967-21sh11:40:403
28405050,0irq/26-eth0-rx-27427-21kthreadcore10:16:043
28405050,0irq/26-eth0-rx-20166-21perl09:51:013
28405050,0irq/26-eth0-rx-20027-21ssh09:05:323
28405050,0irq/26-eth0-rx-15871-21fschecks_time06:41:013
28405050,0irq/26-eth0-rx-15663-21kthreadcore11:51:013
28405050,0irq/26-eth0-rx-0-21swapper/312:06:093
28405050,0irq/26-eth0-rx-0-21swapper/311:18:573
28405050,0irq/26-eth0-rx-0-21swapper/311:01:323
28405050,0irq/26-eth0-rx-0-21swapper/310:27:283
28405050,0irq/26-eth0-rx-0-21swapper/308:40:583
28405050,0irq/26-eth0-rx-0-21swapper/307:15:593
259950,0migration/29165-21idleruntime-cro10:25:442
259950,0migration/26794-21runrttasks08:35:112
259950,0migration/24706-21munin-node07:20:592
259950,0migration/211874-21kthreadcore11:06:032
249950,0watchdog/20-21swapper/211:04:352
249950,0watchdog/20-21swapper/210:48:552
249950,0watchdog/20-21swapper/210:31:552
249950,0watchdog/20-21swapper/210:25:592
249950,0watchdog/20-21swapper/210:10:472
249950,0watchdog/20-21swapper/209:42:392
249950,0watchdog/20-21swapper/209:21:472
249950,0watchdog/20-21swapper/208:23:552
249950,0watchdog/20-21swapper/207:47:032
249950,0watchdog/20-21swapper/207:35:592
249950,0watchdog/20-21swapper/207:31:152
19150,0ktimersoftd/10-21swapper/111:40:511
19150,0ktimersoftd/10-21swapper/109:52:291
179950,0migration/15658-21kthreadcore10:21:041
179950,0migration/15239-21timerwakeupswit11:01:101
179950,0migration/125043-21kthreadcore11:16:031
179950,0migration/118842-21kthreadcore07:31:041
179950,0migration/115695-21kthreadcore07:26:031
169950,0watchdog/17723-21kthreadcore12:06:031
169950,0watchdog/17686-21fschecks_time11:05:591
169950,0watchdog/114323-21kthreadcore10:06:031
151139955,0cyclictest0-21swapper/211:22:072
151139955,0cyclictest0-21swapper/209:16:272
151139952,0cyclictest28191-21munin-node09:10:582
151139950,0cyclictest0-21swapper/209:31:542
151129955,0cyclictest8499-21kworker/1:210:36:021
151129955,0cyclictest8499-21kworker/1:208:56:001
151129955,0cyclictest0-21swapper/109:38:581
151129955,0cyclictest0-21swapper/109:24:571
151129954,0cyclictest8499-21kworker/1:210:40:461
151129954,0cyclictest8499-21kworker/1:210:27:561
151129954,0cyclictest8499-21kworker/1:208:51:321
151129954,0cyclictest8499-21kworker/1:208:44:481
151129954,0cyclictest8041-21awk08:36:011
151129954,0cyclictest5853-21kworker/1:112:03:381
151129954,0cyclictest5853-21kworker/1:111:59:441
151129954,0cyclictest5853-21kworker/1:111:51:161
151129950,0cyclictest0-21swapper/108:05:571
151119955,0cyclictest0-21swapper/011:04:510
151119955,0cyclictest0-21swapper/007:41:090
151119950,0cyclictest2945-21munin-node06:56:000
151119950,0cyclictest0-21swapper/009:15:070
149950,0migration/09110-21kthreadcore12:06:030
149950,0migration/09048-21idleruntime10:46:000
149950,0migration/07500-21kthreadcore08:11:050
149950,0migration/05150-21aten_r1power_fr11:45:550
149950,0migration/04467-21taskset11:44:530
149950,0migration/0372-21kthreadcore06:51:040
149950,0migration/031450-21taskset08:05:350
149950,0migration/028688-21kthreadcore09:31:030
149950,0migration/025615-21kthreadcore11:56:030
149950,0migration/024911-21kthreadcore11:16:030
149950,0migration/024415-21sh10:34:380
149950,0migration/023564-21taskset07:33:410
149950,0migration/020368-21sh11:53:530
149950,0migration/019576-21kthreadcore11:11:050
149950,0migration/016245-21kthreadcore11:31:010
149950,0migration/014714-21kthreadcore09:46:020
149950,0migration/010031-21ksmtuned07:00:580
339940,0migration/39158-21kthreadcore08:36:033
339940,0migration/35341-21kthreadcore08:31:033
339940,0migration/328429-21kthreadcore08:26:053
339940,0migration/320891-21kthreadcore07:31:043
339940,0migration/320755-21kthreadcore07:56:023
339940,0migration/316174-21kthreadcore09:21:023
339940,0migration/314889-21diskmemload11:10:493
339940,0migration/311076-21fschecks_count07:25:593
28415040,0irq/27-eth0-tx-0-21swapper/007:55:590
28405040,0irq/26-eth0-rx-0-21swapper/312:01:213
28405040,0irq/26-eth0-rx-0-21swapper/311:55:493
28405040,0irq/26-eth0-rx-0-21swapper/311:06:303
28405040,0irq/26-eth0-rx-0-21swapper/310:50:513
28405040,0irq/26-eth0-rx-0-21swapper/310:45:573
28405040,0irq/26-eth0-rx-0-21swapper/310:36:033
28405040,0irq/26-eth0-rx-0-21swapper/308:46:063
28405040,0irq/26-eth0-rx-0-21swapper/308:06:033
259940,0migration/215445-21aten_r1power_po06:40:592
259940,0migration/21145-21kthreadcore06:51:052
249940,0watchdog/231942-21wc08:26:112
249940,0watchdog/20-21swapper/208:11:032
249940,0watchdog/20-21swapper/207:51:032
249940,0watchdog/20-21swapper/207:16:072
249940,0watchdog/20-21swapper/207:11:072
249940,0watchdog/20-21swapper/207:02:552
249940,0watchdog/20-21swapper/206:55:592
249940,0watchdog/20-21swapper/206:46:032
179940,0migration/15334-21kthreadcore09:16:011
179940,0migration/131573-21if_eth107:41:011
179940,0migration/128593-21kthreadcore08:26:051
179940,0migration/125734-21kthreadcore07:36:021
179940,0migration/122502-21irqcore11:36:031
179940,0migration/113470-21gltestperf08:16:041
179940,0migration/113470-21gltestperf08:16:041
169940,0watchdog/128916-21cpuspeed_turbos06:50:591
169940,0watchdog/121058-21perl09:05:591
169940,0watchdog/111589-21kthreadcore10:46:031
151139943,0cyclictest6445-21snmpd08:16:312
151139943,0cyclictest6445-21snmpd08:16:312
151139943,0cyclictest3425-21lldpd07:06:012
151139943,0cyclictest25913-21awk08:01:002
151139943,0cyclictest11516-21awk07:26:012
151129944,0cyclictest8499-21kworker/1:211:45:521
151129944,0cyclictest8499-21kworker/1:211:30:481
151129944,0cyclictest8499-21kworker/1:211:25:461
151129944,0cyclictest8499-21kworker/1:211:20:541
151129944,0cyclictest8499-21kworker/1:210:55:521
151129944,0cyclictest8499-21kworker/1:210:51:021
151129944,0cyclictest8499-21kworker/1:210:16:001
151129944,0cyclictest8499-21kworker/1:210:11:001
151129944,0cyclictest8499-21kworker/1:209:45:461
151129944,0cyclictest8499-21kworker/1:209:30:541
151129944,0cyclictest8499-21kworker/1:209:00:541
151129944,0cyclictest8499-21kworker/1:207:20:561
151129944,0cyclictest8499-21kworker/1:206:56:041
151129943,0cyclictest8499-21kworker/1:211:10:461
151129943,0cyclictest8499-21kworker/1:209:55:501
151129943,0cyclictest6445-21snmpd07:50:461
151129943,0cyclictest6445-21snmpd07:18:451
151129943,0cyclictest6445-21snmpd07:09:001
151129943,0cyclictest6445-21snmpd07:00:451
151129943,0cyclictest3584-21awk10:01:011
151129943,0cyclictest3425-21lldpd09:13:211
151129943,0cyclictest3110-21irqbalance08:12:291
151129943,0cyclictest3110-21irqbalance08:00:591
151129943,0cyclictest3110-21irqbalance07:56:591
151129943,0cyclictest3110-21irqbalance07:46:191
151129943,0cyclictest3110-21irqbalance06:45:491
151129943,0cyclictest3110-21irqbalance06:41:391
151129943,0cyclictest23814-21awk07:10:591
151129943,0cyclictest20312-21awk08:20:581
151129943,0cyclictest19939-21awk09:26:001
151129943,0cyclictest18066-21awk10:31:001
151119944,0cyclictest11100-21ssh09:20:380
151119944,0cyclictest0-21swapper/011:20:560
151119944,0cyclictest0-21swapper/010:50:490
151119944,0cyclictest0-21swapper/010:36:090
151119943,0cyclictest8034-21awk11:06:000
151119943,0cyclictest6445-21snmpd10:08:160
151119943,0cyclictest6445-21snmpd07:13:000
151119943,0cyclictest5741-21awk08:55:580
151119943,0cyclictest22228-21awk11:36:010
151119943,0cyclictest13494-21awk09:00:590
151119940,0cyclictest22175-21kthreadcore10:11:040
151119940,0cyclictest0-21swapper/008:30:490
149940,0migration/027387-21kthreadcore10:56:110
149940,0migration/024192-21kthreadcore08:21:010
149940,0migration/01655-21kthreadcore08:51:050
149940,0migration/014013-21kthreadcore07:51:040
28415030,0irq/27-eth0-tx-0-21swapper/007:16:030
28405030,0irq/26-eth0-rx-0-21swapper/308:16:023
28405030,0irq/26-eth0-rx-0-21swapper/308:16:023
28405030,0irq/26-eth0-rx-0-21swapper/308:11:003
28405030,0irq/26-eth0-rx-0-21swapper/307:40:583
28405030,0irq/26-eth0-rx-0-21swapper/307:05:583
28405030,0irq/26-eth0-rx-0-21swapper/306:50:583
249930,0watchdog/20-21swapper/208:05:592
151139932,0cyclictest31601-21awk07:41:012
151129932,0cyclictest595-21expr08:30:571
151119933,0cyclictest0-21swapper/007:39:060
151119932,0cyclictest6445-21snmpd09:07:300
151119932,0cyclictest3568-21awk10:01:010
151119932,0cyclictest3425-21lldpd09:59:010
151119932,0cyclictest22444-21awk08:46:000
151119932,0cyclictest10324-21awk10:26:000
151119931,0cyclictest7892-21tr09:41:020
151119930,0cyclictest7720-21fschecks_time08:35:590
151119930,0cyclictest3464-21runrttasks09:40:080
151119930,0cyclictest2553-21iostat_ios10:21:020
151119930,0cyclictest0-21swapper/007:47:370
149930,0migration/07522-21sh10:44:260
149930,0migration/04837-21sh12:05:260
149930,0migration/025294-21sh09:26:280
151119920,0cyclictest0-21swapper/008:16:020
151119920,0cyclictest0-21swapper/008:16:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional