You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-05-23 - 18:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Thu May 23, 2024 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
271780,0ktimersoftd/230708-21perf06:59:342
461723524,0sleep10-21swapper/106:55:581
291350350,0irq/25-eth10-21swapper/306:54:563
3092523422,0sleep00-21swapper/006:54:520
8225060,0irq/24-0000:00:13279-21cat12:19:382
339960,0migration/38722-21sh11:11:393
339960,0migration/38028-21sh09:49:073
339960,0migration/38028-21sh09:49:073
339960,0migration/33806-21sh09:01:173
339960,0migration/327579-21sh10:43:093
339960,0migration/321817-21sh10:19:393
339960,0migration/312318-21gltestperf10:55:023
28415060,0irq/27-eth0-tx-5227-21munin-node07:00:120
28415060,0irq/27-eth0-tx-3036-21munin-node08:39:580
28415060,0irq/27-eth0-tx-21465-21munin-node08:25:160
28415060,0irq/27-eth0-tx-0-21swapper/212:24:382
28415060,0irq/27-eth0-tx-0-21swapper/212:12:272
28415060,0irq/27-eth0-tx-0-21swapper/212:01:452
28415060,0irq/27-eth0-tx-0-21swapper/211:42:092
28415060,0irq/27-eth0-tx-0-21swapper/211:12:192
28415060,0irq/27-eth0-tx-0-21swapper/210:44:582
28415060,0irq/27-eth0-tx-0-21swapper/210:23:362
28415060,0irq/27-eth0-tx-0-21swapper/210:06:542
28415060,0irq/27-eth0-tx-0-21swapper/209:53:122
28415060,0irq/27-eth0-tx-0-21swapper/209:29:572
28415060,0irq/27-eth0-tx-0-21swapper/209:26:052
28415060,0irq/27-eth0-tx-0-21swapper/209:24:252
28415060,0irq/27-eth0-tx-0-21swapper/209:02:512
28415060,0irq/27-eth0-tx-0-21swapper/008:19:550
28415060,0irq/27-eth0-tx-0-21swapper/007:44:560
28405060,0irq/26-eth0-rx-16923-21sort09:55:003
28405060,0irq/26-eth0-rx-13850-21if_eth1.210:35:023
28405060,0irq/26-eth0-rx-0-21swapper/310:33:113
249960,0watchdog/210501-21latency07:00:072
249960,0watchdog/20-21swapper/209:12:022
149960,0migration/06437-21fschecks_time12:14:560
8225050,0irq/24-0000:00:0-21swapper/211:00:022
50149955,0cyclictest0-21swapper/211:45:052
50149955,0cyclictest0-21swapper/211:20:012
50139955,0cyclictest0-21swapper/110:21:491
50139955,0cyclictest0-21swapper/109:55:011
50139955,0cyclictest0-21swapper/109:36:061
50139954,0cyclictest12762-21awk07:05:131
50139950,0cyclictest0-21swapper/112:14:421
50139950,0cyclictest0-21swapper/112:03:021
50139950,0cyclictest0-21swapper/111:50:251
50139950,0cyclictest0-21swapper/111:34:301
50139950,0cyclictest0-21swapper/109:25:231
339950,0migration/3974-21sh11:06:283
339950,0migration/36743-21kthreadcore09:05:133
339950,0migration/3429-21timerandwakeup08:05:123
339950,0migration/33319-21kthreadcore11:50:073
339950,0migration/330250-21kthreadcore09:20:063
339950,0migration/328685-21latency_hist08:34:443
339950,0migration/32515-21taskset08:36:313
339950,0migration/324495-21taskset10:00:033
339950,0migration/324245-21kthreadcore10:20:023
339950,0migration/319528-21runrttasks07:10:013
339950,0migration/319051-21sh09:35:073
339950,0migration/317576-21kthreadcore08:20:063
339950,0migration/317269-21kthreadcore07:55:113
339950,0migration/314617-21kthreadcore11:15:183
339950,0migration/313221-21df_abs07:29:583
339950,0migration/313109-21kthreadcore11:35:053
339950,0migration/310724-21kthreadcore09:50:133
28415050,0irq/27-eth0-tx-6445-21snmpd08:55:010
28415050,0irq/27-eth0-tx-6372-21iceccd08:35:090
28415050,0irq/27-eth0-tx-14638-21kthreadcore10:35:042
28415050,0irq/27-eth0-tx-1374-21munin-node08:10:120
28415050,0irq/27-eth0-tx-10494-21ssh10:52:022
28415050,0irq/27-eth0-tx-0-21swapper/212:26:492
28415050,0irq/27-eth0-tx-0-21swapper/211:56:372
28415050,0irq/27-eth0-tx-0-21swapper/211:32:342
28415050,0irq/27-eth0-tx-0-21swapper/211:25:582
28415050,0irq/27-eth0-tx-0-21swapper/210:00:102
28415050,0irq/27-eth0-tx-0-21swapper/209:57:072
28415050,0irq/27-eth0-tx-0-21swapper/209:34:462
28415050,0irq/27-eth0-tx-0-21swapper/209:14:492
28415050,0irq/27-eth0-tx-0-21swapper/209:06:402
28415050,0irq/27-eth0-tx-0-21swapper/007:37:240
28405050,0irq/26-eth0-rx-7735-21kthreadcore08:40:013
28405050,0irq/26-eth0-rx-7273-21kthreadcore09:25:063
28405050,0irq/26-eth0-rx-27833-21missed_timers12:05:083
28405050,0irq/26-eth0-rx-26943-21kthreadcore07:15:043
28405050,0irq/26-eth0-rx-15959-21stty11:39:563
28405050,0irq/26-eth0-rx-0-21swapper/312:20:123
28405050,0irq/26-eth0-rx-0-21swapper/311:27:053
28405050,0irq/26-eth0-rx-0-21swapper/311:20:063
28405050,0irq/26-eth0-rx-0-21swapper/310:46:073
28405050,0irq/26-eth0-rx-0-21swapper/310:25:043
28405050,0irq/26-eth0-rx-0-21swapper/309:10:193
28405050,0irq/26-eth0-rx-0-21swapper/308:25:083
28405050,0irq/26-eth0-rx-0-21swapper/307:45:043
28405050,0irq/26-eth0-rx-0-21swapper/307:25:023
259950,0migration/227174-21kthreadcore09:40:082
259950,0migration/220525-21aten_r1power_cu07:59:562
259950,0migration/212575-21sh10:10:162
249950,0watchdog/227413-21cpuspeed_turbos08:04:552
249950,0watchdog/2246142chrt07:10:152
249950,0watchdog/20-21swapper/212:06:382
249950,0watchdog/20-21swapper/211:51:022
249950,0watchdog/20-21swapper/211:37:422
249950,0watchdog/20-21swapper/211:16:102
249950,0watchdog/20-21swapper/211:05:022
249950,0watchdog/20-21swapper/210:55:142
249950,0watchdog/20-21swapper/210:42:022
249950,0watchdog/20-21swapper/210:25:142
249950,0watchdog/20-21swapper/210:16:102
249950,0watchdog/20-21swapper/208:55:022
249950,0watchdog/20-21swapper/208:18:542
249950,0watchdog/20-21swapper/207:44:512
249950,0watchdog/20-21swapper/207:42:312
249950,0watchdog/20-21swapper/207:04:472
19150,0ktimersoftd/10-21swapper/110:59:401
179950,0migration/128175-21munin-node12:05:131
179950,0migration/127224-21taskset10:41:591
149950,0migration/09040-21fschecks_count09:50:030
149950,0migration/08846-21gltestperf08:14:570
149950,0migration/08076-21latency_hist11:34:430
149950,0migration/07976-21latency_hist11:34:430
149950,0migration/07031-21kthreadcore09:05:130
149950,0migration/06386-21sh07:24:570
149950,0migration/06117-21if_eth010:30:020
149950,0migration/05934-21sh10:08:120
149950,0migration/05429-21taskset12:14:340
149950,0migration/031296-21aten_r1power_cu11:49:540
149950,0migration/029969-21kthreadcore07:15:050
149950,0migration/029642-21sh11:45:210
149950,0migration/028255-21kthreadcore07:40:080
149950,0migration/027084-21cpuspeed_turbos11:04:550
149950,0migration/026605-21taskset09:16:230
149950,0migration/025234-21kthreadcore08:00:080
149950,0migration/024901-21latency_hist11:24:430
149950,0migration/024369-21kthreadcore12:25:070
149950,0migration/02434-21kthreadcore09:45:040
149950,0migration/02434-21kthreadcore09:45:040
149950,0migration/023746-21kthreadcore10:20:020
149950,0migration/018313-21kthreadcore08:50:070
149950,0migration/017190-21latency_hist11:19:430
149950,0migration/017019-21kthreadcore10:15:080
149950,0migration/015367-21gltestperf12:00:000
149950,0migration/014721-21irqcore12:20:040
149950,0migration/014712-21kthreadcore10:55:070
149950,0migration/01182-21kthreadcore09:00:100
50149943,0cyclictest6445-21snmpd10:32:422
50149943,0cyclictest6445-21snmpd08:50:112
50149943,0cyclictest1667-21awk09:45:032
50149943,0cyclictest1667-21awk09:45:032
50149943,0cyclictest15722-21awk08:20:022
50149943,0cyclictest12648-21sadc07:29:442
50139944,0cyclictest0-21swapper/111:08:201
50139944,0cyclictest0-21swapper/110:45:471
50139943,0cyclictest7246-21awk07:25:061
50139943,0cyclictest6092-21awk07:00:011
50139943,0cyclictest3948-21awk08:39:581
50139943,0cyclictest3425-21lldpd11:24:101
50139943,0cyclictest3425-21lldpd11:19:021
50139943,0cyclictest3425-21lldpd10:14:531
50139943,0cyclictest3425-21lldpd08:16:071
50139943,0cyclictest3425-21lldpd08:07:081
50139943,0cyclictest3425-21lldpd07:17:571
50139943,0cyclictest29656-21awk08:35:001
50139943,0cyclictest27891-21sadc08:29:441
50139943,0cyclictest26525-21awk09:40:051
50139943,0cyclictest24597-21awk10:00:041
50139943,0cyclictest20171-21awk11:00:051
50139943,0cyclictest16804-21awk11:40:041
50139943,0cyclictest13637-21awk09:09:591
50139943,0cyclictest1310-21awk07:45:001
50139943,0cyclictest10760-21awk08:45:021
50139940,0cyclictest0-21swapper/107:20:311
50129944,0cyclictest0-21swapper/010:46:250
50129943,0cyclictest4728-21awk10:50:010
50129943,0cyclictest3924-21awk09:25:020
50129943,0cyclictest322-21awk07:20:000
50129943,0cyclictest3110-21irqbalance09:20:580
50129943,0cyclictest23426-21sadc09:59:440
50129943,0cyclictest19046-21irqrtprio09:35:070
50129943,0cyclictest13769-21awk10:35:010
50129943,0cyclictest13723-21awk07:30:020
339940,0migration/3439-21kthreadcore12:10:113
28415040,0irq/27-eth0-tx-0-21swapper/008:47:360
28415040,0irq/27-eth0-tx-0-21swapper/007:09:540
28405040,0irq/26-eth0-rx-0-21swapper/312:24:583
28405040,0irq/26-eth0-rx-0-21swapper/312:14:593
28405040,0irq/26-eth0-rx-0-21swapper/311:59:463
28405040,0irq/26-eth0-rx-0-21swapper/311:54:463
28405040,0irq/26-eth0-rx-0-21swapper/311:45:023
28405040,0irq/26-eth0-rx-0-21swapper/311:00:283
28405040,0irq/26-eth0-rx-0-21swapper/310:49:593
28405040,0irq/26-eth0-rx-0-21swapper/310:09:453
28405040,0irq/26-eth0-rx-0-21swapper/310:08:193
28405040,0irq/26-eth0-rx-0-21swapper/309:41:103
28405040,0irq/26-eth0-rx-0-21swapper/309:29:573
28405040,0irq/26-eth0-rx-0-21swapper/309:17:313
249940,0watchdog/20-21swapper/208:44:502
249940,0watchdog/20-21swapper/208:43:022
249940,0watchdog/20-21swapper/208:37:422
249940,0watchdog/20-21swapper/208:34:142
249940,0watchdog/20-21swapper/208:10:232
249940,0watchdog/20-21swapper/207:54:592
249940,0watchdog/20-21swapper/207:49:592
249940,0watchdog/20-21swapper/207:35:272
249940,0watchdog/20-21swapper/207:29:472
249940,0watchdog/20-21swapper/207:20:552
249940,0watchdog/20-21swapper/207:17:192
179940,0migration/1454-21kthreadcore10:05:111
179940,0migration/127478-21timerwakeupswit12:25:171
179940,0migration/123832-21kthreadcore09:15:151
179940,0migration/120696-21irqcore07:35:031
179940,0migration/120474-21latency_hist07:59:441
179940,0migration/119297-21munin-node07:09:581
179940,0migration/11602-21kthreadcore10:25:041
179940,0migration/115608-21latency_hist11:39:431
179940,0migration/112864-21taskset10:34:491
169940,0watchdog/19326-21kthreadcore09:05:141
169940,0watchdog/17813-21ssh09:48:221
169940,0watchdog/17813-21ssh09:48:221
169940,0watchdog/17693-21kthreadcore12:15:021
169940,0watchdog/123893-21munin-node11:45:021
169940,0watchdog/122465-21munin-node08:50:181
169940,0watchdog/115745-21kthreadcore09:30:061
169940,0watchdog/111113-21grep10:10:061
169940,0watchdog/10-21swapper/111:55:061
169940,0watchdog/10-21swapper/108:20:061
169940,0watchdog/10-21swapper/107:50:071
149940,0migration/014963-21irqcore07:55:090
149940,0migration/011592-21kthreadcore11:55:070
50139933,0cyclictest3425-21lldpd11:10:091
50139933,0cyclictest3425-21lldpd08:30:381
50139933,0cyclictest30530-21kworker/1:011:24:591
50139933,0cyclictest30530-21kworker/1:009:50:041
50139932,0cyclictest4467-21grep10:49:581
50139932,0cyclictest28659-21awk09:20:031
50139932,0cyclictest27056-21awk07:40:041
50139932,0cyclictest17397-21kthreadcore07:55:111
50139931,0cyclictest8297-21kthreadcore10:30:071
50129932,0cyclictest7981-21cut07:49:590
50129932,0cyclictest3110-21irqbalance10:09:580
50129932,0cyclictest2784-21grep11:10:060
50129932,0cyclictest25944-21awk11:25:030
50129932,0cyclictest20057-21awk11:00:040
50129932,0cyclictest16250-21sensors09:30:140
50129932,0cyclictest12746-21munin-node09:09:550
50129930,0cyclictest22113-21latency_hist12:04:430
50129930,0cyclictest21139-21perl10:40:000
50129930,0cyclictest0-21swapper/010:00:000
50129930,0cyclictest0-21swapper/009:40:120
50129930,0cyclictest0-21swapper/007:04:450
28415030,0irq/27-eth0-tx-0-21swapper/008:32:330
28405030,0irq/26-eth0-rx-0-21swapper/311:29:483
28405030,0irq/26-eth0-rx-0-21swapper/308:54:573
28405030,0irq/26-eth0-rx-0-21swapper/308:44:553
28405030,0irq/26-eth0-rx-0-21swapper/308:09:573
28405030,0irq/26-eth0-rx-0-21swapper/307:50:143
28405030,0irq/26-eth0-rx-0-21swapper/307:42:233
28405030,0irq/26-eth0-rx-0-21swapper/307:35:003
249930,0watchdog/20-21swapper/208:24:462
169930,0watchdog/10-21swapper/112:19:581
169930,0watchdog/10-21swapper/108:59:461
169930,0watchdog/10-21swapper/108:54:541
169930,0watchdog/10-21swapper/108:10:031
169930,0watchdog/10-21swapper/107:30:071
149930,0migration/022137-21sh11:40:440
50129921,0cyclictest27094-21date08:04:440
50129920,0cyclictest29747-21sh10:24:480
35120,0ktimersoftd/310372-21kthreadcore08:15:043
339920,0migration/312319-21diskstats07:05:093
28405020,0irq/26-eth0-rx-0-21swapper/308:50:063
28405020,0irq/26-eth0-rx-0-21swapper/308:00:033
28405020,0irq/26-eth0-rx-0-21swapper/307:20:373
28405020,0irq/26-eth0-rx-0-21swapper/306:59:563
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional