You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-05-25 - 06:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sat May 25, 2024 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1324123826,0sleep00-21swapper/018:56:470
1322723625,0sleep10-21swapper/118:56:361
291350350,0irq/25-eth10-21swapper/318:55:583
709422622,0sleep20-21swapper/218:54:492
13582991212,0cyclictest698-21sensors22:10:121
135839998,0cyclictest29520-21turbostat21:09:402
28415070,0irq/27-eth0-tx-14938-21ssh20:59:590
28415070,0irq/27-eth0-tx-12967-21munin-node20:35:140
135829970,0cyclictest20104-21cut22:45:041
4160,0ktimersoftd/00-21swapper/023:46:480
339960,0migration/328966-21latency_hist21:29:403
339960,0migration/32038-21sh21:50:223
339960,0migration/318657-21latency_hist22:24:413
28415060,0irq/27-eth0-tx-482-21ssh22:52:482
28415060,0irq/27-eth0-tx-25297-21munin-node20:20:110
28415060,0irq/27-eth0-tx-13814-21munin-node19:00:010
28415060,0irq/27-eth0-tx-0-21swapper/223:54:552
28415060,0irq/27-eth0-tx-0-21swapper/223:51:372
28415060,0irq/27-eth0-tx-0-21swapper/223:40:402
28415060,0irq/27-eth0-tx-0-21swapper/223:37:512
28415060,0irq/27-eth0-tx-0-21swapper/223:12:152
28415060,0irq/27-eth0-tx-0-21swapper/222:57:242
28415060,0irq/27-eth0-tx-0-21swapper/222:41:292
28415060,0irq/27-eth0-tx-0-21swapper/222:32:582
28415060,0irq/27-eth0-tx-0-21swapper/222:25:352
28415060,0irq/27-eth0-tx-0-21swapper/222:14:482
28415060,0irq/27-eth0-tx-0-21swapper/221:51:242
28415060,0irq/27-eth0-tx-0-21swapper/221:42:072
28415060,0irq/27-eth0-tx-0-21swapper/221:23:562
28415060,0irq/27-eth0-tx-0-21swapper/200:19:482
28415060,0irq/27-eth0-tx-0-21swapper/200:19:482
28415060,0irq/27-eth0-tx-0-21swapper/200:18:572
28415060,0irq/27-eth0-tx-0-21swapper/200:07:592
28415060,0irq/27-eth0-tx-0-21swapper/020:33:520
28405060,0irq/26-eth0-rx-31723-21sed00:14:583
28405060,0irq/26-eth0-rx-14418-21perl00:25:023
249960,0watchdog/20-21swapper/222:05:032
249960,0watchdog/20-21swapper/219:00:072
149960,0migration/030968-21sh23:53:010
149960,0migration/010092-21sh21:55:310
135839960,0cyclictest0-21swapper/219:35:122
135819960,0cyclictest0-21swapper/021:37:120
4150,0ktimersoftd/00-21swapper/022:11:530
35150,0ktimersoftd/30-21swapper/323:46:483
339950,0migration/39699-21runrttasks22:17:033
339950,0migration/39275-21bash19:15:273
339950,0migration/37120-21kthreadcore21:55:013
339950,0migration/33307-21irqcore22:35:043
339950,0migration/332397-21fschecks_count20:24:563
339950,0migration/331652-21kthreadcore19:35:023
339950,0migration/33071-21kthreadcore20:00:053
339950,0migration/330408-21fschecks_time21:09:553
339950,0migration/329208-21kthreadcore20:45:033
339950,0migration/326765-21runrttasks21:45:483
339950,0migration/325566-21if_eth1.223:29:593
339950,0migration/324489-21munin-node20:15:113
339950,0migration/323082-21rt-features23:05:183
339950,0migration/314703-21kthreadcore21:20:033
339950,0migration/313902-21kthreadcore19:45:043
339950,0migration/312835-21taskset21:19:133
339950,0migration/312719-21taskset20:34:333
339950,0migration/311362-21kthreadcore23:20:183
339950,0migration/310947-21kthreadcore00:00:023
28415050,0irq/27-eth0-tx-8529-21ssh22:35:442
28415050,0irq/27-eth0-tx-2187-21ssh21:30:062
28415050,0irq/27-eth0-tx-10891-21ssh21:35:322
28415050,0irq/27-eth0-tx-10823-21munin-node22:19:592
28415050,0irq/27-eth0-tx-0-21swapper/223:45:312
28415050,0irq/27-eth0-tx-0-21swapper/223:22:112
28415050,0irq/27-eth0-tx-0-21swapper/223:17:392
28415050,0irq/27-eth0-tx-0-21swapper/223:06:012
28415050,0irq/27-eth0-tx-0-21swapper/222:09:492
28415050,0irq/27-eth0-tx-0-21swapper/221:59:592
28415050,0irq/27-eth0-tx-0-21swapper/221:56:012
28415050,0irq/27-eth0-tx-0-21swapper/221:15:252
28415050,0irq/27-eth0-tx-0-21swapper/221:12:132
28415050,0irq/27-eth0-tx-0-21swapper/200:25:042
28415050,0irq/27-eth0-tx-0-21swapper/200:01:352
28415050,0irq/27-eth0-tx-0-21swapper/020:49:510
28415050,0irq/27-eth0-tx-0-21swapper/020:14:500
28415050,0irq/27-eth0-tx-0-21swapper/019:56:580
28415050,0irq/27-eth0-tx-0-21swapper/019:28:400
28415050,0irq/27-eth0-tx-0-21swapper/019:10:060
28405050,0irq/26-eth0-rx-28729-21smartctl00:10:153
28405050,0irq/26-eth0-rx-25195-21ssh23:09:483
28405050,0irq/26-eth0-rx-22934-21ssh23:25:293
28405050,0irq/26-eth0-rx-2155-21ssh22:54:583
28405050,0irq/26-eth0-rx-16853-21kthreadcore21:40:063
28405050,0irq/26-eth0-rx-12184-21kthreadcore23:00:063
28405050,0irq/26-eth0-rx-11769-21ssh21:59:513
28405050,0irq/26-eth0-rx-0-21swapper/323:51:283
28405050,0irq/26-eth0-rx-0-21swapper/322:50:053
28405050,0irq/26-eth0-rx-0-21swapper/322:32:253
28405050,0irq/26-eth0-rx-0-21swapper/321:38:013
28405050,0irq/26-eth0-rx-0-21swapper/300:20:023
28405050,0irq/26-eth0-rx-0-21swapper/300:20:023
28405050,0irq/26-eth0-rx-0-21swapper/300:08:283
259950,0migration/24897-21latency20:25:082
259950,0migration/22437-21kthreadcore20:50:022
259950,0migration/222627-21kthreadcore19:50:012
259950,0migration/219856-21kthreadcore23:25:002
259950,0migration/219408-21idleruntime-cro20:39:402
249950,0watchdog/20-21swapper/222:47:192
249950,0watchdog/20-21swapper/221:01:552
249950,0watchdog/20-21swapper/219:48:392
249950,0watchdog/20-21swapper/219:21:432
249950,0watchdog/20-21swapper/219:09:592
249950,0watchdog/20-21swapper/219:09:272
19150,0ktimersoftd/10-21swapper/122:41:461
179950,0migration/18949-21taskset22:59:111
179950,0migration/17800-21idleruntime-cro23:39:401
179950,0migration/14913-21cpuspeed_turbos21:34:541
179950,0migration/124337-21munin-node19:30:021
179950,0migration/11775-21timerandwakeup21:50:121
169950,0watchdog/130446-21diskstats19:34:551
169950,0watchdog/10-21swapper/121:40:391
149950,0migration/09517-21taskset22:38:470
149950,0migration/07425-21kthreadcore20:05:060
149950,0migration/07212-21irqcore00:19:580
149950,0migration/07212-21irqcore00:19:580
149950,0migration/05738-21sh23:55:470
149950,0migration/03592-21kthreadcore20:00:050
149950,0migration/032343-21kthreadcore21:30:030
149950,0migration/031654-21needreboot20:45:050
149950,0migration/031227-21kthreadcore21:10:020
149950,0migration/031119-21kthreadcore21:50:020
149950,0migration/028270-21latency00:10:060
149950,0migration/027085-21kthreadcore19:30:060
149950,0migration/026733-21kthreadcore23:30:020
149950,0migration/026369-21irqcore23:10:030
149950,0migration/024444-21kthreadcore20:40:050
149950,0migration/024027-21kthreadcore22:05:040
149950,0migration/022398-21irqcore19:04:580
149950,0migration/01936-21latency_hist22:34:400
149950,0migration/017575-21latency_hist19:49:400
149950,0migration/015219-21kthreadcore21:20:030
149950,0migration/015110-21kthreadcore22:20:020
149950,0migration/014812-21latency23:00:070
149950,0migration/01339-21fschecks_time23:14:560
149950,0migration/012526-21aten_r1power_cu21:39:520
149950,0migration/012305-21kthreadcore22:40:100
135839955,0cyclictest0-21swapper/223:00:102
135839954,0cyclictest26848-21sh21:46:042
135839954,0cyclictest25459-21awk23:29:572
135839950,0cyclictest0-21swapper/200:10:002
135829950,0cyclictest0-21swapper/123:03:121
135829950,0cyclictest0-21swapper/122:39:241
135819955,0cyclictest0-21swapper/022:29:140
135819955,0cyclictest0-21swapper/000:00:120
135819950,0cyclictest0-21swapper/023:09:070
135819950,0cyclictest0-21swapper/021:48:330
339940,0migration/3999-21kthreadcore19:10:033
339940,0migration/32972-21kthreadcore23:35:023
339940,0migration/318013-21cpuspeed_turbos19:49:533
339940,0migration/314208-21kthreadcore20:10:123
28405040,0irq/26-eth0-rx-30128-21kthreadcore22:10:043
28405040,0irq/26-eth0-rx-0-21swapper/323:55:053
28405040,0irq/26-eth0-rx-0-21swapper/323:40:043
28405040,0irq/26-eth0-rx-0-21swapper/323:15:013
28405040,0irq/26-eth0-rx-0-21swapper/322:47:273
28405040,0irq/26-eth0-rx-0-21swapper/322:41:443
28405040,0irq/26-eth0-rx-0-21swapper/322:19:583
28405040,0irq/26-eth0-rx-0-21swapper/322:05:493
28405040,0irq/26-eth0-rx-0-21swapper/321:26:113
28405040,0irq/26-eth0-rx-0-21swapper/321:04:503
28405040,0irq/26-eth0-rx-0-21swapper/320:59:463
28405040,0irq/26-eth0-rx-0-21swapper/320:35:053
28405040,0irq/26-eth0-rx-0-21swapper/320:22:323
28405040,0irq/26-eth0-rx-0-21swapper/319:00:083
259940,0migration/232177-21irqcore20:00:032
259940,0migration/223578-21aten_r1power_vo19:29:532
259940,0migration/221280-21kthreadcore19:25:042
249940,0watchdog/225487-21tr21:25:032
249940,0watchdog/20-21swapper/220:55:152
249940,0watchdog/20-21swapper/220:39:512
249940,0watchdog/20-21swapper/220:19:512
249940,0watchdog/20-21swapper/220:14:592
249940,0watchdog/20-21swapper/220:09:592
249940,0watchdog/20-21swapper/219:55:032
249940,0watchdog/20-21swapper/219:40:032
249940,0watchdog/20-21swapper/219:14:592
179940,0migration/19557-21fschecks_time23:20:121
179940,0migration/17487-21munin-node19:40:031
179940,0migration/15551-21kthreadcore20:50:031
179940,0migration/131978-21kthreadcore22:30:041
179940,0migration/126264-21runrttasks20:44:421
179940,0migration/123630-21kthreadcore20:15:021
179940,0migration/123507-21timerandwakeup19:50:121
179940,0migration/122168-21cpuspeed_turbos21:04:531
179940,0migration/120681-21kthreadcore19:25:031
179940,0migration/115328-21kthreadcore22:00:031
179940,0migration/110959-21kthreadcore20:30:071
169940,0watchdog/16756-21perl00:19:551
169940,0watchdog/16756-21perl00:19:551
169940,0watchdog/130346-21grep21:09:551
169940,0watchdog/125486-21cut21:25:031
169940,0watchdog/125105-21kthreadcore21:45:031
169940,0watchdog/124311-21ssh23:49:591
169940,0watchdog/123133-21ssh23:26:071
169940,0watchdog/122369-21cut22:05:031
169940,0watchdog/118731-21ssh22:24:471
169940,0watchdog/114399-21fschecks_count18:59:591
169940,0watchdog/110682-21fschecks_time19:19:551
169940,0watchdog/110402-21kthreadcore00:00:031
149940,0migration/032564-21aten_r1power_po23:34:540
149940,0migration/024761-21kthreadcore21:05:010
149940,0migration/022053-21runrttasks00:07:500
149940,0migration/020441-21kthreadcore19:50:010
149940,0migration/018492-21latency_hist20:14:400
135839943,0cyclictest6445-21snmpd20:29:422
135839943,0cyclictest6445-21snmpd20:07:122
135839943,0cyclictest27162-21awk20:44:582
135829943,0cyclictest7428-21kworker/1:023:46:161
135829943,0cyclictest7428-21kworker/1:023:39:461
135829943,0cyclictest7428-21kworker/1:023:12:001
135829943,0cyclictest7428-21kworker/1:023:04:441
135829943,0cyclictest7428-21kworker/1:022:20:561
135829943,0cyclictest7428-21kworker/1:021:19:441
135829943,0cyclictest6445-21snmpd23:56:121
135829943,0cyclictest6445-21snmpd21:54:571
135829943,0cyclictest6445-21snmpd21:29:571
135829943,0cyclictest6445-21snmpd20:39:421
135829943,0cyclictest6445-21snmpd19:59:271
135829943,0cyclictest3889-21awk22:15:031
135829943,0cyclictest353-21awk20:25:011
135829943,0cyclictest3425-21lldpd21:14:541
135829943,0cyclictest29043-21awk19:09:581
135829943,0cyclictest26829-21awk22:50:021
135829943,0cyclictest26232-21awk20:19:571
135829943,0cyclictest23698-21kworker/1:100:25:161
135829943,0cyclictest22233-21awk19:04:551
135829943,0cyclictest1655-21awk23:15:001
135829943,0cyclictest14936-21awk20:59:591
135829943,0cyclictest11779-21sadc20:09:401
135819944,0cyclictest0-21swapper/023:24:340
135819944,0cyclictest0-21swapper/022:53:590
135819944,0cyclictest0-21swapper/022:45:570
135819944,0cyclictest0-21swapper/020:26:250
135819944,0cyclictest0-21swapper/019:36:260
135819943,0cyclictest6445-21snmpd19:18:560
135819943,0cyclictest3425-21lldpd00:17:260
135819943,0cyclictest2337-21awk22:55:050
135819943,0cyclictest14559-21awk00:25:030
135819943,0cyclictest11059-21awk19:20:000
28405030,0irq/26-eth0-rx-0-21swapper/320:54:513
28405030,0irq/26-eth0-rx-0-21swapper/320:04:533
28405030,0irq/26-eth0-rx-0-21swapper/319:55:073
28405030,0irq/26-eth0-rx-0-21swapper/319:39:513
169930,0watchdog/10-21swapper/119:15:031
169930,0watchdog/10-21swapper/100:09:551
135829933,0cyclictest7428-21kworker/1:023:29:521
135829933,0cyclictest7428-21kworker/1:020:54:581
135829933,0cyclictest7428-21kworker/1:020:34:541
135829933,0cyclictest7428-21kworker/1:020:10:201
135829933,0cyclictest7428-21kworker/1:019:59:401
135829933,0cyclictest7428-21kworker/1:000:04:421
135829933,0cyclictest23698-21kworker/1:100:14:441
135829932,0cyclictest11844-21hwlatdetect19:44:551
135819933,0cyclictest0-21swapper/023:39:420
135819932,0cyclictest8160-21awk20:54:590
135819932,0cyclictest6445-21snmpd22:15:570
135819932,0cyclictest6190-21awk21:14:590
135819932,0cyclictest22207-21awk21:24:580
135819932,0cyclictest17562-21awk23:24:560
135819932,0cyclictest17499-21sensors22:00:130
135819930,0cyclictest0-21swapper/019:40:090
329920,0watchdog/321510-21aten_r1power_vo19:04:513
329920,0watchdog/30-21swapper/320:50:033
28405020,0irq/26-eth0-rx-0-21swapper/320:44:013
28405020,0irq/26-eth0-rx-0-21swapper/319:30:103
28405020,0irq/26-eth0-rx-0-21swapper/319:25:383
28405020,0irq/26-eth0-rx-0-21swapper/319:19:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional