You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-03-02 - 08:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sat Mar 02, 2024 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2205023725,0sleep30-21swapper/318:58:523
2213823623,0sleep00-21swapper/018:59:530
2201623324,0sleep20-21swapper/218:58:272
2233222824,0sleep10-21swapper/119:02:071
339960,0migration/3830-21sh22:29:243
339960,0migration/317652-21sh22:41:473
339960,0migration/310058-21sh21:54:473
28415060,0irq/27-eth0-tx-20666-21munin-run21:02:440
28415060,0irq/27-eth0-tx-0-21swapper/223:18:552
28415060,0irq/27-eth0-tx-0-21swapper/223:09:272
28415060,0irq/27-eth0-tx-0-21swapper/223:05:032
28415060,0irq/27-eth0-tx-0-21swapper/222:52:452
28415060,0irq/27-eth0-tx-0-21swapper/222:50:182
28415060,0irq/27-eth0-tx-0-21swapper/222:23:472
28415060,0irq/27-eth0-tx-0-21swapper/221:54:402
28415060,0irq/27-eth0-tx-0-21swapper/221:25:412
28415060,0irq/27-eth0-tx-0-21swapper/221:20:032
28415060,0irq/27-eth0-tx-0-21swapper/221:13:272
28415060,0irq/27-eth0-tx-0-21swapper/200:19:352
28415060,0irq/27-eth0-tx-0-21swapper/200:03:222
28415060,0irq/27-eth0-tx-0-21swapper/021:03:300
249960,0watchdog/217072-21grep20:08:112
249960,0watchdog/210882-21ssh22:38:062
249960,0watchdog/20-21swapper/223:01:502
249960,0watchdog/20-21swapper/222:06:542
224579966,0cyclictest20672-21diskmemload21:32:242
224549966,0cyclictest0-21swapper/000:12:590
149960,0migration/032265-21sh23:09:380
149960,0migration/023849-21aten_r1power_vo19:28:070
149960,0migration/017064-21sh22:40:010
149960,0migration/010030-21sh21:33:150
8225050,0irq/24-0000:00:0-21swapper/220:57:442
339950,0migration/39976-21aten_r1power_vo22:58:053
339950,0migration/39002-21munin-node23:58:153
339950,0migration/38928-21kthreadcore20:53:123
339950,0migration/37096-21cpu00:18:053
339950,0migration/37027-21forks20:28:063
339950,0migration/36758-21kthreadcore22:13:093
339950,0migration/36021-21kthreadcore21:33:073
339950,0migration/34403-21aten_r1power_fr21:13:033
339950,0migration/332689-21cpuspeed_turbos23:53:073
339950,0migration/331503-21diskmemload00:13:003
339950,0migration/330759-21kthreadcore20:18:143
339950,0migration/324234-21kthreadcore21:43:123
339950,0migration/324202-21fschecks_time00:08:103
339950,0migration/323032-21kthreadcore22:43:093
339950,0migration/320825-21taskset00:23:533
339950,0migration/320174-21unixbench_multi20:58:193
339950,0migration/31761-21aten_r1power_cu22:53:023
339950,0migration/316315-21fschecks_count00:03:073
339950,0migration/312364-21kthreadcore23:18:183
28415050,0irq/27-eth0-tx-3994-21kthreadcore23:33:082
28415050,0irq/27-eth0-tx-21498-21ssh00:26:182
28415050,0irq/27-eth0-tx-20672-21diskmemload22:08:032
28415050,0irq/27-eth0-tx-19795-21ssh21:43:042
28415050,0irq/27-eth0-tx-18033-21munin-node19:48:090
28415050,0irq/27-eth0-tx-0-21swapper/223:58:142
28415050,0irq/27-eth0-tx-0-21swapper/223:38:042
28415050,0irq/27-eth0-tx-0-21swapper/223:29:152
28415050,0irq/27-eth0-tx-0-21swapper/223:24:282
28415050,0irq/27-eth0-tx-0-21swapper/221:58:002
28415050,0irq/27-eth0-tx-0-21swapper/221:38:282
28415050,0irq/27-eth0-tx-0-21swapper/221:33:582
28415050,0irq/27-eth0-tx-0-21swapper/221:09:212
28415050,0irq/27-eth0-tx-0-21swapper/200:30:122
28415050,0irq/27-eth0-tx-0-21swapper/200:13:462
28415050,0irq/27-eth0-tx-0-21swapper/020:48:160
28415050,0irq/27-eth0-tx-0-21swapper/020:28:120
28415050,0irq/27-eth0-tx-0-21swapper/020:23:060
28405050,0irq/26-eth0-rx-28287-21aten_r1power_en21:28:023
28405050,0irq/26-eth0-rx-25568-21iostat_ios23:48:133
28405050,0irq/26-eth0-rx-0-21swapper/323:45:223
28405050,0irq/26-eth0-rx-0-21swapper/323:42:293
28405050,0irq/26-eth0-rx-0-21swapper/323:29:003
28405050,0irq/26-eth0-rx-0-21swapper/323:24:433
28405050,0irq/26-eth0-rx-0-21swapper/323:13:093
28405050,0irq/26-eth0-rx-0-21swapper/322:37:013
28405050,0irq/26-eth0-rx-0-21swapper/322:23:203
28405050,0irq/26-eth0-rx-0-21swapper/322:08:083
28405050,0irq/26-eth0-rx-0-21swapper/321:58:123
28405050,0irq/26-eth0-rx-0-21swapper/320:03:023
28405050,0irq/26-eth0-rx-0-21swapper/319:58:133
28405050,0irq/26-eth0-rx-0-21swapper/319:43:053
249950,0watchdog/228953-21ps21:48:102
249950,0watchdog/227047-21kthreadcore19:53:112
249950,0watchdog/20-21swapper/223:53:062
249950,0watchdog/20-21swapper/223:48:262
249950,0watchdog/20-21swapper/223:44:062
249950,0watchdog/20-21swapper/222:43:102
249950,0watchdog/20-21swapper/222:33:422
249950,0watchdog/20-21swapper/222:18:182
249950,0watchdog/20-21swapper/222:13:222
249950,0watchdog/20-21swapper/221:03:272
249950,0watchdog/20-21swapper/220:43:232
249950,0watchdog/20-21swapper/220:32:432
249950,0watchdog/20-21swapper/220:18:312
249950,0watchdog/20-21swapper/219:57:512
249950,0watchdog/20-21swapper/219:49:272
249950,0watchdog/20-21swapper/219:38:112
249950,0watchdog/20-21swapper/219:24:352
249950,0watchdog/20-21swapper/219:22:312
249950,0watchdog/20-21swapper/200:09:102
224579950,0cyclictest0-21swapper/223:13:372
224559955,0cyclictest0-21swapper/123:45:361
224559955,0cyclictest0-21swapper/122:28:021
224559955,0cyclictest0-21swapper/122:12:441
224559955,0cyclictest0-21swapper/100:10:021
224559950,0cyclictest0-21swapper/123:09:301
224559950,0cyclictest0-21swapper/121:11:341
224549955,0cyclictest0-21swapper/023:49:290
179950,0migration/132328-21sh23:31:531
179950,0migration/124652-21munin-node22:23:191
179950,0migration/120378-21kthreadcore20:13:091
179950,0migration/120055-21latency_hist21:22:441
169950,0watchdog/12253-21kthreadcore00:13:061
149950,0migration/07506-21latency_hist20:52:440
149950,0migration/0600-21aten_r1power_po23:33:020
149950,0migration/05191-21kthreadcore22:33:090
149950,0migration/03979-21irqcore22:13:080
149950,0migration/03803-21fschecks_count21:53:050
149950,0migration/032670-21taskset23:53:070
149950,0migration/030173-21kthreadcore21:48:110
149950,0migration/030058-21kthreadcore21:28:090
149950,0migration/029386-21kthreadcore19:53:110
149950,0migration/029173-21ntp_offset00:08:170
149950,0migration/027870-21kthreadcore23:28:100
149950,0migration/023360-21kthreadcore22:23:150
149950,0migration/022547-21unixbench-2d23:43:140
149950,0migration/022333-21kthreadcore21:23:120
149950,0migration/017604-21latency_hist23:02:440
149950,0migration/017123-21kthreadcore21:18:120
149950,0migration/01683-21fschecks_count23:13:040
149950,0migration/015515-21munin-node00:23:080
149950,0migration/014093-21taskset00:21:570
149950,0migration/012451-21fschecks_time21:38:050
149950,0migration/010159-21kthreadcore23:38:080
339940,0migration/332219-21latency_hist20:22:443
339940,0migration/331117-21gltestperf19:33:053
339940,0migration/328696-21kthreadcore19:28:163
339940,0migration/320916-21diskmemload21:02:533
28415040,0irq/27-eth0-tx-0-21swapper/222:27:592
28415040,0irq/27-eth0-tx-0-21swapper/019:38:150
28415040,0irq/27-eth0-tx-0-21swapper/019:13:070
28415040,0irq/27-eth0-tx-0-21swapper/019:02:560
28405040,0irq/26-eth0-rx-0-21swapper/323:33:043
28405040,0irq/26-eth0-rx-0-21swapper/323:10:223
28405040,0irq/26-eth0-rx-0-21swapper/323:02:443
28405040,0irq/26-eth0-rx-0-21swapper/322:49:383
28405040,0irq/26-eth0-rx-0-21swapper/322:19:333
28405040,0irq/26-eth0-rx-0-21swapper/322:02:503
28405040,0irq/26-eth0-rx-0-21swapper/321:48:073
28405040,0irq/26-eth0-rx-0-21swapper/321:38:173
28405040,0irq/26-eth0-rx-0-21swapper/321:22:463
28405040,0irq/26-eth0-rx-0-21swapper/321:20:163
28405040,0irq/26-eth0-rx-0-21swapper/321:07:443
28405040,0irq/26-eth0-rx-0-21swapper/319:08:083
28405040,0irq/26-eth0-rx-0-21swapper/300:30:303
259940,0migration/26132-21kthreadcore20:48:112
259940,0migration/24269-21fschecks_time19:13:062
259940,0migration/227123-21kthreadcore19:28:152
259940,0migration/212333-21if_eth1.219:43:082
249940,0watchdog/20-21swapper/220:36:152
249940,0watchdog/20-21swapper/220:23:032
249940,0watchdog/20-21swapper/220:13:072
249940,0watchdog/20-21swapper/220:03:112
249940,0watchdog/20-21swapper/219:08:072
249940,0watchdog/20-21swapper/219:03:152
224579943,0cyclictest3383-21snmpd20:58:032
224579943,0cyclictest31272-21awk19:33:072
224559944,0cyclictest0-21swapper/121:58:051
224559944,0cyclictest0-21swapper/121:37:541
224559944,0cyclictest0-21swapper/119:13:101
224559943,0cyclictest8853-21awk23:58:141
224559943,0cyclictest5256-21awk21:13:071
224559943,0cyclictest3425-21lldpd23:25:461
224559943,0cyclictest3425-21lldpd21:55:161
224559943,0cyclictest3425-21lldpd21:37:161
224559943,0cyclictest3383-21snmpd23:53:481
224559943,0cyclictest3383-21snmpd21:52:181
224559943,0cyclictest27832-21kworker/1:022:44:021
224559943,0cyclictest27832-21kworker/1:022:18:441
224559943,0cyclictest27832-21kworker/1:020:08:061
224559943,0cyclictest15189-21awk20:58:101
224559943,0cyclictest10716-21awk22:58:091
224549944,0cyclictest0-21swapper/022:30:020
224549944,0cyclictest0-21swapper/022:00:040
224549943,0cyclictest8927-21awk23:58:140
224549943,0cyclictest3425-21lldpd22:49:460
224549943,0cyclictest3425-21lldpd21:43:160
224549943,0cyclictest32192-21awk19:58:100
224549943,0cyclictest26644-21awk20:18:110
224549943,0cyclictest2636-21awk22:53:070
224549943,0cyclictest26348-21sadc20:42:440
179940,0migration/19496-21kthreadcore19:38:101
179940,0migration/19404-21kthreadcore20:53:121
179940,0migration/18687-21aten_r1power_po23:38:011
179940,0migration/14630-21kthreadcore23:33:081
179940,0migration/14205-21kthreadcore20:48:111
179940,0migration/14093-21kthreadcore23:13:081
179940,0migration/13507-21irqcore22:33:081
179940,0migration/13295-21taskset19:11:241
179940,0migration/131849-21taskset20:20:371
179940,0migration/131612-21kthreadcore22:08:101
179940,0migration/131131-21hddtemp_smartct19:33:051
179940,0migration/126149-21runrttasks22:05:161
179940,0migration/126101-21kthreadcore00:28:101
179940,0migration/125459-21pmu-power20:38:161
179940,0migration/12487-21gltestperf22:53:051
179940,0migration/123781-21kthreadcore21:23:121
179940,0migration/12139-21diskmemload21:28:451
179940,0migration/121288-21df21:03:061
179940,0migration/119424-21kthreadcore00:03:121
179940,0migration/118596-21idleruntime23:03:091
179940,0migration/116810-21kthreadcore20:33:121
179940,0migration/113984-21kthreadcore19:43:101
179940,0migration/11263-21kthreadcore19:58:131
169940,0watchdog/19408-21aten_r1power_vo23:18:061
169940,0watchdog/18298-21grep20:28:111
169940,0watchdog/127048-21kthreadcore19:53:111
169940,0watchdog/120672-21diskmemload21:46:541
169940,0watchdog/113384-21ssh00:19:181
169940,0watchdog/110978-21perl22:38:061
169940,0watchdog/10-21swapper/122:47:461
149940,0migration/027669-21irqcore20:43:130
149940,0migration/014803-21kthreadcore20:33:110
35130,0ktimersoftd/30-21swapper/320:46:023
28415030,0irq/27-eth0-tx-0-21swapper/020:08:030
28405030,0irq/26-eth0-rx-0-21swapper/320:38:153
28405030,0irq/26-eth0-rx-0-21swapper/320:15:493
28405030,0irq/26-eth0-rx-0-21swapper/320:08:033
28405030,0irq/26-eth0-rx-0-21swapper/319:53:053
28405030,0irq/26-eth0-rx-0-21swapper/319:23:093
28405030,0irq/26-eth0-rx-0-21swapper/319:18:233
28405030,0irq/26-eth0-rx-0-21swapper/319:13:043
28405030,0irq/26-eth0-rx-0-21swapper/319:02:563
249930,0watchdog/20-21swapper/220:37:592
224559933,0cyclictest27832-21kworker/1:023:47:591
224559933,0cyclictest27832-21kworker/1:020:42:461
224559933,0cyclictest27832-21kworker/1:020:22:521
224559933,0cyclictest27832-21kworker/1:020:02:481
224559933,0cyclictest27832-21kworker/1:019:47:441
224559933,0cyclictest27832-21kworker/1:019:27:501
224559933,0cyclictest27832-21kworker/1:019:22:461
224559933,0cyclictest27832-21kworker/1:019:02:561
224559933,0cyclictest27832-21kworker/1:000:22:451
224549933,0cyclictest0-21swapper/019:44:200
224549932,0cyclictest9023-21kthreadcore21:13:100
224549932,0cyclictest6569-21awk20:03:060
224549932,0cyclictest3383-21snmpd19:22:480
224549932,0cyclictest3383-21snmpd00:29:060
224549932,0cyclictest31218-21awk19:33:060
224549932,0cyclictest29380-21munin-node19:08:160
224549932,0cyclictest21860-21awk00:03:180
224549932,0cyclictest20241-21awk22:03:080
224549932,0cyclictest18082-21awk23:23:100
224549932,0cyclictest11827-21awk22:18:110
224549932,0cyclictest10956-21kthreadcore23:18:180
224549932,0cyclictest10758-21awk22:58:090
224549930,0cyclictest0-21swapper/019:19:350
169930,0watchdog/10-21swapper/119:17:551
149930,0migration/027270-21sh22:07:590
149930,0migration/018996-21runrttasks20:12:570
339920,0migration/31864-21iostat_ios20:48:103
28405020,0irq/26-eth0-rx-0-21swapper/320:33:133
28405020,0irq/26-eth0-rx-0-21swapper/319:48:053
28405020,0irq/26-eth0-rx-0-21swapper/319:38:043
224549921,0cyclictest28608-21latency_hist21:07:440
224549921,0cyclictest18138-21latency_hist22:42:440
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional