You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-21 - 09:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Sun Apr 21, 2024 00:43:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1875535996350,9cyclictest1905060-21sendmail20:40:062
187553399616,48cyclictest1914911-21diskmemload23:26:010
1875533996123,7cyclictest2005617-21sh23:05:130
1875535994933,11cyclictest1934718-21sh21:35:172
1875533994634,8cyclictest1920571-21/usr/sbin/munin21:15:400
187553499453,35cyclictest1927775-21ssh21:25:341
1875533994435,6cyclictest1943825-21unixbench-2d21:45:370
187553399442,35cyclictest2030512-21ssh23:35:330
1875535994327,11cyclictest351ktimers/222:00:332
187553399433,33cyclictest1951146-21ssh21:55:250
187553399431,34cyclictest1947367-21ssh21:50:330
187553599422,32cyclictest2026403-21sh23:30:252
1875535994219,16cyclictest351ktimers/221:55:282
1875533994229,9cyclictest77550irq/32-end022:25:340
1875535994132,6cyclictest2011088-21unixbench_multi23:10:392
1875535994131,6cyclictest1911618-21munin-plugin-st21:00:072
1875534994133,4cyclictest0-21swapper/123:30:271
187553499412,29cyclictest2014514-21ssh23:15:271
187553499412,29cyclictest2014514-21ssh23:15:271
1875533994130,7cyclictest872-21mta-sts-daemon21:27:140
187553399403,30cyclictest1966993-21ssh22:15:280
1875533994029,7cyclictest2033795-21fschecks_count23:40:170
1875535993910,4cyclictest171rcu_preempt00:30:302
1875533993930,5cyclictest1880396-21unixbench_singl19:20:340
1875533993929,7cyclictest469-21systemd-logind19:55:060
1875533993927,8cyclictest2050117-21latency00:00:240
187553599386,5cyclictest171rcu_preempt00:00:292
1875535993830,5cyclictest1886956-21unixbench_singl19:40:372
1875535993827,7cyclictest412-21avahi-daemon21:55:072
1875535993826,8cyclictest2007142-21unixbench_singl23:05:382
187553599381,30cyclictest1971118-21ssh22:20:332
1875533993827,7cyclictest1994121-21fschecks_time22:50:210
1875533993821,8cyclictest77550irq/32-end022:30:310
1875534993729,5cyclictest1958556-21fschecks_count22:05:201
1875534993728,6cyclictest2045786-21fschecks_count23:55:211
1875533993729,5cyclictest1913977-21gltestperf21:05:210
1875533993726,7cyclictest1954604-21fschecks_count22:00:210
1875533993725,9cyclictest2075831-21latency00:35:250
1875533993723,8cyclictest2010608-21sh23:10:320
1875535993625,8cyclictest2064576-21fschecks_count00:20:192
187553599362,26cyclictest824082-21snmpd21:40:322
187553599362,26cyclictest824082-21snmpd21:40:322
1875535993613,18cyclictest437-21gpm20:15:312
187553499363,6cyclictest0-21swapper/123:26:011
187553499363,25cyclictest2057798-21ssh00:10:321
1875534993627,6cyclictest1885975-21fschecks_time19:40:211
1875533993628,5cyclictest2054628-21unixbench_singl00:05:380
1875533993627,6cyclictest1938841-21fschecks_count21:40:230
1875533993627,6cyclictest1938841-21fschecks_count21:40:230
1875533993626,7cyclictest1885366-21sendmail19:40:070
1875533993623,9cyclictest1935266-21latency21:35:230
1875535993526,6cyclictest2019044-21grep23:20:382
1875535993526,6cyclictest1912260-21gltestperf21:00:182
1875535993525,7cyclictest284-21systemd-journal22:32:592
1875535993525,7cyclictest284-21systemd-journal21:31:292
1875534993527,5cyclictest2071314-21systemd-run00:30:071
1875534993527,5cyclictest2064975-21missed_timers00:20:231
1875534993527,5cyclictest2002120-21gltestperf23:00:191
1875534993525,7cyclictest1943940-21grep21:45:381
1875533993527,5cyclictest1986244-21fschecks_count22:40:200
1875533993527,5cyclictest1907382-21gltestperf20:45:200
1875533993527,5cyclictest1881155-21gltestperf19:25:190
1875533993524,8cyclictest1909994-21sh20:55:070
1875533993512,16cyclictest77550irq/32-end023:45:300
187553599344,25cyclictest1990692-21ssh22:45:262
187553599343,24cyclictest1986869-21ssh22:40:292
1875535993426,6cyclictest284-21systemd-journal00:05:212
1875535993423,8cyclictest1994575-21needreboot22:50:262
1875535993416,8cyclictest284-21systemd-journal23:55:272
1875534993426,5cyclictest2033931-21grep23:40:181
1875534993424,5cyclictest0-21swapper/121:55:251
187553499341,27cyclictest1920055-21ssh21:15:341
1875533993426,5cyclictest2072045-21gltestperf00:30:220
1875533993426,5cyclictest1993449-21idleruntime-cro22:50:060
1875533993423,7cyclictest2067641-21unin-run00:25:070
1875533993423,7cyclictest1918734-21idleruntime-cro21:15:070
187553599334,20cyclictest1963025-21ssh22:10:262
1875535993325,5cyclictest1901707-21grep20:25:362
1875535993324,6cyclictest284-21systemd-journal00:14:222
1875535993323,6cyclictest502-21in:imklog22:25:142
1875535993322,8cyclictest1914861-21unixbench_singl21:05:372
1875535993321,9cyclictest1876550-21ntp_kernel_pll_19:10:322
1875535993321,5cyclictest0-21swapper/221:49:002
1875535993316,12cyclictest351ktimers/220:40:272
1875535993316,12cyclictest351ktimers/220:40:272
187553499333,23cyclictest1939454-21ssh21:40:321
187553499333,23cyclictest1939454-21ssh21:40:321
1875534993324,6cyclictest2075605-21gltestperf00:35:231
1875534993324,6cyclictest2005714-21aten_r2power_en23:05:161
1875534993324,6cyclictest1898442-21grep20:15:381
1875533993327,3cyclictest0-21swapper/023:20:340
1875535993225,4cyclictest1891868-21unixbench_singl19:55:342
187553499322,22cyclictest2038352-21ssh23:45:271
1875534993221,7cyclictest2030839-21timerandwakeup23:35:391
1875534993220,9cyclictest2053971-21netstat00:05:271
1875533993223,4cyclictest11950irq/38-mmc023:30:310
1875533993223,4cyclictest0-21swapper/023:55:210
187553399321,25cyclictest2002651-21ssh23:00:290
1875535993126,2cyclictest0-21swapper/221:20:222
1875535993123,5cyclictest1958691-21gltestperf22:05:212
1875535993122,6cyclictest284-21systemd-journal00:17:402
1875535993122,6cyclictest2017404-21latency_hist23:20:082
1875535993122,6cyclictest2017404-21latency_hist23:20:082
1875535993121,7cyclictest502-21in:imklog21:11:352
1875534993125,4cyclictest878-21mta-sts-daemon19:45:341
1875534993125,4cyclictest878-21mta-sts-daemon19:45:341
1875534993123,5cyclictest1902377-21fschecks_time20:30:191
1875534993123,5cyclictest1895130-21unixbench_multi20:05:361
1875534993123,5cyclictest1882599-21fschecks_count19:30:201
1875534993122,7cyclictest1918764-21sh21:15:081
1875534993122,4cyclictest0-21swapper/121:20:301
1875534993121,7cyclictest1985985-21aten2.4-expect22:40:161
187553499312,13cyclictest1963165-21ntpq22:10:311
187553499311,25cyclictest1970986-21ssh22:20:291
1875535993022,5cyclictest876-21mta-sts-daemon23:03:482
1875535993022,5cyclictest2022030-21gltestperf23:25:242
1875535993021,6cyclictest284-21systemd-journal21:15:442
1875535993020,7cyclictest284-21systemd-journal00:29:272
1875535993020,6cyclictest2038106-21kernelversion23:45:232
1875534993022,5cyclictest1974129-21telnet22:25:181
1875534993021,6cyclictest2060965-21grep00:15:191
1875534993020,8cyclictest1983766-21sh22:36:121
1875534993018,9cyclictest2045141-21/usr/sbin/munin23:55:081
1875534993018,9cyclictest1896270-21ntp_kernel_pll_20:10:271
187553399302,25cyclictest151ktimers/000:10:300
1875533993021,6cyclictest2001375-21sendmail23:00:070
1875533993021,6cyclictest1877858-21gltestperf19:15:220
1875533993019,7cyclictest2014454-21sh23:15:250
1875533993019,7cyclictest2014454-21sh23:15:250
1875533993018,9cyclictest1899519-21ntp_kernel_err20:20:260
187553399301,21cyclictest1971048-21ssh22:20:300
187553599293,20cyclictest1982887-21ssh22:35:282
187553599292,20cyclictest2030430-21ssh23:35:302
1875535992921,6cyclictest351ktimers/220:05:282
1875535992920,6cyclictest1903375-21unixbench_singl20:30:352
1875535992919,8cyclictest1998182-21gltestperf22:55:192
1875535992919,6cyclictest284-21systemd-journal21:28:452
1875535992914,9cyclictest2033505-21/usr/sbin/munin23:40:122
1875534992923,3cyclictest0-21swapper/122:55:141
1875534992921,5cyclictest529-21gdbus20:00:071
1875534992921,5cyclictest2050896-21unixbench_multi00:00:391
1875534992921,4cyclictest0-21swapper/122:00:341
1875534992919,6cyclictest1878790-21cron19:20:071
1875533992922,4cyclictest1886918-21unixbench_multi19:40:360
187553399292,22cyclictest2061373-21ssh00:15:250
1875533992921,5cyclictest503-21rs:main0
1875533992921,5cyclictest1924216-21timerandwakeup21:20:370
1875533992921,5cyclictest1905748-21gltestperf20:40:200
1875533992921,5cyclictest1905748-21gltestperf20:40:200
1875533992921,5cyclictest1904977-21unixbench_singl20:35:360
1875533992920,7cyclictest1911503-21unixbench_multi20:55:390
1875533992920,6cyclictest1895208-21cron20:10:060
1875533992919,7cyclictest1985597-21munin-plugin-st22:40:060
1875533992918,8cyclictest1891313-21ntp_kernel_pll_19:55:250
187553599283,18cyclictest1967059-21ntpq22:15:302
1875535992821,5cyclictest2075128-21/usr/sbin/munin00:35:152
1875535992820,6cyclictest284-21systemd-journal23:50:422
1875535992820,5cyclictest284-21systemd-journal20:23:592
1875535992820,5cyclictest284-21systemd-journal19:51:142
187553599280,8cyclictest171rcu_preempt20:00:292
187553499286,12cyclictest878-21mta-sts-daemon19:35:291
1875534992821,3cyclictest0-21swapper/121:50:301
1875534992820,6cyclictest0-21swapper/122:15:181
1875534992820,5cyclictest1913919-21gltestperf21:05:211
1875534992820,5cyclictest1909919-21unixbench_singl20:50:361
1875534992819,6cyclictest502-21in:imklog23:15:071
187553399283,12cyclictest1963042-21ntpq22:10:260
1875533992821,4cyclictest1883619-21unixbench_multi19:30:370
1875533992820,5cyclictest1901779-21cron20:30:070
1875533992819,6cyclictest1931043-21fschecks_time21:30:190
1875533992819,5cyclictest1887946-21missed_timers19:45:310
1875533992819,5cyclictest1887946-21missed_timers19:45:310
1875535992721,3cyclictest1877688-21fschecks_count19:15:212
1875535992720,4cyclictest1887659-21gltestperf19:45:282
1875535992720,4cyclictest1887659-21gltestperf19:45:282
1875535992720,3cyclictest0-21swapper/220:10:202
187553499272,22cyclictest1990812-21ssh22:45:301
1875534992722,2cyclictest0-21swapper/122:50:251
187553499272,19cyclictest1931635-21ssh21:30:301
1875534992721,3cyclictest1890280-21debian-sa119:55:061
1875534992717,7cyclictest2017935-21fschecks_count23:20:211
1875534992717,6cyclictest0-21swapper/121:35:351
1875534992715,9cyclictest1881483-21ntp_kernel_pll_19:25:261
187553399272,19cyclictest2042309-21ntpq23:50:260
1875533992719,5cyclictest1902171-21aten_r2power_vo20:30:160
1875533992718,6cyclictest1897524-21gltestperf20:15:220
1875533992718,5cyclictest1958675-21gltestperf22:05:210
187553499264,12cyclictest1914911-21diskmemload00:30:101
1875534992616,7cyclictest1978595-21if_wlan022:30:241
1875533992619,4cyclictest2068359-21gltestperf00:25:220
1875533992617,6cyclictest1912988-21threads21:00:320
1875535992518,5cyclictest1881305-21latency19:25:222
1875535992514,7cyclictest878-21mta-sts-daemon19:30:282
187553499253,17cyclictest1893044-21ntp_states20:00:321
187553599242,5cyclictest171rcu_preempt20:50:292
1875534992417,4cyclictest0-21swapper/120:35:361
187553499241,4cyclictest1899611-21ntpq20:20:301
187553499237,12cyclictest878-21mta-sts-daemon19:10:331
187553499232,16cyclictest1907832-21ntpq20:45:311
187553499232,15cyclictest1879934-21ntpq19:20:291
1875534992317,3cyclictest0-21swapper/120:25:361
1875534992316,3cyclictest0-21swapper/121:00:071
1875533992315,5cyclictest1876164-21gltestperf19:10:250
187553399231,16cyclictest1896313-21ntpq20:10:280
187553599229,10cyclictest1879307-21/usr/sbin/munin19:20:172
187553599221,17cyclictest1884925-21awk19:35:302
187553599211,15cyclictest1907822-21ntpq20:45:302
1875534992114,5cyclictest0-21swapper/119:10:081
1875534992111,7cyclictest1906277-21/usr/sbin/munin20:40:321
1875534992111,7cyclictest1906277-21/usr/sbin/munin20:40:321
1875534992013,4cyclictest1912656-21ntpq21:00:241
187553699182,10cyclictest0-21swapper/323:30:263
187553699181,12cyclictest0-21swapper/321:35:283
187553699181,11cyclictest0-21swapper/323:15:263
187553699181,11cyclictest0-21swapper/323:15:263
187553699181,11cyclictest0-21swapper/319:40:273
187553399184,8cyclictest0-21swapper/019:10:070
187553699174,9cyclictest0-21swapper/300:35:303
1875536991711,4cyclictest0-21swapper/323:35:323
187553699171,12cyclictest0-21swapper/323:26:003
187553699171,12cyclictest0-21swapper/322:25:303
187553699171,12cyclictest0-21swapper/321:50:293
187553699171,12cyclictest0-21swapper/321:45:333
187553699171,12cyclictest0-21swapper/321:00:283
187553699171,12cyclictest0-21swapper/320:40:283
187553699171,12cyclictest0-21swapper/320:40:283
187553699171,12cyclictest0-21swapper/320:20:293
187553699171,11cyclictest0-21swapper/323:40:293
187553699171,11cyclictest0-21swapper/320:55:323
187553699170,14cyclictest0-21swapper/319:10:073
187553699170,13cyclictest0-21swapper/321:40:303
187553699170,13cyclictest0-21swapper/321:40:303
1875535991711,3cyclictest0-21swapper/219:10:082
187553699161,12cyclictest0-21swapper/323:15:063
187553699161,12cyclictest0-21swapper/321:55:253
187553699161,12cyclictest0-21swapper/320:30:253
187553699161,12cyclictest0-21swapper/320:05:313
187553699161,12cyclictest0-21swapper/300:15:303
187553699161,11cyclictest0-21swapper/323:55:333
187553699161,11cyclictest0-21swapper/323:50:283
187553699161,11cyclictest0-21swapper/322:55:263
187553699161,11cyclictest0-21swapper/322:15:273
187553699161,11cyclictest0-21swapper/322:10:323
187553699161,11cyclictest0-21swapper/322:00:283
187553699161,11cyclictest0-21swapper/321:25:323
187553699161,11cyclictest0-21swapper/321:20:293
187553699161,11cyclictest0-21swapper/321:05:283
187553699161,11cyclictest0-21swapper/320:45:263
187553699161,11cyclictest0-21swapper/319:55:253
187553699161,11cyclictest0-21swapper/319:50:283
187553699161,11cyclictest0-21swapper/319:35:243
187553699161,11cyclictest0-21swapper/319:20:273
187553699161,11cyclictest0-21swapper/300:00:283
187553699160,12cyclictest0-21swapper/323:00:303
187553699160,11cyclictest0-21swapper/322:20:313
187553699151,11cyclictest0-21swapper/323:45:273
187553699151,11cyclictest0-21swapper/323:20:323
187553699151,11cyclictest0-21swapper/322:50:293
187553699151,11cyclictest0-21swapper/322:35:253
187553699151,11cyclictest0-21swapper/322:30:283
187553699151,11cyclictest0-21swapper/321:30:273
187553699151,11cyclictest0-21swapper/321:15:303
187553699151,11cyclictest0-21swapper/320:25:313
187553699151,11cyclictest0-21swapper/320:00:283
187553699151,11cyclictest0-21swapper/319:28:353
187553699151,11cyclictest0-21swapper/319:15:223
187553699151,11cyclictest0-21swapper/319:10:333
187553699151,11cyclictest0-21swapper/300:30:263
187553699151,11cyclictest0-21swapper/300:25:293
187553699151,11cyclictest0-21swapper/300:20:333
187553699151,11cyclictest0-21swapper/300:10:233
187553699151,11cyclictest0-21swapper/300:05:203
187553699150,12cyclictest0-21swapper/319:45:343
187553699150,12cyclictest0-21swapper/319:45:343
187553699150,11cyclictest0-21swapper/322:40:263
187553699150,11cyclictest0-21swapper/320:50:303
187553699150,11cyclictest0-21swapper/320:15:293
187553699150,11cyclictest0-21swapper/319:30:283
187553699141,10cyclictest0-21swapper/322:45:143
187553699141,10cyclictest0-21swapper/322:05:083
187553699141,10cyclictest0-21swapper/320:35:163
187553699131,11cyclictest0-21swapper/323:05:293
187553699131,10cyclictest0-21swapper/321:10:243
187553699130,10cyclictest0-21swapper/320:15:083
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional