You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-19 - 00:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack2slot7s.osadl.org (updated Thu Jul 18, 2024 12:42:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
814811995429,22cyclictest903952-21fschecks_time10:40:540
814811995228,5cyclictest962504-21ssh12:16:110
814811994814,13cyclictest814-21mta-sts-daemon11:36:060
814812994738,6cyclictest339-21dbus-daemon09:45:011
814813994638,5cyclictest855397-21systemd-run09:15:002
814813994536,6cyclictest4152088-21cups-browsed12:21:502
814813994435,6cyclictest419331-21systemd-journal08:55:002
814811994436,5cyclictest824282-21systemd-run07:40:000
814811994436,5cyclictest824282-21systemd-run07:40:000
814813994336,3cyclictest0-21swapper/211:31:072
814813994335,5cyclictest885512-21latency_hist10:10:012
814813994335,5cyclictest877109-21systemd-run09:55:012
814813994234,5cyclictest814-21mta-sts-daemon12:15:002
814812994235,4cyclictest954866-21cron12:05:001
814812994232,7cyclictest813-21mta-sts-daemon12:05:551
814811994232,4cyclictest1550ktimers/010:01:110
814813994132,6cyclictest335-21avahi-daemon10:59:522
81481199411,33cyclictest940364-21ssh11:41:090
814813994031,5cyclictest860902-21systemd-run09:25:012
814812994032,5cyclictest836892-21latency_hist08:20:011
814812994031,5cyclictest878275-21needreboot09:56:051
814812994031,5cyclictest878275-21needreboot09:56:051
814811994032,5cyclictest894148-21cron10:25:000
814811994031,7cyclictest814767-21/usr/sbin/munin07:10:010
814811994025,12cyclictest828952-21cron07:55:000
814811994018,13cyclictest151ktimers/011:51:070
814813993933,3cyclictest0-21swapper/212:20:002
814813993930,6cyclictest391-21systemd-logind10:15:002
814813993928,7cyclictest339-21dbus-daemon11:20:542
814811993930,6cyclictest860868-21cron09:25:000
814813993830,5cyclictest973661-21systemd-run12:35:002
814813993830,5cyclictest942143-21systemd-run11:45:002
814813993830,5cyclictest903303-21cat10:40:022
814812993830,5cyclictest915803-21fschecks_count11:00:541
814812993830,5cyclictest909182-21idleruntime-cro10:50:001
814811993828,7cyclictest891193-21idleruntime-cro10:20:000
814813993732,2cyclictest0-21swapper/211:26:022
814813993730,4cyclictest813-21mta-sts-daemon11:17:312
814813993727,6cyclictest874340-21idleruntime-cro09:50:002
814812993729,5cyclictest895473-21ssh10:26:451
814812993728,6cyclictest339-21dbus-daemon11:10:001
814812993728,6cyclictest339-21dbus-daemon11:10:001
814812993728,6cyclictest339-21dbus-daemon10:55:011
81481199374,26cyclictest775-21ntpd12:06:080
814811993730,4cyclictest0-21swapper/011:05:000
814811993724,10cyclictest959971-21grep12:11:160
814811993722,12cyclictest910029-21fschecks_time10:50:560
81481199371,31cyclictest907324-21ssh10:46:010
81481199371,30cyclictest968813-21ssh12:26:040
81481199371,30cyclictest968813-21ssh12:26:040
814811993711,15cyclictest151ktimers/012:21:090
814813993631,2cyclictest0-21swapper/210:20:002
814813993629,3cyclictest0-21swapper/209:45:012
814813993628,5cyclictest936835-21kernelversion11:36:002
814813993628,5cyclictest866101-21true09:35:002
814812993630,3cyclictest0-21swapper/109:20:001
814812993628,5cyclictest894154-21cron10:25:001
814812993628,5cyclictest891188-21latency_hist10:20:001
814812993627,7cyclictest696-21cron11:45:001
814811993628,5cyclictest951539-21ssh12:00:000
814811993628,5cyclictest843149-21idleruntime-cro08:40:000
81481199361,31cyclictest928215-21ssh11:21:100
814813993528,4cyclictest862051-21sh09:26:022
814813993525,8cyclictest821876-21latency07:31:012
814813993525,8cyclictest821876-21latency07:31:012
814812993527,5cyclictest339-21dbus-daemon11:00:011
814812993527,5cyclictest339-21dbus-daemon09:00:001
814812993527,5cyclictest339-21dbus-daemon09:00:001
81481299351,24cyclictest959385-21ssh12:11:041
81481199358,22cyclictest775-21ntpd07:21:010
814811993527,5cyclictest934105-21kernelversion11:30:580
814811993524,8cyclictest852599-21diskmemload10:40:020
81481199351,23cyclictest922412-21diskmemload11:11:070
81481199351,23cyclictest922412-21diskmemload11:11:070
814813993430,2cyclictest0-21swapper/208:25:002
814813993427,5cyclictest968321-21sort12:25:552
814813993427,5cyclictest968321-21sort12:25:552
814813993424,8cyclictest824275-21sh07:40:002
814813993424,8cyclictest824275-21sh07:40:002
814813993424,7cyclictest892764-21switchtime10:21:152
814812993426,5cyclictest4152088-21cups-browsed10:00:551
814812993425,6cyclictest4152088-21cups-browsed12:39:351
814811993427,4cyclictest943698-21switchtime11:46:140
814811993426,5cyclictest853577-21sort09:10:540
814811993424,7cyclictest418-21gdbus09:31:170
814811993424,7cyclictest335-21avahi-daemon09:56:270
814811993424,7cyclictest335-21avahi-daemon09:56:270
814811993423,8cyclictest876022-21unixbench_multi09:51:170
814813993326,4cyclictest906122-21idleruntime-cro10:45:002
814813993325,5cyclictest339-21dbus-daemon10:05:002
814813993324,7cyclictest696-21cron12:40:002
814813993324,6cyclictest4152088-21cups-browsed11:00:252
814813993324,5cyclictest339-21dbus-daemon09:18:222
814812993326,4cyclictest968719-21ssh12:26:011
814812993326,4cyclictest968719-21ssh12:26:011
814812993324,6cyclictest335-21avahi-daemon12:16:381
814811993325,5cyclictest863648-21ssh09:30:010
814811993324,5cyclictest898841-21unixbench_multi10:31:190
814811993324,5cyclictest0-21swapper/009:46:050
814811993310,13cyclictest151ktimers/008:46:030
814813993226,3cyclictest0-21swapper/207:20:002
814813993225,4cyclictest840001-21idleruntime-cro08:30:002
814813993225,3cyclictest0-21swapper/211:10:572
814813993225,3cyclictest0-21swapper/211:10:572
814813993224,5cyclictest955770-21fschecks_time12:05:572
814812993226,3cyclictest0-21swapper/110:11:151
814812993224,5cyclictest841554-21cron08:35:001
814812993224,5cyclictest837995-21sendmail_mailqu08:21:091
814812993224,5cyclictest335-21avahi-daemon09:25:001
814812993221,8cyclictest898828-21unixbench_multi10:31:191
814811993225,4cyclictest849463-21kworker/u8:10+xprtiod09:00:000
814811993225,4cyclictest849463-21kworker/u8:10+xprtiod09:00:000
814811993224,5cyclictest914157-21sed10:56:190
814811993224,5cyclictest852611-21systemd-run09:10:000
814811993217,11cyclictest12550irq/32-eth010:26:480
814813993125,3cyclictest0-21swapper/211:05:502
814813993125,3cyclictest0-21swapper/211:05:502
814813993125,3cyclictest0-21swapper/210:26:352
814813993125,3cyclictest0-21swapper/209:10:002
814813993124,4cyclictest811-21mta-sts-daemon07:15:002
814813993123,4cyclictest0-21swapper/211:55:502
814813993121,8cyclictest943918-21tail11:46:152
814812993125,3cyclictest0-21swapper/111:40:001
814812993123,5cyclictest4152088-21cups-browsed07:44:021
814811993124,4cyclictest811-21mta-sts-daemon11:15:060
814811993123,5cyclictest835279-21systemd-run08:15:010
814811993117,6cyclictest974215-21aten_r2power_fr12:35:530
814813993022,5cyclictest909203-21sh10:50:012
814813993022,5cyclictest868774-21munin-run09:40:002
814813993010,5cyclictest358606250lldpd07:26:172
814813993010,5cyclictest358606250lldpd07:26:172
814812993024,4cyclictest814810-21cyclictest11:10:581
814812993024,4cyclictest814810-21cyclictest11:10:581
814812993024,3cyclictest838443-21kworker/u8:1-nfsiod08:40:011
814812993024,3cyclictest0-21swapper/111:20:001
814812993024,3cyclictest0-21swapper/109:30:001
814812993023,4cyclictest933118-21ssh11:30:011
814812993022,5cyclictest814-21mta-sts-daemon09:10:541
814812993021,6cyclictest832081-21unixbench_singl08:01:141
814812993021,6cyclictest821122-21systemd-run07:30:001
814812993021,6cyclictest821122-21systemd-run07:30:001
814812993020,7cyclictest847858-21/usr/sbin/munin08:55:011
814811993024,3cyclictest0-21swapper/010:06:080
814811993019,8cyclictest952447-21fschecks_time12:00:560
814811993019,8cyclictest868765-21cron09:40:000
814813992922,4cyclictest814-21mta-sts-daemon07:55:512
814813992921,5cyclictest852599-21diskmemload12:01:092
814813992921,5cyclictest844632-21unixbench_multi08:41:442
814813992921,5cyclictest836833-21cron08:20:002
814813992918,8cyclictest878286-21needreboot09:56:062
814813992918,8cyclictest878286-21needreboot09:56:062
814812992921,5cyclictest946992-21unixbench-2d11:51:141
814812992921,5cyclictest885543-21latency_hist10:10:011
814812992921,5cyclictest418-21gdbus07:19:461
814812992921,5cyclictest4152088-21cups-browsed11:57:441
814812992921,5cyclictest4152088-21cups-browsed07:57:091
814812992920,7cyclictest872609-21missed_timers09:46:001
814812992920,6cyclictest877117-21unin-run09:55:011
81481299291,20cyclictest928316-21ssh11:21:151
814811992923,3cyclictest858400-21sendmail09:20:000
814811992922,4cyclictest830546-21readlink08:00:010
814811992922,4cyclictest817-21mta-sts-daemon08:01:110
814811992918,8cyclictest0-21swapper/010:15:000
814813992821,4cyclictest811-21mta-sts-daemon10:30:112
814813992818,8cyclictest834668-21ntp_kernel_pll_08:11:042
814812992820,5cyclictest817-21mta-sts-daemon10:35:591
814812992820,5cyclictest4152088-21cups-browsed07:12:011
814812992820,4cyclictest965515-21ssh12:21:061
814812992819,6cyclictest822676-21munin-plugin-st07:35:001
814812992819,6cyclictest822676-21munin-plugin-st07:35:001
814812992819,6cyclictest335-21avahi-daemon09:10:001
81481299281,14cyclictest864755-21ntpq09:31:071
814811992821,5cyclictest816659-21aten_r2power_en07:15:540
814811992820,5cyclictest844743-21systemd-run08:45:000
814811992819,7cyclictest840770-21latency08:30:570
814811992819,7cyclictest836210-21ntp_kernel_pll_08:16:020
814811992818,7cyclictest816313-21unixbench_singl07:11:120
814811992817,2cyclictest3586050-21snmpd11:06:040
814811992817,2cyclictest3586050-21snmpd11:06:040
814811992812,10cyclictest971869-21ssh12:31:040
814813992721,3cyclictest0-21swapper/210:55:002
814813992720,4cyclictest825355-21proc_pri07:41:132
814813992720,4cyclictest817-21mta-sts-daemon07:51:072
814813992720,3cyclictest0-21swapper/211:54:592
814812992722,2cyclictest0-21swapper/107:25:001
814812992721,3cyclictest945197-21kworker/u8:7-nfsiod12:34:311
814812992721,3cyclictest0-21swapper/108:45:001
814812992720,4cyclictest813-21mta-sts-daemon11:47:241
814812992719,5cyclictest833578-21unixbench_multi08:06:131
814812992719,5cyclictest411-21in:imuxsock07:10:011
814812992718,7cyclictest839653-21switchtime08:26:141
814812992718,6cyclictest934885-21unixbench_multi11:32:161
814811992720,4cyclictest813-21mta-sts-daemon11:27:090
814811992719,5cyclictest833804-21latency_hist08:10:020
814811992719,5cyclictest827328-21unixbench_singl07:46:120
814811992719,5cyclictest821116-21systemd-run07:30:000
814811992719,5cyclictest821116-21systemd-run07:30:000
814811992716,4cyclictest838419-21cron08:25:000
814813992619,4cyclictest849437-21systemctl09:00:002
814813992619,4cyclictest849437-21systemctl09:00:002
814813992619,4cyclictest843143-21awk08:40:002
814813992617,5cyclictest4152088-21cups-browsed07:20:532
814812992621,2cyclictest0-21swapper/108:15:001
814812992621,2cyclictest0-21swapper/107:40:001
814812992621,2cyclictest0-21swapper/107:40:001
814812992616,7cyclictest867147-21missed_timers09:36:071
81481299261,19cyclictest903964-21grep10:40:541
814813992519,4cyclictest850361-21netstat09:01:002
814813992519,4cyclictest850361-21netstat09:01:002
814813992519,3cyclictest0-21swapper/208:06:092
81481399250,5cyclictest171rcu_preempt07:46:002
814811992517,5cyclictest821404-21aten_r2power_en07:30:540
814811992517,5cyclictest821404-21aten_r2power_en07:30:540
814811992516,6cyclictest839871-21tail08:26:160
81481199251,18cyclictest870111-21ssh09:41:120
814813992416,3cyclictest0-21swapper/208:46:062
814812992420,3cyclictest814810-21cyclictest09:02:301
814812992420,3cyclictest814810-21cyclictest09:02:301
814812992418,3cyclictest0-21swapper/107:50:571
814813992316,4cyclictest832065-21unixbench_singl08:01:142
814813992316,3cyclictest0-21swapper/208:35:002
814812992316,3cyclictest0-21swapper/108:50:001
814811992318,4cyclictest814810-21cyclictest09:00:510
814811992318,4cyclictest814810-21cyclictest09:00:510
814812992218,2cyclictest0-21swapper/107:45:541
81481199222,15cyclictest847220-21ntpq08:51:000
81481199222,15cyclictest825209-21ntpq07:41:060
81481399191,15cyclictest1-21systemd07:10:002
81481499182,5cyclictest0-21swapper/308:16:043
81481499179,5cyclictest431ktimers/307:10:003
81481499172,8cyclictest0-21swapper/309:41:093
81481499171,7cyclictest0-21swapper/308:06:023
81481499163,4cyclictest0-21swapper/307:51:003
81481499163,4cyclictest0-21swapper/307:26:073
81481499163,4cyclictest0-21swapper/307:26:073
81481499162,6cyclictest0-21swapper/310:21:033
81481499162,5cyclictest0-21swapper/308:51:033
81481499152,8cyclictest0-21swapper/312:01:063
81481499152,8cyclictest0-21swapper/310:55:593
81481499152,7cyclictest0-21swapper/311:01:013
81481499152,7cyclictest0-21swapper/309:21:053
81481499152,6cyclictest0-21swapper/310:30:563
81481499151,8cyclictest0-21swapper/311:26:033
81481499151,8cyclictest0-21swapper/309:16:093
81481499151,6cyclictest0-21swapper/308:11:083
81481499151,5cyclictest0-21swapper/307:46:003
81481499144,5cyclictest0-21swapper/310:16:053
81481499143,7cyclictest0-21swapper/310:26:463
81481499143,7cyclictest0-21swapper/307:41:063
81481499142,7cyclictest0-21swapper/312:35:573
81481499142,7cyclictest0-21swapper/310:40:543
81481499142,7cyclictest0-21swapper/309:50:583
81481499142,6cyclictest0-21swapper/311:36:093
81481499141,8cyclictest0-21swapper/312:21:123
81481499141,8cyclictest0-21swapper/311:51:073
81481499141,7cyclictest0-21swapper/311:16:053
81481499140,6cyclictest0-21swapper/308:21:023
81481499134,4cyclictest0-21swapper/312:25:553
81481499134,4cyclictest0-21swapper/312:25:553
81481499134,4cyclictest0-21swapper/309:35:593
81481499133,6cyclictest0-21swapper/311:31:073
81481499133,5cyclictest0-21swapper/311:10:573
81481499133,5cyclictest0-21swapper/311:10:573
81481499132,7cyclictest0-21swapper/312:10:553
81481499132,7cyclictest0-21swapper/311:46:053
81481499132,7cyclictest0-21swapper/311:25:003
81481499132,7cyclictest0-21swapper/309:56:193
81481499132,7cyclictest0-21swapper/309:56:193
81481499132,7cyclictest0-21swapper/309:30:003
81481499132,6cyclictest0-21swapper/310:51:063
81481499132,6cyclictest0-21swapper/307:40:003
81481499132,6cyclictest0-21swapper/307:40:003
81481499131,7cyclictest0-21swapper/312:20:003
81481499131,7cyclictest0-21swapper/311:41:103
81481499131,7cyclictest0-21swapper/310:46:043
81481499131,7cyclictest0-21swapper/310:35:573
81481499131,7cyclictest0-21swapper/310:00:573
81481499131,7cyclictest0-21swapper/309:11:023
81481499131,6cyclictest0-21swapper/308:36:053
81481499131,5cyclictest0-21swapper/309:06:033
81481499130,6cyclictest0-21swapper/307:11:013
81481499124,4cyclictest0-21swapper/311:56:003
81481499123,5cyclictest0-21swapper/310:11:013
81481499122,6cyclictest0-21swapper/311:05:593
81481499122,6cyclictest0-21swapper/311:05:593
81481499121,7cyclictest0-21swapper/312:30:583
81481499121,7cyclictest0-21swapper/309:31:073
81481499121,6cyclictest0-21swapper/312:05:593
81481499121,6cyclictest0-21swapper/310:05:583
81481499121,6cyclictest0-21swapper/307:31:073
81481499121,6cyclictest0-21swapper/307:31:073
81481499120,6cyclictest0-21swapper/308:01:043
81481499112,4cyclictest0-21swapper/308:00:023
81481499111,6cyclictest0-21swapper/309:45:513
81481499101,6cyclictest0-21swapper/307:20:003
81481499101,5cyclictest0-21swapper/309:00:013
81481499101,5cyclictest0-21swapper/309:00:013
81481499101,5cyclictest0-21swapper/308:45:533
81481499101,5cyclictest0-21swapper/308:25:573
8148149994,3cyclictest0-21swapper/308:41:013
8148149993,4cyclictest0-21swapper/308:31:033
8148149984,3cyclictest0-21swapper/307:21:373
8148149981,5cyclictest0-21swapper/309:01:023
8148149981,5cyclictest0-21swapper/309:01:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional