You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-05-23 - 20:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot8s.osadl.org (updated Thu May 23, 2024 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4006776992718,7cyclictest4132802-21kthreadcore08:50:182
4006776992714,12cyclictest0-21swapper/211:25:002
4006770992724,2cyclictest4033182-21idleruntime-cro07:30:010
4006776992616,9cyclictest60856-21cron10:20:012
4006774992522,2cyclictest4143390-21sh09:00:001
4006776992411,2cyclictest0-21swapper/209:55:172
4006779992321,1cyclictest182056-21idleruntime-cro11:45:003
4006776992317,5cyclictest203312-21sendmail12:00:002
4006776992311,3cyclictest4083655-21cut08:10:202
4006776992311,3cyclictest0-21swapper/211:40:002
4006776992311,3cyclictest0-21swapper/211:40:002
4006776992311,2cyclictest0-21swapper/212:15:162
4006779992218,3cyclictest533-21gdbus12:40:013
4006776992218,3cyclictest125925-21irqcore11:05:172
4006776992212,9cyclictest0-21swapper/209:30:252
4006776992211,9cyclictest118125-21cron11:00:002
4006776992210,3cyclictest4137864-21gltestperf08:55:162
4006776992210,3cyclictest166896-21kworker/u8:0-xprtiod12:10:202
4006776992210,2cyclictest0-21swapper/211:15:192
4006776992210,2cyclictest0-21swapper/210:30:012
4006779992118,2cyclictest510-21in:imuxsock12:30:003
400677699219,3cyclictest730-21cups-browsed10:25:012
400677699219,3cyclictest0-21swapper/209:45:192
4006770992116,4cyclictest0-21swapper/011:35:000
400677999208,2cyclictest191rcu_preempt11:15:153
4006779992017,2cyclictest224667-21idleruntime-cro12:15:003
4006779992017,2cyclictest0-21swapper/312:00:003
4006779992016,3cyclictest510-21in:imuxsock08:00:013
4006776992011,3cyclictest189540-21cpuspeed_turbos11:50:132
4006774992017,2cyclictest4128229-21kthreadcore08:45:201
4006774992016,3cyclictest4170417-21gltestperf09:20:141
4006774992016,3cyclictest0-21swapper/112:00:001
4006770992017,2cyclictest0-21swapper/011:30:000
4006770992017,2cyclictest0-21swapper/011:30:000
4006770992016,2cyclictest0-21swapper/011:45:000
4006779991917,1cyclictest4169803-21cron09:20:003
4006779991916,2cyclictest353-21systemd-journal11:10:003
4006779991916,2cyclictest253240-21latency_hist12:35:013
4006779991915,2cyclictest0-21swapper/312:10:003
400677699199,2cyclictest0-21swapper/210:45:312
4006776991915,3cyclictest47452-21gltestperf10:10:192
4006774991916,2cyclictest210972-21fschecks_count12:05:131
4006774991915,2cyclictest0-21swapper/112:35:131
4006779991815,2cyclictest0-21swapper/309:45:003
4006779991813,3cyclictest4027716-21head07:25:143
4006776991815,2cyclictest4155523-21diskmemload11:30:132
400677699181,2cyclictest0-21swapper/207:25:012
4006774991815,2cyclictest4100739-21turbostat.cron08:25:001
4006774991813,4cyclictest353-21systemd-journal11:35:001
4006770991815,2cyclictest491-21avahi-daemon08:00:250
4006770991811,5cyclictest353-21systemd-journal08:00:010
4006779991715,1cyclictest0-21swapper/311:30:003
4006779991715,1cyclictest0-21swapper/311:30:003
4006779991713,3cyclictest132629-21cpuspeed_turbos11:10:123
4006776991714,2cyclictest0-21swapper/211:15:002
4006776991712,3cyclictest91073-21irqrtprio10:40:182
4006774991714,2cyclictest90181-21cpuspeed_turbos10:40:121
4006774991714,2cyclictest87949-21ntp_kernel_err10:35:231
4006774991714,2cyclictest4059729-21kthreadcore07:50:191
4006774991714,2cyclictest24367-21unixbench_singl09:50:361
4006774991714,2cyclictest118719-21cut11:00:141
4006774991714,2cyclictest0-21swapper/109:10:001
4006774991713,3cyclictest4027671-21gltestperf07:25:141
4006774991713,3cyclictest182550-21fschecks_count11:45:141
4006774991713,3cyclictest144968-21unixbench_singl11:15:291
4006774991713,2cyclictest0-21swapper/108:35:001
4006774991712,1cyclictest191rcu_preempt12:20:151
4006770991713,3cyclictest18584-21cut09:50:010
400677099170,3cyclictest4044539-21needreboot07:35:260
1169212170,1sleep00-21swapper/010:55:340
400677999166,2cyclictest191rcu_preempt07:40:013
400677999166,2cyclictest191rcu_preempt07:20:013
4006779991613,2cyclictest2609-21cron11:40:003
4006779991613,2cyclictest2609-21cron11:40:003
4006779991613,2cyclictest228504-21kthreadcore12:15:183
4006779991613,2cyclictest111866-21irqcore10:55:203
4006779991612,2cyclictest167831-21cron11:35:003
4006779991611,1cyclictest40978191kthreadcore08:20:223
400677999160,4cyclictest32992-21find10:00:133
4006776991613,2cyclictest510-21in:imuxsock10:05:002
4006776991613,2cyclictest4155523-21diskmemload09:50:262
4006776991613,2cyclictest210476-21munin-plugin-st12:05:002
4006776991612,3cyclictest4039893-21gltestperf07:35:172
4006776991612,2cyclictest0-21swapper/211:50:002
4006774991613,2cyclictest510-21in:imuxsock10:10:011
4006774991613,2cyclictest4087557-21needreboot08:10:251
4006774991613,2cyclictest168381-21fschecks_time11:35:161
4006774991613,2cyclictest168381-21fschecks_time11:35:161
4006774991612,3cyclictest116880-21unixbench_singl10:55:321
4006770991613,2cyclictest144922-21unixbench_multi11:15:290
4006770991612,3cyclictest189156-21cron11:49:590
4006770991611,3cyclictest4089136-21cut08:15:160
400677999159,4cyclictest4073471-21ps08:00:193
4006779991512,2cyclictest54248-21cpuspeed_turbos10:15:143
4006779991512,2cyclictest4026688-21unixbench_singl07:20:263
4006779991512,2cyclictest31052-21sendmail_mailtr09:55:263
4006779991512,2cyclictest20500-21kthreadcore09:50:243
4006779991511,3cyclictest4156780-21tr09:10:163
4006779991511,3cyclictest4156780-21tr09:10:163
4006779991511,3cyclictest4088140-21unixbench_singl08:10:353
4006779991511,3cyclictest4081208-21kthreadcore08:05:223
400677699157,6cyclictest353-21systemd-journal10:10:012
4006776991512,2cyclictest239426-21fschecks_time12:25:142
4006776991511,3cyclictest510-21in:imuxsock07:45:012
4006776991511,3cyclictest4070646-21cpuspeed_turbos08:00:122
4006776991511,3cyclictest4015447-21grep07:15:172
4006776991511,2cyclictest4058690-21cut07:50:152
4006776991510,4cyclictest4088912-21cut08:15:122
400677699150,14cyclictest123402-21strings11:00:222
400677499156,7cyclictest104405-21expr10:50:151
400677499154,2cyclictest191rcu_preempt10:10:221
4006774991512,2cyclictest0-21swapper/107:35:011
4006774991511,3cyclictest0-21swapper/111:10:001
4006774991510,4cyclictest4106682-21sshd08:28:171
4006774991510,3cyclictest4113180-21apt-get08:35:121
400677499150,4cyclictest68585-21latency_hist10:25:011
400677099159,4cyclictest353-21systemd-journal08:08:190
4006770991513,1cyclictest44944-21grep10:05:230
4006770991512,2cyclictest91016-21irqrtprio10:40:180
4006770991512,2cyclictest4153573-21kworker/0:0+kacpid12:20:210
4006770991512,2cyclictest0-21swapper/010:20:000
4006770991511,3cyclictest17250irq/127-ahci[0000:00:17.0]07:45:010
4006770991511,3cyclictest130957-21unixbench_singl11:05:280
4006770991511,2cyclictest4013192-21gltestperf07:15:010
4006770991510,3cyclictest4039265-21turbostat07:35:010
400677999141,4cyclictest2695-21rtkit-daemon10:11:173
4006779991412,1cyclictest75632-21munin-plugin-st10:30:013
4006779991412,1cyclictest0-21swapper/308:30:003
4006779991411,2cyclictest66962-21unixbench_multi10:20:313
4006779991411,2cyclictest4182747-21unixbench_multi09:25:563
4006779991411,2cyclictest4155523-21diskmemload09:20:193
4006779991411,2cyclictest4089073-21gltestperf08:15:163
4006779991411,2cyclictest12115-21fschecks_count09:45:153
400677699149,3cyclictest0-21swapper/208:50:002
400677699148,4cyclictest4039192-21sshd07:34:182
400677699147,5cyclictest533-21gdbus12:25:002
400677699147,5cyclictest4113497-21gltestperf08:35:162
400677699147,5cyclictest353-21systemd-journal09:40:002
4006776991411,2cyclictest510-21in:imuxsock09:25:002
4006776991410,3cyclictest4161399-21unixbench_singl09:10:272
4006776991410,3cyclictest4161399-21unixbench_singl09:10:272
4006776991410,2cyclictest0-21swapper/208:25:262
400677499149,4cyclictest4052097-21df_inode07:45:131
400677499149,4cyclictest4026347-21systemctl07:20:241
4006774991411,2cyclictest76006-21cpuspeed_turbos10:30:131
4006774991411,2cyclictest510-21in:imuxsock09:45:001
4006774991411,2cyclictest4155523-21diskmemload09:25:231
4006774991411,2cyclictest4146569-21kthreadcore09:00:231
4006774991411,2cyclictest4046019-21fschecks_count07:40:141
4006774991411,2cyclictest223329-21unixbench_multi12:10:351
4006774991411,2cyclictest204111-21irqcore12:00:181
4006774991411,2cyclictest0-21swapper/110:25:201
4006774991411,2cyclictest0-21swapper/110:05:001
4006774991410,3cyclictest0-21swapper/107:15:031
400677499140,2cyclictest1925920-21kworker/u8:1+nfsiod09:15:201
400677099147,5cyclictest353-21systemd-journal11:40:000
400677099147,5cyclictest353-21systemd-journal11:40:000
400677099144,2cyclictest66325-21missed_timers10:20:230
4006770991411,2cyclictest53857-21turbostat.cron10:15:010
4006770991411,2cyclictest3226-21unixbench_singl09:35:310
4006770991411,2cyclictest0-21swapper/010:45:180
4006770991411,2cyclictest0-21swapper/008:25:000
4006770991411,1cyclictest0-21swapper/010:40:000
4006770991410,3cyclictest247644-21irqrtprio12:30:180
4006770991410,2cyclictest0-21swapper/011:05:000
400677099140,5cyclictest25619-21munin-plugin-st09:55:000
400677099140,2cyclictest4011993-21open_inodes07:10:260
2236262144,2sleep0191rcu_preempt12:11:270
400677999139,3cyclictest4191703-21cpuspeed_turbos09:35:123
400677999139,3cyclictest4150105-21gltestperf09:05:163
400677999137,4cyclictest353-21systemd-journal09:30:183
400677999135,7cyclictest182580-21uniq11:45:143
400677999133,2cyclictest191rcu_preempt12:01:103
400677999132,3cyclictest2695-21rtkit-daemon12:24:413
400677999132,2cyclictest191rcu_preempt11:50:213
4006779991310,2cyclictest4128229-21tr08:45:203
4006779991310,2cyclictest0-21swapper/310:35:003
400677999130,5cyclictest146924-21grep11:20:183
400677999130,3cyclictest1928258-21kworker/3:3+events_power_efficient10:35:203
400677699139,3cyclictest4163195-21fschecks_count09:15:142
400677699139,3cyclictest4076754-21cpuspeed_turbos08:05:142
400677699137,4cyclictest4179636-21tr09:25:252
4006776991310,2cyclictest4007069-21cpuspeed_turbos07:10:152
400677699130,3cyclictest4149480-21dump-pmu-power09:05:002
400677499139,3cyclictest0-21swapper/107:10:211
400677499139,2cyclictest176800-21kthreadcore11:40:201
400677499138,3cyclictest4069220-21memory07:55:201
400677499133,2cyclictest191rcu_preempt11:25:121
400677499133,2cyclictest191rcu_preempt11:25:121
400677499131,2cyclictest191rcu_preempt12:30:301
4006774991311,1cyclictest1299-21kthreadcore09:35:221
4006774991310,2cyclictest4076748-21cpuspeed_turbos08:05:141
4006774991310,2cyclictest0-21swapper/112:25:191
4006774991310,2cyclictest0-21swapper/111:10:151
400677499130,3cyclictest97492-21gltestperf10:45:151
400677499130,3cyclictest54175-21apt-get10:15:121
400677499130,3cyclictest4131610-21fschecks_count08:50:121
400677499130,3cyclictest4075877-21unixbench-2d08:00:271
400677499130,2cyclictest269698rtkit-daemon08:15:211
400677499130,2cyclictest191rcu_preempt10:00:001
400677499130,2cyclictest191rcu_preempt09:10:161
400677499130,2cyclictest191rcu_preempt09:10:161
400677099139,3cyclictest4137194-21kworker/0:1+kacpi_notify09:07:170
400677099139,3cyclictest260318-21systemctl12:40:000
400677099139,2cyclictest4137255-21cron08:55:000
400677099139,2cyclictest0-21swapper/009:15:000
400677099139,2cyclictest0-21swapper/009:15:000
400677099138,4cyclictest370-21systemd-udevd08:49:520
400677099136,5cyclictest495-21dbus-daemon08:10:320
400677099132,10cyclictest152015-21cat11:20:390
400677099131,3cyclictest353-21systemd-journal08:30:000
400677099131,11cyclictest4164711-21kthreadcore09:15:190
400677099131,10cyclictest11625-21munin-run09:45:000
4006770991310,2cyclictest76060-21cpuspeed_turbos10:30:130
4006770991310,2cyclictest4124078-21needreboot08:40:230
4006770991310,2cyclictest4000303-21kworker/0:0+kacpid07:50:190
4006770991310,2cyclictest193123-21kthreadcore11:50:220
4006770991310,2cyclictest0-21swapper/009:20:140
400677099130,3cyclictest353-21systemd-journal09:05:000
400677999129,2cyclictest533-21gdbus07:15:013
400677999129,2cyclictest4050668-21needreboot07:40:213
400677999129,2cyclictest4033811-21cut07:30:163
400677999129,2cyclictest0-21swapper/310:45:213
400677999126,4cyclictest40062-21apt-get10:05:113
400677999123,1cyclictest269698rtkit-daemon10:50:193
400677999123,1cyclictest191rcu_preempt11:00:253
400677999121,1cyclictest41481831kthreadcore09:00:233
4006779991210,1cyclictest4053387-21kthreadcore07:45:203
4006779991210,1cyclictest0-21swapper/310:40:123
4006779991210,1cyclictest0-21swapper/307:50:173
400677999120,3cyclictest4107491-21sort08:30:173
400677999120,3cyclictest0-21swapper/308:55:193
400677699129,2cyclictest4149817-21cpuspeed_turbos09:05:142
400677699129,2cyclictest0-21swapper/210:35:292
400677699128,3cyclictest76266-21cut10:30:162
400677699128,3cyclictest160695-21latency_hist11:30:002
400677699128,3cyclictest160695-21latency_hist11:30:002
400677699128,2cyclictest0-21swapper/211:40:222
400677699128,2cyclictest0-21swapper/207:45:192
400677499129,2cyclictest4184607-21cpuspeed_turbos09:30:121
400677499122,1cyclictest191rcu_preempt08:40:161
400677499120,2cyclictest227945-21kthreadcore12:15:181
400677499120,1cyclictest191rcu_preempt07:35:201
400677099129,2cyclictest4153573-21kworker/0:0+kacpi_notify09:31:020
400677099129,2cyclictest4052300-21gltestperf07:45:160
400677099129,2cyclictest0-21swapper/012:20:010
400677099129,2cyclictest0-21swapper/009:30:010
400677099129,2cyclictest0-21swapper/008:30:310
400677099127,4cyclictest370-21systemd-udevd08:59:520
400677099127,3cyclictest495-21dbus-daemon11:55:280
400677099122,2cyclictest0-21swapper/012:00:200
4006770991210,1cyclictest0-21swapper/012:25:110
4006770991210,1cyclictest0-21swapper/008:35:210
400677099120,11cyclictest216054-21switchtime12:05:250
400677999116,4cyclictest4133034-21tr08:50:183
400677699119,1cyclictest0-21swapper/212:35:102
400677699117,2cyclictest0-21swapper/208:40:312
400677699116,4cyclictest4107380-21tune2fs08:30:162
400677699116,4cyclictest4095918-21sed08:20:192
400677699116,4cyclictest247400-21sed12:30:172
400677699115,4cyclictest4031636-21tr07:25:182
400677699114,5cyclictest110618-21sshd10:53:182
400677499110,2cyclictest191rcu_preempt11:50:161
400677499110,2cyclictest191rcu_preempt09:45:191
400677499110,1cyclictest191rcu_preempt11:20:221
400677099119,1cyclictest4024157-21kthreadcore07:20:180
400677099118,2cyclictest4153573-21kworker/0:0+kacpid09:55:240
400677099117,3cyclictest4153573-21kworker/0:0+kacpi_notify10:25:020
400677099117,2cyclictest111062-21munin-run10:55:000
400677099111,9cyclictest138884-21ssh11:13:410
400677099111,1cyclictest0-21swapper/010:00:210
400677999105,4cyclictest4124618-21sed08:40:313
400677699107,2cyclictest0-21swapper/207:55:112
400677699105,4cyclictest11295-21pool-udisksd09:43:452
400677699104,4cyclictest0-21swapper/212:05:132
40067799996,2cyclictest491ktimers/308:35:123
40067799981,1cyclictest191rcu_preempt07:10:013
40067769981,5cyclictest4006844-21latency_hist07:10:012
40067709985,2cyclictest4006618-21latency_hist07:10:010
26969880,3rtkit-daemon707-21ntpd07:05:491
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional