You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-27 - 10:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot4s.osadl.org (updated Sat Apr 27, 2024 00:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
375895994440,3cyclictest74-21kswapd023:45:111
375895994238,2cyclictest588805-21kworker/u8:0+events_unbound22:35:141
375903993431,2cyclictest74-21kswapd023:15:003
375895993431,2cyclictest637207-21kworker/u8:1+flush-259:022:55:001
375895993429,3cyclictest779927-21kworker/u8:2+events_unbound00:35:131
375903993328,3cyclictest588775-21latency_hist22:10:013
375903993327,4cyclictest751884-21latency_hist00:00:013
375903993228,3cyclictest74-21kswapd023:40:003
375893993125,3cyclictest381156-21cp19:15:010
375903992923,4cyclictest458144-21latency_hist20:30:013
375895992925,3cyclictest74-21kswapd023:25:001
375895992925,3cyclictest74-21kswapd023:25:001
375895992925,2cyclictest253-21systemd-journal22:10:011
375893992823,3cyclictest670837-21apt-get23:05:100
375893992821,5cyclictest422331-21latency_hist19:55:010
375899992724,2cyclictest74-21kswapd022:10:012
375899992722,3cyclictest484222-21kworker/u8:2+flush-259:021:10:302
375895992723,3cyclictest499817-21kworker/u8:3+flush-259:021:15:001
375893992622,3cyclictest74-21kswapd022:15:010
375903992522,2cyclictest498-21dbus-daemon22:15:013
375903992521,3cyclictest74-21kswapd000:15:013
375899992520,4cyclictest499817-21kworker/u8:3+flush-259:021:20:282
375893992519,4cyclictest253-21systemd-journal23:20:000
375893992519,4cyclictest253-21systemd-journal23:20:000
375903992421,2cyclictest596145-21apt-get22:15:023
375895992421,2cyclictest677957-21kworker/u8:1+flush-259:023:39:181
375895992421,2cyclictest637207-21kworker/u8:1+events_unbound22:55:131
375895992421,2cyclictest386260-21kworker/u8:1+flush-259:021:10:011
375895992420,3cyclictest499817-21kworker/u8:3+flush-259:021:20:201
375895992420,3cyclictest2529989-21kworker/u8:4+events_unbound19:30:001
375899992319,3cyclictest604720-21ssh22:20:162
37589399231,21cyclictest516-21rs:main0
375903992218,3cyclictest648671-21apt-get22:50:133
375899992218,3cyclictest2529989-21kworker/u8:4+flush-259:020:04:592
375899992210,6cyclictest670501-21munin-run23:05:002
375903992119,1cyclictest74-21kswapd021:55:013
375899992117,3cyclictest544364-21cron21:40:012
375899992114,5cyclictest253-21systemd-journal22:15:012
375895992118,2cyclictest771876-21kworker/u8:0+flush-259:000:30:011
375895992118,2cyclictest489312-21cron21:00:001
375895992118,1cyclictest0-21swapper/122:15:001
375895992117,3cyclictest474253-21apt-get20:45:111
375895992117,3cyclictest2529989-21kworker/u8:4+flush-259:019:30:251
375895992117,3cyclictest2529989-21kworker/u8:4+events_unbound19:15:201
375893992114,5cyclictest463469-21cp20:35:010
375903992017,2cyclictest2529989-21kworker/u8:4+flush-259:020:09:593
375899992017,2cyclictest774881-21irqcore00:15:162
375899992015,3cyclictest552117-21apt-get21:45:132
375895992017,2cyclictest615869-21kworker/u8:3+events_unbound23:40:251
375895992017,2cyclictest529604-21kworker/u8:4+flush-259:021:50:261
375895992015,3cyclictest253-21systemd-journal21:56:501
375903991916,2cyclictest74-21kswapd023:25:133
375903991916,2cyclictest611365-21idleruntime-cro22:25:003
375899991915,3cyclictest662097-21kworker/u8:0+flush-259:023:05:152
375899991915,3cyclictest604013-21kworker/u8:1+flush-259:022:25:142
375895991914,3cyclictest678439-21ssh23:10:141
375895991914,3cyclictest514815-21dump-pmu-power21:20:001
375893991915,3cyclictest529604-21kworker/u8:4+flush-259:021:35:010
375893991914,3cyclictest499994-21latency_hist21:10:010
375903991815,2cyclictest447879-21idleruntime-cro20:20:003
375903991814,3cyclictest663598-21ssh23:00:133
375899991815,2cyclictest636276-21kthreadcore22:40:182
375899991815,2cyclictest0-21swapper/222:35:242
375899991814,2cyclictest0-21swapper/200:40:002
375899991812,4cyclictest253-21systemd-journal22:55:002
375895991815,2cyclictest751300-21kworker/u8:2+events_unbound00:02:241
375895991815,2cyclictest615869-21kworker/u8:3+events_unbound23:30:001
375895991815,2cyclictest528022-21kworker/u8:2+events_unbound21:30:151
375895991814,3cyclictest74-21kswapd023:35:001
375895991814,3cyclictest615869-21kworker/u8:3+flush-259:022:29:091
375895991814,2cyclictest0-21swapper/122:50:001
375895991813,3cyclictest253-21systemd-journal19:35:551
375895991812,4cyclictest464111-21diskstats20:35:141
375903991714,2cyclictest74-21kswapd022:00:013
37589999179,5cyclictest685293-21latency_hist23:15:012
375899991714,2cyclictest401ktimers/223:58:432
375899991713,3cyclictest479367-21apt-get20:50:112
375899991713,3cyclictest2904-21cron20:40:002
375899991711,4cyclictest463255-21apt-get20:35:002
375899991710,4cyclictest648309-21latency_hist22:50:002
375895991714,2cyclictest615869-21kworker/u8:3+flush-259:023:00:251
375895991714,2cyclictest543023-21kworker/u8:1+flush-259:021:45:271
375895991714,2cyclictest402076-21kworker/u8:0+events_unbound19:50:241
375895991713,2cyclictest494484-21cut21:04:591
375895991712,3cyclictest597063-21ssh22:15:111
375893991714,2cyclictest707500-21turbostat.cron23:30:000
375893991714,2cyclictest61150irq/132-enp0s29f1:rx-022:56:490
375893991714,2cyclictest61150irq/132-enp0s29f1:rx-021:14:320
375893991714,2cyclictest61150irq/132-enp0s29f1:rx-000:30:260
375893991714,2cyclictest61150irq/132-enp0s29f1:rx-000:25:430
375893991714,2cyclictest61150irq/132-enp0s29f1:rx-000:00:580
375893991713,3cyclictest694189-21kthreadcore23:20:240
375893991713,3cyclictest694189-21kthreadcore23:20:240
375893991713,3cyclictest402076-21kworker/u8:0+flush-259:019:59:590
375893991713,2cyclictest61150irq/132-enp0s29f1:rx-023:10:160
375893991711,4cyclictest618751-21latency_hist22:30:000
375903991614,1cyclictest615757-21needreboot22:25:203
375903991613,2cyclictest491ktimers/322:04:003
375903991613,2cyclictest491ktimers/300:05:103
375903991613,2cyclictest422331-21sh19:54:593
375903991613,1cyclictest640926-21sh22:45:003
375903991612,3cyclictest721921-21kworker/u8:0+flush-259:023:40:113
375903991612,3cyclictest545033-21ssh21:40:163
375903991612,2cyclictest253-21systemd-journal00:00:503
37589999168,5cyclictest391614-21latency_hist19:25:012
375899991612,2cyclictest637207-21kworker/u8:1+events_unbound22:55:202
375899991612,2cyclictest2904-21cron22:20:002
375899991612,2cyclictest0-21swapper/223:25:002
375899991612,2cyclictest0-21swapper/223:25:002
375899991611,4cyclictest781948-21cat00:20:022
375899991611,4cyclictest468851-21kworker/u8:4+flush-259:020:55:172
375895991613,2cyclictest779927-21kworker/u8:2+events_unbound00:33:521
375895991613,2cyclictest751300-21kworker/u8:2+flush-259:000:10:191
375895991613,2cyclictest573296-21kworker/u8:2+flush-259:022:32:241
375895991613,2cyclictest529604-21kworker/u8:4+events_unbound21:40:211
375895991613,2cyclictest468850-21kworker/u8:3+flush-259:020:40:231
375895991613,2cyclictest2529989-21kworker/u8:4+events_unbound19:40:311
375895991612,2cyclictest573985-21kworker/u8:3+flush-259:022:00:251
375895991612,2cyclictest376541-21irqcore19:10:201
375893991614,1cyclictest61150irq/132-enp0s29f1:rx-022:49:000
375893991614,1cyclictest61150irq/132-enp0s29f1:rx-022:40:160
375893991614,1cyclictest61150irq/132-enp0s29f1:rx-022:21:060
375893991614,1cyclictest61150irq/132-enp0s29f1:rx-022:09:430
375893991614,1cyclictest61150irq/132-enp0s29f1:rx-021:53:380
375893991614,1cyclictest61150irq/132-enp0s29f1:rx-020:20:210
375893991614,1cyclictest0-21swapper/023:00:130
375893991613,2cyclictest61150irq/132-enp0s29f1:rx-023:50:020
375893991613,2cyclictest61150irq/132-enp0s29f1:rx-023:40:130
375893991613,2cyclictest61150irq/132-enp0s29f1:rx-022:50:270
375893991613,2cyclictest61150irq/132-enp0s29f1:rx-022:17:410
375893991613,2cyclictest61150irq/132-enp0s29f1:rx-021:49:480
375893991613,2cyclictest61150irq/132-enp0s29f1:rx-021:40:290
375893991613,2cyclictest61150irq/132-enp0s29f1:rx-021:36:250
375893991613,2cyclictest61150irq/132-enp0s29f1:rx-000:38:300
375899991512,2cyclictest559603-21cpuspeed_turbos21:50:152
375899991511,3cyclictest386260-21kworker/u8:1+flush-259:019:45:232
37589599159,4cyclictest427446-21latency_hist20:00:011
375895991512,2cyclictest615869-21kworker/u8:3+flush-259:023:09:341
375895991512,2cyclictest2529989-21kworker/u8:4+events_unbound20:15:511
375895991511,3cyclictest453271-21apt-get20:25:121
375895991511,3cyclictest2529989-21kworker/u8:4+events_unbound20:20:181
375893991513,1cyclictest61150irq/132-enp0s29f1:rx-023:55:180
375893991513,1cyclictest61150irq/132-enp0s29f1:rx-023:30:170
375893991513,1cyclictest61150irq/132-enp0s29f1:rx-022:03:320
375893991513,1cyclictest61150irq/132-enp0s29f1:rx-021:57:090
375893991513,1cyclictest61150irq/132-enp0s29f1:rx-021:26:500
375893991513,1cyclictest61150irq/132-enp0s29f1:rx-020:40:180
375893991513,1cyclictest61150irq/132-enp0s29f1:rx-020:26:250
375893991513,1cyclictest61150irq/132-enp0s29f1:rx-000:20:290
375893991513,1cyclictest61150irq/132-enp0s29f1:rx-000:15:260
375893991513,1cyclictest61150irq/132-enp0s29f1:rx-000:11:550
375893991513,1cyclictest61150irq/132-enp0s29f1:rx-000:09:230
375893991513,1cyclictest0-21swapper/020:59:590
375893991512,2cyclictest61150irq/132-enp0s29f1:rx-023:46:120
375893991512,2cyclictest61150irq/132-enp0s29f1:rx-022:37:020
375893991512,2cyclictest61150irq/132-enp0s29f1:rx-021:15:340
375893991512,2cyclictest61150irq/132-enp0s29f1:rx-020:00:110
375893991511,3cyclictest386260-21kworker/u8:1+flush-259:020:10:240
375893991510,1cyclictest61150irq/132-enp0s29f1:rx-022:30:110
375903991413,1cyclictest491ktimers/323:15:283
375903991413,1cyclictest491ktimers/323:15:283
375903991412,1cyclictest741802-21cat23:50:233
375903991412,1cyclictest491ktimers/321:15:383
375903991411,2cyclictest693185-21tr23:20:143
375903991411,2cyclictest693185-21tr23:20:143
375903991411,2cyclictest615869-21kworker/u8:3+flush-259:023:05:263
375903991411,2cyclictest529604-21kworker/u8:4+flush-259:021:46:143
375903991411,2cyclictest468851-21kworker/u8:4+flush-259:020:55:213
375903991410,3cyclictest514-21in:imuxsock19:45:003
37589999148,4cyclictest253-21systemd-journal20:45:002
37589999148,4cyclictest253-21systemd-journal19:54:592
37589999144,1cyclictest7335812tr23:45:192
375899991411,2cyclictest401ktimers/222:30:122
375899991410,3cyclictest742615-21ssh23:51:282
375899991410,3cyclictest707336-21ssh23:29:442
375899991410,2cyclictest0-21swapper/219:15:002
37589999140,1cyclictest479105-21awk20:49:592
375895991411,2cyclictest751300-21kworker/u8:2+flush-259:000:07:231
375895991411,2cyclictest615869-21kworker/u8:3+flush-259:000:16:401
375895991411,2cyclictest615869-21kworker/u8:3+events_unbound23:55:581
375895991411,2cyclictest604013-21kworker/u8:1+events_unbound22:22:521
375895991410,3cyclictest771876-21kworker/u8:0+events_unbound00:24:061
375895991410,3cyclictest615869-21kworker/u8:3+events_unbound22:42:151
375895991410,3cyclictest529604-21kworker/u8:4+flush-259:021:35:221
375895991410,3cyclictest386260-21kworker/u8:1+events_unbound20:05:171
37589599140,2cyclictest692742-21cut23:20:001
37589599140,2cyclictest692742-21cut23:20:001
375893991413,1cyclictest61950irq/133-enp0s29f1:tx-019:30:260
375893991412,1cyclictest61150irq/132-enp0s29f1:rx-021:21:290
375893991412,1cyclictest61150irq/132-enp0s29f1:rx-020:51:480
375893991412,1cyclictest417217-21turbostat19:49:590
375893991411,2cyclictest61150irq/132-enp0s29f1:rx-019:21:490
375893991411,1cyclictest0-21swapper/021:05:010
37590399139,3cyclictest74-21kswapd023:35:003
37590399139,3cyclictest506633-21ssh21:13:353
37590399139,3cyclictest386260-21kworker/u8:1+flush-259:020:45:003
37590399139,2cyclictest0-21swapper/319:39:593
37590399139,2cyclictest0-21swapper/300:15:093
37590399138,4cyclictest364989-21kworker/u8:0+flush-259:019:10:283
375903991311,1cyclictest491ktimers/322:45:493
375903991311,1cyclictest0-21swapper/322:55:013
375903991310,2cyclictest811568-21idleruntime-cro00:40:003
375903991310,2cyclictest0-21swapper/300:35:013
37589999139,3cyclictest74-21kswapd021:55:102
37589999139,3cyclictest720691-21ssh23:36:592
37589999137,4cyclictest6915641ssh23:17:492
37589999137,4cyclictest6915641ssh23:17:492
37589999137,4cyclictest5278651ssh21:26:492
375899991311,1cyclictest0-21swapper/221:19:592
375899991310,2cyclictest406537-21unixbench_singl19:35:272
375899991310,2cyclictest0-21swapper/220:20:212
375895991310,2cyclictest507427-21kworker/u8:0+events_unbound21:25:121
375895991310,2cyclictest478941-21kworker/u8:0+flush-259:020:50:231
375895991310,2cyclictest386260-21kworker/u8:1+events_unbound20:32:511
375893991312,1cyclictest61150irq/132-enp0s29f1:rx-020:35:260
37590399129,2cyclictest619329-21fschecks_count22:30:143
37590399129,2cyclictest508-21systemd-logind21:10:013
37590399129,2cyclictest507427-21kworker/u8:0+xprtiod21:34:433
37590399129,1cyclictest0-21swapper/320:15:003
37590399128,2cyclictest0-21swapper/319:30:183
37590399127,4cyclictest386748-21/usr/sbin/munin19:20:213
37589999128,3cyclictest386260-21kworker/u8:1+flush-259:020:25:172
37589999128,2cyclictest0-21swapper/200:10:142
37589999122,5cyclictest396756-21munin-run19:30:002
375899991211,1cyclictest401ktimers/221:34:052
37589599129,2cyclictest677957-21kworker/u8:1+events_unbound23:51:491
37589599129,2cyclictest2529989-21kworker/u8:4+events_unbound19:45:551
37589599120,12cyclictest0-21swapper/120:14:551
375893991211,1cyclictest61150irq/132-enp0s29f1:rx-019:39:270
375893991210,1cyclictest61150irq/132-enp0s29f1:rx-019:40:140
37590399119,1cyclictest468112-21sensors20:35:263
37590399119,1cyclictest0-21swapper/321:20:133
37590399119,1cyclictest0-21swapper/320:54:593
37590399118,2cyclictest627566-21kthreadcore22:35:213
37590399118,1cyclictest0-21swapper/320:30:173
37590399118,1cyclictest0-21swapper/300:21:513
37590399117,3cyclictest735582-21ssh23:47:073
37590399117,2cyclictest4745382sleep320:45:173
37590399115,4cyclictest790224-21cut00:25:203
37590399115,4cyclictest529578-21latency_hist21:30:013
37589999119,2cyclictest401ktimers/200:34:052
37589999118,2cyclictest722745-21cpuspeed_turbos23:40:142
37589999118,2cyclictest432891-21cpuspeed_turbos20:05:122
37589999117,3cyclictest796064-21ssh00:28:502
37589999117,3cyclictest677957-21kworker/u8:1+flush-259:000:05:012
37589999115,1cyclictest0-21swapper/200:05:002
37589999110,2cyclictest412054-21munin-run19:45:002
37589599118,2cyclictest430007-21kthreadcore20:00:171
37589399119,1cyclictest61150irq/132-enp0s29f1:rx-020:47:380
37589399119,1cyclictest61150irq/132-enp0s29f1:rx-020:05:130
375893991110,1cyclictest61150irq/132-enp0s29f1:rx-019:15:200
37590399108,1cyclictest493761-21sensors21:00:243
37590399108,1cyclictest491ktimers/321:37:063
37590399105,3cyclictest412603-21fw_forwarded_lo19:45:133
37589999108,1cyclictest493763-21sensors21:00:242
37589999107,2cyclictest536163-21kworker/2:2+events_power_efficient23:30:172
37589999107,1cyclictest0-21swapper/221:40:192
37589999106,2cyclictest574302-21sh22:00:122
37589999105,3cyclictest401128-21/usr/sbin/munin19:30:242
37589999105,3cyclictest386260-21kworker/u8:1+events_unbound20:15:172
37589999104,4cyclictest253-21systemd-journal19:59:512
37589599108,1cyclictest390901-21sensors19:20:251
37589399108,1cyclictest0-21swapper/020:15:180
3759039997,1cyclictest392246-21ethtool19:25:163
3759039996,2cyclictest427802-21cpuspeed_turbos20:00:113
3759039995,3cyclictest448127-21/usr/sbin/munin20:20:223
3759039995,3cyclictest386260-21kworker/u8:1+flush-259:019:55:283
3758999998,1cyclictest432338-21kworker/2:2+events_power_efficient20:11:222
3758999998,1cyclictest332649-21kworker/2:0+events_power_efficient19:19:042
3758999995,3cyclictest253-21systemd-journal21:06:522
3758959990,8cyclictest375958-21cat19:10:001
3759039987,1cyclictest491ktimers/319:15:103
3758999975,1cyclictest0-21swapper/219:10:002
3758939971,3cyclictest375894-21cut19:10:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional