You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-16 - 01:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Wed Apr 15, 2026 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3301611994842,4cyclictest3567857-21https11:25:002
3301616994638,5cyclictest3457461-21latency_hist09:40:003
3301611994438,4cyclictest3400417-21latency_hist08:45:002
3301616993935,3cyclictest3511174-21kworker/u8:2+flush-259:010:35:213
3301616993931,5cyclictest3646486-21latency_hist12:40:013
3301608993833,3cyclictest3301809-21/usr/sbin/munin07:10:241
3301608993833,3cyclictest3301809-21/usr/sbin/munin07:10:231
3301608993631,3cyclictest3359013-21apt-get08:05:131
3301606993633,2cyclictest3567834-21apt-get11:24:590
3301606993630,4cyclictest3446690-21latency_hist09:30:010
3301611993529,4cyclictest3353581-21latency_hist08:00:012
3301606993529,4cyclictest3536056-21ntpq10:50:200
3301616993427,5cyclictest3467346-21ntpq09:45:213
3301608993429,3cyclictest3338415-21apt-get07:45:121
3301616993327,4cyclictest252-21systemd-journal08:10:003
3301608993326,5cyclictest3599715-21cat11:55:011
3301616993226,4cyclictest3291309-21kworker/u8:4+flush-259:007:20:163
3301611993221,9cyclictest3511155-21latency_hist10:30:012
3301606993226,4cyclictest3343290-21latency_hist07:50:010
3301608993125,4cyclictest3311039-21ntpq07:15:191
3301616993025,4cyclictest3494792-21gpgv10:15:013
3301616993024,4cyclictest3516337-21kworker/u8:4+flush-259:012:10:153
3301606993025,4cyclictest252-21systemd-journal08:52:500
3301608992925,3cyclictest3537360-21diskstats10:55:121
3301608992923,4cyclictest3379536-21cp08:25:021
3301616992721,4cyclictest3516337-21kworker/u8:4+flush-259:010:45:013
3301611992723,3cyclictest3610195-21apt-get12:05:102
3301611992721,5cyclictest3542562-21diskstats11:00:152
3301616992620,4cyclictest3479589-21/usr/sbin/munin10:00:233
3301616992618,5cyclictest3431299-21latency_hist09:15:003
3301608992623,2cyclictest3478052-21kworker/u8:2+events_unbound10:20:001
3301608992619,5cyclictest3357863-21ntpq08:00:211
3301606992622,3cyclictest3557172-21kworker/u8:0+events_unbound11:25:240
3301606992622,3cyclictest3473344-21latency_hist09:55:010
3301606992621,4cyclictest3404706-21ntpq08:45:200
3301606992612,12cyclictest3552677-21apt-get11:10:120
3301606992610,5cyclictest3312790-21diskstats07:20:120
3301616992518,5cyclictest3373313-21ntpq08:15:213
3301611992521,3cyclictest3584015-21apt-get11:40:132
3301611992521,2cyclictest3567872-21kworker/u8:1+events_unbound11:30:162
3301606992519,4cyclictest3542128-21latency_hist11:00:000
3301606992519,4cyclictest3435618-21ntpq09:15:220
3301606992519,4cyclictest252-21systemd-journal09:50:000
3301616992420,2cyclictest0-21swapper/308:00:013
3301616992419,3cyclictest3588900-21kworker/u8:2+flush-259:012:05:253
3301616992418,4cyclictest3630705-21kworker/u8:1+flush-259:012:30:003
3301616992417,4cyclictest3395253-21latency_hist08:40:013
3301611992420,3cyclictest3495456-21kworker/u8:1+flush-259:010:35:002
3301611992419,3cyclictest3605034-21apt-get12:00:112
330160699248,4cyclictest3624978-21ntpq12:15:210
3301616992319,3cyclictest3511174-21kworker/u8:2+flush-259:011:10:003
3301616992318,4cyclictest3516337-21kworker/u8:4+flush-259:012:30:183
3301616992317,4cyclictest509-21rs:main3
3301616992317,4cyclictest3505568-21apt-get10:25:003
3301616992317,4cyclictest3478930-21apt-get10:00:013
3301611992319,3cyclictest3599210-21gpgv11:54:592
3301611992319,3cyclictest3516580-21apt-get10:35:132
3301611992319,3cyclictest3364146-21apt-get08:10:112
3301611992317,5cyclictest3478050-21ntpq09:55:212
3301611992317,4cyclictest3531774-21latency_hist10:50:012
3301606992319,3cyclictest3263804-21kworker/u8:2+flush-259:007:13:110
3301606992319,3cyclictest3263804-21kworker/u8:2+flush-259:007:13:110
3301606992318,4cyclictest3440678-21ntpq09:20:190
3301616992218,3cyclictest3588900-21kworker/u8:2+events_unbound12:00:223
3301616992216,4cyclictest3311936-21apt-get07:20:013
3301608992219,2cyclictest3421035-21kworker/u8:0+flush-259:009:20:141
3301608992217,4cyclictest3636192-21latency_hist12:30:011
3301606992215,5cyclictest3456551-21ntpq09:35:200
3301616992117,3cyclictest3421035-21kworker/u8:0+flush-259:009:05:213
3301616992116,4cyclictest3357676-21kworker/u8:1+events_unbound08:20:233
3301616992116,3cyclictest3446711-21kworker/u8:0+flush-259:010:05:183
3301616992115,4cyclictest3446926-21/usr/sbin/munin09:30:143
3301608992117,3cyclictest3547491-21/usr/sbin/munin11:05:231
3301608992117,3cyclictest3457463-21apt-get09:40:001
3301608992115,4cyclictest3557788-21/usr/sbin/munin11:15:131
3301606992118,2cyclictest3489677-21kworker/u8:3+events_unbound10:25:010
3301606992117,3cyclictest252-21systemd-journal12:32:110
3301616992014,4cyclictest3547250-21latency_hist11:05:003
3301608992018,1cyclictest832-21runrttasks07:30:011
3301608992017,2cyclictest3567864-21gpgv11:24:591
3301608992016,3cyclictest3395524-21apt-get08:40:121
3301608992016,3cyclictest252-21systemd-journal09:34:101
3301608992015,3cyclictest509-21rs:main1
3301608992012,7cyclictest0-21swapper/107:55:011
3301616991914,4cyclictest3557172-21kworker/u8:0+flush-259:011:15:213
3301616991913,5cyclictest3343521-21/usr/sbin/munin07:50:133
3301611991916,2cyclictest3431524-21kworker/u8:4+flush-259:009:15:172
3301608991915,3cyclictest3505575-21cut10:25:011
3301608991914,3cyclictest252-21systemd-journal10:46:111
3301608991913,5cyclictest3317520-21latency_hist07:25:001
3301608991913,3cyclictest252-21systemd-journal11:01:121
3301606991913,5cyclictest60350irq/132-enp0s29f1:rx-011:04:140
3301606991913,5cyclictest3384020-21ntpq08:25:200
3301616991815,2cyclictest3291309-21kworker/u8:4+events_unbound07:40:213
3301616991814,3cyclictest3567872-21kworker/u8:1+events_unbound11:50:153
3301616991814,3cyclictest3312592-21kworker/u8:1+flush-259:007:25:203
3301616991814,3cyclictest3291309-21kworker/u8:4+flush-259:007:30:223
3301616991813,4cyclictest3598304-21kworker/u8:0+events_unbound11:55:223
3301616991813,4cyclictest3516337-21kworker/u8:4+flush-259:011:44:153
3301616991812,4cyclictest3599836-21kworker/u8:3+flush-259:012:20:153
3301616991812,4cyclictest3395279-21kworker/u8:1+events_unbound08:40:143
3301616991811,5cyclictest3410696-21latency_hist08:55:013
3301611991816,1cyclictest0-21swapper/209:55:002
3301611991815,2cyclictest3547938-21irqcore11:05:172
3301611991815,2cyclictest3505588-21systemd-run10:25:012
3301611991815,2cyclictest3325214-21kthreadcore07:30:162
3301611991814,3cyclictest509-21rs:main2
3301608991816,1cyclictest0-21swapper/110:55:011
3301606991814,3cyclictest509-21rs:main0
3301606991814,3cyclictest3291309-21kworker/u8:4+flush-259:007:30:150
3301606991813,4cyclictest60350irq/132-enp0s29f1:rx-008:40:200
3301606991812,1cyclictest60350irq/132-enp0s29f1:rx-009:05:220
3301616991715,1cyclictest0-21swapper/308:59:593
3301616991713,3cyclictest3263804-21kworker/u8:2+flush-259:007:15:003
3301616991713,3cyclictest3263804-21kworker/u8:2+flush-259:007:15:003
3301616991712,4cyclictest3348654-21kworker/u8:2+flush-259:008:00:173
3301616991712,3cyclictest3511174-21kworker/u8:2+flush-259:010:45:153
3301616991711,4cyclictest3379973-21base6408:25:013
3301616991710,5cyclictest3369048-21latency_hist08:15:023
3301611991713,3cyclictest509-21rs:main2
330160899179,7cyclictest0-21swapper/112:15:221
330160899175,3cyclictest252-21systemd-journal10:10:011
3301608991714,1cyclictest0-21swapper/110:15:001
3301608991712,3cyclictest3578831-21apt-get11:35:111
3301606991714,2cyclictest3620305-21/usr/sbin/munin12:15:010
3301606991713,3cyclictest3338148-21sh07:45:000
3301606991712,1cyclictest60350irq/132-enp0s29f1:rx-011:50:230
3301606991712,1cyclictest60350irq/132-enp0s29f1:rx-011:41:130
3301606991711,5cyclictest3389246-21ntpq08:30:220
3301606991711,1cyclictest60350irq/132-enp0s29f1:rx-010:48:180
3301616991613,1cyclictest0-21swapper/309:05:003
3301616991611,4cyclictest3557172-21kworker/u8:0+events_unbound11:35:183
3301616991610,5cyclictest3291309-21kworker/u8:4+events_unbound07:35:143
330161199161,6cyclictest3410700-21idleruntime-cro08:54:592
3301611991614,1cyclictest0-21swapper/209:09:592
3301611991612,3cyclictest3478052-21kworker/u8:2+flush-259:010:05:152
330160899167,8cyclictest0-21swapper/112:20:161
330160899166,9cyclictest3480039-21sed10:00:151
330160899164,9cyclictest0-21swapper/111:40:141
330160899164,1cyclictest3414917-21ntpq08:55:201
330160899164,11cyclictest0-21swapper/109:55:011
330160899162,3cyclictest490-21dbus-daemon10:40:011
3301608991612,3cyclictest3348693-21apt-get07:55:131
330160699169,2cyclictest60350irq/132-enp0s29f1:rx-008:55:040
3301606991614,2cyclictest61150irq/133-enp0s29f1:tx-007:25:220
3301606991614,1cyclictest60350irq/132-enp0s29f1:rx-008:23:440
3301606991614,1cyclictest60350irq/132-enp0s29f1:rx-007:50:230
3301606991614,1cyclictest3552417-21turbostat.cron11:09:590
3301606991613,2cyclictest60350irq/132-enp0s29f1:rx-010:39:040
3301606991613,2cyclictest60350irq/132-enp0s29f1:rx-008:10:240
3301616991511,3cyclictest3431524-21kworker/u8:4+flush-259:009:25:213
3301611991511,3cyclictest252-21systemd-journal09:25:122
330160899155,9cyclictest0-21swapper/111:25:231
330160899154,2cyclictest3510662-21grep10:25:241
330160899154,1cyclictest0-21swapper/108:45:161
330160899153,3cyclictest3613088-21cut12:05:181
330160899153,2cyclictest3511136-21cron10:34:561
330160899153,2cyclictest252-21systemd-journal09:20:001
330160899153,1cyclictest0-21swapper/110:00:011
330160899152,2cyclictest507-21in:imuxsock12:00:101
330160899151,5cyclictest3640417-21ntpq12:30:171
3301608991512,2cyclictest507-21in:imuxsock08:54:591
3301608991512,2cyclictest1-21systemd09:50:001
3301608991512,1cyclictest0-21swapper/111:15:001
3301608991510,3cyclictest3421256-21apt-get09:05:111
330160899150,4cyclictest3391381-21tr08:35:151
330160699159,4cyclictest3332037-21ntpq07:35:200
3301606991513,1cyclictest60350irq/132-enp0s29f1:rx-010:41:190
3301606991513,1cyclictest60350irq/132-enp0s29f1:rx-010:28:190
3301606991513,1cyclictest60350irq/132-enp0s29f1:rx-009:56:390
3301606991513,1cyclictest60350irq/132-enp0s29f1:rx-008:35:140
3301606991513,1cyclictest60350irq/132-enp0s29f1:rx-008:15:120
3301606991513,1cyclictest60350irq/132-enp0s29f1:rx-007:55:160
3301606991513,1cyclictest60350irq/132-enp0s29f1:rx-007:18:440
3301606991512,2cyclictest60350irq/132-enp0s29f1:rx-012:05:110
3301606991512,2cyclictest60350irq/132-enp0s29f1:rx-010:05:100
3301606991512,2cyclictest60350irq/132-enp0s29f1:rx-009:10:240
3301606991512,2cyclictest60350irq/132-enp0s29f1:rx-008:05:240
3301606991510,4cyclictest3593150-21ntpq11:45:190
3301606991510,4cyclictest252-21systemd-journal11:32:110
330161699149,3cyclictest3431524-21kworker/u8:4+events_unbound09:15:203
3301616991411,2cyclictest3343299-21munin-plugin-st07:50:003
330161199148,4cyclictest3338149-21latency_hist07:45:012
3301611991411,2cyclictest3343332-21kworker/u8:0+flush-259:007:51:162
3301611991410,3cyclictest3599836-21kworker/u8:3+flush-259:012:15:172
330160899145,1cyclictest0-21swapper/111:45:151
330160899144,4cyclictest3336198-21cut07:40:171
330160899143,1cyclictest0-21swapper/108:25:011
330160899141,4cyclictest3441968-21cpuspeed_turbos09:25:121
330160899141,3cyclictest3323346-21irqcore07:30:141
330160899141,2cyclictest3368584-21grep08:10:241
330160899141,2cyclictest3332054-21ntpq07:35:191
330160899140,4cyclictest252-21systemd-journal10:45:001
330160899140,3cyclictest3646035-21tail12:35:251
330160899140,3cyclictest3573974-21uniq11:30:141
330160899140,3cyclictest3420986-21basename09:05:001
330160899140,3cyclictest3369435-21cpuspeed_turbos08:15:131
3301606991413,1cyclictest60350irq/132-enp0s29f1:rx-010:04:380
3301606991412,1cyclictest60350irq/132-enp0s29f1:rx-011:55:230
3301606991412,1cyclictest60350irq/132-enp0s29f1:rx-011:35:200
3301606991412,1cyclictest60350irq/132-enp0s29f1:rx-011:18:530
3301606991412,1cyclictest60350irq/132-enp0s29f1:rx-009:30:230
3301606991412,1cyclictest60350irq/132-enp0s29f1:rx-008:03:370
3301606991411,2cyclictest60350irq/132-enp0s29f1:rx-012:37:290
330161699139,3cyclictest3516337-21kworker/u8:4+flush-259:011:10:253
330161699139,3cyclictest3462633-21kworker/u8:3+events_unbound09:50:143
3301616991310,2cyclictest3462575-21unixbench_singl09:40:243
330161199139,3cyclictest3494954-21cp10:15:012
330161199139,3cyclictest252-21systemd-journal09:22:542
330161199138,4cyclictest509-21rs:main2
330161199131,3cyclictest3495617-21diskstats10:15:122
3301611991311,1cyclictest0-21swapper/210:55:012
3301611991311,1cyclictest0-21swapper/207:30:012
3301611991310,2cyclictest3589257-21tr11:45:122
3301611991310,2cyclictest3451463-21unixbench_singl09:30:242
330160899132,2cyclictest3426423-21apt-get09:10:121
330160899130,4cyclictest3615651-21grep12:10:131
330160899130,4cyclictest3386186-21grep08:30:171
3301606991311,1cyclictest60350irq/132-enp0s29f1:rx-009:00:220
3301606991310,2cyclictest60350irq/132-enp0s29f1:rx-010:10:160
330161699120,11cyclictest1-21systemd11:49:123
330161199129,2cyclictest3615467-21cpuspeed_turbos12:10:112
330161199129,2cyclictest3462160-21ntp_kernel_pll_09:40:202
330161199129,2cyclictest3386222-21kthreadcore08:30:172
330161199128,3cyclictest3536959-21kworker/u8:3+flush-259:010:55:152
330161199121,9cyclictest3646491-21awk12:40:002
3301611991210,1cyclictest0-21swapper/207:25:002
3301606991210,1cyclictest60350irq/132-enp0s29f1:rx-012:00:220
330161699118,2cyclictest0-21swapper/310:55:013
330161199118,2cyclictest3599181-21apt-get11:55:022
330161199118,2cyclictest3379558-21latency_hist08:25:022
330161199111,9cyclictest3636189-21awk12:30:002
330161199110,9cyclictest3415889-21cat09:00:002
330160699119,1cyclictest60350irq/132-enp0s29f1:rx-012:20:180
330160699117,3cyclictest3458142-21apt-get09:40:100
330161699107,2cyclictest3538221-21kthreadcore10:55:163
330161699105,4cyclictest3568360-21cat11:25:003
330161699104,5cyclictest3489677-21kworker/u8:3+events_unbound10:15:133
330161199107,2cyclictest3416163-21cpuspeed_turbos09:00:132
330161199107,2cyclictest0-21swapper/208:35:192
330161199105,4cyclictest3463280-21apt09:45:102
330161199100,9cyclictest3426394-21sshd09:10:102
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional