You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-05 - 21:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Fri Jun 05, 2026 12:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2910652995145,4cyclictest3061484-21latency_hist09:35:003
2910652994741,4cyclictest3124054-21ntpq10:30:213
2910644994641,3cyclictest3087359-21latency_hist10:00:001
2910647994440,3cyclictest3066848-21kworker/u8:4+flush-259:010:05:182
2910652994235,5cyclictest3171811-21apt-get11:20:013
2910647994235,4cyclictest3156356-21latency_hist11:05:012
2910647994138,2cyclictest2856792-21kworker/u8:1+events_unbound07:20:012
2910647993936,2cyclictest3161747-21kworker/u8:2+events_unbound11:15:242
2910644993832,4cyclictest3197258-21ntpq11:40:201
2910652993734,2cyclictest3035316-21kworker/u8:2+flush-259:009:10:143
2910652993734,2cyclictest2931755-21kworker/u8:3+flush-259:008:05:163
2910652993734,2cyclictest2931755-21kworker/u8:3+flush-259:008:05:163
2910647993633,2cyclictest3029990-21kworker/u8:3+flush-259:010:40:212
2910647993633,2cyclictest2931755-21kworker/u8:3+flush-259:007:40:152
2910652993530,3cyclictest3161747-21kworker/u8:2+flush-259:011:40:153
2910647993532,2cyclictest2972828-21kworker/u8:1+flush-259:009:15:262
2910647993531,3cyclictest3229199-21kworker/u8:3+flush-259:012:20:202
2910647993531,3cyclictest3029990-21kworker/u8:3+flush-259:009:55:202
2910647993531,3cyclictest2947198-21kworker/u8:4+events_unbound08:30:172
2910647993529,4cyclictest2983130-21latency_hist08:20:002
2910640993528,5cyclictest335557-21systemd-journal11:22:270
2910640993528,5cyclictest3055358-21ntpq09:25:220
2910652993429,3cyclictest3254381-21kworker/u8:1+events_unbound12:40:013
2910652993429,3cyclictest3254381-21kworker/u8:1+events_unbound12:40:003
2910647993431,2cyclictest2856792-21kworker/u8:1+flush-259:007:10:212
2910652993327,4cyclictest3239697-21diskstats12:25:133
2910652993327,4cyclictest2957784-21diskstats07:55:133
2910652993326,5cyclictest3248633-21ntpq12:30:193
2910647993330,2cyclictest2972828-21kworker/u8:1+events_unbound09:00:142
2910647993329,3cyclictest3161747-21kworker/u8:2+flush-259:011:10:212
2910647993329,3cyclictest3161747-21kworker/u8:2+events_unbound11:40:122
2910647993329,3cyclictest2856792-21kworker/u8:1+flush-259:007:25:142
2910644993328,4cyclictest3098234-21cp10:09:591
2910644993328,3cyclictest3076977-21latency_hist09:50:001
2910640993329,3cyclictest3161747-21kworker/u8:2+flush-259:011:45:010
2910640993328,3cyclictest3019928-21apt-get08:55:100
2910640993326,5cyclictest335557-21systemd-journal08:20:000
2910647993229,2cyclictest3082383-21kworker/u8:2+events_unbound10:30:012
2910647993229,2cyclictest2926008-21kworker/u8:2+flush-259:007:45:262
2910647993228,3cyclictest3208425-21kworker/u8:1+events_unbound12:10:162
2910647993128,2cyclictest3161747-21kworker/u8:2+events_unbound12:35:202
2910647993128,2cyclictest3161747-21kworker/u8:2+events_unbound12:35:202
2910647993127,3cyclictest3161747-21kworker/u8:2+flush-259:012:15:202
2910647993127,3cyclictest3082383-21kworker/u8:2+events_unbound10:20:152
2910647993127,3cyclictest2972828-21kworker/u8:1+events_unbound08:25:172
2910644993125,4cyclictest2930657-21ntpq07:25:221
2910652993026,3cyclictest2972828-21kworker/u8:1+flush-259:009:25:163
2910647993027,2cyclictest3172469-21kworker/u8:0+events_unbound11:25:172
2910647993027,2cyclictest3098784-21kworker/u8:1+events_unbound10:15:122
2910647993027,2cyclictest2972828-21kworker/u8:1+flush-259:009:25:192
2910647993027,2cyclictest2931755-21kworker/u8:3+events_unbound08:10:202
2910647993026,3cyclictest2972828-21kworker/u8:1+flush-259:008:50:212
2910640993025,4cyclictest3029990-21kworker/u8:3+flush-259:010:10:140
2910647992926,2cyclictest2972828-21kworker/u8:1+flush-259:008:20:212
2910647992925,3cyclictest3161747-21kworker/u8:2+flush-259:012:30:202
2910647992925,3cyclictest2947198-21kworker/u8:4+events_unbound08:45:162
2910647992923,3cyclictest3140429-21latency_hist10:50:012
2910644992926,2cyclictest3229199-21kworker/u8:3+flush-259:012:40:001
2910644992926,2cyclictest3229199-21kworker/u8:3+flush-259:012:40:001
2910644992925,3cyclictest3953151-21kworker/u8:0+flush-259:007:30:161
2910644992923,5cyclictest3061725-21/usr/sbin/munin09:35:141
2910640992924,3cyclictest3066848-21kworker/u8:4+flush-259:010:00:160
2910640992924,3cyclictest2972828-21kworker/u8:1+flush-259:008:35:270
2910652992822,4cyclictest2856792-21kworker/u8:1+flush-259:007:29:593
2910647992826,2cyclictest3066848-21kworker/u8:4+flush-259:010:00:222
2910647992825,2cyclictest3172469-21kworker/u8:0+flush-259:011:30:162
2910647992824,3cyclictest3119738-21kworker/u8:0+events_unbound10:35:212
2910647992824,3cyclictest2972828-21kworker/u8:1+events_unbound09:45:172
2910647992724,2cyclictest3035316-21kworker/u8:2+flush-259:009:10:162
2910647992724,2cyclictest2972828-21kworker/u8:1+flush-259:009:30:212
2910647992723,3cyclictest3172469-21kworker/u8:0+events_unbound11:50:132
2910647992723,3cyclictest3150703-21kworker/u8:0+events_unbound11:05:272
2910652992622,2cyclictest2921470-21apt-get07:20:123
2910647992623,2cyclictest2972828-21kworker/u8:1+flush-259:009:20:192
2910647992622,3cyclictest3161747-21kworker/u8:2+events_unbound11:45:122
2910647992618,5cyclictest3161747-21kworker/u8:2+flush-259:012:00:002
2910644992623,2cyclictest2972828-21kworker/u8:1+flush-259:008:40:011
2910644992620,5cyclictest2971913-21ntpq08:05:221
2910644992620,5cyclictest2971913-21ntpq08:05:211
2910640992622,3cyclictest3130301-21kworker/u8:1+flush-259:010:40:150
2910652992520,3cyclictest3030229-21apt-get09:05:113
2910652992519,4cyclictest335557-21systemd-journal12:10:003
2910644992521,3cyclictest2993714-21apt-get08:30:111
2910640992521,3cyclictest3150703-21kworker/u8:0+flush-259:011:05:250
2910640992520,4cyclictest3087469-21gpgv10:00:000
2910652992420,3cyclictest2972828-21kworker/u8:1+events_unbound09:50:163
2910652992419,3cyclictest3953151-21kworker/u8:0+flush-259:007:30:193
2910647992421,2cyclictest3953151-21kworker/u8:0+events_unbound07:30:192
2910647992421,2cyclictest3208425-21kworker/u8:1+flush-259:012:00:172
2910647992421,2cyclictest2926008-21kworker/u8:2+events_unbound07:50:142
2910647992420,3cyclictest2972828-21kworker/u8:1+events_unbound09:05:122
2910647992416,6cyclictest335557-21systemd-journal10:55:002
2910644992420,2cyclictest3077211-21/usr/sbin/munin09:50:121
2910640992419,4cyclictest3151444-21apt-get11:00:120
2910652992320,2cyclictest3161747-21kworker/u8:2+events_unbound11:10:183
2910652992320,2cyclictest2926008-21kworker/u8:2+flush-259:007:45:203
2910652992320,2cyclictest2926008-21kworker/u8:2+events_unbound07:50:283
2910652992319,3cyclictest3140689-21apt-get10:50:123
2910652992318,3cyclictest2999278-21/usr/sbin/munin08:35:253
2910652992316,5cyclictest3160636-21ntpq11:05:213
2910647992320,2cyclictest3208425-21kworker/u8:1+events_unbound12:05:162
2910647992320,2cyclictest2947198-21kworker/u8:4+events_unbound08:40:252
2910647992319,3cyclictest3953151-21kworker/u8:0+flush-259:007:20:272
2910647992319,3cyclictest2931755-21kworker/u8:3+events_unbound07:55:152
2910647992317,4cyclictest2972798-21latency_hist08:10:012
2910647992317,4cyclictest2972798-21latency_hist08:10:002
2910640992317,4cyclictest3035507-21diskstats09:10:120
2910652992217,3cyclictest3161747-21kworker/u8:2+flush-259:011:25:003
2910640992219,2cyclictest3140654-21kworker/u8:2+flush-259:010:50:250
2910640992217,4cyclictest3098784-21kworker/u8:1+flush-259:010:15:250
2910640992215,5cyclictest3170987-21ntpq11:15:220
2910652992118,2cyclictest3082383-21kworker/u8:2+events_unbound10:20:143
2910652992117,3cyclictest0-21swapper/311:05:003
2910652992114,5cyclictest3040685-21diskstats09:15:163
2910647992118,2cyclictest3156381-21kworker/u8:3+events_unbound11:20:202
2910644992118,2cyclictest3009390-21kworker/u8:0+flush-259:008:50:011
2910644992118,2cyclictest2926008-21kworker/u8:2+flush-259:007:50:171
2910644992117,3cyclictest2926008-21kworker/u8:2+flush-259:007:35:261
2910644992117,2cyclictest1-21systemd09:15:011
2910644992116,4cyclictest335557-21systemd-journal12:00:271
2910644992115,5cyclictest3118843-21ntpq10:25:211
2910640992116,3cyclictest335557-21systemd-journal07:51:280
2910640992115,4cyclictest3233268-21ntpq12:15:190
2910652992017,2cyclictest3161747-21kworker/u8:2+events_unbound12:19:103
2910652992017,2cyclictest3066848-21kworker/u8:4+flush-259:009:40:193
2910652992016,3cyclictest509-21rs:main3
2910652992016,3cyclictest2972828-21kworker/u8:1+events_unbound09:45:233
2910652992015,4cyclictest3187795-21latency_hist11:35:003
2910652992015,4cyclictest2931755-21kworker/u8:3+events_unbound08:00:143
2910652992015,3cyclictest3098784-21kworker/u8:1+events_unbound10:15:223
2910647992017,2cyclictest3029990-21kworker/u8:3+flush-259:009:35:202
2910647992016,3cyclictest3119738-21kworker/u8:0+events_unbound10:30:232
2910640992016,3cyclictest2983392-21apt-get08:20:120
2910640992015,4cyclictest335557-21systemd-journal08:28:260
2910640992013,5cyclictest2936694-21latency_hist07:35:000
2910652991916,2cyclictest3029990-21kworker/u8:3+events_unbound10:45:143
2910652991916,2cyclictest2947198-21kworker/u8:4+events_unbound08:30:253
2910652991915,2cyclictest509-21rs:main3
2910647991915,3cyclictest3029990-21kworker/u8:3+events_unbound10:12:282
2910647991915,3cyclictest3009390-21kworker/u8:0+events_unbound08:55:132
2910647991915,2cyclictest2910750-21latency_hist07:10:022
2910644991916,2cyclictest2972828-21kworker/u8:1+flush-259:008:25:251
2910644991915,2cyclictest3177457-21grep11:25:001
2910644991914,4cyclictest2911070-21/usr/sbin/munin07:10:141
2910640991914,4cyclictest2910922-21cat07:10:020
2910640991912,5cyclictest335557-21systemd-journal12:35:000
2910652991815,2cyclictest3178134-21irqcore11:25:163
2910652991815,2cyclictest3172469-21kworker/u8:0+events_unbound11:50:233
2910652991814,3cyclictest2963139-21kworker/u8:0+events_unbound08:25:153
2910652991811,5cyclictest2978387-21diskstats08:15:123
2910647991815,2cyclictest2963139-21kworker/u8:0+events_unbound08:00:542
2910647991814,3cyclictest2926008-21kworker/u8:2+events_unbound07:37:282
2910644991816,1cyclictest0-21swapper/112:20:001
2910644991814,3cyclictest3161747-21kworker/u8:2+flush-259:011:58:321
2910644991813,3cyclictest3014788-21apt-get08:50:111
2910640991815,2cyclictest2926008-21kworker/u8:2+flush-259:007:45:170
2910640991814,3cyclictest509-21rs:main0
2910652991714,2cyclictest3130073-21sh10:40:003
2910652991714,2cyclictest3029990-21kworker/u8:3+events_unbound10:10:123
2910652991714,2cyclictest2972828-21kworker/u8:1+events_unbound08:43:283
2910652991710,5cyclictest2941878-21latency_hist07:40:013
2910647991714,2cyclictest3140654-21kworker/u8:2+events_unbound10:55:292
2910647991713,3cyclictest2972828-21kworker/u8:1+events_unbound09:54:222
2910644991713,3cyclictest3161747-21kworker/u8:2+flush-259:011:45:241
2910640991714,2cyclictest60350irq/132-enp0s29f1:rx-011:30:210
2910640991714,2cyclictest60350irq/132-enp0s29f1:rx-009:04:120
2910640991714,2cyclictest3229199-21kworker/u8:3+flush-259:012:20:230
2910640991714,2cyclictest3004442-21apt-get08:40:110
2910640991713,3cyclictest2972828-21kworker/u8:1+flush-259:009:15:270
2910640991712,3cyclictest3192035-21ntpq11:35:210
2910652991613,2cyclictest3209668-21kthreadcore11:55:163
2910652991613,2cyclictest3161747-21kworker/u8:2+events_unbound11:45:143
2910652991613,2cyclictest3087388-21apt-config09:59:593
2910652991612,3cyclictest3172469-21kworker/u8:0+flush-259:011:35:233
291064799161,4cyclictest3071827-21munin-run09:45:002
2910647991613,2cyclictest2972828-21kworker/u8:1+events_unbound08:35:182
2910647991612,3cyclictest3172469-21kworker/u8:0+events_unbound11:39:012
2910644991614,1cyclictest3160617-21ntp_kernel_pll_11:05:211
2910644991613,2cyclictest500-21systemd-logind08:25:011
2910644991613,2cyclictest3192061-21ntp_kernel_pll_11:35:211
2910644991613,2cyclictest3172469-21kworker/u8:0+events_unbound11:30:191
2910644991613,2cyclictest3029990-21kworker/u8:3+events_unbound09:20:451
2910644991613,2cyclictest3009390-21kworker/u8:0+flush-259:008:55:151
2910644991611,4cyclictest335557-21systemd-journal10:45:001
2910640991614,1cyclictest60350irq/132-enp0s29f1:rx-011:10:160
2910640991613,2cyclictest60350irq/132-enp0s29f1:rx-010:20:210
2910640991613,2cyclictest60350irq/132-enp0s29f1:rx-007:40:000
2910640991612,3cyclictest335557-21systemd-journal10:30:000
2910640991612,3cyclictest3009390-21kworker/u8:0+flush-259:009:05:150
2910640991610,5cyclictest3222981-21ntpq12:05:210
2910652991513,1cyclictest0-21swapper/310:30:003
2910652991512,2cyclictest3015211-21irqcore08:50:163
2910652991512,2cyclictest2931755-21kworker/u8:3+events_unbound07:40:213
2910644991513,1cyclictest0-21swapper/112:30:001
2910644991511,3cyclictest335557-21systemd-journal07:21:261
2910644991511,3cyclictest2973064-21apt-get08:10:101
2910644991511,3cyclictest2931755-21kworker/u8:3+flush-259:007:55:251
291064499150,4cyclictest30418231kthreadcore09:15:191
2910640991513,2cyclictest61150irq/133-enp0s29f1:tx-007:20:120
2910640991513,2cyclictest60350irq/132-enp0s29f1:rx-010:45:110
2910640991513,1cyclictest60350irq/132-enp0s29f1:rx-011:55:220
2910640991513,1cyclictest60350irq/132-enp0s29f1:rx-007:59:190
2910640991512,1cyclictest0-21swapper/012:05:000
291065299149,4cyclictest3140654-21kworker/u8:2+events_unbound10:57:283
291065299149,4cyclictest3009390-21kworker/u8:0+flush-259:008:45:263
291065299149,4cyclictest2963139-21kworker/u8:0+flush-259:008:13:493
2910652991410,3cyclictest3208425-21kworker/u8:1+events_unbound12:00:233
2910644991410,3cyclictest335557-21systemd-journal11:54:021
2910640991413,1cyclictest60350irq/132-enp0s29f1:rx-012:28:180
2910640991413,1cyclictest60350irq/132-enp0s29f1:rx-009:53:440
2910640991413,1cyclictest60350irq/132-enp0s29f1:rx-009:47:430
2910640991412,1cyclictest60350irq/132-enp0s29f1:rx-010:07:140
2910640991412,1cyclictest60350irq/132-enp0s29f1:rx-008:50:190
2910640991412,1cyclictest60350irq/132-enp0s29f1:rx-008:45:250
2910640991412,1cyclictest60350irq/132-enp0s29f1:rx-008:30:240
2910640991412,1cyclictest60350irq/132-enp0s29f1:rx-008:10:230
2910640991411,2cyclictest60350irq/132-enp0s29f1:rx-011:50:240
2910640991411,2cyclictest60350irq/132-enp0s29f1:rx-011:25:120
2910640991411,2cyclictest60350irq/132-enp0s29f1:rx-008:00:100
2910640991410,3cyclictest3066857-21/usr/sbin/munin09:40:190
291065299138,4cyclictest3229199-21kworker/u8:3+events_unbound12:20:153
2910652991310,2cyclictest2856792-21kworker/u8:1+events_unbound07:15:213
291064499139,3cyclictest3029990-21kworker/u8:3+flush-259:010:50:171
291064499137,4cyclictest335557-21systemd-journal10:15:001
2910644991311,1cyclictest0-21swapper/110:40:001
291064499130,2cyclictest2971-21cron12:35:001
2910640991312,1cyclictest60350irq/132-enp0s29f1:rx-011:46:570
2910640991312,1cyclictest60350irq/132-enp0s29f1:rx-010:30:490
2910640991312,1cyclictest60350irq/132-enp0s29f1:rx-009:35:220
2910640991312,1cyclictest60350irq/132-enp0s29f1:rx-008:06:330
2910640991312,1cyclictest60350irq/132-enp0s29f1:rx-008:06:330
2910640991311,2cyclictest60350irq/132-enp0s29f1:rx-007:18:010
2910640991311,1cyclictest60350irq/132-enp0s29f1:rx-009:30:170
2910640991311,1cyclictest60350irq/132-enp0s29f1:rx-007:40:240
2910640991310,2cyclictest3050707-21apt-key09:25:000
291065299128,3cyclictest3087353-21apt-get10:00:023
2910652991210,1cyclictest3098481-21latency_hist10:10:013
291064499129,2cyclictest3223386-21unixbench_singl12:05:241
291064499129,2cyclictest3167025-21cpuspeed_turbos11:15:131
291064499129,2cyclictest3056255-21idleruntime-cro09:30:011
291064499127,3cyclictest3104417-21apt-get10:15:101
2910644991210,1cyclictest0-21swapper/109:35:001
291064099129,2cyclictest60350irq/132-enp0s29f1:rx-010:55:170
2910640991211,1cyclictest60350irq/132-enp0s29f1:rx-012:10:490
291065299119,1cyclictest3050613-21apt-get09:25:013
291065299118,2cyclictest3208425-21kworker/u8:1+events_unbound12:10:123
291065299118,1cyclictest0-21swapper/307:10:023
291065299117,3cyclictest3953151-21kworker/u8:0+events_unbound07:12:283
291064499118,2cyclictest3161846-21cpuspeed_turbos11:10:121
291064499118,1cyclictest0-21swapper/110:20:161
291064499117,3cyclictest3071823-21turbostat.cron09:45:001
291064499117,2cyclictest2952167-21idleruntime-cro07:49:591
291064499116,4cyclictest3066848-21kworker/u8:4+events_unbound10:00:171
291064099118,2cyclictest60350irq/132-enp0s29f1:rx-007:25:230
291065299107,2cyclictest3025206-21cpuspeed_turbos09:00:103
291064499105,4cyclictest3119978-21apt-get10:30:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional