You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-18 - 18:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Thu Jun 18, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2724936994641,4cyclictest2947699-21kworker/u8:3+flush-259:012:35:000
2724936994540,3cyclictest2989713-21apt-get11:25:100
2724936994538,5cyclictest2911236-21latency_hist10:10:010
2724936994338,3cyclictest3057586-21kworker/u8:2+flush-259:012:40:000
2724940994235,4cyclictest2797493-21latency_hist08:20:001
2724940993831,5cyclictest3015667-21ntpq11:45:211
2724936993834,3cyclictest3021975-21apt-get11:55:100
2724949993631,3cyclictest2777143-21apt-get08:00:113
2724936993528,5cyclictest2782057-21latency_hist08:05:000
2724940993430,2cyclictest3032118-21kworker/u8:0+flush-259:012:15:141
2724936993429,3cyclictest2719990-21kworker/u8:2+flush-259:007:20:010
2724940993328,3cyclictest335557-21systemd-journal08:59:051
2724936993329,3cyclictest2761618-21apt-get07:45:110
2724940993227,3cyclictest2854817-21kworker/u8:6+flush-259:009:40:171
2724940993226,4cyclictest2771935-21/usr/sbin/munin07:55:151
2724936993224,6cyclictest3046761-21ntpq12:15:230
2724940993026,3cyclictest2854279-21gpgv09:15:001
2724936993027,2cyclictest2761377-21kworker/u8:2+flush-259:008:05:240
2724936993026,2cyclictest2906305-21kworker/u8:1+flush-259:010:55:200
2724949992921,6cyclictest335557-21systemd-journal10:20:003
2724936992926,2cyclictest2947699-21kworker/u8:3+events_unbound12:15:010
2724949992821,5cyclictest2974346-21diskstats11:10:143
2724949992817,9cyclictest3021715-21latency_hist11:55:013
2724936992824,3cyclictest2916603-21kworker/u8:0+flush-259:010:40:220
2724945992722,3cyclictest2724894-21kworker/u8:4+flush-259:007:20:152
2724940992722,4cyclictest2750659-21gpgv07:35:011
2724940992722,3cyclictest2875437-21apt-get09:35:111
2724936992723,3cyclictest2838682-21kworker/u8:4+flush-259:009:25:220
2724936992723,3cyclictest2724894-21kworker/u8:4+flush-259:007:40:010
2724936992721,4cyclictest3011133-21cat11:45:000
2724940992621,3cyclictest2927343-21/usr/sbin/munin10:25:161
2724936992522,2cyclictest3032118-21kworker/u8:0+events_unbound12:05:130
2724936992520,4cyclictest2932158-21kworker/u8:2+flush-259:010:40:000
2724949992420,3cyclictest2932158-21kworker/u8:2+flush-259:010:40:183
2724949992420,3cyclictest2735219-21kworker/u8:0+flush-259:007:25:013
2724949992420,2cyclictest0-21swapper/311:40:013
2724945992418,4cyclictest2813374-21diskstats08:35:132
2724936992421,2cyclictest211rcuc/008:55:000
2724936992420,3cyclictest2766517-21kworker/u8:3+events_unbound08:25:010
2724949992318,3cyclictest2979367-21apt-get11:15:103
2724949992317,4cyclictest2740598-21apt-get07:25:133
2724945992319,3cyclictest2719990-21kworker/u8:2+flush-259:007:15:162
2724945992317,4cyclictest335557-21systemd-journal12:35:302
2724940992317,4cyclictest335557-21systemd-journal09:05:151
2724940992317,4cyclictest2890907-21/usr/sbin/munin09:50:141
2724940992317,4cyclictest2719990-21kworker/u8:2+flush-259:007:24:441
2724936992320,2cyclictest2807827-21kworker/u8:1+flush-259:008:45:000
2724936992319,3cyclictest2968372-21kworker/u8:0+events_unbound11:30:280
2724936992319,3cyclictest2947699-21kworker/u8:3+events_unbound11:50:280
2724936992319,3cyclictest2906305-21kworker/u8:1+events_unbound11:03:110
2724936992318,3cyclictest1-21systemd07:10:470
2724949992218,3cyclictest3032118-21kworker/u8:0+flush-259:012:10:143
2724949992212,9cyclictest2973709-21latency_hist11:10:003
2724945992218,3cyclictest2838682-21kworker/u8:4+events_unbound09:25:202
2724940992220,1cyclictest0-21swapper/110:45:001
2724940992218,3cyclictest2766517-21kworker/u8:3+flush-259:007:55:001
2724940992217,3cyclictest2994876-21apt-get11:30:121
2724940992216,4cyclictest3006020-21/usr/sbin/munin11:40:131
2724936992218,3cyclictest2947699-21kworker/u8:3+events_unbound12:00:180
2724936992218,3cyclictest2922228-21apt-get10:20:110
2724936992218,3cyclictest2875839-21kworker/u8:2+flush-259:009:40:220
272494999210,19cyclictest2833059-21grep08:50:243
2724945992117,3cyclictest3047831-21kworker/u8:1+flush-259:012:25:162
2724945992117,3cyclictest2766517-21kworker/u8:3+flush-259:008:15:152
2724945992115,4cyclictest2828620-21/usr/sbin/munin08:50:122
2724940992115,4cyclictest3041524-21ntpq12:10:211
2724936992118,2cyclictest2932158-21kworker/u8:2+flush-259:010:50:230
2724936992117,3cyclictest2719990-21kworker/u8:2+events_unbound07:20:490
2724949992014,4cyclictest2813213-21apt-get08:35:103
2724945992016,3cyclictest2766517-21kworker/u8:3+flush-259:010:00:252
2724945992015,3cyclictest2751253-21cat07:35:012
2724945992014,4cyclictest3016769-21/usr/sbin/munin11:50:212
2724940992016,3cyclictest3058166-21apt-get12:30:111
2724940992015,4cyclictest2870276-21apt-get09:30:111
2724940992015,3cyclictest2968596-21/usr/sbin/munin11:05:241
2724940992013,5cyclictest3067311-21ntpq12:35:181
2724936992017,2cyclictest2968372-21kworker/u8:0+events_unbound11:20:010
2724936992016,3cyclictest2766517-21kworker/u8:3+events_unbound08:37:050
2724936992016,3cyclictest2724894-21kworker/u8:4+events_unbound07:44:520
2724936992016,3cyclictest2724894-21kworker/u8:4+events_unbound07:30:210
2724936992015,3cyclictest2916370-21latency_hist10:15:010
2724949991915,3cyclictest2838682-21kworker/u8:4+flush-259:009:20:003
2724949991914,4cyclictest335557-21systemd-journal12:00:433
2724949991910,7cyclictest0-21swapper/311:05:003
272494999190,3cyclictest2854168-21latency_hist09:15:013
2724945991912,5cyclictest2818344-21/usr/sbin/munin08:40:132
2724940991914,4cyclictest335557-21systemd-journal11:40:011
2724940991914,4cyclictest335557-21systemd-journal10:55:011
2724940991913,5cyclictest2761377-21kworker/u8:2+flush-259:008:05:161
2724940991913,4cyclictest335557-21systemd-journal10:05:241
2724936991916,2cyclictest2854817-21kworker/u8:6+events_unbound09:40:010
2724936991916,2cyclictest2766517-21kworker/u8:3+flush-259:008:15:150
2724936991916,2cyclictest0-21swapper/012:30:000
2724936991915,3cyclictest2854817-21kworker/u8:6+events_unbound09:47:060
2724936991915,3cyclictest2761377-21kworker/u8:2+events_unbound07:50:230
2724949991812,4cyclictest1-21systemd11:42:103
272494999180,4cyclictest3037263-21munin-plugin-st12:10:013
2724945991814,3cyclictest2932158-21kworker/u8:2+events_unbound10:50:202
2724945991812,4cyclictest3026891-21latency_hist12:00:002
2724940991815,2cyclictest2855881-21kthreadcore09:15:181
2724940991814,3cyclictest3047831-21kworker/u8:1+events_unbound12:20:151
2724940991814,2cyclictest2829072-21irqcore08:50:141
2724936991815,2cyclictest2947699-21kworker/u8:3+events_unbound11:10:210
2724936991815,2cyclictest2947699-21kworker/u8:3+events_unbound11:05:100
2724936991815,2cyclictest2766517-21kworker/u8:3+flush-259:009:15:240
2724936991815,2cyclictest2766517-21kworker/u8:3+events_unbound08:34:530
2724936991815,2cyclictest2766517-21kworker/u8:3+events_unbound08:34:520
2724936991814,3cyclictest60350irq/132-enp0s29f1:rx-011:45:210
2724936991814,3cyclictest2916603-21kworker/u8:0+events_unbound10:29:080
2724936991814,3cyclictest2854817-21kworker/u8:6+events_unbound09:20:190
2724936991814,3cyclictest2838131-21kworker/u8:0+events_unbound09:10:000
2724936991814,3cyclictest2735219-21kworker/u8:0+flush-259:008:15:000
272494999178,8cyclictest3058232-21cpuspeed_turbos12:30:123
272494999177,9cyclictest0-21swapper/311:45:243
272494999174,4cyclictest2845791-21cut09:05:173
272494999171,5cyclictest2843811-21latency_hist09:05:003
2724949991713,3cyclictest3016755-21kworker/u8:4+flush-259:011:55:153
2724945991714,2cyclictest2838682-21kworker/u8:4+events_unbound09:15:222
2724945991713,3cyclictest2932158-21kworker/u8:2+flush-259:010:40:202
2724945991713,3cyclictest2906305-21kworker/u8:1+flush-259:011:00:142
2724945991713,3cyclictest2906305-21kworker/u8:1+flush-259:010:55:252
2724945991713,3cyclictest2838682-21kworker/u8:4+flush-259:010:25:242
2724945991713,3cyclictest2745483-21idleruntime-cro07:30:002
2724945991713,2cyclictest1-21systemd09:35:012
272494599170,15cyclictest2776888-21munin-plugin-st08:00:002
2724940991714,2cyclictest2724894-21kworker/u8:4+flush-259:007:25:251
2724940991713,3cyclictest2947699-21kworker/u8:3+flush-259:011:10:151
2724940991711,4cyclictest2808027-21/usr/sbin/munin08:30:121
2724940991711,4cyclictest2808027-21/usr/sbin/munin08:30:121
2724936991714,2cyclictest2838682-21kworker/u8:4+flush-259:009:55:060
2724936991713,3cyclictest2968372-21kworker/u8:0+events_unbound11:35:010
2724936991713,3cyclictest2838131-21kworker/u8:0+events_unbound09:10:180
2724936991713,2cyclictest0-21swapper/008:25:010
272494999166,9cyclictest0-21swapper/307:30:243
272494999165,1cyclictest0-21swapper/308:30:163
272494999165,1cyclictest0-21swapper/308:30:163
272494999164,2cyclictest2827281-21wc08:45:173
272494999161,5cyclictest2735188-21awk07:20:003
272494999161,4cyclictest2911919-21cut10:10:153
272494999161,4cyclictest2787159-21cron08:10:003
272494999161,4cyclictest2766478-21cron07:50:013
272494999161,2cyclictest2958060-21idleruntime-cro10:54:593
272494999161,2cyclictest2927084-21cron10:24:593
272494999161,2cyclictest2822327-21ntpq08:40:203
2724949991612,3cyclictest2874277-21ntp_kernel_pll_09:30:203
2724949991610,4cyclictest2864864-21latency_hist09:25:023
272494999160,6cyclictest2760445-21ntpq07:40:213
2724945991610,4cyclictest2896093-21apt-get09:55:112
2724945991610,4cyclictest2750637-21apt-get07:35:002
2724940991613,2cyclictest2917665-21kthreadcore10:15:161
2724940991610,4cyclictest3036388-21ntpq12:05:211
2724936991613,2cyclictest60350irq/132-enp0s29f1:rx-008:00:000
2724936991613,2cyclictest2766517-21kworker/u8:3+events_unbound09:52:060
2724936991612,3cyclictest2807827-21kworker/u8:1+events_unbound08:55:190
2724936991611,4cyclictest3047866-21apt-get12:20:090
2724936991611,3cyclictest2932480-21/usr/sbin/munin10:30:120
272494999154,3cyclictest2770536-21tr07:50:193
272494999153,3cyclictest2868519-21cut09:25:173
272494999153,2cyclictest2802987-21grep08:25:123
272494999152,2cyclictest2951663-21sshd10:45:193
272494999151,4cyclictest3057079-21ntpq12:25:223
272494999151,4cyclictest2900427-21unixbench-2d09:55:253
272494999151,3cyclictest2886171-21cut09:45:153
272494999151,2cyclictest335557-21systemd-journal07:15:003
272494999151,2cyclictest3051907-21ntpq12:20:203
272494999151,2cyclictest0-21swapper/308:15:003
272494999151,2cyclictest0-21swapper/308:00:003
2724949991512,2cyclictest0-21swapper/309:00:003
272494999150,4cyclictest3064603-21grep12:35:153
272494999150,4cyclictest2934818-21ps10:30:173
272494999150,4cyclictest2905007-21/usr/sbin/munin10:00:193
272494999150,4cyclictest2801655-21grep08:20:183
272494999150,3cyclictest335557-21systemd-journal09:40:203
272494999150,3cyclictest2998928-21ntpq11:30:213
272494999150,3cyclictest2984920-21grep11:20:153
272494999150,3cyclictest2927508-21taskset10:25:133
272494999150,3cyclictest2895806-21cron09:55:003
272494999150,2cyclictest2994609-21turbostat.cron11:30:013
272494599159,4cyclictest2895820-21latency_hist09:55:012
2724945991512,2cyclictest2906305-21kworker/u8:1+events_unbound10:30:162
2724945991512,2cyclictest2766517-21kworker/u8:3+events_unbound09:35:202
2724945991511,3cyclictest2724894-21kworker/u8:4+events_unbound07:40:222
272494099159,5cyclictest2921541-21apt-get10:20:001
272494099159,4cyclictest335557-21systemd-journal12:05:011
272494099159,4cyclictest3025998-21ntpq11:55:181
2724940991512,2cyclictest2983809-21unixbench_singl11:15:241
2724940991512,2cyclictest2912525-21kthreadcore10:10:171
2724940991510,4cyclictest2947920-21/usr/sbin/munin10:45:141
2724936991513,2cyclictest60350irq/132-enp0s29f1:rx-009:30:090
2724936991513,1cyclictest60350irq/132-enp0s29f1:rx-008:45:140
2724936991512,2cyclictest60350irq/132-enp0s29f1:rx-007:25:220
2724936991512,2cyclictest2842933-21ntp_kernel_pll_09:00:190
2724936991512,2cyclictest2766517-21kworker/u8:3+events_unbound10:15:210
2724936991511,3cyclictest2916603-21kworker/u8:0+events_unbound10:45:200
272494999141,3cyclictest513-21gdbus09:40:003
272494999140,3cyclictest3042804-21grep12:15:143
272494999140,3cyclictest2937815-21cpuspeed_turbos10:35:123
272494999140,3cyclictest2755219-21grep07:35:203
272494999140,2cyclictest2962232-21needreboot10:55:203
2724945991411,2cyclictest3010981-21kworker/u8:2+events_unbound11:45:222
2724945991410,3cyclictest2994629-21kworker/u8:1+flush-259:011:35:232
2724945991410,3cyclictest2947699-21kworker/u8:3+flush-259:011:10:142
2724945991410,3cyclictest2885478-21idleruntime-cro09:45:002
2724945991410,3cyclictest2724894-21kworker/u8:4+flush-259:007:49:542
272494099147,5cyclictest2994603-21latency_hist11:30:001
272494099147,5cyclictest2755353-21ntpq07:35:221
2724940991411,2cyclictest2869115-21ntp_kernel_pll_09:25:191
2724940991410,3cyclictest2766517-21kworker/u8:3+events_unbound09:00:211
2724936991411,2cyclictest60350irq/132-enp0s29f1:rx-010:00:120
272494599139,3cyclictest2947699-21kworker/u8:3+events_unbound11:15:222
272494599139,3cyclictest2735219-21kworker/u8:0+flush-259:008:25:182
272494599139,3cyclictest2714682-21kworker/u8:1+flush-259:007:10:152
272494599139,2cyclictest3032118-21kworker/u8:0+flush-259:012:05:062
272494599139,2cyclictest0-21swapper/212:35:002
272494599138,4cyclictest2968372-21kworker/u8:0+flush-259:011:25:152
272494599136,5cyclictest2828378-21latency_hist08:50:002
2724945991310,2cyclictest2838987-21cpuspeed_turbos09:00:102
2724945991310,2cyclictest2801768-21ntp_kernel_pll_08:20:202
2724945991310,2cyclictest2766517-21kworker/u8:3+flush-259:009:05:202
2724945991310,1cyclictest3000106-21awk11:35:002
2724945991310,1cyclictest0-21swapper/209:25:002
272494099139,3cyclictest2730314-21/usr/sbin/munin07:15:141
2724940991310,2cyclictest2863914-21needreboot09:20:191
272494599129,2cyclictest3032061-21cron12:05:012
272494599129,1cyclictest0-21swapper/212:10:182
272494599129,1cyclictest0-21swapper/210:45:162
272494599128,3cyclictest2787416-21/usr/sbin/munin08:10:152
272494099129,2cyclictest3017207-21irqcore11:50:181
272494099129,2cyclictest2757477-21kthreadcore07:40:181
272494099128,3cyclictest2906305-21kworker/u8:1+flush-259:010:30:231
272494099128,3cyclictest2735219-21kworker/u8:0+flush-259:008:30:001
272494099121,9cyclictest2900943-21awk10:00:011
2724940991210,1cyclictest0-21swapper/110:35:131
2724940991210,1cyclictest0-21swapper/108:15:011
2724936991210,1cyclictest60350irq/132-enp0s29f1:rx-011:20:180
272494599118,2cyclictest2834804-21kthreadcore08:55:162
272494099115,4cyclictest2885733-21/usr/sbin/munin09:45:181
272494599100,2cyclictest2921542-21latency_hist10:20:012
272494099107,1cyclictest0-21swapper/111:20:201
272494099106,3cyclictest2766517-21kworker/u8:3+flush-259:010:00:241
27249459997,1cyclictest0-21swapper/211:05:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional