You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-18 - 17:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Fri Sep 18, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2724959996561,3cyclictest2860174-21kworker/u8:1+flush-259:009:24:171
2724957996359,3cyclictest75-21kswapd009:24:010
2724962996057,2cyclictest75-21kswapd008:05:592
2724967995251,1cyclictest2860186-21kworker/u8:4+events_unbound09:24:093
2724967995042,6cyclictest2970075-21cp11:05:003
2724959994642,2cyclictest2829184-21kworker/u8:0+events_unbound09:40:191
2724959994541,3cyclictest2772891-21kworker/u8:1+flush-259:008:00:251
2724967994439,4cyclictest2913059-21latency_hist10:10:013
2724959994338,3cyclictest2692572-21kworker/u8:1+flush-259:007:25:161
2724967994239,2cyclictest2928618-21kworker/u8:2+events_unbound10:28:043
2724959994239,2cyclictest3010743-21kworker/u8:3+flush-259:011:45:241
2724967994138,2cyclictest3016596-21kworker/u8:0+events_unbound11:52:203
2724967994138,2cyclictest3016596-21kworker/u8:0+events_unbound11:52:203
2724967994138,2cyclictest2829184-21kworker/u8:0+events_unbound09:54:033
2724962994136,3cyclictest3027558-21apt-get12:00:112
2724967994037,2cyclictest2829184-21kworker/u8:0+events_unbound09:47:043
2724967994037,2cyclictest2714522-21kworker/u8:3+events_unbound07:24:593
2724967993937,2cyclictest3022054-21kworker/u8:2+events_unbound12:22:203
2724967993937,2cyclictest2740661-21kworker/u8:2+events_unbound08:35:263
2724967993936,2cyclictest3032146-21kworker/u8:1+events_unbound12:19:213
2724967993936,2cyclictest2692572-21kworker/u8:1+events_unbound07:49:083
2724967993933,4cyclictest2876647-21cp09:35:013
2724962993831,4cyclictest2755944-21latency_hist07:40:002
2724967993735,2cyclictest2692572-21kworker/u8:1+events_unbound07:36:583
2724967993734,2cyclictest3001445-21kworker/u8:0+events_unbound11:35:133
2724967993734,2cyclictest2740661-21kworker/u8:2+events_unbound08:15:233
2724967993634,2cyclictest2772891-21kworker/u8:1+events_unbound08:02:593
2724967993634,2cyclictest2755959-21kworker/u8:4+events_unbound07:50:593
2724967993633,2cyclictest2802935-21kworker/u8:3+events_unbound09:17:043
2724962993630,4cyclictest2949304-21kworker/u8:3+flush-259:010:55:002
2724959993631,3cyclictest2755959-21kworker/u8:4+events_unbound07:45:181
2724957993630,4cyclictest2725148-21/usr/sbin/munin07:10:140
2724967993533,2cyclictest3016596-21kworker/u8:0+events_unbound12:28:213
2724967993531,2cyclictest2740661-21kworker/u8:2+events_unbound09:25:203
2724962993528,5cyclictest2938061-21ntpq10:30:202
2724962993528,5cyclictest2772891-21kworker/u8:1+flush-259:008:50:162
2724959993532,2cyclictest2897754-21kworker/u8:3+flush-259:010:15:241
2724959993528,5cyclictest2881619-21latency_hist09:40:011
2724957993530,3cyclictest2881792-21cat09:40:010
2724967993433,1cyclictest2944010-21kworker/u8:1+events_unbound11:09:033
2724967993432,2cyclictest3016596-21kworker/u8:0+events_unbound12:02:183
2724967993432,2cyclictest2975126-21kworker/u8:2+events_unbound11:14:533
2724967993432,2cyclictest2949304-21kworker/u8:3+events_unbound10:51:023
2724967993432,2cyclictest2944010-21kworker/u8:1+events_unbound11:30:233
2724967993432,2cyclictest2714522-21kworker/u8:3+events_unbound07:25:583
2724967993431,2cyclictest3010743-21kworker/u8:3+events_unbound11:59:573
2724967993429,3cyclictest2934040-21apt-get10:30:103
2724959993431,2cyclictest661596-21kworker/u8:4+flush-259:007:10:211
2724959993431,2cyclictest3010743-21kworker/u8:3+flush-259:011:40:241
2724959993431,2cyclictest2766317-21kworker/u8:0+events_unbound07:50:141
2724959993430,3cyclictest2772891-21kworker/u8:1+flush-259:008:35:211
2724967993331,2cyclictest2772891-21kworker/u8:1+events_unbound07:55:223
2724967993330,2cyclictest2980584-21kworker/u8:0+events_unbound11:15:083
2724962993328,4cyclictest2740661-21kworker/u8:2+flush-259:008:30:152
2724962993328,3cyclictest2829184-21kworker/u8:0+flush-259:009:19:592
2724959993330,2cyclictest2897545-21kworker/u8:1+events_unbound10:10:211
2724967993229,2cyclictest2829184-21kworker/u8:0+events_unbound09:07:023
2724967993228,3cyclictest2991120-21latency_hist11:25:003
2724959993229,2cyclictest3010743-21kworker/u8:3+flush-259:011:50:201
2724959993229,2cyclictest3010743-21kworker/u8:3+flush-259:011:50:201
2724957993226,4cyclictest2949280-21latency_hist10:45:010
2724967993128,2cyclictest2802935-21kworker/u8:3+events_unbound09:10:233
2724967993125,5cyclictest2944112-21latency_hist10:40:013
2724962993124,5cyclictest2781337-21ntpq08:00:212
2724959993128,2cyclictest3010743-21kworker/u8:3+flush-259:011:55:201
2724959993127,3cyclictest3016596-21kworker/u8:0+flush-259:012:10:161
2724959993127,3cyclictest2949304-21kworker/u8:3+events_unbound10:55:231
2724959993126,4cyclictest2740661-21kworker/u8:2+flush-259:009:04:591
2724957993126,3cyclictest2808777-21/usr/sbin/munin08:30:120
2724967993028,2cyclictest2740661-21kworker/u8:2+events_unbound08:20:263
2724967993025,4cyclictest3063685-21apt-get12:35:123
2724962993026,3cyclictest661596-21kworker/u8:4+flush-259:007:10:152
2724957993025,4cyclictest2787897-21diskstats08:10:130
2724967992927,2cyclictest2897754-21kworker/u8:3+events_unbound10:04:043
2724967992926,2cyclictest2772891-21kworker/u8:1+events_unbound08:50:223
2724967992926,2cyclictest2755959-21kworker/u8:4+events_unbound07:40:233
2724962992923,4cyclictest2802935-21kworker/u8:3+flush-259:009:00:252
2724962992825,2cyclictest2860174-21kworker/u8:1+flush-259:009:24:062
2724959992826,1cyclictest2887239-21kworker/u8:2+flush-259:009:50:191
2724967992724,2cyclictest2897545-21kworker/u8:1+events_unbound10:10:253
2724967992723,3cyclictest2944010-21kworker/u8:1+events_unbound10:40:033
2724967992723,3cyclictest2802935-21kworker/u8:3+events_unbound08:55:013
2724962992721,4cyclictest3063417-21latency_hist12:35:012
2724959992724,2cyclictest75-21kswapd008:05:591
2724959992724,2cyclictest2802935-21kworker/u8:3+flush-259:009:10:251
2724959992724,2cyclictest2755959-21kworker/u8:4+flush-259:007:40:111
2724959992720,5cyclictest3058237-21latency_hist12:30:011
2724957992722,4cyclictest2876194-21base6409:35:010
2724967992623,2cyclictest2944010-21kworker/u8:1+events_unbound11:25:263
2724962992621,4cyclictest2802935-21kworker/u8:3+flush-259:008:40:152
2724959992623,2cyclictest2975126-21kworker/u8:2+events_unbound11:10:181
2724959992623,2cyclictest2944010-21kworker/u8:1+flush-259:011:25:261
2724959992622,3cyclictest2692572-21kworker/u8:1+events_unbound07:22:591
272495799269,4cyclictest2776362-21/usr/sbin/munin07:55:230
2724967992522,2cyclictest2772891-21kworker/u8:1+events_unbound08:47:023
2724967992522,2cyclictest2740661-21kworker/u8:2+events_unbound08:30:083
2724967992521,3cyclictest2772891-21kworker/u8:1+events_unbound08:08:593
2724962992519,4cyclictest2927674-21ntpq10:20:192
2724959992522,2cyclictest2944010-21kworker/u8:1+events_unbound11:05:011
2724959992522,2cyclictest2897754-21kworker/u8:3+events_unbound09:55:221
2724959992522,2cyclictest2772891-21kworker/u8:1+flush-259:008:20:171
2724959992522,2cyclictest2772891-21kworker/u8:1+events_unbound08:40:021
2724959992522,2cyclictest2772891-21kworker/u8:1+events_unbound08:30:201
2724959992521,3cyclictest3022054-21kworker/u8:2+events_unbound12:30:261
2724959992521,3cyclictest2692572-21kworker/u8:1+events_unbound07:19:021
2724959992518,5cyclictest3047049-21ntpq12:15:201
272495799259,5cyclictest2974647-21ntpq11:05:210
272495799258,6cyclictest3058545-21grep12:30:120
272495799257,13cyclictest0-21swapper/010:15:130
2724957992511,8cyclictest3022350-21apt-get11:55:130
2724957992510,4cyclictest2848936-21ntpq09:05:220
2724957992510,3cyclictest2990281-21ntpq11:20:200
2724967992421,2cyclictest661596-21kworker/u8:4+events_unbound07:19:583
2724967992421,2cyclictest2897545-21kworker/u8:1+events_unbound10:19:323
2724959992421,2cyclictest2959562-21kworker/u8:0+events_unbound11:00:211
2724959992421,2cyclictest2714522-21kworker/u8:3+events_unbound07:33:011
272495799249,9cyclictest2843803-21ntpq09:00:220
272495799249,8cyclictest3041881-21ntpq12:10:210
272495799249,5cyclictest2985901-21/usr/sbin/munin11:20:000
272495799249,4cyclictest3048203-21chrt12:20:110
272495799248,7cyclictest2886336-21ntpq09:40:190
2724957992412,2cyclictest2995394-21ntpq11:25:200
2724967992320,2cyclictest661596-21kworker/u8:4+events_unbound07:14:013
2724967992320,2cyclictest3010743-21kworker/u8:3+events_unbound11:47:193
2724967992320,2cyclictest2740661-21kworker/u8:2+events_unbound09:33:043
2724967992320,2cyclictest2740661-21kworker/u8:2+events_unbound08:14:183
2724959992320,2cyclictest2944010-21kworker/u8:1+events_unbound11:20:211
2724959992320,2cyclictest2928618-21kworker/u8:2+events_unbound10:25:181
2724959992320,2cyclictest2897754-21kworker/u8:3+flush-259:010:30:161
2724959992320,2cyclictest2860186-21kworker/u8:4+flush-259:009:30:241
2724959992319,3cyclictest2928618-21kworker/u8:2+flush-259:010:40:111
2724959992319,3cyclictest2829184-21kworker/u8:0+flush-259:009:45:241
2724959992317,4cyclictest2829423-21apt-get08:50:101
272495799239,7cyclictest2907018-21ntpq10:00:210
272495799239,5cyclictest2828641-21sed08:45:240
272495799239,4cyclictest3021367-21/usr/sbin/munin11:50:220
272495799239,4cyclictest3021367-21/usr/sbin/munin11:50:220
272495799238,8cyclictest3031533-21ntpq12:00:190
272495799238,7cyclictest2953588-21ntpq10:45:200
272495799238,5cyclictest2917316-21gcc10:10:200
272495799238,4cyclictest3037738-21wc12:10:000
272495799237,5cyclictest2755961-21systemd-run07:40:000
2724957992320,2cyclictest2740661-21kworker/u8:2+flush-259:008:05:180
2724957992317,4cyclictest2782236-21latency_hist08:05:000
2724957992311,2cyclictest2958682-21ntpq10:50:220
2724967992219,2cyclictest2944010-21kworker/u8:1+events_unbound10:48:033
2724962992218,3cyclictest3016596-21kworker/u8:0+flush-259:011:55:252
2724962992216,4cyclictest2980963-21apt-get11:15:122
2724959992219,2cyclictest3010743-21kworker/u8:3+events_unbound12:00:111
2724959992219,2cyclictest2740661-21kworker/u8:2+flush-259:009:25:261
2724959992218,3cyclictest2928618-21kworker/u8:2+flush-259:010:45:111
272495799229,4cyclictest2893952-21cut09:50:180
272495799229,4cyclictest2854083-21ntpq09:10:220
272495799229,1cyclictest2808134-21munin-run08:30:010
272495799228,3cyclictest3005885-21/usr/sbin/munin11:35:240
272495799227,4cyclictest3057348-21collect212:25:190
272495799227,4cyclictest3047221-21/usr/sbin/munin12:15:220
272495799227,4cyclictest2901759-21netstat09:55:170
2724957992217,4cyclictest2834334-21latency_hist08:55:000
2724957992210,7cyclictest2766847-21fw_conntrack07:50:140
2724957992210,6cyclictest2855523-21fw_forwarded_lo09:15:140
2724957992210,3cyclictest3016012-21ntpq11:45:210
2724957992210,2cyclictest3067754-21ntpq12:35:210
2724957992210,2cyclictest2891460-21ntpq09:45:200
2724967992118,2cyclictest3010743-21kworker/u8:3+events_unbound11:42:193
2724962992115,4cyclictest2912210-21ntpq10:05:202
2724962992114,5cyclictest335557-21systemd-journal07:50:012
2724959992118,2cyclictest2897545-21kworker/u8:1+events_unbound10:20:201
272495799218,4cyclictest0-21swapper/011:40:210
272495799217,4cyclictest2997134-21irqcore11:30:160
272495799216,4cyclictest2932912-21collect210:25:200
272495799216,4cyclictest2750676-21modprobe07:34:090
272495799216,10cyclictest2817971-21ntpq08:35:210
272495799215,6cyclictest2966612-21cut11:00:170
272495799215,6cyclictest2926594-21cut10:20:170
272495799215,5cyclictest2733810-21kthreadcore07:15:180
2724957992115,4cyclictest2839488-21latency_hist09:00:010
2724967992016,3cyclictest2740661-21kworker/u8:2+events_unbound08:42:033
2724962992016,3cyclictest2740661-21kworker/u8:2+flush-259:009:09:172
2724962992016,3cyclictest2692572-21kworker/u8:1+flush-259:007:20:252
2724962992014,4cyclictest2996509-21/usr/sbin/munin11:30:132
2724962992014,4cyclictest2944530-21diskstats10:40:112
2724962992013,5cyclictest2772020-21gpgv07:55:012
2724959992018,1cyclictest2777090-21sh08:00:011
2724959992017,2cyclictest3032146-21kworker/u8:1+events_unbound12:05:201
2724959992013,5cyclictest335557-21systemd-journal10:40:001
272495799207,4cyclictest2975795-21grep11:10:130
272495799206,8cyclictest2735488-21taskset07:20:110
272495799206,4cyclictest2760308-21modprobe07:40:200
272495799205,5cyclictest2963564-21cut10:55:170
2724957992015,4cyclictest2938926-21latency_hist10:35:010
2724967991916,2cyclictest3016596-21kworker/u8:0+events_unbound12:14:183
2724967991916,2cyclictest2949304-21kworker/u8:3+events_unbound10:55:533
2724967991916,2cyclictest2897754-21kworker/u8:3+events_unbound10:22:113
2724967991915,3cyclictest2692572-21kworker/u8:1+events_unbound07:30:173
2724967991914,4cyclictest3037628-21latency_hist12:10:003
2724962991913,4cyclictest3053062-21latency_hist12:25:002
2724962991913,4cyclictest3006838-21/usr/sbin/munin11:40:142
2724959991916,2cyclictest2829184-21kworker/u8:0+events_unbound09:05:241
2724959991914,2cyclictest2692572-21kworker/u8:1+events_unbound07:35:131
2724959991913,2cyclictest2980924-21kworker/u8:3+events_unbound11:16:181
272495799198,2cyclictest2823095-21ntpq08:40:200
272495799196,4cyclictest2865861-21taskset09:25:100
272495799195,8cyclictest2908297-21df10:05:120
272495799195,5cyclictest2799396-21kthreadcore08:20:180
2724967991816,2cyclictest2860186-21kworker/u8:4+events_unbound09:57:033
2724967991816,2cyclictest2740661-21kworker/u8:2+events_unbound09:00:043
2724962991814,3cyclictest2897754-21kworker/u8:3+flush-259:010:00:202
2724962991813,3cyclictest2802935-21kworker/u8:3+flush-259:009:10:172
2724962991812,4cyclictest2834605-21apt-get08:55:132
2724962991811,5cyclictest335557-21systemd-journal11:20:182
2724959991815,2cyclictest3032146-21kworker/u8:1+flush-259:012:35:141
2724959991815,2cyclictest2740661-21kworker/u8:2+flush-259:008:55:151
2724959991814,3cyclictest2897545-21kworker/u8:1+events_unbound10:05:171
272495799187,2cyclictest2744531-21memory07:25:190
272495799186,2cyclictest2943220-21ntpq10:35:210
272495799185,4cyclictest2763423-21grep07:45:170
2724967991715,2cyclictest3032146-21kworker/u8:1+events_unbound12:33:383
2724962991715,1cyclictest3038312-21irqcore12:10:162
2724962991715,1cyclictest2887239-21kworker/u8:2+flush-259:009:50:132
2724962991713,3cyclictest2756206-21apt-get07:40:102
2724962991712,3cyclictest2902685-21idleruntime-cro10:00:002
2724962991711,4cyclictest335557-21systemd-journal10:17:002
2724959991714,2cyclictest2802935-21kworker/u8:3+events_unbound09:15:561
2724959991713,3cyclictest2944010-21kworker/u8:1+events_unbound10:50:221
2724959991713,3cyclictest2802935-21kworker/u8:3+events_unbound08:25:131
272495799179,5cyclictest2792890-21apt-get08:15:110
2724967991613,2cyclictest2829184-21kworker/u8:0+events_unbound09:41:153
2724967991613,2cyclictest2802935-21kworker/u8:3+events_unbound08:25:313
272496299164,6cyclictest2954414-21awk10:50:002
2724962991612,3cyclictest2959562-21kworker/u8:0+flush-259:011:00:162
2724962991612,3cyclictest2791727-21ntp_kernel_pll_08:10:212
2724962991612,3cyclictest2772891-21kworker/u8:1+events_unbound08:39:592
2724962991612,3cyclictest2692572-21kworker/u8:1+flush-259:007:30:232
2724962991611,3cyclictest2792637-21kworker/u8:0+flush-259:008:27:032
2724959991613,2cyclictest2990596-21kworker/u8:2+events_unbound11:30:251
2724959991613,2cyclictest2740661-21kworker/u8:2+events_unbound08:12:011
2724959991612,3cyclictest3022054-21kworker/u8:2+events_unbound12:21:211
272495799168,5cyclictest2725061-21cat07:10:010
272495799168,5cyclictest2725061-21cat07:10:000
272496299159,4cyclictest2802948-21latency_hist08:25:002
2724962991512,2cyclictest3001445-21kworker/u8:0+flush-259:011:35:212
2724962991512,2cyclictest2876773-21cpuspeed_turbos09:35:152
2724962991511,3cyclictest2771450-21sh07:55:002
2724959991512,2cyclictest2897754-21kworker/u8:3+events_unbound10:03:041
2724959991511,3cyclictest3006598-21latency_hist11:40:001
272496299149,4cyclictest3016596-21kworker/u8:0+events_unbound12:25:152
272496299142,10cyclictest2933780-21munin-run10:30:002
2724962991411,1cyclictest0-21swapper/211:10:132
2724962991410,2cyclictest2939284-21cpuspeed_turbos10:35:142
2724959991410,3cyclictest2772891-21kworker/u8:1+events_unbound08:47:191
272496799132,2cyclictest50-21ksoftirqd/307:10:013
272496799132,2cyclictest50-21ksoftirqd/307:10:013
272496299138,3cyclictest2990596-21kworker/u8:2+flush-259:011:25:102
272496299138,3cyclictest2881610-21apt-get09:40:022
272496299137,4cyclictest335557-21systemd-journal09:30:002
272496299136,5cyclictest2735231-21latency_hist07:20:012
2724959991310,2cyclictest2740661-21kworker/u8:2+events_unbound08:17:531
272496299122,5cyclictest2918243-21munin-run10:15:002
272496299121,8cyclictest2829156-21latency_hist08:50:002
272496299121,4cyclictest3022094-21munin-run11:55:012
272496299121,4cyclictest3022094-21munin-run11:55:002
2724962991210,1cyclictest0-21swapper/211:45:132
272496299119,1cyclictest0-21swapper/209:30:162
272496299118,2cyclictest2971003-21irqcore11:05:162
272496299118,2cyclictest0-21swapper/212:05:192
272496299117,2cyclictest0-21swapper/207:29:592
272496299116,4cyclictest3067859-21wc12:35:232
272496299107,2cyclictest2889768-21kthreadcore09:45:172
272496299106,2cyclictest2944010-21kworker/u8:1+events_unbound10:55:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional