You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-26 - 19:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Sun Jul 26, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2663875995451,2cyclictest2673876-21kworker/u8:3+flush-259:008:10:012
2663872994845,2cyclictest2845609-21kworker/u8:0+events_unbound10:11:211
2663872994541,3cyclictest2845583-21latency_hist10:05:011
2663872994440,3cyclictest2960782-21kworker/u8:3+events_unbound12:23:201
2663880994235,5cyclictest3001921-21latency_hist12:35:013
2663872993835,2cyclictest2772965-21kworker/u8:3+flush-259:009:15:001
2663875993734,2cyclictest2819791-21kworker/u8:2+flush-259:009:55:002
2663880993631,3cyclictest2866498-21/usr/sbin/munin10:25:193
2663880993629,4cyclictest335557-21systemd-journal10:00:283
2663875993427,5cyclictest335557-21systemd-journal11:01:292
2663872993431,2cyclictest2809420-21kworker/u8:1+flush-259:010:50:211
2663872993431,2cyclictest2788714-21kworker/u8:0+events_unbound09:40:221
2663872993431,2cyclictest2772965-21kworker/u8:3+events_unbound09:05:141
2663872993431,2cyclictest2604994-21kworker/u8:1+flush-259:007:10:011
2663872993429,4cyclictest2720875-21apt-get08:05:131
2663870993427,5cyclictest2730023-21ntpq08:10:210
2663880993329,3cyclictest2934098-21kworker/u8:0+flush-259:011:35:013
2663880993329,3cyclictest2715664-21kworker/u8:0+flush-259:008:00:213
2663872993330,2cyclictest2819791-21kworker/u8:2+events_unbound10:10:001
2663870993327,4cyclictest2913496-21apt-get11:10:150
2663872993229,2cyclictest2913033-21store11:10:001
2663872993229,2cyclictest2788714-21kworker/u8:0+events_unbound09:20:171
2663872993229,2cyclictest2658702-21kworker/u8:2+flush-259:007:45:211
2663872993222,3cyclictest2991034-21kworker/u8:0+events_unbound12:25:231
2663870993228,3cyclictest2809420-21kworker/u8:1+flush-259:009:45:260
2663870993227,3cyclictest2887171-21apt-get10:45:100
2663872993128,2cyclictest2730961-21kworker/u8:4+events_unbound09:28:591
2663872993126,4cyclictest2897519-21apt-get10:55:131
2663872993126,4cyclictest2741957-21/usr/sbin/munin08:25:151
2663872993124,2cyclictest2604994-21kworker/u8:1+events_unbound07:10:231
2663870993127,3cyclictest2673876-21kworker/u8:3+events_unbound07:33:300
2663880993026,3cyclictest2866244-21kworker/u8:0+flush-259:010:30:243
2663880993025,4cyclictest2673876-21kworker/u8:3+flush-259:007:20:173
2663880993025,4cyclictest2673876-21kworker/u8:3+flush-259:007:20:163
2663880993023,5cyclictest2959938-21cat11:55:003
2663872993027,2cyclictest2788714-21kworker/u8:0+flush-259:009:35:241
2663872993026,3cyclictest2809420-21kworker/u8:1+flush-259:010:40:211
2663870993026,3cyclictest2819791-21kworker/u8:2+events_unbound09:55:230
2663870993026,3cyclictest2809420-21kworker/u8:1+events_unbound10:00:230
2663870993024,4cyclictest2820177-21diskstats09:40:150
2663875992924,4cyclictest2673876-21kworker/u8:3+flush-259:007:45:002
2663872992926,2cyclictest2934098-21kworker/u8:0+flush-259:011:45:201
2663872992926,2cyclictest2730961-21kworker/u8:4+events_unbound08:50:211
2663872992926,2cyclictest2730961-21kworker/u8:4+events_unbound08:30:171
2663872992926,2cyclictest2699974-21kworker/u8:1+flush-259:008:00:251
2663872992922,2cyclictest2699974-21kworker/u8:1+events_unbound08:15:241
2663870992926,2cyclictest2730961-21kworker/u8:4+flush-259:008:40:210
2663870992925,3cyclictest2881803-21kworker/u8:2+events_unbound11:05:140
2663870992924,4cyclictest335557-21systemd-journal07:17:280
2663875992821,4cyclictest2980883-21latency_hist12:15:002
2663872992825,2cyclictest2996743-21kworker/u8:2+events_unbound12:30:231
2663872992825,2cyclictest2944227-21kworker/u8:4+events_unbound11:59:291
2663872992825,2cyclictest2944227-21kworker/u8:4+events_unbound11:59:281
2663870992824,3cyclictest2970216-21kworker/u8:0+events_unbound12:04:300
2663870992824,3cyclictest2970216-21kworker/u8:0+events_unbound12:04:290
2663870992824,3cyclictest2881803-21kworker/u8:2+events_unbound11:18:290
2663870992822,4cyclictest2929161-21diskstats11:25:140
2663880992723,3cyclictest2730961-21kworker/u8:4+flush-259:008:25:223
2663875992719,5cyclictest2850738-21latency_hist10:10:022
2663872992724,2cyclictest2918426-21kworker/u8:3+flush-259:011:25:161
2663872992724,2cyclictest2819791-21kworker/u8:2+events_unbound09:55:171
2663872992724,2cyclictest2725786-21kworker/u8:2+events_unbound08:20:221
2663872992723,3cyclictest2830343-21/usr/sbin/munin09:50:151
2663872992723,3cyclictest2809420-21kworker/u8:1+flush-259:010:50:001
2663872992721,2cyclictest2944227-21kworker/u8:4+events_unbound12:10:141
2663870992723,3cyclictest2730961-21kworker/u8:4+events_unbound09:21:290
2663870992723,2cyclictest2715698-21apt-get08:00:120
2663880992621,3cyclictest2809420-21kworker/u8:1+events_unbound09:45:003
2663875992622,3cyclictest2658702-21kworker/u8:2+flush-259:007:55:002
2663875992620,3cyclictest2866254-21latency_hist10:25:012
2663872992623,2cyclictest2934098-21kworker/u8:0+events_unbound11:30:181
2663872992622,3cyclictest2809420-21kworker/u8:1+events_unbound09:45:271
2663872992618,6cyclictest335557-21systemd-journal08:45:011
2663870992623,2cyclictest2934098-21kworker/u8:0+flush-259:011:50:260
2663870992623,2cyclictest2918426-21kworker/u8:3+events_unbound11:20:220
2663870992622,3cyclictest3774224-21kworker/u8:0+events_unbound07:13:310
2663870992622,3cyclictest335557-21systemd-journal10:36:280
2663880992518,5cyclictest335557-21systemd-journal12:20:013
2663875992518,5cyclictest2944227-21kworker/u8:4+flush-259:012:30:162
2663875992517,5cyclictest2783744-21cp09:05:012
2663872992522,2cyclictest2944227-21kworker/u8:4+events_unbound12:05:231
2663872992522,2cyclictest2918426-21kworker/u8:3+events_unbound11:35:231
2663872992522,2cyclictest2730961-21kworker/u8:4+events_unbound09:15:161
2663872992518,3cyclictest2673876-21kworker/u8:3+events_unbound07:20:221
2663872992518,3cyclictest2673876-21kworker/u8:3+events_unbound07:20:221
2663870992523,2cyclictest2944225-21kworker/u8:1+events_unbound11:44:290
2663870992522,2cyclictest2881803-21kworker/u8:2+events_unbound11:00:240
2663870992522,2cyclictest2809420-21kworker/u8:1+flush-259:010:50:250
2663870992522,2cyclictest2788714-21kworker/u8:0+events_unbound09:10:280
2663870992521,3cyclictest2944227-21kworker/u8:4+events_unbound12:08:280
2663870992520,3cyclictest335557-21systemd-journal10:20:280
2663880992420,3cyclictest2918426-21kworker/u8:3+flush-259:011:40:003
2663875992421,2cyclictest2949642-21apt-get11:45:112
2663875992419,3cyclictest335557-21systemd-journal09:28:282
2663875992418,4cyclictest335557-21systemd-journal07:25:282
2663872992422,2cyclictest2881803-21kworker/u8:2+events_unbound11:43:301
2663872992421,2cyclictest2918426-21kworker/u8:3+flush-259:011:20:241
2663872992421,2cyclictest2866244-21kworker/u8:0+events_unbound10:35:241
2663872992421,2cyclictest2752111-21kworker/u8:2+events_unbound09:00:191
2663872992420,3cyclictest3774224-21kworker/u8:0+flush-259:007:15:241
2663872992420,3cyclictest2881803-21kworker/u8:2+flush-259:011:00:211
2663870992420,3cyclictest2819791-21kworker/u8:2+events_unbound09:52:280
2663870992420,3cyclictest2788714-21kworker/u8:0+events_unbound09:15:220
2663870992418,4cyclictest2783107-21latency_hist09:05:000
2663880992316,5cyclictest2960258-21cp11:55:003
2663880992316,5cyclictest2960258-21cp11:55:003
266387599235,6cyclictest2772729-21latency_hist08:55:002
2663875992316,5cyclictest2944628-21diskstats11:40:162
2663875992316,4cyclictest2897249-21latency_hist10:55:012
2663872992320,2cyclictest2960782-21kworker/u8:3+events_unbound12:00:161
2663872992320,2cyclictest2960782-21kworker/u8:3+events_unbound12:00:151
2663872992320,2cyclictest2860817-21kworker/u8:3+events_unbound10:25:281
2663872992320,2cyclictest2860817-21kworker/u8:3+events_unbound10:19:101
2663872992316,5cyclictest2757229-21latency_hist08:40:011
2663870992320,2cyclictest2944227-21kworker/u8:4+events_unbound12:30:130
2663870992320,2cyclictest2752111-21kworker/u8:2+events_unbound08:55:170
2663870992320,2cyclictest2699974-21kworker/u8:1+events_unbound08:18:440
2663870992319,3cyclictest2970216-21kworker/u8:0+events_unbound12:10:220
2663870992319,3cyclictest2944227-21kworker/u8:4+events_unbound11:57:290
2663870992319,3cyclictest2944227-21kworker/u8:4+events_unbound11:57:290
2663870992319,3cyclictest2934098-21kworker/u8:0+flush-259:011:30:240
2663870992319,3cyclictest2845609-21kworker/u8:0+flush-259:010:15:260
2663870992319,3cyclictest2752111-21kworker/u8:2+events_unbound08:35:270
2663880992218,3cyclictest2699974-21kworker/u8:1+flush-259:008:35:193
2663880992218,3cyclictest2699974-21kworker/u8:1+flush-259:007:50:003
2663880992218,3cyclictest2658702-21kworker/u8:2+flush-259:007:25:173
2663880992217,3cyclictest2876843-21/usr/sbin/munin10:35:153
2663875992216,4cyclictest2912806-21latency_hist11:10:002
2663872992219,2cyclictest3001948-21kworker/u8:0+flush-259:012:35:251
2663872992219,2cyclictest2960782-21kworker/u8:3+events_unbound12:15:181
2663872992219,2cyclictest2881803-21kworker/u8:2+flush-259:011:20:001
2663872992219,2cyclictest2860817-21kworker/u8:3+events_unbound10:30:281
2663870992219,3cyclictest2730961-21kworker/u8:4+events_unbound09:33:290
2663870992219,2cyclictest2991034-21kworker/u8:0+events_unbound12:25:160
2663870992219,2cyclictest2715664-21kworker/u8:0+events_unbound08:05:170
2663870992219,2cyclictest2658702-21kworker/u8:2+events_unbound07:42:280
2663870992218,3cyclictest3001948-21kworker/u8:0+events_unbound12:35:200
2663870992218,2cyclictest2788695-21latency_hist09:10:000
2663880992114,5cyclictest335557-21systemd-journal11:09:593
2663875992114,4cyclictest2777953-21latency_hist08:59:592
2663872992117,3cyclictest2772977-21/usr/sbin/munin08:55:181
2663880992017,2cyclictest2960782-21kworker/u8:3+flush-259:012:00:213
2663880992017,2cyclictest2960782-21kworker/u8:3+flush-259:012:00:203
2663880992017,2cyclictest2736345-21gpgv08:20:003
2663880992014,4cyclictest335557-21systemd-journal09:00:003
2663880992014,4cyclictest2819791-21kworker/u8:2+flush-259:009:55:163
266387599204,6cyclictest2902462-21grep11:00:002
2663875992016,3cyclictest2960006-21latency_hist11:55:002
2663875992016,2cyclictest0-21swapper/208:20:002
2663872992017,2cyclictest2860817-21kworker/u8:3+events_unbound10:20:231
2663872992017,2cyclictest2689688-21kworker/u8:0+events_unbound07:35:151
2663872992017,2cyclictest2673876-21kworker/u8:3+flush-259:007:40:161
2663872992017,2cyclictest2658702-21kworker/u8:2+flush-259:007:55:261
2663870992017,2cyclictest2730961-21kworker/u8:4+events_unbound08:25:250
2663870992017,2cyclictest2658702-21kworker/u8:2+flush-259:007:50:160
2663870992017,2cyclictest2658702-21kworker/u8:2+flush-259:007:45:230
2663870992016,3cyclictest335557-21systemd-journal10:09:040
2663870992016,3cyclictest2788714-21kworker/u8:0+events_unbound09:40:000
2663870992016,3cyclictest2725786-21kworker/u8:2+flush-259:008:20:240
2663880991915,3cyclictest2725786-21kworker/u8:2+flush-259:008:20:243
2663880991915,3cyclictest2699974-21kworker/u8:1+flush-259:008:10:293
2663880991915,2cyclictest2970216-21kworker/u8:0+events_unbound12:10:173
2663880991914,4cyclictest2881803-21kworker/u8:2+events_unbound11:45:233
266387599193,4cyclictest2761520-21ld08:40:212
2663875991916,2cyclictest2975667-21cron12:10:002
2663875991916,2cyclictest2748180-21kthreadcore08:30:192
2663875991916,2cyclictest0-21swapper/207:20:002
2663872991916,2cyclictest2658702-21kworker/u8:2+flush-259:007:50:191
2663872991915,2cyclictest1-21systemd07:34:591
2663870991916,2cyclictest2730961-21kworker/u8:4+events_unbound08:45:140
2663870991915,3cyclictest2944227-21kworker/u8:4+events_unbound12:16:280
2663870991915,3cyclictest2658702-21kworker/u8:2+events_unbound08:00:000
2663880991815,2cyclictest2809664-21apt-get09:30:143
2663880991815,2cyclictest2695506-21irqcore07:40:173
2663880991814,3cyclictest2991034-21kworker/u8:0+flush-259:012:25:253
2663875991815,2cyclictest2964342-21needreboot11:55:192
2663875991815,2cyclictest2964342-21needreboot11:55:182
2663875991814,3cyclictest2730961-21kworker/u8:4+flush-259:009:35:002
266387599181,15cyclictest1-21systemd08:00:012
2663872991815,2cyclictest2715664-21kworker/u8:0+events_unbound08:10:131
2663870991814,3cyclictest2934098-21kworker/u8:0+events_unbound11:49:290
2663880991715,1cyclictest2897929-21irqcore10:55:183
2663880991714,2cyclictest500-21systemd-logind11:45:003
2663880991714,2cyclictest2780510-21kthreadcore09:00:183
2663880991714,2cyclictest0-21swapper/307:20:003
2663880991714,1cyclictest0-21swapper/310:55:003
2663880991713,3cyclictest2819791-21kworker/u8:2+events_unbound09:45:153
2663880991711,4cyclictest335557-21systemd-journal09:10:003
2663875991713,3cyclictest2970216-21kworker/u8:0+flush-259:012:05:012
2663875991713,3cyclictest2970216-21kworker/u8:0+flush-259:012:05:002
2663872991714,2cyclictest2699974-21kworker/u8:1+events_unbound08:49:301
2663870991713,3cyclictest2960782-21kworker/u8:3+flush-259:012:20:160
2663870991713,3cyclictest2788714-21kworker/u8:0+events_unbound09:25:170
2663870991713,3cyclictest2689688-21kworker/u8:0+flush-259:007:35:260
2663870991712,4cyclictest2939473-21diskstats11:35:150
2663870991712,3cyclictest2901602-21ntpq10:55:240
2663880991613,1cyclictest0-21swapper/307:39:593
2663880991612,3cyclictest2959746-21kworker/u8:1+events_unbound12:25:003
2663880991612,3cyclictest2819791-21kworker/u8:2+events_unbound10:20:023
266387599162,12cyclictest490-21dbus-daemon12:20:012
2663875991611,4cyclictest335557-21systemd-journal12:23:282
2663872991612,3cyclictest2604994-21kworker/u8:1+events_unbound07:26:291
2663870991612,3cyclictest2604994-21kworker/u8:1+events_unbound07:25:300
2663870991610,4cyclictest335557-21systemd-journal10:30:000
2663880991512,2cyclictest2970216-21kworker/u8:0+flush-259:012:05:163
2663880991512,2cyclictest2922643-21ntp_kernel_err11:15:233
2663880991511,3cyclictest2768253-21irqcore08:50:163
266387599154,7cyclictest2798982-21cron09:20:002
266387599153,2cyclictest2943304-21ntpq11:35:222
266387599152,4cyclictest2995871-21ntpq12:25:222
2663875991512,2cyclictest2922662-21ntp_kernel_pll_11:15:232
2663875991511,3cyclictest2923826-21apt-get11:20:112
2663875991510,3cyclictest2809420-21kworker/u8:1+flush-259:010:40:172
2663870991514,1cyclictest60350irq/132-enp0s29f1:rx-010:43:520
2663870991511,3cyclictest2866244-21kworker/u8:0+flush-259:010:30:190
2663880991411,2cyclictest2668513-21unixbench_multi07:10:253
2663880991410,3cyclictest2730961-21kworker/u8:4+flush-259:008:40:243
266387599149,3cyclictest2730961-21kworker/u8:4+flush-259:008:45:152
2663875991411,2cyclictest2855924-21idleruntime-cro10:15:002
266387599140,13cyclictest2861015-21sshd10:19:282
2663872991411,2cyclictest2788714-21kworker/u8:0+events_unbound09:30:171
2663872991410,3cyclictest2881803-21kworker/u8:2+events_unbound11:14:201
2663870991412,1cyclictest61150irq/133-enp0s29f1:tx-008:30:160
266388099139,3cyclictest2699974-21kworker/u8:1+flush-259:007:50:403
266388099135,6cyclictest490-21dbus-daemon10:45:003
2663880991310,2cyclictest2846605-21irqcore10:05:173
266387599139,3cyclictest2871668-21/usr/sbin/munin10:30:162
266387599138,3cyclictest2845586-21sh10:05:002
266387599137,4cyclictest2868822-21kthreadcore10:25:242
266387599136,5cyclictest2892076-21idleruntime-cro10:50:012
266387599132,2cyclictest2746053-21ntpq08:25:232
2663875991310,2cyclictest2803736-21unixbench_singl09:20:242
2663875991310,2cyclictest2737848-21tr08:20:172
2663870991312,1cyclictest60350irq/132-enp0s29f1:rx-010:11:050
266388099129,2cyclictest2930021-21kthreadcore11:25:173
266388099128,2cyclictest0-21swapper/310:15:253
2663880991210,1cyclictest0-21swapper/308:35:013
266388099120,10cyclictest2855918-21awk10:15:003
266387599129,2cyclictest2936431-21kthreadcore11:30:172
266387599126,4cyclictest2824890-21cron09:45:002
266387599126,4cyclictest2730936-21idleruntime-cro08:15:002
266387599126,3cyclictest2694825-21latency_hist07:40:012
266387599125,5cyclictest2782697-21grep09:00:262
266387599125,5cyclictest2756788-21grep08:35:252
266387599121,3cyclictest3007098-21awk12:40:002
266387299129,2cyclictest2934098-21kworker/u8:0+events_unbound11:50:301
2663870991210,1cyclictest60350irq/132-enp0s29f1:rx-007:20:230
2663870991210,1cyclictest60350irq/132-enp0s29f1:rx-007:20:220
266387099120,10cyclictest2663873-21cat07:10:000
266388099119,1cyclictest0-21swapper/312:40:003
266388099118,2cyclictest2834405-21ntp_kernel_pll_09:50:213
266388099118,2cyclictest2721133-21fschecks_time08:05:163
266388099118,1cyclictest0-21swapper/311:20:133
266388099116,3cyclictest335254-21systemd-udevd07:55:203
266387599115,4cyclictest2913615-21tr11:10:152
266387599113,5cyclictest0-21swapper/210:35:262
266387599111,2cyclictest41-21ksoftirqd/211:25:192
266387599111,2cyclictest2789119-21diskstats09:10:182
266387599111,2cyclictest2704699-21users07:45:232
266388099107,2cyclictest2797986-21needreboot09:15:203
266387599105,3cyclictest2664202-21tr07:10:122
266387599100,2cyclictest41-21ksoftirqd/209:55:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional