You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-09 - 15:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Thu Jul 09, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2709480995144,5cyclictest3031002-21latency_hist12:20:002
2709477994642,3cyclictest2917080-21kworker/u8:0+flush-259:011:20:011
2709477994541,3cyclictest2651091-21kworker/u8:3+events_unbound07:20:011
2709475994439,4cyclictest2735180-21latency_hist07:35:010
2709477994340,2cyclictest2676993-21kworker/u8:1+events_unbound07:20:211
2709477994239,2cyclictest2917080-21kworker/u8:0+flush-259:010:30:181
2709477994138,2cyclictest2917080-21kworker/u8:0+flush-259:011:10:251
2709480994034,4cyclictest335557-21systemd-journal11:55:002
2709480994034,4cyclictest2725113-21/usr/sbin/munin07:25:132
2709477994036,3cyclictest2891920-21kworker/u8:3+events_unbound10:50:241
2709477994036,3cyclictest2719945-21kworker/u8:0+events_unbound08:00:011
2709477993936,2cyclictest3015494-21kworker/u8:3+events_unbound12:10:221
2709477993935,3cyclictest2901539-21kworker/u8:1+events_unbound10:20:021
2709477993935,3cyclictest2676993-21kworker/u8:1+events_unbound07:10:111
270947599390,38cyclictest1-21systemd08:27:240
2709477993835,2cyclictest2813236-21kworker/u8:1+events_unbound09:45:251
2709477993733,3cyclictest2781274-21kworker/u8:3+events_unbound08:25:011
2709477993733,3cyclictest2745493-21kworker/u8:1+events_unbound08:05:021
2709477993633,2cyclictest2813236-21kworker/u8:1+events_unbound09:20:201
2709477993632,3cyclictest2942827-21kworker/u8:4+events_unbound11:10:001
2709475993632,3cyclictest2917080-21kworker/u8:0+flush-259:011:00:200
2709477993430,3cyclictest2973780-21kworker/u8:1+flush-259:011:40:251
2709477993430,3cyclictest2973780-21kworker/u8:1+events_unbound12:00:251
2709475993425,6cyclictest2818786-21latency_hist08:55:000
2709485993327,4cyclictest2751043-21latency_hist07:50:013
2709477993330,2cyclictest3015494-21kworker/u8:3+events_unbound12:35:141
2709477993229,2cyclictest2932517-21kworker/u8:2+events_unbound10:55:251
2709477993227,3cyclictest2745688-21gpgv07:45:011
2709477993225,5cyclictest2734285-21ntpq07:30:211
2709475993227,4cyclictest2839638-21apt-get09:15:130
2709477993128,2cyclictest2973808-21kworker/u8:2+events_unbound11:36:241
2709477993128,2cyclictest2973780-21kworker/u8:1+events_unbound11:25:311
2709477993128,2cyclictest2839412-21kworker/u8:2+events_unbound09:25:241
2709477993127,3cyclictest2917080-21kworker/u8:0+flush-259:011:05:011
2709477993127,3cyclictest2807634-21kworker/u8:4+flush-259:008:45:201
2709477993127,3cyclictest2807131-21kworker/u8:0+events_unbound09:36:251
2709475993128,2cyclictest2942827-21kworker/u8:4+flush-259:011:05:210
2709475993124,4cyclictest2714573-21latency_hist07:15:010
2709477993027,2cyclictest2864280-21kworker/u8:2+events_unbound09:40:251
2709477993027,2cyclictest2745493-21kworker/u8:1+events_unbound07:55:231
2709475993026,3cyclictest2917080-21kworker/u8:0+events_unbound11:00:010
2709475993026,3cyclictest2719945-21kworker/u8:0+events_unbound07:45:020
2709480992924,3cyclictest2876076-21/usr/sbin/munin09:50:152
2709480992923,4cyclictest2740338-21latency_hist07:40:012
2709477992926,2cyclictest3020864-21kworker/u8:0+events_unbound12:18:251
2709477992926,2cyclictest2906774-21kworker/u8:2+events_unbound10:25:121
2709477992926,2cyclictest2839412-21kworker/u8:2+flush-259:009:15:171
2709477992926,2cyclictest2676993-21kworker/u8:1+events_unbound07:25:221
2709475992924,4cyclictest3020864-21kworker/u8:0+flush-259:012:15:000
2709485992816,10cyclictest2917320-21apt-get10:30:103
2709480992823,4cyclictest2792827-21/usr/sbin/munin08:30:192
2709477992825,2cyclictest2901539-21kworker/u8:1+events_unbound10:40:241
2709477992825,2cyclictest2781274-21kworker/u8:3+events_unbound08:34:261
2709477992825,2cyclictest2756600-21kworker/u8:2+events_unbound08:20:221
2709477992825,2cyclictest2651091-21kworker/u8:3+events_unbound07:35:221
2709477992824,3cyclictest2891920-21kworker/u8:3+events_unbound10:16:241
2709477992824,3cyclictest2719945-21kworker/u8:0+events_unbound08:13:261
2709485992721,4cyclictest3041253-21latency_hist12:30:013
2709480992714,11cyclictest2874986-21ntpq09:45:232
2709477992724,3cyclictest2745493-21kworker/u8:1+events_unbound07:50:131
2709477992723,3cyclictest2870690-21kworker/u8:3+flush-259:009:50:251
2709477992723,3cyclictest2870690-21kworker/u8:3+events_unbound09:57:251
2709477992723,3cyclictest2807131-21kworker/u8:0+events_unbound09:00:121
2709475992723,3cyclictest2781274-21kworker/u8:3+flush-259:008:30:240
2709480992620,4cyclictest335557-21systemd-journal07:44:592
2709477992623,2cyclictest3015494-21kworker/u8:3+events_unbound12:25:171
2709477992622,3cyclictest2891920-21kworker/u8:3+events_unbound10:07:271
2709477992622,3cyclictest2807634-21kworker/u8:4+events_unbound08:50:231
2709475992622,3cyclictest2891920-21kworker/u8:3+events_unbound10:05:250
2709475992622,3cyclictest2745493-21kworker/u8:1+flush-259:007:54:510
2709475992622,3cyclictest2719945-21kworker/u8:0+flush-259:008:05:210
2709485992520,3cyclictest2876101-21apt-get09:50:123
2709485992519,4cyclictest335557-21systemd-journal10:22:243
2709485992518,5cyclictest335557-21systemd-journal09:00:003
2709477992522,2cyclictest3015494-21kworker/u8:3+events_unbound12:05:231
2709477992522,2cyclictest2917080-21kworker/u8:0+events_unbound11:21:251
2709477992522,2cyclictest2813236-21kworker/u8:1+events_unbound08:55:121
2709477992521,3cyclictest3015494-21kworker/u8:3+events_unbound12:24:261
2709475992519,4cyclictest335557-21systemd-journal12:25:000
2709485992420,3cyclictest2989501-21apt-get11:40:113
2709480992420,3cyclictest3046444-21apt-get12:35:002
2709480992412,10cyclictest3046968-21cat12:35:012
2709477992420,3cyclictest3005331-21kworker/u8:2+flush-259:011:55:211
2709477992420,3cyclictest3005331-21kworker/u8:2+flush-259:011:55:201
2709477992420,3cyclictest2891920-21kworker/u8:3+events_unbound10:45:131
2709475992421,2cyclictest2917080-21kworker/u8:0+flush-259:010:30:230
2709475992420,3cyclictest2973780-21kworker/u8:1+events_unbound11:40:230
2709480992319,3cyclictest2973780-21kworker/u8:1+flush-259:012:25:012
2709480992319,3cyclictest2891920-21kworker/u8:3+flush-259:011:00:002
2709477992319,3cyclictest2781274-21kworker/u8:3+events_unbound08:40:151
2709485992215,6cyclictest0-21swapper/310:15:003
270948599221,20cyclictest3051620-21sshd12:35:253
2709480992219,2cyclictest3005121-21kworker/u8:0+flush-259:012:00:012
2709480992219,2cyclictest3005121-21kworker/u8:0+flush-259:012:00:002
2709480992218,3cyclictest2973808-21kworker/u8:2+flush-259:011:39:552
2709475992219,2cyclictest3047096-21kworker/u8:4+events_unbound12:40:000
2709475992219,2cyclictest2917080-21kworker/u8:0+flush-259:011:10:180
2709475992218,3cyclictest3015494-21kworker/u8:3+events_unbound12:10:000
2709475992217,4cyclictest2756600-21kworker/u8:2+flush-259:008:20:210
270948599218,2cyclictest2860276-21cpuspeed_turbos09:35:113
2709485992118,2cyclictest2719980-21apt-get07:20:123
2709477992118,2cyclictest2973808-21kworker/u8:2+events_unbound11:30:131
2709477992117,3cyclictest2807131-21kworker/u8:0+events_unbound09:05:181
2709475992118,2cyclictest2719945-21kworker/u8:0+events_unbound08:10:220
2709475992117,3cyclictest2973780-21kworker/u8:1+events_unbound11:26:260
2709475992117,3cyclictest2651091-21kworker/u8:3+flush-259:007:40:230
2709477992017,2cyclictest2891920-21kworker/u8:3+events_unbound10:10:131
2709477992017,2cyclictest2864280-21kworker/u8:2+events_unbound10:00:141
2709477992017,2cyclictest2807131-21kworker/u8:0+events_unbound09:10:171
2709477992017,2cyclictest2745493-21kworker/u8:1+events_unbound07:45:191
2709477992016,3cyclictest3041522-21apt-get12:30:121
2709475992017,2cyclictest2891920-21kworker/u8:3+flush-259:010:15:160
2709475992017,2cyclictest2864280-21kworker/u8:2+events_unbound09:40:230
2709475992015,4cyclictest2714998-21diskstats07:15:140
2709485991915,3cyclictest2886361-21kworker/u8:1+flush-259:010:00:263
2709485991913,5cyclictest509-21rs:main3
2709480991915,3cyclictest2901539-21kworker/u8:1+flush-259:010:40:242
2709477991916,2cyclictest2813236-21kworker/u8:1+events_unbound09:30:121
2709477991916,2cyclictest2813236-21kworker/u8:1+events_unbound09:30:121
2709475991916,2cyclictest2891920-21kworker/u8:3+flush-259:010:25:210
2709475991915,3cyclictest3005331-21kworker/u8:2+flush-259:011:55:160
2709475991915,3cyclictest3005331-21kworker/u8:2+flush-259:011:55:150
2709475991915,3cyclictest2942827-21kworker/u8:4+events_unbound11:23:240
2709475991915,3cyclictest2891920-21kworker/u8:3+events_unbound10:47:260
2709475991915,3cyclictest2870690-21kworker/u8:3+events_unbound09:58:260
2709475991913,4cyclictest3046444-21apt-get12:35:010
270948599187,2cyclictest17650irq/131-nvme0q410:40:003
2709485991815,2cyclictest2870690-21kworker/u8:3+flush-259:009:45:143
2709485991814,3cyclictest2719945-21kworker/u8:0+flush-259:008:10:153
2709485991814,3cyclictest2719945-21kworker/u8:0+flush-259:007:40:003
2709485991813,3cyclictest2937628-21cron10:50:003
2709480991816,1cyclictest0-21swapper/209:00:002
2709480991815,1cyclictest0-21swapper/210:40:012
2709480991814,2cyclictest2798417-21irqcore08:35:162
2709480991813,3cyclictest2958500-21/usr/sbin/munin11:10:152
2709477991814,3cyclictest2745493-21kworker/u8:1+events_unbound08:35:081
2709475991814,3cyclictest2901539-21kworker/u8:1+events_unbound10:40:130
2709475991814,3cyclictest2807131-21kworker/u8:0+events_unbound09:37:260
2709475991814,2cyclictest509-21rs:main0
2709475991813,4cyclictest2891920-21kworker/u8:3+flush-259:010:20:200
2709485991713,3cyclictest2864280-21kworker/u8:2+flush-259:009:40:143
2709485991713,3cyclictest2745653-21cat07:45:013
270948599170,1cyclictest29965912cat11:45:193
2709477991711,4cyclictest2994916-21base6411:45:001
2709475991713,3cyclictest2973780-21kworker/u8:1+flush-259:011:35:200
2709475991713,3cyclictest2870690-21kworker/u8:3+events_unbound09:50:150
2709475991713,3cyclictest2807634-21kworker/u8:4+flush-259:008:45:220
2709475991713,3cyclictest2745493-21kworker/u8:1+events_unbound07:55:210
2709475991713,3cyclictest2740326-21idleruntime-cro07:39:590
2709485991613,2cyclictest3043232-21kthreadcore12:30:173
2709485991612,3cyclictest2824219-21apt-get09:00:123
2709480991612,3cyclictest2860186-21/usr/sbin/munin09:35:122
2709475991613,2cyclictest60350irq/132-enp0s29f1:rx-011:30:210
2709475991613,2cyclictest60350irq/132-enp0s29f1:rx-009:22:060
2709475991613,2cyclictest60350irq/132-enp0s29f1:rx-009:01:340
2709475991613,2cyclictest60350irq/132-enp0s29f1:rx-008:59:560
2709475991613,2cyclictest2917080-21kworker/u8:0+events_unbound10:35:190
2709475991613,2cyclictest2839412-21kworker/u8:2+events_unbound09:25:180
2709475991612,3cyclictest2973780-21kworker/u8:1+events_unbound12:15:240
2709475991611,4cyclictest3015494-21kworker/u8:3+events_unbound12:25:130
2709485991512,2cyclictest3025831-21idleruntime-cro12:15:013
2709485991511,3cyclictest2834253-21latency_hist09:10:003
2709485991511,3cyclictest2781274-21kworker/u8:3+flush-259:008:35:163
2709480991512,2cyclictest0-21swapper/209:20:002
2709480991511,3cyclictest335557-21systemd-journal09:14:552
2709480991511,3cyclictest2808288-21apt-get08:45:112
2709480991510,4cyclictest335557-21systemd-journal09:59:242
2709477991512,2cyclictest2995030-21kworker/u8:3+events_unbound11:50:151
2709475991513,1cyclictest60350irq/132-enp0s29f1:rx-010:00:240
2709475991513,1cyclictest60350irq/132-enp0s29f1:rx-007:23:380
2709475991512,2cyclictest60350irq/132-enp0s29f1:rx-012:00:220
2709475991512,2cyclictest60350irq/132-enp0s29f1:rx-010:10:150
2709475991512,2cyclictest60350irq/132-enp0s29f1:rx-009:45:220
2709475991511,3cyclictest2813236-21kworker/u8:1+events_unbound09:30:220
2709475991511,3cyclictest2813236-21kworker/u8:1+events_unbound09:30:220
2709475991511,3cyclictest2781274-21kworker/u8:3+flush-259:008:35:240
2709475991511,3cyclictest2745493-21kworker/u8:1+events_unbound08:00:160
2709475991511,2cyclictest2942089-21sshd10:50:240
2709475991510,4cyclictest2745493-21kworker/u8:1+flush-259:008:40:180
2709485991412,1cyclictest0-21swapper/310:10:003
2709485991412,1cyclictest0-21swapper/309:35:003
2709485991412,1cyclictest0-21swapper/309:35:003
2709485991410,3cyclictest2973808-21kworker/u8:2+flush-259:011:30:003
270948099149,4cyclictest2993477-21ntpq11:40:212
2709480991412,1cyclictest0-21swapper/209:30:002
2709480991411,2cyclictest2769414-21tr08:05:172
2709475991410,3cyclictest2807131-21kworker/u8:0+flush-259:009:10:300
2709475991410,3cyclictest2719945-21kworker/u8:0+flush-259:008:15:160
270948599139,3cyclictest2803103-21/usr/sbin/munin08:40:133
270948599139,3cyclictest2719945-21kworker/u8:0+flush-259:007:30:263
270948599138,4cyclictest2912157-21/usr/sbin/munin10:25:143
270948599135,3cyclictest491ktimers/310:55:113
2709485991311,1cyclictest3010995-21irqcore12:00:143
2709485991311,1cyclictest0-21swapper/308:50:003
2709485991311,1cyclictest0-21swapper/308:05:153
2709485991310,2cyclictest2713645-21ntp_kernel_err07:10:193
2709485991310,2cyclictest0-21swapper/308:05:003
270948099139,3cyclictest3020871-21/usr/sbin/munin12:10:182
270948099139,3cyclictest2751460-21gpgv07:50:012
270948099139,3cyclictest2651091-21kworker/u8:3+flush-259:007:25:002
2709480991311,1cyclictest2905753-21needreboot10:15:202
2709480991310,2cyclictest2856090-21kthreadcore09:30:162
2709480991310,2cyclictest2856090-21kthreadcore09:30:162
2709480991310,2cyclictest2814095-21fschecks_count08:50:142
2709480991310,2cyclictest2778429-21kthreadcore08:15:192
2709480991310,2cyclictest2772695-21irqcore08:10:152
270947799139,3cyclictest2781274-21kworker/u8:3+events_unbound08:19:231
270947599139,3cyclictest2963812-21tr11:15:130
2709475991311,1cyclictest60350irq/132-enp0s29f1:rx-011:53:140
270948599129,2cyclictest2957685-21cat11:05:253
270948599129,2cyclictest2817924-21ntp_kernel_pll_08:50:223
270948599129,2cyclictest2781274-21kworker/u8:3+flush-259:008:25:173
270948599128,3cyclictest3004687-21unixbench_singl11:50:253
270948599126,4cyclictest335557-21systemd-journal11:05:003
2709485991210,1cyclictest0-21swapper/312:00:003
2709485991210,1cyclictest0-21swapper/311:59:593
270948099129,2cyclictest2978520-21unixbench_singl11:25:252
270948099129,2cyclictest2953451-21cpuspeed_turbos11:05:122
270948099126,4cyclictest2828219-21ntpq09:00:222
270948099121,3cyclictest2870662-21awk09:45:012
2709480991210,1cyclictest0-21swapper/210:45:172
2709475991210,1cyclictest60350irq/132-enp0s29f1:rx-007:25:270
270948599119,1cyclictest0-21swapper/309:15:273
270948599118,2cyclictest2793874-21kthreadcore08:30:173
270948599118,2cyclictest2715034-21fschecks_count07:15:133
270948599116,4cyclictest3031269-21apt-get12:20:123
270948599116,3cyclictest2781212-21memory08:15:203
270948599115,4cyclictest513-21gdbus07:55:003
270948099118,2cyclictest3036840-21sort12:25:162
270948099118,2cyclictest2907426-21irqcore10:20:192
270948099118,2cyclictest2782618-21cpuspeed_turbos08:20:132
270948099118,2cyclictest2730342-21cpuspeed_turbos07:30:132
270948099118,2cyclictest2714139-21unixbench_singl07:10:242
270948099118,1cyclictest335557-21systemd-journal10:15:002
270948099116,3cyclictest335557-21systemd-journal11:20:192
270948099115,4cyclictest335557-21systemd-journal07:20:002
270948599108,1cyclictest0-21swapper/310:55:013
270948599108,1cyclictest0-21swapper/309:20:193
270948599107,2cyclictest2834625-21cpuspeed_turbos09:10:123
270948599107,1cyclictest0-21swapper/312:10:003
270948599106,2cyclictest2761708-21awk08:00:003
270948599100,2cyclictest2730009-21awk07:30:003
270948099107,2cyclictest0-21swapper/210:00:192
270948099106,3cyclictest2973780-21kworker/u8:1+flush-259:012:00:232
270947599100,9cyclictest2709479-21latency_hist07:10:010
27094859995,3cyclictest3030084-21ld12:15:213
27094809997,1cyclictest0-21swapper/208:25:202
27094809995,3cyclictest2844876-21cut09:20:132
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional