You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-22 - 20:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Wed Jul 22, 2026 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2428996995042,6cyclictest2559310-21latency_hist09:15:010
2429001994137,3cyclictest2720062-21kworker/u8:4+flush-259:012:05:001
2428996994137,3cyclictest2756115-21kworker/u8:0+flush-259:012:35:200
2428996994035,3cyclictest2548257-21kworker/u8:4+events_unbound09:25:000
2429006993832,4cyclictest2683238-21latency_hist11:15:013
2428996993835,2cyclictest2512335-21kworker/u8:1+flush-259:008:50:010
2429003993733,3cyclictest2626144-21kworker/u8:2+flush-259:010:25:002
2428996993632,3cyclictest2600602-21kworker/u8:3+events_unbound10:33:190
2429003993428,4cyclictest2725039-21latency_hist11:55:002
2429001993429,3cyclictest2460760-21apt-get07:40:131
2429001993328,3cyclictest335557-21systemd-journal08:05:261
2429003993226,4cyclictest335557-21systemd-journal12:00:002
2429006993125,4cyclictest509-21rs:main3
2428996993128,2cyclictest2626144-21kworker/u8:2+flush-259:010:45:210
2428996993025,3cyclictest2590250-21kworker/u8:1+events_unbound09:50:200
2428996993024,4cyclictest2677180-21ntpq11:05:210
2429006992923,4cyclictest2683926-21apt-get11:15:123
2429003992925,3cyclictest2574636-21kworker/u8:2+flush-259:009:40:162
2428996992926,2cyclictest2460725-21kworker/u8:1+events_unbound07:50:230
2428996992926,2cyclictest2460725-21kworker/u8:1+events_unbound07:50:230
2428996992925,3cyclictest2559544-21kworker/u8:0+events_unbound09:15:130
2429006992821,5cyclictest2694026-21latency_hist11:25:013
2429003992824,3cyclictest2413324-21kworker/u8:0+flush-259:007:24:592
2429001992824,3cyclictest335557-21systemd-journal11:20:251
2428996992824,3cyclictest2657402-21kworker/u8:0+events_unbound11:00:260
2428996992824,3cyclictest2512335-21kworker/u8:1+flush-259:008:50:210
2428996992824,2cyclictest2720062-21kworker/u8:4+events_unbound12:00:200
2428996992823,4cyclictest335557-21systemd-journal08:35:010
2429003992723,3cyclictest2662785-21kworker/u8:1+flush-259:011:40:162
2428996992724,2cyclictest2657402-21kworker/u8:0+events_unbound10:50:170
2428996992724,2cyclictest2413324-21kworker/u8:0+events_unbound07:25:200
2428996992723,3cyclictest2714920-21apt-get11:45:120
2428996992721,5cyclictest2626406-21latency_hist10:20:010
2428996992623,2cyclictest2481290-21kworker/u8:2+events_unbound08:05:000
2428996992623,2cyclictest2449761-21kworker/u8:3+events_unbound07:30:130
2428996992622,3cyclictest2418559-21kworker/u8:2+events_unbound07:15:010
2429006992520,3cyclictest2548257-21kworker/u8:4+flush-259:009:25:003
2429001992522,2cyclictest2616334-21apt-get10:10:101
2429001992521,3cyclictest2548257-21kworker/u8:4+flush-259:009:10:151
2429001992520,3cyclictest1607760-21fwupd10:43:511
2428996992521,3cyclictest2720062-21kworker/u8:4+events_unbound11:55:220
2428996992521,3cyclictest2559544-21kworker/u8:0+events_unbound09:25:250
2429006992421,2cyclictest509-21rs:main3
2429006992419,4cyclictest2512335-21kworker/u8:1+flush-259:008:53:273
2429006992319,3cyclictest2413324-21kworker/u8:0+flush-259:007:14:593
2429003992317,4cyclictest2766376-21latency_hist12:35:002
2428996992320,2cyclictest2600602-21kworker/u8:3+events_unbound10:00:230
2428996992320,2cyclictest2460725-21kworker/u8:1+flush-259:008:05:210
2428996992319,3cyclictest2418559-21kworker/u8:2+events_unbound07:15:180
2428996992317,5cyclictest2687956-21ntpq11:15:210
2429003992218,3cyclictest2730438-21kworker/u8:3+flush-259:012:25:002
2429003992218,3cyclictest2626144-21kworker/u8:2+flush-259:010:40:202
2428996992219,2cyclictest2662785-21kworker/u8:1+flush-259:011:40:150
2428996992219,2cyclictest2600602-21kworker/u8:3+events_unbound10:25:130
2428996992219,2cyclictest2559544-21kworker/u8:0+events_unbound09:35:190
2428996992219,2cyclictest2512335-21kworker/u8:1+events_unbound09:05:150
2428996992219,2cyclictest2455361-21kworker/u8:2+flush-259:007:41:210
2429006992117,3cyclictest335557-21systemd-journal12:15:003
2429006992117,3cyclictest2486711-21apt-get08:05:133
242900699211,18cyclictest2756102-21awk12:25:003
2429003992116,4cyclictest335557-21systemd-journal07:47:522
2429001992117,3cyclictest2673122-21/usr/sbin/munin11:05:221
2429001992116,3cyclictest2756076-21cron12:25:001
2428996992118,2cyclictest2689049-21kworker/u8:4+events_unbound11:24:270
2428996992118,2cyclictest2522825-21kworker/u8:2+events_unbound08:55:240
2428996992118,2cyclictest2413324-21kworker/u8:0+events_unbound07:22:250
2428996992117,3cyclictest2626144-21kworker/u8:2+flush-259:010:23:250
242900699209,9cyclictest509-21rs:main3
2429001992017,2cyclictest2559544-21kworker/u8:0+flush-259:010:04:591
2429001992015,4cyclictest2683452-21store11:15:011
2429001992015,4cyclictest2439626-21apt-get07:20:101
2429001992012,6cyclictest335557-21systemd-journal11:35:001
2428996992017,2cyclictest2522825-21kworker/u8:2+flush-259:009:00:130
2428996992017,2cyclictest2496373-21kworker/u8:0+events_unbound08:25:240
2428996992016,3cyclictest2559544-21kworker/u8:0+events_unbound10:08:270
2428996992016,3cyclictest2559544-21kworker/u8:0+events_unbound10:08:270
2428996992014,4cyclictest2605787-21latency_hist10:00:010
2428996992014,4cyclictest2579129-21ntpq09:30:220
2428996992013,5cyclictest2760363-21ntpq12:25:200
2429006991916,2cyclictest2413324-21kworker/u8:0+flush-259:007:25:163
2429006991915,2cyclictest0-21swapper/309:10:013
2429006991913,4cyclictest2616076-21latency_hist10:10:013
2429006991913,4cyclictest2616076-21latency_hist10:10:003
2429006991913,4cyclictest2554370-21apt-get09:10:093
2429006991911,7cyclictest2465406-21kworker/u8:4+flush-259:008:25:013
2429003991916,2cyclictest2662785-21kworker/u8:1+flush-259:011:40:002
2429001991913,4cyclictest2745738-21latency_hist12:15:001
2428996991916,2cyclictest2730438-21kworker/u8:3+events_unbound12:30:160
2428996991916,2cyclictest2730226-21kworker/u8:1+events_unbound12:10:180
2428996991916,2cyclictest2465406-21kworker/u8:4+events_unbound08:24:250
2428996991916,2cyclictest2465406-21kworker/u8:4+events_unbound08:15:030
2428996991912,5cyclictest2470058-21ntpq07:45:220
242900699188,9cyclictest0-21swapper/311:55:003
2429006991814,3cyclictest2657402-21kworker/u8:0+flush-259:010:59:263
2429006991814,3cyclictest2481290-21kworker/u8:2+flush-259:008:04:553
2429006991813,4cyclictest2720062-21kworker/u8:4+flush-259:012:35:013
242900699180,8cyclictest2523380-21sed08:40:153
2429003991816,1cyclictest0-21swapper/208:45:002
2429003991814,3cyclictest2657402-21kworker/u8:0+flush-259:010:50:252
2429001991815,2cyclictest2626144-21kworker/u8:2+flush-259:010:45:161
2429001991814,3cyclictest2522825-21kworker/u8:2+flush-259:009:00:381
2429001991813,4cyclictest2621510-21apt-get10:15:121
2428996991815,2cyclictest2678292-21kworker/u8:0+events_unbound11:25:230
2428996991814,3cyclictest2600602-21kworker/u8:3+events_unbound11:10:200
2428996991812,4cyclictest509-21rs:main0
242899699180,17cyclictest2595459-21unin-run09:49:590
242900699174,3cyclictest2704485-21cat11:35:003
2429006991712,4cyclictest2455361-21kworker/u8:2+flush-259:007:39:593
2429003991714,2cyclictest2600602-21kworker/u8:3+flush-259:009:55:182
2429003991713,3cyclictest2678292-21kworker/u8:0+flush-259:011:20:172
2429003991713,3cyclictest2460725-21kworker/u8:1+flush-259:008:05:182
2429003991710,5cyclictest335557-21systemd-journal11:15:012
242900199177,8cyclictest0-21swapper/112:20:001
2429001991714,2cyclictest2664244-21kthreadcore10:55:181
2429001991713,3cyclictest2646109-21ntp_kernel_err10:35:201
2429001991713,3cyclictest2496373-21kworker/u8:0+flush-259:008:25:261
2429001991712,4cyclictest2522825-21kworker/u8:2+flush-259:008:45:001
2429001991712,4cyclictest2496998-21/usr/sbin/munin08:15:231
2429001991711,5cyclictest335557-21systemd-journal07:52:251
2429001991711,5cyclictest335557-21systemd-journal07:52:241
2429001991711,4cyclictest335557-21systemd-journal10:26:251
2429001991711,4cyclictest2719830-21latency_hist11:50:001
2429001991711,4cyclictest2600877-21apt-get09:55:131
2428996991714,2cyclictest2465406-21kworker/u8:4+events_unbound08:36:260
2428996991713,3cyclictest2574636-21kworker/u8:2+events_unbound09:40:140
242900699164,3cyclictest26409532sleep310:30:203
242900699164,3cyclictest2563830-21sshd09:15:253
242900699164,2cyclictest2709988-21fschecks_time11:40:143
242900699160,6cyclictest832-21runrttasks11:45:123
242900699160,5cyclictest2740585-21apt12:10:003
242900699160,5cyclictest2532985-21grep08:50:013
242900699160,4cyclictest2616738-21irqcore10:10:163
2429003991612,3cyclictest2626144-21kworker/u8:2+flush-259:010:40:002
242900399161,13cyclictest1-21systemd08:00:002
242900199169,5cyclictest335557-21systemd-journal08:25:011
242900199169,5cyclictest335557-21systemd-journal08:02:251
2429001991612,3cyclictest2460725-21kworker/u8:1+flush-259:008:10:271
2429001991612,2cyclictest0-21swapper/107:30:161
2428996991613,2cyclictest60350irq/132-enp0s29f1:rx-012:20:200
2428996991613,2cyclictest60350irq/132-enp0s29f1:rx-011:50:180
2428996991613,2cyclictest60350irq/132-enp0s29f1:rx-011:30:230
2428996991613,2cyclictest2626144-21kworker/u8:2+flush-259:010:43:500
2428996991613,2cyclictest2626144-21kworker/u8:2+flush-259:010:35:220
2428996991613,2cyclictest2460725-21kworker/u8:1+events_unbound08:10:160
2428996991611,3cyclictest2455583-21apt-get07:35:120
242900699156,8cyclictest0-21swapper/310:35:253
242900699156,1cyclictest2661667-21ntpq10:50:213
242900699156,1cyclictest0-21swapper/310:40:263
242900699156,1cyclictest0-21swapper/309:54:593
2429006991512,2cyclictest2496312-21unixbench_singl08:10:263
2429006991511,2cyclictest0-21swapper/309:29:593
242900699150,5cyclictest335557-21systemd-journal07:55:003
242900699150,5cyclictest335557-21systemd-journal07:54:593
242900699150,4cyclictest2585130-21turbostat.cron09:40:003
242900699150,3cyclictest2678077-21idleruntime-cro11:10:003
2429003991511,3cyclictest2455361-21kworker/u8:2+flush-259:007:35:202
2429001991512,2cyclictest2552822-21ntp_kernel_pll_09:05:221
2429001991512,2cyclictest2465406-21kworker/u8:4+flush-259:008:35:271
242899699159,4cyclictest2523018-21diskstats08:40:130
2428996991512,2cyclictest60350irq/132-enp0s29f1:rx-011:39:430
2428996991512,2cyclictest2735674-21apt-get12:05:130
242900699149,4cyclictest2657376-21latency_hist10:50:013
242900699144,1cyclictest0-21swapper/309:35:003
242900699141,5cyclictest2734513-21ntpq12:00:223
2429006991411,2cyclictest2771986-21idleruntime-cro12:40:003
2429006991411,2cyclictest2729776-21unixbench_singl11:55:263
2429006991411,2cyclictest2517901-21fschecks_count08:35:153
242900699140,6cyclictest2454407-21ntpq07:30:203
242900399149,4cyclictest2538391-21apt-get08:55:132
242900399149,3cyclictest335557-21systemd-journal11:30:252
2429003991411,2cyclictest0-21swapper/209:20:172
2429001991410,2cyclictest2727568-21kthreadcore11:55:191
2428996991411,2cyclictest60350irq/132-enp0s29f1:rx-010:55:390
2428996991410,3cyclictest2616592-21fschecks_time10:10:140
242900699137,4cyclictest2595434-21latency_hist09:50:013
242900699135,1cyclictest2434423-21/usr/sbin/munin07:15:263
242900699135,1cyclictest0-21swapper/308:30:213
242900699133,3cyclictest2469396-21cut07:45:203
242900699130,5cyclictest2698362-21ntpq11:25:223
242900399139,3cyclictest2548257-21kworker/u8:4+flush-259:009:11:562
2429003991311,1cyclictest0-21swapper/209:50:002
2429003991311,1cyclictest0-21swapper/208:50:152
2429003991310,2cyclictest2747021-21tr12:15:192
242900199139,3cyclictest2418559-21kworker/u8:2+flush-259:007:15:181
242900199137,4cyclictest2449244-21ntpq07:25:221
2429001991310,2cyclictest2720164-21cpuspeed_turbos11:50:151
2429001991310,1cyclictest0-21swapper/110:54:591
242900699127,4cyclictest335557-21systemd-journal10:25:003
242900699126,4cyclictest2750921-21latency_hist12:20:003
242900699121,4cyclictest2501072-21ntpq08:15:203
242900399129,2cyclictest2662785-21kworker/u8:1+flush-259:011:15:162
242900399129,2cyclictest2564510-21latency_hist09:20:002
242900399129,2cyclictest2522128-21unixbench_multi08:35:252
242900399129,2cyclictest2429289-21cpuspeed_turbos07:10:122
242900399129,2cyclictest0-21swapper/209:54:592
242900399127,3cyclictest2580192-21/usr/sbin/munin09:35:152
2429003991210,1cyclictest2545480-21kthreadcore09:00:192
242900199129,2cyclictest2600602-21kworker/u8:3+events_unbound11:00:251
242900199129,2cyclictest2476560-21fschecks_count07:55:171
242900199129,2cyclictest0-21swapper/109:25:201
242900199126,4cyclictest335557-21systemd-journal11:15:031
242900199120,3cyclictest2766376-21latency_hist12:35:001
242900699116,3cyclictest2761241-21awk12:30:003
242900699113,3cyclictest2605023-21ps09:55:253
242900699110,4cyclictest2542417-21ntpq08:55:223
242900399118,2cyclictest2709522-21kworker/u8:2+flush-259:012:00:002
242900399117,3cyclictest335557-21systemd-journal10:35:002
242900399115,4cyclictest2667740-21cron11:00:002
242900399110,9cyclictest2672893-21awk11:05:002
242899699119,1cyclictest60350irq/132-enp0s29f1:rx-012:15:240
242900699105,3cyclictest2547540-21nfs4_client09:00:213
242900399107,2cyclictest2745298-21unixbench_singl12:10:242
242900399107,2cyclictest0-21swapper/207:15:222
242900399106,3cyclictest2559544-21kworker/u8:0+flush-259:010:05:212
242900399106,3cyclictest2559544-21kworker/u8:0+flush-259:010:05:212
242900399105,3cyclictest335557-21systemd-journal07:35:002
242900199108,1cyclictest0-21swapper/111:25:261
242900199108,1cyclictest0-21swapper/107:35:241
242900199106,2cyclictest2630603-21ntpq10:20:201
242900199106,2cyclictest0-21swapper/109:40:241
242900199102,6cyclictest1-21systemd12:10:001
24290069997,1cyclictest0-21swapper/311:00:193
24290039997,1cyclictest0-21swapper/211:45:142
24290039996,2cyclictest2616743-21sed10:10:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional