You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-12 - 06:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Sun Apr 12, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2501555995045,3cyclictest2553260-21kworker/u8:4+flush-259:022:35:172
2501550994843,4cyclictest252-21systemd-journal20:40:000
2501560994339,3cyclictest2761093-21kworker/u8:0+flush-259:000:10:013
2501552994339,3cyclictest2761093-21kworker/u8:0+flush-259:000:00:261
2501550994225,4cyclictest2843830-21latency_hist00:40:010
2501560994135,4cyclictest2735382-21latency_hist22:55:003
2501550994036,3cyclictest2615662-21kworker/u8:2+flush-259:021:30:240
2501560993935,3cyclictest2345198-21kworker/u8:0+flush-259:019:10:153
2501555993934,4cyclictest2615662-21kworker/u8:2+flush-259:022:30:242
2501555993933,4cyclictest2522148-21latency_hist19:30:012
2501550993936,2cyclictest2797564-21kworker/u8:2+flush-259:000:25:240
2501550993832,4cyclictest2678354-21latency_hist22:00:000
2501550993527,6cyclictest2791532-21ntpq23:45:220
250155099351,32cyclictest1-21systemd23:55:000
2501560993430,3cyclictest2553260-21kworker/u8:4+flush-259:021:10:173
2501560993429,3cyclictest252-21systemd-journal20:04:533
2501552993429,4cyclictest2715043-21apt-get22:35:121
2501560993329,3cyclictest2615662-21kworker/u8:2+flush-259:022:45:173
2501555993329,3cyclictest2615662-21kworker/u8:2+flush-259:022:20:162
2501555993326,5cyclictest2537587-21latency_hist19:45:012
2501552993328,3cyclictest2740736-21/usr/sbin/munin23:00:141
2501552993327,4cyclictest2698879-21apt-get22:20:001
2501550993327,4cyclictest2673193-21latency_hist21:55:000
2501560993227,3cyclictest2797564-21kworker/u8:2+flush-259:000:25:173
2501555993225,5cyclictest2637008-21apt-get21:20:122
2501552993229,2cyclictest2553260-21kworker/u8:4+flush-259:021:40:131
2501552993227,3cyclictest2589806-21apt-get20:35:121
2501550993227,3cyclictest2553260-21kworker/u8:4+flush-259:022:00:150
2501550993226,5cyclictest2777345-21diskstats23:35:130
2501555993126,4cyclictest2652938-21apt-get21:35:122
2501550993125,4cyclictest2573678-21latency_hist20:20:010
2501560993024,4cyclictest2818107-21latency_hist00:15:003
2501552993026,3cyclictest2553260-21kworker/u8:4+flush-259:021:00:261
2501552993025,3cyclictest2459833-21kworker/u8:3+flush-259:019:20:201
2501552993024,4cyclictest2605624-21apt-get20:50:121
2501560992926,2cyclictest2553260-21kworker/u8:4+flush-259:020:40:163
2501560992925,3cyclictest2459833-21kworker/u8:3+flush-259:019:15:153
2501552992925,3cyclictest2553260-21kworker/u8:4+flush-259:022:10:191
2501550992925,3cyclictest2797564-21kworker/u8:2+flush-259:000:10:150
2501550992925,3cyclictest2553260-21kworker/u8:4+flush-259:020:45:180
2501555992824,3cyclictest2698915-21gpgv22:20:002
2501550992820,6cyclictest2521241-21ntpq19:25:210
2501560992722,3cyclictest2610785-21apt-get20:55:123
2501560992720,5cyclictest2699577-21apt-get22:20:113
2501552992725,1cyclictest2720147-21kworker/u8:1+events_unbound23:20:141
2501552992723,3cyclictest2615662-21kworker/u8:2+events_unbound21:05:231
2501552992723,2cyclictest2615662-21kworker/u8:2+flush-259:022:05:191
2501550992724,2cyclictest2761093-21kworker/u8:0+flush-259:000:05:160
2501550992723,3cyclictest2345198-21kworker/u8:0+flush-259:019:35:200
2501552992622,3cyclictest2761093-21kworker/u8:0+events_unbound00:05:181
2501552992621,4cyclictest2553260-21kworker/u8:4+flush-259:020:40:181
2501560992522,2cyclictest2345198-21kworker/u8:0+flush-259:019:35:233
2501555992521,3cyclictest2537854-21apt-get19:45:122
2501552992521,3cyclictest2720147-21kworker/u8:1+flush-259:022:40:241
2501552992520,4cyclictest2615662-21kworker/u8:2+flush-259:021:45:191
2501552992519,3cyclictest2459833-21kworker/u8:3+flush-259:019:25:171
2501560992420,3cyclictest2553260-21kworker/u8:4+flush-259:022:15:173
2501555992418,4cyclictest2579246-21latency_hist20:25:012
2501552992422,1cyclictest2615662-21kworker/u8:2+events_unbound21:30:211
2501552992421,2cyclictest2720147-21kworker/u8:1+flush-259:023:10:131
2501552992421,2cyclictest2615662-21kworker/u8:2+flush-259:022:20:241
2501552992421,2cyclictest2615662-21kworker/u8:2+flush-259:021:20:211
2501552992420,3cyclictest2573722-21kworker/u8:1+events_unbound20:30:241
2501552992420,3cyclictest2553260-21kworker/u8:4+events_unbound21:50:181
2501552992420,3cyclictest2553260-21kworker/u8:4+events_unbound21:10:141
2501550992418,4cyclictest2699308-21gpgv22:20:010
2501550992417,5cyclictest2692863-21ntpq22:10:210
2501550992417,5cyclictest2641925-21latency_hist21:25:010
2501550992417,5cyclictest252-21systemd-journal19:20:000
2501560992319,3cyclictest2797564-21kworker/u8:2+flush-259:000:00:003
2501560992319,3cyclictest2553260-21kworker/u8:4+flush-259:020:45:243
2501560992319,3cyclictest2542967-21/usr/sbin/munin19:50:143
2501560992317,4cyclictest2668054-21latency_hist21:50:013
2501555992318,4cyclictest252-21systemd-journal19:32:202
2501555992317,4cyclictest2558469-21apt-get20:05:122
2501552992320,2cyclictest2553260-21kworker/u8:4+flush-259:020:55:251
2501552992320,2cyclictest2459833-21kworker/u8:3+events_unbound20:25:201
2501552992319,3cyclictest2761093-21kworker/u8:0+events_unbound23:45:151
2501552992319,3cyclictest2615662-21kworker/u8:2+events_unbound22:30:531
2501552992317,4cyclictest252-21systemd-journal23:54:531
2501552992316,5cyclictest2542733-21latency_hist19:50:001
2501560992218,3cyclictest2678621-21apt-get22:00:113
2501560992216,4cyclictest252-21systemd-journal21:30:533
2501555992217,4cyclictest2765753-21ntpq23:20:212
2501552992220,1cyclictest2761093-21kworker/u8:0+events_unbound00:35:221
2501552992219,2cyclictest2636738-21sh21:20:001
2501552992219,2cyclictest2345198-21kworker/u8:0+flush-259:019:10:231
2501552992218,3cyclictest2720147-21kworker/u8:1+flush-259:023:30:151
2501552992218,3cyclictest2720147-21kworker/u8:1+flush-259:023:25:231
2501552992218,3cyclictest2615662-21kworker/u8:2+events_unbound22:59:531
2501552992218,3cyclictest2459833-21kworker/u8:3+events_unbound20:10:011
2501552992216,4cyclictest2704712-21/usr/sbin/munin22:25:121
2501550992216,4cyclictest2833943-21diskstats00:30:160
2501550992216,4cyclictest2662069-21ntpq21:40:220
2501560992117,3cyclictest507-21in:imuxsock23:15:013
2501560992116,4cyclictest2797564-21kworker/u8:2+flush-259:000:30:233
2501560992114,5cyclictest509-21rs:main3
2501560992114,5cyclictest252-21systemd-journal23:35:003
2501555992117,3cyclictest2720147-21kworker/u8:1+flush-259:023:05:252
2501555992117,3cyclictest2720147-21kworker/u8:1+flush-259:023:05:252
2501555992117,3cyclictest2553260-21kworker/u8:4+flush-259:020:15:002
2501552992118,2cyclictest507-21in:imuxsock19:45:001
2501552992118,2cyclictest2553260-21kworker/u8:4+events_unbound20:10:211
2501552992118,2cyclictest2345198-21kworker/u8:0+events_unbound19:55:161
2501552992117,3cyclictest2761093-21kworker/u8:0+events_unbound00:15:541
2501552992117,3cyclictest2725367-21apt-get22:45:131
2501552992115,4cyclictest252-21systemd-journal00:20:531
2501560992017,2cyclictest2459833-21kworker/u8:3+flush-259:020:30:003
2501560992016,3cyclictest2720147-21kworker/u8:1+flush-259:023:05:273
2501560992016,3cyclictest2720147-21kworker/u8:1+flush-259:023:05:273
2501560992016,3cyclictest2459833-21kworker/u8:3+flush-259:019:25:253
2501552992017,2cyclictest2553260-21kworker/u8:4+flush-259:020:45:271
2501552992017,2cyclictest2553260-21kworker/u8:4+events_unbound21:55:141
250155299201,17cyclictest1-21systemd20:04:591
2501560991913,5cyclictest2574342-21/usr/sbin/munin20:20:123
2501560991913,4cyclictest2642493-21gpgv21:25:013
250155599199,9cyclictest2682631-21ntp_kernel_pll_22:00:212
2501555991915,3cyclictest2750989-21/usr/sbin/munin23:10:142
2501555991910,8cyclictest2843345-21unixbench_multi00:35:252
2501552991916,2cyclictest2761093-21kworker/u8:0+events_unbound23:55:251
2501552991916,2cyclictest2553260-21kworker/u8:4+events_unbound20:20:211
2501552991915,3cyclictest2761093-21kworker/u8:0+events_unbound00:11:531
2501552991915,3cyclictest2345198-21kworker/u8:0+flush-259:019:30:231
2501552991914,3cyclictest2553260-21kworker/u8:4+flush-259:021:25:531
2501552991912,5cyclictest252-21systemd-journal00:30:001
2501550991916,2cyclictest2459833-21kworker/u8:3+flush-259:019:40:160
2501550991914,4cyclictest2605501-21gpgv20:50:010
2501550991913,4cyclictest2630668-21ntpq21:10:210
2501560991815,2cyclictest2762154-21irqcore23:20:163
2501560991814,3cyclictest2720147-21kworker/u8:1+flush-259:023:45:003
2501560991814,3cyclictest2459833-21kworker/u8:3+flush-259:020:15:193
2501560991812,4cyclictest2761093-21kworker/u8:0+flush-259:000:15:163
2501555991814,3cyclictest2573722-21kworker/u8:1+events_unbound20:30:192
2501555991813,4cyclictest2506700-21kworker/u8:1+flush-259:019:20:152
2501555991813,3cyclictest2766880-21/usr/sbin/munin23:25:202
2501555991812,5cyclictest0-21swapper/220:19:592
2501555991812,4cyclictest252-21systemd-journal22:06:532
2501555991811,5cyclictest2615662-21kworker/u8:2+flush-259:021:09:242
250155599180,4cyclictest2677572-21/usr/sbin/munin21:55:232
250155599180,1cyclictest0-21swapper/220:50:142
2501552991815,2cyclictest2797564-21kworker/u8:2+events_unbound00:30:291
2501552991815,2cyclictest2776942-21kworker/u8:3+events_unbound23:42:191
2501552991813,4cyclictest2459833-21kworker/u8:3+flush-259:019:15:201
2501552991812,5cyclictest2777184-21apt-get23:35:121
2501560991711,4cyclictest2652913-21/usr/sbin/munin21:35:203
2501560991711,4cyclictest2615662-21kworker/u8:2+flush-259:022:15:003
250155599179,7cyclictest0-21swapper/200:20:012
250155599174,11cyclictest513-21gdbus22:45:002
2501555991713,3cyclictest2720147-21kworker/u8:1+flush-259:023:30:232
2501555991713,3cyclictest2531578-21ntp_kernel_pll_19:35:192
2501555991711,4cyclictest252-21systemd-journal23:47:482
250155599171,10cyclictest2818110-21munin-plugin-st00:15:002
250155599170,15cyclictest2668732-21grep21:50:162
2501552991713,3cyclictest2553260-21kworker/u8:4+events_unbound22:04:531
2501550991714,2cyclictest60350irq/132-enp0s29f1:rx-020:55:130
2501550991712,4cyclictest2626226-21store21:10:010
2501560991611,3cyclictest2615662-21kworker/u8:2+flush-259:023:15:183
250155599166,9cyclictest2547867-21latency_hist19:55:012
250155599164,10cyclictest2778782-21tr23:35:172
250155599162,11cyclictest0-21swapper/223:20:002
250155599161,13cyclictest2693775-21basename22:15:002
250155599161,13cyclictest2663569-21sed21:45:172
250155599161,12cyclictest0-21swapper/220:45:182
2501555991610,5cyclictest2619936-21ntpq21:00:212
2501552991613,2cyclictest2720147-21kworker/u8:1+events_unbound22:52:461
2501552991612,3cyclictest2459833-21kworker/u8:3+flush-259:019:50:211
250155099169,5cyclictest252-21systemd-journal19:55:000
2501550991613,2cyclictest60350irq/132-enp0s29f1:rx-023:40:140
2501550991613,2cyclictest60350irq/132-enp0s29f1:rx-022:50:230
2501550991613,2cyclictest60350irq/132-enp0s29f1:rx-020:30:230
2501550991613,2cyclictest60350irq/132-enp0s29f1:rx-000:00:480
2501550991613,2cyclictest2567576-21needreboot20:10:200
2501550991610,5cyclictest2646663-21ntpq21:25:220
2501550991610,4cyclictest2688628-21latency_hist22:10:000
2501560991512,2cyclictest2673199-21grep21:55:003
2501560991510,4cyclictest2720147-21kworker/u8:1+flush-259:022:40:253
250155599151,12cyclictest2797516-21apt23:55:012
250155599151,12cyclictest2725780-21cut22:45:172
250155599151,12cyclictest2662886-21cron21:45:002
250155599151,11cyclictest2705161-21irqcore22:25:142
250155599150,7cyclictest2646626-21ntpq21:25:222
250155599150,5cyclictest2734538-21/usr/sbin/munin22:50:222
250155599150,4cyclictest2807089-21/usr/sbin/munin00:00:242
250155599150,3cyclictest2798137-21if_enp0s29f123:55:172
250155599150,2cyclictest2745662-21munin-run23:05:012
250155599150,13cyclictest2812944-21awk00:10:012
250155599150,13cyclictest2651711-21head21:30:202
250155599150,13cyclictest2631144-21unixbench_singl21:10:252
250155599150,12cyclictest2833925-21grep00:30:162
250155599150,12cyclictest2782631-21cut23:40:142
250155599150,12cyclictest2735756-21grep22:55:132
250155299159,4cyclictest2459833-21kworker/u8:3+flush-259:020:15:191
2501552991511,3cyclictest2345198-21kworker/u8:0+flush-259:019:35:181
250155099159,5cyclictest2823475-21apt-get00:20:100
2501550991513,1cyclictest60350irq/132-enp0s29f1:rx-023:55:250
2501550991513,1cyclictest60350irq/132-enp0s29f1:rx-023:24:280
2501550991513,1cyclictest60350irq/132-enp0s29f1:rx-023:00:070
2501550991513,1cyclictest60350irq/132-enp0s29f1:rx-022:45:150
2501550991513,1cyclictest60350irq/132-enp0s29f1:rx-022:41:030
2501550991513,1cyclictest60350irq/132-enp0s29f1:rx-020:20:210
2501550991513,1cyclictest60350irq/132-enp0s29f1:rx-020:07:400
2501550991513,1cyclictest60350irq/132-enp0s29f1:rx-020:00:190
2501550991512,2cyclictest60350irq/132-enp0s29f1:rx-023:00:010
2501550991512,2cyclictest60350irq/132-enp0s29f1:rx-022:30:180
2501550991512,2cyclictest60350irq/132-enp0s29f1:rx-020:25:120
2501550991512,2cyclictest60350irq/132-enp0s29f1:rx-019:34:250
2501550991511,3cyclictest2553260-21kworker/u8:4+flush-259:021:16:260
250156099149,4cyclictest2673547-21cut21:55:133
2501560991410,2cyclictest0-21swapper/322:35:183
2501560991410,2cyclictest0-21swapper/320:15:003
250155599141,2cyclictest2548135-21apt-get19:55:132
250155599141,12cyclictest2590212-21ps20:35:172
250155599141,11cyclictest2635862-21ntpq21:15:232
250155599140,5cyclictest2507128-21grep19:15:132
250155599140,4cyclictest2583451-21/usr/sbin/munin20:25:212
250155599140,2cyclictest2828721-21tr00:25:132
250155599140,2cyclictest2557315-21ntpq20:00:202
250155599140,12cyclictest2506229-21unixbench_singl19:10:242
250155599140,11cyclictest2594999-21tr20:40:132
2501552991410,3cyclictest2615662-21kworker/u8:2+flush-259:023:15:251
2501550991412,1cyclictest60350irq/132-enp0s29f1:rx-023:30:270
2501550991412,1cyclictest60350irq/132-enp0s29f1:rx-023:15:200
2501550991412,1cyclictest60350irq/132-enp0s29f1:rx-022:22:050
2501550991412,1cyclictest60350irq/132-enp0s29f1:rx-021:45:170
2501550991412,1cyclictest60350irq/132-enp0s29f1:rx-020:40:100
2501550991412,1cyclictest60350irq/132-enp0s29f1:rx-000:15:170
2501550991411,2cyclictest60350irq/132-enp0s29f1:rx-023:25:240
2501550991411,2cyclictest60350irq/132-enp0s29f1:rx-022:35:130
2501550991410,3cyclictest60350irq/132-enp0s29f1:rx-021:35:210
2501550991410,3cyclictest2553260-21kworker/u8:4+flush-259:021:00:170
250156099138,4cyclictest2761093-21kworker/u8:0+flush-259:023:45:313
250156099138,2cyclictest0-21swapper/321:25:013
2501560991310,2cyclictest2560761-21kthreadcore20:05:173
2501560991310,2cyclictest2533136-21sed19:40:163
250155599131,1cyclictest0-21swapper/219:10:002
250155599130,3cyclictest2823734-21fschecks_time00:20:122
250155299139,3cyclictest2553260-21kworker/u8:4+flush-259:021:35:241
2501550991312,1cyclictest60350irq/132-enp0s29f1:rx-023:06:080
2501550991312,1cyclictest60350irq/132-enp0s29f1:rx-023:06:070
2501550991311,1cyclictest60350irq/132-enp0s29f1:rx-022:25:230
2501550991310,2cyclictest60350irq/132-enp0s29f1:rx-019:58:260
250156099128,2cyclictest2634036-21kthreadcore21:15:193
250156099127,4cyclictest2589526-21cron20:35:003
250155299129,2cyclictest2720147-21kworker/u8:1+flush-259:023:05:271
250155299129,2cyclictest2720147-21kworker/u8:1+flush-259:023:05:261
2501550991210,1cyclictest60350irq/132-enp0s29f1:rx-023:14:160
250156099118,2cyclictest2739692-21grep22:55:223
250156099118,2cyclictest2548240-21cpuspeed_turbos19:55:143
250156099117,2cyclictest2688628-21latency_hist22:10:003
250156099106,2cyclictest0-21swapper/320:50:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional