You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-19 - 06:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Fri Jun 19, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2237621994034,4cyclictest2555000-21latency_hist00:15:003
2237612993831,5cyclictest2528707-21cat23:50:010
2237617993734,2cyclictest2424743-21kworker/u8:4+events_unbound22:11:472
2237612993629,5cyclictest2538569-21ntpq23:55:210
2237617993532,2cyclictest2560203-21kworker/u8:3+events_unbound00:29:112
2237614993531,3cyclictest2450440-21kworker/u8:5+flush-259:022:40:001
2237612993529,5cyclictest2382727-21latency_hist21:30:000
2237617993431,2cyclictest2263053-21kworker/u8:1+events_unbound19:47:102
2237614993429,3cyclictest2570500-21latency_hist00:30:011
2237617993330,2cyclictest2237790-21kworker/u8:0+events_unbound20:08:122
2237617993330,2cyclictest2232423-21kworker/u8:3+events_unbound20:32:102
2237612993327,4cyclictest2507701-21latency_hist23:30:000
2237617993228,3cyclictest2351811-21kworker/u8:3+events_unbound21:32:222
2237614993230,1cyclictest2242414-21fwupd19:11:091
2237614993229,2cyclictest2424534-21kworker/u8:0+flush-259:022:50:001
2237612993225,5cyclictest2549012-21ntpq00:05:230
2237612993225,5cyclictest2485678-21ntpq23:05:220
2237614993026,3cyclictest2330675-21kworker/u8:1+events_unbound20:50:181
2237614993026,3cyclictest2232423-21kworker/u8:3+flush-259:020:05:001
2237617992824,3cyclictest2351811-21kworker/u8:3+events_unbound21:49:572
2237617992824,3cyclictest2253694-21kworker/u8:2+events_unbound21:42:112
2237614992822,4cyclictest509-21rs:main1
2237614992722,3cyclictest335557-21systemd-journal20:56:101
2237612992719,6cyclictest2428784-21ntpq22:10:210
2237617992623,3cyclictest2435092-21kworker/u8:2+events_unbound23:54:132
2237614992622,3cyclictest2465920-21kworker/u8:3+flush-259:023:05:261
2237614992621,3cyclictest2388154-21apt-get21:35:111
2237614992620,4cyclictest335557-21systemd-journal00:04:111
2237617992522,2cyclictest2435092-21kworker/u8:2+events_unbound22:24:552
2237617992522,2cyclictest2253694-21kworker/u8:2+events_unbound22:01:442
2237617992522,2cyclictest2232423-21kworker/u8:3+events_unbound20:44:562
2237617992520,3cyclictest335557-21systemd-journal21:03:112
2237617992519,5cyclictest335557-21systemd-journal22:30:002
2237614992521,3cyclictest2465920-21kworker/u8:3+flush-259:022:50:251
2237614992519,4cyclictest2414208-21latency_hist22:00:011
2237612992521,3cyclictest2382081-21kworker/u8:4+flush-259:021:45:000
2237612992518,5cyclictest2496073-21ntpq23:15:210
2237621992419,3cyclictest2440314-21apt-get22:25:113
223762199241,21cyclictest1-21systemd22:15:003
2237617992421,2cyclictest2435092-21kworker/u8:2+events_unbound23:44:552
2237617992420,3cyclictest2424534-21kworker/u8:0+events_unbound23:34:562
2237614992420,3cyclictest2253694-21kworker/u8:2+flush-259:022:05:001
2237614992420,3cyclictest2232423-21kworker/u8:3+flush-259:020:40:001
2237612992420,3cyclictest2435092-21kworker/u8:2+events_unbound22:20:190
2237612992420,3cyclictest2232424-21kworker/u8:4+events_unbound19:11:090
2237612992420,3cyclictest2232423-21kworker/u8:3+flush-259:019:20:200
223761299241,22cyclictest1-21systemd20:56:100
223762199239,12cyclictest2247907-21dump-pmu-power19:20:003
2237621992317,5cyclictest509-21rs:main3
2237621992317,4cyclictest2549860-21latency_hist00:10:013
2237617992320,2cyclictest2295056-21kworker/u8:1+events_unbound20:23:582
2237612992319,3cyclictest2279526-21kworker/u8:4+events_unbound20:00:230
2237612992318,3cyclictest2435092-21kworker/u8:2+flush-259:023:20:170
2237612992318,3cyclictest2351811-21kworker/u8:3+events_unbound21:35:220
2237621992218,3cyclictest2971-21cron00:30:003
2237621992216,4cyclictest2481423-21latency_hist23:05:013
2237621992215,5cyclictest2518011-21latency_hist23:40:003
2237617992220,2cyclictest2253694-21kworker/u8:2+events_unbound21:08:582
2237617992219,2cyclictest2424534-21kworker/u8:0+events_unbound23:45:442
2237617992219,2cyclictest2351811-21kworker/u8:3+events_unbound21:29:552
2237617992219,2cyclictest2247922-21kworker/u8:1+events_unbound19:22:012
2237617992218,3cyclictest2435092-21kworker/u8:2+events_unbound00:19:222
2237617992218,3cyclictest2435092-21kworker/u8:2+events_unbound00:19:212
2237612992218,3cyclictest2403538-21kworker/u8:1+flush-259:021:55:000
2237621992112,8cyclictest0-21swapper/323:50:003
2237617992118,2cyclictest2486813-21kworker/u8:4+events_unbound23:23:512
2237617992118,2cyclictest2435092-21kworker/u8:2+events_unbound00:11:342
2237617992118,2cyclictest2330675-21kworker/u8:1+events_unbound20:59:592
2237617992118,2cyclictest2279526-21kworker/u8:4+events_unbound19:55:562
2237617992118,2cyclictest2253694-21kworker/u8:2+events_unbound19:27:222
2237617992118,2cyclictest2232423-21kworker/u8:3+events_unbound20:38:122
2237617992116,4cyclictest2232423-21kworker/u8:3+flush-259:020:17:122
2237614992118,2cyclictest2527948-21kworker/u8:1+flush-259:023:55:101
2237614992117,3cyclictest2486850-21apt-get23:10:111
223762199208,10cyclictest0-21swapper/320:05:253
2237621992017,2cyclictest2476265-21sh23:00:003
2237621992016,3cyclictest2232423-21kworker/u8:3+flush-259:020:35:173
2237617992017,2cyclictest2435092-21kworker/u8:2+events_unbound00:30:012
2237617992017,2cyclictest2336444-21kworker/u8:0+events_unbound20:46:162
2237617992017,2cyclictest2253694-21kworker/u8:2+events_unbound22:07:252
2237617992016,3cyclictest2424534-21kworker/u8:0+flush-259:023:15:252
2237614992017,2cyclictest2253694-21kworker/u8:2+flush-259:021:05:261
2237614992016,3cyclictest2554761-21kworker/u8:0+flush-259:000:15:281
2237614992016,3cyclictest2554761-21kworker/u8:0+flush-259:000:15:271
2237612992017,2cyclictest2424743-21kworker/u8:4+events_unbound22:15:130
2237612992016,3cyclictest2435092-21kworker/u8:2+flush-259:023:00:180
223762199199,8cyclictest0-21swapper/319:35:013
223762199199,8cyclictest0-21swapper/319:35:013
2237621991915,3cyclictest335557-21systemd-journal00:18:113
2237621991915,3cyclictest335557-21systemd-journal00:18:103
2237617991916,2cyclictest2534548-21kworker/u8:3+events_unbound00:00:162
2237617991916,2cyclictest2527948-21kworker/u8:1+events_unbound23:59:082
2237617991916,2cyclictest2424534-21kworker/u8:0+events_unbound22:33:272
2237617991916,2cyclictest2403538-21kworker/u8:1+events_unbound21:57:402
2237617991916,2cyclictest2351811-21kworker/u8:3+events_unbound21:53:302
2237617991916,2cyclictest2253694-21kworker/u8:2+events_unbound21:16:592
2237617991915,3cyclictest2237790-21kworker/u8:0+events_unbound19:43:532
2237617991915,3cyclictest2237790-21kworker/u8:0+events_unbound19:43:522
2237614991914,4cyclictest2475384-21ntpq22:55:221
2237614991912,5cyclictest335557-21systemd-journal00:25:001
2237614991912,5cyclictest2459822-21ntpq22:40:201
2237612991915,3cyclictest2253694-21kworker/u8:2+events_unbound20:25:210
2237612991915,3cyclictest2237790-21kworker/u8:0+events_unbound19:55:140
2237612991914,4cyclictest2419640-21apt-get22:05:110
2237621991815,2cyclictest2491739-21cron23:15:003
223762199181,4cyclictest2377580-21dump-pmu-power21:24:593
2237621991812,4cyclictest2340504-21ntpq20:45:233
2237621991811,5cyclictest335557-21systemd-journal23:53:103
2237617991816,2cyclictest2424534-21kworker/u8:0+events_unbound22:15:272
2237617991816,2cyclictest2403538-21kworker/u8:1+events_unbound22:40:512
2237617991815,3cyclictest2575901-21kworker/u8:0+events_unbound00:39:592
2237617991815,2cyclictest2527948-21kworker/u8:1+events_unbound00:08:342
2237617991815,2cyclictest2486345-21kworker/u8:1+events_unbound23:09:592
2237617991815,2cyclictest2461003-21apt-get22:45:102
2237617991815,2cyclictest2424534-21kworker/u8:0+events_unbound23:01:272
2237617991815,2cyclictest2382081-21kworker/u8:4+events_unbound21:35:382
2237617991815,2cyclictest2351811-21kworker/u8:3+events_unbound21:12:472
2237617991815,2cyclictest2330675-21kworker/u8:1+events_unbound21:20:042
2237617991815,2cyclictest2295056-21kworker/u8:1+events_unbound20:28:062
2237617991815,2cyclictest2279526-21kworker/u8:4+events_unbound20:01:212
2237617991815,2cyclictest2263053-21kworker/u8:1+events_unbound19:31:532
2237617991815,2cyclictest2263053-21kworker/u8:1+events_unbound19:31:522
2237617991815,2cyclictest2253694-21kworker/u8:2+events_unbound19:51:082
2237617991815,2cyclictest2232423-21kworker/u8:3+events_unbound20:11:572
2237612991814,3cyclictest2336444-21kworker/u8:0+events_unbound20:45:190
2237612991814,3cyclictest2232423-21kworker/u8:3+flush-259:020:15:170
2237612991813,4cyclictest2232423-21kworker/u8:3+flush-259:020:30:160
223762199177,9cyclictest0-21swapper/321:55:123
223762199171,5cyclictest335557-21systemd-journal23:20:003
223762199171,5cyclictest335557-21systemd-journal21:05:013
2237621991712,3cyclictest2424534-21kworker/u8:0+flush-259:022:55:003
2237621991710,4cyclictest2242745-21latency_hist19:15:003
2237617991714,2cyclictest2512882-21kworker/u8:3+events_unbound23:37:182
2237617991714,2cyclictest2465920-21kworker/u8:3+events_unbound22:50:332
2237617991714,2cyclictest2435092-21kworker/u8:2+events_unbound22:59:512
2237617991714,2cyclictest2435092-21kworker/u8:2+events_unbound22:37:572
2237617991714,2cyclictest2424534-21kworker/u8:0+events_unbound23:26:592
2237617991714,2cyclictest2424534-21kworker/u8:0+events_unbound23:11:212
2237617991714,2cyclictest2237790-21kworker/u8:0+events_unbound19:37:342
2237617991714,2cyclictest2232424-21kworker/u8:4+events_unbound19:17:362
2237617991713,3cyclictest2435092-21kworker/u8:2+events_unbound00:24:342
2237614991714,2cyclictest2274133-21idleruntime-cro19:45:001
2237614991714,2cyclictest2274133-21idleruntime-cro19:44:591
2237614991713,3cyclictest2289682-21cut19:59:591
2237614991712,4cyclictest2367520-21apt-get21:15:101
2237614991710,5cyclictest335557-21systemd-journal21:25:001
2237612991714,2cyclictest60350irq/132-enp0s29f1:rx-022:25:170
2237612991714,2cyclictest2560203-21kworker/u8:3+flush-259:000:30:210
2237612991713,3cyclictest2560203-21kworker/u8:3+events_unbound00:20:230
2237612991713,3cyclictest2330675-21kworker/u8:1+flush-259:020:50:170
2237612991713,3cyclictest2232423-21kworker/u8:3+flush-259:019:25:160
2237612991712,4cyclictest2529020-21apt-get23:50:120
2237612991711,4cyclictest2253694-21kworker/u8:2+flush-259:021:10:210
223762199166,1cyclictest0-21swapper/323:05:153
223762199164,2cyclictest2575587-21sshd00:34:113
223762199161,5cyclictest335557-21systemd-journal19:55:003
223762199161,4cyclictest2575942-21seq00:35:123
223762199161,4cyclictest2424518-21apt22:10:003
223762199161,4cyclictest2310399-21awk20:20:003
2237621991613,2cyclictest2336210-21basename20:45:003
2237621991611,3cyclictest2450659-21/usr/sbin/munin22:35:153
2237621991611,3cyclictest2435105-21/usr/sbin/munin22:20:153
2237621991610,4cyclictest2320975-21/usr/sbin/munin20:30:153
223762199160,4cyclictest2367931-21sort21:15:163
2237617991613,2cyclictest2253694-21kworker/u8:2+events_unbound20:51:242
2237614991613,2cyclictest2445242-21idleruntime-cro22:30:001
2237614991613,2cyclictest2429684-21cron22:15:001
2237614991613,2cyclictest2398692-21munin-run21:45:001
2237614991613,2cyclictest2258200-21idleruntime-cro19:30:001
2237614991613,2cyclictest0-21swapper/123:55:001
2237612991614,1cyclictest60350irq/132-enp0s29f1:rx-019:15:240
2237612991613,2cyclictest60350irq/132-enp0s29f1:rx-022:30:210
2237612991613,2cyclictest60350irq/132-enp0s29f1:rx-020:23:420
2237612991612,3cyclictest2295056-21kworker/u8:1+flush-259:020:05:200
223762199153,3cyclictest2304386-21ntpq20:10:203
223762199151,4cyclictest2539064-21unixbench_singl23:55:253
223762199151,4cyclictest2205578-21munin-node20:00:103
223762199151,3cyclictest0-21swapper/322:20:003
2237621991511,3cyclictest2409059-21cron21:55:003
223762199150,4cyclictest2459932-21tr22:40:223
2237617991512,2cyclictest2237790-21kworker/u8:0+events_unbound19:11:492
223761499159,4cyclictest335557-21systemd-journal22:10:001
2237614991512,2cyclictest0-21swapper/120:45:011
2237614991510,4cyclictest2429964-21apt-get22:15:111
2237614991510,4cyclictest2263367-21apt-get19:35:011
2237612991513,1cyclictest60350irq/132-enp0s29f1:rx-023:41:160
2237612991513,1cyclictest60350irq/132-enp0s29f1:rx-022:40:150
2237612991513,1cyclictest60350irq/132-enp0s29f1:rx-021:05:230
2237612991513,1cyclictest60350irq/132-enp0s29f1:rx-020:40:100
2237612991513,1cyclictest60350irq/132-enp0s29f1:rx-019:46:080
2237612991513,1cyclictest60350irq/132-enp0s29f1:rx-019:35:260
2237612991513,1cyclictest60350irq/132-enp0s29f1:rx-000:15:260
2237612991513,1cyclictest60350irq/132-enp0s29f1:rx-000:15:260
2237612991513,1cyclictest0-21swapper/023:15:000
2237612991512,2cyclictest61150irq/133-enp0s29f1:tx-021:00:230
2237612991512,2cyclictest60350irq/132-enp0s29f1:rx-023:30:110
2237612991512,2cyclictest60350irq/132-enp0s29f1:rx-021:59:490
2237612991512,2cyclictest60350irq/132-enp0s29f1:rx-021:20:190
2237612991512,2cyclictest60350irq/132-enp0s29f1:rx-019:35:000
2237612991512,2cyclictest60350irq/132-enp0s29f1:rx-019:35:000
2237612991512,2cyclictest60350irq/132-enp0s29f1:rx-000:35:130
2237612991511,3cyclictest2560203-21kworker/u8:3+flush-259:000:25:170
2237612991511,3cyclictest2351811-21kworker/u8:3+events_unbound21:45:200
2237612991511,3cyclictest2232423-21kworker/u8:3+flush-259:020:10:210
223762199149,4cyclictest335557-21systemd-journal20:27:073
223762199148,4cyclictest2253694-21kworker/u8:2+flush-259:021:10:153
223762199145,2cyclictest2361190-21ntpq21:05:213
223762199141,6cyclictest0-21swapper/323:25:163
223762199141,4cyclictest2462021-21sed22:45:163
2237614991412,1cyclictest0-21swapper/123:39:591
2237614991412,1cyclictest0-21swapper/122:25:001
2237614991410,2cyclictest311ktimers/121:45:121
223761299149,4cyclictest2232423-21kworker/u8:3+events_unbound19:50:190
223761299148,4cyclictest335557-21systemd-journal22:02:100
223761299148,4cyclictest2527948-21kworker/u8:1+flush-259:000:01:150
2237612991412,1cyclictest60350irq/132-enp0s29f1:rx-021:15:240
2237612991412,1cyclictest60350irq/132-enp0s29f1:rx-020:35:160
2237612991411,2cyclictest60350irq/132-enp0s29f1:rx-000:10:080
2237612991411,2cyclictest2451182-21irqcore22:35:170
2237612991411,2cyclictest2382081-21kworker/u8:4+events_unbound21:30:250
2237612991410,3cyclictest2465920-21kworker/u8:3+flush-259:022:50:240
223762199137,4cyclictest2424534-21kworker/u8:0+flush-259:023:40:173
223762199134,3cyclictest2343550-21cut20:50:183
223762199132,3cyclictest2274409-21apt-get19:45:103
2237621991310,2cyclictest335557-21systemd-journal19:24:553
2237621991310,2cyclictest2562761-21tr00:20:193
2237612991311,2cyclictest60350irq/132-enp0s29f1:rx-019:40:410
2237612991311,2cyclictest60350irq/132-enp0s29f1:rx-019:40:400
2237612991310,2cyclictest2518012-21latency_hist23:39:590
223762199129,2cyclictest0-21swapper/320:25:003
223762199125,5cyclictest335557-21systemd-journal23:34:593
223762199120,10cyclictest2544658-21cron00:05:003
223761499129,2cyclictest2305225-21turbostat.cron20:15:001
223761499128,2cyclictest2496044-21ntpq23:15:201
2237612991210,1cyclictest60350irq/132-enp0s29f1:rx-022:57:010
2237612991210,1cyclictest60350irq/132-enp0s29f1:rx-022:45:140
223762199118,2cyclictest0-21swapper/321:45:133
223762199110,4cyclictest2393765-21apt-get21:40:113
223761499119,1cyclictest0-21swapper/122:35:001
223761499118,2cyclictest2336904-21sed20:45:181
223761499117,2cyclictest2310412-21munin-plugin-st20:20:001
223761499117,2cyclictest2277395-21kthreadcore19:45:171
223761499110,10cyclictest1-21systemd23:50:001
223762199105,3cyclictest2271528-21sed19:40:173
223762199105,3cyclictest2271528-21sed19:40:173
223761499106,3cyclictest311ktimers/119:15:121
223761499100,8cyclictest2295766-21irqcore20:05:171
22376149997,1cyclictest0-21swapper/121:30:171
22376149984,3cyclictest2250973-21cut19:20:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional