You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-10 - 06:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Fri Jul 10, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2199917994742,4cyclictest2157043-21kworker/u8:1+flush-259:019:35:002
2199911994644,1cyclictest2337750-21fwupd21:20:201
2199921994238,3cyclictest2329370-21kworker/u8:0+events_unbound21:20:233
2199917994034,4cyclictest2344588-21latency_hist21:30:012
2199921993734,2cyclictest2225599-21kworker/u8:3+flush-259:019:54:593
2199917993529,4cyclictest2422378-21latency_hist22:45:012
2199911993530,3cyclictest2240997-21latency_hist19:50:001
2199908993529,4cyclictest2463684-21latency_hist23:25:010
2199921993328,4cyclictest2215296-21latency_hist19:25:013
2199921993228,3cyclictest2407161-21/usr/sbin/munin22:30:253
2199911993228,3cyclictest335557-21systemd-journal21:01:291
2199917993124,4cyclictest2329140-21latency_hist21:15:002
2199911993125,4cyclictest2360331-21apt-get21:45:111
2199908993125,4cyclictest2494797-21cat23:55:010
2199921992925,3cyclictest2246381-21kworker/u8:1+flush-259:020:25:243
2199917992824,3cyclictest2338558-21kworker/u8:3+flush-259:021:45:022
2199917992823,3cyclictest2443079-21kworker/u8:3+flush-259:023:35:012
2199911992823,4cyclictest2250458-21ntpq19:55:191
2199911992621,3cyclictest2453632-21/usr/sbin/munin23:15:231
2199908992622,3cyclictest2157043-21kworker/u8:1+flush-259:019:26:010
2199908992622,3cyclictest2157043-21kworker/u8:1+flush-259:019:10:180
2199917992522,2cyclictest2157043-21kworker/u8:1+flush-259:019:40:162
2199917992521,3cyclictest2246381-21kworker/u8:1+flush-259:020:25:002
2199917992419,3cyclictest2236128-21apt-get19:45:112
2199917992418,4cyclictest2500230-21latency_hist00:00:012
2199908992320,2cyclictest2489496-21cron23:49:590
2199917992218,3cyclictest335557-21systemd-journal19:15:272
2199917992218,3cyclictest2401437-21gpgv22:25:002
2199917992217,3cyclictest335557-21systemd-journal23:06:002
2199908992219,2cyclictest2432710-21sh22:55:000
2199908992218,3cyclictest2474030-21kworker/u8:4+flush-259:000:31:060
2199908992218,3cyclictest2308823-21apt-get20:55:110
219990899221,19cyclictest1-21systemd21:01:290
2199921992118,2cyclictest2338558-21kworker/u8:3+flush-259:021:45:183
2199921992117,3cyclictest2338558-21kworker/u8:3+flush-259:021:40:253
2199921992115,5cyclictest2448446-21kworker/u8:1+flush-259:000:00:013
2199917992114,4cyclictest2541652-21latency_hist00:40:002
2199911992118,2cyclictest2494685-21kworker/u8:2+flush-259:000:05:271
2199911992118,2cyclictest2329370-21kworker/u8:0+flush-259:022:05:231
2199911992116,3cyclictest2437867-21latency_hist23:00:011
2199917992017,2cyclictest2338558-21kworker/u8:3+flush-259:021:35:102
2199911992016,3cyclictest2448446-21kworker/u8:1+flush-259:023:25:001
2199908992017,2cyclictest2329370-21kworker/u8:0+flush-259:022:15:000
2199908992017,2cyclictest2251320-21munin-run19:59:590
2199908992017,2cyclictest2225599-21kworker/u8:3+flush-259:020:10:170
2199908992016,3cyclictest2157043-21kworker/u8:1+flush-259:019:15:470
2199921991916,2cyclictest500-21systemd-logind22:30:003
2199921991916,2cyclictest2157043-21kworker/u8:1+events_unbound19:15:003
219991799193,7cyclictest2453394-21munin-run23:15:002
2199917991913,4cyclictest335557-21systemd-journal22:18:512
2199911991915,3cyclictest2422403-21kworker/u8:0+flush-259:022:50:241
2199911991915,3cyclictest2200145-21apt-get19:10:131
2199911991915,2cyclictest0-21swapper/120:29:591
2199911991914,4cyclictest2219572-21ntpq19:25:231
2199908991916,2cyclictest2458569-21kworker/u8:0+flush-259:023:30:160
2199908991915,3cyclictest2510835-21/usr/sbin/munin00:10:250
2199908991913,4cyclictest2531318-21latency_hist00:30:000
2199921991816,1cyclictest0-21swapper/321:55:003
2199921991813,4cyclictest2262012-21diskstats20:10:133
2199921991811,5cyclictest2308984-21diskstats20:55:123
2199917991813,4cyclictest2167711-21kworker/u8:0+flush-259:019:30:012
2199911991815,2cyclictest507-21in:imuxsock21:10:001
2199911991814,2cyclictest2267035-21apt-get20:15:111
2199908991815,2cyclictest2282961-21kworker/u8:4+flush-259:020:30:250
2199908991814,3cyclictest2324020-21kworker/u8:2+flush-259:021:15:290
2199908991812,5cyclictest2260698-21ntpq20:05:210
2199921991714,2cyclictest2167711-21kworker/u8:0+flush-259:019:35:163
2199921991712,3cyclictest509-21rs:main3
2199917991713,3cyclictest2448446-21kworker/u8:1+flush-259:023:25:252
2199917991713,3cyclictest2225599-21kworker/u8:3+flush-259:019:55:222
2199911991715,1cyclictest2396170-21idleruntime-cro22:19:591
2199911991714,2cyclictest2494684-21kworker/u8:0+flush-259:000:05:011
2199911991713,3cyclictest2448446-21kworker/u8:1+flush-259:023:25:181
2199911991712,3cyclictest2495337-21apt-get23:55:141
2199908991715,1cyclictest1-21systemd22:18:510
2199908991714,2cyclictest61150irq/133-enp0s29f1:tx-023:40:250
2199908991714,1cyclictest0-21swapper/019:45:000
2199908991712,4cyclictest2344850-21apt-get21:30:100
2199908991711,4cyclictest335557-21systemd-journal20:51:550
2199908991711,4cyclictest2379807-21ntpq22:00:200
2199921991613,2cyclictest2383261-21kthreadcore22:05:173
2199917991613,2cyclictest2417211-21cron22:40:002
2199917991611,4cyclictest507-21in:imuxsock22:55:002
219991199169,5cyclictest2391011-21latency_hist22:15:001
2199911991612,3cyclictest2417490-21apt-get22:40:121
2199911991612,3cyclictest2246381-21kworker/u8:1+flush-259:020:20:251
2199908991614,1cyclictest60350irq/132-enp0s29f1:rx-020:49:280
2199908991614,1cyclictest60350irq/132-enp0s29f1:rx-020:35:100
2199908991614,1cyclictest60350irq/132-enp0s29f1:rx-019:30:160
2199908991614,1cyclictest0-21swapper/021:55:010
2199908991613,2cyclictest61150irq/133-enp0s29f1:tx-021:20:200
2199908991613,2cyclictest60350irq/132-enp0s29f1:rx-020:20:130
2199908991612,3cyclictest2536696-21kworker/u8:0+flush-259:000:35:260
2199908991612,3cyclictest2276844-21kworker/u8:2+flush-259:020:25:170
2199921991512,2cyclictest2390167-21ntp_offset22:10:223
2199921991512,2cyclictest2371660-21kthreadcore21:55:183
2199921991511,3cyclictest2401322-21apt-get22:24:593
2199921991510,3cyclictest2323985-21cron21:10:013
2199917991513,1cyclictest0-21swapper/220:44:592
2199917991512,2cyclictest2225599-21kworker/u8:3+flush-259:020:10:112
2199917991510,4cyclictest2448446-21kworker/u8:1+flush-259:023:50:162
2199917991510,3cyclictest2251550-21/usr/sbin/munin20:00:202
2199911991513,1cyclictest2406919-21turbostat.cron22:30:001
2199911991510,4cyclictest2375500-21latency_hist22:00:001
2199908991514,1cyclictest60350irq/132-enp0s29f1:rx-023:00:110
2199908991513,1cyclictest60350irq/132-enp0s29f1:rx-023:25:200
2199908991513,1cyclictest60350irq/132-enp0s29f1:rx-022:40:510
2199908991513,1cyclictest60350irq/132-enp0s29f1:rx-022:25:260
2199908991513,1cyclictest60350irq/132-enp0s29f1:rx-022:08:040
2199908991513,1cyclictest60350irq/132-enp0s29f1:rx-021:35:260
2199908991513,1cyclictest60350irq/132-enp0s29f1:rx-020:41:360
2199908991513,1cyclictest60350irq/132-enp0s29f1:rx-000:05:270
2199908991513,1cyclictest60350irq/132-enp0s29f1:rx-000:00:010
2199908991512,2cyclictest60350irq/132-enp0s29f1:rx-000:00:120
2199908991511,3cyclictest2157043-21kworker/u8:1+flush-259:019:35:210
2199908991510,4cyclictest60350irq/132-enp0s29f1:rx-000:15:150
219990899150,2cyclictest2324013-21idleruntime-cro21:10:000
219992199149,4cyclictest2484773-21diskstats23:45:153
219992199148,5cyclictest335557-21systemd-journal22:40:003
2199921991412,1cyclictest0-21swapper/321:35:013
2199921991411,2cyclictest2406691-21kworker/u8:3+flush-259:022:50:163
2199921991411,2cyclictest2375544-21kworker/u8:2+flush-259:023:00:163
2199921991411,1cyclictest0-21swapper/322:40:183
219991799143,6cyclictest2505450-21idleruntime-cro00:05:002
219991799140,12cyclictest335557-21systemd-journal20:25:272
219991199149,3cyclictest2205256-21/usr/sbin/munin19:15:131
219991199148,4cyclictest2469127-21apt-get23:30:111
2199911991411,2cyclictest2416297-21ntp_kernel_err22:35:201
2199911991411,2cyclictest2246381-21kworker/u8:1+events_unbound20:00:241
2199911991410,3cyclictest2516453-21irqcore00:15:161
2199911991410,3cyclictest2338558-21kworker/u8:3+flush-259:021:36:121
2199911991410,3cyclictest2293294-21apt-get20:40:101
2199908991413,1cyclictest60350irq/132-enp0s29f1:rx-022:55:330
2199908991413,1cyclictest60350irq/132-enp0s29f1:rx-000:20:210
2199908991412,1cyclictest60350irq/132-enp0s29f1:rx-023:05:280
2199908991412,1cyclictest60350irq/132-enp0s29f1:rx-021:55:150
2199908991412,1cyclictest60350irq/132-enp0s29f1:rx-021:45:160
2199908991412,1cyclictest60350irq/132-enp0s29f1:rx-021:44:160
2199908991412,1cyclictest60350irq/132-enp0s29f1:rx-021:25:230
2199908991412,1cyclictest60350irq/132-enp0s29f1:rx-020:00:280
2199908991411,2cyclictest60350irq/132-enp0s29f1:rx-022:45:260
2199908991411,2cyclictest60350irq/132-enp0s29f1:rx-022:45:260
2199908991411,2cyclictest60350irq/132-enp0s29f1:rx-022:35:490
2199908991411,2cyclictest60350irq/132-enp0s29f1:rx-021:10:200
2199908991411,2cyclictest60350irq/132-enp0s29f1:rx-020:15:120
2199908991411,2cyclictest2443079-21kworker/u8:3+flush-259:023:15:000
219992199137,5cyclictest2526127-21latency_hist00:25:003
219992199136,5cyclictest335557-21systemd-journal00:10:013
219992199133,8cyclictest2453389-21latency_hist23:15:003
2199921991311,1cyclictest2422887-21grep22:45:163
2199921991311,1cyclictest2422887-21grep22:45:153
2199921991311,1cyclictest2234881-21needreboot19:40:193
219992199131,11cyclictest1-21systemd21:28:313
2199921991310,2cyclictest2222978-21tr19:30:183
2199921991310,1cyclictest0-21swapper/320:30:143
219991799138,3cyclictest2246157-21latency_hist19:55:002
2199917991310,2cyclictest2313271-21unixbench_singl20:55:242
219991199139,3cyclictest2494645-21apt-get23:55:001
219991199139,3cyclictest2443079-21kworker/u8:3+flush-259:000:10:261
219991199138,3cyclictest335557-21systemd-journal23:02:001
2199911991311,1cyclictest0-21swapper/120:14:591
2199911991310,2cyclictest2474030-21kworker/u8:4+flush-259:023:41:071
2199908991311,1cyclictest60350irq/132-enp0s29f1:rx-023:35:170
2199908991311,1cyclictest60350irq/132-enp0s29f1:rx-023:15:270
2199908991311,1cyclictest60350irq/132-enp0s29f1:rx-019:20:160
2199908991310,2cyclictest60350irq/132-enp0s29f1:rx-019:48:460
2199908991310,2cyclictest2245307-21ntp_offset19:50:220
219992199129,2cyclictest2526463-21cpuspeed_turbos00:25:163
219992199129,2cyclictest2443079-21kworker/u8:3+flush-259:023:05:183
219992199129,1cyclictest2465749-21kthreadcore23:25:183
219992199129,1cyclictest0-21swapper/323:50:183
219992199129,1cyclictest0-21swapper/320:35:183
219992199127,4cyclictest2380704-21latency_hist22:05:003
219992199127,3cyclictest2277124-21systemctl20:25:003
219992199126,4cyclictest490-21dbus-daemon22:20:003
219992199121,5cyclictest2109693-21snmpd00:00:243
2199921991210,1cyclictest2539050-21kthreadcore00:35:193
219991799129,2cyclictest2230211-21unixbench_multi19:35:242
219991799128,3cyclictest401ktimers/221:40:132
219991799128,3cyclictest2432373-21kworker/u8:1+flush-259:023:03:052
219991799128,3cyclictest2432373-21kworker/u8:1+flush-259:022:55:282
219991799128,3cyclictest2380969-21apt-get22:05:112
219991199129,2cyclictest2283061-21cpuspeed_turbos20:30:141
219991199129,2cyclictest2256755-21cpuspeed_turbos20:05:131
219991199129,2cyclictest0-21swapper/123:15:011
219991199125,5cyclictest2313710-21awk21:00:001
2199911991210,1cyclictest2524428-21kthreadcore00:20:191
2199908991211,1cyclictest60350irq/132-enp0s29f1:rx-022:23:440
219992199119,1cyclictest0-21swapper/323:25:013
219992199119,1cyclictest0-21swapper/321:15:213
219992199118,2cyclictest2535474-21needreboot00:30:203
219992199118,1cyclictest0-21swapper/320:40:163
219992199117,2cyclictest0-21swapper/320:15:153
219992199116,3cyclictest335557-21systemd-journal22:59:593
219992199116,3cyclictest211-21jbd2/nvme0n1p2-800:15:013
219992199111,2cyclictest2251312-21latency_hist20:00:013
219992199110,3cyclictest2210141-21latency_hist19:20:013
219991799118,2cyclictest2375868-21cpuspeed_turbos22:00:132
219991799115,4cyclictest2407333-21df22:30:142
219991799112,1cyclictest191rcu_preempt20:50:002
219991799111,1cyclictest0-21swapper/221:20:232
219991199119,1cyclictest0-21swapper/120:35:131
219991199115,4cyclictest2225573-21latency_hist19:35:011
219991199111,1cyclictest0-21swapper/122:25:001
219990899119,2cyclictest60350irq/132-enp0s29f1:rx-022:34:080
219992199107,2cyclictest2516110-21cpuspeed_turbos00:15:123
219992199107,2cyclictest2215628-21cpuspeed_turbos19:25:143
219992199105,4cyclictest2456853-21tr23:15:213
219992199105,3cyclictest2303805-21/usr/sbin/munin20:50:143
219992199104,4cyclictest2238957-21kthreadcore19:45:183
219992199100,8cyclictest2329140-21latency_hist21:15:003
219992199100,3cyclictest2474004-21latency_hist23:35:013
219991799108,1cyclictest0-21swapper/222:50:002
219991799108,1cyclictest0-21swapper/222:49:592
219991799108,1cyclictest0-21swapper/222:00:002
219991799107,2cyclictest2505816-21cpuspeed_turbos00:05:152
219991799107,2cyclictest2345108-21fschecks_time21:30:142
219991799106,3cyclictest2474030-21kworker/u8:4+flush-259:000:30:262
219991199100,8cyclictest2349740-21awk21:35:011
219991199100,8cyclictest0-21swapper/100:25:171
21999179996,1cyclictest2515782-21latency_hist00:15:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional