You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-18 - 11:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Wed Mar 18, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2239108993630,4cyclictest2270150-21apt-get19:39:592
2239111993428,4cyclictest252-21systemd-journal22:45:003
2239099993327,4cyclictest2473402-21diskstats22:55:130
2239108993224,5cyclictest2328136-21latency_hist20:35:002
2239099993128,2cyclictest2395421-21kworker/u8:3+flush-259:021:50:020
2239099993125,4cyclictest2338411-21latency_hist20:45:000
2239111993024,4cyclictest2519243-21ntpq23:35:213
2239103993027,2cyclictest2483726-21kworker/u8:3+flush-259:023:20:001
2239103993024,4cyclictest2249824-21/usr/sbin/munin19:20:131
2239103992924,4cyclictest2390504-21apt-get21:35:101
2239099992923,4cyclictest2375252-21diskstats21:20:120
2239108992822,4cyclictest2566590-21latency_hist00:25:012
2239103992825,2cyclictest2431844-21kworker/u8:0+flush-259:022:40:161
2239103992824,3cyclictest2541009-21kworker/u8:1+flush-259:000:00:151
2239108992723,3cyclictest2541009-21kworker/u8:1+flush-259:000:10:002
2239108992722,3cyclictest2478171-21gpgv23:00:002
2239103992621,3cyclictest252-21systemd-journal19:49:591
2239111992521,3cyclictest2280874-21kworker/u8:1+flush-259:020:00:003
2239099992520,4cyclictest2524394-21ntpq23:40:190
2239111992421,2cyclictest2328152-21kworker/u8:2+events_unbound20:55:183
2239103992420,3cyclictest2254747-21kworker/u8:0+flush-259:020:00:221
2239111992319,3cyclictest2416402-21kworker/u8:2+events_unbound22:35:133
2239103992319,2cyclictest0-21swapper/121:25:011
2239103992318,4cyclictest2244662-21/usr/sbin/munin19:15:151
2239103992317,4cyclictest2274838-21ntpq19:40:211
2239099992320,2cyclictest2291427-21kworker/u8:3+events_unbound20:05:140
2239099992319,3cyclictest2281090-21/usr/sbin/munin19:50:140
2239108992219,2cyclictest2566633-21kworker/u8:1+events_unbound00:25:182
2239108992215,5cyclictest2520174-21kworker/u8:3+flush-259:023:55:152
2239108992215,5cyclictest2254747-21kworker/u8:0+flush-259:021:20:162
2239103992219,2cyclictest2473022-21kworker/u8:1+flush-259:023:35:191
2239103992219,2cyclictest2328152-21kworker/u8:2+events_unbound20:50:181
2239103992218,3cyclictest2254747-21kworker/u8:0+flush-259:020:05:241
2239099992217,4cyclictest2468077-21/usr/sbin/munin22:50:170
2239099992216,4cyclictest2466988-21ntpq22:45:200
2239111992118,2cyclictest2254747-21kworker/u8:0+flush-259:020:30:163
2239108992117,3cyclictest2405630-21sh21:50:002
2239108992115,4cyclictest252-21systemd-journal21:50:332
2239108992115,4cyclictest2510102-21/usr/sbin/munin23:30:152
2239108992114,5cyclictest252-21systemd-journal22:05:002
2239103992118,2cyclictest2431844-21kworker/u8:0+events_unbound00:10:141
2239103992118,2cyclictest2082037-21kworker/u8:2+flush-259:019:50:151
2239103992117,3cyclictest2421764-21/usr/sbin/munin22:05:151
2239103992117,3cyclictest2369928-21kworker/u8:1+flush-259:021:55:141
2239099992118,2cyclictest2431844-21kworker/u8:0+events_unbound23:15:240
2239099992117,3cyclictest2572004-21kworker/u8:4+events_unbound00:30:210
2239099992117,3cyclictest2416402-21kworker/u8:2+events_unbound22:30:170
2239099992117,3cyclictest2416402-21kworker/u8:2+events_unbound22:30:160
2239099992115,5cyclictest2270180-21apt-key19:40:000
2239108992016,3cyclictest2254747-21kworker/u8:0+flush-259:020:15:352
2239108992014,4cyclictest2509864-21latency_hist23:30:012
2239103992017,2cyclictest1070830-21kworker/u8:4+flush-259:019:30:111
2239103992016,3cyclictest2572004-21kworker/u8:4+events_unbound00:30:181
2239103992015,4cyclictest2473242-21apt-get22:55:091
2239103992015,3cyclictest2566576-21cron00:25:001
2239099992017,2cyclictest2416402-21kworker/u8:2+flush-259:022:10:250
2239099992016,3cyclictest2369928-21kworker/u8:1+events_unbound22:05:170
2239099992015,4cyclictest2570885-21ntpq00:25:200
2239099992015,4cyclictest252-21systemd-journal21:05:010
2239108991915,2cyclictest2483692-21idleruntime-cro23:05:002
2239108991913,4cyclictest2456670-21ntpq22:35:202
2239103991916,2cyclictest2395421-21kworker/u8:3+flush-259:021:45:231
2239103991915,3cyclictest2504984-21/usr/sbin/munin23:25:131
2239103991915,3cyclictest2483726-21kworker/u8:3+flush-259:023:20:171
2239103991913,5cyclictest2327222-21ntpq20:30:201
2239099991916,2cyclictest2082037-21kworker/u8:2+flush-259:019:45:000
2239099991913,4cyclictest252-21systemd-journal23:40:010
2239111991816,1cyclictest0-21swapper/320:45:013
2239111991815,2cyclictest2473022-21kworker/u8:1+events_unbound23:40:153
2239111991815,2cyclictest0-21swapper/322:50:003
2239111991815,2cyclictest0-21swapper/321:10:013
2239111991815,2cyclictest0-21swapper/320:05:013
2239111991814,3cyclictest2416402-21kworker/u8:2+flush-259:022:55:153
2239111991813,4cyclictest2343566-21latency_hist20:50:003
2239111991813,4cyclictest2318257-21diskstats20:25:153
2239108991811,5cyclictest2239708-21diskstats19:10:122
2239103991815,2cyclictest2525544-21kworker/u8:2+events_unbound23:45:231
2239103991815,2cyclictest2328152-21kworker/u8:2+events_unbound20:40:191
2239103991815,2cyclictest2254747-21kworker/u8:0+events_unbound20:15:241
2239103991812,5cyclictest2534750-21ntpq23:50:201
2239099991815,2cyclictest2541009-21kworker/u8:1+events_unbound00:05:000
2239099991815,2cyclictest2416402-21kworker/u8:2+events_unbound22:20:120
2239099991815,2cyclictest2396065-21tr21:40:160
2239099991811,5cyclictest252-21systemd-journal19:57:340
2239111991714,2cyclictest2276418-21irqcore19:45:153
2239111991714,2cyclictest2254747-21kworker/u8:0+events_unbound21:00:153
2239111991713,3cyclictest252-21systemd-journal22:21:103
2239108991714,2cyclictest509-21rs:main2
2239108991714,2cyclictest2541009-21kworker/u8:1+flush-259:000:00:172
2239108991713,2cyclictest0-21swapper/221:15:012
2239108991710,5cyclictest2322969-21latency_hist20:30:012
223910899170,9cyclictest2530505-21latency_hist23:50:002
2239103991714,2cyclictest2364856-21cpuspeed_turbos21:10:121
2239103991714,2cyclictest2254747-21kworker/u8:0+flush-259:020:35:461
2239103991713,3cyclictest1070830-21kworker/u8:4+events_unbound19:25:151
2239099991713,3cyclictest2520174-21kworker/u8:3+events_unbound23:55:190
223909999170,16cyclictest2244434-21turbostat19:15:010
2239111991613,2cyclictest2468512-21irqcore22:50:143
2239111991613,2cyclictest2431844-21kworker/u8:0+flush-259:000:10:223
2239111991613,2cyclictest2405885-21latency_hist21:50:003
2239111991612,3cyclictest2473022-21kworker/u8:1+flush-259:023:10:173
2239111991612,3cyclictest2254747-21kworker/u8:0+flush-259:020:20:203
2239103991613,3cyclictest2082037-21kworker/u8:2+events_unbound19:55:241
2239103991613,2cyclictest2483249-21unixbench_multi23:00:271
2239103991613,2cyclictest2473022-21kworker/u8:1+events_unbound23:30:221
2239103991613,2cyclictest2426926-21apt-get22:10:131
2239103991613,2cyclictest2385467-21cpuspeed_turbos21:30:131
2239103991613,2cyclictest2369928-21kworker/u8:1+events_unbound22:00:251
2239103991613,2cyclictest2280874-21kworker/u8:1+events_unbound20:10:191
2239103991612,3cyclictest2566633-21kworker/u8:1+flush-259:000:25:201
2239103991611,4cyclictest2364525-21latency_hist21:10:011
2239099991614,1cyclictest60350irq/132-enp0s29f1:rx-022:15:130
2239099991614,1cyclictest60350irq/132-enp0s29f1:rx-020:55:240
2239099991614,1cyclictest60350irq/132-enp0s29f1:rx-020:20:150
2239099991613,2cyclictest2369928-21kworker/u8:1+events_unbound21:45:160
2239099991613,2cyclictest2254747-21kworker/u8:0+events_unbound21:35:120
2239099991612,3cyclictest2520174-21kworker/u8:3+events_unbound00:35:140
2239099991612,3cyclictest2473022-21kworker/u8:1+events_unbound23:00:190
2239099991612,3cyclictest2254747-21kworker/u8:0+flush-259:020:35:160
2239099991612,3cyclictest2082037-21kworker/u8:2+flush-259:019:20:180
2239099991611,4cyclictest2461822-21ntpq22:40:200
2239111991512,2cyclictest2548518-21tr00:05:173
2239111991512,2cyclictest2395421-21kworker/u8:3+flush-259:021:50:213
2239111991511,3cyclictest2416402-21kworker/u8:2+events_unbound22:30:233
2239111991511,3cyclictest2416402-21kworker/u8:2+events_unbound22:30:233
2239111991510,4cyclictest2264986-21latency_hist19:35:013
2239111991510,2cyclictest0-21swapper/319:15:003
223910899152,7cyclictest2275743-21munin-run19:45:002
2239108991513,1cyclictest0-21swapper/222:15:002
2239108991511,2cyclictest0-21swapper/223:40:012
223910899150,9cyclictest2296365-21munin-plugin-st20:05:002
2239103991511,3cyclictest509-21rs:main1
2239103991511,3cyclictest252-21systemd-journal22:25:001
2239103991511,3cyclictest2369928-21kworker/u8:1+flush-259:021:25:031
2239099991514,1cyclictest60350irq/132-enp0s29f1:rx-020:26:040
2239099991513,1cyclictest60350irq/132-enp0s29f1:rx-023:20:140
2239099991513,1cyclictest60350irq/132-enp0s29f1:rx-021:30:190
2239099991513,1cyclictest60350irq/132-enp0s29f1:rx-020:10:210
2239099991513,1cyclictest60350irq/132-enp0s29f1:rx-019:15:230
2239099991513,1cyclictest60350irq/132-enp0s29f1:rx-000:10:170
2239099991512,2cyclictest60350irq/132-enp0s29f1:rx-021:15:170
2239099991512,2cyclictest2369928-21kworker/u8:1+flush-259:022:00:000
2239099991511,3cyclictest2416402-21kworker/u8:2+flush-259:022:30:000
2239099991510,4cyclictest2493982-21latency_hist23:15:010
2239099991510,3cyclictest2515023-21cron23:35:000
223911199149,3cyclictest2577153-21/usr/sbin/munin00:35:213
2239111991411,2cyclictest2431844-21kworker/u8:0+events_unbound23:05:143
2239111991411,2cyclictest2368698-21needreboot21:10:193
2239111991410,3cyclictest252-21systemd-journal23:16:193
2239111991410,3cyclictest2478666-21base6423:00:013
2239111991410,3cyclictest2254747-21kworker/u8:0+flush-259:021:15:133
2239111991410,3cyclictest1070830-21kworker/u8:4+flush-259:019:35:153
223910899147,5cyclictest2305748-21ntpq20:10:212
2239108991411,2cyclictest2504338-21/usr/sbin/munin23:25:002
2239108991411,2cyclictest2394396-21needreboot21:35:182
2239108991410,3cyclictest2431844-21kworker/u8:0+flush-259:022:50:162
2239108991410,3cyclictest2431844-21kworker/u8:0+flush-259:000:15:162
2239108991410,3cyclictest2082037-21kworker/u8:2+flush-259:019:15:322
2239103991412,1cyclictest0-21swapper/119:15:011
2239103991411,2cyclictest1070830-21kworker/u8:4+flush-259:019:35:201
2239103991410,3cyclictest2483940-21/usr/sbin/munin23:05:191
2239103991410,3cyclictest2395421-21kworker/u8:3+flush-259:021:50:151
223909999149,4cyclictest60350irq/132-enp0s29f1:rx-023:45:180
2239099991413,1cyclictest60350irq/132-enp0s29f1:rx-023:51:230
2239099991413,1cyclictest60350irq/132-enp0s29f1:rx-022:00:350
2239099991412,2cyclictest2328152-21kworker/u8:2+events_unbound21:10:140
2239099991412,1cyclictest60350irq/132-enp0s29f1:rx-023:05:190
2239099991412,1cyclictest60350irq/132-enp0s29f1:rx-022:35:120
2239099991412,1cyclictest60350irq/132-enp0s29f1:rx-021:30:010
2239099991412,1cyclictest60350irq/132-enp0s29f1:rx-020:50:480
2239099991412,1cyclictest60350irq/132-enp0s29f1:rx-020:18:330
2239099991412,1cyclictest60350irq/132-enp0s29f1:rx-019:31:030
2239099991412,1cyclictest0-21swapper/023:30:000
2239099991412,1cyclictest0-21swapper/019:10:010
223911199139,3cyclictest2416402-21kworker/u8:2+events_unbound22:25:133
223911199139,3cyclictest2311827-21apt-get20:20:013
223911199139,3cyclictest2082037-21kworker/u8:2+flush-259:019:21:383
2239111991311,1cyclictest0-21swapper/323:55:003
2239111991311,1cyclictest0-21swapper/323:35:003
2239111991310,2cyclictest2971-21cron19:44:593
2239111991310,2cyclictest2567263-21irqcore00:25:153
2239111991310,2cyclictest2344006-21fschecks_count20:50:143
223910899139,3cyclictest2359077-21gpgv21:05:002
223910899139,3cyclictest2254747-21kworker/u8:0+flush-259:021:00:012
223910899138,4cyclictest2431844-21kworker/u8:0+flush-259:022:40:102
223910899132,6cyclictest2488849-21awk23:10:002
2239108991310,2cyclictest2333953-21irqcore20:40:142
2239108991310,2cyclictest0-21swapper/219:24:592
223910399139,3cyclictest2416402-21kworker/u8:2+flush-259:022:15:231
2239103991310,2cyclictest2493964-21idleruntime-cro23:15:001
2239103991310,2cyclictest2359333-21apt-key21:05:011
2239103991310,2cyclictest2254747-21kworker/u8:0+flush-259:021:15:151
2239099991312,1cyclictest60350irq/132-enp0s29f1:rx-019:45:230
2239099991312,1cyclictest60350irq/132-enp0s29f1:rx-019:26:500
2239099991312,1cyclictest60350irq/132-enp0s29f1:rx-000:20:490
2239099991311,1cyclictest60350irq/132-enp0s29f1:rx-021:05:160
223911199129,2cyclictest509-21rs:main3
223911199129,2cyclictest2525544-21kworker/u8:2+events_unbound23:45:153
223911199129,2cyclictest2434360-21kthreadcore22:15:173
223911199126,5cyclictest2561876-21tr00:20:143
223910899129,2cyclictest2285026-21needreboot19:50:182
223910899129,1cyclictest0-21swapper/222:24:592
2239108991210,1cyclictest2279866-21needreboot19:45:182
223910399129,2cyclictest2520174-21kworker/u8:3+flush-259:023:55:211
223910399129,2cyclictest2431844-21kworker/u8:0+events_unbound00:15:131
223910399129,2cyclictest0-21swapper/122:30:001
223910399127,4cyclictest2348945-21/usr/sbin/munin20:55:141
2239103991210,1cyclictest0-21swapper/120:25:171
223909999128,3cyclictest60350irq/132-enp0s29f1:rx-000:05:110
223909999121,2cyclictest2291462-21apt-get20:00:120
2239099991211,1cyclictest60350irq/132-enp0s29f1:rx-020:30:240
223911199119,1cyclictest0-21swapper/320:10:003
223911199118,2cyclictest2391123-21irqcore21:35:153
223911199118,2cyclictest2385509-21cpuspeed_turbos21:30:133
223911199118,2cyclictest2082037-21kworker/u8:2+events_unbound19:25:223
223911199117,3cyclictest2520174-21kworker/u8:3+flush-259:000:00:013
223910899118,2cyclictest2360059-21irqcore21:05:172
223910899118,2cyclictest2297031-21sed20:05:152
223910899118,1cyclictest0-21swapper/220:25:002
223910899117,3cyclictest2520671-21fschecks_time23:40:132
223910899117,2cyclictest17550irq/130-nvme0q300:15:002
223910899116,3cyclictest2467826-21cron22:50:002
223910899116,3cyclictest2328371-21/usr/sbin/munin20:35:242
223910899113,1cyclictest0-21swapper/221:35:012
223910399119,1cyclictest0-21swapper/120:45:151
223910399118,2cyclictest2463163-21fschecks_count22:45:131
223910399118,2cyclictest2451971-21unixbench_multi22:30:261
223910399118,2cyclictest2451971-21unixbench_multi22:30:261
223911199108,1cyclictest0-21swapper/323:20:233
223911199100,2cyclictest2576922-21awk00:35:003
223910399108,1cyclictest0-21swapper/122:35:171
22391089996,2cyclictest2370334-21irqcore21:15:132
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional