You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-08 - 08:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Sat Aug 08, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2339040996158,2cyclictest2463491-21kworker/u8:2+events_unbound21:10:022
2339037994845,2cyclictest2380571-21kworker/u8:4+events_unbound20:00:021
2339040994438,4cyclictest2588253-21latency_hist23:10:012
2339037994436,5cyclictest2624811-21latency_hist23:45:011
2339044994238,3cyclictest2515939-21kworker/u8:0+flush-259:022:15:013
2339037993835,2cyclictest2536630-21kworker/u8:2+events_unbound22:22:001
2339044993733,3cyclictest2469380-21kworker/u8:3+flush-259:021:30:003
2339037993730,5cyclictest335557-21systemd-journal19:45:001
2339044993631,3cyclictest2583102-21latency_hist23:05:003
2339037993633,2cyclictest2515939-21kworker/u8:0+events_unbound22:37:051
2339037993630,4cyclictest509-21rs:main1
2339037993629,5cyclictest335557-21systemd-journal23:02:321
2339040993531,3cyclictest2339225-21kworker/u8:3+flush-259:019:10:252
2339034993529,4cyclictest2458315-21gpgv21:05:010
2339044993327,4cyclictest2573039-21/usr/sbin/munin22:55:133
2339040993326,5cyclictest2416686-21base6420:25:002
2339034993330,2cyclictest211rcuc/022:15:000
2339040993229,2cyclictest335557-21systemd-journal21:47:312
2339040993225,5cyclictest2561596-21ntpq22:40:202
2339040993224,6cyclictest2656251-21latency_hist00:15:012
2339040993224,6cyclictest2656251-21latency_hist00:15:012
2339034993225,5cyclictest2666190-21ntpq00:20:220
2339044993128,2cyclictest2463491-21kworker/u8:2+flush-259:021:20:233
2339040993126,3cyclictest2463785-21gpgv21:09:592
2339040993125,4cyclictest2359723-21kworker/u8:1+events_unbound19:35:242
2339034993126,3cyclictest2630470-21base6423:50:010
2339044993027,2cyclictest2661423-21kworker/u8:1+events_unbound00:30:013
2339044993026,3cyclictest2505563-21kworker/u8:2+flush-259:021:50:243
2339040993027,2cyclictest2588276-21kworker/u8:2+flush-259:023:15:132
2339040993025,3cyclictest2552100-21latency_hist22:35:002
2339044992926,2cyclictest2536630-21kworker/u8:2+flush-259:022:55:013
2339044992925,3cyclictest2485534-21kworker/u8:1+flush-259:021:45:003
2339044992925,3cyclictest2318105-21kworker/u8:0+flush-259:019:30:003
2339040992925,3cyclictest2515939-21kworker/u8:0+flush-259:000:38:442
2339034992925,2cyclictest2588276-21kworker/u8:2+flush-259:023:35:150
2339034992923,4cyclictest2609367-21turbostat23:30:000
2339044992824,3cyclictest2380364-21kworker/u8:3+flush-259:020:00:003
2339040992824,3cyclictest2515939-21kworker/u8:0+events_unbound23:55:122
2339040992823,3cyclictest2380364-21kworker/u8:3+events_unbound20:10:012
2339037992825,2cyclictest2645928-21kworker/u8:3+events_unbound00:14:561
2339037992825,2cyclictest2645928-21kworker/u8:3+events_unbound00:14:551
2339037992822,4cyclictest335557-21systemd-journal21:00:311
2339037992821,5cyclictest2593429-21latency_hist23:15:011
2339034992824,3cyclictest2505768-21kworker/u8:4+flush-259:022:20:010
2339040992721,4cyclictest2500385-21latency_hist21:45:012
2339040992720,5cyclictest2432401-21awk20:40:002
2339037992722,4cyclictest2474782-21apt-get21:20:111
2339037992721,4cyclictest2661925-21cat00:20:011
2339044992622,3cyclictest2359723-21kworker/u8:1+flush-259:021:14:593
2339044992621,4cyclictest2485534-21kworker/u8:1+flush-259:021:34:373
2339040992622,3cyclictest2515939-21kworker/u8:0+events_unbound23:46:322
2339040992622,3cyclictest2505768-21kworker/u8:4+events_unbound21:55:132
2339040992620,3cyclictest2640951-21/usr/sbin/munin00:00:162
2339040992619,5cyclictest2651061-21latency_hist00:10:002
2339034992620,4cyclictest2394905-21ntpq20:00:180
2339040992521,3cyclictest2515939-21kworker/u8:0+events_unbound00:24:062
2339040992520,4cyclictest2588276-21kworker/u8:2+events_unbound23:35:322
2339040992519,4cyclictest2339225-21kworker/u8:3+events_unbound19:26:342
2339037992520,4cyclictest2359723-21kworker/u8:1+flush-259:020:20:181
2339037992520,3cyclictest2614783-21apt-get23:35:111
2339037992520,3cyclictest2515939-21kworker/u8:0+flush-259:022:10:011
2339044992421,2cyclictest2359723-21kworker/u8:1+flush-259:019:30:243
2339040992421,2cyclictest2661423-21kworker/u8:1+flush-259:000:30:272
2339040992420,3cyclictest2505768-21kworker/u8:4+events_unbound22:15:342
2339040992420,3cyclictest2285506-21kworker/u8:1+flush-259:019:15:252
2339037992421,2cyclictest2359723-21kworker/u8:1+events_unbound19:50:011
2339037992417,5cyclictest335557-21systemd-journal19:22:311
2339034992421,2cyclictest2515939-21kworker/u8:0+events_unbound22:55:160
2339034992420,3cyclictest2359723-21kworker/u8:1+flush-259:020:35:330
2339034992419,4cyclictest2509806-21ntpq21:50:210
2339044992318,3cyclictest2458424-21store21:05:023
2339044992317,4cyclictest335557-21systemd-journal19:40:013
2339040992319,3cyclictest2525874-21kworker/u8:1+flush-259:022:20:212
2339040992317,4cyclictest2410495-21ntpq20:15:242
2339037992316,5cyclictest2359710-21latency_hist19:30:001
2339034992319,3cyclictest2677601-21gpgv00:35:010
2339034992319,3cyclictest2645928-21kworker/u8:3+flush-259:000:15:000
2339034992319,3cyclictest2645928-21kworker/u8:3+flush-259:000:14:590
2339034992319,3cyclictest2395853-21kworker/u8:2+flush-259:020:30:280
2339044992219,2cyclictest2593457-21gpgv23:15:003
2339044992218,3cyclictest2536630-21kworker/u8:2+flush-259:022:35:003
2339040992218,3cyclictest2395853-21kworker/u8:2+events_unbound20:33:322
2339040992218,3cyclictest2359723-21kworker/u8:1+flush-259:020:55:252
2339040992218,3cyclictest2318105-21kworker/u8:0+events_unbound19:10:002
2339040992218,2cyclictest2521357-21apt-get22:05:122
2339040992217,4cyclictest2422101-21kworker/u8:3+events_unbound20:46:492
2339040992217,3cyclictest2578169-21/usr/sbin/munin23:00:142
2339037992217,3cyclictest2463491-21kworker/u8:2+flush-259:021:20:001
2339034992219,2cyclictest2515939-21kworker/u8:0+flush-259:023:20:230
2339034992219,2cyclictest2359723-21kworker/u8:1+flush-259:020:45:240
2339044992118,2cyclictest2515939-21kworker/u8:0+flush-259:022:00:163
2339044992116,4cyclictest2604187-21latency_hist23:25:013
2339040992118,2cyclictest2339029-21cyclictest21:05:012
2339040992117,3cyclictest2536630-21kworker/u8:2+flush-259:022:36:372
2339040992117,3cyclictest2525874-21kworker/u8:1+flush-259:022:13:082
2339040992116,4cyclictest2505768-21kworker/u8:4+flush-259:021:55:002
2339040992116,3cyclictest2599250-21/usr/sbin/munin23:20:202
2339037992118,2cyclictest2485534-21kworker/u8:1+events_unbound21:36:401
2339044992017,2cyclictest2515939-21kworker/u8:0+flush-259:023:55:153
2339044992015,4cyclictest2608496-21ntpq23:25:193
2339040992017,2cyclictest2406156-21cron20:15:002
2339040992016,3cyclictest2629586-21kworker/u8:1+events_unbound23:52:322
2339040992016,3cyclictest2588276-21kworker/u8:2+events_unbound23:26:322
2339040992015,4cyclictest335557-21systemd-journal19:35:002
2339040992015,4cyclictest2469621-21apt-get21:15:112
2339040992014,4cyclictest2672174-21latency_hist00:30:012
2339037992017,2cyclictest2515939-21kworker/u8:0+events_unbound22:30:021
2339037992016,3cyclictest509-21rs:main1
2339037992014,4cyclictest2505768-21kworker/u8:4+flush-259:022:40:151
2339034992013,5cyclictest2364893-21turbostat19:35:000
2339044991916,2cyclictest2422101-21kworker/u8:3+flush-259:020:45:003
2339044991916,2cyclictest2380364-21kworker/u8:3+events_unbound20:06:413
2339040991915,3cyclictest2515939-21kworker/u8:0+events_unbound22:25:132
2339040991915,3cyclictest2505768-21kworker/u8:4+events_unbound22:46:312
2339040991915,3cyclictest2479710-21kworker/u8:0+events_unbound21:30:212
2339040991915,3cyclictest2380364-21kworker/u8:3+events_unbound20:00:092
2339037991916,2cyclictest2359723-21kworker/u8:1+events_unbound20:28:181
2339037991915,3cyclictest2500648-21apt-get21:45:121
2339037991915,3cyclictest2422101-21kworker/u8:3+events_unbound20:44:261
2339037991914,1cyclictest191rcu_preempt22:04:331
2339037991913,4cyclictest335557-21systemd-journal00:05:001
2339034991916,2cyclictest2469380-21kworker/u8:3+flush-259:021:22:030
2339034991914,4cyclictest2490294-21apt-get21:35:130
2339044991816,1cyclictest2661392-21cron00:19:593
2339044991815,2cyclictest2505768-21kworker/u8:4+events_unbound21:55:153
2339044991812,4cyclictest2442743-21latency_hist20:50:023
2339044991812,4cyclictest2338963-21latency_hist19:10:003
2339040991815,2cyclictest2655860-21kworker/u8:2+flush-259:000:15:242
2339040991814,3cyclictest2515939-21kworker/u8:0+events_unbound23:30:142
2339040991814,3cyclictest2485534-21kworker/u8:1+flush-259:021:35:252
2339037991816,1cyclictest2505768-21kworker/u8:4+events_unbound22:29:531
2339037991811,5cyclictest335557-21systemd-journal23:54:541
2339034991815,2cyclictest2463491-21kworker/u8:2+flush-259:021:14:590
2339034991815,2cyclictest2339225-21kworker/u8:3+events_unbound19:20:230
2339044991714,2cyclictest2629586-21kworker/u8:1+events_unbound00:00:233
2339044991714,2cyclictest2515939-21kworker/u8:0+events_unbound23:40:203
2339044991712,4cyclictest2583334-21/usr/sbin/munin23:05:233
2339040991714,2cyclictest2515939-21kworker/u8:0+events_unbound22:55:192
2339040991713,3cyclictest2463491-21kworker/u8:2+events_unbound21:20:142
2339040991713,3cyclictest2380571-21kworker/u8:4+events_unbound19:56:402
233903799178,5cyclictest0-21swapper/120:48:231
2339037991715,2cyclictest2359723-21kworker/u8:1+events_unbound21:12:051
2339037991714,2cyclictest2588276-21kworker/u8:2+flush-259:023:15:081
2339037991714,2cyclictest2505563-21kworker/u8:2+events_unbound21:56:551
2339037991714,2cyclictest2359723-21kworker/u8:1+events_unbound19:45:381
2339037991713,3cyclictest2572792-21idleruntime-cro22:55:011
2339034991714,2cyclictest2463491-21kworker/u8:2+events_unbound21:15:150
2339034991714,2cyclictest2339029-21cyclictest20:45:010
2339034991713,3cyclictest2629586-21kworker/u8:1+flush-259:000:00:170
233904499169,5cyclictest335557-21systemd-journal21:09:593
2339044991613,2cyclictest2354538-21latency_hist19:25:003
2339044991612,3cyclictest2656107-21sshd00:13:333
2339044991612,3cyclictest2656107-21sshd00:13:333
2339044991612,3cyclictest2594116-21apt-get23:15:113
2339040991612,3cyclictest2395853-21kworker/u8:2+events_unbound20:42:442
2339040991612,3cyclictest2359723-21kworker/u8:1+events_unbound19:50:172
233903799161,3cyclictest2343324-21ntpq19:10:201
2339037991613,2cyclictest2661423-21kworker/u8:1+events_unbound00:35:481
2339037991612,3cyclictest2344434-21/usr/sbin/munin19:15:151
2339037991611,4cyclictest335557-21systemd-journal23:50:001
2339037991611,3cyclictest2469380-21kworker/u8:3+flush-259:021:25:111
2339034991614,1cyclictest0-21swapper/021:49:590
2339034991613,2cyclictest60350irq/132-enp0s29f1:rx-023:48:560
2339034991613,2cyclictest60350irq/132-enp0s29f1:rx-000:05:140
2339034991612,3cyclictest2375455-21apt-get19:45:130
2339044991513,1cyclictest2615204-21irqcore23:35:163
2339044991512,2cyclictest507-21in:imuxsock19:55:003
2339044991512,2cyclictest2650113-21ntp_kernel_err00:05:183
2339044991512,2cyclictest2359723-21kworker/u8:1+flush-259:020:55:163
2339044991510,3cyclictest2682957-21latency_hist00:40:013
2339040991512,3cyclictest2359723-21kworker/u8:1+flush-259:019:40:162
2339040991512,2cyclictest2588276-21kworker/u8:2+events_unbound23:10:142
2339040991511,3cyclictest2536630-21kworker/u8:2+flush-259:022:50:232
2339040991511,3cyclictest2515939-21kworker/u8:0+events_unbound22:01:312
2339040991511,3cyclictest2395853-21kworker/u8:2+events_unbound20:25:282
233903799159,1cyclictest191rcu_preempt21:05:231
233903799157,4cyclictest0-21swapper/121:50:371
233903799156,2cyclictest0-21swapper/122:10:221
233903799155,1cyclictest191rcu_preempt19:30:181
233903799152,2cyclictest32-21ksoftirqd/123:55:111
2339037991512,2cyclictest2380364-21kworker/u8:3+events_unbound19:55:051
2339037991510,3cyclictest2599275-21apt-get23:20:111
2339034991513,1cyclictest60350irq/132-enp0s29f1:rx-023:40:150
2339034991513,1cyclictest60350irq/132-enp0s29f1:rx-023:30:230
2339034991513,1cyclictest60350irq/132-enp0s29f1:rx-023:10:180
2339034991513,1cyclictest60350irq/132-enp0s29f1:rx-022:20:170
2339034991513,1cyclictest60350irq/132-enp0s29f1:rx-022:05:240
2339034991513,1cyclictest60350irq/132-enp0s29f1:rx-021:55:250
2339034991513,1cyclictest60350irq/132-enp0s29f1:rx-021:40:160
2339034991513,1cyclictest60350irq/132-enp0s29f1:rx-021:32:160
2339034991513,1cyclictest60350irq/132-enp0s29f1:rx-020:54:040
2339034991513,1cyclictest60350irq/132-enp0s29f1:rx-020:27:110
2339034991513,1cyclictest60350irq/132-enp0s29f1:rx-020:15:200
2339034991513,1cyclictest60350irq/132-enp0s29f1:rx-019:10:160
2339034991512,2cyclictest60350irq/132-enp0s29f1:rx-023:20:010
2339034991512,2cyclictest60350irq/132-enp0s29f1:rx-000:38:540
2339034991511,3cyclictest507-21in:imuxsock00:30:000
2339034991511,3cyclictest2516180-21apt-get22:00:110
2339034991511,3cyclictest2416756-21store20:25:010
2339034991511,3cyclictest2359723-21kworker/u8:1+flush-259:019:36:360
2339034991510,1cyclictest60350irq/132-enp0s29f1:rx-023:00:220
233904499149,3cyclictest2447915-21sh20:55:003
2339044991411,2cyclictest2525874-21kworker/u8:1+events_unbound22:20:143
2339044991411,2cyclictest2525378-21ntp_kernel_pll_22:05:213
2339044991411,2cyclictest2515939-21kworker/u8:0+flush-259:023:45:323
2339044991411,2cyclictest2348960-21unixbench_singl19:15:263
2339040991410,3cyclictest2463491-21kworker/u8:2+events_unbound21:29:332
233903799149,4cyclictest335557-21systemd-journal20:10:001
233903799148,2cyclictest191rcu_preempt00:05:221
233903799145,3cyclictest32-21ksoftirqd/123:35:001
233903799143,2cyclictest32-21ksoftirqd/123:25:211
233903799143,2cyclictest32-21ksoftirqd/121:30:111
233903799142,3cyclictest32-21ksoftirqd/100:30:001
233903799142,2cyclictest32-21ksoftirqd/121:40:141
2339037991411,2cyclictest2359723-21kworker/u8:1+events_unbound20:52:371
2339037991410,1cyclictest191rcu_preempt00:16:131
2339034991412,1cyclictest60350irq/132-enp0s29f1:rx-022:45:150
2339034991412,1cyclictest60350irq/132-enp0s29f1:rx-022:30:190
2339034991412,1cyclictest60350irq/132-enp0s29f1:rx-022:25:200
2339034991412,1cyclictest60350irq/132-enp0s29f1:rx-021:25:140
2339034991412,1cyclictest60350irq/132-enp0s29f1:rx-020:55:240
2339034991411,2cyclictest60350irq/132-enp0s29f1:rx-019:25:120
2339034991411,2cyclictest60350irq/132-enp0s29f1:rx-000:17:430
2339034991410,3cyclictest2344658-21grep19:15:150
233904499139,3cyclictest0-21swapper/321:20:013
233904499138,4cyclictest335557-21systemd-journal19:15:003
233904499131,10cyclictest2667045-21awk00:25:003
2339044991310,2cyclictest2672511-21cpuspeed_turbos00:30:133
2339044991310,2cyclictest2411678-21cpuspeed_turbos20:20:133
2339044991310,2cyclictest2395853-21kworker/u8:2+events_unbound20:30:233
2339044991310,2cyclictest2359723-21kworker/u8:1+events_unbound20:25:023
233904499130,12cyclictest2536225-21sshd22:15:313
233904099139,3cyclictest2515939-21kworker/u8:0+flush-259:023:42:072
233904099139,3cyclictest2318105-21kworker/u8:0+events_unbound19:45:292
233904099138,4cyclictest2339225-21kworker/u8:3+events_unbound19:20:142
233904099137,4cyclictest509-21rs:main2
233903799134,5cyclictest32-21ksoftirqd/120:40:011
233903799132,2cyclictest32-21ksoftirqd/120:15:251
2339037991310,2cyclictest2587832-21unixbench_singl23:05:241
2339037991310,1cyclictest191rcu_preempt19:36:021
233903499139,3cyclictest2577951-21kworker/u8:3+flush-259:023:10:000
233903499137,4cyclictest2458833-21cp21:05:010
2339034991311,1cyclictest60350irq/132-enp0s29f1:rx-023:55:190
2339034991311,1cyclictest60350irq/132-enp0s29f1:rx-022:40:130
2339034991311,1cyclictest60350irq/132-enp0s29f1:rx-020:10:120
233904499129,2cyclictest2380364-21kworker/u8:3+events_unbound20:00:123
233904499128,3cyclictest335557-21systemd-journal19:49:553
233904499128,3cyclictest2536630-21kworker/u8:2+flush-259:022:39:083
233904499128,3cyclictest2505768-21kworker/u8:4+flush-259:022:44:363
233904499128,3cyclictest2359723-21kworker/u8:1+events_unbound20:36:313
2339044991210,1cyclictest0-21swapper/322:50:003
233903799122,3cyclictest32-21ksoftirqd/120:10:161
233903799122,2cyclictest32-21ksoftirqd/120:55:231
233903799120,7cyclictest2422318-21/usr/sbin/munin20:30:151
233903499128,3cyclictest2359723-21kworker/u8:1+events_unbound19:55:020
2339034991210,1cyclictest60350irq/132-enp0s29f1:rx-019:51:330
233904499118,2cyclictest2380364-21kworker/u8:3+flush-259:020:10:163
233904499117,3cyclictest2630618-21/usr/sbin/munin23:50:143
2339034991110,1cyclictest60350irq/132-enp0s29f1:rx-019:40:080
233904499107,2cyclictest2490706-21tr21:35:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional