You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-18 - 05:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Thu Dec 18, 2025 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2787597994442,1cyclictest2808037-21fwupd19:28:282
2787592993730,5cyclictest2860742-21latency_hist20:20:000
2787594993529,4cyclictest252-21systemd-journal21:45:001
2787597993125,4cyclictest3057697-21cat23:30:002
2787594993127,3cyclictest2787292-21kworker/u8:3+events_unbound19:28:301
2787592992921,5cyclictest2907275-21latency_hist21:05:010
2787592992920,6cyclictest2844819-21latency_hist20:05:000
2787597992823,3cyclictest2839323-21cp20:00:002
2787602992520,3cyclictest252-21systemd-journal21:20:013
2787597992519,4cyclictest2787516-21latency_hist19:10:002
2787594992421,2cyclictest2892402-21kworker/u8:2+events_unbound20:55:151
2787594992319,3cyclictest3000964-21apt-get22:35:121
2787602992218,3cyclictest2787292-21kworker/u8:3+events_unbound19:40:223
2787602992218,3cyclictest2728254-21kworker/u8:1+flush-259:019:15:173
2787597992218,3cyclictest3026803-21/usr/sbin/munin23:00:152
2787592992218,3cyclictest2927862-21kworker/u8:2+flush-259:021:45:330
2787602992117,3cyclictest2886819-21/usr/sbin/munin20:45:143
2787597992114,5cyclictest2912472-21latency_hist21:10:012
2787594992118,2cyclictest2876502-21kworker/u8:0+events_unbound20:40:241
2787594992118,2cyclictest2876260-21idleruntime-cro20:35:001
2787594992117,3cyclictest2870712-21kworker/u8:4+flush-259:020:30:001
2787594992117,3cyclictest252-21systemd-journal21:30:001
2787592992114,5cyclictest2937957-21ntpq21:30:220
2787602992018,1cyclictest0-21swapper/321:40:003
2787602992015,4cyclictest2845060-21/usr/sbin/munin20:05:163
2787597992017,2cyclictest3058200-21kworker/u8:3+flush-259:000:10:202
2787597992015,4cyclictest252-21systemd-journal20:30:002
2787597992013,5cyclictest509-21rs:main2
2787594992018,1cyclictest2838866-21kworker/u8:0+events_unbound20:00:231
2787594992016,3cyclictest2990603-21kworker/u8:0+flush-259:022:25:171
2787594992013,5cyclictest2839216-21latency_hist20:00:001
278759299205,3cyclictest0-21swapper/021:00:000
2787592992013,5cyclictest252-21systemd-journal22:50:000
2787602991916,3cyclictest2927862-21kworker/u8:2+flush-259:021:50:013
2787602991916,2cyclictest3026547-21cron23:00:013
2787602991916,2cyclictest3026547-21cron23:00:003
2787602991914,4cyclictest2834464-21diskstats19:55:133
2787602991911,7cyclictest0-21swapper/323:20:003
2787597991915,3cyclictest3005866-21kworker/u8:4+flush-259:023:20:202
2787597991915,3cyclictest2959344-21kworker/u8:1+flush-259:021:55:022
2787594991916,2cyclictest2927862-21kworker/u8:2+flush-259:022:50:251
2787594991916,2cyclictest2927862-21kworker/u8:2+flush-259:022:50:251
2787594991916,2cyclictest2927862-21kworker/u8:2+events_unbound22:40:231
2787592991912,5cyclictest2985211-21latency_hist22:20:000
2787592991911,6cyclictest3116019-21latency_hist00:25:010
278759299191,16cyclictest252-21systemd-journal21:25:300
2787602991815,1cyclictest0-21swapper/321:55:003
2787602991812,4cyclictest2817675-21ntpq19:35:213
2787597991813,3cyclictest252-21systemd-journal23:33:562
2787594991815,2cyclictest2959344-21kworker/u8:1+flush-259:022:30:221
2787592991815,2cyclictest2927862-21kworker/u8:2+events_unbound22:09:550
2787592991811,6cyclictest2839223-21apt-get19:59:590
2787592991811,5cyclictest3046308-21ntpq23:15:220
2787592991811,5cyclictest2797878-21latency_hist19:20:010
2787602991714,2cyclictest3105694-21kworker/u8:0+events_unbound00:15:193
2787602991714,2cyclictest2927862-21kworker/u8:2+flush-259:022:10:003
2787602991714,2cyclictest2881436-21journal-offline20:40:003
2787602991713,3cyclictest2932689-21ntp_kernel_err21:25:223
2787597991714,2cyclictest3058200-21kworker/u8:3+flush-259:023:45:242
2787594991714,2cyclictest2823930-21kworker/u8:1+events_unbound19:50:191
2787594991711,4cyclictest2880610-21ntpq20:35:241
278759299173,9cyclictest2818361-21sshd19:37:560
278759299173,10cyclictest3119835-21cut00:25:190
278759299172,4cyclictest2910923-21tr21:05:200
2787592991714,2cyclictest60350irq/132-enp0s29f1:rx-023:35:220
2787592991713,3cyclictest60350irq/132-enp0s29f1:rx-022:10:130
2787592991713,3cyclictest3036484-21kworker/u8:3+flush-259:023:10:140
2787592991713,3cyclictest2959697-21apt-get21:55:100
2787592991710,5cyclictest2968848-21ntpq22:00:230
278759299170,6cyclictest2866060-21latency_hist20:25:000
2787602991614,1cyclictest2922886-21turbostat.cron21:20:003
2787602991614,1cyclictest0-21swapper/320:30:003
2787602991613,2cyclictest3126302-21sh00:35:003
2787602991613,2cyclictest2959344-21kworker/u8:1+flush-259:022:00:243
2787602991613,2cyclictest2808186-21cron19:30:013
2787602991612,2cyclictest0-21swapper/320:20:003
2787602991610,4cyclictest3058397-21diskstats23:30:143
2787597991613,2cyclictest3074085-21latency_hist23:45:002
2787597991613,2cyclictest0-21swapper/219:20:012
2787597991610,4cyclictest2895988-21ntpq20:50:212
2787594991614,1cyclictest0-21swapper/121:55:011
2787594991612,3cyclictest2876502-21kworker/u8:0+events_unbound20:45:251
2787594991612,3cyclictest2823930-21kworker/u8:1+flush-259:020:05:201
2787594991611,3cyclictest3111074-21/usr/sbin/munin00:20:191
278759299167,1cyclictest191rcu_preempt23:45:210
278759299163,4cyclictest3096588-21irqstats00:05:200
278759299163,4cyclictest2994649-21ntpq22:25:220
278759299163,3cyclictest2866178-21apt-get20:25:120
278759299163,2cyclictest3010177-21ntpq22:40:230
278759299162,9cyclictest2892180-21sh20:50:130
278759299162,4cyclictest3090381-21date00:00:010
278759299162,4cyclictest2852467-21grep20:10:150
2787592991613,2cyclictest60350irq/132-enp0s29f1:rx-022:55:180
2787592991613,2cyclictest60350irq/132-enp0s29f1:rx-022:55:180
2787592991613,2cyclictest60350irq/132-enp0s29f1:rx-019:45:170
2787592991612,3cyclictest60350irq/132-enp0s29f1:rx-023:25:130
2787592991611,1cyclictest191rcu_preempt23:40:510
2787592991610,4cyclictest2917938-21/usr/sbin/munin21:15:200
2787602991513,1cyclictest2963698-21ntp_kernel_pll_21:55:203
2787602991512,2cyclictest2892402-21kworker/u8:2+events_unbound21:00:243
2787602991512,2cyclictest2876502-21kworker/u8:0+flush-259:020:55:003
2787602991512,2cyclictest0-21swapper/323:40:003
2787602991511,3cyclictest2823930-21kworker/u8:1+flush-259:020:20:173
2787597991512,2cyclictest2927862-21kworker/u8:2+flush-259:021:25:172
2787597991511,3cyclictest0-21swapper/200:00:002
2787594991513,1cyclictest3111067-21kworker/u8:1+events_unbound00:25:241
2787594991512,2cyclictest3058200-21kworker/u8:3+flush-259:000:10:161
2787594991512,2cyclictest2959344-21kworker/u8:1+events_unbound22:45:241
2787594991512,2cyclictest2927862-21kworker/u8:2+flush-259:022:20:221
2787594991512,2cyclictest2876502-21kworker/u8:0+events_unbound21:20:151
2787594991512,2cyclictest2838866-21kworker/u8:0+flush-259:020:25:001
2787594991512,2cyclictest2792727-21cron19:15:011
2787594991512,1cyclictest0-21swapper/121:15:001
2787594991511,3cyclictest2802122-21ntp_kernel_pll_19:20:221
278759299154,7cyclictest3085448-21df23:55:140
278759299153,7cyclictest2880544-21ntpq20:35:230
278759299153,3cyclictest2829104-21latency_hist19:50:000
278759299151,4cyclictest3058189-21gpg-connect-age23:30:010
2787592991514,1cyclictest60350irq/132-enp0s29f1:rx-023:03:100
2787592991513,1cyclictest60350irq/132-enp0s29f1:rx-023:23:360
2787592991513,1cyclictest60350irq/132-enp0s29f1:rx-021:38:420
2787592991513,1cyclictest60350irq/132-enp0s29f1:rx-019:28:300
2787592991513,1cyclictest60350irq/132-enp0s29f1:rx-019:13:360
2787592991513,1cyclictest60350irq/132-enp0s29f1:rx-000:10:150
2787592991511,3cyclictest60350irq/132-enp0s29f1:rx-020:30:220
2787592991511,2cyclictest0-21swapper/023:55:000
2787592991510,1cyclictest191rcu_preempt20:09:480
2787592991510,1cyclictest191rcu_preempt19:20:030
2787592991510,1cyclictest191rcu_preempt00:19:440
278760299149,4cyclictest2871496-21diskstats20:30:133
2787602991411,2cyclictest3090839-21irqcore00:00:183
2787602991411,2cyclictest2773396-21kworker/u8:0+events_unbound19:10:103
2787602991411,1cyclictest0-21swapper/323:10:013
2787602991410,3cyclictest3005866-21kworker/u8:4+flush-259:023:15:003
2787602991410,3cyclictest252-21systemd-journal22:25:563
2787597991412,1cyclictest0-21swapper/200:39:592
2787597991411,3cyclictest2917371-21kworker/u8:1+flush-259:021:20:232
2787597991411,2cyclictest3080010-21gpgv23:50:012
2787597991411,2cyclictest2838866-21kworker/u8:0+flush-259:020:15:242
2787597991410,3cyclictest3121402-21apt-get00:30:112
2787594991412,1cyclictest3058648-21irqcore23:30:181
2787594991412,1cyclictest3016214-21kworker/u8:3+flush-259:022:55:201
2787594991412,1cyclictest3016214-21kworker/u8:3+flush-259:022:55:201
2787594991411,2cyclictest3106363-21sed00:15:171
2787594991411,2cyclictest3068343-21kworker/u8:2+flush-259:023:55:221
2787594991411,2cyclictest3068343-21kworker/u8:2+flush-259:023:45:151
2787594991411,2cyclictest3026584-21kworker/u8:1+flush-259:023:15:181
2787594991411,2cyclictest2927862-21kworker/u8:2+flush-259:022:15:001
2787594991411,2cyclictest2927862-21kworker/u8:2+flush-259:021:45:261
2787594991411,2cyclictest2876502-21kworker/u8:0+flush-259:021:15:231
2787594991410,3cyclictest2927862-21kworker/u8:2+flush-259:022:05:261
2787594991410,3cyclictest2892402-21kworker/u8:2+flush-259:021:00:171
2787594991410,3cyclictest2728254-21kworker/u8:1+flush-259:019:15:211
278759299142,2cyclictest3002783-21cut22:35:180
278759299141,3cyclictest2823358-21modprobe19:40:560
2787592991412,1cyclictest60350irq/132-enp0s29f1:rx-022:30:190
2787592991412,1cyclictest60350irq/132-enp0s29f1:rx-022:23:260
2787592991412,1cyclictest60350irq/132-enp0s29f1:rx-021:54:060
2787592991412,1cyclictest60350irq/132-enp0s29f1:rx-000:37:140
2787592991411,2cyclictest60350irq/132-enp0s29f1:rx-020:40:240
2787592991411,2cyclictest3018783-21kthreadcore22:50:190
2787592991411,2cyclictest3018783-21kthreadcore22:50:180
2787592991411,1cyclictest191rcu_preempt21:44:470
278759299140,6cyclictest3034167-21kthreadcore23:05:180
278760299139,3cyclictest3079826-21apt-key23:50:023
2787602991311,1cyclictest2985431-21runrttasks22:20:003
2787602991311,1cyclictest0-21swapper/320:45:003
2787602991310,2cyclictest3111183-21cpuspeed_turbos00:20:123
2787602991310,2cyclictest3100813-21cpuspeed_turbos00:10:133
2787602991310,2cyclictest3080546-21irqcore23:50:153
2787602991310,2cyclictest2990603-21kworker/u8:0+events_unbound22:55:003
2787602991310,2cyclictest2990603-21kworker/u8:0+events_unbound22:55:003
2787602991310,2cyclictest2985563-21cpuspeed_turbos22:20:143
2787602991310,2cyclictest2844411-21unixbench_singl20:00:263
2787602991310,2cyclictest2823930-21kworker/u8:1+events_unbound19:45:223
278759799139,3cyclictest509-21rs:main2
278759799137,4cyclictest2855147-21awk20:15:002
2787597991311,1cyclictest0-21swapper/220:40:002
2787597991310,2cyclictest3095675-21cpuspeed_turbos00:05:152
2787597991310,2cyclictest2990603-21kworker/u8:0+flush-259:022:55:222
2787597991310,2cyclictest2990603-21kworker/u8:0+flush-259:022:55:212
2787597991310,2cyclictest2876502-21kworker/u8:0+flush-259:021:35:212
2787597991310,2cyclictest2813269-21kworker/u8:0+events_unbound19:45:002
278759499139,3cyclictest3058200-21kworker/u8:3+flush-259:023:35:191
278759499139,3cyclictest2990603-21kworker/u8:0+events_unbound23:05:211
278759499139,3cyclictest2876502-21kworker/u8:0+flush-259:021:35:261
2787594991310,2cyclictest3051893-21unixbench_multi23:20:271
2787594991310,2cyclictest3026584-21kworker/u8:1+flush-259:000:05:181
2787594991310,2cyclictest2860725-21sh20:20:001
2787594991310,2cyclictest2823930-21kworker/u8:1+events_unbound19:45:141
278759299131,2cyclictest3125406-21ntpq00:30:210
2787592991310,2cyclictest60350irq/132-enp0s29f1:rx-019:30:290
278760299129,2cyclictest2938302-21unixbench_multi21:30:253
278760299129,2cyclictest2927862-21kworker/u8:2+flush-259:022:45:263
278760299129,2cyclictest2892402-21kworker/u8:2+flush-259:021:00:003
278760299129,2cyclictest0-21swapper/323:45:013
278760299128,3cyclictest2990603-21kworker/u8:0+flush-259:023:00:253
278760299128,2cyclictest0-21swapper/320:15:003
2787602991210,1cyclictest0-21swapper/323:25:013
278759799129,2cyclictest3067313-21needreboot23:35:212
278759799129,2cyclictest2975259-21cpuspeed_turbos22:10:132
278759799129,2cyclictest2964932-21cpuspeed_turbos22:00:132
278759799129,2cyclictest2927862-21kworker/u8:2+events_unbound22:35:152
278759799129,2cyclictest2898194-21kthreadcore20:55:172
278759799129,2cyclictest2818144-21unixbench_singl19:35:262
278759799129,2cyclictest2813269-21kworker/u8:0+flush-259:019:45:002
278759799129,2cyclictest0-21swapper/219:30:212
278759799128,3cyclictest2959344-21kworker/u8:1+flush-259:022:20:342
2787597991210,1cyclictest3042076-21sh23:15:002
2787597991210,1cyclictest0-21swapper/222:19:592
2787597991210,1cyclictest0-21swapper/220:45:002
2787597991210,1cyclictest0-21swapper/219:20:172
278759499129,2cyclictest3052612-21kworker/u8:2+flush-259:023:25:321
278759499128,3cyclictest3121471-21cpuspeed_turbos00:30:131
278759499128,3cyclictest2876502-21kworker/u8:0+flush-259:020:50:311
278759499127,4cyclictest2964865-21apt-get22:00:121
278759499121,3cyclictest2823692-21latency_hist19:45:001
2787594991210,1cyclictest3068343-21kworker/u8:2-ext4-rsv-conversion23:50:011
2787594991210,1cyclictest2787292-21kworker/u8:3+events_unbound19:35:241
278759499120,10cyclictest3073934-21unin-run23:45:011
278760299119,1cyclictest0-21swapper/319:30:183
278760299118,2cyclictest0-21swapper/323:25:173
278759799119,1cyclictest0-21swapper/221:40:252
278759799119,1cyclictest0-21swapper/220:04:592
278759799118,2cyclictest2990838-21fschecks_count22:25:142
278759799118,2cyclictest0-21swapper/220:35:002
278759799117,3cyclictest252-21systemd-journal21:20:002
278759799117,2cyclictest0-21swapper/220:45:142
278759499119,1cyclictest0-21swapper/121:05:181
278759499116,3cyclictest2985202-21awk22:20:001
278760299100,3cyclictest3131673-21cat00:40:003
278759799107,2cyclictest0-21swapper/200:00:142
278759799107,1cyclictest0-21swapper/200:15:192
278759799106,3cyclictest2990603-21kworker/u8:0+flush-259:022:40:202
278759499100,8cyclictest2960120-21grep21:55:141
27876029994,3cyclictest3116436-21df00:25:143
27875979997,1cyclictest0-21swapper/222:30:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional