You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-12 - 07:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Thu Mar 12, 2026 00:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3942441994034,4cyclictest4189034-21latency_hist23:05:010
3942448993832,4cyclictest41509-21latency_hist23:50:002
3942448993630,4cyclictest4011180-21kworker/u8:3+flush-259:020:19:592
3942446993431,2cyclictest4173323-21kworker/u8:3+events_unbound23:00:011
3942451993228,3cyclictest3947584-21kworker/u8:2+flush-259:020:00:003
3942448993127,3cyclictest4162838-21apt-get22:40:002
3942446993025,3cyclictest4037458-21diskstats20:40:111
3942446993025,3cyclictest3948207-21/usr/sbin/munin19:15:201
3942441993026,3cyclictest252-21systemd-journal20:45:000
3942451992824,3cyclictest4083504-21kworker/u8:1+flush-259:021:40:003
3942451992823,4cyclictest4016155-21kworker/u8:4+flush-259:022:00:163
3942451992822,4cyclictest4168413-21latency_hist22:45:013
3942446992822,4cyclictest4026688-21latency_hist20:30:011
3942446992722,3cyclictest4093999-21/usr/sbin/munin21:35:211
3942451992622,3cyclictest4031869-21kworker/u8:1+flush-259:021:10:013
3942451992621,3cyclictest4031869-21kworker/u8:1+flush-259:020:55:143
3942448992622,3cyclictest31254-21kworker/u8:2+flush-259:023:40:242
3942446992623,2cyclictest4031869-21kworker/u8:1+flush-259:020:55:111
3942441992622,3cyclictest509-21rs:main0
3942451992519,4cyclictest252-21systemd-journal00:40:013
3942448992520,3cyclictest4098943-21latency_hist21:40:002
3942446992522,2cyclictest4104714-21kworker/u8:2+events_unbound21:45:151
3942446992522,2cyclictest31254-21kworker/u8:2+flush-259:023:40:161
3942446992521,3cyclictest31254-21kworker/u8:2+flush-259:023:50:001
3942448992420,3cyclictest4016155-21kworker/u8:4+flush-259:021:35:002
3942448992420,3cyclictest252-21systemd-journal23:27:172
3942446992419,3cyclictest5467-21kworker/u8:1+events_unbound23:30:131
3942446992419,3cyclictest4130248-21kworker/u8:0+flush-259:022:35:161
3942446992418,4cyclictest252-21systemd-journal22:28:191
3942441992419,4cyclictest3969133-21gpgv19:35:000
3942448992318,4cyclictest62331-21apt-get00:10:112
3942446992319,3cyclictest3922107-21kworker/u8:0+events_unbound19:20:131
3942441992319,3cyclictest4120209-21apt-get22:00:110
3942441992318,3cyclictest3990797-21apt-get19:55:100
3942451992217,3cyclictest252-21systemd-journal19:32:193
3942451992216,4cyclictest509-21rs:main3
3942448992219,2cyclictest4114484-21kworker/u8:1+flush-259:022:15:002
3942448992219,2cyclictest4114484-21kworker/u8:1+flush-259:022:15:002
3942441992219,2cyclictest82909-21apt-get00:30:110
3942441992219,2cyclictest82909-21apt-get00:30:100
3942441992217,3cyclictest4157253-21latency_hist22:35:000
3942448992115,5cyclictest4188153-21ntpq23:00:202
3942448992115,4cyclictest3995700-21latency_hist20:00:012
3942446992118,2cyclictest5467-21kworker/u8:1+flush-259:023:35:191
3942446992117,3cyclictest5467-21kworker/u8:1+events_unbound00:00:121
3942446992117,3cyclictest3947584-21kworker/u8:2+flush-259:020:00:191
3942446992117,3cyclictest3947584-21kworker/u8:2+flush-259:019:40:161
3942448992017,2cyclictest509-21rs:main2
3942448992014,4cyclictest4130232-21latency_hist22:10:002
3942446992017,2cyclictest4016155-21kworker/u8:4+events_unbound22:10:271
3942446992017,2cyclictest4016155-21kworker/u8:4+events_unbound22:10:271
3942446992016,3cyclictest41527-21kworker/u8:3+events_unbound23:55:191
3942446992016,3cyclictest3922107-21kworker/u8:0+events_unbound19:55:151
3942446992015,3cyclictest4016155-21kworker/u8:4+flush-259:022:30:101
3942446992015,3cyclictest3889906-21kworker/u8:1+events_unbound19:10:241
3942446992014,4cyclictest4099209-21apt-get21:40:121
3942451991917,1cyclictest0-21swapper/300:10:003
3942451991916,1cyclictest0-21swapper/320:10:003
3942451991914,3cyclictest4021504-21latency_hist20:25:013
3942448991916,2cyclictest4016155-21kworker/u8:4+flush-259:020:35:012
3942448991912,5cyclictest252-21systemd-journal00:30:012
3942448991912,5cyclictest252-21systemd-journal00:30:002
3942446991916,2cyclictest4178748-21kworker/u8:2+flush-259:023:00:241
3942446991916,2cyclictest41527-21kworker/u8:3+events_unbound00:20:191
3942446991916,2cyclictest4016155-21kworker/u8:4+flush-259:021:30:151
3942446991916,2cyclictest4011180-21kworker/u8:3+events_unbound20:50:171
3942446991915,2cyclictest0-21swapper/120:15:001
3942441991915,3cyclictest4140528-21apt-get22:20:010
3942441991915,3cyclictest3922107-21kworker/u8:0+flush-259:019:50:160
3942441991915,3cyclictest252-21systemd-journal23:58:180
3942441991914,4cyclictest4016155-21kworker/u8:4+flush-259:020:20:000
3942441991912,5cyclictest4168670-21apt-get22:45:110
3942451991816,2cyclictest4000880-21kworker/u8:1+flush-259:020:10:263
3942451991816,1cyclictest0-21swapper/321:55:003
3942451991812,4cyclictest4151669-21latency_hist22:30:003
3942448991815,2cyclictest4150766-21kworker/u8:2+events_unbound22:30:172
3942448991812,4cyclictest92956-21latency_hist00:40:012
3942448991812,4cyclictest4110072-21diskstats21:50:162
3942448991811,5cyclictest3947549-21apt-get19:15:002
3942446991815,2cyclictest4031869-21kworker/u8:1+flush-259:021:05:311
3942446991815,2cyclictest11836-21kworker/u8:3+events_unbound23:25:001
3942446991814,3cyclictest4157106-21kworker/u8:1+events_unbound22:40:151
3942446991814,2cyclictest0-21swapper/100:40:001
3942446991813,3cyclictest459-21apt-get23:10:101
3942441991816,1cyclictest4130226-21sh22:10:000
3942441991815,2cyclictest4162842-21idleruntime-cro22:40:010
3942441991812,5cyclictest14881-21ntpq23:20:190
3942441991811,5cyclictest61159-21ntpq00:05:180
3942451991713,3cyclictest4115050-21apt-get21:55:133
3942451991711,4cyclictest4088625-21latency_hist21:30:003
3942448991714,2cyclictest4016971-21irqcore20:20:172
3942448991713,3cyclictest4130248-21kworker/u8:0+flush-259:022:55:002
3942446991714,2cyclictest82494-21kworker/u8:1+events_unbound00:30:171
3942446991714,2cyclictest82494-21kworker/u8:1+events_unbound00:30:171
3942446991714,2cyclictest62297-21kworker/u8:2+flush-259:000:10:251
3942446991714,2cyclictest4114484-21kworker/u8:1+events_unbound22:05:151
3942446991714,2cyclictest3947584-21kworker/u8:2+events_unbound20:05:261
3942446991714,2cyclictest3922107-21kworker/u8:0+flush-259:019:50:191
3942446991713,3cyclictest5689-21/usr/sbin/munin23:15:211
3942446991713,3cyclictest4031869-21kworker/u8:1+flush-259:021:00:251
3942446991713,3cyclictest4016155-21kworker/u8:4+events_unbound22:02:211
3942446991713,3cyclictest4016155-21kworker/u8:4+events_unbound20:24:211
3942446991713,2cyclictest72334-21kworker/u8:0+flush-259:000:25:271
3942446991713,2cyclictest72334-21kworker/u8:0+flush-259:000:25:261
3942441991715,1cyclictest4031832-21sh20:35:000
3942441991714,2cyclictest60350irq/132-enp0s29f1:rx-023:05:160
3942441991714,2cyclictest60350irq/132-enp0s29f1:rx-021:26:290
3942441991713,4cyclictest60350irq/132-enp0s29f1:rx-021:15:230
3942441991713,3cyclictest507-21in:imuxsock19:45:000
3942441991711,4cyclictest46673-21latency_hist23:55:000
3942441991711,4cyclictest252-21systemd-journal21:00:190
3942451991613,2cyclictest4016155-21kworker/u8:4+flush-259:021:50:003
3942451991612,3cyclictest62297-21kworker/u8:2+flush-259:000:17:053
3942451991612,3cyclictest509-21rs:main3
3942448991611,4cyclictest72331-21cron00:20:002
3942448991611,4cyclictest52028-21/usr/sbin/munin00:00:172
3942448991611,3cyclictest4011180-21kworker/u8:3-ext4-rsv-conversion20:59:592
3942446991614,1cyclictest4146431-21apt-key22:25:001
3942446991613,2cyclictest4184580-21kworker/u8:0+events_unbound23:05:241
3942446991613,2cyclictest4011180-21kworker/u8:3+events_unbound21:11:211
3942446991611,4cyclictest4078238-21kworker/u8:0+flush-259:021:25:251
3942441991614,1cyclictest60350irq/132-enp0s29f1:rx-019:25:110
3942441991613,2cyclictest60350irq/132-enp0s29f1:rx-021:35:180
3942441991613,2cyclictest60350irq/132-enp0s29f1:rx-020:21:240
3942441991613,2cyclictest60350irq/132-enp0s29f1:rx-000:00:190
3942441991613,2cyclictest4048012-21irqcore20:50:150
3942441991610,4cyclictest252-21systemd-journal19:10:190
3942451991511,3cyclictest0-21swapper/319:44:593
394244899158,5cyclictest3974832-21cat19:40:012
394244899158,5cyclictest3969013-21apt-get19:35:002
3942448991513,1cyclictest0-21swapper/219:25:002
3942448991510,3cyclictest4016155-21kworker/u8:4+flush-259:020:55:012
3942446991512,2cyclictest4031822-21cron20:35:001
3942446991511,3cyclictest4130248-21kworker/u8:0+flush-259:022:45:211
3942446991511,3cyclictest4130248-21kworker/u8:0+events_unbound22:50:021
3942446991511,3cyclictest4011180-21kworker/u8:3+events_unbound20:49:211
3942441991513,1cyclictest60350irq/132-enp0s29f1:rx-023:45:190
3942441991513,1cyclictest60350irq/132-enp0s29f1:rx-023:10:060
3942441991513,1cyclictest60350irq/132-enp0s29f1:rx-022:58:460
3942441991513,1cyclictest60350irq/132-enp0s29f1:rx-019:38:480
3942441991513,1cyclictest60350irq/132-enp0s29f1:rx-000:11:040
3942441991512,2cyclictest60350irq/132-enp0s29f1:rx-023:40:110
3942441991512,2cyclictest60350irq/132-enp0s29f1:rx-022:50:230
3942441991512,2cyclictest60350irq/132-enp0s29f1:rx-021:09:350
3942441991512,2cyclictest60350irq/132-enp0s29f1:rx-000:15:220
3942441991512,2cyclictest4114789-21idleruntime-cro21:55:000
3942441991511,3cyclictest3922107-21kworker/u8:0+flush-259:019:15:160
3942441991510,3cyclictest507-21in:imuxsock00:30:010
3942441991510,3cyclictest507-21in:imuxsock00:30:000
3942451991412,1cyclictest0-21swapper/300:04:593
3942451991411,2cyclictest4079586-21sort21:20:163
3942451991411,2cyclictest27351-21kthreadcore23:35:173
3942451991411,2cyclictest0-21swapper/321:40:233
394244899149,3cyclictest4068232-21/usr/sbin/munin21:10:192
394244899145,5cyclictest252-21systemd-journal23:15:002
394244899143,6cyclictest4068009-21awk21:10:012
3942448991411,2cyclictest4026677-21journal-offline20:30:002
3942448991410,3cyclictest3980723-21fschecks_time19:45:152
3942448991410,3cyclictest3980723-21fschecks_time19:45:142
394244699149,3cyclictest4073395-21/usr/sbin/munin21:15:151
394244699147,5cyclictest4015392-21ntpq20:15:221
394244699145,7cyclictest301rcuc/119:30:001
3942446991412,2cyclictest3947584-21kworker/u8:2+events_unbound19:50:011
3942446991412,2cyclictest3947584-21kworker/u8:2+events_unbound19:50:001
3942446991411,2cyclictest4016155-21kworker/u8:4+events_unbound22:17:001
3942446991411,1cyclictest3969371-21latency_hist19:35:011
3942446991410,3cyclictest62297-21kworker/u8:2+flush-259:000:18:341
3942446991410,3cyclictest4016155-21kworker/u8:4+events_unbound21:50:161
3942446991410,3cyclictest3947584-21kworker/u8:2+flush-259:019:38:241
394244199148,1cyclictest4135395-21turbostat22:15:000
394244199148,1cyclictest4135395-21turbostat22:15:000
3942441991412,1cyclictest60350irq/132-enp0s29f1:rx-022:40:190
3942441991412,1cyclictest60350irq/132-enp0s29f1:rx-020:38:500
3942441991412,1cyclictest60350irq/132-enp0s29f1:rx-020:25:330
3942441991412,1cyclictest60350irq/132-enp0s29f1:rx-020:14:530
3942441991412,1cyclictest60350irq/132-enp0s29f1:rx-020:09:330
3942441991412,1cyclictest60350irq/132-enp0s29f1:rx-020:00:130
3942441991412,1cyclictest60350irq/132-enp0s29f1:rx-000:35:330
3942441991412,1cyclictest60350irq/132-enp0s29f1:rx-000:23:530
3942441991411,2cyclictest60350irq/132-enp0s29f1:rx-021:55:260
3942441991411,2cyclictest60350irq/132-enp0s29f1:rx-021:30:110
3942441991411,2cyclictest0-21swapper/022:30:010
3942441991410,2cyclictest0-21swapper/023:20:010
394245199137,4cyclictest51787-21latency_hist00:00:003
394245199133,1cyclictest39847972timerwakeupswit19:45:253
394245199133,1cyclictest39847972timerwakeupswit19:45:253
3942451991311,1cyclictest0-21swapper/322:25:003
3942451991311,1cyclictest0-21swapper/300:14:593
3942451991310,2cyclictest509-21rs:main3
3942451991310,2cyclictest31673-21fschecks_count23:40:133
394244899138,4cyclictest5467-21kworker/u8:1+flush-259:023:30:242
394244899138,3cyclictest4125103-21cron22:05:002
394244899138,3cyclictest4042183-21cron20:45:002
3942448991310,2cyclictest4149083-21kthreadcore22:25:182
3942448991310,2cyclictest4078286-21cron21:20:002
3942448991310,2cyclictest4042525-21cpuspeed_turbos20:45:132
3942448991310,2cyclictest3999051-21sshd20:00:182
3942446991311,1cyclictest5467-21kworker/u8:1+events_unbound00:05:121
394244699130,5cyclictest40348121kthreadcore20:35:171
394244199139,3cyclictest4046466-21ntp_kernel_pll_20:45:210
3942441991312,1cyclictest60350irq/132-enp0s29f1:rx-023:29:590
3942441991311,1cyclictest60350irq/132-enp0s29f1:rx-023:35:240
3942441991311,1cyclictest60350irq/132-enp0s29f1:rx-023:30:150
3942441991311,1cyclictest60350irq/132-enp0s29f1:rx-022:15:230
3942441991311,1cyclictest60350irq/132-enp0s29f1:rx-021:41:170
3942441991311,1cyclictest60350irq/132-enp0s29f1:rx-021:22:350
3942441991311,1cyclictest60350irq/132-enp0s29f1:rx-020:55:140
394245199129,2cyclictest73029-21irqcore00:20:153
394245199129,2cyclictest718-21fschecks_count23:10:143
394245199129,2cyclictest5780-21cpuspeed_turbos23:15:143
394245199129,2cyclictest3922107-21kworker/u8:0+flush-259:019:20:163
394245199129,2cyclictest0-21swapper/323:30:213
394245199128,3cyclictest4130464-21/usr/sbin/munin22:10:143
394245199128,3cyclictest4130464-21/usr/sbin/munin22:10:143
394245199128,3cyclictest3947584-21kworker/u8:2+flush-259:019:37:503
394244899129,2cyclictest72673-21cpuspeed_turbos00:20:112
394244899129,1cyclictest0-21swapper/223:35:222
394244899129,1cyclictest0-21swapper/223:20:252
394244899128,2cyclictest0-21swapper/222:20:002
3942448991210,1cyclictest0-21swapper/219:15:252
394244699129,2cyclictest4078238-21kworker/u8:0+events_unbound21:20:131
394244699128,3cyclictest5467-21kworker/u8:1+events_unbound23:28:181
394245199119,1cyclictest0-21swapper/322:55:153
394245199117,2cyclictest4157246-21awk22:35:003
394245199116,4cyclictest82656-21latency_hist00:30:003
394245199116,4cyclictest82656-21latency_hist00:30:003
394244899119,1cyclictest0-21swapper/219:25:172
394244899118,2cyclictest4179074-21cpuspeed_turbos22:55:112
394244899118,2cyclictest4107051-21kthreadcore21:45:182
394244899118,2cyclictest0-21swapper/220:10:172
394244899117,2cyclictest4103195-21ntpq21:40:222
394244899115,4cyclictest4115209-21diskstats21:55:152
394244699118,2cyclictest4114484-21kworker/u8:1+events_unbound21:55:021
394244699118,2cyclictest20936-21kworker/u8:0+flush-259:023:50:151
394245199108,1cyclictest0-21swapper/323:50:173
394245199107,1cyclictest0-21swapper/320:45:013
394245199106,3cyclictest4016155-21kworker/u8:4+flush-259:020:45:163
394245199100,8cyclictest252-21systemd-journal22:39:183
394244899108,1cyclictest0-21swapper/200:30:122
394244899108,1cyclictest0-21swapper/200:30:122
394244899105,4cyclictest3986095-21sed19:50:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional