You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-16 - 10:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot4s.osadl.org (updated Mon Mar 16, 2026 00:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26705993933,4cyclictest348819-21kworker/u8:2+flush-259:000:25:163
26697993931,5cyclictest348590-21latency_hist00:20:020
26700993834,2cyclictest213770-21/usr/sbin/munin22:10:141
26705993629,5cyclictest252-21systemd-journal21:15:003
26705993629,5cyclictest252-21systemd-journal21:14:593
26701993632,3cyclictest265504-21kworker/u8:2+flush-259:023:35:002
26701993632,3cyclictest265504-21kworker/u8:2+flush-259:023:35:002
26697993629,4cyclictest276253-21diskstats23:10:150
2669799360,34cyclictest1-21systemd22:25:010
26697993528,5cyclictest265906-21diskstats23:00:130
26700993429,4cyclictest301969-21apt-get23:35:131
26705993328,3cyclictest73238-21kworker/u8:3+flush-259:020:40:223
26700993228,3cyclictest312565-21base6423:45:011
26700993128,2cyclictest114488-21kworker/u8:0+flush-259:023:15:001
26700993127,3cyclictest322988-21apt-get23:55:111
26697993123,5cyclictest252-21systemd-journal22:45:000
26705993026,3cyclictest265504-21kworker/u8:2+flush-259:023:20:153
26705993025,3cyclictest114488-21kworker/u8:0+flush-259:021:10:013
26705993025,3cyclictest114488-21kworker/u8:0+flush-259:021:10:003
26705993024,4cyclictest252-21systemd-journal21:44:593
26701992926,2cyclictest328123-21kworker/u8:4+flush-259:000:10:012
26697992924,3cyclictest83360-21kworker/u8:2+flush-259:020:05:150
26697992921,5cyclictest155172-21ntpq21:10:210
26697992921,5cyclictest155172-21ntpq21:10:200
26705992824,3cyclictest249635-21kworker/u8:3+flush-259:023:50:153
26705992822,4cyclictest327890-21latency_hist00:00:013
26700992822,4cyclictest252-21systemd-journal20:19:221
26697992821,5cyclictest252-21systemd-journal21:50:000
26697992820,5cyclictest78376-21latency_hist20:00:010
26701992723,3cyclictest312030-21apt-get23:45:002
26701992723,3cyclictest130024-21gpgv20:50:002
26700992724,2cyclictest224076-21apt-get22:20:121
26700992722,3cyclictest52703-21/usr/sbin/munin19:35:151
26697992722,3cyclictest171562-21gpgv21:30:000
26697992721,4cyclictest161235-21latency_hist21:20:010
26700992623,2cyclictest234319-21kworker/u8:2+flush-259:022:35:201
26701992522,2cyclictest73238-21kworker/u8:3+flush-259:020:15:002
26701992521,3cyclictest182473-21apt-get21:40:112
26697992519,4cyclictest145771-21latency_hist21:05:000
26705992421,2cyclictest73238-21kworker/u8:3+flush-259:020:25:273
26701992420,3cyclictest338525-21apt-get00:10:102
26700992421,2cyclictest197669-21kworker/u8:3+flush-259:022:25:211
26700992420,3cyclictest252-21systemd-journal20:39:591
26705992319,3cyclictest63093-21kworker/u8:0+flush-259:019:50:243
26705992319,3cyclictest114488-21kworker/u8:0+flush-259:022:35:013
26705992319,3cyclictest114488-21kworker/u8:0+flush-259:022:05:213
26700992319,3cyclictest166634-21/usr/sbin/munin21:25:141
26697992313,9cyclictest60350irq/132-enp0s29f1:rx-022:25:200
26705992219,2cyclictest249635-21kworker/u8:3+events_unbound23:00:143
26705992219,2cyclictest197669-21kworker/u8:3+flush-259:022:00:143
26705992219,2cyclictest1952385-21kworker/u8:1+events_unbound19:55:203
26705992218,3cyclictest73238-21kworker/u8:3+flush-259:020:30:203
26705992218,3cyclictest343448-21kworker/u8:0+flush-259:000:20:173
26705992218,3cyclictest239504-21kworker/u8:1+events_unbound22:40:183
26705992218,3cyclictest114488-21kworker/u8:0+flush-259:021:15:143
26701992218,3cyclictest140607-21latency_hist21:00:012
26701992217,3cyclictest252-21systemd-journal22:11:182
26700992219,2cyclictest255205-21kworker/u8:4+flush-259:023:00:001
26700992217,3cyclictest172195-21/usr/sbin/munin21:30:141
2669799228,12cyclictest42171-21turbostat19:25:010
26705992117,3cyclictest83360-21kworker/u8:2+flush-259:020:05:203
26705992117,3cyclictest73238-21kworker/u8:3+events_unbound20:20:203
26705992117,3cyclictest252-21systemd-journal20:35:533
26705992117,3cyclictest218672-21kworker/u8:1+events_unbound22:25:153
26701992118,2cyclictest275853-21kworker/u8:1+flush-259:023:40:002
26701992118,2cyclictest234319-21kworker/u8:2+flush-259:022:50:002
26700992118,2cyclictest114488-21kworker/u8:0+events_unbound22:20:011
26700992116,3cyclictest47573-21apt-get19:30:121
26697992115,4cyclictest260791-21diskstats22:55:140
26697992115,4cyclictest119894-21apt-get20:40:100
26705992016,3cyclictest88745-21kworker/u8:1+events_unbound20:10:253
26705992016,3cyclictest358922-21kworker/u8:1+events_unbound00:35:163
26705992016,3cyclictest218672-21kworker/u8:1+flush-259:022:25:013
26700992017,2cyclictest63093-21kworker/u8:0+events_unbound19:50:221
26700992017,2cyclictest114488-21kworker/u8:0+flush-259:021:39:591
26700992015,4cyclictest291383-21latency_hist23:25:001
26700992014,4cyclictest252-21systemd-journal00:11:231
2669799208,11cyclictest0-21swapper/000:35:000
2669799205,2cyclictest0-21swapper/023:35:260
2669799201,4cyclictest202797-21idleruntime-cro22:00:000
2669799201,4cyclictest119636-21cron20:40:000
26697992014,4cyclictest208796-21diskstats22:05:130
26705991916,2cyclictest63093-21kworker/u8:0+flush-259:020:00:203
26705991916,2cyclictest58336-21irqcore19:40:163
26705991916,2cyclictest1952385-21kworker/u8:1+flush-259:019:15:013
26705991916,2cyclictest114488-21kworker/u8:0+flush-259:020:50:153
26705991915,3cyclictest197669-21kworker/u8:3+flush-259:021:57:303
26701991917,1cyclictest0-21swapper/221:20:002
26701991916,2cyclictest114488-21kworker/u8:0+events_unbound22:20:152
26701991915,3cyclictest250024-21latency_hist22:45:002
26701991913,4cyclictest108407-21ntpq20:25:202
26701991912,5cyclictest290496-21ntpq23:20:202
26700991916,2cyclictest358922-21kworker/u8:1+flush-259:000:30:171
26700991916,2cyclictest249635-21kworker/u8:3+flush-259:023:15:241
2669799195,3cyclictest182612-21cut21:40:120
2669799195,1cyclictest0-21swapper/019:10:120
2669799194,2cyclictest217634-21needreboot22:10:190
26697991915,3cyclictest79043-21irqcore20:00:180
2669799191,4cyclictest255181-21munin-plugin-st22:50:000
2669799191,4cyclictest252-21systemd-journal23:10:000
2669799191,4cyclictest252-21systemd-journal20:55:000
2669799191,4cyclictest244768-21grep22:40:000
2669799191,3cyclictest358895-21apt00:30:000
2669799191,3cyclictest353764-21idleruntime-cro00:25:000
26697991913,4cyclictest125011-21apt-get20:45:110
2669799190,5cyclictest317561-21idleruntime-cro23:50:000
2669799190,4cyclictest51851-21/usr/sbin/munin19:30:250
2669799190,4cyclictest177225-21wc21:35:010
26705991814,3cyclictest343688-21apt-get00:15:113
26705991814,3cyclictest114488-21kworker/u8:0+events_unbound23:35:163
26701991815,2cyclictest31850-21kworker/u8:2+events_unbound19:25:242
26701991815,2cyclictest275853-21kworker/u8:1+events_unbound23:50:242
26701991815,2cyclictest145771-21latency_hist21:05:002
26701991814,3cyclictest249635-21kworker/u8:3+flush-259:023:25:252
26701991813,4cyclictest99272-21/usr/sbin/munin20:20:152
26701991813,4cyclictest114488-21kworker/u8:0+flush-259:021:10:232
26701991813,4cyclictest114488-21kworker/u8:0+flush-259:021:10:222
26700991815,2cyclictest88745-21kworker/u8:1+events_unbound20:45:221
26700991815,2cyclictest207973-21gpgv22:05:011
26700991814,3cyclictest249635-21kworker/u8:3+flush-259:000:05:221
26700991814,3cyclictest245099-21apt-get22:40:111
2669799186,11cyclictest0-21swapper/023:40:130
2669799185,1cyclictest0-21swapper/020:15:140
2669799183,3cyclictest56655-21grep19:35:200
2669799183,3cyclictest368669-21grep00:35:250
2669799182,3cyclictest322759-21grep23:55:000
2669799181,4cyclictest343413-21idleruntime-cro00:15:000
2669799181,4cyclictest285394-21/usr/sbin/munin23:15:210
2669799181,3cyclictest252-21systemd-journal21:55:000
2669799181,3cyclictest252-21systemd-journal21:00:000
2669799181,3cyclictest252-21systemd-journal20:24:590
2669799181,2cyclictest333071-21awk00:05:010
2669799181,2cyclictest150939-21dump-pmu-power21:10:000
2669799181,2cyclictest150939-21dump-pmu-power21:10:000
2669799180,5cyclictest238339-21/usr/sbin/munin22:30:200
2669799180,5cyclictest108566-21/usr/sbin/munin20:25:240
2669799180,4cyclictest63545-21grep19:45:160
2669799180,4cyclictest260372-21idleruntime-cro22:55:000
2669799180,4cyclictest223813-21sh22:20:000
2669799180,3cyclictest46839-21grep19:25:240
26705991714,2cyclictest500-21systemd-logind00:05:003
26705991713,3cyclictest114488-21kworker/u8:0+events_unbound22:15:023
26705991710,5cyclictest252-21systemd-journal19:40:013
26701991714,2cyclictest177096-21kworker/u8:2+flush-259:021:50:162
26701991714,2cyclictest135455-21cron20:54:592
2669799173,3cyclictest288012-21cut23:20:170
2669799171,3cyclictest72715-21tail19:50:240
2669799171,3cyclictest36108-21ntpq19:15:210
2669799171,3cyclictest208314-21cat22:05:000
26697991713,3cyclictest60350irq/132-enp0s29f1:rx-023:30:100
26697991713,3cyclictest60350irq/132-enp0s29f1:rx-023:30:100
26697991711,5cyclictest113559-21ntpq20:30:220
2669799170,4cyclictest58038-21grep19:40:120
2669799170,4cyclictest337811-21tail00:05:240
2669799170,4cyclictest296553-21/usr/sbin/munin23:30:000
2669799170,4cyclictest165427-21grep21:20:180
2669799170,3cyclictest177748-21irqcore21:35:160
26705991613,2cyclictest42177-21kworker/u8:3+flush-259:019:31:073
26705991613,2cyclictest31850-21kworker/u8:2+flush-259:019:26:163
26705991613,2cyclictest2971-21cron23:00:003
26705991613,2cyclictest275853-21kworker/u8:1+events_unbound23:40:203
26705991613,2cyclictest197650-21idleruntime-cro21:55:003
26705991613,2cyclictest156093-21kworker/u8:3+flush-259:021:35:193
26705991613,2cyclictest114488-21kworker/u8:0+events_unbound21:30:153
26705991612,3cyclictest249635-21kworker/u8:3+flush-259:022:50:193
26705991612,3cyclictest249635-21kworker/u8:3+events_unbound22:48:243
26705991612,2cyclictest0-21swapper/300:15:003
26701991613,2cyclictest343448-21kworker/u8:0+flush-259:000:35:002
26701991610,4cyclictest270884-21/usr/sbin/munin23:05:152
26700991613,2cyclictest114488-21kworker/u8:0+events_unbound20:50:241
26700991611,4cyclictest73430-21/usr/sbin/munin19:55:211
2670099160,9cyclictest140620-21awk21:00:001
2669799166,1cyclictest91353-21kthreadcore20:10:190
2669799160,4cyclictest327433-21grep23:55:240
26705991511,3cyclictest275853-21kworker/u8:1+events_unbound23:45:213
26705991511,3cyclictest249635-21kworker/u8:3+events_unbound23:30:003
26705991511,3cyclictest114488-21kworker/u8:0+events_unbound21:22:233
26705991511,2cyclictest197669-21kworker/u8:3+flush-259:022:10:193
26701991512,3cyclictest140632-21kworker/u8:1+flush-259:021:45:282
26701991512,2cyclictest313134-21irqcore23:45:162
26701991511,3cyclictest31850-21kworker/u8:2+flush-259:019:20:232
2670099159,4cyclictest252-21systemd-journal22:55:001
26700991511,3cyclictest307151-21apt-get23:40:111
26700991511,3cyclictest140870-21apt-get21:00:091
26700991511,3cyclictest114488-21kworker/u8:0+flush-259:022:45:151
26700991510,3cyclictest252-21systemd-journal00:26:231
26705991411,2cyclictest88745-21kworker/u8:1+flush-259:020:47:593
26705991411,2cyclictest114488-21kworker/u8:0+events_unbound23:35:013
26705991411,2cyclictest114488-21kworker/u8:0+events_unbound23:35:003
26705991411,2cyclictest114488-21kworker/u8:0+events_unbound21:25:203
26705991410,3cyclictest249635-21kworker/u8:3+events_unbound23:15:133
26705991410,3cyclictest140632-21kworker/u8:1+events_unbound21:46:153
26705991410,2cyclictest0-21swapper/322:39:593
26701991412,1cyclictest0-21swapper/222:05:152
26701991411,2cyclictest171653-21apt-key21:30:012
26700991411,2cyclictest31850-21kworker/u8:2+flush-259:019:20:131
2670599139,3cyclictest265504-21kworker/u8:2+events_unbound23:11:273
26705991310,2cyclictest1952385-21kworker/u8:1+events_unbound19:22:223
2670199138,4cyclictest364449-21diskstats00:35:142
2670199135,7cyclictest0-21swapper/222:05:012
2670199131,6cyclictest119646-21awk20:40:002
26701991312,1cyclictest197669-21kworker/u8:3+events_unbound21:55:232
26701991310,2cyclictest281449-21fschecks_count23:15:132
2670099131,11cyclictest490-21dbus-daemon21:15:001
2670099131,11cyclictest490-21dbus-daemon21:14:591
26700991310,2cyclictest79049-21irqcore20:00:181
26700991310,2cyclictest109662-21cpuspeed_turbos20:30:141
2670599128,3cyclictest124770-21kworker/u8:2+events_unbound21:00:153
2670199129,2cyclictest239600-21cpuspeed_turbos22:35:112
2670199129,2cyclictest123855-21ntp_kernel_pll_20:40:192
2670199128,3cyclictest509-21rs:main2
2670199127,4cyclictest77458-21ntpq19:55:192
2670199120,5cyclictest88693-21idleruntime-cro20:10:012
2670099129,3cyclictest343448-21kworker/u8:0+flush-259:000:35:241
2670099129,2cyclictest301249-21unixbench_multi23:30:251
2670099129,2cyclictest301249-21unixbench_multi23:30:251
2670099129,2cyclictest187831-21fschecks_count21:45:141
2670099129,2cyclictest165920-21unixbench_multi21:20:231
2670099129,1cyclictest0-21swapper/121:40:161
2670099127,4cyclictest252-21systemd-journal20:45:001
2670099125,5cyclictest103694-21grep20:20:241
26700991210,1cyclictest0-21swapper/123:50:131
2670199119,1cyclictest276266-21fschecks_count23:10:142
2670199119,1cyclictest0-21swapper/219:45:152
2670199119,1cyclictest0-21swapper/219:40:212
2670199119,1cyclictest0-21swapper/219:30:132
2670199118,2cyclictest328380-21fschecks_time00:00:152
2670199118,2cyclictest323111-21cpuspeed_turbos23:55:132
2670199118,2cyclictest165988-21unixbench_singl21:20:242
2670199117,3cyclictest63093-21kworker/u8:0+events_unbound20:00:222
2670199111,3cyclictest99037-21latency_hist20:20:012
2670199110,2cyclictest132-1kworker/2:1H+kblockd20:35:002
2670099118,2cyclictest198049-21tr21:55:151
2670099118,2cyclictest0-21swapper/121:15:131
2670599106,3cyclictest31850-21kworker/u8:2+flush-259:019:50:003
2670199104,4cyclictest53014-21fw_forwarded_lo19:35:152
2670099108,1cyclictest311ktimers/120:10:011
2670099107,2cyclictest348951-21cpuspeed_turbos00:20:131
267019996,2cyclictest176641-21grep21:30:252
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional