You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-28 - 08:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Sat Mar 28, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
622033993530,4cyclictest803720-21apt-get22:05:122
622032993226,4cyclictest252-21systemd-journal22:15:001
622032993127,3cyclictest948275-21gpgv00:25:011
622033993026,2cyclictest798317-21kworker/u8:2+flush-259:022:20:172
622032993026,3cyclictest938124-21apt-get00:15:101
622032993024,4cyclictest621992-21latency_hist19:10:001
622032992821,5cyclictest860245-21cp22:59:591
622037992721,4cyclictest788281-21apt-get21:50:123
622037992721,4cyclictest252-21systemd-journal23:46:403
622033992723,3cyclictest927820-21apt-get00:05:132
622032992620,4cyclictest730912-21gpgv20:55:001
622029992622,3cyclictest684296-21kworker/u8:0+flush-259:020:35:160
622037992520,3cyclictest891713-21/usr/sbin/munin23:30:153
622037992515,8cyclictest906936-21kworker/u8:2+flush-259:000:10:003
622033992521,3cyclictest641750-21kworker/u8:2+events_unbound20:10:132
622033992521,3cyclictest252-21systemd-journal22:57:002
622033992520,4cyclictest252-21systemd-journal22:20:002
622033992518,5cyclictest252-21systemd-journal00:00:012
622032992519,5cyclictest959137-21apt-get00:35:121
622037992419,3cyclictest762479-21apt-get21:25:113
622033992318,4cyclictest839538-21latency_hist22:40:012
622032992316,5cyclictest757105-21kworker/u8:1+flush-259:021:30:171
622037992217,3cyclictest252-21systemd-journal00:17:363
622037992214,6cyclictest812930-21ntpq22:10:233
622032992218,3cyclictest710351-21/usr/sbin/munin20:35:141
622029992220,1cyclictest579513-21kworker/0:1+pm19:49:210
622033992116,4cyclictest632519-21/usr/sbin/munin19:20:152
622032992116,3cyclictest948159-21apt-get00:25:021
622037992016,3cyclictest694670-21kworker/u8:3+flush-259:021:10:013
622037992016,3cyclictest694670-21kworker/u8:3+flush-259:021:10:003
622037992013,4cyclictest953736-21latency_hist00:30:013
622033992017,2cyclictest632297-21kworker/u8:1+events_unbound19:45:152
622033992016,3cyclictest720512-21kworker/u8:0+events_unbound20:50:192
622032992015,4cyclictest865884-21apt-get23:05:121
622032992015,3cyclictest252-21systemd-journal21:24:271
622032992014,4cyclictest891472-21latency_hist23:30:011
622032992014,4cyclictest891472-21latency_hist23:30:011
622032992013,5cyclictest252-21systemd-journal19:45:001
622029992017,2cyclictest632297-21kworker/u8:1+flush-259:019:45:000
622037991915,3cyclictest906910-21latency_hist23:45:013
622033991913,5cyclictest949011-21diskstats00:25:142
622033991912,5cyclictest797440-21ntpq21:55:222
62203299192,5cyclictest694317-21modprobe20:16:261
622032991914,4cyclictest891738-21apt-get23:30:111
622029991915,3cyclictest736637-21apt-get21:00:110
622037991815,2cyclictest726337-21irqcore20:50:173
622037991814,3cyclictest757105-21kworker/u8:1+events_unbound21:35:153
622037991813,4cyclictest782874-21kworker/u8:0+flush-259:022:00:163
622033991816,1cyclictest0-21swapper/200:25:012
622033991815,2cyclictest0-21swapper/222:30:002
62203299185,2cyclictest834642-21apt-get22:35:121
62203299183,9cyclictest833499-21ntpq22:30:221
622032991811,5cyclictest642596-21awk19:30:001
622029991815,2cyclictest509-21rs:main0
622029991815,2cyclictest0-21swapper/021:15:000
622029991814,3cyclictest932689-21kworker/u8:4+flush-259:000:20:160
622029991813,3cyclictest885373-21ntpq23:20:190
622037991715,1cyclictest737656-21kthreadcore21:00:173
622037991714,2cyclictest784134-21kthreadcore21:45:163
622037991713,3cyclictest782874-21kworker/u8:0+flush-259:022:10:013
62203399177,7cyclictest704949-21idleruntime-cro20:29:592
622033991715,1cyclictest0-21swapper/223:45:002
622033991714,2cyclictest937865-21sh00:15:002
622033991713,3cyclictest641750-21kworker/u8:2+flush-259:019:40:252
62203299173,4cyclictest652792-21sed19:35:251
622029991714,2cyclictest0-21swapper/022:45:000
622029991713,3cyclictest876179-21kworker/u8:4+flush-259:023:20:000
622029991713,3cyclictest252-21systemd-journal21:49:560
622037991613,2cyclictest715000-21kworker/u8:2+events_unbound20:55:163
622037991613,2cyclictest632297-21kworker/u8:1+flush-259:019:20:213
622037991611,3cyclictest855160-21/usr/sbin/munin22:55:153
622037991610,4cyclictest818892-21latency_hist22:20:003
62203399169,5cyclictest252-21systemd-journal23:34:592
622033991613,2cyclictest906936-21kworker/u8:2+flush-259:000:00:162
622033991611,3cyclictest252-21systemd-journal22:11:542
622033991611,3cyclictest252-21systemd-journal21:16:262
622033991610,5cyclictest890605-21ntpq23:25:202
622033991610,5cyclictest890605-21ntpq23:25:202
62203299162,4cyclictest885385-21collect223:20:191
622029991614,1cyclictest60350irq/132-enp0s29f1:rx-021:55:150
622029991614,1cyclictest60350irq/132-enp0s29f1:rx-019:50:230
622029991614,1cyclictest60350irq/132-enp0s29f1:rx-019:35:150
622029991613,2cyclictest777673-21sh21:40:010
622029991613,2cyclictest60350irq/132-enp0s29f1:rx-020:45:220
62202999161,13cyclictest252-21systemd-journal21:06:260
62202999161,13cyclictest252-21systemd-journal21:06:250
62203799159,4cyclictest839960-21diskstats22:40:113
622037991512,2cyclictest963118-21ntp_kernel_err00:35:203
622037991512,2cyclictest916335-21ntp_offset23:50:233
622037991512,2cyclictest720493-21latency_hist20:44:593
622037991511,3cyclictest824104-21kworker/u8:3+flush-259:022:45:153
622037991510,4cyclictest694670-21kworker/u8:3+flush-259:021:20:303
622033991511,3cyclictest678683-21unixbench_singl20:00:242
622033991511,3cyclictest642721-21gpgv19:30:012
62203299159,4cyclictest715689-21diskstats20:40:111
62203299158,5cyclictest860570-21base6423:00:011
62203299158,5cyclictest252-21systemd-journal20:00:001
622032991511,3cyclictest844668-21kworker/u8:1+flush-259:023:35:171
622032991511,3cyclictest252-21systemd-journal22:01:531
622032991511,2cyclictest0-21swapper/100:10:001
622032991510,3cyclictest788018-21latency_hist21:50:001
62202999159,4cyclictest709225-21ntpq20:30:230
622029991513,2cyclictest60350irq/132-enp0s29f1:rx-020:55:200
622029991513,2cyclictest60350irq/132-enp0s29f1:rx-020:40:080
622029991513,1cyclictest60350irq/132-enp0s29f1:rx-023:50:200
622029991513,1cyclictest60350irq/132-enp0s29f1:rx-022:50:250
622029991513,1cyclictest60350irq/132-enp0s29f1:rx-022:37:260
622029991513,1cyclictest60350irq/132-enp0s29f1:rx-021:26:000
622029991513,1cyclictest60350irq/132-enp0s29f1:rx-020:15:140
622029991513,1cyclictest60350irq/132-enp0s29f1:rx-020:00:230
622029991513,1cyclictest60350irq/132-enp0s29f1:rx-019:30:240
622029991512,2cyclictest952882-21ntp_offset00:25:230
622029991512,2cyclictest931781-21ntp_kernel_pll_00:05:220
622029991512,2cyclictest60350irq/132-enp0s29f1:rx-023:30:130
622029991512,2cyclictest60350irq/132-enp0s29f1:rx-023:25:100
622029991512,2cyclictest60350irq/132-enp0s29f1:rx-023:25:100
622029991512,2cyclictest60350irq/132-enp0s29f1:rx-019:25:200
622029991512,2cyclictest60350irq/132-enp0s29f1:rx-000:15:150
622029991512,2cyclictest60350irq/132-enp0s29f1:rx-000:10:150
622037991412,1cyclictest699788-21sh20:25:003
622037991411,2cyclictest948435-21latency_hist00:25:013
622037991411,2cyclictest824753-21irqcore22:25:163
622037991410,3cyclictest507-21in:imuxsock00:35:003
62203399148,4cyclictest715251-21cron20:40:002
62203399148,4cyclictest252-21systemd-journal22:05:002
62203399147,5cyclictest252-21systemd-journal23:20:002
62203399141,5cyclictest725675-21munin-run20:49:592
62203399141,5cyclictest6492461irqcore19:35:172
622033991411,2cyclictest875024-21ntp_kernel_err23:10:202
622033991411,2cyclictest861154-21sed23:00:162
622033991411,2cyclictest693736-21ntp_kernel_pll_20:15:192
622033991411,1cyclictest0-21swapper/221:00:252
62203299149,4cyclictest746774-21kworker/u8:0+flush-259:021:20:001
62203299148,4cyclictest876181-21/usr/sbin/munin23:15:201
62203299148,4cyclictest772783-21apt-get21:35:121
62203299142,2cyclictest828352-21ntpq22:25:201
622032991410,2cyclictest0-21swapper/100:00:011
62202999149,4cyclictest828360-21ntpq22:25:200
62202999147,4cyclictest632277-21latency_hist19:20:000
622029991413,1cyclictest60350irq/132-enp0s29f1:rx-023:36:140
622029991413,1cyclictest60350irq/132-enp0s29f1:rx-022:18:490
622029991413,1cyclictest60350irq/132-enp0s29f1:rx-020:08:580
622029991412,1cyclictest694670-21kworker/u8:3+flush-259:021:15:150
622029991412,1cyclictest60350irq/132-enp0s29f1:rx-023:55:230
622029991412,1cyclictest60350irq/132-enp0s29f1:rx-023:40:130
622029991412,1cyclictest60350irq/132-enp0s29f1:rx-022:45:260
622029991412,1cyclictest60350irq/132-enp0s29f1:rx-021:20:230
622029991412,1cyclictest60350irq/132-enp0s29f1:rx-020:25:050
622029991412,1cyclictest60350irq/132-enp0s29f1:rx-019:57:370
622029991412,1cyclictest60350irq/132-enp0s29f1:rx-019:10:120
622029991411,2cyclictest60350irq/132-enp0s29f1:rx-023:45:150
622029991411,2cyclictest60350irq/132-enp0s29f1:rx-022:10:120
622029991411,2cyclictest60350irq/132-enp0s29f1:rx-022:03:580
622029991411,2cyclictest60350irq/132-enp0s29f1:rx-021:30:120
622029991411,2cyclictest60350irq/132-enp0s29f1:rx-020:10:140
622029991410,3cyclictest932172-21kworker/u8:3+flush-259:000:30:180
62203799139,3cyclictest780235-21kthreadcore21:40:163
622037991311,1cyclictest0-21swapper/319:30:013
622037991310,2cyclictest881435-21cpuspeed_turbos23:20:133
622037991310,2cyclictest679450-21cpuspeed_turbos20:05:133
62203799130,11cyclictest751911-21awk21:15:003
62203399138,4cyclictest641750-21kworker/u8:2+flush-259:020:05:262
62203399131,2cyclictest782843-21awk21:45:012
622033991311,1cyclictest0-21swapper/223:45:152
622033991310,2cyclictest881435-21cpuspeed_turbos23:20:132
622033991310,2cyclictest747061-21cpuspeed_turbos21:10:152
62203399130,11cyclictest762218-21awk21:25:002
62203299137,4cyclictest668762-21idleruntime-cro19:55:011
62203299132,3cyclictest32-21ksoftirqd/123:15:001
62203299132,2cyclictest797423-21ntpq21:55:201
62203299131,3cyclictest819134-21/usr/sbin/munin22:20:261
622032991310,2cyclictest933000-21cpuspeed_turbos00:10:101
622032991310,2cyclictest902055-21cpuspeed_turbos23:40:111
622032991310,2cyclictest792690-21unixbench_multi21:50:261
622032991310,1cyclictest0-21swapper/122:15:151
622029991312,1cyclictest60350irq/132-enp0s29f1:rx-020:22:450
622029991312,1cyclictest60350irq/132-enp0s29f1:rx-019:20:150
622029991311,1cyclictest60350irq/132-enp0s29f1:rx-023:01:280
622029991311,1cyclictest60350irq/132-enp0s29f1:rx-022:55:170
622029991311,1cyclictest60350irq/132-enp0s29f1:rx-022:20:230
622029991311,1cyclictest60350irq/132-enp0s29f1:rx-021:52:560
622029991311,1cyclictest60350irq/132-enp0s29f1:rx-021:40:200
622029991311,1cyclictest60350irq/132-enp0s29f1:rx-000:36:260
622029991310,2cyclictest61150irq/133-enp0s29f1:tx-023:10:150
622029991310,2cyclictest60350irq/132-enp0s29f1:rx-022:30:130
62203799129,2cyclictest854942-21kworker/u8:2+flush-259:023:01:313
62203799129,2cyclictest663966-21cpuspeed_turbos19:50:133
62203799128,2cyclictest0-21swapper/319:20:003
62203799127,3cyclictest689687-21/usr/sbin/munin20:15:133
62203799121,9cyclictest922318-21awk00:00:003
62203399129,1cyclictest0-21swapper/221:09:592
62203399129,1cyclictest0-21swapper/221:09:592
62203399128,3cyclictest622235-21kworker/u8:2+events_unbound19:10:162
62203399127,4cyclictest777626-21kworker/u8:3+flush-259:021:45:302
62203399127,3cyclictest844668-21kworker/u8:1+events_unbound22:50:182
62203399121,8cyclictest673949-21latency_hist20:00:012
622033991210,1cyclictest0-21swapper/222:40:142
62203299129,2cyclictest840222-21irqcore22:40:131
62203299126,4cyclictest643292-21apt-get19:30:131
62202999129,1cyclictest0-21swapper/020:54:590
622029991211,1cyclictest60350irq/132-enp0s29f1:rx-023:05:200
62203799117,3cyclictest854942-21kworker/u8:2+flush-259:023:05:263
62203799117,2cyclictest0-21swapper/320:35:173
62203799115,4cyclictest725670-21latency_hist20:50:003
62203399119,1cyclictest0-21swapper/222:45:162
62203399117,3cyclictest694670-21kworker/u8:3+flush-259:020:30:152
62203399117,3cyclictest616515-21kworker/u8:3+flush-259:019:15:172
62203399117,2cyclictest769866-21kthreadcore21:30:182
62203399116,4cyclictest932689-21kworker/u8:4+flush-259:000:35:162
62203399115,4cyclictest793141-21cron21:55:012
62203399115,4cyclictest715682-21df_inode20:40:112
62203799107,2cyclictest0-21swapper/321:55:163
62203799106,3cyclictest844668-21kworker/u8:1+flush-259:022:55:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional