You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 17:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Mon Dec 01, 2025 12:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
167334021910,2sleep3481ktimers/307:41:173
18012332730,2sleep3471rcuc/309:51:153
1637248997368,3cyclictest1733621-21kworker/u8:4+flush-8:009:01:251
1637248996559,4cyclictest1902457-21kworker/u8:0+events_unbound11:51:291
19484452590,5chrt0-21swapper/312:21:133
18213892590,1sleep11821393-21tr10:11:161
1637248995854,3cyclictest1822490-21kworker/u8:1+events_unbound11:05:011
1637245995851,5cyclictest1917597-21kworker/u8:2+events_unbound12:26:240
17045382570,1sleep11704542-21cut08:16:101
1637248995752,4cyclictest1627312-21kworker/u8:0+events_unbound07:11:231
1637248995752,3cyclictest1668990-21kworker/u8:2+flush-8:008:26:181
1637248995650,4cyclictest1708749-21kworker/u8:0+events_unbound08:40:001
1637248995649,5cyclictest1902457-21kworker/u8:0+flush-8:011:46:151
1637248995549,4cyclictest1679423-21kworker/u8:1+flush-8:008:01:221
1637248995348,4cyclictest1917597-21kworker/u8:2+events_unbound12:11:181
1637248995348,4cyclictest1877761-21kworker/u8:0+events_unbound11:11:191
1637245995348,4cyclictest1607480-21kworker/u8:3+flush-8:008:36:260
1637248995248,3cyclictest279109-21kworker/u8:1+flush-8:007:21:231
1637248995248,3cyclictest1768735-21kworker/u8:2+events_unbound11:06:091
1637248995246,5cyclictest1902457-21kworker/u8:0+events_unbound12:01:081
1637248995246,4cyclictest1703062-21ntpq08:11:211
1637248995147,3cyclictest1891587-21kworker/u8:2+flush-8:011:21:241
1637248995139,9cyclictest1886570-21ntpq11:16:191
1637248995047,2cyclictest1778351-21kworker/u8:1+events_unbound09:31:251
1637248995045,4cyclictest1822490-21kworker/u8:1+flush-8:010:46:271
1637248995045,4cyclictest1668990-21kworker/u8:2+flush-8:007:46:251
1637248995044,5cyclictest1607480-21kworker/u8:3+flush-8:007:26:111
1637248995043,5cyclictest1607480-21kworker/u8:3+flush-8:008:31:191
1637248995041,8cyclictest1733621-21kworker/u8:4+events_unbound08:56:201
1637248995041,8cyclictest1733621-21kworker/u8:4+events_unbound08:56:191
1637248994945,3cyclictest1607480-21kworker/u8:3+flush-8:007:36:101
1637248994943,4cyclictest1891587-21kworker/u8:2+flush-8:011:26:201
1637248994938,8cyclictest1936513-21ntpq12:06:191
1637248994842,4cyclictest1778351-21kworker/u8:1+events_unbound09:46:221
1637248994742,3cyclictest1902457-21kworker/u8:0+events_unbound11:36:081
1637248994735,10cyclictest282-21systemd-journal07:31:211
1637245994742,4cyclictest1607480-21kworker/u8:3+flush-8:009:01:230
1637245994742,3cyclictest1902457-21kworker/u8:0+flush-8:011:46:220
1637245994736,8cyclictest1642129-21latency_hist07:15:010
1637248994640,4cyclictest1917597-21kworker/u8:2+events_unbound12:26:121
1637248994640,4cyclictest1822490-21kworker/u8:1+events_unbound10:31:211
1637248994639,6cyclictest1822490-21kworker/u8:1+events_unbound10:26:231
1637248994534,8cyclictest1787631-21ntpq09:36:181
1637245994541,3cyclictest1668990-21kworker/u8:2+flush-8:007:56:240
1637248994440,3cyclictest1768735-21kworker/u8:2+events_unbound10:06:241
1637248994439,3cyclictest1917597-21kworker/u8:2+flush-8:012:36:111
1637248994439,3cyclictest1902457-21kworker/u8:0+flush-8:012:21:181
1637248994435,7cyclictest1922604-21/usr/sbin/munin11:56:161
1637248994337,5cyclictest1607480-21kworker/u8:3+events_unbound08:21:121
1637248994337,4cyclictest1822490-21kworker/u8:1+flush-8:010:56:201
1637248994337,4cyclictest1668990-21kworker/u8:2+events_unbound07:41:221
1637245994338,3cyclictest1607480-21kworker/u8:3+flush-8:007:20:000
1637245994337,4cyclictest1927725-21kworker/u8:1+flush-8:012:31:110
163247224328,9sleep10-21swapper/107:05:361
1637252994237,4cyclictest1803550-21kworker/u8:0+flush-8:009:56:232
1637248994238,3cyclictest1768735-21kworker/u8:2+flush-8:010:01:201
1637248994237,4cyclictest1927725-21kworker/u8:1+flush-8:012:15:051
1637245994234,6cyclictest1658896-21latency_hist07:30:010
1637248994136,4cyclictest1822490-21kworker/u8:1+events_unbound11:31:221
1637248994134,5cyclictest1607480-21kworker/u8:3+events_unbound09:16:271
1637245994136,4cyclictest1822490-21kworker/u8:1+events_unbound11:36:230
1637248994035,3cyclictest1788039-21kworker/u8:3+events_unbound09:51:281
1637248994034,4cyclictest1861828-21ntpq10:51:191
1637248994034,4cyclictest1607480-21kworker/u8:3+flush-8:009:26:181
1637245994035,4cyclictest1668990-21kworker/u8:2+events_unbound07:51:080
1637245994035,3cyclictest1679423-21kworker/u8:1+flush-8:008:00:310
1637248993933,4cyclictest279109-21kworker/u8:1+flush-8:007:16:251
1637248993933,4cyclictest1822490-21kworker/u8:1+flush-8:010:41:311
1637245993933,4cyclictest1917597-21kworker/u8:2+flush-8:011:56:090
1637245993929,8cyclictest1777724-21ntpq09:26:200
1637252993833,4cyclictest1679423-21kworker/u8:1+flush-8:008:06:152
1637248993833,4cyclictest1788039-21kworker/u8:3+flush-8:009:41:201
1637248993826,9cyclictest1846958-21ntpq10:36:201
1637245993833,4cyclictest1679423-21kworker/u8:1+flush-8:008:16:230
1637245993833,3cyclictest1679423-21kworker/u8:1+events_unbound08:11:170
1637245993832,5cyclictest1857452-21kworker/u8:3+flush-8:011:00:050
1637245993831,5cyclictest1891587-21kworker/u8:2+flush-8:011:31:230
1637252993731,4cyclictest279109-21kworker/u8:1+flush-8:007:15:052
1637252993729,6cyclictest282-21systemd-journal08:00:002
1637248993733,3cyclictest1679423-21kworker/u8:1+flush-8:008:06:301
1637248993732,4cyclictest1927725-21kworker/u8:1+events_unbound12:31:071
1637248993731,4cyclictest1768735-21kworker/u8:2+flush-8:009:21:151
1637248993731,4cyclictest1607480-21kworker/u8:3+events_unbound08:51:171
1637248993731,4cyclictest1607480-21kworker/u8:3+events_unbound08:51:171
1637248993729,6cyclictest1732794-21ntpq08:41:191
1637245993731,4cyclictest1607480-21kworker/u8:3+flush-8:008:51:290
1637245993731,4cyclictest1607480-21kworker/u8:3+flush-8:008:51:290
163724599370,35cyclictest1862709-21munin-run10:55:010
1637248993632,3cyclictest1607480-21kworker/u8:3+flush-8:008:46:441
1637245993632,3cyclictest1607480-21kworker/u8:3+flush-8:009:06:140
1637245993629,5cyclictest1668990-21kworker/u8:2+flush-8:007:46:310
1637252993528,5cyclictest1788039-21kworker/u8:3+events_unbound09:50:012
1637252993528,5cyclictest1664065-21apt-get07:35:002
1637245993531,3cyclictest1788039-21kworker/u8:3+events_unbound09:51:230
1637245993530,3cyclictest1768735-21kworker/u8:2+events_unbound09:21:120
1637245993526,6cyclictest1931562-21ntpq12:01:170
1637252993429,4cyclictest1753647-21kworker/u8:1+events_unbound09:11:222
1637248993429,4cyclictest1768735-21kworker/u8:2+events_unbound10:16:211
1637248993429,4cyclictest1679423-21kworker/u8:1+events_unbound07:52:001
1637245993429,3cyclictest1803550-21kworker/u8:0+flush-8:010:06:170
1637245993428,4cyclictest1822490-21kworker/u8:1+flush-8:010:26:220
1637248993329,3cyclictest1822490-21kworker/u8:1+flush-8:011:41:281
1637245993328,3cyclictest1822490-21kworker/u8:1+flush-8:010:56:270
1637245993327,4cyclictest279109-21kworker/u8:1+events_unbound07:21:240
1637245993327,4cyclictest1679423-21kworker/u8:1+flush-8:008:06:230
1637245993324,6cyclictest1942309-21latency_hist12:15:010
1637248993228,3cyclictest1778351-21kworker/u8:1+events_unbound09:55:441
1637245993228,3cyclictest1607480-21kworker/u8:3+events_unbound08:21:260
1637245993227,4cyclictest1902457-21kworker/u8:0+flush-8:011:41:100
1637245993227,3cyclictest1822490-21kworker/u8:1+events_unbound10:36:250
1637245993226,4cyclictest1607480-21kworker/u8:3+flush-8:009:16:200
1637245993226,4cyclictest1607480-21kworker/u8:3+flush-8:008:30:010
1637245993225,5cyclictest412-21systemd-logind11:30:000
1637245993224,6cyclictest1763317-21latency_hist09:15:010
1637245993222,8cyclictest1848447-21diskstats10:41:140
1637245993126,4cyclictest1768735-21kworker/u8:2+flush-8:010:01:240
1637245993126,3cyclictest1822490-21kworker/u8:1+flush-8:010:46:100
1637245993125,4cyclictest1877761-21kworker/u8:0+flush-8:011:11:260
1637245993125,4cyclictest1733294-21grep08:41:260
1637245993123,6cyclictest1917597-21kworker/u8:2+flush-8:012:06:210
19463912300,3sleep00-21swapper/012:16:200
1637252993025,4cyclictest1917597-21kworker/u8:2+events_unbound12:11:282
1637252993021,7cyclictest1802933-21unixbench_multi09:51:272
1637245993025,4cyclictest1822490-21kworker/u8:1+events_unbound11:21:170
1637245993021,4cyclictest1778351-21kworker/u8:1+flush-8:009:56:100
1637252992919,8cyclictest282-21systemd-journal08:05:002
1637248992924,3cyclictest1763386-21awk09:15:001
1637245992925,3cyclictest1708749-21kworker/u8:0+events_unbound09:36:230
1637245992925,3cyclictest1607480-21kworker/u8:3+flush-8:007:30:320
1637245992924,3cyclictest1668990-21kworker/u8:2+events_unbound07:41:180
1637245992923,4cyclictest1768735-21kworker/u8:2+flush-8:010:16:170
16324862292,6sleep00-21swapper/007:05:470
16668162280,4sleep20-21swapper/207:36:152
1637252992824,2cyclictest1678875-21unixbench_singl07:46:322
1637252992824,2cyclictest1641781-21sed07:11:272
1637252992819,8cyclictest1933013-21cpuspeed_turbos12:06:092
1637248992824,3cyclictest1768735-21kworker/u8:2+flush-8:010:21:251
1637248992824,3cyclictest1768735-21kworker/u8:2+flush-8:010:21:251
1637248992822,3cyclictest0-21swapper/109:06:071
1637245992824,3cyclictest1768735-21kworker/u8:2+flush-8:010:21:170
1637245992824,3cyclictest1768735-21kworker/u8:2+flush-8:010:21:170
1637245992823,3cyclictest1948133-21sort12:21:120
1637245992822,4cyclictest1902457-21kworker/u8:0+events_unbound11:51:180
163724599282,24cyclictest1837970-21kworker/0:1+events11:15:340
1637245992821,5cyclictest1664875-21diskstats07:36:090
1637245992820,4cyclictest1873115-21ls11:06:090
17860502270,1sleep30-21swapper/309:36:153
17454922270,2sleep3471rcuc/308:56:143
17454922270,2sleep3471rcuc/308:56:143
1637252992722,4cyclictest1822490-21kworker/u8:1+flush-8:010:11:272
1637252992722,3cyclictest1946874-21unixbench_multi12:16:272
1637252992722,3cyclictest1754017-21fschecks_count09:06:122
1637252992721,5cyclictest1811133-21kworker/u8:3+flush-8:010:01:182
1637252992721,4cyclictest1773129-21unixbench_singl09:21:272
1637252992720,6cyclictest1833696-21fschecks_time10:26:112
1637252992719,7cyclictest1699678-21fschecks_time08:11:122
163724599272,24cyclictest1694083-21kworker/0:0+events10:14:220
1637245992722,4cyclictest1607480-21kworker/u8:3+flush-8:008:46:260
163724599271,25cyclictest1837970-21kworker/0:1+events12:39:460
1637252992623,2cyclictest1892073-21unixbench_singl11:21:252
1637252992623,2cyclictest1789070-21cpuspeed_turbos09:41:092
1637252992623,2cyclictest1744090-21cpuspeed_turbos08:56:082
1637252992623,2cyclictest1744090-21cpuspeed_turbos08:56:082
1637252992621,3cyclictest1778351-21kworker/u8:1+flush-8:009:31:232
1637252992621,3cyclictest1713444-21unixbench_multi08:21:272
1637252992619,5cyclictest1927234-21apt-get12:00:002
1637252992618,6cyclictest282-21systemd-journal10:30:572
1637248992622,3cyclictest1668990-21kworker/u8:2+events_unbound07:56:161
1637245992622,3cyclictest1778351-21kworker/u8:1+flush-8:009:46:280
1637245992620,4cyclictest1788039-21kworker/u8:3+events_unbound09:41:160
1637245992620,4cyclictest1733621-21kworker/u8:4+flush-8:008:56:150
1637245992620,4cyclictest1733621-21kworker/u8:4+flush-8:008:56:150
1637252992520,3cyclictest1738236-21grep08:46:292
1637252992519,4cyclictest1897340-21apt-get11:30:002
1637252992519,4cyclictest18912-21cron09:20:012
1637245992520,4cyclictest1718385-21unixbench_multi08:26:250
1637252992415,7cyclictest1957787-21fschecks_count12:31:102
163725299241,22cyclictest1743187-21unixbench_multi08:51:362
163725299241,22cyclictest1743187-21unixbench_multi08:51:362
163724599242,21cyclictest0-21swapper/009:31:240
1637252992319,3cyclictest1718359-21unixbench_multi08:26:252
1637252992319,2cyclictest1679637-21aten2_r5power_v07:51:082
1637252992315,6cyclictest1749151-21fschecks_time09:01:092
1637252992314,7cyclictest1853127-21apt-get10:46:062
163725299231,10cyclictest1876655-21ntpq11:06:182
163724599232,20cyclictest1694083-21kworker/0:0+events10:33:100
163725299225,3cyclictest161rcu_preempt08:35:002
1637252992218,3cyclictest1728672-21idleruntime-cro08:39:592
1637252992218,2cyclictest1951848-21unixbench_multi12:21:252
1637252992217,3cyclictest1952779-21cpuspeed_turbos12:26:092
1637252992217,3cyclictest0-21swapper/209:36:102
18047322210,3sleep3586-21kworker/3:2-events09:56:133
1637252992118,2cyclictest1878156-21fschecks_count11:11:112
1637252992117,2cyclictest0-21swapper/210:25:002
1637252992117,2cyclictest0-21swapper/210:24:592
1637252992116,3cyclictest1917735-21aten2_r5power_c11:51:112
1637252992113,6cyclictest1658882-21cron07:30:002
163725299204,9cyclictest0-21swapper/210:56:142
1637252992016,2cyclictest0-21swapper/210:40:012
1637252992014,4cyclictest1848432-21seq10:41:132
1637252992011,7cyclictest1910335-21tr11:41:162
1637252992011,6cyclictest1778759-21grep09:30:012
1637252992010,8cyclictest1887554-21sed11:20:012
1637252991915,2cyclictest0-21swapper/210:09:592
1637252991914,3cyclictest1729353-21grep08:41:102
1637252991911,6cyclictest1964532-21tr12:36:152
1637252991911,6cyclictest1860477-21cut10:51:152
1637252991910,7cyclictest1899872-21tr11:31:162
1637252991910,7cyclictest1651755-21tr07:21:142
163725299189,7cyclictest1674231-21cron07:45:002
1637252991810,6cyclictest1930332-21tr12:01:152
1637252991810,6cyclictest1905711-21tr11:36:142
1637252991810,6cyclictest1825635-21tr10:16:152
1637252991810,6cyclictest0-21swapper/211:46:152
16325962183,8sleep30-21swapper/307:06:113
16324782184,10sleep20-21swapper/207:05:412
1637252991713,2cyclictest1708447-21unixbench_multi08:16:262
1637252991712,3cyclictest0-21swapper/211:01:162
19575292156,5sleep30-21swapper/312:31:093
18531012142,8sleep30-21swapper/310:46:073
19624272132,7sleep30-21swapper/312:36:073
19223162132,6sleep30-21swapper/311:55:013
18580652132,7sleep30-21swapper/310:51:083
18132582132,6sleep30-21swapper/310:05:013
17734482132,7sleep30-21swapper/309:25:003
17386122132,6sleep30-21swapper/308:50:013
17386122132,6sleep30-21swapper/308:50:003
17288062132,6sleep30-21swapper/308:40:023
16539452132,6sleep30-21swapper/307:25:013
19528142122,6sleep30-21swapper/312:26:113
19332192122,6sleep30-21swapper/312:06:133
19177202122,5sleep30-21swapper/311:51:123
19028632122,6sleep30-21swapper/311:36:073
18979072122,6sleep30-21swapper/311:31:093
18972482122,6sleep30-21swapper/311:29:053
18728992122,6sleep30-21swapper/311:06:053
18688792121,7sleep30-21swapper/311:01:163
18376482122,6sleep30-21swapper/310:26:283
18233812122,6sleep30-21swapper/310:16:053
18225212122,6sleep30-21swapper/310:11:273
18086462122,6sleep30-21swapper/310:01:083
17981102123,6sleep30-21swapper/309:46:313
17885362122,5sleep30-21swapper/309:40:013
17795552122,6sleep30-21swapper/309:31:143
17590822123,5sleep30-21swapper/309:11:123
17197152122,6sleep30-21swapper/308:31:153
17181542122,6sleep30-21swapper/308:26:253
17039522122,5sleep30-21swapper/308:15:013
16986002122,6sleep30-21swapper/308:06:303
16748212122,6sleep30-21swapper/307:46:163
16645982122,6sleep30-21swapper/307:36:053
16374792122,6sleep30-21swapper/307:11:093
19435652112,5sleep30-21swapper/312:16:163
19413652112,5sleep30-21swapper/312:11:193
19276752112,5sleep30-21swapper/312:00:023
19131762112,5sleep30-21swapper/311:46:143
19121922112,5sleep30-21swapper/311:42:053
18922432112,5sleep30-21swapper/311:23:073
18828472112,5sleep30-21swapper/311:16:093
18781062112,5sleep30-21swapper/311:11:103
18675482112,5sleep30-21swapper/310:58:593
18486382112,5sleep30-21swapper/310:41:173
18472552112,6sleep30-21swapper/310:36:283
18381082111,6sleep30-21swapper/310:30:013
18319972112,5sleep30-21swapper/310:21:193
18319972112,5sleep30-21swapper/310:21:193
17747132112,5sleep30-21swapper/309:26:173
17676122112,5sleep30-21swapper/309:16:253
17537572112,5sleep30-21swapper/309:06:103
17493432112,5sleep30-21swapper/309:01:113
17278412111,6sleep30-21swapper/308:36:243
17100692112,6sleep30-21swapper/308:21:143
17000822111,6sleep30-21swapper/308:11:163
16936152112,5sleep30-21swapper/308:01:253
16844132112,5sleep30-21swapper/307:55:153
16792072112,5sleep30-21swapper/307:50:013
16597072112,5sleep30-21swapper/307:31:103
16542962112,5sleep30-21swapper/307:26:093
16422042112,5sleep30-21swapper/307:15:013
17338632102,5sleep30-21swapper/308:45:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional