You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-11 - 03:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Sun Jan 11, 2026 00:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1577003214455,84sleep30-21swapper/319:06:443
172301621350,2sleep00-21swapper/021:25:120
157696329780,9sleep10-21swapper/119:06:151
157714528057,6sleep00-21swapper/019:08:310
157227727661,9sleep20-21swapper/219:05:032
1577382992823,2cyclictest0-21swapper/321:40:113
157738299224,11cyclictest1730568-21taskset21:30:173
157738299222,10cyclictest17977821taskset22:35:223
157738299221,17cyclictest1833458-21turbostat23:10:003
1577382992118,1cyclictest0-21swapper/323:55:113
157738299211,16cyclictest1600369-21sleep19:30:013
1577382992015,3cyclictest0-21swapper/320:25:133
1577382992014,4cyclictest0-21swapper/319:45:193
157738299201,15cyclictest228-1kworker/3:1H+kblockd23:18:193
157738299201,15cyclictest1632285-21turbostat19:59:593
157738299201,14cyclictest0-21swapper/323:29:243
157738299194,10cyclictest1912627-21taskset00:25:123
157738299193,2cyclictest0-21swapper/323:40:203
157738299193,13cyclictest0-21swapper/321:35:113
157738299193,10cyclictest1643308-21taskset20:10:013
157738299193,10cyclictest1622020-21taskset19:50:013
157738299192,14cyclictest0-21swapper/300:05:213
157738299191,14cyclictest0-21swapper/321:25:113
157738299191,13cyclictest0-21swapper/320:15:123
157738299190,15cyclictest1669943-21sleep20:35:003
157738299183,11cyclictest1928065-21turbostat00:40:003
157738299183,11cyclictest1859636-21sleep23:35:003
157738299183,11cyclictest1749651-21chrt21:50:133
157738299183,11cyclictest1643092-21sleep20:09:593
157738299183,11cyclictest15850irq/127-ahci[0000:00:17.0]20:22:483
157738299183,10cyclictest1907415-21taskset00:20:103
157738299183,10cyclictest1816496-21taskset22:50:203
1577382991815,1cyclictest0-21swapper/321:10:193
1577382991812,2cyclictest0-21swapper/320:35:133
157738299181,14cyclictest0-21swapper/323:10:193
157738299181,14cyclictest0-21swapper/321:55:193
157738299181,13cyclictest1886453-21chrt00:00:113
157738299181,13cyclictest1827967-21turbostat23:04:593
157738299181,13cyclictest1582604-21turbostat19:15:003
157738299181,13cyclictest1582604-21turbostat19:14:593
157738299181,13cyclictest0-21swapper/323:50:123
157738299181,13cyclictest0-21swapper/322:15:113
157738299181,13cyclictest0-21swapper/321:20:113
157738299180,14cyclictest0-21swapper/322:30:093
157738299177,3cyclictest16801502sleep320:41:193
157738299176,8cyclictest1743889-21turbostat21:49:553
157738299175,9cyclictest1812155-21turbostat22:50:013
157738299174,2cyclictest0-21swapper/322:05:203
157738299173,9cyclictest1680775-21taskset20:45:123
157738299173,11cyclictest1611462-21chrt19:40:033
157738299173,10cyclictest1875740-21chrt23:50:003
157738299173,10cyclictest1717369-21turbostat21:20:003
157738299173,10cyclictest1695093-21chrt20:55:183
157738299172,11cyclictest1610673-21chrt19:36:573
157738299172,10cyclictest1701339-21taskset21:04:553
157738299171,13cyclictest228-1kworker/3:1H+kblockd19:31:493
157738299171,13cyclictest0-21swapper/320:50:143
157738299171,12cyclictest228-1kworker/3:1H+kblockd19:22:353
157738299171,12cyclictest1865127-21turbostat23:40:003
157738299171,12cyclictest1701627-21chrt21:05:123
157738299171,12cyclictest0-21swapper/322:40:213
18268452161,9sleep21826933-21ntpq23:00:182
17600492160,1sleep10-21swapper/122:00:141
157738299164,3cyclictest0-21swapper/323:22:223
157738299163,9cyclictest1906809-21taskset00:16:143
157738299163,9cyclictest1637430-21taskset20:04:123
157738299161,9cyclictest1764641-21taskset22:03:243
1577382991613,2cyclictest0-21swapper/319:15:013
1577382991612,2cyclictest0-21swapper/300:30:253
157738299161,13cyclictest0-21swapper/322:25:213
157738299160,13cyclictest0-21swapper/322:55:213
18756022151,9sleep21875737-21cat23:50:002
17551482151,9sleep01755961-21grep21:55:160
16441212151,6sleep01645791-21kthreadcore20:10:180
16380332151,3sleep21638052-21apt-get20:05:122
16156162151,9sleep21615695-21ps19:40:232
1577382991512,2cyclictest0-21swapper/300:10:173
157738299150,12cyclictest0-21swapper/322:20:113
157738299150,12cyclictest0-21swapper/322:10:273
18134912141,4sleep11815178-21cut22:50:161
17598392141,4sleep01760048-21expr22:00:120
17341822144,5sleep01734496-21kthreadcore21:35:150
16635872140,2sleep2421rcuc/220:25:192
16436042141,8sleep11643324-21/usr/sbin/munin20:10:151
19076832131,8sleep11907865-21gltestperf00:20:131
18504942131,8sleep21852193-21kthreadcore23:25:152
17917802131,7sleep21791985-21sed22:30:122
15814782132,5sleep01581566-21ntpq19:10:190
15814782132,5sleep01581566-21ntpq19:10:180
18764312121,7sleep11877429-21kthreadcore23:50:161
18499562126,3sleep10-21swapper/123:25:141
18442202120,1sleep00-21swapper/023:20:130
18389372121,5sleep01838956-21apt-get23:15:110
17179192121,7sleep11717955-21fschecks_count21:20:141
16491542121,7sleep11649891-21cut20:15:171
15947982120,2sleep20-21swapper/219:21:352
19183482111,6sleep01918431-21kernelversion00:30:150
19070592111,7sleep10-21swapper/100:19:221
18546442111,5sleep11854663-21apt-get23:30:121
18111062111,5sleep21811152-21ntpq22:45:202
18109932111,5sleep11811105-21ntpq22:45:191
17862802111,4sleep21786298-21apt-get22:25:122
17493732111,4sleep11749391-21apt-get21:50:121
17116322110,1sleep20-21swapper/221:12:502
17068322111,5sleep00-21swapper/021:10:110
17063682111,3sleep072-21kcompactd021:07:260
16914672111,5sleep11691590-21perl20:55:121
16707402111,2sleep00-21swapper/020:35:160
16167592116,1sleep10-21swapper/119:45:101
19235972101,4sleep01923667-21irqstats00:35:150
19218932101,4sleep21921949-21ntp_states00:30:202
19129012101,4sleep11913109-21hddtemp_smartct00:25:151
18966642101,5sleep1553-21dbus-daemon00:10:001
18759792101,4sleep21876150-21diskstats23:50:132
18711262101,4sleep11871194-21irqstats23:45:151
18484592101,4sleep11848464-21sensors_temp23:20:261
18216972101,4sleep01821743-21ntpq22:55:190
17810662101,4sleep01781085-21apt-get22:20:080
17743932101,2sleep00-21swapper/022:10:180
17601122101,4sleep21760320-21hddtemp_smartct22:00:152
17585712100,1sleep10-21swapper/121:55:201
17548692101,5sleep21755019-21head21:55:132
17336692100,1sleep10-21swapper/121:35:121
17216512101,4sleep01721697-21ntpq21:20:210
16331372104,2sleep134-21ksoftirqd/120:00:171
16215382100,1sleep20-21swapper/219:47:032
15848032101,4sleep21584822-21apt-get19:15:092
15815732101,5sleep11581626-21ntpq19:10:201
15815732101,5sleep11581626-21ntpq19:10:191
1923118291,4sleep11923262-21cstates00:35:131
1908979291,4sleep01910732-21tr00:20:160
1901566290,1sleep20-21swapper/200:10:482
1900887291,3sleep01900928-21ntpq00:10:180
1898357293,3sleep11900100-21kthreadcore00:10:151
1893248291,4sleep01895004-21cut00:05:180
1886117291,5sleep04061759-21cupsd00:00:010
1875862291,3sleep01875881-21apt-get23:50:100
1837811290,1sleep00-21swapper/023:10:230
1827709290,1sleep10-21swapper/123:02:071
1812939291,4sleep01813490-21cut22:50:150
1802011291,4sleep11802224-21df_inode22:40:131
1792060291,4sleep01793059-21kthreadcore22:30:140
1785902290,1sleep20-21swapper/222:23:312
1779835291,3sleep21779891-21ntpq22:15:222
1776383291,4sleep11777742-21grep22:15:171
1750231291,4sleep21751892-21kthreadcore21:50:172
1744158291,4sleep21744249-21grep21:45:112
1733512291,3sleep20-21swapper/221:35:012
1733379290,6sleep00-21swapper/021:34:560
1728386291,2sleep10-21swapper/121:30:121
1713572291,4sleep01715276-21cut21:15:150
1694952291,4sleep21695092-21ntpq20:55:182
1690923291,3sleep01690952-21munin-run20:55:000
1689907291,4sleep11689953-21/usr/sbin/munin20:50:201
1664944291,3sleep01664963-21apt-get20:30:100
1655705291,4sleep11657472-21grep20:20:171
1654434291,4sleep21654599-21grep20:20:142
1653028291,3sleep01653213-21users20:15:280
1638002291,4sleep01637507-21apt-get20:05:020
1632533291,3sleep00-21swapper/020:00:120
1631905291,4sleep072-21kcompactd019:55:400
1631222291,4sleep11631266-21gcc19:55:211
1628000291,4sleep21629630-21cut19:55:172
1617048291,4sleep01617176-21gltestperf19:45:140
1613491291,4sleep11615262-21tr19:40:181
1610915291,4sleep01610935-21munin-run19:40:000
1604633291,4sleep21604679-21ntpq19:30:212
1600606291,3sleep11600624-21apt-get19:30:121
1595880291,4sleep21596385-21cat19:25:152
1582238290,1sleep20-21swapper/219:10:452
1582238290,1sleep20-21swapper/219:10:442
1927357283,1sleep20-21swapper/200:35:262
1907393281,2sleep20-21swapper/200:20:102
1906105281,3sleep01906148-21ntpq00:15:200
1890486280,4sleep01890539-21ntpq00:00:200
1886431280,2sleep20-21swapper/200:00:112
1885494281,3sleep21885735-21grep23:55:252
1867191281,3sleep11868979-21kthreadcore23:40:181
1861808281,3sleep01862179-21kthreadcore23:35:160
1860640281,3sleep21860709-21irqstats23:35:132
1855217281,3sleep21856588-21kthreadcore23:30:172
1854938281,3sleep01855148-21hddtemp_smartct23:30:150
1828757281,2sleep10-21swapper/123:05:141
1807437283,2sleep0221rcuc/022:45:130
1802495281,3sleep01803094-21cut22:40:170
1796728280,1sleep20-21swapper/222:35:182
1793063281,3sleep11794818-21kthreadcore22:30:151
1771770281,3sleep21772611-21kthreadcore22:10:172
1771109282,3sleep11771769-21kthreadcore22:10:151
1769194281,3sleep11769283-21ntpq22:05:191
1753449280,1sleep00-21swapper/021:50:220
1748248280,5sleep10-21swapper/121:45:221
1738903280,2sleep20-21swapper/221:40:112
1728990281,3sleep21730567-21cut21:30:172
1727847281,3sleep21728164-21latency_hist21:30:002
1726181281,3sleep11726727-21sort21:25:181
1701311280,1sleep20-21swapper/221:04:352
1681265281,3sleep21682070-21kthreadcore20:45:162
1679756281,3sleep21679948-21sed20:40:262
1674225281,3sleep11674284-21ntp_offset20:35:211
1670460280,4sleep21670641-21expr20:35:142
1668961281,3sleep21669018-21ntpq20:30:202
1665515281,3sleep11666646-21kthreadcore20:30:151
1658451281,3sleep01658464-21systemctl20:20:240
1647716281,3sleep21647930-21unixbench_singl20:10:282
1642710280,2sleep10-21swapper/120:05:351
1622566281,3sleep01622829-21kthreadcore19:50:140
1612056281,3sleep01613490-21cat19:40:170
1610326281,3sleep21610542-21users19:35:252
1606160281,3sleep11606262-21perl19:35:121
1598071281,3sleep01599182-21memory19:25:170
1589822281,3sleep01589926-21latency_hist19:20:000
1912605270,2sleep21912624-21apt-get00:25:122
1906773270,5sleep20-21swapper/200:15:462
1896643270,1sleep20-21swapper/200:09:452
1865966271,2sleep00-21swapper/023:40:160
1859607271,2sleep11860105-21gpgv23:35:001
1854051270,1sleep00-21swapper/023:25:490
1843620270,1sleep20-21swapper/223:16:212
1839621271,2sleep10-21swapper/123:15:161
1822549270,1sleep20-21swapper/222:58:192
1822373270,2sleep10-21swapper/122:56:041
1804798270,3sleep20-21swapper/222:40:182
1801436270,1sleep00-21swapper/022:37:170
1786268270,4sleep10-21swapper/122:25:031
1785757270,1sleep10-21swapper/122:21:411
1779738270,4sleep00-21swapper/022:15:200
1765872271,2sleep20-21swapper/222:05:152
1765272271,2sleep01764774-21apt-get22:05:020
1722350270,1sleep20-21swapper/221:21:232
1717066270,1sleep20-21swapper/221:16:212
1707638271,2sleep10-21swapper/121:10:161
1697031271,2sleep10-21swapper/121:00:181
1696913271,2sleep01697029-21kthreadcore21:00:170
1687262271,3sleep21688994-21kthreadcore20:50:162
1684681270,4sleep10-21swapper/120:45:231
1676187271,3sleep01677852-21kthreadcore20:40:170
1659469271,2sleep00-21swapper/020:25:020
1637182270,1sleep20-21swapper/220:01:032
1600931271,3sleep00-21swapper/019:30:150
1922770260,1sleep10-21swapper/100:34:121
1917482260,4sleep00-21swapper/000:28:470
1890763261,2sleep10-21swapper/100:00:251
1885176260,2sleep10-21swapper/123:55:211
1865684260,2sleep20-21swapper/223:40:152
1844196260,1sleep20-21swapper/223:20:112
1838540260,1sleep20-21swapper/223:13:182
1834308261,2sleep10-21swapper/123:10:161
1828450260,2sleep00-21swapper/023:05:010
1801450260,1sleep10-21swapper/122:37:261
1743760260,1sleep10-21swapper/121:43:311
1743522260,1sleep00-21swapper/021:40:290
1716663261,2sleep10-21swapper/121:15:241
1701605261,1sleep21701624-21apt-get21:05:122
1692438261,2sleep00-21swapper/020:55:150
1663219260,2sleep10-21swapper/120:25:191
1649892261,2sleep20-21swapper/220:15:182
1626665260,1sleep10-21swapper/119:50:481
1600138260,4sleep10-21swapper/119:27:221
1590102260,2sleep10-21swapper/119:20:101
1870916250,1sleep00-21swapper/023:45:140
1825625250,1sleep00-21swapper/023:00:170
1815179250,1sleep20-21swapper/222:50:162
1685693250,1sleep00-21swapper/020:49:540
1680176250,1sleep10-21swapper/120:41:371
1626068250,2sleep20-21swapper/219:50:212
1589649250,1sleep10-21swapper/119:17:481
1828786240,1sleep20-21swapper/223:05:162
1786561240,1sleep00-21swapper/022:25:140
1743928240,1sleep00-21swapper/021:45:020
1701901240,1sleep10-21swapper/121:05:141
1590101241,1sleep00-21swapper/019:20:090
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional