You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-27 - 04:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Sat Dec 27, 2025 00:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1497422213160,66sleep30-21swapper/319:09:303
149737627858,14sleep00-21swapper/019:08:550
149734327762,9sleep20-21swapper/219:08:292
149729927559,10sleep10-21swapper/119:07:561
15743002300,6sleep11574305-21gltestperf20:20:131
15743002300,6sleep11574305-21gltestperf20:20:131
1497585992822,3cyclictest18053232chrt00:00:003
1497585992522,1cyclictest0-21swapper/320:40:003
1497585992511,10cyclictest1762845-21turbostat23:20:003
1497585992319,1cyclictest0-21swapper/319:25:123
1497585992317,2cyclictest0-21swapper/320:00:003
149758599231,16cyclictest1630490-21taskset21:10:263
1497585992310,11cyclictest0-21swapper/319:45:273
1497585992219,1cyclictest0-21swapper/300:10:153
149758599219,9cyclictest1669709-21chrt21:50:143
149758599214,14cyclictest15581882sleep320:05:123
149758599213,12cyclictest15739002sleep320:20:123
149758599213,12cyclictest15739002sleep320:20:123
149758599212,13cyclictest17460732sleep323:00:213
149758599202,12cyclictest1673366-21taskset21:55:113
149758599202,11cyclictest1608173-21taskset20:50:183
1497585992017,1cyclictest0-21swapper/320:25:173
1497585992017,1cyclictest0-21swapper/319:35:133
1497585991916,1cyclictest0-21swapper/323:36:193
1497585991916,1cyclictest0-21swapper/323:10:123
1497585991916,1cyclictest0-21swapper/322:40:163
1497585991916,1cyclictest0-21swapper/319:40:173
1497585991915,2cyclictest0-21swapper/323:45:123
1497585991915,2cyclictest0-21swapper/321:45:183
1497585991911,4cyclictest16105262sleep320:55:123
149758599191,14cyclictest0-21swapper/300:20:103
149758599191,13cyclictest0-21swapper/321:15:113
149758599191,11cyclictest1510300-21taskset19:20:113
15260502182,4sleep21526069-21apt-get19:35:102
149758599181,4cyclictest1600337-21taskset20:45:133
1497585991813,2cyclictest0-21swapper/321:30:003
149758599181,14cyclictest0-21swapper/322:50:003
149758599181,13cyclictest1752319-21turbostat23:10:013
149758599181,13cyclictest1594823-21chrt20:40:123
149758599181,13cyclictest0-21swapper/323:30:213
149758599181,12cyclictest1502769-21sleep19:15:003
149758599181,10cyclictest1845813-21taskset00:35:163
149758599174,10cyclictest0-21swapper/321:40:173
149758599173,2cyclictest0-21swapper/320:15:203
149758599172,10cyclictest1683790-21taskset22:05:133
149758599171,6cyclictest1810643-21turbostat00:05:003
1497585991715,1cyclictest0-21swapper/321:35:203
1497585991714,1cyclictest0-21swapper/321:00:153
1497585991714,1cyclictest0-21swapper/320:05:003
1497585991714,1cyclictest0-21swapper/300:30:183
1497585991713,2cyclictest0-21swapper/320:10:133
149758599171,14cyclictest0-21swapper/321:30:263
149758599171,13cyclictest0-21swapper/322:25:203
149758599171,12cyclictest1773336-21turbostat23:30:003
149758599171,12cyclictest1773336-21turbostat23:30:003
149758599171,12cyclictest1510019-21turbostat19:20:003
149758599170,13cyclictest0-21swapper/320:30:223
17357202161,11sleep21731642-21/usr/sbin/munin22:50:222
149758599161,9cyclictest1694275-21taskset22:15:073
149758599161,9cyclictest1546526-21taskset19:53:023
1497585991612,2cyclictest0-21swapper/300:25:113
149758599161,13cyclictest0-21swapper/323:50:163
149758599161,12cyclictest1826335-21turbostat00:20:013
149758599161,11cyclictest1814837-21chrt00:05:213
149758599161,11cyclictest0-21swapper/321:05:123
149758599160,13cyclictest0-21swapper/322:55:193
149758599160,12cyclictest0-21swapper/323:40:113
149758599160,12cyclictest0-21swapper/321:20:103
149758599160,10cyclictest1704746-21taskset22:24:333
149758599160,10cyclictest1704746-21taskset22:24:333
16986872150,3sleep20-21swapper/222:15:262
16527772151,9sleep11652949-21cat21:35:181
15842152150,3sleep20-21swapper/220:30:002
15743862151,9sleep21575530-21cut20:20:162
15743862151,9sleep21575530-21cut20:20:152
149758599158,3cyclictest16800242sleep322:00:173
149758599151,9cyclictest1767730-21taskset23:20:323
149758599151,9cyclictest1736409-21taskset22:52:313
149758599151,9cyclictest1715447-21taskset22:33:313
149758599151,9cyclictest1525464-21taskset19:30:493
149758599150,13cyclictest0-21swapper/322:35:213
149758599150,12cyclictest0-21swapper/322:10:143
17731072142,8sleep2800-21snmpd23:27:222
17731072142,8sleep2800-21snmpd23:27:222
17593172140,9sleep01761022-21tr23:15:170
17573612141,9sleep21757364-21sshd23:12:272
17047812141,9sleep11704828-21apt-get22:25:001
17047812141,9sleep11704828-21apt-get22:25:001
15374912144,5sleep01539144-21cut19:45:180
18318542131,8sleep11832061-21expr00:25:121
17673882131,7sleep21767580-21sed23:20:242
17423902131,8sleep11742585-21expr23:00:151
16267932131,8sleep21627513-21grep21:10:182
16188602131,4sleep11619588-21ls21:00:191
15898062131,4sleep11589937-21grep20:35:141
17917592121,7sleep21793528-21grep23:45:172
17566082123,4sleep11756653-21ntpq23:10:211
17336952121,7sleep11735382-21latency22:50:171
17163962121,1sleep00-21swapper/022:35:170
16840502121,7sleep01684118-21fschecks_time22:05:150
16514352121,7sleep21651531-21timerandwakeup21:30:262
16111592121,7sleep11612756-21kthreadcore20:55:181
16006072121,7sleep11601117-21grep20:45:161
15580192120,8sleep21558187-21grep20:05:122
15406032121,7sleep21540733-21plymouth19:45:242
17991182111,5sleep21799172-21ntpq23:50:222
17671082111,5sleep11767157-21ntpq23:20:191
16738532110,7sleep21673345-21/usr/sbin/munin21:55:152
16576672110,2sleep00-21swapper/021:40:100
16458722111,5sleep01645909-21ntpq21:25:190
16431232111,6sleep11644811-21kthreadcore21:25:161
16316312111,4sleep01631754-21diskstats21:15:130
16300642110,6sleep10-21swapper/121:10:211
16209962111,3sleep20-21swapper/221:05:132
16040312111,5sleep21604080-21ntpq20:45:212
15987712111,4sleep21598809-21ntpq20:40:202
15671042115,3sleep134-21ksoftirqd/120:10:201
15617282111,5sleep01561774-21ntpq20:05:180
15407822111,6sleep11540969-21timerwakeupswit19:45:271
15199652111,4sleep01520162-21users19:25:260
18427972101,4sleep01842868-21irqstats00:35:140
18266202101,4sleep11826878-21df_inode00:20:111
17827462101,4sleep01782792-21ntpq23:35:170
17688422101,4sleep01768913-21irqstats23:25:160
17688422101,4sleep01768913-21irqstats23:25:150
17096882101,4sleep01709935-21users22:25:270
16943032101,5sleep01694556-21df22:15:130
16837682101,4sleep21683787-21apt-get22:05:132
16629012101,4sleep11662920-21apt-get21:45:121
16523912101,4sleep21652410-21apt-get21:35:152
16252782101,5sleep01625451-21sort21:05:270
16197742101,4sleep01619823-21ntpq21:00:220
15895632101,4sleep01589805-21df_inode20:35:130
15353522101,5sleep11535466-21/usr/sbin/munin19:40:221
1842018290,1sleep00-21swapper/000:34:460
1830559291,4sleep01830601-21ntpq00:20:180
1821898291,4sleep01823300-21cat00:15:170
1820108291,4sleep21820162-21ntpq00:10:222
1806454291,4sleep21808090-21kthreadcore00:00:162
1805589291,2sleep10-21swapper/100:00:111
1794833291,4sleep01794879-21systemctl23:50:000
1794525290,1sleep10-21swapper/123:46:091
1788588293,3sleep1553-21dbus-daemon23:40:231
1784346291,4sleep21784602-21df23:40:122
1773601291,4sleep01773778-21expr23:30:120
1763107291,3sleep01763126-21apt-get23:20:110
1738509291,4sleep01740234-21cut22:55:170
1737153291,4sleep11737322-21grep22:55:141
1735615291,3sleep01735664-21ntpq22:50:200
1709258291,4sleep11709315-21ntpq22:25:211
1699764291,3sleep01699243-21apt-get22:20:020
1699764291,3sleep01699243-21apt-get22:20:020
1699104290,1sleep10-21swapper/122:18:231
1689734291,4sleep01691362-21cut22:10:170
1672190291,4sleep01672278-21ps21:50:200
1668701291,4sleep21669708-21cat21:50:142
1659822291,4sleep21661468-21cat21:40:162
1651088291,3sleep01651139-21ntpq21:30:200
1625099291,4sleep11625227-21systemctl21:05:241
1615431290,1sleep20-21swapper/220:59:272
1584827291,4sleep11585025-21grep20:30:171
1572302291,4sleep21572376-21grep20:15:182
1568530290,1sleep10-21swapper/120:15:011
1552701291,4sleep21552771-21irqstats20:00:152
1547514291,4sleep01548750-21grep19:55:160
1545771291,4sleep21545816-21pmu-power19:50:212
1536237291,4sleep21536263-21perf19:45:002
1514092291,4sleep11514226-21netstat19:20:181
1501780291,4sleep21501830-21ld19:10:212
1842869281,3sleep11842902-21kthreadcore00:35:151
1814888281,3sleep01814936-21ls00:05:230
1799067281,3sleep11799116-21ntpq23:50:211
1783765281,3sleep21783917-21apt-key23:40:002
1777643280,1sleep20-21swapper/223:30:222
1751347281,2sleep00-21swapper/023:05:190
1742586281,3sleep21742669-21irqstats23:00:162
1736863281,3sleep21736881-21apt22:55:112
1726410281,3sleep01726623-21cstates22:45:150
1725401281,3sleep21725430-21mailstats22:40:242
1717638281,3sleep11719407-21grep22:35:181
1710569281,2sleep11710588-21apt-get22:30:131
1692825281,3sleep21692981-21/usr/sbin/munin22:10:182
1668132280,3sleep11668151-21apt-get21:50:091
1658271281,4sleep11659821-21cut21:40:151
1647176281,3sleep11647333-21cstates21:30:131
1637300281,3sleep01638698-21kthreadcore21:20:140
1610236280,1sleep20-21swapper/220:54:382
1604378281,3sleep01604636-21grep20:45:270
1588251281,4sleep21588302-21grep20:30:222
1552376280,4sleep00-21swapper/020:00:130
1551231281,4sleep21551455-21grep19:55:262
1516451281,3sleep21518091-21kthreadcore19:25:162
1502831281,3sleep11503233-21cpu19:15:011
1847011270,1sleep20-21swapper/200:37:002
1831832270,2sleep00-21swapper/000:25:110
1815693270,1sleep10-21swapper/100:08:411
1810872270,2sleep21810890-21apt-get00:05:132
1804996270,1sleep10-21swapper/123:56:271
1784680270,1sleep00-21swapper/023:40:140
1780060271,3sleep10-21swapper/123:35:151
1778279270,1sleep10-21swapper/123:31:501
1768914271,2sleep10-21swapper/123:25:161
1768914271,2sleep10-21swapper/123:25:161
1762607270,1sleep10-21swapper/123:17:151
1752307270,4sleep00-21swapper/023:10:000
1751619271,3sleep10-21swapper/123:05:241
1714842270,1sleep20-21swapper/222:30:262
1709947270,1sleep20-21swapper/222:25:292
1683381270,1sleep10-21swapper/122:03:281
1620585270,1sleep20-21swapper/221:03:082
1611037270,3sleep01611158-21cut20:55:180
1605295270,2sleep10-21swapper/120:50:131
1599442270,1sleep00-21swapper/020:40:300
1583392271,3sleep01583632-21grep20:25:270
1574105271,3sleep00-21swapper/020:20:130
1574105271,3sleep00-21swapper/020:20:130
1567701270,1sleep00-21swapper/020:10:520
1561776270,3sleep11561830-21ntpq20:05:201
1545667270,3sleep01545715-21ldconfig19:50:190
1522742271,2sleep10-21swapper/119:30:161
1516060271,2sleep10-21swapper/119:25:151
1502737271,3sleep01502829-21latency_hist19:15:000
1838738261,2sleep10-21swapper/100:30:181
1837378260,3sleep20-21swapper/200:30:152
1829569261,2sleep20-21swapper/200:20:162
1823301261,2sleep10-21swapper/100:15:181
1820057260,1sleep10-21swapper/100:10:211
1816051260,1sleep00-21swapper/000:10:040
1809427260,2sleep01809557-21strings00:00:180
1800006260,1sleep00-21swapper/023:54:170
1743746260,2sleep00-21swapper/023:00:180
1726052260,1sleep00-21swapper/022:44:070
1724968260,2sleep10-21swapper/122:40:191
1720491260,1sleep20-21swapper/222:35:522
1688999260,1sleep10-21swapper/122:10:091
1688515260,1sleep10-21swapper/122:07:081
1683279260,1sleep20-21swapper/222:02:082
1678101260,1sleep10-21swapper/121:57:301
1673537261,2sleep00-21swapper/021:55:130
1667839260,1sleep20-21swapper/221:49:342
1667673260,1sleep00-21swapper/021:47:270
1656700261,2sleep00-21swapper/021:35:290
1646198261,2sleep20-21swapper/221:25:252
1640428260,2sleep21640601-21strings21:20:162
1636282260,1sleep20-21swapper/221:17:552
1595100260,3sleep10-21swapper/120:40:151
1593420260,2sleep20-21swapper/220:35:192
1583918260,4sleep10-21swapper/120:28:241
1572517260,1sleep00-21swapper/020:15:200
1567937260,1sleep20-21swapper/220:13:542
1545989260,3sleep10-21swapper/119:50:241
1535545260,2sleep00-21swapper/019:40:250
1521367261,2sleep00-21swapper/019:30:150
1515237260,1sleep00-21swapper/019:24:500
1510278260,1sleep20-21swapper/219:20:112
1506355261,2sleep00-21swapper/019:15:160
1503264261,2sleep20-21swapper/219:15:022
1498324260,2sleep10-21swapper/119:10:161
1833999250,2sleep20-21swapper/200:25:182
1825427250,2sleep20-21swapper/200:15:232
1805281250,2sleep20-21swapper/200:00:012
1801302250,2sleep00-21swapper/023:55:170
1761724250,1sleep20-21swapper/223:15:192
1751207250,1sleep20-21swapper/223:05:172
1728580250,2sleep10-21swapper/122:45:201
1699964250,1sleep20-21swapper/222:20:162
1699964250,1sleep20-21swapper/222:20:162
1678865250,1sleep00-21swapper/022:00:140
1636892250,1sleep10-21swapper/121:20:111
1629218250,2sleep00-21swapper/021:10:200
1609111250,1sleep00-21swapper/020:50:200
1589199250,1sleep00-21swapper/020:34:160
1556574250,2sleep10-21swapper/120:00:261
1546683250,1sleep10-21swapper/119:55:011
1528278250,1sleep00-21swapper/019:35:160
1524510250,1sleep20-21swapper/219:30:172
1726125240,1sleep20-21swapper/222:45:022
1710562240,1sleep00-21swapper/022:30:080
1631755240,1sleep10-21swapper/121:15:151
1525840240,1sleep10-21swapper/119:35:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional