You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-30 - 19:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackfslot7s.osadl.org (updated Thu Oct 30, 2025 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
152536999284814,2830cyclictest1554361-21kworker/2:0+events20:50:222
152536999284814,2830cyclictest1554361-21kworker/2:0+events20:50:222
152536499284210,2829cyclictest1604507-21kworker/0:2+events23:20:230
152536999280713,2790cyclictest1619101-21kworker/2:3+events00:05:232
152536999275613,2739cyclictest1587066-21kworker/2:2+events22:25:172
152536999272513,2708cyclictest1597793-21kworker/2:0+events22:55:142
152537499270513,2688cyclictest1546442-21kworker/3:0+events20:15:213
152536999257212,2555cyclictest1627309-21kworker/2:0+events00:25:142
152536999257112,2556cyclictest1562012-21kworker/2:2+events21:20:222
152537499255912,2544cyclictest1622229-21kworker/3:1+events00:10:003
152537499255611,2541cyclictest1584600-21kworker/3:0+events22:15:153
152536999255412,2539cyclictest1602885-21kworker/2:1+events23:15:012
152537499255112,2535cyclictest1539916-21kworker/3:2+events20:05:223
152537499247913,2463cyclictest1528641-21kworker/3:0+events19:33:283
152536499247412,2459cyclictest1528607-21kworker/0:0+events19:20:290
152537499245411,2441cyclictest1546442-21kworker/3:0+events20:35:193
152536499245411,2439cyclictest1581519-21kworker/0:1+events22:02:360
152536999245112,2436cyclictest1599375-21kworker/2:1+events23:00:122
152536999243512,2420cyclictest1612281-21kworker/2:1+events23:37:322
152536999243212,2417cyclictest1551147-21kworker/2:2+events20:38:012
152536999242212,2408cyclictest1620950-21kworker/2:0+events00:10:222
152536999242012,2405cyclictest1583355-21kworker/2:0+events22:20:162
152536999242012,2405cyclictest1562012-21kworker/2:2+events21:31:302
152536999241812,2402cyclictest1562012-21kworker/2:2+events21:50:222
152536999241810,2405cyclictest1591280-21kworker/2:0+events22:37:242
152536999241712,2401cyclictest1541705-21kworker/2:1+events20:29:122
152536999241312,2398cyclictest1594555-21kworker/2:1+events22:51:012
152536999240812,2392cyclictest1576448-21kworker/2:1+events21:46:102
152536999240611,2392cyclictest1627309-21kworker/2:0+events00:22:542
152536999240611,2392cyclictest1618826-21kworker/2:2+events00:03:502
152536499240612,2390cyclictest1583177-21kworker/0:2+events22:16:250
152536999240411,2390cyclictest1554361-21kworker/2:0+events20:46:332
15253749924028,2391cyclictest1528641-21kworker/3:0+events19:39:313
152536999240012,2386cyclictest1627309-21kworker/2:0+events00:30:072
152536999239712,2382cyclictest1576448-21kworker/2:1+events22:05:182
152536499239712,2382cyclictest1618617-21kworker/0:0+events23:56:360
152536499239212,2376cyclictest1584785-21kworker/0:0+events22:30:340
152536999239011,2374cyclictest1541705-21kworker/2:1+events20:08:482
152536499239012,2375cyclictest1589889-21kworker/0:1+events22:39:170
152536499238911,2375cyclictest1591493-21kworker/0:2+events22:55:370
152536499238910,2375cyclictest1627492-21kworker/0:1+events00:33:090
152536499238711,2372cyclictest1599394-21kworker/0:1+events23:14:140
152536499238412,2369cyclictest1541491-21kworker/0:0+events20:08:250
152536499237711,2362cyclictest1550966-21kworker/0:1+events20:46:110
15253699923769,2364cyclictest1576448-21kworker/2:1+events21:56:052
152536999237211,2358cyclictest1533499-21kworker/2:0+events19:41:012
152536499237212,2356cyclictest1622042-21kworker/0:1+events00:17:360
152536999237111,2357cyclictest1628913-21kworker/2:1+events00:38:332
152537499236511,2351cyclictest1605970-21kworker/3:2+events23:24:573
152536499235410,2341cyclictest1575472-21kworker/0:2+events21:48:050
152537499235212,2336cyclictest1602670-21kworker/3:0+events23:08:343
152536499234812,2333cyclictest1591493-21kworker/0:2+events22:40:470
152537499234712,2332cyclictest1566680-21kworker/3:1+events21:26:293
15253749923408,2329cyclictest1555779-21kworker/3:0+events20:53:273
15253749923408,2329cyclictest1555779-21kworker/3:0+events20:53:273
152537499234012,2325cyclictest1617163-21kworker/3:0+events23:56:143
152537499233612,2322cyclictest1552515-21kworker/3:2+events20:40:203
152537499233012,2314cyclictest1593031-21kworker/3:1+events22:41:313
152537499232912,2314cyclictest1571790-21kworker/3:2+events21:48:273
152537499232912,2313cyclictest1571790-21kworker/3:2+events21:39:373
152536699232912,2313cyclictest1594807-21kworker/1:2+events22:52:091
152537499232812,2313cyclictest1563717-21kworker/3:0+events21:11:583
152537499232611,2312cyclictest1618857-21kworker/3:1+events00:01:583
152537499232611,2312cyclictest1541614-21kworker/3:1+events19:59:423
152537499232512,2310cyclictest1628764-21kworker/3:1+events00:27:533
152537499232412,2309cyclictest1588133-21kworker/3:0+events22:22:373
15253749923239,2311cyclictest1536069-21kworker/3:1+events19:43:393
152537499231812,2302cyclictest1536069-21kworker/3:1+events19:51:373
152536499231512,2300cyclictest1584785-21kworker/0:0+events22:25:390
152537499231012,2295cyclictest1571790-21kworker/3:2+events21:44:153
152537499230912,2294cyclictest1612898-21kworker/3:1+events23:46:393
152536999230812,2293cyclictest1535154-21kworker/2:2+events19:38:462
152536499230610,2294cyclictest1575472-21kworker/0:2+events21:50:440
152536499230511,2291cyclictest1559479-21kworker/0:1+events21:01:540
152537499230112,2285cyclictest1541614-21kworker/3:1+events20:02:213
152536499230112,2286cyclictest1630771-21kworker/0:2+events00:39:400
152537499229811,2283cyclictest1604305-21kworker/3:1+events23:34:103
152537499228511,2271cyclictest1620622-21kworker/3:0+events00:14:323
152536499228212,2266cyclictest1537977-21kworker/0:2+events20:11:550
152536499227710,2263cyclictest1622042-21kworker/0:1+events00:08:530
152536499227112,2256cyclictest1595977-21kworker/0:1+events22:46:490
15253669922639,2251cyclictest1624975-21kworker/1:0+events00:16:061
15253699922539,2241cyclictest1557267-21kworker/2:1+events21:09:162
152536999223610,2224cyclictest1617224-21kworker/2:1+events23:56:572
152536999223610,2223cyclictest1623485-21kworker/2:1+events00:16:522
15253649922328,2221cyclictest1530266-21kworker/0:1+events19:28:570
152536499223210,2219cyclictest1550966-21kworker/0:1+events20:32:160
152537499223110,2218cyclictest1620622-21kworker/3:0+events00:31:393
152537499223010,2217cyclictest1596342-21kworker/3:2+events22:52:563
152537499222912,2214cyclictest1566680-21kworker/3:1+events21:24:113
152536999222711,2213cyclictest1562012-21kworker/2:2+events21:40:002
152536999222611,2212cyclictest1565229-21kworker/2:0+events21:14:592
152536999222610,2213cyclictest1607531-21kworker/2:2+events23:27:142
152536999222310,2210cyclictest1530387-21kworker/2:1+events19:31:132
152536999222310,2210cyclictest1526654-21kworker/2:2+events19:22:232
15253749922228,2210cyclictest1563919-21kworker/3:3+events21:17:173
152536999222011,2206cyclictest1615623-21kworker/2:2+events23:50:542
152536999221911,2205cyclictest1613994-21kworker/2:0+events23:43:382
15253699922189,2206cyclictest1568461-21kworker/2:1+events21:28:022
15253699922178,2206cyclictest1551147-21kworker/2:2+events20:44:382
152537499221512,2200cyclictest1576659-21kworker/3:1+events21:52:413
152536999221511,2201cyclictest1533499-21kworker/2:0+events19:50:302
15253699922128,2201cyclictest1530387-21kworker/2:1+events19:28:342
15253699922037,2193cyclictest1576448-21kworker/2:1+events22:03:232
152536999220311,2189cyclictest1541705-21kworker/2:1+events20:13:502
15253699921979,2185cyclictest1540035-21kworker/2:2+events19:57:462
152537499219610,2183cyclictest1604305-21kworker/3:1+events23:19:123
15253699921957,2187cyclictest1597793-21kworker/2:0+events23:15:222
152536699219312,2179cyclictest1581637-21kworker/1:0+events22:06:031
152536499219311,2179cyclictest1541491-21kworker/0:0+events20:03:530
15253699921928,2181cyclictest1613994-21kworker/2:0+events23:49:432
152536999219211,2178cyclictest1533499-21kworker/2:0+events19:47:492
152536999219210,2178cyclictest1526654-21kworker/2:2+events19:18:572
152536499219111,2177cyclictest1537977-21kworker/0:2+events19:59:190
152536499219011,2176cyclictest1571970-21kworker/0:2+events21:39:140
152536499219010,2177cyclictest1604507-21kworker/0:2+events23:29:560
152536499219010,2177cyclictest1566868-21kworker/0:0+events21:28:250
152536999218913,2173cyclictest1549404-21kworker/2:0+events20:22:592
152536499218911,2175cyclictest1530266-21kworker/0:1+events19:38:000
152536499218910,2175cyclictest1575472-21kworker/0:2+events21:58:480
152536999218711,2173cyclictest1596158-21kworker/2:2+events22:49:522
152536499218710,2174cyclictest1546541-21kworker/0:0+events20:28:490
15253699921869,2174cyclictest1576448-21kworker/2:1+events22:13:202
15253649921869,2174cyclictest1555902-21kworker/0:0+events20:51:090
15253649921869,2174cyclictest1555902-21kworker/0:0+events20:51:090
15253649921868,2175cyclictest1546541-21kworker/0:0+events20:36:500
15253649921868,2175cyclictest1537977-21kworker/0:2+events19:46:170
152536999218511,2172cyclictest1587066-21kworker/2:2+events22:30:112
152536499218511,2171cyclictest1584785-21kworker/0:0+events22:23:220
152536499218511,2170cyclictest1601020-21kworker/0:2+events23:05:330
152536499218511,2170cyclictest1566868-21kworker/0:0+events21:21:080
152536499218410,2171cyclictest1597760-21kworker/0:0+events23:04:020
15253649921828,2171cyclictest1614469-21kworker/0:0+events23:49:000
15253649921827,2172cyclictest1563562-21kworker/0:2+events21:13:060
152536499218210,2169cyclictest1549284-21kworker/0:2+events20:21:270
15253699921815,2173cyclictest1557267-21kworker/2:1+events20:59:112
152536999218111,2167cyclictest1562012-21kworker/2:2+events21:18:512
152536999218110,2168cyclictest1594555-21kworker/2:1+events22:43:462
152536499218012,2165cyclictest1568491-21kworker/0:1+events21:30:430
152536499218011,2167cyclictest1618617-21kworker/0:0+events00:01:340
15253649921799,2166cyclictest1591493-21kworker/0:2+events22:53:420
152536499217911,2165cyclictest1563562-21kworker/0:2+events21:18:040
15253749921789,2166cyclictest1563717-21kworker/3:0+events21:08:543
152536499217810,2165cyclictest1604507-21kworker/0:2+events23:17:400
152536499217511,2161cyclictest1627492-21kworker/0:1+events00:28:160
152536499217510,2162cyclictest1623645-21kworker/0:2+events00:14:560
152536999216611,2152cyclictest1526654-21kworker/2:2+events19:14:192
152536999216611,2152cyclictest1526654-21kworker/2:2+events19:14:192
152536499216311,2148cyclictest1610709-21kworker/0:0+events23:35:170
152536999215311,2139cyclictest1557267-21kworker/2:1+events21:01:312
152536999214611,2132cyclictest1576448-21kworker/2:1+events22:16:462
15253749921459,2133cyclictest1609168-21kworker/3:0+events23:40:133
152536499214511,2130cyclictest1555902-21kworker/0:0+events20:59:340
152537499214111,2127cyclictest1588133-21kworker/3:0+events22:30:563
152537499214012,2125cyclictest1546442-21kworker/3:0+events20:31:333
152537499213811,2125cyclictest1623892-21kworker/3:2+events00:18:433
152537499213811,2124cyclictest1544867-21kworker/3:1+events20:11:083
152537499213711,2123cyclictest1580160-21kworker/3:1+events22:02:153
152537499213410,2121cyclictest1530298-21kworker/3:2+events19:24:433
15253649921349,2121cyclictest1606320-21kworker/0:1+events23:34:340
152537499213311,2119cyclictest1548309-21kworker/3:2+events20:21:073
152537499213011,2116cyclictest1612898-21kworker/3:1+events23:51:163
152537499212911,2115cyclictest1586207-21kworker/3:1+events22:26:023
152537499212811,2114cyclictest1611039-21kworker/3:2+events23:37:563
152537499212811,2114cyclictest1530298-21kworker/3:2+events19:28:113
152537499212711,2113cyclictest1588133-21kworker/3:0+events22:46:263
152537499212611,2112cyclictest1580160-21kworker/3:1+events22:09:293
152537499212611,2112cyclictest1546442-21kworker/3:0+events20:29:353
152537499212511,2111cyclictest1588133-21kworker/3:0+events22:38:553
152537499211812,2103cyclictest1559110-21kworker/3:1+events20:56:513
152536699211511,2101cyclictest1602931-21kworker/1:1+events23:16:321
152536499211210,2100cyclictest1537977-21kworker/0:2+events19:52:020
152537499211010,2097cyclictest1528641-21kworker/3:0+events19:19:193
15253649921094,2102cyclictest1623645-21kworker/0:2+events00:24:280
152536699210311,2089cyclictest1570450-21kworker/1:2+events21:51:071
152537499210110,2088cyclictest1605970-21kworker/3:2+events23:28:453
152537499210110,2088cyclictest1571790-21kworker/3:2+events21:58:253
15253649920969,2083cyclictest1573726-21kworker/0:1+events21:43:060
152536699208811,2074cyclictest1576870-21kworker/1:1+events22:03:001
152537499207710,2064cyclictest1555779-21kworker/3:0+events20:46:563
15253699920757,2065cyclictest1551147-21kworker/2:2+events20:32:392
152537499207410,2061cyclictest1571790-21kworker/3:2+events21:34:573
152536999207010,2057cyclictest1540035-21kworker/2:2+events20:04:592
15253669920697,2059cyclictest1602931-21kworker/1:1+events23:13:511
152536499206810,2055cyclictest1524929-21kworker/0:2+events19:16:150
152536999206510,2052cyclictest1609734-21kworker/2:0+events23:33:242
15253699920639,2051cyclictest1597793-21kworker/2:0+events23:07:052
15253699920629,2050cyclictest1562012-21kworker/2:2+events21:42:192
152537499206010,2046cyclictest1538284-21kworker/3:0+events19:47:263
152537499205711,2043cyclictest1620622-21kworker/3:0+events00:21:443
152536699205411,2040cyclictest1581637-21kworker/1:0+events22:49:281
152536999204810,2035cyclictest1607531-21kworker/2:2+events23:24:332
152536699203611,2022cyclictest1615445-21kworker/1:1+events00:32:261
152536699203610,2023cyclictest1510377-21kworker/1:0+events20:11:311
15253649920358,2023cyclictest1544835-21kworker/0:1+events20:15:440
152536499203110,2018cyclictest1612547-21kworker/0:2+events23:50:300
152536499202910,2016cyclictest1563562-21kworker/0:2+events21:07:450
152536499202810,2015cyclictest1606320-21kworker/0:1+events23:44:020
152536499202810,2015cyclictest1583177-21kworker/0:2+events22:12:580
152537499202711,2013cyclictest1593031-21kworker/3:1+events22:58:173
152536499202410,2011cyclictest1518394-21kworker/0:1+events19:12:270
152536499202410,2011cyclictest1518394-21kworker/0:1+events19:12:260
152536499201410,2001cyclictest1532135-21kworker/0:0+events19:32:210
15253649920089,1991cyclictest1535361-21kworker/0:0+events19:40:390
15253649920087,1998cyclictest1583177-21kworker/0:2+events22:09:070
15253649919889,1976cyclictest1550966-21kworker/0:1+events20:44:150
15253749919869,1973cyclictest1631938-21kworker/3:2+events00:38:563
15253749919856,1976cyclictest1559110-21kworker/3:1+events21:03:513
152537499197810,1966cyclictest1602670-21kworker/3:0+events23:13:083
15253749919747,1964cyclictest1584600-21kworker/3:0+events22:12:113
15253749919738,1962cyclictest1593031-21kworker/3:1+events23:02:523
15253669919595,1951cyclictest1605754-21kworker/1:0+events23:45:341
15253669919486,1938cyclictest1552601-21kworker/1:0+events21:03:281
152536699194810,1936cyclictest1546173-21kworker/1:2+events20:45:021
15253669919457,1935cyclictest1562166-21kworker/1:2+events21:08:311
152537499193110,1918cyclictest1521738-21kworker/3:2+events19:14:423
152537499193110,1918cyclictest1521738-21kworker/3:2+events19:14:423
15253669918955,1887cyclictest1510377-21kworker/1:0+events19:15:501
152536699152149,2cyclictest1546173-21kworker/1:2+pm20:44:521
1523829214795,41sleep10-21swapper/119:05:151
15238262147105,23sleep30-21swapper/319:05:133
15248542140108,29sleep20-21swapper/219:06:152
15248422140107,28sleep00-21swapper/019:06:060
152536699341,29cyclictest1569433-21ntpq21:25:181
1525366992511,12cyclictest873-10wireplumber21:00:001
1525366992312,4cyclictest744-21snmpd23:01:441
152536699220,21cyclictest0-21swapper/123:35:121
152536699220,20cyclictest1623963-21awk00:15:011
152536699220,20cyclictest1616764-21proc_pri23:50:211
152536699220,11cyclictest1622296-21sleep00:10:001
1525366992117,2cyclictest1576814-21idleruntime-cro21:50:001
152536699211,18cyclictest0-21swapper/122:25:011
152536699211,17cyclictest1593180-21sendmail-msp22:40:001
152536699211,16cyclictest1585267-21grep22:15:151
152536699201,17cyclictest1559155-21idleruntime-cro20:55:001
152536699201,17cyclictest1559155-21idleruntime-cro20:55:001
152536699201,16cyclictest1589999-21idleruntime-cro22:30:001
152536699201,16cyclictest1580235-21systemctl22:00:011
152536699201,14cyclictest1625708-21apt-get00:20:011
152536699200,15cyclictest1547337-21/usr/sbin/munin20:15:191
152536699200,15cyclictest0-21swapper/123:40:171
152536699191,16cyclictest1540923-21ntpq19:55:171
152536699191,15cyclictest1573637-21munin-run21:40:001
152536699191,15cyclictest1554415-21plymouth20:40:001
152536699191,14cyclictest1573922-21apt-get21:40:111
1525366991910,7cyclictest1536822-21taskset19:42:561
152536699190,18cyclictest1567727-21/usr/sbin/munin21:20:211
152536699190,18cyclictest1530929-21taskset19:25:091
152536699190,17cyclictest0-21swapper/123:00:011
152536699190,15cyclictest1538839-21apt-get19:50:131
152536699186,10cyclictest1530431-21http19:25:001
1525366991811,5cyclictest1606549-21df_inode23:20:141
152536699181,13cyclictest1601065-21apt-get23:05:021
152536699181,12cyclictest0-21swapper/122:30:181
152536699180,9cyclictest1534062-21/usr/sbin/munin19:35:151
152536699180,15cyclictest1631467-21ntpq00:35:181
152536699180,10cyclictest1608714-21grep23:25:201
152536699175,10cyclictest16189852chrt23:59:591
1525366991714,2cyclictest1609713-21gdbus23:30:011
152536699171,15cyclictest0-21swapper/120:09:111
152536699171,13cyclictest1619347-21apt-get00:00:111
152536699171,13cyclictest1570696-21packagekitd21:30:011
152536699171,13cyclictest1567078-21tr21:20:001
152536699171,13cyclictest1551251-21basename20:29:591
1525366991710,5cyclictest1538226-21taskset19:45:451
152536699170,5cyclictest1565217-21chrt21:11:591
152536699170,10cyclictest1526999-21cat19:15:011
152536699170,10cyclictest1526999-21cat19:15:011
152536699161,12cyclictest1542498-21ntpq20:00:181
152536699161,11cyclictest1583858-21df_inode22:10:141
1525366991610,4cyclictest1628949-21runrttasks00:27:461
152536699160,15cyclictest1532488-21/usr/sbin/munin19:30:211
152536699160,14cyclictest1593915-21ntpq22:40:171
152536699160,14cyclictest1549454-21sshd20:22:431
1525366991513,1cyclictest0-21swapper/120:32:211
1525369991210,1cyclictest1547674-21taskset20:15:252
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional