You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-13 - 16:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Fri Mar 13, 2026 12:44:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
165802399510502,5cyclictest1752218-21turbostat11:39:550
165802999505498,4cyclictest1695542-21turbostat08:59:552
165803399462455,4cyclictest1755679-21turbostat11:45:003
165803399461453,5cyclictest1762607-21turbostat12:04:593
165802999461452,6cyclictest1667452-21turbostat07:39:562
165802599461453,5cyclictest1686531-21turbostat08:34:551
165802599461453,5cyclictest1686531-21turbostat08:34:551
165803399460453,4cyclictest1688574-21turbostat08:35:003
165803399460453,4cyclictest1688574-21turbostat08:35:003
165803399459452,4cyclictest1745296-21turbostat11:14:593
165803399455447,5cyclictest1743563-21turbostat11:10:003
165802399452445,4cyclictest1747030-21turbostat11:24:550
165803399446439,4cyclictest1659709-21turbostat07:15:003
165802999446439,4cyclictest1674424-21turbostat07:59:552
165803399444437,4cyclictest1672691-21turbostat07:49:593
165803399444437,4cyclictest1672691-21turbostat07:49:593
165802399443435,5cyclictest1707743-21turbostat09:34:550
165803399442435,4cyclictest1720156-21turbostat10:09:553
165803399442435,4cyclictest1720156-21turbostat10:09:553
165803399442435,4cyclictest1697276-21turbostat09:00:003
165803399442433,6cyclictest1718430-21turbostat10:04:553
165802599441432,6cyclictest1741526-21turbostat11:09:551
165802599441432,6cyclictest1732577-21turbostat10:44:551
165802999440433,4cyclictest1755679-21turbostat11:49:552
165802999440432,5cyclictest1736340-21turbostat10:54:552
165802599436429,4cyclictest1757415-21turbostat11:54:561
165802599436429,4cyclictest1757415-21turbostat11:54:551
165802399436393,29cyclictest1748757-21turbostat11:29:560
165802399436393,29cyclictest1748757-21turbostat11:29:560
165803399433426,4cyclictest1718430-21turbostat10:00:003
165802999433426,4cyclictest1657917-21turbostat07:14:552
165803399432424,5cyclictest1748757-21turbostat11:25:013
165803399432423,6cyclictest1683080-21turbostat08:24:553
165803399431422,6cyclictest1674424-21turbostat07:55:003
165803399428421,4cyclictest1759155-21turbostat11:59:553
165802599428421,4cyclictest1665706-21turbostat07:34:551
165802599428403,21cyclictest1681344-21turbostat08:19:561
165803399426418,5cyclictest1690300-21turbostat08:40:013
165803399425416,6cyclictest1766069-21turbostat12:15:013
165803399424417,4cyclictest1693816-21turbostat08:50:003
165802599423415,5cyclictest1700771-21turbostat09:14:551
165803399419412,4cyclictest1716698-21turbostat09:55:003
165802999419408,8cyclictest1762607-21turbostat12:09:552
165802999418411,4cyclictest1699008-21turbostat09:09:552
165803399417408,6cyclictest1670966-21turbostat07:45:003
165802399417408,6cyclictest1669235-21turbostat07:44:550
165803399416408,5cyclictest1734614-21turbostat10:49:553
165802599415408,4cyclictest1679611-21turbostat08:14:551
165802599415408,4cyclictest1679611-21turbostat08:14:551
165803399414407,4cyclictest1747030-21turbostat11:20:003
165803399413404,6cyclictest1693816-21turbostat08:54:553
165802999413404,6cyclictest1676147-21turbostat08:04:552
165802599413406,4cyclictest1716698-21turbostat09:59:551
165803399411404,4cyclictest1683080-21turbostat08:20:003
165803399410402,5cyclictest1739795-21turbostat11:00:003
165803399410378,29cyclictest1769532-21turbostat12:25:003
165802399410403,4cyclictest1743563-21turbostat11:14:550
165802599407400,4cyclictest1690300-21turbostat08:44:551
165803399406399,4cyclictest1727385-21turbostat10:29:553
165802399404395,6cyclictest1723913-21turbostat10:19:550
165803399403396,4cyclictest1741526-21turbostat11:05:003
165803399401392,6cyclictest1707743-21turbostat09:30:003
165803399400393,4cyclictest1712931-21turbostat09:45:003
165803399400365,21cyclictest1712931-21turbostat09:49:553
165803399398391,4cyclictest1681344-21turbostat08:15:013
165803399398391,4cyclictest1681344-21turbostat08:15:013
165803399398371,24cyclictest1759155-21turbostat11:55:003
165803399398371,24cyclictest1759155-21turbostat11:55:003
165802999397390,4cyclictest1662245-21turbostat07:24:552
165802399396389,4cyclictest1771261-21turbostat12:34:550
165802999394387,4cyclictest1714661-21turbostat09:54:562
165802999393384,6cyclictest1729118-21turbostat10:34:552
165803399391384,4cyclictest1663981-21turbostat07:25:003
165803399390383,4cyclictest1711201-21turbostat09:40:003
165803399390383,4cyclictest1669235-21turbostat07:40:003
165802999390383,4cyclictest1705965-21turbostat09:29:552
165802399389366,19cyclictest1769532-21turbostat12:29:550
165802599387376,8cyclictest1764343-21turbostat12:14:551
165803399386379,4cyclictest1705965-21turbostat09:25:003
165803399384377,4cyclictest1667452-21turbostat07:35:003
165802399380374,4cyclictest1725650-21turbostat10:24:550
165802999379371,5cyclictest1672691-21turbostat07:54:552
165802599379371,5cyclictest1702503-21turbostat09:19:551
165803399378372,3cyclictest1702503-21turbostat09:15:003
165803399378371,4cyclictest1677877-21turbostat08:05:003
165803399377351,23cyclictest1725650-21turbostat10:20:003
165803399375351,21cyclictest1750481-21turbostat11:30:003
165803399375351,21cyclictest1750481-21turbostat11:30:003
165802599374365,6cyclictest1739795-21turbostat11:04:551
165802999373367,3cyclictest1730845-21turbostat10:39:552
165803399372365,4cyclictest1730845-21turbostat10:35:003
165803399371365,3cyclictest1692091-21turbostat08:45:003
165803399371362,6cyclictest1734614-21turbostat10:45:003
165802399366358,5cyclictest1663981-21turbostat07:29:550
165802399364358,3cyclictest1688574-21turbostat08:39:550
165803399361355,3cyclictest1773307-21turbostat12:34:593
165802999361355,3cyclictest1704239-21turbostat09:24:552
165802399359351,5cyclictest1750481-21turbostat11:34:550
165802999358351,4cyclictest1738069-21turbostat10:59:552
165802399357351,3cyclictest1773307-21turbostat12:39:550
165802999356348,5cyclictest1753950-21turbostat11:44:552
165802999354347,4cyclictest1659709-21turbostat07:19:552
165802999353345,5cyclictest1684801-21turbostat08:29:552
165803399352344,5cyclictest1723913-21turbostat10:15:003
165803399352344,5cyclictest1723913-21turbostat10:15:003
165802399352346,3cyclictest1709472-21turbostat09:39:550
165802399351345,3cyclictest1766069-21turbostat12:19:560
165802399351345,3cyclictest1766069-21turbostat12:19:560
165803399350344,3cyclictest1764343-21turbostat12:10:003
165803399349324,22cyclictest1700771-21turbostat09:10:003
165803399346339,4cyclictest1775033-21turbostat12:40:013
165802999344337,4cyclictest1721875-21turbostat10:14:552
165802999344337,4cyclictest1721875-21turbostat10:14:552
165803399341335,4cyclictest1662245-21turbostat07:20:003
165803399341334,4cyclictest1753950-21turbostat11:40:003
165803399340334,3cyclictest1767810-21turbostat12:20:003
165803399340334,3cyclictest1767810-21turbostat12:20:003
165803399340333,4cyclictest1665706-21turbostat07:30:003
165803399339313,23cyclictest1697276-21turbostat09:04:553
165803399333325,5cyclictest1732577-21turbostat10:40:003
165803399330323,4cyclictest1704239-21turbostat09:20:003
165802399329307,19cyclictest1745296-21turbostat11:19:550
165803399325318,4cyclictest1686531-21turbostat08:30:003
165802599324318,3cyclictest1711201-21turbostat09:44:551
165803399316309,4cyclictest1676147-21turbostat08:00:013
165803399314308,3cyclictest1752218-21turbostat11:35:003
165802999310304,3cyclictest1677877-21turbostat08:09:552
165803399246220,22cyclictest1738069-21turbostat10:55:003
165803399179165,10cyclictest1757415-21turbostat11:50:013
165803399171156,11cyclictest1771261-21turbostat12:30:003
165803399164156,5cyclictest1727385-21turbostat10:25:003
169447821090,6sleep10-21swapper/108:50:181
1657701210977,22sleep00-21swapper/007:07:270
1657875210774,23sleep10-21swapper/107:09:421
1657801210774,23sleep30-21swapper/307:08:473
1657785210774,23sleep20-21swapper/207:08:342
16580339910194,4cyclictest1679611-21turbostat08:10:003
16580339910190,8cyclictest1709472-21turbostat09:35:003
17352642820,4chrt1735263-21/usr/sbin/munin10:45:182
17660852750,4sleep21766075-21munin-run12:15:002
197399580,7rtkit-daemon1972-21rtkit-daemon11:24:321
165802999547,17cyclictest34-21ksoftirqd/210:10:002
165802999547,17cyclictest34-21ksoftirqd/210:10:002
165802999545,32cyclictest2193-21gdbus09:10:212
1658029995243,5cyclictest331rcuc/212:20:232
1658023995211,28cyclictest1695543-21sh08:55:000
1658029995116,19cyclictest34-21ksoftirqd/212:30:022
1658029995017,18cyclictest34-21ksoftirqd/210:41:222
1658023994927,17cyclictest1756430-21irqstats11:45:220
17559392470,9sleep126-21ksoftirqd/111:45:131
17730762460,14chrt26-21ksoftirqd/112:32:301
1658029994625,8cyclictest34-21ksoftirqd/211:33:462
1658025994624,14cyclictest26-21ksoftirqd/112:05:011
165802999458,18cyclictest34-21ksoftirqd/211:05:242
165802999458,16cyclictest34-21ksoftirqd/210:00:002
1658029994512,12cyclictest34-21ksoftirqd/209:50:022
1658029994510,19cyclictest34-21ksoftirqd/209:45:022
165802999448,20cyclictest34-21ksoftirqd/210:25:012
165802999444,24cyclictest34-21ksoftirqd/207:35:012
1658029994433,8cyclictest34-21ksoftirqd/207:50:012
1658029994433,8cyclictest34-21ksoftirqd/207:50:012
1658029994411,18cyclictest34-21ksoftirqd/211:20:022
1658029994410,19cyclictest34-21ksoftirqd/211:05:002
1658029994410,19cyclictest34-21ksoftirqd/209:20:022
165802599442,12cyclictest1747010-21cron11:20:001
165802999437,20cyclictest34-21ksoftirqd/210:00:232
165802999437,19cyclictest34-21ksoftirqd/211:50:132
165802999437,19cyclictest34-21ksoftirqd/211:50:132
165802999437,18cyclictest34-21ksoftirqd/211:30:002
165802999437,18cyclictest34-21ksoftirqd/211:30:002
165802999436,10cyclictest34-21ksoftirqd/208:25:002
1658029994322,8cyclictest34-21ksoftirqd/209:35:012
1658025994332,7cyclictest1137-21runrttasks07:38:091
1658025994322,13cyclictest26-21ksoftirqd/109:05:001
165802399431,38cyclictest1972-21rtkit-daemon12:22:460
16844192420,6chrt0-21swapper/108:20:281
165802999428,17cyclictest34-21ksoftirqd/212:35:012
165802999427,20cyclictest34-21ksoftirqd/211:25:022
165802999427,17cyclictest34-21ksoftirqd/210:20:002
165802999427,11cyclictest34-21ksoftirqd/208:40:252
165802999426,18cyclictest34-21ksoftirqd/208:45:292
1658029994222,7cyclictest34-21ksoftirqd/207:43:322
1658029994213,16cyclictest34-21ksoftirqd/211:55:132
1658025994221,12cyclictest26-21ksoftirqd/109:25:021
1658025994213,7cyclictest141rcu_preempt10:40:021
165802399428,17cyclictest1720566-21cut10:05:130
165802399428,17cyclictest1720566-21cut10:05:130
1658023994216,8cyclictest13-21ksoftirqd/009:50:120
165802999419,16cyclictest34-21ksoftirqd/208:30:272
165802999419,16cyclictest34-21ksoftirqd/208:30:272
165802999419,16cyclictest34-21ksoftirqd/208:15:132
165802999417,19cyclictest34-21ksoftirqd/208:35:172
165802999417,17cyclictest34-21ksoftirqd/211:13:222
165802999417,11cyclictest34-21ksoftirqd/212:15:132
165802999417,11cyclictest34-21ksoftirqd/212:15:132
1658029994119,8cyclictest34-21ksoftirqd/211:35:112
1658029994116,7cyclictest141rcu_preempt10:30:002
1658025994127,6cyclictest26-21ksoftirqd/110:25:221
1658025994124,13cyclictest26-21ksoftirqd/107:15:041
1658025994121,6cyclictest26-21ksoftirqd/109:50:281
1658025994120,13cyclictest26-21ksoftirqd/108:35:091
1658025994119,18cyclictest1700562-21apport09:07:341
165802399412,35cyclictest1972-21rtkit-daemon12:03:320
1658023994120,16cyclictest1717325-21df_inode09:55:140
165802999407,16cyclictest34-21ksoftirqd/212:40:012
165802999407,11cyclictest34-21ksoftirqd/212:05:002
165802999406,16cyclictest34-21ksoftirqd/207:25:162
1658025994029,7cyclictest26-21ksoftirqd/107:44:091
1658025994022,14cyclictest26-21ksoftirqd/107:55:011
165802399403,11cyclictest141rcu_preempt10:47:570
1658023994024,11cyclictest1702483-21cron09:15:000
1658023994023,12cyclictest956-21systemd-logind09:29:070
1658023994022,14cyclictest1680236-21df08:10:150
1658023994022,14cyclictest1680236-21df08:10:140
165802399401,12cyclictest1743035-21grep11:05:280
165802999397,12cyclictest34-21ksoftirqd/209:35:122
165802999395,19cyclictest34-21ksoftirqd/208:53:062
165802999391,11cyclictest1697697-21grep09:00:142
1658025993923,12cyclictest1709494-21systemd-run09:35:001
1658025993922,13cyclictest26-21ksoftirqd/110:10:091
1658025993922,13cyclictest26-21ksoftirqd/110:10:091
1658025993921,14cyclictest26-21ksoftirqd/108:05:271
1658025993920,14cyclictest26-21ksoftirqd/110:05:001
1658023993923,12cyclictest1734589-21cron10:45:000
1658023993923,12cyclictest1713557-21df09:45:160
1658023993917,9cyclictest13-21ksoftirqd/007:19:340
17211682380,6chrt4497-21PK-Backend10:05:241
17211682380,6chrt4497-21PK-Backend10:05:241
16736242380,7chrt4497-21PK-Backend07:50:210
165802999381,16cyclictest34-21ksoftirqd/208:13:492
165802999381,16cyclictest34-21ksoftirqd/208:13:492
165802599383,11cyclictest141rcu_preempt12:25:301
165802599383,10cyclictest26-21ksoftirqd/112:24:561
1658025993822,12cyclictest1686583-21apt-config08:30:011
1658025993822,11cyclictest26-21ksoftirqd/109:30:001
1658023993824,10cyclictest1686998-21gdbus08:30:030
1658023993824,10cyclictest1686998-21gdbus08:30:030
1658023993822,11cyclictest438-20systemd-journal08:40:250
1658023993820,13cyclictest1723568-40apt-check10:11:210
1658023993820,13cyclictest1723568-40apt-check10:11:200
1658023993815,10cyclictest13-21ksoftirqd/011:58:000
165802399381,33cyclictest1972-21rtkit-daemon11:42:490
165802599374,7cyclictest141rcu_preempt12:00:001
1658025993727,5cyclictest26-21ksoftirqd/110:55:021
1658025993721,6cyclictest26-21ksoftirqd/109:50:021
1658025993720,13cyclictest26-21ksoftirqd/110:16:381
1658025993719,14cyclictest26-21ksoftirqd/108:55:301
1658025993719,13cyclictest438-20systemd-journal11:25:211
1658025993719,13cyclictest438-20systemd-journal11:25:211
1658025993719,13cyclictest26-21ksoftirqd/111:35:111
1658025993719,13cyclictest26-21ksoftirqd/109:40:021
1658025993719,13cyclictest26-21ksoftirqd/108:00:021
1658025993719,13cyclictest26-21ksoftirqd/107:45:231
1658025993719,13cyclictest26-21ksoftirqd/107:45:231
1658025993718,14cyclictest26-21ksoftirqd/110:35:021
165802599371,33cyclictest0-21swapper/111:12:081
165802399378,12cyclictest0-21swapper/007:38:060
165802399376,16cyclictest13-21ksoftirqd/009:23:210
1658023993720,13cyclictest2500-40pool-tracker-mi10:53:590
1658023993719,14cyclictest1739062-21ntpq10:55:230
1658023993719,13cyclictest1682098-21kernelversion08:15:200
165802399371,20cyclictest1720138-21cron10:05:000
165802399371,15cyclictest16714242seq07:45:130
165802399371,15cyclictest16714242seq07:45:130
165802599368,13cyclictest141rcu_preempt12:15:211
165802599368,13cyclictest141rcu_preempt12:15:211
165802599365,6cyclictest141rcu_preempt07:20:261
1658025993619,13cyclictest26-21ksoftirqd/110:55:261
1658025993619,13cyclictest26-21ksoftirqd/108:45:141
1658025993619,12cyclictest1754265-21awk11:40:131
1658025993618,14cyclictest26-21ksoftirqd/112:05:211
1658025993618,14cyclictest26-21ksoftirqd/110:50:011
1658025993617,14cyclictest26-21ksoftirqd/108:00:021
165802399365,16cyclictest13-21ksoftirqd/008:23:330
1658023993623,10cyclictest1732347-21sshd10:37:210
165802399362,10cyclictest141rcu_preempt12:10:210
165802399361,16cyclictest1692350-21apt08:45:120
1658025993519,11cyclictest1659695-21apt-get07:15:021
1658025993518,12cyclictest26-21ksoftirqd/111:30:161
1658025993517,14cyclictest26-21ksoftirqd/112:35:111
1658025993517,14cyclictest26-21ksoftirqd/110:23:101
165802399352,16cyclictest0-21swapper/011:02:000
1658023993520,11cyclictest1696538-21ntpq08:55:240
1658023993519,12cyclictest1728961-21sshd10:28:210
1658023993519,12cyclictest1712618-21seq09:41:160
1658023993519,12cyclictest1678622-21awk08:05:180
1658023993514,11cyclictest1677162-21ntpq08:00:230
165802399351,13cyclictest1700349-21users09:05:290
1658023993510,5cyclictest141rcu_preempt12:05:270
1658025993416,6cyclictest26-21ksoftirqd/107:25:141
165802399349,8cyclictest141rcu_preempt07:30:240
165802399342,17cyclictest1658605-21df_inode07:10:140
165802399342,10cyclictest1662868-21df07:20:120
1658023993420,10cyclictest13-21ksoftirqd/009:15:250
1658023993317,12cyclictest1685301-21expr08:25:150
165802399323,11cyclictest141rcu_preempt11:50:220
165802399323,11cyclictest141rcu_preempt11:50:220
165802399322,9cyclictest1675048-21df07:55:140
1658023993217,11cyclictest1697506-21seq09:00:070
1658023993214,14cyclictest2345-21gdbus10:30:210
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional