You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-06 - 22:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Fri Mar 06, 2026 12:44:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
187049199504497,4cyclictest1890216-21turbostat08:09:553
187049199502494,5cyclictest1916531-21turbostat09:20:003
187049199473454,16cyclictest1930677-21turbostat10:00:003
187049199465456,6cyclictest1983092-21turbostat12:34:553
187049199462454,5cyclictest1957076-21turbostat11:14:593
187049199462454,5cyclictest1957076-21turbostat11:14:593
187049199462450,9cyclictest1981363-21turbostat12:24:593
187049199462450,9cyclictest1981363-21turbostat12:24:593
187049199460450,6cyclictest1886761-21turbostat07:55:003
187049199458449,6cyclictest1977893-21turbostat12:15:003
187049199458436,19cyclictest1891954-21turbostat08:14:553
187049199457446,8cyclictest1958812-21turbostat11:19:593
187048499457449,5cyclictest1900651-21turbostat08:39:552
187049199456434,19cyclictest1950116-21turbostat10:54:593
187048399456424,18cyclictest1906140-21turbostat08:54:551
187049199455448,4cyclictest1904412-21turbostat08:45:003
187049199451444,4cyclictest1965731-21turbostat11:39:593
187049199451444,4cyclictest1888474-21turbostat08:04:543
187049199449442,4cyclictest1951846-21turbostat11:04:553
187049199449441,4cyclictest1921724-21turbostat09:35:003
187049199448441,4cyclictest1935950-21turbostat10:19:553
187049199447439,5cyclictest1946656-21turbostat10:45:003
187048499447424,20cyclictest1885026-21turbostat07:54:562
187049199445437,5cyclictest1921724-21turbostat09:39:553
187047799445424,8cyclictest1907869-21turbostat08:59:550
187049199442434,5cyclictest1881569-21turbostat07:40:003
187048499442435,4cyclictest1893668-21turbostat08:19:562
187049199440433,4cyclictest1941160-21turbostat10:34:553
187049199440433,4cyclictest1918259-21turbostat09:25:013
187049199440432,5cyclictest1977893-21turbostat12:19:563
187049199440432,5cyclictest1977893-21turbostat12:19:563
187049199438427,8cyclictest1909592-21turbostat09:04:553
187048399433425,5cyclictest1895410-21turbostat08:24:561
187048399433425,5cyclictest1895410-21turbostat08:24:561
187049199432424,5cyclictest1943193-21turbostat10:39:553
187049199430421,6cyclictest1970939-21turbostat11:55:013
187049199430421,6cyclictest1898875-21turbostat08:34:553
187049199424417,4cyclictest1902385-21turbostat08:40:013
187049199422415,4cyclictest1913067-21turbostat09:14:553
187049199422415,4cyclictest1913067-21turbostat09:14:553
187049199421414,4cyclictest1981363-21turbostat12:29:543
187047799419412,4cyclictest1919994-21turbostat09:34:550
187049199417396,5cyclictest1951846-21turbostat11:00:003
187049199416409,4cyclictest1967470-21turbostat11:44:593
187049199416409,4cyclictest1874588-21turbostat07:24:543
187049199416396,17cyclictest1928652-21turbostat09:55:013
187049199411404,4cyclictest1941160-21turbostat10:30:003
187049199411404,4cyclictest1926917-21turbostat09:50:013
187049199408401,4cyclictest1970939-21turbostat11:59:563
187049199408401,4cyclictest1970939-21turbostat11:59:553
187049199408401,4cyclictest1909592-21turbostat09:00:003
187048499408401,4cyclictest1879786-21turbostat07:39:542
187049199406399,4cyclictest1962277-21turbostat11:34:553
187047799401393,5cyclictest1886761-21turbostat07:59:560
187047799401393,5cyclictest1886761-21turbostat07:59:550
187049199399392,4cyclictest1895410-21turbostat08:20:003
187049199399390,6cyclictest1976163-21turbostat12:10:003
187049199395386,6cyclictest1906140-21turbostat08:49:593
187049199391384,4cyclictest1918259-21turbostat09:29:563
187049199390382,5cyclictest1923459-21turbostat09:44:543
187049199389382,4cyclictest1935950-21turbostat10:15:013
187049199385376,6cyclictest1878051-21turbostat07:34:563
187049199384377,4cyclictest1883296-21turbostat07:49:553
187048499384377,4cyclictest1937697-21turbostat10:24:552
187048499384377,4cyclictest1872357-21turbostat07:19:552
187048499382354,25cyclictest1902385-21turbostat08:44:562
187049199377370,4cyclictest1872357-21turbostat07:15:003
187049199376370,3cyclictest1932450-21turbostat10:09:553
187048499373365,5cyclictest1928652-21turbostat09:59:562
187048399371346,22cyclictest1976163-21turbostat12:14:561
187049199367361,3cyclictest1883296-21turbostat07:45:003
187049199365358,4cyclictest1960550-21turbostat11:29:563
187049199364357,4cyclictest1960550-21turbostat11:25:013
187049199362357,3cyclictest1969198-21turbostat11:50:003
187048499362355,4cyclictest1953578-21turbostat11:09:562
187049199360354,3cyclictest1911332-21turbostat09:09:553
187049199356350,3cyclictest1878051-21turbostat07:30:003
187048499356349,4cyclictest1914798-21turbostat09:19:542
187048499355347,5cyclictest1870269-21turbostat07:14:542
187048399355348,4cyclictest1930677-21turbostat10:04:561
187048399355348,4cyclictest1930677-21turbostat10:04:561
187049199350343,4cyclictest1984826-21turbostat12:39:563
187048499348341,4cyclictest1904412-21turbostat08:49:562
187049199346340,3cyclictest1948386-21turbostat10:50:003
187049199344339,3cyclictest1939431-21turbostat10:25:003
187047799342336,3cyclictest1969198-21turbostat11:54:560
187048499341334,4cyclictest1876319-21turbostat07:29:552
187049199333326,4cyclictest1874588-21turbostat07:20:003
187049199333311,19cyclictest1955299-21turbostat11:10:013
187047799331323,5cyclictest1946656-21turbostat10:49:550
187048499326320,3cyclictest1881569-21turbostat07:44:552
187048499322315,4cyclictest1958812-21turbostat11:24:542
187049199321313,5cyclictest1972668-21turbostat12:04:553
187048499319313,3cyclictest1957076-21turbostat11:19:542
187048499319312,4cyclictest1950116-21turbostat10:59:542
187049199314308,3cyclictest1897146-21turbostat08:29:553
187048499314307,4cyclictest1926917-21turbostat09:54:562
187049199235221,10cyclictest1932450-21turbostat10:05:003
187049199235221,10cyclictest1932450-21turbostat10:04:593
1973991350,92rtkit-daemon0-21swapper/107:09:581
1870149211076,24sleep20-21swapper/207:08:412
190616821090,4chrt1906167-21unin-run08:50:000
1870248210874,23sleep00-21swapper/007:09:560
1870160210775,22sleep30-21swapper/307:08:503
18704919910093,4cyclictest1888474-21turbostat07:59:593
18704919910093,4cyclictest1888474-21turbostat07:59:593
197399990,6rtkit-daemon1972-21rtkit-daemon09:56:251
1870484999891,4cyclictest1967470-21turbostat11:49:542
1870491999588,4cyclictest1897146-21turbostat08:24:593
1870491999588,4cyclictest1897146-21turbostat08:24:593
1870491999386,4cyclictest1907869-21turbostat08:54:593
19528172850,5sleep00-21swapper/011:00:240
197399790,5rtkit-daemon4497-21PK-Backend08:02:091
197399770,24rtkit-daemon4497-21PK-Backend07:28:051
197399760,5rtkit-daemon4497-21PK-Backend09:46:081
19186902740,7sleep01918694-21cpuspeed_turbos09:25:130
197399730,5rtkit-daemon0-21swapper/107:40:551
197399610,9rtkit-daemon1972-21rtkit-daemon10:07:240
197399600,8rtkit-daemon1972-21rtkit-daemon11:32:200
197399570,9rtkit-daemon1972-21rtkit-daemon09:50:160
187047799570,53cyclictest1972-21rtkit-daemon11:55:400
187047799570,53cyclictest1972-21rtkit-daemon11:55:390
197399560,9rtkit-daemon1972-21rtkit-daemon09:37:520
197399560,8rtkit-daemon1972-21rtkit-daemon10:20:370
197399550,7rtkit-daemon1972-21rtkit-daemon10:30:010
187047799540,50cyclictest1972-21rtkit-daemon09:56:240
1870483995217,19cyclictest1909629-21unin-run09:00:001
187047799520,47cyclictest1972-21rtkit-daemon09:45:030
197399510,8rtkit-daemon1972-21rtkit-daemon08:39:371
197399510,5rtkit-daemon0-21swapper/209:28:312
1870477995023,8cyclictest13-21ksoftirqd/011:12:560
1870477995023,8cyclictest13-21ksoftirqd/011:12:560
187047799500,47cyclictest1972-21rtkit-daemon10:19:210
19013722480,9sleep00-21swapper/008:35:190
1870477994810,6cyclictest141rcu_preempt10:04:420
1870477994810,6cyclictest141rcu_preempt10:04:410
187047799461,23cyclictest1985180-21latency_hist12:35:030
1870477994425,6cyclictest13-21ksoftirqd/012:35:220
187047799442,20cyclictest438-20systemd-journal10:15:000
187048499435,5cyclictest141rcu_preempt10:28:372
1870484994330,9cyclictest34-21ksoftirqd/208:33:522
1870477994325,14cyclictest13-21ksoftirqd/008:25:210
1870477994324,14cyclictest13-21ksoftirqd/010:30:030
1870477994324,14cyclictest13-21ksoftirqd/007:55:000
1870477994319,21cyclictest13-21ksoftirqd/009:05:220
19507702420,8chrt4497-21PK-Backend10:55:171
187048499428,17cyclictest438-20systemd-journal12:19:552
187048499428,17cyclictest438-20systemd-journal12:19:552
187048499426,13cyclictest34-21ksoftirqd/212:35:272
1870484994229,8cyclictest34-21ksoftirqd/210:04:392
1870484994229,8cyclictest34-21ksoftirqd/210:04:392
1870484994227,5cyclictest34-21ksoftirqd/211:50:262
1870484994213,14cyclictest141rcu_preempt08:26:202
1870477994224,14cyclictest13-21ksoftirqd/011:39:590
1870477994224,13cyclictest13-21ksoftirqd/008:40:090
1870477994223,5cyclictest13-21ksoftirqd/012:00:540
1870477994210,7cyclictest141rcu_preempt11:30:010
197399410,32rtkit-daemon1870470-21cyclictest11:47:390
197399410,27rtkit-daemon4497-21PK-Backend07:49:441
18973802412,12sleep126-21ksoftirqd/108:25:111
1870484994118,9cyclictest34-21ksoftirqd/212:05:012
1870484994110,15cyclictest141rcu_preempt08:05:172
187048399411,34cyclictest438-20systemd-journal11:55:401
187048399411,34cyclictest438-20systemd-journal11:55:401
1870477994124,5cyclictest13-21ksoftirqd/010:55:000
1870477994124,13cyclictest13-21ksoftirqd/010:44:230
1870477994124,13cyclictest13-21ksoftirqd/009:01:070
1870477994123,6cyclictest13-21ksoftirqd/012:10:260
1870477994123,6cyclictest13-21ksoftirqd/007:10:120
1870477994123,14cyclictest13-21ksoftirqd/012:15:120
1870477994123,14cyclictest13-21ksoftirqd/012:15:120
1870477994123,14cyclictest13-21ksoftirqd/008:50:380
1870477994123,13cyclictest13-21ksoftirqd/007:15:060
1870477994122,14cyclictest13-21ksoftirqd/007:25:240
187047799411,20cyclictest1876310-21perf07:25:000
197399400,26rtkit-daemon0-21swapper/108:30:231
187048499407,10cyclictest1943444-21/usr/sbin/munin10:35:132
187048499406,13cyclictest141rcu_preempt11:04:012
187048499404,11cyclictest141rcu_preempt12:05:292
1870484994024,13cyclictest34-21ksoftirqd/207:55:202
1870484994024,13cyclictest34-21ksoftirqd/207:55:192
1870484994022,14cyclictest34-21ksoftirqd/209:32:272
1870484994022,13cyclictest956-21systemd-logind10:05:572
1870483994025,11cyclictest923-21dbus-daemon10:30:041
1870483994023,13cyclictest3034-21update-notifier07:11:311
1870477994029,8cyclictest13-21ksoftirqd/008:33:500
1870477994023,5cyclictest13-21ksoftirqd/007:45:280
1870477994023,13cyclictest13-21ksoftirqd/012:25:240
1870477994023,13cyclictest13-21ksoftirqd/010:35:330
1870477994022,5cyclictest13-21ksoftirqd/011:40:250
1870477994022,14cyclictest13-21ksoftirqd/012:25:010
1870477994022,14cyclictest13-21ksoftirqd/012:25:010
1870477994022,14cyclictest13-21ksoftirqd/009:16:400
1870477994022,14cyclictest13-21ksoftirqd/008:15:180
1870477994022,13cyclictest13-21ksoftirqd/007:40:180
1870477994020,5cyclictest13-21ksoftirqd/008:05:300
1870477994014,22cyclictest1881551-21cron07:40:000
19503922390,4sleep00-21swapper/010:55:130
187048499397,15cyclictest34-21ksoftirqd/209:05:142
187048499394,5cyclictest141rcu_preempt08:56:132
1870484993928,7cyclictest1935977-21systemd10:15:022
1870484993926,5cyclictest34-21ksoftirqd/211:30:092
1870484993924,11cyclictest34-21ksoftirqd/208:25:012
1870484993924,11cyclictest34-21ksoftirqd/208:25:002
1870484993922,6cyclictest34-21ksoftirqd/209:00:302
1870484993922,13cyclictest34-21ksoftirqd/212:20:132
1870484993922,13cyclictest34-21ksoftirqd/212:20:132
1870484993922,13cyclictest34-21ksoftirqd/208:50:282
1870484993918,13cyclictest34-21ksoftirqd/207:22:362
187048499391,15cyclictest19271032date09:50:012
1870483993923,11cyclictest438-20systemd-journal09:15:401
187048399392,11cyclictest1137-21runrttasks12:20:011
187048399392,11cyclictest1137-21runrttasks12:20:011
1870477993923,5cyclictest13-21ksoftirqd/012:05:280
1870477993923,5cyclictest13-21ksoftirqd/007:30:150
1870477993922,6cyclictest13-21ksoftirqd/009:40:150
1870477993922,5cyclictest13-21ksoftirqd/008:10:250
1870477993922,14cyclictest13-21ksoftirqd/011:15:310
1870477993922,13cyclictest13-21ksoftirqd/011:20:290
1870477993921,6cyclictest13-21ksoftirqd/009:12:580
1870477993921,6cyclictest13-21ksoftirqd/009:12:580
1870477993921,13cyclictest13-21ksoftirqd/009:20:140
187047799391,14cyclictest438-20systemd-journal08:25:000
187047799391,14cyclictest438-20systemd-journal08:24:590
187048499389,6cyclictest141rcu_preempt09:40:192
187048499386,10cyclictest34-21ksoftirqd/212:25:122
1870484993821,13cyclictest34-21ksoftirqd/211:55:272
1870484993821,13cyclictest34-21ksoftirqd/211:55:262
1870484993821,13cyclictest34-21ksoftirqd/210:30:062
1870484993821,13cyclictest34-21ksoftirqd/208:02:272
1870484993821,13cyclictest34-21ksoftirqd/207:30:482
1870483993823,11cyclictest1906122-21cron08:50:001
1870477993820,5cyclictest13-21ksoftirqd/011:05:280
197399370,26rtkit-daemon4497-21PK-Backend08:13:011
197399370,26rtkit-daemon4497-21PK-Backend07:51:541
187048499376,10cyclictest34-21ksoftirqd/212:31:452
1870484993722,11cyclictest1961266-21seq11:25:202
1870484993721,13cyclictest34-21ksoftirqd/210:51:462
1870484993720,6cyclictest34-21ksoftirqd/207:45:132
1870484993720,13cyclictest34-21ksoftirqd/210:45:262
1870484993719,14cyclictest34-21ksoftirqd/209:20:102
187048399372,31cyclictest1917167-21df_inode09:20:141
1870483993710,12cyclictest141rcu_preempt08:00:001
1870483993710,12cyclictest141rcu_preempt08:00:001
1870477993721,11cyclictest13-21ksoftirqd/008:00:020
1870484993625,7cyclictest1945860-21needreboot10:40:212
187048499362,17cyclictest1977551-21taskset12:10:482
1870484993620,13cyclictest34-21ksoftirqd/209:35:052
1870484993620,12cyclictest34-21ksoftirqd/211:39:592
1870484993620,12cyclictest1967185-21sshd11:41:402
1870484993619,13cyclictest1-21systemd09:10:402
1870484993619,13cyclictest1-21systemd09:10:392
1870483993622,10cyclictest1895155-21cron08:17:011
1870483993620,12cyclictest1980623-21ntpq12:20:241
1870483993620,12cyclictest1980623-21ntpq12:20:231
1870483993620,11cyclictest438-20systemd-journal08:40:401
1870483993619,12cyclictest438-20systemd-journal07:39:591
187048399361,10cyclictest0-21swapper/112:30:341
1870484993522,9cyclictest34-21ksoftirqd/210:15:232
1870484993518,13cyclictest34-21ksoftirqd/208:14:362
1870484993513,19cyclictest34-21ksoftirqd/211:12:592
1870484993513,19cyclictest34-21ksoftirqd/211:12:582
1870483993527,4cyclictest4497-21PK-Backend12:05:041
1870483993520,11cyclictest1967465-21grep11:45:001
187048399351,9cyclictest4497-21PK-Backend11:10:101
187048399351,9cyclictest4497-21PK-Backend11:10:101
1870483993518,13cyclictest1033-21gdbus10:05:561
187048399351,14cyclictest4497-21PK-Backend09:37:581
187048399341,9cyclictest4497-21PK-Backend11:47:451
1870483993419,11cyclictest1912605-21cat09:05:301
187048399341,8cyclictest0-21swapper/109:26:061
1870483993418,12cyclictest1-21systemd11:20:401
187048399341,5cyclictest4497-21PK-Backend12:26:201
187048399341,30cyclictest4497-21PK-Backend09:02:441
1870483993411,19cyclictest1986387-40lsb_release12:35:291
1870483993326,4cyclictest4497-21PK-Backend12:01:561
1870483993326,4cyclictest4497-21PK-Backend07:21:071
1870483993322,7cyclictest1940177-21kernelversion10:25:221
187048399331,8cyclictest4497-21PK-Backend11:28:531
187048399331,8cyclictest4497-21PK-Backend11:17:471
187048399331,8cyclictest4497-21PK-Backend09:12:431
187048399331,8cyclictest4497-21PK-Backend09:12:431
187048399331,8cyclictest4497-21PK-Backend08:09:391
1870483993314,16cyclictest4497-21PK-Backend11:10:001
187048399331,29cyclictest4497-21PK-Backend11:31:351
187048399331,29cyclictest4497-21PK-Backend10:20:181
187048399331,10cyclictest4497-21PK-Backend07:34:471
187048399321,9cyclictest4497-21PK-Backend10:40:471
1870483993216,12cyclictest1947289-21df10:45:151
1870483993215,12cyclictest1952114-21cpu11:00:131
187048399321,4cyclictest4497-21PK-Backend11:52:321
187048399321,29cyclictest0-21swapper/111:38:101
187048399321,28cyclictest4497-21PK-Backend09:42:461
187048399321,28cyclictest4497-21PK-Backend09:33:411
187048399321,28cyclictest4497-21PK-Backend07:15:281
187048399321,13cyclictest4497-21PK-Backend10:17:381
187048399321,13cyclictest4497-21PK-Backend09:51:171
187048399311,8cyclictest4497-21PK-Backend10:36:291
197399290,23rtkit-daemon4497-21PK-Backend10:14:361
187048399281,14cyclictest4497-21PK-Backend10:51:511
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional