You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-06 - 07:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot8s.osadl.org (updated Sat Dec 06, 2025 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9155689913450,1342cyclictest9931392sleep33
9155689912910,1288cyclictest11901992sleep33
915568995530,550cyclictest921760-21chrt3
915568993910,388cyclictest1024609-21turbostat3
10059502300,2sleep244-21ksoftirqd/22
915568992421,1cyclictest0-21swapper/33
915568992118,1cyclictest0-21swapper/33
915568992017,1cyclictest0-21swapper/33
91556899197,9cyclictest1311877-21turbostat3
129541821810,6sleep10-21swapper/11
91556899176,7cyclictest9856482sleep33
91556899176,7cyclictest10439492sleep33
915568991613,1cyclictest0-21swapper/33
190898160,4rtkit-daemon1277641-21ssh1
91556899156,7cyclictest1335982-21turbostat3
91556899155,7cyclictest1335753-21taskset3
91556899151,7cyclictest0-21swapper/33
915568991512,1cyclictest0-21swapper/33
915568991512,1cyclictest0-21swapper/33
915568991512,1cyclictest0-21swapper/33
91556899151,11cyclictest1261248-21taskset3
91556899151,11cyclictest1123587-21taskset3
91556899151,11cyclictest1091115-21taskset3
91556899145,6cyclictest11160262sleep33
91556899141,9cyclictest955535-21taskset3
915568991413,0cyclictest0-21swapper/33
915568991412,1cyclictest0-21swapper/33
915568991412,1cyclictest0-21swapper/33
915568991412,1cyclictest0-21swapper/33
915568991412,1cyclictest0-21swapper/33
915568991412,1cyclictest0-21swapper/33
91556899141,11cyclictest961849-21taskset3
91556899141,11cyclictest961849-21taskset3
915568991411,1cyclictest0-21swapper/33
915568991411,1cyclictest0-21swapper/33
915568991411,1cyclictest0-21swapper/33
915568991411,1cyclictest0-21swapper/33
91556899141,10cyclictest943321-21taskset3
91556899141,10cyclictest1398738-21taskset3
91556899141,10cyclictest1350808-21taskset3
91556899141,10cyclictest1212232-21taskset3
91556899141,10cyclictest1168099-21taskset3
91556899141,10cyclictest1141205-21taskset3
91556899141,10cyclictest1098724-21taskset3
91556899140,6cyclictest0-21swapper/33
91556899135,6cyclictest9737292sleep33
91556899134,7cyclictest1270586-21taskset3
91556899133,7cyclictest1320192-21taskset3
91556899131,9cyclictest930415-21taskset3
91556899131,9cyclictest1287819-21chrt3
91556899131,9cyclictest1068532-21taskset3
915568991312,0cyclictest0-21swapper/33
915568991312,0cyclictest0-21swapper/33
915568991311,1cyclictest0-21swapper/33
915568991311,1cyclictest0-21swapper/33
915568991311,1cyclictest0-21swapper/33
915568991311,1cyclictest0-21swapper/33
915568991311,1cyclictest0-21swapper/33
915568991311,1cyclictest0-21swapper/33
91556899131,10cyclictest980176-21taskset3
91556899131,10cyclictest942411-21taskset3
91556899131,10cyclictest1245700-21taskset3
915568991311,0cyclictest0-21swapper/33
915568991310,1cyclictest0-21swapper/33
915568991310,1cyclictest0-21swapper/33
91556899130,10cyclictest1188074-21taskset3
91556899130,10cyclictest1157601-21taskset3
190898130,6rtkit-daemon921933-21munin-node0
190898130,6rtkit-daemon921933-21munin-node0
13701542130,11sleep21371863-21cut2
9174752120,4sleep1919484-21cut1
91556899121,9cyclictest1220947-21taskset3
91556899121,8cyclictest1149511-21taskset3
91556899120,9cyclictest1376702-21taskset3
91556899120,9cyclictest1141506-21taskset3
9094462121,6sleep30-21swapper/33
10251012122,6sleep01025134-21apt-get0
190898110,7rtkit-daemon1254831-21ssh2
190898110,7rtkit-daemon1157132-21ssh1
190898110,6rtkit-daemon1101092-21latency_hist0
12444822111,7sleep21244586-21cut2
12089602111,8sleep11209756-21kthreadcore1
11490272110,1sleep00-21swapper/00
190898100,5rtkit-daemon1310354-21ssh1
190898100,5rtkit-daemon1157717-21apt-get1
190898100,3rtkit-daemon438-21wpa_supplicant1
190898100,3rtkit-daemon1359430-21ssh0
190898100,3rtkit-daemon1124119-21ssh0
190898100,1rtkit-daemon0-21swapper/22
190898100,1rtkit-daemon0-21swapper/11
13683302101,6sleep10-21swapper/11
13683302101,6sleep10-21swapper/11
13579252100,7sleep11352516-21/usr/sbin/munin1
13344762100,1sleep00-21swapper/00
12718942100,7sleep01272036-21ps0
11241382100,1sleep20-21swapper/22
11176302101,7sleep11117435-21/usr/sbin/munin1
930418291,4sleep1930451-21apt-get1
929238291,5sleep2929297-21ntp_states2
19089890,5rtkit-daemon438-21wpa_supplicant0
19089890,5rtkit-daemon319-21systemd-udevd0
19089890,5rtkit-daemon1928-21sendmail-mta1
19089890,4rtkit-daemon1285668-21ssh1
19089890,4rtkit-daemon1179873-21ssh1
19089890,4rtkit-daemon1108650-21ssh0
19089890,4rtkit-daemon1086151-21irqcore0
19089890,4rtkit-daemon1076054-21ssh0
19089890,4rtkit-daemon1012478-21df_inode0
19089890,1rtkit-daemon0-21swapper/22
1326785290,6sleep21326793-21ssh2
1278022290,3sleep21278030-21ssh2
1263470293,3sleep01263505-21apt-get0
1141560292,3sleep21141592-21apt-get2
1063332291,5sleep11064279-21ps1
1049280291,6sleep21049475-21/usr/sbin/munin2
1043986291,5sleep00-21swapper/00
987057281,4sleep2987145-21tune2fs2
962665280,6sleep2963537-21cut2
962665280,6sleep2963537-21cut2
943120281,4sleep2943320-21hddtemp_smartct2
19089880,5rtkit-daemon667-21snmpd0
19089880,5rtkit-daemon433-21hwrng0
19089880,5rtkit-daemon1928-21sendmail-mta0
19089880,4rtkit-daemon998830-21users0
19089880,4rtkit-daemon921933-21munin-node0
19089880,4rtkit-daemon877434-21munin-node0
19089880,4rtkit-daemon674-21lldpd0
19089880,4rtkit-daemon671-21ntpd2
19089880,4rtkit-daemon671-21ntpd1
19089880,4rtkit-daemon667-21snmpd0
19089880,4rtkit-daemon438-21wpa_supplicant0
19089880,4rtkit-daemon1400024-21ssh2
19089880,4rtkit-daemon1374858-21ssh0
19089880,4rtkit-daemon1348499-21kthreadcore1
19089880,4rtkit-daemon1334938-21runrttasks2
19089880,4rtkit-daemon1198592-21ssh1
19089880,4rtkit-daemon1141046-21ssh0
19089880,4rtkit-daemon1019971-21sed0
19089880,3rtkit-daemon924300-21fschecks_count0
19089880,3rtkit-daemon921933-21munin-node2
19089880,3rtkit-daemon1367416-21ssh0
19089880,3rtkit-daemon1367416-21ssh0
19089880,1rtkit-daemon0-21swapper/22
19089880,1rtkit-daemon0-21swapper/11
19089880,1rtkit-daemon0-21swapper/00
19089880,1rtkit-daemon0-21swapper/00
19089880,1rtkit-daemon0-21swapper/00
19089880,1rtkit-daemon0-21swapper/00
1377165281,4sleep21377921-21cut2
1358822280,2sleep21358830-21ssh2
1336517280,6sleep11336722-21hddtemp_smartct1
1327917280,5sleep11327922-21cron1
1311731281,4sleep01311740-21sshd0
1286948280,5sleep01286956-21ssh0
1239186281,3sleep01239225-21apt-get0
1212593280,1sleep00-21swapper/00
1200358280,5sleep21202348-21kthreadcore2
1198607281,3sleep01198644-21apt-get0
1190692281,4sleep01191286-21/usr/sbin/munin0
1173994281,4sleep01174198-21hddtemp_smartct0
1130749280,5sleep01130816-21ntp_states0
1114689281,4sleep01114812-21python30
1083219280,6sleep11083226-21sh1
1025137281,4sleep11025283-21/usr/sbin/munin1
1000481281,5sleep11000941-21kthreadcore1
986619270,4sleep2986662-21munin-run2
19089870,4rtkit-daemon671-21ntpd0
19089870,4rtkit-daemon667-21snmpd1
19089870,4rtkit-daemon667-21snmpd0
19089870,3rtkit-daemon942295-21users0
19089870,3rtkit-daemon921933-21munin-node2
19089870,3rtkit-daemon921933-21munin-node2
19089870,3rtkit-daemon921933-21munin-node1
19089870,3rtkit-daemon921916-21kworker/0:3+kacpid0
19089870,3rtkit-daemon1188429-21ssh0
19089870,3rtkit-daemon1068818-21diskmemload1
19089870,3rtkit-daemon1068116-21smartctl0
19089870,3rtkit-daemon1055751-21runrttasks0
1368668270,3sleep11368704-21apt-get1
1365812270,5sleep21360581-21/usr/sbin/munin2
1365812270,5sleep21360581-21/usr/sbin/munin2
1328252270,4sleep11328407-21df_inode1
1311060270,1sleep20-21swapper/22
1287889270,4sleep01287897-21ssh0
1281007271,3sleep21282122-21cut2
1255691270,4sleep01255888-21/usr/sbin/munin0
1246905270,1sleep11246913-21ssh1
1236774270,4sleep11236959-21users1
1235852270,4sleep01236327-21/usr/sbin/munin0
1223240270,5sleep21223272-21/usr/sbin/munin2
1190216270,4sleep21190224-21ssh2
1182211271,4sleep21182370-21/usr/sbin/munin2
1182178271,2sleep10-21swapper/11
1156467270,1sleep00-21swapper/00
1141885270,4sleep11142059-21ssh1
1085255271,3sleep11085289-21apt-get1
1074931270,1sleep10-21swapper/11
1031437270,4sleep11031490-21wc1
1018308270,4sleep21018400-21grep2
993972260,4sleep1994826-21kthreadcore1
992042261,3sleep1992088-21ps1
969145260,4sleep1969520-21kthreadcore1
961751261,2sleep1961784-21apt-get1
961751261,2sleep1961784-21apt-get1
955499261,2sleep20-21swapper/22
944602260,4sleep1946613-21kthreadcore1
938608261,2sleep10-21swapper/11
921068260,3sleep2915716-21/usr/sbin/munin2
911698261,2sleep10-21swapper/11
19089860,3rtkit-daemon473-21gmain0
1408723260,4sleep11408731-21ssh1
1408330260,1sleep00-21swapper/00
1336723260,3sleep21337121-21irqcore2
1321709260,4sleep00-21swapper/00
1255053260,1sleep10-21swapper/11
1249887261,3sleep01251924-21kthreadcore0
1229905260,3sleep01229907-21ssh0
1222288260,1sleep0171rcu_preempt0
1216548261,3sleep21218160-21ps2
1213012260,4sleep21068818-21diskmemload2
1131328260,1sleep10-21swapper/11
1108409260,4sleep11108415-21ssh1
1101686262,2sleep20-21swapper/22
1093304260,3sleep21093339-21apt-get2
1086747261,3sleep21088357-21cat2
1069138261,2sleep21069177-21apt-get2
1049691260,1sleep10-21swapper/11
1027656261,2sleep20-21swapper/22
1010945260,4sleep11011074-21sensors1
1001602261,2sleep20-21swapper/22
998360250,3sleep2998461-21basename2
955222250,2sleep2955283-21systemd-run2
19089850,1rtkit-daemon0-21swapper/00
1390281251,2sleep10-21swapper/11
1350675251,2sleep20-21swapper/22
1343257250,3sleep00-21swapper/00
1312150251,2sleep01068818-21diskmemload0
1295990250,2sleep00-21swapper/00
1270606250,3sleep20-21swapper/22
1261660250,1sleep20-21swapper/22
1172447250,3sleep00-21swapper/00
1164987250,3sleep01164994-21sh0
1140065250,3sleep11140073-21ssh1
1115667251,2sleep10-21swapper/11
1100032250,2sleep11100040-21ssh1
1083501250,3sleep20-21swapper/22
1062322250,1sleep10-21swapper/11
1056508250,1sleep20-21swapper/22
1056127250,1sleep10-21swapper/11
1043631250,1sleep10-21swapper/11
1024261251,2sleep10-21swapper/11
1017772251,1sleep10-21swapper/11
983028240,2sleep10-21swapper/11
974322240,1sleep20-21swapper/22
973917240,3sleep20-21swapper/22
961380240,3sleep10-21swapper/11
954466240,1sleep10-21swapper/11
936058240,3sleep20-21swapper/22
1394408240,2sleep10-21swapper/11
1384815240,1sleep20-21swapper/22
1376953240,2sleep10-21swapper/11
1352058240,2sleep00-21swapper/00
1297474240,1sleep20-21swapper/22
1294102240,2sleep21294110-21ssh2
1159153240,2sleep20-21swapper/22
1150379240,2sleep20-21swapper/22
1109372240,2sleep20-21swapper/22
1064285240,1sleep20-21swapper/22
1056170240,3sleep20-21swapper/22
1040375240,2sleep20-21swapper/22
978084230,1sleep10-21swapper/11
936868230,1sleep20-21swapper/22
929097230,1sleep10-21swapper/11
909440230,1sleep20-21swapper/22
1032902230,1sleep20-21swapper/22
1018760230,1sleep20-21swapper/22
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional