You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-21 - 20:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Sun Dec 21, 2025 12:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1088353214254,83sleep30-21swapper/307:05:473
108866829261,10sleep10-21swapper/107:09:461
108381728973,9sleep20-21swapper/207:05:102
108866228560,20sleep00-21swapper/007:09:410
1088943993027,1cyclictest0-21swapper/308:59:593
108894399251,20cyclictest1413101-21turbostat12:15:003
1088943992421,1cyclictest0-21swapper/309:05:143
1088943992117,2cyclictest0-21swapper/310:55:003
108894399211,12cyclictest1179610-21taskset08:30:193
108894399202,14cyclictest1413363-21chrt12:15:123
1088943992017,1cyclictest0-21swapper/307:25:173
1088943992016,2cyclictest0-21swapper/311:30:013
108894399201,14cyclictest1397386-21chrt12:00:143
1088943991916,1cyclictest0-21swapper/312:25:013
1088943991916,1cyclictest0-21swapper/311:30:153
1088943991916,1cyclictest0-21swapper/311:20:173
1088943991916,1cyclictest0-21swapper/311:15:163
1088943991916,1cyclictest0-21swapper/309:20:143
108894399191,15cyclictest0-21swapper/308:15:213
108894399191,14cyclictest0-21swapper/307:20:123
1088943991910,4cyclictest13865732sleep311:50:123
108894399185,6cyclictest1291159-21taskset10:17:053
108894399184,7cyclictest1185569-21taskset08:36:053
1088943991815,1cyclictest0-21swapper/311:55:183
1088943991815,1cyclictest0-21swapper/310:10:173
1088943991815,1cyclictest0-21swapper/307:50:163
1088943991814,2cyclictest0-21swapper/311:40:013
1088943991814,2cyclictest0-21swapper/307:50:003
108894399181,14cyclictest0-21swapper/310:55:113
108894399181,14cyclictest0-21swapper/310:30:143
108894399181,13cyclictest1439320-21turbostat12:40:003
108894399181,13cyclictest1101485-21taskset07:19:493
108894399181,11cyclictest1376339-21taskset11:40:133
108894399180,13cyclictest0-21swapper/309:45:133
108894399176,7cyclictest1254307-21taskset09:43:413
108894399172,10cyclictest1168863-21taskset08:20:213
1088943991715,1cyclictest0-21swapper/311:15:003
1088943991714,2cyclictest0-21swapper/312:25:123
1088943991714,2cyclictest0-21swapper/308:40:163
1088943991714,1cyclictest0-21swapper/309:50:223
1088943991714,1cyclictest0-21swapper/308:25:133
1088943991713,2cyclictest0-21swapper/309:10:173
1088943991712,3cyclictest0-21swapper/309:15:123
108894399171,13cyclictest0-21swapper/307:40:123
108894399171,12cyclictest1344304-21turbostat11:10:003
108894399171,10cyclictest1317168-21taskset10:40:253
108894399170,13cyclictest0-21swapper/308:45:273
115465421610,3sleep01154760-21fschecks_time08:10:130
108894399161,9cyclictest1149106-21taskset08:05:073
108894399161,3cyclictest0-21swapper/312:05:123
1088943991613,1cyclictest0-21swapper/310:05:163
108894399161,12cyclictest0-21swapper/312:30:203
108894399161,12cyclictest0-21swapper/307:55:113
108894399161,10cyclictest1158629-21taskset08:10:233
108894399161,10cyclictest1093847-21taskset07:10:283
1088943991610,4cyclictest0-21swapper/309:35:113
108894399160,13cyclictest0-21swapper/311:45:253
108894399160,13cyclictest0-21swapper/310:20:273
108894399160,13cyclictest0-21swapper/310:20:263
108894399160,13cyclictest0-21swapper/307:35:153
108894399160,13cyclictest0-21swapper/307:30:153
12938632156,2sleep2171rcu_preempt10:20:172
12938632156,2sleep2171rcu_preempt10:20:172
12376422151,9sleep01237691-21ntpq09:25:210
12120742151,9sleep01212194-21cut09:05:000
11688062151,2sleep20-21swapper/208:20:212
10908172151,9sleep11092576-21cut07:10:191
108894399151,9cyclictest1338677-21taskset11:04:093
108894399151,9cyclictest1323079-21taskset10:49:353
108894399151,9cyclictest1238349-21taskset09:26:383
1088943991512,1cyclictest0-21swapper/310:00:283
108894399150,13cyclictest0-21swapper/308:00:143
108894399150,12cyclictest0-21swapper/310:35:183
108894399150,12cyclictest0-21swapper/310:25:143
108894399150,12cyclictest0-21swapper/309:30:203
108894399150,12cyclictest0-21swapper/308:50:183
13771712141,8sleep21378875-21cut11:40:172
13760602141,2sleep10-21swapper/111:40:131
13221332141,4sleep01322182-21ntpq10:45:210
12110682141,3sleep11211117-21sshd09:00:211
11919692146,4sleep11193035-21cat08:45:171
108894399141,2cyclictest0-21swapper/309:55:283
108894399140,12cyclictest0-21swapper/309:00:153
13538832130,6sleep11353934-21open_files11:15:231
13392852131,6sleep01339304-21apt-get11:05:120
12321142131,8sleep11232169-21gcc09:20:201
11068032131,6sleep01106830-21munin-run07:25:000
13609812121,7sleep11362565-21kthreadcore11:25:171
13451272121,6sleep21346416-21kthreadcore11:10:152
13171762121,7sleep01317420-21sed10:40:270
12798792121,5sleep20-21swapper/210:05:232
12024092121,7sleep21203359-21kthreadcore08:55:162
11867152120,3sleep1321rcuc/108:40:171
11491352121,7sleep11149395-21sh08:05:141
11224932120,7sleep01122448-31find07:38:030
11076312121,3sleep11108600-21kthreadcore07:25:171
14238802111,4sleep01423899-21apt-get12:25:130
14028562111,6sleep11403015-21cstates12:05:131
14025692111,4sleep21402311-21apt-get12:05:022
14016182110,7sleep01401797-21cat12:00:270
13603772111,5sleep01360396-21apt-get11:25:120
13511062111,6sleep01352843-21kthreadcore11:15:180
13063982111,5sleep11306446-21ntpq10:30:221
12601722111,6sleep11260318-21gltestperf09:50:141
12546962111,5sleep21254952-21df09:45:142
12441352111,5sleep10-21swapper/109:35:121
12179132111,5sleep11218122-21hddtemp_smartct09:10:141
11708972111,6sleep20-21swapper/208:25:142
11175582111,5sleep21117576-21apt-get07:35:122
11061132111,6sleep11106150-21switchtime07:20:241
14130782101,4sleep21413127-21munin-run12:15:012
13973642101,3sleep20-21swapper/212:00:142
13865962101,4sleep01386849-21df11:50:140
13748142101,4sleep01374868-21ntpq11:35:220
13601132101,5sleep0581-21gdbus11:25:010
13593562100,1sleep10-21swapper/111:20:241
13450552101,4sleep11345126-21irqstats11:10:141
13387442101,5sleep01338983-21sed11:05:000
13291282100,6sleep11330129-21kthreadcore10:55:161
13077902101,4sleep21307913-21diskstats10:35:122
13029942101,2sleep00-21swapper/010:30:170
13012112101,4sleep11301256-21ntpq10:25:211
12703552101,3sleep21270374-21apt-get10:00:132
12149472101,4sleep01216198-21memory09:05:200
12083922104,3sleep2421rcuc/209:00:182
12004582101,5sleep21200575-21ntpq08:50:202
11918522101,4sleep01191968-21irqstats08:45:160
11861872101,4sleep21186443-21df08:40:132
11708252101,4sleep11170896-21irqstats08:25:131
11321512101,4sleep21132201-21ntpq07:45:222
11229652101,5sleep20-21swapper/207:40:132
11229442101,3sleep11122962-21apt-get07:40:121
11162802101,4sleep01116331-21ntpq07:30:210
1438120291,4sleep11438259-21ntp_192.168.11412:35:201
1434327291,3sleep21434346-21apt-get12:35:132
1433171291,4sleep21433248-21pmu-power12:30:232
1402302290,5sleep11402566-21sed12:05:021
1395707291,3sleep21395788-21ntpq11:55:202
1389277291,4sleep11390411-21memory11:50:191
1386308292,2sleep1171rcu_preempt11:49:581
1371190291,4sleep21371341-21/usr/sbin/munin11:35:162
1327614291,5sleep21327703-21timerandwakeup10:50:272
1311552291,3sleep01311609-21ntpq10:35:190
1286961291,5sleep01287857-21cut10:15:180
1275541291,3sleep11275346-21apt-get10:05:021
1274411291,4sleep01270356-21/usr/sbin/munin10:00:230
1254674291,2sleep10-21swapper/109:45:141
1253399290,3sleep11253447-21ntpq09:40:201
1250651291,4sleep21252383-21grep09:40:172
1248275291,4sleep01248394-21gzip09:35:230
1244708291,4sleep21246021-21cut09:35:162
1239436291,4sleep01240148-21grep09:30:170
1232170291,3sleep21232222-21ntpq09:20:222
1212338291,3sleep21212357-21apt-get09:05:142
1206794291,4sleep11207002-21latency_hist09:00:001
1190006290,5sleep00-21swapper/008:40:200
1180905291,4sleep21181061-21cstates08:35:122
1179718291,4sleep11179765-21ls08:30:221
1170300291,3sleep01169801-21apt-get08:25:040
1168753291,4sleep11168804-21ntpq08:20:201
1163875291,4sleep01164128-21awk08:15:260
1152245291,4sleep21152999-21cat08:05:192
1137209291,3sleep21137322-21ntpq07:50:192
1114986291,4sleep21116138-21memory07:30:192
1102069291,4sleep21102199-21gltestperf07:20:142
1089278291,4sleep01089307-21grep07:10:130
1429139281,3sleep00-21swapper/012:30:130
1428714283,3sleep2431ktimers/212:28:142
1425215281,3sleep10-21swapper/112:25:181
1418225280,1sleep10-21swapper/112:18:311
1406804280,4sleep01406850-21ld12:05:200
1381338281,3sleep00-21swapper/011:45:120
1370395280,1sleep00-21swapper/011:33:300
1334345281,3sleep11334833-21grep11:00:161
1330131281,3sleep21331904-21cut10:55:172
1327267280,1sleep00-21swapper/010:50:210
1318304280,1sleep10-21swapper/110:45:141
1302720281,3sleep21302823-21fschecks_time10:30:152
1299912281,3sleep21301016-21memory10:25:182
1297179281,2sleep00-21swapper/010:25:130
1292524281,3sleep11293861-21kthreadcore10:20:161
1292524281,3sleep11293861-21kthreadcore10:20:161
1291946280,1sleep00-21swapper/010:20:080
1291946280,1sleep00-21swapper/010:20:070
1265632281,3sleep01266038-21kthreadcore09:55:190
1233629280,3sleep21233124-21apt-get09:25:022
1228118280,1sleep00-21swapper/009:20:110
1222963281,3sleep01223168-21diskstats09:15:140
1217622281,2sleep21217641-21apt-get09:10:122
1199704281,3sleep11200456-21missed_timers08:50:191
1196656281,3sleep01196915-21df_inode08:50:140
1160147281,3sleep20-21swapper/208:15:172
1154930281,3sleep21155719-21cut08:10:152
1148162281,4sleep21148166-21mailstats08:00:252
1138598281,2sleep11138617-21apt-get07:55:121
1133668281,3sleep11133751-21grep07:50:141
1133411281,3sleep01133667-21df07:50:130
1100805280,2sleep10-21swapper/107:15:231
1089547281,4sleep21089680-21tail07:10:162
1423436270,1sleep10-21swapper/112:22:401
1420302272,2sleep20-21swapper/212:20:172
1393204271,3sleep00-21swapper/011:55:180
1392114271,3sleep11392230-21fschecks_time11:55:151
1386564270,1sleep20-21swapper/211:50:082
1370352270,1sleep10-21swapper/111:32:591
1369550271,2sleep20-21swapper/211:30:232
1356326271,3sleep21358046-21kthreadcore11:20:172
1334834271,2sleep20-21swapper/211:00:172
1323980271,3sleep10-21swapper/110:50:181
1311611271,2sleep10-21swapper/110:35:201
1274463271,3sleep1553-21dbus-daemon10:00:241
1261459271,3sleep00-21swapper/009:50:170
1242973270,1sleep20-21swapper/209:30:232
1164526270,1sleep10-21swapper/108:19:361
1159162270,1sleep10-21swapper/108:13:021
1149674271,3sleep00-21swapper/008:05:160
1148131271,3sleep10-21swapper/108:00:241
1138620271,3sleep20-21swapper/207:55:132
1128155271,1sleep10-21swapper/107:45:141
1122499270,3sleep11122402-31sort07:38:071
1117068270,1sleep10-21swapper/107:32:101
1111130271,2sleep20-21swapper/207:25:222
1438926260,4sleep00-21swapper/012:35:310
1416476261,2sleep00-21swapper/012:15:190
1413341261,2sleep20-21swapper/212:15:122
1413054260,1sleep00-21swapper/012:14:420
1412528260,1sleep10-21swapper/112:10:261
1365228260,1sleep20-21swapper/211:28:462
1349813260,1sleep20-21swapper/211:15:132
1349467260,1sleep00-21swapper/011:13:570
1339307260,2sleep10-21swapper/111:05:141
1322911260,1sleep20-21swapper/210:47:272
1313451261,2sleep20-21swapper/210:40:172
1291232260,1sleep20-21swapper/210:18:012
1291118260,1sleep10-21swapper/110:16:351
1281997261,2sleep01283677-21kthreadcore10:10:170
1266041261,2sleep10-21swapper/109:55:201
1263738260,2sleep20-21swapper/209:50:192
1259562260,1sleep00-21swapper/009:49:180
1254172260,4sleep00-21swapper/009:41:590
1226948261,2sleep10-21swapper/109:15:231
1221930260,1sleep00-21swapper/009:10:250
1217113260,4sleep10-21swapper/109:06:571
1193036260,3sleep21194807-21kthreadcore08:45:182
1185531260,4sleep00-21swapper/008:35:340
1180883260,2sleep10-21swapper/108:35:111
1179468260,2sleep20-21swapper/208:30:192
1176754261,2sleep00-21swapper/008:30:180
1143878260,1sleep00-21swapper/008:00:110
1110988260,2sleep00-21swapper/007:25:200
1101499260,1sleep00-21swapper/007:19:590
1100018261,2sleep21100442-21grep07:15:162
1418623250,1sleep00-21swapper/012:20:110
1385432250,1sleep20-21swapper/211:45:222
1380883250,1sleep00-21swapper/011:42:520
1328559250,2sleep00-21swapper/010:55:110
1316762250,1sleep10-21swapper/110:40:201
1286130250,1sleep20-21swapper/210:15:002
1283681250,1sleep10-21swapper/110:10:181
1269855250,1sleep20-21swapper/209:57:212
1240150250,2sleep10-21swapper/109:30:181
1237695250,1sleep10-21swapper/109:25:211
1227599250,2sleep20-21swapper/209:17:042
1205139250,2sleep00-21swapper/008:55:180
1169779250,1sleep00-21swapper/008:24:520
1132576250,1sleep00-21swapper/007:45:290
1126795250,1sleep00-21swapper/007:40:190
1429313240,1sleep10-21swapper/112:30:141
1370770240,1sleep10-21swapper/111:35:071
1339270240,1sleep20-21swapper/211:05:032
1276214240,1sleep00-21swapper/010:05:160
1138894240,1sleep00-21swapper/007:55:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional