You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-03 - 16:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Sat Jan 03, 2026 12:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1286562213654,77sleep30-21swapper/307:08:293
128651629183,5sleep20-21swapper/207:07:532
128648928872,10sleep10-21swapper/107:07:321
128594027656,13sleep00-21swapper/007:05:250
1286796992421,1cyclictest0-21swapper/312:35:003
1286796992421,1cyclictest0-21swapper/311:00:003
128679699203,14cyclictest1495610-21taskset10:24:553
128679699201,9cyclictest1574190-21turbostat11:39:593
128679699201,7cyclictest1431900-21turbostat09:24:593
1286796992017,1cyclictest0-21swapper/309:10:303
1286796992017,1cyclictest0-21swapper/307:37:313
1286796992016,2cyclictest0-21swapper/312:20:133
1286796992016,2cyclictest0-21swapper/307:15:013
128679699201,15cyclictest1578434-21taskset11:40:183
128679699191,14cyclictest1384021-21taskset08:37:303
128679699191,14cyclictest0-21swapper/309:15:203
1286796991815,1cyclictest0-21swapper/308:50:193
1286796991814,2cyclictest0-21swapper/309:05:143
128679699181,13cyclictest1563990-21chrt11:30:123
128679699181,13cyclictest0-21swapper/311:15:113
128679699181,12cyclictest1326521-21taskset07:44:553
15962582171,11sleep01597441-21kthreadcore12:00:160
13586662171,5sleep01358782-21irqstats08:15:150
1286796991715,1cyclictest0-21swapper/311:50:173
1286796991714,1cyclictest0-21swapper/310:40:113
1286796991714,1cyclictest0-21swapper/310:40:013
1286796991714,1cyclictest0-21swapper/309:00:123
1286796991714,1cyclictest0-21swapper/307:50:143
1286796991713,2cyclictest0-21swapper/310:10:113
128679699171,13cyclictest0-21swapper/311:20:153
128679699170,13cyclictest0-21swapper/311:00:203
128679699170,13cyclictest0-21swapper/309:39:033
128679699170,10cyclictest1361685-21taskset08:15:173
15959762161,11sleep21596180-21grep12:00:152
13384252161,7sleep21340134-21kthreadcore07:55:172
13255752161,9sleep01325631-21ntpq07:40:210
12945172161,10sleep10-21swapper/107:15:121
128679699169,4cyclictest15012752sleep310:30:123
128679699161,9cyclictest1400262-21taskset08:55:053
128679699161,9cyclictest1373988-21taskset08:30:013
128679699161,9cyclictest1368724-21taskset08:25:073
128679699161,9cyclictest1310628-21taskset07:30:003
1286796991613,1cyclictest0-21swapper/311:25:123
1286796991613,1cyclictest0-21swapper/310:50:173
1286796991613,1cyclictest0-21swapper/310:50:173
1286796991613,1cyclictest0-21swapper/309:55:173
1286796991613,1cyclictest0-21swapper/309:30:123
1286796991613,1cyclictest0-21swapper/307:25:293
1286796991613,1cyclictest0-21swapper/307:20:203
1286796991612,2cyclictest0-21swapper/312:15:213
1286796991612,2cyclictest0-21swapper/311:45:113
1286796991612,2cyclictest0-21swapper/311:10:113
1286796991612,2cyclictest0-21swapper/310:25:203
128679699161,12cyclictest1611163-21turbostat12:15:013
128679699161,12cyclictest0-21swapper/308:40:123
128679699161,11cyclictest0-21swapper/312:25:123
128679699161,10cyclictest1636028-21taskset12:35:183
128679699161,10cyclictest1479936-21taskset10:10:003
128679699160,13cyclictest0-21swapper/312:05:213
128679699160,13cyclictest0-21swapper/311:55:193
128679699160,13cyclictest0-21swapper/310:03:103
128679699160,13cyclictest0-21swapper/308:45:143
128679699160,10cyclictest1600372-21taskset12:01:033
128679699160,10cyclictest1347318-21taskset08:02:553
16101002151,3sleep11610151-21ntpq12:10:201
15169002151,9sleep11516919-21apt-get10:45:101
14485132150,10sleep11447995-21apt-get09:40:011
128679699151,9cyclictest1368312-21taskset08:23:043
128679699151,9cyclictest1352377-21taskset08:05:393
128679699151,9cyclictest1298614-21taskset07:15:213
128679699151,12cyclictest0-21swapper/307:45:133
128679699151,11cyclictest1432431-21chrt09:25:123
128679699151,10cyclictest1485367-21chrt10:15:103
128679699150,13cyclictest0-21swapper/310:45:113
128679699150,12cyclictest0-21swapper/309:50:143
128679699150,12cyclictest0-21swapper/309:45:123
128679699150,12cyclictest0-21swapper/309:45:123
128679699150,12cyclictest0-21swapper/309:40:163
128679699150,11cyclictest0-21swapper/307:55:123
15891422141,9sleep11589188-21ntpq11:50:211
14787322141,9sleep21478848-21collect210:05:202
13897552140,2sleep20-21swapper/208:45:112
1286796991411,1cyclictest0-21swapper/308:10:133
128679699140,11cyclictest0-21swapper/311:05:203
15688752131,8sleep21568878-21sshd11:34:292
14751552130,9sleep11475241-21grep10:05:151
14596362131,7sleep01460909-21kthreadcore09:50:180
13845242131,8sleep01384603-21grep08:40:120
13779912131,4sleep11378040-21ntpq08:30:191
13684372130,1sleep20-21swapper/208:24:402
13567982133,5sleep11356939-21ntpq08:10:191
13477622135,4sleep21347820-21wc08:05:112
15802762121,8sleep01580377-21fschecks_time11:45:130
15677892121,7sleep11567902-21ntpq11:30:191
14872142121,7sleep01489021-21kthreadcore10:15:150
14171542121,7sleep21418084-21kthreadcore09:10:202
13972772120,2sleep10-21swapper/108:50:181
13630312120,1sleep10-21swapper/108:18:011
13598812121,7sleep21361682-21kthreadcore08:15:172
13267862121,5sleep21326805-21apt-get07:45:112
13146472120,8sleep21314701-21ntpq07:30:202
13000822121,6sleep11299563-21apt-get07:20:011
16101532111,5sleep21610199-21ntpq12:10:222
15943542110,7sleep21594402-21ntpq11:55:172
15626812111,4sleep11562722-21ntpq11:25:181
15562452111,5sleep01557358-21memory11:20:160
15525702111,6sleep11552830-21unixbench_multi11:15:241
14591732111,5sleep11459349-21diskstats09:50:151
14425202110,1sleep00-21swapper/009:32:350
14165852111,6sleep11417153-21tr09:10:191
14105282111,7sleep2800-21snmpd09:04:592
13207452111,5sleep11320706-31find07:37:331
13145172111,5sleep01314598-21ntpq07:30:180
13007132111,6sleep01302182-21kthreadcore07:20:160
16367052101,4sleep21636759-21ntpq12:35:222
16313902101,4sleep11631437-21ntpq12:30:201
16205192101,4sleep01620624-21ntpq12:20:170
16105522101,4sleep01610769-21users12:10:270
15695372101,5sleep01569730-21/usr/sbin/munin11:35:150
15689182101,4sleep01568938-21munin-run11:35:000
15535432101,4sleep21553562-21apt-get11:20:102
15532622101,4sleep01553334-21munin-run11:20:000
15422882101,4sleep21542417-21users11:05:282
15328932101,4sleep21533076-21grep11:00:142
15067022101,4sleep01506911-21hddtemp_smartct10:35:110
15051092101,5sleep11505157-21ntpq10:30:191
15014492101,4sleep01501658-21hddtemp_smartct10:30:150
14540972101,4sleep01454310-21hddtemp_smartct09:45:140
14540972101,4sleep01454310-21hddtemp_smartct09:45:130
14318762101,5sleep21431888-21apt-get09:24:592
14262692101,5sleep01426298-21munin-run09:20:000
14198662101,4sleep01420007-21ntpq09:10:220
14157652101,4sleep21415816-21munin-run09:10:012
14108092101,4sleep11411070-21df_inode09:05:111
14058292101,4sleep11405953-21perl09:00:131
13832392101,4sleep01383286-21ntpq08:35:200
13683092100,4sleep10-21swapper/108:23:021
13158972101,4sleep01316155-21df_inode07:35:110
13106422101,4sleep11310660-21apt-get07:30:111
13097232101,5sleep21309991-21sed07:25:282
13045272101,4sleep21304771-21users07:20:262
12869822101,4sleep11287001-21apt-get07:10:111
1617243291,3sleep11617996-21cut12:20:141
1585509291,4sleep01585722-21hddtemp_smartct11:50:150
1578478291,4sleep01578525-21ntpq11:40:210
1552126291,4sleep21552239-21ntpq11:15:182
1543051291,2sleep00-21swapper/011:10:100
1517190291,3sleep01517400-21hddtemp_smartct10:45:130
1485635291,4sleep11485845-21hddtemp_smartct10:15:121
1484148291,3sleep21484201-21ntp_states10:10:202
1480121291,3sleep01480140-21apt-get10:10:100
1478904291,4sleep01478957-21ld10:05:210
1442996291,4sleep21443068-21cpuspeed_turbos09:35:132
1442974293,3sleep11286789-21cyclictest09:35:121
1432409291,2sleep20-21swapper/209:25:122
1409580291,3sleep01409637-21ntpq09:00:200
1402593290,1sleep20-21swapper/208:55:172
1383294291,4sleep11379252-21/usr/sbin/munin08:35:201
1375589294,2sleep20-21swapper/208:30:162
1364078291,4sleep01365653-21kthreadcore08:20:160
1332474291,4sleep11332590-21irqstats07:50:161
1631269281,2sleep01631389-21ntpq12:30:190
1628932281,3sleep21630692-21kthreadcore12:30:172
1627030280,1sleep00-21swapper/012:28:390
1626007281,3sleep21626120-21ntpq12:25:202
1611402280,2sleep10-21swapper/112:15:121
1605849280,1sleep10-21swapper/112:08:591
1599706281,3sleep11599760-21modprobe12:00:231
1584950280,1sleep10-21swapper/111:49:551
1527347280,3sleep11527366-21apt-get10:55:131
1506433283,2sleep20-21swapper/210:35:092
1500753280,1sleep20-21swapper/210:28:122
1499897281,2sleep00-21swapper/010:25:210
1469628280,2sleep10-21swapper/110:00:131
1437722280,2sleep10-21swapper/109:30:111
1436770280,1sleep00-21swapper/009:25:250
1422925281,3sleep11424683-21kthreadcore09:15:161
1400294281,3sleep10-21swapper/108:55:131
1325871280,4sleep11326121-21sed07:40:251
1320756280,3sleep21320661-31sort07:37:382
1291726280,1sleep20-21swapper/207:11:452
1636559271,2sleep00-21swapper/012:35:210
1632675271,3sleep11632725-21/usr/sbin/munin12:35:121
1621401270,1sleep20-21swapper/212:22:042
1616402271,2sleep00-21swapper/012:20:000
1616231270,1sleep20-21swapper/212:17:502
1600943270,2sleep20-21swapper/212:05:112
1585822271,2sleep20-21swapper/211:50:162
1580557271,3sleep20-21swapper/211:45:152
1579226270,1sleep10-21swapper/111:42:071
1569180270,2sleep10-21swapper/111:35:121
1562724271,2sleep20-21swapper/211:25:192
1547984270,1sleep20-21swapper/211:14:132
1547854270,1sleep10-21swapper/111:12:321
1540184270,1sleep10-21swapper/111:05:191
1532271270,1sleep00-21swapper/010:59:240
1527069270,1sleep00-21swapper/010:54:480
1527069270,1sleep00-21swapper/010:54:470
1516516270,1sleep10-21swapper/110:43:271
1501180270,3sleep21501272-21cpuspeed_turbos10:30:122
1500732271,1sleep10-21swapper/110:27:551
1495349270,1sleep20-21swapper/210:21:392
1484820270,1sleep10-21swapper/110:11:361
1464330270,2sleep10-21swapper/109:55:131
1463780270,1sleep20-21swapper/209:51:212
1457811271,2sleep20-21swapper/209:45:192
1457811271,2sleep20-21swapper/209:45:192
1453806270,2sleep10-21swapper/109:45:111
1453806270,2sleep10-21swapper/109:45:101
1452321270,2sleep01452495-21strings09:40:170
1449116271,2sleep20-21swapper/209:40:142
1437744270,2sleep21438000-21df09:30:122
1411354271,3sleep01412302-21kthreadcore09:05:130
1395643271,2sleep00-21swapper/008:50:180
1393829270,4sleep00-21swapper/008:45:220
1389163270,2sleep10-21swapper/108:41:301
1374003271,2sleep00-21swapper/008:30:100
1372826270,4sleep10-21swapper/108:25:221
1357890270,1sleep00-21swapper/008:14:150
1352982270,2sleep20-21swapper/208:10:132
1351589270,3sleep11351702-21ntpq08:05:181
1337278270,2sleep10-21swapper/107:55:121
1336793270,1sleep20-21swapper/207:52:132
1626792260,1sleep10-21swapper/112:25:391
1605721260,1sleep00-21swapper/012:07:220
1595094260,1sleep10-21swapper/111:56:051
1574758261,2sleep20-21swapper/211:40:142
1563486260,1sleep00-21swapper/011:27:140
1558240260,1sleep10-21swapper/111:21:301
1537839261,2sleep00-21swapper/011:05:130
1532622261,2sleep00-21swapper/011:00:120
1532086260,1sleep20-21swapper/210:57:032
1526992260,4sleep10-21swapper/110:53:491
1526992260,4sleep10-21swapper/110:53:481
1516922261,2sleep20-21swapper/210:45:112
1516408260,4sleep20-21swapper/210:42:032
1515949260,3sleep00-21swapper/010:40:250
1506411260,2sleep10-21swapper/110:35:081
1495426260,1sleep10-21swapper/110:22:381
1494646260,1sleep00-21swapper/010:20:200
1474532260,1sleep20-21swapper/210:03:592
1469209260,1sleep00-21swapper/009:58:210
1469023260,1sleep20-21swapper/209:55:582
1447797260,4sleep00-21swapper/009:37:360
1437409260,4sleep10-21swapper/109:29:231
1431810260,4sleep10-21swapper/109:24:101
1428684261,2sleep00-21swapper/009:20:160
1421268260,4sleep20-21swapper/209:15:062
1400781261,2sleep00-21swapper/008:55:160
1390049260,2sleep10-21swapper/108:45:141
1372582260,2sleep20-21swapper/208:25:192
1369749261,2sleep00-21swapper/008:25:170
1351813261,2sleep00-21swapper/008:05:210
1347179260,1sleep10-21swapper/108:01:111
1346928261,2sleep20-21swapper/208:00:262
1342227260,1sleep00-21swapper/007:59:440
1331466260,3sleep10-21swapper/107:46:191
1326343261,3sleep2431ktimers/207:42:432
1310356260,4sleep10-21swapper/107:29:431
1305427261,2sleep00-21swapper/007:25:130
1299452260,1sleep00-21swapper/007:18:440
1294508260,1sleep20-21swapper/207:15:062
1291759260,1sleep00-21swapper/007:12:110
1594462250,1sleep00-21swapper/011:55:200
1570355250,1sleep20-21swapper/211:35:172
1534316251,1sleep10-21swapper/111:00:171
1521853250,1sleep20-21swapper/210:50:002
1521853250,1sleep20-21swapper/210:50:002
1485345251,1sleep20-21swapper/210:15:102
1473490250,1sleep00-21swapper/010:00:200
1399393250,2sleep20-21swapper/208:50:282
1384791250,1sleep20-21swapper/208:40:142
1383688250,1sleep20-21swapper/208:35:252
1343118250,2sleep01344606-21cut08:00:160
1336890250,3sleep00-21swapper/007:53:280
1326717250,1sleep00-21swapper/007:45:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional