You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-15 - 11:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot8s.osadl.org (updated Mon Dec 15, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12946119912260,1223cyclictest14319992sleep320:54:573
12946119912151,1208cyclictest1644830-21sleep23:40:003
12946119911281,1123cyclictest15223142sleep322:04:563
134475321100,1sleep30-21swapper/319:45:173
1294611992219,1cyclictest0-21swapper/323:20:013
1294611992011,7cyclictest0-21swapper/321:09:423
129461199201,14cyclictest1593530-21taskset23:00:133
1294611992011,0cyclictest0-21swapper/322:22:183
1294611992010,1cyclictest0-21swapper/321:13:223
1294611991916,1cyclictest0-21swapper/321:35:403
1294611991911,0cyclictest0-21swapper/321:54:273
301798180,6rtkit-daemon1406672-21apt-get20:35:171
1294611991810,1cyclictest0-21swapper/300:20:343
16383792171,4sleep01638424-21munin-run23:35:010
1294611991714,1cyclictest0-21swapper/322:25:133
1294611991613,1cyclictest0-21swapper/323:10:003
1294611991513,1cyclictest0-21swapper/322:57:373
1294611991513,1cyclictest0-21swapper/321:44:303
1294611991513,1cyclictest0-21swapper/321:22:533
1294611991513,1cyclictest0-21swapper/319:10:013
1294611991512,2cyclictest0-21swapper/319:27:283
1294611991512,1cyclictest0-21swapper/323:29:083
1294611991512,1cyclictest0-21swapper/321:25:553
129461199151,10cyclictest1381256-21taskset20:15:143
129461199151,10cyclictest1381256-21taskset20:15:133
129461199150,11cyclictest1309763-21turbostat19:19:593
301798140,5rtkit-daemon447-21systemd-logind20:03:331
129461199149,2cyclictest0-21swapper/319:25:003
129461199146,6cyclictest14578782sleep321:15:133
1294611991412,1cyclictest0-21swapper/323:24:323
1294611991412,1cyclictest0-21swapper/320:30:113
1294611991412,1cyclictest0-21swapper/320:08:563
1294611991412,1cyclictest0-21swapper/320:00:153
1294611991412,0cyclictest0-21swapper/321:30:393
1294611991411,1cyclictest0-21swapper/323:57:343
1294611991411,1cyclictest0-21swapper/322:34:333
1294611991411,1cyclictest0-21swapper/321:55:123
1294611991411,1cyclictest0-21swapper/319:45:003
129461199141,10cyclictest1683541-21taskset00:10:013
129461199141,10cyclictest1580128-21taskset22:46:033
129461199141,10cyclictest1535108-21taskset22:13:583
129461199141,10cyclictest1499210-21taskset21:45:213
129461199141,10cyclictest1425258-21taskset20:45:263
129461199141,10cyclictest1406675-21chrt20:35:183
129461199141,10cyclictest1400001-21chrt20:30:013
129461199141,10cyclictest1387414-21taskset20:20:103
129461199141,10cyclictest1335674-21taskset19:39:253
129461199140,11cyclictest1301192-21taskset19:15:013
129461199140,10cyclictest1719329-21taskset00:35:213
16657872134,6sleep21667450-21kthreadcore23:55:192
129461199134,7cyclictest13235752sleep319:30:153
129461199132,8cyclictest1682693-21taskset00:05:243
129461199131,9cyclictest1676799-21taskset00:03:353
129461199131,9cyclictest1650386-21chrt23:40:203
129461199131,9cyclictest1566983-21chrt22:35:543
129461199131,9cyclictest1566983-21chrt22:35:533
129461199131,9cyclictest1541464-21taskset22:18:103
129461199131,9cyclictest1438667-21chrt21:00:093
129461199131,9cyclictest1418474-21taskset20:40:233
129461199131,9cyclictest1356034-21taskset19:55:183
1294611991312,0cyclictest0-21swapper/323:53:263
1294611991312,0cyclictest0-21swapper/300:15:313
1294611991311,1cyclictest0-21swapper/323:46:543
1294611991310,1cyclictest0-21swapper/322:40:183
1294611991310,1cyclictest0-21swapper/320:11:593
1294611991310,1cyclictest0-21swapper/300:30:233
129461199130,10cyclictest1708710-21chrt00:25:553
129461199130,10cyclictest1638289-21chrt23:33:513
301798120,7rtkit-daemon1516501-21sed22:00:151
14953052120,3sleep11495446-21ntpq21:40:181
129461199121,9cyclictest1586607-21taskset22:52:123
129461199121,9cyclictest1354750-21taskset19:51:273
1294611991210,1cyclictest0-21swapper/320:55:193
129461199120,9cyclictest1606545-21chrt23:10:023
129461199120,9cyclictest1522990-21chrt22:05:283
301798110,4rtkit-daemon636-21snmpd00:25:311
301798110,3rtkit-daemon34-21ksoftirqd/123:49:461
301798100,7rtkit-daemon317-21systemd-udevd21:33:331
301798100,6rtkit-daemon636-21snmpd20:09:521
301798100,6rtkit-daemon1406008-21users20:30:241
301798100,5rtkit-daemon1553603-21ntp_states22:25:181
15663822101,5sleep21566458-21ntp_states22:35:212
15663822101,5sleep21566458-21ntp_states22:35:202
13418272100,7sleep01341829-21sshd19:40:310
30179890,5rtkit-daemon636-21snmpd00:15:231
30179890,5rtkit-daemon3205-21sendmail-mta21:15:051
30179890,5rtkit-daemon3205-21sendmail-mta21:11:051
30179890,5rtkit-daemon1301343-21munin-node22:16:561
30179890,4rtkit-daemon1645551-21sed23:40:151
30179890,4rtkit-daemon1588071-21/usr/sbin/munin22:55:151
30179890,4rtkit-daemon1574137-21apt-config22:45:001
30179890,4rtkit-daemon1322648-21apt-get19:30:001
30179890,3rtkit-daemon1413585-21sed20:40:171
30179890,1rtkit-daemon0-21swapper/122:30:171
30179890,1rtkit-daemon0-21swapper/122:05:261
1606040291,5sleep01606104-21munin-run23:10:010
1560023290,7sleep01560099-21awk22:30:230
1503301290,7sleep21503343-21grep21:50:132
1444217290,6sleep01444434-21users21:00:230
1400524290,1sleep01400687-21interrupts20:30:150
1348553290,7sleep01348554-21sshd19:49:310
1316451291,4sleep11316484-21apt-get19:25:131
1303592291,5sleep21303745-21diskstats19:15:122
1294716290,6sleep01294348-21apt-get19:10:020
30179880,5rtkit-daemon636-21snmpd19:52:081
30179880,4rtkit-daemon667-21lldpd23:38:121
30179880,4rtkit-daemon667-21lldpd19:21:131
30179880,4rtkit-daemon638-21ntpd00:05:121
30179880,4rtkit-daemon636-21snmpd20:20:111
30179880,4rtkit-daemon636-21snmpd19:58:461
30179880,4rtkit-daemon636-21snmpd19:15:101
30179880,4rtkit-daemon636-21snmpd19:11:111
30179880,4rtkit-daemon636-21snmpd00:12:071
30179880,4rtkit-daemon492-21gmain20:26:511
30179880,4rtkit-daemon448-21wpa_supplicant23:30:081
30179880,4rtkit-daemon430-21avahi-daemon00:01:471
30179880,4rtkit-daemon3205-21sendmail-mta21:58:451
30179880,4rtkit-daemon3200908-21packagekitd21:07:211
30179880,4rtkit-daemon1497960-21sed21:45:201
30179880,4rtkit-daemon1489661-21seq21:36:531
30179880,4rtkit-daemon1472033-21cut21:25:161
30179880,4rtkit-daemon1301343-21munin-node23:13:091
30179880,4rtkit-daemon1301343-21munin-node22:51:241
30179880,4rtkit-daemon1301343-21munin-node22:38:481
30179880,4rtkit-daemon1301343-21munin-node22:38:471
30179880,4rtkit-daemon1301343-21munin-node20:55:101
30179880,4rtkit-daemon1301343-21munin-node19:39:431
30179880,4rtkit-daemon1301343-21munin-node00:31:021
30179880,1rtkit-daemon0-21swapper/123:28:481
30179880,1rtkit-daemon0-21swapper/123:19:271
30179880,1rtkit-daemon0-21swapper/122:14:541
30179880,1rtkit-daemon0-21swapper/121:51:361
30179880,1rtkit-daemon0-21swapper/120:46:181
30179880,1rtkit-daemon0-21swapper/120:15:551
30179880,1rtkit-daemon0-21swapper/120:15:551
1721224281,4sleep01721468-21users00:35:270
1691435281,4sleep21693297-21grep00:15:172
1670767280,5sleep01670919-21df_inode00:00:130
1658336281,4sleep11659151-21sed23:50:161
1645264281,4sleep01645464-21hddtemp_smartct23:40:130
1644683280,5sleep21644684-21sshd23:38:352
1624866281,4sleep01624977-21/usr/sbin/munin23:20:230
1606557281,4sleep01606590-21apt-get23:10:120
1599330283,4sleep1376-21hwrng23:01:131
1593962280,5sleep21594949-21irqstats23:00:172
1530470281,5sleep01532188-21kthreadcore22:10:180
1522318281,4sleep21522335-21munin-run22:05:002
1516176281,4sleep01516322-21df22:00:140
1506941281,4sleep01508393-21memory21:50:200
1486406281,4sleep21488592-21tr21:35:442
1464683281,4sleep11464953-21sed21:20:161
1457872280,6sleep21457876-21find21:15:112
1342844280,5sleep11343548-21sed19:45:161
1310053280,4sleep01310182-21cut19:20:140
1303951280,5sleep01304469-21sed19:15:150
30179870,4rtkit-daemon636-21snmpd23:55:061
30179870,4rtkit-daemon636-21snmpd20:10:291
30179870,3rtkit-daemon638-21ntpd23:05:161
30179870,3rtkit-daemon636-21snmpd21:01:001
30179870,3rtkit-daemon636-21snmpd00:35:041
30179870,3rtkit-daemon492-21gmain19:40:591
30179870,3rtkit-daemon3205-21sendmail-mta19:05:551
30179870,3rtkit-daemon1301343-21munin-node23:20:031
30179870,3rtkit-daemon1301343-21munin-node22:22:411
30179870,3rtkit-daemon1301343-21munin-node20:52:481
30179870,3rtkit-daemon1301343-21munin-node00:23:041
30179870,2rtkit-daemon638-21ntpd22:45:241
1717261270,5sleep21719327-21tr00:35:212
1682574270,4sleep21682690-21cat00:05:232
1677198270,1sleep00-21swapper/000:05:130
1634750270,4sleep21636915-21grep23:30:182
1630950270,4sleep21631090-21awk23:25:212
1614426270,4sleep01616033-21grep23:15:180
1606746270,4sleep21606861-21fw_packets23:10:142
1575734271,3sleep01577171-21grep22:45:160
1567983270,4sleep01568479-21cut22:40:160
1542279270,4sleep01542541-21cut22:20:160
1525181271,4sleep21527360-21kthreadcore22:05:322
1483400271,3sleep01483262-21apt-get21:35:020
1399962271,4sleep21400000-21systemd-run20:30:012
1392580273,2sleep00-21swapper/020:20:200
1301136270,4sleep21301190-21latency_hist19:15:012
1294140270,1sleep20-21swapper/219:07:302
1711388261,2sleep00-21swapper/000:30:170
1644631260,1sleep00-21swapper/023:37:530
1599344260,1sleep00-21swapper/023:01:230
1592316260,3sleep21592374-21ntp_states22:55:202
1585930260,3sleep01585988-21ntp_states22:50:220
1569407260,4sleep21571497-21kthreadcore22:40:172
1567085260,1sleep00-21swapper/022:37:120
1567085260,1sleep00-21swapper/022:37:110
1478521261,3sleep01479719-21cut21:30:160
1477335260,4sleep21477093-21/usr/sbin/munin21:30:152
1464954260,4sleep21465908-21kthreadcore21:20:172
1451903260,3sleep21452879-21cut21:10:142
1424844260,3sleep01424935-21switchtime20:45:200
1419478261,2sleep20-21swapper/220:45:102
1413128260,4sleep21413284-21diskstats20:40:152
1407035260,4sleep21407392-21irqcore20:35:212
1394880260,3sleep01396059-21cut20:25:180
1380954261,2sleep00-21swapper/020:15:120
1380954261,2sleep00-21swapper/020:15:110
1350643260,3sleep21352749-21grep19:50:182
1324387261,3sleep00-21swapper/019:30:160
1709288250,2sleep20-21swapper/200:30:112
1702901250,2sleep21702952-21grep00:25:112
1700810250,3sleep01701636-21cat00:20:200
1696155250,2sleep00-21swapper/000:20:000
1689527250,1sleep20-21swapper/200:12:412
1675838250,3sleep20-21swapper/200:00:182
1670105250,1sleep00-21swapper/023:55:450
1663771250,1sleep00-21swapper/023:51:070
1657330250,1sleep00-21swapper/023:46:030
1627313251,2sleep00-21swapper/023:25:190
1593127250,1sleep00-21swapper/022:58:500
1543619251,2sleep20-21swapper/222:20:182
1535921250,2sleep20-21swapper/222:15:182
1515730250,1sleep20-21swapper/221:58:182
1510437251,2sleep01509680-21/usr/sbin/munin21:55:160
1501403250,2sleep01501828-21strings21:45:230
1496222250,1sleep00-21swapper/021:43:300
1490966251,2sleep20-21swapper/221:40:152
1476803250,1sleep00-21swapper/021:29:360
1470288250,1sleep00-21swapper/021:23:000
1451024250,1sleep20-21swapper/221:07:512
1445057250,2sleep00-21swapper/021:05:130
1440340250,2sleep21442436-21kthreadcore21:00:172
1432258250,2sleep20-21swapper/220:55:132
1431773250,1sleep20-21swapper/220:52:052
1425907251,2sleep00-21swapper/020:50:130
1418997250,1sleep00-21swapper/020:42:050
1392743250,2sleep20-21swapper/220:20:222
1386991250,1sleep20-21swapper/220:18:072
1386991250,1sleep20-21swapper/220:18:062
1379941251,2sleep00-21swapper/020:10:230
1374231250,1sleep00-21swapper/020:09:200
1374008250,1sleep20-21swapper/220:06:292
1367210250,3sleep01367441-21unixbench_multi20:00:210
1361772251,2sleep20-21swapper/220:00:122
1361295250,1sleep00-21swapper/019:57:500
1361250250,1sleep20-21swapper/219:57:152
1348843250,1sleep00-21swapper/019:50:080
1343549250,2sleep20-21swapper/219:45:172
1336988251,2sleep20-21swapper/219:40:162
1329598250,2sleep00-21swapper/019:35:130
1327725250,1sleep20-21swapper/219:30:182
1317908250,2sleep20-21swapper/219:25:182
1289713250,2sleep01290924-21kthreadcore19:05:120
1708301240,1sleep00-21swapper/000:25:210
1697425240,2sleep21698664-21cut00:20:192
1659152241,1sleep20-21swapper/223:50:172
1645805240,2sleep21646789-21kthreadcore23:40:162
1620859240,2sleep21622157-21cat23:20:162
1600083240,1sleep20-21swapper/223:05:132
1586523240,3sleep20-21swapper/222:51:082
1555015240,1sleep20-21swapper/222:30:162
1554253240,3sleep00-21swapper/022:27:250
1549784240,2sleep21551736-21kthreadcore22:25:162
1523329240,2sleep01524046-21irqstats22:05:300
1498052240,2sleep21499209-21kthreadcore21:45:212
1472157240,2sleep20-21swapper/221:25:172
1459365241,2sleep01461382-21kthreadcore21:15:170
1438281240,3sleep00-21swapper/020:58:360
1412002240,2sleep01412128-21/usr/sbin/munin20:35:270
1401675240,1sleep20-21swapper/220:30:162
1374542240,1sleep20-21swapper/220:10:132
1329779240,2sleep21329945-21gltestperf19:35:162
1322327240,3sleep00-21swapper/019:26:080
1683748230,1sleep00-21swapper/000:10:150
1655909230,1sleep20-21swapper/223:45:212
1618915230,2sleep20-21swapper/223:17:162
1579336230,1sleep20-21swapper/222:45:192
1535640230,1sleep00-21swapper/022:15:160
1534509230,1sleep20-21swapper/222:10:252
1454087230,1sleep00-21swapper/021:10:170
1310216230,1sleep20-21swapper/219:20:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional