You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-12 - 03:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8s.osadl.org (updated Tue Nov 11, 2025 00:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
147052421040,3sleep1321rcuc/119:15:211
1458301991916,1cyclictest0-21swapper/321:15:003
16898062180,7sleep016-21ksoftirqd/022:00:010
1458301991714,1cyclictest0-21swapper/322:15:003
1458301991714,1cyclictest0-21swapper/300:20:203
17059402161,4sleep01706092-21df22:10:120
157898160,4rtkit-daemon1538279-21cut20:10:180
1458301991613,1cyclictest0-21swapper/321:36:403
1458301991613,1cyclictest0-21swapper/321:30:163
1458301991613,1cyclictest0-21swapper/320:20:183
1458301991613,1cyclictest0-21swapper/319:20:013
19386972151,4sleep01938733-21apt-get00:35:150
19286112150,4sleep1331ktimers/100:25:501
19285852150,1chrt0-21swapper/200:25:462
1458301991512,1cyclictest0-21swapper/322:40:203
1458301991512,1cyclictest0-21swapper/300:00:013
1458301991412,1cyclictest0-21swapper/321:41:133
1458301991412,1cyclictest0-21swapper/319:40:193
1458301991412,1cyclictest0-21swapper/300:00:183
1458301991411,1cyclictest0-21swapper/323:00:213
1458301991411,1cyclictest0-21swapper/322:35:213
1458301991411,1cyclictest0-21swapper/322:25:003
1458301991411,1cyclictest0-21swapper/320:00:013
1458301991411,1cyclictest0-21swapper/319:10:183
1458301991411,1cyclictest0-21swapper/319:10:183
1458301991411,1cyclictest0-21swapper/300:10:153
157898130,3rtkit-daemon0-21swapper/121:19:171
1458301991310,1cyclictest0-21swapper/323:50:213
1458301991310,1cyclictest0-21swapper/323:30:183
1458301991310,1cyclictest0-21swapper/323:22:453
1458301991310,1cyclictest0-21swapper/322:45:143
1458301991310,1cyclictest0-21swapper/322:30:023
1458301991310,1cyclictest0-21swapper/321:55:013
1458301991310,1cyclictest0-21swapper/320:35:143
1458301991310,1cyclictest0-21swapper/320:20:023
1458301991310,1cyclictest0-21swapper/320:05:113
18075032120,5sleep01807648-21switchtime23:10:230
157898120,3rtkit-daemon505-21wpa_supplicant00:02:002
145830199129,1cyclictest0-21swapper/323:40:143
145830199129,1cyclictest0-21swapper/323:16:363
145830199129,1cyclictest0-21swapper/323:14:093
145830199129,1cyclictest0-21swapper/322:55:183
145830199129,1cyclictest0-21swapper/322:55:003
145830199129,1cyclictest0-21swapper/321:20:203
145830199129,1cyclictest0-21swapper/319:50:003
145830199129,1cyclictest0-21swapper/300:25:153
145830199121,8cyclictest1936468-21taskset00:30:283
145830199121,8cyclictest1704243-21taskset22:06:573
145830199121,8cyclictest1578062-21taskset20:40:383
145830199121,8cyclictest1528396-21taskset20:04:363
145830199121,8cyclictest1515688-21taskset19:50:453
145830199121,8cyclictest1491052-21taskset19:32:303
145830199121,7cyclictest1713953-21taskset22:15:153
145830199121,7cyclictest1609624-21turbostat21:10:003
1458301991210,1cyclictest0-21swapper/323:35:153
1458301991210,1cyclictest0-21swapper/321:55:433
145830199120,8cyclictest1828759-21taskset23:25:203
14522422122,5sleep01452275-21apt-get19:05:110
19322312114,5sleep01933554-21cut00:30:180
157898110,6rtkit-daemon1713910-21/usr/sbin/munin22:15:251
145830199119,1cyclictest0-21swapper/320:10:183
145830199118,1cyclictest0-21swapper/323:05:163
145830199118,1cyclictest0-21swapper/322:30:203
145830199118,1cyclictest0-21swapper/321:15:173
145830199118,1cyclictest0-21swapper/320:45:003
145830199118,1cyclictest0-21swapper/320:25:233
145830199118,1cyclictest0-21swapper/300:35:183
145830199114,5cyclictest1672775-21taskset21:48:143
145830199111,7cyclictest1906372-21taskset00:15:053
145830199111,7cyclictest1895088-21taskset00:05:193
145830199111,7cyclictest1858364-21taskset23:45:123
145830199111,7cyclictest1858364-21taskset23:45:123
145830199111,7cyclictest1633920-21taskset21:25:143
145830199111,7cyclictest1633920-21taskset21:25:143
145830199111,7cyclictest1491725-21taskset19:35:143
145830199111,7cyclictest1472913-21taskset19:20:123
17941482100,8sleep21794183-21acpi23:05:122
17223852103,5sleep21722459-21fschecks_count22:20:162
16978432101,5sleep11697878-21apt-get22:05:131
16658982101,5sleep11665933-21apt-get21:45:161
157898100,6rtkit-daemon1649883-21/usr/sbin/munin21:35:261
157898100,5rtkit-daemon1738244-21apt-get22:30:131
157898100,5rtkit-daemon1597523-21apt-get21:00:140
157898100,5rtkit-daemon1510133-21apt-get19:50:160
157898100,3rtkit-daemon724-21snmpd23:25:292
157898100,3rtkit-daemon724-21snmpd22:42:341
157898100,3rtkit-daemon538-21gmain20:56:200
157898100,3rtkit-daemon505-21wpa_supplicant00:06:212
157898100,2rtkit-daemon1913914-21ssh00:19:332
15225032102,4sleep01522536-21apt-get20:00:130
145830199107,1cyclictest0-21swapper/320:55:243
145830199107,1cyclictest0-21swapper/320:50:153
145830199107,1cyclictest0-21swapper/319:25:163
145830199101,6cyclictest1689928-21taskset22:00:123
145830199101,6cyclictest1597475-21taskset21:00:023
145830199100,3cyclictest15605172sleep320:30:173
1890397290,2sleep00-21swapper/000:05:130
1880515291,6sleep21880523-21ssh23:56:302
1753624290,1sleep01753629-21sh22:39:210
1671645290,3sleep01671652-21sh21:45:350
15789890,5rtkit-daemon1864773-21ssh23:47:002
15789890,5rtkit-daemon1864773-21ssh23:47:002
15789890,4rtkit-daemon724-21snmpd20:45:220
15789890,4rtkit-daemon501-21rsyslogd20:30:150
15789890,4rtkit-daemon1696453-21ssh22:02:331
15789890,4rtkit-daemon1658265-21ssh21:40:161
1572482291,4sleep21572514-21apt-get20:40:142
1567251290,1sleep21568197-21kthreadcore20:35:172
1531354290,7sleep00-21swapper/020:05:180
1920868280,4sleep01920876-21ssh00:22:050
1914441281,3sleep20-21swapper/200:20:142
1865681280,2sleep01609622-21diskmemload23:49:020
1865681280,2sleep01609622-21diskmemload23:49:020
1848970281,4sleep11848978-21ssh23:37:371
1754234281,3sleep00-21swapper/022:40:150
1745143281,5sleep01745151-21ssh22:33:160
1590340281,4sleep21590536-21unixbench-2d20:50:242
15789880,4rtkit-daemon98157-21packagekitd19:48:560
15789880,4rtkit-daemon98157-21packagekitd19:41:560
15789880,4rtkit-daemon727-21ntpd21:12:561
15789880,4rtkit-daemon724-21snmpd23:51:142
15789880,4rtkit-daemon724-21snmpd23:32:122
15789880,4rtkit-daemon724-21snmpd23:10:092
15789880,4rtkit-daemon724-21snmpd21:55:511
15789880,4rtkit-daemon724-21snmpd21:29:271
15789880,4rtkit-daemon724-21snmpd21:29:261
15789880,4rtkit-daemon724-21snmpd20:25:540
15789880,4rtkit-daemon724-21snmpd20:17:120
15789880,4rtkit-daemon724-21snmpd00:37:182
15789880,4rtkit-daemon538-21gmain19:05:041
15789880,4rtkit-daemon387-21systemd-udevd21:30:061
15789880,4rtkit-daemon387-21systemd-udevd00:10:062
15789880,4rtkit-daemon368-21systemd-journal22:53:062
15789880,4rtkit-daemon368-21systemd-journal21:52:381
15789880,4rtkit-daemon1824985-21ssh23:22:472
15789880,4rtkit-daemon1778105-21kworker/u8:2+nfsiod23:01:482
15789880,4rtkit-daemon1610-21sendmail-mta23:40:202
15789880,4rtkit-daemon1610-21sendmail-mta22:21:301
15789880,4rtkit-daemon1577375-21missed_timers20:40:200
15789880,4rtkit-daemon1472946-21apt-get19:20:141
15789880,4rtkit-daemon1464555-21munin-node23:18:052
15789880,4rtkit-daemon1464555-21munin-node22:55:302
15789880,4rtkit-daemon1464555-21munin-node22:35:301
15789880,4rtkit-daemon1464555-21munin-node22:28:541
15789880,4rtkit-daemon1464555-21munin-node21:22:311
15789880,4rtkit-daemon1464555-21munin-node21:07:481
15789880,4rtkit-daemon1464555-21munin-node20:20:490
15789880,4rtkit-daemon1464555-21munin-node19:30:090
15789880,4rtkit-daemon1464555-21munin-node00:31:042
15789880,3rtkit-daemon727-21ntpd22:10:141
15789880,1rtkit-daemon0-21swapper/022:45:320
15789880,1rtkit-daemon0-21swapper/019:55:380
1552377281,4sleep11552515-21awk20:20:211
1491506281,4sleep01491539-21apt-get19:35:120
1463584281,4sleep01463709-21python319:10:250
1463584281,4sleep01463709-21python319:10:240
1458863280,6sleep11459107-21cut19:10:171
1458863280,6sleep11459107-21cut19:10:171
1458151280,2sleep30-21swapper/319:09:553
1930708270,1sleep10-21swapper/100:30:071
1922156270,3sleep01922177-21apt-get00:25:010
1903298270,4sleep11903489-21netstat00:10:191
1882071271,4sleep01882188-21sed00:00:000
1880117270,1sleep10-21swapper/123:55:291
1866395270,1sleep00-21swapper/023:50:110
1850392272,2sleep11850427-21apt-get23:40:141
1832918270,5sleep01832928-21ssh23:27:380
1823970270,1sleep00-21swapper/023:20:330
1816407270,1sleep00-21swapper/023:16:360
1793748270,5sleep11793756-21ssh23:04:431
1776785270,4sleep11776793-21ssh22:52:381
1768180270,5sleep11768188-21ssh22:46:081
1727993270,4sleep01728003-21ssh22:20:380
1721429270,4sleep21721437-21ssh22:19:452
1689884271,3sleep20-21swapper/222:00:122
1656444270,1sleep00-21swapper/021:37:220
1648024270,5sleep01648032-21ssh21:31:210
1641604270,4sleep00-21swapper/021:30:000
1641604270,4sleep00-21swapper/021:30:000
1631829270,4sleep21631837-21ssh21:20:532
1623817270,5sleep01623825-21ssh21:15:510
1612010270,4sleep01614027-21kthreadcore21:10:170
1591324271,3sleep11591479-21diskstats20:55:151
15789870,3rtkit-daemon727-21ntpd22:50:000
15789870,3rtkit-daemon724-21snmpd20:35:080
15789870,3rtkit-daemon505-21wpa_supplicant19:26:271
15789870,3rtkit-daemon1610-21sendmail-mta21:04:291
15789870,3rtkit-daemon1585523-21if_eno120:50:150
15789870,3rtkit-daemon1464555-21munin-node23:35:072
15789870,3rtkit-daemon1464555-21munin-node19:28:490
1547516271,3sleep20-21swapper/220:20:142
1491685270,4sleep21491724-21grep19:35:142
1921492260,1sleep10-21swapper/100:23:301
1911822260,3sleep01912060-21grep00:15:280
1905168260,1sleep00-21swapper/000:12:510
1866404261,3sleep11866410-21ssh23:50:121
1857619260,1sleep00-21swapper/023:43:570
1848392262,2sleep01458293-21cyclictest23:36:170
1840620260,1sleep00-21swapper/023:31:550
1834294261,2sleep10-21swapper/123:30:121
1815972260,4sleep11815980-21ssh23:15:331
1802628260,4sleep11803130-21irqcore23:10:161
1799963260,1sleep00-21swapper/023:05:380
1792656260,1sleep00-21swapper/023:02:180
1778121260,1sleep00-21swapper/022:55:080
1766004260,3sleep21767112-21memory22:45:162
1761262260,4sleep21761263-21sshd22:43:362
1736589260,4sleep00-21swapper/022:27:030
1731658261,3sleep21733399-21kthreadcore22:25:212
1703941260,1sleep00-21swapper/022:06:110
1688289260,1sleep20-21swapper/221:57:082
1679914260,1sleep00-21swapper/021:51:120
1673903261,2sleep20-21swapper/221:50:142
1667375260,4sleep21668991-21grep21:45:212
1657883260,1sleep0191rcu_preempt21:40:060
1642151261,3sleep21642351-21hddtemp_smartct21:30:152
1625910260,1sleep00-21swapper/021:20:030
1619346261,3sleep21620582-21kthreadcore21:15:162
1585261260,3sleep11585306-21fschecks_count20:50:141
1580545260,3sleep21582551-21cut20:45:182
1561805260,3sleep11563828-21tr20:30:191
1523607260,4sleep11524741-21kthreadcore20:00:181
1516022261,2sleep10-21swapper/119:55:011
1503915260,3sleep20-21swapper/219:45:122
1499879261,2sleep10-21swapper/119:40:181
1492603260,4sleep11493716-21tr19:35:171
1487062261,2sleep10-21swapper/119:30:211
1479154260,4sleep21479248-21cpuspeed_turbos19:25:132
1478705260,1sleep00-21swapper/019:22:580
1466775261,3sleep01466887-21/usr/sbin/munin19:15:170
1451800260,2sleep20-21swapper/219:05:012
1906453251,2sleep10-21swapper/100:15:161
1891872251,2sleep10-21swapper/100:05:181
1833984250,3sleep11834009-21idleruntime-cro23:30:001
1818467250,2sleep11818590-21ssh23:20:141
1799390250,2sleep10-21swapper/123:05:231
1778170250,2sleep10-21swapper/122:55:131
1738146250,3sleep21738154-21ssh22:30:062
1713893250,3sleep01713901-21ssh22:15:130
1711272250,2sleep20-21swapper/222:10:222
1687621250,3sleep00-21swapper/021:55:340
1652580251,2sleep20-21swapper/221:35:192
1641334250,3sleep20-21swapper/221:29:262
1641334250,3sleep20-21swapper/221:29:262
1604533251,2sleep20-21swapper/221:05:182
1579638251,2sleep10-21swapper/120:45:171
1566277250,2sleep11566310-21apt-get20:35:121
1541298251,1sleep20-21swapper/220:15:132
1535594250,3sleep21536567-21irqstats20:10:162
1528683250,2sleep20-21swapper/220:05:132
1511024250,3sleep21511869-21grep19:50:212
1502965250,3sleep21503146-21timerwakeupswit19:40:232
1478640250,1sleep20-21swapper/219:22:122
1472023250,3sleep21472267-21users19:15:272
1459108250,3sleep21459956-21cut19:10:182
1459108250,3sleep21459956-21cut19:10:182
1946189240,2sleep11946197-21ssh00:39:371
1888829240,2sleep11609622-21diskmemload00:02:101
1887550240,2sleep00-21swapper/000:00:200
1859851240,2sleep10-21swapper/123:45:201
1859851240,2sleep10-21swapper/123:45:191
1746227240,2sleep20-21swapper/222:35:152
1699304240,2sleep20-21swapper/222:05:172
1665108240,2sleep21665116-21ssh21:43:572
1615842240,2sleep21615850-21ssh21:10:592
1607498240,1sleep00-21swapper/021:05:200
1597885240,2sleep20-21swapper/221:00:172
1577333240,2sleep10-21swapper/120:40:221
1559104240,2sleep11553936-21/usr/sbin/munin20:25:221
1541681240,2sleep11541961-21cut20:15:161
1596405230,1sleep20-21swapper/220:55:242
1563829230,1sleep20-21swapper/220:30:202
1553923230,1sleep20-21swapper/220:25:032
1535115230,1sleep10-21swapper/120:10:031
1528958230,1sleep10-21swapper/120:05:151
1522682230,1sleep20-21swapper/220:00:152
1521871230,1sleep20-21swapper/219:55:302
1516277230,1sleep10-21swapper/119:55:131
1509493230,1sleep10-21swapper/119:45:351
1485130230,1sleep20-21swapper/219:30:032
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional