You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-10 - 21:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Sat Jan 10, 2026 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
14105512149101,29sleep00-21swapper/019:09:050
14105312142109,28sleep20-21swapper/219:08:512
14103122142109,28sleep10-21swapper/119:06:021
14102552141109,27sleep30-21swapper/319:05:213
1410902992519,4cyclictest1517024-21idleruntime-cro00:40:012
1410894992318,3cyclictest0-21swapper/000:00:000
141090299221,8cyclictest1497782-21perf23:39:592
1410902992213,2cyclictest191rcu_preempt23:10:002
1410898992211,7cyclictest1449510-21sshd21:06:001
1410894992212,8cyclictest1412303-21awk19:15:010
141089499220,19cyclictest1466678-21ntpq22:00:140
141090299218,12cyclictest1479152-21/usr/sbin/munin22:40:152
1410902992113,5cyclictest1491451-21systemctl23:19:592
1410902992110,4cyclictest43-21ksoftirqd/221:20:002
1410898992116,4cyclictest1416936-21ntp_kernel_pll_19:25:191
141090499204,7cyclictest1440150-10dpkg20:40:003
1410904992015,3cyclictest1484893-21sendmail22:59:593
1410904992012,5cyclictest1472508-21apt-get22:20:113
141090299209,6cyclictest1478576-21tr22:40:012
1410902992016,3cyclictest1509958-21ntp_offset00:15:182
1410902992012,5cyclictest1425873-21dpkg19:55:112
1410902992010,2cyclictest0-21swapper/220:40:002
141089899204,4cyclictest1456083-21seq21:29:141
1410898992017,2cyclictest656-21systemd-logind20:40:001
1410898992012,6cyclictest1-21systemd00:25:011
1410894992010,5cyclictest1510438-10sshd00:16:550
1410904991916,2cyclictest13336-21JS19:58:003
1410904991915,2cyclictest1462741-21cron21:50:003
1410904991910,6cyclictest1488030-21apt-get23:10:003
141090299195,3cyclictest191rcu_preempt21:50:002
1410902991915,2cyclictest0-21swapper/223:00:002
141090299191,3cyclictest1430353-21munin-run20:10:012
1410902991910,4cyclictest1428447-10sshd20:00:572
141090299190,17cyclictest1450597-21ldconfig21:10:162
1410898991910,4cyclictest1444991-10sshd20:53:551
141089899190,7cyclictest1441134-21ntpq20:40:141
1410894991915,3cyclictest1-21systemd23:09:590
1410894991915,2cyclictest1414305-21idleruntime-cro19:20:000
1410894991914,3cyclictest758-21mta-sts-daemon21:20:130
1410894991914,3cyclictest758-21mta-sts-daemon21:20:120
1410894991912,5cyclictest1441935-21sh20:45:000
1410894991912,2cyclictest0-21swapper/021:55:130
1410904991814,2cyclictest0-21swapper/319:40:003
1410904991811,5cyclictest1510674-21chrt00:19:543
1410904991811,5cyclictest1433447-21taskset20:16:093
141090499180,16cyclictest1505630-21chrt00:00:503
141090499180,16cyclictest1435023-21sshd20:20:513
141090499180,16cyclictest1412152-21taskset19:13:233
141090299182,2cyclictest0-21swapper/223:00:142
1410902991815,2cyclictest659-21in:imuxsock22:20:012
1410902991814,3cyclictest421ktimers/220:10:112
1410902991811,5cyclictest1476047-21memory22:30:152
1410902991810,6cyclictest1458017-21awk21:35:002
141090299180,5cyclictest1469720-21meminfo22:10:152
141090299180,16cyclictest1464232-21taskset21:53:592
141090299180,16cyclictest1434261-21/usr/sbin/munin20:20:122
141090299180,16cyclictest1412079-21chrt19:12:272
141090299180,15cyclictest1454822-21taskset21:25:102
1410898991811,5cyclictest1457681-21seq21:31:021
141089899180,16cyclictest1460823-21runrttasks21:40:381
141089499187,6cyclictest1453009-21basename21:19:590
141089499187,6cyclictest1440101-21systemctl20:39:590
141089499187,4cyclictest1456469-10gpg-connect-age21:30:010
141089499186,10cyclictest1429247-21/usr/sbin/munin20:05:130
141089499181,8cyclictest1512307-10awk00:25:000
1410894991811,5cyclictest1478829-21apt-get22:40:110
1410894991811,5cyclictest1470636-21sh22:15:010
141089499180,16cyclictest1468757-21taskset22:06:120
141089499180,16cyclictest1420687-21taskset19:37:590
1410904991712,3cyclictest760-21mta-sts-daemon20:10:113
1410904991711,4cyclictest1507225-21taskset00:06:083
1410904991711,4cyclictest1468691-21taskset22:05:213
1410904991711,4cyclictest1425431-21sshd19:52:573
1410904991711,4cyclictest1416978-21/usr/sbin/munin19:25:203
141090499171,14cyclictest13236-21gnome-shell22:12:583
141090499171,11cyclictest1428779-21munin-run20:05:003
1410904991710,5cyclictest1412968-21taskset19:15:103
141090499170,5cyclictest1497382-21seq23:35:203
141090499170,5cyclictest1474469-21meminfo22:25:153
141090299178,5cyclictest1512306-21basename00:24:592
141090299178,3cyclictest191rcu_preempt00:10:002
141090299177,7cyclictest1500930-21apt-get23:50:002
141090299177,2cyclictest191rcu_preempt22:00:012
141090299175,9cyclictest1417868-21ls19:30:002
1410902991715,1cyclictest659-21in:imuxsock20:45:062
141090299170,4cyclictest1425531-21runrttasks19:54:132
141090299170,15cyclictest1508012-21diskstats00:10:122
141090299170,15cyclictest1432415-21runrttasks20:15:022
141089899179,6cyclictest1425989-21expr19:55:131
141089899178,5cyclictest355-21systemd-journal22:05:001
141089899176,5cyclictest1465081-10ld21:55:161
141089899174,11cyclictest13236-21gnome-shell22:27:581
1410898991715,1cyclictest0-21swapper/119:44:491
1410898991712,3cyclictest0-21swapper/123:25:001
1410898991711,1cyclictest1476215-21/usr/sbin/munin22:30:171
1410898991710,6cyclictest1423280-21/usr/sbin/munin19:45:161
141089899170,4cyclictest1459475-21chrt21:38:471
141089899170,15cyclictest1500855-21taskset23:49:111
141089899170,15cyclictest1414198-21taskset19:19:101
141089499176,6cyclictest1501681-21hddtemp_smartct23:50:120
1410894991715,1cyclictest1448502-21/usr/sbin/munin21:05:200
1410894991712,4cyclictest1456170-21awk21:30:000
1410894991711,4cyclictest795-21runrttasks23:27:260
141089499170,4cyclictest1427679-21/usr/sbin/munin20:00:150
141089499170,15cyclictest13236-21gnome-shell21:12:590
141090499168,5cyclictest1422466-21grep19:45:003
141090499161,7cyclictest1444012-21smartctl20:50:133
141090499161,3cyclictest1437730-21ntpq20:30:193
1410904991610,4cyclictest1472080-21taskset22:18:233
1410904991610,4cyclictest1458243-21taskset21:35:083
141090499160,5cyclictest1478241-21taskset22:36:213
141090499160,14cyclictest743-21sshd00:26:553
141090299168,4cyclictest1505636-10sshd00:00:552
141090299167,6cyclictest1512557-21apt-get00:25:102
141090299167,6cyclictest1435626-21runrttasks20:25:112
141090299167,3cyclictest191rcu_preempt00:35:012
141090299162,12cyclictest13236-21gnome-shell23:30:592
1410902991611,3cyclictest0-21swapper/221:25:002
1410902991611,3cyclictest0-21swapper/221:24:592
141090299161,12cyclictest1414293-21systemctl19:20:012
1410902991610,4cyclictest1419516-21seq19:35:042
141090299160,9cyclictest645-21dbus-daemon21:00:002
141090299160,15cyclictest1480377-21/usr/sbin/munin22:45:142
141090299160,14cyclictest1482695-21/usr/sbin/munin22:50:192
141090299160,14cyclictest1448281-21/usr/sbin/munin21:04:592
141089899169,5cyclictest1489819-21runrttasks23:14:361
141089899169,5cyclictest1484617-21chrt22:57:041
141089899167,6cyclictest1490726-21smartctl23:15:201
141089899166,7cyclictest1506821-21smartctl00:05:181
141089899166,7cyclictest1437730-21ntpq20:30:191
141089899166,7cyclictest1430356-21ls20:10:011
1410898991612,2cyclictest1496935-21ntpq23:35:151
1410898991612,2cyclictest1443182-21sshd20:45:551
1410898991612,2cyclictest0-21swapper/123:05:161
141089899161,13cyclictest1480964-21ntpq22:45:171
141089899161,12cyclictest1504405-21munin-run00:00:011
141089899161,10cyclictest1494878-21apt-get23:30:111
1410898991610,4cyclictest1494275-21sshd23:25:551
1410898991610,4cyclictest1477535-21/usr/sbin/munin22:35:161
1410898991610,4cyclictest1431583-21sshd20:10:491
141089899160,5cyclictest1501657-21df_inode23:50:111
141089899160,14cyclictest743-21sshd00:12:551
141089899160,14cyclictest1514107-21munin-node00:30:111
141089899160,14cyclictest1505842-21taskset00:03:341
141089899160,14cyclictest1415766-21runrttasks19:20:261
141089499169,5cyclictest1462761-21awk21:50:010
141089499169,2cyclictest1434025-21/usr/sbin/munin20:20:150
1410894991616,0cyclictest0-21swapper/020:10:200
141089499161,13cyclictest794-21snmpd00:09:560
141089499161,13cyclictest13236-21gnome-shell19:54:590
1410894991610,5cyclictest1515681-21/usr/sbin/munin00:35:200
1410894991610,4cyclictest795-21runrttasks20:17:470
1410894991610,0cyclictest0-21swapper/000:30:130
141089499160,4cyclictest795-21runrttasks22:48:340
141089499160,3cyclictest1491702-21/usr/sbin/munin23:20:180
141090499159,4cyclictest1467473-21sshd22:04:563
141090499156,5cyclictest645-21dbus-daemon21:09:593
141090499155,6cyclictest1511584-21cut00:20:183
1410904991512,2cyclictest1462992-21/usr/sbin/munin21:50:143
1410904991512,1cyclictest1464565-21/usr/sbin/munin21:55:133
1410904991511,3cyclictest1514107-21/usr/sbin/munin00:30:143
141090499151,10cyclictest1446674-21munin-run21:00:003
141090499150,4cyclictest1502679-21taskset23:54:033
141090299158,2cyclictest1416126-21cron19:25:012
141090299157,6cyclictest1445094-21munin-run20:55:002
141090299156,4cyclictest1449739-10sshd21:08:552
141090299156,2cyclictest191rcu_preempt23:50:022
1410902991513,1cyclictest0-21swapper/223:25:002
1410902991512,2cyclictest381-21systemd-udevd22:26:492
1410902991511,2cyclictest1467489-21idleruntime-cro22:05:012
141090299151,10cyclictest13065-10wireplumber23:45:002
141090299150,5cyclictest1459220-21chrt21:35:332
141089899157,5cyclictest1464010-21sshd21:51:091
141089899156,6cyclictest1469844-21ntpq22:10:161
141089899153,8cyclictest1468302-21ntpq22:05:171
1410898991512,2cyclictest760-21mta-sts-daemon00:15:171
1410898991512,2cyclictest331ktimers/123:40:111
1410898991511,2cyclictest1451399-21cron21:15:001
141089899151,10cyclictest1480143-21munin-run22:45:011
141089899151,10cyclictest1473821-21munin-run22:25:001
1410898991510,4cyclictest1453236-21chrt21:20:011
1410898991510,4cyclictest1453236-21chrt21:20:001
141089499159,4cyclictest1513573-21taskset00:26:260
141089499156,7cyclictest1415049-21hddtemp_smartct19:20:120
141089499156,5cyclictest1490579-21cat23:15:180
141089499151,4cyclictest794-21snmpd20:25:230
1410894991512,2cyclictest650-21polkitd23:49:590
1410894991512,2cyclictest0-21swapper/022:25:140
1410894991512,2cyclictest0-21swapper/020:56:020
141089499150,14cyclictest13236-21gnome-shell19:43:090
141089499150,13cyclictest1472480-21/usr/sbin/munin22:20:170
141090499149,4cyclictest1451123-21chrt21:11:293
141090499149,3cyclictest1475943-21/usr/sbin/munin22:30:143
141090499148,4cyclictest753-21ntpd23:20:163
141090499146,5cyclictest1441023-21memory20:40:133
141090499145,7cyclictest1490692-21ntpq23:15:183
141090499145,5cyclictest1-21systemd23:50:013
141090499144,6cyclictest1508370-21ntpq00:10:173
1410904991412,1cyclictest794-21snmpd19:34:053
1410904991411,2cyclictest1503005-21/usr/sbin/munin23:55:133
141090499141,12cyclictest1486216-21chrt23:02:073
1410904991411,2cyclictest1426576-21/usr/sbin/munin19:55:193
1410904991410,3cyclictest642-21cron22:55:003
141090499140,4cyclictest1452910-21taskset21:19:113
141090499140,2cyclictest1454234-21seq21:20:453
141090499140,2cyclictest1454234-21seq21:20:443
141090499140,1cyclictest1480878-21needreboot22:45:173
141090499140,1cyclictest1429769-21/usr/sbin/munin20:05:203
141090499140,12cyclictest794-21snmpd21:43:103
141090499140,12cyclictest1423110-21/usr/sbin/munin19:45:153
141090499140,12cyclictest1415296-21strings19:20:153
141090299149,4cyclictest1430365-21kworker/u16:0+events_unbound20:30:192
141090299148,3cyclictest795-21runrttasks19:45:112
141090299147,4cyclictest1421145-21apt-get19:40:112
141090299146,5cyclictest1489472-21cat23:10:202
1410902991412,1cyclictest0-21swapper/221:40:182
1410902991412,1cyclictest0-21swapper/220:40:172
1410902991411,2cyclictest381-21systemd-udevd22:08:492
141090299140,1cyclictest0-21swapper/223:55:182
141089899146,5cyclictest661-21rs:main1
141089899145,6cyclictest1516220-21ntpq00:35:161
141089899141,9cyclictest1435624-21apt-get20:25:111
1410898991412,1cyclictest1462752-21idleruntime-cro21:50:011
1410898991412,1cyclictest1418910-21taskset19:30:221
1410898991412,1cyclictest0-21swapper/120:00:151
1410898991412,1cyclictest0-21swapper/119:15:001
141089899141,1cyclictest1445864-21/usr/sbin/munin20:55:161
1410898991410,2cyclictest1485700-21ntpq23:00:181
1410898991410,2cyclictest0-21swapper/120:15:011
141089899140,1cyclictest1447407-21/usr/sbin/munin21:00:161
141089899140,13cyclictest1425462-21chrt19:53:201
141089899140,13cyclictest0-21swapper/100:25:101
141089899140,12cyclictest1471883-21taskset22:15:561
141089499147,5cyclictest1489220-21awk23:10:190
141089499147,5cyclictest1484051-21ld22:55:170
141089499146,5cyclictest355-21systemd-journal23:01:290
141089499141,9cyclictest1499362-21munin-run23:45:000
141089499141,9cyclictest1497775-21awk23:40:000
141089499141,9cyclictest1477827-21nvmesmart_nvme022:35:190
1410894991412,1cyclictest1417573-21taskset19:28:250
1410894991411,2cyclictest1475632-21/usr/sbin/munin22:30:190
1410894991410,3cyclictest1417945-21taskset19:30:070
141089499140,1cyclictest1463997-21taskset21:50:580
141089499140,12cyclictest1481952-21/usr/sbin/munin22:50:160
141090499138,3cyclictest795-21runrttasks21:25:583
141090499138,3cyclictest1436601-21taskset20:25:543
141090499136,4cyclictest1442760-21ld20:45:183
141090499135,5cyclictest1495023-21expr23:30:133
141090499131,8cyclictest1499365-21perf23:44:593
1410904991310,2cyclictest1478804-21/usr/sbin/munin22:40:143
141090499130,12cyclictest1489519-21taskset23:10:463
141090499130,11cyclictest1493491-21df23:25:133
141090299137,4cyclictest1418768-21find19:30:202
141090299135,5cyclictest1493884-21cat23:25:182
141089899138,4cyclictest1434263-21/usr/sbin/munin20:20:131
1410898991310,2cyclictest1451886-21/usr/sbin/munin21:15:131
1410898991310,2cyclictest1420542-21chrt19:36:061
141089499139,2cyclictest0-21swapper/021:05:000
141089499131,8cyclictest1505247-21cut00:00:180
141089499131,8cyclictest1460463-21grep21:40:170
141089499131,8cyclictest1444006-21hddtemp_smartct20:50:130
1410894991312,1cyclictest0-21swapper/020:30:380
1410894991311,1cyclictest0-21swapper/000:10:100
141089499130,12cyclictest0-21swapper/019:56:570
141089499130,11cyclictest795-21runrttasks21:39:420
141090499126,4cyclictest645-21dbus-daemon21:30:023
141090499121,8cyclictest1447184-21smartctl21:00:153
141090299120,10cyclictest1472725-21smartctl22:20:132
141089499129,2cyclictest1470880-21/usr/sbin/munin22:15:110
141089499128,2cyclictest1494839-21taskset23:30:020
141089499121,8cyclictest1442210-21apt-get20:45:130
141089499120,12cyclictest0-21swapper/019:45:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional