You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-25 - 16:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Mon May 25, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1399949994744,2cyclictest1550821-21kworker/u8:2+events_unbound09:40:140
1399949994643,2cyclictest1608327-21kworker/u8:1+flush-259:010:35:240
1399949994642,3cyclictest1676698-21kworker/u8:3+events_unbound11:35:230
1399949994638,2cyclictest1676698-21kworker/u8:3+events_unbound11:50:200
1399951994439,4cyclictest1659644-21ntpq11:15:201
1399949994341,2cyclictest1514713-21kworker/u8:1+flush-259:009:25:240
1399949994340,2cyclictest1535344-21kworker/u8:0+events_unbound11:20:240
1399949994239,2cyclictest1379374-21kworker/u8:3+events_unbound07:20:160
1399949994138,2cyclictest1707526-21kworker/u8:2+events_unbound12:35:230
1399949994138,2cyclictest1639519-21kworker/u8:2+flush-259:011:30:220
1399949994138,2cyclictest1582033-21kworker/u8:3+events_unbound10:40:230
1399949994138,2cyclictest1582033-21kworker/u8:3+events_unbound10:10:200
1399949994138,2cyclictest1379374-21kworker/u8:3+events_unbound07:30:220
1399949994037,2cyclictest1550821-21kworker/u8:2+flush-259:009:55:230
1399949994037,2cyclictest1514740-21kworker/u8:4+flush-259:009:05:240
1399959993935,3cyclictest1582033-21kworker/u8:3+flush-259:010:10:213
1399959993934,3cyclictest1592438-21apt-get10:15:103
1399949993936,2cyclictest1728138-21kworker/u8:3+events_unbound12:30:230
1399949993935,3cyclictest1676698-21kworker/u8:3+events_unbound12:00:230
1399949993935,3cyclictest1495310-21kworker/u8:3+events_unbound08:50:140
1399949993933,4cyclictest1712667-21latency_hist12:10:010
1399959993832,4cyclictest1581800-21latency_hist10:05:013
1399954993834,3cyclictest1634573-21/usr/sbin/munin10:55:222
1399959993732,3cyclictest1628956-21kworker/u8:1+events_unbound11:05:193
1399949993734,2cyclictest1535344-21kworker/u8:0+flush-259:009:30:230
1399959993632,3cyclictest1535344-21kworker/u8:0+flush-259:009:50:013
1399951993632,3cyclictest1723217-21apt-get12:20:121
1399951993631,3cyclictest1473553-21cat08:20:011
1399949993633,2cyclictest1707526-21kworker/u8:2+events_unbound12:20:220
1399949993633,2cyclictest1535344-21kworker/u8:0+events_unbound10:20:230
1399949993632,3cyclictest1550821-21kworker/u8:2+events_unbound09:35:210
1399959993529,4cyclictest1426157-21latency_hist07:35:023
1399954993530,3cyclictest1478623-21latency_hist08:25:012
1399949993531,3cyclictest1390229-21kworker/u8:0+events_unbound07:11:280
1399949993529,4cyclictest1479053-21diskstats08:25:140
1399959993430,3cyclictest1550821-21kworker/u8:2+events_unbound10:05:243
1399949993431,2cyclictest1462965-21kworker/u8:2+events_unbound08:10:150
1399959993330,2cyclictest1639519-21kworker/u8:2+events_unbound11:45:193
1399959993329,3cyclictest1676698-21kworker/u8:3+flush-259:012:00:213
1399959993329,3cyclictest1655598-21kworker/u8:4+flush-259:011:35:003
1399949993330,2cyclictest1728138-21kworker/u8:3+events_unbound12:25:160
1399949993330,2cyclictest1707526-21kworker/u8:2+events_unbound12:15:240
1399959993228,3cyclictest1514713-21kworker/u8:1+flush-259:009:25:203
1399959993227,3cyclictest1379374-21kworker/u8:3+flush-259:007:20:193
1399949993129,2cyclictest1639519-21kworker/u8:2+flush-259:011:15:230
1399949993129,2cyclictest1446838-21kworker/u8:1+events_unbound08:30:230
1399949993129,2cyclictest1436573-21kworker/u8:2+flush-259:008:00:140
1399949993128,2cyclictest1582033-21kworker/u8:3+events_unbound10:45:250
1399949993128,2cyclictest1514740-21kworker/u8:4+events_unbound09:15:260
1399949993128,2cyclictest1514740-21kworker/u8:4+events_unbound09:00:150
1399949993127,3cyclictest1639519-21kworker/u8:2+events_unbound11:45:110
1399949993127,3cyclictest1535344-21kworker/u8:0+events_unbound11:40:210
1399959993026,3cyclictest1514713-21kworker/u8:1+flush-259:009:20:163
1399949993028,2cyclictest1535344-21kworker/u8:0+events_unbound09:45:250
1399949993028,2cyclictest1495310-21kworker/u8:3+events_unbound09:10:240
1399949993027,2cyclictest1550821-21kworker/u8:2+flush-259:010:25:250
1399949993027,2cyclictest1421020-21kworker/u8:1+events_unbound07:40:240
1399949993026,3cyclictest1535344-21kworker/u8:0+flush-259:010:15:250
1399959992925,3cyclictest1628956-21kworker/u8:1+flush-259:011:00:153
1399959992925,3cyclictest1628956-21kworker/u8:1+flush-259:011:00:153
1399959992925,3cyclictest1535344-21kworker/u8:0+flush-259:011:10:433
1399959992925,3cyclictest1535344-21kworker/u8:0+flush-259:011:10:433
1399949992925,3cyclictest1535344-21kworker/u8:0+events_unbound11:27:310
1399949992925,3cyclictest1535344-21kworker/u8:0+events_unbound11:00:200
1399949992925,3cyclictest1535344-21kworker/u8:0+events_unbound11:00:200
1399949992925,3cyclictest1436573-21kworker/u8:2+flush-259:007:54:500
1399949992925,3cyclictest1390229-21kworker/u8:0+events_unbound07:15:120
1399959992824,3cyclictest1495310-21kworker/u8:3+events_unbound09:10:283
1399954992822,4cyclictest335557-21systemd-journal09:35:142
1399951992823,4cyclictest1441901-21/usr/sbin/munin07:50:211
1399949992825,2cyclictest1426841-21kworker/u8:0+events_unbound08:05:210
1399949992824,3cyclictest1649217-21kworker/u8:3+events_unbound11:10:270
1399949992824,3cyclictest1649217-21kworker/u8:3+events_unbound11:10:270
1399949992824,3cyclictest1446838-21kworker/u8:1+events_unbound08:36:300
1399959992724,2cyclictest1495310-21kworker/u8:3+flush-259:008:45:203
1399959992724,2cyclictest1426841-21kworker/u8:0+flush-259:007:50:253
1399959992723,3cyclictest1623848-21apt-get10:45:113
1399959992723,3cyclictest1426841-21kworker/u8:0+flush-259:008:05:153
1399959992722,4cyclictest1676698-21kworker/u8:3+events_unbound11:35:173
1399959992722,3cyclictest1535344-21kworker/u8:0+events_unbound10:40:213
1399954992724,2cyclictest1639519-21kworker/u8:2+flush-259:011:20:152
1399951992720,5cyclictest1431310-21latency_hist07:40:021
1399949992724,2cyclictest1446838-21kworker/u8:1+events_unbound08:23:300
1399949992723,3cyclictest1495310-21kworker/u8:3+events_unbound08:40:250
1399959992623,2cyclictest1608327-21kworker/u8:1+flush-259:010:35:263
1399959992623,2cyclictest1462965-21kworker/u8:2+flush-259:008:40:163
1399959992622,3cyclictest1436573-21kworker/u8:2+events_unbound07:45:243
1399959992622,3cyclictest1390229-21kworker/u8:0+flush-259:007:10:233
1399959992622,2cyclictest1535344-21kworker/u8:0+flush-259:009:40:233
1399959992620,4cyclictest1508647-21ntpq08:50:223
1399951992620,4cyclictest1415828-21latency_hist07:25:011
1399951992619,5cyclictest1570601-21ntpq09:50:211
1399949992623,2cyclictest1462965-21kworker/u8:2+events_unbound08:15:160
1399949992623,2cyclictest1436573-21kworker/u8:2+events_unbound07:55:110
1399949992623,2cyclictest1384782-21kworker/u8:4+events_unbound07:28:300
1399959992522,3cyclictest1639519-21kworker/u8:2+events_unbound11:50:023
1399959992522,2cyclictest1514713-21kworker/u8:1+events_unbound09:18:293
1399959992522,2cyclictest1462965-21kworker/u8:2+flush-259:008:15:003
1399959992521,3cyclictest1426841-21kworker/u8:0+events_unbound07:40:153
1399959992517,3cyclictest1535344-21kworker/u8:0+events_unbound09:30:193
1399949992522,2cyclictest1550821-21kworker/u8:2+events_unbound09:50:270
1399949992522,2cyclictest1514713-21kworker/u8:1+events_unbound09:20:300
1399949992521,3cyclictest1421020-21kworker/u8:1+events_unbound07:39:310
1399949992519,4cyclictest1613287-21latency_hist10:34:590
1399959992421,2cyclictest1728138-21kworker/u8:3+flush-259:012:30:203
1399959992420,3cyclictest1722069-21kworker/u8:0+events_unbound12:15:523
1399959992420,3cyclictest1379374-21kworker/u8:3+events_unbound07:25:213
1399954992420,3cyclictest1550821-21kworker/u8:2+flush-259:009:50:182
1399954992417,5cyclictest1601645-21ntpq10:20:202
1399951992418,4cyclictest335557-21systemd-journal07:30:011
1399949992421,2cyclictest1707526-21kworker/u8:2+events_unbound12:10:250
1399949992420,3cyclictest1676698-21kworker/u8:3+events_unbound11:56:290
1399949992419,4cyclictest1399889-21latency_hist07:10:010
1399959992320,2cyclictest1446838-21kworker/u8:1+flush-259:008:30:253
1399959992318,3cyclictest1390229-21kworker/u8:0+flush-259:007:15:223
1399954992318,4cyclictest335557-21systemd-journal08:19:282
1399954992318,4cyclictest1582033-21kworker/u8:3+events_unbound10:45:172
1399954992317,4cyclictest1618395-21latency_hist10:40:002
1399951992318,4cyclictest335557-21systemd-journal11:05:001
1399951992318,4cyclictest335557-21systemd-journal11:05:001
1399951992318,3cyclictest1514740-21kworker/u8:4+flush-259:009:00:161
1399949992320,2cyclictest1550821-21kworker/u8:2+events_unbound10:04:300
1399949992320,2cyclictest1488967-21kworker/u8:0+flush-259:008:55:090
1399959992219,2cyclictest1686837-21kworker/u8:1+events_unbound11:55:203
1399959992219,2cyclictest1582033-21kworker/u8:3+events_unbound10:30:223
1399959992219,2cyclictest1535344-21kworker/u8:0+flush-259:011:25:213
1399954992218,3cyclictest1488967-21kworker/u8:0+flush-259:008:55:012
1399951992218,3cyclictest1452230-21/usr/sbin/munin08:00:231
1399951992217,3cyclictest1405773-21apt-get07:15:111
1399959992118,2cyclictest1592402-21kworker/u8:4+events_unbound10:25:033
1399959992118,2cyclictest1514740-21kworker/u8:4+events_unbound09:05:163
1399959992118,2cyclictest1446838-21kworker/u8:1+flush-259:008:40:003
1399959992117,3cyclictest1686837-21kworker/u8:1+flush-259:012:40:003
1399959992117,3cyclictest1550821-21kworker/u8:2+events_unbound10:23:313
1399959992117,3cyclictest1535344-21kworker/u8:0+flush-259:011:25:003
1399959992117,3cyclictest1488967-21kworker/u8:0+events_unbound08:55:113
1399959992117,3cyclictest1421020-21kworker/u8:1+flush-259:007:39:443
139995199216,5cyclictest1482931-21ntpq08:25:221
139995199216,4cyclictest1466948-21ntpq08:10:201
1399949992118,2cyclictest1535344-21kworker/u8:0+events_unbound10:05:140
1399949992118,2cyclictest1504016-21kworker/u8:1+events_unbound08:46:310
1399959992018,2cyclictest1707717-21kworker/u8:4+events_unbound12:08:303
1399959992017,2cyclictest1582033-21kworker/u8:3+flush-259:010:50:153
1399959992017,2cyclictest1436573-21kworker/u8:2+events_unbound08:00:173
1399959992016,3cyclictest1550821-21kworker/u8:2+events_unbound09:52:303
1399959992016,2cyclictest0-21swapper/311:00:003
1399959992014,4cyclictest1728355-21/usr/sbin/munin12:25:203
1399959992013,3cyclictest1462965-21kworker/u8:2+events_unbound08:15:183
139995499204,4cyclictest1612383-21ld10:30:212
1399954992018,1cyclictest1644659-21sh11:05:002
1399954992018,1cyclictest1644659-21sh11:05:002
1399954992015,4cyclictest1462474-21cat08:10:012
139995199209,10cyclictest1623122-21sed10:40:241
139995199206,4cyclictest1534427-21ntpq09:15:231
1399951992016,3cyclictest1473031-21apt-get08:19:591
1399951992014,5cyclictest0-21swapper/109:10:001
1399949992016,3cyclictest1628956-21kworker/u8:1+events_unbound10:59:500
1399949992016,3cyclictest1426841-21kworker/u8:0+events_unbound07:49:410
1399959991916,2cyclictest1550821-21kworker/u8:2+events_unbound09:55:203
1399959991915,3cyclictest1707526-21kworker/u8:2+flush-259:012:10:233
1399959991914,3cyclictest335557-21systemd-journal11:40:383
139995499195,4cyclictest1486986-21cut08:30:172
1399954991916,2cyclictest1733277-21idleruntime-cro12:30:002
1399954991916,2cyclictest1634134-21gpgv10:55:002
1399954991916,2cyclictest1467859-21cron08:15:002
1399954991916,1cyclictest1677141-21irqcore11:35:152
1399954991915,3cyclictest1535566-21/usr/sbin/munin09:20:222
1399954991915,2cyclictest509-21rs:main2
1399954991913,4cyclictest1561156-21latency_hist09:45:012
139995199197,10cyclictest1671289-21awk11:30:001
139995199195,4cyclictest1717671-21seq12:13:111
1399951991917,1cyclictest0-21swapper/112:10:001
1399951991914,3cyclictest335557-21systemd-journal10:55:011
1399951991913,4cyclictest335557-21systemd-journal09:10:011
1399954991814,3cyclictest1436573-21kworker/u8:2+flush-259:007:55:222
1399954991814,2cyclictest1535344-21kworker/u8:0+flush-259:011:40:162
1399954991811,5cyclictest1535309-21awk09:20:002
139995199185,7cyclictest1612929-21seq10:30:461
139995199185,5cyclictest1680887-21nvmesmart_nvme011:35:211
139995199185,4cyclictest1504173-21chrt08:47:331
139995199184,5cyclictest1597116-21runrttasks10:17:191
1399949991814,3cyclictest1582033-21kworker/u8:3+events_unbound10:50:070
1399959991714,2cyclictest1707526-21kworker/u8:2+events_unbound12:20:163
1399959991714,2cyclictest1436573-21kworker/u8:2+events_unbound07:55:203
1399959991713,3cyclictest1514740-21kworker/u8:4+events_unbound09:00:023
1399959991713,3cyclictest1462965-21kworker/u8:2+flush-259:008:20:133
139995499174,4cyclictest1722905-21taskset12:19:362
139995499174,3cyclictest1572052-21fw_conntrack09:55:152
1399954991713,3cyclictest1462965-21kworker/u8:2+flush-259:008:25:182
1399954991711,5cyclictest1409795-21ntpq07:15:222
1399954991710,5cyclictest1581795-21idleruntime-cro10:05:012
139995199178,6cyclictest1535712-21tr09:20:151
139995199177,8cyclictest1707487-21latency_hist12:05:011
139995199175,3cyclictest1509221-21seq08:51:091
1399951991715,1cyclictest1556682-21irqcore09:40:161
1399951991713,3cyclictest1514694-21latency_hist09:00:001
1399951991711,4cyclictest1421240-21apt-get07:30:111
1399951991710,5cyclictest1581785-21cron10:05:011
1399959991613,2cyclictest1639519-21kworker/u8:2+events_unbound11:15:133
1399959991611,4cyclictest1535344-21kworker/u8:0+events_unbound09:35:123
1399959991611,4cyclictest1462965-21kworker/u8:2+events_unbound08:29:393
139995499169,5cyclictest335557-21systemd-journal09:50:012
139995499166,8cyclictest1525062-21latency_hist09:10:012
139995499166,8cyclictest1399889-21latency_hist07:10:002
139995499165,2cyclictest1445929-21ntpq07:50:222
139995499164,3cyclictest1513852-21ntpq08:55:222
1399954991614,1cyclictest1421020-21kworker/u8:1+flush-259:007:45:002
1399954991613,2cyclictest1697514-21cpuspeed_turbos11:55:122
139995199169,5cyclictest1484275-21grep08:30:141
139995199168,5cyclictest1462681-21cat08:10:011
139995199166,2cyclictest1540935-21diskstats09:25:131
139995199164,3cyclictest1655292-21chrt11:14:181
139995199164,3cyclictest1655292-21chrt11:14:181
139995199163,3cyclictest1738408-21runrttasks12:34:151
1399951991613,2cyclictest1639519-21kworker/u8:2+flush-259:011:05:101
1399951991612,3cyclictest1628989-21apt-get10:50:121
1399951991612,3cyclictest0-21swapper/111:25:001
1399951991612,2cyclictest0-21swapper/110:40:001
1399951991611,3cyclictest1550779-21idleruntime-cro09:35:001
1399951991610,5cyclictest1576638-21sh10:00:011
1399951991610,4cyclictest1698299-21cut11:55:161
1399951991610,4cyclictest1556000-21cut09:40:001
1399949991613,2cyclictest1649217-21kworker/u8:3+events_unbound11:06:350
139995499159,4cyclictest1738486-21sh12:35:002
139995499159,4cyclictest1676472-21basename11:35:002
139995499159,4cyclictest1645755-21irqcore11:05:182
139995499159,4cyclictest1597352-21munin-run10:20:002
139995499158,5cyclictest1670302-21needreboot11:25:182
139995499158,5cyclictest1623597-21basename10:45:002
139995499158,5cyclictest1587672-21ps10:10:162
139995499158,5cyclictest1545969-21tr09:30:112
139995499158,5cyclictest1494475-21tr08:40:122
139995499158,4cyclictest1515369-21irqcore09:00:172
139995499158,4cyclictest1421375-21cpuspeed_turbos07:30:122
139995499158,4cyclictest1-21systemd11:20:002
139995499155,6cyclictest1743251-21users12:35:252
139995499154,3cyclictest1529942-21taskset09:11:592
1399954991510,3cyclictest1707482-21sh12:05:012
1399954991510,3cyclictest1457136-21cron08:04:592
139995199159,5cyclictest1719118-21tr12:15:171
139995199159,4cyclictest1682098-21cpuspeed_turbos11:40:131
139995199159,4cyclictest1675484-21tail11:30:181
139995199159,3cyclictest1692408-21cpuspeed_turbos11:50:121
139995199158,5cyclictest1494745-21irqcore08:40:141
139995199157,5cyclictest0-21swapper/107:10:351
139995199156,6cyclictest1447307-21fschecks_time07:55:121
139995199156,5cyclictest1607232-21ntpq10:25:211
139995199155,2cyclictest1691140-21ntpq11:45:201
139995199155,2cyclictest1437729-21irqstats07:45:161
1399951991510,3cyclictest1628708-21cron10:50:001
1399951991510,3cyclictest1598031-21sort10:20:171
1399951991510,3cyclictest1588275-21tr10:10:171
1399951991510,3cyclictest1566352-21cron09:50:011
139995499149,3cyclictest1608094-21cron10:30:002
139995499148,5cyclictest1501804-21sed08:45:162
139995499148,4cyclictest1697183-21apt11:55:002
139995499148,4cyclictest1687194-21cut11:45:112
139995499148,4cyclictest1654391-21tail11:10:192
139995499148,4cyclictest1654391-21tail11:10:192
139995499148,4cyclictest1545606-21cron09:29:592
139995499148,4cyclictest1436862-21cpuspeed_turbos07:45:122
139995499148,4cyclictest1420988-21idleruntime-cro07:30:012
139995499147,3cyclictest1583094-21grep10:05:182
139995499145,4cyclictest1430407-21ntp_kernel_pll_07:35:202
139995499144,3cyclictest1404900-21seq07:13:002
139995499144,2cyclictest1711791-21ntpq12:05:202
139995499144,2cyclictest1411216-21fw_conntrack07:20:142
1399954991410,3cyclictest335557-21systemd-journal12:24:552
139995199149,3cyclictest1582485-21grep10:05:161
139995199148,4cyclictest1431832-21fschecks_time07:40:141
139995199148,3cyclictest1738841-21cpuspeed_turbos12:35:131
139995199147,5cyclictest1729463-21kthreadcore12:25:171
139995199147,4cyclictest1489294-21grep08:35:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional