You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-28 - 01:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Sat Jun 27, 2026 12:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
157081023440,3sleep375-21kworker/3:1+events08:00:243
151689021800,5sleep3451irq_work/307:05:333
16848452980,1sleep11684851-21tr09:55:221
16319312910,2chrt0-21swapper/009:00:230
1517706996459,4cyclictest1658138-21kworker/u8:1+events_unbound10:15:331
1517700995950,6cyclictest1668452-21latency_hist09:40:000
1517700995547,6cyclictest1629042-21kworker/u8:0+flush-8:009:10:320
1517706995347,5cyclictest1718705-21kworker/u8:2+flush-8:011:00:301
1517706995247,4cyclictest1742441-21kworker/u8:0+flush-8:011:05:351
1517706995247,4cyclictest1658138-21kworker/u8:1+flush-8:010:10:301
1517706995238,11cyclictest1704890-21systemd-journal10:06:271
1517706995044,4cyclictest1466061-21kworker/u8:2+flush-8:007:25:261
1517706995043,5cyclictest1658138-21kworker/u8:1+flush-8:010:35:181
1517708994938,9cyclictest1704890-21systemd-journal12:15:272
1517706994943,4cyclictest1629042-21kworker/u8:0+flush-8:009:15:311
1517706994943,4cyclictest1584000-21latency_hist08:15:011
1517706994943,4cyclictest1584000-21latency_hist08:15:001
1517706994938,8cyclictest1704890-21systemd-journal09:33:271
1517700994940,7cyclictest1847971-21cp12:40:000
1517706994844,3cyclictest1475054-21kworker/u8:3+events_unbound08:15:341
1517706994844,3cyclictest1475054-21kworker/u8:3+events_unbound08:15:341
1517706994841,6cyclictest1718705-21kworker/u8:2+flush-8:010:55:321
1517706994841,4cyclictest1658138-21kworker/u8:1+events_unbound10:50:231
1517700994839,6cyclictest1797258-21ntpq11:45:260
1517708994743,3cyclictest1546112-21kworker/u8:0+flush-8:007:40:312
1517706994741,5cyclictest1792373-21kworker/u8:0+events_unbound12:20:331
1517706994741,4cyclictest1792261-21ntpq11:40:271
1517706994741,4cyclictest1748473-21kworker/u8:3+events_unbound12:00:341
1517706994732,10cyclictest1704890-21systemd-journal07:11:251
1517708994637,7cyclictest1579051-21/usr/sbin/munin08:10:002
1517706994640,5cyclictest1718705-21kworker/u8:2+events_unbound10:30:301
1517706994638,5cyclictest1629042-21kworker/u8:0+events_unbound09:10:291
1517706994637,7cyclictest1843139-21/usr/sbin/munin12:35:281
1517706994637,6cyclictest1613069-21ntpq08:40:291
1517700994637,6cyclictest1738361-21latency_hist10:50:010
1517708994538,5cyclictest1792373-21kworker/u8:0+flush-8:012:00:322
1517706994538,5cyclictest1629042-21kworker/u8:0+flush-8:009:05:291
1517706994536,7cyclictest1658138-21kworker/u8:1+flush-8:009:45:291
1517706994530,11cyclictest1704890-21systemd-journal08:00:001
1517700994537,6cyclictest1683793-21/usr/sbin/munin09:55:330
1517700994337,4cyclictest1723662-21apt-get10:35:140
1517700994335,5cyclictest1763140-21latency_hist11:15:010
1517700994335,5cyclictest1763140-21latency_hist11:15:000
1517706994235,5cyclictest1522562-21kworker/u8:1+flush-8:007:40:341
1517706994235,5cyclictest1475054-21kworker/u8:3+flush-8:010:25:321
1517700994235,5cyclictest1466061-21kworker/u8:2+flush-8:007:30:290
1517700994234,6cyclictest1475054-21kworker/u8:3+flush-8:009:40:310
1517700994234,5cyclictest1704890-21systemd-journal12:35:010
1517706994135,4cyclictest1837260-21kworker/u8:3+events_unbound12:30:311
1517706994134,5cyclictest1658138-21kworker/u8:1+events_unbound09:50:171
1517700994137,3cyclictest1792373-21kworker/u8:0+flush-8:011:55:320
1517708994035,4cyclictest1475054-21kworker/u8:3+flush-8:007:20:012
1517706994035,3cyclictest1466061-21kworker/u8:2+events_unbound07:15:321
1517706994032,6cyclictest1599077-21kworker/u8:2+flush-8:008:55:311
1517700994033,4cyclictest1658138-21kworker/u8:1+flush-8:010:20:310
1517706993934,4cyclictest1748473-21kworker/u8:3+flush-8:011:45:311
15173762394,6sleep10-21swapper/107:08:131
1517706993832,4cyclictest1792373-21kworker/u8:0+events_unbound12:15:281
1517706993829,6cyclictest1704890-21systemd-journal08:39:261
1517700993826,9cyclictest1704890-21systemd-journal12:25:260
1517708993731,4cyclictest1792373-21kworker/u8:0+flush-8:012:10:312
1517706993731,4cyclictest1704890-21systemd-journal12:13:271
1517706993730,5cyclictest1522562-21kworker/u8:1+events_unbound08:30:271
1517700993733,3cyclictest1778228-21apt-key11:30:010
1517700993733,3cyclictest1778228-21apt-key11:30:010
1517700993723,4cyclictest1649236-21fschecks_time09:20:190
1517706993631,3cyclictest1748473-21kworker/u8:3+events_unbound11:50:271
1517706993631,3cyclictest1475054-21kworker/u8:3+events_unbound09:20:311
1517706993630,4cyclictest1475054-21kworker/u8:3+flush-8:010:00:291
1517700993631,3cyclictest1466061-21kworker/u8:2+events_unbound07:15:020
1517700993628,6cyclictest1704890-21systemd-journal08:05:000
1517700993621,13cyclictest1579662-21fschecks_time08:10:180
1517700993621,13cyclictest1579662-21fschecks_time08:10:180
1517708993525,8cyclictest1704890-21systemd-journal09:15:012
1517706993530,4cyclictest1522562-21kworker/u8:1+events_unbound07:45:291
1517706993524,8cyclictest1704890-21systemd-journal11:40:001
1517700993530,4cyclictest1792373-21kworker/u8:0+flush-8:012:20:240
1517700993522,11cyclictest1689120-21fschecks_time10:00:190
1517700993521,12cyclictest1743886-21fschecks_time10:55:190
1517700993520,13cyclictest1719055-21fschecks_time10:30:210
1517708993427,5cyclictest1704890-21systemd-journal08:40:012
1517708993423,8cyclictest1592980-21ntpq08:20:282
1517708993423,8cyclictest1592980-21ntpq08:20:272
1517706993428,5cyclictest1629042-21kworker/u8:0+events_unbound09:25:291
1517706993428,5cyclictest1475054-21kworker/u8:3+events_unbound09:40:291
1517706993426,5cyclictest1704890-21systemd-journal08:21:251
1517706993426,5cyclictest1704890-21systemd-journal08:21:251
1517706993421,9cyclictest1704890-21systemd-journal10:44:271
1517700993430,3cyclictest1475054-21kworker/u8:3+flush-8:008:15:320
1517700993430,3cyclictest1475054-21kworker/u8:3+flush-8:008:15:320
1517700993427,5cyclictest1569079-21cron07:59:590
1517700993424,8cyclictest1614568-21fschecks_time08:45:190
15174042344,9sleep20-21swapper/207:08:352
1517706993328,3cyclictest1812228-21kworker/u8:1+flush-8:012:25:231
1517706993327,4cyclictest1475054-21kworker/u8:3+events_unbound08:00:301
1517706993317,11cyclictest1704890-21systemd-journal12:09:251
1517700993327,4cyclictest1718438-21turbostat.cron10:29:590
1517700993326,5cyclictest1822997-21latency_hist12:15:020
1517700993326,5cyclictest1658138-21kworker/u8:1+flush-8:011:35:360
1517700993227,3cyclictest1718705-21kworker/u8:2+flush-8:010:52:580
1517700993225,5cyclictest1648588-21cron09:20:000
1517700993224,6cyclictest1704890-21systemd-journal12:05:570
15165502323,22sleep01516598-21ntpq07:05:260
1517708993127,2cyclictest1788243-21cron11:39:592
1517708993123,6cyclictest1704890-21systemd-journal11:50:532
1517706993127,3cyclictest1466061-21kworker/u8:2+flush-8:007:20:211
1517706993125,4cyclictest1729152-21kworker/u8:3+events_unbound10:45:201
1517706993125,4cyclictest1-21systemd08:55:011
1517706993124,6cyclictest1718705-21kworker/u8:2+events_unbound11:10:321
1517706993124,6cyclictest1718705-21kworker/u8:2+events_unbound11:10:311
1517706993124,5cyclictest1718705-21kworker/u8:2+flush-8:011:15:341
1517706993124,5cyclictest1718705-21kworker/u8:2+flush-8:011:15:331
1517700993124,5cyclictest1629042-21kworker/u8:0+flush-8:009:05:340
1517708993025,4cyclictest1522562-21kworker/u8:1+flush-8:008:50:342
1517706993024,4cyclictest1466061-21kworker/u8:2+flush-8:007:30:291
1517700993025,3cyclictest1699126-21cpuspeed_turbos10:10:170
1517700993024,4cyclictest1599643-21fschecks_time08:30:190
1517700993022,6cyclictest1704890-21systemd-journal11:45:010
1517700993022,6cyclictest1658138-21kworker/u8:1+flush-8:009:45:280
1517700993022,6cyclictest1609000-21cron08:40:010
1517708992924,4cyclictest1748473-21kworker/u8:3+flush-8:011:55:282
1517708992924,4cyclictest1629042-21kworker/u8:0+flush-8:009:25:342
1517708992924,3cyclictest1558754-21unixbench_multi07:45:342
1517708992920,7cyclictest1704890-21systemd-journal08:15:002
1517708992920,7cyclictest1704890-21systemd-journal08:14:592
1517706992924,4cyclictest1658138-21kworker/u8:1+flush-8:011:20:181
1517706992924,3cyclictest1658138-21kworker/u8:1+flush-8:011:25:261
1517706992924,3cyclictest1658138-21kworker/u8:1+flush-8:011:25:251
1517706992912,12cyclictest1704890-21systemd-journal08:06:271
1517700992924,3cyclictest1808897-21irqcore12:00:210
1517700992923,4cyclictest1559813-21fschecks_time07:50:190
1517700992923,4cyclictest1466061-21kworker/u8:2+flush-8:007:20:300
1517700992921,6cyclictest1704890-21systemd-journal07:44:590
1517700992918,9cyclictest1704890-21systemd-journal08:25:000
1517700992918,9cyclictest1704890-21systemd-journal08:24:590
1517706992825,2cyclictest1549306-21systemd-run07:40:011
1517700992824,3cyclictest1770592-21kthreadcore11:20:220
1517700992823,4cyclictest1546112-21kworker/u8:0+flush-8:007:35:310
1517700992823,3cyclictest1628798-21munin-run09:00:010
1517700992823,3cyclictest1623451-21unixbench_singl08:50:350
1517700992822,4cyclictest1679150-21fschecks_count09:50:190
151770099281,25cyclictest1658202-21sshd09:26:260
1517708992723,3cyclictest1778897-21fschecks_time11:30:202
1517708992723,3cyclictest1778897-21fschecks_time11:30:202
1517708992723,2cyclictest1733908-21fschecks_count10:45:182
1517708992722,4cyclictest1748473-21kworker/u8:3+events_unbound11:15:302
1517708992722,4cyclictest1748473-21kworker/u8:3+events_unbound11:15:292
1517708992722,4cyclictest1677820-21kworker/u8:0+flush-8:010:05:342
1517708992722,3cyclictest1534731-21aten2_r5power_p07:25:172
1517708992721,4cyclictest1847863-21systemctl12:40:012
1517706992723,3cyclictest1677820-21kworker/u8:0+events_unbound10:20:321
1517706992723,2cyclictest1632046-21kthreadcore09:00:231
1517706992722,3cyclictest1668447-21apt09:39:591
1517700992723,2cyclictest1613010-21needreboot08:40:270
1517700992721,4cyclictest1827016-21needreboot12:15:260
1517700992719,6cyclictest1598113-21ntpq08:25:270
1517700992719,6cyclictest1598113-21ntpq08:25:260
1517708992622,3cyclictest1762706-21unixbench_multi11:10:362
1517708992622,3cyclictest1762706-21unixbench_multi11:10:362
1517708992621,4cyclictest1658138-21kworker/u8:1+flush-8:011:20:362
1517708992621,3cyclictest1833506-21fschecks_count12:25:162
1517708992621,3cyclictest1743754-21cpuspeed_turbos10:55:182
1517708992619,5cyclictest1570301-21sort08:00:222
1517708992616,8cyclictest1618924-21/usr/sbin/munin08:50:012
1517706992621,3cyclictest0-21swapper/107:50:211
1517706992620,4cyclictest1614253-21apt-get08:45:141
1517700992621,3cyclictest1753737-21fschecks_count11:05:190
1517700992620,4cyclictest1748473-21kworker/u8:3+flush-8:011:50:300
1517700992619,5cyclictest1538354-21needreboot07:25:250
1517708992521,2cyclictest1813591-21fschecks_count12:05:182
1517708992521,2cyclictest1673559-21latency_hist09:45:002
1517700992519,4cyclictest1658138-21kworker/u8:1+events_unbound10:15:280
1517708992420,2cyclictest1693775-21latency_hist10:05:012
1517708992419,3cyclictest1688493-21sendmail09:59:592
1517706992420,2cyclictest1808080-21sh12:00:011
1517700992420,2cyclictest0-21swapper/007:45:240
1517700992419,3cyclictest1778589-21runrttasks11:30:170
1517700992419,3cyclictest1778589-21runrttasks11:30:160
1517700992418,4cyclictest1752754-21unixbench_multi11:00:340
1517708992319,3cyclictest1718705-21kworker/u8:2+flush-8:010:40:212
1517708992319,2cyclictest1828520-21cpuspeed_turbos12:20:182
1517708992319,2cyclictest1709124-21fschecks_count10:20:192
1517706992318,3cyclictest0-21swapper/108:25:231
1517706992318,3cyclictest0-21swapper/108:25:231
1517700992316,5cyclictest0-21swapper/010:05:330
151770899222,14cyclictest1648885-21apt-get09:20:152
1517708992219,2cyclictest1588494-21unixbench_multi08:15:352
1517708992219,2cyclictest1588494-21unixbench_multi08:15:342
1517708992218,2cyclictest1797784-21unixbench_singl11:45:352
1517708992217,4cyclictest1522562-21kworker/u8:1+flush-8:008:25:192
1517708992217,4cyclictest1522562-21kworker/u8:1+flush-8:008:25:192
1517700992217,4cyclictest1519147-21kthreadcore07:10:220
1517700992216,4cyclictest1732087-21kthreadcore10:40:240
1517700992216,4cyclictest1660971-21kthreadcore09:30:250
1517708992118,2cyclictest1559729-21fschecks_count07:50:182
1517708992117,2cyclictest1644148-21cpuspeed_turbos09:15:192
1517708992116,4cyclictest1708545-21idleruntime-cro10:19:592
1517708992115,4cyclictest0-21swapper/211:30:012
1517708992115,4cyclictest0-21swapper/211:30:012
1517706992117,2cyclictest0-21swapper/111:30:351
1517706992117,2cyclictest0-21swapper/111:30:341
1517700992117,3cyclictest1522562-21kworker/u8:1+flush-8:008:05:190
1517700992116,4cyclictest1767697-21unixbench_multi11:15:360
1517700992116,4cyclictest1767697-21unixbench_multi11:15:360
16239922200,3chrt75-21kworker/3:1-events08:55:013
151770899209,3cyclictest0-21swapper/208:30:242
151770899207,3cyclictest16117542cut08:40:262
151770899203,12cyclictest1634009-21apt-get09:05:152
1517708992017,2cyclictest1569082-21sh08:00:002
1517708992016,2cyclictest0-21swapper/211:05:012
1517708992015,4cyclictest1522562-21kworker/u8:1+flush-8:007:35:192
151770899198,4cyclictest0-21swapper/208:56:182
151770899195,4cyclictest0-21swapper/207:12:232
151770899192,12cyclictest1667568-21ntpq09:35:282
1517708991814,2cyclictest0-21swapper/211:10:002
1517708991810,6cyclictest1741477-21cut10:50:242
151770899178,7cyclictest1633101-21grep09:00:332
151770899176,8cyclictest1791404-21tr11:40:252
151770899172,4cyclictest0-21swapper/209:30:262
1517708991712,4cyclictest0-21swapper/210:30:232
151770899162,10cyclictest1539604-21apt-get07:30:152
1517708991613,2cyclictest0-21swapper/209:52:112
1517708991612,2cyclictest1839035-21irqrtprio12:30:232
1517708991612,2cyclictest0-21swapper/210:40:002
1517708991612,2cyclictest0-21swapper/207:20:172
151770899161,11cyclictest1713768-21apt-get10:25:142
16448562155,6sleep30-21swapper/309:15:253
151770899157,6cyclictest1699499-21sed10:10:202
151770899150,3cyclictest16766161kthreadcore09:45:242
18181182132,6sleep30-21swapper/312:10:013
18151022132,7sleep30-21swapper/312:05:253
18031332132,6sleep30-21swapper/311:55:023
17882412132,7sleep30-21swapper/311:40:003
17691372132,7sleep30-21swapper/311:20:223
17274302132,7sleep30-21swapper/310:35:263
16836962132,6sleep30-21swapper/309:55:023
16776402132,7sleep30-21swapper/309:45:263
16607762133,6sleep30-21swapper/309:30:263
16540922132,8sleep30-21swapper/309:25:193
16389442132,7sleep30-21swapper/309:10:153
18469502122,6sleep30-21swapper/312:35:313
18370182122,6sleep30-21swapper/312:25:273
18309722122,6sleep30-21swapper/312:20:263
18070972123,6sleep30-21swapper/311:55:253
17932362122,6sleep30-21swapper/311:45:023
17677022122,6sleep30-21swapper/311:15:363
17677022122,6sleep30-21swapper/311:15:363
17433862122,6sleep30-21swapper/310:55:023
17386252122,6sleep30-21swapper/310:50:153
17184632122,5sleep30-21swapper/310:30:013
17176762122,6sleep30-21swapper/310:25:323
16991482122,6sleep30-21swapper/310:10:193
16973052122,6sleep30-21swapper/310:05:263
16887742122,6sleep30-21swapper/310:00:153
16726822122,6sleep30-21swapper/309:40:323
16637392122,6sleep30-21swapper/309:35:163
16497682122,6sleep30-21swapper/309:20:233
16337582122,5sleep30-21swapper/309:05:023
16294922122,6sleep30-21swapper/309:00:203
16239922122,6sleep30-21swapper/308:55:033
16148042122,6sleep30-21swapper/308:45:233
16138522122,6sleep30-21swapper/308:43:273
15998122122,6sleep30-21swapper/308:30:223
15943972122,6sleep30-21swapper/308:25:173
15943972122,6sleep30-21swapper/308:25:163
15881902122,6sleep30-21swapper/308:15:343
15881902122,6sleep30-21swapper/308:15:333
15805332122,6sleep30-21swapper/308:10:243
15805332122,6sleep30-21swapper/308:10:243
15751882120,6sleep30-21swapper/308:05:203
15640942122,6sleep30-21swapper/307:54:263
15499722122,6sleep30-21swapper/307:40:223
15395612122,5sleep30-21swapper/307:30:023
15346622122,6sleep30-21swapper/307:25:173
15299392122,6sleep30-21swapper/307:20:203
15217572122,6sleep30-21swapper/307:10:323
18385762112,5sleep30-21swapper/312:30:203
18236062112,6sleep30-21swapper/312:15:203
18127482112,5sleep30-21swapper/312:01:283
17885442112,5sleep30-21swapper/311:40:173
17788922112,5sleep30-21swapper/311:30:213
17788922112,5sleep30-21swapper/311:30:213
17734912112,5sleep30-21swapper/311:25:203
17734912112,5sleep30-21swapper/311:25:193
17630562112,6sleep30-21swapper/311:14:043
17630562112,6sleep30-21swapper/311:14:043
17531962112,5sleep30-21swapper/311:05:003
17484862112,5sleep30-21swapper/311:00:163
17336032112,5sleep30-21swapper/310:45:033
17286662112,5sleep30-21swapper/310:40:173
17088072112,6sleep30-21swapper/310:20:143
17043072112,5sleep30-21swapper/310:15:203
16827252112,5sleep30-21swapper/309:50:333
16081422112,6sleep30-21swapper/308:35:313
15894722112,5sleep30-21swapper/308:20:193
15894722112,5sleep30-21swapper/308:20:183
15686382112,5sleep30-21swapper/307:55:353
15583692112,5sleep30-21swapper/307:45:313
15445662112,5sleep30-21swapper/307:35:163
15284442112,6sleep30-21swapper/307:15:293
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional