You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-14 - 22:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sat Mar 14, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1515255994233,2cyclictest1826339-21kworker/u8:4+events_unbound12:25:212
1515255994037,2cyclictest1455979-21kworker/u8:3+events_unbound07:10:152
1515255993934,3cyclictest1515177-21latency_hist07:10:002
1515255993835,2cyclictest1826339-21kworker/u8:4+flush-259:012:10:212
1515255993835,2cyclictest1659863-21kworker/u8:2+events_unbound10:00:002
1515255993832,4cyclictest1732883-21latency_hist10:40:012
1515250993832,3cyclictest1805114-21latency_hist11:50:000
1515255993734,2cyclictest1644359-21kworker/u8:0+flush-259:009:20:232
1515255993732,3cyclictest1546104-21apt-get07:40:022
1515255993633,2cyclictest1659863-21kworker/u8:2+events_unbound10:25:142
1515255993532,2cyclictest1711661-21kworker/u8:1+events_unbound10:55:182
1515255993532,2cyclictest1520383-21kworker/u8:0+events_unbound07:20:182
1515255993431,3cyclictest1711661-21kworker/u8:1+flush-259:010:45:162
1515255993431,2cyclictest1789695-21kworker/u8:2+events_unbound11:40:192
1515255993431,2cyclictest1618552-21kworker/u8:4+flush-259:009:05:192
1515250993430,3cyclictest1618552-21kworker/u8:4+flush-259:009:05:240
1515255993330,2cyclictest1799992-21kworker/u8:3+events_unbound11:50:162
1515255993329,3cyclictest1530645-21kworker/u8:2+events_unbound07:30:232
1515255993325,5cyclictest1717052-21latency_hist10:25:012
1515250993329,3cyclictest1530645-21kworker/u8:2+flush-259:007:30:000
1515255993228,3cyclictest1799992-21kworker/u8:3+flush-259:012:05:162
1515252993227,4cyclictest1711661-21kworker/u8:1+flush-259:011:30:151
1515255993128,2cyclictest1613388-21kworker/u8:2+events_unbound08:55:192
1515255993127,3cyclictest1815401-21kworker/u8:0+events_unbound12:15:132
1515255993127,3cyclictest1711661-21kworker/u8:1+events_unbound11:25:172
1515250993126,4cyclictest1764167-21apt-get11:10:110
1515255993028,2cyclictest1799992-21kworker/u8:3+events_unbound11:55:122
1515255993028,2cyclictest1799992-21kworker/u8:3+events_unbound11:55:112
1515255993028,2cyclictest1732906-21kworker/u8:3+events_unbound11:15:232
1515255993027,2cyclictest1520383-21kworker/u8:0+events_unbound08:15:002
1515255993027,2cyclictest1520383-21kworker/u8:0+events_unbound07:25:192
1515252993026,3cyclictest1795073-21apt-get11:40:121
1515255992925,3cyclictest1540983-21kworker/u8:3+events_unbound07:45:142
1515252992924,4cyclictest1525744-21/usr/sbin/munin07:20:141
1515252992923,4cyclictest1712148-21/usr/sbin/munin10:20:141
1515255992825,2cyclictest1644359-21kworker/u8:0+events_unbound10:15:202
1515255992824,3cyclictest1675779-21kworker/u8:3+events_unbound10:05:182
1515255992824,3cyclictest1644359-21kworker/u8:0+events_unbound09:40:152
1515255992824,3cyclictest1540983-21kworker/u8:3+events_unbound07:50:142
1515252992821,5cyclictest1727770-21latency_hist10:35:011
1515250992821,5cyclictest1613366-21latency_hist08:45:010
1515260992723,2cyclictest1659863-21kworker/u8:2+flush-259:010:35:003
1515255992724,2cyclictest1711661-21kworker/u8:1+flush-259:011:10:222
1515255992724,2cyclictest1572259-21kworker/u8:1+events_unbound08:15:222
1515250992723,3cyclictest1815649-21apt-get12:00:130
1515250992722,3cyclictest1593058-21apt-get08:25:110
1515255992623,2cyclictest1618552-21kworker/u8:4+events_unbound09:10:222
1515252992620,5cyclictest1658908-21ntpq09:25:201
1515252992619,5cyclictest1670596-21latency_hist09:40:011
1515252992619,5cyclictest1670596-21latency_hist09:40:001
1515260992521,2cyclictest1836746-21apt-get12:20:123
1515260992520,4cyclictest1659823-21apt-get09:30:013
1515260992518,5cyclictest252-21systemd-journal08:40:203
1515255992522,2cyclictest1711661-21kworker/u8:1+flush-259:011:20:242
1515255992522,2cyclictest1711661-21kworker/u8:1+events_unbound11:30:142
1515255992522,2cyclictest1618552-21kworker/u8:4+events_unbound09:25:142
1515255992522,2cyclictest1520383-21kworker/u8:0+flush-259:008:00:002
1515250992520,4cyclictest1825801-21gpgv12:10:000
1515260992419,4cyclictest1676179-21diskstats09:45:123
1515260992418,4cyclictest252-21systemd-journal07:35:003
1515260992418,4cyclictest1836484-21latency_hist12:20:003
1515255992421,2cyclictest1582538-21kworker/u8:2+flush-259:008:20:252
1515252992419,3cyclictest1567094-21latency_hist08:00:011
1515250992419,4cyclictest1540983-21kworker/u8:3+flush-259:007:45:200
1515250992418,4cyclictest1815386-21latency_hist12:00:010
1515250992418,4cyclictest1815386-21latency_hist12:00:010
1515255992320,2cyclictest1494445-21kworker/u8:1+events_unbound07:15:142
1515250992320,2cyclictest1826339-21kworker/u8:4+events_unbound12:25:180
1515250992320,2cyclictest1711661-21kworker/u8:1+events_unbound10:40:200
1515250992319,3cyclictest1520383-21kworker/u8:0+events_unbound08:10:140
1515250992318,4cyclictest252-21systemd-journal07:55:000
1515260992218,3cyclictest1540983-21kworker/u8:3+flush-259:008:25:213
1515260992217,3cyclictest1748019-40mandb10:51:203
1515255992218,3cyclictest1711661-21kworker/u8:1+flush-259:010:51:232
1515255992218,3cyclictest1711661-21kworker/u8:1+events_unbound10:40:132
1515255992216,4cyclictest1773912-21kworker/u8:0+flush-259:011:35:242
1515252992217,4cyclictest1711661-21kworker/u8:1+flush-259:010:50:241
1515252992217,4cyclictest1659863-21kworker/u8:2+flush-259:010:10:141
1515250992218,3cyclictest1789695-21kworker/u8:2+flush-259:011:40:200
1515250992217,3cyclictest1515466-21apt-get07:10:110
1515260992118,2cyclictest1540983-21kworker/u8:3+flush-259:008:05:003
1515260992117,3cyclictest1634240-21kworker/u8:1+flush-259:009:15:003
1515260992117,3cyclictest1582538-21kworker/u8:2+events_unbound08:20:223
1515255992118,2cyclictest1540983-21kworker/u8:3+flush-259:008:00:222
1515255992118,2cyclictest1540983-21kworker/u8:3+events_unbound08:30:242
1515252992115,4cyclictest1701605-21latency_hist10:10:011
1515250992118,2cyclictest1659863-21kworker/u8:2+events_unbound10:30:230
1515260992017,2cyclictest1704171-21kthreadcore10:10:153
1515260992016,3cyclictest1613388-21kworker/u8:2+flush-259:008:45:183
1515255992017,2cyclictest1815401-21kworker/u8:0+events_unbound12:20:142
1515255992017,2cyclictest1659863-21kworker/u8:2+events_unbound10:30:192
1515255992017,2cyclictest1644359-21kworker/u8:0+events_unbound09:35:192
1515255992017,2cyclictest1644359-21kworker/u8:0+events_unbound09:35:182
1515255992017,2cyclictest1598170-21kworker/u8:1+flush-259:008:35:192
1515252992016,3cyclictest1820519-21cron12:05:001
1515252992016,3cyclictest1541209-21apt-get07:35:131
1515252992016,2cyclictest0-21swapper/111:00:201
1515252992013,5cyclictest252-21systemd-journal09:55:011
1515250992017,2cyclictest1530645-21kworker/u8:2+flush-259:007:30:170
1515250992016,3cyclictest1732906-21kworker/u8:3+flush-259:011:00:010
1515250992016,3cyclictest1675779-21kworker/u8:3+events_unbound10:00:270
1515250992016,3cyclictest1618552-21kworker/u8:4+events_unbound09:10:150
1515250992016,3cyclictest1582538-21kworker/u8:2+events_unbound08:20:130
1515250992015,4cyclictest1753770-21kworker/u8:2+events_unbound11:01:190
1515250992014,4cyclictest252-21systemd-journal11:30:000
1515260991916,3cyclictest1711661-21kworker/u8:1+flush-259:010:45:283
1515260991916,2cyclictest1659863-21kworker/u8:2+events_unbound10:35:163
1515260991913,4cyclictest1623927-21/usr/sbin/munin08:55:133
1515260991911,5cyclictest252-21systemd-journal10:10:013
1515255991916,2cyclictest1815401-21kworker/u8:0+flush-259:012:00:212
1515255991916,2cyclictest1644359-21kworker/u8:0+events_unbound09:30:232
1515255991916,2cyclictest1613388-21kworker/u8:2+flush-259:008:50:182
1515255991915,3cyclictest1613388-21kworker/u8:2+events_unbound09:00:192
1515252991912,5cyclictest252-21systemd-journal09:50:001
1515250991915,3cyclictest1815401-21kworker/u8:0+events_unbound12:15:190
1515250991915,3cyclictest1799992-21kworker/u8:3+events_unbound11:50:150
1515260991815,2cyclictest1598170-21kworker/u8:1+flush-259:008:35:153
1515260991815,2cyclictest1546128-21gpgv07:40:003
1515260991814,3cyclictest1789695-21kworker/u8:2+flush-259:011:45:243
1515260991814,3cyclictest1659871-21systemd-run09:30:003
1515260991814,3cyclictest1613388-21kworker/u8:2+events_unbound08:50:203
1515260991814,3cyclictest1598170-21kworker/u8:1+flush-259:008:35:003
1515255991816,1cyclictest1520383-21kworker/u8:0+events_unbound08:05:192
1515255991815,2cyclictest1753770-21kworker/u8:2+events_unbound11:00:242
1515252991811,5cyclictest509-21rs:main1
1515252991811,5cyclictest1717647-21cat10:25:011
1515250991815,3cyclictest1520383-21kworker/u8:0+events_unbound07:55:150
1515250991815,2cyclictest1644359-21kworker/u8:0+events_unbound09:50:110
1515250991815,2cyclictest1613388-21kworker/u8:2+events_unbound08:55:220
1515250991814,3cyclictest1644359-21kworker/u8:0+events_unbound09:36:200
1515250991814,3cyclictest1644359-21kworker/u8:0+events_unbound09:36:190
1515250991810,7cyclictest1618552-21kworker/u8:4+flush-259:009:19:540
1515260991712,3cyclictest252-21systemd-journal10:45:003
1515255991714,2cyclictest1815401-21kworker/u8:0+events_unbound12:33:202
1515255991711,4cyclictest1680943-21latency_hist09:50:002
1515252991713,3cyclictest1773912-21kworker/u8:0+flush-259:011:35:191
1515252991712,3cyclictest1711661-21kworker/u8:1+flush-259:011:24:591
1515250991714,2cyclictest60350irq/132-enp0s29f1:rx-007:20:210
1515250991714,2cyclictest1665639-21kworker/u8:1+events_unbound09:40:250
1515250991713,3cyclictest1711661-21kworker/u8:1+events_unbound11:38:200
1515250991713,3cyclictest1711661-21kworker/u8:1+events_unbound10:45:210
1515250991713,3cyclictest1659863-21kworker/u8:2+events_unbound10:10:150
1515250991713,3cyclictest1540983-21kworker/u8:3+events_unbound08:33:220
1515250991712,3cyclictest1650050-21hddtemp_smartct09:20:150
1515250991712,3cyclictest1540983-21kworker/u8:3+events_unbound07:40:140
1515250991711,4cyclictest1546205-21cp07:40:000
1515260991613,2cyclictest1732906-21kworker/u8:3+flush-259:011:15:243
1515260991612,3cyclictest1520383-21kworker/u8:0+events_unbound07:40:033
1515260991612,2cyclictest1644359-21kworker/u8:0+flush-259:009:35:183
1515260991612,2cyclictest1644359-21kworker/u8:0+flush-259:009:35:183
1515255991612,3cyclictest1826339-21kworker/u8:4+events_unbound12:35:072
1515255991612,3cyclictest1799992-21kworker/u8:3+events_unbound11:49:212
1515255991612,3cyclictest1659863-21kworker/u8:2+events_unbound10:00:242
1515252991614,1cyclictest0-21swapper/112:20:001
1515252991612,2cyclictest0-21swapper/107:50:001
1515252991611,3cyclictest1738335-21apt-get10:45:121
1515250991613,2cyclictest60350irq/132-enp0s29f1:rx-009:58:060
1515250991613,2cyclictest60350irq/132-enp0s29f1:rx-008:45:210
1515250991612,3cyclictest60350irq/132-enp0s29f1:rx-011:20:130
1515250991612,3cyclictest1773912-21kworker/u8:0+events_unbound11:32:200
1515250991612,3cyclictest1773912-21kworker/u8:0+events_unbound11:17:190
1515250991612,3cyclictest1644359-21kworker/u8:0+flush-259:010:54:280
1515260991513,1cyclictest0-21swapper/312:04:593
1515260991512,2cyclictest1583805-21kthreadcore08:15:183
1515260991512,2cyclictest0-21swapper/310:05:003
1515260991511,3cyclictest1826339-21kworker/u8:4+flush-259:012:35:213
1515260991511,3cyclictest1799992-21kworker/u8:3+flush-259:011:50:203
1515260991511,3cyclictest1613388-21kworker/u8:2+flush-259:009:05:003
1515255991513,2cyclictest1753770-21kworker/u8:2+events_unbound11:05:252
1515255991512,2cyclictest1644359-21kworker/u8:0+events_unbound09:50:222
1515255991511,3cyclictest1675779-21kworker/u8:3+events_unbound10:14:372
1515252991512,2cyclictest1761312-21kthreadcore11:05:181
1515250991513,1cyclictest60350irq/132-enp0s29f1:rx-010:25:170
1515250991513,1cyclictest60350irq/132-enp0s29f1:rx-009:25:480
1515250991513,1cyclictest60350irq/132-enp0s29f1:rx-008:50:150
1515250991513,1cyclictest60350irq/132-enp0s29f1:rx-007:15:190
1515250991512,3cyclictest1592814-21kworker/u8:0+events_unbound09:00:120
1515250991511,3cyclictest1659863-21kworker/u8:2+events_unbound10:20:240
1515250991511,3cyclictest1598170-21kworker/u8:1+events_unbound08:35:180
151526099149,3cyclictest509-21rs:main3
151526099148,5cyclictest252-21systemd-journal11:10:003
151526099148,4cyclictest252-21systemd-journal07:21:203
151526099148,4cyclictest1826533-21diskstats12:10:133
1515260991412,1cyclictest1779354-21turbostat11:25:003
1515260991411,2cyclictest1815401-21kworker/u8:0+flush-259:012:30:193
1515260991411,2cyclictest1716118-21ntp_kernel_err10:20:203
1515255991410,3cyclictest1644359-21kworker/u8:0+flush-259:009:15:262
1515255991410,3cyclictest1540983-21kworker/u8:3+events_unbound08:26:162
151525299149,4cyclictest1613388-21kworker/u8:2+flush-259:008:50:161
151525299148,4cyclictest1826202-21cat12:10:001
1515252991410,3cyclictest1540983-21kworker/u8:3+flush-259:007:40:251
1515250991412,1cyclictest60350irq/132-enp0s29f1:rx-012:35:260
1515250991412,1cyclictest60350irq/132-enp0s29f1:rx-012:20:590
1515250991412,1cyclictest60350irq/132-enp0s29f1:rx-010:35:260
1515250991412,1cyclictest60350irq/132-enp0s29f1:rx-010:05:120
1515250991411,2cyclictest61150irq/133-enp0s29f1:tx-011:05:160
1515250991411,2cyclictest61150irq/133-enp0s29f1:tx-008:05:130
1515250991411,2cyclictest60350irq/132-enp0s29f1:rx-012:30:120
1515250991410,3cyclictest1618552-21kworker/u8:4+events_unbound09:30:160
151526099139,3cyclictest1773912-21kworker/u8:0+flush-259:011:35:233
151526099139,3cyclictest1675779-21kworker/u8:3+events_unbound10:15:113
151526099139,3cyclictest1561468-21unixbench_multi07:50:243
151526099136,5cyclictest252-21systemd-journal11:29:593
1515260991311,1cyclictest1654655-21idleruntime-cro09:24:593
1515260991310,2cyclictest1797362-21kthreadcore11:40:183
1515260991310,2cyclictest1711661-21kworker/u8:1+flush-259:010:55:243
1515260991310,2cyclictest1659863-21kworker/u8:2+flush-259:009:55:163
151525299139,3cyclictest1856647-21unixbench_singl12:35:251
151525299139,3cyclictest1644359-21kworker/u8:0+flush-259:009:30:231
151525299139,3cyclictest1540983-21kworker/u8:3+flush-259:008:25:131
151525299139,3cyclictest1540983-21kworker/u8:3+flush-259:007:54:571
151525299139,2cyclictest0-21swapper/108:45:161
1515252991310,2cyclictest1766443-21kthreadcore11:10:171
1515252991310,2cyclictest1711661-21kworker/u8:1-events_unbound11:25:161
151525099139,3cyclictest1644359-21kworker/u8:0+flush-259:010:15:180
151525099139,3cyclictest1540983-21kworker/u8:3+events_unbound08:16:220
1515250991312,1cyclictest60350irq/132-enp0s29f1:rx-009:49:380
1515250991310,2cyclictest1815401-21kworker/u8:0+flush-259:012:10:160
151526099129,2cyclictest503-21tpm2-abrmd12:00:003
151526099129,2cyclictest503-21tpm2-abrmd12:00:003
151526099128,3cyclictest1455979-21kworker/u8:3+events_unbound07:10:183
151526099126,4cyclictest1535774-21awk07:30:013
1515260991210,1cyclictest0-21swapper/309:40:163
151525599128,3cyclictest1613388-21kworker/u8:2+events_unbound08:45:542
151525299129,2cyclictest1691614-21cpuspeed_turbos10:00:131
151525299129,2cyclictest1603439-21cpuspeed_turbos08:35:131
151525299129,2cyclictest1592319-21unixbench_multi08:20:251
151525299129,2cyclictest1567368-21apt-get08:00:101
151525299128,3cyclictest1789695-21kworker/u8:2+flush-259:011:45:301
1515252991210,1cyclictest0-21swapper/107:15:001
1515250991210,1cyclictest60350irq/132-enp0s29f1:rx-008:00:190
151526099118,1cyclictest1711661-21kworker/u8:1-events_unbound11:30:163
151525599119,2cyclictest1598170-21kworker/u8:1+events_unbound08:40:222
151525299119,1cyclictest0-21swapper/112:10:001
151525299118,2cyclictest1686362-21/usr/sbin/munin09:55:181
151525299118,2cyclictest0-21swapper/109:00:151
151526099107,2cyclictest1553070-21kthreadcore07:45:173
151526099107,2cyclictest0-21swapper/312:05:173
151526099106,2cyclictest0-21swapper/311:00:243
151525299108,1cyclictest0-21swapper/108:40:161
151525299107,2cyclictest1582864-21cpuspeed_turbos08:15:131
151525299105,3cyclictest1773592-21cat11:15:231
151525299100,8cyclictest1644330-21latency_hist09:15:011
15152529996,2cyclictest0-21swapper/112:25:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional