You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-17 - 12:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Sun May 17, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1286121994542,2cyclictest1363686-21apt-get20:25:110
1286121994542,2cyclictest1363686-21apt-get20:25:110
1286124994135,4cyclictest1332701-21/usr/sbin/munin19:55:151
1286121993936,2cyclictest1613359-21kworker/u8:3+flush-259:000:30:210
1286121993731,4cyclictest1378035-21ntpq20:35:200
1286127993634,1cyclictest1472014-21fwupd22:07:582
1286131993530,3cyclictest1560934-21base6423:35:003
1286127993530,3cyclictest1373978-21kworker/u8:0+flush-259:020:55:182
1286124993531,3cyclictest1426054-21apt-get21:25:131
1286121993429,3cyclictest1451564-21latency_hist21:50:010
1286124993328,3cyclictest1487291-21ntpq22:20:231
1286121993227,3cyclictest1597439-21latency_hist00:10:010
1286121993226,5cyclictest1440403-21ntpq21:35:220
1286127993127,3cyclictest1232914-21kworker/u8:1+flush-259:019:20:012
1286127993124,5cyclictest1305851-21ntpq19:25:222
1286121993128,2cyclictest1436111-21kworker/u8:3+flush-259:022:00:160
1286127993024,4cyclictest1307016-21apt-get19:30:252
1286121993027,2cyclictest1477396-21kworker/u8:2+flush-259:023:00:180
1286121993026,3cyclictest1472205-21latency_hist22:10:000
1286131992924,3cyclictest252-21systemd-journal22:10:573
1286131992922,5cyclictest1488968-21diskstats22:25:163
1286127992924,4cyclictest1576715-21kworker/u8:4+flush-259:023:55:172
1286121992926,2cyclictest1373978-21kworker/u8:0+flush-259:020:55:280
1286121992924,4cyclictest1347013-21ntpq20:05:210
1286127992823,4cyclictest1534980-21kworker/u8:1+flush-259:000:05:152
1286124992825,3cyclictest1477396-21kworker/u8:2+flush-259:023:00:251
1286124992822,4cyclictest252-21systemd-journal00:15:011
1286121992824,3cyclictest1410607-21apt-get21:10:120
1286131992721,4cyclictest1420649-21latency_hist21:20:013
1286124992723,3cyclictest1477396-21kworker/u8:2+flush-259:022:20:001
1286121992724,2cyclictest1352726-21kworker/u8:4+flush-259:020:30:160
1286127992610,7cyclictest1477362-21munin-run22:15:002
1286124992622,3cyclictest1363652-21kworker/u8:3+flush-259:020:50:001
1286124992619,5cyclictest1362504-21ntpq20:20:211
1286124992619,5cyclictest1342771-21latency_hist20:05:011
1286121992623,2cyclictest1436111-21kworker/u8:3+flush-259:021:40:250
1286121992623,2cyclictest1415117-21kworker/u8:1+events_unbound21:25:140
1286121992622,2cyclictest1280971-21kworker/u8:2+events_unbound19:40:000
1286124992522,2cyclictest1373978-21kworker/u8:0+events_unbound21:45:241
1286121992522,2cyclictest1415117-21kworker/u8:1+flush-259:021:30:590
1286121992522,2cyclictest1280971-21kworker/u8:2+flush-259:019:30:010
1286121992518,5cyclictest1508288-21ntpq22:40:220
1286131992420,3cyclictest1561468-21apt-get23:35:123
1286124992421,2cyclictest1597258-21kworker/u8:3+flush-259:000:15:161
1286124992418,4cyclictest252-21systemd-journal19:42:141
1286131992317,4cyclictest1602656-21https00:15:003
1286131992315,6cyclictest1483169-21cp22:20:013
1286127992319,3cyclictest1519755-21apt-get22:55:122
1286127992318,4cyclictest1384382-21kworker/u8:2+flush-259:021:05:002
1286127992318,3cyclictest1363652-21kworker/u8:3+flush-259:020:50:152
1286127992318,3cyclictest1286345-21apt-get19:10:132
128612499235,8cyclictest1394927-21latency_hist20:55:001
1286124992319,3cyclictest1613359-21kworker/u8:3+events_unbound00:30:161
1286121992317,4cyclictest1482540-21apt-get22:20:000
128613199227,4cyclictest1513383-21ntpq22:45:193
128613199227,4cyclictest1513383-21ntpq22:45:193
1286131992217,3cyclictest252-21systemd-journal21:01:573
1286131992215,4cyclictest1347934-21latency_hist20:10:013
1286127992215,5cyclictest1445515-21ntpq21:40:212
1286127992214,6cyclictest1561049-21cat23:35:022
1286124992218,3cyclictest1436111-21kworker/u8:3+events_unbound21:40:131
1286124992218,3cyclictest1264764-21kworker/u8:4+flush-259:019:10:251
1286124992216,4cyclictest1311874-21kworker/u8:1+flush-259:019:35:171
1286121992218,3cyclictest1456738-21kworker/u8:1+flush-259:021:55:160
1286121992218,3cyclictest1363652-21kworker/u8:3+events_unbound20:45:000
1286121992217,4cyclictest1436111-21kworker/u8:3+events_unbound21:50:220
128613199216,4cyclictest1409425-21ntpq21:05:223
1286124992117,3cyclictest1498850-21kworker/u8:0+flush-259:022:40:161
1286124992117,3cyclictest1280971-21kworker/u8:2+events_unbound19:15:141
1286121992118,3cyclictest1587080-21kworker/u8:0+events_unbound00:15:230
1286121992118,2cyclictest1373978-21kworker/u8:0+flush-259:021:20:210
1286121992117,3cyclictest1597258-21kworker/u8:3+events_unbound00:10:130
1286121992117,3cyclictest1534980-21kworker/u8:1+events_unbound00:30:000
1286121992117,3cyclictest1456738-21kworker/u8:1+flush-259:022:30:240
128613199204,4cyclictest1534036-21ntpq23:05:193
1286131992015,3cyclictest1410582-21/usr/sbin/munin21:10:233
1286131992014,4cyclictest1623952-21apt-get00:35:113
1286127992017,2cyclictest0-21swapper/222:05:002
1286124992017,2cyclictest1477396-21kworker/u8:2+flush-259:022:35:151
1286124992015,4cyclictest1321179-21kworker/u8:0+events_unbound19:45:221
1286124992015,3cyclictest252-21systemd-journal23:12:411
1286124992015,3cyclictest1327204-21kworker/u8:2+flush-259:020:10:221
1286121992017,2cyclictest1327204-21kworker/u8:2+events_unbound20:12:180
1286121992016,3cyclictest1509181-21kworker/u8:1+flush-259:022:55:200
1286121992016,3cyclictest1-21systemd00:00:010
128613199195,11cyclictest1394923-21latency_hist20:55:013
1286127991916,2cyclictest1602881-21latency_hist00:15:002
1286127991916,2cyclictest1550449-21cron23:25:002
1286124991916,2cyclictest1373978-21kworker/u8:0+flush-259:021:35:001
1286124991915,3cyclictest1477396-21kworker/u8:2+flush-259:023:43:571
1286124991915,3cyclictest1352726-21kworker/u8:4+events_unbound20:15:211
1286124991914,3cyclictest1472465-21apt-get22:10:111
1286121991916,2cyclictest1327204-21kworker/u8:2+flush-259:020:00:160
1286121991915,3cyclictest1587080-21kworker/u8:0+events_unbound00:20:220
1286121991915,3cyclictest1311874-21kworker/u8:1+events_unbound19:44:580
1286127991814,3cyclictest1581901-21turbostat.cron23:55:002
1286127991814,3cyclictest1405439-21/usr/sbin/munin21:05:162
1286127991813,3cyclictest1327537-21/usr/sbin/munin19:50:152
1286127991812,4cyclictest1514551-21/usr/sbin/munin22:50:242
1286127991810,6cyclictest1575823-21ntpq23:45:222
1286124991815,2cyclictest1587080-21kworker/u8:0+events_unbound00:22:571
1286124991815,2cyclictest1373978-21kworker/u8:0+events_unbound21:20:241
1286124991815,2cyclictest1309283-21kthreadcore19:30:301
1286124991814,3cyclictest1321179-21kworker/u8:0+flush-259:020:05:201
1286124991813,3cyclictest1456738-21kworker/u8:1+flush-259:022:25:161
1286121991815,2cyclictest1509181-21kworker/u8:1+flush-259:022:50:160
1286121991815,2cyclictest1280971-21kworker/u8:2+flush-259:019:30:310
1286121991814,2cyclictest60350irq/132-enp0s29f1:rx-000:00:100
1286121991813,4cyclictest1264764-21kworker/u8:4+flush-259:019:20:160
128613199176,9cyclictest1293775-21grep19:15:193
128613199174,10cyclictest1472205-21latency_hist22:10:003
1286127991711,4cyclictest1613775-21diskstats00:25:172
1286124991714,2cyclictest1472205-21sh22:10:001
1286124991714,1cyclictest0-21swapper/121:55:001
1286124991713,3cyclictest1534980-21kworker/u8:1+events_unbound23:20:171
1286124991713,3cyclictest1400298-21/usr/sbin/munin21:00:151
1286121991714,2cyclictest1555317-21kworker/u8:0+flush-259:023:35:160
1286121991713,3cyclictest1534980-21kworker/u8:1+flush-259:000:35:250
1286121991713,3cyclictest1534980-21kworker/u8:1+events_unbound23:20:230
1286121991713,3cyclictest1321179-21kworker/u8:0+events_unbound19:49:590
1286121991712,4cyclictest1373978-21kworker/u8:0+events_unbound21:15:220
1286121991712,3cyclictest1498850-21kworker/u8:0+flush-259:022:46:020
1286121991712,3cyclictest1498850-21kworker/u8:0+flush-259:022:46:020
128613199166,8cyclictest1529937-21cat23:05:003
128613199163,4cyclictest1587469-21df00:00:123
1286131991612,3cyclictest1384408-21store20:45:013
1286127991613,2cyclictest252-21systemd-journal20:00:002
1286127991613,2cyclictest1623698-21munin-plugin-st00:34:592
128612499164,6cyclictest1400072-21basename21:00:001
1286124991611,3cyclictest0-21swapper/122:50:181
1286121991614,1cyclictest60350irq/132-enp0s29f1:rx-020:50:140
1286121991614,1cyclictest60350irq/132-enp0s29f1:rx-019:55:190
1286121991613,2cyclictest60350irq/132-enp0s29f1:rx-023:50:160
1286121991613,2cyclictest60350irq/132-enp0s29f1:rx-023:10:000
1286121991612,3cyclictest1291485-21apt-get19:15:120
1286121991612,2cyclictest1487225-21ntp_kernel_pll_22:20:220
1286121991612,2cyclictest1405210-21turbostat21:05:000
128613199159,4cyclictest1555578-21latency_hist23:30:013
128613199159,3cyclictest1388914-21ntpq20:45:223
128613199156,6cyclictest1499341-21cut22:35:143
128613199156,6cyclictest1363689-21taskset20:25:123
128613199156,6cyclictest1363689-21taskset20:25:123
128613199155,8cyclictest1327306-21latency_hist19:50:003
128613199154,3cyclictest1539249-21ntpq23:10:213
128613199153,7cyclictest1452105-21fw_conntrack21:50:133
128613199153,5cyclictest1445682-21sensors_temp21:40:243
1286131991512,2cyclictest1300633-21ntp_kernel_err19:20:203
1286131991510,3cyclictest1341885-21ntpq20:00:203
1286131991510,3cyclictest0-21swapper/319:25:183
128612799154,5cyclictest0-21swapper/219:45:002
1286127991511,3cyclictest252-21systemd-journal19:25:002
1286127991511,2cyclictest1477396-21kworker/u8:2+flush-259:023:05:152
1286124991511,3cyclictest509-21rs:main1
1286124991511,3cyclictest509-21rs:main1
1286124991510,4cyclictest1587034-21cron00:00:011
1286121991513,2cyclictest60350irq/132-enp0s29f1:rx-023:12:470
1286121991513,1cyclictest60350irq/132-enp0s29f1:rx-023:30:380
1286121991513,1cyclictest60350irq/132-enp0s29f1:rx-019:10:370
1286121991512,2cyclictest60350irq/132-enp0s29f1:rx-023:45:140
1286121991512,2cyclictest60350irq/132-enp0s29f1:rx-023:25:200
1286121991511,3cyclictest1534980-21kworker/u8:1+events_unbound23:15:230
128612199150,14cyclictest1358282-21turbostat20:20:000
128612199150,13cyclictest1477360-21turbostat22:15:000
128613199148,4cyclictest1550444-21turbostat.cron23:25:013
128613199148,4cyclictest1520168-21grep22:55:173
128613199148,4cyclictest1264764-21kworker/u8:4+flush-259:019:10:183
128613199147,5cyclictest1374299-21/usr/sbin/munin20:35:143
128613199146,6cyclictest1618624-21sort00:30:003
128613199146,6cyclictest1482737-21grep22:20:003
128613199146,5cyclictest1608586-21grep00:20:123
128613199145,2cyclictest1424909-21ntpq21:20:213
128613199144,3cyclictest1461870-21latency_hist22:00:003
128613199144,2cyclictest1369030-21df_inode20:30:143
128613199144,1cyclictest1431385-21tune2fs21:30:143
128613199143,3cyclictest1577242-21fw_conntrack23:50:153
128612799148,4cyclictest1529853-21latency_hist23:05:002
1286127991411,2cyclictest1608891-21irqcore00:20:162
1286127991411,2cyclictest1565926-21unixbench_singl23:35:252
1286127991410,3cyclictest1456738-21kworker/u8:1+flush-259:022:15:212
1286127991410,3cyclictest1415117-21kworker/u8:1+flush-259:021:30:262
1286127991410,3cyclictest0-21swapper/222:30:002
128612499149,4cyclictest1524648-21latency_hist23:00:001
1286124991411,2cyclictest1363659-21/usr/sbin/munin20:25:161
1286124991411,2cyclictest1363659-21/usr/sbin/munin20:25:161
1286124991410,3cyclictest1587080-21kworker/u8:0+events_unbound00:00:201
1286124991410,3cyclictest1373978-21kworker/u8:0+events_unbound20:40:231
1286124991410,2cyclictest1461866-21sh22:00:001
1286121991412,1cyclictest60350irq/132-enp0s29f1:rx-022:35:140
1286121991412,1cyclictest60350irq/132-enp0s29f1:rx-020:40:190
1286121991412,1cyclictest60350irq/132-enp0s29f1:rx-019:50:250
1286121991411,2cyclictest60350irq/132-enp0s29f1:rx-023:40:140
1286121991411,2cyclictest60350irq/132-enp0s29f1:rx-021:05:150
1286121991411,2cyclictest0-21swapper/020:25:000
128613199138,3cyclictest490-21dbus-daemon00:00:003
128613199138,3cyclictest1352196-21ntpq20:10:183
128613199137,4cyclictest1515002-21grep22:50:173
128613199137,4cyclictest1426515-21irqcore21:25:183
128613199137,4cyclictest1317671-21irqcore19:40:173
128613199136,5cyclictest1621065-21/usr/sbin/munin00:30:193
128613199136,5cyclictest1494361-21ps22:30:173
128613199136,5cyclictest1362424-21grep20:20:203
128613199136,4cyclictest1462213-21cut22:00:133
128613199133,6cyclictest1436485-21df21:35:133
128613199133,5cyclictest1311447-21users19:30:383
128613199133,2cyclictest1603474-21df00:15:153
128613199133,2cyclictest1592806-21fw_conntrack00:05:143
128613199133,2cyclictest1544192-21memory23:15:213
128613199133,2cyclictest1357408-21ntpq20:15:203
128612799138,4cyclictest1358511-21kworker/u8:0+flush-259:020:20:172
128612799138,3cyclictest1430929-21awk21:30:002
1286127991310,2cyclictest2971-21cron22:00:002
1286127991310,2cyclictest1540443-21cpuspeed_turbos23:15:142
1286127991310,2cyclictest1496237-21kthreadcore22:30:172
1286127991310,2cyclictest1342790-21kworker/u8:1+flush-259:020:35:092
1286124991311,1cyclictest1477396-21kworker/u8:2+events_unbound23:15:171
1286124991310,2cyclictest1373978-21kworker/u8:0+flush-259:021:35:211
1286121991311,1cyclictest60350irq/132-enp0s29f1:rx-022:26:390
128613199125,5cyclictest1395602-21irqcore20:55:183
128613199125,4cyclictest1312153-21wc19:35:123
128612799129,2cyclictest1343291-21fschecks_time20:05:152
128612799126,4cyclictest1384270-21cp20:45:002
128612499129,2cyclictest1264764-21kworker/u8:4+flush-259:019:20:271
128612499128,3cyclictest1534980-21kworker/u8:1+flush-259:023:25:161
128612799118,2cyclictest1425298-21unixbench_multi21:20:242
128612799118,2cyclictest1348258-21cpuspeed_turbos20:10:112
128612799118,1cyclictest0-21swapper/221:35:162
128612799117,3cyclictest1534980-21kworker/u8:1+flush-259:023:25:212
128612799117,3cyclictest1311874-21kworker/u8:1+events_unbound19:45:162
128612799111,2cyclictest1509154-21awk22:45:002
128612499118,2cyclictest1571867-21cpuspeed_turbos23:45:131
128612799106,2cyclictest0-21swapper/222:45:152
128612799106,2cyclictest0-21swapper/222:45:152
128612799100,8cyclictest1368617-21latency_hist20:30:012
128612799100,8cyclictest1368617-21latency_hist20:30:002
128612499107,3cyclictest1477396-21kworker/u8:2+events_unbound23:30:241
128612499107,2cyclictest1352726-21kworker/u8:4+flush-259:020:35:001
128612499106,3cyclictest1477396-21kworker/u8:2+events_unbound23:05:141
12861279991,7cyclictest61-21kcompactd020:30:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional