You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-19 - 19:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Fri Dec 19, 2025 12:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1361471994341,1cyclictest1453919-21fwupd08:38:112
1361471994137,3cyclictest1625907-21kworker/u8:2+flush-259:012:35:002
1361460994035,4cyclictest1346122-21kworker/u8:1+flush-259:008:15:000
1361471993934,3cyclictest1552777-21latency_hist10:15:002
1361467993731,4cyclictest1625867-21latency_hist11:25:011
1361460993733,3cyclictest1443807-21kworker/u8:0+events_unbound08:38:110
1361467993630,4cyclictest252-21systemd-journal11:40:001
1361467993629,5cyclictest1407757-21latency_hist07:55:001
1361471993429,3cyclictest1459520-21base6408:45:002
1361467993429,4cyclictest1692864-21kworker/u8:4+flush-259:012:40:001
1361476993128,2cyclictest1428316-21kworker/u8:2+flush-259:008:40:193
1361460993124,5cyclictest1521758-21latency_hist09:45:010
1361467993027,2cyclictest1340559-21kworker/u8:0+flush-259:007:15:261
1361460993025,4cyclictest1382037-21kworker/u8:0+flush-259:007:40:000
1361460993024,4cyclictest1408187-21diskstats07:55:130
1361476992925,3cyclictest1604886-21kworker/u8:2+flush-259:011:05:233
1361467992924,3cyclictest1532330-21/usr/sbin/munin09:55:141
1361467992924,3cyclictest1532330-21/usr/sbin/munin09:55:131
1361460992924,3cyclictest1382037-21kworker/u8:0+events_unbound07:40:230
1361460992924,3cyclictest1382037-21kworker/u8:0+events_unbound07:40:220
1361460992922,5cyclictest1500209-21ntpq09:20:210
136146099291,26cyclictest1402616-21true07:50:000
1361476992824,3cyclictest1470336-21kworker/u8:1+flush-259:010:00:003
1361476992824,3cyclictest1470336-21kworker/u8:1+flush-259:009:59:593
1361476992821,4cyclictest1532091-21latency_hist09:55:003
1361460992825,2cyclictest1428316-21kworker/u8:2+events_unbound09:10:200
1361460992824,3cyclictest1656773-21kworker/u8:1+flush-259:012:15:210
1361460992824,3cyclictest1656773-21kworker/u8:1+flush-259:012:15:210
1361460992723,3cyclictest1620471-21kworker/u8:1+flush-259:011:20:270
1361460992721,4cyclictest1672452-21/usr/sbin/munin12:10:130
1361471992622,3cyclictest1625907-21kworker/u8:2+flush-259:011:55:002
1361471992621,4cyclictest1453928-21kworker/u8:3+flush-259:008:45:242
1361467992620,4cyclictest1442943-21ntpq08:25:211
1361467992618,5cyclictest1579006-21diskstats10:40:141
1361471992521,3cyclictest1656773-21kworker/u8:1+flush-259:012:10:002
1361471992518,5cyclictest1588035-21ntpq10:45:212
1361460992522,2cyclictest1532318-21kworker/u8:0+flush-259:010:05:210
1361460992522,2cyclictest1532318-21kworker/u8:0+flush-259:010:05:200
136146099251,22cyclictest490-21dbus-daemon12:30:000
1361476992419,3cyclictest1470336-21kworker/u8:1+flush-259:010:45:003
1361476992419,3cyclictest1412897-21kworker/u8:3+flush-259:008:00:183
1361467992418,4cyclictest1526914-21latency_hist09:50:011
1361460992420,3cyclictest1636436-21apt-get11:35:120
1361460992416,5cyclictest252-21systemd-journal10:50:000
1361476992318,3cyclictest252-21systemd-journal10:14:593
1361476992318,3cyclictest1470336-21kworker/u8:1+events_unbound10:00:163
1361471992319,3cyclictest1656773-21kworker/u8:1+flush-259:012:00:242
1361467992315,5cyclictest1614718-21ntpq11:10:241
1361460992317,4cyclictest1470336-21kworker/u8:1+events_unbound09:35:210
1361476992218,3cyclictest1610688-21kworker/u8:0+flush-259:011:45:243
1361476992218,3cyclictest1480670-21apt-get09:05:123
1361471992218,3cyclictest1620471-21kworker/u8:1+flush-259:011:25:192
1361467992218,3cyclictest1454396-21apt-get08:40:121
1361467992217,3cyclictest1361663-21apt-get07:10:121
1361467992216,5cyclictest1515737-21ntpq09:35:211
1361460992219,2cyclictest1552793-21kworker/u8:3+flush-259:010:15:200
1361460992218,3cyclictest1340559-21kworker/u8:0+events_unbound07:10:220
1361460992217,3cyclictest1656773-21kworker/u8:1+events_unbound12:05:190
1361460992217,3cyclictest1626300-21diskstats11:25:170
1361471992116,3cyclictest509-21rs:main2
1361471992116,3cyclictest252-21systemd-journal10:54:092
1361460992117,3cyclictest1532318-21kworker/u8:0+flush-259:010:30:200
1361460992117,3cyclictest1443807-21kworker/u8:0+events_unbound09:00:250
1361460992115,4cyclictest1651631-21latency_hist11:50:010
1361460992114,5cyclictest252-21systemd-journal08:45:020
1361460992114,5cyclictest1418465-21diskstats08:05:150
1361476992017,2cyclictest1687679-21kworker/u8:1+events_unbound12:30:143
1361476992016,3cyclictest509-21rs:main3
1361476992015,4cyclictest1346122-21kworker/u8:1+flush-259:007:40:173
1361476992015,4cyclictest1346122-21kworker/u8:1+flush-259:007:40:173
1361476992014,4cyclictest252-21systemd-journal10:05:113
1361476992014,4cyclictest252-21systemd-journal10:05:113
1361476992014,4cyclictest1656773-21kworker/u8:1+events_unbound11:55:223
1361476992014,4cyclictest1475277-21latency_hist09:00:013
1361471992015,3cyclictest1620998-21apt-get11:20:132
1361467992016,3cyclictest1423014-21kworker/u8:0+flush-259:008:15:141
1361467992016,2cyclictest1495962-21kworker/u8:3+events_unbound09:55:001
1361467992014,4cyclictest1485589-21latency_hist09:10:011
1361467992014,4cyclictest1438639-21latency_hist08:25:011
1361460992017,2cyclictest1346122-21kworker/u8:1+flush-259:007:25:200
1361460992016,3cyclictest1610688-21kworker/u8:0+events_unbound11:50:220
1361460992016,3cyclictest1604886-21kworker/u8:2+events_unbound11:14:100
1361460992015,4cyclictest1610688-21kworker/u8:0+events_unbound11:30:200
1361460992014,4cyclictest1666169-21ntpq12:00:220
1361476991916,2cyclictest1453919-21fwupd08:38:113
1361476991915,3cyclictest1656773-21kworker/u8:1+flush-259:012:15:183
1361476991915,3cyclictest1656773-21kworker/u8:1+flush-259:012:15:173
1361476991915,3cyclictest1346122-21kworker/u8:1+flush-259:007:30:213
1361471991914,3cyclictest252-21systemd-journal07:55:002
1361467991913,4cyclictest1656745-21latency_hist11:55:011
1361460991916,2cyclictest1495962-21kworker/u8:3+events_unbound09:50:000
1361460991915,3cyclictest1428316-21kworker/u8:2+events_unbound08:20:250
1361460991915,3cyclictest1372389-21irqcore07:20:150
1361460991914,3cyclictest1661885-21cron11:59:590
1361476991815,2cyclictest1600986-21kthreadcore11:00:183
1361476991814,3cyclictest252-21systemd-journal07:50:003
1361476991814,3cyclictest1667050-21kworker/u8:0+flush-259:012:05:003
1361476991810,6cyclictest252-21systemd-journal10:20:003
1361471991815,2cyclictest1371725-21kworker/u8:2+flush-259:008:05:152
1361471991813,3cyclictest1428316-21kworker/u8:2+events_unbound09:10:022
1361467991814,3cyclictest1656773-21kworker/u8:1+events_unbound12:15:171
1361467991814,3cyclictest1656773-21kworker/u8:1+events_unbound12:15:171
1361467991812,5cyclictest1646445-21latency_hist11:45:001
1361467991812,4cyclictest1589563-21cat10:50:011
1361467991812,4cyclictest1520939-21ntpq09:40:241
1361460991815,2cyclictest1568296-21munin-plugin-st10:30:010
1361460991814,3cyclictest1532318-21kworker/u8:0+flush-259:010:02:460
1361460991814,3cyclictest1470336-21kworker/u8:1+flush-259:010:42:510
1361476991713,3cyclictest1470336-21kworker/u8:1+flush-259:010:50:203
1361471991715,1cyclictest1636167-21sh11:34:592
1361471991712,4cyclictest509-21rs:main2
1361471991712,4cyclictest509-21rs:main2
136147199171,14cyclictest1-21systemd07:48:092
1361460991714,2cyclictest60350irq/132-enp0s29f1:rx-007:52:040
1361460991714,2cyclictest1642000-21irqcore11:40:170
1361460991714,2cyclictest1495949-21latency_hist09:20:000
1361460991713,3cyclictest1532318-21kworker/u8:0+flush-259:010:38:170
1361460991713,3cyclictest1532318-21kworker/u8:0+events_unbound09:55:210
1361460991713,3cyclictest1532318-21kworker/u8:0+events_unbound09:55:200
1361460991713,3cyclictest1443807-21kworker/u8:0+flush-259:008:50:250
1361460991713,3cyclictest1443807-21kworker/u8:0+events_unbound08:55:140
1361460991712,3cyclictest1428316-21kworker/u8:2+events_unbound08:40:120
1361460991712,3cyclictest1423014-21kworker/u8:0+flush-259:008:15:150
1361476991613,2cyclictest1652323-21tr11:50:173
1361476991613,2cyclictest1366546-21idleruntime-cro07:14:593
1361476991613,2cyclictest0-21swapper/311:00:013
1361476991612,3cyclictest1604886-21kworker/u8:2+events_unbound11:15:213
1361476991612,3cyclictest1443807-21kworker/u8:0+flush-259:008:50:163
1361476991612,3cyclictest1428316-21kworker/u8:2+flush-259:009:00:193
1361471991612,3cyclictest1656773-21kworker/u8:1+flush-259:012:10:162
1361471991612,3cyclictest1470336-21kworker/u8:1+flush-259:010:20:012
1361471991612,3cyclictest1470336-21kworker/u8:1+flush-259:009:15:162
1361471991612,3cyclictest1443807-21kworker/u8:0+flush-259:008:55:152
1361471991612,3cyclictest1382278-21apt-get07:30:102
1361471991611,3cyclictest252-21systemd-journal10:38:102
1361471991610,5cyclictest1408025-21apt-get07:55:122
1361471991610,4cyclictest1375979-21ntpq07:20:192
1361467991613,2cyclictest1489939-21ntp_offset09:10:221
1361467991613,1cyclictest0-21swapper/111:05:001
1361467991612,3cyclictest1428316-21kworker/u8:2+flush-259:008:35:261
1361467991611,3cyclictest252-21systemd-journal10:30:011
1361467991610,4cyclictest252-21systemd-journal11:49:591
1361467991610,4cyclictest1505429-21ntpq09:25:211
1361467991610,3cyclictest252-21systemd-journal12:20:011
1361467991610,3cyclictest252-21systemd-journal12:20:001
1361460991613,2cyclictest60350irq/132-enp0s29f1:rx-010:55:210
1361460991613,2cyclictest60350irq/132-enp0s29f1:rx-009:25:030
1361460991613,2cyclictest60350irq/132-enp0s29f1:rx-007:19:050
1361460991613,2cyclictest1687679-21kworker/u8:1+flush-259:012:30:200
1361460991613,2cyclictest1470336-21kworker/u8:1+flush-259:010:50:210
1361460991612,3cyclictest1443807-21kworker/u8:0+events_unbound09:08:100
1361476991513,1cyclictest0-21swapper/309:25:003
1361476991511,3cyclictest1346122-21kworker/u8:1+events_unbound08:25:183
1361476991511,2cyclictest0-21swapper/310:35:013
1361471991511,3cyclictest1453928-21kworker/u8:3+flush-259:008:55:012
1361471991511,3cyclictest1346122-21kworker/u8:1+flush-259:008:10:272
1361471991511,3cyclictest1340559-21kworker/u8:0+flush-259:007:10:262
1361471991511,2cyclictest1537262-21cron10:00:002
1361471991511,2cyclictest1537262-21cron10:00:002
1361471991510,3cyclictest252-21systemd-journal10:03:092
1361467991512,1cyclictest0-21swapper/107:50:001
1361467991511,3cyclictest1470336-21kworker/u8:1+flush-259:010:10:251
1361467991511,3cyclictest1453928-21kworker/u8:3+flush-259:008:48:161
1361467991511,3cyclictest1346122-21kworker/u8:1+flush-259:007:25:241
1361460991513,2cyclictest60350irq/132-enp0s29f1:rx-008:27:100
1361460991513,1cyclictest60350irq/132-enp0s29f1:rx-009:32:430
1361460991512,2cyclictest1382037-21kworker/u8:0+flush-259:007:34:530
136146099151,1cyclictest1448951-21turbostat08:35:010
1361460991511,3cyclictest1412897-21kworker/u8:3+flush-259:008:00:160
1361476991410,3cyclictest1620471-21kworker/u8:1+flush-259:011:25:163
1361476991410,3cyclictest1610688-21kworker/u8:0+flush-259:011:35:143
1361476991410,3cyclictest1443807-21kworker/u8:0+flush-259:008:30:523
1361476991410,2cyclictest0-21swapper/309:50:003
1361471991410,3cyclictest1583993-21kworker/u8:3+flush-259:011:01:462
1361471991410,3cyclictest1495962-21kworker/u8:3+events_unbound09:30:132
136146799149,3cyclictest1542689-21apt-get10:05:121
136146799149,3cyclictest1542689-21apt-get10:05:111
136146799148,4cyclictest1448943-21latency_hist08:35:011
1361467991411,2cyclictest1495461-21unixbench_multi09:15:251
1361467991411,1cyclictest1594933-21tr10:55:141
136146099149,4cyclictest1599964-21apt-get11:00:120
1361460991412,1cyclictest60350irq/132-enp0s29f1:rx-012:35:190
1361460991412,1cyclictest60350irq/132-enp0s29f1:rx-010:10:240
1361460991410,3cyclictest1682257-21kworker/u8:0+flush-259:012:20:200
1361460991410,3cyclictest1682257-21kworker/u8:0+flush-259:012:20:200
1361460991410,3cyclictest1583993-21kworker/u8:3+flush-259:011:19:450
1361460991410,3cyclictest1470336-21kworker/u8:1+events_unbound11:05:120
136147699139,3cyclictest1346122-21kworker/u8:1+flush-259:007:25:163
136147699138,4cyclictest252-21systemd-journal09:14:593
1361476991311,1cyclictest834-21snmpd09:27:423
1361476991311,1cyclictest0-21swapper/311:20:243
1361476991310,2cyclictest1470336-21kworker/u8:1+flush-259:010:40:013
136147199139,3cyclictest503-21tpm2-abrmd09:54:592
136147199139,3cyclictest1428316-21kworker/u8:2+flush-259:008:20:162
1361471991310,2cyclictest1568979-21irqcore10:30:162
136146799139,2cyclictest1532318-21kworker/u8:0+flush-259:010:35:201
136146799136,5cyclictest1631025-21latency_hist11:30:001
136146099139,3cyclictest1495962-21kworker/u8:3+events_unbound09:50:150
1361460991310,2cyclictest1470336-21kworker/u8:1+flush-259:010:20:000
136147699129,2cyclictest1685072-21kthreadcore12:20:183
136147699129,2cyclictest1685072-21kthreadcore12:20:173
136147699129,2cyclictest1635743-21unixbench_singl11:30:273
136147699129,2cyclictest1340559-21kworker/u8:0+events_unbound07:15:143
136147699129,2cyclictest0-21swapper/308:45:203
136147699128,3cyclictest1470336-21kworker/u8:1+events_unbound10:20:193
136147699126,4cyclictest1677345-21latency_hist12:15:013
136147699125,5cyclictest252-21systemd-journal08:16:103
136147199129,2cyclictest1470336-21kworker/u8:1+flush-259:009:20:162
136147199127,4cyclictest1470336-21kworker/u8:1+flush-259:010:56:072
136147199127,3cyclictest1522000-21/usr/sbin/munin09:45:162
136146799129,2cyclictest1616241-21irqcore11:15:151
136146799129,2cyclictest1570836-21kthreadcore10:30:181
136146799129,2cyclictest0-21swapper/112:30:191
136146799129,1cyclictest0-21swapper/108:10:171
136146799128,3cyclictest1589371-21gpg-connect-age10:50:011
136147699117,2cyclictest1424078-21irqcore08:10:183
136147699110,9cyclictest1423170-21awk08:10:003
136147199118,2cyclictest1604886-21kworker/u8:2+flush-259:011:15:192
136147199118,2cyclictest1558342-21cpuspeed_turbos10:20:132
136147199118,2cyclictest1516179-21unixbench_singl09:35:252
136147199118,2cyclictest0-21swapper/208:30:222
136147199117,3cyclictest1610655-21cat11:10:022
136147199117,3cyclictest1521585-21sshd09:43:102
136147199117,2cyclictest1433463-21latency_hist08:20:002
136147199114,5cyclictest1691983-21ntpq12:25:202
136147199111,8cyclictest1583755-21awk10:45:012
136147199111,8cyclictest1506305-21basename09:29:592
136146799119,1cyclictest0-21swapper/109:20:251
136146799118,2cyclictest1412897-21kworker/u8:3+flush-259:008:05:001
136146799115,4cyclictest509-21rs:main1
136146799114,2cyclictest1677355-21munin-run12:14:591
136147699107,2cyclictest1667360-21cpuspeed_turbos12:05:143
136147699107,2cyclictest0-21swapper/307:50:193
136147699104,4cyclictest1564670-21tr10:25:173
136147199108,1cyclictest0-21swapper/208:25:122
136147199107,2cyclictest1563767-21sed10:25:162
136147199106,3cyclictest1702291-21ntp_kernel_pll_12:35:202
136147199100,2cyclictest1547611-21awk10:10:002
136147199100,2cyclictest1547611-21awk10:10:002
13614769997,1cyclictest0-21swapper/312:25:123
13614719996,2cyclictest252-21systemd-journal11:40:002
13614719996,2cyclictest1681615-21ntp_kernel_pll_12:15:212
13614719996,2cyclictest1681615-21ntp_kernel_pll_12:15:212
13614719995,3cyclictest1610140-21gpgv11:10:002
13614719994,3cyclictest1371790-21cat07:20:002
13614769986,1cyclictest0-21swapper/311:40:183
13614719986,1cyclictest0-21swapper/207:25:112
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional