You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-11 - 21:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Thu Dec 11, 2025 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1754151994238,3cyclictest1968087-21kworker/u8:1+flush-259:010:50:152
1754155994137,3cyclictest2076964-21/usr/sbin/munin12:20:143
1754151994037,2cyclictest1748957-21kworker/u8:4+flush-259:007:40:012
1754151993929,9cyclictest1895097-21kworker/u8:2+flush-259:010:00:012
1754147993936,2cyclictest1968087-21kworker/u8:1+events_unbound11:05:151
1754155993835,2cyclictest1843089-21kworker/u8:1+flush-259:009:25:173
1754151993734,2cyclictest2004363-21kworker/u8:4+events_unbound11:25:002
1754151993734,2cyclictest1791286-21kworker/u8:1+events_unbound08:10:232
1754151993633,2cyclictest2028892-21kworker/u8:0+events_unbound12:20:202
1754151993633,2cyclictest2028892-21kworker/u8:0+events_unbound11:40:232
1754147993633,2cyclictest1822539-21kworker/u8:3+events_unbound08:25:291
1754147993532,2cyclictest2028892-21kworker/u8:0+events_unbound11:40:131
1754147993531,3cyclictest1874553-21gpgv09:05:011
1754151993330,2cyclictest2066446-21kworker/u8:1+events_unbound12:15:242
1754151993329,3cyclictest2046001-21kworker/u8:4+events_unbound12:05:192
1754151993329,2cyclictest2019621-21kworker/u8:1+events_unbound11:35:192
1754147993330,2cyclictest1947387-21kworker/u8:2+events_unbound10:40:191
1754147993330,2cyclictest1843089-21kworker/u8:1+events_unbound09:45:251
1754151993230,1cyclictest1748957-21kworker/u8:4+events_unbound07:40:142
1754147993228,3cyclictest1807265-21apt-get08:00:121
1754151993128,2cyclictest1832828-21kworker/u8:0+flush-259:008:30:242
1754147993128,2cyclictest1700720-21kworker/u8:2+events_unbound07:10:141
1754147993127,3cyclictest1843089-21kworker/u8:1+flush-259:009:30:001
1754147993127,2cyclictest1957762-21kworker/u8:5+events_unbound10:38:301
1754151993026,3cyclictest1968087-21kworker/u8:1+events_unbound10:55:222
1754147993027,2cyclictest1822539-21kworker/u8:3+events_unbound09:05:241
1754147993027,2cyclictest1780938-21kworker/u8:3+events_unbound07:55:171
1754147993026,2cyclictest1994062-21kworker/u8:3+events_unbound11:25:221
1754155992922,5cyclictest1817378-21latency_hist08:10:013
1754151992926,2cyclictest1916057-21kworker/u8:0+events_unbound09:45:282
1754151992926,2cyclictest1900654-21kworker/u8:3+events_unbound10:30:142
1754151992926,2cyclictest1807232-21kworker/u8:2+events_unbound08:35:182
1754151992925,3cyclictest1759246-21kworker/u8:0+events_unbound07:30:222
1754151992923,4cyclictest2029910-21latency_hist11:35:002
1754151992921,3cyclictest1843089-21kworker/u8:1+events_unbound09:15:222
1754147992926,2cyclictest1874487-21kworker/u8:0+events_unbound09:10:281
1754147992926,2cyclictest1807232-21kworker/u8:2+events_unbound08:35:291
1754147992926,2cyclictest1780287-21apt-get07:35:011
1754147992924,4cyclictest2698871-21systemd-journal08:14:291
1754151992825,2cyclictest2046001-21kworker/u8:4+events_unbound12:10:152
1754151992825,2cyclictest1994062-21kworker/u8:3+flush-259:011:25:232
1754151992824,3cyclictest1843089-21kworker/u8:1+events_unbound09:30:162
1754151992821,2cyclictest1843089-21kworker/u8:1+events_unbound09:35:162
1754147992826,2cyclictest1994062-21kworker/u8:3+events_unbound11:33:301
1754147992825,2cyclictest2046001-21kworker/u8:4+events_unbound11:50:281
1754155992724,2cyclictest1822539-21kworker/u8:3+events_unbound08:45:223
1754155992723,3cyclictest1875102-21/usr/sbin/munin09:05:153
1754155992722,3cyclictest1999016-21latency_hist11:05:003
1754155992721,4cyclictest2698871-21systemd-journal11:50:293
1754151992725,2cyclictest1700720-21kworker/u8:2+events_unbound07:15:252
1754151992721,4cyclictest2056523-21diskstats12:00:152
1754147992725,1cyclictest1994062-21kworker/u8:3+events_unbound11:15:271
1754147992724,2cyclictest2066446-21kworker/u8:1+events_unbound12:18:301
1754147992724,2cyclictest2028892-21kworker/u8:0+events_unbound12:35:251
1754147992724,2cyclictest1843089-21kworker/u8:1+events_unbound09:35:301
1754147992724,2cyclictest1822539-21kworker/u8:3+events_unbound08:40:181
1754147992724,2cyclictest1801429-21gpgv07:55:011
1754155992622,3cyclictest2066446-21kworker/u8:1+flush-259:012:30:233
1754155992622,3cyclictest2019264-21kworker/u8:0+flush-259:011:25:003
1754155992622,3cyclictest1848887-21/usr/sbin/munin08:40:243
1754155992622,3cyclictest1843177-21gpgv08:35:003
1754155992620,4cyclictest1801552-21base6407:55:013
1754151992623,2cyclictest2028892-21kworker/u8:0+events_unbound12:25:192
1754151992623,2cyclictest2004363-21kworker/u8:4+events_unbound11:15:212
1754151992623,2cyclictest1968087-21kworker/u8:1+flush-259:011:05:302
1754151992623,2cyclictest1947387-21kworker/u8:2+events_unbound10:15:212
1754151992623,2cyclictest1843089-21kworker/u8:1+events_unbound09:25:252
1754151992622,3cyclictest1700720-21kworker/u8:2+flush-259:007:20:062
1754151992621,2cyclictest1822731-21kworker/u8:4+events_unbound08:15:162
1754151992620,4cyclictest2698871-21systemd-journal08:00:002
1754147992624,2cyclictest1931506-21kworker/u8:0+events_unbound10:05:141
1754147992624,2cyclictest1900654-21kworker/u8:3+events_unbound09:53:291
1754147992623,2cyclictest2028892-21kworker/u8:0+events_unbound12:25:161
1754147992623,2cyclictest2004363-21kworker/u8:4+events_unbound11:14:311
1754147992623,2cyclictest1822539-21kworker/u8:3+events_unbound08:45:161
1754147992622,3cyclictest2698871-21systemd-journal08:55:311
1754155992522,2cyclictest1759246-21kworker/u8:0+events_unbound07:30:143
1754155992519,4cyclictest1947601-21/usr/sbin/munin10:15:143
1754151992523,2cyclictest1931506-21kworker/u8:0+flush-259:010:20:252
1754151992522,2cyclictest2028892-21kworker/u8:0+flush-259:011:45:152
1754151992522,2cyclictest1994062-21kworker/u8:3+flush-259:011:00:152
1754147992522,2cyclictest1874487-21kworker/u8:0+events_unbound09:20:151
1754147992522,2cyclictest1748957-21kworker/u8:4+events_unbound07:40:221
175414399256,6cyclictest1780978-21apt07:35:120
1754143992520,4cyclictest2071848-21apt-get12:15:120
1754155992420,2cyclictest1931507-21gpgv10:00:013
1754151992421,2cyclictest1874487-21kworker/u8:0+events_unbound09:10:142
1754147992422,2cyclictest1843089-21kworker/u8:1+events_unbound10:13:291
1754147992421,2cyclictest1957762-21kworker/u8:5+events_unbound10:48:311
1754147992421,2cyclictest1780938-21kworker/u8:3+events_unbound07:45:201
175414399246,6cyclictest1864690-21smartctl08:55:200
1754155992320,2cyclictest1748957-21kworker/u8:4+events_unbound07:15:173
1754155992319,3cyclictest1807232-21kworker/u8:2+flush-259:008:15:003
1754155992319,3cyclictest1764605-21kworker/u8:1+flush-259:007:20:193
1754151992321,2cyclictest1947387-21kworker/u8:2+events_unbound10:35:182
1754151992320,2cyclictest1900654-21kworker/u8:3+flush-259:009:40:162
1754151992320,2cyclictest1843089-21kworker/u8:1+flush-259:009:00:152
1754151992320,2cyclictest1700720-21kworker/u8:2+events_unbound07:10:252
1754151992318,4cyclictest1880002-21latency_hist09:10:002
1754147992320,2cyclictest1843089-21kworker/u8:1+events_unbound09:30:231
1754147992320,2cyclictest1807232-21kworker/u8:2+events_unbound08:34:291
1754147992319,3cyclictest1759246-21kworker/u8:0+events_unbound07:25:031
175414399238,3cyclictest2060379-21ntpq12:00:220
1754143992320,2cyclictest1931476-21apt-get09:59:590
1754143992318,4cyclictest2698871-21systemd-journal09:50:000
1754155992218,3cyclictest2028892-21kworker/u8:0+flush-259:011:35:003
1754155992217,4cyclictest1874487-21kworker/u8:0+events_unbound09:20:183
1754151992218,3cyclictest2004373-21/usr/sbin/munin11:10:152
1754147992219,2cyclictest2066446-21kworker/u8:1+events_unbound12:21:311
1754147992219,2cyclictest2046001-21kworker/u8:4+events_unbound12:10:291
175414399226,4cyclictest1987820-21ntpq10:50:210
175414399225,5cyclictest2080985-21ntpq12:20:200
175414399225,5cyclictest1951633-21ntpq10:15:220
1754143992219,2cyclictest2035029-21cron11:40:000
1754143992216,4cyclictest1962930-21latency_hist10:30:010
1754155992114,5cyclictest2698871-21systemd-journal10:24:563
1754151992118,2cyclictest1947387-21kworker/u8:2+events_unbound10:40:202
1754151992118,2cyclictest1854016-21kworker/u8:4+events_unbound08:55:192
1754151992118,2cyclictest1843089-21kworker/u8:1+events_unbound09:20:232
1754151992118,2cyclictest1759246-21kworker/u8:0+events_unbound07:25:242
1754151992117,3cyclictest1791286-21kworker/u8:1+events_unbound07:45:132
1754147992119,2cyclictest1900654-21kworker/u8:3+events_unbound10:30:261
1754147992118,2cyclictest2028892-21kworker/u8:0+events_unbound12:00:291
1754147992118,2cyclictest1806868-21kworker/u8:0+events_unbound08:09:561
175414399215,6cyclictest2039982-21find11:40:240
175414399215,6cyclictest1828794-21cut08:20:150
175414399215,5cyclictest1944352-21cut10:10:170
1754143992117,3cyclictest2019621-21kworker/u8:1+flush-259:011:30:000
1754155992017,2cyclictest1780938-21kworker/u8:3+flush-259:007:55:223
1754155992016,3cyclictest1895097-21kworker/u8:2+events_unbound09:50:153
1754155992016,3cyclictest1807232-21kworker/u8:2+flush-259:008:20:243
1754155992014,4cyclictest2040872-21/usr/sbin/munin11:45:223
1754151992017,2cyclictest2046001-21kworker/u8:4+events_unbound11:50:162
1754151992017,2cyclictest1822539-21kworker/u8:3+flush-259:008:40:222
1754151992017,2cyclictest1822539-21kworker/u8:3+flush-259:008:25:252
1754151992017,2cyclictest1780938-21kworker/u8:3+events_unbound07:50:162
1754147992015,4cyclictest1957682-21apt-get10:24:561
175414399206,7cyclictest1899754-21ntpq09:25:220
175414399206,4cyclictest2019029-21sed11:20:240
175414399205,4cyclictest2086288-21/usr/sbin/munin12:25:270
175414399203,6cyclictest2013378-21taskset11:15:190
1754143992016,3cyclictest1931506-21kworker/u8:0+flush-259:010:05:170
1754143992016,3cyclictest1764605-21kworker/u8:1+events_unbound07:35:010
1754155991916,2cyclictest2097324-21munin-plugin-st12:40:003
1754155991915,3cyclictest1988920-21kworker/u8:2+flush-259:010:55:243
1754151991914,3cyclictest2092411-21/usr/sbin/munin12:35:172
175414799197,2cyclictest0-21swapper/109:15:181
1754147991918,1cyclictest1900654-21kworker/u8:3+flush-259:009:40:271
1754147991916,2cyclictest1994062-21kworker/u8:3+events_unbound11:00:261
1754147991910,1cyclictest191rcu_preempt08:15:191
175414399195,4cyclictest1925482-21ntpq09:50:210
175414399194,9cyclictest1863214-21ntpq08:50:210
175414399194,5cyclictest1814475-21grep08:05:150
175414399194,5cyclictest1799703-21cut07:50:180
175414399194,4cyclictest1854057-21http08:45:130
175414399194,4cyclictest1811524-21sed08:00:240
175414399193,5cyclictest2065600-21ntpq12:05:240
175414399193,5cyclictest1843059-21apt-get08:35:020
175414399192,6cyclictest1824760-21grep08:15:190
175414399192,6cyclictest1755504-21cut07:10:160
175414399192,5cyclictest1791325-21http07:45:140
1754143991917,1cyclictest1822513-21turbostat.cron08:15:010
1754155991816,1cyclictest0-21swapper/308:05:003
1754155991815,2cyclictest2046001-21kworker/u8:4+flush-259:012:05:203
1754155991815,2cyclictest1822731-21kworker/u8:4+flush-259:008:15:263
1754155991814,3cyclictest1947387-21kworker/u8:2+events_unbound10:40:033
1754155991814,3cyclictest1822539-21kworker/u8:3+flush-259:008:25:243
1754155991813,4cyclictest2028892-21kworker/u8:0+events_unbound11:40:173
1754155991812,5cyclictest1868372-21ntpq08:55:253
1754151991815,2cyclictest1806868-21kworker/u8:0+events_unbound08:07:282
1754151991814,3cyclictest1895097-21kworker/u8:2+events_unbound09:50:132
175414799189,1cyclictest191rcu_preempt10:25:251
1754147991815,2cyclictest1759246-21kworker/u8:0+events_unbound07:20:101
1754147991814,3cyclictest1931506-21kworker/u8:0+events_unbound10:16:291
1754147991814,2cyclictest0-21swapper/110:00:001
175414399185,1cyclictest1890599-21apt-get09:20:140
175414399184,4cyclictest2068448-21tr12:10:170
175414399184,4cyclictest1980068-21kthreadcore10:45:180
175414399184,4cyclictest1932116-21gpgv10:00:010
175414399184,4cyclictest1910493-21who09:35:260
175414399184,4cyclictest1765659-21kthreadcore07:20:180
175414399183,5cyclictest1834804-21cut08:25:190
175414399183,5cyclictest1763153-21cut07:15:210
175414399183,4cyclictest1789344-21cut07:40:180
175414399183,10cyclictest1993488-21taskset10:55:540
175414399182,6cyclictest2053417-21cat11:55:190
175414399182,6cyclictest1802799-21cut07:55:190
175414399182,4cyclictest1887803-21grep09:15:180
1754155991714,2cyclictest1968087-21kworker/u8:1+events_unbound11:05:233
1754155991712,3cyclictest2028892-21kworker/u8:0+events_unbound12:00:173
1754151991714,2cyclictest1947387-21kworker/u8:2+flush-259:010:25:212
1754151991714,2cyclictest1807232-21kworker/u8:2+events_unbound08:00:132
1754151991712,2cyclictest2028892-21kworker/u8:0+events_unbound12:30:142
1754147991715,2cyclictest2019621-21kworker/u8:1+events_unbound11:39:551
1754147991715,2cyclictest1748957-21kworker/u8:4+events_unbound07:35:271
1754147991714,2cyclictest2046001-21kworker/u8:4+events_unbound12:09:551
1754147991714,2cyclictest1968087-21kworker/u8:1+events_unbound10:50:151
175414399175,4cyclictest2087247-21find12:30:110
175414399174,4cyclictest1997791-21kthreadcore11:00:190
175414399174,4cyclictest1842491-21cat08:30:250
175414399173,4cyclictest1954365-21grep10:20:180
175414399173,4cyclictest1851372-21kthreadcore08:40:200
175414399173,4cyclictest1772334-21cut07:25:170
175414399172,6cyclictest2004399-21apt-get11:10:120
175414399172,5cyclictest2093822-21grep12:35:200
1754143991714,2cyclictest61450irq/134-enp0s29f1:tx-010:40:160
175415599169,5cyclictest1967144-21ntpq10:30:213
1754155991613,2cyclictest1931506-21kworker/u8:0+flush-259:010:05:213
1754155991612,3cyclictest1968087-21kworker/u8:1+flush-259:010:50:203
1754155991612,3cyclictest1947387-21kworker/u8:2+events_unbound10:35:243
1754155991612,3cyclictest1843089-21kworker/u8:1+events_unbound09:00:153
1754155991611,4cyclictest1947387-21kworker/u8:2+events_unbound10:45:173
1754151991613,2cyclictest1931506-21kworker/u8:0+events_unbound10:09:562
1754151991613,2cyclictest1843089-21kworker/u8:1+events_unbound08:49:562
175415199160,15cyclictest1947389-21unin-run10:15:012
175414799162,3cyclictest32-21ksoftirqd/108:20:151
1754147991613,2cyclictest2046001-21kworker/u8:4+events_unbound11:56:311
1754147991613,2cyclictest1900654-21kworker/u8:3+events_unbound10:00:261
1754147991613,2cyclictest1854016-21kworker/u8:4+events_unbound08:51:281
1754147991612,3cyclictest2066446-21kworker/u8:1+events_unbound12:31:291
1754147991612,3cyclictest2004363-21kworker/u8:4+flush-259:011:20:161
1754147991612,3cyclictest1968087-21kworker/u8:1+events_unbound10:58:171
175414399166,8cyclictest1874640-21cat09:05:000
175414399165,2cyclictest1884318-21ntpq09:10:230
175414399162,5cyclictest1914129-21tr09:40:190
175414399162,4cyclictest2043276-21grep11:45:210
175414399162,4cyclictest2029001-21ld11:30:220
1754155991512,2cyclictest1843089-21kworker/u8:1+flush-259:009:45:163
1754155991512,2cyclictest1807232-21kworker/u8:2+events_unbound08:35:233
1754155991511,3cyclictest1721016-21kworker/u8:3+events_unbound07:10:133
1754151991512,2cyclictest1957762-21kworker/u8:5+events_unbound10:45:512
175414799159,5cyclictest1763553-21ntpq07:15:231
175414399154,9cyclictest19056542sleep009:33:580
1754143991513,1cyclictest60650irq/133-enp0s29f1:rx-011:51:580
1754143991513,1cyclictest60650irq/133-enp0s29f1:rx-011:08:250
1754155991411,2cyclictest2019621-21kworker/u8:1+events_unbound11:35:293
1754155991410,3cyclictest2076295-21unixbench_singl12:15:253
175414799148,2cyclictest191rcu_preempt11:45:201
175414399146,6cyclictest1967409-21/usr/sbin/munin10:30:250
175414399141,4cyclictest1874756-21apt-key09:05:010
175415599139,3cyclictest2046001-21kworker/u8:4+flush-259:012:10:183
175415599138,4cyclictest2004363-21kworker/u8:4+flush-259:011:15:203
175415599128,3cyclictest2004363-21kworker/u8:4+flush-259:011:14:343
175415599128,3cyclictest1900654-21kworker/u8:3+events_unbound09:40:183
175415599128,3cyclictest1791286-21kworker/u8:1+events_unbound07:45:153
175415599128,2cyclictest1780938-21kworker/u8:3+events_unbound07:35:193
1754151991210,2cyclictest1900654-21kworker/u8:3+flush-259:010:00:182
175415599118,2cyclictest1942768-21fschecks_time10:10:143
175415599116,4cyclictest1854016-21kworker/u8:4+events_unbound08:50:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional