You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-16 - 22:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Sun Aug 16, 2026 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2098997995147,3cyclictest2083638-21kworker/u8:0+flush-259:008:55:013
2098997995146,3cyclictest2333161-21latency_hist10:55:013
2098989994844,3cyclictest2400094-21kworker/u8:2+flush-259:012:05:011
2098989994844,3cyclictest2400094-21kworker/u8:2+flush-259:012:05:001
2098997994239,2cyclictest2181459-21kworker/u8:3+events_unbound09:03:473
2098997994037,2cyclictest2322227-21kworker/u8:3+events_unbound11:12:283
2098997994037,2cyclictest2197743-21kworker/u8:1+events_unbound09:08:473
2098992994037,2cyclictest2358912-21kworker/u8:0+events_unbound11:55:242
2098987994036,3cyclictest2270388-21kworker/u8:2+events_unbound10:05:200
2098997993835,2cyclictest2197743-21kworker/u8:1+events_unbound09:14:283
2098997993733,3cyclictest2280699-21kworker/u8:1+events_unbound10:25:073
2098997993633,2cyclictest2083638-21kworker/u8:0+events_unbound08:33:413
2098987993630,4cyclictest2411280-21cp12:10:010
2098997993532,2cyclictest2140116-21kworker/u8:1+events_unbound08:00:173
2098997993528,5cyclictest335557-21systemd-journal09:20:013
2098992993528,5cyclictest2317595-21latency_hist10:40:012
2098997993431,2cyclictest2140116-21kworker/u8:1+events_unbound08:05:013
2098992993429,3cyclictest2228815-21apt-get09:15:122
2098989993430,3cyclictest2275788-21apt-get10:00:121
2098997993325,5cyclictest335557-21systemd-journal07:20:003
2098992993328,3cyclictest2328172-21/usr/sbin/munin10:50:152
2098992993328,3cyclictest2197743-21kworker/u8:1+flush-259:008:45:212
2098989993327,4cyclictest2270368-21latency_hist09:55:001
2098987993329,3cyclictest2083638-21kworker/u8:0+flush-259:007:55:210
2098997993229,2cyclictest2083638-21kworker/u8:0+events_unbound08:21:273
2098997993225,5cyclictest2426632-21latency_hist12:25:003
2098992993229,2cyclictest2400094-21kworker/u8:2+flush-259:012:10:152
2098992993227,3cyclictest2333386-21kworker/u8:1+flush-259:011:05:252
2098992993226,4cyclictest335557-21systemd-journal09:40:002
2098989993227,4cyclictest2280699-21kworker/u8:1+flush-259:010:20:011
2098987993228,3cyclictest2239538-21kworker/u8:3+flush-259:010:00:000
2098989993127,3cyclictest2358912-21kworker/u8:0+flush-259:011:30:161
2098989993127,3cyclictest2083638-21kworker/u8:0+flush-259:008:25:251
2098989993126,3cyclictest2213084-21latency_hist09:00:001
2098989993125,4cyclictest2109305-21latency_hist07:20:011
2098987993128,2cyclictest2197743-21kworker/u8:1+flush-259:009:00:230
2098987993127,3cyclictest2171321-21kworker/u8:2+flush-259:008:20:200
2098989993024,4cyclictest2442200-21latency_hist12:40:011
2098989993024,4cyclictest2358006-21ntpq11:15:231
2098987993024,4cyclictest2212173-21ntpq08:55:210
2098997992923,4cyclictest2233972-21apt-get09:20:123
2098997992922,5cyclictest2124815-21latency_hist07:35:013
2098992992926,2cyclictest2140116-21kworker/u8:1+events_unbound07:50:232
2098992992925,3cyclictest2312438-21kworker/u8:0+events_unbound10:40:232
2098992992925,3cyclictest2249917-21kworker/u8:0+flush-259:009:40:252
2098992992924,3cyclictest2083638-21kworker/u8:0+events_unbound07:35:192
2098992992922,5cyclictest2186782-21latency_hist08:35:012
2098989992924,3cyclictest2161201-21/usr/sbin/munin08:10:211
2098987992926,2cyclictest2140116-21kworker/u8:1+flush-259:008:10:210
2098987992925,3cyclictest2322227-21kworker/u8:3+flush-259:011:00:190
2098987992923,5cyclictest2380038-21diskstats11:40:110
2098997992825,2cyclictest2427723-21kworker/u8:2+events_unbound12:27:563
2098997992822,4cyclictest509-21rs:main3
2098992992824,3cyclictest2410858-21kworker/u8:1+flush-259:012:30:012
2098992992823,3cyclictest2384721-21kworker/u8:2+flush-259:011:45:252
2098992992823,3cyclictest2280699-21kworker/u8:1+flush-259:010:30:222
2098989992825,2cyclictest2083638-21kworker/u8:0+flush-259:007:55:281
2098987992821,5cyclictest335557-21systemd-journal08:50:010
2098997992724,2cyclictest2333386-21kworker/u8:1+events_unbound11:41:213
2098992992723,3cyclictest2197743-21kworker/u8:1+events_unbound09:05:012
2098992992723,3cyclictest2083638-21kworker/u8:0+events_unbound07:11:272
2098997992623,2cyclictest2389997-21kworker/u8:1+events_unbound11:55:413
2098997992623,2cyclictest2322227-21kworker/u8:3+events_unbound11:15:363
2098997992623,2cyclictest2140116-21kworker/u8:1+events_unbound08:10:303
2098997992620,4cyclictest2322751-21latency_hist10:45:013
2098992992621,4cyclictest2135099-21latency_hist07:45:012
2098992992621,3cyclictest2270388-21kworker/u8:2+flush-259:010:05:212
2098989992623,2cyclictest2280699-21kworker/u8:1+flush-259:010:10:211
2098989992620,4cyclictest509-21rs:main1
2098987992623,2cyclictest2129988-21kworker/u8:3+events_unbound07:45:160
2098997992522,2cyclictest2410858-21kworker/u8:1+events_unbound12:31:143
2098997992519,4cyclictest2405731-21apt-get12:05:123
2098992992521,3cyclictest2083638-21kworker/u8:0+events_unbound08:43:432
2098992992518,5cyclictest2363174-21ntpq11:20:212
2098989992521,3cyclictest2426865-21/usr/sbin/munin12:25:141
2098989992521,3cyclictest2312438-21kworker/u8:0+flush-259:010:35:261
2098989992521,3cyclictest2238913-21gpgv09:25:001
2098987992521,3cyclictest2181459-21kworker/u8:3+flush-259:009:05:250
2098987992520,3cyclictest2410872-21store12:10:000
2098997992421,2cyclictest2427723-21kworker/u8:2+events_unbound12:35:013
2098997992421,2cyclictest2322227-21kworker/u8:3+events_unbound11:49:563
2098997992421,2cyclictest2280699-21kworker/u8:1+events_unbound10:39:563
2098997992421,2cyclictest2181459-21kworker/u8:3+events_unbound08:55:533
2098997992421,2cyclictest2094055-21kworker/u8:1+events_unbound07:12:593
2098997992421,2cyclictest2083638-21kworker/u8:0+events_unbound08:43:333
2098992992421,2cyclictest2400094-21kworker/u8:2+events_unbound12:05:012
2098992992421,2cyclictest2322227-21kworker/u8:3+flush-259:011:25:162
2098992992421,2cyclictest2181459-21kworker/u8:3+flush-259:008:50:252
2098992992421,2cyclictest2094055-21kworker/u8:1+events_unbound07:34:272
2098992992420,3cyclictest2140116-21kworker/u8:1+events_unbound08:17:272
2098992992420,3cyclictest2083638-21kworker/u8:0+flush-259:007:55:162
2098992992420,3cyclictest2083638-21kworker/u8:0+events_unbound08:28:402
2098992992419,4cyclictest2416476-21kworker/u8:0+events_unbound12:39:292
2098992992418,4cyclictest2286068-21/usr/sbin/munin10:10:142
2098989992421,2cyclictest2083638-21kworker/u8:0+flush-259:008:15:201
2098989992419,4cyclictest2359188-21apt-get11:20:121
2098987992420,3cyclictest2181459-21kworker/u8:3+flush-259:008:50:170
2098997992321,2cyclictest2410858-21kworker/u8:1+events_unbound12:14:293
2098997992320,2cyclictest2207892-21kworker/u8:5+events_unbound09:31:203
2098997992320,2cyclictest2094055-21kworker/u8:1+events_unbound07:20:403
2098997992320,2cyclictest2083638-21kworker/u8:0+events_unbound08:37:193
2098997992318,3cyclictest2322227-21kworker/u8:3+flush-259:011:25:173
2098997992317,4cyclictest2421426-21latency_hist12:20:013
2098992992319,3cyclictest2322227-21kworker/u8:3+flush-259:011:50:212
2098992992319,3cyclictest2280699-21kworker/u8:1+events_unbound10:23:272
2098992992319,3cyclictest2239538-21kworker/u8:3+events_unbound10:15:272
2098992992319,3cyclictest2140116-21kworker/u8:1+events_unbound08:08:342
2098992992318,3cyclictest2114728-21apt-get07:25:142
2098989992319,3cyclictest2083638-21kworker/u8:0+flush-259:008:45:011
2098987992319,3cyclictest2301693-21latency_hist10:25:010
2098987992318,3cyclictest335557-21systemd-journal08:03:310
2098997992220,2cyclictest2322227-21kworker/u8:3+events_unbound11:36:473
2098997992220,2cyclictest2239538-21kworker/u8:3+events_unbound09:55:013
2098997992219,2cyclictest2358912-21kworker/u8:0+events_unbound11:53:213
2098997992219,2cyclictest2312438-21kworker/u8:0+events_unbound10:45:023
2098997992219,2cyclictest2083638-21kworker/u8:0+events_unbound07:58:203
2098997992218,3cyclictest509-21rs:main3
2098997992218,3cyclictest2312438-21kworker/u8:0+events_unbound10:59:043
2098997992218,3cyclictest2181459-21kworker/u8:3+events_unbound08:48:033
2098997992218,3cyclictest2083638-21kworker/u8:0+events_unbound08:27:223
2098992992218,3cyclictest2410858-21kworker/u8:1+events_unbound12:15:252
2098992992218,3cyclictest2358912-21kworker/u8:0+events_unbound11:34:262
2098992992218,3cyclictest2333386-21kworker/u8:1+events_unbound11:41:272
2098992992218,3cyclictest2233937-21kworker/u8:2+events_unbound09:20:212
2098992992218,3cyclictest2094055-21kworker/u8:1+flush-259:007:15:152
2098992992218,3cyclictest2094055-21kworker/u8:1+events_unbound07:22:272
2098992992217,3cyclictest2421689-21apt-get12:20:112
2098992992217,3cyclictest2333386-21kworker/u8:1+events_unbound10:55:162
2098992992216,3cyclictest2140116-21kworker/u8:1+events_unbound08:00:192
2098992992210,4cyclictest2218182-21idleruntime-cro09:05:002
2098989992219,2cyclictest2094055-21kworker/u8:1+flush-259:007:20:161
2098989992218,3cyclictest2094055-21kworker/u8:1+events_unbound07:10:191
2098989992217,4cyclictest2223636-21/usr/sbin/munin09:10:141
2098987992219,2cyclictest335557-21systemd-journal07:30:270
2098987992219,2cyclictest2421655-21kworker/u8:4+flush-259:012:20:490
2098997992119,1cyclictest2358912-21kworker/u8:0+events_unbound12:00:113
2098997992119,1cyclictest2358912-21kworker/u8:0+events_unbound12:00:103
2098997992119,1cyclictest2322227-21kworker/u8:3+events_unbound11:22:173
2098997992118,2cyclictest2280699-21kworker/u8:1+events_unbound10:14:283
2098997992118,2cyclictest2270388-21kworker/u8:2+events_unbound10:02:393
2098997992118,2cyclictest2239538-21kworker/u8:3+events_unbound09:45:083
2098997992118,2cyclictest2239538-21kworker/u8:3+events_unbound09:45:083
2098997992118,2cyclictest2104140-21kworker/u8:2+events_unbound07:40:493
2098997992118,2cyclictest2083638-21kworker/u8:0+events_unbound07:54:203
2098997992117,3cyclictest2129988-21kworker/u8:3+events_unbound07:46:583
2098992992118,2cyclictest2322227-21kworker/u8:3+events_unbound11:00:182
2098992992117,3cyclictest2322227-21kworker/u8:3+events_unbound11:17:262
2098992992117,3cyclictest2197743-21kworker/u8:1+events_unbound09:10:222
2098989992117,3cyclictest335557-21systemd-journal09:25:261
2098989992116,4cyclictest335557-21systemd-journal12:21:281
2098987992117,3cyclictest2260060-21kworker/u8:1+flush-259:009:50:180
2098987992116,4cyclictest335557-21systemd-journal11:48:270
2098987992114,5cyclictest2113548-21ntpq07:20:210
2098997992017,2cyclictest2333386-21kworker/u8:1+events_unbound11:31:193
2098997992017,2cyclictest2280699-21kworker/u8:1+events_unbound10:20:343
2098997992017,2cyclictest2249917-21kworker/u8:0+events_unbound09:41:353
2098997992017,2cyclictest2140116-21kworker/u8:1+events_unbound08:19:083
2098997992017,2cyclictest2094055-21kworker/u8:1+events_unbound07:29:353
2098992992016,3cyclictest2239538-21kworker/u8:3+events_unbound10:25:202
2098992992015,3cyclictest2358912-21kworker/u8:0+events_unbound12:00:262
2098992992015,3cyclictest2358912-21kworker/u8:0+events_unbound12:00:252
2098989992017,2cyclictest2333386-21kworker/u8:1+flush-259:010:55:191
2098989992017,2cyclictest2140116-21kworker/u8:1+flush-259:008:00:011
2098989992016,3cyclictest2207892-21kworker/u8:5+flush-259:009:10:001
2098989992015,3cyclictest335557-21systemd-journal11:59:591
2098987992016,3cyclictest2301710-21kworker/u8:2+flush-259:010:46:070
2098987992015,3cyclictest2290982-21latency_hist10:15:010
2098997991916,2cyclictest2322227-21kworker/u8:3+events_unbound11:00:413
2098997991916,2cyclictest2270388-21kworker/u8:2+events_unbound10:06:523
2098997991916,2cyclictest2104140-21kworker/u8:2+events_unbound07:37:023
2098997991915,3cyclictest2260060-21kworker/u8:1+events_unbound09:50:093
2098997991915,3cyclictest2239538-21kworker/u8:3+events_unbound09:28:553
2098992991915,3cyclictest2333386-21kworker/u8:1+flush-259:011:10:222
2098992991915,3cyclictest2207892-21kworker/u8:5+events_unbound09:29:512
2098992991915,3cyclictest2083638-21kworker/u8:0+events_unbound07:48:272
2098992991915,2cyclictest2239538-21kworker/u8:3+events_unbound09:45:252
2098992991915,2cyclictest2239538-21kworker/u8:3+events_unbound09:45:252
2098989991916,2cyclictest2420457-21head12:15:211
2098989991915,2cyclictest2218200-21turbostat.cron09:05:001
2098987991916,2cyclictest2436834-21apt-get12:33:190
2098987991916,2cyclictest2280699-21kworker/u8:1+flush-259:010:15:280
2098987991915,3cyclictest2104140-21kworker/u8:2+events_unbound07:25:200
2098987991915,3cyclictest2094055-21kworker/u8:1+flush-259:007:15:260
2098987991914,4cyclictest509-21rs:main0
2098997991815,2cyclictest2333386-21kworker/u8:1+events_unbound11:07:053
2098997991815,2cyclictest2239538-21kworker/u8:3+events_unbound10:31:183
2098992991816,1cyclictest2301710-21kworker/u8:2+events_unbound10:45:242
2098992991815,2cyclictest2333386-21kworker/u8:1+events_unbound11:35:132
2098992991814,3cyclictest2171321-21kworker/u8:2+flush-259:008:35:112
2098992991812,4cyclictest335557-21systemd-journal09:55:002
2098989991815,2cyclictest2301776-21apt-key10:25:001
2098989991814,3cyclictest2254852-21idleruntime-cro09:40:001
2098989991814,3cyclictest2239538-21kworker/u8:3+flush-259:009:45:261
2098989991814,3cyclictest2239538-21kworker/u8:3+flush-259:009:45:261
2098989991813,3cyclictest2255129-21apt-get09:40:121
2098989991812,4cyclictest335557-21systemd-journal12:07:271
2098987991815,2cyclictest2140924-21irqcore07:50:160
2098987991814,3cyclictest2249926-21/usr/sbin/munin09:35:200
2098987991813,4cyclictest2244508-21latency_hist09:30:000
2098992991713,3cyclictest2239538-21kworker/u8:3+events_unbound09:30:172
2098992991713,3cyclictest2197743-21kworker/u8:1+events_unbound08:55:192
2098989991714,2cyclictest2358912-21kworker/u8:0+flush-259:011:45:151
2098989991714,2cyclictest2280699-21kworker/u8:1+flush-259:010:40:181
2098989991713,3cyclictest2140116-21kworker/u8:1+flush-259:008:05:161
2098989991711,4cyclictest335557-21systemd-journal11:55:001
2098987991715,1cyclictest2358910-21cron11:19:590
2098987991714,2cyclictest60350irq/132-enp0s29f1:rx-012:15:230
2098987991714,2cyclictest2405456-21idleruntime-cro12:05:010
2098987991714,2cyclictest2405456-21idleruntime-cro12:05:000
2098989991614,1cyclictest0-21swapper/112:15:011
2098987991614,1cyclictest60350irq/132-enp0s29f1:rx-010:57:490
2098987991614,1cyclictest60350irq/132-enp0s29f1:rx-010:41:280
2098987991614,1cyclictest0-21swapper/009:25:000
2098987991613,2cyclictest60350irq/132-enp0s29f1:rx-012:35:130
2098987991613,2cyclictest60350irq/132-enp0s29f1:rx-011:35:250
2098987991613,2cyclictest60350irq/132-enp0s29f1:rx-008:15:290
2098987991613,2cyclictest60350irq/132-enp0s29f1:rx-008:07:550
2098987991613,2cyclictest2280699-21kworker/u8:1+flush-259:010:25:260
2098987991613,2cyclictest2227540-21needreboot09:10:200
2098992991512,2cyclictest509-21rs:main2
2098992991510,4cyclictest2239538-21kworker/u8:3+events_unbound10:04:282
2098989991512,2cyclictest2197743-21kworker/u8:1+flush-259:009:15:171
2098987991513,1cyclictest60350irq/132-enp0s29f1:rx-008:25:160
2098987991512,2cyclictest60350irq/132-enp0s29f1:rx-012:28:280
2098987991512,2cyclictest60350irq/132-enp0s29f1:rx-011:25:020
2098987991512,2cyclictest60350irq/132-enp0s29f1:rx-009:30:200
2098987991512,2cyclictest2276210-21irqcore10:00:170
2098987991510,4cyclictest335557-21systemd-journal10:36:260
2098992991411,2cyclictest2171321-21kworker/u8:2+events_unbound08:23:222
2098989991411,2cyclictest2171321-21kworker/u8:2+flush-259:008:35:011
2098989991411,2cyclictest2171321-21kworker/u8:2+flush-259:008:20:271
2098989991410,3cyclictest2322227-21kworker/u8:3+flush-259:011:26:011
2098987991412,1cyclictest60350irq/132-enp0s29f1:rx-011:55:250
2098987991412,1cyclictest60350irq/132-enp0s29f1:rx-011:05:290
2098987991411,2cyclictest60350irq/132-enp0s29f1:rx-009:44:430
2098987991411,2cyclictest60350irq/132-enp0s29f1:rx-008:40:130
2098987991411,2cyclictest2260728-21irqcore09:45:190
2098987991411,2cyclictest2260728-21irqcore09:45:180
209899799132,3cyclictest50-21ksoftirqd/307:10:013
209899299139,3cyclictest2239538-21kworker/u8:3+flush-259:009:57:382
209898999139,3cyclictest2379852-21/usr/sbin/munin11:40:181
209898999139,3cyclictest2270609-21/usr/sbin/munin09:55:151
2098989991310,2cyclictest2197843-21cpuspeed_turbos08:45:141
2098987991312,1cyclictest60350irq/132-enp0s29f1:rx-008:37:170
2098987991311,1cyclictest60350irq/132-enp0s29f1:rx-011:20:160
2098987991311,1cyclictest60350irq/132-enp0s29f1:rx-011:10:530
2098987991311,1cyclictest60350irq/132-enp0s29f1:rx-008:33:260
2098987991311,1cyclictest60350irq/132-enp0s29f1:rx-007:11:270
209898999129,2cyclictest2333386-21kworker/u8:1+flush-259:011:00:261
209898999118,2cyclictest2312438-21kworker/u8:0-ext4-rsv-conversion10:55:001
209898999117,3cyclictest335557-21systemd-journal10:10:001
209898999107,2cyclictest2353788-21sh11:15:001
209898999107,2cyclictest2244873-21tr09:30:151
209898999102,1cyclictest191rcu_preempt08:35:211
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional