You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-18 - 21:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Tue Nov 18, 2025 12:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2264581994940,6cyclictest2379626-21idleruntime-cro12:40:001
2264576994839,6cyclictest2358929-21auditctl11:40:000
2264585994736,7cyclictest2312451-21cut09:25:162
2264581994719,25cyclictest2316686-21unixbench_multi09:35:301
2264581994719,25cyclictest2316686-21unixbench_multi09:35:301
2264581994623,18cyclictest0-21swapper/110:40:201
2264585994534,6cyclictest215-21dbus-daemon10:30:152
2264581994519,23cyclictest0-21swapper/111:35:171
226458199441,9cyclictest22671462cstates07:15:181
226458199441,9cyclictest22671462cstates07:15:171
2264590994334,5cyclictest2349948-21grep11:10:243
226458199431,9cyclictest2375582-21/usr/sbin/munin12:25:251
226458199431,9cyclictest2375582-21/usr/sbin/munin12:25:251
226458199431,11cyclictest2283832-21cron08:02:011
2264581994214,24cyclictest0-21swapper/109:00:001
2264581994214,24cyclictest0-21swapper/109:00:001
2264585994133,5cyclictest2379617-21munin-plugin-st12:40:002
226458199411,9cyclictest2365055-21/usr/sbin/munin11:55:251
226458199411,9cyclictest2284634-21/usr/sbin/munin08:05:191
226458199411,8cyclictest2279199-21/usr/sbin/munin07:50:181
226458199411,10cyclictest2363958-21cron11:54:001
226458199411,10cyclictest2288505-21/usr/sbin/munin08:15:241
226458199411,10cyclictest2288505-21/usr/sbin/munin08:15:241
2264585994032,4cyclictest0-21swapper/212:00:002
2264581994028,8cyclictest440-21cron08:30:001
226458199401,9cyclictest2335824-21tr10:30:221
226458199401,9cyclictest2330865-21cron10:16:011
226458199401,9cyclictest2307487-21/usr/sbin/munin09:10:211
2264581994011,26cyclictest2355648-21date11:30:011
2264581994011,25cyclictest0-21swapper/109:20:271
2264576994032,6cyclictest121ktimers/012:35:170
2264590993931,5cyclictest2329497-21apt10:15:013
226458199393,6cyclictest2278068-21ntp_states07:45:231
226458199393,10cyclictest2281468-21ntp_states07:55:211
226458199391,9cyclictest2360992-21/usr/sbin/munin11:45:161
226458199391,9cyclictest2360475-21cron11:43:001
226458199391,9cyclictest2360475-21cron11:43:001
226458199391,8cyclictest2343138-21/usr/sbin/munin10:50:271
2264581993912,24cyclictest2371323-21fschecks_time12:15:171
2264581993912,23cyclictest0-21swapper/111:30:151
226458199391,10cyclictest2265304-21/usr/sbin/munin07:10:251
2264590993830,4cyclictest0-21swapper/311:35:263
226458199389,5cyclictest2349699-21/usr/sbin/munin11:10:241
226458199389,3cyclictest0-21swapper/109:45:261
226458199383,7cyclictest630-21snmpd10:28:401
226458199381,9cyclictest2366088-21/usr/sbin/munin12:00:151
226458199381,9cyclictest2340866-21memory10:45:221
226458199381,9cyclictest2312829-21ntp_states09:25:201
2264581993817,17cyclictest1-21systemd12:30:311
2264581993816,18cyclictest0-21swapper/111:00:231
226458199381,5cyclictest2332779-21seq10:20:431
2264576993830,5cyclictest1-21systemd08:15:020
2264576993830,5cyclictest1-21systemd08:15:010
226458199378,3cyclictest0-21swapper/108:40:281
226458199372,9cyclictest2314540-21ntp_states09:30:251
226458199372,9cyclictest2314540-21ntp_states09:30:251
226458199371,9cyclictest2368270-21ntp_states12:05:231
226458199371,9cyclictest2352476-21diskstats11:20:161
226458199371,9cyclictest2328221-21/usr/sbin/munin10:10:181
226458199371,9cyclictest2268886-21diskstats07:20:151
226458199371,8cyclictest2298765-21ntp_states08:45:231
226458199371,8cyclictest2290360-21which08:20:241
226458199371,8cyclictest2276195-21/usr/sbin/munin07:40:211
2264581993716,18cyclictest2294127-21unixbench_singl08:30:261
226458199371,10cyclictest2270605-21fschecks_count07:25:171
2264576993729,4cyclictest0-21swapper/012:05:300
2264576993729,4cyclictest0-21swapper/008:50:010
2264590993624,7cyclictest0-21swapper/308:15:283
2264590993624,7cyclictest0-21swapper/308:15:283
2264585993624,5cyclictest2276627-21grep07:40:242
226458199363,5cyclictest2286146-21diskstats08:10:171
226458199362,9cyclictest2274323-21/usr/sbin/munin07:35:211
226458199361,9cyclictest2326700-21ntp_states10:05:231
226458199361,8cyclictest2323058-21ntp_states09:55:221
226458199361,8cyclictest2305389-21fschecks_count09:05:181
226458199361,7cyclictest2347421-21grep11:05:191
226458199361,7cyclictest2345280-21sh11:00:001
226458199361,7cyclictest2345280-21sh11:00:001
226458199361,7cyclictest2304510-21grep09:00:261
226458199361,7cyclictest2300929-21unixbench_singl08:50:291
226458199361,6cyclictest2370037-21uname12:10:241
226458199361,10cyclictest2294768-21/usr/sbin/munin08:35:171
2264576993630,3cyclictest0-21swapper/010:40:000
2264576993626,6cyclictest2318399-21grep09:40:270
2264576993626,6cyclictest2318399-21grep09:40:270
2264590993527,4cyclictest0-21swapper/308:00:013
2264585993528,4cyclictest2347009-21cut11:05:002
2264585993527,4cyclictest0-21swapper/208:10:182
2264585993524,7cyclictest0-21swapper/209:45:182
226458199359,5cyclictest231ktimers/107:30:141
226458199355,4cyclictest2337179-21fschecks_time10:35:181
226458199352,7cyclictest630-21snmpd12:23:051
226458199351,9cyclictest2310239-21taskset09:19:161
226458199351,8cyclictest2320786-21/usr/sbin/munin09:50:151
226458199351,6cyclictest2350847-21fschecks_time11:15:171
226458199351,5cyclictest2317599-21grep09:40:181
226458199351,5cyclictest2317599-21grep09:40:181
2264576993529,3cyclictest0-21swapper/009:55:010
2264576993525,6cyclictest2334434-21grep10:25:260
2264590993427,4cyclictest2331211-21munin-plugin-st10:20:013
2264590993426,4cyclictest0-21swapper/311:55:003
2264590993426,4cyclictest0-21swapper/307:30:163
2264585993426,4cyclictest0-21swapper/209:20:172
226458199341,8cyclictest2324546-21diskstats10:00:171
2264576993427,4cyclictest0-21swapper/012:20:010
2264590993325,4cyclictest0-21swapper/312:25:263
2264590993325,4cyclictest0-21swapper/312:25:263
2264590993325,4cyclictest0-21swapper/311:20:273
2264590993323,5cyclictest0-21swapper/312:10:173
2264590993322,7cyclictest0-21swapper/308:15:013
2264585993325,4cyclictest0-21swapper/209:20:012
2264585993324,5cyclictest0-21swapper/210:15:182
2264576993322,7cyclictest2365284-21grep11:55:260
2264590993225,4cyclictest0-21swapper/312:15:273
2264590993225,3cyclictest0-21swapper/311:15:273
2264590993223,5cyclictest0-21swapper/307:20:133
2264585993222,6cyclictest2353822-21sh11:25:002
2264585993222,6cyclictest0-21swapper/210:00:272
2264585993222,6cyclictest0-21swapper/208:00:282
2264585993218,10cyclictest2268534-21tr07:20:002
2264585993218,10cyclictest2268534-21tr07:20:002
2264576993227,3cyclictest0-21swapper/009:20:130
2264576993225,4cyclictest2270254-21idleruntime-cro07:25:000
2264576993224,4cyclictest0-21swapper/010:10:280
2264590993123,4cyclictest0-21swapper/309:15:273
2264585993120,7cyclictest2308210-21grep09:10:282
2264576993123,5cyclictest0-21swapper/011:00:000
2264576993123,5cyclictest0-21swapper/011:00:000
2264576993121,7cyclictest2352129-21sendmail11:20:010
2264590993024,3cyclictest0-21swapper/308:20:263
2264590993023,4cyclictest0-21swapper/309:05:273
2264590993022,4cyclictest0-21swapper/311:00:003
2264590993022,4cyclictest0-21swapper/311:00:003
2264585993023,4cyclictest440-21cron11:40:002
2264585993019,7cyclictest0-21swapper/212:15:272
2264585993013,5cyclictest131rcu_preempt11:50:142
2264576993022,4cyclictest0-21swapper/010:20:260
2264576993020,6cyclictest2287485-21sh08:15:010
2264590992923,3cyclictest0-21swapper/308:55:013
2264590992922,3cyclictest0-21swapper/312:00:173
2264590992921,4cyclictest0-21swapper/312:35:293
2264590992921,4cyclictest0-21swapper/311:05:303
2264590992921,4cyclictest0-21swapper/310:40:203
2264590992917,8cyclictest391ktimers/310:30:143
2264585992921,4cyclictest0-21swapper/212:20:272
2264585992921,4cyclictest0-21swapper/209:00:002
2264585992921,4cyclictest0-21swapper/209:00:002
2264585992920,6cyclictest0-21swapper/212:30:302
2264585992920,5cyclictest0-21swapper/211:40:272
2264585992920,5cyclictest0-21swapper/211:40:272
2264585992919,7cyclictest311ktimers/207:50:112
2264585992917,9cyclictest440-21cron12:30:002
2264585992917,9cyclictest440-21cron12:30:002
2264585992917,7cyclictest0-21swapper/207:20:252
2264576992924,3cyclictest2282380-21cron08:00:010
2264576992921,4cyclictest0-21swapper/010:45:170
2264576992920,4cyclictest0-21swapper/010:30:270
2264590992820,4cyclictest0-21swapper/310:50:273
2264590992820,4cyclictest0-21swapper/310:45:173
2264590992818,6cyclictest0-21swapper/310:10:013
226458599289,4cyclictest22921322expr08:25:272
2264585992820,5cyclictest0-21swapper/211:05:192
2264585992820,4cyclictest0-21swapper/208:15:192
2264585992820,4cyclictest0-21swapper/208:15:192
2264585992819,6cyclictest2304243-21grep09:00:252
2264585992819,4cyclictest0-21swapper/209:55:012
2264576992822,3cyclictest0-21swapper/009:00:000
2264576992822,3cyclictest0-21swapper/009:00:000
2264576992821,3cyclictest0-21swapper/008:50:290
2264576992821,3cyclictest0-21swapper/007:45:010
2264576992818,7cyclictest121ktimers/009:35:160
2264576992818,7cyclictest121ktimers/009:35:160
2264576992817,7cyclictest0-21swapper/011:20:270
2264590992719,4cyclictest0-21swapper/310:20:263
2264590992719,4cyclictest0-21swapper/309:10:273
2264590992718,5cyclictest0-21swapper/307:39:593
2264590992715,8cyclictest2297493-21grep08:40:283
2264585992720,4cyclictest311ktimers/207:32:452
2264585992720,4cyclictest0-21swapper/210:45:282
2264585992718,5cyclictest0-21swapper/211:45:282
2264585992716,7cyclictest2271937-21auditctl07:30:002
2264585992715,8cyclictest2327582-21sh10:10:002
2264576992721,3cyclictest0-21swapper/011:15:010
2264576992720,3cyclictest0-21swapper/011:45:280
2264576992719,5cyclictest0-21swapper/008:40:280
2264576992719,5cyclictest0-21swapper/007:10:300
2264576992718,6cyclictest0-21swapper/011:10:000
2264576992717,6cyclictest0-21swapper/010:00:010
2264590992619,4cyclictest2347005-21idleruntime-cro11:05:003
2264590992618,5cyclictest0-21swapper/309:25:253
2264590992616,6cyclictest0-21swapper/309:00:003
2264590992616,6cyclictest0-21swapper/309:00:003
2264585992619,4cyclictest0-21swapper/207:35:272
2264585992618,5cyclictest0-21swapper/211:30:242
2264585992618,4cyclictest0-21swapper/207:45:162
2264585992616,7cyclictest2294066-21grep08:30:262
2264585992613,8cyclictest2324176-21gpgv10:00:012
226457699261,7cyclictest2327622-21sed10:10:000
2264576992616,7cyclictest2325890-21ls10:05:000
2264590992521,2cyclictest0-21swapper/310:25:143
2264590992519,4cyclictest2284853-21ntp_kernel_err08:05:233
2264590992519,3cyclictest0-21swapper/310:35:203
2264590992519,3cyclictest0-21swapper/307:50:143
2264590992518,4cyclictest0-21swapper/307:20:003
2264590992518,4cyclictest0-21swapper/307:20:003
2264590992518,3cyclictest0-21swapper/308:25:293
2264590992517,5cyclictest0-21swapper/311:55:263
2264590992516,6cyclictest2324003-21grep10:00:013
2264590992516,5cyclictest0-21swapper/309:20:273
2264590992515,7cyclictest2317152-21sendmail09:40:003
2264590992515,7cyclictest2317152-21sendmail09:40:003
2264585992519,3cyclictest0-21swapper/209:35:302
2264585992519,3cyclictest0-21swapper/209:35:292
2264585992518,3cyclictest0-21swapper/209:10:002
2264585992517,5cyclictest0-21swapper/211:15:272
2264585992516,6cyclictest0-21swapper/208:55:012
2264585992515,7cyclictest2317591-21grep09:40:182
2264585992515,7cyclictest2317591-21grep09:40:182
2264585992512,8cyclictest215-21dbus-daemon12:05:282
2264576992518,4cyclictest2356781-21tail11:30:240
2264576992517,5cyclictest0-21swapper/010:40:190
226457699251,4cyclictest2284902-21ntp_states08:05:240
2264576992514,8cyclictest2284121-21cat08:05:010
2264590992417,3cyclictest0-21swapper/311:30:003
2264590992416,4cyclictest0-21swapper/307:25:263
2264590992415,6cyclictest2294865-21ls08:35:163
2264590992412,9cyclictest1747-21packagekitd07:45:053
226459099241,12cyclictest637-21runrttasks11:33:013
2264585992417,3cyclictest0-21swapper/211:10:202
2264585992415,7cyclictest311ktimers/210:10:052
2264585992415,6cyclictest2282384-21cut08:00:012
2264585992412,8cyclictest2296251-21sendmail08:40:012
2264576992417,3cyclictest0-21swapper/012:27:180
2264576992417,3cyclictest0-21swapper/012:27:170
226457699241,19cyclictest0-21swapper/007:39:170
226457699241,10cyclictest2306704-21awk09:09:590
2264590992317,3cyclictest0-21swapper/312:05:303
2264590992317,3cyclictest0-21swapper/309:40:183
2264590992317,3cyclictest0-21swapper/309:40:183
2264590992317,3cyclictest0-21swapper/309:30:193
2264590992317,3cyclictest0-21swapper/309:30:183
2264590992314,5cyclictest0-21swapper/312:30:233
2264590992313,7cyclictest2324408-21seq10:00:073
2264590992312,8cyclictest2319820-21cat09:45:263
2264585992317,3cyclictest0-21swapper/211:25:232
2264585992316,5cyclictest2360526-21kworker/u8:2+events_unbound12:14:292
2264585992316,4cyclictest0-21swapper/210:55:312
2264585992316,4cyclictest0-21swapper/210:55:302
2264585992315,4cyclictest0-21swapper/210:50:182
2264585992314,4cyclictest440-21cron09:35:002
2264585992314,4cyclictest440-21cron09:35:002
2264576992318,3cyclictest0-21swapper/007:25:260
2264576992317,3cyclictest0-21swapper/011:55:000
2264576992317,3cyclictest0-21swapper/007:50:190
2264576992316,3cyclictest0-21swapper/008:29:340
2264576992314,6cyclictest2374059-21grep12:20:270
226457699231,3cyclictest2377919-21munin-run12:35:000
226457699231,3cyclictest0-21swapper/007:45:040
2264576992311,8cyclictest1-21systemd11:40:270
2264576992311,8cyclictest1-21systemd11:40:270
226457699231,18cyclictest0-21swapper/009:17:440
2264590992216,4cyclictest0-21swapper/309:50:323
2264590992216,3cyclictest0-21swapper/311:45:223
2264590992216,3cyclictest0-21swapper/308:30:143
2264590992214,5cyclictest2277050-21cron07:45:013
2264590992214,4cyclictest0-21swapper/307:10:303
2264585992216,3cyclictest0-21swapper/207:10:292
2264585992215,5cyclictest2330825-21kworker/u8:1+events_unbound10:21:042
2264585992214,5cyclictest0-21swapper/210:25:252
2264585992213,6cyclictest2338834-21/usr/sbin/munin10:40:192
226458599220,6cyclictest131rcu_preempt12:00:142
2264576992218,2cyclictest0-21swapper/009:30:260
2264576992218,2cyclictest0-21swapper/009:30:260
2264576992218,2cyclictest0-21swapper/008:30:150
2264576992216,3cyclictest0-21swapper/009:49:590
2264576992216,3cyclictest0-21swapper/008:40:010
2264576992211,8cyclictest2371063-21sed12:15:010
226459099211,11cyclictest0-21swapper/308:01:463
2264585992112,6cyclictest311ktimers/208:20:112
2264585992111,6cyclictest0-21swapper/208:05:282
2264576992114,5cyclictest121ktimers/009:00:140
2264576992113,4cyclictest2342345-21fschecks_time10:50:180
226457699211,18cyclictest0-21swapper/011:27:240
226457699211,18cyclictest0-21swapper/007:16:120
226457699211,18cyclictest0-21swapper/007:16:120
2264576992110,8cyclictest2366110-21df12:00:160
226457699210,19cyclictest0-21swapper/011:01:130
2264590992013,4cyclictest0-21swapper/311:40:173
2264590992013,4cyclictest0-21swapper/311:40:173
2264585992013,5cyclictest311ktimers/210:35:232
226457699201,17cyclictest0-21swapper/007:30:470
2264590991912,4cyclictest2303848-21if_eth109:00:193
2264585991912,5cyclictest2292799-21kworker/u8:2+events_unbound08:40:192
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional