You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-16 - 20:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TI
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot6s.osadl.org (updated Mon Jun 16, 2025 12:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
694740991152,83cyclictest743032-21/usr/sbin/munin11:50:252
694740999310,50cyclictest0-21swapper/209:35:022
69474099892,40cyclictest746411-21/usr/sbin/munin12:10:232
694738998779,5cyclictest22-21rcuc/110:50:011
69474099768,52cyclictest311ktimers/207:40:012
69473599755,47cyclictest25250irq/422-mmc110:50:050
694740997039,25cyclictest0-21swapper/212:25:322
69474099701,57cyclictest0-21swapper/209:05:172
69474099701,57cyclictest0-21swapper/209:05:172
694735997056,11cyclictest15-21rcuc/010:50:010
69473599701,63cyclictest0-21swapper/007:40:110
69474099682,59cyclictest720904-21/usr/sbin/munin09:40:202
69474099681,43cyclictest0-21swapper/208:00:012
69474099671,52cyclictest0-21swapper/211:30:142
69474099671,52cyclictest0-21swapper/211:05:152
69474099671,51cyclictest0-21swapper/207:10:122
69473599671,53cyclictest0-21swapper/012:25:140
69474099661,54cyclictest0-21swapper/208:05:132
69474099652,42cyclictest730069-21latency_hist10:35:012
69474099651,50cyclictest0-21swapper/208:45:182
69474099641,52cyclictest0-21swapper/208:35:162
69474099641,52cyclictest0-21swapper/208:35:162
69474099641,51cyclictest0-21swapper/209:55:152
69474099631,47cyclictest0-21swapper/208:25:132
69474099621,47cyclictest0-21swapper/208:30:132
69474099621,47cyclictest0-21swapper/207:25:122
69474099611,48cyclictest0-21swapper/211:45:202
694738996112,16cyclictest718279-21/usr/sbin/munin09:25:211
69474099602,46cyclictest0-21swapper/208:40:142
69474099601,48cyclictest0-21swapper/212:35:142
69474099601,46cyclictest0-21swapper/211:55:162
69474099601,46cyclictest0-21swapper/211:55:162
69474099593,25cyclictest0-21swapper/210:20:022
694740995921,34cyclictest0-21swapper/212:00:312
69474099591,46cyclictest0-21swapper/211:00:152
69474099591,46cyclictest0-21swapper/207:35:122
69474099584,45cyclictest696323-21/usr/sbin/munin07:15:252
69474099575,38cyclictest0-21swapper/207:45:162
69474099572,46cyclictest707262-21/usr/sbin/munin08:20:192
69474099572,46cyclictest707262-21/usr/sbin/munin08:20:192
69474099571,49cyclictest721745-21/usr/sbin/munin09:45:112
69474099571,49cyclictest721745-21/usr/sbin/munin09:45:112
69474099571,48cyclictest718282-21apt-get09:25:182
69474099571,45cyclictest0-21swapper/210:20:152
69474099571,45cyclictest0-21swapper/209:00:182
69474099571,45cyclictest0-21swapper/209:00:172
694735995715,37cyclictest131ktimers/011:35:190
69474099562,47cyclictest703824-21apt-get08:00:152
69474099562,45cyclictest702185-21ssh07:50:192
69474099561,48cyclictest713710-21/usr/sbin/munin08:55:322
69474099561,45cyclictest0-21swapper/210:50:152
69474099561,26cyclictest717213-21cron09:20:002
69474099561,22cyclictest0-21swapper/209:25:012
69474099561,22cyclictest0-21swapper/209:25:012
69474099551,48cyclictest739431-21sshd11:25:352
69474099551,47cyclictest722809-21/usr/sbin/munin09:50:252
69474099551,47cyclictest706403-21/usr/sbin/munin08:15:172
69474099551,46cyclictest696893-21apt-get07:20:142
69474099551,45cyclictest724843-21/usr/sbin/munin10:00:322
69474099551,41cyclictest738536-21/usr/sbin/munin11:20:322
69474099543,44cyclictest747676-21/usr/sbin/munin12:15:322
69474099542,44cyclictest733649-21apt-get10:55:172
69474099542,44cyclictest733649-21apt-get10:55:172
69474099541,47cyclictest731671-21sshd10:44:352
69474099541,47cyclictest712264-21apt-get08:50:202
69474099541,47cyclictest712264-21apt-get08:50:202
69474099541,46cyclictest750010-21/usr/sbin/munin12:30:262
69474099541,46cyclictest730637-21/usr/sbin/munin10:35:342
69474099541,45cyclictest745575-21apt-get12:05:152
69474099541,45cyclictest741837-21/usr/sbin/munin11:40:342
69474099541,45cyclictest741837-21/usr/sbin/munin11:40:342
69474099541,45cyclictest698734-21/usr/sbin/munin07:30:192
69474099541,44cyclictest0-21swapper/210:10:152
69474099541,43cyclictest736531-21/usr/sbin/munin11:10:282
69474099531,46cyclictest729134-21sshd10:29:352
69474099531,46cyclictest0-21swapper/208:10:252
694738995340,9cyclictest696678-21sendmail07:20:001
69474099523,41cyclictest748222-21/usr/sbin/munin12:20:212
69474099522,43cyclictest740796-21memory11:35:272
69474099521,45cyclictest715734-21/usr/sbin/munin09:10:152
69474099521,44cyclictest737371-21/usr/sbin/munin11:15:242
694738995240,8cyclictest744514-21sendmail12:00:011
694738995240,8cyclictest744514-21sendmail12:00:011
69474099511,42cyclictest725126-21apt-get10:05:142
69474099501,43cyclictest731950-21/usr/sbin/munin10:45:202
694735995043,5cyclictest736076-21munin-plugin-st11:09:590
69474099491,45cyclictest720006-21/usr/sbin/munin09:35:192
694738994921,21cyclictest0-21swapper/110:05:011
69473599493,10cyclictest0-21swapper/010:15:010
69473599486,16cyclictest0-21swapper/007:15:010
694735994837,6cyclictest0-21swapper/011:25:300
69473599471,29cyclictest0-21swapper/010:40:010
694738994633,8cyclictest715451-21grep09:05:341
694738994633,8cyclictest715451-21grep09:05:341
694738994633,4cyclictest0-21swapper/110:54:591
694738994630,9cyclictest705265-21grep08:05:301
694735994637,4cyclictest20750irq/251-8000000.ethernet11:20:340
69473599461,25cyclictest0-21swapper/007:10:000
69473599452,40cyclictest716211-21grep09:10:320
694738994427,11cyclictest231ktimers/112:30:181
69473599448,5cyclictest27250irq/343-wl18xx10:05:300
694738994335,5cyclictest712064-21munin-plugin-st08:50:001
694738994332,7cyclictest719705-21grep09:30:341
694738994331,8cyclictest710366-21apt08:40:001
694738994331,8cyclictest710366-21apt08:40:001
694738994324,12cyclictest0-21swapper/111:35:281
694735994337,4cyclictest0-21swapper/009:30:200
694735994324,15cyclictest720405-21sendmail_mailqu09:35:290
69473599431,9cyclictest0-21swapper/012:40:010
694738994234,5cyclictest740297-21munin-plugin-st11:35:001
694738994227,8cyclictest0-21swapper/111:40:331
694738994227,8cyclictest0-21swapper/111:40:331
694738994223,9cyclictest0-21swapper/110:55:291
694738994223,9cyclictest0-21swapper/110:55:291
694735994234,5cyclictest20950irq/209-8000000.ethernet08:20:190
694735994234,5cyclictest20950irq/209-8000000.ethernet08:20:190
69473599422,34cyclictest0-21swapper/012:20:000
69473599421,5cyclictest0-21swapper/012:00:020
69473599421,5cyclictest0-21swapper/012:00:010
69473899413,3cyclictest0-21swapper/112:20:001
694738994133,4cyclictest717992-21unixbench_multi09:20:311
694738994133,4cyclictest717992-21unixbench_multi09:20:311
694738994133,4cyclictest168350irq/127-8000000.ethernet-tx110:25:281
694738994132,5cyclictest0-21swapper/111:10:341
694735994132,5cyclictest0-21swapper/010:30:300
694735994131,3cyclictest731641-21unixbench_singl10:40:300
69473599411,38cyclictest0-21swapper/009:45:000
694738994024,9cyclictest0-21swapper/110:40:281
69473899401,30cyclictest0-21swapper/108:25:231
694735994031,5cyclictest20950irq/209-8000000.ethernet12:05:320
694735994030,7cyclictest0-21swapper/008:40:010
694735994030,7cyclictest0-21swapper/008:40:000
694735994020,15cyclictest131ktimers/011:40:170
694735994020,15cyclictest131ktimers/011:40:170
69473599401,33cyclictest0-21swapper/010:15:160
69473899397,26cyclictest749287-21phc2sys12:25:281
694738993918,15cyclictest0-21swapper/111:25:001
694735993933,4cyclictest717221-21cat09:20:010
694735993931,5cyclictest20950irq/209-8000000.ethernet09:45:200
694735993931,5cyclictest20950irq/209-8000000.ethernet09:45:200
69473599392,3cyclictest715192-21meminfo09:05:290
69473599392,3cyclictest715192-21meminfo09:05:290
69474099381,30cyclictest694851-21latency_hist07:10:012
69473899384,4cyclictest0-21swapper/107:45:011
694738993827,7cyclictest707811-21grep08:20:311
694738993827,7cyclictest707811-21grep08:20:311
694738993824,10cyclictest0-21swapper/111:00:301
694735993831,5cyclictest734298-21sendmail11:00:000
694735993831,5cyclictest734298-21sendmail11:00:000
694735993831,4cyclictest20950irq/209-8000000.ethernet08:25:180
694735993830,4cyclictest0-21swapper/008:59:590
694735993828,7cyclictest699585-21apt-get07:35:050
694735993826,8cyclictest748656-21grep12:20:280
694735993824,8cyclictest742724-21grep11:45:330
69473599381,35cyclictest0-21swapper/012:00:320
69473599381,35cyclictest0-21swapper/011:10:180
69473899374,6cyclictest0-21swapper/108:45:011
694738993731,3cyclictest0-21swapper/112:35:301
694738993728,6cyclictest714503-21grep09:00:331
694738993728,6cyclictest714503-21grep09:00:331
694738993727,5cyclictest0-21swapper/110:00:231
694738993725,4cyclictest0-21swapper/107:55:011
694738993724,7cyclictest704422-21unixbench_singl08:00:291
694738993723,10cyclictest231ktimers/111:15:181
69473899372,29cyclictest709461-21unixbench_singl08:30:301
694738993722,8cyclictest0-21swapper/111:25:311
69473899372,28cyclictest0-21swapper/111:05:241
694735993727,6cyclictest0-21swapper/008:00:300
694735993724,9cyclictest0-21swapper/009:00:330
694735993724,9cyclictest0-21swapper/009:00:330
69473599372,32cyclictest0-21swapper/009:25:210
694735993722,9cyclictest0-21swapper/011:50:350
69473599371,4cyclictest0-21swapper/008:15:020
69473599371,34cyclictest0-21swapper/010:00:240
69473599371,2cyclictest0-21swapper/007:50:300
694738993630,3cyclictest168350irq/127-8000000.ethernet-tx110:35:321
694738993629,3cyclictest168350irq/127-8000000.ethernet-tx107:35:291
694738993628,4cyclictest168350irq/127-8000000.ethernet-tx110:20:281
694738993624,4cyclictest0-21swapper/107:55:201
69473899362,32cyclictest0-21swapper/109:55:291
69473899362,29cyclictest0-21swapper/107:20:261
69473899361,7cyclictest0-21swapper/109:00:001
694735993629,4cyclictest20950irq/209-8000000.ethernet08:05:150
694735993628,5cyclictest0-21swapper/008:10:310
694735993625,7cyclictest0-21swapper/008:50:330
694735993625,7cyclictest0-21swapper/008:50:330
69473599361,33cyclictest0-21swapper/010:25:230
69473599361,2cyclictest0-21swapper/007:45:020
694738993528,3cyclictest701619-21phc2sys07:45:281
694738993519,10cyclictest0-21swapper/107:30:261
69473899351,29cyclictest0-21swapper/110:10:191
69473899351,27cyclictest0-21swapper/112:00:261
694735993528,4cyclictest20950irq/209-8000000.ethernet08:35:000
694735993528,4cyclictest131ktimers/011:30:310
694735993527,4cyclictest0-21swapper/008:45:250
69473899342,29cyclictest729682-21phc2sys-jitter10:30:301
694738993421,8cyclictest712799-21grep08:50:331
694738993421,8cyclictest712799-21grep08:50:331
69473899341,4cyclictest0-21swapper/112:25:011
694735993426,5cyclictest131ktimers/009:50:260
694735993426,4cyclictest0-21swapper/009:55:300
694735993425,6cyclictest703032-21/usr/sbin/munin07:55:170
694738993323,6cyclictest0-21swapper/111:45:341
694738993322,8cyclictest0-21swapper/110:15:291
69473599332,5cyclictest698085-21/usr/sbin/munin07:25:260
694735993325,5cyclictest750126-21phc2sys12:30:290
694735993325,4cyclictest20950irq/209-8000000.ethernet07:15:310
694735993323,7cyclictest711125-21grep08:40:310
694735993321,8cyclictest0-21swapper/007:20:310
69473599331,29cyclictest728131-21mailstats10:20:310
69473899323,20cyclictest705502-21/usr/sbin/munin08:10:211
69473899321,25cyclictest1666-21snmpd11:53:551
694735993226,3cyclictest734604-21grep11:00:180
694735993225,5cyclictest131ktimers/011:15:310
694735993222,5cyclictest0-21swapper/009:20:250
694735993222,5cyclictest0-21swapper/009:20:240
69473599321,3cyclictest0-21swapper/007:30:260
694738993120,8cyclictest720604-21sendmail09:40:001
694738993120,5cyclictest0-21swapper/112:15:011
69473899311,4cyclictest0-21swapper/107:10:001
694735993110,15cyclictest131ktimers/012:10:160
694738993021,4cyclictest0-21swapper/107:10:261
694738993019,6cyclictest722195-21wc09:45:281
694738993019,6cyclictest722195-21wc09:45:271
694738993011,3cyclictest27350irq/343-s-wl18xx09:16:231
694738992918,7cyclictest0-21swapper/107:25:241
694738992820,4cyclictest0-21swapper/109:10:151
694738992813,10cyclictest721454-21grep09:40:301
69473899281,2cyclictest0-21swapper/109:55:001
69473899277,4cyclictest27350irq/343-s-wl18xx08:19:311
694738992715,7cyclictest0-21swapper/112:05:261
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional