You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-09 - 07:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TI
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot6s.osadl.org (updated Fri May 09, 2025 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
115955998620,29cyclictest157073-21/usr/sbin/munin23:10:191
11595599781,58cyclictest0-21swapper/120:25:191
115957997711,32cyclictest129033-21/usr/sbin/munin20:25:202
115957997312,38cyclictest133199-21gpg-connect-age20:50:012
115957997312,38cyclictest133199-21gpg-connect-age20:50:012
11595799729,55cyclictest0-21swapper/222:25:012
11595599642,32cyclictest119768-21/usr/sbin/munin19:30:191
115957996136,18cyclictest0-21swapper/222:50:132
115957996131,16cyclictest0-21swapper/223:10:142
115957995931,14cyclictest0-21swapper/223:55:142
115957995846,8cyclictest156540-21/usr/sbin/munin23:05:262
115957995838,14cyclictest0-21swapper/223:45:152
115957995838,12cyclictest141843-21apt-get21:40:162
115957995837,15cyclictest0-21swapper/220:50:122
115957995831,15cyclictest0-21swapper/200:15:172
115957995831,15cyclictest0-21swapper/200:15:172
11595799574,30cyclictest0-21swapper/220:15:012
115957995730,14cyclictest0-21swapper/223:00:142
115957995637,13cyclictest0-21swapper/223:35:132
115957995539,11cyclictest140152-21apt-get21:30:162
115957995538,12cyclictest151946-21apt-get22:40:172
115957995440,10cyclictest152785-21apt-get22:45:172
115957995434,13cyclictest0-21swapper/223:15:112
115957995432,11cyclictest0-21swapper/223:40:162
11595599541,19cyclictest0-21swapper/122:25:011
115957995338,10cyclictest136849-21diskstats21:10:202
115957995335,11cyclictest0-21swapper/200:25:152
115954995341,9cyclictest138263-21sendmail21:19:590
115954995341,9cyclictest138263-21sendmail21:19:590
115957995234,10cyclictest167425-21/usr/sbin/munin00:10:212
115957995232,14cyclictest0-21swapper/220:05:142
115957995227,13cyclictest0-21swapper/222:30:142
115957995224,17cyclictest0-21swapper/220:00:142
115957995138,5cyclictest128198-21apt-get20:20:142
115957995138,5cyclictest128198-21apt-get20:20:132
115957995137,8cyclictest160955-21/usr/sbin/munin23:30:292
115957995137,8cyclictest158551-21apt-get23:20:072
115957995137,8cyclictest158551-21apt-get23:20:072
115957995134,9cyclictest166203-21/usr/sbin/munin00:00:342
115957995133,11cyclictest138151-21grep21:15:282
115957995133,11cyclictest138151-21grep21:15:282
115954995125,4cyclictest27250irq/343-wl18xx19:50:200
115957995044,3cyclictest123973-21apt-get19:55:122
115957995040,6cyclictest139026-21grep21:20:312
115957995037,7cyclictest144705-21/usr/sbin/munin21:55:252
115957995034,10cyclictest133959-21apt-get20:55:042
115957995034,10cyclictest127772-21/usr/sbin/munin20:15:262
115957995032,11cyclictest164197-21/usr/sbin/munin23:50:232
11595499501,11cyclictest0-21swapper/021:25:010
115957994940,5cyclictest151089-21apt-get22:35:172
115957994933,10cyclictest143587-21/usr/sbin/munin21:50:222
115957994931,11cyclictest148120-21/usr/sbin/munin22:15:282
115957994929,10cyclictest0-21swapper/219:20:102
115957994928,13cyclictest145206-21munin-node22:00:132
115957994928,13cyclictest142760-21/usr/sbin/munin21:45:202
115957994926,17cyclictest130088-21/usr/sbin/munin20:30:232
115957994923,18cyclictest167015-21head00:05:292
115957994920,3cyclictest139615-21memory21:25:312
11595799487,4cyclictest0-21swapper/200:35:022
115957994839,5cyclictest149415-21apt-get22:25:172
115957994839,5cyclictest149415-21apt-get22:25:162
115957994835,7cyclictest131785-21/usr/sbin/munin20:40:222
115957994835,7cyclictest121757-21/usr/sbin/munin19:40:232
115957994830,10cyclictest0-21swapper/219:30:142
115957994829,12cyclictest117720-21/usr/sbin/munin19:15:292
115957994825,14cyclictest147241-21/usr/sbin/munin22:10:272
115957994815,26cyclictest1-21systemd23:25:182
11595599481,26cyclictest0-21swapper/123:55:011
115954994844,2cyclictest0-21swapper/019:30:130
115954994840,4cyclictest0-21swapper/021:35:130
115954994815,20cyclictest131ktimers/022:45:140
115957994741,3cyclictest131312-21/usr/sbin/munin20:35:272
115957994740,3cyclictest123761-21sshd19:54:302
115957994733,7cyclictest154797-21/usr/sbin/munin22:55:282
115957994731,10cyclictest122295-21/usr/sbin/munin19:45:132
115957994731,10cyclictest116426-21/usr/sbin/munin19:10:252
115957994730,12cyclictest0-21swapper/221:05:132
115957994729,11cyclictest141211-21/usr/sbin/munin21:35:222
115957994724,8cyclictest146597-21tail22:05:302
115955994727,11cyclictest140988-21apt-get21:35:171
115957994639,3cyclictest169052-21apt-get00:20:132
115957994639,3cyclictest118935-21apt-get19:25:132
115957994639,3cyclictest118935-21apt-get19:25:132
11595599465,35cyclictest156237-21/usr/sbin/munin23:05:201
115954994639,3cyclictest0-21swapper/021:55:100
115954994633,5cyclictest0-21swapper/023:35:290
115954994613,30cyclictest0-21swapper/000:05:210
115957994538,3cyclictest120660-21ssh19:35:182
115957994537,4cyclictest171563-21apt-get00:35:172
115957994529,12cyclictest0-21swapper/221:00:132
115955994536,6cyclictest0-21swapper/121:20:001
115955994536,6cyclictest0-21swapper/121:20:001
115954994537,5cyclictest134889-21sendmail21:00:000
11595599449,17cyclictest0-21swapper/123:40:011
115955994428,12cyclictest0-21swapper/120:15:011
115954994434,8cyclictest1263-21cron21:05:000
11595499442,6cyclictest0-21swapper/023:20:010
11595499433,9cyclictest0-21swapper/022:20:010
115954994334,6cyclictest20950irq/209-8000000.ethernet22:54:590
115954994333,7cyclictest0-21swapper/000:20:290
115954994333,6cyclictest130466-21grep20:30:300
115955994234,5cyclictest135743-21munin-plugin-st21:04:591
115955994226,10cyclictest0-21swapper/100:35:291
11595499425,6cyclictest0-21swapper/023:25:320
115954994235,4cyclictest20750irq/251-8000000.ethernet22:15:000
115954994233,7cyclictest127988-21sendmail20:20:000
11595499421,4cyclictest128267-21diskstats20:20:160
11595499421,4cyclictest128267-21diskstats20:20:160
11595499421,37cyclictest131ktimers/021:05:280
11595599417,28cyclictest0-21swapper/100:05:231
115955994129,7cyclictest139034-21grep21:20:311
115955994126,8cyclictest0-21swapper/121:25:351
115955994126,11cyclictest0-21swapper/122:15:301
115955994124,11cyclictest116651-21grep19:10:281
115955994123,10cyclictest132909-21/usr/sbin/munin20:45:311
115955994123,10cyclictest132909-21/usr/sbin/munin20:45:311
11595599411,20cyclictest0-21swapper/123:15:181
11595499411,4cyclictest0-21swapper/023:05:130
115955994033,3cyclictest123136-21apt-get19:50:111
115955994028,8cyclictest141960-21ssh21:40:201
115955994028,8cyclictest0-21swapper/123:45:301
115955994027,8cyclictest0-21swapper/121:45:281
115955994027,5cyclictest129868-21/usr/sbin/munin20:30:311
11595499402,3cyclictest148777-21perl22:20:200
11595499401,37cyclictest0-21swapper/020:45:240
11595499401,37cyclictest0-21swapper/020:45:240
115955993924,8cyclictest152956-21lsblk22:45:221
115955993923,9cyclictest121855-21/usr/sbin/munin19:40:251
115955993921,11cyclictest118299-21/usr/sbin/munin19:20:191
115955993920,9cyclictest152050-21/usr/sbin/munin22:40:201
11595599391,2cyclictest0-21swapper/122:40:001
115954993929,6cyclictest0-21swapper/019:10:270
115954993929,5cyclictest139952-21grep21:30:000
115954993929,5cyclictest0-21swapper/023:20:340
115954993929,5cyclictest0-21swapper/023:20:340
115954993925,10cyclictest0-21swapper/019:55:180
11595499392,30cyclictest163865-21ls23:50:010
11595499391,4cyclictest0-21swapper/023:15:010
115954993911,4cyclictest0-21swapper/019:35:170
115957993825,10cyclictest116043-21latency_hist19:10:012
115955993830,4cyclictest0-21swapper/122:50:291
115955993824,8cyclictest0-21swapper/123:00:131
115955993822,8cyclictest168530-21/usr/sbin/munin00:15:291
115955993822,8cyclictest168530-21/usr/sbin/munin00:15:291
115955993819,13cyclictest154671-21/usr/sbin/munin22:55:231
115955993819,12cyclictest167566-21/usr/sbin/munin00:10:241
115955993818,13cyclictest120916-21/usr/sbin/munin19:35:261
115955993818,13cyclictest119133-21perl19:25:231
115955993818,13cyclictest119133-21perl19:25:231
115955993817,9cyclictest0-21swapper/122:00:151
11595499383,29cyclictest152514-21head22:40:290
115954993830,6cyclictest20950irq/209-8000000.ethernet20:05:300
115954993829,5cyclictest0-21swapper/022:05:290
115954993823,11cyclictest0-21swapper/020:35:240
11595499381,3cyclictest143807-21load21:50:270
11595599375,27cyclictest0-21swapper/119:15:261
115955993728,5cyclictest0-21swapper/120:55:311
115955993727,7cyclictest0-21swapper/122:25:301
115955993727,7cyclictest0-21swapper/122:25:301
115955993725,5cyclictest0-21swapper/122:30:281
115955993724,9cyclictest0-21swapper/122:05:301
115955993724,7cyclictest144083-21/usr/sbin/munin21:50:311
115955993723,7cyclictest133453-21/usr/sbin/munin20:50:181
115955993722,8cyclictest0-21swapper/123:40:281
115955993722,7cyclictest122510-21/usr/sbin/munin19:45:241
115955993722,10cyclictest131619-21/usr/sbin/munin20:40:161
115955993718,12cyclictest166026-21/usr/sbin/munin00:00:311
115955993718,12cyclictest124406-21/usr/sbin/munin19:55:271
115955993717,13cyclictest130957-21/usr/sbin/munin20:35:211
11595499373,5cyclictest121996-21grep19:40:270
115954993732,3cyclictest0-21swapper/021:15:000
115954993731,4cyclictest0-21swapper/019:25:140
115954993731,4cyclictest0-21swapper/019:25:140
115954993731,3cyclictest166054-21mailstats00:00:320
115954993727,6cyclictest162380-21tune2fs23:40:220
11595499372,32cyclictest0-21swapper/023:30:280
11595499371,34cyclictest131ktimers/021:40:270
11595499371,31cyclictest0-21swapper/023:55:030
115955993628,5cyclictest0-21swapper/123:30:311
115955993628,4cyclictest0-21swapper/123:25:311
115955993624,7cyclictest144409-21dropbear21:55:151
115955993623,7cyclictest170723-21/usr/sbin/munin00:30:281
115955993618,11cyclictest125005-21/usr/sbin/munin20:00:211
115955993617,11cyclictest128568-21/usr/sbin/munin20:20:241
115955993617,11cyclictest128568-21/usr/sbin/munin20:20:231
115955993613,12cyclictest0-21swapper/120:05:131
115954993629,4cyclictest20950irq/209-8000000.ethernet20:00:160
115954993629,4cyclictest0-21swapper/019:45:130
115955993516,12cyclictest169131-21/usr/sbin/munin00:20:191
115954993528,5cyclictest20950irq/209-8000000.ethernet19:20:220
115954993528,4cyclictest25450irq/423-mmc223:00:010
115954993528,4cyclictest19950irq/27-20000000.i2c00:15:310
115954993528,4cyclictest19950irq/27-20000000.i2c00:15:310
115954993514,19cyclictest20750irq/251-8000000.ethernet21:45:290
11595499351,3cyclictest0-21swapper/000:30:010
11595499351,32cyclictest0-21swapper/000:30:240
11595499351,2cyclictest0-21swapper/023:05:010
115955993422,8cyclictest158860-21munin-node23:20:161
115955993422,8cyclictest158860-21munin-node23:20:161
115955993421,8cyclictest137403-21sshd21:13:291
115955993414,10cyclictest127339-21/usr/sbin/munin20:15:111
11595599341,27cyclictest136000-21df21:05:171
115954993427,3cyclictest0-21swapper/020:45:000
115954993425,5cyclictest20750irq/251-8000000.ethernet19:15:300
115954993424,6cyclictest0-21swapper/023:55:010
11595499341,3cyclictest0-21swapper/022:25:180
11595499341,3cyclictest0-21swapper/022:25:180
115955993326,4cyclictest164816-21/usr/sbin/munin23:55:211
115955993320,7cyclictest146939-21df22:10:171
11595499333,6cyclictest172191-21cron00:40:000
115954993327,4cyclictest20950irq/209-8000000.ethernet20:25:290
115954993323,7cyclictest0-21swapper/022:30:290
11595499331,6cyclictest133625-21memory20:50:230
11595499331,18cyclictest0-21swapper/022:35:210
115955993220,7cyclictest140203-21/usr/sbin/munin21:30:171
115954993225,4cyclictest20750irq/251-8000000.ethernet20:15:010
115954993224,4cyclictest20750irq/251-8000000.ethernet21:30:190
115955993120,8cyclictest169887-21/usr/sbin/munin00:25:121
115954993125,4cyclictest20950irq/209-8000000.ethernet22:00:170
115955993021,4cyclictest116004-21wc19:10:011
115954993019,8cyclictest0-21swapper/000:10:260
11595499261,2cyclictest0-21swapper/019:10:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional