You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 00:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TI
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot6s.osadl.org (updated Sat Nov 15, 2025 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
397289997928,46cyclictest429896-21grep10:15:290
397289997727,42cyclictest0-21swapper/012:25:290
39728999741,6cyclictest0-21swapper/010:40:010
39728999712,2cyclictest0-21swapper/008:40:010
397289996830,30cyclictest0-21swapper/011:40:300
397289996635,28cyclictest453927-21sendmail12:40:000
39728999663,2cyclictest0-21swapper/009:05:000
397289996522,33cyclictest0-21swapper/012:30:310
397289996418,37cyclictest0-21swapper/008:15:210
39729199631,34cyclictest0-21swapper/111:40:011
397289996329,29cyclictest416134-21sendmail09:00:000
397289996223,33cyclictest0-21swapper/010:05:010
397289996223,33cyclictest0-21swapper/010:05:010
397289996120,36cyclictest0-21swapper/007:15:290
397291996052,5cyclictest0-21swapper/109:15:011
397289996022,28cyclictest0-21swapper/007:35:280
39728999601,4cyclictest0-21swapper/009:20:210
39728999592,2cyclictest0-21swapper/008:25:010
397289995919,5cyclictest421413-21latency_hist09:30:010
397289995915,40cyclictest0-21swapper/011:25:010
39728999582,5cyclictest429070-21apt-get10:15:010
397289995825,27cyclictest425642-21grep09:50:290
397289995823,30cyclictest0-21swapper/010:30:210
397289995748,5cyclictest0-21swapper/009:05:130
39728999571,49cyclictest437770-21latency_hist11:05:010
39728999571,3cyclictest404923-21/usr/sbin/munin07:50:280
397294995610,27cyclictest30-21rcuc/209:15:012
397289995644,8cyclictest449703-21grep12:15:000
39728999562,5cyclictest448646-21unixbench_multi12:05:270
39728999562,5cyclictest448646-21unixbench_multi12:05:270
397289995619,28cyclictest0-21swapper/007:25:290
397289995618,33cyclictest0-21swapper/011:45:270
39728999553,15cyclictest446118-21tail11:50:310
397289995519,5cyclictest0-21swapper/012:15:300
397289995514,7cyclictest0-21swapper/008:45:280
39728999551,39cyclictest0-21swapper/008:05:120
39728999551,25cyclictest406567-21ps08:00:260
397289995322,24cyclictest0-21swapper/010:20:310
39728999531,3cyclictest411068-21munin-run08:30:010
397291995242,6cyclictest452980-21grep12:30:311
397291995237,9cyclictest0-21swapper/108:25:001
397289995228,18cyclictest20050irq/27-20000000.i2c12:00:300
39728999522,25cyclictest0-21swapper/007:40:240
397289995222,25cyclictest0-21swapper/010:50:010
397289995222,25cyclictest0-21swapper/010:50:010
397289995215,5cyclictest0-21swapper/007:20:300
397289995215,5cyclictest0-21swapper/007:20:300
39728999521,4cyclictest451376-21users12:20:300
39728999521,4cyclictest451376-21users12:20:300
39728999521,25cyclictest431717-21sort10:30:000
397291995144,4cyclictest423161-21munin-plugin-st09:40:001
397291995143,5cyclictest411910-21munin-plugin-st08:35:001
397289995140,5cyclictest398094-21munin-plugin-st07:15:000
397289995117,3cyclictest0-21swapper/008:50:210
397289995116,6cyclictest0-21swapper/007:55:300
39728999511,26cyclictest0-21swapper/009:55:260
397291995037,9cyclictest0-21swapper/109:35:001
397291995037,9cyclictest0-21swapper/109:35:001
397289995026,20cyclictest21250irq/208-8000000.ethernet09:35:130
397289995021,5cyclictest0-21swapper/007:45:250
39728999496,15cyclictest0-21swapper/008:40:010
39728999491,26cyclictest0-21swapper/011:25:230
397289994823,19cyclictest24150irq/422-mmc109:15:010
39728999481,4cyclictest433906-21meminfo10:40:270
397289994737,5cyclictest21250irq/208-8000000.ethernet07:10:010
397289994722,4cyclictest419308-21irqstats09:15:270
39728999472,18cyclictest0-21swapper/011:15:010
397289994720,4cyclictest47-21kauditd11:17:000
397291994631,9cyclictest0-21swapper/109:00:011
397289994621,4cyclictest21250irq/208-8000000.ethernet10:50:150
397289994619,3cyclictest0-21swapper/010:55:160
397289994615,5cyclictest0-21swapper/007:35:010
397294994533,8cyclictest0-21swapper/209:55:302
39729199454,3cyclictest421550-21gpg-connect-age09:30:011
397291994527,12cyclictest0-21swapper/110:35:301
397294994432,8cyclictest0-21swapper/212:29:592
397294994428,10cyclictest0-21swapper/210:50:292
39729199431,9cyclictest0-21swapper/110:45:001
39728999431,3cyclictest0-21swapper/010:10:000
397294994233,5cyclictest0-21swapper/209:50:292
397294994233,5cyclictest0-21swapper/207:10:282
397291994224,12cyclictest0-21swapper/109:20:281
397294994133,4cyclictest0-21swapper/209:05:302
397294994127,10cyclictest0-21swapper/207:55:302
397291994130,7cyclictest0-21swapper/112:15:021
397291994128,8cyclictest0-21swapper/109:45:291
397291994128,8cyclictest0-21swapper/109:45:291
397291994128,6cyclictest0-21swapper/110:35:011
397291994125,11cyclictest0-21swapper/111:30:321
397289994123,15cyclictest0-21swapper/009:45:170
397289994123,15cyclictest0-21swapper/009:45:170
397289994119,3cyclictest0-21swapper/011:30:140
397294994027,9cyclictest0-21swapper/211:15:282
397294994024,6cyclictest311ktimers/207:30:012
397294994023,10cyclictest0-21swapper/208:20:292
397291994034,3cyclictest426534-21sendmail10:00:001
397289994032,4cyclictest21050irq/252-8000000.ethernet09:40:170
397289994019,7cyclictest437907-21/usr/sbin/munin11:05:260
39729499398,3cyclictest0-21swapper/209:30:012
397294993928,6cyclictest0-21swapper/212:00:312
397294993925,10cyclictest0-21swapper/210:30:302
397294993924,11cyclictest311ktimers/212:10:152
397294993924,11cyclictest311ktimers/208:35:152
397291993929,6cyclictest0-21swapper/112:25:291
397291993928,8cyclictest0-21swapper/110:15:001
397291993926,9cyclictest0-21swapper/110:55:301
397291993925,8cyclictest0-21swapper/110:50:301
397291993925,7cyclictest0-21swapper/109:00:301
397291993925,10cyclictest404203-21grep07:45:301
39728999391,2cyclictest0-21swapper/009:35:010
39728999391,2cyclictest0-21swapper/009:35:010
397294993821,11cyclictest311ktimers/210:00:162
397294993821,11cyclictest311ktimers/210:00:152
397294993821,10cyclictest0-21swapper/211:55:292
397294993819,14cyclictest311ktimers/209:00:162
39729199385,25cyclictest0-21swapper/109:40:291
397291993826,8cyclictest0-21swapper/107:35:301
397291993823,10cyclictest0-21swapper/109:15:321
397291993823,10cyclictest0-21swapper/108:00:281
39729199380,5cyclictest0-21swapper/107:10:011
397294993730,4cyclictest0-21swapper/209:20:002
397294993727,5cyclictest428208-21kworker/u8:1-ext4-rsv-conversion10:15:002
397294993723,7cyclictest411810-21grep08:30:302
397294993721,12cyclictest311ktimers/211:00:162
39729499371,30cyclictest4481311/usr/sbin/munin12:05:262
39729499371,30cyclictest4481311/usr/sbin/munin12:05:252
397291993725,7cyclictest0-21swapper/111:45:291
397291993723,11cyclictest0-21swapper/107:55:211
39729199371,31cyclictest0-21swapper/108:15:291
39729199371,2cyclictest0-21swapper/109:05:271
39728999371,2cyclictest0-21swapper/012:00:000
397294993627,5cyclictest453086-21munin-plugin-st12:35:002
397294993623,7cyclictest423098-21grep09:35:292
397294993621,10cyclictest311ktimers/210:15:132
397294993620,12cyclictest311ktimers/208:50:122
397291993626,6cyclictest441036-21grep11:20:291
397291993625,7cyclictest0-21swapper/111:10:301
397291993623,8cyclictest413549-21grep08:40:301
397291993620,10cyclictest0-21swapper/107:10:201
39729199361,2cyclictest0-21swapper/112:35:011
39729199361,29cyclictest22-21rcuc/109:50:301
39729199361,29cyclictest0-21swapper/110:15:291
397294993528,3cyclictest168450irq/134-8000000.ethernet-tx207:15:112
397294993527,5cyclictest0-21swapper/207:45:012
397294993526,6cyclictest441918-21grep11:25:312
397294993526,4cyclictest0-21swapper/212:20:212
397294993526,4cyclictest0-21swapper/212:20:212
397294993525,5cyclictest0-21swapper/211:10:222
397294993525,4cyclictest0-21swapper/208:05:212
397294993524,7cyclictest0-21swapper/210:40:312
397294993520,10cyclictest0-21swapper/208:40:282
39729499351,7cyclictest30-21rcuc/211:20:272
397291993528,4cyclictest168350irq/126-8000000.ethernet-tx107:20:301
397291993528,4cyclictest168350irq/126-8000000.ethernet-tx107:20:301
397291993525,5cyclictest0-21swapper/112:05:281
397291993525,5cyclictest0-21swapper/112:05:281
397291993525,5cyclictest0-21swapper/111:55:221
397291993524,7cyclictest0-21swapper/111:25:311
397291993515,11cyclictest435138-21apt-config10:50:011
397291993515,11cyclictest435138-21apt-config10:50:011
397289993527,6cyclictest443689-21cron11:40:010
397289993525,5cyclictest408490-21grep08:10:290
397294993425,6cyclictest444470-21grep11:40:292
397294993420,7cyclictest0-21swapper/207:35:222
397294993417,13cyclictest311ktimers/210:20:142
397294993415,14cyclictest311ktimers/211:30:192
397291993426,4cyclictest168350irq/126-8000000.ethernet-tx110:00:271
397291993426,4cyclictest168350irq/126-8000000.ethernet-tx110:00:271
397291993422,8cyclictest0-21swapper/112:00:301
397294993325,5cyclictest0-21swapper/208:55:292
397294993325,4cyclictest0-21swapper/210:50:012
397294993325,4cyclictest0-21swapper/210:50:012
397294993323,5cyclictest0-21swapper/208:15:272
397294993322,8cyclictest0-21swapper/207:20:312
397294993322,8cyclictest0-21swapper/207:20:312
397294993320,10cyclictest0-21swapper/211:05:262
397294993318,12cyclictest4242141/usr/sbin/munin09:45:222
397294993318,12cyclictest4242141/usr/sbin/munin09:45:222
397291993324,5cyclictest400863-21sed07:25:281
397291993323,7cyclictest401647-21grep07:30:301
397291993323,7cyclictest231ktimers/108:50:121
397291993322,7cyclictest0-21swapper/111:00:321
397291993322,7cyclictest0-21swapper/108:35:291
397294993219,9cyclictest0-21swapper/207:50:302
39729499321,4cyclictest450199-21meminfo12:15:252
397291993226,4cyclictest0-21swapper/108:10:291
397291993225,3cyclictest0-21swapper/107:50:301
397291993223,6cyclictest0-21swapper/110:20:311
397291993220,9cyclictest0-21swapper/112:20:291
397291993220,9cyclictest0-21swapper/112:20:291
397294993127,2cyclictest0-21swapper/210:55:202
397294993124,4cyclictest0-21swapper/209:40:322
397294993121,7cyclictest408468-21grep08:10:292
397294993121,6cyclictest410980-21grep08:25:282
397294993113,5cyclictest168450irq/134-8000000.ethernet-tx207:45:202
39729499311,24cyclictest0-21swapper/210:05:192
397291993123,5cyclictest0-21swapper/110:05:281
397294993020,7cyclictest453922-21apt12:40:002
397294993020,6cyclictest0-21swapper/208:45:312
39729499301,4cyclictest0-21swapper/208:00:232
397291993023,4cyclictest0-21swapper/111:45:001
397291993020,7cyclictest0-21swapper/112:15:271
397291993017,8cyclictest0-21swapper/111:20:011
397294992922,4cyclictest420491-21head09:20:302
397294992922,4cyclictest0-21swapper/211:45:272
397294992920,5cyclictest446127-21sed11:50:322
39729499291,26cyclictest0-21swapper/211:35:292
397291992922,4cyclictest168350irq/126-8000000.ethernet-tx107:40:301
397291992922,3cyclictest0-21swapper/108:45:221
397291992922,3cyclictest0-21swapper/108:30:011
397291992913,11cyclictest231ktimers/108:05:131
39729199291,26cyclictest0-21swapper/111:05:291
397289992921,4cyclictest0-21swapper/008:30:270
397294992819,7cyclictest0-21swapper/207:30:312
397291992821,4cyclictest0-21swapper/110:25:281
397291992819,7cyclictest231ktimers/111:50:191
39729499278,14cyclictest431607-21sed10:25:312
397294992720,4cyclictest0-21swapper/209:30:312
397294992720,4cyclictest0-21swapper/209:30:312
397294992718,5cyclictest433269-21grep10:35:302
397291992618,4cyclictest0-21swapper/107:15:281
397294992115,3cyclictest397386-21latency_hist07:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional