You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-08 - 23:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TI
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot6s.osadl.org (updated Sat Nov 08, 2025 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
256199099110104,4cyclictest131ktimers/010:10:010
256199099862,37cyclictest15-21rcuc/012:20:010
256199099862,37cyclictest15-21rcuc/012:20:010
256199999787,46cyclictest0-21swapper/210:20:012
2561990997872,4cyclictest131ktimers/009:05:010
2561999997127,13cyclictest32-21ksoftirqd/211:00:222
2561999997116,29cyclictest0-21swapper/207:25:012
2561994997063,4cyclictest231ktimers/109:40:011
2562002996829,34cyclictest0-21swapper/307:15:003
2562002996827,36cyclictest0-21swapper/308:15:223
2562002996729,33cyclictest0-21swapper/307:50:313
2562002996622,33cyclictest0-21swapper/308:25:303
2562002996442,17cyclictest2573213-21munin-plugin-st08:15:003
256200299644,17cyclictest2577397-21head08:35:303
2562002996424,35cyclictest2597967-21sh10:39:593
2562002996422,33cyclictest0-21swapper/310:50:273
2562002996318,34cyclictest0-21swapper/312:20:293
2562002996318,34cyclictest0-21swapper/312:20:283
2561994996356,4cyclictest2561999-21cyclictest07:10:001
2562002996223,28cyclictest0-21swapper/308:40:273
2562002996223,28cyclictest0-21swapper/308:40:273
2562002996221,31cyclictest0-21swapper/311:30:293
256200299602,15cyclictest2609254-21apt-get11:45:173
2562002996017,34cyclictest0-21swapper/312:10:293
256200299594,11cyclictest2613209-21grep12:05:293
256200299594,11cyclictest2613209-21grep12:05:293
2562002995936,19cyclictest0-21swapper/312:25:323
256200299593,13cyclictest2614835-21grep12:15:273
2562002995921,29cyclictest0-21swapper/308:05:233
2562002995920,33cyclictest2612325-21grep12:00:303
2562002995920,32cyclictest0-21swapper/307:20:283
2562002995917,8cyclictest0-21swapper/311:25:323
2562002995916,35cyclictest2595276-21grep10:20:303
2562002995916,35cyclictest2595276-21grep10:20:303
2562002995915,35cyclictest0-21swapper/310:29:593
2562002995914,36cyclictest0-21swapper/310:40:273
2562002995913,4cyclictest0-21swapper/310:05:283
2562002995911,6cyclictest0-21swapper/311:40:023
256200299583,11cyclictest2586725-21grep09:30:283
256200299583,11cyclictest2586725-21grep09:30:283
2562002995827,25cyclictest0-21swapper/312:35:303
256200299582,14cyclictest2563829-21sed07:15:273
2562002995816,7cyclictest0-21swapper/309:35:263
2562002995814,33cyclictest0-21swapper/311:40:303
256200299581,13cyclictest2565624-21latency_hist07:30:003
256199999587,15cyclictest0-21swapper/212:20:012
256199999587,15cyclictest0-21swapper/212:20:002
256200299573,9cyclictest2590974-21head09:55:293
256200299573,8cyclictest2566423-21grep07:30:333
2562002995722,28cyclictest0-21swapper/309:45:203
2562002995721,31cyclictest0-21swapper/311:20:183
256200299564,10cyclictest2601733-21cut11:00:243
256200299563,10cyclictest2599367-21cut10:45:413
256200299556,10cyclictest2611511-21head11:55:313
256200299553,15cyclictest2574844-21unixbench_singl08:20:293
256200299553,15cyclictest2574844-21unixbench_singl08:20:293
256200299553,11cyclictest2580845-21head08:55:323
256200299553,10cyclictest2582548-21grep09:05:293
2562002995523,26cyclictest0-21swapper/310:10:273
256200299552,12cyclictest2581292-21fschecks_time09:00:253
256200299552,11cyclictest2585041-21unixbench_multi09:20:283
2561999995540,9cyclictest0-21swapper/210:45:432
256200299546,9cyclictest2588469-21grep09:40:313
256200299543,9cyclictest2591848-21grep10:00:293
256200299543,7cyclictest2570133-21lsblk07:55:243
256200299543,13cyclictest2596667-21cut10:30:223
256200299542,12cyclictest2604678-21sed11:15:293
2562002995419,26cyclictest0-21swapper/308:30:213
2561999995444,5cyclictest0-21swapper/212:20:002
256200299539,7cyclictest2590025-21mailstats09:50:293
256200299537,5cyclictest0-21swapper/311:50:223
256200299533,9cyclictest2601250-21grep10:55:323
256200299533,9cyclictest2601250-21grep10:55:323
2561999995345,5cyclictest2600510-21munin-plugin-st10:55:002
2561994995343,6cyclictest2584293-21run-parts09:20:011
256200299524,9cyclictest2579971-21grep08:50:303
256200299524,14cyclictest2617434-21head12:30:303
256200299522,15cyclictest2582944-21cut09:10:223
2561994995227,14cyclictest231ktimers/110:30:171
256200299516,10cyclictest2594411-21tail10:15:283
256200299513,8cyclictest2602960-21grep11:05:293
2562002995115,26cyclictest0-21swapper/309:30:013
2561999995112,8cyclictest32-21ksoftirqd/211:30:012
256199999511,22cyclictest0-21swapper/208:20:012
2561994995132,12cyclictest231ktimers/108:00:131
2561994995131,12cyclictest231ktimers/108:25:171
256200299503,8cyclictest2570987-21fschecks_count08:00:213
256200299503,15cyclictest2579009-21ptp4l-jitter08:45:293
256200299503,13cyclictest2567244-21head07:35:293
2562002995015,26cyclictest0-21swapper/307:50:013
256199499507,13cyclictest2607261-21apt-get11:35:021
2561994995026,11cyclictest231ktimers/112:10:161
256200299495,11cyclictest2584218-21grep09:15:293
2561990994938,7cyclictest2585061-21grep09:20:280
256200299483,9cyclictest2567703-21tune2fs07:40:223
2561999994836,8cyclictest2591043-21sendmail10:00:002
2561990994839,7cyclictest131ktimers/011:45:150
256200299474,7cyclictest2603792-21grep11:10:293
2561994994740,4cyclictest0-21swapper/107:25:001
256199499473,9cyclictest24-21ksoftirqd/107:25:011
2561994994735,8cyclictest0-21swapper/112:35:301
2561994994732,9cyclictest0-21swapper/107:15:011
2561994994713,12cyclictest22-21rcuc/107:50:021
256199099475,12cyclictest15-21rcuc/011:45:010
256199999469,28cyclictest2586819-21cron09:35:012
256199999469,28cyclictest2586819-21cron09:35:002
2561994994630,8cyclictest0-21swapper/110:50:281
2561994994628,11cyclictest231ktimers/108:40:131
2561994994628,11cyclictest231ktimers/108:40:131
2561994994625,14cyclictest231ktimers/112:15:141
256199999456,6cyclictest2577434-21apt-get08:40:012
2561999994533,8cyclictest0-21swapper/209:05:292
2561999994533,8cyclictest0-21swapper/207:35:282
2561999994518,13cyclictest0-21swapper/211:15:132
2561994994530,9cyclictest0-21swapper/109:20:291
2561990994539,4cyclictest0-21swapper/007:45:010
2561990994517,26cyclictest14-21rcu_preempt11:25:250
2561999994431,8cyclictest0-21swapper/211:35:012
2561994994432,8cyclictest0-21swapper/111:25:011
2561994994432,8cyclictest0-21swapper/110:35:301
2561994994431,8cyclictest0-21swapper/109:05:281
2561990994435,6cyclictest2568993-21munin-plugin-st07:50:010
2561990994433,6cyclictest21250irq/208-8000000.ethernet08:05:290
2561990994430,10cyclictest0-21swapper/007:15:010
256199099442,3cyclictest0-21swapper/008:30:000
2561999994320,15cyclictest2567750-21/usr/sbin/munin07:40:232
2561994994330,8cyclictest0-21swapper/111:50:291
2561994994324,14cyclictest0-21swapper/110:40:301
2561994994323,13cyclictest231ktimers/111:55:161
2561990994334,5cyclictest0-21swapper/008:15:290
2561999994225,11cyclictest2596706-21/usr/sbin/munin10:30:252
2561999994223,11cyclictest0-21swapper/211:10:272
2561999994217,13cyclictest0-21swapper/208:40:152
2561999994217,13cyclictest0-21swapper/208:40:152
2561994994233,5cyclictest14-21rcu_preempt12:20:221
2561994994233,5cyclictest14-21rcu_preempt12:20:221
2561994994228,9cyclictest0-21swapper/111:40:301
2561994994226,10cyclictest0-21swapper/111:15:301
2561994994225,12cyclictest231ktimers/108:05:141
256199099425,10cyclictest12-21ksoftirqd/012:15:000
2561990994234,5cyclictest2596100-21unixbench_multi10:25:310
2561990994233,5cyclictest2611572-21apt12:00:000
2561990994232,7cyclictest2616649-21munin-plugin-st12:30:000
2561990994230,8cyclictest0-21swapper/008:40:010
2561990994228,8cyclictest0-21swapper/007:15:270
2561990994221,16cyclictest15-21rcuc/009:45:140
2561990994217,20cyclictest14-21rcu_preempt12:35:190
2561990994211,25cyclictest0-21swapper/008:20:180
2561990994211,25cyclictest0-21swapper/008:20:180
2561999994134,3cyclictest2580908-21latency_hist09:00:002
2561999994130,6cyclictest2584235-21grep09:15:292
2561999994129,8cyclictest0-21swapper/207:30:342
2561999994114,21cyclictest2585138-21cron09:25:002
2561994994132,7cyclictest231ktimers/109:10:131
256199499411,3cyclictest0-21swapper/108:50:001
2561990994137,2cyclictest0-21swapper/011:30:120
2561990994134,4cyclictest2569835-21munin-plugin-st07:55:010
2561990994129,7cyclictest0-21swapper/010:40:300
2561990994127,9cyclictest0-21swapper/011:00:310
2561990994127,10cyclictest0-21swapper/010:00:290
2561990994124,9cyclictest0-21swapper/010:35:280
2561999994033,4cyclictest2596105-21unixbench_multi10:25:312
256199499403,17cyclictest0-21swapper/108:20:001
2561994994029,8cyclictest0-21swapper/107:20:001
2561994994028,8cyclictest0-21swapper/112:25:321
2561994994025,8cyclictest0-21swapper/109:45:281
2561994994021,9cyclictest0-21swapper/110:55:311
2561994994021,9cyclictest0-21swapper/110:55:311
2561994994010,25cyclictest231ktimers/109:35:291
2561990994028,8cyclictest0-21swapper/009:25:280
2561990994027,9cyclictest0-21swapper/008:55:320
2561990994027,8cyclictest131ktimers/008:30:150
2561990994025,9cyclictest0-21swapper/011:50:300
256199099401,2cyclictest0-21swapper/010:20:000
2561999993929,6cyclictest311ktimers/208:05:262
2561999993929,6cyclictest2601231-21grep10:55:322
2561999993929,6cyclictest2601231-21grep10:55:322
2561999993925,8cyclictest2597990-21sendmail-msp10:39:592
2561999993924,9cyclictest2592095-21apt-get10:05:162
2561999993923,11cyclictest0-21swapper/209:50:002
2561999993922,10cyclictest2585346-21apt-get09:25:172
256199999391,31cyclictest0-21swapper/212:25:212
2561994993926,8cyclictest0-21swapper/108:20:301
2561994993926,8cyclictest0-21swapper/108:20:301
2561994993925,9cyclictest0-21swapper/108:40:011
2561990993926,9cyclictest2603828-21unixbench_singl11:10:290
256199099392,32cyclictest0-21swapper/011:35:310
256199099391,32cyclictest0-21swapper/011:00:020
256199099391,32cyclictest0-21swapper/011:00:020
2561990993912,25cyclictest0-21swapper/012:15:190
2561999993828,6cyclictest2570608-21grep07:55:312
2561999993827,5cyclictest0-21swapper/208:00:222
2561999993824,8cyclictest0-21swapper/210:20:312
2561999993824,8cyclictest0-21swapper/210:20:312
2561999993823,8cyclictest0-21swapper/208:35:012
2561999993820,12cyclictest0-21swapper/207:50:162
2561999993820,11cyclictest0-21swapper/212:35:302
2561999993810,16cyclictest311ktimers/209:40:142
2561994993826,8cyclictest0-21swapper/109:25:291
2561994993826,8cyclictest0-21swapper/107:35:291
2561994993825,8cyclictest0-21swapper/112:05:291
2561994993825,8cyclictest0-21swapper/112:05:291
2561994993825,8cyclictest0-21swapper/110:05:281
2561994993825,8cyclictest0-21swapper/108:10:311
2561994993824,9cyclictest231ktimers/111:00:151
2561994993824,9cyclictest0-21swapper/110:45:441
2561994993824,8cyclictest2595236-21xargs10:20:301
2561994993824,8cyclictest2595236-21xargs10:20:301
2561994993824,10cyclictest231ktimers/108:30:161
2561990993828,7cyclictest0-21swapper/008:00:010
2561990993827,7cyclictest0-21swapper/010:10:290
2561990993823,8cyclictest0-21swapper/009:15:280
2561999993728,5cyclictest0-21swapper/212:00:302
2561999993725,8cyclictest0-21swapper/210:00:292
2561999993718,12cyclictest0-21swapper/211:05:272
2561999993715,10cyclictest0-21swapper/212:10:132
2561994993731,4cyclictest2561986-21cyclictest12:00:281
2561994993726,7cyclictest0-21swapper/109:50:291
2561994993726,7cyclictest0-21swapper/107:55:321
2561994993724,9cyclictest0-21swapper/109:55:281
256199099376,27cyclictest0-21swapper/009:50:300
256199099374,25cyclictest0-21swapper/009:05:180
2561990993730,4cyclictest21250irq/208-8000000.ethernet09:10:220
2561990993730,4cyclictest131ktimers/007:25:180
2561990993729,5cyclictest131ktimers/010:30:040
256199099372,4cyclictest2617426-21unixbench_singl12:30:300
2561990993723,7cyclictest0-21swapper/012:00:300
2561999993627,4cyclictest0-21swapper/211:45:002
2561999993626,6cyclictest2573178-21grep08:10:322
2561999993625,6cyclictest0-21swapper/210:40:292
2561999993618,12cyclictest2563587-21/usr/sbin/munin07:15:252
256199999361,27cyclictest2576582-21kworker/u8:0+rpciod09:04:292
2561994993628,5cyclictest0-21swapper/110:30:011
2561994993628,4cyclictest0-21swapper/110:15:291
2561994993624,8cyclictest0-21swapper/109:30:291
2561994993624,8cyclictest0-21swapper/109:30:291
2561994993622,10cyclictest0-21swapper/111:35:281
2561990993629,4cyclictest21250irq/208-8000000.ethernet09:30:250
2561990993629,4cyclictest21250irq/208-8000000.ethernet09:30:250
2561990993629,4cyclictest21250irq/208-8000000.ethernet08:50:310
2561990993627,6cyclictest2571434-21grep08:00:300
2561990993627,4cyclictest0-21swapper/007:35:290
2561990993626,7cyclictest131ktimers/011:20:180
2561990993625,7cyclictest15-21rcuc/010:20:170
2561990993625,7cyclictest15-21rcuc/010:20:170
2561990993622,8cyclictest131ktimers/010:50:140
2561999993524,9cyclictest2565281-21/usr/sbin/munin07:25:272
2561999993523,7cyclictest2605565-21sshd11:21:312
2561999993522,9cyclictest2578580-21munin-node08:45:122
2561994993522,8cyclictest0-21swapper/108:45:301
2561994993522,7cyclictest2568134-21sshd07:41:311
2561990993522,8cyclictest0-21swapper/011:05:280
2561990993518,12cyclictest0-21swapper/008:45:300
2561999993426,4cyclictest2607550-21/usr/sbin/munin11:35:302
2561999993425,5cyclictest311ktimers/209:35:182
2561999993423,7cyclictest2568351-21/usr/sbin/munin07:45:172
2561999993420,9cyclictest2580047-21sshd08:52:302
2561999993414,10cyclictest0-21swapper/209:50:142
2561994993427,4cyclictest2568960-21unixbench_singl07:45:311
2561990993428,4cyclictest21250irq/208-8000000.ethernet09:40:270
2561990993421,8cyclictest131ktimers/011:15:160
2561990993421,8cyclictest0-21swapper/010:45:440
2561999993324,6cyclictest311ktimers/211:50:142
2561999993323,5cyclictest2616919-21/usr/sbin/munin12:30:192
2561999993318,9cyclictest2611527-21grep11:55:312
2561999993314,13cyclictest2562704-21unixbench_multi07:10:292
256199999331,2cyclictest0-21swapper/208:25:022
256199999331,2cyclictest0-21swapper/208:25:022
2561994993325,5cyclictest14-21rcu_preempt07:30:311
2561990993327,4cyclictest21250irq/208-8000000.ethernet08:10:240
2561990993323,8cyclictest2591055-21sendmail-msp10:00:000
2561990993322,6cyclictest131ktimers/012:05:210
2561990993322,6cyclictest131ktimers/012:05:210
2561990993318,10cyclictest0-21swapper/007:25:000
256199099331,4cyclictest0-21swapper/009:40:010
256199099331,26cyclictest0-21swapper/008:40:130
256199099331,26cyclictest0-21swapper/008:40:130
2561999993219,7cyclictest2612847-21/usr/sbin/munin12:05:232
2561999993219,7cyclictest2612847-21/usr/sbin/munin12:05:232
2561999993217,12cyclictest2609864-21/usr/sbin/munin11:45:292
2561994993227,4cyclictest2561986-21cyclictest11:10:281
2561994993222,4cyclictest0-21swapper/110:00:261
2561994993221,7cyclictest0-21swapper/112:30:301
2561990993221,8cyclictest131ktimers/007:30:190
2561999993125,3cyclictest2593239-21/usr/sbin/munin10:10:242
2561999993123,5cyclictest2575170-21df08:25:192
2561999993117,9cyclictest0-21swapper/209:15:022
2561994993121,6cyclictest0-21swapper/110:10:301
2561994993119,7cyclictest22-21rcuc/108:55:211
2561994993024,4cyclictest14-21rcu_preempt11:05:211
2561994992921,5cyclictest231ktimers/111:50:011
2561994992918,5cyclictest22-21rcuc/111:25:161
2561994992820,5cyclictest22-21rcuc/109:00:321
256199999278,11cyclictest0-21swapper/207:10:012
256199099271,7cyclictest15-21rcuc/007:10:000
2562002992512,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional