You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-14 - 09:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TI
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot6s.osadl.org (updated Sat Mar 14, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2959328991053,2cyclictest0-21swapper/023:55:010
295932899791,8cyclictest2971029-21tune2fs20:15:200
295932899762,8cyclictest2964479-21/usr/sbin/munin19:35:270
295932899742,3cyclictest2992501-21apt-get22:20:150
295932899732,6cyclictest2997710-21cut22:50:210
295932899721,6cyclictest2976353-21/usr/sbin/munin20:45:220
295932899721,6cyclictest2976353-21/usr/sbin/munin20:45:210
2959328997122,7cyclictest20650irq/252-8000000.ethernet21:50:110
2959328997122,7cyclictest20650irq/252-8000000.ethernet21:50:110
295932899711,5cyclictest1660-21snmpd00:25:020
295932899702,4cyclictest3000399-21meminfo23:05:240
295932899702,4cyclictest2977927-21fschecks_time20:55:200
2959331996856,7cyclictest0-21swapper/122:45:311
2959328996828,8cyclictest2960151-21munin-node19:15:010
295932899681,5cyclictest2990521-21apt-config22:10:010
295932899676,4cyclictest3002174-21processes23:15:240
2959328996633,30cyclictest20850irq/208-8000000.ethernet00:10:280
2959328996535,27cyclictest3004663-21phc2sys-jitter23:30:240
2959328996524,16cyclictest0-21swapper/023:20:250
295932899651,4cyclictest2996779-21/usr/sbin/munin22:45:190
295932899641,36cyclictest0-21swapper/000:10:010
295933299637,26cyclictest32-21ksoftirqd/223:10:182
295933299632,22cyclictest2990506-21apt-get22:10:022
2959331996348,5cyclictest231ktimers/120:15:011
295932899635,5cyclictest2966333-21apt-get19:50:010
295932899634,4cyclictest1258-21in:imuxsock23:10:010
2959328996328,30cyclictest0-21swapper/022:55:290
2959328996326,6cyclictest3015959-21grep00:35:270
295932899631,10cyclictest2980947-21latency_hist21:15:010
2959328996310,24cyclictest131ktimers/020:10:230
2959332996247,12cyclictest311ktimers/222:15:192
2959332996211,17cyclictest30-21rcuc/220:35:012
295932899621,6cyclictest2965567-21cat19:45:010
2959332996113,27cyclictest32-21ksoftirqd/223:50:012
2959328996129,4cyclictest20850irq/208-8000000.ethernet21:05:180
295932899611,6cyclictest2966694-21diskstats19:50:190
295932899611,4cyclictest3003901-21ptp4l-jitter23:25:270
2959328996035,22cyclictest2988656-21apt-get22:00:000
295932899602,4cyclictest0-21swapper/022:25:180
295932899601,4cyclictest2987040-21grep21:50:010
2959332995952,4cyclictest32-21ksoftirqd/223:20:172
2959328995929,4cyclictest20850irq/208-8000000.ethernet21:30:170
295932899591,4cyclictest2961778-21memory19:20:220
295932899581,6cyclictest3012111-21tune2fs00:15:180
2959332995718,13cyclictest32-21ksoftirqd/220:45:172
2959332995718,13cyclictest32-21ksoftirqd/220:45:172
295932899571,30cyclictest2973208-21apt-config20:30:000
2959328995628,4cyclictest20850irq/208-8000000.ethernet22:05:020
2959328995628,4cyclictest20850irq/208-8000000.ethernet22:05:010
2959332995517,15cyclictest30-21rcuc/220:20:012
295933199543,38cyclictest0-21swapper/119:25:201
2959332995245,4cyclictest2996523-21munin-plugin-st22:45:002
2959328995233,15cyclictest131ktimers/019:55:250
2959331995142,6cyclictest2974108-21munin-plugin-st20:35:001
2959328995129,4cyclictest0-21swapper/020:23:330
2959328995129,4cyclictest0-21swapper/020:23:330
2959328995129,3cyclictest0-21swapper/020:50:160
2959331995038,7cyclictest2961149-21grep19:15:271
2959331995011,15cyclictest14-21rcu_preempt19:50:011
2959328995032,3cyclictest0-21swapper/019:25:010
2959332994939,5cyclictest3008332-21munin-plugin-st23:55:012
295933299493,25cyclictest32-21ksoftirqd/221:45:012
295932899492,7cyclictest3008400-21latency_hist23:55:000
2959332994834,9cyclictest0-21swapper/200:40:002
295933199483,34cyclictest3002310-21unixbench_multi23:15:261
2959331994827,10cyclictest0-21swapper/122:35:001
295933299479,9cyclictest32-21ksoftirqd/219:40:012
2959331994734,9cyclictest3012652-21sendmail00:20:001
2959328994728,4cyclictest20850irq/208-8000000.ethernet22:30:200
2959328994728,4cyclictest20850irq/208-8000000.ethernet00:30:150
2959332994634,7cyclictest2963680-21grep19:30:272
295933199469,16cyclictest0-21swapper/122:45:011
295932899461,2cyclictest0-21swapper/000:05:000
2959332994534,7cyclictest2973206-21grep20:30:002
2959332994528,10cyclictest3009178-21sendmail00:00:012
2959331994531,10cyclictest0-21swapper/122:25:281
2959331994530,9cyclictest0-21swapper/123:45:271
2959328994533,4cyclictest0-21swapper/019:15:030
2959332994432,7cyclictest0-21swapper/219:15:012
295933199447,11cyclictest0-21swapper/119:15:011
2959331994425,10cyclictest0-21swapper/119:50:281
295933199441,6cyclictest2976627-21unixbench_multi20:45:271
295933199441,6cyclictest2976627-21unixbench_multi20:45:271
295933199441,4cyclictest0-21swapper/122:40:011
2959328994437,3cyclictest0-21swapper/021:35:250
2959328994430,9cyclictest0-21swapper/020:00:270
2959328994429,10cyclictest0-21swapper/021:00:280
295932899441,37cyclictest0-21swapper/022:15:020
2959332994319,13cyclictest0-21swapper/222:10:132
2959331994333,6cyclictest3015096-21grep00:30:271
2959332994227,9cyclictest0-21swapper/222:50:252
2959331994234,5cyclictest0-21swapper/123:10:271
2959331994231,6cyclictest0-21swapper/119:30:281
2959331994230,9cyclictest0-21swapper/122:25:001
2959331994226,10cyclictest0-21swapper/121:05:161
295932899425,9cyclictest2982213-21fschecks_count21:20:190
2959328994228,9cyclictest2999780-21grep23:00:260
2959328994228,9cyclictest2999780-21grep23:00:260
295932899422,34cyclictest3006545-21sed23:40:290
295932899422,34cyclictest3006545-21sed23:40:280
2959332994123,9cyclictest0-21swapper/221:35:252
2959331994131,5cyclictest0-21swapper/100:00:011
2959331994130,6cyclictest0-21swapper/120:10:001
295932899412,33cyclictest2974569-21sh20:35:200
2959332994030,4cyclictest2980948-21apt-get21:15:022
2959332994018,12cyclictest0-21swapper/220:20:272
2959332994018,12cyclictest0-21swapper/220:20:272
2959332994014,20cyclictest2-21kthreadd21:09:192
2959331994025,10cyclictest0-21swapper/121:00:261
2959331994023,8cyclictest231ktimers/121:10:271
295933199401,6cyclictest2986004-21unixbench_multi21:40:271
295932899401,5cyclictest2992286-21munin-run22:20:000
295932899401,37cyclictest0-21swapper/023:35:200
2959332993925,6cyclictest2998931-21grep22:55:282
2959332993923,8cyclictest0-21swapper/223:35:272
2959332993921,10cyclictest2994179-21/usr/sbin/munin22:30:282
295933199396,21cyclictest3003476-21/usr/sbin/munin23:25:201
2959331993929,6cyclictest0-21swapper/119:55:301
2959331993929,6cyclictest0-21swapper/119:40:281
2959331993929,4cyclictest0-21swapper/123:05:291
2959331993928,7cyclictest3008299-21unixbench_singl23:50:281
2959331993924,10cyclictest0-21swapper/100:15:021
295933199391,31cyclictest2983103-21cut21:25:201
295932899391,4cyclictest0-21swapper/021:25:160
2959332993829,4cyclictest0-21swapper/221:45:202
2959332993828,5cyclictest2989525-21grep22:00:282
2959332993828,5cyclictest2989525-21grep22:00:282
2959332993827,5cyclictest0-21swapper/219:50:292
2959332993822,11cyclictest3003479-21apt-get23:25:162
2959332993821,10cyclictest3014539-21apt-get00:30:142
2959332993814,19cyclictest2-21kthreadd23:17:202
2959332993812,14cyclictest2959325-21cyclictest21:50:142
2959332993812,14cyclictest2959325-21cyclictest21:50:132
2959331993823,7cyclictest0-21swapper/100:20:211
2959331993821,10cyclictest0-21swapper/120:25:271
2959331993819,9cyclictest0-21swapper/123:40:281
2959331993819,9cyclictest0-21swapper/123:40:281
2959328993830,4cyclictest167350irq/118-8000000.ethernet-tx020:05:150
295932899381,35cyclictest0-21swapper/022:35:120
295932899381,35cyclictest0-21swapper/021:40:110
295932899381,35cyclictest0-21swapper/019:30:130
295932899381,35cyclictest0-21swapper/000:25:270
295933299376,24cyclictest2960585-21munin-node19:15:052
2959332993724,9cyclictest3000097-21munin-node23:05:112
2959332993724,8cyclictest0-21swapper/223:40:192
2959332993724,8cyclictest0-21swapper/223:40:192
2959332993714,11cyclictest0-21swapper/221:25:122
2959332993713,16cyclictest167550irq/134-8000000.ethernet-tx220:10:162
2959332993711,3cyclictest2-21kthreadd23:02:182
2959332993711,3cyclictest2-21kthreadd23:02:182
2959332993711,21cyclictest2-21kthreadd20:39:212
2959331993725,8cyclictest0-21swapper/120:55:281
2959331993725,7cyclictest0-21swapper/100:05:281
295933199372,2cyclictest0-21swapper/122:10:011
295933199372,29cyclictest0-21swapper/120:40:001
2959331993720,9cyclictest14-21rcu_preempt22:00:011
2959331993720,9cyclictest14-21rcu_preempt22:00:011
2959328993725,8cyclictest0-21swapper/020:40:290
2959332993628,4cyclictest2964856-21apt-get19:40:122
2959332993626,7cyclictest1660-21snmpd00:14:192
2959332993626,4cyclictest0-21swapper/200:25:272
2959332993623,9cyclictest0-21swapper/222:40:002
2959332993614,13cyclictest2982333-21/usr/sbin/munin21:20:212
2959332993610,14cyclictest0-21swapper/220:55:122
295933199363,7cyclictest2987732-21grep21:50:281
295933199363,7cyclictest2987732-21grep21:50:281
2959331993625,8cyclictest0-21swapper/100:00:281
2959331993625,7cyclictest2964534-21grep19:35:281
2959331993624,8cyclictest0-21swapper/121:45:281
2959331993623,8cyclictest0-21swapper/100:25:191
2959331993621,10cyclictest0-21swapper/122:50:201
295932899361,3cyclictest0-21swapper/022:40:120
2959332993525,5cyclictest0-21swapper/220:00:272
2959332993525,5cyclictest0-21swapper/219:20:262
2959332993524,7cyclictest1321-21sendmail-mta23:32:562
295933299352,24cyclictest2973168-21kworker/u8:2+rpciod21:12:192
2959332993521,8cyclictest0-21swapper/220:50:272
2959332993520,10cyclictest2993343-21apt-get22:25:172
2959332993512,15cyclictest3010749-21/usr/sbin/munin00:05:272
2959331993525,7cyclictest0-21swapper/123:30:241
2959331993525,6cyclictest2982630-21unixbench_multi21:20:271
2959331993522,6cyclictest1660-21snmpd20:15:171
2959331993519,12cyclictest0-21swapper/121:35:001
295933199351,10cyclictest0-21swapper/120:05:011
295932899351,3cyclictest2973469-21apt-get20:30:130
2959332993428,4cyclictest311ktimers/200:00:172
2959332993423,7cyclictest0-21swapper/219:25:282
2959332993423,5cyclictest0-21swapper/221:30:242
295933299342,23cyclictest3011692-21kworker/u8:0+rpciod00:21:212
295933199342,5cyclictest0-21swapper/120:25:011
295933199342,5cyclictest0-21swapper/120:25:011
295933199341,2cyclictest0-21swapper/119:25:011
2959328993425,5cyclictest2981815-21grep21:15:270
2959328993421,8cyclictest0-21swapper/023:45:200
2959332993326,3cyclictest0-21swapper/219:55:212
2959332993325,4cyclictest2987995-21apt-get21:55:142
295933299331,23cyclictest3011692-21kworker/u8:0+rpciod00:17:192
295933199332,4cyclictest2977572-21munin-run20:55:001
2959332993227,3cyclictest311ktimers/219:45:162
295933299321,26cyclictest0-21swapper/222:20:292
295933199323,22cyclictest2981818-21sed21:15:281
2959331993222,7cyclictest0-21swapper/122:55:281
2959332993124,4cyclictest311ktimers/220:05:172
2959331993124,3cyclictest3003215-21unixbench_singl23:20:271
2959332993017,7cyclictest2978683-21/usr/sbin/munin21:00:162
295933199307,11cyclictest2990796-21/usr/sbin/munin22:10:171
2959331993020,6cyclictest3015947-21grep00:35:271
2959328993019,6cyclictest131ktimers/019:10:010
2959332992923,4cyclictest311ktimers/220:40:172
2959332992914,10cyclictest0-21swapper/222:45:132
2959331992919,6cyclictest3005679-21grep23:35:271
2959331992821,5cyclictest231ktimers/120:40:291
2959331992819,7cyclictest0-21swapper/123:00:181
2959331992819,7cyclictest0-21swapper/123:00:181
2959331992618,6cyclictest0-21swapper/121:35:201
2959331992618,4cyclictest0-21swapper/121:55:261
2959331992418,4cyclictest2992279-21sendmail22:20:001
295933199235,10cyclictest0-21swapper/119:10:011
295933299213,12cyclictest2959378-21latency_hist19:10:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional