You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-28 - 10:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TI
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot6s.osadl.org (updated Sun Dec 28, 2025 00:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2701731998168,10cyclictest311ktimers/220:15:022
2701729997327,42cyclictest0-21swapper/100:00:301
2701729997148,19cyclictest2742884-21munin-plugin-st23:10:001
2701729997148,19cyclictest2742884-21munin-plugin-st23:09:591
270173199702,25cyclictest0-21swapper/223:50:012
270172999684,48cyclictest0-21swapper/120:00:001
2701729996625,33cyclictest0-21swapper/122:05:001
270172999662,50cyclictest0-21swapper/123:30:151
270172999652,58cyclictest2757435-21apt-get00:35:011
270172999651,53cyclictest0-21swapper/121:15:111
270172999651,53cyclictest0-21swapper/121:15:111
2701728996517,42cyclictest2749906-21latency_hist23:50:010
2701729996429,30cyclictest2730775-21grep21:55:281
2701729996421,37cyclictest2727521-21grep21:35:311
2701728996422,40cyclictest2714674-21latency_hist20:25:000
270172999632,44cyclictest2701723-21cyclictest22:55:161
270172999631,48cyclictest0-21swapper/119:15:121
2701728996224,35cyclictest2720605-21cut20:55:290
2701728996221,4cyclictest2732746-21munin-run22:10:010
2701729996122,31cyclictest2725001-21grep21:20:321
2701728996122,3cyclictest2735255-21/usr/sbin/munin22:20:310
270172999601,47cyclictest0-21swapper/120:15:151
2701728996023,33cyclictest0-21swapper/023:10:270
2701728996018,39cyclictest0-21swapper/022:40:290
270172999592,48cyclictest2732108-21apt-get22:05:151
2701729995917,34cyclictest2702460-21head19:10:301
2701728995916,7cyclictest2702190-21memory19:10:270
270172999582,48cyclictest2715163-21/usr/sbin/munin20:25:281
270172999581,48cyclictest2754464-21/usr/sbin/munin00:15:211
270172999581,46cyclictest0-21swapper/123:45:171
2701728995812,40cyclictest2705465-21latency_hist19:30:000
2701729995736,17cyclictest0-21swapper/122:15:001
270172999571,47cyclictest2715781-21apt-get20:30:191
270172999571,44cyclictest0-21swapper/119:45:151
270172999570,48cyclictest2741406-21apt-get23:00:121
2701728995720,3cyclictest2714906-21df20:25:180
270172999563,46cyclictest2736074-21head22:25:291
2701729995629,22cyclictest0-21swapper/120:15:001
270172999562,46cyclictest2721879-21apt-get21:05:141
270172999561,48cyclictest2748645-21/usr/sbin/munin23:40:281
270172999561,47cyclictest2729787-21/usr/sbin/munin21:50:251
270172999561,47cyclictest2706749-21/usr/sbin/munin19:35:271
270172999561,47cyclictest2706749-21/usr/sbin/munin19:35:271
270172999561,47cyclictest2705558-21apt-get19:30:171
270172999561,46cyclictest2718413-21apt-get20:45:141
2701728995620,8cyclictest2725099-21latency_hist21:24:590
2701728995616,3cyclictest2716970-21irqstats20:35:250
2701728995615,38cyclictest0-21swapper/023:20:200
2701728995615,38cyclictest0-21swapper/023:20:200
270172999552,46cyclictest2708936-21apt-get19:50:171
270172999552,46cyclictest2708936-21apt-get19:50:171
270172999551,47cyclictest2710633-21apt-get20:00:181
270172999551,46cyclictest2756296-21/usr/sbin/munin00:25:271
270172999551,46cyclictest2727884-21apt-get21:40:051
270172999551,46cyclictest2719339-21apt-get20:50:131
2701728995540,9cyclictest0-21swapper/023:30:130
2701728995536,14cyclictest0-21swapper/021:10:120
270172999542,45cyclictest2745266-21/usr/sbin/munin23:20:281
270172999542,45cyclictest2745266-21/usr/sbin/munin23:20:281
270172999542,43cyclictest2703879-21apt-get19:20:161
270172999542,43cyclictest2703879-21apt-get19:20:151
2701729995418,26cyclictest0-21swapper/121:45:221
270172999541,46cyclictest2723159-21/usr/sbin/munin21:10:261
270172999541,46cyclictest2720635-21/usr/sbin/munin20:55:291
270172999541,46cyclictest2717155-21/usr/sbin/munin20:35:291
270172999541,46cyclictest0-21swapper/122:20:161
270172999541,45cyclictest2737393-21/usr/sbin/munin22:35:231
270172999541,45cyclictest2711811-21/usr/sbin/munin20:05:281
270172999541,41cyclictest0-21swapper/122:15:121
2701728995428,13cyclictest0-21swapper/022:45:130
2701728995425,4cyclictest2743995-21diskstats23:15:190
2701728995417,3cyclictest2734397-21users22:15:280
2701728995410,18cyclictest0-21swapper/000:35:010
270172999531,47cyclictest2714270-21/usr/sbin/munin20:20:231
270172999531,45cyclictest2753761-21/usr/sbin/munin00:10:271
270172999531,43cyclictest2755104-21apt-get00:20:141
270172999531,43cyclictest2745626-21apt-get23:25:171
270172999531,43cyclictest2717571-21apt-get20:40:151
2701728995338,11cyclictest2707896-21munin-plugin-st19:45:000
2701728995334,14cyclictest2746070-21/usr/sbin/munin23:25:310
2701728995324,3cyclictest2739977-21/usr/sbin/munin22:50:250
2701728995324,3cyclictest2739977-21/usr/sbin/munin22:50:250
2701728995322,28cyclictest2725918-21date21:30:000
2701728995322,28cyclictest2708982-21/usr/sbin/munin19:50:170
2701728995322,28cyclictest2708982-21/usr/sbin/munin19:50:170
2701731995242,6cyclictest2746266-21munin-plugin-st23:30:002
270172999521,44cyclictest2750148-21/usr/sbin/munin23:50:211
270172999521,44cyclictest2750148-21/usr/sbin/munin23:50:201
270172999521,44cyclictest2738867-21/usr/sbin/munin22:45:101
270172999521,44cyclictest2726146-21/usr/sbin/munin21:30:171
270172999521,44cyclictest2726146-21/usr/sbin/munin21:30:171
270172999521,44cyclictest2725516-21/usr/sbin/munin21:25:251
270172999521,44cyclictest2704977-21irqstats19:25:231
270172999521,44cyclictest0-21swapper/121:00:301
270172999521,44cyclictest0-21swapper/121:00:301
270172999521,43cyclictest2744224-21memory23:15:261
270172999521,43cyclictest2743158-21/usr/sbin/munin23:10:171
270172999521,42cyclictest2738028-21apt-get22:40:131
270172999521,42cyclictest0-21swapper/122:50:141
270172999521,42cyclictest0-21swapper/122:50:141
2701728995216,33cyclictest2750219-21kernelversion23:50:220
2701728995216,33cyclictest2750219-21kernelversion23:50:220
2701731995141,4cyclictest2745426-21munin-plugin-st23:25:002
2701731995141,4cyclictest2745426-21munin-plugin-st23:25:002
270172999512,42cyclictest2707253-21munin-node19:40:131
270172999511,43cyclictest2747382-21/usr/sbin/munin23:35:191
270172999511,43cyclictest2747382-21/usr/sbin/munin23:35:191
2701728995133,15cyclictest167350irq/118-8000000.ethernet-tx019:45:320
270172999501,42cyclictest2736651-21/usr/sbin/munin22:30:261
270172999501,41cyclictest2750858-21munin-node23:55:121
2701728995037,10cyclictest2752233-21/usr/sbin/munin00:00:300
2701728995037,10cyclictest2711783-21/usr/sbin/munin20:05:290
2701728995020,3cyclictest2747306-21apt-get23:35:120
2701728995020,3cyclictest2747306-21apt-get23:35:120
270173199496,18cyclictest14-21rcu_preempt22:50:012
270173199496,18cyclictest14-21rcu_preempt22:50:002
270172999493,39cyclictest2752857-21/usr/sbin/munin00:05:261
270172999492,42cyclictest2701778-21latency_hist19:10:011
270172999492,42cyclictest2701778-21latency_hist19:10:001
2701728994940,7cyclictest2731469-21/usr/sbin/munin22:00:260
2701728994940,4cyclictest167350irq/118-8000000.ethernet-tx022:30:160
2701728994930,15cyclictest2726176-21/usr/sbin/munin21:30:190
2701728994930,15cyclictest2726176-21/usr/sbin/munin21:30:190
2701728994928,4cyclictest0-21swapper/021:45:010
2701728994922,3cyclictest2723624-21df_inode21:15:170
2701728994922,3cyclictest2723624-21df_inode21:15:170
2701728994921,3cyclictest2718135-21/usr/sbin/munin20:40:290
2701728994920,3cyclictest2741665-21irqstats23:00:220
2701728994918,29cyclictest0-21swapper/021:45:170
2701728994915,4cyclictest2706007-21find19:30:270
2701728994912,34cyclictest2719046-21latency_hist20:50:020
2701728994834,10cyclictest0-21swapper/021:00:130
2701728994834,10cyclictest0-21swapper/021:00:130
2701728994826,6cyclictest2755806-21latency_hist00:25:010
2701728994817,7cyclictest0-21swapper/000:15:270
2701729994716,28cyclictest2757724-21/usr/sbin/munin00:35:171
2701728994735,8cyclictest2735688-21/usr/sbin/munin22:25:210
2701728994735,8cyclictest2713329-21/usr/sbin/munin20:15:210
2701728994735,8cyclictest0-21swapper/021:35:150
2701728994734,9cyclictest2710368-21grep19:55:290
2701728994733,11cyclictest2748210-21/usr/sbin/munin23:40:170
2701728994731,13cyclictest2712765-21/usr/sbin/munin20:10:280
2701728994730,13cyclictest2742400-21cut23:05:200
2701728994730,13cyclictest2742400-21cut23:05:200
2701728994725,15cyclictest2756716-21latency_hist00:30:010
2701728994717,27cyclictest2730600-21dropbear21:55:230
2701731994637,5cyclictest2711274-21munin-plugin-st20:05:002
2701728994635,8cyclictest2758029-21/usr/sbin/munin00:35:260
2701728994635,8cyclictest2719907-21head20:50:290
2701728994632,10cyclictest2733539-21grep22:10:290
2701728994626,13cyclictest2751273-21ptp4l23:55:280
2701728994617,3cyclictest2737810-21apt22:40:010
2701728994539,3cyclictest2716014-21/usr/sbin/munin20:30:250
2701728994530,9cyclictest2703426-21cut19:15:280
2701728994530,12cyclictest2706829-21/usr/sbin/munin19:35:270
2701728994530,12cyclictest2706829-21/usr/sbin/munin19:35:270
2701728994520,23cyclictest0-21swapper/022:55:160
2701728994515,3cyclictest2754129-21ls00:15:000
2701728994438,3cyclictest2704208-21grep19:20:260
2701728994438,3cyclictest2704208-21grep19:20:260
2701728994437,5cyclictest2722283-21/usr/sbin/munin21:05:270
2701728994430,11cyclictest2753110-21sed00:05:300
2701728994430,10cyclictest2710932-21/usr/sbin/munin20:00:280
2701731994332,6cyclictest0-21swapper/223:05:282
2701731994332,6cyclictest0-21swapper/223:05:282
2701731994332,6cyclictest0-21swapper/219:30:292
2701731994312,27cyclictest311ktimers/200:20:182
2701731994233,6cyclictest2714656-21munin-plugin-st20:25:012
2701728994223,16cyclictest1660-21snmpd21:52:030
2701731994121,14cyclictest0-21swapper/200:00:292
270173199407,15cyclictest2709773-21/usr/sbin/munin19:55:202
270173199405,15cyclictest2748161-21/usr/sbin/munin23:40:192
2701731994034,4cyclictest30-21rcuc/221:20:012
2701731994034,4cyclictest30-21rcuc/221:20:012
2701731994028,6cyclictest0-21swapper/221:35:302
2701731994028,6cyclictest0-21swapper/200:35:302
2701728994029,5cyclictest2701855-21grep19:10:020
2701728994029,5cyclictest2701855-21grep19:10:020
2701731993928,6cyclictest0-21swapper/221:25:302
270173199392,20cyclictest30-21rcuc/222:25:022
2701731993825,8cyclictest0-21swapper/223:55:012
2701731993825,8cyclictest0-21swapper/223:55:012
2701731993730,4cyclictest0-21swapper/220:40:002
2701731993726,7cyclictest0-21swapper/222:10:292
2701731993726,5cyclictest0-21swapper/219:20:302
2701731993726,5cyclictest0-21swapper/219:20:302
2701731993723,8cyclictest0-21swapper/200:30:292
270173199371,12cyclictest0-21swapper/222:20:012
2701731993625,5cyclictest0-21swapper/222:00:332
2701731993624,4cyclictest0-21swapper/223:55:292
2701731993620,8cyclictest0-21swapper/221:20:322
2701731993523,8cyclictest0-21swapper/220:55:292
2701731993521,9cyclictest0-21swapper/222:55:212
270173199351,9cyclictest0-21swapper/222:45:012
270173199351,8cyclictest0-21swapper/219:50:002
270173199351,28cyclictest0-21swapper/219:15:262
2701731993426,5cyclictest2707897-21munin-plugin-st19:44:592
2701731993425,5cyclictest0-21swapper/223:15:292
2701731993425,4cyclictest0-21swapper/222:30:212
270173199341,28cyclictest0-21swapper/220:15:182
270173199335,19cyclictest0-21swapper/222:40:012
2701731993318,9cyclictest2728521-21cron21:45:002
270173199331,11cyclictest30-21rcuc/223:35:012
2701731993310,12cyclictest2741403-21/usr/sbin/munin23:00:182
2701731993226,3cyclictest2730571-21sh21:55:222
2701731993223,4cyclictest0-21swapper/221:50:232
2701731993222,5cyclictest2728728-21/usr/sbin/munin21:45:262
2701731993215,9cyclictest0-21swapper/220:05:322
2701731993125,4cyclictest0-21swapper/223:10:162
2701731993124,4cyclictest0-21swapper/219:50:282
2701731993124,4cyclictest0-21swapper/219:50:282
2701731993123,6cyclictest2716564-21apt-get20:35:012
2701731993121,5cyclictest0-21swapper/200:05:302
2701731993116,8cyclictest0-21swapper/222:05:222
270173199311,28cyclictest0-21swapper/220:40:202
270173199311,27cyclictest2715475-21sshd20:27:452
270173199311,25cyclictest0-21swapper/221:09:312
2701731993013,12cyclictest311ktimers/200:25:132
270173199301,24cyclictest0-21swapper/222:25:182
2701731992922,4cyclictest0-21swapper/200:15:252
2701731992921,6cyclictest14-21rcu_preempt20:50:242
2701731992919,7cyclictest311ktimers/200:10:272
2701731992824,2cyclictest0-21swapper/223:35:182
2701731992824,2cyclictest0-21swapper/223:35:182
2701731992819,6cyclictest0-21swapper/221:00:272
2701731992819,6cyclictest0-21swapper/221:00:272
2701731992818,6cyclictest2698430-21kworker/u8:5-ext4-rsv-conversion19:30:012
2701731992817,5cyclictest0-21swapper/221:30:212
2701731992817,5cyclictest0-21swapper/221:30:212
2701731992813,8cyclictest0-21swapper/220:50:022
2701731992718,6cyclictest0-21swapper/219:10:282
2701731992616,7cyclictest0-21swapper/221:10:232
270173199258,10cyclictest2706395-21/usr/sbin/munin19:35:192
270173199258,10cyclictest2706395-21/usr/sbin/munin19:35:192
2701731992415,5cyclictest0-21swapper/222:45:252
2701731992212,5cyclictest0-21swapper/219:10:012
2701731992212,5cyclictest0-21swapper/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional