You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-08-08 - 18:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Atom E3825 @1333 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of shadow in rack #9, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rack9slot4s.osadl.org (updated Sun Jul 31, 2022 12:59:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
266219912061178,23cyclictest28517-21munin-run07:30:001
262652580497,40sleep10-21swapper/107:21:531
261512462417,30sleep00-21swapper/007:20:410
2662099269234,27cyclictest28568-21munin-node07:30:000
159721140,4sleep1201rcuc/109:05:291
26621996920,16cyclictest9201-21grep08:00:201
26621996920,16cyclictest13123-21taskset12:25:531
26621996920,16cyclictest11129-21/usr/sbin/munin08:05:331
26621996920,16cyclictest11129-21/usr/sbin/munin08:05:321
26621996919,14cyclictest29387-21/usr/sbin/munin08:55:211
26621996821,17cyclictest31479-21cat11:50:201
26621996721,15cyclictest3242-21cat07:45:201
26621996720,16cyclictest7879-21logger07:55:551
26621996720,16cyclictest12585-21cat11:00:201
26621996719,14cyclictest6634-21hddtemp_smartct12:10:191
26621996620,16cyclictest20505-21cat08:30:211
26621996619,16cyclictest2259-21chrt12:00:071
26621996521,14cyclictest6151-21runrttasks09:18:121
26621996521,14cyclictest6151-21runrttasks09:18:111
26621996520,15cyclictest838-21smartctl11:55:211
26621996520,15cyclictest18665-21chrt08:25:201
26621996520,14cyclictest3685-21find09:10:321
26621996519,14cyclictest8856-21grep10:50:221
26621996518,16cyclictest13821-21chrt08:12:041
26621996518,14cyclictest14627-21hddtemp_smartct09:40:291
26621996421,15cyclictest17435-21runrttasks11:14:111
26621996420,15cyclictest8195-21cat12:15:131
26621996420,15cyclictest16741-21cat08:20:201
26621996420,14cyclictest3298-21runrttasks10:20:311
26621996419,16cyclictest27540-21cat10:15:241
26621996320,14cyclictest30959-21runrttasks07:34:171
26621996318,14cyclictest10767-21awk12:20:281
26621996317,13cyclictest847-21sh10:30:191
26621996220,15cyclictest26216-21runrttasks08:45:201
26621996219,15cyclictest6528-21test09:20:101
26621996219,15cyclictest22389-21cat08:35:211
26621996219,15cyclictest17027-21smartctl-wrappe09:45:351
26621996219,14cyclictest28761-21runrttasks11:43:571
26621996218,14cyclictest5082-21smartctl07:50:211
26621996217,16cyclictest16959-21chrt12:36:411
26621996121,21cyclictest27799-21df_inode08:50:151
26621996120,14cyclictest31211-21apt-config07:35:021
26621996120,14cyclictest19758-21/usr/sbin/munin09:55:211
26621996020,22cyclictest17996-21diskstats11:15:241
26621996019,15cyclictest6371-21/usr/sbin/munin10:45:281
26621996019,14cyclictest10736-21grep10:55:201
26621996019,10cyclictest19585-21/usr/sbin/munin11:20:201
26621996018,15cyclictest25938-21smartctl10:10:201
26621995922,15cyclictest3298-21runrttasks07:44:201
26621995919,14cyclictest2500-21/usr/sbin/munin10:35:391
26621995917,14cyclictest5517-21smartctl10:40:311
26621995823,10cyclictest14269-21fschecks_count11:05:211
26621995819,10cyclictest31627-21hddtemp_smartct10:25:191
26621995818,11cyclictest21674-21/usr/sbin/munin10:00:321
26621995816,15cyclictest25747-21smartctl11:35:221
26621995815,21cyclictest31388-21aten_r4power_vo09:00:141
26621995722,15cyclictest18203-21fschecks_count09:50:171
26621995718,16cyclictest13646-21runrttasks09:37:231
26621995718,13cyclictest3298-21runrttasks09:26:151
26621995716,13cyclictest23349-21/usr/sbin/munin11:30:131
26621995622,10cyclictest14181-21hddtemp_smartct12:30:181
26621995619,15cyclictest24543-21awk08:40:261
26621995619,15cyclictest24543-21awk08:40:261
26621995536,11cyclictest14275-21apt-get08:15:111
26621995521,12cyclictest22103-21ntpq12:50:251
26621995521,11cyclictest23533-21taskset10:05:021
26621995518,16cyclictest17476-21taskset12:40:101
26621995420,15cyclictest10603-21df09:30:221
26621995321,10cyclictest29328-21df11:45:151
26621995320,14cyclictest4441-21cat12:05:201
26621995318,9cyclictest20105-21memory12:45:291
26621995317,13cyclictest21759-21df11:25:161
2662099389,14cyclictest27964-21iostat_ios10:15:330
2662099361,22cyclictest24537-21/usr/sbin/munin10:05:300
26620993611,13cyclictest29844-21hwlatdetect07:30:190
2662099359,13cyclictest2201-21latency_hist12:00:010
80712346,10sleep03-21ksoftirqd/010:49:530
26620993420,8cyclictest1286-21iostat_ios07:40:210
2662099341,21cyclictest8926-21hddtemp_smartct09:25:220
192772341,9sleep00-21swapper/009:51:460
2662099331,19cyclictest6937-21grep12:10:240
2662099331,13cyclictest0-21swapper/012:34:450
2662099325,5cyclictest81rcu_preempt10:55:160
2662099323,14cyclictest16410-21irqstats11:10:230
26620993225,4cyclictest15600-21cron08:16:000
26620993213,6cyclictest9433-21ntp_states08:00:250
2662099321,12cyclictest0-21swapper/012:42:390
48802311,4sleep04979-21grep07:50:190
2662099318,12cyclictest30104-21open_files10:20:330
2662099317,9cyclictest0-21swapper/008:47:010
2662099316,3cyclictest81rcu_preempt12:21:240
2662099312,11cyclictest14814-21ps11:05:320
2662099311,4cyclictest81rcu_preempt11:17:090
2662099311,4cyclictest81rcu_preempt11:01:170
2662099311,4cyclictest81rcu_preempt10:12:410
26620993111,16cyclictest1578-21open_files09:05:280
26620993110,10cyclictest21346-21tr12:50:120
324052301,5sleep00-21swapper/011:52:410
2662099306,4cyclictest81rcu_preempt10:41:030
2662099302,4cyclictest81rcu_preempt10:26:530
2662099301,4cyclictest81rcu_preempt11:22:370
2662099301,4cyclictest81rcu_preempt08:57:220
2662099301,4cyclictest81rcu_preempt08:50:140
26620993012,14cyclictest4398-21taskset09:14:370
26620993011,5cyclictest11147-21ntp_states09:30:340
2662099300,4cyclictest81rcu_preempt08:42:310
2662099300,4cyclictest81rcu_preempt08:42:300
2662099298,17cyclictest4161-21gdbus12:15:150
2662099294,19cyclictest31543-21diskstats07:35:190
2662099292,15cyclictest0-21swapper/007:55:240
2662099291,4cyclictest81rcu_preempt12:36:270
26620992912,6cyclictest16684-21/usr/sbin/munin08:20:190
26620992912,5cyclictest3285-21iostat_ios07:45:220
2662099291,17cyclictest4161-21gdbus12:07:110
2662099291,10cyclictest0-21swapper/011:45:190
26620992910,6cyclictest14959-21ntp_states09:40:360
26620992910,5cyclictest28371-21unixbench_multi11:40:500
218172291,5sleep00-21swapper/008:35:070
214942291,5sleep00-21swapper/008:33:200
2662099282,14cyclictest12642-21ntp_states12:25:270
2662099282,13cyclictest19706-21df_inode12:45:210
2662099282,13cyclictest1797-21grep10:30:360
2662099281,4cyclictest81rcu_preempt10:04:110
26620992812,5cyclictest21073-21taskset09:55:490
2662099281,24cyclictest18476-21fschecks_time08:25:170
2662099281,14cyclictest12074-21runrttasks08:08:290
2662099281,14cyclictest12074-21runrttasks08:08:290
26620992810,6cyclictest8747-21perl10:50:180
26620992810,4cyclictest2882-21cstates10:35:480
268382271,5sleep00-21swapper/011:38:470
2662099279,5cyclictest13020-21/usr/sbin/munin09:35:350
2662099272,13cyclictest23653-21diskstats11:30:160
2662099272,13cyclictest2269-21apt-get12:00:110
2662099272,12cyclictest6546-21apt-get09:20:210
2662099271,3cyclictest81rcu_preempt09:16:070
2662099271,3cyclictest81rcu_preempt09:16:070
2662099271,14cyclictest32157-21/usr/sbin/munin09:00:280
2662099271,14cyclictest13273-21/usr/sbin/munin08:10:280
2662099271,13cyclictest17225-21/usr/sbin/munin09:45:370
2662099269,5cyclictest21764-21df_inode11:25:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional