You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-10-01 - 07:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TI
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6s.osadl.org (updated Sun Oct 01, 2023 00:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2932069998915,37cyclictest0-21swapper/023:05:140
293207199741,69cyclictest928-21dropbear21:30:281
2932069997232,12cyclictest2947209-21grep20:35:390
2932069997025,16cyclictest2965100-21sendmail_mailst22:20:350
293207199692,7cyclictest2944917-21apt-get20:25:221
2932071996914,51cyclictest1549-21snmpd00:31:151
2932069996930,10cyclictest2939778-21apt-get19:55:180
293207199682,6cyclictest2972980-21/usr/sbin/munin23:30:381
293207199682,11cyclictest2964695-21/usr/sbin/munin22:20:211
293207199682,10cyclictest2957198-21/usr/sbin/munin21:35:351
293207199681,63cyclictest2963165-21/usr/sbin/munin22:10:341
293207199681,12cyclictest2958133-21/usr/sbin/munin21:40:331
2932069996830,34cyclictest2976183-21apt-get23:50:050
2932069996827,13cyclictest230-21systemd-journal19:40:270
293207199673,6cyclictest2961220-21/usr/sbin/munin22:00:391
293207199673,6cyclictest2940702-21/usr/sbin/munin20:00:241
293207199672,10cyclictest2978273-21/usr/sbin/munin00:00:281
293207199672,10cyclictest2969007-21/usr/sbin/munin22:55:271
293207199671,63cyclictest928-21dropbear23:52:091
2932069996734,4cyclictest2963526-21users22:10:410
293207199663,6cyclictest2971548-21sshd23:16:391
293207199662,8cyclictest2952884-21memory21:10:341
293207199662,7cyclictest2935500-21memory19:25:341
293207199662,7cyclictest2935500-21memory19:25:341
293207199662,6cyclictest2946135-21/usr/sbin/munin20:30:361
293207199662,10cyclictest2979276-21/usr/sbin/munin00:10:301
293207199662,10cyclictest2950371-21/usr/sbin/munin20:55:371
293207199652,9cyclictest2941793-21meminfo20:05:361
293207199652,7cyclictest2947545-21/usr/sbin/munin20:40:271
293207199652,7cyclictest2947545-21/usr/sbin/munin20:40:271
293207199652,7cyclictest2944247-21perl20:20:291
293207199651,12cyclictest2959554-21/usr/sbin/munin21:50:241
293207199651,11cyclictest2960647-21/usr/sbin/munin21:55:331
2932069996535,4cyclictest2971543-21users23:15:330
2932069996533,4cyclictest2960646-21meminfo21:55:330
293207199642,7cyclictest2948385-21/usr/sbin/munin20:45:261
293207199642,6cyclictest2977629-21memory23:55:331
293207199642,6cyclictest2975847-21/usr/sbin/munin23:45:361
293207199642,6cyclictest2975847-21/usr/sbin/munin23:45:361
293207199642,6cyclictest2970151-21meminfo23:05:281
293207199642,6cyclictest2963832-21diskstats22:15:281
293207199642,6cyclictest2938607-21/usr/sbin/munin19:45:381
293207199642,10cyclictest2951937-21perl21:05:311
293207199641,10cyclictest2958708-21df21:45:241
293207199641,10cyclictest2958708-21df21:45:241
293207199641,10cyclictest2937152-21/usr/sbin/munin19:35:331
2932069996441,14cyclictest12-21ksoftirqd/022:45:010
2932069996422,39cyclictest2943858-21tr20:20:000
2932071996332,6cyclictest156250irq/142-800000019:30:321
293207199632,7cyclictest2974437-21users23:35:381
293207199631,9cyclictest2939775-21/usr/sbin/munin19:55:141
293207199631,12cyclictest1549-21snmpd19:43:151
293207199631,11cyclictest2972757-21sshd23:28:391
2932069996327,32cyclictest2966711-21lsblk22:35:250
2932069996323,36cyclictest2956045-21wc21:30:020
2932069996319,39cyclictest2976066-21/usr/sbin/munin23:45:400
2932069996319,39cyclictest2976066-21/usr/sbin/munin23:45:400
293207199623,6cyclictest2942341-21/usr/sbin/munin20:10:391
293207199622,6cyclictest2966859-21/usr/sbin/munin22:35:281
293207199622,5cyclictest2932983-21munin-run19:15:001
293207199621,11cyclictest2954626-21open_inodes21:20:351
2932069996230,28cyclictest2968808-21/usr/sbin/munin22:55:200
2932069996225,10cyclictest2941753-21irqstats20:05:340
2932071996133,25cyclictest156250irq/142-800000000:10:011
293207199612,8cyclictest2965736-21/usr/sbin/munin22:25:321
293207199612,12cyclictest2946822-21if_err_eth120:35:321
293207199611,11cyclictest2974701-21apt-get23:40:181
293207199611,11cyclictest2974701-21apt-get23:40:181
293207199611,10cyclictest1549-21snmpd22:52:381
2932069996124,33cyclictest2944974-21df20:25:280
293207199603,7cyclictest2934663-21/usr/sbin/munin19:20:351
293207199602,7cyclictest2980690-21/usr/sbin/munin00:20:271
293207199602,7cyclictest2980690-21/usr/sbin/munin00:20:271
293207199602,7cyclictest2953644-21/usr/sbin/munin21:15:311
293207199602,6cyclictest2933246-21irqstats19:15:021
293207199601,9cyclictest2950965-21/usr/sbin/munin21:00:271
293207199601,8cyclictest2939247-21apt-get19:50:061
2932071996011,44cyclictest0-21swapper/123:10:061
2932069996045,11cyclictest2961728-21/usr/sbin/munin22:00:370
2932069996030,3cyclictest2964377-21sshd22:16:390
2932069996027,4cyclictest2973828-21apt-get23:35:210
2932069996027,29cyclictest1549-21snmpd00:01:010
2932069996023,9cyclictest2958436-21cron21:45:010
2932069996023,33cyclictest2941010-21proc_pri20:00:370
293207199592,8cyclictest2955776-21sshd21:26:571
293207199592,7cyclictest2971578-21apt-get23:20:051
293207199592,6cyclictest1549-21snmpd00:28:291
293207199591,11cyclictest1549-21snmpd20:17:001
2932069995947,9cyclictest2933550-21/usr/sbin/munin19:15:310
2932069995944,11cyclictest2953649-21/usr/sbin/munin21:15:310
2932069995931,10cyclictest2947556-21/usr/sbin/munin20:40:300
2932069995931,10cyclictest2947556-21/usr/sbin/munin20:40:300
2932069995927,4cyclictest2952353-21sshd21:06:390
2932069995926,4cyclictest2967859-21tune2fs22:45:250
2932069995926,4cyclictest2967859-21tune2fs22:45:250
293206999592,54cyclictest2932536-21dropbear19:10:290
2932069995921,35cyclictest928-21dropbear21:30:270
293207199582,7cyclictest2949245-21diskstats20:50:251
293207199581,9cyclictest1549-21snmpd22:45:571
293207199581,9cyclictest1549-21snmpd22:45:571
293207199581,9cyclictest1549-21snmpd22:33:501
293207199581,8cyclictest1549-21snmpd00:16:051
293207199581,7cyclictest1549-21snmpd22:42:561
2932069995845,10cyclictest2942414-21/usr/sbin/munin20:10:270
2932069995842,13cyclictest2946403-21munin-run20:35:000
2932069995825,8cyclictest1549-21snmpd20:54:310
2932069995823,4cyclictest2954609-21meminfo21:20:330
293207199572,6cyclictest2962368-21memory22:05:371
293207199571,10cyclictest1549-21snmpd00:36:531
2932069995744,10cyclictest1549-21snmpd22:26:130
2932069995740,14cyclictest2959582-21/usr/sbin/munin21:50:270
2932069995732,11cyclictest12-21ksoftirqd/019:10:020
2932069995725,3cyclictest2972187-21df23:25:200
2932069995723,4cyclictest2962502-21sed22:05:400
2932069995723,30cyclictest2950950-21sh21:00:240
2932069995718,37cyclictest0-21swapper/000:20:300
2932069995718,37cyclictest0-21swapper/000:20:300
293207199561,9cyclictest1549-21snmpd23:04:001
2932069995643,10cyclictest2950280-21irqstats20:55:340
2932069995636,16cyclictest2981929-21/usr/sbin/munin00:30:290
2932069995627,9cyclictest2979326-21/usr/sbin/munin00:10:190
2932069995625,9cyclictest2944151-21/usr/sbin/munin20:20:270
2932069995623,29cyclictest2977873-21awk23:55:370
2932069995623,29cyclictest2968519-21sshd22:51:400
2932069995623,28cyclictest2948933-21/usr/sbin/munin20:45:400
2932069995543,9cyclictest2934412-21/usr/sbin/munin19:20:240
2932069995541,10cyclictest2957371-21/usr/sbin/munin21:35:390
2932069995535,16cyclictest2936479-21/usr/sbin/munin19:30:360
2932069995527,10cyclictest2939117-21/usr/sbin/munin19:45:370
2932069995440,11cyclictest2973317-21/usr/sbin/munin23:30:340
2932069995431,19cyclictest2935517-21/usr/sbin/munin19:25:350
2932069995431,19cyclictest2935517-21/usr/sbin/munin19:25:350
2932069995341,9cyclictest2958655-21/usr/sbin/munin21:45:190
2932069995341,9cyclictest2958655-21/usr/sbin/munin21:45:190
2932069995340,11cyclictest2952798-21/usr/sbin/munin21:10:300
2932069995244,4cyclictest2974998-21meminfo23:40:320
2932069995244,4cyclictest2974998-21meminfo23:40:320
2932069995238,10cyclictest1549-21snmpd00:37:240
2932069995223,3cyclictest2980142-21sshd00:18:390
293207199511,14cyclictest2932102-21apt-config19:10:011
2932069995123,23cyclictest2969809-21latency_hist23:05:010
2932069995042,5cyclictest2937083-21ssh19:35:310
2932069995038,9cyclictest1549-21snmpd00:25:590
2932069994935,10cyclictest1549-21snmpd22:32:110
293207499481,22cyclictest0-21swapper/219:40:282
2932069994836,8cyclictest1549-21snmpd19:54:560
2932069994833,10cyclictest1549-21snmpd00:09:530
2932069994823,21cyclictest2970724-21latency_hist23:15:010
2932069994735,9cyclictest1549-21snmpd23:22:270
293207499441,22cyclictest0-21swapper/220:20:252
2932074994220,3cyclictest0-21swapper/222:34:392
293207499411,19cyclictest0-21swapper/222:25:532
293207499304,23cyclictest0-21swapper/222:35:322
293207499302,24cyclictest0-21swapper/221:00:342
293207499296,16cyclictest0-21swapper/223:50:042
293207499291,25cyclictest0-21swapper/220:30:382
293207499283,22cyclictest0-21swapper/220:50:322
293207499282,23cyclictest0-21swapper/221:25:362
293207499282,22cyclictest0-21swapper/220:15:262
2932074992819,5cyclictest0-21swapper/200:24:282
2932074992819,5cyclictest0-21swapper/200:24:282
293207499281,24cyclictest0-21swapper/220:35:402
2932074992722,3cyclictest0-21swapper/223:05:322
293207499272,21cyclictest0-21swapper/221:20:332
293207499271,23cyclictest0-21swapper/200:09:332
293207499271,22cyclictest0-21swapper/222:48:392
293207499271,22cyclictest0-21swapper/222:48:392
293207499271,22cyclictest0-21swapper/221:40:392
293207499271,22cyclictest0-21swapper/220:25:262
293207499265,14cyclictest0-21swapper/219:55:002
293207499262,5cyclictest0-21swapper/221:05:362
2932074992620,4cyclictest0-21swapper/220:40:382
2932074992620,4cyclictest0-21swapper/220:40:382
293207499261,22cyclictest0-21swapper/220:12:382
293207499261,21cyclictest0-21swapper/221:54:392
293207499255,13cyclictest0-21swapper/222:15:002
293207499252,20cyclictest0-21swapper/200:29:392
2932074992520,3cyclictest0-21swapper/222:59:392
2932074992518,4cyclictest0-21swapper/221:33:392
2932074992518,4cyclictest0-21swapper/220:46:532
293207499251,3cyclictest0-21swapper/220:05:292
293207499251,21cyclictest0-21swapper/223:26:392
293207499251,21cyclictest0-21swapper/222:05:352
293207499251,21cyclictest0-21swapper/220:55:332
293207499251,21cyclictest0-21swapper/219:45:302
293207499251,21cyclictest0-21swapper/200:04:402
293207499251,20cyclictest0-21swapper/223:41:402
293207499251,20cyclictest0-21swapper/223:41:402
293207499251,20cyclictest0-21swapper/200:19:402
293207499251,20cyclictest0-21swapper/200:14:392
293207499244,16cyclictest0-21swapper/222:20:352
293207499242,19cyclictest0-21swapper/222:00:472
2932074992418,4cyclictest0-21swapper/223:57:142
293207499241,3cyclictest0-21swapper/223:14:392
293207499241,3cyclictest0-21swapper/222:41:392
293207499241,20cyclictest0-21swapper/223:38:322
293207499241,20cyclictest0-21swapper/221:35:322
293207499241,20cyclictest0-21swapper/221:14:392
293207499241,19cyclictest0-21swapper/223:20:002
293207499231,19cyclictest0-21swapper/222:50:582
293207499231,19cyclictest0-21swapper/221:47:392
293207499231,19cyclictest0-21swapper/221:47:392
2932074992217,3cyclictest0-21swapper/219:27:382
2932074992217,3cyclictest0-21swapper/219:27:382
2932074992215,4cyclictest0-21swapper/219:20:392
293207499221,18cyclictest0-21swapper/223:30:332
293207499221,18cyclictest0-21swapper/200:36:172
293207499215,11cyclictest0-21swapper/221:15:392
293207499214,11cyclictest0-21swapper/219:10:042
293207499213,13cyclictest0-21swapper/200:30:292
2932074992115,4cyclictest0-21swapper/219:38:382
293207499211,3cyclictest0-21swapper/219:32:332
293207499211,15cyclictest0-21swapper/219:20:012
293207499211,12cyclictest0-21swapper/219:10:012
293207499204,11cyclictest0-21swapper/223:20:042
293207499201,3cyclictest0-21swapper/221:55:462
293207499194,9cyclictest0-21swapper/223:45:342
293207499194,9cyclictest0-21swapper/223:45:342
293207499193,11cyclictest0-21swapper/219:55:302
293207499183,11cyclictest0-21swapper/220:00:292
293207499183,10cyclictest0-21swapper/223:05:002
293207499173,11cyclictest0-21swapper/222:15:322
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional