You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-08-01 - 18:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

ARMv7 Processor rev 3 (v7l), Linux 4.19.97-rt24-v7l (Profile)

Latency plot of shadow in rack #7, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rack7slot2s.osadl.org (updated Tue Jul 20, 2021 00:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2252021410,4sleep022522-21fschecks_count21:33:560
2252021410,4sleep022522-21fschecks_count21:33:560
614021170,3sleep0469599cyclictest20:48:410
614021170,3sleep0469599cyclictest20:48:410
90942980,2sleep00-21swapper/019:24:030
309272950,1sleep0151rcuc/023:49:110
294042710,1sleep00-21swapper/021:53:420
104872680,2sleep10-21swapper/122:29:091
4704996532,8cyclictest26-21ksoftirqd/122:48:541
273192650,5sleep1470499cyclictest20:14:091
129482610,1sleep10-21swapper/122:38:411
4704995529,9cyclictest26-21ksoftirqd/119:43:531
4704995434,7cyclictest26-21ksoftirqd/119:18:531
267402530,1sleep20-21swapper/223:33:422
4704995227,6cyclictest26-21ksoftirqd/122:03:541
235082490,5sleep2471299cyclictest21:34:062
235082490,5sleep2471299cyclictest21:34:062
87102480,1sleep38830-21cat20:54:043
245162450,2sleep20-21swapper/220:08:562
58742440,3sleep35876-21latency_hist22:18:413
470499442,9cyclictest121rcu_preempt23:53:511
469599443,5cyclictest131rcu_sched19:18:530
4695994431,8cyclictest10-21ksoftirqd/021:28:530
160122440,2sleep00-21swapper/022:44:070
470499414,9cyclictest121rcu_preempt22:38:561
470499414,9cyclictest121rcu_preempt22:38:561
47692400,4sleep3472199cyclictest20:44:023
47692400,4sleep3472199cyclictest20:44:023
4712994019,7cyclictest34-21ksoftirqd/219:53:532
4704994032,5cyclictest26-21ksoftirqd/100:03:531
4695994026,10cyclictest10-21ksoftirqd/021:13:550
472199390,5cyclictest13464-21/usr/sbin/munin21:09:033
31092390,1sleep10-21swapper/122:09:031
472199381,5cyclictest17022-21apt-get21:18:553
472199381,31cyclictest0-21swapper/320:08:533
472199381,2cyclictest0-21swapper/322:18:533
472199380,4cyclictest31692-21sort20:28:593
4712993814,6cyclictest34-21ksoftirqd/219:43:522
469599386,5cyclictest121rcu_preempt19:43:530
469599381,7cyclictest121rcu_preempt22:03:540
472199371,3cyclictest0-21swapper/323:18:543
472199371,3cyclictest0-21swapper/300:08:543
472199370,4cyclictest29891-21grep20:23:553
472199370,4cyclictest29891-21grep20:23:553
472199370,3cyclictest9605-21apt-get22:28:553
469599374,23cyclictest31304-21snmpd20:18:540
4695993713,4cyclictest121rcu_preempt22:48:540
472199362,2cyclictest9866-21apt-get20:58:513
472199361,2cyclictest0-21swapper/321:08:113
472199361,2cyclictest0-21swapper/321:08:113
472199360,2cyclictest0-21swapper/323:09:083
4695993623,9cyclictest10-21ksoftirqd/022:08:530
472199351,3cyclictest7838-21apt-get22:23:553
472199351,3cyclictest0-21swapper/319:53:533
472199351,2cyclictest4874-21apt-get19:13:533
472199350,3cyclictest2589-21df_inode22:08:553
472199350,3cyclictest12330-21sed22:34:123
472199350,2cyclictest0-21swapper/322:03:593
4712993525,6cyclictest331ktimersoftd/222:58:522
4712993525,6cyclictest331ktimersoftd/222:58:522
44302350,2sleep00-21swapper/022:13:560
472199341,4cyclictest0-21swapper/323:48:593
472199341,3cyclictest0-21swapper/321:58:543
472199340,3cyclictest19275-21fschecks_time22:58:563
472199340,3cyclictest19275-21fschecks_time22:58:563
472199340,3cyclictest14035-21diskstats00:33:563
472199340,3cyclictest0-21swapper/321:24:073
472199340,3cyclictest0-21swapper/320:03:593
472199340,2cyclictest0-21swapper/321:33:593
472199340,2cyclictest0-21swapper/321:33:593
472199340,2cyclictest0-21swapper/321:28:593
4695993424,6cyclictest10-21ksoftirqd/022:38:560
4695993424,6cyclictest10-21ksoftirqd/022:38:560
297712340,1sleep00-21swapper/020:23:540
297712340,1sleep00-21swapper/020:23:540
263422340,5sleep3472199cyclictest21:44:043
100622340,2sleep00-21swapper/020:58:550
472199331,2cyclictest0-21swapper/322:38:553
472199331,2cyclictest0-21swapper/322:38:553
472199331,2cyclictest0-21swapper/321:53:503
472199330,3cyclictest7311-21processes19:19:033
472199330,3cyclictest0-21swapper/319:23:553
472199330,2cyclictest0-21swapper/323:59:083
4712993324,7cyclictest34-21ksoftirqd/222:48:542
4704993323,6cyclictest26-21ksoftirqd/120:58:511
4695993324,6cyclictest10-21ksoftirqd/000:03:530
4695993317,10cyclictest10-21ksoftirqd/019:58:540
472199322,3cyclictest3157-21apt-get00:03:533
472199322,2cyclictest26190-21apt-get20:13:563
472199321,4cyclictest0-21swapper/321:48:533
472199320,3cyclictest17851-21vmstat22:49:083
472199320,3cyclictest15600-21uname21:13:593
472199320,3cyclictest0-21swapper/323:57:283
472199320,3cyclictest0-21swapper/319:30:233
472199320,2cyclictest0-21swapper/319:34:033
4712993224,6cyclictest331ktimersoftd/222:03:532
4695993222,7cyclictest10-21ksoftirqd/020:53:530
472199311,2cyclictest12161-21apt-get00:28:543
472199310,3cyclictest26708-21apt-get23:33:443
472199310,3cyclictest0-21swapper/300:19:283
472199310,2cyclictest0-21swapper/319:48:273
472199301,5cyclictest11938-21awk00:28:423
472199301,3cyclictest0-21swapper/321:38:533
4721993011,17cyclictest7682-21threads00:14:073
472199300,3cyclictest16875-21chrt00:39:323
472199300,3cyclictest0-21swapper/320:40:353
469599304,4cyclictest131rcu_sched20:28:550
44222305,19sleep00-21swapper/019:11:490
472199291,5cyclictest6385-21fschecks_count20:48:573
472199291,4cyclictest14819-21tail19:39:043
472199291,3cyclictest0-21swapper/323:18:423
472199290,4cyclictest29673-21latency_hist20:23:413
472199290,3cyclictest0-21swapper/322:55:083
472199290,3cyclictest0-21swapper/319:50:273
4704992914,9cyclictest26-21ksoftirqd/119:28:511
4695992923,4cyclictest10-21ksoftirqd/022:58:510
4695992923,4cyclictest10-21ksoftirqd/022:58:510
472199281,5cyclictest930-21sort20:33:553
472199281,5cyclictest29215-21cron23:43:413
472199281,5cyclictest21060-21grep19:58:573
472199281,3cyclictest0-21swapper/323:48:413
472199281,24cyclictest0-21swapper/323:33:423
4712992818,7cyclictest13372sleep223:58:552
470499287,4cyclictest26-21ksoftirqd/119:53:531
469599284,8cyclictest121rcu_preempt21:03:530
469599284,8cyclictest121rcu_preempt21:03:530
472199271,3cyclictest0-21swapper/323:08:413
472199271,2cyclictest0-21swapper/322:43:583
4695992713,8cyclictest10-21ksoftirqd/000:18:560
42722275,16sleep10-21swapper/119:09:521
42582275,16sleep20-21swapper/219:09:412
38442276,11sleep30-21swapper/319:09:053
162472270,5sleep0469599cyclictest00:39:000
472199261,3cyclictest0-21swapper/323:28:413
471299261,8cyclictest0-21swapper/220:23:592
471299261,8cyclictest0-21swapper/220:23:592
4712992614,5cyclictest34-21ksoftirqd/221:43:412
4704992611,6cyclictest26-21ksoftirqd/100:28:421
469599261,3cyclictest131rcu_sched23:23:410
4704992513,3cyclictest26-21ksoftirqd/121:39:041
4712992411,9cyclictest0-21swapper/222:08:522
470499248,3cyclictest26-21ksoftirqd/121:23:421
4704992410,3cyclictest26-21ksoftirqd/121:58:421
4704992410,3cyclictest26-21ksoftirqd/121:28:421
4704992410,3cyclictest26-21ksoftirqd/100:18:411
469599243,14cyclictest31304-21snmpd19:33:520
20732240,5sleep1470499cyclictest23:59:051
470499231,3cyclictest131rcu_sched21:48:421
469599232,3cyclictest0-21swapper/022:03:410
469599231,2cyclictest131rcu_sched22:28:410
471299223,3cyclictest131rcu_sched00:13:412
471299222,6cyclictest34-21ksoftirqd/223:33:432
470499224,3cyclictest6152-21latency_hist20:48:411
470499224,3cyclictest6152-21latency_hist20:48:411
4704992214,5cyclictest26-21ksoftirqd/100:43:421
469599226,4cyclictest121rcu_preempt19:53:540
4695992217,3cyclictest10-21ksoftirqd/000:13:410
471299215,10cyclictest34-21ksoftirqd/219:28:512
471299213,3cyclictest15033-21sed21:13:412
471299211,5cyclictest0-21swapper/219:43:412
4712992112,7cyclictest34-21ksoftirqd/222:18:532
4712992112,3cyclictest34-21ksoftirqd/221:48:422
4712992111,7cyclictest34-21ksoftirqd/220:58:502
4704992113,6cyclictest22648-21users23:09:101
4704992110,7cyclictest26-21ksoftirqd/123:18:551
4704992110,3cyclictest26-21ksoftirqd/100:36:401
471299206,7cyclictest331ktimersoftd/221:53:512
4712992015,3cyclictest34-21ksoftirqd/222:43:412
4712992015,3cyclictest34-21ksoftirqd/222:43:412
470499209,3cyclictest26-21ksoftirqd/123:18:421
470499208,3cyclictest26-21ksoftirqd/120:04:041
4704992014,3cyclictest26-21ksoftirqd/119:18:411
4704992011,7cyclictest22345-21/usr/sbin/munin21:34:061
4704992011,7cyclictest22345-21/usr/sbin/munin21:34:061
4695992016,2cyclictest10-21ksoftirqd/020:43:410
4695992012,4cyclictest0-21swapper/020:13:560
4712991914,3cyclictest31822-21cat23:53:422
471299191,3cyclictest372-21dbus-daemon23:43:412
4712991912,4cyclictest6644-21apt-get19:18:552
4712991911,4cyclictest0-21swapper/221:13:552
470499197,9cyclictest0-21swapper/121:53:511
4704991914,3cyclictest26-21ksoftirqd/121:33:421
4704991911,6cyclictest26-21ksoftirqd/119:23:521
469599193,10cyclictest0-21swapper/021:53:510
469599191,2cyclictest131rcu_sched21:39:040
4695991910,7cyclictest2918-40mandb00:03:410
4712991812,4cyclictest0-21swapper/200:29:002
4712991811,5cyclictest15975-21tail22:44:072
4712991810,3cyclictest34-21ksoftirqd/223:23:412
470499188,3cyclictest26-21ksoftirqd/119:40:571
4704991812,4cyclictest26-21ksoftirqd/120:33:411
4704991811,5cyclictest26-21ksoftirqd/122:18:411
469599185,9cyclictest0-21swapper/022:28:560
469599183,3cyclictest131rcu_sched23:58:420
469599182,2cyclictest131rcu_sched00:28:420
4695991814,3cyclictest10-21ksoftirqd/000:31:320
469599181,3cyclictest131rcu_sched23:48:410
4695991813,3cyclictest10-21ksoftirqd/023:13:420
471299177,8cyclictest18556-21sendmail21:23:412
471299175,10cyclictest0-21swapper/200:03:542
471299171,6cyclictest15481-21latency_hist00:38:412
4712991713,3cyclictest34-21ksoftirqd/223:48:412
4712991713,3cyclictest34-21ksoftirqd/221:59:042
4712991713,2cyclictest331ktimersoftd/222:13:532
471299170,13cyclictest23178-21ls23:13:422
470499173,3cyclictest26-21ksoftirqd/122:24:061
4704991713,2cyclictest26-21ksoftirqd/123:48:411
4704991713,2cyclictest26-21ksoftirqd/100:19:101
4704991712,3cyclictest26-21ksoftirqd/120:28:411
4695991713,2cyclictest10-21ksoftirqd/019:29:040
4695991712,3cyclictest10-21ksoftirqd/020:06:390
4695991711,4cyclictest10-21ksoftirqd/020:53:410
314832170,3sleep10-21swapper/123:50:341
266182170,1sleep00-21swapper/023:32:480
170282171,3sleep20-21swapper/200:41:332
108512171,3sleep20-21swapper/222:30:272
471299163,10cyclictest0-21swapper/221:08:122
471299163,10cyclictest0-21swapper/221:08:122
471299161,5cyclictest8908-21grep22:24:072
471299161,2cyclictest0-21swapper/220:31:252
4712991612,3cyclictest34-21ksoftirqd/223:04:432
4712991612,3cyclictest34-21ksoftirqd/219:33:582
4712991611,3cyclictest34-21ksoftirqd/219:59:012
471299161,10cyclictest10542-21grep00:23:562
470499161,5cyclictest0-21swapper/120:48:541
470499161,4cyclictest7542-21cron22:23:411
470499161,4cyclictest31306-21basename20:28:411
470499161,3cyclictest0-21swapper/119:48:541
4704991611,3cyclictest26-21ksoftirqd/120:03:001
4704991611,3cyclictest26-21ksoftirqd/100:31:341
4704991611,3cyclictest14994-21sh21:13:411
469599167,6cyclictest0-21swapper/019:13:530
469599164,7cyclictest0-21swapper/020:08:540
469599163,10cyclictest562-21runrttasks23:33:540
469599161,2cyclictest131rcu_sched23:18:420
469599161,2cyclictest131rcu_sched21:28:420
4695991612,2cyclictest10-21ksoftirqd/021:48:420
4695991612,2cyclictest10-21ksoftirqd/021:13:410
4695991612,2cyclictest10-21ksoftirqd/019:53:410
4695991612,2cyclictest10-21ksoftirqd/019:39:040
4695991612,2cyclictest10-21ksoftirqd/000:34:000
469599161,13cyclictest18585-21latency_hist21:23:420
4695991611,3cyclictest10-21ksoftirqd/022:19:270
469599161,12cyclictest26084-21latency_hist23:28:410
469599161,10cyclictest6816-21apt-get00:13:550
4695991610,4cyclictest10-21ksoftirqd/022:38:420
278222160,4sleep2471299cyclictest21:48:572
162492160,3sleep10-21swapper/122:46:181
471299159,4cyclictest0-21swapper/220:18:582
471299151,4cyclictest955-21cpuspeed20:33:552
4712991512,2cyclictest34-21ksoftirqd/220:53:412
4712991511,3cyclictest34-21ksoftirqd/220:15:432
4712991511,3cyclictest34-21ksoftirqd/219:24:582
471299151,10cyclictest9515-21awk20:58:422
470499154,3cyclictest26-21ksoftirqd/120:18:531
470499153,5cyclictest26-21ksoftirqd/121:49:261
470499153,4cyclictest0-21swapper/122:58:561
470499153,4cyclictest0-21swapper/122:58:561
470499151,4cyclictest27049-21/usr/sbin/munin23:34:011
470499151,10cyclictest1777-21timerandwakeup20:34:061
4704991510,3cyclictest26-21ksoftirqd/123:31:321
4704991510,3cyclictest26-21ksoftirqd/121:15:001
469599159,4cyclictest10-21ksoftirqd/023:43:420
469599152,4cyclictest0-21swapper/020:33:560
131582150,2sleep10-21swapper/119:34:261
471299148,4cyclictest34-21ksoftirqd/220:39:062
471299148,4cyclictest0-21swapper/221:28:532
471299147,2cyclictest34-21ksoftirqd/200:14:072
471299142,8cyclictest4874-21apt-get19:13:522
471299141,9cyclictest676-21munin-node19:48:532
471299141,6cyclictest32118-21fschecks_count23:53:542
471299141,5cyclictest34-21ksoftirqd/200:22:592
471299141,10cyclictest676-21munin-node20:03:542
4712991410,3cyclictest11859-21missed_timers22:34:042
4712991410,3cyclictest0-21swapper/220:44:102
4712991410,3cyclictest0-21swapper/220:44:102
470499149,3cyclictest26-21ksoftirqd/123:08:411
470499149,3cyclictest11796-21fschecks_count21:03:561
470499149,3cyclictest11796-21fschecks_count21:03:561
470499148,4cyclictest4215-21date20:43:411
470499148,3cyclictest26-21ksoftirqd/120:58:421
4704991410,2cyclictest29210-21apt23:43:411
470499140,10cyclictest18847-21latency_hist22:58:411
469599149,3cyclictest20911-21seq23:04:210
471299131,10cyclictest18952-21ls22:58:422
470499137,4cyclictest26095-21latency_hist23:28:411
4704991310,2cyclictest26-21ksoftirqd/100:09:071
471299128,3cyclictest23301-21seq23:14:222
471299126,5cyclictest0-21swapper/221:24:002
470499128,3cyclictest25941-21idleruntime-cro20:13:411
469599128,3cyclictest18718-21sshd22:57:150
471299118,2cyclictest562-21runrttasks23:27:252
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional