You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-06-16 - 21:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

AMD Embedded R-Series RX-421BD Radeon R7, Linux 4.18.7-rt5 (Profile)

Latency plot of shadow in rack #4, slot #1
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rack4slot1s.osadl.org (updated Wed Mar 10, 2021 12:43:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9032147110,27sleep10-21swapper/107:08:421
1078214795,22sleep00-21swapper/007:10:570
9092145115,21sleep20-21swapper/207:08:472
1081214080,10sleep30-21swapper/307:11:003
140799862,71cyclictest30217-21ntp_kernel_err10:12:481
140799860,1cyclictest10010-31munin-html11:17:501
1407998572,11cyclictest0-21swapper/110:29:591
1412998483,0cyclictest0-21swapper/210:55:442
1407998472,11cyclictest0-21swapper/112:13:091
1407998470,12cyclictest0-21swapper/110:42:451
140799840,82cyclictest0-21swapper/109:27:511
1419998382,0cyclictest0-21swapper/307:25:103
140799830,82cyclictest0-21swapper/109:37:501
1407998272,9cyclictest0-21swapper/107:15:251
1407998269,11cyclictest22576-31munin-html12:22:491
140799820,1cyclictest0-21swapper/111:27:091
140799820,0cyclictest0-21swapper/108:42:241
1407998172,8cyclictest0-21swapper/109:22:581
1407998072,6cyclictest0-21swapper/108:02:591
140799800,79cyclictest0-21swapper/111:46:501
140799800,72cyclictest0-21swapper/111:03:471
140799800,72cyclictest0-21swapper/108:33:081
140799800,2cyclictest20720-21runrttasks10:01:561
140799800,0cyclictest0-21swapper/112:31:331
140799800,0cyclictest0-21swapper/112:11:551
140799800,0cyclictest0-21swapper/111:09:441
140399800,79cyclictest0-21swapper/011:52:020
141299790,78cyclictest0-21swapper/208:17:512
1407997978,0cyclictest32622sleep109:33:141
1407997972,6cyclictest0-21swapper/108:42:391
1407997971,7cyclictest0-21swapper/111:33:141
1407997971,7cyclictest0-21swapper/107:46:231
1407997970,8cyclictest0-21swapper/110:39:331
1407997968,10cyclictest0-21swapper/110:52:411
1407997967,10cyclictest17832-31munin-graph09:07:501
140799790,78cyclictest0-21swapper/107:59:411
140799790,77cyclictest0-21swapper/110:18:171
140799790,72cyclictest0-21swapper/107:56:131
140799790,0cyclictest0-21swapper/111:48:111
140799790,0cyclictest0-21swapper/108:26:331
140799790,0cyclictest0-21swapper/108:09:211
1403997977,1cyclictest24305-21taskset10:06:350
140399796,71cyclictest19725-21uname09:57:470
140399790,77cyclictest0-21swapper/010:42:540
1412997874,3cyclictest26894-21apache209:58:282
1407997872,5cyclictest0-21swapper/109:43:231
1407997872,4cyclictest0-21swapper/109:16:071
1407997871,6cyclictest0-21swapper/109:18:111
1407997870,7cyclictest0-21swapper/110:33:391
1407997870,7cyclictest0-21swapper/110:04:261
1407997870,7cyclictest0-21swapper/107:50:471
1407997870,7cyclictest0-21swapper/107:33:561
1407997869,9cyclictest0-21swapper/112:20:111
1407997868,9cyclictest0-21swapper/111:00:591
1407997867,10cyclictest7396-21ntp_states07:17:481
1407997864,13cyclictest0-21swapper/109:47:421
1407997864,13cyclictest0-21swapper/109:47:421
140799780,77cyclictest0-21swapper/109:53:201
140799780,76cyclictest0-21swapper/107:38:061
140799780,72cyclictest0-21swapper/112:35:021
140799780,70cyclictest0-21swapper/107:32:111
1403997876,1cyclictest101ktimersoftd/011:27:530
141999774,2cyclictest0-21swapper/308:42:403
1412997773,3cyclictest27386-21/usr/sbin/munin08:37:382
141299771,3cyclictest10746-21/usr/share/muni09:47:452
141299771,3cyclictest10746-21/usr/share/muni09:47:452
1407997771,5cyclictest0-21swapper/111:53:121
1407997770,7cyclictest0-21swapper/108:12:531
140799770,0cyclictest0-21swapper/108:49:161
140399771,3cyclictest6922-31munin-html11:12:510
140399771,3cyclictest31817-31munin-graph09:27:510
141999761,72cyclictest7337-21apt-get08:07:403
1412997672,2cyclictest21464-21wc09:12:502
141299761,72cyclictest27132-21python10:07:532
141299761,3cyclictest8015-21grep11:17:402
141299760,75cyclictest0-21swapper/208:57:502
1407997672,3cyclictest0-21swapper/112:02:381
1407997669,5cyclictest0-21swapper/110:47:491
1407997667,8cyclictest0-21swapper/111:58:031
140799760,0cyclictest0-21swapper/111:13:241
1403997674,1cyclictest0-21swapper/007:33:130
1403997672,2cyclictest873-21runrttasks08:15:350
1403997672,2cyclictest256-21systemd-journal07:45:350
1403997672,2cyclictest17532-21apache_accesses08:22:400
1403997671,2cyclictest23882-21apt-get12:27:410
1412997571,3cyclictest19211-31munin-html12:17:492
1412997571,2cyclictest30953-21apache_processe08:42:392
1412997571,2cyclictest1535-31munin-graph10:17:512
141299751,72cyclictest4206-21munin-node09:56:122
141299751,72cyclictest24398-21munin-limits07:42:482
141299751,72cyclictest0-21swapper/212:32:532
141299751,2cyclictest17266-21munin-limits11:27:502
141299750,72cyclictest16222-21/usr/sbin/munin10:42:522
1407997570,5cyclictest0-21swapper/107:23:261
1407997569,5cyclictest0-21swapper/109:02:541
1407997568,6cyclictest0-21swapper/110:23:011
1407997568,5cyclictest0-21swapper/110:07:431
1407997567,7cyclictest0-21swapper/112:37:551
1403997572,2cyclictest15831-21taskset10:41:580
1403997572,1cyclictest2204-21ntp_states07:57:470
1403997572,1cyclictest20725-31munin-graph11:32:530
1403997571,2cyclictest31771-21munin_stats11:02:460
1403997571,2cyclictest0-21swapper/010:17:480
140399750,73cyclictest32695-21df12:37:480
140399750,73cyclictest21895-21entropy10:47:490
140399750,72cyclictest15007-21ntp_kernel_pll_12:12:450
141999741,71cyclictest15658-31munin-html12:12:493
141999741,71cyclictest0-21swapper/307:32:443
141999740,73cyclictest0-21swapper/307:31:523
141999740,72cyclictest7750-21apache_processe09:42:403
141999740,72cyclictest3761-21/usr/share/muni09:37:443
141999740,72cyclictest26918-31munin-graph10:07:523
141999740,72cyclictest2488-21/usr/sbin/munin11:07:453
141999740,72cyclictest1870-31munin-html11:52:493
141999740,72cyclictest0-21swapper/312:18:563
141999740,72cyclictest0-21swapper/311:17:413
1412997471,2cyclictest30598-21diskstats11:47:462
1412997471,1cyclictest0-21swapper/207:32:392
1412997470,2cyclictest0-21swapper/210:37:412
1412997469,3cyclictest4947-21mailstats10:22:492
1412997469,3cyclictest2984-31munin-graph09:32:512
141299741,2cyclictest0-21swapper/211:37:512
141299740,3cyclictest26830-21ntp_states10:57:422
141299740,2cyclictest29793-21/usr/share/muni11:02:442
141299740,1cyclictest17684-21df_inode09:07:482
1407997472,1cyclictest13890-21apache_processe08:17:381
1407997472,1cyclictest0-21swapper/111:27:521
1403997472,1cyclictest7953-21apt-get09:42:420
1403997472,1cyclictest30947-21apache_volume08:42:380
1403997472,1cyclictest2984-31munin-graph09:32:520
1403997472,1cyclictest25822-21/usr/sbin/munin07:47:400
1403997472,1cyclictest1978-21gmain10:16:320
1403997472,1cyclictest10047-21apache_volume12:07:380
1403997472,1cyclictest0-21swapper/010:27:470
1403997472,1cyclictest0-21swapper/009:02:400
1403997471,1cyclictest26225-21/usr/sbin/munin09:22:400
140399740,72cyclictest23006-21apache_accesses10:52:390
140399740,72cyclictest22550-31munin-html12:22:500
140399740,72cyclictest0-21swapper/011:01:540
140399740,1cyclictest10051-31munin-html11:17:500
1419997371,1cyclictest19938-31munin-html09:57:493
1419997370,2cyclictest0-21swapper/310:42:503
1419997370,1cyclictest28335-21df_inode09:22:483
1419997369,1cyclictest9415-21apache_accesses10:32:393
141999730,72cyclictest873-21runrttasks09:18:293
141999730,71cyclictest4293-21/usr/share/muni11:12:493
141999730,71cyclictest30320-21df_inode10:12:483
141999730,71cyclictest23397-21python10:02:483
141999730,71cyclictest21982-31munin-html10:47:503
141999730,71cyclictest0-21swapper/309:02:523
141999730,71cyclictest0-21swapper/308:03:213
141999730,1cyclictest28809-21munin_stats11:47:393
141999730,1cyclictest26805-21sshd08:33:003
141999730,1cyclictest24422-31munin-graph11:37:513
141999730,1cyclictest17794-31munin-html09:07:503
141999730,1cyclictest16258-31munin-html08:17:503
141999730,1cyclictest0-21swapper/310:41:523
1412997371,1cyclictest341ktimersoftd/208:28:312
1412997371,1cyclictest0-21swapper/207:52:502
1412997370,2cyclictest26894-21apache209:39:322
1412997370,2cyclictest0-21swapper/208:00:552
1412997370,1cyclictest26169-21df_inode08:32:482
1412997370,1cyclictest1310-21munin_stats11:07:402
1412997369,2cyclictest15639-21ntp_states12:12:492
1412997369,2cyclictest0-21swapper/210:27:472
141299731,1cyclictest0-21swapper/211:42:432
141299730,71cyclictest691-21snmpd07:39:482
141299730,71cyclictest32212-21vmstat11:52:402
141299730,71cyclictest30520-31munin-graph10:12:502
141299730,71cyclictest11362-21/usr/sbin/munin11:22:482
141299730,2cyclictest5423-21/usr/sbin/munin07:17:512
141299730,1cyclictest9482-31munin-graph08:07:502
141299730,1cyclictest26025-31munin-graph12:27:502
141299730,1cyclictest22520-21df10:02:442
141299730,1cyclictest0-21swapper/210:52:382
141299730,0cyclictest0-21swapper/208:06:222
140799730,72cyclictest0-21swapper/111:38:001
140799730,72cyclictest0-21swapper/108:28:271
140799730,1cyclictest0-21swapper/108:52:481
1403997372,0cyclictest0-21swapper/011:52:540
1403997372,0cyclictest0-21swapper/010:22:590
1403997372,0cyclictest0-21swapper/008:32:580
1403997372,0cyclictest0-21swapper/008:02:440
1403997371,1cyclictest28623-21smartctl08:37:440
1403997371,1cyclictest0-21swapper/011:07:400
1403997371,1cyclictest0-21swapper/009:17:510
1403997371,1cyclictest0-21swapper/008:47:520
1403997370,1cyclictest6227-21df_inode08:52:430
1403997370,1cyclictest30065-21df07:52:440
1403997370,1cyclictest11163-31munin-graph08:57:510
1403997370,1cyclictest0-21swapper/008:07:400
140399730,72cyclictest8876-31munin-graph12:02:510
140399730,72cyclictest0-21swapper/012:32:430
140399730,72cyclictest0-21swapper/012:18:570
140399730,72cyclictest0-21swapper/011:58:040
140399730,72cyclictest0-21swapper/011:43:200
140399730,72cyclictest0-21swapper/011:38:030
140399730,72cyclictest0-21swapper/011:22:560
140399730,72cyclictest0-21swapper/010:32:400
140399730,72cyclictest0-21swapper/010:07:450
140399730,72cyclictest0-21swapper/009:53:160
140399730,72cyclictest0-21swapper/009:47:380
140399730,72cyclictest0-21swapper/009:47:380
140399730,72cyclictest0-21swapper/009:37:410
140399730,72cyclictest0-21swapper/009:07:520
140399730,72cyclictest0-21swapper/008:28:110
140399730,72cyclictest0-21swapper/008:17:450
140399730,72cyclictest0-21swapper/007:27:460
140399730,72cyclictest0-21swapper/007:23:200
140399730,72cyclictest0-21swapper/007:18:270
140399730,72cyclictest0-21swapper/007:13:440
140399730,1cyclictest0-21swapper/009:12:380
140399730,1cyclictest0-21swapper/007:37:500
1419997271,0cyclictest0-21swapper/307:19:403
1419997270,1cyclictest6464-21apache_processe12:02:383
1419997270,1cyclictest31281-31munin-html07:52:493
1419997270,1cyclictest19003-21apache_processe09:12:393
1419997270,1cyclictest0-21swapper/309:47:393
1419997270,1cyclictest0-21swapper/309:47:393
141999721,70cyclictest0-21swapper/312:37:393
141999720,71cyclictest28716-21df_inode10:57:493
141999720,71cyclictest0-21swapper/312:28:013
141999720,71cyclictest0-21swapper/310:21:073
141999720,71cyclictest0-21swapper/308:47:403
141999720,70cyclictest873-21runrttasks10:28:253
141999720,70cyclictest32467-31munin-html11:02:503
141999720,70cyclictest29465-21apache_accesses09:27:383
141999720,70cyclictest23704-21/usr/sbin/munin07:42:453
141999720,70cyclictest22568-21/usr/sbin/munin12:22:493
141999720,70cyclictest0-21swapper/308:27:433
141999720,2cyclictest19271-21/usr/sbin/munin08:22:463
141999720,1cyclictest27542-21users08:37:393
141999720,1cyclictest11916-21/usr/sbin/munin12:07:463
141999720,1cyclictest0-21swapper/309:52:453
141999720,0cyclictest29474-31munin-graph12:32:533
141999720,0cyclictest0-21swapper/309:34:113
1412997270,1cyclictest4592-21/usr/sbin/munin11:12:402
141299721,1cyclictest0-21swapper/210:32:492
141299720,71cyclictest19498-21ntp_states08:22:482
141299720,71cyclictest0-21swapper/211:58:352
141299720,70cyclictest0-21swapper/209:27:482
141299720,1cyclictest9634-21ntp_kernel_err09:42:482
141299720,1cyclictest4214-21/usr/sbin/munin08:47:482
141299720,1cyclictest14107-31munin-graph07:27:502
141299720,1cyclictest0-21swapper/212:22:392
141299720,1cyclictest0-21swapper/208:12:502
141299720,0cyclictest429-31munin-graph12:37:512
141299720,0cyclictest0-21swapper/212:04:442
141299720,0cyclictest0-21swapper/211:33:372
140799720,1cyclictest11022-31munin-html08:57:501
1419997169,1cyclictest7536-31munin-html08:52:493
1419997169,1cyclictest5088-31munin-html10:22:513
1419997169,1cyclictest45422chrt07:15:453
1419997168,2cyclictest25820-21vmstat07:47:393
141999710,69cyclictest27439-31munin-html11:42:503
141999710,69cyclictest21166-31munin-html07:37:503
141999710,69cyclictest0-21swapper/310:52:533
141999710,1cyclictest16635-21memory11:27:463
141999710,1cyclictest11321-21diskstats08:12:433
141999710,1cyclictest0-21swapper/311:32:483
141999710,0cyclictest0-21swapper/311:58:293
141999710,0cyclictest0-21swapper/311:26:013
1412997171,0cyclictest341ktimersoftd/207:22:372
141299710,69cyclictest0-21swapper/209:17:452
141299710,0cyclictest0-21swapper/212:07:402
141299710,0cyclictest0-21swapper/209:22:412
141299710,0cyclictest0-21swapper/209:02:482
141299710,0cyclictest0-21swapper/208:53:082
141299710,0cyclictest0-21swapper/207:48:022
141299710,0cyclictest0-21swapper/207:13:582
141999700,69cyclictest0-21swapper/307:57:383
141999660,0cyclictest8497-21munin-update08:57:493
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional