You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-07-03 - 19:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Core i7-M620 @2667 MHz, Linux 4.19.106-rt45+ (Profile)

Latency plot of shadow in rack #3, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack3slot6s.osadl.org (updated Sun Jul 03, 2022 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26608211595,15sleep00-21swapper/007:09:430
26340211595,15sleep20-21swapper/207:06:162
26448210787,14sleep30-21swapper/307:07:423
26277210585,15sleep10-21swapper/107:05:271
263242760,0sleep10-21swapper/108:35:121
194852700,0sleep10-21swapper/111:10:171
2666399221,3cyclictest1168-21apt08:55:103
2666299210,19cyclictest0-21swapper/208:25:122
2666399200,2cyclictest0-21swapper/312:30:113
2666399200,2cyclictest0-21swapper/311:40:133
26660992017,2cyclictest0-21swapper/010:25:130
26663991910,2cyclictest0-21swapper/312:25:253
2666399190,2cyclictest0-21swapper/307:15:223
26661991914,3cyclictest0-21swapper/109:59:591
2666399182,6cyclictest0-21swapper/308:45:243
2666399181,9cyclictest28507-21munin-run07:15:003
2666399180,8cyclictest10434-21pzem_current07:50:193
2666399180,2cyclictest26875-21munin-node3
2666399180,11cyclictest0-21swapper/310:30:123
26660991815,2cyclictest0-21swapper/009:35:120
2666399173,6cyclictest286-21jbd2/sda1-808:25:003
26663991713,2cyclictest0-21swapper/310:56:593
26663991713,2cyclictest0-21swapper/310:46:593
26661991714,2cyclictest0-21swapper/110:05:581
2666399167,2cyclictest0-21swapper/307:38:583
2666399162,6cyclictest0-21swapper/312:08:593
2666399162,6cyclictest0-21swapper/308:37:593
2666399160,7cyclictest0-21swapper/310:07:593
2666399160,2cyclictest0-21swapper/311:48:083
26662991612,2cyclictest0-21swapper/211:25:162
2666199165,7cyclictest3563-21Xorg11:25:591
26661991612,2cyclictest0-21swapper/109:08:581
26661991612,2cyclictest0-21swapper/109:04:591
2666199160,2cyclictest0-21swapper/112:30:121
26660991613,2cyclictest24998-21kworker/u8:008:34:040
2666099160,11cyclictest0-21swapper/010:35:040
2666399158,2cyclictest0-21swapper/311:50:183
2666399158,2cyclictest0-21swapper/309:13:383
2666399158,1cyclictest0-21swapper/309:25:153
2666399151,5cyclictest0-21swapper/309:05:003
26663991510,3cyclictest0-21swapper/307:30:163
26662991510,3cyclictest18858-21cut09:40:232
2666199152,7cyclictest0-21swapper/111:40:001
2666199152,7cyclictest0-21swapper/107:20:001
2666199151,3cyclictest13662-21diskstats10:55:121
2666199151,3cyclictest10717-21apt-get12:15:141
26661991510,3cyclictest0-21swapper/108:20:131
2666199150,13cyclictest13153-21sed08:00:001
26660991512,2cyclictest32067-21kworker/u8:107:36:040
26660991512,2cyclictest32067-21kworker/u8:107:26:030
26660991512,2cyclictest24998-21kworker/u8:008:26:030
26660991512,2cyclictest22101-21kworker/u8:011:49:040
26660991512,2cyclictest16126-21kworker/u8:009:53:040
26660991511,2cyclictest0-21swapper/011:10:110
2666399149,3cyclictest6416-21munin-node3
2666399140,13cyclictest0-21swapper/311:10:183
2666399140,12cyclictest25005-21memory08:30:193
2666299149,3cyclictest0-21swapper/207:55:112
2666299142,7cyclictest0-21swapper/209:55:002
2666299142,6cyclictest0-21swapper/212:39:592
2666299142,6cyclictest0-21swapper/208:35:002
2666299141,7cyclictest0-21swapper/211:05:002
2666299140,2cyclictest0-21swapper/208:40:262
2666299140,12cyclictest30779-21cut08:45:192
2666299140,12cyclictest10581-21cut10:45:182
2666299140,12cyclictest0-21swapper/212:10:182
2666299140,12cyclictest0-21swapper/208:10:212
2666199142,7cyclictest0-21swapper/109:50:011
2666199142,6cyclictest0-21swapper/110:15:011
2666199141,2cyclictest28220-21apt-get11:35:121
2666199140,12cyclictest0-21swapper/108:30:181
26660991411,2cyclictest6631-21kworker/u8:109:13:040
26660991411,2cyclictest32067-21kworker/u8:107:58:040
26660991411,2cyclictest32067-21kworker/u8:107:22:040
26660991411,2cyclictest28595-21kworker/u8:210:22:030
26660991411,2cyclictest28595-21kworker/u8:210:15:040
26660991411,2cyclictest28595-21kworker/u8:210:11:040
26660991411,2cyclictest18488-21kworker/u8:211:19:040
26660991411,2cyclictest16126-21kworker/u8:009:46:040
2666099140,9cyclictest0-21swapper/009:09:040
2666099140,12cyclictest30088-21cpu11:40:140
2666399138,3cyclictest3814-21cat11:55:143
2666399138,3cyclictest15997-21munin-node3
26663991310,2cyclictest0-21swapper/309:15:203
26663991310,2cyclictest0-21swapper/309:15:193
26663991310,2cyclictest0-21swapper/307:45:113
2666399130,2cyclictest0-21swapper/310:50:113
2666399130,12cyclictest0-21swapper/312:20:173
2666399130,12cyclictest0-21swapper/312:15:223
2666299139,2cyclictest0-21swapper/209:20:202
2666299132,6cyclictest0-21swapper/209:40:012
2666299131,2cyclictest0-21swapper/212:15:222
2666299131,10cyclictest22313-21cron08:25:002
2666299130,2cyclictest0-21swapper/210:25:172
2666299130,2cyclictest0-21swapper/208:05:202
2666299130,11cyclictest0-21swapper/210:55:192
2666299130,11cyclictest0-21swapper/207:45:212
2666299130,11cyclictest0-21swapper/207:20:152
2666199135,6cyclictest0-21swapper/112:05:121
2666199132,8cyclictest0-21swapper/111:31:591
2666199132,8cyclictest0-21swapper/107:50:581
2666199132,7cyclictest0-21swapper/110:50:001
2666199132,6cyclictest0-21swapper/110:25:001
2666199131,3cyclictest0-21swapper/110:30:181
2666199131,3cyclictest0-21swapper/110:25:141
26661991310,2cyclictest0-21swapper/109:30:311
26661991310,2cyclictest0-21swapper/108:00:241
26661991310,2cyclictest0-21swapper/107:45:221
2666099139,2cyclictest26938-21ntp_kernel_err08:35:170
26660991310,2cyclictest7575-21kworker/u8:010:56:030
26660991310,2cyclictest32067-21kworker/u8:108:00:040
26660991310,2cyclictest32067-21kworker/u8:107:43:040
26660991310,2cyclictest32067-21kworker/u8:107:31:040
26660991310,2cyclictest29408-21kworker/u8:209:02:040
26660991310,2cyclictest29408-21kworker/u8:208:44:040
26660991310,2cyclictest24998-21kworker/u8:007:11:040
26660991310,2cyclictest22101-21kworker/u8:012:00:030
26660991310,2cyclictest22101-21kworker/u8:011:56:030
26660991310,2cyclictest20484-21kworker/u8:208:22:040
26660991310,2cyclictest18488-21kworker/u8:211:09:040
26660991310,2cyclictest16126-21kworker/u8:009:40:030
26660991310,2cyclictest10191-21kworker/u8:212:28:030
26660991310,2cyclictest0-21swapper/011:00:000
26660991310,2cyclictest0-21swapper/010:45:000
26660991310,2cyclictest0-21swapper/007:50:240
2666099130,2cyclictest0-21swapper/012:06:040
2666099130,2cyclictest0-21swapper/011:35:120
2666099130,2cyclictest0-21swapper/009:20:110
2666099130,12cyclictest0-21swapper/010:05:210
2666099130,11cyclictest25394-21apt-get10:00:150
2666399128,2cyclictest0-21swapper/310:00:133
2666399127,3cyclictest374-21timerandwakeup08:50:243
2666399122,7cyclictest0-21swapper/308:03:583
2666399122,5cyclictest0-21swapper/308:29:593
2666399120,2cyclictest0-21swapper/308:10:203
2666399120,2cyclictest0-21swapper/307:20:203
2666399120,11cyclictest0-21swapper/311:25:143
2666399120,11cyclictest0-21swapper/311:20:253
2666399120,11cyclictest0-21swapper/309:40:133
2666299129,2cyclictest0-21swapper/208:55:112
2666299127,3cyclictest7218-21sshd07:42:082
2666299122,7cyclictest0-21swapper/212:07:592
2666299122,7cyclictest0-21swapper/211:06:592
2666299122,7cyclictest0-21swapper/210:35:592
2666299122,7cyclictest0-21swapper/210:16:012
2666299121,9cyclictest28003-21munin-run11:35:002
2666299121,6cyclictest0-21swapper/211:49:592
2666299120,2cyclictest0-21swapper/212:20:222
2666199128,3cyclictest0-21swapper/108:07:591
2666199128,2cyclictest0-21swapper/107:25:181
2666199127,3cyclictest18105-21munin-node1
2666199127,3cyclictest17127-21munin-node1
2666199127,3cyclictest0-21swapper/108:55:221
2666199122,7cyclictest0-21swapper/111:47:011
2666199122,7cyclictest0-21swapper/108:54:591
2666199122,7cyclictest0-21swapper/108:15:011
2666199121,9cyclictest9813-21sed09:15:251
2666199121,9cyclictest9813-21sed09:15:251
2666199121,9cyclictest0-21swapper/111:50:121
2666199121,9cyclictest0-21swapper/111:15:141
2666199121,9cyclictest0-21swapper/110:50:241
2666199121,9cyclictest0-21swapper/110:15:141
2666199121,9cyclictest0-21swapper/110:00:261
2666199121,3cyclictest0-21swapper/112:10:151
2666199121,3cyclictest0-21swapper/111:55:141
2666199121,3cyclictest0-21swapper/108:45:171
2666199120,2cyclictest0-21swapper/112:00:141
2666199120,2cyclictest0-21swapper/109:10:121
2666199120,1cyclictest0-21swapper/109:50:011
2666199120,11cyclictest0-21swapper/111:00:141
2666199120,11cyclictest0-21swapper/109:35:201
2666099129,2cyclictest32067-21kworker/u8:108:06:040
2666099129,2cyclictest29408-21kworker/u8:208:59:040
2666099129,2cyclictest10191-21kworker/u8:212:14:040
2666099129,2cyclictest0-21swapper/010:30:270
2666099129,2cyclictest0-21swapper/008:11:590
2666099120,8cyclictest0-21swapper/009:31:040
2666099120,2cyclictest0-21swapper/009:15:180
2666099120,2cyclictest0-21swapper/009:15:170
2666099120,2cyclictest0-21swapper/007:15:350
2666099120,11cyclictest0-21swapper/011:25:220
2666099120,11cyclictest0-21swapper/009:25:260
2666399118,2cyclictest0-21swapper/310:15:583
2666399118,2cyclictest0-21swapper/307:55:233
2666399117,2cyclictest22455-21munin-run11:20:003
2666399112,6cyclictest0-21swapper/311:02:593
2666399112,6cyclictest0-21swapper/310:38:023
2666399112,6cyclictest0-21swapper/308:19:593
2666399112,5cyclictest0-21swapper/310:25:003
2666399110,9cyclictest29408-21grep11:35:263
2666399110,10cyclictest0-21swapper/311:09:073
2666299118,2cyclictest0-21swapper/211:10:142
2666299118,2cyclictest0-21swapper/210:10:212
2666299113,6cyclictest0-21swapper/208:03:372
2666299113,5cyclictest0-21swapper/208:51:022
2666299112,6cyclictest0-21swapper/211:43:592
2666299112,6cyclictest0-21swapper/209:26:582
2666299112,6cyclictest0-21swapper/207:51:022
2666299111,8cyclictest9522-21latency_hist10:45:002
2666299111,7cyclictest0-21swapper/208:16:592
2666299110,10cyclictest0-21swapper/210:30:272
2666199117,2cyclictest14929-21iostat_ios12:25:151
2666199117,2cyclictest0-21swapper/108:25:211
2666199117,2cyclictest0-21swapper/107:40:171
2666199117,2cyclictest0-21swapper/107:35:191
2666199116,3cyclictest6035-21munin-node1
2666199116,3cyclictest18703-21ntpq09:40:181
2666199116,3cyclictest0-21swapper/112:20:121
2666199116,3cyclictest0-21swapper/108:40:171
2666199111,8cyclictest8317-21hddtemp_smartct10:40:131
2666199111,8cyclictest0-21swapper/107:20:111
2666199111,7cyclictest4532-21gdm-simple-gree11:23:581
2666199111,3cyclictest0-21swapper/107:10:161
2666199110,10cyclictest0-21swapper/107:30:171
2666099118,2cyclictest0-21swapper/012:35:130
2666099118,2cyclictest0-21swapper/012:15:200
2666099118,2cyclictest0-21swapper/011:30:170
2666099118,2cyclictest0-21swapper/011:20:250
2666099118,2cyclictest0-21swapper/007:49:590
2666099111,6cyclictest0-21swapper/010:40:230
2666099110,9cyclictest3323-21snmpd10:54:110
2666099110,7cyclictest0-21swapper/008:51:040
2666099110,2cyclictest0-21swapper/011:54:040
183112111,7sleep318403-21fschecks_count12:35:143
2666399107,2cyclictest0-21swapper/308:40:233
2666399102,6cyclictest0-21swapper/307:40:253
2666399102,5cyclictest0-21swapper/309:33:593
2666399100,9cyclictest0-21swapper/310:14:593
2666399100,6cyclictest0-21swapper/309:55:233
2666399100,6cyclictest0-21swapper/309:35:263
2666299107,2cyclictest0-21swapper/210:05:162
2666299104,4cyclictest29843-21sendmail11:40:002
2666299104,4cyclictest28554-21wc07:14:592
2666299103,6cyclictest0-21swapper/210:50:222
2666299102,7cyclictest0-21swapper/210:01:592
2666299102,6cyclictest0-21swapper/209:10:252
2666299102,6cyclictest0-21swapper/209:00:232
2666299102,6cyclictest0-21swapper/208:35:232
2666299101,6cyclictest0-21swapper/211:20:222
2666299101,6cyclictest0-21swapper/209:59:002
2666299100,7cyclictest0-21swapper/207:37:582
2666199105,3cyclictest18610-21cron08:15:001
2666199100,3cyclictest10600-21munin-node1
2666099107,2cyclictest0-21swapper/012:20:140
2666099107,2cyclictest0-21swapper/008:45:260
2666099102,6cyclictest0-21swapper/008:15:220
2666099101,5cyclictest0-21swapper/010:00:010
4579290,1sleep20-21swapper/211:55:262
266639996,2cyclictest0-21swapper/310:29:183
266639996,2cyclictest0-21swapper/309:50:273
266639996,2cyclictest0-21swapper/309:05:143
266639994,3cyclictest8542-21wc10:40:153
266639992,4cyclictest0-21swapper/312:14:023
266639991,6cyclictest0-21swapper/309:45:123
266629996,2cyclictest0-21swapper/212:30:192
266629996,2cyclictest0-21swapper/209:05:202
266629994,3cyclictest66712sleep212:03:272
266629994,3cyclictest29423-21cut07:15:312
266629994,3cyclictest20907-21grep11:15:112
266629994,3cyclictest1746-21hwlatdetect11:50:152
266629994,3cyclictest14882-21if_err_eth012:25:152
266629994,3cyclictest1102-21proc_pri10:20:212
266629994,3cyclictest0-21swapper/207:25:122
266629990,6cyclictest0-21swapper/209:48:592
266619994,3cyclictest12593-21cpuspeed_turbos09:25:121
266609990,2cyclictest0-21swapper/012:30:030
266639985,2cyclictest0-21swapper/307:25:083
266639983,3cyclictest10736-21tr09:20:113
266629983,3cyclictest2311-21hwlatdetect07:30:182
266629983,3cyclictest14279-21chrt09:30:022
266629980,7cyclictest0-21swapper/209:15:062
266629980,7cyclictest0-21swapper/209:15:062
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional