You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-06-05 - 12:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel

x86 Intel Atom D510 @1667 MHz, Linux 3.12.61-rt81 (Profile)

Latency plot of shadow in rack #1, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot6s.osadl.org (updated Mon Jun 05, 2023 00:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
234599890,4rtkit-daemon2344-21rtkit-daemon18:54:193
234599890,4rtkit-daemon2344-21rtkit-daemon18:54:193
234599850,12rtkit-daemon0-21swapper/319:14:103
234599820,7rtkit-daemon0-21swapper/220:01:530
234599790,13rtkit-daemon37-21ksoftirqd/319:59:093
234599780,11rtkit-daemon34-21rcuc/319:44:173
234599770,6rtkit-daemon34-21rcuc/317:14:073
234599720,6rtkit-daemon0-21swapper/319:36:023
234599720,6rtkit-daemon0-21swapper/319:36:023
234599720,10rtkit-daemon0-21swapper/319:19:183
234599710,6rtkit-daemon0-21swapper/318:59:113
234599710,5rtkit-daemon2344-21rtkit-daemon20:04:161
234599700,7rtkit-daemon0-21swapper/319:40:393
234599700,7rtkit-daemon0-21swapper/319:25:173
234599700,6rtkit-daemon0-21swapper/319:12:033
234599690,6rtkit-daemon0-21swapper/317:13:003
2263799695,52cyclictest0-21swapper/222:03:342
234599680,8rtkit-daemon0-21swapper/319:48:543
234599680,6rtkit-daemon0-21swapper/319:31:553
234599680,6rtkit-daemon0-21swapper/319:04:183
234599680,5rtkit-daemon0-21swapper/319:54:313
234599680,4rtkit-daemon2344-21rtkit-daemon18:34:571
234599670,4rtkit-daemon2344-21rtkit-daemon20:40:441
22637996718,23cyclictest10134-21crond18:58:402
22637996718,23cyclictest10134-21crond18:58:402
234599660,5rtkit-daemon2344-21rtkit-daemon22:34:101
234599660,4rtkit-daemon2344-21rtkit-daemon22:15:111
234599660,4rtkit-daemon2344-21rtkit-daemon18:31:191
234599650,5rtkit-daemon2344-21rtkit-daemon21:08:211
234599650,4rtkit-daemon2344-21rtkit-daemon21:28:131
234599650,4rtkit-daemon2344-21rtkit-daemon20:49:141
234599650,4rtkit-daemon2344-21rtkit-daemon20:34:071
234599650,4rtkit-daemon2344-21rtkit-daemon20:19:301
234599650,4rtkit-daemon2344-21rtkit-daemon20:14:231
234599650,4rtkit-daemon2344-21rtkit-daemon20:14:231
234599650,4rtkit-daemon2344-21rtkit-daemon20:02:081
234599650,4rtkit-daemon2344-21rtkit-daemon19:09:551
234599650,4rtkit-daemon2344-21rtkit-daemon18:44:411
234599640,4rtkit-daemon2344-21rtkit-daemon22:31:101
234599640,4rtkit-daemon2344-21rtkit-daemon22:05:411
234599640,4rtkit-daemon2344-21rtkit-daemon22:01:191
234599640,4rtkit-daemon2344-21rtkit-daemon21:55:421
234599640,4rtkit-daemon2344-21rtkit-daemon21:41:051
234599640,4rtkit-daemon2344-21rtkit-daemon20:59:511
234599640,4rtkit-daemon2344-21rtkit-daemon20:27:371
234599640,4rtkit-daemon2344-21rtkit-daemon18:39:571
234599640,4rtkit-daemon2344-21rtkit-daemon17:18:371
234599630,8rtkit-daemon0-21swapper/121:52:341
234599630,4rtkit-daemon2344-21rtkit-daemon18:49:261
234599620,9rtkit-daemon0-21swapper/118:21:271
234599620,9rtkit-daemon0-21swapper/117:30:441
234599620,7rtkit-daemon0-21swapper/121:21:581
234599620,7rtkit-daemon0-21swapper/120:45:371
234599620,3rtkit-daemon18-21rcuc/118:18:421
234599620,3rtkit-daemon18-21rcuc/118:18:421
234599620,3rtkit-daemon0-21swapper/120:56:441
234599620,3rtkit-daemon0-21swapper/120:56:441
234599620,3rtkit-daemon0-21swapper/118:13:431
234599620,3rtkit-daemon0-21swapper/117:21:071
2263099624,49cyclictest0-21swapper/019:47:160
312312613,51sleep00-21swapper/020:49:070
234599610,9rtkit-daemon0-21swapper/118:27:271
234599610,9rtkit-daemon0-21swapper/117:55:361
234599610,9rtkit-daemon0-21swapper/117:38:141
234599610,9rtkit-daemon0-21swapper/117:38:141
234599610,9rtkit-daemon0-21swapper/117:25:291
234599610,8rtkit-daemon0-21swapper/121:12:511
234599610,8rtkit-daemon0-21swapper/118:53:491
234599610,8rtkit-daemon0-21swapper/118:53:491
234599610,7rtkit-daemon0-21swapper/117:59:511
234599610,7rtkit-daemon0-21swapper/117:52:281
234599610,3rtkit-daemon0-21swapper/122:08:411
234599610,3rtkit-daemon0-21swapper/121:34:351
234599610,3rtkit-daemon0-21swapper/121:34:351
234599610,2rtkit-daemon0-21swapper/122:28:031
214762619,43sleep20-21swapper/217:04:122
234599600,5rtkit-daemon0-21swapper/121:28:501
234599600,4rtkit-daemon11554-21users21:14:211
234599600,3rtkit-daemon0-21swapper/120:31:221
234599600,2rtkit-daemon0-21swapper/122:20:331
234599600,2rtkit-daemon0-21swapper/118:04:281
234599600,2rtkit-daemon0-21swapper/117:41:591
2263899605,34cyclictest0-21swapper/322:14:053
234599590,2rtkit-daemon0-21swapper/120:12:451
2263099594,29cyclictest0-21swapper/017:27:500
64772583,47sleep20-21swapper/221:04:172
234599580,4rtkit-daemon18-21rcuc/121:44:121
22637995838,13cyclictest0-21swapper/220:24:062
2263799583,47cyclictest0-21swapper/218:29:192
2263099583,37cyclictest23733-21latency_hist21:43:350
112752583,46sleep30-21swapper/320:08:553
22637995736,14cyclictest0-21swapper/222:14:062
2263099574,27cyclictest0-21swapper/017:39:200
2263099573,13cyclictest0-21swapper/021:48:590
234599560,4rtkit-daemon2344-21rtkit-daemon21:27:580
234599550,4rtkit-daemon2344-21rtkit-daemon21:55:270
2263799555,43cyclictest0-21swapper/221:44:012
22637995534,14cyclictest0-21swapper/221:34:012
22637995534,14cyclictest0-21swapper/221:34:012
22637995534,14cyclictest0-21swapper/219:38:562
22637995529,7cyclictest23733-21latency_hist21:43:342
2263099554,33cyclictest0-21swapper/020:58:520
22630995538,10cyclictest28125-21df20:43:560
306152543,10sleep10-21swapper/119:39:101
234599540,5rtkit-daemon2344-21rtkit-daemon22:30:550
2263899546,29cyclictest0-21swapper/320:29:103
22638995436,11cyclictest10549-21unixbench-2d20:04:163
22637995433,14cyclictest0-21swapper/220:04:102
22637995429,11cyclictest0-21swapper/221:00:352
2263099544,31cyclictest0-21swapper/017:29:210
2263099544,24cyclictest6062-21yum-updatesd-he22:14:100
2263099544,23cyclictest0-21swapper/018:29:160
2263099543,13cyclictest0-21swapper/019:24:190
270862532,9sleep00-21swapper/017:15:130
234599530,5rtkit-daemon2344-21rtkit-daemon21:08:060
2263899535,42cyclictest0-21swapper/321:49:103
22638995320,14cyclictest0-21swapper/320:59:103
2263099534,28cyclictest0-21swapper/022:34:120
22630995334,11cyclictest7579-21grep21:08:560
22630995322,13cyclictest0-21swapper/017:49:200
13252532,9sleep00-21swapper/018:39:020
132122532,9sleep20-21swapper/221:19:042
44772522,9sleep00-21swapper/017:37:590
44772522,9sleep00-21swapper/017:37:590
248852522,9sleep00-21swapper/017:12:310
2263899529,31cyclictest11083-21open_inodes21:14:143
2263899525,28cyclictest0-21swapper/322:18:553
22638995235,10cyclictest25852-21crond18:23:413
2263799524,42cyclictest0-21swapper/218:08:412
2263799524,31cyclictest0-21swapper/217:14:032
22637995235,12cyclictest0-21swapper/220:34:132
2263599524,43cyclictest0-21swapper/119:35:161
2263599524,43cyclictest0-21swapper/119:35:161
2263099524,29cyclictest0-21swapper/019:23:380
2263099524,28cyclictest0-21swapper/021:58:550
22630995221,12cyclictest0-21swapper/018:09:110
22630995217,13cyclictest0-21swapper/021:47:100
224352524,6sleep00-21swapper/017:06:350
2263899518,38cyclictest0-21swapper/321:29:123
2263899515,39cyclictest0-21swapper/318:24:103
2263899515,38cyclictest0-21swapper/320:14:173
2263899515,38cyclictest0-21swapper/320:14:173
2263899515,28cyclictest0-21swapper/320:44:173
2263899514,29cyclictest0-21swapper/321:54:053
2263899514,29cyclictest0-21swapper/320:55:253
2263899514,29cyclictest0-21swapper/320:55:253
2263899514,28cyclictest0-21swapper/321:39:113
2263799514,40cyclictest0-21swapper/222:03:322
22637995121,13cyclictest0-21swapper/222:18:532
2263799512,44cyclictest0-21swapper/217:11:482
2263599515,30cyclictest0-21swapper/118:59:141
2263599514,39cyclictest0-21swapper/117:48:441
22635995122,14cyclictest0-21swapper/119:18:401
2263099514,28cyclictest0-21swapper/020:14:220
2263099514,28cyclictest0-21swapper/020:14:220
2263099514,28cyclictest0-21swapper/018:34:130
2263099514,28cyclictest0-21swapper/017:59:150
2263099514,28cyclictest0-21swapper/017:44:150
2263099514,27cyclictest0-21swapper/018:24:060
22630995135,10cyclictest1282-21grep22:03:520
22630995120,12cyclictest0-21swapper/021:18:590
213212512,9sleep00-21swapper/021:38:090
213212512,9sleep00-21swapper/021:38:090
20662512,9sleep30-21swapper/317:32:543
172402512,42sleep00-21swapper/019:13:330
149472512,43sleep20-21swapper/222:31:002
84872504,39sleep00-21swapper/018:54:010
84872504,39sleep00-21swapper/018:54:010
59732502,9sleep00-21swapper/019:54:410
2263899504,38cyclictest0-21swapper/322:23:593
22638995033,10cyclictest17734-21unixbench_singl20:19:173
22638995033,10cyclictest14651-21cpuspeed_turbos21:23:523
2263899503,30cyclictest0-21swapper/318:18:423
2263899503,30cyclictest0-21swapper/318:18:423
22638995021,22cyclictest16621-21munin-node18:04:073
22638995018,14cyclictest0-21swapper/322:05:003
2263799504,27cyclictest0-21swapper/218:59:162
22637995035,10cyclictest0-21swapper/219:14:012
22637995034,11cyclictest0-21swapper/221:58:332
22637995033,11cyclictest25796-21cut19:29:082
22637995032,11cyclictest960-21tr18:38:572
2263799503,41cyclictest0-21swapper/219:48:582
22637995026,18cyclictest4540-21crond17:38:432
22637995026,18cyclictest4540-21crond17:38:432
22637995024,7cyclictest29986-21latency_hist20:48:362
22637995022,9cyclictest18125-21latency_hist20:23:382
22637995022,8cyclictest16383-21latency_hist18:03:432
2263599505,30cyclictest0-21swapper/119:54:081
2263099504,28cyclictest0-21swapper/019:08:400
2263099504,27cyclictest0-21swapper/021:29:180
2263099504,27cyclictest0-21swapper/020:33:570
2263099504,27cyclictest0-21swapper/020:19:100
2263099503,28cyclictest0-21swapper/020:09:140
2263099503,27cyclictest0-21swapper/020:39:120
115482502,9sleep30-21swapper/317:52:373
56382492,9sleep30-21swapper/322:12:593
234599490,5rtkit-daemon2344-21rtkit-daemon18:44:190
2263899494,38cyclictest0-21swapper/321:19:063
2263899494,26cyclictest0-21swapper/318:35:023
22638994931,11cyclictest26885-21wc20:39:113
22638994930,12cyclictest0-21swapper/317:43:443
2263899493,39cyclictest0-21swapper/322:28:563
2263899492,41cyclictest0-21swapper/317:59:103
2263799495,38cyclictest0-21swapper/221:09:192
2263799494,38cyclictest0-21swapper/221:14:202
2263799494,28cyclictest0-21swapper/218:19:162
2263799494,27cyclictest0-21swapper/217:39:182
22637994934,10cyclictest0-21swapper/222:08:532
2263799492,42cyclictest0-21swapper/217:49:152
2263599496,24cyclictest0-21swapper/119:24:551
2263599494,29cyclictest0-21swapper/119:19:081
2263599494,17cyclictest0-21swapper/119:53:381
2263099494,27cyclictest0-21swapper/021:14:180
2263099494,27cyclictest0-21swapper/020:29:200
2263099494,27cyclictest0-21swapper/020:24:040
2263099494,27cyclictest0-21swapper/019:53:380
2263099494,27cyclictest0-21swapper/018:14:220
2263099494,27cyclictest0-21swapper/018:14:220
2263099494,26cyclictest0-21swapper/019:34:020
2263099494,26cyclictest0-21swapper/019:34:020
2263099493,27cyclictest0-21swapper/022:19:020
22630994919,12cyclictest0-21swapper/019:00:060
294352482,40sleep30-21swapper/317:19:523
2263899484,36cyclictest0-21swapper/321:38:353
2263899484,36cyclictest0-21swapper/321:38:353
2263899482,41cyclictest0-21swapper/317:44:273
2263799484,37cyclictest0-21swapper/221:29:022
2263799484,27cyclictest0-21swapper/220:28:592
2263799484,27cyclictest0-21swapper/219:19:012
2263799484,27cyclictest0-21swapper/218:44:002
22637994833,11cyclictest0-21swapper/217:44:252
2263799483,38cyclictest0-21swapper/218:14:172
2263799483,38cyclictest0-21swapper/218:14:172
22637994826,11cyclictest0-21swapper/217:19:142
22637994819,12cyclictest0-21swapper/218:10:472
2263599484,27cyclictest23059-21cpuspeed_turbos17:09:031
2263099484,26cyclictest0-21swapper/019:29:110
2263099483,38cyclictest12736-21sed17:54:060
2263099483,28cyclictest0-21swapper/022:24:000
22630994815,15cyclictest0-21swapper/022:09:070
14032483,39sleep10-21swapper/119:48:171
75482472,40sleep00-21swapper/018:50:510
302232472,9sleep20-21swapper/220:48:492
2263899476,34cyclictest0-21swapper/318:39:083
2263899476,34cyclictest0-21swapper/317:34:093
2263899476,34cyclictest0-21swapper/317:34:093
22638994731,10cyclictest23772-21fschecks_time20:34:023
2263799474,36cyclictest0-21swapper/220:08:562
2263799474,34cyclictest0-21swapper/219:53:572
2263799474,26cyclictest0-21swapper/219:08:232
22637994732,10cyclictest0-21swapper/218:38:412
22637994731,11cyclictest0-21swapper/217:33:262
22637994727,13cyclictest0-21swapper/222:24:092
2263599474,26cyclictest0-21swapper/119:30:111
22635994726,6cyclictest201ktimersoftd/119:08:411
2263099474,26cyclictest0-21swapper/018:19:000
2263099473,26cyclictest0-21swapper/018:04:000
22630994716,13cyclictest0-21swapper/020:54:090
22630994716,13cyclictest0-21swapper/020:54:090
295362462,9sleep00-21swapper/017:21:100
289082462,9sleep20-21swapper/219:36:122
289082462,9sleep20-21swapper/219:36:122
287882463,8sleep328896-21munin-node18:29:013
2263899464,36cyclictest0-21swapper/317:58:423
22638994629,10cyclictest30380-21cpuspeed_turbos20:48:553
22638994629,10cyclictest17096-21unixbench_multi22:34:143
22638994628,11cyclictest4958-21sed18:44:203
2263799464,35cyclictest0-21swapper/217:54:072
2263099464,25cyclictest0-21swapper/019:39:010
2263099463,26cyclictest0-21swapper/019:18:390
75032452,9sleep20-21swapper/218:50:162
59382452,36sleep30-21swapper/321:04:063
2263899454,14cyclictest0-21swapper/320:25:203
2263899453,35cyclictest0-21swapper/322:03:333
22638994524,14cyclictest0-21swapper/317:24:063
2263799458,30cyclictest0-21swapper/219:58:582
2263799454,35cyclictest0-21swapper/219:24:132
2263799454,34cyclictest0-21swapper/219:44:142
2263799454,33cyclictest0-21swapper/222:33:512
2263799454,26cyclictest0-21swapper/217:24:072
22637994525,14cyclictest0-21swapper/219:09:122
22637994524,15cyclictest0-21swapper/221:48:532
2263099454,14cyclictest0-21swapper/020:04:090
2263899442,37cyclictest0-21swapper/321:10:283
2263799444,33cyclictest0-21swapper/220:39:102
2263799444,33cyclictest0-21swapper/220:13:562
2263799444,33cyclictest0-21swapper/220:13:562
2263799444,33cyclictest0-21swapper/218:23:572
158232442,36sleep20-21swapper/221:24:082
76152432,36sleep30-21swapper/318:51:423
234599430,23rtkit-daemon34-21rcuc/317:04:083
2263799433,33cyclictest0-21swapper/220:54:072
2263799433,33cyclictest0-21swapper/220:54:072
2263899423,33cyclictest0-21swapper/318:09:083
22638994224,11cyclictest24638-21fschecks_time21:43:593
22635994019,13cyclictest1199-21runrttasks17:08:451
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional