You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-27 - 05:37

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #a, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rackaslot0.osadl.org (updated Mon Mar 27, 2023 00:44:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
245672890,1sleep11054599cyclictest21:18:241
202172890,1sleep21054699cyclictest23:53:122
128522880,1sleep119-21rcuc/122:58:101
36832860,1sleep01054499cyclictest23:45:570
106972630,1sleep01718-21runrttasks22:11:540
277662620,1sleep327765-21bash21:10:033
92672590,0sleep00-21swapper/022:31:570
38962580,0sleep30-21swapper/300:38:123
143382580,0sleep1201ktimersoftd/121:28:571
218862570,0sleep221872-21sshd21:29:372
57442550,1sleep25743-21sshd21:42:392
146132540,0sleep10-21swapper/122:41:041
146132540,0sleep10-21swapper/122:41:041
7112530,0sleep20-21swapper/221:53:422
270592530,1sleep10-21swapper/121:41:381
243732530,0sleep326938-21diskmemload21:47:103
10302520,0sleep20-21swapper/200:22:042
289372510,0sleep30-21swapper/322:42:223
289372510,0sleep30-21swapper/322:42:213
248132510,0sleep00-21swapper/021:50:060
259202500,0sleep00-21swapper/021:32:480
197692500,1sleep219677-21sshd21:55:222
310872490,1sleep131066-21sshd23:54:101
1027824922,9sleep00-21swapper/019:09:540
991724825,19sleep30-21swapper/319:05:283
23942480,1sleep310542-21cyclictest21:50:543
130802480,0sleep30-21swapper/323:43:593
1012224825,19sleep20-21swapper/219:07:562
257772470,0sleep010-21rcuc/000:05:310
255602470,0sleep20-21swapper/222:04:362
1009224725,19sleep10-21swapper/119:07:351
4512460,0sleep20-21swapper/222:16:452
8812450,0sleep20-21swapper/200:18:562
242112450,0sleep30-21swapper/323:39:153
179442450,0sleep10-21swapper/100:36:171
102872420,0sleep20-21swapper/200:04:092
102882380,0sleep30-21swapper/321:17:043
133162350,1sleep113319-21sshd23:09:441
14182240,0sleep30-21swapper/322:57:083
10545992220,1cyclictest12046-21sshd22:14:591
281542210,0sleep328156-21id00:12:043
10546992119,2cyclictest0-21swapper/223:32:572
10544992118,2cyclictest31277-21bash21:44:590
86592200,0sleep30-21swapper/323:15:023
10546992018,1cyclictest0-21swapper/221:34:102
10544992018,1cyclictest2513-21sshd23:31:330
322552190,0sleep00-21swapper/000:31:200
233612180,0sleep123353-21sshd21:21:041
10546991817,1cyclictest0-21swapper/222:46:442
10544991817,1cyclictest114750irq/25-eth022:46:470
10544991816,1cyclictest20362-21sshd22:41:350
10544991816,1cyclictest20362-21sshd22:41:350
41912170,1sleep24166-21sshd00:31:462
10547991710,6cyclictest16417-21bash00:17:143
10545991716,1cyclictest820-21sshd23:02:491
10544991715,1cyclictest11567-21sshd21:48:550
10544991714,2cyclictest17559-21ls00:30:000
1054499171,15cyclictest24879-21sshd23:36:220
1054499170,4cyclictest0-21swapper/023:28:430
1054499162,2cyclictest0-21swapper/021:38:550
1054499160,4cyclictest0-21swapper/023:05:480
1054499160,4cyclictest0-21swapper/022:55:380
1054499160,4cyclictest0-21swapper/021:15:300
1054499160,12cyclictest0-21swapper/021:20:050
1054499160,11cyclictest0-21swapper/023:15:430
235092150,0sleep30-21swapper/300:24:173
219792150,0sleep3381ktimersoftd/300:08:233
10547991513,1cyclictest4357-21sshd22:08:273
10547991513,1cyclictest18075-21sshd22:47:073
1054799150,2cyclictest0-21swapper/321:40:213
1054699150,14cyclictest26433-21bash00:08:502
1054599151,1cyclictest6631-21sshd21:48:281
1054499159,3cyclictest0-21swapper/023:55:150
1054499158,3cyclictest14812-21sshd00:20:150
1054499158,3cyclictest0-21swapper/023:51:020
1054499158,3cyclictest0-21swapper/023:41:120
1054499158,3cyclictest0-21swapper/022:06:340
1054499156,0cyclictest724-40kipmi019:10:200
10544991510,1cyclictest3785-21sshd21:10:410
10544991510,1cyclictest26938-21diskmemload21:55:480
1054499150,3cyclictest26938-21diskmemload23:11:370
1054499150,14cyclictest11379-21sshd22:52:190
1054499150,14cyclictest0-21swapper/000:36:430
1054499150,11cyclictest9281-21sshd22:17:330
1054499150,11cyclictest18759-21sshd00:11:070
1054499150,11cyclictest15025-21sshd22:35:210
1054499150,10cyclictest7865-21sshd22:26:030
179702140,0sleep317969-21bash22:26:593
1054799140,13cyclictest0-21swapper/321:55:513
1054699149,4cyclictest0-21swapper/223:01:252
1054699149,3cyclictest0-21swapper/222:56:322
1054599149,4cyclictest16945-21munin-node22:15:231
1054599149,1cyclictest114750irq/25-eth023:39:471
1054599149,1cyclictest0-21swapper/123:31:221
1054599149,1cyclictest0-21swapper/122:49:401
1054599149,1cyclictest0-21swapper/100:01:151
1054599144,1cyclictest17280-21sshd21:55:101
1054599141,9cyclictest114750irq/25-eth022:39:391
10545991410,1cyclictest114750irq/25-eth022:25:341
1054599140,1cyclictest28515-21sshd22:22:101
1054599140,1cyclictest0-21swapper/122:50:421
1054599140,1cyclictest0-21swapper/121:12:201
1054599140,13cyclictest21393-21sshd00:17:431
1054599140,13cyclictest0-21swapper/123:55:391
1054499147,6cyclictest0-21swapper/000:03:280
10544991412,1cyclictest0-21swapper/021:27:180
1054499140,13cyclictest0-21swapper/023:20:280
1054499140,13cyclictest0-21swapper/000:15:210
1054499140,11cyclictest26938-21diskmemload23:00:300
193022130,0sleep319300-21bash22:21:183
158182130,0sleep00-21swapper/022:20:590
1054799138,4cyclictest28776-21sshd23:48:143
1054799138,2cyclictest0-21swapper/323:52:403
1054799131,1cyclictest114750irq/25-eth022:31:073
10547991311,1cyclictest27556-21sshd00:34:093
10547991311,1cyclictest26689-21sshd22:36:253
10547991311,1cyclictest0-21swapper/323:09:353
1054799131,11cyclictest0-21swapper/322:52:313
1054799130,4cyclictest0-21swapper/323:23:093
1054799130,12cyclictest30529-21sshd23:28:223
1054799130,12cyclictest29902-21sshd00:27:593
1054799130,12cyclictest1579-21sshd22:03:253
1054799130,12cyclictest0-21swapper/323:10:013
1054799130,11cyclictest0-21swapper/322:15:473
1054699138,4cyclictest8454-21sshd23:20:392
10546991311,1cyclictest691-21sshd23:48:382
10546991311,1cyclictest31425-21cp00:37:412
10546991311,1cyclictest19715-21bash23:35:532
10546991311,1cyclictest14399-21sshd22:32:262
1054699131,11cyclictest0-21swapper/222:40:212
1054699131,11cyclictest0-21swapper/222:40:212
1054699130,9cyclictest30486-21sshd22:07:542
1054699130,12cyclictest715-21cp21:13:222
1054699130,12cyclictest6104-21sshd22:54:452
1054699130,12cyclictest3438-21sshd23:25:542
1054699130,12cyclictest26938-21diskmemload21:20:432
1054699130,12cyclictest1579-21sshd23:15:092
1054699130,12cyclictest15278-21sshd21:46:202
1054699130,12cyclictest0-21swapper/223:08:482
1054699130,10cyclictest0-21swapper/223:13:142
1054699130,10cyclictest0-21swapper/221:18:002
1054599139,1cyclictest0-21swapper/121:37:201
1054599138,4cyclictest4693-21sshd23:40:201
1054599138,2cyclictest0-21swapper/123:25:561
1054599138,1cyclictest0-21swapper/122:08:221
1054599138,1cyclictest0-21swapper/122:00:491
1054599137,2cyclictest114750irq/25-eth023:45:101
1054599132,10cyclictest0-21swapper/123:20:071
10545991312,1cyclictest32513-21sshd00:25:101
1054599130,1cyclictest114750irq/25-eth000:32:171
1054599130,12cyclictest26910-21cp21:50:161
1054599130,12cyclictest24687-21id21:32:421
1054599130,10cyclictest0-21swapper/100:22:071
10544991311,1cyclictest8330-21cp22:00:090
1054799129,2cyclictest0-21swapper/321:38:543
1054799129,1cyclictest0-21swapper/322:12:583
1054799128,3cyclictest0-21swapper/323:32:593
1054799128,3cyclictest0-21swapper/321:30:073
1054799128,1cyclictest0-21swapper/321:21:433
1054799121,10cyclictest0-21swapper/323:01:063
1054799120,2cyclictest11538-21bash21:25:433
1054799120,11cyclictest0-21swapper/300:00:023
1054699129,2cyclictest0-21swapper/223:58:192
1054699129,2cyclictest0-21swapper/223:40:152
1054699129,2cyclictest0-21swapper/222:35:132
1054699129,2cyclictest0-21swapper/222:20:402
1054699129,2cyclictest0-21swapper/222:10:272
1054699129,2cyclictest0-21swapper/221:37:372
1054699129,2cyclictest0-21swapper/200:26:012
1054699127,2cyclictest934-21sshd00:12:352
1054699120,11cyclictest555-21sshd22:25:272
1054599129,2cyclictest0-21swapper/100:06:301
1054599128,3cyclictest0-21swapper/100:11:401
1054599121,10cyclictest0-21swapper/120:00:161
10545991210,1cyclictest21-21ksoftirqd/123:17:281
1054599120,1cyclictest114750irq/25-eth023:10:381
1054599120,11cyclictest1491-21nfsd22:31:041
1054799110,1cyclictest0-21swapper/320:06:453
1054799110,11cyclictest0-21swapper/319:51:323
1054799110,11cyclictest0-21swapper/319:25:093
1054799110,10cyclictest0-21swapper/323:55:043
1054799110,10cyclictest0-21swapper/319:15:013
1054699110,1cyclictest0-21swapper/221:08:342
1054699110,1cyclictest0-21swapper/220:04:122
1054599119,1cyclictest0-21swapper/121:05:161
1054599110,1cyclictest0-21swapper/121:00:431
1054599110,11cyclictest0-21swapper/120:42:011
1054599110,11cyclictest0-21swapper/119:13:231
1054599110,10cyclictest0-21swapper/120:52:561
1054499119,1cyclictest0-21swapper/020:45:160
1054499119,1cyclictest0-21swapper/019:15:010
1054499111,9cyclictest0-21swapper/021:00:150
1054499111,1cyclictest0-21swapper/020:00:160
1054499110,8cyclictest0-21swapper/020:50:120
1054499110,7cyclictest0-21swapper/019:30:180
1054799109,1cyclictest0-21swapper/321:03:333
1054799108,1cyclictest0-21swapper/320:30:193
1054799100,1cyclictest0-21swapper/319:57:003
1054799100,10cyclictest0-21swapper/321:09:213
1054799100,10cyclictest0-21swapper/320:57:513
1054799100,10cyclictest0-21swapper/320:53:333
1054799100,10cyclictest0-21swapper/320:42:253
1054799100,10cyclictest0-21swapper/320:20:183
1054799100,10cyclictest0-21swapper/319:36:533
1054799100,10cyclictest0-21swapper/319:31:333
1054799100,10cyclictest0-21swapper/319:14:003
1054799100,0cyclictest0-21swapper/320:17:543
1054699103,7cyclictest0-21swapper/219:40:572
1054699100,9cyclictest0-21swapper/221:00:162
1054699100,9cyclictest0-21swapper/220:50:172
1054699100,10cyclictest0-21swapper/220:48:142
1054699100,10cyclictest0-21swapper/220:30:342
1054699100,10cyclictest0-21swapper/220:15:002
1054599109,1cyclictest0-21swapper/120:30:031
1054599109,1cyclictest0-21swapper/119:45:271
1054599102,5cyclictest0-21swapper/120:06:571
1054599100,9cyclictest0-21swapper/119:15:011
1054599100,1cyclictest0-21swapper/120:37:431
1054599100,1cyclictest0-21swapper/119:53:271
1054599100,10cyclictest0-21swapper/120:57:141
1054599100,10cyclictest0-21swapper/119:31:441
1054599100,10cyclictest0-21swapper/119:20:071
1054499105,5cyclictest0-21swapper/020:55:150
1054499100,8cyclictest0-21swapper/019:50:420
1054499100,6cyclictest0-21swapper/020:20:000
105479998,1cyclictest0-21swapper/320:11:513
105479998,1cyclictest0-21swapper/319:47:073
105479990,9cyclictest0-21swapper/320:35:173
105469998,1cyclictest0-21swapper/219:36:122
105469998,1cyclictest0-21swapper/219:18:192
105469990,9cyclictest0-21swapper/220:40:062
105469990,9cyclictest0-21swapper/219:30:032
105469990,8cyclictest0-21swapper/220:13:102
105469990,8cyclictest0-21swapper/219:26:412
105459998,1cyclictest0-21swapper/119:41:501
105459997,1cyclictest16013-21fschecks_time20:45:161
105449998,1cyclictest0-21swapper/020:27:550
105449992,4cyclictest0-21swapper/020:05:060
105449991,8cyclictest0-21swapper/019:37:120
105449990,9cyclictest0-21swapper/019:22:520
105449990,8cyclictest0-21swapper/020:35:200
105449990,1cyclictest0-21swapper/020:20:140
6859280,4sleep121-21ksoftirqd/120:25:211
105479987,1cyclictest0-21swapper/319:43:313
105479987,1cyclictest0-21swapper/319:23:253
105469987,1cyclictest0-21swapper/220:21:102
105469987,1cyclictest0-21swapper/219:55:222
105469987,1cyclictest0-21swapper/219:53:382
105469987,1cyclictest0-21swapper/219:11:072
105459987,1cyclictest0-21swapper/119:55:191
105449987,1cyclictest0-21swapper/021:07:040
105449980,6cyclictest0-21swapper/019:55:020
105449980,6cyclictest0-21swapper/019:45:440
105449980,6cyclictest0-21swapper/019:40:150
105449980,5cyclictest0-21swapper/020:40:450
105449980,5cyclictest0-21swapper/020:30:180
105449980,4cyclictest0-21swapper/019:25:060
105449980,3cyclictest0-21swapper/020:10:200
105479977,0cyclictest0-21swapper/320:00:173
105479976,1cyclictest0-21swapper/320:48:353
105479970,6cyclictest0-21swapper/320:25:153
105469975,1cyclictest7601-21kworker/2:020:35:162
105469971,5cyclictest0-21swapper/219:20:122
105459977,0cyclictest0-21swapper/120:11:561
105459976,1cyclictest0-21swapper/119:35:201
105459971,5cyclictest0-21swapper/120:20:151
105459971,5cyclictest0-21swapper/120:15:161
105459970,6cyclictest0-21swapper/119:27:121
105469962,4cyclictest0-21swapper/220:25:182
105469960,5cyclictest0-21swapper/220:55:142
105469960,5cyclictest0-21swapper/220:05:302
105469960,2cyclictest0-21swapper/219:45:422
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional