You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2022-01-23 - 20:05

Intel(R) Core(TM) i7 CPU E 610 @ 2.53GHz, Linux 5.4.123-rt59 (Profile)

Latency plot of system in rack #5, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4.osadl.org (updated Sun Jan 23, 2022 12:44:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1466729889,5sleep10-21swapper/107:07:151
1354729074,10sleep30-21swapper/307:05:243
1475528374,5sleep20-21swapper/207:08:242
1477827757,15sleep00-21swapper/007:08:420
160432750,0sleep20-21swapper/212:35:272
79132680,1sleep07914-21expr12:30:160
309462670,1sleep230947-21sh09:27:342
286452660,0sleep30-21swapper/309:47:433
146702640,0sleep30-21swapper/310:21:463
150632510,0sleep10-21swapper/111:28:551
68992280,0sleep10-21swapper/109:33:191
125532250,0sleep00-21swapper/009:15:210
1509899241,21cyclictest3595-21diskmemload10:50:210
15098992320,2cyclictest0-21swapper/011:48:420
1510299221,20cyclictest0-21swapper/108:36:061
1509899221,20cyclictest0-21swapper/010:20:070
1509899221,20cyclictest0-21swapper/007:31:200
1509899220,20cyclictest0-21swapper/008:20:220
1511199210,20cyclictest0-21swapper/310:39:373
15102992118,2cyclictest227850irq/25-eth010:18:141
1510299210,2cyclictest0-21swapper/110:30:211
1510299210,20cyclictest0-21swapper/110:48:071
1509899211,18cyclictest0-21swapper/009:48:110
1509899210,20cyclictest0-21swapper/010:25:520
1509899210,19cyclictest27869-21chrt11:59:450
1509899210,19cyclictest20714-21ssh11:32:230
121042210,0sleep20-21swapper/212:32:552
1511199200,19cyclictest0-21swapper/309:57:223
1511199200,19cyclictest0-21swapper/309:30:213
1511199200,19cyclictest0-21swapper/307:52:363
1511199200,19cyclictest0-21swapper/307:42:203
15102992017,2cyclictest227850irq/25-eth009:20:221
15102992017,2cyclictest227850irq/25-eth007:25:281
15102992010,9cyclictest0-21swapper/108:55:181
1510299200,19cyclictest0-21swapper/111:54:231
1510299200,19cyclictest0-21swapper/110:22:061
1510299200,14cyclictest0-21swapper/108:23:061
1509899201,2cyclictest9770-21top09:11:550
1509899200,5cyclictest0-21swapper/010:40:180
1509899200,18cyclictest14363-21ssh09:38:160
1509899200,18cyclictest0-21swapper/010:16:150
1509899200,14cyclictest0-21swapper/008:45:260
1510299194,14cyclictest0-21swapper/112:00:171
15102991916,2cyclictest0-21swapper/112:10:161
15102991915,2cyclictest227850irq/25-eth011:12:031
1510299191,2cyclictest0-21swapper/107:30:161
15098991914,3cyclictest14191-21ssh11:50:160
1509899190,2cyclictest0-21swapper/011:35:290
1511199180,17cyclictest0-21swapper/310:30:163
15102991815,2cyclictest227850irq/25-eth011:33:501
1509899189,2cyclictest0-21swapper/008:25:310
15098991815,2cyclictest0-21swapper/011:16:290
15098991813,3cyclictest24111-21ssh10:06:310
15098991813,3cyclictest21243-21ssh09:42:410
1509899181,15cyclictest9488-21ssh11:02:410
1509899181,15cyclictest8281-21ssh11:24:180
1509899180,2cyclictest0-21swapper/009:24:060
1509899180,1cyclictest0-21swapper/011:26:330
1511199171,13cyclictest0-21swapper/309:17:453
1510299171,9cyclictest0-21swapper/112:24:221
15102991715,1cyclictest227850irq/25-eth012:39:451
15102991715,1cyclictest227850irq/25-eth011:24:061
15102991714,2cyclictest227850irq/25-eth011:57:121
15102991714,2cyclictest227850irq/25-eth010:55:101
15102991714,2cyclictest227850irq/25-eth009:43:531
1510299171,15cyclictest0-21swapper/108:12:501
1510299170,2cyclictest0-21swapper/110:40:181
1509899171,3cyclictest31661-21ssh10:11:290
1509899171,3cyclictest19338-21ssh10:03:540
15098991713,2cyclictest0-21swapper/012:10:070
15098991713,2cyclictest0-21swapper/009:29:030
15098991712,3cyclictest4103-21ssh12:27:250
15098991712,3cyclictest31511-21ssh12:24:300
15098991712,3cyclictest2756-21ssh11:41:590
15098991712,3cyclictest17967-21ssh10:45:440
1509899171,14cyclictest5470-21ssh09:32:030
1509899171,14cyclictest4761-21ssh10:37:060
1509899171,14cyclictest1374-21ssh09:50:540
1509899170,3cyclictest24722-21ssh11:13:070
1509899170,15cyclictest0-21swapper/010:30:160
1509899170,15cyclictest0-21swapper/009:55:140
1511199161,13cyclictest0-21swapper/312:36:173
15102991613,2cyclictest227850irq/25-eth012:27:341
15102991613,2cyclictest227850irq/25-eth011:38:151
15102991613,2cyclictest227850irq/25-eth011:03:221
15102991613,2cyclictest227850irq/25-eth010:52:091
15102991613,2cyclictest227850irq/25-eth010:26:581
15102991613,2cyclictest227850irq/25-eth010:11:161
15102991613,2cyclictest227850irq/25-eth010:07:081
15102991613,2cyclictest227850irq/25-eth009:58:441
15102991613,2cyclictest227850irq/25-eth009:52:501
15102991613,2cyclictest227850irq/25-eth009:45:071
15102991613,2cyclictest227850irq/25-eth009:36:341
15102991613,2cyclictest227850irq/25-eth009:26:591
15102991613,2cyclictest227850irq/25-eth009:13:221
1510299161,13cyclictest0-21swapper/110:00:211
1509899167,8cyclictest0-21swapper/012:15:160
15098991613,2cyclictest0-21swapper/012:08:350
15098991613,2cyclictest0-21swapper/011:05:140
1509899161,13cyclictest8899-21grep08:05:320
15098991611,3cyclictest32517-21ssh10:55:530
1509899160,14cyclictest0-21swapper/012:35:530
1509899160,14cyclictest0-21swapper/012:01:520
1511199151,13cyclictest0-21swapper/309:54:233
1511199151,12cyclictest0-21swapper/310:05:223
1511199150,13cyclictest0-21swapper/312:05:213
1511199150,13cyclictest0-21swapper/311:19:393
15106991510,3cyclictest16029-21ssh10:22:592
15106991510,3cyclictest15135-21ntp_offset08:20:272
15102991513,1cyclictest227850irq/25-eth009:15:071
15102991512,2cyclictest227850irq/25-eth012:30:381
15102991512,2cyclictest227850irq/25-eth012:07:511
15102991512,2cyclictest227850irq/25-eth011:41:111
15102991512,2cyclictest227850irq/25-eth011:15:321
15102991512,2cyclictest227850irq/25-eth011:05:401
15102991512,2cyclictest227850irq/25-eth010:35:511
1510299150,4cyclictest0-21swapper/112:15:241
15098991512,2cyclictest0-21swapper/007:15:220
15098991512,2cyclictest0-21swapper/007:15:220
1509899151,13cyclictest0-21swapper/007:43:510
1509899150,14cyclictest0-21swapper/009:00:140
37162140,0sleep30-21swapper/311:20:283
15111991411,2cyclictest0-21swapper/310:40:283
1511199141,12cyclictest0-21swapper/309:20:443
1511199140,8cyclictest0-21swapper/310:18:223
1511199140,2cyclictest0-21swapper/311:03:063
1511199140,13cyclictest0-21swapper/311:45:213
1511199140,13cyclictest0-21swapper/311:30:593
1511199140,13cyclictest0-21swapper/311:29:303
1511199140,13cyclictest0-21swapper/310:00:233
1511199140,13cyclictest0-21swapper/309:42:073
1511199140,13cyclictest0-21swapper/309:26:513
1511199140,13cyclictest0-21swapper/308:10:233
1510699141,9cyclictest26-21rcuc/211:18:172
15106991411,2cyclictest0-21swapper/212:00:272
15106991411,2cyclictest0-21swapper/209:30:192
15106991411,2cyclictest0-21swapper/207:30:172
1510699140,3cyclictest20984-21ssh11:32:372
1510699140,12cyclictest6205-21ssh09:55:112
1510699140,12cyclictest6090-21ssh11:44:492
1510699140,12cyclictest0-21swapper/211:20:202
1510299145,8cyclictest0-21swapper/108:28:051
15102991411,2cyclictest227850irq/25-eth011:46:221
1510299140,8cyclictest0-21swapper/107:51:051
1510299140,13cyclictest0-21swapper/108:50:221
15098991412,1cyclictest0-21swapper/008:54:150
15098991411,2cyclictest0-21swapper/007:58:110
15098991411,2cyclictest0-21swapper/007:37:050
1509899140,13cyclictest0-21swapper/009:05:220
310552130,1sleep20-21swapper/207:45:142
15111991311,1cyclictest0-21swapper/312:03:503
1511199131,10cyclictest0-21swapper/311:52:373
15111991310,2cyclictest0-21swapper/312:10:073
15111991310,2cyclictest0-21swapper/311:55:113
15111991310,2cyclictest0-21swapper/311:35:113
15111991310,2cyclictest0-21swapper/311:11:523
15111991310,2cyclictest0-21swapper/310:56:543
15111991310,2cyclictest0-21swapper/310:11:523
15111991310,2cyclictest0-21swapper/309:35:273
15111991310,2cyclictest0-21swapper/308:40:163
15111991310,2cyclictest0-21swapper/307:21:203
1511199130,2cyclictest0-21swapper/310:45:293
1511199130,2cyclictest0-21swapper/307:10:203
1511199130,12cyclictest0-21swapper/312:15:573
1511199130,12cyclictest0-21swapper/311:06:573
1511199130,12cyclictest0-21swapper/310:25:013
1511199130,12cyclictest0-21swapper/309:10:443
1511199130,11cyclictest0-21swapper/312:30:183
1510699139,2cyclictest12976-21ssh10:42:532
1510699138,4cyclictest5544-21kworker/u8:312:10:452
15106991310,2cyclictest3790-21apt-get10:15:182
15106991310,2cyclictest28160-21kworker/u8:010:05:402
15106991310,2cyclictest0-21swapper/212:29:442
15106991310,2cyclictest0-21swapper/212:15:072
15106991310,2cyclictest0-21swapper/211:25:162
15106991310,2cyclictest0-21swapper/208:10:252
1510699130,12cyclictest23268-21ssh09:22:172
1510699130,12cyclictest0-21swapper/212:20:192
1510699130,12cyclictest0-21swapper/211:13:162
1510299139,2cyclictest227850irq/25-eth008:40:301
1510299138,3cyclictest3536-21if_eth007:55:251
15102991310,2cyclictest227850irq/25-eth007:38:101
1509899138,3cyclictest9770-21top07:49:420
1509899132,6cyclictest0-21swapper/008:05:010
15098991310,2cyclictest15144-21copy07:10:060
1511199129,2cyclictest0-21swapper/312:25:083
1511199129,2cyclictest0-21swapper/310:50:263
1511199129,2cyclictest0-21swapper/308:35:363
1511199129,2cyclictest0-21swapper/308:21:353
1511199129,2cyclictest0-21swapper/307:26:353
1511199121,9cyclictest0-21swapper/308:47:213
1511199121,9cyclictest0-21swapper/307:30:213
1511199120,1cyclictest0-21swapper/311:40:333
1511199120,11cyclictest0-21swapper/312:23:393
1511199120,11cyclictest0-21swapper/309:05:523
1511199120,11cyclictest0-21swapper/308:55:243
1511199120,11cyclictest0-21swapper/307:55:213
1511199120,11cyclictest0-21swapper/307:49:273
1510699129,2cyclictest9002-21ssh11:45:482
1510699129,2cyclictest5544-21kworker/u8:309:45:042
1510699129,2cyclictest3437-21ssh10:35:542
1510699129,2cyclictest0-21swapper/211:37:382
1510699129,2cyclictest0-21swapper/211:02:222
1510699129,2cyclictest0-21swapper/210:46:172
1510699129,2cyclictest0-21swapper/210:00:552
1510699129,2cyclictest0-21swapper/209:52:052
1510699129,2cyclictest0-21swapper/209:12:092
1510699129,2cyclictest0-21swapper/208:50:182
1510699129,2cyclictest0-21swapper/207:55:332
1510699127,3cyclictest15556-21gzip11:50:272
1510699120,11cyclictest15439-21ssh11:06:252
1510699120,11cyclictest0-21swapper/212:06:002
1510699120,11cyclictest0-21swapper/211:55:482
1510699120,11cyclictest0-21swapper/210:57:292
1510699120,11cyclictest0-21swapper/210:52:382
1510699120,11cyclictest0-21swapper/210:26:142
1510699120,11cyclictest0-21swapper/209:40:302
1510699120,11cyclictest0-21swapper/209:37:012
1510699120,11cyclictest0-21swapper/207:40:182
1510299129,2cyclictest0-21swapper/108:45:261
1510299129,2cyclictest0-21swapper/107:12:501
1510299120,10cyclictest8590-21munin-node1
1510299120,10cyclictest0-21swapper/108:18:051
1509899129,2cyclictest0-21swapper/008:35:360
1509899129,2cyclictest0-21swapper/008:15:180
1509899121,9cyclictest0-21swapper/008:56:220
1509899120,11cyclictest0-21swapper/008:30:190
1509899120,11cyclictest0-21swapper/008:10:200
1509899120,10cyclictest5356-21sshd07:21:450
1511199119,1cyclictest0-21swapper/308:05:263
1511199118,2cyclictest0-21swapper/307:15:243
1511199118,2cyclictest0-21swapper/307:15:233
1511199110,10cyclictest0-21swapper/308:50:273
1511199110,10cyclictest0-21swapper/308:26:003
1511199110,10cyclictest0-21swapper/308:00:263
1510699119,1cyclictest0-21swapper/210:30:252
1510699119,1cyclictest0-21swapper/209:05:222
1510699119,1cyclictest0-21swapper/208:05:262
1510699119,1cyclictest0-21swapper/207:15:332
1510699119,1cyclictest0-21swapper/207:15:322
1510699117,3cyclictest3232-21snmpd09:15:372
1510699117,3cyclictest3232-21snmpd07:29:202
1510699116,3cyclictest3232-21snmpd07:52:512
1510699116,3cyclictest30151-21cat08:55:202
1510699112,8cyclictest11-21rcu_preempt08:45:252
1510699110,1cyclictest0-21swapper/210:10:262
1510699110,10cyclictest0-21swapper/208:00:202
1510699110,10cyclictest0-21swapper/207:35:202
1510299113,6cyclictest0-21swapper/108:04:161
1510299110,9cyclictest0-21swapper/107:15:191
1510299110,9cyclictest0-21swapper/107:15:191
1510299110,10cyclictest0-21swapper/109:00:181
1509899110,9cyclictest683-21wget07:50:110
1509899110,9cyclictest0-21swapper/007:25:100
1509899110,3cyclictest22941-21munin-node0
1511199107,2cyclictest0-21swapper/308:15:213
1510699108,1cyclictest0-21swapper/209:00:302
1510699106,3cyclictest3232-21snmpd08:34:502
1510699100,1cyclictest0-21swapper/208:15:242
1510299106,3cyclictest227850irq/25-eth007:20:161
1510299100,9cyclictest0-21swapper/109:05:241
1510299100,9cyclictest0-21swapper/107:40:121
1510299100,2cyclictest0-21swapper/107:45:161
151119991,5cyclictest0-21swapper/309:00:303
151119990,8cyclictest0-21swapper/308:30:223
151069995,3cyclictest3232-21snmpd08:44:212
151069995,3cyclictest3232-21snmpd07:12:052
151069990,3cyclictest21442-21chrt07:23:102
151029996,2cyclictest227850irq/25-eth008:30:201
151069985,2cyclictest0-21swapper/208:25:052
151069983,3cyclictest20746-21aten_r5power_en08:35:162
151119970,6cyclictest0-21swapper/307:35:553
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional