You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-12-04 - 02:14

x86 Intel Atom E3845 @1910 MHz, Linux 4.19.37-rt19 (Profile)

Latency plot of system in rack #d, slot #1
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Up99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackdslot1.osadl.org (updated Sat Dec 03, 2022 12:46:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
941721680,6sleep32125899cyclictest12:30:013
1176911660,13ptp4l0-21swapper/107:05:121
1176911650,16ptp4l0-21swapper/007:07:040
1176911490,12ptp4l0-21swapper/207:05:392
1176911470,12ptp4l0-21swapper/307:06:223
263322740,3sleep20-21swapper/208:30:012
304242690,6sleep030428-21needreboot10:55:280
117691660,3ptp4l0-21swapper/212:12:142
9922640,2sleep00-21swapper/007:35:140
117691640,10ptp4l0-21swapper/008:45:320
117691620,2ptp4l0-21swapper/208:40:262
117691620,2ptp4l0-21swapper/012:15:010
117691620,17ptp4l30296-21apt-get12:05:153
117691620,16ptp4l0-21swapper/310:37:333
117691620,10ptp4l0-21swapper/011:20:330
21256996119,7cyclictest25-21ksoftirqd/111:07:111
117691610,5ptp4l437-21sed09:50:333
55198600,2rtkit-daemon550-21rtkit-daemon12:22:383
117691600,2ptp4l0-21swapper/209:25:182
117691600,1ptp4l0-21swapper/311:30:153
117691600,10ptp4l0-21swapper/011:02:550
21255995927,5cyclictest9-21ksoftirqd/010:50:280
117691590,5ptp4l29472-21cstates09:45:153
21255995825,5cyclictest9-21ksoftirqd/007:45:150
21255995824,5cyclictest9-21ksoftirqd/011:35:000
117691580,1ptp4l0-21swapper/311:37:153
117691580,1ptp4l0-21swapper/307:52:403
117691580,1ptp4l0-21swapper/307:52:403
21256995752,3cyclictest231rcuc/107:26:341
21255995725,5cyclictest9-21ksoftirqd/010:00:000
21255995725,5cyclictest9-21ksoftirqd/009:20:200
21255995725,5cyclictest9-21ksoftirqd/007:55:300
117691570,2ptp4l0-21swapper/311:23:223
117691570,2ptp4l0-21swapper/207:50:222
117691570,2ptp4l0-21swapper/207:50:222
117691570,1ptp4l0-21swapper/208:31:572
2125699560,53cyclictest241ktimersoftd/108:10:291
21255995626,5cyclictest9-21ksoftirqd/012:05:210
21255995624,5cyclictest9-21ksoftirqd/011:40:300
21255995624,4cyclictest9-21ksoftirqd/008:35:130
21255995623,5cyclictest9-21ksoftirqd/009:40:200
117691560,2ptp4l0-21swapper/212:23:512
117691560,2ptp4l0-21swapper/009:50:010
117691560,1ptp4l0-21swapper/308:55:153
117691560,1ptp4l0-21swapper/209:41:442
21255995524,5cyclictest9-21ksoftirqd/012:20:230
21255995523,7cyclictest9-21ksoftirqd/010:25:280
21255995523,5cyclictest9-21ksoftirqd/008:50:270
21255995523,5cyclictest9-21ksoftirqd/007:25:190
117691550,3ptp4l0-21swapper/110:05:171
117691550,2ptp4l0-21swapper/210:25:302
117691550,2ptp4l0-21swapper/110:25:351
117691550,1ptp4l0-21swapper/309:57:223
117691550,1ptp4l0-21swapper/209:24:212
2125599543,2cyclictest121rcu_preempt08:30:180
21255995426,4cyclictest9-21ksoftirqd/010:45:250
21255995422,5cyclictest9-21ksoftirqd/009:35:410
21255995422,5cyclictest9-21ksoftirqd/007:50:150
21255995422,5cyclictest9-21ksoftirqd/007:50:140
21255995421,5cyclictest9-21ksoftirqd/012:25:220
117691540,4ptp4l216-21systemd-journal08:42:553
117691540,2ptp4l0-21swapper/010:40:320
117691540,2ptp4l0-21swapper/008:12:550
117691540,14ptp4l0-21swapper/210:04:062
21255995323,5cyclictest9-21ksoftirqd/007:30:200
21255995322,5cyclictest9-21ksoftirqd/008:20:210
21255995321,5cyclictest9-21ksoftirqd/009:05:270
117691530,2ptp4l0-21swapper/310:34:593
117691530,2ptp4l0-21swapper/310:34:583
117691530,2ptp4l0-21swapper/212:35:172
117691530,2ptp4l0-21swapper/208:15:502
2125699529,7cyclictest25-21ksoftirqd/109:35:011
2125699528,8cyclictest25-21ksoftirqd/108:45:181
2125699521,7cyclictest30261-21/usr/sbin/munin08:35:311
21255995222,5cyclictest9-21ksoftirqd/009:50:110
21255995221,7cyclictest9-21ksoftirqd/011:20:020
21255995221,5cyclictest9-21ksoftirqd/011:50:290
21255995221,5cyclictest9-21ksoftirqd/009:00:180
21255995220,5cyclictest9-21ksoftirqd/011:55:340
21255995220,5cyclictest9-21ksoftirqd/011:55:330
117691520,2ptp4l0-21swapper/312:19:173
117691520,2ptp4l0-21swapper/311:14:353
117691520,2ptp4l0-21swapper/208:00:292
117691520,2ptp4l0-21swapper/009:32:550
117691520,1ptp4l0-21swapper/312:14:433
117691520,1ptp4l0-21swapper/309:32:013
117691520,1ptp4l0-21swapper/211:49:432
117691520,1ptp4l0-21swapper/011:29:510
2125699519,7cyclictest25-21ksoftirqd/112:10:001
2125699518,7cyclictest25-21ksoftirqd/109:25:231
21256995127,11cyclictest25-21ksoftirqd/111:40:281
2125699510,48cyclictest241ktimersoftd/107:40:281
2125599515,9cyclictest9-21ksoftirqd/010:19:590
117691510,6ptp4l0-21swapper/211:15:332
117691510,2ptp4l0-21swapper/211:56:212
117691510,2ptp4l0-21swapper/211:56:202
117691510,2ptp4l0-21swapper/209:58:132
117691510,2ptp4l0-21swapper/010:11:090
117691510,2ptp4l0-21swapper/008:08:010
117691510,2ptp4l0-21swapper/007:14:320
117691510,1ptp4l0-21swapper/311:27:103
117691510,10ptp4l0-21swapper/312:39:093
2125699508,7cyclictest25-21ksoftirqd/110:55:011
2125699507,7cyclictest25-21ksoftirqd/110:55:321
2125699503,9cyclictest241ktimersoftd/112:35:281
2125699503,8cyclictest241ktimersoftd/108:15:191
2125699503,8cyclictest241ktimersoftd/108:00:171
2125699503,7cyclictest241ktimersoftd/111:15:161
2125699503,7cyclictest241ktimersoftd/109:50:301
2125699500,9cyclictest241ktimersoftd/109:10:221
21255995036,4cyclictest9-21ksoftirqd/008:45:000
21255995022,8cyclictest9-21ksoftirqd/007:40:140
21255995020,5cyclictest9-21ksoftirqd/012:10:270
21255995020,4cyclictest9-21ksoftirqd/010:00:210
21255995019,5cyclictest9-21ksoftirqd/011:05:300
117691500,2ptp4l0-21swapper/311:42:033
117691500,2ptp4l0-21swapper/309:40:383
117691500,2ptp4l0-21swapper/309:08:003
117691500,2ptp4l0-21swapper/211:52:072
117691500,2ptp4l0-21swapper/211:13:102
117691500,2ptp4l0-21swapper/208:08:542
117691500,2ptp4l0-21swapper/008:01:370
117691500,1ptp4l0-21swapper/310:57:593
117691500,1ptp4l0-21swapper/308:45:223
117691500,12ptp4l0-21swapper/209:51:002
2125699498,7cyclictest25-21ksoftirqd/110:10:211
2125699495,9cyclictest241ktimersoftd/107:50:321
2125699495,9cyclictest241ktimersoftd/107:50:311
2125699493,8cyclictest241ktimersoftd/109:55:131
2125699493,7cyclictest241ktimersoftd/110:35:171
2125699493,7cyclictest241ktimersoftd/109:40:371
2125699492,8cyclictest241ktimersoftd/109:00:011
2125699492,8cyclictest241ktimersoftd/108:20:341
2125699492,7cyclictest241ktimersoftd/110:20:171
2125699490,8cyclictest20977-21cstates11:45:151
2125699490,7cyclictest241ktimersoftd/108:05:221
2125699490,45cyclictest241ktimersoftd/109:20:001
2125599494,7cyclictest9-21ksoftirqd/012:05:000
21255994921,4cyclictest9-21ksoftirqd/011:45:270
117691490,3ptp4l41-21ksoftirqd/308:54:563
117691490,2ptp4l0-21swapper/309:00:493
117691490,2ptp4l0-21swapper/210:50:562
117691490,2ptp4l0-21swapper/210:48:262
117691490,2ptp4l0-21swapper/209:16:072
117691490,2ptp4l0-21swapper/010:36:070
117691490,1ptp4l0-21swapper/309:27:433
117691490,1ptp4l0-21swapper/308:09:503
117691490,1ptp4l0-21swapper/211:05:082
117691490,1ptp4l0-21swapper/107:10:251
2125699483,9cyclictest241ktimersoftd/109:05:281
2125699483,8cyclictest241ktimersoftd/111:35:231
2125699483,8cyclictest241ktimersoftd/111:30:351
2125699483,8cyclictest241ktimersoftd/108:25:251
2125699483,7cyclictest241ktimersoftd/112:10:281
2125699483,10cyclictest241ktimersoftd/107:15:211
2125699482,8cyclictest241ktimersoftd/112:15:141
2125699482,7cyclictest241ktimersoftd/112:25:231
2125699482,7cyclictest241ktimersoftd/111:20:011
2125699480,45cyclictest241ktimersoftd/109:20:211
2125599484,7cyclictest121rcu_preempt07:20:270
2125599483,7cyclictest9-21ksoftirqd/010:20:260
117691480,6ptp4l321ktimersoftd/210:14:492
117691480,2ptp4l0-21swapper/311:18:563
117691480,2ptp4l0-21swapper/307:28:513
117691480,2ptp4l0-21swapper/212:18:082
117691480,2ptp4l0-21swapper/211:27:542
117691480,2ptp4l0-21swapper/211:21:542
117691480,2ptp4l0-21swapper/210:57:572
117691480,2ptp4l0-21swapper/210:21:562
117691480,2ptp4l0-21swapper/209:10:232
117691480,2ptp4l0-21swapper/208:55:012
117691480,2ptp4l0-21swapper/110:30:291
117691480,2ptp4l0-21swapper/110:30:281
117691480,1ptp4l0-21swapper/311:50:473
117691480,1ptp4l0-21swapper/308:13:383
117691480,1ptp4l0-21swapper/211:42:402
117691480,1ptp4l0-21swapper/211:02:082
117691480,1ptp4l0-21swapper/210:07:162
117691480,1ptp4l0-21swapper/209:47:392
117691480,1ptp4l0-21swapper/209:37:072
117691480,1ptp4l0-21swapper/209:02:092
117691480,1ptp4l0-21swapper/208:54:252
117691480,1ptp4l0-21swapper/208:14:132
117691480,1ptp4l0-21swapper/207:55:502
117691480,1ptp4l0-21swapper/110:16:351
117691480,12ptp4l4453-21ntp_states08:50:271
2125699474,8cyclictest241ktimersoftd/107:55:141
2125699474,7cyclictest25302-21latency_hist11:55:011
2125699473,8cyclictest241ktimersoftd/109:45:131
2125699472,6cyclictest241ktimersoftd/111:55:201
2125699472,6cyclictest241ktimersoftd/111:55:201
21256994720,8cyclictest25-21ksoftirqd/110:47:481
2125699470,9cyclictest241ktimersoftd/108:30:321
2125699470,7cyclictest241ktimersoftd/112:34:591
2125699470,44cyclictest241ktimersoftd/107:45:201
2125599474,8cyclictest9-21ksoftirqd/009:10:270
2125599472,9cyclictest9-21ksoftirqd/011:35:190
21255994728,5cyclictest9-21ksoftirqd/008:15:190
21255994717,5cyclictest9-21ksoftirqd/008:55:150
21255994716,11cyclictest9-21ksoftirqd/010:30:200
21255994716,11cyclictest9-21ksoftirqd/010:30:190
117691470,8ptp4l0-21swapper/207:25:572
117691470,2ptp4l0-21swapper/308:36:403
117691470,2ptp4l0-21swapper/211:36:292
117691470,2ptp4l0-21swapper/208:47:382
117691470,1ptp4l0-21swapper/208:22:372
117691470,1ptp4l0-21swapper/107:38:021
117691470,10ptp4l0-21swapper/308:34:313
2125699465,8cyclictest241ktimersoftd/112:00:371
2125699464,8cyclictest241ktimersoftd/110:00:381
2125699460,7cyclictest26903-21sed07:20:251
2125699460,6cyclictest241ktimersoftd/109:35:381
2125699460,44cyclictest241ktimersoftd/111:30:011
2125599467,9cyclictest9-21ksoftirqd/011:15:000
2125599467,4cyclictest9-21ksoftirqd/009:15:140
2125599465,4cyclictest121rcu_preempt07:20:000
21255994622,9cyclictest9-21ksoftirqd/010:05:310
117691460,8ptp4l0-21swapper/207:46:362
117691460,8ptp4l0-21swapper/207:41:132
117691460,8ptp4l0-21swapper/207:31:292
117691460,8ptp4l0-21swapper/207:22:112
117691460,2ptp4l0-21swapper/310:06:433
117691460,2ptp4l0-21swapper/307:15:223
117691460,2ptp4l0-21swapper/211:32:492
117691460,2ptp4l0-21swapper/210:41:452
117691460,2ptp4l0-21swapper/210:35:482
117691460,2ptp4l0-21swapper/109:03:141
117691460,1ptp4l0-21swapper/312:34:353
117691460,1ptp4l0-21swapper/209:34:162
117691460,1ptp4l0-21swapper/209:06:472
2125699451,9cyclictest241ktimersoftd/112:20:221
2125699450,6cyclictest241ktimersoftd/111:10:181
2125599457,4cyclictest9-21ksoftirqd/008:25:310
2125599456,4cyclictest121rcu_preempt12:30:230
117691450,8ptp4l0-21swapper/210:19:552
117691450,8ptp4l0-21swapper/207:37:152
117691450,8ptp4l0-21swapper/009:27:390
117691450,3ptp4l33-21ksoftirqd/207:19:162
117691450,2ptp4l0-21swapper/311:03:063
117691450,2ptp4l0-21swapper/307:57:263
117691450,2ptp4l0-21swapper/210:32:112
117691450,2ptp4l0-21swapper/210:32:102
117691450,2ptp4l0-21swapper/012:37:170
21256994410,10cyclictest25-21ksoftirqd/107:34:221
117691440,1ptp4l0-21swapper/212:32:452
117691440,1ptp4l0-21swapper/208:37:482
2125699434,5cyclictest241ktimersoftd/110:40:181
117691430,2ptp4l0-21swapper/308:24:343
117691430,2ptp4l0-21swapper/212:29:422
117691430,1ptp4l0-21swapper/310:41:563
117691430,1ptp4l0-21swapper/308:02:113
117691430,1ptp4l0-21swapper/212:09:442
2125699425,4cyclictest25-21ksoftirqd/111:00:331
2125699420,8cyclictest241ktimersoftd/108:40:241
117691420,2ptp4l0-21swapper/310:45:523
117691420,2ptp4l0-21swapper/309:39:323
117691420,2ptp4l0-21swapper/309:14:563
117691420,2ptp4l0-21swapper/308:17:143
117691420,1ptp4l0-21swapper/212:01:112
117691410,5ptp4l27491-21diskstats10:50:173
117691410,5ptp4l17053-21apt-get09:20:023
117691410,4ptp4l4209-21df_inode10:00:233
117691410,2ptp4l0-21swapper/207:12:322
117691400,4ptp4l27769-21apt-get12:00:193
117691400,4ptp4l21224-21fschecks_count11:45:183
117691400,4ptp4l16297-21irqstats10:25:253
117691400,2ptp4l0-21swapper/311:06:123
117691400,2ptp4l0-21swapper/310:20:273
117691400,2ptp4l0-21swapper/308:25:063
117691400,2ptp4l0-21swapper/307:20:163
117691400,1ptp4l0-21swapper/311:56:523
117691400,1ptp4l0-21swapper/311:56:513
117691390,4ptp4l9371-21meminfo10:10:233
117691390,2ptp4l0-21swapper/309:19:373
117691390,2ptp4l0-21swapper/307:40:273
117691380,2ptp4l0-21swapper/307:40:003
117691370,2ptp4l0-21swapper/307:30:193
117691360,2ptp4l0-21swapper/310:15:183
117691360,2ptp4l0-21swapper/307:45:203
117691280,1ptp4l0-21swapper/307:13:583
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional