You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-05-12 - 14:28

Intel(R) Atom(TM) CPU E3845 @ 1.91GHz, Linux 4.19.37-rt19 (Profile)

Latency plot of system in rack #d, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Up99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackdslot2.osadl.org (updated Wed May 12, 2021 00:46:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
531021690,3sleep11168799cyclictest21:00:141
1268911370,107ptp4l401ktimersoftd/319:07:243
1089929574,10sleep20-21swapper/219:05:502
1100229068,10sleep00-21swapper/019:07:060
1111228766,9sleep10-21swapper/119:08:321
123191780,4getstats1168999cyclictest22:18:373
318662770,2sleep10-21swapper/122:15:021
175032750,2sleep20-21swapper/219:55:152
126891750,2ptp4l401ktimersoftd/322:25:353
11687997468,3cyclictest25-21ksoftirqd/100:10:131
126891720,1ptp4l401ktimersoftd/300:32:493
126891710,2ptp4l401ktimersoftd/319:35:103
126891700,2ptp4l401ktimersoftd/323:10:023
11687996659,4cyclictest25-21ksoftirqd/122:40:131
126891630,1ptp4l401ktimersoftd/300:10:293
4722620,2sleep00-21swapper/020:15:080
11689996154,4cyclictest41-21ksoftirqd/319:20:133
11687996148,8cyclictest25-21ksoftirqd/119:20:131
126891600,1ptp4l401ktimersoftd/322:51:003
126891600,1ptp4l401ktimersoftd/322:51:003
11687995748,5cyclictest25-21ksoftirqd/120:40:141
11687995746,7cyclictest25-21ksoftirqd/120:00:131
11687995443,7cyclictest25-21ksoftirqd/122:30:131
11685995342,6cyclictest9-21ksoftirqd/021:35:130
179932520,2sleep10-21swapper/119:55:181
11687995242,7cyclictest25-21ksoftirqd/100:35:131
11689995142,5cyclictest41-21ksoftirqd/323:20:143
11688995042,5cyclictest33-21ksoftirqd/223:15:132
11688995041,5cyclictest33-21ksoftirqd/200:15:122
11687995037,6cyclictest25-21ksoftirqd/119:50:121
11688994942,4cyclictest33-21ksoftirqd/220:10:122
126891480,1ptp4l401ktimersoftd/320:41:573
903024739,4sleep125-21ksoftirqd/121:05:121
126891470,1ptp4l401ktimersoftd/323:55:003
126891460,1ptp4l401ktimersoftd/323:05:113
11688994638,5cyclictest33-21ksoftirqd/223:25:132
11688994638,5cyclictest33-21ksoftirqd/220:00:132
11687994633,8cyclictest25-21ksoftirqd/119:10:131
11687994633,8cyclictest25-21ksoftirqd/119:10:131
11687994631,7cyclictest25-21ksoftirqd/123:50:151
126891450,1ptp4l401ktimersoftd/300:35:153
11688994538,4cyclictest33-21ksoftirqd/220:20:132
11688994537,5cyclictest33-21ksoftirqd/220:05:142
11685994525,17cyclictest9-21ksoftirqd/021:45:530
126891440,1ptp4l401ktimersoftd/321:00:283
126891440,1ptp4l401ktimersoftd/319:57:343
126891440,1ptp4l401ktimersoftd/300:05:213
126891440,1ptp4l401ktimersoftd/300:05:213
11688994438,3cyclictest33-21ksoftirqd/219:10:142
11688994438,3cyclictest33-21ksoftirqd/219:10:142
11687994430,8cyclictest25-21ksoftirqd/121:20:121
11685994437,4cyclictest11903-21cron23:50:000
11685994417,6cyclictest101ktimersoftd/000:00:130
126891430,1ptp4l401ktimersoftd/322:57:093
126891430,1ptp4l401ktimersoftd/322:57:083
126891430,1ptp4l401ktimersoftd/319:25:053
11689994337,3cyclictest41-21ksoftirqd/323:45:153
11687994337,3cyclictest25-21ksoftirqd/122:25:061
11687994336,4cyclictest25-21ksoftirqd/100:15:111
126891420,1ptp4l401ktimersoftd/323:35:113
126891420,1ptp4l401ktimersoftd/323:35:003
126891420,1ptp4l401ktimersoftd/319:50:033
126891420,1ptp4l401ktimersoftd/300:20:153
11688994234,5cyclictest33-21ksoftirqd/222:45:142
11688994231,9cyclictest33-21ksoftirqd/222:35:072
11685994223,4cyclictest9-21ksoftirqd/000:39:020
126891410,1ptp4l401ktimersoftd/321:30:263
126891410,1ptp4l401ktimersoftd/319:30:493
11688994134,4cyclictest33-21ksoftirqd/221:25:012
11685994123,5cyclictest9-21ksoftirqd/023:36:500
126891400,1ptp4l401ktimersoftd/320:30:133
1168899403,4cyclictest121rcu_preempt21:24:592
11688994033,4cyclictest33-21ksoftirqd/222:05:122
11687994035,3cyclictest25-21ksoftirqd/120:50:011
11687994033,4cyclictest25-21ksoftirqd/119:40:091
11685994034,4cyclictest9-21ksoftirqd/019:45:010
83912390,6sleep38394-21missed_timers21:45:093
11689993933,3cyclictest41-21ksoftirqd/300:30:003
11688993934,3cyclictest33-21ksoftirqd/221:50:002
11688993933,4cyclictest33-21ksoftirqd/221:55:142
11688993932,4cyclictest33-21ksoftirqd/223:50:152
11688993932,4cyclictest33-21ksoftirqd/221:10:282
11688993932,4cyclictest33-21ksoftirqd/220:45:012
11687993934,3cyclictest25-21ksoftirqd/122:10:121
11685993932,4cyclictest9-21ksoftirqd/023:50:040
11685993922,4cyclictest9-21ksoftirqd/000:12:070
126891380,1ptp4l401ktimersoftd/322:35:193
126891380,1ptp4l401ktimersoftd/320:00:153
126891380,1ptp4l401ktimersoftd/319:15:363
11688993832,4cyclictest33-21ksoftirqd/223:45:142
11688993832,4cyclictest33-21ksoftirqd/223:40:012
11688993831,4cyclictest33-21ksoftirqd/223:09:442
11688993831,4cyclictest33-21ksoftirqd/220:50:002
11688993830,4cyclictest33-21ksoftirqd/223:25:002
11687993832,4cyclictest25-21ksoftirqd/121:40:141
11687993832,3cyclictest25-21ksoftirqd/123:20:021
11685993820,12cyclictest1784-21cpuspeed_turbos00:15:140
126891370,1ptp4l401ktimersoftd/320:50:253
126891370,13ptp4l0-21swapper/323:40:033
126891370,13ptp4l0-21swapper/321:38:123
11688993731,4cyclictest33-21ksoftirqd/223:35:142
11688993731,4cyclictest33-21ksoftirqd/222:20:132
11688993731,4cyclictest33-21ksoftirqd/219:45:142
11688993730,4cyclictest33-21ksoftirqd/222:30:002
11688993730,4cyclictest33-21ksoftirqd/222:00:122
11688993730,4cyclictest33-21ksoftirqd/219:20:012
11688993730,4cyclictest33-21ksoftirqd/200:20:142
11688993730,4cyclictest33-21ksoftirqd/200:05:192
11688993730,4cyclictest33-21ksoftirqd/200:05:192
11688993729,5cyclictest33-21ksoftirqd/222:15:142
11687993732,3cyclictest25-21ksoftirqd/123:45:041
11687993731,4cyclictest25-21ksoftirqd/120:35:141
11687993731,3cyclictest25-21ksoftirqd/122:35:141
11687993730,4cyclictest25-21ksoftirqd/120:25:131
11687993730,4cyclictest25-21ksoftirqd/120:25:131
11687993730,4cyclictest25-21ksoftirqd/100:04:441
11685993726,6cyclictest9-21ksoftirqd/022:05:120
179582360,2sleep00-21swapper/020:35:190
126891360,1ptp4l401ktimersoftd/323:50:013
126891360,1ptp4l401ktimersoftd/322:01:393
126891360,1ptp4l401ktimersoftd/321:11:373
126891360,13ptp4l0-21swapper/320:13:523
11689993629,4cyclictest41-21ksoftirqd/319:40:393
11689993628,5cyclictest41-21ksoftirqd/323:27:043
11688993631,3cyclictest33-21ksoftirqd/219:35:122
11688993630,4cyclictest33-21ksoftirqd/221:50:132
11688993630,4cyclictest33-21ksoftirqd/221:15:022
11688993630,4cyclictest33-21ksoftirqd/220:30:142
11688993630,4cyclictest33-21ksoftirqd/200:40:002
11688993630,3cyclictest33-21ksoftirqd/221:00:012
11688993630,3cyclictest33-21ksoftirqd/219:30:142
11688993629,4cyclictest33-21ksoftirqd/220:35:012
11688993629,4cyclictest33-21ksoftirqd/220:15:012
11688993629,4cyclictest33-21ksoftirqd/219:20:122
11687993630,4cyclictest25-21ksoftirqd/119:45:121
11687993629,4cyclictest25-21ksoftirqd/123:25:071
11687993629,4cyclictest25-21ksoftirqd/123:00:121
11687993629,4cyclictest25-21ksoftirqd/122:49:591
11687993629,4cyclictest22628-21kworker/1:000:25:161
11687993628,5cyclictest25-21ksoftirqd/120:30:141
11685993629,4cyclictest9-21ksoftirqd/022:17:460
11685993629,4cyclictest9-21ksoftirqd/021:27:030
126891350,1ptp4l401ktimersoftd/323:19:363
126891350,1ptp4l401ktimersoftd/322:11:533
126891350,1ptp4l401ktimersoftd/322:09:583
126891350,1ptp4l401ktimersoftd/321:53:033
126891350,1ptp4l401ktimersoftd/321:06:103
11688993529,4cyclictest33-21ksoftirqd/223:30:072
11688993529,4cyclictest33-21ksoftirqd/221:40:092
11688993529,4cyclictest33-21ksoftirqd/220:50:142
11688993529,4cyclictest33-21ksoftirqd/220:25:142
11688993529,4cyclictest33-21ksoftirqd/220:25:142
11688993529,4cyclictest33-21ksoftirqd/219:40:142
11688993529,4cyclictest33-21ksoftirqd/200:25:142
11688993529,3cyclictest33-21ksoftirqd/200:34:092
11688993528,4cyclictest33-21ksoftirqd/221:05:012
11688993528,4cyclictest33-21ksoftirqd/219:54:282
11688993527,5cyclictest210-21systemd-journal21:03:442
1168799354,3cyclictest121rcu_preempt22:05:251
11687993530,3cyclictest25-21ksoftirqd/123:10:141
11687993530,3cyclictest25-21ksoftirqd/122:00:141
11687993530,3cyclictest25-21ksoftirqd/120:20:001
11687993529,3cyclictest25-21ksoftirqd/121:19:591
11687993529,3cyclictest25-21ksoftirqd/120:10:281
11687993528,4cyclictest25-21ksoftirqd/122:55:021
11687993528,4cyclictest25-21ksoftirqd/122:55:021
11687993528,4cyclictest25-21ksoftirqd/120:56:161
11687993528,4cyclictest25-21ksoftirqd/120:06:331
11687993528,4cyclictest25-21ksoftirqd/119:28:561
11687993528,4cyclictest25-21ksoftirqd/100:30:021
11687993528,4cyclictest25-21ksoftirqd/100:20:011
11685993529,4cyclictest9-21ksoftirqd/021:24:550
11685993529,3cyclictest9-21ksoftirqd/021:02:560
11685993529,3cyclictest9-21ksoftirqd/020:58:000
11685993528,4cyclictest9-21ksoftirqd/022:46:120
11685993528,4cyclictest9-21ksoftirqd/022:38:250
11685993528,4cyclictest9-21ksoftirqd/019:36:570
11685993528,4cyclictest9-21ksoftirqd/019:24:210
126891340,1ptp4l401ktimersoftd/322:24:223
126891340,1ptp4l401ktimersoftd/320:23:073
126891340,12ptp4l401ktimersoftd/320:06:433
11689993428,4cyclictest41-21ksoftirqd/323:03:063
11689993428,4cyclictest41-21ksoftirqd/321:20:143
11689993428,4cyclictest41-21ksoftirqd/321:15:023
11689993428,3cyclictest41-21ksoftirqd/322:45:083
11689993428,3cyclictest41-21ksoftirqd/320:38:133
11689993427,4cyclictest41-21ksoftirqd/321:57:293
11689993427,4cyclictest41-21ksoftirqd/321:41:243
11689993427,4cyclictest41-21ksoftirqd/320:25:473
11689993427,4cyclictest41-21ksoftirqd/320:25:473
11689993427,4cyclictest41-21ksoftirqd/300:16:593
11688993428,4cyclictest33-21ksoftirqd/222:55:032
11688993428,4cyclictest33-21ksoftirqd/222:55:022
11688993428,4cyclictest33-21ksoftirqd/222:10:142
11688993428,4cyclictest33-21ksoftirqd/221:35:082
11688993428,4cyclictest33-21ksoftirqd/200:00:152
11688993427,4cyclictest33-21ksoftirqd/223:10:122
11688993427,4cyclictest33-21ksoftirqd/223:00:152
11688993427,4cyclictest33-21ksoftirqd/222:40:012
11687993429,3cyclictest25-21ksoftirqd/119:40:001
11687993429,3cyclictest25-21ksoftirqd/119:15:321
11687993428,4cyclictest25-21ksoftirqd/123:55:131
11687993428,4cyclictest25-21ksoftirqd/121:50:021
11687993428,3cyclictest25-21ksoftirqd/119:30:491
11687993427,4cyclictest25-21ksoftirqd/123:40:311
11687993427,4cyclictest25-21ksoftirqd/122:53:281
11687993427,4cyclictest25-21ksoftirqd/122:53:281
11687993427,4cyclictest25-21ksoftirqd/121:25:141
11687993425,6cyclictest25-21ksoftirqd/121:45:271
11685993427,5cyclictest9-21ksoftirqd/020:26:450
11685993427,5cyclictest9-21ksoftirqd/020:26:440
11685993427,4cyclictest9-21ksoftirqd/022:56:550
11685993427,4cyclictest9-21ksoftirqd/022:56:550
11685993427,4cyclictest9-21ksoftirqd/022:11:310
11685993427,4cyclictest9-21ksoftirqd/020:31:140
11685993427,4cyclictest9-21ksoftirqd/019:26:290
11685993427,4cyclictest9-21ksoftirqd/019:14:290
11685993427,4cyclictest9-21ksoftirqd/019:14:290
11685993427,4cyclictest9-21ksoftirqd/000:34:210
11689993327,4cyclictest41-21ksoftirqd/320:55:313
11689993326,4cyclictest41-21ksoftirqd/322:44:343
11689993326,4cyclictest41-21ksoftirqd/319:47:453
11688993328,3cyclictest33-21ksoftirqd/222:25:082
11688993327,4cyclictest33-21ksoftirqd/223:55:132
11688993326,4cyclictest33-21ksoftirqd/222:50:152
11688993326,4cyclictest33-21ksoftirqd/222:50:152
11688993326,4cyclictest33-21ksoftirqd/221:32:462
11687993328,3cyclictest25-21ksoftirqd/123:20:321
11687993328,3cyclictest25-21ksoftirqd/100:05:121
11687993328,3cyclictest25-21ksoftirqd/100:05:121
11687993327,4cyclictest25-21ksoftirqd/121:30:441
11687993326,4cyclictest25-21ksoftirqd/123:35:141
11687993326,4cyclictest25-21ksoftirqd/120:53:591
11687993325,5cyclictest25-21ksoftirqd/122:24:051
11687993325,5cyclictest25-21ksoftirqd/120:17:331
11685993327,4cyclictest9-21ksoftirqd/023:09:250
11685993327,4cyclictest9-21ksoftirqd/019:50:440
11685993327,3cyclictest9-21ksoftirqd/023:24:290
11685993327,3cyclictest9-21ksoftirqd/023:20:000
11685993327,3cyclictest9-21ksoftirqd/021:55:070
11685993326,4cyclictest9-21ksoftirqd/023:58:470
11685993326,4cyclictest9-21ksoftirqd/022:32:390
11685993326,4cyclictest9-21ksoftirqd/022:21:180
11685993326,4cyclictest9-21ksoftirqd/021:12:420
11685993326,4cyclictest9-21ksoftirqd/020:51:290
11685993326,4cyclictest9-21ksoftirqd/019:34:520
11685993326,4cyclictest9-21ksoftirqd/019:17:380
11685993325,5cyclictest9-21ksoftirqd/022:28:580
11689993227,3cyclictest41-21ksoftirqd/320:45:123
11688993225,4cyclictest33-21ksoftirqd/219:25:042
11687993226,3cyclictest25-21ksoftirqd/121:58:161
11685993226,4cyclictest9-21ksoftirqd/023:00:100
11685993226,3cyclictest9-21ksoftirqd/020:41:390
11685993226,3cyclictest9-21ksoftirqd/019:56:480
11685993225,4cyclictest9-21ksoftirqd/023:31:480
11685993225,4cyclictest9-21ksoftirqd/023:27:430
11685993225,4cyclictest9-21ksoftirqd/022:42:060
11685993225,4cyclictest9-21ksoftirqd/022:04:210
11685993225,4cyclictest9-21ksoftirqd/000:06:140
11685993225,4cyclictest9-21ksoftirqd/000:06:140
11687993125,4cyclictest25-21ksoftirqd/123:30:151
11687993125,4cyclictest25-21ksoftirqd/123:05:361
11685993126,3cyclictest9-21ksoftirqd/020:45:370
11685993125,3cyclictest9-21ksoftirqd/021:43:470
11685993124,4cyclictest9-21ksoftirqd/021:31:120
11685993124,4cyclictest9-21ksoftirqd/021:17:110
11685993124,4cyclictest9-21ksoftirqd/020:11:260
11685993124,4cyclictest9-21ksoftirqd/020:03:410
11685993124,4cyclictest9-21ksoftirqd/019:45:140
11689993025,3cyclictest41-21ksoftirqd/321:25:003
11689993025,3cyclictest41-21ksoftirqd/300:00:093
11689993024,3cyclictest41-21ksoftirqd/319:13:453
11689993024,3cyclictest41-21ksoftirqd/319:13:443
11688993024,4cyclictest33-21ksoftirqd/200:10:082
1168799303,3cyclictest121rcu_preempt21:11:451
11687993024,4cyclictest25-21ksoftirqd/121:35:021
126891290,1ptp4l401ktimersoftd/320:19:313
11685992924,3cyclictest9-21ksoftirqd/020:08:440
11685992923,4cyclictest9-21ksoftirqd/021:07:510
11685992923,3cyclictest9-21ksoftirqd/021:51:180
11685992822,3cyclictest9-21ksoftirqd/022:53:500
11685992822,3cyclictest9-21ksoftirqd/022:53:490
11685992821,4cyclictest9-21ksoftirqd/023:12:210
11685992821,4cyclictest9-21ksoftirqd/020:20:300
11689992722,3cyclictest41-21ksoftirqd/322:30:183
11685992721,4cyclictest9-21ksoftirqd/000:28:460
11685992619,4cyclictest9-21ksoftirqd/000:24:480
11685992317,4cyclictest9-21ksoftirqd/023:40:570
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional