You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-09-30 - 19:19
/usr/bin/Xorg /usr/bin/Xorg

x86 Intel Atom E3845 @1910 MHz, Linux 4.19.37-rt19 (Profile)

Latency plot of system in rack #d, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Up99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackdslot2.osadl.org (updated Fri Sep 30, 2022 12:46:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
554981790,10rtkit-daemon553-21rtkit-daemon07:08:190
1127921550,7sleep11734499cyclictest08:05:171
133891137114,10phc2sys0-21swapper/307:07:173
554981270,99rtkit-daemon18948-21apache207:07:221
167942839,61sleep20-21swapper/207:06:392
65542730,3sleep20-21swapper/212:30:272
17344997336,8cyclictest25-21ksoftirqd/109:25:221
17343997361,6cyclictest9-21ksoftirqd/009:50:120
17344997024,10cyclictest25-21ksoftirqd/108:20:211
17344996731,10cyclictest25-21ksoftirqd/109:10:011
17344996531,9cyclictest25-21ksoftirqd/109:00:001
17344996431,11cyclictest25-21ksoftirqd/107:55:261
17344996430,11cyclictest25-21ksoftirqd/111:10:221
17344996428,10cyclictest25-21ksoftirqd/109:00:301
17344996348,4cyclictest25-21ksoftirqd/108:10:171
17344996246,4cyclictest25-21ksoftirqd/107:40:221
17344996226,10cyclictest25-21ksoftirqd/107:35:221
17344996220,10cyclictest25-21ksoftirqd/110:58:381
17344996154,4cyclictest25-21ksoftirqd/111:40:051
17344996154,4cyclictest25-21ksoftirqd/111:40:041
17344996127,7cyclictest25-21ksoftirqd/111:45:201
17344996125,10cyclictest25-21ksoftirqd/110:40:191
17344996053,4cyclictest25-21ksoftirqd/108:45:051
17344996027,10cyclictest25-21ksoftirqd/107:15:331
17344996016,9cyclictest25-21ksoftirqd/109:15:251
17344995924,10cyclictest25-21ksoftirqd/109:25:011
17344995924,10cyclictest25-21ksoftirqd/108:40:261
17344995922,8cyclictest25-21ksoftirqd/108:18:311
17344995916,8cyclictest25-21ksoftirqd/110:00:171
17343995947,5cyclictest9-21ksoftirqd/010:35:120
17343995922,4cyclictest9-21ksoftirqd/010:10:230
1734499587,9cyclictest25-21ksoftirqd/109:30:141
17344995823,10cyclictest25-21ksoftirqd/107:30:201
17344995821,7cyclictest25-21ksoftirqd/110:30:151
1734399587,9cyclictest9-21ksoftirqd/008:28:320
17343995814,4cyclictest121rcu_preempt08:00:210
17344995739,4cyclictest25-21ksoftirqd/110:10:001
17344995724,10cyclictest25-21ksoftirqd/110:45:151
35662560,3sleep23567-21sed08:55:332
1734499564,8cyclictest25-21ksoftirqd/108:35:141
17344995643,4cyclictest25-21ksoftirqd/112:05:271
17344995641,4cyclictest25-21ksoftirqd/107:45:311
17344995638,4cyclictest25-21ksoftirqd/107:20:211
17344995636,4cyclictest25-21ksoftirqd/111:00:181
1734499562,3cyclictest121rcu_preempt10:50:321
17344995620,11cyclictest25-21ksoftirqd/108:50:331
17343995627,8cyclictest9-21ksoftirqd/007:50:250
167852560,3sleep10-21swapper/110:35:151
1734499557,4cyclictest25-21ksoftirqd/111:25:221
1734499553,4cyclictest25-21ksoftirqd/111:15:241
1734499546,9cyclictest25-21ksoftirqd/111:20:171
17344995437,4cyclictest25-21ksoftirqd/107:30:001
1734499543,4cyclictest121rcu_preempt08:25:161
17344995421,9cyclictest25-21ksoftirqd/111:05:141
1734399546,4cyclictest14128-21apt-get11:40:010
1734399546,4cyclictest14128-21apt-get11:40:010
17343995442,4cyclictest9-21ksoftirqd/007:45:230
17343995424,4cyclictest9-21ksoftirqd/011:50:280
1734499536,4cyclictest25-21ksoftirqd/110:10:271
1734499533,4cyclictest25-21ksoftirqd/108:00:191
1734499532,3cyclictest121rcu_preempt07:50:151
17344995320,11cyclictest25-21ksoftirqd/112:10:201
17344995319,10cyclictest25-21ksoftirqd/112:30:151
1734499531,8cyclictest25-21ksoftirqd/112:00:141
1734499531,3cyclictest121rcu_preempt07:10:291
1734399535,5cyclictest12883-21unixbench_singl09:15:310
17343995343,3cyclictest9-21ksoftirqd/009:55:360
17343995338,4cyclictest9-21ksoftirqd/009:30:150
17343995336,4cyclictest9-21ksoftirqd/009:05:300
1734399533,5cyclictest11795-21cron11:35:000
1734399531,7cyclictest9-21ksoftirqd/008:10:230
1734499525,3cyclictest25-21ksoftirqd/111:50:151
17344995246,4cyclictest25-21ksoftirqd/109:10:321
17344995246,4cyclictest25-21ksoftirqd/109:10:321
17344995245,4cyclictest25-21ksoftirqd/111:30:141
1734499524,3cyclictest121rcu_preempt09:50:251
17343995241,4cyclictest9-21ksoftirqd/010:00:230
1734499518,4cyclictest121rcu_preempt09:40:291
1734499514,8cyclictest121rcu_preempt11:35:141
1734499514,8cyclictest121rcu_preempt11:35:131
17344995138,5cyclictest25-21ksoftirqd/110:25:251
17344995138,5cyclictest25-21ksoftirqd/110:25:251
1734499512,9cyclictest25-21ksoftirqd/111:55:211
17344995119,10cyclictest25-21ksoftirqd/109:45:281
1734499510,3cyclictest121rcu_preempt09:55:361
17343995138,5cyclictest9-21ksoftirqd/010:55:010
17343995138,4cyclictest9-21ksoftirqd/007:55:260
17343995136,4cyclictest9-21ksoftirqd/011:15:000
17343995134,4cyclictest9-21ksoftirqd/009:00:150
17343995118,4cyclictest9-21ksoftirqd/011:40:190
17343995118,4cyclictest9-21ksoftirqd/011:40:180
17343995117,7cyclictest9-21ksoftirqd/008:55:330
1734499505,8cyclictest25-21ksoftirqd/112:35:251
17344995029,10cyclictest25-21ksoftirqd/112:25:331
1734499501,8cyclictest25-21ksoftirqd/109:35:151
1734499500,3cyclictest121rcu_preempt10:15:181
17343995040,3cyclictest9-21ksoftirqd/007:25:320
17343995036,4cyclictest9-21ksoftirqd/007:35:180
17343995036,4cyclictest9-21ksoftirqd/007:10:280
17343995033,5cyclictest9-21ksoftirqd/011:25:160
17343995029,10cyclictest9-21ksoftirqd/008:20:250
1734499496,3cyclictest121rcu_preempt08:30:241
1734499491,9cyclictest25-21ksoftirqd/110:20:171
1734399495,5cyclictest9-21ksoftirqd/010:15:240
17343994939,3cyclictest9-21ksoftirqd/011:05:210
17343994935,4cyclictest9-21ksoftirqd/008:35:140
17343994934,4cyclictest9-21ksoftirqd/011:20:170
17343994933,4cyclictest9-21ksoftirqd/007:40:380
17343994932,4cyclictest9-21ksoftirqd/008:40:230
17343994932,4cyclictest9-21ksoftirqd/007:20:340
1734399491,9cyclictest204-21systemd-journal11:49:310
17343994913,4cyclictest9-21ksoftirqd/009:21:210
258342480,2sleep00-21swapper/010:55:140
1734399484,3cyclictest121rcu_preempt10:05:230
17343994843,3cyclictest9-21ksoftirqd/008:15:180
1734399483,3cyclictest121rcu_preempt07:15:250
17343994830,4cyclictest9-21ksoftirqd/009:50:020
17343994829,4cyclictest9-21ksoftirqd/008:45:260
1734499473,8cyclictest25-21ksoftirqd/112:20:311
1734399474,3cyclictest121rcu_preempt10:30:280
1734399474,3cyclictest121rcu_preempt07:30:290
17343994740,4cyclictest9-21ksoftirqd/009:10:190
17343994740,4cyclictest9-21ksoftirqd/009:10:180
17343994736,3cyclictest9-21ksoftirqd/009:40:200
17343994736,3cyclictest9-21ksoftirqd/009:25:160
1734399473,4cyclictest121rcu_preempt08:30:140
17343994732,4cyclictest9-21ksoftirqd/011:55:220
17343994731,4cyclictest9-21ksoftirqd/012:30:250
17343994713,4cyclictest9-21ksoftirqd/012:00:280
133991470,7ptp4l1734699cyclictest07:42:063
1734499463,8cyclictest25-21ksoftirqd/112:15:221
1734399464,3cyclictest121rcu_preempt10:20:130
17343994639,4cyclictest9-21ksoftirqd/008:50:160
17343994636,3cyclictest9-21ksoftirqd/011:15:220
17343994632,4cyclictest9-21ksoftirqd/012:15:000
1734399460,3cyclictest121rcu_preempt10:40:290
17343994540,3cyclictest9-21ksoftirqd/009:35:130
17343994540,3cyclictest9-21ksoftirqd/008:05:200
1734399453,3cyclictest121rcu_preempt11:00:150
17343994531,4cyclictest9-21ksoftirqd/010:25:300
17343994531,4cyclictest9-21ksoftirqd/010:25:290
133991450,1ptp4l401ktimersoftd/310:35:153
133991450,1ptp4l401ktimersoftd/310:20:043
1734399444,3cyclictest9-21ksoftirqd/012:07:430
17343994427,5cyclictest9-21ksoftirqd/012:35:140
17343994426,5cyclictest9-21ksoftirqd/010:45:280
1734399440,8cyclictest9-21ksoftirqd/012:17:470
133991440,1ptp4l401ktimersoftd/312:23:303
133991440,1ptp4l401ktimersoftd/312:01:423
133991440,1ptp4l401ktimersoftd/308:40:533
133991440,1ptp4l401ktimersoftd/308:11:353
133991430,1ptp4l401ktimersoftd/309:20:243
133991420,39ptp4l391rcuc/307:20:233
17346994134,4cyclictest41-21ksoftirqd/308:05:103
17346994123,15cyclictest41-21ksoftirqd/310:49:393
17343994135,3cyclictest9-21ksoftirqd/012:25:200
17345994034,4cyclictest33-21ksoftirqd/209:35:012
17343994027,3cyclictest9-21ksoftirqd/012:20:210
133991400,1ptp4l401ktimersoftd/307:50:223
17346993932,4cyclictest41-21ksoftirqd/309:30:113
17346993911,4cyclictest28845-21latency_hist07:35:013
17346993831,4cyclictest41-21ksoftirqd/311:24:433
17346993811,4cyclictest23264-21sed08:30:213
17345993832,4cyclictest33-21ksoftirqd/210:45:222
17345993831,4cyclictest33-21ksoftirqd/208:05:292
17346993730,4cyclictest41-21ksoftirqd/309:59:213
17346993721,13cyclictest41-21ksoftirqd/311:36:003
17346993721,13cyclictest41-21ksoftirqd/311:36:003
17346993712,4cyclictest31249-21expr12:15:163
17346993711,4cyclictest16682-21idleruntime-cro11:45:013
17346993711,4cyclictest16682-21idleruntime-cro11:45:003
17345993730,4cyclictest33-21ksoftirqd/207:58:152
17345993730,4cyclictest33-21ksoftirqd/207:23:192
17345993728,5cyclictest33-21ksoftirqd/211:20:132
1734699366,5cyclictest0-21swapper/312:30:123
17346993630,3cyclictest41-21ksoftirqd/310:02:053
17346993630,3cyclictest41-21ksoftirqd/307:15:063
17346993629,4cyclictest41-21ksoftirqd/311:05:533
17346993629,4cyclictest41-21ksoftirqd/307:37:123
17345993630,4cyclictest33-21ksoftirqd/208:22:212
17345993630,3cyclictest33-21ksoftirqd/210:50:162
17345993629,4cyclictest33-21ksoftirqd/210:25:242
17345993629,4cyclictest33-21ksoftirqd/210:25:242
17345993629,4cyclictest33-21ksoftirqd/208:11:522
1734699353,7cyclictest20883-21apt-get09:35:123
17346993529,4cyclictest41-21ksoftirqd/309:52:303
17346993528,4cyclictest41-21ksoftirqd/309:42:423
17346993527,5cyclictest41-21ksoftirqd/309:03:283
17346993527,5cyclictest41-21ksoftirqd/308:37:103
17346993517,15cyclictest41-21ksoftirqd/307:55:113
17346993513,3cyclictest26260-21/usr/sbin/munin12:05:273
17346993510,5cyclictest20285-21grep11:50:223
17345993530,3cyclictest33-21ksoftirqd/210:33:522
17345993530,3cyclictest33-21ksoftirqd/209:35:222
17345993529,4cyclictest33-21ksoftirqd/211:35:292
17345993529,4cyclictest33-21ksoftirqd/211:35:282
17345993529,3cyclictest33-21ksoftirqd/207:43:122
17345993528,4cyclictest33-21ksoftirqd/211:57:582
17345993528,4cyclictest33-21ksoftirqd/211:53:482
17345993528,4cyclictest33-21ksoftirqd/210:59:452
17345993528,4cyclictest33-21ksoftirqd/209:54:452
17345993528,4cyclictest33-21ksoftirqd/208:53:452
17345993528,4cyclictest33-21ksoftirqd/208:15:142
17345993527,5cyclictest33-21ksoftirqd/210:16:422
1734599351,16cyclictest0-21swapper/210:02:542
1734699343,8cyclictest25836-21apt-get10:55:153
17346993428,4cyclictest41-21ksoftirqd/310:17:033
17346993428,3cyclictest41-21ksoftirqd/308:25:593
17346993427,4cyclictest41-21ksoftirqd/312:36:453
17346993427,4cyclictest41-21ksoftirqd/312:10:083
17346993427,4cyclictest41-21ksoftirqd/307:48:333
17346993412,4cyclictest28106-21/usr/sbin/munin11:00:273
17346993410,5cyclictest6571-21/usr/sbin/munin09:05:153
17346993410,4cyclictest4903-21latency_hist11:20:003
17345993429,3cyclictest33-21ksoftirqd/210:10:222
17345993428,4cyclictest33-21ksoftirqd/208:45:012
17345993428,4cyclictest33-21ksoftirqd/207:25:432
17345993427,4cyclictest33-21ksoftirqd/211:30:042
17345993427,4cyclictest33-21ksoftirqd/211:00:142
17345993427,4cyclictest33-21ksoftirqd/209:42:372
133991340,10ptp4l401ktimersoftd/310:41:273
1734699333,6cyclictest14141-21apt-get10:30:153
17346993327,4cyclictest41-21ksoftirqd/311:10:423
17346993327,4cyclictest41-21ksoftirqd/308:24:133
17346993327,3cyclictest41-21ksoftirqd/310:27:483
17346993327,3cyclictest41-21ksoftirqd/310:27:473
17346993327,3cyclictest41-21ksoftirqd/307:26:003
17346993326,4cyclictest41-21ksoftirqd/312:25:383
17346993326,4cyclictest41-21ksoftirqd/310:07:273
17346993326,4cyclictest41-21ksoftirqd/309:16:143
17346993326,4cyclictest41-21ksoftirqd/309:13:323
17346993326,4cyclictest41-21ksoftirqd/309:13:323
17345993327,4cyclictest33-21ksoftirqd/211:05:212
17345993327,4cyclictest33-21ksoftirqd/209:45:202
17345993327,4cyclictest33-21ksoftirqd/208:39:262
17345993327,4cyclictest33-21ksoftirqd/208:27:382
17345993327,3cyclictest33-21ksoftirqd/210:43:332
17345993327,3cyclictest33-21ksoftirqd/209:10:222
17345993327,3cyclictest33-21ksoftirqd/209:10:212
17345993326,4cyclictest33-21ksoftirqd/212:11:232
17345993326,4cyclictest33-21ksoftirqd/209:06:042
17345993326,4cyclictest33-21ksoftirqd/207:52:292
17346993227,3cyclictest41-21ksoftirqd/311:49:313
17346993226,3cyclictest41-21ksoftirqd/309:49:343
17346993226,3cyclictest41-21ksoftirqd/308:48:043
17346993225,4cyclictest41-21ksoftirqd/311:57:203
17346993225,4cyclictest41-21ksoftirqd/309:30:013
17346993225,4cyclictest41-21ksoftirqd/308:18:203
17345993227,3cyclictest33-21ksoftirqd/212:16:222
17345993226,4cyclictest33-21ksoftirqd/211:42:122
17345993226,4cyclictest33-21ksoftirqd/211:42:122
17345993226,4cyclictest33-21ksoftirqd/209:28:592
17345993226,4cyclictest33-21ksoftirqd/208:33:562
17345993226,4cyclictest33-21ksoftirqd/208:00:212
17345993226,3cyclictest33-21ksoftirqd/210:23:062
17345993225,4cyclictest33-21ksoftirqd/212:36:402
17345993225,4cyclictest33-21ksoftirqd/211:45:582
17345993225,4cyclictest33-21ksoftirqd/210:36:572
1734599321,3cyclictest121rcu_preempt10:05:272
17346993126,3cyclictest41-21ksoftirqd/308:03:183
17346993124,4cyclictest41-21ksoftirqd/308:50:033
17345993126,3cyclictest33-21ksoftirqd/209:15:312
17345993124,4cyclictest33-21ksoftirqd/211:26:252
17345993124,4cyclictest33-21ksoftirqd/207:18:522
17345993124,4cyclictest33-21ksoftirqd/207:11:462
17346993025,3cyclictest41-21ksoftirqd/307:11:443
17346993024,3cyclictest41-21ksoftirqd/310:13:473
17346993023,4cyclictest41-21ksoftirqd/308:57:383
17345993025,3cyclictest33-21ksoftirqd/209:55:272
17345993025,3cyclictest33-21ksoftirqd/207:35:312
17345993024,4cyclictest33-21ksoftirqd/209:01:222
17345993024,4cyclictest33-21ksoftirqd/208:40:192
17346992922,4cyclictest41-21ksoftirqd/311:31:223
17345992923,4cyclictest33-21ksoftirqd/209:20:242
17345992923,3cyclictest33-21ksoftirqd/211:13:052
17345992922,4cyclictest33-21ksoftirqd/211:15:352
17345992921,5cyclictest33-21ksoftirqd/212:25:002
17346992822,4cyclictest41-21ksoftirqd/310:54:403
17345992823,3cyclictest33-21ksoftirqd/207:45:302
17345992821,4cyclictest33-21ksoftirqd/207:33:262
17345992720,4cyclictest33-21ksoftirqd/212:27:542
17345992720,4cyclictest33-21ksoftirqd/212:05:252
17345992518,4cyclictest33-21ksoftirqd/212:02:402
133991240,1ptp4l401ktimersoftd/311:25:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional