You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2019-06-27 - 11:38

Intel(R) Atom(TM) CPU E3845 @ 1.91GHz, Linux 4.19.37-rt20 (Profile)

Latency plot of system in rack #b, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
The worst-case latency is NOT higher under extreme CPU load.
Results of all systems under extreme CPU load are combined on this summary graph.
The worst-case latency is about 20.90 times higher under graphics load.
Results of all systems with accelerated graphics are combined on this summary graph.
Characteristics of the highest latencies:
System rackbslot6.osadl.org (updated Thu Jun 27, 2019 00:43:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2094121830,5sleep1965799cyclictest21:38:271
104591980,4ptp4l401ktimersoftd/319:16:183
235592700,3sleep20-21swapper/223:33:442
93002674,10sleep00-21swapper/019:16:520
176522580,5sleep217654-21ptp4l-jitter20:38:332
176522580,5sleep217654-21ptp4l-jitter20:38:332
232772570,2sleep30-21swapper/300:13:003
232772570,2sleep30-21swapper/300:13:003
208372570,4sleep1965799cyclictest22:53:311
313412560,3sleep09-21ksoftirqd/023:06:230
965899528,12cyclictest13098-21ssh22:07:422
100082520,2sleep10-21swapper/100:33:381
965899517,12cyclictest14745-21ssh21:31:032
169542490,1sleep30-21swapper/323:28:153
965899488,13cyclictest13326-21ssh00:00:032
965999451,14cyclictest15055-21ssh22:08:343
965899458,11cyclictest2930-21ssh23:10:422
965999430,13cyclictest11355-21cat20:28:153
137462420,5sleep0965699cyclictest23:23:300
965899408,19cyclictest6587-21apt-get23:53:232
104591390,1ptp4l401ktimersoftd/322:16:443
104591390,1ptp4l401ktimersoftd/321:42:133
965899388,15cyclictest18894-21snmp_o5powerctl23:28:382
965899388,12cyclictest18629-21awk22:13:292
104591380,1ptp4l401ktimersoftd/323:58:203
104591380,1ptp4l401ktimersoftd/323:51:343
104591380,1ptp4l401ktimersoftd/323:43:473
104591380,1ptp4l401ktimersoftd/323:38:063
104591380,1ptp4l401ktimersoftd/323:06:183
104591380,1ptp4l401ktimersoftd/322:54:433
104591380,1ptp4l401ktimersoftd/322:49:033
104591380,1ptp4l401ktimersoftd/322:03:283
104591380,1ptp4l401ktimersoftd/321:45:463
104591380,1ptp4l401ktimersoftd/321:32:283
104591380,1ptp4l401ktimersoftd/300:35:383
104591380,0ptp4l401ktimersoftd/300:31:133
965999370,12cyclictest19853-21ssh21:37:483
104591370,1ptp4l401ktimersoftd/323:02:023
104591370,1ptp4l401ktimersoftd/321:55:483
104591370,1ptp4l401ktimersoftd/321:20:403
104591370,1ptp4l401ktimersoftd/320:42:413
104591370,1ptp4l401ktimersoftd/320:42:413
104591370,1ptp4l401ktimersoftd/300:43:053
104591360,1ptp4l401ktimersoftd/323:22:163
104591360,1ptp4l401ktimersoftd/322:29:263
104591360,1ptp4l401ktimersoftd/322:29:263
104591360,1ptp4l401ktimersoftd/319:38:213
104591360,1ptp4l401ktimersoftd/300:43:473
104591360,1ptp4l401ktimersoftd/300:28:113
104591360,0ptp4l401ktimersoftd/322:24:513
104591360,0ptp4l401ktimersoftd/321:23:423
104591360,0ptp4l401ktimersoftd/300:01:163
965899358,12cyclictest26240-21ssh22:59:402
965899356,16cyclictest10532-21unixbench_singl23:18:412
965899355,12cyclictest20305-21/usr/sbin/munin21:38:392
9658993519,11cyclictest0-21swapper/200:38:252
9657993528,4cyclictest25-21ksoftirqd/123:58:241
104591350,1ptp4l401ktimersoftd/300:06:453
965999340,17cyclictest4512-1kworker/u9:022:01:533
965899346,15cyclictest14490-21snmp_o5powerctl23:23:392
965899346,14cyclictest19150-21ls19:33:342
104591340,1ptp4l401ktimersoftd/323:09:593
965899338,10cyclictest2859-21diskmemload00:03:342
965899337,11cyclictest12334-21needreboot22:43:342
965899336,15cyclictest13705-21ssh22:08:222
965899335,12cyclictest27211-21cut22:23:292
965899335,12cyclictest1080-21sed20:03:352
965899328,9cyclictest11835-21apt-get19:23:162
965899328,21cyclictest18942-21ssh22:52:162
965899325,15cyclictest10458-21sshd21:25:592
965899325,12cyclictest17206-21sed21:33:312
9658993220,6cyclictest19730-21apt-get22:53:222
965899315,14cyclictest32206-21ssh22:28:352
965899315,14cyclictest32206-21ssh22:28:352
965899315,13cyclictest5899-21ssh23:13:362
965899314,12cyclictest31402-21ssh00:21:452
9658993119,6cyclictest821-21apt-get21:53:212
9658993117,11cyclictest0-21swapper/220:23:202
92142316,12sleep10-21swapper/119:15:441
9658993015,10cyclictest0-21swapper/221:58:212
9658993012,12cyclictest0-21swapper/200:43:252
9657993025,3cyclictest25-21ksoftirqd/123:53:301
104591300,1ptp4l401ktimersoftd/321:50:023
965999290,17cyclictest3862-21pluginstate22:33:323
965999290,16cyclictest23662-21/usr/sbin/munin00:13:283
965999290,16cyclictest23662-21/usr/sbin/munin00:13:283
965999290,15cyclictest28407-21apt-get21:03:213
965899293,12cyclictest5874-21rm22:36:502
965899292,12cyclictest0-21swapper/200:16:112
965899292,12cyclictest0-21swapper/200:16:112
965899292,10cyclictest0-21swapper/221:48:282
9658992916,7cyclictest8584-21apt-get00:33:252
9658992914,10cyclictest0-21swapper/223:03:222
9658992913,12cyclictest0-21swapper/223:38:242
9658992913,12cyclictest0-21swapper/219:58:192
78062295,10sleep20-21swapper/219:13:272
965999280,16cyclictest18579-21ls23:28:343
9658992813,11cyclictest0-21swapper/221:08:192
9658992810,14cyclictest26559-21grep19:48:362
9658992810,13cyclictest0-21swapper/223:48:232
9657992823,3cyclictest25-21ksoftirqd/100:38:321
9657992822,3cyclictest25-21ksoftirqd/120:38:291
9657992822,3cyclictest25-21ksoftirqd/120:38:291
9657992822,3cyclictest25-21ksoftirqd/119:23:141
965999270,13cyclictest3521-21sed20:08:353
965999270,13cyclictest27066-21sed20:58:323
965899275,8cyclictest2859-21diskmemload23:43:262
965899274,9cyclictest29927-21ls21:03:382
9658992716,6cyclictest22034-21apt-get19:41:282
9658992712,11cyclictest0-21swapper/200:28:262
9657992722,3cyclictest25-21ksoftirqd/122:58:291
9657992721,3cyclictest25-21ksoftirqd/123:38:311
965899262,11cyclictest0-21swapper/220:13:262
9658992620,3cyclictest33-21ksoftirqd/220:33:492
9658992620,3cyclictest33-21ksoftirqd/220:33:492
9658992619,4cyclictest33-21ksoftirqd/219:54:002
9658992614,8cyclictest23645-21tr20:53:202
9658992612,7cyclictest6856-21apt-get22:38:222
9658992612,11cyclictest11940-21gltestperf20:28:252
9658992612,10cyclictest1466-21snmp_o5powerctl00:23:372
9658992610,14cyclictest4298-21needreboot21:18:292
9657992621,3cyclictest25-21ksoftirqd/121:58:301
9657992621,3cyclictest25-21ksoftirqd/121:33:401
9657992620,4cyclictest25-21ksoftirqd/120:48:211
9657992620,3cyclictest25-21ksoftirqd/119:29:341
9657992619,4cyclictest25-21ksoftirqd/100:28:261
9656992621,3cyclictest9-21ksoftirqd/023:08:320
9656992620,3cyclictest9-21ksoftirqd/020:52:040
9656992620,3cyclictest9-21ksoftirqd/000:15:250
9656992620,3cyclictest9-21ksoftirqd/000:15:250
104591260,1ptp4l401ktimersoftd/323:38:453
104591260,1ptp4l401ktimersoftd/322:42:393
965999251,9cyclictest31795-21latency_hist20:03:143
965999250,11cyclictest320-21sshd21:11:593
965999250,11cyclictest23029-21if_enp1s019:43:243
965899252,10cyclictest0-21swapper/219:28:272
9658992520,3cyclictest33-21ksoftirqd/221:43:302
9658992518,4cyclictest2440-21sed20:08:222
9658992513,8cyclictest22952-21sed19:43:232
9658992512,9cyclictest22935-21irqrtprio22:18:282
9658992512,9cyclictest20211-21awk20:43:362
9658992512,7cyclictest9740-21apt-get19:18:162
9657992520,3cyclictest25-21ksoftirqd/122:03:401
9657992520,3cyclictest25-21ksoftirqd/120:08:351
9657992520,3cyclictest25-21ksoftirqd/119:23:141
9657992519,3cyclictest25-21ksoftirqd/123:48:241
9657992519,3cyclictest25-21ksoftirqd/122:13:301
9657992519,3cyclictest25-21ksoftirqd/119:44:081
9657992518,4cyclictest25-21ksoftirqd/120:46:231
9657992518,4cyclictest25-21ksoftirqd/119:52:301
965799250,12cyclictest178432tr19:33:191
9656992520,3cyclictest9-21ksoftirqd/020:04:450
9656992519,3cyclictest9-21ksoftirqd/021:00:450
9656992519,3cyclictest9-21ksoftirqd/020:48:060
9656992519,3cyclictest9-21ksoftirqd/020:40:050
9656992519,3cyclictest9-21ksoftirqd/020:40:050
9656992519,3cyclictest9-21ksoftirqd/020:10:140
9656992519,3cyclictest9-21ksoftirqd/019:27:190
9656992518,4cyclictest9-21ksoftirqd/022:06:080
9656992518,4cyclictest9-21ksoftirqd/021:10:510
9656992518,4cyclictest9-21ksoftirqd/020:27:510
9656992518,4cyclictest7916-21grep23:53:340
965699251,3cyclictest121rcu_preempt21:38:030
104591250,1ptp4l401ktimersoftd/322:23:043
9659992418,3cyclictest41-21ksoftirqd/320:05:293
965999241,7cyclictest2859-21diskmemload00:18:383
9659992414,7cyclictest6644-21/usr/sbin/munin20:18:343
9659992414,7cyclictest4670-21fschecks_count20:13:223
965999240,10cyclictest0-21swapper/320:48:323
965899247,12cyclictest0-21swapper/221:13:212
9658992412,9cyclictest26694-21tr20:58:302
9658992412,9cyclictest22791-21grep20:48:382
965799246,12cyclictest30413-21ssh23:43:241
965799244,13cyclictest22241-21ssh22:18:211
9657992419,3cyclictest25-21ksoftirqd/122:43:311
9657992418,3cyclictest25-21ksoftirqd/121:18:271
9657992418,3cyclictest25-21ksoftirqd/120:04:541
9657992417,4cyclictest25-21ksoftirqd/119:39:251
9656992419,3cyclictest9-21ksoftirqd/019:38:290
9656992419,3cyclictest9-21ksoftirqd/019:35:080
9656992418,3cyclictest9-21ksoftirqd/022:48:080
9656992418,3cyclictest9-21ksoftirqd/020:56:450
9656992418,3cyclictest9-21ksoftirqd/020:22:170
9656992418,3cyclictest9-21ksoftirqd/019:52:070
9656992418,3cyclictest9-21ksoftirqd/000:28:320
9656992417,4cyclictest9-21ksoftirqd/022:25:360
9656992417,4cyclictest9-21ksoftirqd/020:37:260
9656992417,4cyclictest9-21ksoftirqd/020:37:260
79482240,2sleep0965699cyclictest23:17:160
22402240,20sleep12248-21ssh23:09:151
1091912418,3phc2sys41-21ksoftirqd/319:18:213
104591240,1ptp4l401ktimersoftd/323:13:373
104591240,1ptp4l401ktimersoftd/319:53:323
104591240,1ptp4l401ktimersoftd/319:48:173
9659992313,7cyclictest19797-21latency_hist19:38:143
965899237,13cyclictest6644-21/usr/sbin/munin20:18:252
9658992310,9cyclictest20186-21cut00:08:332
9658992310,9cyclictest20186-21cut00:08:332
965799234,13cyclictest19465-21ssh00:08:251
965799234,13cyclictest19465-21ssh00:08:251
9657992318,3cyclictest25-21ksoftirqd/123:33:331
9657992318,3cyclictest25-21ksoftirqd/123:18:411
9657992318,3cyclictest25-21ksoftirqd/122:08:331
9657992318,3cyclictest25-21ksoftirqd/120:33:351
9657992318,3cyclictest25-21ksoftirqd/120:33:351
9657992318,3cyclictest25-21ksoftirqd/120:28:251
9657992317,3cyclictest25-21ksoftirqd/123:28:321
9657992317,3cyclictest25-21ksoftirqd/122:34:201
9657992317,3cyclictest25-21ksoftirqd/122:23:291
9657992317,3cyclictest25-21ksoftirqd/121:08:201
9657992316,4cyclictest25-21ksoftirqd/119:53:261
9656992318,3cyclictest9-21ksoftirqd/020:13:350
9656992318,3cyclictest9-21ksoftirqd/000:03:420
9656992316,4cyclictest9-21ksoftirqd/021:13:200
96292230,3sleep1965799cyclictest22:40:391
104591230,1ptp4l401ktimersoftd/322:43:253
104591230,1ptp4l401ktimersoftd/321:13:213
104591230,1ptp4l401ktimersoftd/320:53:183
104591230,1ptp4l401ktimersoftd/320:43:213
104591230,1ptp4l401ktimersoftd/320:33:333
104591230,1ptp4l401ktimersoftd/320:33:333
104591230,1ptp4l401ktimersoftd/320:28:193
104591230,1ptp4l401ktimersoftd/319:28:343
104591230,1ptp4l401ktimersoftd/319:23:183
9657992217,3cyclictest25-21ksoftirqd/121:23:301
9657992216,4cyclictest25-21ksoftirqd/120:58:111
9657992216,3cyclictest25-21ksoftirqd/120:58:561
9657992216,3cyclictest25-21ksoftirqd/120:18:341
9657992216,3cyclictest25-21ksoftirqd/120:15:221
9657992215,4cyclictest18489-21sh00:43:361
9656992217,3cyclictest9-21ksoftirqd/021:48:300
9656992216,3cyclictest9-21ksoftirqd/022:40:540
9656992216,3cyclictest9-21ksoftirqd/019:45:090
9656992216,3cyclictest9-21ksoftirqd/019:21:100
9657992116,3cyclictest25-21ksoftirqd/123:13:271
9657992116,3cyclictest25-21ksoftirqd/122:49:221
9657992116,3cyclictest25-21ksoftirqd/100:13:421
9657992116,3cyclictest25-21ksoftirqd/100:13:421
9657992115,3cyclictest2647-21grep21:53:391
9657992114,4cyclictest29293-21grep21:48:241
9657992113,5cyclictest32437-21apt-get00:23:241
965699212,16cyclictest13004-21cpuspeed_turbos00:38:270
9656992116,3cyclictest9-21ksoftirqd/023:38:280
9656992116,3cyclictest9-21ksoftirqd/023:33:440
9656992115,4cyclictest9-21ksoftirqd/021:06:170
9656992114,4cyclictest9-21ksoftirqd/020:30:480
9656992114,4cyclictest2218-21/usr/sbin/munin23:48:280
9656992114,4cyclictest17867-21/usr/sbin/munin22:13:200
174322210,2sleep00-21swapper/019:33:140
965799202,3cyclictest313432ssh23:06:231
9657992015,3cyclictest25-21ksoftirqd/123:23:251
9657992015,3cyclictest25-21ksoftirqd/100:18:351
9657992014,4cyclictest25-21ksoftirqd/100:03:321
9657992014,3cyclictest26499-21vmstat21:43:421
9657992014,3cyclictest25-21ksoftirqd/120:27:531
9656992015,3cyclictest9-21ksoftirqd/023:58:340
9656992015,3cyclictest9-21ksoftirqd/022:48:220
9656992015,3cyclictest9-21ksoftirqd/019:57:560
9656992015,3cyclictest20059-21cut22:53:260
9656992013,4cyclictest9-21ksoftirqd/020:01:050
9656992013,4cyclictest339-21sh22:29:070
9656992013,4cyclictest339-21sh22:29:070
9656992013,4cyclictest24963-21sh22:20:430
9656992013,4cyclictest17547-21fschecks_time00:43:290
965699201,11cyclictest32442-21rm00:23:240
965799193,10cyclictest30756-21apt-get22:28:211
965799193,10cyclictest30756-21apt-get22:28:211
9657991913,4cyclictest25-21ksoftirqd/121:28:271
9657991913,4cyclictest25-21ksoftirqd/121:13:271
9657991913,3cyclictest25-21ksoftirqd/121:05:211
9657991913,3cyclictest25-21ksoftirqd/119:58:261
9656991914,3cyclictest9-21ksoftirqd/023:28:330
9656991914,3cyclictest9-21ksoftirqd/022:58:290
9656991914,3cyclictest24571-21/usr/sbin/munin21:43:300
9656991913,4cyclictest886-21wc21:53:220
9656991913,4cyclictest7355-21sh21:59:330
9656991913,3cyclictest30532-21cut23:43:250
9656991913,3cyclictest30213-21sh00:19:110
9656991913,3cyclictest14997-21sh21:31:350
9656991913,3cyclictest10087-21sh21:25:100
9656991912,4cyclictest3124-21cat21:18:190
9656991912,4cyclictest2547-21acpi22:33:200
9656991912,4cyclictest15057-21sendmail_mailqu22:08:340
965699191,15cyclictest10375-21unixbench_singl00:33:440
965699181,4cyclictest0-21swapper/000:08:320
965699181,4cyclictest0-21swapper/000:08:320
9656991812,4cyclictest11179-21sh23:19:580
9656991812,3cyclictest20468-21cat21:38:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional