You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-08-04 - 21:43

Intel(R) Atom(TM) CPU E3845 @ 1.91GHz, Linux 4.19.37-rt20 (Profile)

Latency plot of system in rack #b, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rackbslot6.osadl.org (updated Wed Aug 04, 2021 12:43:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
873911040,34ptp4l401ktimersoftd/309:06:353
44542720,3sleep00-21swapper/014:32:330
87391710,1ptp4l401ktimersoftd/312:35:023
87391700,1ptp4l401ktimersoftd/312:30:493
87391680,2ptp4l26099-21kworker/3:012:19:533
87391680,1ptp4l401ktimersoftd/312:48:023
78172670,1sleep10-21swapper/111:49:431
78172670,1sleep10-21swapper/111:49:431
87391650,1ptp4l401ktimersoftd/313:16:553
70332624,10sleep10-21swapper/109:09:311
65062625,42sleep00-21swapper/009:05:470
87391610,1ptp4l401ktimersoftd/314:01:013
87391610,1ptp4l401ktimersoftd/313:55:483
87391590,1ptp4l401ktimersoftd/313:08:183
87391590,1ptp4l401ktimersoftd/311:39:513
87391570,1ptp4l401ktimersoftd/312:59:243
87391570,1ptp4l401ktimersoftd/309:45:393
41262570,2sleep30-21swapper/314:32:003
87391550,1ptp4l401ktimersoftd/312:52:043
7300995510,13cyclictest7349-21irqrtprio14:35:441
7300995410,13cyclictest8469-21sed10:15:491
7300995410,13cyclictest8469-21sed10:15:491
730099539,13cyclictest4042-21acpi13:25:371
730099539,13cyclictest4042-21acpi13:25:371
730099519,13cyclictest4265-21cat12:51:101
7300995111,11cyclictest839-21diskmemload11:35:151
69792514,35sleep20-21swapper/209:08:522
160922490,2sleep20-21swapper/211:57:192
160922490,2sleep20-21swapper/211:57:192
730099489,12cyclictest6780-21sh11:48:211
730099489,12cyclictest32022-21sh13:53:341
7300994811,12cyclictest20910-21sh13:41:171
730099478,18cyclictest18480-21mailstats13:05:521
730099478,15cyclictest16879-21snmp_o5powerctl14:11:001
730099467,15cyclictest839-21diskmemload11:34:161
730099466,13cyclictest0-21swapper/113:46:031
730099466,13cyclictest0-21swapper/111:17:181
52872460,2sleep20-21swapper/211:45:512
730099457,11cyclictest31243-21ssh12:12:501
730099455,11cyclictest0-21swapper/114:07:231
730099448,17cyclictest32519-21sh12:47:021
730099448,16cyclictest1649-21ssh13:21:501
730099447,12cyclictest1405-21sh11:10:101
87391430,1ptp4l401ktimersoftd/314:28:393
730099437,15cyclictest28568-21ssh14:24:341
87391420,1ptp4l401ktimersoftd/313:46:063
87391420,1ptp4l401ktimersoftd/310:42:173
730099428,9cyclictest10736-21diskstats12:25:371
7300994212,26cyclictest10701-21expr09:15:301
730099414,10cyclictest0-21swapper/113:19:381
87391400,0ptp4l401ktimersoftd/312:25:173
730099399,27cyclictest0-21swapper/110:00:461
87391380,1ptp4l401ktimersoftd/314:20:543
87391380,1ptp4l401ktimersoftd/313:42:433
87391380,1ptp4l401ktimersoftd/311:51:443
87391380,1ptp4l401ktimersoftd/311:51:443
87391380,1ptp4l401ktimersoftd/311:36:013
87391380,1ptp4l401ktimersoftd/311:20:373
87391380,1ptp4l401ktimersoftd/311:09:553
87391380,0ptp4l401ktimersoftd/311:26:573
7300993812,9cyclictest22460-21munin-run12:04:371
87391370,2ptp4l26099-21kworker/3:013:34:333
87391370,1ptp4l401ktimersoftd/313:28:203
87391370,1ptp4l401ktimersoftd/313:28:203
87391370,1ptp4l401ktimersoftd/313:02:073
87391370,1ptp4l401ktimersoftd/312:41:193
87391370,1ptp4l401ktimersoftd/312:19:113
730099377,26cyclictest0-21swapper/109:30:381
87391360,1ptp4l401ktimersoftd/314:10:103
87391360,1ptp4l401ktimersoftd/312:13:173
87391360,1ptp4l401ktimersoftd/309:33:193
87391360,0ptp4l401ktimersoftd/313:10:323
87391360,0ptp4l401ktimersoftd/311:15:033
730099369,24cyclictest0-21swapper/110:40:401
730099369,14cyclictest21656-21snmp_o5powerctl14:15:591
7300993612,8cyclictest12189-21/usr/sbin/munin11:20:371
7300993610,12cyclictest839-21diskmemload13:35:461
87391350,12ptp4l401ktimersoftd/313:39:193
87391350,0ptp4l401ktimersoftd/312:00:573
87391350,0ptp4l401ktimersoftd/309:35:513
730099358,17cyclictest839-21diskmemload12:55:061
7300993523,6cyclictest7445-21apt-get09:10:261
7300993523,6cyclictest4525-21apt-get10:10:291
7300993512,9cyclictest9463-21apt-get10:20:291
7300993512,9cyclictest9463-21apt-get10:20:291
87391340,2ptp4l26099-21kworker/3:013:22:133
87391340,2ptp4l26099-21kworker/3:010:05:553
87391340,0ptp4l401ktimersoftd/314:36:313
87391340,0ptp4l401ktimersoftd/314:18:293
87391340,0ptp4l401ktimersoftd/312:08:153
730099349,23cyclictest0-21swapper/110:30:431
730099349,23cyclictest0-21swapper/110:30:431
730099344,13cyclictest0-21swapper/112:08:421
7300993422,6cyclictest21680-21apt-get10:45:261
7300993422,5cyclictest15288-21apt-get09:25:291
7300993417,12cyclictest0-21swapper/112:35:311
87391330,1ptp4l401ktimersoftd/311:31:523
730099336,12cyclictest0-21swapper/112:21:051
7300993315,14cyclictest32185-21/usr/sbin/munin11:40:401
7300993311,8cyclictest8949-21apt-get13:30:341
87391320,1ptp4l1901-21kworker/3:109:40:243
87391320,10ptp4l401ktimersoftd/310:47:003
87391320,0ptp4l401ktimersoftd/314:05:213
87391320,0ptp4l401ktimersoftd/310:17:563
87391320,0ptp4l401ktimersoftd/310:17:563
730099329,20cyclictest839-21diskmemload11:58:041
730099329,20cyclictest839-21diskmemload11:58:041
730099329,10cyclictest839-21diskmemload14:03:031
730099326,12cyclictest0-21swapper/110:55:361
730099325,17cyclictest0-21swapper/110:25:301
87391310,1ptp4l401ktimersoftd/313:52:583
87391310,1ptp4l401ktimersoftd/311:59:183
87391310,1ptp4l401ktimersoftd/311:59:183
87391310,1ptp4l401ktimersoftd/310:58:113
87391310,1ptp4l401ktimersoftd/309:53:173
87391310,1ptp4l401ktimersoftd/309:11:293
87391310,0ptp4l401ktimersoftd/311:49:033
87391310,0ptp4l401ktimersoftd/310:51:333
87391310,0ptp4l401ktimersoftd/310:35:063
730099319,8cyclictest839-21diskmemload14:25:141
730099316,10cyclictest0-21swapper/114:34:131
7300993116,11cyclictest0-21swapper/113:55:351
87391300,1ptp4l26099-21kworker/3:010:19:453
87391300,1ptp4l26099-21kworker/3:010:19:453
7301993014,13cyclictest8237-21sh14:01:522
730099305,11cyclictest0-21swapper/113:03:271
730099305,11cyclictest0-21swapper/112:15:491
730099304,11cyclictest0-21swapper/112:40:331
730099303,10cyclictest0-21swapper/109:40:461
7300993015,11cyclictest0-21swapper/111:25:341
730199294,3cyclictest121rcu_preempt14:28:432
730099293,9cyclictest0-21swapper/109:55:341
7300992920,5cyclictest21878-21apt-get13:10:381
7300992918,6cyclictest2138-21apt-get10:05:301
7300992915,10cyclictest0-21swapper/109:20:251
730099283,10cyclictest0-21swapper/112:33:571
7299992818,7cyclictest5152-21ssh11:13:380
87391270,1ptp4l401ktimersoftd/309:55:023
87391270,0ptp4l401ktimersoftd/310:00:103
730099273,9cyclictest0-21swapper/110:35:401
730099273,22cyclictest0-21swapper/109:38:221
7300992715,9cyclictest31339-21grep11:05:321
7300992714,9cyclictest0-21swapper/109:45:261
87391260,1ptp4l401ktimersoftd/310:09:393
87391260,1ptp4l401ktimersoftd/309:22:353
87391260,0ptp4l401ktimersoftd/311:02:023
7301992618,5cyclictest30492-21grep11:00:572
7300992615,8cyclictest27733-21fschecks_time09:50:311
7300992615,8cyclictest25411-21snmp_o5powerctl10:50:391
7300992614,9cyclictest28721-21taskset10:59:431
729999262,20cyclictest30779-21cut12:45:460
7299992612,4cyclictest3387-21cut12:50:530
87391250,1ptp4l401ktimersoftd/309:25:313
87391250,10ptp4l401ktimersoftd/309:18:313
87391250,0ptp4l401ktimersoftd/310:24:363
730199253,3cyclictest121rcu_preempt12:36:152
7301992517,4cyclictest31790-21sh09:59:342
7301992516,5cyclictest6553-21rm14:35:362
729999254,13cyclictest28763-21ssh13:50:450
729999252,20cyclictest25291-21sed14:20:420
87391240,1ptp4l401ktimersoftd/311:05:313
7301992417,4cyclictest25397-21sh12:06:092
7301992417,4cyclictest1127-21tune2fs12:15:492
729999243,13cyclictest63-21kswapd014:10:370
7299992419,3cyclictest9-21ksoftirqd/014:35:370
7299992418,3cyclictest9-21ksoftirqd/012:04:380
87391230,1ptp4l401ktimersoftd/310:29:393
87391230,1ptp4l401ktimersoftd/310:29:393
730199239,9cyclictest26708-21ssh13:15:342
730199238,12cyclictest17823-21grep09:30:362
730199237,11cyclictest6537-21ssh11:15:302
730199232,8cyclictest12176-21apt-get13:00:372
7301992316,4cyclictest16906-21sh13:05:322
7301992315,4cyclictest33-21ksoftirqd/213:34:432
7301992313,6cyclictest25790-21vmstat13:46:062
729999233,12cyclictest21888-21ssh13:10:380
7299992317,3cyclictest23911-21grep13:45:360
7299992316,4cyclictest8271-21/usr/sbin/munin11:50:430
7299992316,4cyclictest8271-21/usr/sbin/munin11:50:430
284182230,2sleep00-21swapper/011:36:410
13052230,2sleep20-21swapper/213:55:372
104222231,4sleep010499-21apt-get09:15:280
7301992216,3cyclictest22696-21sh14:17:122
7301992215,4cyclictest33-21ksoftirqd/210:21:142
7301992215,4cyclictest33-21ksoftirqd/210:21:142
7301992215,4cyclictest33-21ksoftirqd/210:16:492
7301992215,4cyclictest33-21ksoftirqd/210:16:492
7301992215,4cyclictest33-21ksoftirqd/210:05:412
7301992215,4cyclictest20449-21sendmail_mailst10:40:412
7301992215,4cyclictest18637-21sh13:40:272
7301992214,5cyclictest21178-21sh11:30:292
7301992214,4cyclictest26141-21mailstats09:45:392
7301992213,5cyclictest30940-21irqrtprio12:45:472
7301992213,5cyclictest17571-21rm11:25:432
7301992210,9cyclictest1989-21sh12:49:552
729999228,11cyclictest26798-21cat12:41:080
729999222,17cyclictest10990-21ssh12:25:400
7299992217,3cyclictest9-21ksoftirqd/014:15:480
7299992216,4cyclictest9-21ksoftirqd/013:24:440
7299992216,4cyclictest9-21ksoftirqd/013:24:440
7299992216,3cyclictest9-21ksoftirqd/010:55:290
7299992216,3cyclictest9-21ksoftirqd/009:11:220
7299992215,4cyclictest9-21ksoftirqd/013:20:510
7299992215,4cyclictest9-21ksoftirqd/009:26:520
7299992215,4cyclictest23021-21timerwakeupswit11:30:550
7299992215,3cyclictest9-21ksoftirqd/009:54:160
7299992214,5cyclictest849-21runrttasks11:26:050
7299992214,4cyclictest18926-21cpuspeed_turbos13:40:360
7299992213,5cyclictest7952-21sendmail_mailtr11:15:420
729999221,17cyclictest30870-21ls14:26:010
730199215,11cyclictest10824-21unixbench_singl13:30:552
730199213,4cyclictest26100-21kworker/2:112:09:522
7301992115,3cyclictest25430-21sh12:40:412
7301992114,4cyclictest7156-21sh12:21:002
7301992114,4cyclictest7051-21sh12:55:132
7301992114,4cyclictest5580-21sh14:34:272
7301992114,4cyclictest5514-21ptp4l13:25:582
7301992114,4cyclictest5514-21ptp4l13:25:582
7301992114,4cyclictest33-21ksoftirqd/210:24:442
7301992114,4cyclictest33-21ksoftirqd/209:42:482
7301992114,4cyclictest32734-21vmstat11:40:492
7301992114,4cyclictest29314-21sh11:38:162
7301992114,4cyclictest28450-21sh13:50:202
7301992114,4cyclictest28341-21seq14:24:102
7301992114,4cyclictest21136-21sh12:02:272
7301992114,4cyclictest14647-21sh11:22:582
7301992114,4cyclictest12955-21sh14:06:532
7301992113,5cyclictest7821-21sh11:49:442
7301992113,5cyclictest7821-21sh11:49:442
7301992113,5cyclictest15524-21cstates12:30:392
7301992113,4cyclictest12521-21cron09:19:342
7299992115,3cyclictest9-21ksoftirqd/011:00:570
7299992115,3cyclictest15599-21ls11:24:370
7299992115,3cyclictest13912-21/usr/sbin/munin13:35:440
7299992114,4cyclictest9-21ksoftirqd/011:08:180
7299992114,4cyclictest9-21ksoftirqd/010:53:210
7299992114,4cyclictest23691-21diskstats12:05:410
7299992114,4cyclictest14614-21sh10:30:290
7299992114,4cyclictest14614-21sh10:30:290
7299992114,4cyclictest1403-21sh13:55:400
7299992114,4cyclictest10626-21sed13:30:530
7299992113,5cyclictest9-21ksoftirqd/010:35:060
7299992113,5cyclictest13950-21seq10:28:550
7299992112,5cyclictest20337-21cstates12:35:340
729999210,18cyclictest32301-21ssh12:14:380
730199205,9cyclictest26454-21apt-get10:55:312
730199204,10cyclictest1688-21apt-get11:10:332
7301992014,3cyclictest33-21ksoftirqd/209:26:072
7301992014,3cyclictest31166-21/usr/sbin/munin11:05:372
7301992014,3cyclictest28227-21sed09:50:352
7301992013,4cyclictest862-21phc2sys10:00:432
7301992013,4cyclictest24392-21grep10:50:302
7301992013,4cyclictest22302-21fw_packets13:10:422
7301992013,4cyclictest17942-21sh14:12:142
7301992013,4cyclictest16680-21/usr/sbin/munin10:35:512
7301992013,4cyclictest11455-21sh12:25:482
7301992012,5cyclictest15097-21latency_hist09:24:342
7301992012,4cyclictest31851-21cstates13:20:322
7301992011,5cyclictest16407-21idleruntime-cro10:34:352
7301992011,5cyclictest16407-21idleruntime-cro10:34:352
729999209,8cyclictest22117-21idleruntime-cro09:39:340
7299992014,4cyclictest9-21ksoftirqd/010:49:300
7299992014,4cyclictest24776-21/usr/sbin/munin09:45:280
7299992014,3cyclictest9386-21/usr/sbin/munin10:20:360
7299992014,3cyclictest9386-21/usr/sbin/munin10:20:360
7299992014,3cyclictest9-21ksoftirqd/010:01:250
7299992014,3cyclictest5737-21sh14:00:320
7299992014,3cyclictest3295-21proc_pri10:05:420
7299992014,3cyclictest27796-21sed13:15:500
7299992014,3cyclictest2509-21timerwakeupswit12:16:120
7299992014,3cyclictest13231-21cut09:20:300
7299992013,4cyclictest8045-21phc2sys10:15:410
7299992013,4cyclictest8045-21phc2sys10:15:410
7299992013,4cyclictest5571-21ssh11:46:210
7299992013,4cyclictest5427-21/usr/sbin/munin12:20:300
7299992013,4cyclictest31979-21tail11:40:380
7299992013,4cyclictest23578-21open_inodes09:40:400
7299992013,4cyclictest12626-21gltestperf13:00:440
7299992013,3cyclictest9-21ksoftirqd/009:59:250
7301991912,4cyclictest8937-21wc09:10:442
7301991912,4cyclictest33-21ksoftirqd/210:45:202
7301991912,4cyclictest33-21ksoftirqd/210:11:272
7299991913,3cyclictest9-21ksoftirqd/009:29:410
7299991913,3cyclictest17229-21cat13:05:400
7299991912,4cyclictest15487-21cpuspeed_turbos12:30:380
7299991912,4cyclictest10725-21grep14:05:390
7299991911,5cyclictest20627-21wc10:40:470
730199180,4cyclictest20073-21taskset09:35:292
729999185,6cyclictest0-21swapper/010:10:290
7299991812,3cyclictest14434-21needreboot11:55:390
7299991812,3cyclictest14434-21needreboot11:55:390
7299991812,3cyclictest101ktimersoftd/012:55:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional