You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-24 - 04:25

x86 Intel Atom E3845 @1910 MHz, Linux 4.19.37-rt19 (Profile)

Latency plot of system in rack #d, slot #1
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Up99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rackdslot1.osadl.org (updated Fri Mar 24, 2023 00:46:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1176911830,8ptp4l0-21swapper/119:06:421
1176911640,9ptp4l0-21swapper/319:09:413
1728821500,9sleep11540599cyclictest21:30:181
1176911480,13ptp4l0-21swapper/219:07:542
1176911450,15ptp4l0-21swapper/019:09:050
117691740,2ptp4l0-21swapper/223:55:312
117691710,2ptp4l0-21swapper/322:45:143
15407996954,7cyclictest41-21ksoftirqd/321:00:143
15407996855,7cyclictest41-21ksoftirqd/300:00:153
15407996854,7cyclictest41-21ksoftirqd/323:00:193
15407996854,7cyclictest41-21ksoftirqd/319:50:133
15407996753,7cyclictest41-21ksoftirqd/321:30:133
15407996653,6cyclictest41-21ksoftirqd/321:10:203
15405996526,11cyclictest25-21ksoftirqd/121:04:591
117691640,5ptp4l1159-21snmpd19:10:312
15407996351,6cyclictest41-21ksoftirqd/320:10:123
1540499637,13cyclictest10280-21/usr/sbin/munin20:05:260
1540499637,13cyclictest10280-21/usr/sbin/munin20:05:260
1540499636,13cyclictest18796-21ps23:50:430
15407996251,6cyclictest41-21ksoftirqd/321:05:123
15405996227,5cyclictest25-21ksoftirqd/120:40:191
15405996226,8cyclictest25-21ksoftirqd/122:40:251
15405996226,8cyclictest25-21ksoftirqd/122:40:251
1540499624,14cyclictest13098-21fschecks_time22:30:210
15405996125,8cyclictest25-21ksoftirqd/120:45:171
117691610,2ptp4l0-21swapper/220:36:272
258072600,5sleep1231rcuc/119:30:301
15407996049,6cyclictest41-21ksoftirqd/322:50:143
15405996027,5cyclictest25-21ksoftirqd/121:35:221
15405996026,8cyclictest25-21ksoftirqd/123:00:231
15405996026,8cyclictest25-21ksoftirqd/121:45:431
15405996025,7cyclictest25-21ksoftirqd/122:10:181
15405996025,7cyclictest25-21ksoftirqd/120:10:241
15405996024,8cyclictest25-21ksoftirqd/120:25:001
15405996023,8cyclictest25-21ksoftirqd/119:25:191
1540499604,14cyclictest22923-21cpuspeed_turbos19:25:140
1540499602,8cyclictest117691ptp4l21:25:000
1540499602,8cyclictest117691ptp4l21:25:000
117691600,2ptp4l0-21swapper/122:30:151
241532590,2sleep30-21swapper/319:25:303
15405995926,8cyclictest25-21ksoftirqd/123:15:331
15405995926,5cyclictest25-21ksoftirqd/119:15:001
15405995926,5cyclictest25-21ksoftirqd/100:00:001
15405995923,8cyclictest25-21ksoftirqd/122:20:201
1540499594,15cyclictest14634-21grep21:25:180
1540499594,15cyclictest14634-21grep21:25:170
117691590,2ptp4l0-21swapper/000:37:030
117691590,14ptp4l0-21swapper/221:45:552
15407995848,5cyclictest41-21ksoftirqd/323:45:173
15407995817,5cyclictest41-21ksoftirqd/319:35:273
15405995827,5cyclictest25-21ksoftirqd/100:05:291
15405995826,5cyclictest25-21ksoftirqd/123:40:221
15405995825,8cyclictest25-21ksoftirqd/123:20:141
15405995825,8cyclictest25-21ksoftirqd/121:25:321
15405995825,8cyclictest25-21ksoftirqd/121:25:321
15405995825,8cyclictest25-21ksoftirqd/121:15:311
15405995825,7cyclictest25-21ksoftirqd/122:25:321
15405995825,5cyclictest25-21ksoftirqd/120:40:011
15405995825,5cyclictest25-21ksoftirqd/119:40:001
15405995825,10cyclictest25-21ksoftirqd/100:35:521
15405995824,8cyclictest25-21ksoftirqd/121:50:231
15405995824,8cyclictest25-21ksoftirqd/121:10:241
15405995824,5cyclictest25-21ksoftirqd/120:00:291
15405995824,5cyclictest25-21ksoftirqd/120:00:291
15405995824,11cyclictest25-21ksoftirqd/119:50:301
15405995823,8cyclictest25-21ksoftirqd/123:30:211
15405995823,5cyclictest25-21ksoftirqd/120:55:251
117691580,2ptp4l0-21swapper/221:59:262
117691580,2ptp4l0-21swapper/023:55:070
117691580,2ptp4l0-21swapper/000:15:000
15407995744,8cyclictest41-21ksoftirqd/323:20:133
15405995729,4cyclictest25-21ksoftirqd/122:15:161
15405995728,4cyclictest25-21ksoftirqd/123:55:011
15405995727,5cyclictest25-21ksoftirqd/122:35:221
15405995727,5cyclictest25-21ksoftirqd/122:35:211
15405995725,5cyclictest25-21ksoftirqd/123:45:361
15405995724,8cyclictest25-21ksoftirqd/119:55:301
15405995724,5cyclictest25-21ksoftirqd/122:05:371
15405995724,10cyclictest25-21ksoftirqd/119:15:321
15405995723,8cyclictest25-21ksoftirqd/100:10:381
15405995723,8cyclictest25-21ksoftirqd/100:00:231
15405995723,11cyclictest25-21ksoftirqd/120:15:161
15405995722,8cyclictest25-21ksoftirqd/100:30:201
15405995721,8cyclictest25-21ksoftirqd/121:40:291
15405995720,9cyclictest25-21ksoftirqd/120:30:011
1540499574,14cyclictest5788-21cpuspeed_turbos22:15:160
1540499573,14cyclictest19992-21df_abs22:45:170
1540499573,14cyclictest16305-21/usr/sbin/munin20:20:260
117691570,6ptp4l33-21ksoftirqd/220:51:592
117691570,1ptp4l0-21swapper/222:45:352
117691570,11ptp4l0-21swapper/200:36:272
1540799567,9cyclictest41-21ksoftirqd/321:25:003
1540799567,9cyclictest41-21ksoftirqd/321:25:003
15407995646,5cyclictest41-21ksoftirqd/319:10:123
15407995643,6cyclictest41-21ksoftirqd/322:55:183
15407995623,8cyclictest41-21ksoftirqd/322:10:353
15405995627,4cyclictest25-21ksoftirqd/120:50:331
15405995626,4cyclictest25-21ksoftirqd/122:45:261
15405995625,5cyclictest25-21ksoftirqd/121:55:221
15405995624,5cyclictest25-21ksoftirqd/122:00:221
15405995623,9cyclictest25-21ksoftirqd/123:10:221
15405995623,8cyclictest25-21ksoftirqd/123:35:181
15405995622,8cyclictest25-21ksoftirqd/120:05:141
15405995622,8cyclictest25-21ksoftirqd/120:05:141
1540499566,12cyclictest9389-21sshd21:13:250
117691560,1ptp4l0-21swapper/200:15:482
15407995545,5cyclictest41-21ksoftirqd/319:20:163
15407995544,6cyclictest41-21ksoftirqd/321:45:193
15405995524,5cyclictest25-21ksoftirqd/120:30:231
15405995524,5cyclictest25-21ksoftirqd/119:45:201
15405995524,5cyclictest25-21ksoftirqd/100:25:181
15405995523,5cyclictest25-21ksoftirqd/100:20:201
15405995522,8cyclictest25-21ksoftirqd/122:55:401
15405995522,8cyclictest25-21ksoftirqd/119:20:241
1540499555,13cyclictest1721-21gnome-shell19:35:300
1540499555,12cyclictest32191-21meminfo23:10:280
1540499553,13cyclictest18585-21cut19:15:260
1540499552,14cyclictest2104-21cat19:50:000
1540499552,14cyclictest12864-21apt-get23:40:170
1540499552,13cyclictest22683-21awk20:30:340
117691550,2ptp4l0-21swapper/322:20:593
117691550,2ptp4l0-21swapper/319:45:173
117691550,1ptp4l0-21swapper/223:41:232
117691550,1ptp4l0-21swapper/223:20:162
117691550,1ptp4l0-21swapper/223:15:142
117691550,1ptp4l0-21swapper/021:55:220
15407995442,6cyclictest41-21ksoftirqd/320:15:143
15407995420,10cyclictest41-21ksoftirqd/322:00:293
1540499543,13cyclictest2819-21missed_timers00:25:290
117691540,4ptp4l16499-21ls22:35:402
117691540,4ptp4l16499-21ls22:35:402
117691540,2ptp4l0-21swapper/223:39:492
117691540,1ptp4l0-21swapper/222:21:502
15405995326,4cyclictest25-21ksoftirqd/100:15:251
1540499536,11cyclictest117691ptp4l00:10:360
1540499532,13cyclictest31500-21unixbench_multi19:40:310
1540499532,13cyclictest20766-21df_abs19:20:190
1540499532,12cyclictest26714-21systemctl00:07:160
117691530,2ptp4l0-21swapper/223:45:332
117691530,2ptp4l0-21swapper/021:30:240
117691530,1ptp4l0-21swapper/300:08:213
117691530,15ptp4l33-21ksoftirqd/222:04:162
117691530,11ptp4l0-21swapper/223:02:312
15405995225,4cyclictest25-21ksoftirqd/123:25:171
15405995224,4cyclictest25-21ksoftirqd/121:20:261
15405995224,4cyclictest25-21ksoftirqd/121:20:251
15405995221,5cyclictest25-21ksoftirqd/123:05:291
1540499521,14cyclictest315-21sshd22:01:270
1540499521,14cyclictest22189-21gpgv00:00:030
1540499521,13cyclictest3163-21sed22:10:000
1540499521,12cyclictest30268-21latency_hist20:50:000
117691520,6ptp4l9-21ksoftirqd/023:27:430
117691520,2ptp4l0-21swapper/219:35:312
117691520,1ptp4l0-21swapper/223:53:282
15407995139,6cyclictest41-21ksoftirqd/319:15:173
15407995120,5cyclictest41-21ksoftirqd/319:40:133
15407995119,5cyclictest41-21ksoftirqd/300:10:253
15405995121,5cyclictest25-21ksoftirqd/122:50:221
1540499518,13cyclictest10746-21missed_timers21:15:260
1540499512,13cyclictest25642-21/usr/sbin/munin20:40:290
1540499511,14cyclictest20152-21sed21:35:230
1540499511,13cyclictest24674-21df_abs22:55:220
1540499511,12cyclictest4118-21/usr/sbin/munin00:30:160
1540499511,12cyclictest13512-21latency_hist20:14:590
117691510,2ptp4l0-21swapper/320:20:023
117691510,2ptp4l0-21swapper/223:09:142
117691510,1ptp4l0-21swapper/021:40:180
15407995039,5cyclictest41-21ksoftirqd/320:45:143
1540599505,5cyclictest25-21ksoftirqd/121:09:591
1540599505,10cyclictest25-21ksoftirqd/119:35:231
1540499505,11cyclictest26743-21taskset19:30:530
1540499502,13cyclictest27289-21ntp_states21:50:350
1540499501,13cyclictest18778-21timerandwakeup22:40:420
1540499501,13cyclictest18778-21timerandwakeup22:40:410
117691500,2ptp4l0-21swapper/322:30:593
117691500,2ptp4l0-21swapper/219:16:592
117691500,1ptp4l0-21swapper/323:42:103
117691500,1ptp4l0-21swapper/223:10:262
15407994919,4cyclictest41-21ksoftirqd/321:55:133
15407994916,10cyclictest41-21ksoftirqd/321:25:363
15407994916,10cyclictest41-21ksoftirqd/321:25:353
15407994915,10cyclictest41-21ksoftirqd/320:35:333
1540499495,10cyclictest1403-21seq00:23:360
1540499494,11cyclictest30647-21taskset23:06:020
1540499492,14cyclictest25012-21unixbench_singl20:35:370
117791490,10getstats0-21swapper/023:30:360
117691490,9ptp4l0-21swapper/222:40:422
117691490,9ptp4l0-21swapper/222:40:412
117691490,2ptp4l0-21swapper/223:30:112
117691490,2ptp4l0-21swapper/023:18:550
117691490,2ptp4l0-21swapper/020:53:230
117691490,1ptp4l0-21swapper/222:05:272
15407994816,11cyclictest41-21ksoftirqd/300:15:233
1540499485,12cyclictest5147-21taskset22:12:270
1540499484,12cyclictest0-21swapper/020:56:500
1540499481,10cyclictest11712-21ls22:25:400
117691480,8ptp4l0-21swapper/222:34:502
117691480,2ptp4l33-21ksoftirqd/220:12:182
117691480,2ptp4l0-21swapper/323:57:153
117691480,2ptp4l0-21swapper/323:34:393
117691480,2ptp4l0-21swapper/220:09:202
117691480,2ptp4l0-21swapper/220:09:202
117691480,1ptp4l0-21swapper/222:18:282
117691480,1ptp4l0-21swapper/222:13:512
117691480,1ptp4l0-21swapper/200:28:112
117691480,1ptp4l0-21swapper/200:00:102
117691480,12ptp4l16225-21irqrtprio19:10:200
15407994734,4cyclictest41-21ksoftirqd/320:40:303
15407994712,9cyclictest41-21ksoftirqd/323:35:203
1540499472,13cyclictest15996-21sshd23:45:270
1540499471,11cyclictest24153-21latency_hist22:55:000
117691470,8ptp4l0-21swapper/221:39:472
117691470,8ptp4l0-21swapper/221:22:582
117691470,8ptp4l0-21swapper/221:22:572
117691470,8ptp4l0-21swapper/023:00:330
117691470,2ptp4l0-21swapper/221:18:492
117691470,2ptp4l0-21swapper/200:31:462
117691470,1ptp4l0-21swapper/222:52:242
117691470,1ptp4l0-21swapper/219:21:502
117691470,11ptp4l0-21swapper/220:30:252
1540799461,9cyclictest19321-21unattended-upgr21:35:153
15407994614,10cyclictest41-21ksoftirqd/300:20:323
15407994613,5cyclictest41-21ksoftirqd/323:05:333
117691460,8ptp4l0-21swapper/221:52:542
117691460,8ptp4l0-21swapper/220:22:362
117691460,8ptp4l0-21swapper/019:59:440
117691460,7ptp4l321ktimersoftd/219:56:052
117691460,2ptp4l0-21swapper/222:28:142
117691460,2ptp4l0-21swapper/221:03:022
117691460,2ptp4l0-21swapper/220:28:442
117691460,2ptp4l0-21swapper/021:00:590
117691460,1ptp4l0-21swapper/323:27:303
117691460,10ptp4l0-21swapper/022:20:260
15407994515,10cyclictest41-21ksoftirqd/323:15:363
117691450,8ptp4l0-21swapper/019:54:100
117691450,7ptp4l0-21swapper/023:35:420
117691450,7ptp4l0-21swapper/023:23:530
117691450,2ptp4l0-21swapper/322:15:423
117691450,2ptp4l0-21swapper/200:11:512
117691450,2ptp4l0-21swapper/020:04:130
117691450,2ptp4l0-21swapper/020:04:130
15407994413,11cyclictest41-21ksoftirqd/321:55:003
117691440,8ptp4l0-21swapper/220:44:232
117691440,8ptp4l0-21swapper/219:49:472
117691440,2ptp4l0-21swapper/322:39:223
117691440,2ptp4l0-21swapper/322:39:213
117691440,2ptp4l0-21swapper/319:32:143
117691440,2ptp4l0-21swapper/021:45:270
117691440,2ptp4l0-21swapper/021:06:250
117691440,1ptp4l0-21swapper/319:55:463
117691440,1ptp4l0-21swapper/300:31:133
117691440,1ptp4l0-21swapper/220:58:332
1540799438,9cyclictest41-21ksoftirqd/322:25:283
15407994320,11cyclictest41-21ksoftirqd/321:20:013
117691430,1ptp4l0-21swapper/322:43:023
117691430,1ptp4l0-21swapper/322:43:013
117691430,1ptp4l0-21swapper/300:27:213
117691430,1ptp4l0-21swapper/200:24:292
117691430,1ptp4l0-21swapper/200:06:002
117691430,1ptp4l0-21swapper/020:16:440
1540799428,5cyclictest41-21ksoftirqd/321:40:353
117691420,5ptp4l7778-21memory20:00:253
117691420,5ptp4l7778-21memory20:00:243
117691420,4ptp4l33-21ksoftirqd/221:09:232
117691420,4ptp4l15403-21cyclictest20:28:593
117691420,3ptp4l29397-21apt-get19:40:052
117691420,3ptp4l0-21swapper/219:30:212
117691420,2ptp4l0-21swapper/220:48:212
117691420,2ptp4l0-21swapper/022:39:590
117691420,2ptp4l0-21swapper/022:39:580
117691420,1ptp4l0-21swapper/323:51:083
117691420,1ptp4l0-21swapper/322:09:263
117691420,1ptp4l0-21swapper/221:12:092
1540799415,4cyclictest121rcu_preempt21:00:003
117691410,2ptp4l0-21swapper/323:10:193
117691410,2ptp4l0-21swapper/223:29:122
117691410,2ptp4l0-21swapper/222:56:352
117691410,2ptp4l0-21swapper/221:41:422
117691410,2ptp4l0-21swapper/221:31:252
15407994024,12cyclictest41-21ksoftirqd/320:50:293
15407994011,5cyclictest41-21ksoftirqd/320:30:173
117691400,5ptp4l9619-21fschecks_count20:05:163
117691400,5ptp4l9619-21fschecks_count20:05:163
117691400,5ptp4l7941-21ntp_states20:00:262
117691400,5ptp4l7941-21ntp_states20:00:262
117691400,4ptp4l2203-21apt-get19:50:132
117691400,4ptp4l15964-21apt-config20:20:002
117691400,2ptp4l0-21swapper/221:25:112
117691400,2ptp4l0-21swapper/221:25:112
117691400,2ptp4l0-21swapper/219:25:272
117691400,2ptp4l0-21swapper/020:25:320
117691390,2ptp4l0-21swapper/300:35:503
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional