You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-05-29 - 13:21

x86 Intel Xeon E3-1220 @3100 MHz, Linux 4.19.1-rt3 (Profile)

Latency plot of system in rack #2, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rack2slot0.osadl.org (updated Sun May 29, 2022 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2016626648,6sleep00-21swapper/019:05:200
243892530,0sleep20-21swapper/223:56:262
63882520,0sleep20-21swapper/221:58:312
227752510,0sleep30-21swapper/323:07:213
253142490,0sleep30-21swapper/300:05:013
12892440,0sleep2311rcuc/222:37:402
2090823525,6sleep10-21swapper/119:09:371
278292310,0sleep327830-21bash21:39:433
2062322614,7sleep30-21swapper/319:06:073
2069122311,7sleep20-21swapper/219:06:582
118932170,4sleep00-21swapper/000:25:350
233682160,1sleep10-21swapper/122:59:341
149572160,0sleep00-21swapper/021:12:040
57082150,0sleep00-21swapper/023:35:250
23302140,0sleep30-21swapper/323:34:563
2105399130,11cyclictest5443-21sshd21:17:483
21051991313,0cyclictest7528-21sensors00:00:181
21051991312,0cyclictest5426-21lspci21:25:431
21051991312,0cyclictest3843-21lspci23:26:571
200242130,0sleep10-21swapper/100:11:311
2105399120,11cyclictest18550-21id00:19:293
21051991211,0cyclictest32491-21lspci21:40:311
21051991211,0cyclictest11371-21lspci21:11:051
2105199120,1cyclictest0-21swapper/122:13:171
2105099120,1cyclictest14556-21sshd23:29:530
2105099120,11cyclictest27937-21sshd23:41:040
2105099120,11cyclictest0-21swapper/021:34:330
2105399110,10cyclictest0-21swapper/323:58:083
2105399110,10cyclictest0-21swapper/323:29:393
2105299119,1cyclictest11148-21bash00:25:232
2105299110,10cyclictest0-21swapper/221:16:532
2105299110,10cyclictest0-21swapper/200:09:042
2105199110,1cyclictest19517-21lspci22:17:581
2105199110,10cyclictest0-21swapper/123:32:571
2105199110,10cyclictest0-21swapper/123:17:381
2105199110,10cyclictest0-21swapper/122:33:571
2105199110,10cyclictest0-21swapper/119:20:171
2105099119,1cyclictest7234-21sshd22:47:130
2105099119,1cyclictest17843-21bash23:22:230
2105099110,10cyclictest0-21swapper/023:50:030
2105099110,10cyclictest0-21swapper/022:05:330
2105099110,10cyclictest0-21swapper/021:51:240
2105099110,10cyclictest0-21swapper/020:40:230
2105099110,10cyclictest0-21swapper/020:15:130
2105099110,10cyclictest0-21swapper/000:13:030
2105399109,0cyclictest0-21swapper/323:01:093
2105399109,0cyclictest0-21swapper/322:58:483
2105399100,9cyclictest0-21swapper/322:45:083
2105399100,9cyclictest0-21swapper/322:20:383
2105399100,0cyclictest0-21swapper/323:14:083
2105399100,0cyclictest0-21swapper/321:13:593
2105299109,0cyclictest0-21swapper/223:44:342
2105299109,0cyclictest0-21swapper/222:34:232
2105299109,0cyclictest0-21swapper/222:26:342
2105299109,0cyclictest0-21swapper/222:10:032
2105299109,0cyclictest0-21swapper/221:45:432
2105299109,0cyclictest0-21swapper/200:33:542
2105299109,0cyclictest0-21swapper/200:10:432
2105299108,1cyclictest5774-21bash21:42:032
2105299100,0cyclictest19147-21sshd22:01:342
2105199109,0cyclictest8088-21lspci22:06:551
2105199109,0cyclictest0-21swapper/123:42:281
2105199109,0cyclictest0-21swapper/122:28:081
2105199109,0cyclictest0-21swapper/100:22:171
2105199108,1cyclictest614-21dbus-daemon00:34:081
2105199108,1cyclictest3649-21sshd22:54:381
2105199100,9cyclictest0-21swapper/100:26:571
2105199100,0cyclictest0-21swapper/123:46:571
2105199100,0cyclictest0-21swapper/121:54:381
2105199100,0cyclictest0-21swapper/121:37:371
2105199100,0cyclictest0-21swapper/121:31:071
2105099109,0cyclictest0-21swapper/023:06:040
2105099109,0cyclictest0-21swapper/022:56:530
2105099109,0cyclictest0-21swapper/021:16:540
2105099109,0cyclictest0-21swapper/020:55:130
210539998,0cyclictest0-21swapper/323:49:483
210539998,0cyclictest0-21swapper/322:51:493
210539998,0cyclictest0-21swapper/321:51:083
210539998,0cyclictest0-21swapper/321:46:583
210539998,0cyclictest0-21swapper/321:30:183
210539998,0cyclictest0-21swapper/300:35:583
210539998,0cyclictest0-21swapper/300:28:493
210539998,0cyclictest0-21swapper/300:21:083
210539996,2cyclictest27036-21sshd23:40:483
210529998,0cyclictest0-21swapper/222:19:232
210529998,0cyclictest0-21swapper/221:21:532
210519998,0cyclictest0-21swapper/123:37:471
210519998,0cyclictest0-21swapper/122:03:371
210519998,0cyclictest0-21swapper/119:39:571
210519998,0cyclictest0-21swapper/100:05:581
210519990,0cyclictest0-21swapper/119:59:271
210509998,0cyclictest0-21swapper/023:55:040
210509998,0cyclictest0-21swapper/022:33:030
210509998,0cyclictest0-21swapper/021:56:430
210509998,0cyclictest0-21swapper/019:56:330
210509998,0cyclictest0-21swapper/000:31:440
210509990,0cyclictest0-21swapper/021:38:230
210539985,2cyclictest598-21avahi-daemon23:35:523
210539985,2cyclictest598-21avahi-daemon22:27:473
210539985,2cyclictest598-21avahi-daemon21:20:173
210539985,2cyclictest598-21avahi-daemon00:11:283
210539985,2cyclictest598-21avahi-daemon00:07:443
210529987,0cyclictest0-21swapper/222:54:442
210529987,0cyclictest0-21swapper/221:26:442
210529987,0cyclictest0-21swapper/200:00:532
210529986,1cyclictest13991-21bash23:05:132
210529985,2cyclictest598-21avahi-daemon23:32:122
210529985,2cyclictest598-21avahi-daemon23:15:032
210529985,2cyclictest598-21avahi-daemon23:12:022
210529985,2cyclictest598-21avahi-daemon23:02:542
210529985,2cyclictest598-21avahi-daemon22:58:362
210529985,2cyclictest598-21avahi-daemon22:44:292
210529985,2cyclictest598-21avahi-daemon22:23:182
210529985,2cyclictest598-21avahi-daemon00:17:402
210519987,0cyclictest0-21swapper/123:51:081
210519987,0cyclictest0-21swapper/123:07:181
210519985,2cyclictest614-21dbus-daemon00:35:091
210519985,2cyclictest598-21avahi-daemon23:22:491
210519985,2cyclictest598-21avahi-daemon22:49:351
210509987,0cyclictest0-21swapper/023:47:530
210509987,0cyclictest0-21swapper/000:21:140
210509987,0cyclictest0-21swapper/000:06:340
210509985,2cyclictest598-21avahi-daemon23:13:370
210509985,2cyclictest598-21avahi-daemon22:43:400
210509985,2cyclictest598-21avahi-daemon21:48:470
210509985,2cyclictest598-21avahi-daemon21:25:360
210539976,0cyclictest0-21swapper/322:43:283
210539976,0cyclictest0-21swapper/321:56:283
210539975,1cyclictest633-21polkitd21:41:393
210539975,1cyclictest614-21dbus-daemon23:16:573
210539975,1cyclictest598-21avahi-daemon23:51:093
210539975,1cyclictest598-21avahi-daemon22:30:023
210539975,1cyclictest598-21avahi-daemon21:26:163
210539975,1cyclictest0-21swapper/319:50:183
210539974,2cyclictest970-21snmpd20:31:163
210539974,2cyclictest614-21dbus-daemon22:37:013
210539974,2cyclictest614-21dbus-daemon22:08:363
210539974,2cyclictest598-21avahi-daemon23:21:383
210539974,2cyclictest598-21avahi-daemon22:15:093
210539974,2cyclictest598-21avahi-daemon22:11:373
210539974,2cyclictest598-21avahi-daemon22:01:123
210539974,2cyclictest598-21avahi-daemon00:31:573
210529976,0cyclictest0-21swapper/219:40:132
210529975,1cyclictest598-21avahi-daemon23:45:062
210529975,1cyclictest598-21avahi-daemon23:36:492
210529975,1cyclictest598-21avahi-daemon23:25:362
210529975,1cyclictest598-21avahi-daemon23:20:032
210529975,1cyclictest598-21avahi-daemon22:45:442
210529975,1cyclictest598-21avahi-daemon22:08:482
210529975,1cyclictest598-21avahi-daemon21:50:482
210529975,1cyclictest598-21avahi-daemon21:35:162
210529975,1cyclictest598-21avahi-daemon21:31:012
210529975,1cyclictest598-21avahi-daemon00:36:412
210529974,2cyclictest614-21dbus-daemon20:10:002
210529974,2cyclictest598-21avahi-daemon23:50:152
210529973,3cyclictest598-21avahi-daemon00:21:552
210529973,1cyclictest0-21swapper/220:51:272
210529972,3cyclictest19821-21sshd21:13:222
210519976,0cyclictest0-21swapper/120:34:071
210519975,1cyclictest598-21avahi-daemon23:56:531
210519975,1cyclictest598-21avahi-daemon23:13:201
210519975,1cyclictest598-21avahi-daemon23:01:531
210519975,1cyclictest598-21avahi-daemon22:40:071
210519975,1cyclictest598-21avahi-daemon22:20:401
210519975,1cyclictest598-21avahi-daemon21:55:571
210519975,1cyclictest598-21avahi-daemon00:15:191
210519974,2cyclictest598-21avahi-daemon22:36:231
210519974,2cyclictest598-21avahi-daemon21:47:191
210519971,2cyclictest25-21ksoftirqd/121:17:131
210519970,1cyclictest121rcu_preempt21:10:001
210509976,0cyclictest0-21swapper/020:34:330
210509975,1cyclictest598-21avahi-daemon23:30:520
210509975,1cyclictest598-21avahi-daemon23:01:530
210509975,1cyclictest598-21avahi-daemon22:36:050
210509975,1cyclictest598-21avahi-daemon22:25:090
210509975,1cyclictest598-21avahi-daemon22:21:520
210509975,1cyclictest598-21avahi-daemon22:10:390
210509975,1cyclictest598-21avahi-daemon00:39:350
210509975,1cyclictest598-21avahi-daemon00:15:560
210509975,1cyclictest598-21avahi-daemon00:01:140
210509974,2cyclictest614-21dbus-daemon23:15:280
210509974,2cyclictest598-21avahi-daemon22:50:110
210509974,2cyclictest598-21avahi-daemon22:17:280
30667260,0sleep00-21swapper/020:50:140
210539963,2cyclictest970-21snmpd20:45:053
210539963,2cyclictest970-21snmpd19:26:413
210539963,2cyclictest970-21snmpd19:17:113
210539963,2cyclictest32468-21diskstats20:55:143
210539960,5cyclictest0-21swapper/319:34:283
210529964,1cyclictest970-21snmpd19:18:082
210529964,1cyclictest633-21polkitd20:00:002
210529964,1cyclictest614-21dbus-daemon21:10:002
210529964,1cyclictest598-21avahi-daemon20:49:552
210529963,2cyclictest970-21snmpd20:17:132
210529962,1cyclictest0-21swapper/219:10:272
210519964,1cyclictest633-21polkitd21:20:591
210519964,1cyclictest25-21ksoftirqd/119:50:001
210519964,1cyclictest25-21ksoftirqd/119:30:001
210519963,2cyclictest970-21snmpd20:13:011
210519963,2cyclictest970-21snmpd19:54:481
210519963,2cyclictest970-21snmpd19:30:321
210519960,5cyclictest0-21swapper/120:24:371
210519960,3cyclictest25-21ksoftirqd/120:40:001
210509965,0cyclictest0-21swapper/019:31:330
210509964,1cyclictest614-21dbus-daemon21:20:060
210509964,1cyclictest614-21dbus-daemon19:30:000
210509964,1cyclictest31357-21sshd21:40:190
210509963,2cyclictest614-21dbus-daemon22:00:430
210509960,5cyclictest0-21swapper/021:08:030
210509960,5cyclictest0-21swapper/020:39:130
210539954,0cyclictest0-21swapper/320:40:283
210539954,0cyclictest0-21swapper/320:28:283
210539953,1cyclictest970-21snmpd21:08:533
210539953,1cyclictest970-21snmpd20:22:533
210539953,1cyclictest970-21snmpd20:00:063
210539953,1cyclictest970-21snmpd19:55:133
210539953,1cyclictest970-21snmpd19:20:023
210539953,1cyclictest410-21open_inodes19:35:173
210539953,1cyclictest30616-21cat20:50:133
210539953,1cyclictest25233-21timerandwakeup20:35:213
210539953,1cyclictest16591-21needreboot20:15:153
210539953,1cyclictest14326-21cat20:10:143
210539953,1cyclictest0-21swapper/321:00:173
210539952,2cyclictest970-21snmpd19:42:263
210539952,2cyclictest4051-21gltestperf19:45:163
210539952,2cyclictest13619-21chrt20:09:563
210529953,1cyclictest970-21snmpd20:55:052
210529953,1cyclictest970-21snmpd20:27:312
210529953,1cyclictest970-21snmpd19:56:002
210529953,1cyclictest970-21snmpd19:51:572
210529953,1cyclictest970-21snmpd19:45:132
210529953,1cyclictest970-21snmpd19:27:532
210529953,1cyclictest598-21avahi-daemon20:20:012
210529953,1cyclictest32455-21cat19:35:142
210529953,1cyclictest29934-21/usr/sbin/munin19:30:212
210529953,1cyclictest26349-21cat19:20:142
210529953,1cyclictest14363-21cat20:10:152
210529952,2cyclictest970-21snmpd20:40:012
210529952,2cyclictest22355-21grep20:30:122
210529950,2cyclictest1881-21/usr/sbin/munin21:00:162
210529950,1cyclictest24053-21/usr/sbin/munin20:35:142
210519954,0cyclictest0-21swapper/120:29:571
210519953,1cyclictest970-21snmpd20:56:201
210519953,1cyclictest970-21snmpd20:17:041
210519953,1cyclictest970-21snmpd20:05:481
210519953,1cyclictest970-21snmpd19:16:411
210519953,1cyclictest30650-21cat20:50:131
210519953,1cyclictest21724-21cat19:10:131
210519953,1cyclictest1647-21/usr/sbin/munin19:40:181
210519953,1cyclictest10659-21ntp_states20:00:171
210519952,2cyclictest2315-21cat21:00:161
210519950,0cyclictest0-21swapper/120:43:171
210509953,1cyclictest970-21snmpd21:00:230
210509953,1cyclictest970-21snmpd20:45:070
210509953,1cyclictest970-21snmpd20:21:010
210509953,1cyclictest970-21snmpd20:10:300
210509953,1cyclictest970-21snmpd20:05:420
210509953,1cyclictest970-21snmpd19:50:330
210509953,1cyclictest614-21dbus-daemon20:25:020
210509953,1cyclictest598-21avahi-daemon19:24:020
210509953,1cyclictest31714-21kworker/0:019:36:280
210509953,1cyclictest24859-21sendmail_mailtr19:15:190
210509952,2cyclictest970-21snmpd20:00:400
210509952,2cyclictest970-21snmpd19:12:120
210509950,0cyclictest0-21swapper/019:45:130
210539942,1cyclictest21544-21tail19:10:123
210519942,1cyclictest0-21swapper/120:45:131
210509942,1cyclictest0-21swapper/019:40:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional