You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-12-05 - 03:25

x86 Intel Xeon E3-1220 @3100 MHz, Linux 4.19.1-rt3 (Profile)

Latency plot of system in rack #2, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack2slot0.osadl.org (updated Mon Dec 05, 2022 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
191202570,0sleep00-21swapper/021:17:230
55252540,0sleep30-21swapper/321:46:223
321742540,0sleep10-21swapper/121:53:121
28242520,0sleep00-21swapper/021:21:200
104302520,0sleep30-21swapper/321:23:233
206182510,0sleep20-21swapper/222:55:222
79882490,0sleep3391rcuc/323:57:223
357323827,7sleep10-21swapper/119:06:291
373323312,7sleep00-21swapper/019:08:280
36442319,7sleep30-21swapper/319:07:233
362622816,7sleep20-21swapper/219:07:102
5912170,2sleep0396799cyclictest22:25:440
246232170,0sleep00-21swapper/022:56:160
91172150,0sleep20-21swapper/223:00:262
309472150,0sleep30-21swapper/321:44:543
256642150,0sleep30-21swapper/300:01:473
119702150,0sleep20-21swapper/222:36:472
166852140,0sleep10-21swapper/121:49:251
154772140,0sleep10-21swapper/121:16:241
397799130,11cyclictest17907-21sshd23:35:253
3973991312,0cyclictest23171-21lspci00:09:311
397799120,1cyclictest0-21swapper/323:13:243
397699120,1cyclictest0-21swapper/223:46:202
3973991211,0cyclictest33150irq/27-ahci[00022:05:111
3973991211,0cyclictest27666-21lspci23:54:211
3973991210,1cyclictest10146-21id23:33:371
396799120,1cyclictest0-21swapper/023:13:250
397799119,1cyclictest0-21swapper/323:29:153
3977991110,0cyclictest0-21swapper/323:45:453
397799110,10cyclictest27332-21lspci00:26:343
397799110,10cyclictest0-21swapper/323:51:553
397799110,10cyclictest0-21swapper/323:06:553
397799110,10cyclictest0-21swapper/323:06:553
397799110,10cyclictest0-21swapper/321:27:343
3976991110,0cyclictest0-21swapper/221:46:102
3976991110,0cyclictest0-21swapper/221:41:202
397699110,1cyclictest3524-21bash23:40:102
397699110,10cyclictest0-21swapper/223:21:002
397699110,10cyclictest0-21swapper/222:11:202
397699110,10cyclictest0-21swapper/221:55:302
397699110,10cyclictest0-21swapper/200:08:302
3973991110,0cyclictest2403-21lspci22:18:141
397399110,1cyclictest24314-21sshd22:23:471
397399110,1cyclictest17739-21lspci23:27:161
397399110,10cyclictest0-21swapper/121:25:061
396799119,1cyclictest20987-21sshd21:42:100
396799110,10cyclictest0-21swapper/023:32:110
396799110,10cyclictest0-21swapper/023:05:510
396799110,10cyclictest0-21swapper/023:05:500
396799110,10cyclictest0-21swapper/021:14:410
396799110,10cyclictest0-21swapper/000:32:510
264782110,1sleep126479-21bash22:40:221
397799109,0cyclictest0-21swapper/323:15:453
397799109,0cyclictest0-21swapper/321:11:243
397799109,0cyclictest0-21swapper/300:39:453
397799108,1cyclictest0-21swapper/322:25:443
397799100,9cyclictest0-21swapper/322:49:353
397799100,0cyclictest0-21swapper/322:03:543
397799100,0cyclictest0-21swapper/321:32:553
397699109,0cyclictest0-21swapper/223:50:302
397699109,0cyclictest0-21swapper/222:50:192
397699109,0cyclictest0-21swapper/222:24:202
397699109,0cyclictest0-21swapper/221:38:502
397699109,0cyclictest0-21swapper/221:25:392
397699109,0cyclictest0-21swapper/221:12:002
397699109,0cyclictest0-21swapper/200:21:302
397699108,1cyclictest4475-21grep00:04:492
397699100,0cyclictest0-21swapper/223:25:302
397699100,0cyclictest0-21swapper/223:05:302
397699100,0cyclictest0-21swapper/223:05:302
397699100,0cyclictest0-21swapper/200:10:202
397399109,0cyclictest12229-21lspci22:36:511
397399109,0cyclictest0-21swapper/122:25:571
397399109,0cyclictest0-21swapper/100:17:261
397399109,0cyclictest0-21swapper/100:04:271
397399107,2cyclictest614-21dbus-daemon00:25:071
397399100,0cyclictest20032-21sshd23:11:271
397399100,0cyclictest0-21swapper/123:58:171
397399100,0cyclictest0-21swapper/123:45:571
397399100,0cyclictest0-21swapper/123:39:471
397399100,0cyclictest0-21swapper/122:04:071
397399100,0cyclictest0-21swapper/121:40:461
397399100,0cyclictest0-21swapper/121:38:171
396799109,0cyclictest0-21swapper/023:58:510
396799109,0cyclictest0-21swapper/022:33:100
396799109,0cyclictest0-21swapper/000:12:210
396799100,9cyclictest0-21swapper/022:48:310
396799100,0cyclictest10283-21sshd21:39:410
396799100,0cyclictest0-21swapper/023:52:400
396799100,0cyclictest0-21swapper/021:26:410
39779998,0cyclictest0-21swapper/321:38:043
39779998,0cyclictest0-21swapper/319:15:353
39779998,0cyclictest0-21swapper/319:14:143
39779998,0cyclictest0-21swapper/300:32:153
39779997,1cyclictest24087-21lspci21:59:253
39779990,0cyclictest0-21swapper/320:53:353
39769998,0cyclictest0-21swapper/223:56:402
39769998,0cyclictest0-21swapper/223:38:092
39769998,0cyclictest0-21swapper/223:16:302
39769998,0cyclictest0-21swapper/222:30:402
39769998,0cyclictest0-21swapper/219:20:202
39769998,0cyclictest0-21swapper/200:29:002
39769997,1cyclictest14643-21idleruntime-cro19:35:002
39769990,0cyclictest0-21swapper/219:45:292
39739998,0cyclictest0-21swapper/122:45:271
39739998,0cyclictest0-21swapper/121:59:371
39739990,8cyclictest0-21swapper/119:17:461
39739990,0cyclictest0-21swapper/119:55:561
39679998,0cyclictest0-21swapper/022:07:300
39679998,0cyclictest0-21swapper/020:21:010
39679998,0cyclictest0-21swapper/000:38:310
39679997,1cyclictest968-21sshd23:16:510
39779987,0cyclictest0-21swapper/322:32:053
39779987,0cyclictest0-21swapper/321:19:153
39779987,0cyclictest0-21swapper/320:20:153
39779987,0cyclictest0-21swapper/320:04:343
39769987,0cyclictest0-21swapper/223:32:002
39769987,0cyclictest0-21swapper/200:34:402
39769986,1cyclictest0-21swapper/222:48:102
39769985,2cyclictest598-21avahi-daemon22:40:292
39769985,2cyclictest598-21avahi-daemon21:53:442
39769985,2cyclictest598-21avahi-daemon00:36:092
39769985,2cyclictest598-21avahi-daemon00:15:402
39739987,0cyclictest0-21swapper/122:32:171
39739987,0cyclictest0-21swapper/120:35:161
39739987,0cyclictest0-21swapper/100:38:071
39739987,0cyclictest0-21swapper/100:32:271
39739986,1cyclictest598-21avahi-daemon21:30:251
39739985,2cyclictest598-21avahi-daemon23:40:231
39739985,2cyclictest598-21avahi-daemon23:09:491
39739985,2cyclictest598-21avahi-daemon23:09:481
39739985,2cyclictest598-21avahi-daemon22:58:291
39739985,2cyclictest598-21avahi-daemon21:24:561
39739985,2cyclictest598-21avahi-daemon00:11:311
39679987,0cyclictest0-21swapper/023:44:310
39679987,0cyclictest0-21swapper/022:39:410
39679987,0cyclictest0-21swapper/022:03:010
39679987,0cyclictest0-21swapper/000:23:310
39679985,2cyclictest598-21avahi-daemon23:35:330
39679985,2cyclictest598-21avahi-daemon22:54:060
39679985,2cyclictest598-21avahi-daemon22:22:480
39679985,2cyclictest598-21avahi-daemon21:50:220
39779976,0cyclictest0-21swapper/319:54:353
39779976,0cyclictest0-21swapper/300:15:253
39779975,1cyclictest598-21avahi-daemon23:20:283
39779975,1cyclictest598-21avahi-daemon22:55:333
39779975,1cyclictest598-21avahi-daemon22:50:583
39779975,1cyclictest598-21avahi-daemon22:41:563
39779975,1cyclictest598-21avahi-daemon22:11:403
39779975,1cyclictest598-21avahi-daemon22:05:013
39779975,1cyclictest598-21avahi-daemon21:51:013
39779975,1cyclictest598-21avahi-daemon00:05:153
39779974,2cyclictest968-21sshd23:41:413
39779974,2cyclictest614-21dbus-daemon22:20:133
39779974,2cyclictest598-21avahi-daemon23:30:553
39779974,2cyclictest598-21avahi-daemon22:35:123
39779974,2cyclictest598-21avahi-daemon22:16:033
39779973,3cyclictest614-21dbus-daemon23:02:033
39779973,3cyclictest614-21dbus-daemon00:21:403
39779970,6cyclictest0-21swapper/320:45:453
39769975,1cyclictest598-21avahi-daemon22:27:492
39769975,1cyclictest598-21avahi-daemon22:19:042
39769975,1cyclictest598-21avahi-daemon22:06:172
39769975,1cyclictest598-21avahi-daemon22:01:412
39769975,1cyclictest598-21avahi-daemon21:31:082
39769974,2cyclictest598-21avahi-daemon23:11:062
39769974,2cyclictest598-21avahi-daemon21:20:382
39769974,2cyclictest598-21avahi-daemon21:17:082
39769970,6cyclictest0-21swapper/219:58:202
39739976,0cyclictest0-21swapper/123:20:371
39739976,0cyclictest0-21swapper/120:50:151
39739975,1cyclictest598-21avahi-daemon23:15:451
39739975,1cyclictest598-21avahi-daemon23:00:181
39739975,1cyclictest598-21avahi-daemon22:10:211
39739975,1cyclictest598-21avahi-daemon00:22:221
39739974,2cyclictest598-21avahi-daemon22:53:201
39679976,0cyclictest0-21swapper/023:21:210
39679975,1cyclictest598-21avahi-daemon23:47:170
39679975,1cyclictest598-21avahi-daemon23:26:460
39679975,1cyclictest598-21avahi-daemon22:42:220
39679975,1cyclictest598-21avahi-daemon21:57:450
39679975,1cyclictest598-21avahi-daemon21:46:120
39679975,1cyclictest598-21avahi-daemon21:34:220
39679975,1cyclictest598-21avahi-daemon00:25:310
39679975,1cyclictest598-21avahi-daemon00:17:510
39679975,1cyclictest598-21avahi-daemon00:05:070
39679975,1cyclictest14864-21kworker/0:220:55:590
39679974,2cyclictest598-21avahi-daemon22:10:190
39679971,3cyclictest121rcu_preempt23:00:410
32373270,1sleep332430-21cp00:11:373
39779965,0cyclictest0-21swapper/320:29:143
39779965,0cyclictest0-21swapper/319:45:043
39779963,2cyclictest970-21snmpd21:09:083
39779963,2cyclictest970-21snmpd20:33:193
39779963,2cyclictest970-21snmpd20:06:243
39779960,0cyclictest0-21swapper/319:20:443
39769963,2cyclictest970-21snmpd20:06:152
39769960,5cyclictest0-21swapper/221:10:002
39769960,5cyclictest0-21swapper/220:56:402
39769960,5cyclictest0-21swapper/220:33:402
39769960,5cyclictest0-21swapper/220:14:092
39769960,5cyclictest0-21swapper/219:16:292
39769960,0cyclictest0-21swapper/219:26:292
39769960,0cyclictest0-21swapper/219:11:182
39739964,1cyclictest970-21snmpd20:48:101
39739964,1cyclictest614-21dbus-daemon21:00:011
39739964,1cyclictest27587-21mii-tool20:05:191
39739963,2cyclictest970-21snmpd21:00:471
39739963,2cyclictest970-21snmpd20:16:271
39739963,2cyclictest598-21avahi-daemon21:10:031
39739961,2cyclictest131rcu_sched19:35:001
39739960,1cyclictest25-21ksoftirqd/119:39:591
39679965,0cyclictest6959-21kworker/0:120:45:150
39679965,0cyclictest20714-21kworker/0:019:20:030
39679964,1cyclictest18103-21sshd00:00:040
39679963,2cyclictest970-21snmpd19:37:260
39679963,2cyclictest970-21snmpd19:29:340
39679963,2cyclictest614-21dbus-daemon22:15:380
39779953,1cyclictest970-21snmpd20:39:033
39779953,1cyclictest970-21snmpd20:16:583
39779953,1cyclictest970-21snmpd19:29:203
39779953,1cyclictest614-21dbus-daemon19:35:003
39779953,1cyclictest15613-21cat20:55:213
39779953,1cyclictest15343-21cat19:35:173
39779952,2cyclictest970-21snmpd20:40:193
39779952,2cyclictest18605-21taskset21:00:533
39779952,2cyclictest1078-21ntpd20:10:173
39779952,2cyclictest0-21swapper/319:55:153
39769954,0cyclictest0-21swapper/221:02:292
39769954,0cyclictest0-21swapper/219:43:102
39769953,1cyclictest970-21snmpd20:42:372
39769953,1cyclictest970-21snmpd20:39:512
39769953,1cyclictest970-21snmpd20:28:182
39769953,1cyclictest970-21snmpd20:20:312
39769953,1cyclictest970-21snmpd20:02:502
39769953,1cyclictest970-21snmpd19:54:122
39769953,1cyclictest31667-21cat20:15:162
39769953,1cyclictest15258-21gltestperf19:35:172
39769953,1cyclictest13574-21cat20:50:132
39769953,1cyclictest11501-21cat20:45:152
39739953,1cyclictest970-21snmpd21:07:471
39739953,1cyclictest970-21snmpd20:25:031
39739953,1cyclictest970-21snmpd20:20:431
39739953,1cyclictest970-21snmpd20:10:511
39739953,1cyclictest970-21snmpd19:52:351
39739953,1cyclictest970-21snmpd19:40:091
39739953,1cyclictest970-21snmpd19:25:041
39739953,1cyclictest970-21snmpd19:24:051
39739953,1cyclictest970-21snmpd19:11:011
39739953,1cyclictest614-21dbus-daemon20:00:001
39739953,1cyclictest5395-21cat20:30:141
39739953,1cyclictest19414-21cat19:45:191
39739953,1cyclictest1078-21ntpd20:40:181
39679955,0cyclictest6959-21kworker/0:120:40:280
39679955,0cyclictest6959-21kworker/0:120:35:050
39679955,0cyclictest28888-21kworker/0:220:25:380
39679955,0cyclictest20714-21kworker/0:020:00:280
39679955,0cyclictest20714-21kworker/0:019:55:090
39679955,0cyclictest20714-21kworker/0:019:45:090
39679955,0cyclictest20714-21kworker/0:019:40:260
39679955,0cyclictest20714-21kworker/0:019:31:210
39679955,0cyclictest14864-21kworker/0:221:00:140
39679954,1cyclictest28888-21kworker/0:220:10:230
39679954,0cyclictest20714-21kworker/0:019:10:060
39679954,0cyclictest0-21swapper/020:15:000
39679953,1cyclictest970-21snmpd20:50:510
39679953,1cyclictest6302-21gltestperf19:15:000
39679953,1cyclictest5415-21mii-tool20:30:140
39679953,1cyclictest27182-21cpuspeed_turbos20:05:160
39679953,1cyclictest21450-21cat19:50:140
39679952,2cyclictest970-21snmpd21:06:130
39779942,1cyclictest17226-21head19:40:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional