You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-05-31 - 02:48

x86 Intel Xeon E3-1220 @3100 MHz, Linux 4.19.1-rt3 (Profile)

Latency plot of system in rack #2, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack2slot0.osadl.org (updated Wed May 31, 2023 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3872550,0sleep30-21swapper/323:47:113
71112540,0sleep0111rcuc/023:16:010
297172540,0sleep00-21swapper/022:32:590
305752530,0sleep20-21swapper/222:00:332
194762510,0sleep10-21swapper/123:51:581
280622500,0sleep20-21swapper/221:27:332
249502480,0sleep10-21swapper/122:40:011
229532470,0sleep00-21swapper/021:34:320
248932440,0sleep20-21swapper/223:45:182
21522380,1sleep2619-21systemd-logind00:28:282
456823626,6sleep10-21swapper/119:08:461
287623514,6sleep20-21swapper/219:05:102
434523111,6sleep30-21swapper/319:06:033
448822918,7sleep00-21swapper/019:07:460
4769992120,0cyclictest19176-21lspci00:08:180
4769992120,0cyclictest19176-21lspci00:08:180
4769991716,0cyclictest21443-21kworker/0:321:56:570
305292170,0sleep30-21swapper/321:20:123
249672170,0sleep10-21swapper/100:25:501
50582160,0sleep05061-21bash23:07:280
4769991616,0cyclictest24004-21kworker/0:200:25:510
4769991615,0cyclictest21443-21kworker/0:321:53:290
80682150,0sleep20-21swapper/219:15:232
4769991514,0cyclictest4203-21kworker/0:223:45:590
4769991514,0cyclictest3984-21kworker/0:022:40:270
4769991514,0cyclictest18536-21kworker/0:022:19:190
176942150,0sleep10-21swapper/122:13:451
96772140,0sleep10-21swapper/100:38:281
4769991413,0cyclictest32127-21lspci23:14:260
4769991413,0cyclictest25145-21kworker/0:023:21:530
4769991413,0cyclictest14317-21lspci23:26:000
4769991412,1cyclictest3245-21kworker/0:000:36:350
113652140,0sleep30-21swapper/323:25:253
99112130,0sleep30-21swapper/321:47:213
4922130,0sleep10-21swapper/121:28:521
4769991313,0cyclictest23860-21kworker/0:300:15:370
4784991210,1cyclictest621-21dbus-daemon22:13:503
4779991210,1cyclictest26264-21lspci22:24:042
477999120,1cyclictest11646-21sshd21:55:542
477499120,11cyclictest0-21swapper/100:04:461
4769991212,0cyclictest23860-21kworker/0:323:56:430
4769991211,0cyclictest26324-21lspci21:10:470
4769991211,0cyclictest101ktimersoftd/000:10:090
38462120,0sleep2321ktimersoftd/221:37:422
215142120,0sleep20-21swapper/222:47:062
172722120,2sleep0476999cyclictest20:55:310
478499110,1cyclictest0-21swapper/323:51:503
478499110,10cyclictest0-21swapper/323:20:503
478499110,10cyclictest0-21swapper/322:22:393
477999110,10cyclictest0-21swapper/223:39:042
477999110,10cyclictest0-21swapper/223:30:542
477999110,10cyclictest0-21swapper/222:41:342
477999110,10cyclictest0-21swapper/219:45:232
477999110,10cyclictest0-21swapper/200:05:542
477999110,10cyclictest0-21swapper/200:05:542
477999110,0cyclictest0-21swapper/223:50:442
477999110,0cyclictest0-21swapper/222:51:542
477499110,10cyclictest0-21swapper/122:33:071
477499110,10cyclictest0-21swapper/122:04:571
477499110,10cyclictest0-21swapper/119:30:361
477499110,10cyclictest0-21swapper/100:12:571
4769991110,0cyclictest13343-21kworker/0:021:39:260
476999110,1cyclictest0-21swapper/022:57:590
476999110,10cyclictest0-21swapper/020:10:280
476999110,0cyclictest0-21swapper/000:24:190
155172110,0sleep10-21swapper/123:59:191
478499109,0cyclictest0-21swapper/323:42:103
478499109,0cyclictest0-21swapper/323:36:003
478499109,0cyclictest0-21swapper/323:08:003
478499109,0cyclictest0-21swapper/322:50:503
478499109,0cyclictest0-21swapper/322:35:503
478499109,0cyclictest0-21swapper/322:17:403
478499109,0cyclictest0-21swapper/321:27:403
478499109,0cyclictest0-21swapper/321:17:203
478499109,0cyclictest0-21swapper/300:10:003
478499109,0cyclictest0-21swapper/300:09:593
478499109,0cyclictest0-21swapper/300:04:503
478499108,1cyclictest17486-21sshd21:41:003
478499100,0cyclictest0-21swapper/322:42:303
478499100,0cyclictest0-21swapper/322:27:403
477999109,0cyclictest0-21swapper/223:25:342
477999109,0cyclictest0-21swapper/223:06:342
477999109,0cyclictest0-21swapper/223:02:542
477999109,0cyclictest0-21swapper/222:33:442
477999109,0cyclictest0-21swapper/221:40:542
477999109,0cyclictest0-21swapper/221:21:532
477999109,0cyclictest0-21swapper/200:20:142
477999100,0cyclictest0-21swapper/222:29:042
477999100,0cyclictest0-21swapper/219:54:342
477999100,0cyclictest0-21swapper/200:14:042
477499109,0cyclictest0-21swapper/123:40:271
477499109,0cyclictest0-21swapper/122:19:371
477499100,0cyclictest0-21swapper/123:28:371
477499100,0cyclictest0-21swapper/122:57:471
477499100,0cyclictest0-21swapper/122:29:371
477499100,0cyclictest0-21swapper/121:32:171
476999109,0cyclictest0-21swapper/023:34:090
476999109,0cyclictest0-21swapper/022:05:290
476999109,0cyclictest0-21swapper/022:04:180
476999106,3cyclictest13343-21kworker/0:021:22:510
4769991010,0cyclictest31772-21kworker/0:220:40:210
476999100,0cyclictest0-21swapper/022:11:190
476999100,0cyclictest0-21swapper/021:28:580
47849998,0cyclictest0-21swapper/321:54:503
47849998,0cyclictest0-21swapper/300:30:203
47849998,0cyclictest0-21swapper/300:16:003
47799998,0cyclictest0-21swapper/221:54:442
47799998,0cyclictest0-21swapper/221:48:332
47799998,0cyclictest0-21swapper/220:46:432
47799998,0cyclictest0-21swapper/219:35:042
47749998,0cyclictest0-21swapper/123:48:171
47749998,0cyclictest0-21swapper/123:36:461
47749998,0cyclictest0-21swapper/121:54:071
47749998,0cyclictest0-21swapper/121:44:071
47699998,0cyclictest0-21swapper/023:54:390
47699998,0cyclictest0-21swapper/022:53:290
47699998,0cyclictest0-21swapper/021:05:190
47849987,0cyclictest0-21swapper/322:55:203
47849987,0cyclictest0-21swapper/300:35:403
47849986,1cyclictest21631-21bash23:20:003
47849985,2cyclictest617-21polkitd00:12:093
47849985,2cyclictest606-21avahi-daemon21:34:133
47849984,3cyclictest617-21polkitd22:46:293
47849983,4cyclictest628-21gdbus23:55:093
47799987,0cyclictest0-21swapper/221:16:042
47799985,2cyclictest606-21avahi-daemon22:37:102
47799984,3cyclictest628-21gdbus22:06:162
47749987,0cyclictest0-21swapper/123:13:471
47749987,0cyclictest0-21swapper/122:53:171
47749986,1cyclictest606-21avahi-daemon23:24:141
47749985,2cyclictest606-21avahi-daemon23:07:001
47749985,2cyclictest606-21avahi-daemon21:15:531
47699988,0cyclictest4666-21kworker/0:219:12:060
47699988,0cyclictest11405-21kworker/0:019:29:470
47699987,0cyclictest27485-21lspci21:19:270
47699987,0cyclictest21454-21lspci23:03:300
47699987,0cyclictest0-21swapper/022:46:590
47699987,0cyclictest0-21swapper/022:25:080
47699987,0cyclictest0-21swapper/022:20:090
47699987,0cyclictest0-21swapper/020:45:180
47699986,1cyclictest9-21ksoftirqd/000:31:530
47699980,4cyclictest3790-21systemd-cgroups23:40:000
47849976,0cyclictest0-21swapper/323:31:303
47849976,0cyclictest0-21swapper/323:02:403
47849976,0cyclictest0-21swapper/321:10:403
47849976,0cyclictest0-21swapper/300:23:103
47849975,1cyclictest617-21polkitd22:02:263
47849975,1cyclictest606-21avahi-daemon22:30:143
47849974,2cyclictest606-21avahi-daemon21:36:523
47799976,0cyclictest0-21swapper/223:18:342
47799976,0cyclictest0-21swapper/222:55:342
47799976,0cyclictest0-21swapper/219:20:432
47799975,1cyclictest606-21avahi-daemon23:11:352
47799975,1cyclictest606-21avahi-daemon22:13:302
47799974,2cyclictest628-21gdbus19:59:592
47799973,3cyclictest11495-21gdbus00:33:172
47799972,2cyclictest654-21gdbus23:57:322
47799972,2cyclictest628-21gdbus22:15:412
47799972,1cyclictest0-21swapper/200:18:022
47799972,1cyclictest0-21swapper/200:04:542
47749976,0cyclictest0-21swapper/121:47:561
47749976,0cyclictest0-21swapper/100:19:571
47749975,1cyclictest617-21polkitd23:01:131
47749975,1cyclictest617-21polkitd22:47:541
47749975,1cyclictest606-21avahi-daemon22:23:101
47749974,2cyclictest606-21avahi-daemon22:09:051
47749974,2cyclictest606-21avahi-daemon21:22:581
47749974,2cyclictest606-21avahi-daemon21:13:281
47749974,2cyclictest606-21avahi-daemon00:06:301
47749974,2cyclictest606-21avahi-daemon00:06:301
47699976,0cyclictest31772-21kworker/0:220:20:290
47699976,0cyclictest29343-21lspci21:44:140
47699976,0cyclictest0-21swapper/021:47:190
47699974,2cyclictest606-21avahi-daemon00:01:250
47699972,1cyclictest17634-21sshd22:38:030
47849964,1cyclictest6232-21mii-tool20:30:213
47849964,1cyclictest606-21avahi-daemon22:05:113
47849964,1cyclictest606-21avahi-daemon00:25:333
47849963,2cyclictest621-21dbus-daemon19:15:003
47849963,2cyclictest619-21systemd-logind21:56:153
47849961,2cyclictest621-21dbus-daemon23:10:343
47849960,5cyclictest0-21swapper/319:45:503
47799965,0cyclictest0-21swapper/220:44:242
47799964,1cyclictest617-21polkitd21:30:072
47799964,1cyclictest606-21avahi-daemon20:35:002
47799964,1cyclictest606-21avahi-daemon00:35:042
47799964,1cyclictest4175-21sshd23:40:032
47799963,2cyclictest606-21avahi-daemon23:20:012
47799962,1cyclictest0-21swapper/221:10:162
47799962,1cyclictest0-21swapper/220:54:332
47799962,1cyclictest0-21swapper/220:00:322
47799960,5cyclictest0-21swapper/219:40:542
47799960,1cyclictest0-21swapper/221:03:542
47749965,0cyclictest0-21swapper/121:06:471
47749965,0cyclictest0-21swapper/119:41:271
47749964,1cyclictest9885-21sshd00:22:091
47749964,1cyclictest6840-21sshd23:15:571
47749964,1cyclictest617-21polkitd21:56:021
47749964,1cyclictest617-21polkitd21:35:061
47749964,1cyclictest606-21avahi-daemon23:30:001
47749964,1cyclictest5100-21sshd22:35:011
47749964,1cyclictest3476-21cut20:25:001
47749963,2cyclictest969-21snmpd20:26:211
47749963,2cyclictest969-21snmpd20:02:231
47749963,2cyclictest606-21avahi-daemon00:30:111
47749960,5cyclictest0-21swapper/120:52:371
47699966,0cyclictest27702-21kworker/0:120:05:410
47699966,0cyclictest25613-21kworker/0:020:02:330
47699966,0cyclictest17708-21kworker/0:121:01:380
47699966,0cyclictest15439-21kworker/0:119:48:300
47699965,0cyclictest9-21ksoftirqd/020:40:000
47699963,2cyclictest606-21avahi-daemon23:40:000
47699960,5cyclictest0-21swapper/020:18:490
9204250,0sleep20-21swapper/220:35:322
47849954,0cyclictest0-21swapper/321:01:103
47849953,1cyclictest969-21snmpd21:05:163
47849953,1cyclictest969-21snmpd20:59:243
47849953,1cyclictest969-21snmpd20:44:433
47849953,1cyclictest969-21snmpd20:26:123
47849953,1cyclictest969-21snmpd20:16:033
47849953,1cyclictest969-21snmpd20:11:483
47849953,1cyclictest969-21snmpd20:05:473
47849953,1cyclictest969-21snmpd20:02:533
47849953,1cyclictest969-21snmpd19:36:473
47849953,1cyclictest969-21snmpd19:32:013
47849953,1cyclictest8282-21cat20:35:223
47849953,1cyclictest24333-21cat19:55:253
47849953,1cyclictest2144-21cat20:20:273
47849953,1cyclictest18231-21cat19:40:243
47849953,1cyclictest11960-21cpuspeed_turbos20:45:163
47849952,2cyclictest969-21snmpd20:51:223
47849952,2cyclictest10863-21sed19:20:353
47849950,0cyclictest0-21swapper/319:15:093
47799954,0cyclictest0-21swapper/221:05:042
47799954,0cyclictest0-21swapper/220:21:432
47799953,1cyclictest969-21snmpd20:11:122
47799953,1cyclictest969-21snmpd20:05:302
47799953,1cyclictest969-21snmpd19:25:252
47799953,1cyclictest969-21snmpd19:14:192
47799953,1cyclictest606-21avahi-daemon19:31:112
47799953,1cyclictest32485-21cat20:15:232
47799952,2cyclictest621-21dbus-daemon20:54:592
47799952,1cyclictest5237-21sshd20:26:432
47749954,0cyclictest0-21swapper/120:32:371
47749953,1cyclictest969-21snmpd21:01:341
47749953,1cyclictest969-21snmpd20:56:401
47749953,1cyclictest969-21snmpd20:18:121
47749953,1cyclictest969-21snmpd20:13:021
47749953,1cyclictest969-21snmpd20:07:301
47749953,1cyclictest969-21snmpd19:36:531
47749953,1cyclictest969-21snmpd19:26:431
47749953,1cyclictest969-21snmpd19:20:061
47749953,1cyclictest8667-21awk20:35:251
47749953,1cyclictest8049-21cat19:15:231
47749953,1cyclictest24299-21cat19:55:251
47749952,2cyclictest621-21dbus-daemon19:49:541
47749952,2cyclictest10159-21tail20:40:221
47749950,4cyclictest0-21swapper/119:52:171
47699955,0cyclictest31772-21kworker/0:220:25:050
47699955,0cyclictest21542-21kworker/0:219:57:310
47699955,0cyclictest21542-21kworker/0:219:50:350
47699955,0cyclictest11605-21kworker/0:020:52:290
47699955,0cyclictest11405-21kworker/0:019:34:490
47699953,1cyclictest969-21snmpd20:31:000
47699953,1cyclictest969-21snmpd19:41:320
47699953,1cyclictest969-21snmpd19:21:210
47699952,2cyclictest7677-21cut19:15:200
47849942,1cyclictest22339-21uname19:50:223
47849940,1cyclictest0-21swapper/319:25:343
47749942,1cyclictest969-21snmpd20:45:181
47749942,1cyclictest5096-21cpuspeed_turbos19:10:191
47699942,1cyclictest15966-21/usr/sbin/munin19:35:230
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional