You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-06-01 - 14:29

x86 Intel Xeon E3-1220 @3100 MHz, Linux 4.19.1-rt3 (Profile)

Latency plot of system in rack #2, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack2slot0.osadl.org (updated Thu Jun 01, 2023 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
98652530,0sleep10-21swapper/109:11:021
41022530,1sleep34106-21bash11:17:323
218272520,0sleep10-21swapper/110:33:521
95142500,0sleep00-21swapper/011:02:470
1934624932,7sleep30-21swapper/307:06:123
164702480,0sleep20-21swapper/212:25:242
136632480,0sleep10-21swapper/110:00:001
223192470,0sleep20-21swapper/212:35:092
276772460,0sleep00-21swapper/010:03:190
25942390,0sleep10-21swapper/111:09:131
1930823630,3sleep20-21swapper/207:05:432
1955722716,7sleep10-21swapper/107:08:461
192942255,7sleep00-21swapper/007:05:330
204242160,0sleep10-21swapper/111:37:441
19771991615,0cyclictest8178-21kworker/0:310:51:210
324562150,0sleep20-21swapper/212:05:092
1979199150,1cyclictest0-21swapper/310:37:383
19771991515,0cyclictest7330-21kworker/0:309:28:190
19771991515,0cyclictest6211-21kworker/0:210:16:210
19771991514,0cyclictest28667-21kworker/0:109:52:050
19771991513,1cyclictest0-21swapper/012:05:560
48322140,1sleep14837-21switchtime10:45:301
250602140,0sleep30-21swapper/309:54:493
1978299140,0cyclictest0-21swapper/211:13:082
19771991414,0cyclictest8178-21kworker/0:310:45:490
19771991414,0cyclictest7330-21kworker/0:309:36:290
19771991414,0cyclictest523-40kipmi010:11:470
19771991414,0cyclictest2635-21kworker/0:009:15:410
19771991414,0cyclictest17599-21kworker/0:110:32:170
19771991413,0cyclictest29648-21lspci09:47:500
159072140,0sleep30-21swapper/311:20:243
245362130,0sleep20-21swapper/210:18:312
19771991313,0cyclictest26598-21kworker/0:208:55:230
19771991313,0cyclictest26598-21kworker/0:208:50:010
19771991313,0cyclictest18431-21kworker/0:209:59:210
19771991312,0cyclictest23597-21kworker/0:011:41:180
19771991312,0cyclictest12904-21kworker/0:111:05:230
52532120,0sleep30-21swapper/310:45:323
1978299120,1cyclictest12587-21systemd-cgroups10:39:382
19771991212,0cyclictest8595-21kworker/0:312:31:320
19771991212,0cyclictest26362-21kworker/0:007:25:050
19771991212,0cyclictest19733-21kworker/0:307:10:530
19771991210,1cyclictest0-21swapper/012:02:350
1979199119,1cyclictest0-21swapper/311:34:483
1979199110,1cyclictest2261-21bash10:20:483
1979199110,10cyclictest0-21swapper/311:54:083
1979199110,10cyclictest0-21swapper/309:32:283
1979199110,0cyclictest0-21swapper/310:55:283
1979199110,0cyclictest0-21swapper/310:40:483
1979199110,0cyclictest0-21swapper/310:19:583
1978299110,1cyclictest2760-21lspci11:00:582
1978299110,10cyclictest0-21swapper/210:07:382
1978299110,10cyclictest0-21swapper/209:56:582
1978299110,10cyclictest0-21swapper/209:29:582
1978299110,10cyclictest0-21swapper/207:30:382
1978299110,0cyclictest0-21swapper/210:54:182
1978299110,0cyclictest0-21swapper/210:25:482
1977899119,1cyclictest1369-21sshd11:49:181
1977899110,1cyclictest24486-21sshd10:02:281
1977899110,1cyclictest22317-21lspci10:57:581
1977899110,1cyclictest0-21swapper/110:37:481
1977899110,1cyclictest0-21swapper/110:13:581
1977899110,10cyclictest9799-21sshd09:42:581
1977899110,10cyclictest0-21swapper/110:25:081
1977899110,10cyclictest0-21swapper/109:39:581
1977899110,0cyclictest0-21swapper/111:28:581
1977899110,0cyclictest0-21swapper/111:03:281
19771991111,0cyclictest26598-21kworker/0:209:01:590
19771991111,0cyclictest2635-21kworker/0:009:24:310
19771991111,0cyclictest23597-21kworker/0:011:20:490
19771991111,0cyclictest19733-21kworker/0:307:18:090
19771991111,0cyclictest14417-21kworker/0:208:20:430
19771991111,0cyclictest10408-21kworker/0:212:11:140
1977199110,10cyclictest0-21swapper/011:52:250
212602100,0sleep20-21swapper/211:30:002
1979199109,0cyclictest0-21swapper/311:55:383
1979199109,0cyclictest0-21swapper/310:10:283
1979199109,0cyclictest0-21swapper/309:39:183
1979199108,1cyclictest19598-21irqstats11:05:183
1979199100,0cyclictest0-21swapper/309:55:383
1979199100,0cyclictest0-21swapper/309:20:083
1979199100,0cyclictest0-21swapper/309:15:283
1978299109,0cyclictest0-21swapper/212:03:292
1978299109,0cyclictest0-21swapper/211:58:592
1978299109,0cyclictest0-21swapper/211:41:092
1978299109,0cyclictest0-21swapper/211:15:282
1978299109,0cyclictest0-21swapper/209:40:382
1978299109,0cyclictest0-21swapper/209:33:482
1978299108,1cyclictest20044-21bash09:21:282
1978299107,2cyclictest2400-21kworker/2:109:19:252
1978299100,0cyclictest0-21swapper/212:23:192
1978299100,0cyclictest0-21swapper/209:51:482
1977899109,0cyclictest0-21swapper/112:32:281
1977899109,0cyclictest0-21swapper/111:50:491
1977899109,0cyclictest0-21swapper/111:40:281
1977899109,0cyclictest0-21swapper/111:20:181
1977899109,0cyclictest0-21swapper/110:40:581
1977899100,9cyclictest0-21swapper/109:33:081
1977899100,0cyclictest0-21swapper/112:24:081
1977899100,0cyclictest0-21swapper/112:00:581
1977899100,0cyclictest0-21swapper/109:54:081
1977899100,0cyclictest0-21swapper/109:45:581
1977199109,0cyclictest30469-21kworker/0:207:40:310
1977199109,0cyclictest0-21swapper/009:11:550
1977199101,1cyclictest9-21ksoftirqd/010:55:220
19771991010,0cyclictest8595-21kworker/0:312:36:440
19771991010,0cyclictest8254-21kworker/0:008:13:270
19771991010,0cyclictest8178-21kworker/0:310:44:260
19771991010,0cyclictest26362-21kworker/0:007:31:510
19771991010,0cyclictest2153-21kworker/0:107:57:510
19771991010,0cyclictest19733-21kworker/0:307:21:170
19771991010,0cyclictest14417-21kworker/0:208:29:130
19771991010,0cyclictest14417-21kworker/0:208:15:000
1977199100,9cyclictest0-21swapper/008:32:350
1977199100,9cyclictest0-21swapper/008:07:250
1977199100,0cyclictest0-21swapper/012:18:150
1977199100,0cyclictest0-21swapper/010:26:250
7409290,0sleep20-21swapper/207:55:392
197919998,0cyclictest0-21swapper/309:42:183
197919998,0cyclictest0-21swapper/309:10:483
197919990,0cyclictest0-21swapper/309:28:383
197829998,0cyclictest0-21swapper/211:26:302
197829998,0cyclictest0-21swapper/211:22:092
197829998,0cyclictest0-21swapper/210:55:282
197829998,0cyclictest0-21swapper/210:42:482
197829998,0cyclictest0-21swapper/210:24:082
197829998,0cyclictest0-21swapper/209:46:382
197829998,0cyclictest0-21swapper/209:11:182
197829998,0cyclictest0-21swapper/208:45:182
197789999,0cyclictest0-21swapper/110:06:581
197789998,0cyclictest0-21swapper/110:24:181
197789998,0cyclictest0-21swapper/107:12:171
197719999,0cyclictest728-21kworker/0:311:57:220
197719999,0cyclictest2635-21kworker/0:009:07:010
197719999,0cyclictest2153-21kworker/0:107:46:030
197719999,0cyclictest20726-21kworker/0:008:41:100
197719999,0cyclictest20726-21kworker/0:008:35:590
197719998,0cyclictest0-21swapper/012:26:550
197719998,0cyclictest0-21swapper/011:14:550
197719998,0cyclictest0-21swapper/010:24:450
197719998,0cyclictest0-21swapper/009:30:350
197719998,0cyclictest0-21swapper/008:46:350
197719997,1cyclictest0-21swapper/011:30:150
197719991,1cyclictest131rcu_sched11:35:040
197719990,0cyclictest0-21swapper/007:35:050
18898290,1sleep018901-21nfsd411:45:240
197919987,0cyclictest0-21swapper/312:29:493
197919987,0cyclictest0-21swapper/312:06:083
197919987,0cyclictest0-21swapper/311:28:283
197919987,0cyclictest0-21swapper/310:04:183
197919987,0cyclictest0-21swapper/309:45:183
197829987,0cyclictest0-21swapper/210:12:582
197829987,0cyclictest0-21swapper/208:14:282
197829984,3cyclictest606-21avahi-daemon10:31:392
197789987,0cyclictest0-21swapper/112:09:081
197789987,0cyclictest0-21swapper/108:20:281
197789986,1cyclictest25-21ksoftirqd/108:00:001
197789986,1cyclictest17427-21lspci09:20:481
197789986,1cyclictest0-21swapper/108:15:381
197789985,2cyclictest606-21avahi-daemon11:15:581
197719988,0cyclictest2153-21kworker/0:107:54:330
197719983,2cyclictest617-21polkitd10:37:140
197719981,2cyclictest9-21ksoftirqd/009:40:570
197719981,1cyclictest131rcu_sched11:25:090
197719980,2cyclictest617-21polkitd10:05:170
197719980,1cyclictest9-21ksoftirqd/011:15:510
3739270,0sleep30-21swapper/312:38:343
197919976,0cyclictest0-21swapper/311:39:583
197919976,0cyclictest0-21swapper/311:10:483
197919975,1cyclictest617-21polkitd10:06:193
197919974,2cyclictest972-21sshd10:26:303
197919974,2cyclictest606-21avahi-daemon12:21:023
197919974,2cyclictest606-21avahi-daemon10:31:003
197919973,3cyclictest29412-21kworker/u8:211:02:003
197919972,2cyclictest621-21dbus-daemon11:42:043
197829976,0cyclictest0-21swapper/208:25:272
197829975,1cyclictest606-21avahi-daemon12:16:552
197829975,1cyclictest606-21avahi-daemon11:52:492
197829975,1cyclictest606-21avahi-daemon10:47:152
197829973,1cyclictest0-21swapper/212:12:062
197829972,3cyclictest18456-21sshd10:00:522
197829972,2cyclictest14234-21sshd11:36:042
197789976,0cyclictest0-21swapper/111:55:181
197789976,0cyclictest0-21swapper/109:25:281
197789976,0cyclictest0-21swapper/108:57:581
197789975,1cyclictest606-21avahi-daemon12:35:221
197789975,1cyclictest606-21avahi-daemon11:31:411
197789975,1cyclictest606-21avahi-daemon09:15:101
197719977,0cyclictest8254-21kworker/0:008:01:190
197719970,2cyclictest619-21systemd-logind12:20:310
18318270,5sleep21-21systemd11:05:062
197919964,1cyclictest621-21dbus-daemon12:15:153
197919964,1cyclictest621-21dbus-daemon12:10:033
197919964,1cyclictest621-21dbus-daemon08:09:553
197919964,1cyclictest5840-21sshd12:30:413
197919963,2cyclictest969-21snmpd08:55:583
197919963,2cyclictest969-21snmpd08:19:063
197919963,2cyclictest969-21snmpd08:12:093
197919963,2cyclictest969-21snmpd07:33:173
197919963,2cyclictest606-21avahi-daemon11:45:203
197919963,2cyclictest606-21avahi-daemon10:50:173
197919961,3cyclictest578-13auditd12:00:013
197919960,5cyclictest0-21swapper/308:04:373
197919960,5cyclictest0-21swapper/307:17:383
197829965,0cyclictest0-21swapper/207:45:482
197829963,2cyclictest621-21dbus-daemon11:45:012
197829963,2cyclictest606-21avahi-daemon09:35:222
197829962,1cyclictest0-21swapper/212:30:412
197789965,0cyclictest0-21swapper/112:25:381
197789965,0cyclictest0-21swapper/108:49:581
197789964,1cyclictest621-21dbus-daemon10:51:021
197789964,1cyclictest617-21polkitd10:15:041
197789964,1cyclictest606-21avahi-daemon12:15:411
197789964,1cyclictest606-21avahi-daemon11:10:171
197789964,1cyclictest32014-21grep07:35:361
197789964,1cyclictest1119-21ntpd08:40:331
197789963,2cyclictest621-21dbus-daemon12:10:331
197789960,5cyclictest0-21swapper/108:51:181
197789960,5cyclictest0-21swapper/107:25:461
197919953,1cyclictest969-21snmpd09:00:193
197919953,1cyclictest969-21snmpd08:50:553
197919953,1cyclictest969-21snmpd08:44:403
197919953,1cyclictest969-21snmpd08:35:213
197919953,1cyclictest969-21snmpd08:25:543
197919953,1cyclictest969-21snmpd07:41:113
197919953,1cyclictest969-21snmpd07:37:443
197919953,1cyclictest969-21snmpd07:21:053
197919953,1cyclictest969-21snmpd07:10:093
197919953,1cyclictest617-21polkitd07:50:003
197919953,1cyclictest4461-21sa109:10:003
197919953,1cyclictest27307-21cat08:45:213
197919953,1cyclictest21230-21mii-tool08:30:243
197919952,2cyclictest26617-21chrt07:25:173
197919950,1cyclictest7277-21awk07:55:353
197829953,1cyclictest969-21snmpd09:05:222
197829953,1cyclictest969-21snmpd08:40:582
197829953,1cyclictest969-21snmpd08:35:152
197829953,1cyclictest969-21snmpd08:31:092
197829953,1cyclictest969-21snmpd08:20:422
197829953,1cyclictest969-21snmpd08:04:452
197829953,1cyclictest969-21snmpd07:50:442
197829953,1cyclictest969-21snmpd07:11:222
197829953,1cyclictest824-21cat07:40:272
197829953,1cyclictest31188-21sh08:55:222
197829953,1cyclictest31183-21cat07:35:232
197829953,1cyclictest1667-21sensors_temp09:00:272
197829953,1cyclictest1119-21ntpd08:50:212
197829952,2cyclictest26701-21sort07:25:212
197829952,2cyclictest25100-21interrupts07:20:202
197829952,2cyclictest15862-21wc08:15:372
197829950,2cyclictest22441-21which07:15:022
197829950,1cyclictest0-21swapper/208:05:232
197789954,0cyclictest0-21swapper/109:00:381
197789954,0cyclictest0-21swapper/108:27:581
197789953,1cyclictest986-21lldpd08:32:331
197789953,1cyclictest969-21snmpd09:05:311
197789953,1cyclictest969-21snmpd08:37:241
197789953,1cyclictest969-21snmpd08:10:061
197789953,1cyclictest969-21snmpd07:46:171
197789953,1cyclictest969-21snmpd07:32:111
197789953,1cyclictest969-21snmpd07:24:201
197789953,1cyclictest969-21snmpd07:15:041
197789953,1cyclictest2037-21sshd07:43:341
17740250,0sleep317748-21sensors08:20:353
197919943,0cyclictest969-21snmpd07:45:113
197789942,1cyclictest8684-21cstates08:00:211
197789942,1cyclictest4516-21cat07:50:191
197789942,1cyclictest11170-21latency08:05:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional