You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-05-12 - 13:26

Intel(R) Xeon(R) CPU E3-1220L V2 @ 2.30GHz, Linux 4.18.7-rt5 (Profile)

Latency plot of system in rack #9, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack9slot0.osadl.org (updated Tue May 11, 2021 12:43:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2377321470,2sleep0691599cyclictest10:24:560
2363521170,1sleep330813-21gdbus10:48:403
69962840,0sleep10-21swapper/111:26:261
138932770,1sleep113895-21sshd10:09:021
132562750,1sleep3597-21systemd-journal10:04:113
66232740,1sleep36626-21grep09:12:173
66232740,1sleep36626-21grep09:12:173
141492730,1sleep20-21swapper/210:21:502
72042700,1sleep3597-21systemd-journal10:28:443
58652700,0sleep20-21swapper/211:43:142
214902690,1sleep01473-21nfsd09:54:340
85802680,0sleep20-21swapper/212:20:502
326152670,2sleep332616-21bash11:39:023
277282670,0sleep00-21swapper/011:29:250
265482670,1sleep126550-21bash11:51:431
182952670,1sleep1241rcuc/110:37:401
297592660,1sleep01472-21nfsd09:19:270
274272660,0sleep00-21swapper/009:05:000
319122650,1sleep231905-21sshd10:43:222
302892650,1sleep330291-21bash10:03:073
158122650,1sleep115811-21sshd10:56:071
23922640,0sleep10-21swapper/112:12:591
155762630,0sleep10-21swapper/111:03:371
129762630,1sleep012979-21nfsd412:04:090
295732620,0sleep029569-21systemd11:38:510
230732620,0sleep00-21swapper/010:52:420
279092610,0sleep10-21swapper/111:44:251
34612600,0sleep03463-21sshd09:40:230
314562600,0sleep00-21swapper/010:55:100
257212600,0sleep10-21swapper/112:23:421
251322590,0sleep20-21swapper/208:39:022
638525122,8sleep10-21swapper/107:00:311
430224923,8sleep20-21swapper/206:58:562
642124733,8sleep00-21swapper/007:00:590
641424632,8sleep30-21swapper/307:00:543
691599394,34cyclictest0-21swapper/008:27:000
200822340,0sleep220061-21cp11:01:582
239342320,1sleep223933-21bash10:02:382
215982280,1sleep221596-21sshd12:33:092
64062250,1sleep36404-21bash11:26:243
220412250,1sleep1241rcuc/111:34:351
122522240,2sleep312253-21sshd09:25:153
112382230,0sleep111210-21sshd10:44:011
49612220,0sleep20-21swapper/211:26:192
210452220,1sleep3838-21NetworkManager11:13:493
184942220,1sleep3431ktimersoftd/311:58:593
152522220,1sleep215255-21grepconf.sh09:43:382
68742210,0sleep00-21swapper/011:16:430
4222210,0sleep20-21swapper/209:44:542
310422210,0sleep20-21swapper/212:18:302
225882210,0sleep20-21swapper/211:51:282
209522210,0sleep3421rcuc/309:23:313
145882210,1sleep0101ktimersoftd/010:21:520
130742210,1sleep116466-21nscd10:21:441
39112200,2sleep13871-21sshd10:39:041
248852200,1sleep324883-21bash11:36:423
212252200,0sleep321196-21sshd09:51:493
140612200,1sleep1241rcuc/111:41:451
131062190,2sleep2691799cyclictest08:14:112
6918991814,3cyclictest29717-21systemd09:05:083
6917991818,0cyclictest12550irq/23-ehci_hcd08:27:002
6916991815,2cyclictest30813-21gdbus11:09:151
691599180,16cyclictest0-21swapper/009:47:510
144652180,0sleep30-21swapper/309:43:353
6917991714,2cyclictest12876-21kill10:57:502
691799170,16cyclictest12502-21kworker/2:410:50:362
691699177,2cyclictest26-21ksoftirqd/111:18:061
691699171,15cyclictest1172-21systemd12:24:061
691899166,7cyclictest0-21swapper/312:23:293
691899160,15cyclictest0-21swapper/309:44:063
6917991613,1cyclictest27839-21sshd12:08:462
6916991614,1cyclictest26-21ksoftirqd/109:23:491
6916991610,2cyclictest26-21ksoftirqd/111:56:291
691699160,14cyclictest0-21swapper/109:48:151
691699160,0cyclictest0-21swapper/108:59:051
691599168,4cyclictest23030-21munin-run07:33:490
691599160,14cyclictest597-21systemd-journal10:15:090
42482160,0sleep10-21swapper/109:12:071
42482160,0sleep10-21swapper/109:12:071
24192160,1sleep32378-21cp12:29:573
114072160,1sleep0818-21dbus-daemon12:28:340
691899150,13cyclictest28048-21bash10:30:293
6917991512,2cyclictest10290-21bash09:27:362
691799150,14cyclictest350-21bash10:05:392
691699157,1cyclictest26-21ksoftirqd/112:29:191
691699157,1cyclictest26-21ksoftirqd/109:27:431
6916991512,2cyclictest866-21sshd12:07:131
6916991512,2cyclictest30953-21rm10:49:081
6916991512,2cyclictest29360-21kworker/u8:210:30:341
6916991512,2cyclictest28551-21sshd11:59:351
691699151,1cyclictest251ktimersoftd/111:07:091
691699150,1cyclictest111rcu_preempt11:33:481
691699150,1cyclictest111rcu_preempt09:16:461
691599151,3cyclictest0-21swapper/012:12:510
6915991512,2cyclictest9663-21systemd11:45:140
691599150,14cyclictest27541-21grep11:08:150
64022150,1sleep16403-21sshd09:40:361
225242150,0sleep00-21swapper/011:57:150
18932150,0sleep20-21swapper/207:54:042
112512150,1sleep211248-21sshd11:20:462
691899140,2cyclictest32329-21bash10:10:303
691899140,1cyclictest30813-21gdbus09:29:573
691899140,13cyclictest25940-21systemd09:57:433
691899140,13cyclictest10225-21sshd12:15:273
691899140,12cyclictest817-21systemd-logind11:54:183
691799143,10cyclictest877-21polkitd09:06:122
6917991412,1cyclictest7227-21sshd11:13:022
6917991411,2cyclictest9151-21sshd09:38:232
691799140,1cyclictest0-21swapper/211:04:122
691799140,12cyclictest1-21systemd11:35:372
691699146,1cyclictest26-21ksoftirqd/110:15:461
691699141,1cyclictest26-21ksoftirqd/111:18:591
691599147,4cyclictest14762-21bash09:10:250
691599147,4cyclictest14762-21bash09:10:250
6915991411,2cyclictest32550-21sshd10:45:160
691599140,1cyclictest0-21swapper/011:59:520
691599140,13cyclictest0-21swapper/011:01:020
691899132,8cyclictest1469-21nfsd10:54:053
691899131,2cyclictest597-21systemd-journal11:46:133
691899131,2cyclictest20753-40dnf11:32:443
6918991311,1cyclictest1750-21sshd11:04:423
691899131,11cyclictest15416-21sshd11:49:113
6918991311,1cyclictest1-21systemd10:49:043
691899130,5cyclictest30813-21gdbus12:26:073
691899130,2cyclictest24036-21systemd11:15:493
691899130,1cyclictest15633-21systemd-cgroups12:04:173
691899130,12cyclictest3769-21kworker/3:110:41:563
691899130,11cyclictest13794-21sshd11:20:543
691799133,1cyclictest30594-21kworker/2:311:46:112
691799132,10cyclictest9840-21grepconf.sh12:24:352
6917991311,1cyclictest818-21dbus-daemon11:32:442
6917991311,1cyclictest19877-21sshd10:37:472
6917991311,1cyclictest0-21swapper/209:30:552
691799130,1cyclictest0-21swapper/211:58:582
691799130,12cyclictest25514-21bash11:57:272
691799130,12cyclictest21132-21sshd10:12:092
691699136,3cyclictest26-21ksoftirqd/110:28:111
691699135,2cyclictest26-21ksoftirqd/110:03:511
691699132,9cyclictest26-21ksoftirqd/112:14:091
6916991312,0cyclictest251ktimersoftd/109:36:171
6916991312,0cyclictest20861-21kworker/1:309:50:171
6916991312,0cyclictest18464-21kworker/1:209:31:031
691699130,12cyclictest817-21systemd-logind10:00:131
691599139,2cyclictest818-21dbus-daemon09:49:040
691599133,1cyclictest0-21swapper/012:29:020
691599132,2cyclictest22631-21sshd12:14:110
691599131,9cyclictest0-21swapper/011:24:170
6915991311,1cyclictest7944-21systemd11:20:350
6915991311,1cyclictest3701-21sshd10:41:240
6915991311,1cyclictest29280-21expr09:29:010
6915991311,1cyclictest18658-21grepconf.sh11:13:410
6915991310,2cyclictest597-21systemd-journal11:35:520
6915991310,2cyclictest23774-21bash10:35:260
691599130,12cyclictest26860-21bash09:16:220
691899121,10cyclictest5957-21systemd10:59:143
691899120,10cyclictest30813-21gdbus12:10:133
691899120,0cyclictest0-21swapper/308:49:183
691799128,2cyclictest11268-21sshd10:44:012
691799126,5cyclictest11655-21sshd10:13:592
6917991210,1cyclictest1473-21nfsd09:18:192
691799120,11cyclictest0-21swapper/209:57:312
691799120,11cyclictest0-21swapper/208:34:062
691799120,10cyclictest14359-21sshd11:15:152
691799120,10cyclictest11603-21kill12:13:332
691699125,1cyclictest26-21ksoftirqd/109:54:581
6916991211,0cyclictest22873-21kworker/1:108:07:411
6916991210,1cyclictest9520-21kworker/1:007:15:421
6916991210,1cyclictest13507-21sshd09:04:131
691699120,11cyclictest3740-21date07:58:491
6915991210,1cyclictest13645-21systemd-cgroups09:59:350
6915991210,1cyclictest10250-21systemd-cgroups10:06:200
691599120,2cyclictest12649-21sshd11:49:040
691599120,11cyclictest0-21swapper/010:32:060
691599120,10cyclictest1-21systemd12:19:040
691899115,1cyclictest817-21systemd-logind10:34:403
691899114,5cyclictest862-21nscd10:21:433
691899110,8cyclictest0-21swapper/310:16:353
691899110,6cyclictest4914-21ls07:59:093
691899110,2cyclictest12309-21sshd09:15:113
691899110,0cyclictest0-21swapper/307:09:023
691799115,2cyclictest817-21systemd-logind10:24:002
691799113,3cyclictest817-21systemd-logind10:29:422
691799110,10cyclictest30811-21packagekitd09:11:522
691799110,10cyclictest30811-21packagekitd09:11:522
691799110,10cyclictest19319-21systemctl09:49:042
691799110,10cyclictest0-21swapper/208:04:082
6916991110,0cyclictest9520-21kworker/1:007:09:051
6916991110,0cyclictest0-21swapper/107:30:081
691699110,10cyclictest0-21swapper/108:36:211
691699110,0cyclictest0-21swapper/107:59:031
691599117,2cyclictest877-21polkitd09:24:040
691599116,1cyclictest21676-21munin-run08:33:490
691599110,10cyclictest0-21swapper/008:59:450
691899100,9cyclictest0-21swapper/307:23:493
691899100,1cyclictest1-21systemd09:34:033
691799109,0cyclictest0-21swapper/207:39:022
691799109,0cyclictest0-21swapper/207:17:332
691799109,0cyclictest0-21swapper/207:09:392
691799100,9cyclictest25086-21systemd-cgroups09:19:032
691799100,0cyclictest0-21swapper/208:59:142
691799100,0cyclictest0-21swapper/208:48:172
691799100,0cyclictest0-21swapper/207:25:542
691699109,0cyclictest0-21swapper/108:56:401
691699108,1cyclictest24148-21kworker/1:207:03:541
691699108,1cyclictest22873-21kworker/1:108:11:091
691699100,9cyclictest0-21swapper/108:24:501
691699100,9cyclictest0-21swapper/107:35:071
691699100,5cyclictest0-21swapper/108:14:071
691599109,0cyclictest0-21swapper/007:22:170
691599109,0cyclictest0-21swapper/007:08:550
691599108,1cyclictest0-21swapper/008:58:490
691599107,2cyclictest24695-21uname10:09:530
691599100,8cyclictest29141-21kworker/0:309:36:070
691599100,0cyclictest0-21swapper/008:07:280
69189990,1cyclictest20510-21latency_hist07:28:493
69169998,0cyclictest22873-21kworker/1:107:51:551
69169998,0cyclictest0-21swapper/107:40:211
69169998,0cyclictest0-21swapper/107:25:221
69169997,1cyclictest22873-21kworker/1:108:30:131
69159998,0cyclictest0-21swapper/008:36:100
69159998,0cyclictest0-21swapper/007:41:480
69159998,0cyclictest0-21swapper/007:27:030
69159998,0cyclictest0-21swapper/007:17:450
69189980,0cyclictest0-21swapper/307:46:553
69179987,0cyclictest0-21swapper/208:33:382
69179987,0cyclictest0-21swapper/207:21:512
69169987,0cyclictest23968-21kworker/1:208:50:511
69169987,0cyclictest0-21swapper/107:20:361
69159981,2cyclictest0-21swapper/008:09:050
69159980,5cyclictest0-21swapper/007:54:070
69189975,1cyclictest9048-21atenall_r9power08:08:593
69189973,3cyclictest87450irq/26-p4p1-rx-07:18:493
69189970,2cyclictest0-21swapper/308:54:073
69189970,2cyclictest0-21swapper/308:29:043
69189970,2cyclictest0-21swapper/308:24:053
69189970,2cyclictest0-21swapper/308:04:073
69189970,2cyclictest0-21swapper/307:49:093
69189970,2cyclictest0-21swapper/307:34:063
69189970,2cyclictest0-21swapper/307:29:073
69189970,1cyclictest26110-21smartctl08:39:123
69189970,1cyclictest0-21swapper/307:39:073
69189970,0cyclictest0-21swapper/307:06:083
69179970,1cyclictest32254-21aten_r9power_fr08:54:012
69179970,1cyclictest29202-21fw_conntrack07:44:062
69179970,1cyclictest23947-21fschecks_time07:34:032
69179970,1cyclictest1473-21nfsd08:00:422
69179970,1cyclictest1473-21nfsd07:29:112
69179970,1cyclictest14646-21fschecks_count08:19:052
69179970,1cyclictest10164-21needreboot08:09:062
69179970,1cyclictest0-21swapper/208:51:342
69179970,1cyclictest0-21swapper/207:49:102
69179970,1cyclictest0-21swapper/207:04:082
69169972,1cyclictest121rcu_sched07:44:061
69169971,3cyclictest121rcu_sched08:19:031
69169971,1cyclictest121rcu_sched08:44:001
69169970,1cyclictest121rcu_sched08:43:491
69159975,1cyclictest23276-21/usr/sbin/munin07:34:010
69159970,6cyclictest1473-21nfsd07:48:350
69159970,2cyclictest0-21swapper/008:49:120
69159970,2cyclictest0-21swapper/007:04:080
69159970,1cyclictest4482-21perl07:59:040
69159970,1cyclictest0-21swapper/008:44:430
69159970,1cyclictest0-21swapper/008:19:060
69189964,1cyclictest1629-21sed07:54:023
69189962,2cyclictest0-21swapper/308:59:043
69189962,2cyclictest0-21swapper/308:14:063
69189960,5cyclictest0-21swapper/308:34:043
69189960,5cyclictest0-21swapper/308:22:143
69189960,2cyclictest0-21swapper/308:44:083
69159960,2cyclictest0-21swapper/008:39:070
69159960,2cyclictest0-21swapper/007:49:090
69159960,0cyclictest0-21swapper/008:14:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional