You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2022-01-26 - 05:43

Intel(R) Core(TM) i3-2310E CPU @ 2.10GHz, Linux 5.10.47-rt46 (Profile)

Latency plot of system in rack #8, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by a total of 2 SMIs that occured during the measurement.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rack8slot2.osadl.org (updated Wed Jan 26, 2022 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1015799131129,1cyclictest0-21swapper/219:22:222
9587210842,31sleep20-21swapper/219:09:002
9829210544,49sleep10-21swapper/119:12:081
457421020,1sleep20-21swapper/222:58:082
1695921020,0sleep00-21swapper/023:10:370
1015599102100,1cyclictest0-21swapper/019:12:520
974429642,43sleep00-21swapper/019:11:030
148142950,0sleep10-21swapper/122:35:101
964029147,33sleep30-21swapper/319:09:413
194292870,0sleep00-21swapper/023:46:530
10158998379,1cyclictest0-21swapper/319:12:523
169062770,0sleep30-21swapper/322:38:053
10156996355,7cyclictest0-21swapper/119:12:521
10157993735,1cyclictest0-21swapper/219:12:522
260352340,0sleep10-21swapper/123:53:121
271302310,0sleep00-21swapper/023:21:080
1015799310,1cyclictest0-21swapper/219:38:062
101462300,1sleep30-21swapper/300:09:533
160972290,0sleep20-21swapper/221:30:172
28752280,1sleep228-21ksoftirqd/222:23:092
10156992717,5cyclictest8832-21kworker/u8:123:48:141
113792260,0sleep00-21swapper/023:38:110
10158992623,2cyclictest43750irq/33-enp3s0-r23:13:033
204242250,0sleep20-21swapper/223:48:032
1015599250,24cyclictest0-21swapper/000:23:070
1015699240,23cyclictest0-21swapper/122:58:041
1015599241,21cyclictest0-21swapper/021:48:040
151582220,0sleep00-21swapper/021:28:470
1015899220,2cyclictest0-21swapper/320:03:073
1015899220,20cyclictest0-21swapper/323:58:043
1015599221,1cyclictest0-21swapper/022:33:100
1015899210,1cyclictest0-21swapper/321:03:043
1015699210,20cyclictest0-21swapper/100:28:041
1015599211,1cyclictest0-21swapper/000:03:040
1015599204,8cyclictest25744-21apt-get19:47:470
10155991913,1cyclictest141rcu_preempt22:18:030
193822180,1sleep20-21swapper/223:13:082
1015799167,3cyclictest0-21swapper/222:51:422
1015599164,9cyclictest28490-21/usr/sbin/munin21:08:170
1015899151,7cyclictest29260-21apt-get22:17:493
1015799155,5cyclictest0-21swapper/221:38:332
1015699150,9cyclictest9395-21ssh22:29:181
1015599151,8cyclictest29668-21munin-run23:57:480
1015599150,14cyclictest0-21swapper/019:18:090
46812140,3sleep14682-21fschecks_count00:38:091
1015899147,2cyclictest34-21ksoftirqd/321:32:453
1015899146,6cyclictest32238-21fschecks_time00:33:083
1015899145,3cyclictest29370-21kernelversion21:08:143
1015799144,5cyclictest0-21swapper/220:08:122
1015799144,5cyclictest0-21swapper/220:03:092
1015799144,5cyclictest0-21swapper/219:58:122
1015699148,5cyclictest0-21swapper/123:44:021
1015699145,8cyclictest0-21swapper/119:48:221
1015699144,5cyclictest17865-21hddtemp_smartct00:18:101
1015699143,8cyclictest205-21jbd2/sda1-823:18:181
1015699140,6cyclictest31350-21sh23:25:071
1015599145,6cyclictest2087-21sendmail00:02:480
1015599144,5cyclictest0-21swapper/019:25:470
1015899135,4cyclictest0-21swapper/321:49:493
1015899135,3cyclictest0-21swapper/323:30:223
1015899135,3cyclictest0-21swapper/323:30:223
1015899135,3cyclictest0-21swapper/320:25:403
1015899134,8cyclictest0-21swapper/323:33:113
1015899131,7cyclictest14364-21grep23:08:093
1015799134,7cyclictest2367-21ssh22:56:002
1015799134,4cyclictest30742-21irqstats22:18:102
1015799132,9cyclictest0-21swapper/223:03:332
1015699135,2cyclictest0-21swapper/120:31:341
1015699134,8cyclictest0-21swapper/123:07:101
1015699134,8cyclictest0-21swapper/100:05:361
1015699133,5cyclictest0-21swapper/122:46:531
1015699132,6cyclictest24477-21grep22:12:481
1015599135,4cyclictest28441-21ssh23:22:550
1015599130,5cyclictest19560-21idleruntime-cro22:07:480
279922120,1sleep228-21ksoftirqd/200:28:122
1015899125,6cyclictest0-21swapper/321:33:453
1015899125,6cyclictest0-21swapper/320:59:203
1015899124,3cyclictest25160-21ssh21:39:033
1015899124,3cyclictest0-21swapper/322:43:103
1015899124,3cyclictest0-21swapper/322:28:503
1015899124,3cyclictest0-21swapper/320:18:113
1015899124,3cyclictest0-21swapper/320:18:113
1015899124,3cyclictest0-21swapper/300:03:353
1015899123,7cyclictest0-21swapper/320:34:073
1015899120,5cyclictest25337-21gltestperf22:13:083
1015799129,2cyclictest0-21swapper/220:58:102
1015799127,3cyclictest7742-21grep22:28:082
1015799127,3cyclictest10469-21tr21:58:102
1015799126,4cyclictest15082-21ls23:08:172
1015799124,6cyclictest0-21swapper/220:33:462
1015799124,6cyclictest0-21swapper/200:11:242
1015799124,2cyclictest0-21swapper/221:14:032
1015799123,4cyclictest8723-21ssh21:56:132
1015799122,5cyclictest0-21swapper/200:33:102
1015799121,6cyclictest16411-21munin-run22:37:472
1015799120,9cyclictest17973-21ssh23:44:312
1015799120,8cyclictest7394-21ssh00:40:522
1015699126,4cyclictest18430-21ntp_states19:28:161
1015699125,5cyclictest611-21gdbus20:23:171
1015699125,5cyclictest0-21swapper/121:42:121
1015699124,5cyclictest2024-21ssh00:35:021
1015699124,5cyclictest0-21swapper/123:17:341
1015699124,5cyclictest0-21swapper/120:06:201
1015699124,5cyclictest0-21swapper/119:34:081
1015699124,2cyclictest0-21swapper/123:29:001
1015699124,2cyclictest0-21swapper/123:29:001
1015699123,8cyclictest0-21swapper/121:42:581
1015699123,8cyclictest0-21swapper/120:58:051
1015699122,7cyclictest13878-21basename20:33:101
1015699121,6cyclictest30832-21expr19:58:081
1015699121,6cyclictest20128-21df20:48:081
1015699121,6cyclictest14920-21munin-run19:22:481
1015699120,9cyclictest19392-21ssh22:07:341
1015599129,2cyclictest0-21swapper/022:53:070
1015599127,4cyclictest32407-21latency_hist21:47:480
1015599127,3cyclictest603-21runrttasks20:03:210
1015599126,4cyclictest24272-21head21:38:160
1015599125,5cyclictest13528-21find23:07:480
1015599125,5cyclictest0-21swapper/019:49:150
1015599124,5cyclictest5231-21aten2_r8power_c21:53:040
1015599123,6cyclictest21615-21taskset19:37:480
1015599122,9cyclictest30423-21diskmemload21:15:440
1015599122,5cyclictest0-21swapper/000:28:090
1015599121,5cyclictest141rcu_preempt20:58:050
1015599121,10cyclictest0-21swapper/022:43:170
220112110,0sleep30-21swapper/320:53:063
1015899117,3cyclictest19950-21ssh22:08:063
1015899117,2cyclictest4809-21cut00:38:103
1015899116,3cyclictest603-21runrttasks00:19:353
1015899116,3cyclictest31101-21latency_hist22:52:473
1015899116,3cyclictest30222-21vmstat23:23:183
1015899116,3cyclictest17073-21cron19:27:473
1015899115,4cyclictest611-21gdbus00:13:183
1015899115,4cyclictest5858-21grep21:53:073
1015899115,4cyclictest3595-21awk20:08:173
1015899115,4cyclictest27498-21grep00:28:083
1015899115,4cyclictest24531-21sh23:52:243
1015899115,4cyclictest24091-21tr19:43:053
1015899115,4cyclictest17968-21df20:43:063
1015899115,4cyclictest14078-21sh22:01:533
1015899115,4cyclictest13604-21sed19:18:093
1015899115,4cyclictest0-21swapper/322:53:033
1015899114,5cyclictest23290-21ntpq00:23:173
1015899114,5cyclictest11047-21apt-get20:28:053
1015899113,6cyclictest0-21swapper/319:33:323
1015899111,8cyclictest22257-21grep19:38:103
1015899111,5cyclictest8346-21ssh21:23:053
1015899110,8cyclictest0-21swapper/322:59:073
1015899110,2cyclictest0-21swapper/320:38:123
1015899110,1cyclictest0-21swapper/323:40:183
1015799116,3cyclictest30645-21tr23:58:082
1015799116,3cyclictest29775-21pmu-power23:23:152
1015799116,3cyclictest25291-21aten2_r8power_e23:53:072
1015799115,4cyclictest9439-21grep20:23:102
1015799115,4cyclictest5607-21ntpq20:13:132
1015799115,4cyclictest23699-21grep00:23:202
1015799115,3cyclictest32164-21kworker/u8:121:23:392
1015799114,6cyclictest6252-21sshd23:33:082
1015799114,6cyclictest0-21swapper/222:38:442
1015799114,5cyclictest831-21diskstats21:48:072
1015799114,5cyclictest26441-21aten2_r8power_v21:03:052
1015799114,5cyclictest0-21swapper/223:30:532
1015799114,5cyclictest0-21swapper/223:30:532
1015799114,5cyclictest0-21swapper/221:18:052
1015799113,6cyclictest20015-21ssh21:33:392
1015799113,2cyclictest0-21swapper/222:13:232
1015799112,8cyclictest0-21swapper/220:19:102
1015799112,8cyclictest0-21swapper/220:19:102
1015799111,2cyclictest0-21swapper/223:18:172
1015799110,5cyclictest2362-21ssh00:02:572
1015799110,4cyclictest157022ssh00:16:032
1015699117,4cyclictest0-21swapper/121:19:041
1015699116,3cyclictest30423-21diskmemload23:33:131
1015699115,5cyclictest0-21swapper/121:22:481
1015699115,5cyclictest0-21swapper/120:44:301
1015699115,4cyclictest8690-21ntpq00:08:161
1015699114,5cyclictest31981-21ssh22:19:041
1015699114,5cyclictest30931-21/usr/sbin/munin21:13:041
1015699114,5cyclictest0-21swapper/120:08:041
1015699114,5cyclictest0-21swapper/100:13:451
1015699113,8cyclictest0-21swapper/100:02:041
1015699113,5cyclictest0-21swapper/122:26:501
1015699113,5cyclictest0-21swapper/121:53:021
1015699113,5cyclictest0-21swapper/121:48:101
1015699113,5cyclictest0-21swapper/121:12:161
1015699113,5cyclictest0-21swapper/121:03:281
1015699113,5cyclictest0-21swapper/119:38:041
1015699112,4cyclictest16768-21/usr/sbin/munin22:38:161
1015699112,4cyclictest11695-21memory23:38:141
1015699112,3cyclictest22-21ksoftirqd/121:28:071
1015699112,1cyclictest0-21swapper/119:52:581
1015699111,7cyclictest16130-21ssh23:09:181
1015699110,9cyclictest21953-21gdbus20:52:491
1015699110,6cyclictest11111-21rm21:58:161
1015599117,2cyclictest0-21swapper/022:58:170
1015599116,4cyclictest25330-21fw_packets22:13:080
1015599116,3cyclictest16783-21sh00:17:470
1015599115,4cyclictest238-21systemd-journal23:48:100
1015599114,5cyclictest21589-21grep22:08:210
1015599114,5cyclictest0-21swapper/020:39:370
1015599114,5cyclictest0-21swapper/019:58:360
1015599113,5cyclictest0-21swapper/000:33:100
1015599113,3cyclictest0-21swapper/023:36:580
1015599113,3cyclictest0-21swapper/020:47:590
1015599113,3cyclictest0-21swapper/020:23:340
1015599113,3cyclictest0-21swapper/000:39:120
1015599112,4cyclictest0-21swapper/020:08:120
1015599110,9cyclictest18782-21gltestperf21:33:070
221912100,1sleep00-21swapper/020:53:070
1015899106,2cyclictest603-21runrttasks21:13:053
1015899105,3cyclictest29255-21sh21:43:153
1015899105,3cyclictest15822-21gltestperf22:03:193
1015899104,4cyclictest3281-21/usr/sbin/munin21:18:023
1015899104,4cyclictest26424-21cut19:48:053
1015899104,4cyclictest24428-21/usr/sbin/munin23:18:103
1015899103,5cyclictest590-21snmpd22:23:053
1015899102,4cyclictest0-21swapper/320:48:113
1015899101,5cyclictest25229-21ssh23:53:053
1015899101,2cyclictest0-21swapper/319:58:073
1015799107,2cyclictest19021-21unixbench_singl20:43:172
1015799106,2cyclictest20482-21if_err_eno120:48:122
1015799105,4cyclictest0-21swapper/219:29:052
1015799105,3cyclictest11522-21fschecks_count20:28:092
1015799104,5cyclictest0-21swapper/219:48:112
1015799104,5cyclictest0-21swapper/200:21:392
1015799104,4cyclictest29058-21if_enp5s019:53:112
1015799104,4cyclictest24877-21cut19:43:132
1015799102,7cyclictest0-21swapper/219:23:092
1015799101,7cyclictest30428-21grep21:12:472
1015799101,5cyclictest11367-21ssh23:38:112
1015799101,4cyclictest0-21swapper/220:38:122
1015799100,8cyclictest15063-21/usr/sbin/munin22:03:192
1015799100,5cyclictest28185-21grep21:43:042
1015799100,5cyclictest21310-21gpgv22:42:482
1015799100,3cyclictest19926-21/usr/sbin/munin22:08:222
1015699106,3cyclictest24531-21sh00:24:341
1015699105,5cyclictest0-21swapper/122:15:161
1015699105,5cyclictest0-21swapper/119:45:081
1015699104,5cyclictest896-21ssh22:53:361
1015699104,5cyclictest0-21swapper/122:48:331
1015699104,5cyclictest0-21swapper/120:18:041
1015699104,5cyclictest0-21swapper/120:18:041
1015699104,5cyclictest0-21swapper/120:13:001
1015699101,1cyclictest0-21swapper/121:33:021
1015699101,1cyclictest0-21swapper/120:42:221
1015699101,1cyclictest0-21swapper/119:22:501
1015599106,2cyclictest3935-21cut21:18:070
1015599105,3cyclictest8640-21tr20:22:480
1015599105,3cyclictest8640-21tr20:22:480
1015599105,3cyclictest7248-21run-parts00:07:550
1015599105,3cyclictest30423-21diskmemload00:20:150
1015599104,4cyclictest2267-21apt-get22:23:050
1015599104,4cyclictest17461-21gltestperf22:38:090
1015599104,4cyclictest0-21swapper/019:56:270
1015599103,5cyclictest590-21snmpd23:28:070
1015599103,5cyclictest590-21snmpd23:28:070
1015599103,5cyclictest20032-21processes23:13:160
1015599103,5cyclictest17962-21sed19:28:100
1015599102,4cyclictest11034-21acpi20:28:050
1015599102,3cyclictest0-21swapper/020:13:010
1015599101,2cyclictest0-21swapper/022:47:580
1015599101,2cyclictest0-21swapper/021:57:590
1015599100,9cyclictest15128-21awk20:37:480
1015599100,9cyclictest0-21swapper/020:43:070
101589994,3cyclictest17766-21df19:28:083
101589993,4cyclictest5182-21basename20:13:083
101589993,4cyclictest16630-21missed_timers23:43:153
101589992,4cyclictest8350-21apt-get23:02:483
101589991,7cyclictest0-21swapper/319:53:103
101589990,5cyclictest12100-21ssh22:33:043
101579996,2cyclictest22040-21aten2_r8power_e20:53:072
101579994,4cyclictest0-21swapper/219:32:502
101559996,2cyclictest30423-21diskmemload22:27:550
101559994,4cyclictest8289-21apt-get21:23:040
101559994,3cyclictest26916-21grep21:03:090
101559993,5cyclictest590-21snmpd19:32:590
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional