You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-12-06 - 22:18

Intel(R) Core(TM) i5-8365UE CPU @ 1.60GHz, Linux 5.10.41-rt42 (Profile)

Latency plot of system in rack #a, slot #8
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rackaslot8.osadl.org (updated Mon Dec 06, 2021 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
256142300274,16sleep20-21swapper/207:06:412
257032281250,20sleep00-21swapper/007:07:430
257322277250,18sleep10-21swapper/107:08:051
255662274215,47sleep30-21swapper/307:06:043
35932670,0sleep00-21swapper/012:30:290
154612650,1sleep30-21swapper/309:05:173
130732640,0sleep30-21swapper/312:36:113
8352610,0sleep30-21swapper/311:32:343
264282600,0sleep30-21swapper/310:50:203
308622490,0sleep00-21swapper/012:09:340
319542200,0sleep00-21swapper/010:15:380
47222190,0sleep00-21swapper/011:54:050
22882190,0sleep10-21swapper/109:59:101
145712180,0sleep10-21swapper/109:47:001
10582180,1sleep10-21swapper/112:29:571
251242160,0sleep00-21swapper/010:11:540
120602160,0sleep10-21swapper/111:39:281
311602140,0sleep10-21swapper/112:09:471
35032130,0sleep30-21swapper/310:18:333
320732130,0sleep10-21swapper/109:19:521
260019994,3cyclictest283-21systemd-journal08:31:330
9794280,0sleep30-21swapper/310:40:363
9136280,0sleep30-21swapper/307:40:203
260149985,2cyclictest34-21ksoftirqd/312:00:003
260149983,3cyclictest283-21systemd-journal11:55:003
260149980,7cyclictest16473-21diskmemload09:15:113
260099983,3cyclictest283-21systemd-journal08:25:322
260099982,3cyclictest26025-21munin-run08:20:002
260069984,3cyclictest514-21in:imuxsock07:15:011
260019986,1cyclictest0-21swapper/012:24:190
260019983,3cyclictest283-21systemd-journal10:45:230
260019983,3cyclictest283-21systemd-journal10:06:070
260019982,5cyclictest0-21swapper/012:40:010
1624280,0sleep30-21swapper/310:35:523
260149975,1cyclictest34-21ksoftirqd/310:00:003
260149973,3cyclictest665-21snmpd11:14:513
260149973,3cyclictest665-21snmpd08:49:303
260149973,3cyclictest2371-21cat12:30:153
260149970,2cyclictest283-21systemd-journal08:20:003
260099975,2cyclictest28-21ksoftirqd/210:00:002
260099974,2cyclictest283-21systemd-journal07:57:302
260099973,3cyclictest283-21systemd-journal12:40:002
260099973,3cyclictest283-21systemd-journal11:47:592
260099973,3cyclictest283-21systemd-journal10:30:222
260099972,3cyclictest283-21systemd-journal09:34:202
260099970,3cyclictest16206-21sed10:25:232
260069974,2cyclictest283-21systemd-journal07:37:111
260069973,3cyclictest9292-21ntpq12:15:241
260069973,3cyclictest7360-21memory11:55:161
260069973,3cyclictest665-21snmpd08:40:171
260069972,4cyclictest0-21swapper/110:45:001
260069972,3cyclictest4189-21ssh11:53:411
260069971,3cyclictest9899-21munin-run07:45:011
260069970,5cyclictest283-21systemd-journal08:02:231
260069970,3cyclictest283-21systemd-journal10:57:361
260069970,1cyclictest16473-21diskmemload12:04:401
260019974,2cyclictest508-21dbus-daemon11:59:440
260019974,2cyclictest12-21ksoftirqd/007:20:010
260019973,3cyclictest9659-21sshd07:42:260
260019973,3cyclictest665-21snmpd11:30:420
260019972,3cyclictest30764-21dump-pmu-power08:30:010
24641270,0sleep20-21swapper/209:15:102
260149966,0cyclictest0-21swapper/312:28:003
260149966,0cyclictest0-21swapper/311:45:173
260149966,0cyclictest0-21swapper/311:20:463
260149966,0cyclictest0-21swapper/310:23:313
260149963,2cyclictest29451-21df07:15:153
260149963,2cyclictest26933-21missed_timers09:35:193
260149963,2cyclictest22718-21ssh11:26:073
260149963,2cyclictest20388-21ssh12:21:413
260149963,2cyclictest18973-21cat09:30:293
260149963,2cyclictest18817-21ssh10:27:163
260149963,2cyclictest11410-21tune2fs09:45:253
260149962,3cyclictest9030-21cat09:25:133
260149962,3cyclictest665-21snmpd10:31:093
260149962,3cyclictest665-21snmpd08:56:303
260149962,3cyclictest665-21snmpd08:31:073
260149962,3cyclictest665-21snmpd08:03:323
260149962,3cyclictest665-21snmpd07:48:313
260149962,3cyclictest665-21snmpd07:36:123
260149962,3cyclictest31642-21expr07:20:143
260149962,3cyclictest30899-21wc08:30:003
260149962,3cyclictest23882-21ssh09:52:313
260149962,3cyclictest20008-21cut08:05:143
260149962,3cyclictest18221-21cat11:05:133
260149962,3cyclictest14876-21ls11:40:213
260149962,3cyclictest0-21swapper/311:36:283
260149962,3cyclictest0-21swapper/310:13:573
260149962,1cyclictest0-21swapper/312:15:363
260149962,1cyclictest0-21swapper/312:10:273
260149962,1cyclictest0-21swapper/312:06:233
260149962,1cyclictest0-21swapper/311:16:433
260149962,1cyclictest0-21swapper/310:00:153
260149960,6cyclictest0-21swapper/311:02:353
260149960,6cyclictest0-21swapper/311:02:353
260149960,6cyclictest0-21swapper/310:05:063
260149960,6cyclictest0-21swapper/308:10:143
260149960,3cyclictest283-21systemd-journal10:57:443
260149960,3cyclictest283-21systemd-journal07:10:263
260149960,2cyclictest728-21df_inode09:20:143
260149960,1cyclictest0-21swapper/312:00:463
260149960,0cyclictest0-21swapper/310:45:383
260149960,0cyclictest0-21swapper/309:11:563
260149960,0cyclictest0-21swapper/309:11:553
260149960,0cyclictest0-21swapper/307:50:223
260099966,0cyclictest0-21swapper/212:28:522
260099966,0cyclictest0-21swapper/212:10:072
260099966,0cyclictest0-21swapper/212:01:492
260099966,0cyclictest0-21swapper/211:57:362
260099966,0cyclictest0-21swapper/210:40:512
260099966,0cyclictest0-21swapper/209:47:342
260099964,2cyclictest28-21ksoftirqd/211:05:022
260099963,2cyclictest665-21snmpd11:04:322
260099963,2cyclictest665-21snmpd11:04:322
260099963,2cyclictest665-21snmpd07:38:482
260099963,2cyclictest4765-21cat07:30:222
260099963,2cyclictest31959-21ssh10:15:392
260099963,2cyclictest29472-21latency11:30:212
260099963,2cyclictest283-21systemd-journal07:24:242
260099963,2cyclictest27543-21chrt12:07:042
260099963,2cyclictest1395-21cat09:20:192
260099963,2cyclictest0-21swapper/212:34:502
260099963,1cyclictest141rcu_preempt12:15:262
260099962,3cyclictest665-21snmpd12:22:462
260099962,3cyclictest665-21snmpd11:25:592
260099962,3cyclictest665-21snmpd10:12:022
260099962,3cyclictest665-21snmpd09:05:292
260099962,3cyclictest665-21snmpd07:10:192
260099962,3cyclictest5854-21ssh10:57:482
260099962,3cyclictest3747-21cat08:40:172
260099962,3cyclictest3336-21expr11:15:192
260099962,3cyclictest283-21systemd-journal07:54:312
260099962,3cyclictest27347-21grep09:35:212
260099962,3cyclictest20492-21cut09:50:272
260099962,3cyclictest19328-21sh10:46:282
260099962,3cyclictest12549-21cat09:00:162
260099962,3cyclictest12361-21cat11:20:152
260099962,1cyclictest0-21swapper/211:41:192
260099962,1cyclictest0-21swapper/210:20:232
260099962,1cyclictest0-21swapper/209:10:412
260099962,1cyclictest0-21swapper/209:10:412
260099960,6cyclictest0-21swapper/211:14:042
260099960,6cyclictest0-21swapper/210:50:362
260099960,6cyclictest0-21swapper/209:41:402
260099960,0cyclictest0-21swapper/211:50:292
260099960,0cyclictest0-21swapper/208:11:272
260069966,0cyclictest0-21swapper/112:21:221
260069966,0cyclictest0-21swapper/112:11:301
260069964,1cyclictest22548-21cut11:45:161
260069963,2cyclictest31287-21ssh10:15:231
260069963,2cyclictest23443-21sshd09:33:321
260069963,2cyclictest19744-21ssh11:25:031
260069963,2cyclictest17688-21cat09:10:171
260069963,2cyclictest17688-21cat09:10:161
260069963,2cyclictest16781-21ssh10:25:411
260069963,2cyclictest15754-21sh11:22:091
260069963,1cyclictest0-21swapper/109:41:101
260069962,3cyclictest665-21snmpd09:22:561
260069962,3cyclictest665-21snmpd09:10:001
260069962,3cyclictest665-21snmpd08:57:291
260069962,3cyclictest665-21snmpd08:37:021
260069962,3cyclictest665-21snmpd08:25:141
260069962,3cyclictest665-21snmpd08:18:551
260069962,3cyclictest665-21snmpd07:22:001
260069962,3cyclictest6392-21expr10:20:141
260069962,3cyclictest5193-21sed10:00:201
260069962,2cyclictest283-21systemd-journal12:30:091
260069962,1cyclictest0-21swapper/111:33:401
260069962,1cyclictest0-21swapper/111:10:101
260069962,1cyclictest0-21swapper/111:01:411
260069962,1cyclictest0-21swapper/111:01:411
260069962,1cyclictest0-21swapper/110:47:291
260069962,1cyclictest0-21swapper/109:37:441
260069962,1cyclictest0-21swapper/109:27:461
260069961,3cyclictest23788-21munin-run08:15:001
260069960,6cyclictest0-21swapper/111:15:201
260069960,6cyclictest0-21swapper/111:05:361
260069960,6cyclictest0-21swapper/110:10:521
260069960,5cyclictest13319-21/usr/sbin/munin11:40:181
260019966,0cyclictest16473-21diskmemload09:57:210
260019966,0cyclictest0-21swapper/011:25:010
260019966,0cyclictest0-21swapper/009:40:310
260019963,2cyclictest9851-21sh10:21:520
260019963,2cyclictest907-21ssh10:55:150
260019963,2cyclictest665-21snmpd10:39:110
260019963,2cyclictest665-21snmpd09:01:120
260019963,2cyclictest5962-21df_inode08:45:240
260019963,2cyclictest508-21dbus-daemon09:20:390
260019963,2cyclictest28723-21sh09:36:030
260019963,2cyclictest26691-21ssh09:15:300
260019963,2cyclictest18559-21sh11:05:140
260019963,2cyclictest16473-21diskmemload09:45:080
260019962,3cyclictest8297-11brltty11:20:530
260019962,3cyclictest683-21lldpd07:28:470
260019962,3cyclictest665-21snmpd12:15:310
260019962,3cyclictest665-21snmpd12:14:130
260019962,3cyclictest665-21snmpd10:40:290
260019962,3cyclictest665-21snmpd10:30:590
260019962,3cyclictest665-21snmpd09:50:140
260019962,3cyclictest665-21snmpd09:29:590
260019962,3cyclictest665-21snmpd09:14:410
260019962,3cyclictest665-21snmpd09:14:410
260019962,3cyclictest665-21snmpd08:52:590
260019962,3cyclictest665-21snmpd08:23:440
260019962,3cyclictest665-21snmpd08:05:430
260019962,3cyclictest665-21snmpd07:39:430
260019962,3cyclictest665-21snmpd07:31:100
260019962,3cyclictest4135-21cat10:00:140
260019962,3cyclictest283-21systemd-journal09:06:330
260019962,3cyclictest283-21systemd-journal08:10:370
260019962,3cyclictest26533-21ntpq12:25:180
260019962,3cyclictest20772-21sh09:31:340
260019962,3cyclictest15687-21cat10:25:190
260019962,3cyclictest1-21systemd08:00:060
260019961,4cyclictest25176-21rm11:46:340
260019960,6cyclictest0-21swapper/011:40:470
260019960,6cyclictest0-21swapper/011:39:350
260019960,5cyclictest10131-21/usr/sbin/munin07:45:230
260019960,4cyclictest283-21systemd-journal08:40:120
260019960,0cyclictest0-21swapper/011:18:530
260149952,2cyclictest665-21snmpd09:40:023
260149952,2cyclictest665-21snmpd08:50:063
260149952,2cyclictest514-21in:imuxsock08:20:253
260149952,2cyclictest508-21dbus-daemon08:40:123
260149952,2cyclictest1634-21fw_packets07:25:153
260149952,2cyclictest15175-21expr07:55:133
260149951,3cyclictest665-21snmpd08:35:103
260149951,3cyclictest12954-21cat09:00:193
260099954,1cyclictest0-21swapper/208:45:022
260099952,2cyclictest665-21snmpd10:35:022
260099952,2cyclictest665-21snmpd10:05:042
260099952,2cyclictest31177-21grep08:30:132
260099952,2cyclictest28736-21sh07:15:012
260099952,2cyclictest26453-21dpkg08:20:002
260099952,2cyclictest19536-21gpg-connect-age08:05:002
260099952,2cyclictest17109-21apt08:00:142
260099952,1cyclictest0-21swapper/210:00:102
260099952,1cyclictest0-21swapper/209:25:112
260099951,3cyclictest665-21snmpd08:50:352
260099951,3cyclictest283-21systemd-journal07:42:152
260099951,3cyclictest1803-21cat07:25:152
260099951,3cyclictest10685-21cat08:55:152
260099951,3cyclictest10498-21expr07:45:142
260099951,2cyclictest4518-21rm11:35:052
260099950,3cyclictest283-21systemd-journal08:38:012
260069953,1cyclictest665-21snmpd07:30:061
260069952,2cyclictest9948-21ssh12:35:021
260069952,2cyclictest508-21dbus-daemon10:50:021
260069952,2cyclictest32091-21processes08:30:191
260069952,2cyclictest28723-21uname07:15:011
260069952,2cyclictest27421-21date08:20:181
260069952,2cyclictest19462-21apt-key08:05:001
260069952,2cyclictest14665-21dpkg07:55:001
260069952,2cyclictest13150-21fschecks_count10:05:161
260069951,3cyclictest8600-21wc08:50:171
260069951,3cyclictest1801-21cat07:25:151
260069951,3cyclictest12357-21/usr/sbin/munin07:50:141
260069951,3cyclictest12258-21/usr/sbin/munin09:00:181
260069951,3cyclictest1-21systemd07:48:561
260069951,1cyclictest0-21swapper/110:35:021
260069951,1cyclictest0-21swapper/109:50:181
260069950,5cyclictest0-21swapper/108:45:121
260069950,3cyclictest283-21systemd-journal10:30:111
260019953,1cyclictest25258-21swap08:15:220
260019952,2cyclictest26804-21ssh11:10:070
260019952,2cyclictest24779-21sh10:50:120
260019952,2cyclictest2063-21switchtime08:35:210
260019952,2cyclictest13598-21switchtime07:50:210
260019952,2cyclictest10509-21uniq08:55:140
260019951,3cyclictest9717-21expr11:00:140
260019951,3cyclictest9717-21expr11:00:140
260019951,3cyclictest31443-21cpuspeed_turbos07:20:140
260019951,3cyclictest15286-21grep07:55:130
260019950,4cyclictest15751-21ssh12:00:220
260019950,3cyclictest283-21systemd-journal07:10:040
260149940,3cyclictest3367-21dpkg07:30:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional