You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-07-05 - 17:48

x86 Intel Core i5-8265UE @1600 MHz, Linux 5.10.41-rt42 (Profile)

Latency plot of system in rack #a, slot #8
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackaslot8.osadl.org (updated Tue Jul 05, 2022 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
172442306279,18sleep20-21swapper/207:08:452
171632291260,21sleep00-21swapper/007:07:570
170122285254,21sleep10-21swapper/107:06:011
172752266233,21sleep30-21swapper/307:09:103
174342570,0sleep30-21swapper/309:37:403
79572560,0sleep10-21swapper/107:55:191
79572560,0sleep10-21swapper/107:55:191
169402210,0sleep10-21swapper/111:03:111
46762180,0sleep20-21swapper/211:38:082
269862180,0sleep30-21swapper/311:52:423
91792170,1sleep229741-11brltty10:36:132
191952170,0sleep00-21swapper/010:43:210
286462150,0sleep00-21swapper/009:23:350
88222140,0sleep10-21swapper/110:57:441
7772140,0sleep00-21swapper/011:14:150
51182140,0sleep10-21swapper/111:38:311
311212140,0sleep10-21swapper/109:25:121
138482140,0sleep00-21swapper/010:18:290
1744099100,3cyclictest18627-21ntp_states07:10:190
25459290,0sleep10-21swapper/110:05:011
174529997,1cyclictest8807-21dump-pmu-power08:00:013
174529997,1cyclictest8807-21dump-pmu-power08:00:013
174529994,3cyclictest283-21systemd-journal07:34:243
174529990,8cyclictest6580-21ssh11:17:463
174439992,4cyclictest17432-21latencyplot07:15:001
174409998,1cyclictest12-21ksoftirqd/012:40:000
174409994,3cyclictest283-21systemd-journal08:30:420
174529984,3cyclictest283-21systemd-journal09:47:263
174489986,2cyclictest28-21ksoftirqd/209:20:012
174489986,1cyclictest28-21ksoftirqd/210:00:022
174489986,1cyclictest28-21ksoftirqd/210:00:022
174489980,7cyclictest6094-21ssh09:51:222
174489980,5cyclictest22768-21latency_hist07:20:002
174439986,1cyclictest12304-21ssh11:21:111
174439985,2cyclictest22-21ksoftirqd/110:00:021
174439985,2cyclictest22-21ksoftirqd/110:00:011
174439980,7cyclictest19994-21turbostat08:25:001
174409987,1cyclictest12-21ksoftirqd/009:45:020
174409986,1cyclictest28231-21fschecks_count12:15:150
174409984,3cyclictest283-21systemd-journal10:36:140
174409983,3cyclictest283-21systemd-journal12:22:080
174409983,3cyclictest24234-21turbostat11:30:000
19673270,0sleep10-21swapper/109:17:151
174529977,0cyclictest341ksoftirqd/308:35:013
174529976,1cyclictest7893-21diskmemload12:05:313
174529974,2cyclictest283-21systemd-journal09:18:593
174529973,3cyclictest7590-21lldpd08:47:413
174529973,3cyclictest283-21systemd-journal10:53:443
174529973,3cyclictest283-21systemd-journal09:30:203
174529972,3cyclictest283-21systemd-journal11:59:123
174489974,2cyclictest508-21dbus-daemon08:49:282
174489973,3cyclictest10103-21sed08:00:172
174489973,2cyclictest283-21systemd-journal11:49:272
174489973,2cyclictest283-21systemd-journal11:49:262
174439975,1cyclictest22764-21latency_hist07:20:021
174439974,2cyclictest508-21dbus-daemon12:19:371
174439974,2cyclictest508-21dbus-daemon09:36:101
174439974,2cyclictest3420-21idleruntime-cro09:00:001
174439973,3cyclictest508-21dbus-daemon09:35:001
174439973,3cyclictest283-21systemd-journal12:22:321
174439973,3cyclictest283-21systemd-journal11:31:401
174439973,3cyclictest26649-21memory10:05:171
174439973,3cyclictest16597-21snmpd10:27:141
174439973,3cyclictest16597-21snmpd08:37:261
174439971,2cyclictest508-21dbus-daemon07:40:011
174439970,7cyclictest24676-21sshd12:34:341
174409976,1cyclictest121ksoftirqd/012:00:000
174409975,1cyclictest16597-21snmpd09:57:300
174409975,1cyclictest16597-21snmpd09:57:300
174409974,2cyclictest19851-21sshd08:23:480
174409973,3cyclictest0-21swapper/009:18:150
174409973,1cyclictest1197-21turbostat08:55:010
174409971,3cyclictest1732-21rm09:26:490
174409971,1cyclictest141rcu_preempt11:15:110
26537260,0sleep30-21swapper/307:25:183
174529966,0cyclictest0-21swapper/312:32:413
174529966,0cyclictest0-21swapper/311:00:403
174529963,2cyclictest508-21dbus-daemon10:19:343
174529963,2cyclictest4323-21ssh10:33:333
174529963,2cyclictest28515-21cat12:15:153
174529963,2cyclictest283-21systemd-journal12:20:023
174529963,2cyclictest283-21systemd-journal08:14:513
174529963,2cyclictest26221-21ssh10:26:083
174529963,2cyclictest2042-21ssh09:27:073
174529963,2cyclictest16597-21snmpd10:20:453
174529963,2cyclictest16597-21snmpd07:22:173
174529963,2cyclictest12251-21sh12:25:273
174529963,2cyclictest11900-21sh09:55:193
174529963,2cyclictest11900-21sh09:55:193
174529963,2cyclictest10723-21seq09:10:533
174529963,1cyclictest0-21swapper/311:45:313
174529963,1cyclictest0-21swapper/311:45:313
174529962,3cyclictest440-21wc08:50:193
174529962,3cyclictest2878-21sed08:55:183
174529962,3cyclictest283-21systemd-journal10:13:433
174529962,3cyclictest283-21systemd-journal08:37:373
174529962,3cyclictest283-21systemd-journal07:38:403
174529962,3cyclictest19666-21http07:15:003
174529962,3cyclictest18951-21ntpq08:20:163
174529962,3cyclictest18771-21cut10:00:183
174529962,3cyclictest16597-21snmpd11:25:203
174529962,3cyclictest16597-21snmpd10:37:453
174529962,3cyclictest16597-21snmpd10:08:113
174529962,3cyclictest16597-21snmpd09:53:013
174529962,3cyclictest16597-21snmpd09:08:023
174529962,3cyclictest16597-21snmpd08:41:573
174529962,3cyclictest16597-21snmpd08:06:133
174529962,3cyclictest16597-21snmpd08:02:223
174529962,3cyclictest14490-21cut10:40:153
174529962,3cyclictest1-21systemd09:20:453
174529962,3cyclictest0-21swapper/311:34:093
174529962,1cyclictest0-21swapper/311:20:033
174529962,1cyclictest0-21swapper/310:55:143
174529962,1cyclictest0-21swapper/310:46:243
174529962,1cyclictest0-21swapper/309:41:423
174529961,3cyclictest283-21systemd-journal11:38:463
174529961,3cyclictest283-21systemd-journal11:12:313
174529961,3cyclictest283-21systemd-journal11:06:393
174529961,3cyclictest283-21systemd-journal09:04:033
174529960,6cyclictest0-21swapper/312:03:503
174529960,5cyclictest0-21swapper/312:37:083
174529960,0cyclictest0-21swapper/312:10:343
174529960,0cyclictest0-21swapper/311:40:293
174489966,0cyclictest0-21swapper/212:14:372
174489966,0cyclictest0-21swapper/211:25:182
174489966,0cyclictest0-21swapper/209:21:472
174489963,2cyclictest668-21runrttasks10:25:262
174489963,2cyclictest5698-21ssh09:30:062
174489963,2cyclictest492-21ssh10:52:082
174489963,2cyclictest31533-21ssh09:46:492
174489963,2cyclictest29741-11brltty07:45:142
174489963,2cyclictest283-21systemd-journal07:23:332
174489963,2cyclictest20794-21cut09:40:132
174489963,2cyclictest12918-21tr12:05:122
174489963,2cyclictest0-21swapper/208:35:002
174489963,1cyclictest0-21swapper/212:30:142
174489963,1cyclictest0-21swapper/212:26:402
174489963,1cyclictest0-21swapper/209:35:362
174489962,3cyclictest8206-21wc11:40:172
174489962,3cyclictest32690-21expr07:40:122
174489962,3cyclictest29741-11brltty11:17:052
174489962,3cyclictest283-21systemd-journal11:56:422
174489962,3cyclictest283-21systemd-journal11:50:272
174489962,3cyclictest283-21systemd-journal08:20:372
174489962,3cyclictest27114-21cat11:10:112
174489962,3cyclictest26962-21sendmail-msp08:40:002
174489962,3cyclictest25421-21seq10:47:082
174489962,3cyclictest16597-21snmpd12:23:202
174489962,3cyclictest16597-21snmpd12:01:162
174489962,3cyclictest16597-21snmpd11:31:102
174489962,3cyclictest16597-21snmpd11:22:132
174489962,3cyclictest16597-21snmpd11:01:222
174489962,3cyclictest16597-21snmpd10:15:562
174489962,3cyclictest16597-21snmpd10:06:122
174489962,3cyclictest16597-21snmpd10:01:122
174489962,3cyclictest16597-21snmpd09:02:292
174489962,3cyclictest16597-21snmpd08:11:342
174489962,3cyclictest16597-21snmpd07:50:482
174489962,3cyclictest16597-21snmpd07:10:022
174489962,3cyclictest1-21systemd10:22:532
174489962,1cyclictest0-21swapper/212:15:072
174489962,1cyclictest0-21swapper/210:31:212
174489961,4cyclictest17341-21sshd08:15:372
174489960,6cyclictest0-21swapper/209:10:212
174489960,6cyclictest0-21swapper/208:09:572
174489960,3cyclictest7903-21awk09:10:012
174489960,3cyclictest19478-21/usr/sbin/munin11:05:212
174439963,2cyclictest508-21dbus-daemon11:11:401
174439963,2cyclictest508-21dbus-daemon08:18:121
174439963,2cyclictest32077-21sh10:51:221
174439963,2cyclictest29741-11brltty10:40:241
174439963,2cyclictest283-21systemd-journal07:51:551
174439963,2cyclictest26276-21chrt09:21:271
174439963,2cyclictest24690-21idleruntime-cro08:35:001
174439963,2cyclictest22931-21sh11:50:121
174439963,2cyclictest16597-21snmpd10:35:101
174439963,2cyclictest16597-21snmpd10:18:111
174439963,2cyclictest11736-21in:imuxsock07:45:371
174439963,2cyclictest11210-21ssh12:04:071
174439962,3cyclictest502-21systemd-logind11:28:201
174439962,3cyclictest32036-21df08:50:121
174439962,3cyclictest16597-21snmpd12:35:331
174439962,3cyclictest16597-21snmpd11:40:051
174439962,3cyclictest16597-21snmpd11:17:521
174439962,3cyclictest16597-21snmpd10:47:101
174439962,3cyclictest16597-21snmpd09:50:021
174439962,3cyclictest16597-21snmpd09:46:381
174439962,3cyclictest16597-21snmpd09:00:511
174439962,3cyclictest16597-21snmpd08:46:291
174439962,3cyclictest16597-21snmpd08:07:171
174439962,3cyclictest10030-21cat08:00:161
174439962,1cyclictest0-21swapper/112:13:271
174439962,1cyclictest0-21swapper/111:07:161
174439962,1cyclictest0-21swapper/110:10:341
174439962,1cyclictest0-21swapper/109:40:121
174439961,4cyclictest25249-21apt-get07:25:111
174439961,4cyclictest16597-21snmpd12:09:051
174439960,6cyclictest0-21swapper/111:47:401
174439960,6cyclictest0-21swapper/111:47:401
174439960,6cyclictest0-21swapper/110:20:121
174439960,6cyclictest0-21swapper/109:11:591
174439960,4cyclictest15507-21basename08:15:021
174439960,0cyclictest0-21swapper/111:57:021
174409964,1cyclictest12-21ksoftirqd/011:50:010
174409963,2cyclictest993-21ssh09:48:280
174409963,2cyclictest6791-21cat12:00:240
174409963,2cyclictest508-21dbus-daemon07:36:290
174409963,2cyclictest31748-21ssh11:35:010
174409963,2cyclictest283-21systemd-journal07:29:550
174409963,2cyclictest26033-21ssh10:25:590
174409963,2cyclictest25635-21sh10:05:060
174409963,2cyclictest19162-21ssh09:39:110
174409963,2cyclictest17241-21tr10:20:190
174409963,2cyclictest14993-21cat11:45:160
174409963,2cyclictest14993-21cat11:45:160
174409963,2cyclictest14433-21ssh11:01:010
174409963,2cyclictest12977-21expr12:05:120
174409963,2cyclictest11736-21in:imuxsock11:21:150
174409963,2cyclictest10874-21ssh11:42:010
174409963,2cyclictest0-21swapper/009:10:370
174409962,3cyclictest508-21dbus-daemon09:35:010
174409962,3cyclictest508-21dbus-daemon09:02:490
174409962,3cyclictest29579-21sshd07:33:400
174409962,3cyclictest283-21systemd-journal09:06:090
174409962,3cyclictest22128-21cat07:15:200
174409962,3cyclictest16597-21snmpd11:37:350
174409962,3cyclictest16597-21snmpd10:30:270
174409962,3cyclictest16597-21snmpd08:56:300
174409962,3cyclictest16597-21snmpd08:44:050
174409962,3cyclictest16597-21snmpd08:39:290
174409962,3cyclictest16597-21snmpd08:18:430
174409962,3cyclictest16597-21snmpd08:12:340
174409962,3cyclictest16597-21snmpd08:07:080
174409962,1cyclictest0-21swapper/010:55:380
174409962,1cyclictest0-21swapper/010:45:160
174409961,3cyclictest283-21systemd-journal09:50:460
174409961,3cyclictest283-21systemd-journal08:46:340
174409960,6cyclictest77-21kswapd012:25:160
174409960,0cyclictest0-21swapper/012:10:030
174529952,2cyclictest5280-21cat07:50:153
174529952,2cyclictest32299-21cat07:40:113
174529952,2cyclictest2332-21seq07:45:133
174529952,2cyclictest20474-21/usr/sbin/munin08:25:133
174529951,3cyclictest21206-21tune2fs07:15:133
174529951,3cyclictest15726-21/usr/sbin/munin08:15:173
174489953,1cyclictest28085-21expr07:30:122
174489953,1cyclictest0-21swapper/210:10:122
174489953,1cyclictest0-21swapper/207:25:122
174489952,2cyclictest7037-21cat07:55:132
174489952,2cyclictest7037-21cat07:55:132
174489952,2cyclictest5027-21gltestperf10:55:182
174489952,2cyclictest31944-21expr08:50:122
174489952,2cyclictest31096-21expr09:25:112
174489952,2cyclictest27862-21cat08:40:142
174489952,2cyclictest20511-21cpu08:25:122
174489952,2cyclictest1931-21uniq08:55:122
174489952,2cyclictest15288-21awk10:40:212
174489951,3cyclictest30938-21ntpq07:35:182
174489951,3cyclictest16597-21snmpd12:35:352
174439952,2cyclictest6303-21sh09:05:131
174439952,2cyclictest32092-21cat10:30:151
174439952,2cyclictest28106-21expr07:30:111
174439952,2cyclictest27592-21entropy08:40:131
174439952,2cyclictest16597-21snmpd08:25:041
174439952,2cyclictest10200-21sh12:25:041
174439951,3cyclictest32294-21/usr/sbin/munin07:40:161
174439951,3cyclictest16597-21snmpd07:20:031
174409953,1cyclictest21757-21ls08:25:190
174409952,2cyclictest6960-21grep07:55:140
174409952,2cyclictest6960-21grep07:55:140
174409952,2cyclictest4557-21gdbus07:50:010
174409952,2cyclictest29186-21dpkg10:50:110
174409952,2cyclictest24316-21threads07:20:200
174409952,2cyclictest19830-21chrt11:05:180
174409952,2cyclictest19790-21timerandwakeup10:00:260
174409952,2cyclictest1188-21expr07:40:180
174409952,1cyclictest0-21swapper/012:30:020
174409952,1cyclictest0-21swapper/010:10:200
174409951,3cyclictest10043-21ntp_states08:00:180
174409951,2cyclictest283-21systemd-journal07:45:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional