You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-09-12 - 10:13
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8.osadl.org (updated Thu Sep 12, 2024 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
293582296263,20sleep20-21swapper/219:07:482
293622274210,21sleep10-21swapper/119:07:511
292252264229,22sleep30-21swapper/319:06:123
291552263230,21sleep00-21swapper/019:05:260
294712640,0sleep30-21swapper/323:14:483
254422590,0sleep00-21swapper/023:57:170
287932580,0sleep30-21swapper/322:50:213
41912560,0sleep30-21swapper/322:55:533
23912560,0sleep20-21swapper/222:55:132
195572560,0sleep30-21swapper/323:07:063
148742540,0sleep00-21swapper/023:26:300
319882530,0sleep20-21swapper/223:39:332
68712180,0sleep30-21swapper/321:48:103
102032170,0sleep10-21swapper/100:11:031
221462150,0sleep30-21swapper/300:21:353
156902140,0sleep10-21swapper/121:55:141
138962130,0sleep00-21swapper/023:50:010
314182100,0sleep20-21swapper/221:41:522
296609995,3cyclictest19982-21systemd-journal21:58:003
296609994,3cyclictest19982-21systemd-journal19:44:553
296589995,2cyclictest19982-21systemd-journal22:23:282
296529997,2cyclictest22-21ksoftirqd/119:40:001
296489990,8cyclictest17372-21ssh23:51:250
296609986,2cyclictest34-21ksoftirqd/300:00:003
296609984,3cyclictest19982-21systemd-journal21:13:403
296609983,3cyclictest508-21dbus-daemon23:00:353
296609983,3cyclictest19982-21systemd-journal19:35:013
296589984,3cyclictest19982-21systemd-journal23:18:312
296589984,3cyclictest19982-21systemd-journal19:52:002
296589983,3cyclictest19982-21systemd-journal21:15:512
296589983,3cyclictest19982-21systemd-journal20:16:122
296589983,1cyclictest0-21swapper/200:02:402
296529986,1cyclictest12903-21sh23:25:151
296529984,3cyclictest3108-21munin-run22:10:011
296529983,3cyclictest19982-21systemd-journal22:55:101
296529983,3cyclictest19982-21systemd-journal20:49:351
296529983,3cyclictest19982-21systemd-journal00:20:151
296489987,1cyclictest12-21ksoftirqd/022:00:020
296489984,3cyclictest508-21dbus-daemon22:45:020
296489984,3cyclictest19982-21systemd-journal19:34:200
296489983,3cyclictest19982-21systemd-journal21:39:400
29340280,0sleep00-21swapper/023:36:590
296609975,1cyclictest34-21ksoftirqd/322:15:013
296609975,1cyclictest34-21ksoftirqd/321:10:003
296609974,2cyclictest34-21ksoftirqd/319:15:013
296609974,2cyclictest11736-21in:imuxsock23:47:183
296609974,2cyclictest11736-21in:imuxsock00:15:003
296609973,3cyclictest7743-21ntpq22:35:163
296609973,3cyclictest30654-21snmpd22:20:113
296609973,3cyclictest27735-21apt-get20:15:123
296609973,3cyclictest19982-21systemd-journal19:16:013
296609973,3cyclictest1752-21cron20:30:003
296609973,3cyclictest13643-21users23:25:193
296609973,3cyclictest1-21systemd21:03:453
296609973,3cyclictest11938-21ntpq20:50:173
296609973,2cyclictest19982-21systemd-journal00:39:533
296609973,1cyclictest0-21swapper/300:30:113
296609972,3cyclictest19982-21systemd-journal22:09:413
296589974,2cyclictest9061-21ls20:45:122
296589974,2cyclictest508-21dbus-daemon19:55:212
296589974,2cyclictest29741-11brltty00:11:312
296589974,2cyclictest11736-21in:imuxsock22:49:442
296589973,3cyclictest508-21dbus-daemon00:20:012
296589973,3cyclictest19982-21systemd-journal23:25:002
296589973,3cyclictest19982-21systemd-journal21:25:082
296589973,3cyclictest0-21swapper/223:57:102
296589973,3cyclictest0-21swapper/200:05:372
296589973,1cyclictest0-21swapper/223:47:012
296589973,1cyclictest0-21swapper/223:29:292
296589973,1cyclictest0-21swapper/222:35:332
296589973,1cyclictest0-21swapper/222:14:482
296589973,1cyclictest0-21swapper/200:35:552
296589972,3cyclictest19982-21systemd-journal22:31:232
296589972,3cyclictest19982-21systemd-journal21:50:002
296589972,1cyclictest0-21swapper/222:51:032
296589970,4cyclictest31762-21avahi-daemon21:40:002
296529974,2cyclictest11736-21in:imuxsock20:11:111
296529973,3cyclictest4931-21cat20:35:141
296529973,3cyclictest30654-21snmpd23:01:451
296529973,3cyclictest30654-21snmpd22:10:331
296529973,3cyclictest29741-11brltty20:18:421
296529973,3cyclictest19982-21systemd-journal21:51:211
296529973,3cyclictest19982-21systemd-journal20:07:261
296529973,1cyclictest141rcu_preempt21:40:011
296529973,1cyclictest0-21swapper/123:08:101
296529971,4cyclictest19982-21systemd-journal00:25:161
296489975,2cyclictest12-21ksoftirqd/020:00:000
296489975,1cyclictest4643-21diskstats20:35:140
296489974,2cyclictest502-21systemd-logind00:11:550
296489974,2cyclictest2424-21sh00:32:460
296489974,2cyclictest19982-21systemd-journal23:32:090
296489973,3cyclictest584-21memory20:25:170
296489973,3cyclictest508-21dbus-daemon20:40:490
296489973,3cyclictest508-21dbus-daemon19:44:560
296489973,3cyclictest31071-21smart_sda19:10:180
296489973,3cyclictest30654-21snmpd22:09:300
296489973,3cyclictest30654-21snmpd20:46:100
296489973,3cyclictest2546-21apt-get00:05:110
296489973,3cyclictest20253-21diskmemload21:42:280
296489973,3cyclictest17141-21cron22:20:010
296489973,3cyclictest15699-21packagekitd21:00:010
296489973,3cyclictest0-21swapper/023:19:020
296489973,3cyclictest0-21swapper/023:03:220
296489973,3cyclictest0-21swapper/022:32:580
296489973,3cyclictest0-21swapper/022:27:540
296489973,3cyclictest0-21swapper/022:00:250
296489973,3cyclictest0-21swapper/021:19:480
296489973,3cyclictest0-21swapper/000:15:560
296489973,1cyclictest0-21swapper/023:14:420
296489973,1cyclictest0-21swapper/022:56:490
296489973,1cyclictest0-21swapper/022:51:060
21652270,0sleep00-21swapper/020:00:150
296609966,0cyclictest0-21swapper/300:26:233
296609966,0cyclictest0-21swapper/300:02:043
296609964,1cyclictest23649-21df_inode20:05:133
296609963,2cyclictest30241-21cpuspeed_turbos23:15:123
296609963,2cyclictest29191-21ssh23:36:493
296609963,2cyclictest23366-21processes22:00:193
296609963,2cyclictest21228-21latency_hist23:55:003
296609963,2cyclictest18909-21sh22:43:533
296609963,2cyclictest13642-21tune2fs20:55:123
296609963,2cyclictest11736-21in:imuxsock19:45:203
296609963,1cyclictest0-21swapper/300:06:143
296609962,3cyclictest9612-21grep21:50:153
296609962,3cyclictest6731-21cat20:40:123
296609962,3cyclictest508-21dbus-daemon23:24:223
296609962,3cyclictest370-21if_enp4s022:30:163
296609962,3cyclictest30654-21snmpd22:18:053
296609962,3cyclictest30654-21snmpd21:41:063
296609962,3cyclictest30654-21snmpd21:16:263
296609962,3cyclictest30654-21snmpd20:21:403
296609962,3cyclictest27501-21ls20:15:013
296609962,3cyclictest21037-21awk20:00:123
296609962,3cyclictest19982-21systemd-journal19:57:043
296609962,3cyclictest10237-21df_inode19:35:133
296609962,1cyclictest0-21swapper/322:45:243
296609962,1cyclictest0-21swapper/321:30:493
296609962,1cyclictest0-21swapper/321:20:373
296609962,1cyclictest0-21swapper/300:15:193
296609961,4cyclictest2558-21apt-get19:20:143
296609961,3cyclictest19982-21systemd-journal23:40:383
296609961,3cyclictest19982-21systemd-journal19:52:503
296609960,5cyclictest8505-21grep21:25:133
296609960,4cyclictest19982-21systemd-journal23:31:273
296609960,2cyclictest19982-21systemd-journal22:26:183
296589966,0cyclictest0-21swapper/222:00:382
296589964,1cyclictest23473-21df_inode23:10:162
296589963,2cyclictest31595-21sh22:06:132
296589963,2cyclictest11736-21in:imuxsock20:40:462
296589962,3cyclictest5748-21cat19:25:162
296589962,3cyclictest30654-21snmpd23:50:032
296589962,3cyclictest30654-21snmpd22:41:162
296589962,3cyclictest30654-21snmpd21:31:172
296589962,3cyclictest30654-21snmpd21:00:322
296589962,3cyclictest30654-21snmpd20:32:272
296589962,3cyclictest30654-21snmpd20:29:392
296589962,3cyclictest30654-21snmpd20:02:002
296589962,3cyclictest30654-21snmpd19:45:112
296589962,3cyclictest30654-21snmpd19:20:132
296589962,3cyclictest30654-21snmpd19:18:322
296589962,3cyclictest24202-21ntpq20:05:162
296589962,3cyclictest12664-21sed19:40:152
296589962,3cyclictest1-21systemd23:30:372
296589962,3cyclictest0-21swapper/223:43:392
296589962,3cyclictest0-21swapper/223:05:152
296589962,3cyclictest0-21swapper/200:29:112
296589962,2cyclictest19982-21systemd-journal19:31:092
296589962,1cyclictest0-21swapper/222:25:212
296589962,1cyclictest0-21swapper/222:18:452
296589962,1cyclictest0-21swapper/221:51:252
296589962,1cyclictest0-21swapper/221:11:062
296589962,1cyclictest0-21swapper/200:20:542
296589961,3cyclictest19982-21systemd-journal20:20:002
296589961,3cyclictest19982-21systemd-journal19:35:192
296589961,2cyclictest2639-21ssh21:20:152
296589960,5cyclictest15638-21/usr/sbin/munin21:55:142
296589960,2cyclictest668-21runrttasks23:00:132
296589960,0cyclictest0-21swapper/200:30:232
296529966,0cyclictest0-21swapper/123:45:071
296529964,1cyclictest28102-21sed22:50:171
296529964,1cyclictest20253-21diskmemload22:45:131
296529964,1cyclictest15022-21irqstats19:45:151
296529963,2cyclictest8175-21rm21:25:011
296529963,2cyclictest6015-21systemctl23:20:181
296529963,2cyclictest5074-21ssh21:46:201
296529963,2cyclictest502-21systemd-logind21:21:421
296529963,2cyclictest31211-21rm00:01:101
296529963,2cyclictest30654-21snmpd23:13:111
296529963,2cyclictest30654-21snmpd00:30:071
296529963,2cyclictest28852-21systemctl21:15:181
296529963,2cyclictest23120-21needreboot22:00:181
296529963,2cyclictest19919-21uname23:30:171
296529963,2cyclictest16252-21perl21:00:151
296529963,2cyclictest11736-21in:imuxsock20:52:301
296529963,2cyclictest11736-21in:imuxsock00:35:091
296529962,3cyclictest899-21hwlatdetect19:15:141
296529962,3cyclictest7172-21cat20:40:151
296529962,3cyclictest508-21dbus-daemon19:57:011
296529962,3cyclictest508-21dbus-daemon19:20:181
296529962,3cyclictest3752-21find00:05:181
296529962,3cyclictest30654-21snmpd23:55:281
296529962,3cyclictest30654-21snmpd22:25:131
296529962,3cyclictest30654-21snmpd22:21:401
296529962,3cyclictest30654-21snmpd20:28:541
296529962,3cyclictest2955-21ntp_kernel_pll_20:30:161
296529962,3cyclictest28194-21apt-get21:40:121
296529962,3cyclictest18950-21cat21:05:141
296529962,3cyclictest15086-21fschecks_time23:50:131
296529962,3cyclictest14095-21expr00:15:121
296529962,3cyclictest13795-21expr22:40:121
296529962,2cyclictest502-21systemd-logind19:32:241
296529962,1cyclictest0-21swapper/123:42:311
296529962,1cyclictest0-21swapper/123:35:481
296529962,1cyclictest0-21swapper/122:35:571
296529961,4cyclictest0-21swapper/121:31:431
296529961,3cyclictest19982-21systemd-journal19:51:111
296529960,5cyclictest31837-21munin-run19:15:011
296529960,5cyclictest14403-21smart_sda20:55:171
296529960,2cyclictest7661-21lldpd23:17:521
296529960,0cyclictest0-21swapper/122:19:131
296489963,2cyclictest7661-21lldpd21:27:000
296489963,2cyclictest28221-21cut20:15:130
296489963,2cyclictest23242-21ssh21:10:550
296489963,2cyclictest2250-21rm23:40:250
296489963,2cyclictest2116-21taskset19:18:190
296489963,2cyclictest16561-21apt-get19:50:110
296489963,2cyclictest11736-21in:imuxsock19:26:500
296489962,3cyclictest6940-21cat22:35:130
296489962,3cyclictest4757-21processes22:10:190
296489962,3cyclictest4000-21munin-run20:35:010
296489962,3cyclictest30654-21snmpd20:51:580
296489962,3cyclictest30654-21snmpd20:23:330
296489962,3cyclictest30654-21snmpd20:06:470
296489962,3cyclictest29057-21expr00:00:110
296489962,3cyclictest20850-21cat00:20:170
296489962,3cyclictest16759-21if_enp4s023:05:150
296489962,3cyclictest13941-21cat20:55:130
296489962,3cyclictest11314-21cat22:15:140
296489962,3cyclictest0-21swapper/022:45:030
296489962,3cyclictest0-21swapper/021:46:250
296489962,1cyclictest0-21swapper/023:20:380
296489962,1cyclictest0-21swapper/000:35:240
296489961,4cyclictest26174-21rm00:25:160
296489961,4cyclictest19982-21systemd-journal21:53:260
296489961,3cyclictest19982-21systemd-journal19:24:280
296489960,5cyclictest2345-21gltestperf21:20:140
296489960,3cyclictest15120-21ssh21:30:130
296609952,2cyclictest4905-21cmp19:25:013
296609952,2cyclictest2522-21cut20:30:133
296609952,2cyclictest22048-21df21:35:133
296609951,3cyclictest30654-21snmpd20:37:033
296609950,1cyclictest8742-21apt-get20:45:113
296589955,0cyclictest0-21swapper/220:35:132
296589952,2cyclictest30112-21expr19:10:122
296589952,2cyclictest13991-21cut20:55:152
296589951,3cyclictest30654-21snmpd20:10:062
296589951,3cyclictest18224-21gdbus21:05:022
296589950,1cyclictest10943-21/usr/sbin/munin20:50:152
296529952,2cyclictest30654-21snmpd21:10:011
296529952,2cyclictest30212-21expr20:20:131
296529951,3cyclictest21333-21expr20:00:131
296529951,3cyclictest12540-21fschecks_count19:40:151
296529950,3cyclictest19982-21systemd-journal22:30:091
296529950,3cyclictest19982-21systemd-journal19:25:001
296489952,2cyclictest9911-21cpuspeed19:35:130
296489952,2cyclictest19982-21systemd-journal20:10:080
296489952,2cyclictest18827-21grep21:05:130
296489952,2cyclictest14732-21forks19:45:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional