You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-04 - 02:17
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot3.osadl.org (updated Fri Jul 04, 2025 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
297427999140,12cyclictest0-21swapper/321:55:143
2974279991312,0cyclictest0-21swapper/319:15:123
2974279991311,0cyclictest0-21swapper/320:05:003
297427999130,12cyclictest0-21swapper/300:10:143
297427999130,11cyclictest0-21swapper/321:15:153
297427999129,1cyclictest0-21swapper/323:45:123
2974279991211,0cyclictest0-21swapper/323:00:143
297427999120,1cyclictest0-21swapper/321:10:153
297427999120,11cyclictest0-21swapper/323:55:203
297427999120,11cyclictest0-21swapper/323:35:143
297427999120,11cyclictest0-21swapper/323:10:133
297427999120,11cyclictest0-21swapper/322:40:173
297427999120,11cyclictest0-21swapper/300:05:133
2974279991111,0cyclictest0-21swapper/322:30:013
297427999110,8cyclictest0-21swapper/322:15:203
297427999110,10cyclictest0-21swapper/323:05:133
297427999110,10cyclictest0-21swapper/321:40:203
297427999110,10cyclictest0-21swapper/321:35:203
297427999110,0cyclictest0-21swapper/323:50:113
297427999110,0cyclictest0-21swapper/320:40:133
297427999109,1cyclictest0-21swapper/322:00:143
297427999109,0cyclictest0-21swapper/320:30:133
2974279991010,0cyclictest0-21swapper/322:45:143
2974279991010,0cyclictest0-21swapper/320:35:143
2974279991010,0cyclictest0-21swapper/320:19:553
2974279991010,0cyclictest0-21swapper/300:00:143
297427999100,9cyclictest0-21swapper/322:10:193
297427999100,9cyclictest0-21swapper/321:20:203
29742799999,0cyclictest0-21swapper/323:40:153
29742799999,0cyclictest0-21swapper/322:35:183
29742799999,0cyclictest0-21swapper/321:50:193
29742799999,0cyclictest0-21swapper/321:25:153
29742799999,0cyclictest0-21swapper/320:45:273
29742799999,0cyclictest0-21swapper/320:25:013
29742799999,0cyclictest0-21swapper/319:35:193
29742799999,0cyclictest0-21swapper/319:10:133
29742799999,0cyclictest0-21swapper/300:30:143
29742799999,0cyclictest0-21swapper/300:15:173
29742799990,8cyclictest0-21swapper/323:17:043
29742799990,0cyclictest0-21swapper/300:20:143
29742799988,0cyclictest0-21swapper/323:20:213
29742799988,0cyclictest0-21swapper/322:55:153
29742799988,0cyclictest0-21swapper/319:50:003
29742799988,0cyclictest0-21swapper/319:30:143
29742799988,0cyclictest0-21swapper/319:25:163
29742799987,0cyclictest0-21swapper/300:25:193
29742799982,1cyclictest17350irq/126-ahci[0000:00:17.0]23:25:103
29742799977,0cyclictest0-21swapper/322:50:213
29742799977,0cyclictest0-21swapper/322:25:023
29742799977,0cyclictest0-21swapper/320:50:183
29742799977,0cyclictest0-21swapper/319:40:153
29742799977,0cyclictest0-21swapper/300:35:143
29742799971,2cyclictest0-21swapper/320:10:153
29742799970,5cyclictest0-21swapper/323:30:123
2972489271,4sleep30-21swapper/319:05:143
3061036260,0sleep03061051-21munin-run22:45:010
29742799966,0cyclictest0-21swapper/321:30:133
29742799966,0cyclictest0-21swapper/320:55:183
29742799966,0cyclictest0-21swapper/319:20:183
29742799965,0cyclictest0-21swapper/320:05:143
29742799964,1cyclictest30190732chrt21:04:143
29742799964,1cyclictest0-21swapper/320:25:133
29742799963,3cyclictest0-21swapper/322:32:173
29742799963,1cyclictest0-21swapper/322:05:133
29742799960,5cyclictest0-21swapper/319:50:133
13989860,3rtkit-daemon762-21ntpd19:05:020
3058786253,2sleep1760-21snmpd22:38:511
29742799955,0cyclictest0-21swapper/321:45:123
29742799955,0cyclictest0-21swapper/319:55:173
29742799953,0cyclictest0-21swapper/321:05:143
3102920240,0sleep00-21swapper/000:21:590
2973438241,1sleep20-21swapper/219:05:212
13989840,2rtkit-daemon614-21wpa_supplicant23:42:012
13989840,2rtkit-daemon614-21wpa_supplicant23:33:012
13989840,2rtkit-daemon614-21wpa_supplicant21:36:350
13989840,2rtkit-daemon2976281-21munin-node21:52:160
13989840,2rtkit-daemon2974273-21cyclictest20:30:010
13989840,2rtkit-daemon1-21systemd22:51:072
13989840,2rtkit-daemon1-21systemd22:29:072
13989840,2rtkit-daemon1-21systemd21:18:540
3093046230,1sleep03093072-21apt-get00:00:150
3075983230,1sleep23076009-21apt-get23:20:132
3065477230,1sleep13065503-21apt-get22:55:151
2972836231,0sleep10-21swapper/119:05:161
13989830,2rtkit-daemon762-21ntpd19:42:150
13989830,2rtkit-daemon760-21snmpd22:12:380
13989830,2rtkit-daemon614-21wpa_supplicant00:27:332
13989830,2rtkit-daemon3021060-21diskmemload22:01:100
13989830,2rtkit-daemon1399-21sendmail-mta22:38:142
13989830,2rtkit-daemon1399-21sendmail-mta22:20:480
13989830,2rtkit-daemon1-21systemd23:26:042
13989830,1rtkit-daemon762-21ntpd19:24:110
13989830,1rtkit-daemon760-21snmpd00:39:422
13989830,1rtkit-daemon614-21wpa_supplicant21:42:450
13989830,1rtkit-daemon331-21plymouthd22:25:020
13989830,1rtkit-daemon331-21plymouthd21:57:430
13989830,1rtkit-daemon331-21plymouthd21:27:230
13989830,1rtkit-daemon331-21plymouthd00:17:062
13989830,1rtkit-daemon3013374-21latency_hist20:50:010
13989830,1rtkit-daemon2983481-21processes19:30:210
13989830,1rtkit-daemon2976281-21munin-node23:50:192
13989830,1rtkit-daemon2976281-21munin-node23:35:532
13989830,1rtkit-daemon2976281-21munin-node22:05:290
13989830,1rtkit-daemon1-21systemd23:55:192
13989830,1rtkit-daemon1-21systemd23:49:272
13989830,1rtkit-daemon1-21systemd23:10:222
13989830,1rtkit-daemon1-21systemd22:56:272
13989830,1rtkit-daemon1-21systemd22:31:012
13989830,1rtkit-daemon1-21systemd20:55:060
13989830,1rtkit-daemon1-21systemd19:45:240
13989830,1rtkit-daemon1-21systemd19:26:230
13989830,1rtkit-daemon1-21systemd00:20:302
13989830,1rtkit-daemon1-21systemd00:04:302
13989830,0rtkit-daemon331-21plymouthd00:30:492
13989830,0rtkit-daemon0-21swapper/022:16:220
13989830,0rtkit-daemon0-21swapper/019:15:030
3104998220,1sleep00-21swapper/000:26:530
3096785220,0sleep00-21swapper/000:08:220
3092512220,0sleep00-21swapper/023:57:540
3081224220,1sleep03081469-21users23:30:210
3077641220,0sleep00-21swapper/023:23:280
3070992220,1sleep00-21swapper/023:05:230
3048321220,1sleep10-21swapper/122:14:051
3026253220,1sleep23026383-21/usr/sbin/munin21:20:222
3025114220,1sleep13025132-21systemctl21:20:021
3016380220,1sleep23016505-21python320:55:202
3002553220,1sleep13002660-21latency20:20:171
3000315220,1sleep13000580-21diskstats20:15:171
2998854220,1sleep02999008-21ntp_states20:10:200
2997984220,1sleep12998002-21/usr/sbin/munin20:10:001
2992904220,1sleep12993068-21cat19:55:161
2990977220,1sleep22991054-21interrupts19:50:162
13989820,1rtkit-daemon762-21ntpd23:00:202
13989820,1rtkit-daemon762-21ntpd21:00:120
13989820,1rtkit-daemon762-21ntpd00:10:112
13989820,1rtkit-daemon760-21snmpd23:15:042
13989820,1rtkit-daemon760-21snmpd20:20:570
13989820,1rtkit-daemon684-21gmain23:05:062
13989820,1rtkit-daemon684-21gmain20:30:340
13989820,1rtkit-daemon614-21wpa_supplicant20:15:410
13989820,1rtkit-daemon2976281-21munin-node21:10:030
13989820,1rtkit-daemon2976281-21munin-node20:00:360
13989820,1rtkit-daemon2976281-21munin-node19:52:460
13989820,1rtkit-daemon1-21systemd22:45:482
13989820,1rtkit-daemon1-21systemd00:06:022
13989820,0rtkit-daemon331-21plymouthd22:40:112
13989820,0rtkit-daemon331-21plymouthd21:46:130
13989820,0rtkit-daemon331-21plymouthd20:50:320
13989820,0rtkit-daemon0-21swapper/021:32:200
13989820,0rtkit-daemon0-21swapper/021:20:200
13989820,0rtkit-daemon0-21swapper/021:05:030
13989820,0rtkit-daemon0-21swapper/020:40:220
13989820,0rtkit-daemon0-21swapper/020:35:260
13989820,0rtkit-daemon0-21swapper/020:05:260
13989820,0rtkit-daemon0-21swapper/019:55:200
13989820,0rtkit-daemon0-21swapper/019:35:040
13989820,0rtkit-daemon0-21swapper/019:10:000
3109327210,0sleep00-21swapper/000:38:080
3108131210,0sleep10-21swapper/100:35:171
3106966210,0sleep10-21swapper/100:30:451
3106942210,0sleep00-21swapper/000:30:320
3103949210,0sleep10-21swapper/100:25:181
3102776210,0sleep10-21swapper/100:20:441
3100933210,0sleep00-21swapper/000:18:070
3099312210,0sleep13099348-21aten_r1power_cu00:15:161
3098775210,0sleep00-21swapper/000:12:350
3098315210,0sleep10-21swapper/100:10:251
3095888210,0sleep10-21swapper/100:05:191
3093570210,0sleep10-21swapper/100:00:191
3091071210,0sleep10-21swapper/123:55:151
3089735210,0sleep10-21swapper/123:50:181
3088571210,0sleep00-21swapper/023:50:010
3086885210,0sleep10-21swapper/123:45:151
3086216210,0sleep00-21swapper/023:45:020
3085763210,1sleep00-21swapper/023:41:010
3084827210,0sleep10-21swapper/123:40:181
3082780210,0sleep00-21swapper/023:35:160
3082355210,0sleep10-21swapper/123:35:131
3081470210,0sleep10-21swapper/123:30:221
3079705210,0sleep00-21swapper/023:28:120
3077818210,0sleep10-21swapper/123:25:011
3076173210,0sleep10-21swapper/123:20:161
3074328210,0sleep10-21swapper/123:15:161
3074241210,0sleep00-21swapper/023:15:150
3073325210,1sleep00-21swapper/023:12:310
3072636210,0sleep10-21swapper/123:10:201
3069945210,0sleep10-21swapper/123:05:151
3069018210,0sleep00-21swapper/023:01:260
3068395210,0sleep10-21swapper/123:00:191
3065636210,0sleep00-21swapper/022:55:180
3065205210,0sleep10-21swapper/122:54:561
3064226210,0sleep00-21swapper/022:50:220
3062771210,0sleep10-21swapper/122:46:541
3062264210,0sleep00-21swapper/022:45:210
3059201210,0sleep10-21swapper/122:40:141
3057531210,0sleep00-21swapper/022:35:160
3055903210,0sleep10-21swapper/122:30:211
3053660210,0sleep10-21swapper/122:25:171
3051855210,0sleep13052075-21grep22:20:201
3050413210,0sleep10-21swapper/122:19:001
3048432210,0sleep20-21swapper/222:15:012
3046752210,0sleep20-21swapper/222:10:172
3045909210,0sleep10-21swapper/122:06:161
3044498210,0sleep20-21swapper/222:05:122
3043938210,0sleep10-21swapper/122:02:191
3043266210,0sleep20-21swapper/222:00:202
3041669210,0sleep20-21swapper/221:55:572
3040285210,0sleep10-21swapper/121:55:081
3039885210,0sleep10-21swapper/121:53:391
3039833210,0sleep20-21swapper/221:53:122
3036985210,0sleep20-21swapper/221:45:192
3036887210,0sleep10-21swapper/121:45:181
3035274210,0sleep20-21swapper/221:40:242
3033793210,0sleep10-21swapper/121:40:021
3031829210,0sleep10-21swapper/121:35:181
3030967210,0sleep20-21swapper/221:30:222
3029138210,0sleep10-21swapper/121:30:021
3029130210,0sleep20-21swapper/221:29:582
3027981210,0sleep10-21swapper/121:25:171
3023749210,0sleep20-21swapper/221:15:182
3021437210,0sleep20-21swapper/221:10:172
3021343210,0sleep10-21swapper/121:10:161
3019568210,1sleep10-21swapper/121:05:161
3017917210,0sleep20-21swapper/221:00:212
3015664210,0sleep10-21swapper/120:55:151
3014664210,0sleep20-21swapper/220:50:242
3012285210,0sleep20-21swapper/220:45:232
3011160210,0sleep10-21swapper/120:41:181
3011111210,0sleep20-21swapper/220:40:392
3009379210,0sleep20-21swapper/220:38:202
3006031210,0sleep23006331-21cstates20:30:142
3005893210,0sleep10-21swapper/120:30:011
3005176210,0sleep10-21swapper/120:25:211
3002661210,0sleep20-21swapper/220:20:192
3001120210,0sleep20-21swapper/220:15:222
2998357210,0sleep10-21swapper/120:10:171
2997394210,0sleep20-21swapper/220:05:212
2994403210,0sleep10-21swapper/120:00:161
2994130210,0sleep20-21swapper/220:00:012
2993070210,0sleep20-21swapper/219:55:172
2990712210,0sleep10-21swapper/119:50:151
2990030210,0sleep20-21swapper/219:49:562
2988371210,0sleep10-21swapper/119:45:151
2986546210,0sleep20-21swapper/219:40:162
2986456210,0sleep10-21swapper/119:40:151
2985422210,0sleep20-21swapper/219:35:192
2983064210,0sleep10-21swapper/119:30:181
2981050210,0sleep10-21swapper/119:25:171
2979627210,0sleep10-21swapper/119:20:211
2974891210,0sleep10-21swapper/119:10:171
2974448210,0sleep20-21swapper/219:10:142
3054973200,0sleep00-21swapper/022:30:040
3050529200,0sleep20-21swapper/222:20:022
3031639200,0sleep20-21swapper/221:35:032
3025374200,0sleep10-21swapper/121:20:051
3019372200,0sleep20-21swapper/221:05:052
3017457200,0sleep10-21swapper/121:00:031
3013602200,0sleep10-21swapper/120:50:041
3011684200,0sleep10-21swapper/120:45:021
3007830200,0sleep10-21swapper/120:35:031
3003990200,0sleep20-21swapper/220:25:022
2998019200,0sleep20-21swapper/220:10:012
2984502200,0sleep10-21swapper/119:35:021
2982586200,0sleep20-21swapper/219:30:032
2980668200,0sleep20-21swapper/219:25:032
2978753200,0sleep20-21swapper/219:20:022
2976811200,0sleep20-21swapper/219:15:022
2976115200,0sleep10-21swapper/119:15:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional