You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-20 - 14:26
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot3.osadl.org (updated Sat Apr 20, 2024 12:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304248799130,1cyclictest0-21swapper/309:00:123
3042487991210,1cyclictest0-21swapper/307:30:113
304248799120,1cyclictest0-21swapper/308:55:123
304248799120,11cyclictest0-21swapper/308:10:103
304248799120,11cyclictest0-21swapper/307:55:063
30422602122,7sleep30-21swapper/307:09:123
3042487991111,0cyclictest0-21swapper/310:35:083
3042487991111,0cyclictest0-21swapper/309:30:113
3042487991110,1cyclictest0-21swapper/310:10:083
304248799110,1cyclictest0-21swapper/307:40:013
304248799110,11cyclictest0-21swapper/312:23:003
304248799110,11cyclictest0-21swapper/311:15:213
304248799110,11cyclictest0-21swapper/309:45:133
304248799110,11cyclictest0-21swapper/308:05:113
304248799110,0cyclictest0-21swapper/311:48:523
304248799110,0cyclictest0-21swapper/311:10:513
304248799110,0cyclictest0-21swapper/308:50:113
304248799110,0cyclictest0-21swapper/307:45:113
304248799110,0cyclictest0-21swapper/307:10:103
3042487991010,0cyclictest0-21swapper/312:25:173
3042487991010,0cyclictest0-21swapper/311:55:103
3042487991010,0cyclictest0-21swapper/311:05:113
3042487991010,0cyclictest0-21swapper/310:30:223
3042487991010,0cyclictest0-21swapper/309:20:153
3042487991010,0cyclictest0-21swapper/308:35:013
304248799100,10cyclictest0-21swapper/311:39:563
304248799100,10cyclictest0-21swapper/311:25:013
304248799100,10cyclictest0-21swapper/310:55:143
304248799100,10cyclictest0-21swapper/310:25:083
304248799100,0cyclictest0-21swapper/308:20:113
304248799100,0cyclictest0-21swapper/307:25:183
30424879999,0cyclictest0-21swapper/309:10:173
30424879999,0cyclictest0-21swapper/309:05:363
30424879998,1cyclictest0-21swapper/309:55:093
30424879990,9cyclictest0-21swapper/311:30:223
30424879990,9cyclictest0-21swapper/308:35:093
30424879980,8cyclictest0-21swapper/312:15:123
30424879980,0cyclictest0-21swapper/309:25:103
13129880,3rtkit-daemon3027614-21munin-node07:05:550
30424879977,0cyclictest0-21swapper/312:05:093
30424879977,0cyclictest0-21swapper/311:50:183
30424879977,0cyclictest0-21swapper/309:42:213
30424879977,0cyclictest0-21swapper/309:35:203
30424879977,0cyclictest0-21swapper/308:25:283
30424879977,0cyclictest0-21swapper/307:52:323
30424879977,0cyclictest0-21swapper/307:41:323
30424879970,7cyclictest0-21swapper/308:15:173
30424879966,0cyclictest0-21swapper/310:40:183
30424879966,0cyclictest0-21swapper/307:20:213
30424879966,0cyclictest0-21swapper/307:15:123
30424879955,0cyclictest0-21swapper/312:30:113
30424879955,0cyclictest0-21swapper/310:52:463
30424879953,2cyclictest3078670-21turbostat08:45:003
30424879951,4cyclictest0-21swapper/312:00:273
30424879950,4cyclictest0-21swapper/308:45:103
3042162252,2sleep10-21swapper/107:07:581
13129850,3rtkit-daemon0-21swapper/010:00:390
13129850,2rtkit-daemon375-21systemd-udevd12:20:300
30424879944,0cyclictest0-21swapper/311:26:413
30424879944,0cyclictest0-21swapper/310:20:133
30424879944,0cyclictest0-21swapper/310:10:003
30424879940,4cyclictest0-21swapper/312:15:003
30424879940,4cyclictest0-21swapper/309:50:123
30424879940,3cyclictest0-21swapper/312:35:113
30424879940,3cyclictest0-21swapper/310:17:323
30424879940,3cyclictest0-21swapper/310:00:143
30424879940,3cyclictest0-21swapper/308:00:103
30424879940,0cyclictest0-21swapper/309:15:113
3040647240,1sleep23040673-21apt-get07:05:112
13129840,2rtkit-daemon717-21lldpd12:01:380
13129840,2rtkit-daemon717-21lldpd09:41:410
13129840,2rtkit-daemon356-21systemd-journal11:33:530
13129840,2rtkit-daemon356-21systemd-journal10:22:300
13129840,2rtkit-daemon3071694-21irqstats08:25:180
3169535230,1sleep20-21swapper/212:20:122
3067304230,1sleep23067330-21apt-get08:15:112
30424879931,1cyclictest0-21swapper/311:40:103
30424879930,2cyclictest0-21swapper/310:45:093
30424879930,0cyclictest0-21swapper/311:00:063
13129830,2rtkit-daemon714-21ntpd11:10:160
13129830,2rtkit-daemon617-21gmain11:45:070
13129830,2rtkit-daemon3044476-21munin-node12:10:210
13129830,2rtkit-daemon1234-21cron11:27:010
13129830,2rtkit-daemon1-21systemd12:25:230
13129830,2rtkit-daemon1-21systemd09:15:220
13129830,1rtkit-daemon749-21gmain08:03:520
13129830,1rtkit-daemon717-21lldpd09:56:330
13129830,1rtkit-daemon717-21lldpd08:41:320
13129830,1rtkit-daemon717-21lldpd08:09:190
13129830,1rtkit-daemon717-21lldpd07:46:310
13129830,1rtkit-daemon717-21lldpd07:26:150
13129830,1rtkit-daemon717-21lldpd07:18:010
13129830,1rtkit-daemon714-21ntpd09:05:140
13129830,1rtkit-daemon617-21gmain12:39:310
13129830,1rtkit-daemon617-21gmain10:05:110
13129830,1rtkit-daemon539-21wpa_supplicant09:35:260
13129830,1rtkit-daemon538-21systemd-logind07:12:290
13129830,1rtkit-daemon375-21systemd-udevd08:54:290
13129830,1rtkit-daemon356-21systemd-journal11:01:530
13129830,1rtkit-daemon356-21systemd-journal10:50:300
13129830,1rtkit-daemon356-21systemd-journal08:30:010
13129830,1rtkit-daemon3191333-21snmpd12:15:550
13129830,1rtkit-daemon3191333-21snmpd11:35:250
13129830,1rtkit-daemon3191333-21snmpd11:23:190
13129830,1rtkit-daemon3191333-21snmpd09:10:020
13129830,1rtkit-daemon3128054-21mailstats10:40:190
13129830,1rtkit-daemon3048277-21users07:20:230
13129830,1rtkit-daemon3044476-21munin-node08:15:080
13129830,1rtkit-daemon1-21systemd12:33:040
13129830,1rtkit-daemon1-21systemd11:57:250
13129830,1rtkit-daemon1-21systemd11:50:560
13129830,1rtkit-daemon1-21systemd11:15:360
13129830,1rtkit-daemon1-21systemd10:55:170
13129830,1rtkit-daemon1-21systemd10:35:530
13129830,1rtkit-daemon1-21systemd10:34:160
13129830,1rtkit-daemon1-21systemd10:10:590
13129830,1rtkit-daemon1-21systemd07:55:580
13129830,1rtkit-daemon1-21systemd07:40:470
13129830,0rtkit-daemon0-21swapper/010:26:430
13129830,0rtkit-daemon0-21swapper/008:20:030
3176657220,1sleep20-21swapper/212:35:212
3168466220,1sleep10-21swapper/112:15:231
3168088220,1sleep20-21swapper/212:15:202
3155000220,1sleep13155201-21/usr/sbin/munin11:45:131
3149656220,1sleep13149755-21users11:30:241
3147923220,0sleep23147932-21apt-get11:30:012
3139910220,1sleep13140163-21df11:10:151
3120286220,1sleep13120471-21cat10:25:011
3117055220,1sleep00-21swapper/010:15:190
3116058220,0sleep10-21swapper/110:14:561
3111862220,1sleep20-21swapper/210:05:012
3111342220,0sleep10-21swapper/110:00:271
3102079220,1sleep13102104-21irqstats09:40:181
3097974220,1sleep10-21swapper/109:30:181
3095365220,1sleep13095476-21/usr/sbin/munin09:25:131
3094990220,1sleep13095097-21cat09:25:011
3092846220,1sleep00-21swapper/009:20:040
3089584220,1sleep13089692-21cat09:10:211
3086365220,1sleep23086522-21latency_hist09:05:012
3082743220,1sleep23082769-21apt-get08:55:132
3070644221,1sleep13191333-21snmpd08:21:221
3069252220,0sleep20-21swapper/208:20:112
3061536220,1sleep10-21swapper/108:00:131
3056534220,1sleep20-21swapper/207:45:012
3055740220,1sleep13055893-21sed07:40:401
13129820,1rtkit-daemon714-21ntpd12:05:240
13129820,1rtkit-daemon617-21gmain09:00:140
13129820,1rtkit-daemon539-21wpa_supplicant07:51:100
13129820,1rtkit-daemon524-21avahi-daemon08:35:050
13129820,1rtkit-daemon3191333-21snmpd09:31:330
13129820,1rtkit-daemon3191333-21snmpd08:55:180
13129820,1rtkit-daemon3088349-21diskmemload11:40:100
13129820,1rtkit-daemon3044476-21munin-node11:05:120
13129820,1rtkit-daemon269977-21sendmail-mta10:45:190
13129820,1rtkit-daemon269977-21sendmail-mta09:45:080
13129820,1rtkit-daemon1-21systemd08:45:250
13129820,1rtkit-daemon1-21systemd07:30:090
13129820,0rtkit-daemon0-21swapper/009:50:170
13129820,0rtkit-daemon0-21swapper/009:26:030
13129820,0rtkit-daemon0-21swapper/008:10:030
13129820,0rtkit-daemon0-21swapper/007:35:110
3175846210,0sleep10-21swapper/112:35:121
3173832210,0sleep20-21swapper/212:30:132
3173768210,0sleep10-21swapper/112:30:121
3172101210,0sleep20-21swapper/212:25:162
3172063210,0sleep10-21swapper/112:25:151
3169530210,0sleep10-21swapper/112:20:111
3166061210,0sleep10-21swapper/112:10:231
3165413210,0sleep20-21swapper/212:10:162
3163268210,0sleep20-21swapper/212:05:152
3163200210,0sleep10-21swapper/112:05:101
3161259210,0sleep10-21swapper/112:00:301
3161130210,0sleep20-21swapper/212:00:272
3158974210,0sleep20-21swapper/211:55:112
3158726210,0sleep10-21swapper/111:55:021
3157106210,0sleep20-21swapper/211:50:142
3156618210,0sleep10-21swapper/111:50:011
3155202210,0sleep20-21swapper/211:45:142
3152832210,0sleep10-21swapper/111:40:131
3152392210,0sleep20-21swapper/211:40:022
3150575210,0sleep20-21swapper/211:35:142
3150546210,0sleep10-21swapper/111:35:131
3146335210,0sleep10-21swapper/111:25:161
3145819210,0sleep20-21swapper/211:25:012
3144766210,0sleep20-21swapper/211:20:212
3143971210,0sleep10-21swapper/111:20:121
3142263210,0sleep20-21swapper/211:15:162
3141972210,0sleep10-21swapper/111:15:151
3139762210,0sleep20-21swapper/211:10:122
3137640210,0sleep10-21swapper/111:05:111
3137623210,0sleep20-21swapper/211:05:042
3135813210,0sleep20-21swapper/211:00:202
3135523210,0sleep10-21swapper/111:00:031
3134101210,0sleep10-21swapper/110:55:221
3133421210,0sleep20-21swapper/210:55:152
3131741210,0sleep20-21swapper/210:50:152
3131691210,0sleep10-21swapper/110:50:141
3129237210,0sleep10-21swapper/110:45:101
3129194210,0sleep20-21swapper/210:45:032
3127109210,0sleep20-21swapper/210:40:112
3126574210,0sleep10-21swapper/110:40:011
3124827210,0sleep20-21swapper/210:35:102
3124487210,0sleep10-21swapper/110:35:011
3122633210,0sleep20-21swapper/210:30:102
3122384210,0sleep10-21swapper/110:30:011
3120767210,0sleep10-21swapper/110:25:131
3120532210,0sleep20-21swapper/210:25:102
3118163210,0sleep20-21swapper/210:20:012
3116311210,0sleep10-21swapper/110:15:081
3116070210,0sleep20-21swapper/210:15:012
3114483210,0sleep20-21swapper/210:10:132
3112145210,0sleep20-21swapper/210:05:132
3112116210,0sleep13112142-21apt-get10:05:121
3108387210,0sleep10-21swapper/109:55:151
3107976210,0sleep20-21swapper/209:55:122
3106044210,0sleep10-21swapper/109:50:151
3105787210,0sleep20-21swapper/209:50:122
3103874210,0sleep23104092-21expr09:45:152
3103682210,0sleep10-21swapper/109:45:101
3101770210,0sleep20-21swapper/209:40:152
3099823210,0sleep10-21swapper/109:35:141
3099456210,0sleep20-21swapper/209:35:112
3097337210,0sleep20-21swapper/209:30:032
3095231210,0sleep20-21swapper/209:25:062
3094738210,0sleep20-21swapper/209:22:542
3090882210,0sleep20-21swapper/209:15:142
3090803210,0sleep10-21swapper/109:15:131
3088341210,0sleep20-21swapper/209:10:012
3087379210,0sleep20-21swapper/209:05:172
3087252210,0sleep10-21swapper/109:05:161
3084687210,1sleep10-21swapper/109:00:121
3082973210,0sleep10-21swapper/108:55:161
3080908210,0sleep20-21swapper/208:50:122
3080841210,0sleep10-21swapper/108:50:111
3080203210,0sleep10-21swapper/108:45:271
3079015210,0sleep20-21swapper/208:45:122
3077230210,0sleep10-21swapper/108:40:151
3076973210,0sleep20-21swapper/208:40:122
3075147210,0sleep20-21swapper/208:35:122
3075053210,0sleep10-21swapper/108:35:111
3073834210,0sleep20-21swapper/208:30:192
3073650210,0sleep10-21swapper/108:30:181
3072161210,0sleep20-21swapper/208:25:232
3071181210,0sleep10-21swapper/108:25:141
3067060210,0sleep10-21swapper/108:15:021
3066435210,0sleep10-21swapper/108:10:211
3065380210,0sleep20-21swapper/208:10:102
3063722210,0sleep20-21swapper/208:05:142
3063213210,0sleep10-21swapper/108:05:011
3062713210,0sleep20-21swapper/208:00:222
3060256210,0sleep10-21swapper/107:55:121
3059717210,0sleep20-21swapper/207:55:092
3059107210,0sleep10-21swapper/107:52:001
3058458210,0sleep20-21swapper/207:50:012
3057514210,0sleep10-21swapper/107:45:201
3053355210,0sleep20-21swapper/207:35:162
3052736210,0sleep10-21swapper/107:35:031
3052484210,0sleep20-21swapper/207:34:522
3051696210,0sleep10-21swapper/107:30:191
3049150210,0sleep10-21swapper/107:25:151
3048895210,0sleep20-21swapper/207:25:122
3047416210,0sleep20-21swapper/207:20:162
3046970210,0sleep10-21swapper/107:20:111
3046205210,0sleep20-21swapper/207:15:232
3045178210,0sleep10-21swapper/107:15:141
3043941210,0sleep10-21swapper/107:10:221
3042640210,0sleep20-21swapper/207:10:112
3054847200,0sleep20-21swapper/207:40:052
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional