You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-24 - 02:04
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot3.osadl.org (updated Fri Oct 24, 2025 00:43:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
295022899140,12cyclictest0-21swapper/300:15:143
2950228991311,1cyclictest0-21swapper/322:50:133
295022899130,1cyclictest0-21swapper/323:10:143
295022899130,11cyclictest0-21swapper/323:35:153
295022899130,11cyclictest0-21swapper/323:00:133
2950228991210,0cyclictest0-21swapper/322:15:193
295022899120,1cyclictest0-21swapper/321:15:133
295022899120,1cyclictest0-21swapper/319:55:123
295022899120,11cyclictest0-21swapper/323:20:143
295022899120,11cyclictest0-21swapper/322:20:213
295022899120,11cyclictest0-21swapper/322:00:163
2950228991111,0cyclictest0-21swapper/321:30:133
2950228991111,0cyclictest0-21swapper/321:15:013
2950228991110,1cyclictest69450irq/121-eno120:03:593
295022899110,1cyclictest0-21swapper/300:35:203
295022899110,10cyclictest0-21swapper/323:25:233
295022899110,10cyclictest0-21swapper/321:25:203
295022899109,0cyclictest0-21swapper/320:30:153
295022899104,1cyclictest0-21swapper/319:15:133
2950228991010,0cyclictest69450irq/121-eno123:47:553
2950228991010,0cyclictest69450irq/121-eno123:34:463
2950228991010,0cyclictest69450irq/121-eno121:20:173
2950228991010,0cyclictest69450irq/121-eno120:15:133
2950228991010,0cyclictest0-21swapper/323:50:163
2950228991010,0cyclictest0-21swapper/321:55:213
2950228991010,0cyclictest0-21swapper/300:25:163
2950228991010,0cyclictest0-21swapper/300:10:213
295022899100,9cyclictest69450irq/121-eno122:56:243
295022899100,9cyclictest69450irq/121-eno122:49:503
295022899100,9cyclictest69450irq/121-eno122:40:263
295022899100,9cyclictest69450irq/121-eno122:05:193
295022899100,9cyclictest69450irq/121-eno119:49:333
295022899100,9cyclictest0-21swapper/322:35:133
295022899100,9cyclictest0-21swapper/319:35:193
295022899100,10cyclictest0-21swapper/320:50:133
295022899100,0cyclictest0-21swapper/320:05:013
295022899100,0cyclictest0-21swapper/319:10:153
29502289999,0cyclictest69450irq/121-eno123:42:063
29502289999,0cyclictest69450irq/121-eno123:05:163
29502289999,0cyclictest69450irq/121-eno121:52:323
29502289999,0cyclictest69450irq/121-eno121:40:193
29502289999,0cyclictest69450irq/121-eno119:42:033
29502289999,0cyclictest69450irq/121-eno100:31:033
29502289999,0cyclictest69450irq/121-eno100:08:273
29502289999,0cyclictest69450irq/121-eno100:00:223
29502289999,0cyclictest0-21swapper/319:50:143
29502289998,0cyclictest0-21swapper/321:00:143
29502289998,0cyclictest0-21swapper/320:45:133
29502289997,1cyclictest2991484-21turbostat20:59:563
29502289990,9cyclictest0-21swapper/321:45:233
29502289990,8cyclictest69450irq/121-eno122:30:163
29502289990,8cyclictest69450irq/121-eno119:25:173
29502289988,0cyclictest69450irq/121-eno123:57:583
29502289988,0cyclictest69450irq/121-eno122:27:103
29502289988,0cyclictest69450irq/121-eno121:39:043
29502289988,0cyclictest69450irq/121-eno119:30:143
29502289988,0cyclictest69450irq/121-eno100:24:043
29502289988,0cyclictest0-21swapper/320:10:183
29502289987,0cyclictest0-21swapper/323:15:173
29502289982,1cyclictest0-21swapper/320:25:143
29502289980,0cyclictest0-21swapper/321:08:243
29502289970,6cyclictest0-21swapper/320:35:123
29502289970,0cyclictest0-21swapper/320:40:243
2950074271,3sleep30-21swapper/319:09:553
2949792271,1sleep20-21swapper/219:06:172
3032346260,0sleep00-21swapper/022:32:410
2956418264,1sleep32956441-21turbostat19:25:003
29502289965,1cyclictest69450irq/121-eno120:20:173
29502289960,5cyclictest69450irq/121-eno122:10:293
3067175250,0sleep00-21swapper/023:55:140
3083492240,0sleep10-21swapper/100:31:021
3022392240,0sleep13022457-21basename22:10:131
2994877240,0sleep20-21swapper/221:00:212
2949266241,1sleep10-21swapper/119:05:201
3049639230,2sleep13049673-21unin-run23:15:011
3020495230,2sleep13020741-21diskstats22:05:161
3001290230,0sleep10-21swapper/121:17:491
2999748230,1sleep20-21swapper/221:15:142
2979710230,1sleep02979731-21apt-get20:25:010
2974194230,1sleep02974220-21apt-get20:10:120
2968173230,1sleep20-21swapper/219:55:132
3086171220,0sleep20-21swapper/200:38:552
3082314220,0sleep00-21swapper/000:30:190
3081736220,1sleep00-21swapper/000:28:560
3079236220,0sleep10-21swapper/100:22:371
3064057220,1sleep10-21swapper/123:45:301
3050800220,1sleep23050943-21/usr/sbin/munin23:15:222
3046518220,1sleep23046678-21mailstats23:05:202
3044311220,1sleep03044405-21ntp_states23:00:190
3039431220,0sleep00-21swapper/022:50:140
3036568220,0sleep00-21swapper/022:40:320
3035837220,1sleep23036002-21grep22:40:182
3034708220,1sleep03034754-21sendmail-msp22:40:010
3027982220,0sleep00-21swapper/022:21:040
3026023220,0sleep00-21swapper/022:17:230
3022834220,1sleep03022913-21irqstats22:10:160
3019113220,1sleep03019257-21sed22:00:220
3017825220,1sleep00-21swapper/021:59:030
3015706220,1sleep00-21swapper/021:53:460
2995832220,1sleep02995858-21apt-get21:05:140
2984165220,1sleep10-21swapper/120:35:151
2978822220,1sleep22978957-21systemctl20:20:232
2977775221,1sleep016-21ksoftirqd/020:20:010
2976726220,1sleep22976880-21ntp_states20:15:182
2964481220,1sleep00-21swapper/019:45:140
2957288220,1sleep22957442-21ntp_states19:25:192
2950821220,1sleep22950903-21irqstats19:10:182
2948410221,0sleep00-21swapper/019:05:030
3084733210,0sleep00-21swapper/000:35:170
3083950210,0sleep10-21swapper/100:35:021
3082147210,0sleep20-21swapper/200:30:182
3080034210,0sleep10-21swapper/100:25:171
3079734210,0sleep20-21swapper/200:25:012
3079099210,0sleep20-21swapper/200:21:262
3078652210,0sleep00-21swapper/000:20:240
3077072210,0sleep10-21swapper/100:17:081
3075700210,0sleep00-21swapper/000:15:150
3075565210,0sleep20-21swapper/200:15:142
3075192210,0sleep10-21swapper/100:14:001
3074888210,0sleep00-21swapper/000:11:190
3073221210,0sleep20-21swapper/200:10:012
3072988210,0sleep10-21swapper/100:08:001
3071899210,1sleep20-21swapper/200:05:212
3071551210,0sleep00-21swapper/000:05:190
3070075210,0sleep20-21swapper/200:00:202
3069924210,0sleep10-21swapper/100:00:191
3069311210,0sleep00-21swapper/000:00:140
3068593210,0sleep10-21swapper/123:56:231
3068539210,0sleep20-21swapper/223:55:562
3066613210,0sleep10-21swapper/123:54:391
3064829210,0sleep23064855-21apt-get23:50:162
3064609210,0sleep00-21swapper/023:50:020
3064304210,0sleep20-21swapper/223:47:432
3063702210,0sleep00-21swapper/023:45:220
3062263210,0sleep10-21swapper/123:43:061
3061033210,0sleep20-21swapper/223:40:152
3060109210,0sleep00-21swapper/023:40:010
3058710210,0sleep10-21swapper/123:35:181
3058462210,0sleep00-21swapper/023:35:160
3058277210,0sleep20-21swapper/223:35:152
3057583210,0sleep20-21swapper/223:31:152
3056552210,0sleep00-21swapper/023:30:170
3056167210,0sleep10-21swapper/123:30:061
3054624210,0sleep10-21swapper/123:25:211
3054222210,0sleep20-21swapper/223:25:182
3054100210,0sleep00-21swapper/023:25:160
3053179210,0sleep00-21swapper/023:20:230
3052007210,1sleep20-21swapper/223:20:132
3051729210,0sleep10-21swapper/123:20:001
3050050210,0sleep10-21swapper/123:15:171
3049936210,0sleep00-21swapper/023:15:160
3049368210,0sleep20-21swapper/223:12:422
3047897210,0sleep00-21swapper/023:10:150
3046452210,0sleep10-21swapper/123:05:201
3045749210,0sleep00-21swapper/023:05:150
3043813210,0sleep10-21swapper/123:00:161
3043057210,1sleep00-21swapper/022:57:260
3041784210,1sleep23041963-21/usr/sbin/munin22:55:162
3039502210,0sleep10-21swapper/122:50:151
3039410210,0sleep20-21swapper/222:50:042
3038947210,0sleep10-21swapper/122:48:031
3038733210,1sleep00-21swapper/022:46:100
3035693210,0sleep13035827-21latency22:40:171
3034188210,0sleep10-21swapper/122:35:301
3032962210,0sleep20-21swapper/222:35:152
3031303210,0sleep10-21swapper/122:30:181
3030870210,0sleep20-21swapper/222:30:152
3030408210,0sleep10-21swapper/122:29:011
3030058210,1sleep23030060-21ssh22:25:582
3030047210,0sleep00-21swapper/022:25:530
3028357210,0sleep20-21swapper/222:24:222
3027907210,0sleep10-21swapper/122:20:251
3025456210,0sleep20-21swapper/222:15:222
3024477210,0sleep10-21swapper/122:15:091
3022142210,0sleep20-21swapper/222:10:012
3021621210,0sleep20-21swapper/222:05:292
3020294210,1sleep00-21swapper/022:05:070
3018716210,0sleep10-21swapper/122:00:191
3017524210,0sleep20-21swapper/221:56:252
3017479210,0sleep10-21swapper/121:56:001
3014014210,0sleep20-21swapper/221:50:122
3013749210,0sleep20-21swapper/221:49:552
3013552210,0sleep10-21swapper/121:48:101
3013412210,1sleep00-21swapper/021:46:580
3011384210,0sleep20-21swapper/221:42:322
3011240210,0sleep00-21swapper/021:41:140
3010560210,0sleep10-21swapper/121:40:191
3009341210,0sleep10-21swapper/121:37:581
3008749210,0sleep00-21swapper/021:35:200
3008509210,0sleep20-21swapper/221:35:192
3006569210,0sleep00-21swapper/021:30:210
3005508210,0sleep00-21swapper/021:29:570
3004393210,0sleep20-21swapper/221:25:192
3003777210,0sleep10-21swapper/121:25:141
3003388210,0sleep00-21swapper/021:23:360
3003238211,0sleep20-21swapper/221:22:062
3002024210,0sleep10-21swapper/121:20:161
2999871210,0sleep00-21swapper/021:15:160
2999501210,0sleep10-21swapper/121:15:011
2998116210,0sleep20-21swapper/221:10:182
2997841210,0sleep00-21swapper/021:10:160
2995581210,0sleep20-21swapper/221:05:002
2994102210,0sleep10-21swapper/121:00:161
2993957210,0sleep00-21swapper/021:00:150
2993486210,0sleep00-21swapper/020:57:590
2989966210,0sleep20-21swapper/220:50:162
2988339210,0sleep20-21swapper/220:45:162
2988260210,0sleep10-21swapper/120:45:151
2987985210,0sleep00-21swapper/020:45:140
2986760210,0sleep20-21swapper/220:40:242
2986606210,0sleep10-21swapper/120:40:231
2986485210,0sleep00-21swapper/020:40:210
2984430210,0sleep20-21swapper/220:35:162
2984057210,0sleep00-21swapper/020:35:140
2983557210,0sleep20-21swapper/220:31:592
2980221210,0sleep20-21swapper/220:25:142
2980204210,0sleep10-21swapper/120:25:011
2978049210,0sleep00-21swapper/020:20:170
2977828210,0sleep10-21swapper/120:20:021
2976108210,0sleep10-21swapper/120:15:131
2974348210,0sleep20-21swapper/220:10:132
2973948210,0sleep10-21swapper/120:10:011
2973586210,0sleep10-21swapper/120:05:251
2973452210,0sleep00-21swapper/020:05:240
2973079210,0sleep20-21swapper/220:05:222
2970264210,0sleep10-21swapper/120:00:161
2969599210,1sleep10-21swapper/119:57:001
2966413210,0sleep10-21swapper/119:50:161
2966289210,0sleep00-21swapper/019:50:150
2964346210,0sleep20-21swapper/219:45:122
2963781210,0sleep10-21swapper/119:40:591
2962434210,0sleep20-21swapper/219:40:122
2961975210,0sleep10-21swapper/119:37:391
2960926210,0sleep20-21swapper/219:35:162
2960525210,0sleep00-21swapper/019:35:140
2959928210,0sleep20-21swapper/219:30:592
2959882210,0sleep10-21swapper/119:30:231
2959193210,0sleep00-21swapper/019:30:180
2957573210,0sleep00-21swapper/019:25:210
2955404210,0sleep00-21swapper/019:20:200
2955179210,0sleep20-21swapper/219:20:192
2952795210,0sleep20-21swapper/219:15:152
2952766210,0sleep10-21swapper/119:15:141
2952051210,0sleep00-21swapper/019:15:010
2950573210,0sleep10-21swapper/119:10:171
2950418210,0sleep00-21swapper/019:10:160
3043613200,0sleep20-21swapper/223:00:052
3041503200,0sleep10-21swapper/122:55:031
3037320200,0sleep20-21swapper/222:45:042
3018196200,0sleep20-21swapper/222:00:052
3013999200,0sleep10-21swapper/121:50:031
3005758200,0sleep20-21swapper/221:30:022
3005522200,0sleep10-21swapper/121:30:021
2995817200,0sleep10-21swapper/121:05:031
2991971200,0sleep20-21swapper/220:55:052
2991970200,0sleep10-21swapper/120:55:041
2989794200,0sleep10-21swapper/120:50:031
2989793200,0sleep00-21swapper/020:50:020
2982119200,0sleep10-21swapper/120:30:021
2981884200,0sleep00-21swapper/020:30:010
2970095200,0sleep00-21swapper/020:00:020
2969838200,0sleep20-21swapper/220:00:012
2968161200,0sleep00-21swapper/019:55:030
2966011200,0sleep20-21swapper/219:50:012
2964335200,0sleep10-21swapper/119:45:031
2962422200,0sleep00-21swapper/019:40:020
2956654200,0sleep10-21swapper/119:25:021
2954740200,0sleep10-21swapper/119:20:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional