You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 21:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Wed Nov 19, 2025 12:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
275991990,89ptp4l0-21swapper/107:05:571
134892790,0sleep10-21swapper/107:20:251
134892790,0sleep10-21swapper/107:20:251
147926756,9sleep40-21swapper/407:09:046
136126552,10sleep00-21swapper/007:07:360
124726553,10sleep50-21swapper/507:05:537
152826452,10sleep110-21swapper/1107:09:473
139426453,9sleep100-21swapper/1007:07:582
3060826351,10sleep70-21swapper/707:05:069
131526352,9sleep60-21swapper/607:06:548
322852540,0sleep100-21swapper/1009:50:272
260012540,0sleep100-21swapper/1011:10:002
207262540,0sleep90-21swapper/911:00:2211
197299190,1cyclictest0-21swapper/1008:00:192
196699192,11cyclictest30548-21diskmemload11:25:0011
1952991914,5cyclictest0-21swapper/610:59:558
193899193,10cyclictest0-21swapper/312:30:005
196699182,10cyclictest0-21swapper/912:25:0011
275991170,0ptp4l0-21swapper/110:00:171
275991170,0ptp4l0-21swapper/109:05:151
275991170,0ptp4l0-21swapper/108:35:161
1972991716,1cyclictest0-21swapper/1008:54:552
196699173,8cyclictest0-21swapper/910:00:0111
194899173,8cyclictest0-21swapper/510:05:017
192499173,8cyclictest0-21swapper/007:30:010
192499173,8cyclictest0-21swapper/007:30:010
275991160,2ptp4l10140-21sed08:40:231
275991160,0ptp4l0-21swapper/110:50:001
275991160,0ptp4l0-21swapper/107:42:481
197299160,16cyclictest0-21swapper/1009:15:232
194299160,15cyclictest0-21swapper/409:55:186
193899163,8cyclictest0-21swapper/311:40:005
193899162,9cyclictest0-21swapper/310:40:005
193899162,9cyclictest0-21swapper/310:35:005
192499164,7cyclictest0-21swapper/010:38:190
192499162,9cyclictest0-21swapper/007:35:010
192499162,9cyclictest0-21swapper/007:35:010
275991150,1ptp4l21-21ksoftirqd/109:25:201
275991150,0ptp4l0-21swapper/112:05:001
275991150,0ptp4l0-21swapper/110:50:201
275991150,0ptp4l0-21swapper/108:05:251
275991150,0ptp4l0-21swapper/107:55:241
1972991514,1cyclictest0-21swapper/1012:20:202
196699150,3cyclictest0-21swapper/910:15:1811
195299151,10cyclictest24689-21kworker/u24:010:25:008
194899153,6cyclictest0-21swapper/512:00:007
1942991512,2cyclictest29764-21turbostat11:15:006
194299150,14cyclictest0-21swapper/411:50:226
193899152,8cyclictest0-21swapper/311:55:005
193899152,8cyclictest0-21swapper/310:55:005
192499154,6cyclictest0-21swapper/009:45:010
275991140,1ptp4l30548-21diskmemload09:40:181
275991140,1ptp4l2883-21fschecks_time07:50:151
275991140,1ptp4l21-21ksoftirqd/111:50:321
275991140,1ptp4l21-21ksoftirqd/111:25:151
275991140,1ptp4l21-21ksoftirqd/109:17:271
275991140,1ptp4l21-21ksoftirqd/109:13:401
275991140,0ptp4l0-21swapper/111:55:231
275991140,0ptp4l0-21swapper/108:20:131
195299142,8cyclictest0-21swapper/612:00:008
194299142,7cyclictest0-21swapper/408:00:016
194299140,9cyclictest0-21swapper/411:50:006
194299140,6cyclictest0-21swapper/410:00:256
193899143,7cyclictest0-21swapper/309:10:015
193899141,9cyclictest0-21swapper/312:25:005
193899140,13cyclictest6664-21chrt10:43:025
192499142,8cyclictest0-21swapper/011:00:270
192499141,7cyclictest0-21swapper/007:37:350
192499141,7cyclictest0-21swapper/007:37:350
303912130,0sleep00-21swapper/011:15:140
275991130,0ptp4l0-21swapper/110:35:001
275991130,0ptp4l0-21swapper/107:25:331
275991130,0ptp4l0-21swapper/107:25:331
197799132,7cyclictest0-21swapper/1112:35:263
197799132,6cyclictest0-21swapper/1108:35:013
196699132,7cyclictest0-21swapper/908:35:2511
196699132,7cyclictest0-21swapper/908:30:0111
195299132,7cyclictest0-21swapper/608:55:278
194899132,7cyclictest0-21swapper/511:45:277
194899132,6cyclictest0-21swapper/510:15:017
193899130,12cyclictest0-21swapper/310:15:195
192499131,7cyclictest0-21swapper/010:30:000
275991120,5ptp4l21-21ksoftirqd/111:35:231
275991120,4ptp4l21-21ksoftirqd/108:15:001
275991120,3ptp4l26989-21ls11:10:161
275991120,3ptp4l19703-21grep08:55:141
275991120,3ptp4l15671-21users10:10:271
275991120,1ptp4l21-21ksoftirqd/110:40:151
275991120,0ptp4l17178-21if_err_enp4s008:50:171
275991120,0ptp4l0-21swapper/112:20:191
275991120,0ptp4l0-21swapper/111:03:031
275991120,0ptp4l0-21swapper/110:22:591
275991120,0ptp4l0-21swapper/109:50:211
197799122,6cyclictest0-21swapper/1107:40:003
197799122,6cyclictest0-21swapper/1107:40:003
197799120,11cyclictest0-21swapper/1112:09:553
196699129,2cyclictest0-21swapper/910:10:1711
196699122,6cyclictest0-21swapper/908:15:0111
196699121,7cyclictest0-21swapper/907:40:2411
194899120,8cyclictest0-21swapper/511:05:007
193899120,1cyclictest0-21swapper/311:15:165
193899120,11cyclictest31554-21cut07:45:165
193899120,11cyclictest0-21swapper/312:34:055
192542120,0sleep20-21swapper/208:10:204
155792120,0sleep90-21swapper/907:25:2211
155792120,0sleep90-21swapper/907:25:2211
275991110,4ptp4l21-21ksoftirqd/109:55:221
275991110,3ptp4l9716-21aten2_r3power_p08:00:151
275991110,2ptp4l821-21users11:15:281
275991110,2ptp4l3191-21unixbench_singl12:00:251
275991110,2ptp4l12990-21latency_hist11:35:001
275991110,1ptp4l0-21swapper/109:00:221
275991110,0ptp4l0-21swapper/112:30:271
275991110,0ptp4l0-21swapper/112:25:001
275991110,0ptp4l0-21swapper/110:35:191
275991110,0ptp4l0-21swapper/109:22:521
195299112,5cyclictest0-21swapper/611:45:008
195299112,5cyclictest0-21swapper/611:40:008
195299111,6cyclictest0-21swapper/608:00:018
194899110,7cyclictest0-21swapper/511:20:007
193899112,7cyclictest0-21swapper/312:05:005
193899110,10cyclictest4975-21expr08:35:145
193899110,10cyclictest20530-21cut11:00:215
193899110,10cyclictest20339-21irqrtprio07:30:215
193899110,10cyclictest20339-21irqrtprio07:30:215
193899110,10cyclictest1946-21sed07:10:015
193399110,2cyclictest0-21swapper/207:15:224
193399110,2cyclictest0-21swapper/207:15:224
192999110,10cyclictest14246-21cut09:30:131
14422112,7sleep80-21swapper/807:08:3210
276091100,3phc2sys30515-21fschecks_time09:50:184
275991100,2ptp4l11171-21gunzip10:05:231
275991100,2ptp4l1020-21runrttasks11:05:231
275991100,1ptp4l3280-21cut07:10:151
275991100,1ptp4l3280-21cut07:10:151
275991100,1ptp4l29399-21cstates08:25:131
275991100,1ptp4l26065-21sed10:25:221
275991100,1ptp4l24088-21users08:15:271
275991100,1ptp4l21-21ksoftirqd/107:35:161
275991100,1ptp4l21-21ksoftirqd/107:35:161
275991100,1ptp4l19284-21head09:35:171
275991100,1ptp4l15792-21df10:55:151
275991100,1ptp4l13091-21fw_conntrack08:45:151
275991100,1ptp4l13077-21fw_conntrack12:15:171
275991100,1ptp4l0-21swapper/109:45:161
275991100,0ptp4l0-21swapper/112:35:171
275991100,0ptp4l0-21swapper/111:40:131
275991100,0ptp4l0-21swapper/108:30:171
197299106,3cyclictest32392-21cat12:00:002
197299102,5cyclictest0-21swapper/1011:30:242
197299100,9cyclictest12527-21cstates12:15:152
196699102,7cyclictest0-21swapper/908:20:0111
195299102,5cyclictest0-21swapper/611:10:008
194299108,1cyclictest0-21swapper/408:15:176
194299103,6cyclictest0-21swapper/408:25:246
193899101,8cyclictest25083-21fschecks_count10:25:185
193899100,9cyclictest9503-21if_err_enp4s012:10:195
193899100,9cyclictest9063-21grep08:40:185
193899100,9cyclictest7832-21tr10:45:145
193899100,9cyclictest7651-21sed07:55:205
193899100,9cyclictest7556-21kernelversion11:25:195
193899100,9cyclictest6903-21cut09:20:165
193899100,9cyclictest6599-21open_files07:15:015
193899100,9cyclictest6599-21open_files07:15:015
193899100,9cyclictest6465-21latency_hist09:20:015
193899100,9cyclictest6267-21sed12:05:205
193899100,9cyclictest5324-21grep10:00:145
193899100,9cyclictest4837-21switchtime07:10:235
193899100,9cyclictest4837-21switchtime07:10:235
193899100,9cyclictest4416-21chrt09:58:335
193899100,9cyclictest364-21cpuspeed08:30:165
193899100,9cyclictest3231-21cat11:20:215
193899100,9cyclictest31456-21cpuspeed_turbos09:10:145
193899100,9cyclictest29691-21cpuspeed09:50:155
193899100,9cyclictest28995-21cpuspeed11:55:145
193899100,9cyclictest27993-21expr12:35:155
193899100,9cyclictest27705-21sed11:10:195
193899100,9cyclictest27692-21tail07:40:165
193899100,9cyclictest25462-21grep08:20:145
193899100,9cyclictest24629-21gltestperf09:00:195
193899100,9cyclictest23253-21cstates07:35:165
193899100,9cyclictest23253-21cstates07:35:165
193899100,9cyclictest22800-21cpu11:05:165
193899100,9cyclictest21872-21chrt09:40:075
193899100,9cyclictest21614-21expr08:15:155
193899100,9cyclictest21504-21gltestperf10:20:175
193899100,9cyclictest20976-21cat08:55:195
193899100,9cyclictest18148-21aten2_r3power_v09:35:135
193899100,9cyclictest17893-21cstates08:10:135
193899100,9cyclictest17229-21proc_pri07:25:295
193899100,9cyclictest17229-21proc_pri07:25:295
193899100,9cyclictest16885-21fschecks_time08:50:155
193899100,9cyclictest14281-21aten2_r3power_e09:30:135
193899100,9cyclictest14227-21expr08:05:145
193899100,9cyclictest13653-21sed07:20:285
193899100,9cyclictest13653-21sed07:20:285
193899100,9cyclictest13187-21basename12:15:185
193899100,9cyclictest12940-21cat10:10:155
193899100,9cyclictest12001-21aten2_r3power_p08:45:125
193899100,9cyclictest11881-21grep09:25:195
193899100,9cyclictest11343-21cpuspeed10:50:145
193899100,9cyclictest11315-21switchtime10:05:265
193899100,9cyclictest10879-21sed08:00:195
193899100,4cyclictest4605-21timerandwakeup07:50:245
193899100,1cyclictest0-21swapper/311:40:065
193899100,10cyclictest0-21swapper/309:45:095
193399100,9cyclictest0-21swapper/211:00:154
15252105,4sleep90-21swapper/907:09:4511
27599190,1ptp4l8257-21cpuspeed12:10:151
27599190,0ptp4l0-21swapper/111:45:141
27599190,0ptp4l0-21swapper/111:20:171
27599190,0ptp4l0-21swapper/110:15:141
27599190,0ptp4l0-21swapper/107:30:161
27599190,0ptp4l0-21swapper/107:30:161
27599190,0ptp4l0-21swapper/107:15:011
27599190,0ptp4l0-21swapper/107:15:011
19779997,1cyclictest0-21swapper/1109:45:163
19729998,1cyclictest741rcuc/1010:15:252
19729996,2cyclictest26914-21cpuspeed_turbos07:40:132
19729990,4cyclictest0-21swapper/1010:44:552
19729990,1cyclictest0-21swapper/1010:55:212
19669990,9cyclictest0-21swapper/909:00:1711
19619990,8cyclictest589-21cat07:45:2510
19619990,8cyclictest16762-21perl12:20:1910
19619990,1cyclictest0-21swapper/808:25:2010
19529998,1cyclictest0-21swapper/612:35:268
19529998,1cyclictest0-21swapper/609:25:208
19529992,4cyclictest0-21swapper/611:45:008
19529990,9cyclictest0-21swapper/610:25:288
19489997,1cyclictest24216-21smartctl09:40:257
19429991,7cyclictest0-21swapper/408:35:186
19429990,9cyclictest0-21swapper/410:10:206
19389990,8cyclictest9929-21tr11:30:135
19389990,8cyclictest28780-21basename08:25:125
19389990,8cyclictest21096-21chrt11:45:095
19339998,1cyclictest0-21swapper/212:20:284
19339998,1cyclictest0-21swapper/211:05:254
19339990,9cyclictest0-21swapper/211:40:244
19339990,9cyclictest0-21swapper/208:20:224
19339990,8cyclictest31302-21perl11:15:174
19339990,1cyclictest0-21swapper/212:10:154
19339990,1cyclictest0-21swapper/210:25:004
19299990,9cyclictest0-21swapper/107:45:141
19249992,4cyclictest0-21swapper/010:10:160
19249990,9cyclictest0-21swapper/010:15:200
19249990,8cyclictest0-21swapper/011:05:180
9498080,1rtkit-daemon17769-21sed12:20:238
19779987,1cyclictest0-21swapper/1110:40:153
19779987,1cyclictest0-21swapper/1110:15:143
19779987,1cyclictest0-21swapper/1109:55:223
19779987,1cyclictest0-21swapper/1107:40:203
19779980,8cyclictest0-21swapper/1109:20:193
19779980,1cyclictest0-21swapper/1110:45:203
19779980,1cyclictest0-21swapper/1109:30:283
19729987,1cyclictest741rcuc/1012:38:372
19729987,1cyclictest741rcuc/1008:46:552
19729987,1cyclictest741rcuc/1007:15:332
19729987,1cyclictest741rcuc/1007:15:332
19729987,1cyclictest0-21swapper/1009:10:162
19729987,1cyclictest0-21swapper/1008:25:142
19729984,1cyclictest741rcuc/1009:25:012
19729980,8cyclictest0-21swapper/1012:30:252
19729980,8cyclictest0-21swapper/1012:05:252
19729980,8cyclictest0-21swapper/1011:45:212
19729980,8cyclictest0-21swapper/1010:50:222
19729980,8cyclictest0-21swapper/1010:05:012
19729980,8cyclictest0-21swapper/1009:55:252
19729980,1cyclictest0-21swapper/1011:25:232
19729980,1cyclictest0-21swapper/1009:05:242
19729980,1cyclictest0-21swapper/1008:55:142
19669987,1cyclictest0-21swapper/908:50:1611
19669987,1cyclictest0-21swapper/907:55:1511
19669987,1cyclictest0-21swapper/907:33:2511
19669987,1cyclictest0-21swapper/907:33:2511
19669980,8cyclictest0-21swapper/910:38:2111
19669980,8cyclictest0-21swapper/909:25:1911
19669980,8cyclictest0-21swapper/907:50:1611
19669980,1cyclictest0-21swapper/909:10:0111
19669980,1cyclictest0-21swapper/907:10:2411
19669980,1cyclictest0-21swapper/907:10:2411
19619987,1cyclictest30394-40dnf12:38:3510
19619987,1cyclictest0-21swapper/811:37:0110
19619987,1cyclictest0-21swapper/810:55:1810
19619986,2cyclictest30325-21df_inode09:50:1710
19619980,8cyclictest0-21swapper/810:10:1610
19619980,8cyclictest0-21swapper/808:15:2210
19619980,1cyclictest0-21swapper/811:50:2210
19619980,1cyclictest0-21swapper/811:40:1410
19619980,1cyclictest0-21swapper/811:25:2410
19619980,1cyclictest0-21swapper/811:15:2310
19619980,1cyclictest0-21swapper/809:25:1610
19619980,1cyclictest0-21swapper/808:35:2110
19619980,1cyclictest0-21swapper/808:10:1510
19579987,1cyclictest0-21swapper/711:35:209
19579987,1cyclictest0-21swapper/708:32:599
19579980,7cyclictest0-21swapper/711:40:019
19579980,7cyclictest0-21swapper/710:59:309
19579980,7cyclictest0-21swapper/707:46:539
19579980,7cyclictest0-21swapper/707:11:019
19579980,7cyclictest0-21swapper/707:11:019
19579980,1cyclictest0-21swapper/712:28:159
19579980,1cyclictest0-21swapper/711:30:169
19579980,1cyclictest0-21swapper/711:29:329
19579980,1cyclictest0-21swapper/710:50:179
19579980,1cyclictest0-21swapper/710:45:269
19579980,1cyclictest0-21swapper/709:55:179
19579980,1cyclictest0-21swapper/709:20:259
19579980,1cyclictest0-21swapper/707:20:219
19579980,1cyclictest0-21swapper/707:20:219
19529987,1cyclictest0-21swapper/612:06:298
19529987,1cyclictest0-21swapper/610:50:258
19529987,1cyclictest0-21swapper/608:40:208
19529980,8cyclictest0-21swapper/609:10:018
19529980,8cyclictest0-21swapper/609:00:168
19529980,8cyclictest0-21swapper/607:50:018
19529980,8cyclictest0-21swapper/607:15:258
19529980,8cyclictest0-21swapper/607:15:258
19529980,3cyclictest0-21swapper/611:30:008
19529980,1cyclictest0-21swapper/611:50:278
19529980,1cyclictest0-21swapper/607:20:218
19529980,1cyclictest0-21swapper/607:20:218
19489987,1cyclictest0-21swapper/507:25:297
19489987,1cyclictest0-21swapper/507:25:297
19489981,5cyclictest0-21swapper/512:15:007
19489981,4cyclictest0-21swapper/509:55:017
19489980,1cyclictest0-21swapper/509:00:017
19429987,1cyclictest0-21swapper/409:30:176
19429987,1cyclictest0-21swapper/408:30:286
19429987,1cyclictest0-21swapper/407:35:216
19429987,1cyclictest0-21swapper/407:35:216
19429982,3cyclictest0-21swapper/410:30:266
19429980,8cyclictest0-21swapper/412:20:276
19429980,8cyclictest0-21swapper/411:35:286
19429980,8cyclictest0-21swapper/411:00:186
19429980,8cyclictest0-21swapper/410:55:136
19429980,8cyclictest0-21swapper/410:20:176
19429980,8cyclictest0-21swapper/409:15:186
19429980,8cyclictest0-21swapper/409:10:196
19429980,8cyclictest0-21swapper/409:05:196
19429980,8cyclictest0-21swapper/408:55:186
19429980,8cyclictest0-21swapper/408:05:216
19429980,7cyclictest15244-21latency_hist08:50:016
19429980,1cyclictest0-21swapper/409:45:206
19429980,1cyclictest0-21swapper/409:40:246
19429980,1cyclictest0-21swapper/408:50:216
19339987,1cyclictest0-21swapper/211:10:224
19339987,1cyclictest0-21swapper/210:25:234
19339987,1cyclictest0-21swapper/210:00:134
19339987,1cyclictest0-21swapper/209:30:194
19339987,1cyclictest0-21swapper/209:25:144
19339987,1cyclictest0-21swapper/208:00:194
19339987,1cyclictest0-21swapper/207:54:364
19339980,8cyclictest0-21swapper/212:35:154
19339980,8cyclictest0-21swapper/212:25:294
19339980,8cyclictest0-21swapper/212:00:154
19339980,8cyclictest0-21swapper/210:55:174
19339980,8cyclictest0-21swapper/210:35:194
19339980,8cyclictest0-21swapper/209:24:144
19339980,8cyclictest0-21swapper/209:15:244
19339980,8cyclictest0-21swapper/209:10:144
19339980,8cyclictest0-21swapper/209:00:254
19339980,7cyclictest29306-21cstates08:25:134
19339980,1cyclictest0-21swapper/212:30:234
19339980,1cyclictest0-21swapper/211:55:284
19339980,1cyclictest0-21swapper/211:34:584
19339980,1cyclictest0-21swapper/211:20:234
19339980,1cyclictest0-21swapper/210:50:164
19339980,1cyclictest0-21swapper/210:10:354
19339980,1cyclictest0-21swapper/208:40:194
19339980,1cyclictest0-21swapper/208:35:134
19339980,1cyclictest0-21swapper/208:30:274
19339980,1cyclictest0-21swapper/208:17:554
19339980,1cyclictest0-21swapper/207:40:194
19339980,1cyclictest0-21swapper/207:22:364
19339980,1cyclictest0-21swapper/207:22:364
19249987,1cyclictest0-21swapper/011:20:280
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional