You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 20:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Wed Nov 26, 2025 12:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2759911020,83ptp4l0-21swapper/107:06:361
1860526749,10sleep50-21swapper/507:08:097
1873926654,10sleep70-21swapper/707:09:589
1868826557,6sleep110-21swapper/1107:09:143
1855026553,10sleep60-21swapper/607:07:228
1845126555,8sleep00-21swapper/007:05:500
1778726555,7sleep100-21swapper/1007:05:242
1873526353,8sleep40-21swapper/407:09:556
239632570,0sleep110-21swapper/1107:15:033
1911899270,3cyclictest0-21swapper/1009:20:162
1909099242,17cyclictest0-21swapper/407:55:016
1910599222,11cyclictest0-21swapper/711:40:019
67652210,0chrt0-21swapper/408:15:136
1912399191,17cyclictest0-21swapper/1107:35:163
1910999192,10cyclictest0-21swapper/811:05:0010
1909699183,8cyclictest0-21swapper/510:00:017
1907599182,10cyclictest0-21swapper/110:40:001
275991170,0ptp4l0-21swapper/110:30:221
1910599173,8cyclictest0-21swapper/711:55:009
1910599172,9cyclictest0-21swapper/708:25:259
1908099173,8cyclictest0-21swapper/211:00:004
275991160,2ptp4l24731-21memory11:25:201
275991160,1ptp4l21-21ksoftirqd/108:20:161
275991160,0ptp4l0-21swapper/111:30:101
275991160,0ptp4l0-21swapper/109:00:221
275991160,0ptp4l0-21swapper/108:10:171
1911899162,8cyclictest0-21swapper/1007:25:012
1911499163,8cyclictest0-21swapper/912:20:0011
1911499163,8cyclictest0-21swapper/912:20:0011
1911499163,7cyclictest0-21swapper/909:45:0011
1910599163,8cyclictest0-21swapper/712:10:249
1910599163,8cyclictest0-21swapper/712:10:249
1907599164,7cyclictest0-21swapper/108:55:001
1907599162,8cyclictest0-21swapper/112:10:011
1907599162,8cyclictest0-21swapper/112:10:001
1907599162,8cyclictest0-21swapper/111:00:001
187112161,7sleep80-21swapper/807:09:3510
275991150,1ptp4l21-21ksoftirqd/111:15:141
275991150,0ptp4l0-21swapper/112:10:131
275991150,0ptp4l0-21swapper/112:10:131
275991150,0ptp4l0-21swapper/112:00:121
275991150,0ptp4l0-21swapper/112:00:121
275991150,0ptp4l0-21swapper/111:55:221
275991150,0ptp4l0-21swapper/110:40:201
275991150,0ptp4l0-21swapper/110:15:171
275991150,0ptp4l0-21swapper/109:10:161
275991150,0ptp4l0-21swapper/108:05:251
1912399152,9cyclictest0-21swapper/1110:30:013
1910599153,7cyclictest0-21swapper/708:12:489
1910599152,8cyclictest0-21swapper/710:20:009
1910599152,8cyclictest0-21swapper/708:45:009
1909699152,8cyclictest0-21swapper/510:10:007
1908599154,6cyclictest0-21swapper/312:20:015
1908599154,6cyclictest0-21swapper/312:20:005
1908599153,7cyclictest0-21swapper/307:50:015
1908599152,7cyclictest0-21swapper/308:05:015
1908099153,8cyclictest0-21swapper/209:00:004
1907099150,2cyclictest0-21swapper/011:55:180
275991140,0ptp4l0-21swapper/109:45:161
275991140,0ptp4l0-21swapper/109:43:491
275991140,0ptp4l0-21swapper/109:30:251
275991140,0ptp4l0-21swapper/109:09:191
1912399143,8cyclictest23956-21kworker/u24:212:00:003
1911899140,13cyclictest0-21swapper/1008:15:142
1911499143,7cyclictest0-21swapper/910:20:0011
1911499143,7cyclictest0-21swapper/908:15:0111
1911499142,7cyclictest23956-21kworker/u24:208:30:0011
1911499142,7cyclictest0-21swapper/909:50:0111
1911499142,7cyclictest0-21swapper/909:30:0111
1911499142,7cyclictest0-21swapper/909:10:0111
1910999140,13cyclictest0-21swapper/811:30:1510
1910599143,6cyclictest0-21swapper/710:05:009
1910599142,8cyclictest0-21swapper/712:30:009
1910599142,7cyclictest0-21swapper/711:15:009
1909699142,8cyclictest0-21swapper/507:15:017
1908599142,7cyclictest0-21swapper/307:30:015
1908099143,7cyclictest0-21swapper/212:00:004
1907599140,8cyclictest0-21swapper/109:20:001
1907599140,8cyclictest0-21swapper/107:25:021
1907599140,13cyclictest0-21swapper/109:20:171
275991130,0ptp4l0-21swapper/108:45:151
275991130,0ptp4l0-21swapper/107:40:141
275991130,0ptp4l0-21swapper/107:15:151
191591130,0getstats0-21swapper/207:25:064
1912399132,7cyclictest0-21swapper/1107:55:013
1912399132,6cyclictest0-21swapper/1112:25:003
1912399132,6cyclictest0-21swapper/1112:25:003
1912399131,7cyclictest0-21swapper/1109:55:243
1911899130,9cyclictest225221gltestperf12:05:162
1911899130,9cyclictest225221gltestperf12:05:162
1911899130,12cyclictest0-21swapper/1008:51:332
1910599132,6cyclictest0-21swapper/707:30:029
1910599131,7cyclictest0-21swapper/711:30:009
19105991310,2cyclictest32225-21turbostat08:50:009
1910099130,7cyclictest0-21swapper/609:15:018
1909699132,7cyclictest0-21swapper/508:15:027
1908599132,7cyclictest0-21swapper/310:05:235
1908599132,6cyclictest0-21swapper/307:15:265
1908099132,7cyclictest0-21swapper/207:15:014
1908099132,6cyclictest0-21swapper/210:15:004
1907599133,6cyclictest0-21swapper/111:00:001
1907599132,8cyclictest0-21swapper/109:30:011
1907599132,7cyclictest0-21swapper/110:00:001
1907599132,7cyclictest0-21swapper/107:50:011
276091120,3phc2sys0-21swapper/207:07:024
275991120,4ptp4l0-21swapper/109:35:211
275991120,3ptp4l0-21swapper/112:30:231
275991120,1ptp4l21-21ksoftirqd/109:59:411
275991120,0ptp4l0-21swapper/110:28:271
275991120,0ptp4l0-21swapper/109:51:351
275991120,0ptp4l0-21swapper/107:26:551
1912399123,5cyclictest0-21swapper/1109:45:003
1912399122,6cyclictest0-21swapper/1111:30:003
1912399121,7cyclictest0-21swapper/1112:35:003
1911899120,11cyclictest0-21swapper/1007:15:022
1911499122,6cyclictest0-21swapper/912:15:0011
1911499122,6cyclictest0-21swapper/912:15:0011
1911499122,6cyclictest0-21swapper/910:50:0011
1910599122,7cyclictest0-21swapper/712:30:009
1910599122,6cyclictest0-21swapper/710:10:259
1910099123,6cyclictest0-21swapper/607:15:018
1910099122,7cyclictest0-21swapper/607:10:258
1910099121,6cyclictest0-21swapper/612:10:008
1910099121,6cyclictest0-21swapper/612:10:008
1909699121,7cyclictest0-21swapper/511:25:007
1909099120,7cyclictest0-21swapper/411:25:006
1908599122,8cyclictest0-21swapper/311:20:005
1908599121,7cyclictest0-21swapper/310:15:005
1907599121,6cyclictest0-21swapper/111:45:001
1907099129,2cyclictest12323-21turbostat10:30:000
276091110,0phc2sys0-21swapper/209:32:254
275991110,5ptp4l0-21swapper/110:45:401
275991110,2ptp4l22219-21unixbench-2d07:10:261
275991110,2ptp4l21066-21needreboot07:50:451
275991110,2ptp4l20224-21gltestperf11:20:201
275991110,1ptp4l12943-21sed11:10:181
275991110,1ptp4l0-21swapper/110:50:251
275991110,1ptp4l0-21swapper/110:20:321
275991110,1ptp4l0-21swapper/108:35:121
275991110,0ptp4l0-21swapper/111:40:251
275991110,0ptp4l0-21swapper/111:35:011
275991110,0ptp4l0-21swapper/110:10:131
1911899117,3cyclictest0-21swapper/1007:40:142
1911899113,7cyclictest17355-21cut07:45:212
1911899110,7cyclictest315301fschecks_time11:35:152
1911899110,4cyclictest0-21swapper/1010:55:192
1911899110,11cyclictest0-21swapper/1011:46:482
1911899110,11cyclictest0-21swapper/1011:15:182
1911899110,11cyclictest0-21swapper/1010:35:212
1911899110,11cyclictest0-21swapper/1007:53:372
1911899110,10cyclictest0-21swapper/1010:10:582
1911899110,10cyclictest0-21swapper/1009:44:032
1911899110,10cyclictest0-21swapper/1007:11:372
1911499113,4cyclictest0-21swapper/910:40:0011
1911499112,5cyclictest0-21swapper/908:45:0011
1911499110,10cyclictest0-21swapper/911:10:0011
1910099113,4cyclictest0-21swapper/610:15:008
1910099110,6cyclictest0-21swapper/608:20:008
1909099112,6cyclictest0-21swapper/411:15:256
1908099110,5cyclictest0-21swapper/207:45:014
1908099110,10cyclictest17555-21fschecks_count10:35:144
1907099114,6cyclictest0-21swapper/012:05:000
1907099114,6cyclictest0-21swapper/012:05:000
1907099112,8cyclictest0-21swapper/011:05:000
1907099112,8cyclictest0-21swapper/009:50:000
1907099112,7cyclictest0-21swapper/012:20:230
1907099112,7cyclictest0-21swapper/012:20:230
1907099112,7cyclictest0-21swapper/009:40:000
1907099112,5cyclictest0-21swapper/009:55:010
275991100,6ptp4l21-21ksoftirqd/108:25:281
275991100,2ptp4l26548-21if_p34p108:40:201
275991100,2ptp4l24187-21/usr/sbin/munin07:55:171
275991100,1ptp4l4791-21perl07:30:151
275991100,1ptp4l0-21swapper/112:30:001
275991100,1ptp4l0-21swapper/108:15:151
275991100,0ptp4l0-21swapper/112:35:151
275991100,0ptp4l0-21swapper/112:20:191
275991100,0ptp4l0-21swapper/112:20:191
275991100,0ptp4l0-21swapper/111:05:141
275991100,0ptp4l0-21swapper/108:30:131
1911899106,3cyclictest28648-21sh09:25:212
1911899104,5cyclictest28570-21tr10:50:132
1911899104,5cyclictest2475-21cstates11:40:132
1911899104,5cyclictest11244-21irqrtprio11:50:202
1911899100,9cyclictest338-21tr12:20:122
1911899100,9cyclictest338-21tr12:20:122
1911899100,9cyclictest28097-21sed11:30:172
1911899100,9cyclictest10494-21cut10:25:172
1911899100,9cyclictest0-21swapper/1010:41:482
1911899100,10cyclictest0-21swapper/1012:26:132
1911899100,10cyclictest0-21swapper/1009:50:182
1911899100,10cyclictest0-21swapper/1009:49:592
1911899100,10cyclictest0-21swapper/1008:58:032
1911899100,10cyclictest0-21swapper/1008:12:262
1911899100,10cyclictest0-21swapper/1007:27:412
1911499107,2cyclictest23400-21ls11:25:1411
1910999100,5cyclictest0-21swapper/809:40:0010
1910599101,5cyclictest0-21swapper/710:05:009
1910599100,9cyclictest0-21swapper/709:10:009
1910099107,2cyclictest9841-21cut12:30:168
1910099106,3cyclictest7962-21cat11:45:198
1910099102,6cyclictest0-21swapper/609:30:338
1910099102,6cyclictest0-21swapper/607:40:018
1910099102,5cyclictest0-21swapper/611:50:258
1910099101,6cyclictest0-21swapper/608:00:018
1910099100,9cyclictest21993-21cat08:35:128
1909699106,3cyclictest26311-21/usr/sbin/munin10:45:197
1909699106,3cyclictest10732-21fschecks_count08:20:167
1909699100,9cyclictest5226-21df_inode12:25:137
1909099101,6cyclictest0-21swapper/407:50:016
1908599100,9cyclictest23552-21latency_hist09:20:005
1908599100,2cyclictest24580-21latency07:55:195
1908099102,5cyclictest0-21swapper/210:50:004
1908099102,5cyclictest0-21swapper/208:45:004
1908099102,5cyclictest0-21swapper/207:55:024
1907099102,7cyclictest0-21swapper/010:20:310
1907099102,7cyclictest0-21swapper/009:40:340
1907099102,6cyclictest0-21swapper/010:40:000
1907099101,8cyclictest0-21swapper/010:00:000
1907099101,6cyclictest0-21swapper/008:30:000
27599190,0ptp4l0-21swapper/112:15:021
27599190,0ptp4l0-21swapper/112:15:011
27599190,0ptp4l0-21swapper/111:50:141
27599190,0ptp4l0-21swapper/110:05:131
27599190,0ptp4l0-21swapper/108:55:131
27599190,0ptp4l0-21swapper/108:00:131
27599190,0ptp4l0-21swapper/107:35:011
191189998,1cyclictest0-21swapper/1009:05:122
191189990,9cyclictest0-21swapper/1012:10:012
191189990,9cyclictest0-21swapper/1012:10:012
191189990,9cyclictest0-21swapper/1008:42:442
191189990,9cyclictest0-21swapper/1008:34:022
191189990,8cyclictest9451-21sed09:00:142
191189990,8cyclictest7796-21grep07:35:122
191189990,8cyclictest5732-21cstates10:20:202
191189990,8cyclictest5301-21irqrtprio11:00:182
191189990,8cyclictest4868-21grep07:30:152
191189990,8cyclictest3263-21strings10:15:192
191189990,8cyclictest31304-21fschecks_count08:05:152
191189990,8cyclictest31237-21cut09:30:112
191189990,8cyclictest3089-21tr09:35:122
191189990,8cyclictest29890-21fschecks_count12:15:162
191189990,8cyclictest29890-21fschecks_count12:15:152
191189990,8cyclictest29748-21sort08:45:152
191189990,8cyclictest27054-21cpuspeed_turbos08:00:142
191189990,8cyclictest25899-21grep10:05:122
191189990,8cyclictest24755-21cpuspeed_turbos10:45:122
191189990,8cyclictest24283-21cut10:00:192
191189990,8cyclictest23818-21fschecks_time11:25:162
191189990,8cyclictest23328-21cstates07:55:132
191189990,8cyclictest22834-21irqrtprio08:35:172
191189990,8cyclictest21104-21grep09:55:252
191189990,8cyclictest19301-21tr11:20:172
191189990,8cyclictest17756-21tr12:00:122
191189990,8cyclictest17756-21tr12:00:122
191189990,8cyclictest16532-21grep11:55:242
191189990,8cyclictest16364-21grep09:10:142
191189990,8cyclictest14035-21grep08:25:132
191189990,8cyclictest13758-21sed12:35:172
191189990,8cyclictest10860-21fschecks_time08:20:162
191189990,8cyclictest10397-21grep12:30:192
191189990,8cyclictest10321-21sed11:05:232
191189990,8cyclictest0-21swapper/1009:15:122
191189990,3cyclictest0-21swapper/1011:12:292
191189990,3cyclictest0-21swapper/1010:34:172
191149998,1cyclictest0-21swapper/908:10:0111
191149992,4cyclictest0-21swapper/909:25:0011
191149991,5cyclictest0-21swapper/912:30:0011
191149990,9cyclictest0-21swapper/912:30:1711
191149990,9cyclictest0-21swapper/907:35:1211
191149990,1cyclictest0-21swapper/907:10:1811
191099990,1cyclictest621rcuc/809:25:1910
191059998,1cyclictest0-21swapper/707:40:019
191059997,1cyclictest0-21swapper/712:35:179
191059990,8cyclictest0-21swapper/712:15:229
191059990,8cyclictest0-21swapper/712:15:229
191009997,1cyclictest0-21swapper/608:45:218
191009992,5cyclictest0-21swapper/611:10:238
191009990,8cyclictest573-21perl10:55:168
190969991,5cyclictest0-21swapper/508:50:007
190969990,9cyclictest0-21swapper/509:50:007
190969990,1cyclictest0-21swapper/509:50:227
190909990,9cyclictest0-21swapper/409:00:176
190859997,1cyclictest0-21swapper/312:10:145
190859997,1cyclictest0-21swapper/312:10:135
190859992,4cyclictest0-21swapper/308:45:245
190859990,8cyclictest0-21swapper/308:15:495
190859990,1cyclictest0-21swapper/309:25:195
190809998,1cyclictest0-21swapper/209:15:134
190809990,8cyclictest15386-21diskmemload11:25:244
190809990,2cyclictest16394-21hddtemp_smartct07:45:164
190809990,1cyclictest0-21swapper/212:30:124
190809990,1cyclictest0-21swapper/212:10:124
190809990,1cyclictest0-21swapper/212:10:124
190709996,2cyclictest6967-21turbostat07:39:550
190709996,2cyclictest14488-40dnf11:13:260
190709990,8cyclictest0-21swapper/009:20:230
190709990,1cyclictest0-21swapper/007:55:240
18445291,6sleep90-21swapper/907:05:4711
14200290,3sleep614340-21irqrtprio10:30:198
9498080,4rtkit-daemon0-21swapper/307:06:335
191239985,2cyclictest1960-21df10:15:133
191239985,2cyclictest0-21swapper/1109:05:233
191239980,8cyclictest0-21swapper/1111:10:173
191239980,8cyclictest0-21swapper/1111:05:183
191239980,7cyclictest0-21swapper/1108:06:443
191239980,1cyclictest0-21swapper/1109:45:173
191149987,1cyclictest0-21swapper/911:35:1811
191149987,1cyclictest0-21swapper/907:15:2411
191149980,8cyclictest0-21swapper/912:00:1411
191149980,8cyclictest0-21swapper/912:00:1411
191149980,8cyclictest0-21swapper/907:25:1411
191149980,1cyclictest0-21swapper/908:30:1611
191149980,1cyclictest0-21swapper/908:15:2011
191099987,1cyclictest63-21ksoftirqd/807:21:3910
191099987,1cyclictest621rcuc/810:55:1910
191099987,1cyclictest0-21swapper/811:05:2310
191099987,1cyclictest0-21swapper/808:00:1310
191099987,1cyclictest0-21swapper/807:45:2510
191099987,1cyclictest0-21swapper/807:37:2810
191099981,7cyclictest0-21swapper/810:36:4810
191099980,8cyclictest0-21swapper/811:25:2410
191099980,8cyclictest0-21swapper/810:25:1410
191099980,8cyclictest0-21swapper/808:27:2710
191099980,8cyclictest0-21swapper/807:55:0110
191099980,1cyclictest0-21swapper/812:25:0010
191099980,1cyclictest0-21swapper/812:25:0010
191099980,1cyclictest0-21swapper/811:15:1910
191099980,1cyclictest0-21swapper/807:25:1310
191059981,6cyclictest0-21swapper/711:35:009
191059980,8cyclictest0-21swapper/710:20:319
191059980,8cyclictest0-21swapper/708:50:179
191059980,8cyclictest0-21swapper/708:05:179
191059980,7cyclictest0-21swapper/708:15:479
191059980,1cyclictest0-21swapper/707:50:429
191009987,1cyclictest31354-21fschecks_count11:35:158
191009987,1cyclictest0-21swapper/610:25:218
191009987,1cyclictest0-21swapper/609:00:178
191009981,5cyclictest0-21swapper/609:05:238
191009980,8cyclictest0-21swapper/612:25:148
191009980,8cyclictest0-21swapper/608:55:168
191009980,1cyclictest0-21swapper/611:25:258
191009980,1cyclictest0-21swapper/610:44:348
191009980,1cyclictest0-21swapper/608:20:138
190969987,1cyclictest0-21swapper/512:20:197
190969987,1cyclictest0-21swapper/512:20:187
190969987,1cyclictest0-21swapper/512:15:017
190969987,1cyclictest0-21swapper/512:15:007
190969987,1cyclictest0-21swapper/510:20:237
190969987,1cyclictest0-21swapper/509:15:147
190969984,3cyclictest28364-21irqstats11:30:177
190969980,8cyclictest0-21swapper/508:50:157
190969980,1cyclictest0-21swapper/512:00:157
190969980,1cyclictest0-21swapper/512:00:147
190969980,1cyclictest0-21swapper/510:40:177
190909987,1cyclictest0-21swapper/412:40:006
190909987,1cyclictest0-21swapper/412:10:196
190909987,1cyclictest0-21swapper/412:10:196
190909987,1cyclictest0-21swapper/410:00:006
190909980,7cyclictest0-21swapper/409:15:146
190909980,3cyclictest20752-21crond08:35:006
190909980,1cyclictest0-21swapper/411:55:186
190909980,1cyclictest0-21swapper/410:30:206
190859985,2cyclictest0-21swapper/311:45:145
190859980,8cyclictest0-21swapper/309:50:145
190859980,8cyclictest0-21swapper/309:00:205
190859980,1cyclictest0-21swapper/310:15:175
190859980,1cyclictest0-21swapper/310:00:205
190859980,1cyclictest0-21swapper/309:10:005
190859980,1cyclictest0-21swapper/308:10:015
190859980,1cyclictest0-21swapper/307:45:015
190809987,1cyclictest0-21swapper/207:53:484
190809980,8cyclictest0-21swapper/212:20:234
190809980,8cyclictest0-21swapper/212:20:234
190809980,8cyclictest0-21swapper/210:35:004
190809980,8cyclictest0-21swapper/208:00:154
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional