You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-11 - 08:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Thu Dec 11, 2025 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
275991980,88ptp4l0-21swapper/119:06:451
2000928270,9sleep100-21swapper/1019:05:182
1886826965,2sleep60-21swapper/619:05:138
2193726755,10sleep50-21swapper/519:08:247
2191526655,9sleep110-21swapper/1119:08:063
2193026554,9sleep00-21swapper/019:08:190
2193626452,10sleep40-21swapper/419:08:236
2199126049,9sleep70-21swapper/719:09:149
315062580,0sleep60-21swapper/622:50:138
244992530,0sleep70-21swapper/719:50:189
2238099232,16cyclictest0-21swapper/421:50:016
2238099200,6cyclictest0-21swapper/419:55:006
2240899192,6cyclictest0-21swapper/1022:25:012
275991170,0ptp4l0-21swapper/122:50:011
2239599173,8cyclictest0-21swapper/720:10:009
2239599173,8cyclictest0-21swapper/720:10:009
22395991713,2cyclictest0-21swapper/721:50:189
2238999173,8cyclictest0-21swapper/620:20:008
2238599170,16cyclictest0-21swapper/522:10:187
22380991713,2cyclictest0-21swapper/422:10:186
304902160,0sleep1201rcuc/123:30:121
275991160,1ptp4l21-21ksoftirqd/100:15:171
275991160,0ptp4l0-21swapper/122:25:171
275991160,0ptp4l0-21swapper/100:30:151
2239999162,9cyclictest0-21swapper/800:40:0010
2239599164,7cyclictest0-21swapper/723:20:019
2239599163,9cyclictest7194-21kworker/u24:023:30:029
2239599162,8cyclictest0-21swapper/700:25:009
2238999160,3cyclictest0-21swapper/619:20:188
2236799164,8cyclictest0-21swapper/220:20:014
2236799164,8cyclictest0-21swapper/220:20:014
2236799163,9cyclictest0-21swapper/222:45:014
2236799163,8cyclictest0-21swapper/200:00:014
2236799162,9cyclictest0-21swapper/219:45:004
2236399163,7cyclictest0-21swapper/121:35:011
2235899165,7cyclictest0-21swapper/021:25:010
2235899160,2cyclictest0-21swapper/021:10:160
123432160,0sleep80-21swapper/822:25:1210
275991150,0ptp4l0-21swapper/121:10:151
275991150,0ptp4l0-21swapper/119:20:181
275991150,0ptp4l0-21swapper/100:10:211
2239599153,7cyclictest0-21swapper/719:15:009
2239599152,8cyclictest0-21swapper/720:00:009
2239599150,2cyclictest0-21swapper/721:20:189
2238999152,8cyclictest0-21swapper/621:10:008
2238599154,10cyclictest0-21swapper/523:10:017
2238599152,8cyclictest0-21swapper/521:20:017
2238099150,10cyclictest0-21swapper/422:05:016
2236399153,7cyclictest0-21swapper/120:50:011
2236399153,7cyclictest0-21swapper/119:45:001
2236399152,8cyclictest0-21swapper/120:10:001
2236399152,8cyclictest0-21swapper/120:10:001
2236399151,7cyclictest0-21swapper/123:55:001
2235899153,7cyclictest0-21swapper/021:35:000
275991140,1ptp4l21-21ksoftirqd/121:00:141
275991140,1ptp4l21-21ksoftirqd/120:10:201
275991140,0ptp4l0-21swapper/122:00:181
275991140,0ptp4l0-21swapper/121:20:211
275991140,0ptp4l0-21swapper/119:55:221
2239999142,7cyclictest0-21swapper/821:45:0010
2239999142,7cyclictest0-21swapper/800:30:0010
22399991411,2cyclictest24824-21turbostat22:00:0010
2239599143,7cyclictest0-21swapper/721:00:249
2239599141,8cyclictest0-21swapper/723:00:249
2239599140,8cyclictest0-21swapper/700:39:559
2238999142,7cyclictest7194-21kworker/u24:020:45:008
2238999142,7cyclictest7194-21kworker/u24:020:45:008
2238999142,7cyclictest0-21swapper/621:30:268
2238599143,7cyclictest0-21swapper/522:55:017
2238599142,7cyclictest0-21swapper/523:50:007
2238099142,7cyclictest0-21swapper/421:05:016
2238099141,7cyclictest0-21swapper/420:30:246
2238099141,7cyclictest0-21swapper/420:30:246
2236799142,8cyclictest0-21swapper/223:10:244
2236799142,8cyclictest0-21swapper/200:35:004
22367991411,2cyclictest0-21swapper/223:25:184
2236399143,7cyclictest0-21swapper/122:50:011
2236399143,7cyclictest0-21swapper/100:00:001
2236399142,8cyclictest0-21swapper/120:55:011
2236399142,7cyclictest0-21swapper/121:00:001
2235899143,6cyclictest0-21swapper/022:00:010
2235899142,8cyclictest0-21swapper/019:25:250
275991130,1ptp4l21-21ksoftirqd/123:48:511
275991130,0ptp4l0-21swapper/120:15:201
22408991310,3cyclictest0-21swapper/1022:10:182
2239999132,7cyclictest0-21swapper/820:30:0110
2239999132,7cyclictest0-21swapper/820:30:0010
2239999132,6cyclictest0-21swapper/822:15:0110
2239599133,6cyclictest0-21swapper/722:25:029
2239599132,7cyclictest0-21swapper/723:15:019
2239599130,12cyclictest0-21swapper/720:49:559
2238999132,7cyclictest0-21swapper/600:05:008
2238999132,6cyclictest0-21swapper/600:10:008
2238599132,6cyclictest0-21swapper/521:45:217
2236399132,6cyclictest0-21swapper/122:20:011
2235899134,5cyclictest0-21swapper/000:27:170
2235899132,7cyclictest0-21swapper/022:10:010
275991120,3ptp4l23536-21timerwakeupswit20:30:241
275991120,3ptp4l23536-21timerwakeupswit20:30:241
275991120,1ptp4l21-21ksoftirqd/120:20:111
275991120,1ptp4l21-21ksoftirqd/120:20:111
275991120,0ptp4l0-21swapper/122:35:161
275991120,0ptp4l0-21swapper/122:14:411
275991120,0ptp4l0-21swapper/121:45:171
275991120,0ptp4l0-21swapper/121:35:161
275991120,0ptp4l0-21swapper/120:40:171
275991120,0ptp4l0-21swapper/120:40:171
275991120,0ptp4l0-21swapper/119:15:151
2241399122,6cyclictest0-21swapper/1120:00:263
2240899123,6cyclictest0-21swapper/1022:10:012
2239999121,7cyclictest0-21swapper/819:15:2310
2239599122,6cyclictest0-21swapper/720:40:009
2239599122,6cyclictest0-21swapper/720:40:009
2239599121,7cyclictest0-21swapper/721:10:239
2238999122,6cyclictest0-21swapper/619:50:218
2238599122,6cyclictest0-21swapper/519:15:007
2238599120,11cyclictest0-21swapper/500:20:237
2238099122,6cyclictest0-21swapper/423:40:256
2238099122,6cyclictest0-21swapper/421:35:256
2238099122,6cyclictest0-21swapper/400:25:256
2238099121,7cyclictest0-21swapper/419:30:246
2236399121,6cyclictest0-21swapper/122:55:261
2235899122,6cyclictest0-21swapper/000:10:000
195702121,8sleep80-21swapper/819:05:1510
275991110,6ptp4l21-21ksoftirqd/120:00:281
275991110,2ptp4l4720-21df00:20:141
275991110,2ptp4l29976-21df_inode22:05:141
275991110,1ptp4l0-21swapper/123:09:161
275991110,1ptp4l0-21swapper/120:35:151
275991110,1ptp4l0-21swapper/120:35:141
275991110,0ptp4l0-21swapper/121:54:131
275991110,0ptp4l0-21swapper/119:14:091
275991110,0ptp4l0-21swapper/100:27:161
2241399111,6cyclictest0-21swapper/1119:55:013
2240899112,5cyclictest0-21swapper/1021:20:252
2240899111,6cyclictest0-21swapper/1023:45:002
2240899110,10cyclictest7989-21sed21:35:192
2239999111,7cyclictest0-21swapper/823:45:2210
2239999110,10cyclictest6307-21switchtime20:50:2410
2239599113,6cyclictest0-21swapper/720:20:019
2239599110,3cyclictest19120-21users22:30:249
2238999111,7cyclictest7194-21kworker/u24:023:15:018
2238999110,10cyclictest0-21swapper/619:55:228
2238599111,7cyclictest0-21swapper/520:15:007
2238099112,6cyclictest0-21swapper/420:15:246
2238099111,6cyclictest0-21swapper/423:25:256
2238099110,10cyclictest3210-21awk20:50:006
2236799117,3cyclictest27542-21ps22:00:224
2236799113,7cyclictest0-21swapper/221:55:244
2236799110,3cyclictest19754-21cstates21:10:134
2236799110,10cyclictest0-21swapper/223:35:004
191591110,0getstats0-21swapper/220:26:034
191591110,0getstats0-21swapper/220:26:034
275991100,2ptp4l5885-21cpuspeed_turbos23:40:151
275991100,1ptp4l7779-21diskstats19:30:151
275991100,1ptp4l32643-21missed_timers21:25:191
275991100,1ptp4l22041-21cstates21:55:141
275991100,1ptp4l20321-21latency19:45:211
275991100,1ptp4l17487-21cut00:35:201
275991100,1ptp4l10233-21irqstats22:20:171
275991100,0ptp4l0-21swapper/121:40:131
275991100,0ptp4l0-21swapper/121:09:461
275991100,0ptp4l0-21swapper/119:25:191
2241399102,5cyclictest0-21swapper/1123:20:023
2241399102,5cyclictest0-21swapper/1122:00:013
2241399102,4cyclictest0-21swapper/1120:30:013
2241399102,4cyclictest0-21swapper/1120:30:013
2240899100,9cyclictest0-21swapper/1023:15:142
2239999101,5cyclictest0-21swapper/820:40:2310
2239999101,5cyclictest0-21swapper/820:40:2310
2239599100,8cyclictest653-21irqrtprio22:50:199
2239599100,1cyclictest0-21swapper/722:30:009
2238999102,5cyclictest0-21swapper/620:15:008
2238999100,9cyclictest6303-21sed20:50:248
2238999100,9cyclictest26787-21processes20:35:218
2238999100,9cyclictest26787-21processes20:35:218
2238599102,6cyclictest0-21swapper/521:25:017
2238599100,9cyclictest0-21swapper/500:25:157
2237299100,9cyclictest0-21swapper/321:05:135
2237299100,9cyclictest0-21swapper/319:15:125
2237299100,9cyclictest0-21swapper/300:15:245
2237299100,1cyclictest0-21swapper/323:00:145
2236799100,9cyclictest0-21swapper/221:35:154
2235899102,5cyclictest0-21swapper/022:40:010
2235899102,5cyclictest0-21swapper/021:00:000
191591100,0getstats0-21swapper/220:16:034
191591100,0getstats0-21swapper/220:11:034
27599190,2ptp4l21-21ksoftirqd/122:30:141
27599190,1ptp4l21-21ksoftirqd/119:35:111
27599190,0ptp4l0-21swapper/123:35:121
27599190,0ptp4l0-21swapper/123:25:141
27599190,0ptp4l0-21swapper/123:20:141
27599190,0ptp4l0-21swapper/123:15:141
27599190,0ptp4l0-21swapper/123:10:151
27599190,0ptp4l0-21swapper/123:00:161
27599190,0ptp4l0-21swapper/122:40:111
27599190,0ptp4l0-21swapper/121:15:181
27599190,0ptp4l0-21swapper/120:25:151
27599190,0ptp4l0-21swapper/120:25:151
27599190,0ptp4l0-21swapper/119:50:111
27599190,0ptp4l0-21swapper/100:05:161
27599190,0ptp4l0-21swapper/100:00:131
224139995,3cyclictest25625-21awk23:25:013
224139992,5cyclictest0-21swapper/1122:50:253
224139990,9cyclictest0-21swapper/1119:55:153
224139990,8cyclictest16817-21users21:45:223
224139990,1cyclictest22828-21unixbench-2d22:35:253
224139990,1cyclictest0-21swapper/1123:45:153
224139990,1cyclictest0-21swapper/1121:15:003
224139990,1cyclictest0-21swapper/1120:10:003
224139990,1cyclictest0-21swapper/1120:10:003
224089998,1cyclictest0-21swapper/1019:15:002
224089991,8cyclictest0-21swapper/1023:55:152
224049998,1cyclictest0-21swapper/921:40:1611
224049998,1cyclictest0-21swapper/900:10:1811
224049997,2cyclictest7643-21vmstat22:15:2511
224049990,9cyclictest0-21swapper/922:35:1811
224049990,8cyclictest31157-21latency_hist22:50:0111
224049990,1cyclictest0-21swapper/923:35:1211
224049990,1cyclictest0-21swapper/900:00:1311
223999996,2cyclictest395-21cat20:45:1310
223999991,5cyclictest0-21swapper/823:55:0010
223999990,3cyclictest0-21swapper/819:44:5510
223999990,1cyclictest0-21swapper/821:10:1210
223959990,9cyclictest0-21swapper/700:25:169
223899990,2cyclictest6179-21ptp4l00:20:228
223899990,1cyclictest0-21swapper/619:45:208
223859997,1cyclictest0-21swapper/521:55:137
223859992,6cyclictest0-21swapper/520:05:017
223859990,9cyclictest0-21swapper/519:40:177
223859990,8cyclictest0-21swapper/519:15:187
223859990,1cyclictest0-21swapper/520:05:227
223859990,1cyclictest0-21swapper/520:05:227
223809997,1cyclictest0-21swapper/421:20:196
223809990,1cyclictest0-21swapper/422:55:266
223729996,2cyclictest0-21swapper/319:31:035
223729990,1cyclictest0-21swapper/321:15:135
223679990,8cyclictest0-21swapper/222:10:234
223679990,8cyclictest0-21swapper/221:30:164
223679990,8cyclictest0-21swapper/220:30:154
223679990,8cyclictest0-21swapper/220:30:154
223679990,3cyclictest22810-21/usr/sbin/munin00:00:184
223589992,4cyclictest0-21swapper/023:35:170
223589991,6cyclictest0-21swapper/019:25:000
223589991,5cyclictest0-21swapper/023:15:010
223589991,5cyclictest0-21swapper/019:55:020
223589990,9cyclictest0-21swapper/021:25:000
224139981,4cyclictest0-21swapper/1123:40:003
224139981,4cyclictest0-21swapper/1122:25:253
224139981,4cyclictest0-21swapper/1120:35:003
224139981,4cyclictest0-21swapper/1120:35:003
224139980,8cyclictest0-21swapper/1122:40:123
224139980,8cyclictest0-21swapper/1121:50:183
224139980,5cyclictest0-21swapper/1119:35:003
224139980,1cyclictest0-21swapper/1123:50:263
224139980,1cyclictest0-21swapper/1121:15:153
224139980,1cyclictest0-21swapper/1120:45:213
224139980,1cyclictest0-21swapper/1120:15:253
224089987,1cyclictest0-21swapper/1020:00:252
224089982,4cyclictest0-21swapper/1022:40:012
224089981,5cyclictest0-21swapper/1022:00:252
224089980,7cyclictest1009-21snmpd22:44:372
224089980,1cyclictest8209-21/usr/sbin/munin23:00:152
224089980,1cyclictest0-21swapper/1019:55:182
224049987,1cyclictest0-21swapper/921:45:1311
224049987,1cyclictest0-21swapper/921:05:1811
224049980,1cyclictest0-21swapper/923:30:1611
224049980,1cyclictest0-21swapper/922:00:2311
224049980,1cyclictest0-21swapper/920:30:1511
224049980,1cyclictest0-21swapper/920:30:1511
224049980,1cyclictest0-21swapper/919:45:1711
223999985,2cyclictest31732-21munin-run00:15:0010
223999980,8cyclictest0-21swapper/820:30:1210
223999980,8cyclictest0-21swapper/820:30:1210
223999980,1cyclictest0-21swapper/823:00:1610
223999980,1cyclictest0-21swapper/822:50:2010
223999980,1cyclictest0-21swapper/822:30:2310
223959980,8cyclictest0-21swapper/723:35:129
223959980,8cyclictest0-21swapper/722:05:169
223959980,8cyclictest0-21swapper/720:20:139
223959980,8cyclictest0-21swapper/720:20:139
223959980,7cyclictest26352-21sh21:20:009
223959980,1cyclictest0-21swapper/723:20:219
223959980,1cyclictest0-21swapper/722:10:259
223959980,1cyclictest0-21swapper/721:55:209
223959980,1cyclictest0-21swapper/719:40:279
223959980,1cyclictest0-21swapper/719:25:149
223959980,1cyclictest0-21swapper/719:15:189
223899987,1cyclictest0-21swapper/623:50:228
223899987,1cyclictest0-21swapper/623:05:268
223899987,1cyclictest0-21swapper/620:22:228
223899987,1cyclictest0-21swapper/620:22:228
223899980,8cyclictest0-21swapper/623:00:248
223899980,8cyclictest0-21swapper/619:40:208
223899980,8cyclictest0-21swapper/619:30:138
223899980,1cyclictest0-21swapper/622:00:278
223899980,1cyclictest0-21swapper/621:55:238
223899980,1cyclictest0-21swapper/621:50:208
223899980,1cyclictest0-21swapper/619:35:178
223899980,1cyclictest0-21swapper/600:15:138
223859987,1cyclictest0-21swapper/523:50:177
223859987,1cyclictest0-21swapper/523:00:127
223859987,1cyclictest0-21swapper/522:15:147
223859987,1cyclictest0-21swapper/521:30:217
223859987,1cyclictest0-21swapper/521:25:187
223859987,1cyclictest0-21swapper/519:50:107
223859980,8cyclictest0-21swapper/522:25:217
223859980,8cyclictest0-21swapper/519:45:247
223859980,1cyclictest147412sleep500:33:127
223859980,1cyclictest0-21swapper/522:55:197
223859980,1cyclictest0-21swapper/522:05:217
223859980,1cyclictest0-21swapper/522:00:157
223859980,1cyclictest0-21swapper/519:26:037
223809987,1cyclictest0-21swapper/423:20:166
223809986,1cyclictest0-21swapper/419:15:246
223809982,5cyclictest12565-21crond23:49:596
223809980,8cyclictest0-21swapper/420:10:156
223809980,8cyclictest0-21swapper/400:25:006
223809980,7cyclictest9333-21sort23:00:236
223809980,7cyclictest18831-21cat00:40:006
223809980,7cyclictest0-21swapper/420:55:216
223809980,1cyclictest0-21swapper/421:50:016
223809980,1cyclictest0-21swapper/421:05:146
223809980,1cyclictest0-21swapper/400:00:176
223729987,1cyclictest0-21swapper/322:00:205
223729987,1cyclictest0-21swapper/300:30:195
223729987,1cyclictest0-21swapper/300:25:185
223729985,2cyclictest2444-21turbostat19:29:555
223729985,2cyclictest17937-21turbostat23:19:555
223729985,2cyclictest0-21swapper/322:35:155
223729981,6cyclictest0-21swapper/322:30:175
223729980,8cyclictest0-21swapper/323:40:155
223729980,1cyclictest0-21swapper/322:55:215
223679987,1cyclictest0-21swapper/222:35:184
223679986,1cyclictest0-21swapper/223:05:204
223679985,2cyclictest18713-21turbostat00:40:004
223679985,2cyclictest0-21swapper/221:30:004
223679981,6cyclictest0-21swapper/222:15:174
223679980,8cyclictest0-21swapper/223:50:174
223679980,8cyclictest0-21swapper/222:26:234
223679980,8cyclictest0-21swapper/221:40:134
223679980,8cyclictest0-21swapper/220:40:124
223679980,8cyclictest0-21swapper/220:40:124
223679980,8cyclictest0-21swapper/220:00:264
223679980,7cyclictest11673-21/usr/sbin/munin23:45:224
223679980,7cyclictest0-21swapper/223:35:174
223679980,7cyclictest0-21swapper/220:50:144
223679980,2cyclictest0-21swapper/219:55:184
223679980,1cyclictest0-21swapper/220:05:164
223679980,1cyclictest0-21swapper/220:05:154
223679980,1cyclictest0-21swapper/200:20:214
223589981,4cyclictest0-21swapper/020:00:010
223589980,8cyclictest0-21swapper/022:50:150
21799281,5sleep30-21swapper/319:06:225
21741281,5sleep90-21swapper/919:05:4011
636270,0sleep110-21swapper/1122:10:133
224139976,1cyclictest0-21swapper/1123:55:273
224139976,1cyclictest0-21swapper/1123:10:313
224139976,1cyclictest0-21swapper/1123:02:303
224139976,1cyclictest0-21swapper/1122:47:443
224139976,1cyclictest0-21swapper/1122:05:223
224139976,1cyclictest0-21swapper/1121:35:583
224139976,1cyclictest0-21swapper/1121:32:553
224139976,1cyclictest0-21swapper/1120:39:133
224139976,1cyclictest0-21swapper/1120:39:123
224139976,1cyclictest0-21swapper/1120:11:413
224139976,1cyclictest0-21swapper/1119:40:203
224139976,1cyclictest0-21swapper/1119:25:583
224139976,1cyclictest0-21swapper/1100:31:163
224139976,1cyclictest0-21swapper/1100:25:163
224139976,1cyclictest0-21swapper/1100:22:433
224139976,1cyclictest0-21swapper/1100:16:223
224139970,7cyclictest0-21swapper/1123:41:113
224139970,7cyclictest0-21swapper/1123:30:043
224139970,7cyclictest0-21swapper/1123:25:383
224139970,7cyclictest0-21swapper/1123:07:233
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional