You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-08 - 13:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Tue Jul 08, 2025 12:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2759911140,58ptp4l0-21swapper/107:05:521
173582690,0sleep40-21swapper/407:25:236
172026658,6sleep110-21swapper/1107:09:293
322222650,0sleep00-21swapper/012:00:000
155526554,9sleep00-21swapper/007:07:170
145426554,9sleep50-21swapper/507:05:467
3273926452,9sleep40-21swapper/407:05:216
156226453,9sleep70-21swapper/707:07:249
3081126150,9sleep100-21swapper/1007:05:032
152226150,9sleep60-21swapper/607:06:478
298652580,0sleep80-21swapper/811:55:2010
218452580,0sleep80-21swapper/807:30:2710
117962570,0sleep10-21swapper/109:25:201
214799194,9cyclictest0-21swapper/210:50:244
214799193,10cyclictest0-21swapper/208:00:004
216899181,16cyclictest0-21swapper/608:45:218
214799184,9cyclictest0-21swapper/209:30:014
214799183,10cyclictest0-21swapper/209:00:014
214799183,10cyclictest0-21swapper/209:00:004
308222177,7sleep80-21swapper/807:05:1310
275991170,1ptp4l21-21ksoftirqd/110:20:161
219099173,9cyclictest0-21swapper/1108:35:003
219099173,8cyclictest0-21swapper/1111:15:013
214799173,9cyclictest0-21swapper/209:40:004
214799173,9cyclictest0-21swapper/209:40:004
216899163,7cyclictest0-21swapper/612:29:598
216199163,7cyclictest0-21swapper/512:05:017
216199162,9cyclictest0-21swapper/509:55:007
214799163,8cyclictest0-21swapper/210:35:024
214799163,8cyclictest0-21swapper/210:20:244
214799163,8cyclictest0-21swapper/209:55:014
13382160,0sleep30-21swapper/312:00:165
275991150,1ptp4l21-21ksoftirqd/108:55:261
275991150,1ptp4l21-21ksoftirqd/108:55:251
275991150,1ptp4l21-21ksoftirqd/108:05:011
217299153,8cyclictest0-21swapper/711:25:009
217299153,7cyclictest0-21swapper/710:25:009
217299152,8cyclictest0-21swapper/709:00:009
217299152,8cyclictest0-21swapper/709:00:009
2168991513,1cyclictest0-21swapper/608:10:018
216199153,8cyclictest0-21swapper/507:55:217
216199152,8cyclictest0-21swapper/512:15:007
214799153,8cyclictest0-21swapper/211:35:004
214799153,8cyclictest0-21swapper/209:10:014
214799153,8cyclictest0-21swapper/208:20:004
2147991512,2cyclictest0-21swapper/207:30:214
214799150,14cyclictest0-21swapper/208:30:204
213899152,9cyclictest0-21swapper/007:40:000
213899152,8cyclictest0-21swapper/010:20:000
275991140,1ptp4l9613-21sshd09:20:251
275991140,1ptp4l9613-21sshd09:20:251
275991140,1ptp4l21-21ksoftirqd/108:30:161
275991140,1ptp4l21-21ksoftirqd/107:52:251
275991140,1ptp4l21-21ksoftirqd/107:34:211
275991140,0ptp4l0-21swapper/107:15:011
244252140,1sleep10-21swapper/111:50:011
219099143,7cyclictest0-21swapper/1108:20:013
219099142,7cyclictest0-21swapper/1110:35:003
218199143,7cyclictest0-21swapper/910:25:0211
217799140,8cyclictest0-21swapper/809:20:0110
217299142,7cyclictest0-21swapper/708:15:009
216899142,7cyclictest0-21swapper/610:00:018
216899142,7cyclictest0-21swapper/608:15:008
216899140,14cyclictest0-21swapper/608:20:158
216199142,6cyclictest0-21swapper/510:05:007
214399142,7cyclictest0-21swapper/111:10:011
214399142,7cyclictest0-21swapper/110:05:011
214399142,7cyclictest0-21swapper/108:50:001
214399142,7cyclictest0-21swapper/107:25:261
213899143,7cyclictest0-21swapper/009:45:000
213899143,7cyclictest0-21swapper/009:45:000
275991130,1ptp4l21-21ksoftirqd/112:23:171
275991130,1ptp4l21-21ksoftirqd/107:40:151
219099132,7cyclictest0-21swapper/1109:10:003
217799132,7cyclictest0-21swapper/811:20:0110
217799131,8cyclictest0-21swapper/812:15:2510
217299134,6cyclictest0-21swapper/710:15:009
217299132,6cyclictest0-21swapper/711:00:269
217299131,8cyclictest0-21swapper/712:25:019
216899134,8cyclictest0-21swapper/611:45:018
216899133,7cyclictest0-21swapper/612:00:018
216199132,7cyclictest0-21swapper/510:55:017
216199132,7cyclictest0-21swapper/510:35:287
216199132,7cyclictest0-21swapper/508:25:007
215299131,2cyclictest0-21swapper/309:20:175
215299131,2cyclictest0-21swapper/309:20:175
214799132,8cyclictest0-21swapper/209:30:014
214799132,8cyclictest0-21swapper/209:30:014
214799132,7cyclictest0-21swapper/209:45:014
214799132,7cyclictest0-21swapper/209:45:014
3112120,0sleep30-21swapper/310:35:195
275991120,4ptp4l21-21ksoftirqd/110:45:221
275991120,3ptp4l15221-21wc12:20:001
275991120,1ptp4l21-21ksoftirqd/112:25:181
275991120,1ptp4l21-21ksoftirqd/112:04:391
275991120,1ptp4l21-21ksoftirqd/108:20:001
275991120,1ptp4l21-21ksoftirqd/107:12:561
275991120,0ptp4l0-21swapper/112:07:391
275991120,0ptp4l0-21swapper/111:38:481
275991120,0ptp4l0-21swapper/110:55:281
219099122,7cyclictest0-21swapper/1112:00:253
219099122,6cyclictest0-21swapper/1110:10:003
218199122,6cyclictest0-21swapper/912:25:0111
217299122,6cyclictest0-21swapper/710:10:009
216899122,6cyclictest0-21swapper/611:45:018
216899120,12cyclictest0-21swapper/609:35:238
216899120,12cyclictest0-21swapper/609:35:228
216199122,6cyclictest0-21swapper/511:35:017
215299121,2cyclictest0-21swapper/308:35:195
215299120,1cyclictest0-21swapper/311:25:005
215299120,11cyclictest0-21swapper/309:30:015
214799122,7cyclictest0-21swapper/211:50:014
214799120,7cyclictest0-21swapper/207:25:004
214399120,11cyclictest0-21swapper/108:50:001
213899122,6cyclictest0-21swapper/010:30:020
275991110,1ptp4l21-21ksoftirqd/109:45:131
275991110,1ptp4l21-21ksoftirqd/109:45:131
275991110,1ptp4l21-21ksoftirqd/109:00:201
275991110,1ptp4l0-21swapper/110:40:161
275991110,1ptp4l0-21swapper/109:35:161
275991110,1ptp4l0-21swapper/109:35:161
275991110,0ptp4l0-21swapper/112:35:181
275991110,0ptp4l0-21swapper/111:20:141
218199113,6cyclictest0-21swapper/909:35:2811
218199113,6cyclictest0-21swapper/909:35:2811
217799112,6cyclictest0-21swapper/808:05:0110
217799111,6cyclictest23732-21kworker/u24:208:20:0010
216899117,3cyclictest1390-21gltestperf12:00:168
216899110,11cyclictest0-21swapper/609:43:558
216899110,11cyclictest0-21swapper/609:43:558
216899110,11cyclictest0-21swapper/608:50:158
216899110,11cyclictest0-21swapper/607:39:338
216899110,10cyclictest0-21swapper/609:46:298
216899110,10cyclictest0-21swapper/609:46:288
216899110,10cyclictest0-21swapper/608:35:168
216199111,6cyclictest0-21swapper/511:40:017
214799118,2cyclictest0-21swapper/208:35:214
214799118,2cyclictest0-21swapper/207:45:004
214799118,2cyclictest0-21swapper/207:20:004
214399110,2cyclictest0-21swapper/109:55:191
213899118,2cyclictest25841-21tune2fs11:50:130
276091100,2phc2sys0-21swapper/207:07:084
275991100,4ptp4l21-21ksoftirqd/108:23:261
275991100,1ptp4l30626-21diskmemload10:15:241
275991100,1ptp4l21-21ksoftirqd/110:02:261
275991100,1ptp4l21-21ksoftirqd/107:39:261
275991100,1ptp4l15438-21perl09:30:171
275991100,1ptp4l15438-21perl09:30:161
275991100,1ptp4l0-21swapper/111:05:221
275991100,0ptp4l0-21swapper/112:30:171
275991100,0ptp4l0-21swapper/109:40:181
275991100,0ptp4l0-21swapper/109:40:171
275991100,0ptp4l0-21swapper/108:25:291
219099107,2cyclictest17770-21memory08:50:183
219099100,9cyclictest2775-21df_inode07:50:183
218699100,9cyclictest0-21swapper/1011:10:182
218199101,6cyclictest0-21swapper/911:30:2611
218199100,6cyclictest0-21swapper/912:10:0111
217799102,5cyclictest0-21swapper/812:40:0010
217799102,5cyclictest0-21swapper/810:20:0110
217799100,1cyclictest0-21swapper/810:15:2410
217299109,1cyclictest0-21swapper/711:29:559
217299107,2cyclictest6318-21taskset12:05:209
217299106,3cyclictest14790-21gltestperf08:05:209
217299102,4cyclictest0-21swapper/708:30:009
217299100,9cyclictest0-21swapper/707:55:229
217299100,5cyclictest0-21swapper/708:25:009
216899104,5cyclictest6090-21munin-setwarn07:15:008
216899103,6cyclictest14861-21if_p34p111:35:228
216899100,9cyclictest7820-21gltestperf09:20:168
216899100,9cyclictest7820-21gltestperf09:20:158
216899100,9cyclictest5461-21munin-run07:55:008
216899100,9cyclictest409-21cut09:55:008
216899100,9cyclictest388-21timerandwakeup11:15:268
216899100,9cyclictest30790-21cpuspeed_turbos07:45:158
216899100,9cyclictest29398-21cpuspeed_turbos08:25:228
216899100,9cyclictest24826-21sed12:30:208
216899100,9cyclictest23862-21cut11:05:198
216899100,9cyclictest22091-21uniq08:15:158
216899100,9cyclictest21597-21gltestperf10:20:158
216899100,9cyclictest2049-21timerandwakeup10:35:278
216899100,9cyclictest19414-21cpuspeed_turbos07:30:158
216899100,9cyclictest15619-21grep07:25:158
216899100,9cyclictest14435-21irqrtprio10:10:198
216899100,9cyclictest10962-21unixbench_multi12:10:268
216899100,4cyclictest0-21swapper/611:11:038
216899100,10cyclictest0-21swapper/612:07:078
216899100,10cyclictest0-21swapper/611:33:568
216899100,10cyclictest0-21swapper/611:20:238
216899100,10cyclictest0-21swapper/610:55:568
216899100,10cyclictest0-21swapper/610:05:338
216899100,10cyclictest0-21swapper/607:40:148
216199109,1cyclictest0-21swapper/509:05:127
216199102,5cyclictest0-21swapper/508:45:007
215799100,2cyclictest30474-21cat10:30:276
215299104,5cyclictest0-21swapper/308:45:005
214799100,2cyclictest13364-21sed08:05:004
214799100,1cyclictest0-21swapper/209:40:284
214799100,1cyclictest0-21swapper/209:40:274
214399100,9cyclictest0-21swapper/111:50:131
213899107,2cyclictest1020-21runrttasks08:20:200
9498090,0rtkit-daemon0-21swapper/910:25:4611
30878291,6sleep90-21swapper/907:05:1411
27599190,2ptp4l30441-21cpuspeed11:15:151
27599190,1ptp4l21-21ksoftirqd/112:10:021
27599190,1ptp4l21-21ksoftirqd/111:55:191
27599190,1ptp4l21-21ksoftirqd/111:40:131
27599190,1ptp4l21-21ksoftirqd/111:30:011
27599190,1ptp4l21-21ksoftirqd/111:25:041
27599190,1ptp4l21-21ksoftirqd/110:50:131
27599190,1ptp4l21-21ksoftirqd/110:35:171
27599190,1ptp4l21-21ksoftirqd/109:10:131
27599190,1ptp4l21-21ksoftirqd/108:40:151
27599190,1ptp4l21-21ksoftirqd/108:35:141
27599190,1ptp4l21-21ksoftirqd/108:10:201
27599190,1ptp4l21-21ksoftirqd/108:00:161
27599190,1ptp4l21-21ksoftirqd/107:55:121
27599190,1ptp4l21-21ksoftirqd/107:45:211
27599190,1ptp4l21-21ksoftirqd/107:20:211
27599190,0ptp4l0-21swapper/111:00:141
27599190,0ptp4l0-21swapper/110:30:141
27599190,0ptp4l0-21swapper/110:25:181
27599190,0ptp4l0-21swapper/110:10:141
27599190,0ptp4l0-21swapper/109:50:141
27599190,0ptp4l0-21swapper/109:15:171
27599190,0ptp4l0-21swapper/109:05:191
21909990,8cyclictest25336-21timerandwakeup07:35:243
21909990,8cyclictest16347-21perl07:25:173
21819991,6cyclictest0-21swapper/912:15:0211
21819991,5cyclictest0-21swapper/910:15:0011
21819991,5cyclictest0-21swapper/908:30:0011
21779991,5cyclictest0-21swapper/812:30:0010
21779991,5cyclictest0-21swapper/811:10:0110
21779991,5cyclictest0-21swapper/808:30:0010
21779990,9cyclictest0-21swapper/809:00:2210
21779990,1cyclictest28578-21smartctl09:05:1510
21779990,1cyclictest0-21swapper/809:30:0010
21729996,2cyclictest0-21swapper/710:45:169
21729992,4cyclictest0-21swapper/709:25:019
21729992,4cyclictest0-21swapper/709:25:009
21729992,4cyclictest0-21swapper/708:55:009
21729991,5cyclictest0-21swapper/712:15:019
21729990,9cyclictest0-21swapper/708:39:559
21729990,9cyclictest0-21swapper/707:24:559
21729990,1cyclictest0-21swapper/709:35:229
21729990,1cyclictest0-21swapper/709:35:229
21729990,1cyclictest0-21swapper/708:15:249
21689990,9cyclictest0-21swapper/612:20:108
21689990,8cyclictest9834-21irqrtprio08:40:188
21689990,8cyclictest9693-21switchtime10:45:248
21689990,8cyclictest7220-21aten2_r3power_v07:15:018
21689990,8cyclictest6484-21tr07:55:108
21689990,8cyclictest6224-21cut11:25:168
21689990,8cyclictest4141-21expr10:40:148
21689990,8cyclictest31919-21expr09:10:138
21689990,8cyclictest3151-21cpuspeed_turbos09:15:148
21689990,8cyclictest28427-21expr10:30:148
21689990,8cyclictest28356-21fschecks_time09:05:138
21689990,8cyclictest27550-21cpuspeed_turbos12:35:158
21689990,8cyclictest25355-21tr11:50:118
21689990,8cyclictest24782-21basename09:00:218
21689990,8cyclictest24441-21wc10:25:168
21689990,8cyclictest19943-21cat08:55:148
21689990,8cyclictest19943-21cat08:55:148
21689990,8cyclictest19885-21perl11:00:168
21689990,8cyclictest17742-21gltestperf10:15:158
21689990,8cyclictest15533-21sed09:30:178
21689990,8cyclictest15533-21sed09:30:178
21689990,8cyclictest1484-21tune2fs08:30:188
21689990,8cyclictest1344-21cpuspeed_turbos09:55:138
21689990,8cyclictest12886-21fschecks_time12:15:158
21689990,8cyclictest11551-21uname08:00:228
21689990,8cyclictest11434-21cut07:20:168
21689990,8cyclictest11415-21cpuspeed_turbos10:50:148
21689990,8cyclictest10578-21aten2_r3power_v09:25:168
21619998,1cyclictest0-21swapper/507:40:187
21619998,1cyclictest0-21swapper/507:15:257
21619997,1cyclictest0-21swapper/510:50:247
21619997,1cyclictest0-21swapper/509:30:007
21619996,2cyclictest0-21swapper/512:34:597
21619990,1cyclictest0-21swapper/511:00:157
21579997,2cyclictest9326-21hddtemp_smartct12:10:176
21579997,1cyclictest677-21cat09:10:196
21579990,1cyclictest0-21swapper/411:45:206
21529992,6cyclictest0-21swapper/311:05:015
21529992,6cyclictest0-21swapper/310:45:255
21479998,1cyclictest0-21swapper/212:25:264
21479991,5cyclictest0-21swapper/208:45:174
21479990,8cyclictest18246-21/usr/sbin/munin08:10:164
21479990,8cyclictest0-21swapper/210:10:144
21479990,8cyclictest0-21swapper/208:40:154
21479990,8cyclictest0-21swapper/208:05:234
21479990,1cyclictest0-21swapper/209:55:144
21909980,8cyclictest0-21swapper/1110:55:163
21909980,7cyclictest30626-21diskmemload11:18:433
21909980,3cyclictest0-21swapper/1111:49:553
21909980,2cyclictest0-21swapper/1110:50:003
21909980,1cyclictest0-21swapper/1108:20:203
21869987,1cyclictest0-21swapper/1011:35:252
21869987,1cyclictest0-21swapper/1010:25:212
21869987,1cyclictest0-21swapper/1008:00:252
21869987,1cyclictest0-21swapper/1007:15:212
21869985,3cyclictest0-21swapper/1009:45:012
21869985,3cyclictest0-21swapper/1009:45:002
21869980,8cyclictest0-21swapper/1012:25:142
21869980,1cyclictest0-21swapper/1009:33:012
21869980,1cyclictest0-21swapper/1009:33:002
21819982,3cyclictest0-21swapper/910:00:0011
21819981,4cyclictest0-21swapper/911:00:0011
21819980,8cyclictest0-21swapper/908:25:2711
21819980,1cyclictest0-21swapper/911:45:1511
21819980,1cyclictest0-21swapper/911:35:2811
21819980,1cyclictest0-21swapper/907:45:2111
21779987,1cyclictest0-21swapper/810:25:1910
21779986,1cyclictest16455-21smartctl07:25:1810
21779982,4cyclictest0-21swapper/807:15:0010
21779980,1cyclictest0-21swapper/812:30:1810
21779980,1cyclictest0-21swapper/809:10:2010
21779980,1cyclictest0-21swapper/808:20:2110
21729987,1cyclictest0-21swapper/712:34:349
21729987,1cyclictest0-21swapper/710:35:249
21729987,1cyclictest0-21swapper/709:45:249
21729987,1cyclictest0-21swapper/709:45:239
21729987,1cyclictest0-21swapper/707:10:149
21729982,4cyclictest0-21swapper/707:55:009
21729981,5cyclictest0-21swapper/710:35:009
21729980,8cyclictest0-21swapper/710:00:259
21729980,8cyclictest0-21swapper/709:30:179
21729980,8cyclictest0-21swapper/709:30:179
21729980,8cyclictest0-21swapper/709:00:179
21729980,1cyclictest0-21swapper/712:35:219
21729980,1cyclictest0-21swapper/711:35:259
21729980,1cyclictest0-21swapper/710:25:249
21729980,1cyclictest0-21swapper/709:15:149
21619987,1cyclictest0-21swapper/512:05:257
21619987,1cyclictest0-21swapper/511:45:147
21619980,8cyclictest0-21swapper/511:25:217
21619980,1cyclictest0-21swapper/508:10:187
21579987,1cyclictest0-21swapper/410:55:186
21579987,1cyclictest0-21swapper/409:05:236
21579987,1cyclictest0-21swapper/408:55:146
21579987,1cyclictest0-21swapper/408:55:146
21579980,7cyclictest0-21swapper/407:55:126
21579980,2cyclictest24862-21grep09:40:286
21579980,2cyclictest24862-21grep09:40:286
21579980,1cyclictest0-21swapper/408:20:136
21529986,1cyclictest0-21swapper/307:15:195
21529981,6cyclictest0-21swapper/312:05:255
21529980,1cyclictest0-21swapper/312:15:225
21529980,1cyclictest0-21swapper/311:15:175
21479987,1cyclictest0-21swapper/212:20:194
21479987,1cyclictest0-21swapper/211:40:134
21479980,8cyclictest0-21swapper/211:00:144
21479980,8cyclictest0-21swapper/210:05:164
21479980,2cyclictest26222-21awk11:50:144
21479980,2cyclictest25484-21awk12:30:224
21479980,2cyclictest0-21swapper/212:00:144
21479980,1cyclictest0-21swapper/212:05:164
21479980,1cyclictest0-21swapper/209:20:014
21479980,1cyclictest0-21swapper/209:20:014
21479980,1cyclictest0-21swapper/208:50:204
21389987,1cyclictest0-21swapper/010:50:150
21389980,8cyclictest0-21swapper/011:00:210
21389980,8cyclictest0-21swapper/008:50:110
21389980,7cyclictest0-21swapper/011:09:280
1599281,5sleep30-21swapper/307:07:565
21909976,1cyclictest0-21swapper/1112:30:203
21909976,1cyclictest0-21swapper/1112:10:163
21909976,1cyclictest0-21swapper/1111:50:233
21909976,1cyclictest0-21swapper/1111:40:453
21909976,1cyclictest0-21swapper/1111:38:133
21909976,1cyclictest0-21swapper/1110:35:063
21909976,1cyclictest0-21swapper/1110:25:233
21909976,1cyclictest0-21swapper/1110:16:173
21909976,1cyclictest0-21swapper/1109:45:313
21909976,1cyclictest0-21swapper/1109:45:313
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional