You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-04 - 05:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Tue Aug 04, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2834911050,97ptp4l0-21swapper/119:05:551
2143226856,10sleep100-21swapper/1019:07:032
2147826554,9sleep40-21swapper/419:07:456
2161926353,8sleep110-21swapper/1119:09:523
2161326052,6sleep50-21swapper/519:09:477
1835125647,7sleep70-21swapper/719:05:129
2145425443,9sleep60-21swapper/619:07:238
2150225342,9sleep00-21swapper/019:08:050
2203199310,1cyclictest0-21swapper/1022:02:192
2199599193,11cyclictest0-21swapper/219:10:004
2199599193,11cyclictest0-21swapper/200:10:014
2199099192,11cyclictest0-21swapper/122:55:011
156422190,1sleep111009-21runrttasks19:40:193
283491180,1ptp4l21-21ksoftirqd/121:50:101
283491180,0ptp4l0-21swapper/120:10:221
2199599184,9cyclictest0-21swapper/221:05:014
2199599183,10cyclictest0-21swapper/220:35:004
2199599183,10cyclictest0-21swapper/220:25:004
2199599183,10cyclictest0-21swapper/219:55:004
2199099183,8cyclictest0-21swapper/122:25:011
2199099183,8cyclictest0-21swapper/122:25:001
283491170,1ptp4l21-21ksoftirqd/122:05:161
22031991713,2cyclictest0-21swapper/1019:35:162
2202299173,8cyclictest0-21swapper/819:45:0010
2200899172,8cyclictest0-21swapper/519:45:387
2199599174,8cyclictest0-21swapper/222:10:014
2199599173,9cyclictest0-21swapper/222:00:014
2199599173,10cyclictest0-21swapper/223:40:014
2199099173,8cyclictest0-21swapper/120:40:001
2199099173,8cyclictest0-21swapper/100:25:011
283491160,1ptp4l21-21ksoftirqd/122:50:011
2203599164,8cyclictest0-21swapper/1120:24:593
2203599163,7cyclictest0-21swapper/1120:30:003
2203199162,9cyclictest0-21swapper/1023:30:012
2202699160,1cyclictest0-21swapper/922:02:1911
2201799163,8cyclictest0-21swapper/722:55:019
2201399163,8cyclictest0-21swapper/620:45:018
2200499160,4cyclictest0-21swapper/423:30:016
2199599164,8cyclictest0-21swapper/219:15:264
2199599163,8cyclictest0-21swapper/223:40:224
2199599163,8cyclictest0-21swapper/200:40:014
2199099164,7cyclictest0-21swapper/100:10:011
2199099163,8cyclictest0-21swapper/123:50:001
2199099163,8cyclictest0-21swapper/121:30:001
2199099161,14cyclictest0-21swapper/100:35:151
2198699163,8cyclictest0-21swapper/023:40:010
2198699163,8cyclictest0-21swapper/000:30:010
2198699163,8cyclictest0-21swapper/000:05:010
214562167,7sleep80-21swapper/819:07:2510
283491150,1ptp4l21-21ksoftirqd/119:20:201
283491150,0ptp4l0-21swapper/121:35:131
2201799153,7cyclictest0-21swapper/722:20:009
2199599153,8cyclictest0-21swapper/222:05:264
2199599150,9cyclictest0-21swapper/220:05:004
2198699153,7cyclictest0-21swapper/022:26:550
2198699153,7cyclictest0-21swapper/022:26:550
2198699153,7cyclictest0-21swapper/022:15:010
2198699153,7cyclictest0-21swapper/019:26:140
2198699152,7cyclictest0-21swapper/021:15:000
2203599142,7cyclictest0-21swapper/1100:25:013
2201399141,8cyclictest0-21swapper/623:45:018
2200899143,8cyclictest0-21swapper/519:55:007
2199099142,7cyclictest0-21swapper/121:45:011
2199099142,7cyclictest0-21swapper/120:50:001
2199099142,7cyclictest0-21swapper/119:35:231
2198699143,7cyclictest0-21swapper/019:20:000
2198699143,6cyclictest0-21swapper/020:05:000
2198699142,7cyclictest0-21swapper/020:00:000
2198699141,8cyclictest0-21swapper/019:45:000
166552140,0sleep30-21swapper/321:05:195
283491130,4ptp4l0-21swapper/122:30:241
283491130,4ptp4l0-21swapper/122:30:241
283491130,1ptp4l21-21ksoftirqd/121:30:151
283491130,0ptp4l0-21swapper/123:55:201
283491130,0ptp4l0-21swapper/123:50:201
283491130,0ptp4l0-21swapper/122:45:251
283491130,0ptp4l0-21swapper/122:45:251
283491130,0ptp4l0-21swapper/122:00:131
283491130,0ptp4l0-21swapper/120:25:131
283491130,0ptp4l0-21swapper/119:15:141
283491130,0ptp4l0-21swapper/100:00:181
2203199132,7cyclictest0-21swapper/1019:25:232
2202299132,7cyclictest0-21swapper/820:20:0010
2201399132,7cyclictest0-21swapper/620:55:018
2200899130,1cyclictest0-21swapper/523:20:147
2200899130,1cyclictest0-21swapper/522:20:127
2200899130,1cyclictest0-21swapper/522:20:127
2200499132,7cyclictest0-21swapper/423:50:236
2199599133,7cyclictest0-21swapper/221:45:004
2199599132,8cyclictest0-21swapper/200:20:014
2199599132,7cyclictest18046-21diskmemload22:45:014
2199599132,7cyclictest18046-21diskmemload22:45:014
2198699130,8cyclictest0-21swapper/000:25:010
283491120,4ptp4l21-21ksoftirqd/121:10:151
283491120,3ptp4l3657-21cstates20:50:131
283491120,1ptp4l21-21ksoftirqd/122:10:131
283491120,1ptp4l21-21ksoftirqd/120:56:181
283491120,0ptp4l0-21swapper/123:25:251
283491120,0ptp4l0-21swapper/123:15:221
283491120,0ptp4l0-21swapper/122:15:201
283491120,0ptp4l0-21swapper/119:55:131
283491120,0ptp4l0-21swapper/119:12:421
283491120,0ptp4l0-21swapper/100:15:171
2203599122,6cyclictest0-21swapper/1100:20:013
2203599122,6cyclictest0-21swapper/1100:00:253
2203599120,11cyclictest0-21swapper/1123:50:003
2203199122,6cyclictest0-21swapper/1023:10:252
2202299122,6cyclictest0-21swapper/820:00:0010
2202299121,6cyclictest0-21swapper/822:40:2410
2201799128,3cyclictest0-21swapper/721:10:179
2201799123,2cyclictest1009-21runrttasks21:19:499
2201799122,7cyclictest8682-21kworker/u24:121:49:599
2201799122,6cyclictest0-21swapper/700:25:249
2201799120,4cyclictest32513-21irqrtprio19:20:199
2201799120,3cyclictest32436-21sh20:45:149
2201799120,11cyclictest0-21swapper/723:20:249
2200899129,2cyclictest24072-21turbostat22:00:007
2200899123,6cyclictest0-21swapper/522:40:017
2200899123,6cyclictest0-21swapper/522:40:017
2200499122,6cyclictest0-21swapper/420:25:236
2199099123,6cyclictest0-21swapper/120:20:261
2199099122,6cyclictest0-21swapper/121:15:241
2199099120,7cyclictest0-21swapper/119:35:001
2198699122,7cyclictest0-21swapper/000:35:010
283491110,2ptp4l13228-21munin-run23:10:001
283491110,1ptp4l0-21swapper/123:40:221
283491110,1ptp4l0-21swapper/122:40:001
283491110,1ptp4l0-21swapper/122:39:591
283491110,1ptp4l0-21swapper/121:40:181
283491110,1ptp4l0-21swapper/120:30:171
283491110,0ptp4l0-21swapper/123:35:121
2203599112,6cyclictest0-21swapper/1123:30:243
2203599111,6cyclictest0-21swapper/1100:30:013
2203199112,6cyclictest0-21swapper/1023:40:012
2203199110,2cyclictest3699-21sed22:55:152
2203199110,10cyclictest0-21swapper/1023:20:002
2202299110,5cyclictest0-21swapper/800:05:0110
2201799115,6cyclictest0-21swapper/719:50:229
2201799113,4cyclictest11960-21if_err_p34p100:30:189
2201799113,3cyclictest7289-21cat23:40:229
2201799113,3cyclictest3196-21ptp4l-jitter20:05:199
2201799113,3cyclictest29951-21cut20:00:149
2201799113,2cyclictest13641-21latency_hist19:40:009
2201799112,3cyclictest3885-21wc21:30:239
2201799112,3cyclictest24287-21cat22:00:009
2201799112,3cyclictest17310-21cat23:55:179
2201799111,3cyclictest15320-21fschecks_time00:35:139
2201799110,3cyclictest690-21wc23:35:139
2201799110,3cyclictest28973-21cstates22:05:159
2201799110,3cyclictest19279-21sensors_fan21:50:229
2201799110,3cyclictest13144-21grep23:50:139
2201399112,6cyclictest0-21swapper/600:00:008
2201399112,5cyclictest0-21swapper/623:35:258
2200899117,3cyclictest27184-21irqrtprio19:55:197
2200899113,2cyclictest16702-21switchtime00:35:227
2200899110,1cyclictest0-21swapper/522:41:227
2200499118,2cyclictest17036-21turbostat23:14:596
2199599118,2cyclictest13836-21turbostat00:34:594
2199599118,2cyclictest0-21swapper/219:20:164
2199599113,7cyclictest0-21swapper/219:35:004
283491100,2ptp4l29942-21grep20:00:131
283491100,2ptp4l29400-21irqrtprio20:40:171
283491100,1ptp4l3567-21tune2fs19:25:141
283491100,1ptp4l27932-21expr00:10:111
283491100,1ptp4l21-21ksoftirqd/123:10:551
283491100,1ptp4l11218-21cat21:00:111
283491100,1ptp4l10379-21irqrtprio20:15:181
283491100,1ptp4l0-21swapper/123:30:241
283491100,1ptp4l0-21swapper/123:00:141
283491100,1ptp4l0-21swapper/122:20:121
283491100,1ptp4l0-21swapper/122:20:121
283491100,1ptp4l0-21swapper/119:50:171
283491100,1ptp4l0-21swapper/119:40:251
283491100,0ptp4l0-21swapper/122:40:181
283491100,0ptp4l0-21swapper/121:55:061
283491100,0ptp4l0-21swapper/121:05:151
283491100,0ptp4l0-21swapper/100:27:121
247732100,0sleep40-21swapper/422:40:186
2203599100,9cyclictest0-21swapper/1119:30:133
2203599100,3cyclictest19039-21irqrtprio23:15:183
2203199100,9cyclictest10414-21awk21:00:002
2203199100,9cyclictest0-21swapper/1023:20:112
2203199100,9cyclictest0-21swapper/1019:20:222
2201799103,4cyclictest0-21swapper/721:40:019
2201799103,3cyclictest11859-21cut21:00:139
2201799103,2cyclictest1271-21latency_hist21:30:019
2201799103,2cyclictest10224-21cpuspeed_turbos23:05:139
2201799103,1cyclictest8830-21sendmail_mailqu19:30:209
2201799102,3cyclictest23364-21fschecks_time19:10:149
2201799102,3cyclictest19133-21fw_conntrack19:45:309
2201799102,2cyclictest14840-21gzip20:20:239
2201799102,1cyclictest185672sleep722:33:579
2201799102,1cyclictest185672sleep722:33:579
2201799101,3cyclictest26086-21cut22:00:199
2201799101,3cyclictest16103-21missed_timers19:40:209
2201799100,9cyclictest0-21swapper/721:10:009
2201799100,4cyclictest28078-21grep20:40:129
2201799100,4cyclictest27682-21irqrtprio21:20:189
2201799100,3cyclictest8812-21aten2_r3power_p20:15:139
2201799100,3cyclictest877-21sed00:15:199
2201799100,3cyclictest8516-21sh22:20:119
2201799100,3cyclictest8516-21sh22:20:119
2201799100,3cyclictest8272-21fw_forwarded_lo20:55:159
2201799100,3cyclictest793-21tune2fs22:10:139
2201799100,3cyclictest7384-21idleruntime23:00:189
2201799100,3cyclictest6494-21cat19:30:009
2201799100,3cyclictest6210-21expr21:35:139
2201799100,3cyclictest5778-21switchtime20:50:259
2201799100,3cyclictest3280-21expr00:20:149
2201799100,3cyclictest30105-21grep23:30:159
2201799100,3cyclictest30101-21cat00:10:239
2201799100,3cyclictest27515-21expr22:45:169
2201799100,3cyclictest27515-21expr22:45:169
2201799100,3cyclictest27455-21cat20:40:009
2201799100,3cyclictest26293-21expr19:55:139
2201799100,3cyclictest26271-21cat19:15:019
2201799100,3cyclictest24376-21expr00:05:159
2201799100,3cyclictest2389-21wc22:55:119
2201799100,3cyclictest22629-21cat22:40:009
2201799100,3cyclictest22629-21cat22:40:009
2201799100,3cyclictest21579-21sed00:00:199
2201799100,3cyclictest20459-21cpuspeed_turbos20:30:119
2201799100,3cyclictest17612-21aten2_r3power_p23:15:129
2201799100,3cyclictest17264-21fschecks_count20:25:149
2201799100,3cyclictest14003-21sort23:10:139
2201799100,3cyclictest11985-21expr22:25:149
2201799100,3cyclictest11985-21expr22:25:139
2201399101,6cyclictest0-21swapper/619:50:008
2201399100,9cyclictest759-21taskset21:28:168
2201399100,10cyclictest0-21swapper/621:30:188
2200899100,9cyclictest7346-21gltestperf23:00:177
2200899100,7cyclictest0-21swapper/523:55:207
2200899100,7cyclictest0-21swapper/523:11:287
2200899100,7cyclictest0-21swapper/520:34:147
2200899100,6cyclictest0-21swapper/522:32:537
2200899100,6cyclictest0-21swapper/522:32:527
2200899100,6cyclictest0-21swapper/519:18:337
2200899100,6cyclictest0-21swapper/500:16:037
2200899100,1cyclictest0-21swapper/523:18:427
2200899100,1cyclictest0-21swapper/523:05:497
2200899100,1cyclictest0-21swapper/522:47:327
2200899100,1cyclictest0-21swapper/522:47:317
2200899100,1cyclictest0-21swapper/522:10:367
2200899100,1cyclictest0-21swapper/521:49:217
2200899100,1cyclictest0-21swapper/521:37:487
2200899100,1cyclictest0-21swapper/521:33:277
2200899100,1cyclictest0-21swapper/520:52:007
2200899100,1cyclictest0-21swapper/520:48:027
2200899100,1cyclictest0-21swapper/520:10:297
2200899100,1cyclictest0-21swapper/520:07:517
2200899100,1cyclictest0-21swapper/519:37:337
2200899100,1cyclictest0-21swapper/519:26:547
2200899100,1cyclictest0-21swapper/500:27:407
2200899100,1cyclictest0-21swapper/500:22:487
2200899100,1cyclictest0-21swapper/500:12:127
2200899100,1cyclictest0-21swapper/500:06:257
2200899100,10cyclictest0-21swapper/520:40:247
2199599100,9cyclictest0-21swapper/220:10:124
2198699108,1cyclictest0-21swapper/020:50:190
2198699102,4cyclictest0-21swapper/023:10:010
188842100,7sleep90-21swapper/919:05:1411
28349190,1ptp4l21-21ksoftirqd/119:45:011
28349190,1ptp4l21-21ksoftirqd/100:30:131
28349190,0ptp4l0-21swapper/123:20:121
28349190,0ptp4l0-21swapper/121:20:131
28349190,0ptp4l0-21swapper/120:05:121
220359997,1cyclictest0-21swapper/1121:00:173
220319990,8cyclictest0-21swapper/1019:45:292
220319990,1cyclictest0-21swapper/1020:40:182
220319990,1cyclictest0-21swapper/1020:20:172
220319990,1cyclictest0-21swapper/1000:35:132
220269998,1cyclictest0-21swapper/920:40:1211
220269996,2cyclictest0-21swapper/919:15:0011
220269990,8cyclictest1009-21runrttasks23:45:2711
220229991,4cyclictest0-21swapper/821:15:0110
220229990,8cyclictest0-21swapper/820:15:0010
220179990,3cyclictest8624-21cpuspeed_turbos23:45:159
220179990,3cyclictest5213-21cat20:10:119
220179990,3cyclictest26243-21fschecks_time23:25:149
220179990,3cyclictest23234-21awk22:40:119
220139998,1cyclictest0-21swapper/621:20:148
220139992,6cyclictest115650irq/18-p34p121:53:028
220139992,6cyclictest115650irq/18-p34p100:14:298
220139992,5cyclictest0-21swapper/619:15:008
220139991,5cyclictest0-21swapper/623:00:018
220139991,5cyclictest0-21swapper/620:10:008
220139990,9cyclictest0-21swapper/622:55:238
220139990,9cyclictest0-21swapper/621:35:158
220139990,8cyclictest11910-21tr22:25:148
220139990,8cyclictest11910-21tr22:25:138
220139990,3cyclictest3189-21cstates19:25:128
220139990,3cyclictest28681-21cpuspeed_turbos22:05:158
220139990,3cyclictest25750-21wc22:40:248
220139990,3cyclictest12108-21irqrtprio00:30:188
220089993,2cyclictest28972-21timerwakeupswit21:20:267
220089991,5cyclictest0-21swapper/522:10:017
220089990,6cyclictest0-21swapper/523:30:387
220089990,6cyclictest0-21swapper/520:37:237
220089990,6cyclictest0-21swapper/520:28:087
220089990,6cyclictest0-21swapper/519:13:407
220089990,6cyclictest0-21swapper/500:02:077
220089990,1cyclictest0-21swapper/522:17:157
220049998,1cyclictest0-21swapper/423:30:166
220049998,1cyclictest0-21swapper/423:15:256
220049996,2cyclictest21918-21fw_forwarded_lo21:55:156
220049992,5cyclictest0-21swapper/422:25:266
220049992,5cyclictest0-21swapper/422:25:266
220049991,5cyclictest14750irq/20-ata_piix00:40:016
220049991,5cyclictest0-21swapper/423:10:016
220049990,1cyclictest0-21swapper/420:50:246
219959996,2cyclictest22747-40updatedb00:00:294
219959990,9cyclictest0-21swapper/220:55:004
219959990,1cyclictest13004-21strings21:00:184
219959990,1cyclictest0-21swapper/223:15:134
219959990,1cyclictest0-21swapper/219:40:144
219869996,2cyclictest28502-21turbostat23:34:550
219869992,6cyclictest0-21swapper/021:20:250
219869990,9cyclictest0-21swapper/022:55:110
219869990,9cyclictest0-21swapper/021:55:210
219869990,8cyclictest13933-21chrt21:00:520
219869990,8cyclictest0-21swapper/022:00:130
219869990,1cyclictest0-21swapper/021:15:220
219869990,1cyclictest0-21swapper/020:20:170
21325293,5sleep30-21swapper/319:05:325
220359980,1cyclictest0-21swapper/1120:35:213
220359980,1cyclictest0-21swapper/1119:15:263
220319987,1cyclictest0-21swapper/1019:15:162
220319980,8cyclictest0-21swapper/1022:15:142
220319980,8cyclictest0-21swapper/1021:25:132
220319980,1cyclictest0-21swapper/1022:40:192
220319980,1cyclictest0-21swapper/1020:05:232
220319980,1cyclictest0-21swapper/1000:00:202
220269987,1cyclictest0-21swapper/923:35:2011
220269987,1cyclictest0-21swapper/922:35:1411
220269987,1cyclictest0-21swapper/922:35:1311
220269987,1cyclictest0-21swapper/922:05:1311
220269987,1cyclictest0-21swapper/921:10:2111
220269987,1cyclictest0-21swapper/921:05:1411
220269987,1cyclictest0-21swapper/920:35:1411
220269987,1cyclictest0-21swapper/919:55:2111
220269987,1cyclictest0-21swapper/900:25:1811
220269986,1cyclictest0-21swapper/923:10:3511
220269980,8cyclictest0-21swapper/923:30:5411
220269980,8cyclictest0-21swapper/923:20:1211
220269980,8cyclictest0-21swapper/923:15:2011
220269980,8cyclictest0-21swapper/923:05:1311
220269980,8cyclictest0-21swapper/922:40:2311
220269980,8cyclictest0-21swapper/900:15:1611
220269980,1cyclictest0-21swapper/921:40:1611
220269980,1cyclictest0-21swapper/921:35:2211
220269980,1cyclictest0-21swapper/921:30:2311
220269980,1cyclictest0-21swapper/900:10:1211
220229987,1cyclictest0-21swapper/821:25:1410
220229985,2cyclictest24078-21awk22:00:0010
220229981,5cyclictest2325-21taskset20:46:2310
220229980,8cyclictest0-21swapper/823:20:1410
220139987,1cyclictest0-21swapper/621:15:268
220139987,1cyclictest0-21swapper/620:25:198
220139987,1cyclictest0-21swapper/619:10:228
220139987,1cyclictest0-21swapper/600:20:138
220139982,5cyclictest115650irq/18-p34p123:31:428
220139982,5cyclictest115650irq/18-p34p123:15:088
220139982,5cyclictest115650irq/18-p34p123:10:588
220139982,5cyclictest115650irq/18-p34p122:21:568
220139982,5cyclictest115650irq/18-p34p122:21:568
220139982,5cyclictest115650irq/18-p34p121:45:048
220139982,5cyclictest115650irq/18-p34p100:05:158
220139981,6cyclictest115650irq/18-p34p122:30:168
220139981,6cyclictest115650irq/18-p34p122:30:168
220139981,6cyclictest115650irq/18-p34p100:00:158
220139980,7cyclictest28659-21date23:30:018
220139980,7cyclictest16343-21irqrtprio21:05:188
220139980,7cyclictest0-21swapper/622:15:238
220139980,3cyclictest31411-21cpuspeed00:15:148
220139980,3cyclictest14761-21/usr/sbin/munin23:50:238
220139980,2cyclictest16084-21irqstats00:35:178
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional