Image matting is an essential image processing technology due to its wide range of applications. Samplingbased image matting is one of the main branches of image matting research that estimates alpha mattes by selecting the best pixel pairs. It is essentially a large-scale multi-peak optimization problem of pixel pairs. Previous study shows that particle swarm optimization (PSO) can effectively optimize the pixel pairs. However, it still suffers from premature convergence problem which often occurs in pixel pair optimization that involves a large number of local optima. To address this problem, this work presents a parameter-free strategy for PSO called adaptive convergence speed controller (ACSC). ACSC monitors and conditionally controls the particles by competitive pixel pair recombination operator (CPPRO) and pixel pair reset operator (PPRO) during the iteration. ACSC performs CPPRO to improve the competitiveness of a particle when the performance of most of the pixel pairs is worse than that of the best-so-far solution. PPRO is performed to avoid premature convergence when the alpha mattes regarding two selected particles are highly similar. Experimental results show that ACSC significantly enhances the performance of PSO for image matting and provides competitive alpha mattes comparing with state-of-the-art evolutionary algorithms.