Studies on high ability in mathematics rarely take a teacher’s perspective. The purpose of our study is to add such a perspective, which we will do by using positioning theory to analyze elementary teachers’ discussions on mathematical tasks, aiming to challenge all pupils, including the highly able. The study is conducted in the context of a two-year long teacher professional development program on high ability. Teachers expressed both their teaching and the mathematical tasks as helpful in orchestrating teaching suitable for highly able pupils. They highlight the opportunities given by some tasks as well as the importance of guiding highly able pupils to go further in such tasks. However, they expressed their own limited mathematical knowledge and time needed for pupils with learning difficulties as obstacles to orchestrate teaching for the highly able. The results show that it is important to, in close cooperation with teachers, further explore how to orchestrate teaching that challenges highly able pupils.