In this paper, we derive sharp lower bounds, also known as quantum speed limits, for the time it takes to transform a quantum system into a state such that an observable assumes its lowest average value. We assume that the system is initially in an incoherent state relative to the observable and that the state evolves according to a von Neumann equation with a Hamiltonian whose bandwidth is uniformly bounded. The transformation time depends intricately on the observable's and the initial state's eigenvalue spectrum and the relative constellation of the associated eigenspaces. The problem of finding quantum speed limits consequently divides into different cases requiring different strategies. We derive quantum speed limits in a large number of cases, and we simultaneously develop a method to break down complex cases into manageable ones. The derivations involve both combinatorial and differential geometric techniques. We also study multipartite systems and show that allowing correlations between the parts can speed up the transformation time. In a final section, we use the quantum speed limits to obtain upper bounds on the power with which energy can be extracted from quantum batteries.