Near-field and radiation coupling between nearby radiating elements is unavoidable, and it is considered a limiting factor for applications in wireless communications and active sensing. This article proposes a density-based topology optimization approach to design decoupling networks for such systems. The decoupling network is designed by formulating an optimization problem that considers both energy transmission and reflection at the network ports. We replace the radiating elements by their time-domain impulse response for efficient computations and to enable the solution of the design problem using gradient-based optimization methods. We use the adjoint-field method to compute the gradients of the optimization objectives. Additionally, nonlinear filters are applied during the optimization procedure to impose minimum-size control on the optimized designs. We demonstrate the concept by designing the decoupling network for a two-element planar antenna array; the antenna is designed in a separate optimization problem. The optimized decoupling networks provide a signal path that destructively interferes with the coupling between the radiating elements while preserving their individual matching to the feeding ports. Compact decoupling networks capable of suppressing the mutual coupling by more than 10 dB between two closely separated planar antennas operating around 2.45 GHz are presented and validated experimentally.