The linearized collision operator of the Boltzmann equation can in a natural way be written as a sum of a positive multiplication operator, the collision frequency, and an integral operator. Compactness of the integral operator for monatomic single species is a classical result, while corresponding results for mixtures and polyatomic single species where the polyatomicity is modeled by a discrete internal energy variable, are more recently obtained. In this work the compactness of the integral operator for polyatomic single species, for which the number of internal degrees of freedom is greater or equal to two and the polyatomicity is modeled by a continuous internal energy variable, is studied. Compactness of the integral operator is obtained by proving that its terms are, or, at least, can be approximated by, Hilbert-Schmidt integral operators, under some assumptions on the collision kernel. Self-adjointness of the linearized collision operator follows. Moreover, bounds on -including coercivity of -the collision frequency, are obtained for some particular collision kernels -corresponding to hard sphere like models, but also hard potential with cut-off like models. Then it follows that the linearized collision operator is a Fredholm operator.