The quality of service monitoring forms a key element of the current light-handed regulation at Australian airports. The ACCC (Australian Competition and Consumer Commission) evaluates and publicly reports the quality of service levels of the four largest airports on a yearly basis to pressure airports to maintain an acceptable service performance. This article aims to provide an in-depth analysis of the methodology used by the ACCC. This analysis includes a critical review of the methodology based on secondary information in combination with primary research (i.e., data from 21 semi-structured interviews) that considers the current perception of the methodology among key stakeholder groups. The research finds that the methodology used by the ACCC is underpinned by some limitations, putting in question its effectiveness, reliability and validity. Particularly, its weak design does not allow for a comprehensive interpretation of the reported results or a reliable comparison across monitored airports, thus reduces transparency. Stakeholders pointed out that it is not possible to evaluate whether an airport undertakes infrastructure investments that ensure both the efficiency of ongoing airport operations and appropriate levels of service quality. These limitations add to the perception that the ACCC in its current function is not a ‘credible threat’ to airports with market power. Recommendations and future research directions are provided to address the identified limitations.