Mitigation measures to reduce underwater levels of radiated noise are of increasing interest. An example implementation is the Vancouver Fraser Port Authority’s ECHO (Enhancing Cetacean Habitat and Observation) Program that has initiated voluntary, seasonal vessel slowdowns since 2017. ECHO’s goal is to reduce the acoustic disturbance of commercial shipping on endangered whales, whose critical habitat coincides with major shipping routes. This initiative requires the cooperation of many stakeholders, and it is therefore imperative that acoustic monitoring along shipping routes suitably evaluates the effectiveness of such mitigation measures. However, doing so is not straightforward, as levels of ambient noise fluctuate not just in response to vessel slowdowns but also with confounding noise sources of environmental, biological, or anthropogenic origin (e.g., noise from currents, wind, marine mammals, small boats, etc.). Here, methods are presented for how to appropriately compare noise levels during baseline and mitigation periods, using data from one acoustic monitoring station as an example. Factors to consider when evaluating the effectiveness of mitigation measures are highlighted, including investigating suitable covariates and thresholds of these, both of which may vary between sites and seasonally at the same site. Useful ways of communicating results to a variety of stakeholders are also presented.