Rapid acoustic surveys aim to estimate biodiversity based on the diversity of sounds produced by animal communities, and interest in this approach among conservation planners is increasing. Several indices of acoustic diversity have been proposed as proxies for species richness. However, in the field, the animal activity may be responsible for acoustic diversity to varying degrees. To evaluate how measures of acoustic diversity may depart from actual species richness, we selected seven acoustic indices and applied them to simulated recordings of bird assemblages under different field conditions. For a given sound, defined here as the song composition emitted by a species assemblage, we determined if the indices were i) only driven by animal sounds, ii) insensitive to species identity, and iii) independent of species evenness. Under the field conditions evaluated, none of the indices fulfilled the three criteria necessary for a perfect proxy of species richness. However, some indices may be appropriate as a measure of biodiversity under a more broad definition including phylogenetic and/or functional aspects of diversity. We provide recommendations for the application of these indices for biodiversity measurement under field conditions, such as the application of appropriate audio filters, the increase of the repetition rate of the recordings, and the identification of the main taxonomic groups occurring in the recorded communities.