Miller was born Johnnie Lucille Ann Collier in Chireno in Nacogdoches County in east Texas, the daughter of Clara Emma (née Birdwell) and John Alfred Collier, a criminal lawyer who represented the Barrow Gang, Machine Gun Kelly, and Baby Face Nelson, among others. Miller's maternal grandmother was Cherokee. Miller's father insisted on the name Johnnie because he had wanted a boy, but she was often called Annie. She took up dancing to exercise her legs to help her rickets. She was considered a child dance prodigy. In an interview featured in a "behind the scenes" documentary on the making of the compilation That's Entertainment III, she said that Eleanor Powell was an early inspiration.