Hirsch began acting at the age of eight, appearing in minor roles on television shows and made-for-television films, including Kindred: the Embraced; Sabrina, the Teenage Witch and NYPD Blue. He made his film debut with the 2002 drama, The Dangerous Lives of Altar Boys, where he starred opposite Kieran Culkin in the story of two Catholic school boys. His next role was in the prep school drama, The Emperor's Club, which was released later in 2002. Both films received generally positive reviews, but performed only moderately at the box office.

Hirsch was subsequently cast opposite Elisha Cuthbert in the 2004 teen comedy, The Girl Next Door, and as a result gained renown among teenage audiences. The Mudge Boy, an independent film that he had starred in before making Girl Next Door, was given a limited release in the summer of 2004. Hirsch next appeared in Imaginary Heroes, a drama about a dysfunctional family, which received a limited release in February 2005 but failed to reach