Carter made it clear in the 2004 off-season that he wanted to be traded from the Raptors. On December 17, 2004, Raptors General Manager Rob Babcock traded Carter to the New Jersey Nets for Alonzo Mourning, Eric Williams, Aaron Williams and two future first-round draft picks. The trade to the Nets, combined with Carter's continually declining popularity with Raptors fans, also put an end to his charity basketball games in Toronto, with the last one being played in the summer of that year.

In early January 2005, he stated during a television interview with TNT's John Thompson to not giving effort in his last months as a Raptor; when asked if he always played hard, Carter replied, "In years past, no. I was fortunate to have the talent. You get spoiled when you're able to do a lot of things. You see that you don't have to work at it." However, it was shown later that his comments were miscontrued and edited for television. Thompson himself later stated, "“That boy never said to