Hoyt was born to Carl and Maxine Fortenberry in Bon Temps, Louisiana. His father committed suicide when Hoyt was 10-years-old. His mother finagled the insurance company with whom Carl had had a life insurance policy, and claimed that her husband had been murdered by a burglar. She repeated this same story to Hoyt, who grew up believing that his mother was too afraid to live alone because of an unknown assailant.
Hoyt was originally the shortest kid in his elementary school, and his best friend Jason Stackhouse often defended him against bullies. However, he experienced a rapid growth spurt in the eighth grade, and grew ten inches in height in a single school year, which left him physically awkward and uncoordinated. Again, Jason defended him from bullies who now picked on him for being too tall.