In the last days of the Civil War, Vermonter Norman Pelham receives a head wound that leaves him unconscious and separated from his regiment. He is found and nursed back to health by Leah, an escaped slave. When the war ends they travel north, back to his family's farm. Though the community thinks it strange that a native son has returned with a mulatto wife, Norman and Leah don't care much about what others think. In true Yankee fashion they keep to themselves, too busy with tending their animals and fields to bother with their neighbors. Leah gives birth to three children: beautiful Abigail, sturdy Prudence, and an emotional, impulsive boy named Jamie. For all her outward contentment, though, Leah is tormented by questions in her past that eventually compel her to journey back to Sweetboro, North Carolina, the town from which she had fled. Returning home a few weeks later, she is dejected, silent, and utterly changed. Incapable of sharing her experience with anyone, including her husband, she devastates her family by committing suicide.

Constantly teased by other boys about his mixed blood and his mother's death, Jamie leaves home as a teenager and cuts off all ties with his family. In the granite town of Barre, Vermont, he falls in love with a French-Canadian torch singer named Joey and works briefly for a Mafia bootlegger. During one incident, while collecting money that is owed to him, Jamie commits an act of violence that will return to haunt him later in life. He and Joey settle in a resort town in New Hampshire's White Mountains, where they marry and have two children. Jamie builds a bootlegging business of his own, and, when his wife and daughter die in the influenza epidemic of 1918, Jamie is left alone to raise his young son, Foster. When Jamie dies, Foster finds letters between his father and his aunt Abigail, prompting him to visit his aunts in Vermont. Having been told nothing about his father's past or his own family history, Foster is driven to uncover the mystery behind Leah's suicide, eventually traveling to North Carolina and confronting the man and the lies that caused his grandmother's death.

As in the novels of William Faulkner and Toni Morrison, In the Fall is marked with the violence and betrayal, shame and self-hatred of slavery. And for those who have suffered from its taint, coming to terms with the past is a task both dangerous and necessary.