Yorkshire in the 1880's: Joe Skinner marries Lily Whitmore, the woman he has long admired, to give a name to her illegitimate child by Lionel Fillmore, the opportunistic son of an impoverished aristocrat. Lionel, however, has his sights set on Victoria, the naive cousin of hard-working Bridget Mordaunt, and the wealth he wrongly assumes is hers. When Joe's shiftless brother Fred threatens his marriage plans, Lionel murders him and the blame falls on Joe. Bridget's warm regard for Joe sets her on a quest to prove his innocence, the pursuit of which reveals the sordid manipulations and evil that surround Lionel. Just when Lionel believes his crime will never be discovered, Douglas, his gentle sculptor brother finds the murder weapon---and the killer's identity.
In the 1830's in northern England, Riah Millican, a widow with three children, takes a job as housekeeper to a reclusive former teacher, Percival Miller. Miller makes Riah the gift of a black velvet gown, and even educates her children. But when Riah discovers the reason behind Miller's gifts, she vows to leave his house, but Miller has a hold on her, even after his death, when he leaves his house to her on the condition that she never marry. Riah's daughter, Biddy, grows up and becomes a laundress in a large house where her education keeps her from fitting in and makes her a target. But it also catches the eye of a son of the house, and with Miller's legacies, Biddy may yet find her way to happiness.