Caswell County was founded in 1777 from the northern portion of Orange County. The county received its name from Richard Caswell, who was the Governor of North Carolina from 1776 to 1780. Originally because of the vast size of the county, the county seat was Leasburg, but after all the divisions of the county the finalization in 1792, Caswell Court House became the seat and has remained so under its current name of Yanceyville.
Caswell County has produced many political leaders and at one time there had been so many that it was said jokingly that in order to be a successful candidate, one had to be associated with Caswell County legislators. Bartlett Yancey, Jr. was the Speaker of the North Carolina Senate from 1817 to 1827 and it was during this same time that Romulus Mitchell Saunders was Speaker of the North Carolina House of Commons. Among other leaders have been Archibald Debow Murphey, Bedford Brown, Calvin Graves, John Kerr, Jacob Thompson.
The Slade family is responsible for discovering the bright-leaf tobacco curing process, which revolutionized the tobacco industry and brought great wealth to the residents and community of Caswell County.
Caswell County is divided into nine townships; Anderson, Dan River, Hightowers, Leasburg, Locust Hill, Milton, Pelham, Stoney Creek and Yanceyville.