Baxter is on Farm Road 804 and U.S. Highway 175 five miles southeast of Athens in south central Henderson County. The site, like much of Henderson County, was settled during the 1850s. Baxter became a station for the Texas and New Orleans Railroad in 1900, and a Baxter post office operated from 1901 to 1912. In the early 1900s the community consisted of the railroad depot, a general store, a feed store, a cannery, and a school. There were two churches, a Methodist church near the school and a Baptist church near the crossroads where the cannery was located. In 1945 two businesses and a population of fifty-six were reported. The railroad was abandoned in the late 1970s. The community's population was reported as twenty from 1950 through 2000. In 2004 the former school building was used as a community center and the Baxter Baptist church was located on U.S. 175. There was also a volunteer fire department and a water supply company.