Sumpter, a ghost town five miles east of Groveton in central Trinity County, served as the first seat of government after the county was established in February 1850. The home of Solomon Adams, the first White settler in the area, was designated the temporary site for various courts until permanent facilities could be erected. Meanwhile, the state legislature appointed the first commissioners to organize the town, which the legislators named for either the community or the county of Sumpter, Alabama. Although a post office opened in December 1851, the town was not formally laid out until November 20, 1855. With the hope that the area would become a railroad center, business and professional men flocked to Sumpter, which was incorporated on January 11, 1862.
Following the Civil War the railroad bypassed Sumpter, and the businesses began to leave. The courthouse burned on November 2, 1872, and the town of Trinity became county seat on May 30, 1873. The post office at Sumpter closed in 1874. The lumber companies, having cut down the forests, departed, and farmers sold out and moved. The number of school children decreased to twenty-six in 1884 and twelve in 1895. Even houses were moved. Sumpter, once the largest town in the county and sometime home of outlaw John Wesley Hardin, became a ghost town. A cemetery, a few large trees, and a historical marker in a field remain at the site.