Brownwood is on Pecan Bayou at the intersection of U.S. highways 67, 84, and 377, Farm Road 2524, and the Atchison, Topeka and Santa Fe Railway in south central Brown County. The city and the county are named for Henry Stevenson Brown. The area was settled by farmers and cattle ranchers like Welcome W. Chandler and J. H. Fowler. When the sparsely populated county was organized in 1857, the hamlet of Brownwood was chosen as county seat. A post office was opened in the town the following year. The town was originally located on the east side of Pecan Bayou, but in the late 1860s a land-title dispute and problems with an inadequate water supply induced the residents to move to a sixty-acre site on the west side of the bayou donated by Greenleaf Fisk. Brownwood Masonic Lodge was chartered in 1865. As late as 1872 Brownwood was a small community of two stores, a log courthouse, and about five dwellings. In 1873 John Y. Rankin purchased land around the business district and began to build homes in what became known as the Rankin Addition. In 1876, when the town had an estimated 120 inhabitants and Cumberland Presbyterian, Presbyterian, and Baptist churches, the first bank was opened and a schoolhouse was built that also served as a town hall and a church. Because Brownwood lay on a feeder line of the Western Trail, stores and saloons served the needs of the cowboys who drove the herds through town. A cotton gin was built in town in 1877 as the state of Texas began to offer the land to farmers.
The 1880s and 1890s were decades of dramatic growth for the community, as the population increased from 725 in 1880 to 2,176 in 1890 and 3,965 in 1900. The town became a center of the Farmers' Alliance with the building of the West Texas District Alliance Cotton Yard and the establishment of an alliance paper, the weekly Freemans Journal. Other newspapers that have been published in the community include the Brownwood Gazette, Bulletin, and Appeal, the Pecan Valley News, the Texas Immigration and Stock Farmer, Living Issues, and the Brown County Banner. In 1885 the Brownwood Daily Bee became the town's first daily paper. The courthouse burned in 1880, and a new one was completed in 1884. In 1884, when Brownwood incorporated, the town had two banks, nine general stores, five saloons, two hotels, and steam cotton and grist mills. The following year the Gulf, Colorado and Santa Fe Railroad built through Brownwood, and in 1886 the town built its first waterworks. An opera house was built in the late 1880s. By 1890 the town had five churches, an icehouse, a fire department, and a sanitarium. A second railroad, the Fort Worth and Rio Grande, built through Brownwood in 1891. There were also significant developments in education during these years. Several local schools were consolidated to form the Brownwood Independent School District in 1883. In 1889 two colleges opened their doors in Brownwood—Daniel Baker College, founded by the Presbyterians, and Howard Payne College, a Baptist institution. Daniel Baker closed, and its campus became part of Howard Payne College in 1953.
By 1900, the city was dominated by the cotton industry. It supported the West Texas Compress Company, the Brownwood Cotton Oil Mill, and sixteen cotton gins. Severe flooding that year covered parts of the town with up to ten feet of water. Prohibition became popular in the town in the 1890s, and in 1903, after a series of violent incidents, the town's saloons were closed. Equally rowdy drinking clubs sprang up in response to the saloon closings, and a stretch of one street became known as "Battle Row." The population of Brownwood continued to climb, reaching 6,967 in 1910 and 8,223 in 1920, and in spite of the boll weevil infestation and other problems in the cotton industry, the city was the largest cotton-buying center west of Fort Worth by 1920. A third railroad, the Brownwood North and South, was built in 1912 to connect Brownwood with the Brown County community of May, but the short-lived railroad lost money and was abandoned in 1927. Brownwood went through two boom periods in the first half of the twentieth century, the first stimulated by the oil industry, the second by the building of a military installation during World War II. Oil was first discovered near Brownwood in 1917, but the town did not become a major oil-industry site until the 1920s. The population of the city shot up from 8,223 in 1920 to 12,789 in 1930, and estimates from the late 1920s indicate it might have been as high as 15,000. At one time during the boom the city had twenty-five manufacturing and industrial plants in operation. Brownwood's growth was also reflected in the changing percentage of county residents who lived in the town; though fewer than a quarter of Brown County residents lived in Brownwood in 1900, almost half lived in the city by 1930. The pace of development slowed in the 1930s, though the city benefited from the completion of Lake Brownwood in 1933 and several area WPA projects. In 1940 Brownwood had 13,398 inhabitants, some 52 percent of the Brown County population.
In the fall of 1940, as part of the military buildup associated with the peacetime draft, a military-training installation, Camp Bowie, was built to the south and southwest of town. The camp eventually became the largest military training center in Texas, and had a major impact on the city of Brownwood. By December 1940 the 13,500 workers at the camp exceeded the 1940 population of the city; by March 1941 Brownwood's population was counted at 22,479, and by some estimates the wartime population eventually reached considerably more than 50,000. The massive influx of workers, the families of military personnel, and soldiers seeking off-base housing created a severe housing shortage in Brownwood, which started the boom in September of 1940 with a mere 200 vacant dwelling units. City residents coped by turning a skating rink into a dormitory, by allowing patrons to bed down in a movie theater after the final show, by subdividing the existing housing stock, and by the building of additional housing units. Though much of the development in town was ephemeral, the wartime boom left the city with better roads and streets, improved sanitation and medical facilities, a municipal airport, and a supply of new housing and business units.
With the closing of Camp Bowie in 1946 Brownwood's wartime gains began to slip away. Attempts to sustain the growth rate of the war years were blocked by the seven-year drought of the late 1940s and early 1950s and its deleterious impact on the agribusiness of the region. In spite of the postwar depression, the city numbered 20,140 inhabitants, 71 percent of the county population, in 1950. Business revolved around wool, oil, poultry, livestock, peanuts, and pecans. Part of the business district burned down in 1953. Brownwood's population declined during the 1950s by more than 15 percent to 16,974 in 1960, and has remained relatively static over the next forty years, with 17,368 inhabitants in 1970, 19,203 in 1980, 18,387 in 1990, and 18,813 in 2000. The growth of nearby Early, located at the old site of Brownwood on the east side of Pecan Bayou has kept the combined population of the two cities at around 20,000; they comprised almost three-quarters of the county inhabitants in 1970. After the old city auditorium, the Memorial Hall, burned in 1960, the city built Brownwood Coliseum in 1963. In the 1970s Brownwood manufactured industrial and transportation equipment, furniture, clothing, woolen goods, crushed stone, livestock drenches, feeds, and also food, glass, plastic, and leather products. In the 1980s important businesses included meat packing, commercial printing, and the manufacture of plumbing fixtures, leather gloves, oilfield machinery and construction equipment. In 2000, with the development of other county communities, the Brownwood-Early area held only 57 percent of Brown County's population, but the city remained an important distributing center for the county and the region. Area attractions included Lake Brownwood State Recreation Area, Camp Bowie Memorial Park, and the Brown County Museum of History.