Peach Point Plantation, settled in December 1832, was the home of James Franklin Perry and his wife, Emily Austin Bryan Perry, and was considered by Emily's brother, Stephen F. Austin, to be his only home in Texas. It is situated between Jones Creek and the Brazos River, ten miles south of Brazoria on State Highway 36, and was named by Austin for the wild peach trees growing there. Of all the property in his colony, Peach Point was Austin's first choice as a homesite. Soon after his sister's family arrived in Texas on August 14, 1831, Austin drew up suggested plans for the Perrys' home. James began clearing land and built a cabin near Chocolate Bayou, where they lived several months before moving to Peach Point. In the construction at Peach Point Perry followed a number of Austin's suggestions about materials and design of the residence. In accordance with his later instructions, two rooms built at the east end of the house were set aside for his use as a bedroom and office. Austin also provided numerous seeds, slips, cuttings, and plants for the grounds.
Both the family and servants at the plantation were hard hit by cholera and other illnesses between June and October of 1833. During part of that time, no physicians or neighbors were available to check on them, and at the plantation itself almost no family members or servants were well enough to care for the sick. During the Runaway Scrape in the spring of 1836, after taking the family to San Jacinto Bay, Perry and several of his male slaves went on to Galveston Bay to help build fortifications. At least in part because of the absence of both family and slaves during this critical period, the entire cotton crop at Peach Point that year was only twenty-two bales. After Austin's death in West Columbia on December 27, 1836, his body was brought down the Brazos River aboard the steamboat Yellow Stone for burial in the family cemetery at Peach Point. In 1910 Austin's remains were moved to the State Cemetery in Austin.
In the earliest years cotton was the primary cash crop at Peach Point, and corn and vegetables were grown for use on the plantation. Hogs, poultry, and cattle were raised there to provide meat, dairy products, and eggs for the family and servants. Excess quantities of farm products ranging from kegs of eggs to hard soap were sold to Robert Mills and other merchants in the area for credit toward the purchase of items that could not be readily produced on the plantation. Perry began to experiment with raising sugarcane and processing sugar by 1845. Under his direction and later that of his son, Stephen, sugar became the plantation's primary crop in the 1850s, when a sugar mill was built. A brick wall, cistern, and some of the metal piping for that mill remained on the property in the late 1980s. A communal kitchen prepared food for both family and slaves, but the slaves were also allowed to cook over fireplaces in their cabins. They had access to a variety of foods, including sugar and molasses made on the plantation, produce from their own kitchen gardens, and eggs from chickens they raised themselves. They were allowed to farm small plots for gardens, and some had an acre on which they raised cotton that was sold and credited to their accounts with merchants. They were paid a dollar a day when they were required to work on Sundays in peak periods, such as sugar-making time. Most work clothes for the slaves were made on the plantation, of heavy-duty fabrics purchased in large quantities.
Rutherford B. Hayes, a classmate of Emily Perry's son, Guy M. Bryan, visited Peach Point in January 1848. He described the home as being "delightfully situated in the edge of the timber, looking out upon a plain on the south extending five or eight miles to the Gulf" and mentioned the "large beautiful flower garden in front." During the Civil War and Reconstruction, Peach Point, like other Texas plantations, declined sharply in productivity and profitability, and much of the land was sold to satisfy debts. Discovery of oil on the property in the 1930s brought better times, and some of the original Peach Point land was bought back by S. S. Perry, Sr., and S. S. Perry, Jr. With the exception of the two rooms set aside for Stephen F. Austin's use, the plantation house was destroyed by a hurricane in 1909. By 1948 those two rooms had deteriorated badly. S. S. Perry, Sr., and his daughter, May Perry Hamill, undertook the painstaking restoration of these rooms, which are now furnished primarily with original Austin and Perry family heirlooms and are shown by the present owners to visitors on special occasions. A new residence, built at Peach Point by Perry, Sr., in 1949, allows family members once again to live within a few feet of the original homesite.