James Francis Miller, congressman, stockman, and banker, son of Isaac and Susan (Swan) Miller, was born in Tennessee on August 1, 1832. He moved to Texas with his parents in 1845 and later attended Rutersville College, near La Grange. At the age of nineteen he was a teacher in Gonzales. He studied law and was admitted to the bar in 1854; in 1855 he was admitted to practice before the Texas Supreme Court. In 1857 he opened a practice in Gonzales. In 1860 he married Elmyra (Almira) Matthews, daughter of Gonzales pioneer William A. Matthews. At the beginning of the Civil War he enlisted as a private in Company I, Eighth Texas Cavalry, also known as Terry's Texas Rangers. Miller saw action at Shiloh, Perryville, Chickamauga, and several minor engagements. He was captured and later paroled, at Charlotte, North Carolina. While he was away fighting for the South, Elmyra died (in 1862), leaving an infant son who died the following year.
After the war Miller returned to Gonzales to reestablish his law practice. On May 10, 1868, he married Julia Turner Batchelor, daughter of Amasa Turner and widow of a Confederate soldier. Miller retired from the practice of law and in 1868 founded a banking house in Gonzales, in partnership with William B. Sayers. He later became the first president of the Texas Bankers Association. About the same time he began raising Durham, Holstein, and Jersey cattle. Highly successful in that endeavor, Miller later served as the first president of the Texas Live Stock Association. In 1883 he was elected to the United States Congress; he was reelected in 1885. During his first term he served on the Banking and Currency and Mines and Mining committees. During his second term he served as chairman of the Committee on Banking and Currency, retained his seat on the Committee on Mines and Mining, and was a member of the Committee on Education. Miller was a thirty-second-degree Scottish Rite Mason. He was made a Knight Templar in Houston and was a founder and first commander of the Gonzales Commandery. In 1873 he was elected grand master of Masons in Texas and in 1878 was elected grand commander of Knights Templar in Texas. He served as president of the council of high priests and as president of the board of directors of the Masonic Widows and Orphans' Home. He and his wife were Presbyterians. Miller died in Gonzales on July 3, 1902, and was buried in the Masonic Cemetery.