James Duff, freighter and Confederate irregular, was born in Logierait, Perthshire, Scotland about 1828. He emigrated to the United States where, in 1849, he enlisted as a private in the United States army. Promoted to sergeant, he left the army in 1854 to become an army sutler at Fort Belknap. He was a San Antonio merchant by 1856 when his wagons were used to transport a large meteorite (now known as the Wichita County Iron) to Austin. His wagons were also involved with the removal, directed by Robert S. Neighbors, of Texas Comanches to Indian Territory in 1859. Just before the Civil War, he married Harriet Paul, daughter of army officer and future Union army general Gabriel Paul. The couple had two daughters.
When the secession crisis occurred Duff was active in seizing San Antonio, and its army garrison, for the state. Duff served as commander of an irregular Texas Confederate military unit, Duff's Partisan Rangers. Unionism in the Hill Country brought about his assignment to that area in May 1862. He left his duty station at San Antonio, camped on the Pedernales a few miles west of Fredericksburg, and declared martial law in several precincts of Kerr and Gillespie counties. He dismissed the Unionist enrolling officer, Jacob Kuechler, and began what many Hill Country people regarded as a reign of unjustified terror. Captain Duff learned of Fritz Tegener's battalion and its planned departure for Mexico and sent part of his troops in a pursuit that culminated in the battle of the Nueces on August 10, 1862. One participant in the encounter testified to Duff's presence at the "massacre" and his refusal to provide medical assistance to the defeated Germans. Another story had Duff arriving there after the battle. Harassment of Unionists continued in the Fredericksburg area until Duff's return to San Antonio. Duff's command was later expanded into the Thirty-third Texas Cavalry; he served on the Texas coast throughout the remainder of the war.
After the war Duff fled to Mexico, then returned to his native Scotland where in 1871 he was living on a Logierait farm with his family. He returned to the United States in the 1870s and settled in Denver, Colorado, where he managed the Colorado Mortgage and Investment Company and, in general, handled English investment in that city. He proved a tireless promoter of Denver and launched several canal companies to bring water to local farmers. Around 1887 he moved to London, England, where he died on April 16, 1900.