James H. Irby was born on December 31, 1793 in Laurens, South Carolina. He attended South Carolina College graduating in 1816 and was admitted to the bar in 1817. Irby had a partnership with his cousin, William B. Henderson for a number of years, dissolving in 1855. His partnership with J.O. Hunter lasted for 1856-1857. Irby’s last partnership was with Rutherford B. Todd from 1858 until his death in 1860.
Irby was active in local and state politics for many years. He was a member of the South Carolina House of Representatives, 1826-1830, 1832-1840, 1848-1852; lieutenant governor, 1852-1854; and state senator, 1854 and 1855.
Irby owned a large plantation and was said to have employed a great number of slaves.
Irby was married to Henrietta Earle (1818-1892) and they had eleven children. Their son John Laurens Manning Irby (1854 – 1900) served in the S.C. House of Representatives from 1886 to 1892, serving as speaker in 1890, and was a U.S. Senator from March 4, 1891, to March 4, 1897.
 Laurensville Herald, 1848-1855
 McCrady, Edward, and Samuel A. Ashe. Cyclopedia of Eminent and Representative Men of the Carolinas of the Nineteenth Century With a Brief Historical Introduction on South Carolina. Madison, Wis: Brant & Fuller, 1892. p. 219.