There are at least two James M Yanceys floating around north Georgia during the late 1800s and early 1900s. The James M Yancey who was my great-great-grandfather was fostered by Washington and Mary Collins (and is listed as James Collins on the 1870 census). He resided in both Milton and Forsyth Counties over his lifetime, and is buried in the Shady Grove Baptist Church cemetery in Cumming (link to FindAGrave here).
He married Mary E Brannon in the mid-1880s, and they had three children who lived to adulthood: Eddie, born in 1888; Clifford, born in 1890; and Pearlie, born in 1892.
The Inferior Court Records for Milton County show that James M Yancey served as Constable for the 842nd militia district in 1895 and 1896 (link to Rootsweb here, records A299 and A317).
James M Yancey's death certificate can be found in Georgia's Virtual Vault (link here). His birthdates differ on his certificate (6 May 1861) and on his tombstone (1862).