He was first married to Sybil Tarbell, on 7 Sept 1784 in Pepperell, MA. She died on 27 Jun 1804, and he remarried her sister Amy on 1 Dec 1804. They were the daughters of David and Bathsheba (Woods) Tarbell. I believe "Emma", the name of his wife buried beside him, is probably an alternate spelling for Amy, his second wife.
Some time following the 1810 Pepperell, Mass census, he left for Maine, and settled in the town of Solon.
Map on right shows the location of the Jewett Cemetery in Madison, Maine, where James and Amy/Emma Shattuck is buried.
GPS co-ords are :44.890414, -69.792478
Google map (above) shows where highlighted lot from map on left is located in Solon on current map.