The oldest graves in my neck of the woods are generally aligned east-west, in line with the rising and setting of the sun. Thomas S. Klatka found in his study of Roanoke County, Virginia, burying grounds that the orientation of subsequent graves often depended on when the first hole in a cemetery was dug.
Individual graves were rarely dug on a precise compass orientation, but rather they were generally oriented toward the position of the rising sun on the eastern horizon. Additional variability was introduced into this procedure since the exact position of the sun rising over the eastern horizon changes throughout the year. 
For instance, at the latitude of Roanoke the rising sun moves from approximately 60 degrees east of true north during the summer solstice in June to approximately 120 degrees east of true north during the winter solstice in December (U.S. Naval Oceanographic Office 1964). As a result, the exact orientation of graves tended to vary from any northeast through any southeast direction depending on the time of year when the graves were excavated. Following cemetery establishment and excavation of the initial grave shaft, the long axes of subsequent graves within a cemetery generally ran parallel with only minor variation. This pattern often persisted even in cemeteries that were active for lengthy periods of time. While graves within a cemetery were usually oriented parallel to one another, the overall orientation of graves between cemeteries tended to differ more markedly. As a general rule, the orientation of graves within cemeteries tends to reflect the time of year when individual cemeteries were founded.
Rayne, Louisiana, was
recognized by Ripley's Believe It or Not for having the "only cemetery in the U.S. that faces north and south"—St. Joseph's Catholic Cemetery. (Klatka notes that Catholic cemeteries are less likely to follow the east-west tradition.)
Perhaps the gravedigger did not have a compass. Perhaps the priest did not oversee the work of a common laborer. Whatever the case, the most commonly accepted version of what happened is that the graves were mislaid and before the mistake was discovered, too many people had been buried; the expense of reburials (not to mention the effect it would have had on the grieving families) was too great a cost. The citizens allowed the cemetery to remain as it had originally been placed, albeit at the expense of being a rarity in the civilized Western world. [Link]
The
only cemetery in the U.S. that faces north-south? That's a claim that just begs to be refuted. I can certainly think of cemeteries where the "east-west rule" was thrown out the window. (When the
garden cemeteries of the 19th century were designed, aesthetics outweighed celestial considerations in the placement of graves. Just
look at the layout of Mount Auburn Cemetery in Cambridge, Mass.) I can't, though, think offhand of a cemetery I have visited where the graves were
all oriented north-south. Can you?
