Scoring a house in the charming golf community, with its grand, historical homes, is nearly as rare as shooting a hole-in-one.