Though the Carter Center has been a celebrated base for diplomacy and disease eradication, it is worth remembering the controversies that surrounded its construction.