I wasn't alive when Carter was president, so I can only go off of my perceptions. From what I gather, most of the issues during his presidency were based on his inexperience and disdain for getting down in the mud (sadly, politics is a dirty business.) Of course, when it comes to the American hostages in Iran, the Reagan campaign likely sabotaged their release (they just happened to be released shortly after noon on the day Reagan was inaugurated.) When asked about his presidency, Carter said "We kept the peace, we never went to war, we told the truth, never lied to the American people, never misled the American people, so in general terms, I feel perfectly at ease with my term." Diplomacy is extremely important to me, so the fact that Carter wasn't a war monger like his successors is enough to place him head and shoulders above them in my book.
Carter also seemed to have genuine concern for the American people, and his work with Habitat for Humanity speaks volumes about his character. I think it's safe to say that his post-presidency was more productive overall than his presidency, but at least he made it a point to do good in the world. That's more than you can say for most politicians.