I work at an airport in California (that shall remain nameless), so I've had the opportunity to check out a lot of different places. It's funny, now that I know the inner workings of them, I notice stuff a lot better than I used to as just a casual traveler.
With that said, my least favorite is George Bush Intercontinental Airport (Houston, TX). The layout is atrocious if you have to change planes there. United flights are spread out across several concourses, with no easy way of getting from one end to another. And the last few sections on the end look like their just a series of modular buildings that a lot of schools have for overflow classes.
Beyond that, here are some other thoughts.
LAX is huge and outdated. Every time I go through there, they are perpetually halfway through some major rebuilding project that seems to have no function other than to keep construction workers employed. I prefer to fly in and out of Bob Hope Airport in Burbank, despite the somewhat frightening take-off and landing flight paths.
Orlando International sucks for one primary reason, the layout. They put the TSA checkpoint right in the tightest corridor of the terminal. It's already a bottleneck getting to the trams, but throw those smurfs in the mix and it's often an incredible headache.
Hartsfield-Jackson (Atlanta) is a massive airport. A lot of people don't realize it, but its actually the busiest passenger airport in the entire world. However, it really isn't that difficult to navigate between terminals. For as many people as they process through security, the lines are usually very reasonable.