Ran into a problem this morning with Amazon’s Elastic Load Balancer. If you want to have multiple availability zones, say us-east-1a and us-east-1b, behind your elastic load balancer, be sure to have at least one healthy instance running in each. What happens otherwise is inbound requests will “dead end” and serve up 503 errors. This is because the DNS actually resolves to each zone regardless of the health checks and before passing requests to the actual load balancer for that zone. In otherwords, the zones are unaware of the status of the machines in different zones. 🙁
From Paul@AWS on the Amazon Discussion Forum:
The output from your describe call shows that you have two zones enabled:
but you only have instances behind one of them. Whenever your client happens to get directed to the empty zone (which happens at DNS resolution time), it will have a dead-end.
The solution is to either add instances in the additional zone or disable that extra zone.
You can read the full thread that tipped me off this morning to the issue I was experiencing.