Large Scale Custom and Transactional Email

As anybody who's run a website which sends any more than a trivial amount of email can attest it is incredibly difficult to ensure that the email you send is actually received by your users.  In our modern world where an estimated 80% of email is spam ISPs and email hosting providers are using increasingly complex and restrictive techniques to filter the messages coming across their servers.  This makes it very easy for your legitimate emails to get mislabeled as junk or to get your server(s) blocked.  If this didn't make things hard enough it is increasingly important to make each email one sends highly customized for each user to increase value for users and increase the likelihood that a user will actually click on a link in the email.  This requires a massive amount of infrastructure and can make optimizing emails to avoid the spam filters that much harder.  So what is one to do?