I'm doing the same thing, and I chose FastMail. It was easy to set up with a custom domain, and I've been making heavy use of aliases and plus addressing which I like. At least for my purposes, the web client is every bit as capable as the Gmail interface. I haven't tried to import mail from Gmail yet, but it looks straightforward.

The big drawback I think is that Gmail was the interface for Hangouts/Voice, so I either need to find good alternatives for those things, or keep my Gmail account around for that purpose.

I guess it feels a little weird to be paying for a personal email account, but for a single user it's not awful, and I end up owning the entire address. I think it's worth it.