It's not just about metadata being visible. Duplicity is designed to work when the remote machine doesn't have the private key to decrypt your files. If you use EncFS instead, anybody who compromises the server with your backups can also get the key to decrypt them.
EDIT: Never mind, I just saw your other comment and learned about reverse mode. That's a simple but clever feature.
Nah, they can't get your private key if you transfer just the encrypted files. That's the whole point, really, but obviously they can get it if you mount it on the server itself (and thus disclose your passphrase).
EDIT: Never mind, I just saw your other comment and learned about reverse mode. That's a simple but clever feature.