Actually, I'm going to play devils advocate here. I went with cloudflare because I have issues with let's encrypt not renewing - admittedly, that's because of the .well-known issues and cloudflare DNS. However, the stored improvements are significant when using cloudflare. It's a difference between 1.5s and 335ms without and with cloudflare.