DursunCan for Apache there's this https://github.com/phanan/htaccess#force-https-behind-a-proxy
I believe the Canonical Url extension will also work behind a proxy.
You can also consider enabling "force HTTPS" in Cloudflare, in which case Cloudflare will redirect all HTTP requests to HTTPS right at the edge.
I would recommend using "full + strict" HTTPS in Cloudflare and use one of the Cloudflare-generated Origin Certificates on your host to secure the connection between Cloudflare and your server.