facepalm - sorry what a dumb mistake on my part.
So I installed both mediaembed and autovideo, and turned the extensions on in admin.
So here's an example where I'm trying to embed. The embedding doesn't work on the post, but, curiously if I hit "edit" I see the embeds. So I know the extension is working, but for some reason it's not showing up in the actual post.