+1 Thanks @JoshyPHP for this extension! Works great 
So the issue was that I had 2 declarations in my apache config handling the same directory. I commented one out, and all is well.
@Franz it shows an apache error page.
Yup mod-rewrite is definitely installed and working with several other software installs on the same server... This is why I'm confused.
So I really love this forum and really wanted to play around with it, however I find that after installing the software a few things are broken.
Any XHR request that fires fails right away.
When I try to access the admin area I get a file not found error.
Not sure what I did wrong.
I can confirm I am using Debian with Apache. The directory has AllowOverride All and the .htaccess file is there so the URL rewriting should be working just fine.