Finally found out the issues, and this could be a bug too. My forum had 0 tags. Even though I had set the minimum number of tags required for both primary and secondary tags to 0, I couldn't start the discussion.
I created a tag now - "General" and the button allows me to start discussions. So I guess you can't start a discussion unless there is at least 1 tag in your forum irrespective of how you have configured your tag settings (in my case it was 0).
I am not sure if there is a particular reason for the mandate that the forum should have at least one tag created.
Out of topic, but still in topic: wouldn't it be a better idea to allow the OP to add the 'solved' tag to their posts?