Oops I thought this was a duplicate of your other question, but there's actually a bit of difference.
Same answer though, if you just care about looks, you can use CSS.
If you want to block the feature, an extension will be necessary. Not sure what controller / events are used for password changing from the settings. Maybe there's a whole route that can be easily blocked.