xinian520 the max key limit seems to have increased in MySQL 5.7 (source: https://stackoverflow.com/questions/1814532/1071-specified-key-was-too-long-max-key-length-is-767-bytes)
767 bytes is the stated prefix limitation for InnoDB tables in MySQL version 5.6 (and prior versions). It's 1,000 bytes long for MyISAM tables. In MySQL version 5.7 and upwards this limit has been increased to 3072 bytes.
Try disabling the extension, dropping the table again, updating MySQL, and enabling the extension. The error happens because the extension is using the syntax from higher MySQL version.
We had this error at FreeFlarum, if you'd dig deeper in the logs I am sure that the "Specified key too long" error is there.
Simply: update MySQL to be at least version 5.7
FYI you could change database collation instead, but this ruins emoji support