Still no way to remove all users from user field?

I found an easier way to do this because I have too many users as well. If you click on one name, hold down the Shift key, scroll down and click on the final name (you have to leave at least one name) and they will all be highlighted. Then click the Remove User button. Worked like a charm for me!