Still no way to remove all users from user field?

Not that I know of explicitly, no. Programatically my guess would have been API_SetFieldProperties or the REST Update Field, but neither call out user-lists by name. The JSON might be your best bet at throwing darts at the wall to see if you can see the 'options' key to an empty array but if it follows the HTTP format that's likely only for text based options. Beyond that I've never seen anything in the API. 

This probably doesn't help, but if its a limited use case you could just have the default list show and uncheck the green check in the user tab to allow them the show in user lists. Doesn't alleviate the overall problem and potentially causes more, but at least its faster than the method for a user field to add/remove users. 

I found an easier way to do this because I have too many users as well. If you click on one name, hold down the Shift key, scroll down and click on the final name (you have to leave at least one name) and they will all be highlighted. Then click the Remove User button. Worked like a charm for me!