Check if any of your Roles have an access level called "Basic Access with Sharing". That would mean that while users in those Roles are not full Admins they can invite other users.
No. Other than myself, the other roles only have basic access.
Maybe these other users got a link from someone else in an email and clicked the link. That would have triggered an email to you to grant access. Maybe you did grant access that way?
I also always hide the user table and test as another role to everyone except me or trusted individuals.
Do the users actually have a role assigned?