Thanks all... like many Quick Base problems, as soon as I wrote the post I figured out my solution. Here's what I did:
On the master table I created a summary field called "# of replies by this user" which totals the number of child records that match the criteria "posted by = the current user" ('posted by' is a user field on the child record.
Then I added an if statement to the Add Reply formula URL field:
If([# of replies by this user] > 0,"", URLRoot() etc...)
This hides the button if the user has already added 1 reply. I also set the role permissions to hide the Add buttons for that table. So without any button to add a record once the've added a reply... voila, one reply only.
I know this isn't API foolproof, but we're only using the UI to add records.
