Beta Testing Best Practices

Question

Hi everyone,I'm nearing the end of initial development on a new app. I've collected a diverse group of users to begin beta testing some functionality next week.I'm wondering - what are your best practices for working with beta users? Do you do more of a controlled test, giving users specific tasks to complete? Or do you keep it open, to see what they do/can break?------------------------------Heather Bryant------------------------------

laurathacker · Answer

I find my users work a lot differently to how we "expect" them to work; and will always find a way to break something or not enter "required" data that is not marked required. Make sure your beta testers understand what you need from them (screenshots, URL links, videos etc.) so you can debug more efficiently. Also, doing a training session with them to explain how a new function/system works is super helpful; and will often raise questions about a scenario you did not predict or did not know could happen. I always remind everyone that "if it looks wrong" to raise a flag and not assume that it is correct. Better to raise a flag and find out what the "rules" of the system are than to ignore a major flaw in a calculation of some kind or form behavior. I am sure you can give your beta users some "tasks" to do; but you really want them to use the system the way "they" use it; which is not always how you understand they use it.

------------------------------
Laura Thacker (IDS)
laura@intelligentdbs.com
(626) 771 0454
------------------------------

babipanjikar · Answer

I usually do a control testing with set of users briefing them on the purpose of the app and how it should behave and then work with them on testing. This gives me the flexibility to identify issues they may face while working on testing. Then I go with open test with mix set of users including few from first run.

------------------------------
Babi Panjikar
------------------------------

heatherbryant2 · Answer

"but you really want them to use the system the way "they" use it; which is not always how you understand they use it." - this part hit home, Laura! You are right.

I set up a short training call to demo the functionality the way I would *think* they would use it. Now we'll see what they come back with! :)

------------------------------
Heather Bryant
------------------------------

laurathacker · Answer

Heather, this has been a time-and-time again thing I see. Without a deliberate workflow and locking down all the flexibility of Quick Base, you will find users prefer different ways to do things. Some people like to live and function from reports; while others like to live/function in records and make actions from inside a record.

The biggest training item no-one seems to know about or understand is "Favorites". I live in favorites.... so can users if they navigate around their reports or records frequently. Most however, it seems, never even look at that top row on the menu bar!

------------------------------
Laura Thacker (IDS)
laura@intelligentdbs.com
(626) 771 0454
------------------------------

Forum Discussion

Beta Testing Best Practices

Related Content