I recently did a talk at a University for 150 budding entrepreneurs about the importance of finding good problems testing everything you can whilst building your product and business to make sure you continue to do the right thing.
Following the talk a couple of the participants got in touch. The following is an email conversation I had with Kishan Patel.
Thanks for the awesome questions Kishan.
You spoke of writing problems down, in regards to finding your best solution from these, how did you go about testing them all? or did you see which were the “best” problems and test only those?
There are several reasons for getting into the habit of writing problems down. The first is that your brain is not really the best place to store things - of course, it can and does store memories very well, but it’s highly likely that you will forget things as quickly as you first thought of them unless you write them down. David Allen in the book Getting Things Done says “Your mind is for having ideas, not holding them” - the first thing to do when you see a problem or have an idea is to get it out of your brain, as quickly as possible. That way you at least stand a chance of thinking about it again and maybe solving it sometime.
The second reason is a way of practising and building your creative muscle - if you get into the habit of writing lists of 10 things, for example, you will find it much easier to come up with things in the future. I believe the biggest thing holding people back from being creative is not working out their creative muscle.
When it comes to testing the ideas or problems I tend to be pretty brutal with most of the ideas I have. I never delete or erase any ideas, but I only develop the ones that I view as being the best, in terms of their viability and desirability. I also give ideas time - I let them just sit there for a day or two and find that if I go back to it and it still looks interesting, there might be something in it. Then I do the most simple tests I can think of, it’s a bit of a stretch to say it’s a user interview, but I ask people at any opportunity if they have a similar problem. At this stage, I never talk about the solution though, only the problem.
How did you find the problem to solve? Was it mainly through testing all avenues, or through your own understanding and experience of a problem you had experienced or witnessed?
Finding good problems can happen in a number of ways, but I tend to let my own experiences lead the way. It’s taken a long time, but I know what I’m good at, what I enjoy doing and what motivates me pretty well these days, so I tend to look for problems in familiar areas. If you’ve never read Paul Graham’s essays on starting companies, it’s well worth looking at. In How To Get Good Startup Ideas, he says “The way to get startup ideas is not to try to think of startup ideas. It’s to look for problems, preferably problems you have yourself.”
You also need to train yourself to notice. Most people are oblivious to the problems all around them. They may get frustrated by things not working as they’d hoped but rather than take notice of it, they move on to the next thing and try to forget the bad experience. This also takes time, so if you’re looking for problems and ideas, take your time to let them be noticed.
There’s almost a self-test that you can do here too - ask yourself a bunch of questions:
Is this likely to be a problem that lots of people have? Is the pain point big enough to switch or buy something else? Is there a way to build a solution to this? (There are often lots of ways, so keep an open mind here) Could this be a big business one day? If you get positive results from these questions, your next one should be: What’s the quickest way I can test these?
How did you weight aspects of your testing, was there any specific element within the UI/UX you had to factor higher than the others - and did this cause some problems between what you wanted the product to do, and what it was capable of it?
I’m presuming you mean weighting in terms of priority? If so, I would say an assumption may is the best way to prioritise your experiments.
Assumption mapping is essentially listing out all the things you do not know, all of your assumptions. This can come from doing things like Business Model Canvas, Value Proposition Design and using a bunch of other tools.
The idea behind Assumption Mapping is to plot all the assumptions on a 2x2 matrix where the x-axis shows the level of certainty or evidence you have for something. The more certain you are the further left it goes, the more uncertain the further right. The y-axis is the level of importance. The more important it is, the high it goes.
What you are looking for is all the assumptions you have that are in the top right quadrant, in other words, the most important and highest risk assumptions. They are the ones you should test.
Once you’ve done that, you need to design the experiment and run it. You need to know what goal you are looking for:
What is your hypothesis? - what do you believe? What will you test? How will you measure it? What is your success criteria? Remember the purpose of testing business ideas, UX, product decisions or anything else is to reduce the risk of the idea and gather evidence that supports or refutes your hypothesis.
When you first tested the products with customers, did you identify target people, have the “public” (anyone) test it, or was there internal testing prior to public testing, to work out kinks?
This one is quite subjective depending on the product, the test, where we are in the product pipeline etc. I would, however, say that you have to test product ideas with people who you think need this. There’s obviously no point in testing a product for new parents on kids or elderly people. But I would keep your audience as broad as you can in the early stages. When we first testing our Future Builders ideas, we kept the targeting very broad then narrowed down as we got some results. This way you’re able to find out things you didn’t know were important. One interesting one for us was that we had a surprisingly high volume of students and recent sign up to our landing page. We weren’t targeting them and we didn’t think of them as a target demographic, but now we’re aware of it, we’re figuring out new experiments to deepen our understanding of their needs.
Now that we are building out our live product, we’re very targeted when it comes to selecting people to run tests with and we very often do internal tests first. Just giving a product to someone at work that not been involved or a friend can be a great way to work out things out initially. It’s also way cheaper that way too.