I let people rate how much they like different things on a scale of 1-10. How do I actually tell if people like one thing more than another thing if the sample sizes are different? This is not about any real scientific study, more like a personal test :)
For example, if one thing got voted on 10 times and has an average value of 6.5, and another thing got voted on 6 times and has a 6.1, is the 6.5 thing actually more liked? Or is this small sample size still so random that it could with a high chance go both ways?
I’ve never done anything like this, if someone could explain it or direct me to the correct key words/links, that would be hugely appreciated :)
I’ve read up a bit on p-value determination, but I’m not sure what my “null hypothesis” is here actually, numerically. If I’d put it in words I guess my hypothesis would be “this thing is more liked than the other thing”, but honestly, it seems like my specific case would be much simpler than all the stuff I’m reading here :D
People are inherently bad at rating things. Why not run a “This or that?” style study instead?
Given a list of items to rate, pair them up randomly. Ask a person which item they like better out of each pair. Run through Final Four type eliminations until you get down to their number one preference.
Run through this process for each person, beginning with different random pairings every time.
Record data on all the choices - not just the final ones. You should be able to get good data like that.
For example, there will probably be a thing that is so disliked that it gets eliminated in the first round more frequently than anything else. The inverse will likely be true of a highly-preferred item. And I am sure you can identify other insights as well.
Sounds like a good idea, however my participants neither have the attention span nor do I have the resources to do anything else :) after all, like I said, it’s just a small personal thing :)