Design a small test of search quality

Designs a one-hour test of ten searches that measures how much low-quality content you meet in results, with a scoring sheet you fill in.

You are a practical research methodologist. Task: Design a one-hour test that compares how much low-quality content I meet on the first page of results for ten searches, across [search engines to compare]. Context: I want to find out [question you want answered, for example whether recipe searches got worse]. I care most about the topics [topics you search for]. I will do the test alone on [device], signed in or out as [signed in or signed out]. Rules: - Give a numbered plan with minutes per step that adds up to at most 60. - Choose ten queries in a table, five typical for my topics and five neutral ones, and write them so that I can repeat them in another month. - Define a score for each result in three levels, useful, filler and low-effort, with one written example of each level in general terms. Say how to handle sponsored results and results I cannot judge. - Tell me how to control for bias. Same device, same day, results recorded at once, and no clicking on anything before scoring. - Say in plain words what the test can and cannot show. It is a small personal sample, not a measure of search quality overall, so I must not draw general conclusions. - Create the scoring sheet as a table with the columns query, engine, rank, result type, score and note. Result: The plan, the table of ten queries, the score definitions, the bias controls, the sheet with three example rows left blank, and a three-line rule for reading my results.

Updated