# How to compare three AI prototypes before choosing one Canonical URL: https://agentsidequests.com/guides/compare-ai-prototypes/ Markdown URL: https://agentsidequests.com/guides/compare-ai-prototypes/index.md Get genuinely different AI prototypes, try them with the same inputs, separate working behavior from personal preference, and choose what to build next. Give your agent one user problem, a fixed set of sample inputs and three different ways to solve it. Try the same situations in every version, then separate what worked, what you preferred and what remains untested. Ask for a comparison that helps you choose the next experiment, with links to the actual prototypes. By Jordan Bench / Agent Sidequests Published: 2026-10-13 Updated: 2026-10-13 ## Decide what the comparison should help you choose Start with a question such as: Which interaction helps two people agree on a short activity? That is more useful than asking for three impressive apps. Bring the intended users, one essential behavior, a small sample dataset, the device you will use, and a total work limit. Keep the result small enough to try yourself. This guide uses a fictional activity picker. Two people select which activities they are willing to do; the prototype must choose from their overlap or explain that there is no shared choice. The examples and comparison below are invented to demonstrate the method. We have not built or tested these prototypes. Keeping agent-built work reviewable comes up in public agent-building discussions. Reddit user GreenCurious9895 describes separate workspaces, assigned file ownership and an approved brief for overnight work, alongside review overhead. It is a developer self-report, not evidence that more agents or longer runs produce better personal projects. Our contribution is the small comparison method below. ## Give each version a different interaction Write one sentence explaining what each approach is trying to make easier. Approve those differences before the build. Keep the user problem, allowed inputs and required result the same; let the interaction change. New colors, fonts or a different mascot do not answer a new design question. For the activity picker, all three versions use the same six invented activities: paper planes, postcard drawing, a word game, a photo scavenger hunt, a tiny dance lesson and a tabletop puzzle. None needs a real account or personal information. Three proposed interactions for the same fictional activity picker | Version | Interaction hypothesis | What stays fixed | | --- | --- | --- | | A: shared shortlist | Seeing every shared choice makes it easier to agree. | Both people set their allowed activities; only their overlap can be chosen. | | B: pairwise choices | Choosing between two shared options at a time makes the decision easier. | The same allowed activities, shared choices and no-match rule. | | C: surprise wheel | A random choice from the shared options makes deciding feel playful. | The same allowed activities, shared choices and no-match rule. | ## Write the shared scenarios before building Keep a short scenario sheet alongside the brief. Every version gets the same starting selections and the same required result. Judge the outcome, while allowing each interface to reach it differently. If a requirement changes, update the sheet and recheck all affected versions. Use these three cases for the fictional picker. They are acceptance criteria, not claims that any app passed. A random wheel may return different valid activities; the requirement is membership in the shared set, not a predetermined winner. 1. Shared choices: person one allows paper planes, postcard drawing and the word game; person two allows postcard drawing, the word game and the puzzle. The final choice must be postcard drawing or the word game. Record which one appeared and how you reached it. 2. No overlap: person one allows only paper planes; person two allows only the puzzle. The app must show that no shared choice exists. It must not quietly remove a preference or invent a result. 3. Changed mind: person one allows postcard drawing and the word game; person two initially allows only postcard drawing. Complete that choice, then have person two switch to allowing only the word game. The displayed postcard result must be cleared or visibly invalidated. The next completed choice must be the word game. ## Keep the builds separate and the effort comparable Ask for an A, B and C folder, editable files, and one index that opens the versions. Give each the same scope and a stated share of the work budget, reserving time for review. Permit one bounded repair pass per version. Record extra help or extra work so a heavily revised version is not presented as an equal first attempt. If the agent works inside an existing repository, use separate supported workspaces from the same recorded starting revision. Git documents worktrees as a way to maintain multiple working trees. They separate file checkouts; they do not by themselves isolate databases, services or account permissions. For this small lab, use sample data and avoid shared live services. Fornace's claude-overnight README describes separate branches, saved run records and work limits for unattended coding. We read the documentation but did not run or audit the tool. You do not need that orchestrator for this method: one capable agent can build the versions sequentially. Confirm background execution before relying on an overnight run; otherwise do the lab in an active session. ## Record behavior separately from preference Open the delivered versions on the intended device and run the shared scenarios. Note the exact version, input, action and result. Let an agent record checks it can actually perform, then try the interaction yourself. If it cannot open a browser, keep those checks pending instead of converting code inspection into a claim of working behavior. Anthropic's evaluation guidance distinguishes the conversation record from what actually happened in the environment and discusses human checks alongside automated grading. That distinction informs our notes. This little lab is an editorial review method, not a benchmark or a validated usability study. After the required cases, record your own reaction: which choice felt understandable, which action surprised you, and where you hesitated. An agent can offer a design opinion, but it cannot supply your preference or invent feedback from people who never tried the prototype. Keep these kinds of evidence in separate fields | Field | What to write | | --- | --- | | Observed behavior | Version, scenario, actual result and a link or note that lets you repeat the check. | | Human preference | Who tried it, the interaction they preferred and their stated reason; leave blank until someone tries it. | | Blocker or gap | A missing file, unavailable browser, failed interaction or device that has not been checked. | | Extra work | Repairs, changed inputs or added help that affected only this version. | ## A worked comparison: which activity picker gets another pass? Imagine the same person tries all three versions using the scenario sheet. The fictional notes below show why the prettiest result need not be the next one to develop. They are not observations from a real run. Illustrative comparison notes; no prototypes were executed | Version | Fictional check note | Fictional preference note | Next decision | | --- | --- | --- | --- | | A: shared shortlist | All three cases behave as specified in this imagined run. | The reviewer likes seeing the available choices together. | Try this version with the other person before adding features. | | B: pairwise choices | Shared choices and no overlap work; after a changed selection, it retains the old result. | The reviewer likes choosing between two options. | Repair the stale result, then repeat the same cases before comparing preference. | | C: surprise wheel | The delivered page cannot be opened in the available environment. | The screenshot looks playful; the interaction has not been tried. | Mark not comparable yet. Preserve it and resolve the opening problem or defer it. | ## Choose a next experiment and keep the alternatives A useful conclusion names the condition behind the choice: continue with A because it meets the required cases and the reviewer values a visible shortlist; revisit B after the stale-result repair. Keep C unjudged until it can be opened. A technical failure is a real limit of the delivered version, but it does not establish whether its interaction idea is appealing. Ask for a short lab notebook linking the unchanged brief, scenario sheet, exact versions, observations, repairs and next decision. If all versions pass and you have no preference, leave a tie and define the next question to try. Avoid a fabricated score out of 100. One person's small trial does not establish general usability or which AI model is best. For Muse, Instinct, Dots or Grok Bot, check for file creation, coding and browser interaction tools in your actual account. Background execution matters only if you leave the run unattended. If code execution is unavailable, ask for three labeled interaction sketches and a walkthrough, clearly separated from working prototypes. Stop at the agreed budget with the files preserved; the lab does not need to publish or deploy anything. ## Copy a prompt and try it Each idea has a complete starter prompt, the inputs to bring, and the tools it needs. - [Go to sleep with an idea. Wake up to three working versions.](): Three tiny prototypes and a lab notebook - [Recover the useful part of a failed experiment](): A learning review and next-test brief - [Turn a vague ambition into a one-week experiment](): A one-week experiment brief ## Common questions ### Do I need three agents or three different AI models? No. One agent can build separate versions sequentially. Keep the brief, sample inputs and work limits consistent. This compares your prototype approaches; it does not isolate model quality or prove that parallel agents are better. ### What if all three prototypes are almost identical? Return to the interaction hypotheses before spending more build time. Ask what different user decision each approach makes easier. If you cannot name a meaningful difference, reduce the comparison to two useful alternatives. ### Should a prototype that fails a check automatically lose? Record the specific failure and its effect on the required job. Use the agreed repair allowance, then repeat the shared cases. If the prototype cannot be tried, mark it not comparable yet. You can choose a usable version now while leaving the untested interaction undecided. ### Can I use the overnight prototype prompt during the day? Yes. The comparison method needs saved files and inspectable behavior, not a particular schedule. Run it in an active session if background execution is unavailable, and check the finished files before choosing what to develop. ## Sources and further reading [GreenCurious9895: preferred review and debugging workflow — checked October 13, 2026]() Source limitations: Firsthand developer self-report with review overhead and evolving setup. No system or results were independently checked. The page displayed a relative age, so no exact publication date is asserted. [Fornace: claude-overnight repository README — checked October 13, 2026]() Source limitations: Maintainer-authored description of its own coding orchestrator. We did not install, run or audit it, verify its limits, or adopt its automatic merging behavior. It does not establish compatibility with the four personal assistants. [Git: worktree documentation — checked October 13, 2026]() Source limitations: Official version-control documentation supports separate working trees. It is not a claim of security isolation, agent permissions or prototype quality. [Anthropic: Demystifying evals for AI agents — January 9, 2026; checked October 13]() Source limitations: Vendor engineering guidance. Its distinction between recorded claims and outcomes informs this original comparison method; the worked example remains untested. These are practical editorial examples. The prompts have not been independently run-tested across every assistant. A reported use or product description does not establish that a suggested workflow succeeded. ## Keep exploring - [How to move an ongoing project into a new AI chat]() - [How to get AI email drafts that sound like you]() - [Personal AI agent vs chatbot: which should you use?]() - [How to set up an AI morning briefing from email, calendar and notes]() - [How to use an AI agent for job-search alerts that fit your next career move]() - [AI personal assistant use cases: choose a job with a useful finish]() - [AI life admin prompts for the paperwork you keep putting off]() - [Recurring AI agent tasks that tell you when something matters]() - [Creative AI agent projects you can open, play and make your own]() - [How to adapt a personal AI agent prompt for Muse, Instinct, Dots or Grok Bot]() - [How to check an AI-created file before you use or share it]() - [How to use AI to quiz you on your own notes]() For platform setup, read these HTML assistant guides: [Muse](), [Instinct](), [Dots](), [Grok Bot](). [Editorial approach]()