Just like with any product, validating early how the AI will interact with users is crucial. Prototyping allows for this, helping product shapers conduct rapid testing of assumptions, uncover flaws, and iterate on solutions before committing to costly development.
This involves adopting a fail-fast approach, which is especially important in AI, where the complexity of systems and the uncertainty of outcomes can quickly turn model development into a chaotic jumble. Focusing on end users and their reactions from the beginning will help you refine your success criteria, learn from failures quickly, and make necessary adjustments without significant resource waste.
But it comes with its challenges: we don’t yet have the proper tools to prototype AI applications, and there are nuances you need to take into account.
Let’s dive in.
Why prototype an AI solution?
AI often aims to personalize or automate, directly impacting user workflows. You might think as it “automates away” processes from users, user validation and prototype building is not required. However, this is a misconception. Prototyping helps uncover how users perceive and interact with these capabilities in realistic contexts. By exposing your product idea to real users early on, this approach ensures that the solutions are not just theoretically sound but also practically beneficial for users.
“By building quick, real-feeling (and often wholly simulated) versions of products and features, teams can be nimble, explore various directions, fail early, and pivot repeatedly—all before going too far through the entire costly, messy ML development process.”
— Simulating Intelligence (Google Design Blog)
Moreover, prototyping with simulated AI can reveal if current technology can actually deliver the intended functionality and meet user expectations for accuracy, error handling, and bias mitigation. This process often uncovers limitations in current technologies, highlighting areas that need refinement.
Early-stage prototyping
Valid scenarios
Early-stage prototyping scenarios are identified by the following:
- There is a clear understanding of the business value the AI solution will provide, the specific end users it will help, and the overall project goals. If you don’t have this yet, you can go through an AI Design Sprint to define it.
- Data exploration is in the initial stages: the ML team is still identifying relevant datasets or figuring out how to utilize existing data.
Prototyping goals & guiding questions for this stage
Use low-fidelity methods to answer broad questions:
- Does the core concept resonate with users?
- Can users rely on a product like this for their daily work?
- What needs to be explained regarding the scope and the way of working of the ML Model?
- Is the prediction or system output being presented in the most useful way for end-users?
- Are there any fundamental flaws in the proposed workflow?
- Is there a better way to incorporate this prediction into user tasks?
Methods & Tools for this stage
At this stage, quick iterations are essential. Low-fidelity prototypes help teams explore a wide range of ideas without investing too much time or resources. Here are some hand-picked methods you can use to iterate:
1. The Wizard of Oz
Replace complex AI systems with human “wizards” working behind the scenes. This is particularly effective for:
- Conversational UIs (chatbots)
- Personalized recommendation engines
- Systems retrieving real-time information
“…a user interacts with an interface that appears to be autonomous but is (fully or partially) controlled by a human.”
The Wizard of Oz Method in UX (NNGroup)
You can learn more about this technique in the article from Nielsen Normed quoted above.
2. Mocked-up realistic data
Even if you don’t have the datasets yet, you need to think of realistic, coherent cases of data inputs and outputs to be able to perform a test. This is a significant nuance compared to traditional prototyping: content, presented as data, is crucial. Spend time refining the data to be used during the test.
3. Figma or similar
Prototyping tools like Figma or Protopie have interactive elements that help simulate basic interactions with a Graphical User Interface (GUI), allowing you to present something that looks like a ‘working product’ even if it’s not built yet. Figma is the de facto tool for product designers, but it still needs improvements to make validation more realistic, such as allowing for real input in their input fields.
The challenges of early-stage prototyping
- User Perception: Users may find it hard to imagine how the AI will function in the final product based on early prototypes. Showing coherent content and presenting realistic scenarios the user can relate to will help cover these shortcomings.
- Limited Fidelity: Early prototypes may not accurately represent AI’s final behavior, leading to misleading feedback. When summarizing feedback, you need to take into account this.
- Focus on content: Input and output data is presented as the prototype contents and your test subjects will pay special attention to it. Your team of designers might be used to test designs focused on the design and workflow aspects, so you need to prepare for this and ask for the team to adapt.
- Cover critical edge cases: you can identify the most critical edge cases through the design process, and prepare both the prototype and test script to cover them. An example of these edge cases can be, in the cases of a system with prompting functionality, an ambiguous prompt. This will allow you to explore the limits early, build trust, and learn when to incorporate graceful degradation.
- No proper tools yet: as I commented at the beginning of this article, there aren’t tools designed specifically to cover early-stage prototyping of AI applications. The best we can do is stitch things together, for example: connecting data from spreadsheets to Figma screens, and have people helping us with the Wizard of Oz format.
Mid- and late-stage prototyping
But what about mid-stage or even late-stage prototyping? Are there any best practices we can rely on to ensure we’re building our product correctly?
Stay tuned for part 2 of this series and explore the vast world of AI products with Arionkoder. And if you’re beginning to build your product, or want to refine your current product, reach out to us at hello@arionkoder.com for a free consultation!