Introducing AI Prototyping Projects

I’ve had the privilege of helping a number of teams explore AI concepts through AI prototyping projects lasting anywhere from a few days to a few weeks. In this article I’ll talk you through what AI prototyping projects are, the value they offer, and how to effectively start, operate, and conclude them.

What are AI Prototyping Projects?

First, let’s start with a definition of what an AI Prototyping Project is:

AI prototyping projects are short-lived development projects focused on artificial intelligence applications that are designed to demonstrate the viability of a concept in an interactive way while identifying risks and key obstacles that would prevent the AI product from succeeding when converted from a prototype to a real software project that will be deployed for others to interact with.

There are a few key aspects to this:

• Short-lived: your prototyping project should have a fixed duration of anywhere from a few days to a few sprints at most. Constraining AI prototyping projects to a single week is advisable in most cases as it focuses your efforts on the most critical aspects of discovery, but more complex scenarios may require more time.
• Development projects: AI projects should require some degree of custom software engineering effort or work from a related discipline. While AI prototyping projects aren’t about radical innovation of new technologies, they typically involve combining existing technologies in new ways, which often requires engineering effort.
• Interactive demonstrations: a successful AI project will be usable in at least some of its intended use-cases. This usually involves taking an external input (such as text from a user) and producing an output. Your prototype may not handle all cases, but it should handle some scenarios in a demonstrable way.
• Identifying Risks and Obstacles: while the prior point was about supporting a “happy path”, an AI Prototyping project is also focused on finding the limits of an AI solution and projecting obstacles the team may encounter developing and scaling the prototype into a full project or maintaining it in production.
• Considering project promotion: an AI Prototyping project should conclude with an evaluation of the viability of the project as a full project and a rough estimate of the scope of such efforts and the degree of success the team is likely to achieve. For example, you should consider if you expect the developed system will have a sufficient degree of accuracy for users to interact with it? Will a small scale prototype on a subset of data and users work well when scaled up to work on additional data and with additional users?

We’ll talk more about each of these areas throughout this article, but let’s help the idea stick by talking about a few sample AI Prototyping Projects I’ve worked on.

Examples of AI Prototyping Projects

The AI prototyping projects I’ve worked on have run an average of 3 days long and ranged from solo endeavors to projects taken on with a team of 6 specialists.

While I can’t share details on all of these projects, some sample projects I’ve worked on included:

• Evaluating the performance of an automated document processing solution for automated extraction of data from a variety of different document formats.
• Building an AI chat system using retrieval augmented generation (RAG) for knowledge search within a specific problem and business domain.
• Training an AI system to recognize users via a webcam, listen to their speech, and respond to them in a conversational manner for entertainment purposes.
• Exploring the performance characteristics and pre-processing needed for a machine learning solution built for a specific problem facing an organization and its pre-existing data.

Each of these projects had a fixed duration, involved technical skills to accomplish parts of it, and concluded with a study of long-term viability and recommendations for next steps.

Some of these solutions were documented and archived, waiting for other developments in the business to make them more feasible. Others were transformed into larger prototyping projects with larger scale and scope and then deployed to the real world where users are successfully interacting with them.

While not all of these projects became real things that made it to the hands of end-users, I view each of these efforts as successful because they let us evaluate a proposed AI system in a brief period of time and make guided decisions on what should happen next.

Is your idea a good fit for an AI Prototyping project?

If you’re reading this, it’s likely that you’re interested in AI prototyping projects of your own.

AI Prototyping projects are a good move if you’re looking to:

• Constrain the scope of AI Prototyping to reduce cost or business impact
• Prove the viability of the core capabilities of your AI system in a short period of time
• Identify risks that may hinder the application being converted to a real-world project

However, not every project can be an AI prototyping project. In order to succeed you’ll need a few key factors including:

• Team members with the prerequisite skills, tools, and knowledge needed to get started
• Isolation from external distractions during the prototyping process
• A well-defined idea with specific capabilities and a high-level technical direction
• Organizational support for the results of a successful working prototype
• An organization that is cautious to put half-baked or unready systems into production
• Basic AI literacy at the organizational leadership level

In other words, if you are asking your team to work on solving a problem in a short period of time, you need to be sure you’ve allowed your team to grow in the months and years leading up to the project to the point where they’ll have the skills needed to take on the project. Additionally, they’ll need independence and autonomy to carry out their work, plus additional resources such as external computing resources like a dedicated Azure or AWS resources.

Finally, your team needs to know that if they get something promising working at the end that the organization will not immediately ship a prototype built for technology demonstration purposes without further refinement, scalability work, accuracy improvements, and an exploration of potential bias and ethical concerns.

Additionally, if the team meets unexpected obstacles or lower than expected performance, they need to have the safety and security to be able to share this with the organization without negative repercussions. Typically these mixed results will come with additional recommendations such as additional data, time, resources, or recommendations of other work that needs to be done in order to make the project possible in the future.

If you have a project that’s well defined and a team that’s able to work on them, then you may be ready to start defining your goals and launching your project.

In our next article in this series we'll cover Launching AI Prototyping Projects