Building An AI Agent With Google Vertex AI

Ever since autonomous agents became a hot topic in the field of generative AI, I have always been fascinated by the use cases and practical applications that users can build with this technology. In fact, I have created my own AI agents before like this one that stalk anyone on the internet.

However, the process of creating AI agents over a year ago was highly technical, and the AI itself wasn’t that impressive, often leading to hallucinations and agents going in circles during conversations.

Fast forward to today, creating AI agents on platforms like the ones that Google recently launched in Vertex AI is a lot simpler and the quality of responses we get from them is far more decent.

In this article, I am going to walk you through the process of building AI agents on Vertex AI. Before we get into the actual steps, let’s talk about some terminologies some may not be familiar yet.

What are AI agents?

AI agents are a type of artificial intelligence that can operate autonomously to achieve specific goals.

Unlike traditional AI models that require human input and prompts for each interaction, AI agents are designed to think and act independently once they are given an objective.

What is Vertex AI?

Google Cloud Vertex AI is a unified artificial intelligence platform that offers all of Google’s cloud services under one roof. It provides pre-trained APIs for various applications, including video, vision, and natural language processing, enabling easy integration with existing applications.

Okay, now that you’re familiar with AI agents and Vertex AI, you can start creating your own AI agents.

Generative AI Publication is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Subscribed

Step #1: Get into Google Agent Builder

Log into Google Cloud Console and navigate to the Agent Builder dashboard.

Click on the “Create a new app” button to start creating an application.

Step #2: Create an AI Agent

Select this option to start creating your AI agent. Be patient, as this step may take some time to complete.

Built using natural language, agents can answer questions from data, connect with business systems through tools, and more

Once your agent is successfully created, you’ll be redirected to a screen where you can give your agent a unique name, like “Title Generator.”

It’s worth noting that the best language model currently available in Vertex AI is gemini-1.0-pro-001, which is a bit disappointing considering Google’s recent promotion of the more advanced Gemini 1.5 model.

Nonetheless, let’s continue.

Step #3: Set the AI Agent’s Goal

An agent goal is a high-level description of what the agent should accomplish. You don’t have to be very specific on the details of what the agent needs to do at this point.

For our example, let’s say we want our AI agent to research a topic, gather relevant information from the internet, and generate titles for an article. We can set the goal as “To generate 5 title suggestions for a specific topic.”

To generate 5 title suggestions for a specific topic

Step #4: Set the Instructions

Agent instructions define the process that should be taken to accomplish the agent's goal.

Each step contains a natural language instruction that may contain any of the following:

• A basic instruction that the LLM can understand.
• An instruction to route the user to another agent. Agents are referenced using the form ${AGENT: agent_name}.
• An instruction to use a specific tool. Tools are referenced using the form ${TOOL: tool_name}.
• An instruction to route the user to a Dialogflow CX flow. Flows are referenced using the form ${FLOW: flow_name}.

Each step description starts with -, and you can define subinstructions using indentation.

Here’s an example of my instructions to the AI research assistant example:

- greet the user and ask them what topic they need help with
 - Always generate 5 title suggestions
 - Keep the title simple
 - Keep the title short and attention-grabbing
- Once the titles are generated, choose the best one for you and provide an explanation what makes it a good choice

Note: Make it a habit to always click on the “save” button to avoid losing any progress.

Step #4: Add Examples

Each agent should have one or more examples (recommended to have at least four). These examples are sample conversations between an end-user and the agent app, including the dialogue and actions performed by the agent app. These are effectively few-shot prompt examples for the LLM.

Here are 5 title suggestions for an article AGI:
1. The Promise and Perils of AGI: Exploring the Potential of Human-Level AI
2. Beyond the Hype: What Can AGI Realistically Do?
3. The Ethics of AGI: Ensuring Artificial Intelligence Aligns with Human Values
4. Could AGI Become Conscious? The Philosophical Debate
5. AGI and the Future of Work: How Will Artificial Intelligence Transform Our Jobs?

Without enough examples, an agent is likely to result in unpredictable behavior. If your agent is not responding or behaving in the manner you expect, missing or poorly defined examples are likely the cause. Try improving your examples or adding new ones.

Prompt: Please provide 5 title suggestions for an article about AI agents.

Response: Here are 5 titles that you could use in your article about AI agents:
1. AI Agents: The Future of Customer Service
2. How AI Agents Are Changing the Way We Do Business
3. The Benefits of Using AI Agents in Your Business
4. AI Agents: A Guide for Beginners
5. The Ultimate Guide to AI Agents

Save that example and let’s go back to the AI agent dashboard to try it out.

Step #5: Let’s try it out

Now that your agent is set up, it’s time to put it to the test. Let’s try a prompt like:

Prompt: title suggestions about financial freedom

While the agent provided five relevant title suggestions, it failed to select the best one and explain its reasoning. This indicates that our examples should showcase this capability, or the agent will overlook it entirely.

My first impressions

I tried creating agents a couple more times but I was just not getting the results I expected from them. I’m sorry Google but throughout my experience of making my first AI agent with Vertex AI, it wasn’t that great.

• Finding the AI agent builder was already a pain because it was buried deep somewhere in the mess of Google Console.
• There are way too many refusals and hallucinations due to the old model. Why can’t they allow users to use Gemini 1.5?
• It’s subpar with Microsoft’s Autogen 2.0 or CrewAI.

Final Thoughts

Overall, it was great to see Google release an AI agent builder. It’s been years since AI agents were introduced in the AI world, Microsoft’s Autogen is already on version 2.0, so it’s only right that Google releases one of its own.

But this new AI builder is still in beta and needs a lot more tweaking and improvements. Getting a good agent out of this tool is still a hit-and-miss. This isn’t a ChatGPT moment for AI agents yet.

Anyway, I will keep an eye on the developments of Google’s AI agent builder and review it again once it gets out of the beta phase.

‍