In my last blog post I showed how to access agents from different providers other than Google using Vertex AI Model Garden. As the reader, you may be wondering what are some unique ways that I can use these other models in conjunction with Gemini? In this post we are going to go over how to use a continuous improvement loop to derive better outputs by using different models using Genkit. Read on to find out more.
Kaizen aka continuous improvement
Kaizen, also known as continuous improvement, is a system that is designed to derive better outputs from a brief iteration cycle of output and critiques. This pattern is really popular with AI agents and is a way for the AI to critique itself to then come up with a new result.
Continuous improvement loops with model garden
When adding model garden to the mix, we are able to use a multitude of different models to improve the output even more. Each model has its own weight and biases so to derive a better output, we can use one model as the writer and then a different model from another model provider as the critic. The two different weights between the models will point out differences in the output deriving a better result as each one has their own weights and biases that elevate the results.
What does this look like in Genkit?
Let’s demo what a continuous improvement loop looks like. I am basing my continuous improvement loop off of this blog post from Firebase where they used the ADK to generate a continuous improvement loop. In Genkit, we will need to start by getting the Genkit beta SDK. This package includes features that the team is still iterating on so keep an eye out for updates to Genkit to see if these features make it into the released package. From the beta package we are going to specifically use the chat and session tools. Sessions allow us to create persistent sessions while chat allows us to build off the session history without explicitly grabbing the session each time. In my example I am just going to use a JSON session store running locally on my computer. You may want to consider using a more robust backend like Firestore or Data Connect to host your session history.
import { genkit, z, type SessionStore, type SessionData } from "genkit/beta";
import { readFile, writeFile } from "node:fs/promises";
class JsonSessionStore<S = any> implements SessionStore<S> {
async get(sessionId: string): Promise<SessionData<S> | undefined> {
try {
const s = await readFile(`${sessionId}.json`, { encoding: 'utf8' });
return JSON.parse(s);
} catch {
return undefined;
}
}
async save(sessionId: string, sessionData: SessionData<S>): Promise<void> {
const s = JSON.stringify(sessionData);
await writeFile(`${sessionId}.json`, s, { encoding: 'utf8' });
}
}The next thing that we want to do is instantiate Genkit. Remember that we imported genkit in the code sample above, so this version of genkit is coming from the beta package. When we instantiate Genkit here we are instantiating vertexModelGarden and vertexAI plugins. The model garden plugin will give us access to models from Anthropic, Meta, and Mistral where the vertexAI plugin will grant us access to the Gemini models.
import { vertexModelGarden } from "@genkit-ai/vertexai/modelgarden";
import { vertexAI } from "@genkit-ai/google-genai";
export const ai = genkit({
plugins: [
vertexModelGarden({
projectId: "MY_PROJECT",
location: "us-central1",
}),
vertexAI({
projectId: "MY_PROJECT",
location: "global",
}),
],
});Next, I am going to define a tool. The tool will be our looping agent. Here we can supply some input to the agent about the user’s goals and get a workout designed for that user without any input. For now, we give the tool a name. In my case it is the personal_training_staff, and then we give a description of a personal training staff. Finally, we give an input schema and an output schema so the models that use this tool know what to expect.
export const personal_training_staff = ai.defineTool(
{
name: "personal_training_staff",
description: "Personal training staff. Use this to get help from the personal training staff to build a workout.",
inputSchema: z.object({
usersFitnessGoals: z.string().describe(`
A brief summary of the user's goals and current fitness level as well as any
information that should be provided when building a fitness plan.`),
}),
outputSchema: z.string(),
},
async ({ usersFitnessGoals }) => {
//ommitted for brevity
}
);The inner loop
Now that we have the boilerplate out of the way, lets focus on the important part of adopting continuous improvement loops, the looping!
I start by defining an exit flag and a number of max iterations that the loop can use. If the max iterations is hit in the loop or the exit value is flipped to true, the loop ends and we send the history of the conversation back to the first agent.
I then define my loop and inside my loop I have two generate calls. The first generate call is the writer of the fitness plan and I am using Mistral Medium 3 from Vertex AI Model Garden. This comes up with a workout plan based on the prompt and conversation history. The results from this generation are stored in the conversation history and then passed to the Gemini 3 pro preview model.
// ommitted for brevity
let exit = false;
const MAX_ITERATIONS = 5;
let result = `Users Fitness Goals from Reception: ${usersFitnessGoals}`;
for (let i = 1; (i <= MAX_ITERATIONS) && !exit; i++) {
// start with the lifting_friend
const lifting_friend_output = await ai.generate({
model: vertexModelGarden.model("mistral-medium-3"),
prompt: result,
system: `
You are a world class physical fitness coach. You know how to make your clients
successful based on their previous workouts and what they are trying to
accomplish. If the user wants to target a specific muscle group, you know how to
find the appropriate muscles to target. You also know how to find the correct
rep range and the correct weight to put on the bar based on previous workouts.
You are not afraid to give advice to help make your clients successful.
Use the clients previous input to guide the design of the current program and
any critiques to modify the program be sure to incorporate.
`,
});
result += `\n\nLifting Friedn Advice: \n ${lifting_friend_output.text}`;
//ommitted for brevity
return `Fitness Staff Discussion: ${result}`;
//ommitted for brevityThe gemini 3 pro model gets the response in the result object and uses that as its prompt. You will notice that in this generate call I am inline defining a tool to call exit_loop for when all work is completed and the workout cannot be improved anymore. This merely sets the exit variable that I defined earlier to true allowing the loop to finish processing and exit prior to the max iterations being called.
const lifting_pro_output = await ai.generate({
model: vertexAI.model("gemini-3-pro-preview"),
tools: [ai.defineTool({
name: "exit_loop",
description: "All work is complete, the workout cannot be improved.",
inputSchema: z.object({}).describe("Exits the loop"),
outputSchema: z.object({}),
}, async () => {
exit = true;
return {};
})],
prompt: result,
system: `
You are a world class physical fitness coach. You take workouts given to you and
look for issues with them. You know what your clients want and sometimes they
can be... over ambitious. Use your best judgement to help give them critical
feedback on the workouts selected. Make sure the client can reach their goals
safely with their workout and don't hold back on the feedback from the
previously designed workout.
If you get the best possible workout from lifting_friend and you have no
comments, call exit_loop and show the client their workout. Give a reason why
there is no feedback required before calling exit_loop.`,
}),
});
result += `\n\nLifting Pro Feedback: \n ${lifting_pro_output.text}`;
}
Entering the loop
To enter the loop we use a chat session flow where we use that special beta
feature chat that we imported earlier. The chat feature allows us to utilize
a session history which we can get from our JsonSessionStore. In the chat
feature, we interact with Gemini 3 Flash and have it fetch details from the
user before handing off the user request to the personal training staff.
export const chatSessionFlow = ai.defineFlow(
{
name: "chatSessionFlow",
inputSchema: z.object({
sessionId: z.string().optional().describe("Optional session ID to continue an existing chat"),
message: z.string().describe("User input message"),
}),
outputSchema: z.object({
sessionId: z.string().describe("The ID of the session used"),
response: z.string().describe("The response from the chat session"),
}),
},
async ({ sessionId, message }) => {
const store = new JsonSessionStore();
let session;
if (sessionId) {
session = await ai.loadSession(sessionId, { store });
} else {
session = ai.createSession({ store });
}
const chat = session.chat({
model: vertexAI.model("gemini-3-flash-preview"),
tools: [personal_training_staff],
system: `
You are a world class physical fitness coach.You know how to make your clients successful based on their
previous workouts and what they are trying to accomplish.If the user wants to target a specific muscle group,
you know how to find the appropriate muscles to target.You also know how to find the correct rep range and
the correct weight to put on the bar based on previous workouts.You are not afraid to give advice to help make your
clients successful.
Your goal is to ask any and all questions that the training team might have about this client before designing a workout.
When you feel that there is sufficient information to design a workout for this client, send them to the personal_training_staff`,
});
const response = await chat.send(message);
return {
sessionId: session.id,
response: response.text
};
}
);
Expand to see a full continous improvement loop
import { genkit, z, type SessionStore, type SessionData } from "genkit/beta";
import { readFile, writeFile } from "node:fs/promises";
import { vertexModelGarden } from "@genkit-ai/vertexai/modelgarden";
import { vertexAI } from "@genkit-ai/google-genai";
class JsonSessionStore<S = any> implements SessionStore<S> {
async get(sessionId: string): Promise<SessionData<S> | undefined> {
try {
const s = await readFile(`${sessionId}.json`, { encoding: 'utf8' });
return JSON.parse(s);
} catch {
return undefined;
}
}
async save(sessionId: string, sessionData: SessionData<S>): Promise<void> {
const s = JSON.stringify(sessionData);
await writeFile(`${sessionId}.json`, s, { encoding: 'utf8' });
}
}
export const ai = genkit({
plugins: [
vertexModelGarden({
projectId: "MY_PROJECT",
location: "us-central1",
}),
vertexAI({
projectId: "MY_PROJECT",
location: "global",
})
],
});
export const personal_training_staff = ai.defineTool(
{
name: "personal_training_staff",
description: "Personal training staff. Use this to get help from the personal training staff to build a workout.",
inputSchema: z.object({
usersFitnessGoals: z.string().describe(`
A brief summary of the user's goals and current fitness level as well as any
information that should be provided when building a fitness plan.`),
}),
outputSchema: z.string(),
},
async ({ usersFitnessGoals }) => {
let exit = false;
const MAX_ITERATIONS = 5;
let result = `Users Fitness Goals from Reception: ${usersFitnessGoals}`;
for (let i = 1; (i <= MAX_ITERATIONS) && !exit; i++) {
// start with the lifting_friend
const lifting_friend_output = await ai.generate({
model: vertexModelGarden.model("mistral-medium-3"),
prompt: result,
system: `
You are a world class physical fitness coach. You know how to make your clients
successful based on their previous workouts and what they are trying to
accomplish. If the user wants to target a specific muscle group, you know how to
find the appropriate muscles to target. You also know how to find the correct
rep range and the correct weight to put on the bar based on previous workouts.
You are not afraid to give advice to help make your clients successful.
Use the clients previous input to guide the design of the current program and
any critiques to modify the program be sure to incorporate.
`,
});
result += `\n\nLifting Friedn Advice: \n ${lifting_friend_output.text}`;
const lifting_pro_output = await ai.generate({
model: vertexAI.model("gemini-3-pro-preview"),
tools: [ai.defineTool({
name: "exit_loop",
description: "All work is complete, the workout cannot be improved.",
inputSchema: z.object({}).describe("Exits the loop"),
outputSchema: z.object({}),
}, async () => {
exit = true;
return {};
})],
prompt: result,
system: `
You are a world class physical fitness coach. You take workouts given to you and
look for issues with them. You know what your clients want and sometimes they
can be... over ambitious. Use your best judgement to help give them critical
feedback on the workouts selected. Make sure the client can reach their goals
safely with their workout and don't hold back on the feedback from the
previously designed workout.
If you get the best possible workout from lifting_friend and you have no
comments, call exit_loop and show the client their workout. Give a reason why
there is no feedback required before calling exit_loop.`,
});
result += `\n\nLifting Pro Feedback: \n ${lifting_pro_output.text}`;
}
return `Fitness Staff Discussion: ${result}`;
}
);
export const chatSessionFlow = ai.defineFlow(
{
name: "chatSessionFlow",
inputSchema: z.object({
sessionId: z.string().optional().describe("Optional session ID to continue an existing chat"),
message: z.string().describe("User input message"),
}),
outputSchema: z.object({
sessionId: z.string().describe("The ID of the session used"),
response: z.string().describe("The response from the chat session"),
}),
},
async ({ sessionId, message }) => {
const store = new JsonSessionStore();
let session;
if (sessionId) {
session = await ai.loadSession(sessionId, { store });
} else {
session = ai.createSession({ store });
}
const chat = session.chat({
model: vertexAI.model("gemini-3-flash-preview"),
tools: [personal_training_staff],
system: `
You are a world class physical fitness coach.You know how to make your clients successful based on their
previous workouts and what they are trying to accomplish.If the user wants to target a specific muscle group,
you know how to find the appropriate muscles to target.You also know how to find the correct rep range and
the correct weight to put on the bar based on previous workouts.You are not afraid to give advice to help make your
clients successful.
Your goal is to ask any and all questions that the training team might have about this client before designing a workout.
When you feel that there is sufficient information to design a workout for this client, send them to the personal_training_staff`,
});
const response = await chat.send(message);
return {
sessionId: session.id,
response: response.text
};
}
);Conclusion
Continuous improvement loops are a powerful agent tool that allows users to derive better outputs by running in a writer and critic loop. By using different models with different weights we are able to derive even better outputs since the biases of each model will provide a different set of outputs than if we were to use the same model for both the critic and the writer. Utilizing Vertex AI Model Garden we are able to quickly gain access to additional models from different providers without the need to worry about API key storage and rotation, we just use a service account with least privileged access.
Are you planning on trying this in your next project? Let the Genkit team know what you think!
