Being able to swap models and try out different models has a lot of appeal. Being able to bounce between Claude, Mistral, Gemini, and Llama without having to worry about rotating your API keys, having an API key leak accidentally, or from trying to track your different quotas from different consoles is a boon as well. In this article we are going to take a look at a Genkit plugin that allows you to switch models all through one plugin and provider.

Model Garden from Vertex AI

Model Garden is a collection of AI and ML models that allows you to run some partner models in a serverless environment. If the serverless deployment is not available, you also have the option to host the model through Vertex AI and it handles a lot of the configuration for you. The serverless models available at the time of this writing come from Anthropic (Claude), Meta (Llama), and Mistral AI (Mistral). All you need to do is go into model garden, select your model, and then enable it.

How to get the model into Genkit?

To get the model into Genkit you must have the Vertex AI plugin for Genkit installed in your project and then configure the modelGarden plugin which comes within that package. Once it’s there, you call generate in one of your flows and pass in the model with the vertexModelGarden.model() method and pass in the model name which can be found at the end of the model id. In my case, since I wanted to use a mistral model, I can look at the model id on the publisher page which is publishers/mistralai/models/mistral-medium-3 and then get the mistral-medium-3 model from there.

import { genkit, z } from "genkit";
import { vertexModelGarden } from "@genkit-ai/vertexai/modelgarden";

export const ai = genkit({
    plugins: [
        vertexModelGarden({
            projectId: "YOUR_PROJECT_ID",
            location: "DATA_CENTER_LOCATION",
        }),
    ],
});

export const generateSeaShantyFlow = ai.defineFlow(
    {
        name: "generateSeaShantyFlow",
        inputSchema: z.string().describe("Topic for the sea shanty"),
        outputSchema: z.string().describe("The generated sea shanty"),
    },
    async (topic) => {
        const response = await ai.generate({
            model: vertexModelGarden.model("mistral-medium-3"),
            prompt: `Write a lively sea shanty about ${topic}.`,
            config: { temperature: 0.8 },
        });
        return response.text;
    }
);

Conclusion

With one plugin, you have now enabled a wide variety of different models without needing to remember or require a key for each model. Since these models run on Google infrastructure you get the same value and protections offered by Vertex AI and can use different models as you see fit.

Give model garden a try and let the Genkit team know what you think!

Learn more by visiting the documentation at genkit.dev!