Most enterprises already run a dozen disconnected agent pilots, each with its own retrieval stack, its own service account and its own idea of what “approved” means. Gemini Enterprise lets you replace that sprawl with one platform: the Gemini Enterprise app as the place employees find and use agents, the Gemini Enterprise Agent Platform as the place developers build them, and one governance plane — Agent Registry, Agent Identity, Agent Gateway, Model Armor, IAM, VPC Service Controls, CMEK and audit — that every agent passes through, however it was built. This post sets out the layered reference architecture, the landing zone and environment layout, the operating model between a central AI platform team and domain agent teams, a catalog of agentic use cases mapped to building blocks, a guide to choosing between Agent Designer, Agent Studio and ADK, the path of one governed agent call, and a 90-day rollout. Parts 2 to 5 of the series go deep on the knowledge base, the agentic harness, real-time context and L1 resolution.
Gemini Enterprise on Google Cloud · a five-part series
1. Reference architecture ·
2. Enterprise knowledge bases ·
3. The agentic harness ·
4. Real-time context ·
5. Agentic L1 resolution
Why standardize on one platform
The first wave of enterprise generative AI followed a pattern: HR built a policy bot on one vector database, IT a ticket summarizer on another, sales bought a CRM copilot, and engineering wired an open-source framework to a shared API key. Each worked in a demo. Together they produced four problems no single team could solve.
- Duplicated plumbing. Every pilot rebuilt connectors, chunking, embeddings, evaluation and logging.
- Inconsistent access control. Some assistants respected document permissions; some indexed everything under one service account and showed every employee every answer.
- No inventory. Nobody could answer “which agents do we run, what can they touch, who owns them, and what did they do last Tuesday?” That is the first question an auditor, a regulator or a board risk committee asks.
- No path from pilot to production. Without shared environments, evaluation gates and an operating model, pilots either stayed pilots or went live with nothing between them and production systems.
Standardizing on Gemini Enterprise does not mean every agent must be built the same way. It means every agent — prebuilt, no-code, low-code or custom code — is discovered in the same place, runs with its own identity, reaches tools and data through the same enforcement points, and is evaluated and observed the same way. The platform choice is really a governance choice. The rest of this post is about making that concrete.
The principle in one sentence: let teams build agents in whatever way fits the job, but give every agent one front door, one identity model, one gateway to tools and one set of controls.
The reference architecture
Google describes Gemini Enterprise as an end-to-end system with two halves. The Gemini Enterprise app is where employees discover, create, share and run agents. The Gemini Enterprise Agent Platform is the developer platform — the evolution of Vertex AI, through which all Vertex AI services are now delivered. Around those two halves we draw four horizontal layers, a governance plane that cuts through all of them, and an operations layer underneath.
Layer 1 — Experience
The Gemini Enterprise app is the front door. Employees search and chat across connected data, open agents from the Agent Gallery (Google-built, partner agents from Google Cloud Marketplace, and your own registered agents), build simple agents themselves in the no-code Agent Designer, manage the work agents are doing for them in Inbox, and collaborate with colleagues and agents in shared Projects. Canvas, Gemini Notebook Enterprise and a customizable RAG agent round out the app. Make it the default surface: if an agent exists, employees should find it in the gallery, not on a wiki page.
A customer portal, a chat widget in a field-service tool or a Slack or Teams support channel are embedded channels. The same agents serve them, typically through an API layer you own (often on Cloud Run) that calls the agent on Agent Runtime and passes the end user’s identity through. One agent, many channels, one set of controls — never fork an agent to serve a second channel.
Layer 2 — Agents
Four kinds of agents live here. Prebuilt agents come from the Agent Gallery and partners. No-code agents are built by business users in Agent Designer. Low-code agents are assembled on the Agent Studio canvas by technical analysts and developers. Custom agents are written with the Agent Development Kit (ADK), a model-agnostic, code-first framework, and hosted on Agent Runtime (formerly Vertex AI Agent Engine), with Agent Platform Sessions for conversation state and Memory Bank for persistent memory across sessions. Agents built on other stacks, or owned by partners, join through the A2A protocol and can be registered into the app wherever they are hosted.
All four kinds surface in the same app and fall under the same registry and identity model, however differently they were built. Part 3 covers the code-first side in depth.
Layer 3 — Tools and actions
Agents become useful when they act. Every tool call and every agent-to-agent call goes through Agent Gateway, the policy enforcement point for MCP and A2A traffic, which uses IAP and access policies. Behind the gateway sit MCP servers (Google provides a template that connects an agent to a BigQuery MCP server usable only by that agent’s identity; you will write others for your own APIs), systems of record such as ServiceNow, ERP and CRM, and workflows in Application Integration, Workflows or Eventarc for multi-step actions that should run deterministically rather than be improvised by a model. When an agent must act as a particular user, an OAuth authorization resource lets it do so with that user’s permissions rather than a broad service account.
Layer 4 — Knowledge and context
Two kinds of context feed agents. Unstructured knowledge — policies, procedures, runbooks, contracts, tickets — comes in through prebuilt connectors for Confluence, Jira, SharePoint, ServiceNow, Google Workspace and others. Connectors either index content into the Gemini Enterprise search index or federate queries to the source without copying, and both modes return permission-aware results. For specialist corpora you need to chunk and tune yourself, RAG Engine and Vector Search sit alongside. Structured and live context — balances, claim status, inventory, incidents — comes from BigQuery (including continuous queries) and from streams on Pub/Sub, Datastream CDC or Google Cloud Managed Service for Apache Kafka. Part 2 covers the knowledge base; Part 4 covers real-time context.
The governance plane and operations
The governance plane is drawn vertically because it is not a step in the flow. Agent Registry is the catalog of approved agents, tools and MCP servers. Agent Identity gives each agent a unique cryptographic identity used for access control and audit. Model Armor screens prompts and responses for prompt injection, tool poisoning and sensitive-data leakage. Cloud IAM decides who can use what, VPC Service Controls draws a perimeter around the data, CMEK keeps encryption keys under your control in Cloud KMS, and Cloud Logging holds the audit trail. Underneath, operations covers agent evaluation (including multi-turn autoraters and online evaluation), Agent Simulation with synthetic multi-turn users, observability through logging, tracing and the Unified Trace Viewer, and cost.
Components mapped to Google Cloud services
This is the table we hand to architecture review boards. It names the service that fills each role and the decision the platform team owns.
| Role | Google Cloud service | Platform team decides |
|---|---|---|
| Employee front door | Gemini Enterprise app: Agent Gallery, Inbox, Projects, Canvas | Edition and licensing per population; which agents are visible to which groups |
| No-code building | Agent Designer | Who may build, what data and actions designer agents may use, when they need review |
| Low-code building | Agent Studio, Agent Garden templates | Approved templates and tool catalog |
| Code-first building | ADK, Model Garden, Managed Agents API | Reference project layout, CI/CD, evaluation gates |
| Agent hosting | Agent Runtime, Agent Platform Sessions, Code Execution sandbox | Project per domain, scaling and quota, region |
| Unstructured knowledge | Connectors, Gemini Enterprise search index, RAG Engine, Vector Search | Index vs federate per source; permission model; freshness targets |
| Structured and live context | BigQuery, Pub/Sub, Datastream, Dataflow, Managed Service for Apache Kafka | Curated, agent-ready datasets and topics; masking |
| Tool access | Agent Gateway, MCP servers, Application Integration, Workflows | Tool contracts, authority limits, which agents reach which tools |
| Catalog | Agent Registry | Approval criteria and lifecycle states for agents, tools and MCP servers |
| Identity | Agent Identity, Cloud IAM, OAuth authorization resources | One identity per agent; when an agent acts on behalf of a user |
| Content safety | Model Armor | Templates and thresholds per risk tier |
| Data protection | VPC Service Controls, CMEK with Cloud KMS, Secret Manager, Dataplex | Perimeter design, key ownership, data classification |
| Quality | Agent evaluation, Agent Simulation, prompt optimization | Golden sets, pass thresholds, release gates |
| Observability | Cloud Logging, Cloud Monitoring, Unified Trace Viewer, Error Reporting | Trace standards, SLOs, cost attribution labels |
Landing zone and environment layout
Agents are software with unusual privileges, so they deserve the same environment discipline as any production system — plus a few things specific to AI. The layout below is the one we start from. It assumes an existing Google Cloud organization with a landing zone; if you do not have one, build that first.
The decisions behind the layout
- Separate projects per environment, not just separate agents. Dev, test and prod each get their own Gemini Enterprise app configuration and their own Agent Runtime projects. Dev connectors point at masked or synthetic data; only prod connectors see live content.
- One Agent Runtime project per domain per environment. Claims, finance and IT each get
agents-claims-prod-style projects. Quotas, budgets, IAM and cost attribution follow the project boundary. - Shared services in their own folder. The governance project holds Agent Registry configuration, Model Armor templates and the policy-as-code repository’s deployed state. The shared data project holds curated, agent-ready BigQuery datasets that several domains read. The network hub holds the Shared VPC and controlled egress.
- One VPC Service Controls perimeter around the AI projects, the data projects and the key project, with explicit ingress rules for the identity provider and the SaaS connectors and explicit egress rules for any external tool an agent may call.
- Keys and logs outside the workload projects. CMEK keys live in a dedicated Cloud KMS project; logs route to a central sink with retention set by compliance. A team that can deploy an agent should not be able to delete its audit trail.
- Everything as code. Projects, IAM bindings, perimeters, connectors where the API allows it, agent registration and Model Armor templates are all in Terraform or deployment pipelines. Promotion moves the same version forward; nobody rebuilds it by hand in the console.
Operating model: platform team and domain agent teams
The architecture fails without a clear split of responsibilities. The pattern that works is a small central AI platform team that owns the rails and many domain agent teams that own the agents. The platform team is not a bottleneck that builds every agent; it is the team that makes building a safe agent the easiest option.
| Responsibility | AI platform team | Domain agent team |
|---|---|---|
| Landing zone, perimeter, keys | Owns | Consumes |
| Gemini Enterprise app configuration, editions, connectors | Owns | Requests sources |
| Agent Registry, approval workflow, Model Armor templates | Owns | Submits agents and tools |
| Shared MCP servers and tool catalog | Owns common tools | Builds domain tools to the standard |
| Reference ADK project, CI/CD, evaluation harness | Owns | Uses; adds domain golden sets |
| The agent: prompts, logic, tools, authority limits | Reviews | Owns |
| Business outcome and adoption | Reports | Owns |
| Production incidents | Platform incidents | Agent behavior incidents |
Intake and approval of new agents
Every new agent, however it is built, follows the same short path. The depth of each step scales with risk.
Intake
A one-page request: the job to be done, the users, the data sources, the tools and actions, the business owner, and the measure of success. The platform team assigns a risk tier — for example, tier 1 read-only over non-sensitive content, tier 2 read access to sensitive data, tier 3 writes to systems of record or acts externally.
Build path
The decision guide below picks the build path: Gallery agent, Agent Designer, Agent Studio, ADK or A2A.
Evaluate
The domain team supplies a golden set of real questions and expected behavior. Tier 2 and 3 agents also run through Agent Simulation with adversarial and edge-case synthetic users before anyone else sees them.
Approve and register
Security and the business owner sign off at the level the tier requires. The agent is entered in Agent Registry with its owner, version, identity, tools and authority limits, and made visible in the Gemini Enterprise app to the groups that should use it.
Operate and review
Online evaluation and traces run continuously, tier 3 agents are reviewed quarterly, and unused agents are retired rather than left running.
Agentic use cases across the enterprise
Once the rails exist, the backlog fills fast. The table is a representative catalog from the functions we see asking first, with the build path and the building blocks each one leans on. Four rows are the subjects of the rest of this series.
| Use case | What the agent does | Build path | Key building blocks |
|---|---|---|---|
| Enterprise knowledge base | Answers policy, product and procedure questions with citations, respecting document permissions (Part 2) | App + connectors; RAG Engine for specialist corpora | Connectors, search index, permission-aware answers, Gemini Notebook Enterprise |
| Agentic harness for engineering and ops | Multi-agent workflows for code review, release notes, runbook execution and incident triage (Part 3) | ADK | Agent Runtime, Memory Bank, Code Execution sandbox, Agent Gateway, Agent Identity |
| Real-time context agents | Answer “what is happening now” on orders, payments, shipments or claims from live data (Part 4) | ADK | Pub/Sub or Kafka, Datastream, BigQuery continuous queries, BigQuery MCP server |
| L1 support resolution | Diagnoses and resolves routine IT and customer tickets within authority limits, escalates the rest (Part 5) | ADK, surfaced in app and ServiceNow | ServiceNow connector, knowledge base, MCP tools, Agent Gateway, Inbox |
| Underwriting assistant | Assembles submission, loss history and appetite guidelines; drafts a risk summary for the underwriter | ADK or Agent Studio | Document extraction, BigQuery, knowledge base, Projects |
| Claims intake and triage | Reads first notice of loss and documents, checks coverage, routes and sets an initial priority | ADK | Real-time claim context, coverage knowledge, workflows, Agent Identity |
| Finance close | Reconciliation exceptions, variance commentary drafts, close checklist status | Agent Studio or ADK | BigQuery MCP server, ERP tools via gateway, Canvas for commentary |
| Procurement | Intake to the right buying channel, supplier and contract lookup, draft requisitions | Agent Designer to start; ADK for writes | Contract repository connector, ERP tools, approval workflow |
| HR service | Benefits and leave questions, policy lookups, onboarding checklists | Agent Designer or Gallery | HR policy knowledge, permission-aware search, HR system tools for tier 3 actions |
| Sales account prep | Account briefs from CRM, recent tickets, contracts and news; next-step drafts | Agent Designer or Agent Studio | CRM connector, knowledge base, Projects, Canvas |
| Contract review | Compares a contract to playbook positions and flags deviations | ADK | RAG Engine over playbooks, document extraction, Model Armor sensitive-data checks |
| Data and analytics questions | Answers governed metric questions and explains the query it ran | ADK | BigQuery, Dataplex metadata, BigQuery MCP server |
Two patterns are visible in the table. First, almost every use case reuses the knowledge layer and a small number of shared tools — which is exactly why building those once, centrally, pays off. Second, the build path tends to move rightward over time: procurement starts as a Designer agent that answers questions and becomes an ADK agent when the business wants it to raise requisitions. The architecture lets that happen without changing the front door or the controls.
Build or configure: Agent Designer, Agent Studio or ADK
The most expensive mistake in a new agent program is writing code that a configuration would have covered; the second most expensive is pushing a no-code tool past what it was designed for. The decision tree below is the one our intake step uses.
| Path | Who builds | Good fit | Watch for |
|---|---|---|---|
| Gallery / partner agent | Nobody — configure | Common jobs: research, document drafting, meeting follow-up, standard SaaS workflows | Data residency and what the partner agent sends where; check its registry entry and tools |
| Agent Designer | Business users, analysts | Personal and team assistants over connected data; drafting; low-risk actions | Sprawl: hundreds of near-duplicate agents. Promote the good ones, retire the rest |
| Agent Studio | Technical analysts, developers | Multi-step flows with branching, using catalog tools and templates from Agent Garden | Logic that outgrows the canvas; untestable branching. Move to ADK when tests matter |
| ADK on Agent Runtime | Software engineers | Custom tools, multi-agent orchestration, writes to systems of record, strict evaluation and versioning | Building from scratch what a Designer agent would have done in an afternoon |
| A2A registration | Owning team or partner | Agents already built on another framework or cloud | Identity and gateway policy still apply; registration is not an exemption |
Walkthrough: one governed agent call
Architecture diagrams hide the thing that matters most: what actually happens between an employee’s question and the tool call it triggers. Here is one request through an IT support agent built with ADK, registered in the Gemini Enterprise app, that can look up and update ServiceNow incidents through an MCP server. Part 5 develops the full L1 agent.
The employee asks
“My VPN keeps dropping since this morning — is there an incident, and can you log a ticket if not?” Corporate SSO tells the app who is asking and which groups they belong to.
The app checks the user may use this agent
Admins set which users can use each registered agent. If the employee is allowed, the app invokes the agent on Agent Runtime with the conversation and the user’s context.
The prompt is screened
Model Armor checks the input for prompt injection and sensitive data before the model reasons over it.
The agent calls a tool under its own identity
The agent decides it needs search_incidents and possibly create_incident. It calls them through Agent Gateway using its Agent Identity. Where the ticket must be raised as the employee, an OAuth authorization lets the agent act on the user’s behalf, with that user’s permissions.
The gateway enforces policy
Agent Gateway checks that this agent identity is allowed to reach this MCP server and this tool under the access policy, consistent with its Agent Registry entry. An agent registered for read-only incident search that tries to call close_incident is refused at the gateway, regardless of what its prompt says.
The tool runs and the result returns
The MCP server queries ServiceNow, finds an open VPN incident affecting the employee’s region, and returns it. The gateway logs the call, and the agent links the employee to it instead of opening a duplicate.
The response is screened and returned
Model Armor screens the outgoing answer for sensitive data. The employee sees the incident, its status and the workaround from the knowledge base, with citations. The whole chain — user, agent identity, version, tools, policy decisions and outcome — is in Cloud Logging and visible as one trace.
Security, identity and governance
The controls are only useful if each one has a clear job. This is how we divide them.
- Users: Cloud IAM and the identity provider. Which employees may use which agents, mapped to groups from the corporate directory.
- Agents: Agent Identity. Every agent has its own identity. Least privilege applies to it exactly as it would to a person: read on the datasets it needs, call rights on the tools it needs, nothing else.
- On-behalf-of access: OAuth authorization. When an action must carry the user’s permissions (their calendar, their tickets, their documents), the agent acts on the user’s behalf rather than holding a broad grant itself.
- Tools: Agent Registry and Agent Gateway. Only registered tools and MCP servers are reachable, and only by agents whose policy allows it. Authority limits — maximum refund, which ticket states an agent may set — are enforced in the tool implementation as well, so a manipulated model cannot talk its way past them.
- Content: Model Armor. Templates per risk tier for prompt injection, tool poisoning and sensitive-data leakage, applied to inputs, outputs and tool responses.
- Data: permissions, perimeter and keys. Connectors return permission-aware results, VPC Service Controls stops data leaving the perimeter, CMEK keeps keys in your Cloud KMS, and Dataplex classification tells you which datasets an agent may be granted at all.
- Evidence: audit logs. Admin activity, data access and traces in a central sink agent teams cannot modify.
A useful test: pick any agent in production and ask, within five minutes, who owns it, which version is live, what it can reach, who can use it, and what it did yesterday. If any answer needs a meeting, the governance plane is not finished.
Evaluation and operations
An agent that passed review in March can be wrong in June because the documents changed, the model changed or the users started asking different questions. Operations is how you notice.
- Offline evaluation before every release. Golden sets per agent, scored with multi-turn autoraters for correctness, groundedness, tool-call accuracy and policy adherence. Releases that score below the agreed threshold do not promote from test to prod.
- Agent Simulation for the cases you did not think of. Synthetic multi-turn users with varied goals and adversarial intent cover far more ground than manual testing; use it for every tier 3 agent.
- Online evaluation in production. Sample live conversations and score them continuously; alert when quality drifts.
- Tracing as the default debugging tool. Every request carries a trace through the app, the agent, the model calls and the tool calls. When something goes wrong, the Unified Trace Viewer shows which step failed and with what input.
- Cost per agent and per team. Project-per-domain layout plus labels gives you spend by agent and by business unit. Track cost per resolved task, not just total spend.
- SLOs that matter to users. Latency to first token, task success rate, escalation rate, and for context agents, data freshness. Cloud Monitoring alerting on these, routed to the owning domain team.
Where teams want a single view across agents built on several stacks, an LLM tracing and evaluation layer such as our InsightLense can sit alongside Google’s observability; the architecture does not require it.
A 90-day rollout
The fastest programs we have seen run two tracks in parallel: the platform team builds the rails while two or three domain teams deliver the first agents on them. Waiting for a perfect platform before shipping any agent loses executive patience; shipping agents with no platform creates the sprawl you were trying to avoid.
| Phase | Platform track | Agent track | Exit criteria |
|---|---|---|---|
| Days 1–30 Foundations | Landing zone projects, perimeter design, CMEK, log sink. Gemini Enterprise app in dev and test. First connectors (for example SharePoint and Confluence) in index or federated mode. Registry approval workflow and risk tiers agreed. | Pick two or three use cases: one knowledge base, one Designer agent, one ADK agent with a read-only tool. Write golden sets. | An employee pilot group uses the knowledge base in test with permission-aware answers; first ADK agent registered in dev. |
| Days 31–60 First production | Prod app and connectors. Agent Gateway and the first shared MCP servers. Model Armor templates per tier. Reference ADK project and CI/CD with evaluation gate. Trace and cost dashboards. | Knowledge base and Designer agent go live for a wider group. ADK agent passes simulation and reaches prod read-only. | Three agents in prod, each with an owner, an identity, a registry entry and an evaluation baseline. |
| Days 61–90 Scale the pattern | Self-service intake. Shared tool catalog published. Real-time context feed (for example Datastream to BigQuery) for the next wave. First quarterly governance review. | First agent with write actions (tier 3) behind authority limits. Two or three new domain teams onboard using the reference project. | New agent from intake to prod in weeks, not months; adoption and outcome metrics reported per agent. |
VerticalServe has spent seven years building data and AI platforms on Google Cloud — including an enterprise GenAI platform for a global retailer and Vertex AI platforms for a large technology company and a large utility — and runs production agentic systems in insurance. The 90-day shape above is what we use when we stand up a Gemini Enterprise program with a client’s platform team.
Failure modes
1. One service account for every agent. Without Agent Identity per agent you cannot scope access, cannot tell agents apart in the logs and cannot contain a compromise.
2. Indexing everything as an admin. A connector configured to bypass source permissions turns the knowledge base into a data-leak engine. Permission-aware access is the point; test it with real users from different groups before launch.
3. Tools reached around the gateway. An ADK agent with a hard-coded API key to a system of record is invisible to Agent Gateway, Registry and audit. Block direct egress at the perimeter so the gateway is the only route.
4. Designer sprawl with no lifecycle. No-code building is valuable, and it will produce hundreds of agents. Without ownership, review and retirement, the gallery becomes a junk drawer and users stop trusting it.
5. No evaluation gate. Agents promoted on a demo and a thumbs-up regress silently when documents, models or prompts change.
6. Perimeter designed last. VPC Service Controls retrofitted after twenty agents are live breaks connectors and tool calls in ways that take weeks to untangle. Design the perimeter with the landing zone.
7. Platform team as the only builder. If every agent queues behind one central team, domains go around it. Own the rails centrally; let domains own their agents.
Readiness checklist
| Area | Ready when |
|---|---|
| Landing zone | Dev, test and prod projects exist per domain, built from the same infrastructure code |
| Perimeter | VPC Service Controls perimeter covers AI, data and key projects with explicit ingress and egress rules |
| Keys and logs | CMEK in a dedicated Cloud KMS project; audit logs in a central sink agent teams cannot modify |
| Connectors | Priority sources connected with index or federate chosen per source; permission-awareness tested with real users |
| Registry | Approval workflow and risk tiers defined; every production agent, tool and MCP server registered with an owner |
| Identity | One Agent Identity per agent with least privilege; on-behalf-of access uses OAuth authorization |
| Gateway | All tool and agent-to-agent traffic routes through Agent Gateway; direct egress blocked |
| Content safety | Model Armor templates per risk tier on inputs, outputs and tool responses |
| Evaluation | Golden sets and thresholds per agent in CI; Agent Simulation for tier 3; online evaluation in prod |
| Operating model | Platform and domain responsibilities written down; intake-to-prod path published |
The bottom line
Gemini Enterprise gives an enterprise the pieces for one agent platform: an app employees actually use, several ways to build agents for several kinds of builder, and a governance plane that applies to all of them. The pieces do not assemble themselves. The value comes from the decisions around them — one front door, one identity per agent, one gateway to every tool, one perimeter, one evaluation gate — and from an operating model that lets domain teams move fast on rails the platform team owns.
Get those decisions right in the first 90 days and the tenth agent costs a fraction of the first, because the connectors, tools, controls and evidence already exist. The rest of this series goes one level down: permission-aware knowledge bases, the agentic harness, real-time context (which builds on our earlier post on Kafka as the context layer for agents) and agentic L1 resolution.