A lakehouse answers “what happened?”. A Streamhouse answers “what is true right now?”, with the same governance, and serves the answer to applications, dashboards and AI agents. Confluent’s version adds Query as a fifth pillar next to Stream, Connect, Process and Govern. Query has three ways in: Tableflow (streams as open tables for the lakehouse), Lightning Tables (low-latency SQL and API access to current state, now generally available on AWS) and the Real-Time Context Engine (current context for agents over MCP). It complements the lakehouse rather than replacing it. Our Kafka POD has been building this pattern by hand for years, with Flink-derived views, CDC, and Iceberg and Delta sinks feeding a separate serving database or cache. The Streamhouse removes much of that custom plumbing. Below: the architecture, the use cases, what it improves, which lakehouse benefits carry over, the caveats, and how we help.
What was announced
On November 4, 2026, at Current 2026 in San Francisco, Confluent (now an IBM company) introduced its Streamhouse architecture. Confluent defines a Streamhouse as a data architecture that lets AI agents and applications access and act on a governed view of the current state of the business, at machine speed, and run at production service levels.
The announcement has four parts:
| Capability | What it does |
|---|---|
| Query, the fifth pillar | Joins Stream (Kafka), Connect (managed and custom connectors), Process (Apache Flink) and Govern (schemas, lineage, catalog, quality, access control). It serves the current state of the business to whoever needs it, through the interface that fits. |
| Lightning Tables | A read-only serving layer that materialises Kafka data into durable tables and keeps them current. Applications query it with SQL or an API, and agents through MCP. In Confluent’s internal testing: about 100 ms from event to queryable, 10–100 ms query latency and 1,000+ queries per second. Generally available on Confluent Cloud on AWS. |
| Tableflow | Exposes topics as open tables (Apache Iceberg) for lakehouses and warehouses, so analytics reads the same governed data without a separate ingestion job. |
| Real-Time Context Engine | Serves current business context to AI agents over the Model Context Protocol, with OAuth, customer-managed keys and usage metrics. |
Confluent also announced a schema compatibility policy that catches breaking changes before they reach Flink, Tableflow or Lightning Tables. It also announced IBM Granite time-series models running inside Flink for forecasting and anomaly detection, and integration with watsonx.data, which queries Tableflow’s Iceberg tables without copying them.
A pattern, not just a product. In September, Aiven, Confluent, Redpanda, StreamNative and Ververica formed a Streamhouse Working Group and published a shared definition. It allows open-source, commercial or mixed implementations. That matters to us because we deliver on Confluent, Redpanda, Apache Kafka and MSK, and the architecture is the same across all of them.
The “what is true right now” gap
Confluent’s launch example is an airline. A flight is cancelled, and a rebooking agent has to offer the passenger a new connection. Flight operations knows where the aircraft is, scheduling knows the new departure time, and reservations knows which seats are free. Each system updates on its own clock. When the agent reads them at different moments, it can book a connection the aircraft cannot make.
We see the same gap in every industry we work in. An insurance claims agent needs the policy as endorsed this morning, not as of last night’s load. A fraud check needs the card’s last five swipes, not yesterday’s aggregate. A rental pricing engine needs current fleet availability. The lakehouse does a good job of answering questions about history. It was never designed to serve the current state of an entity in milliseconds to thousands of callers.
So teams built a second path, which usually looks like this:
- Kafka carries the events.
- A Flink or Kafka Streams job builds the current-state view.
- That view is written to a side database or cache (Redis, Postgres, DynamoDB, Cassandra) so applications can query it.
- A separate sink lands the same events in the lakehouse for analytics.
- Each AI use case then builds its own extract for retrieval or agent context.
Every arrow in that list is a pipeline someone has to own, secure, monitor and keep in sync with schema changes. The Streamhouse is an attempt to make that second path a first-class part of the streaming platform.
The Streamhouse reference architecture
Streamhouse and lakehouse: complements, not competitors
Confluent positions the Streamhouse as an operational complement to the warehouse and lakehouse, and that matches what we see in client estates. The lakehouse is still the right home for history, exploration and model training. The Streamhouse is the right home for the current state of each entity, served fast and consistently.
| Lakehouse | Streamhouse | |
|---|---|---|
| Question it answers | What happened, and why? | What is true right now? |
| Freshness | Minutes to hours (minutes with streaming ingestion) | Sub-second to seconds |
| Typical reader | Analysts, BI, data science, model training | Applications, operational dashboards, AI agents |
| Access pattern | Large scans, joins, aggregates | Key lookups and small filtered queries at high concurrency |
| Storage of record | Open tables (Iceberg, Delta) on object storage | The event log, plus materialised current-state tables |
| Shared foundation | Open formats, one schema contract, one governance model and one lineage graph. With Tableflow, the stream writes the lakehouse table directly. | |
How we have been building this pattern for clients
The Streamhouse gives a name and a managed product to an architecture our Kafka POD has been assembling from open components for years. A few examples from our case-study library:
Real-time point-of-sale fraud detection for a credit card company
Card transactions streamed through Kafka, with stateful processing that keeps a current profile per card and scores each swipe against it. That per-card current state is exactly what a Lightning Table now serves directly. We later extended the pattern to an AI fraud-investigation agent on Redpanda.
Real-time rental pricing on Confluent Kafka
Fleet availability, demand and competitor signals joined in the stream to keep a current price per location and vehicle class. Pricing services read the current state, and the full history landed in the lake for model training: the two read paths a Streamhouse formalises.
Real-time call insights for a contact center, and viewership analytics on Confluent Cloud
Streaming events enriched with AI/ML inference while the call or session is still live, with the results served to supervisors and agents in the moment and kept for trend analysis afterwards.
Streaming into Iceberg and Delta, and Kafka as the context layer for agents
We have written up the mechanics in Streaming Into the Lakehouse (exactly-once, CDC upserts, table maintenance) and in Kafka as the context layer for AI agents. Together with mainframe offload to Kafka and our MSK-to-Confluent Cloud migrations, these are the foundations a Streamhouse sits on.
In each engagement the hardest parts were the same: agreeing on schemas and ownership, keeping the side database in sync with the stream, and proving to auditors who could see what. Our accelerators exist to shorten that work.
What the Streamhouse makes better
Compared with the hand-built stage-2 pattern, here is where we expect the biggest improvements for clients:
- Fewer systems to run. For suitable workloads, Lightning Tables replaces the Redis, Postgres or DynamoDB side store that teams build only to make stream state queryable. That means one less pipeline, one less security review and one less sync job to debug.
- Derive once, reuse everywhere. A customer 360, claim state or order state is built once in Flink and read by apps, dashboards, the lakehouse and agents, instead of each team rebuilding it from raw topics.
- History and current state agree. Tableflow writes the lakehouse table from the same stream that feeds the current-state table, so the operational and analytical views stop drifting apart.
- Agent-ready by default. MCP access through the Real-Time Context Engine gives agents governed, current context without a bespoke retrieval service per use case.
- Governance follows the data. Schemas, lineage and access policy are applied at capture and enforced through every read path, and the new compatibility policy blocks breaking changes before they reach consumers. For agents that act on data, this matters more than it ever did for dashboards.
- A built-in feature store. Features computed in Flink are stored for ML serving, which narrows the gap between training data in the lakehouse and inference data in production.
- Reach into legacy systems. With IBM, the platform reaches further into IBM Z mainframes and enterprise databases, the sources that are often the real system of record in insurance and banking.
Use cases where it pays off first
| Industry | Current-state question | Who reads it |
|---|---|---|
| Insurance | What is this policy’s coverage, endorsements and open claims right now? Is this FNOL a duplicate? | Claims agents, broker portals, underwriting copilots |
| Banking & payments | What are this card’s last transactions, risk score and dispute status? | Fraud engine, investigation agents, customer service |
| Retail & CPG | What stock is available at this store, and where is this order? | Order-change agents, store apps, replenishment |
| Rental & mobility | Which vehicles are available here, and what is the current price? | Pricing services, booking apps, fleet operations |
| Travel & logistics | Where is the aircraft or shipment, and can this connection still be made? | Rebooking agents, customer notifications |
| Contact centers | What has this customer done in the last ten minutes across channels? | Live agent assist, supervisors |
| Manufacturing & utilities | What is this asset’s current state, and is it drifting from normal? | Operations dashboards, anomaly detection (for example, Granite time-series models in Flink) |
The best first use case is a narrow current-state lookup that today depends on a side database: order status, policy status or card profile. It is easy to measure (latency, freshness, one system retired) and easy to expand once it works.
The lakehouse benefits that carry over
Clients adopted the lakehouse for a consistent set of reasons. Each one has a direct counterpart in the Streamhouse:
| What companies get from the lakehouse | How it applies to the Streamhouse |
|---|---|
| One copy of the data instead of a lake plus several warehouse extracts | One stream feeds history (Tableflow), current state (Lightning Tables) and agents (MCP), so there are no side copies to reconcile |
| Open formats and no lock-in to one query engine | Iceberg tables from the stream are readable by Databricks, Snowflake, Trino, Spark and watsonx.data. The Streamhouse definition is shared across five vendors. |
| Unified governance: one catalog, lineage and access model | The schema contract, lineage and access policy are set at capture and enforced on every read path, including agent access |
| Lower cost from cheap object storage and retired ETL | Fewer ETL jobs, caches and serving databases. History stays on object storage, and only current state is materialised for fast reads. |
| BI and ML on the same platform | Analytics, ML features and operational apps share one derived view. Training data and inference data come from the same pipeline. |
| Faster time to insight | New consumers subscribe to existing governed views instead of commissioning a pipeline. At launch, Accenture described time-to-insight dropping “from months to days” on its Streamhouse. |
| Reliable, ACID tables | Exactly-once processing and durable, schema-enforced tables, with the event log kept for replay and audit |
What to plan for
We are enthusiastic about the pattern, but we also want clients to go in clear-eyed. Independent analysts have raised the same points we would:
1. Read-only by design. Lightning Tables serve state. They do not enforce transactions. A reservation or payment still needs its own check in the operational system before an agent acts.
2. Current is not the same as consistent. Teams still have to decide which source is authoritative for each field, how late events are handled, and when incomplete data should block an action rather than allow it.
3. Deployment coverage. Lightning Tables is generally available on Confluent Cloud on AWS today. Self-managed, Azure and GCP estates need a bridge plan: a Flink-maintained serving store now, then migration when coverage arrives.
4. Vendor latency figures are not end to end. The 100 ms and 10–100 ms numbers come from internal testing of the serving layer. Measure source-to-action latency in your own environment.
5. Agents need an audit trail. Current state cannot reconstruct a past decision. Log what context an agent read and when, and map source-system permissions to agent access explicitly.
6. Continuous processing has a running cost. Always-on Flink, storage and query capacity need a cost model per use case, and a clear owner between the data platform and AI budgets.
How VerticalServe helps you adopt the Streamhouse
Our Kafka POD is 3–6 senior streaming engineers working inside your environment. For Streamhouse adoption we offer:
- Streamhouse readiness assessment (2–3 weeks). We inventory your topics, side databases, caches and lakehouse ingestion jobs, identify which current-state lookups are candidates, and produce a sequenced roadmap with a cost model.
- Schema and data-contract foundation. Schema Registry compatibility policy, topic ownership and naming, and PII classification, so every read path inherits governance. Our Schema UI and Stream DLP & Catalog accelerators speed this up.
- Flink-derived business entities. We design and build the customer, policy, order or card views once, with correct windowing, state sizing and reprocessing strategy.
- Tableflow and open-table integration. Streams landed as Iceberg (or Delta through our Databricks POD) and wired into Databricks, Snowflake, Trino or watsonx.data, with catalog and lineage in place.
- Lightning Tables pilot. We pick one side database to retire, run the old and new paths in parallel, reconcile them, and cut over only on evidence.
- Agent context over MCP. We connect the Real-Time Context Engine to your agents (Gemini Enterprise, Claude, Copilot or custom), with OAuth scopes, decision logging and guardrails for stale or partial data.
- Vendor-neutral delivery. On Redpanda, Apache Kafka or MSK, or where Lightning Tables is not yet available, we build the same architecture with Flink and an open serving layer, so the design carries over when you are ready to move.
- Migration to Confluent Cloud. Our automated migration tooling moves producers, consumers, Streams apps and Flink jobs from self-managed Kafka or MSK, run in parallel with reconciliation before cutover.
- Operate and enable. Lag and freshness SLOs, DR rehearsal, cost tuning, runbooks and paired delivery until your team runs it.
Once current state is governed and queryable, our AI products can read it. InsightCompose, for example, turns governed data into verified insights, dashboards and data APIs, each published as a versioned endpoint in one catalog. Pointed at Streamhouse tables, those insights describe the business as it is now, not as of last night.
A practical first 90 days
Weeks 1–3: Assess
Map current-state lookups and the side stores behind them. Choose one use case with a measurable win. Agree on freshness and latency targets with the people who read the data.
Weeks 4–6: Foundation
Set schema contracts and compatibility policy. Build the Flink-derived entity. Land it with Tableflow so analytics sees the same data from day one.
Weeks 7–10: Serve
Stand up a Lightning Table (or the open equivalent). Run it in parallel with the existing side store, reconcile, and measure end-to-end latency.
Weeks 11–13: Act
Cut the application over, retire the old store, and connect the first agent through MCP with decision logging. Hand over runbooks and SLOs.
Streamhouse readiness checklist
| Area | Ready when |
|---|---|
| Use case | One current-state lookup chosen, with a freshness target and a side store to retire |
| Sources | The system of record is streaming (CDC or events), with topics keyed by entity ID |
| Contracts | Schemas registered, with compatibility rules that block breaking changes |
| Derived views | Each entity built once in Flink, with an owner and a documented definition |
| Authority | The authoritative source per field and the rules for late events are written down |
| Read paths | Each consumer mapped to Tableflow, Lightning Tables or MCP |
| Access | Source permissions mapped to app and agent scopes; agent reads logged |
| Writes | Transactional actions still checked in the operational system |
| Observability | Lag, freshness and query-latency SLOs alerting end to end |
| Cost | Processing, storage and query cost modelled per use case, with an owner |
The bottom line
The lakehouse gave enterprises one governed copy of their history. The Streamhouse applies the same idea to the present: one governed stream that serves history to analytics, current state to applications and context to AI agents, without a fleet of side databases in between.
For teams already running Kafka and Flink, it is an evolution rather than a rebuild. Most of the hard work is the same work we have done with clients for years: schema contracts, Flink-derived entities, clean streaming into open tables, and governance that holds up to an auditor. Start with one current-state lookup, prove it, retire a system, and then let agents read from it.
Product details, availability and performance figures are as announced by Confluent on November 4, 2026 and reported in its press release and independent analyst coverage. Verify current availability for your cloud and region.