Google Cloud Expands AI Agent Observability and Controls

Google deploys a unified framework to track, govern, and evaluate AI agents in BigQuery and Looker. Some components are generally available, while others are in preview, featuring privacy safeguards, cost ceilings, and open-source evaluation tools for both native and custom agents.

Evaluate Before Deployment and Correlate with Production Logs

The open-source tool Prism allows for testing conversational analytics agents via the API, whether they rely on BigQuery tables or Looker explorations. It manages reusable test suites and regex assertions, as well as the discovery of multi-region agents behind an access layer such as Cloud IAP. Prism can stream evaluation executions into daily-partitioned BigQuery tables, placing pre-deployment benchmark scores alongside production agent_events logs for joint analysis in BigQuery and Looker. Meanwhile, first-party agent evaluations in BigQuery are available in preview via the API and a guided Colab notebook, leveraging the evaluation service of the Gemini Enterprise agent platform with support for a customer-specific service account.

Protect Data While Tracing Agent Reasoning

Administrators can enable Cloud Monitoring and Cloud Trace at the project or organization level. Cloud Monitoring provides aggregated metrics such as request volumes, latency, and token usage, while Cloud Trace offers step-by-step spans for each request, with observability scopes to aggregate multiple projects. By default, sanitized 1P metrics and spans exclude prompts, responses, internal thoughts, and generated SQL, allowing for the separation of operational telemetry and content. If finer auditing is necessary, conversation logging can be enabled per agent: prompts, responses, and telemetry per turn are streamed to the BigQuery agent_events table via the BigQuery Storage Write API, inheriting IAM permissions, CMEK keys, and VPC Service Controls. Safeguards apply a fail-closed check at each turn and skip logging in the presence of row-level security, column policy tags, or dynamic masking; a fail-open execution maintains sessions in case of errors.

A Unified Telemetry Framework Covers Data and Custom Agents

A unified telemetry framework now encompasses both 1P data agents and custom agents. Conversational analytics agents automatically emit telemetry, and agent-to-agent flows to BigQuery agents in Gemini Enterprise log Cloud Monitoring metrics, Cloud Trace spans, and conversation logs in the relevant project. Agents built with ADK or LangGraph can, with a single line of instrumentation, stream the same structured telemetry to BigQuery. Prompts and responses from 1P logging are stored in BigQuery tables owned by the client. The conversational analytics API is offered in general availability, with administrative audit logs systematically active and optional read audit logs that can be enabled, while ensuring compatibility with VPC Service Controls, CMEK, as well as regional or multi-regional data residency. BigQuery Studio offers predefined dashboards covering usage, active sessions, knowledge sources, and latency. In Looker (Google Cloud core and original), system activity tables, available generally, track engagement and token usage, while the Responses and Feedback tab provides a preview of user evaluations and query reviews. It is also possible to create reports using an open-source Data Studio template or deploy a ready-to-use Looker Block.

Token FinOps: Detailed Tracking, Native Dashboards, and Active Ceilings

Promotional pricing offers are available for conversational analytics, enabling the establishment of safeguards. At each turn of conversation, BigQuery logs the counts of input, output, thought, cache tokens, as well as the total, in addition to latency. System activity tables in Looker provide an estimate of input and output tokens for BI loads, while the monitoring tab in BigQuery Studio allows for visualizing expected usage and engagement. The agent_events data can be queried in SQL and presented graphically using a seven-page open-source Data Studio template or via a user-hosted Streamlit application. To avoid unexpected charges, big_query_max_billed_bytes limits per query, based on dry-runs, allow blocking overly large analyses without incurring on-demand billing. In preview, daily quotas for input and output tokens apply per project, user, or agent, configurable in the Google Cloud console and via gcloud CLI.

Diagnostics in the Warehouse and Instrumentation of Custom Agents

Developers of custom agents on ADK or LangGraph can use the open-source BigQueryAgentAnalyticsPlugin; a one-line configuration allows streaming complete execution trees, temporal telemetry, and multimodal loads to BigQuery. Logs landing natively in agent_events require no additional export to diagnose failures. AI BigQuery functions like AI.GENERATE or AI.EMBED with VECTOR_SEARCH serve for error summarization and incident grouping, while the Python SDK bigquery-agent-analytics and a Colab notebook facilitate post-hoc semantic drift detection and sentiment analysis. Google recommends adding the SDK to agents, launching the open-source Data Studio report template or the Streamlit app, and exploring analysis in the warehouse with a Colab notebook. It also suggests running automated accuracy suites with Prism, then configuring scan ceilings and daily token quotas. For 1P agents, enabling Cloud Monitoring and Cloud Trace, opting in for conversation logging, and tracking BI token consumption via Looker are advised.