Forward audit logs to your SIEM
The AI Gateway emits structured JSON audit events to pod standard output. It writes one event for each model interaction, one for each rejected authentication attempt, and one for each model list request that model access filters down to nothing.
Events record decisions and metadata only. They never carry prompts, completions, credentials, request or response bodies, token claims, or email addresses.
Every event carries the fields required by NIST SP 800-53 controls AU-2, AU-3, and AU-8: a unique audit ID, a UTC timestamp, the outcome, the component, the source, an optional target, and an extra block for forensic correlation. Events for authenticated requests also carry the subject.
For the legacy AI Gateway, see legacy gateway audit logs.
Enable audit events
Audit events are off by default. Turn them on for the gateway:
ai-gateway-v2:
audit:
enabled: true
Setting global.stacklok.audit.enabled: true also turns them on, and while it's
true, audit.enabled: false has no effect. The change applies when the
gateway pods roll out; see
How configuration changes apply.
Enabled audit events cover every interaction without sampling, and there are no per-event toggles. Filter and route events in your log pipeline.
What gets captured
| Event type | Fires when | Outcome |
|---|---|---|
aigw.v2.interaction.completed | An authenticated generation request reaches its final outcome, including an early refusal | success, denied, failure, or error |
aigw.v2.authentication.failed | The gateway rejects a credential on a protected route, or can't reach a service it needs to validate it | failure, or error when unreachable |
aigw.v2.models.no_grant | Model access filters a non-empty model list to nothing, and the caller receives a 200 | denied |
Interaction events cover the OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and Gemini generate and stream-generate endpoints. A streaming interaction emits one event when it finishes, never one per frame. Health endpoints emit no events, and a model list request that returns at least one model emits none.
Event schema
{
"time": "2026-10-16T14:22:31Z",
"level": "INFO",
"msg": "audit_event",
"logger": "audit",
"audit_id": "b3d4f7a2-8c11-4f6e-9b0a-1c2d3e4f5a6b",
"type": "aigw.v2.interaction.completed",
"logged_at": "2026-10-16T14:22:31.418Z",
"outcome": "success",
"component": "v2",
"source": { "type": "local", "value": "ai-gateway-v2-6d8f9c7b5-x2k4q" },
"subjects": { "user": "00u1a2b3c4d5e6f7g8h9", "department": "engineering" },
"target": {
"model": "claude-sonnet-4-6",
"provider": "anthropic",
"route": "/v1/messages"
},
"metadata": {
"extra": {
"schema_version": 1,
"request_id": "c0ffee01-2a3b-4c5d-6e7f-8091a2b3c4d5",
"interaction_id": "7e1d5390-41e5-4f71-b3c2-9a0815793c4a",
"trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
"span_id": "00f067aa0ba902b7",
"http_method": "POST",
"http_status": 200,
"dialect": "anthropic-messages",
"streaming": true,
"duration_ms": 8421,
"terminal_stage": "delivery",
"reason_code": "delivered",
"usage_present": true,
"input_tokens": 1834,
"output_tokens": 412
}
}
}
Subjects appear only after authentication succeeds. user is the caller's
canonical subject, and virtual_key_id identifies the virtual key on
virtual-key requests. Directory group labels named in directory.groupLabels,
such as department, are added alongside them. A rejected authentication event
carries no subjects, except an auth_identity_unresolved refusal, which names
the verified caller in user, or in the service account subjects described
next.
A service account's
events name its workload instead of a user. service_account is the verified
subject, in the form system:serviceaccount:<NAMESPACE>:<NAME>,
service_account_authority identifies the trusted issuer that verified it, and
service_account_uid is the ServiceAccount's UID, which tells a recreated
ServiceAccount apart from the one it replaced.
Targets carry model, provider, and route, each only once the gateway
has resolved it. provider is the provider kind from the gateway
configuration: openai, openrouter, anthropic, bedrock-anthropic,
bedrock-openai, gemini-aistudio, or vertex. route is the endpoint path
template.
Source is the gateway pod that wrote the event. Events carry no client IP address or user agent.
Common extras are schema_version, request_id, http_status, and, when
tracing is on, trace_id and span_id for OpenTelemetry correlation. The
gateway also returns request_id to the client in the X-Request-ID response
header. Authentication events add auth_method: jwt, virtual_key,
workload, or unknown, and reason_code, which says why authentication
failed, such as auth_identity_unresolved.
Interaction events add interaction_id, http_method, dialect, streaming,
duration_ms, terminal_stage, reason_code, and usage_present, plus
input_tokens and output_tokens when the provider reported usage.
terminal_stage is the stage the interaction ended in: validation, routing,
governance, provider, metering, or delivery. reason_code says what
ended it, such as delivered or governance_denied. When a governance hook
denies or fails a request, hook names the hook entry.
Required governance hooks can report annotations under hook_annotations, keyed
by hook entry name. The budgets entries report the charged subject and amount,
and the scanning entry reports the entity types it acted on, such as
detection_types for a block and redact_types for a redaction.
Separate audit records from application logs
Each event is one line of JSON on the pod's standard output. Every audit record
carries "logger":"audit" and "msg":"audit_event", so shippers should filter
on that pair to separate audit records from ordinary application logs on the
same stream.
Ship them
Each example below filters on those same two keys. Adapt hostnames, credentials,
and <NAMESPACE> for your environment.
Fluent Bit to Splunk
[INPUT]
Name tail
Path /var/log/containers/*<NAMESPACE>*.log
multiline.parser docker, cri
Tag kube.audit.*
Refresh_Interval 5
[FILTER]
Name grep
Match kube.audit.*
Regex log "logger":"audit"
Regex log "msg":"audit_event"
[FILTER]
Name parser
Match kube.audit.*
Key_Name log
Parser json
Reserve_Data On
[OUTPUT]
Name splunk
Match kube.audit.*
Host <SPLUNK_HEC_HOST>
Port 8088
Splunk_Token ${SPLUNK_HEC_TOKEN}
TLS On
event_source aigw-audit
event_index aigw_audit
A record passes the grep filter only when both Regex rules match, so the
filter works whatever order the two keys appear in.
Fluent Bit to Elastic
Keep the input and both filters from the Splunk example above, which tag records
kube.audit.*, and swap the output. The Match value has to be that same tag,
or the output receives nothing:
[OUTPUT]
Name es
Match kube.audit.*
Host <ELASTICSEARCH_HOST>
Port 9200
HTTP_User ${ES_USER}
HTTP_Passwd ${ES_PASSWORD}
tls On
Logstash_Format On
Logstash_Prefix aigw-audit
Replace_Dots On
Vector to Kafka
sources:
kubernetes_logs:
type: kubernetes_logs
extra_namespace_label_selector: 'kubernetes.io/metadata.name=<NAMESPACE>'
transforms:
parse_json:
type: remap
inputs: [kubernetes_logs]
source: |
. = parse_json!(.message)
if .logger != "audit" || .msg != "audit_event" { abort }
sinks:
kafka:
type: kafka
inputs: [parse_json]
bootstrap_servers: '<KAFKA_BROKERS>'
topic: aigw-audit
encoding:
codec: json
compression: snappy
Promtail to Loki
Promtail's match stage selects on labels, so this pipeline promotes logger
and msg to labels, drops every line that isn't an audit record, then removes
those two labels.
scrape_configs:
- job_name: aigw-audit
kubernetes_sd_configs:
- role: pod
relabel_configs:
- source_labels: [__meta_kubernetes_namespace]
regex: <NAMESPACE>
action: keep
- source_labels:
- __meta_kubernetes_pod_uid
- __meta_kubernetes_pod_container_name
separator: /
target_label: __path__
replacement: /var/log/pods/*$1/*.log
pipeline_stages:
- cri: {}
- json:
expressions:
logger: logger
msg: msg
type: type
outcome: outcome
component: component
- labels:
logger:
msg:
- match:
selector: '{logger!="audit"}'
action: drop
- match:
selector: '{msg!="audit_event"}'
action: drop
- labeldrop:
- logger
- msg
- labels:
type:
outcome:
component:
On nodes that use the Docker runtime, replace cri: {} with docker: {}.
Promtail has reached end of life. For new installs, use Grafana Alloy, which
converts this configuration with alloy convert --source-format=promtail.
CloudWatch and rsyslog
The CloudWatch agent cannot filter on a JSON key natively. Either ingest everything and apply a log-group metric filter:
{ $.logger = "audit" && $.msg = "audit_event" }
or run Fluent Bit alongside it with the cloudwatch_logs output, using the
filters above so only audit lines reach the dedicated log group.
For rsyslog, match both markers before forwarding:
if $msg contains '"logger":"audit"' and $msg contains '"msg":"audit_event"' then {
action(type="omfwd"
target="<SIEM_HOST>"
port="514"
protocol="tcp"
template="RSYSLOG_SyslogProtocol23Format")
stop
}
Retention, tamper evidence, and GDPR
Configure retention, tamper evidence, and privacy controls in your ingest and storage systems. Use lifecycle policies and write-once storage where your compliance requirements call for them.
Apply field-level redaction before events reach long-term storage. The gateway does not retain or replay emitted audit events.
Log the text behind a denial
Audit events record that a governance hook denied a request, but not the text
that tripped it. To see that text when you review a PII block or a prompt
injection verdict, turn on failureLogs.includePrompt:
ai-gateway-v2:
failureLogs:
includePrompt: true
The logged text is raw user content, including the PII the scanner detected. Restrict access to the gateway's standard error, including Kubernetes pod log access, and set retention in your log pipeline to what your data handling policy allows for that content.
The chart won't render failureLogs.includePrompt: true while
failureLogs.includeMessage is false, because that setting asks for
metadata-only records.
The PII scanner and the prompt injection guardrail then append the text behind
each denial to the cause they return, after ; prompt: . The gateway writes
that cause to an operational failure record on standard error, separate from
audit events. The scanner appends each message that carried a blocked entity,
prefixed with its path in the request body. The guardrail appends the text it
screened. Each appends at most 15 KiB, and the gateway logs up to 16 KiB of a
cause instead of the usual 1 KiB. The adapter pods roll out when the setting
changes.
A refused request leaves one ai-gateway-v2: request failed record. A request
the guardrail flags in Monitor mode is still forwarded and leaves one
ai-gateway-v2: governance would deny record. Both records carry:
request_idandinteraction_id, which match the audit event. The client also receivesrequest_idin theX-Request-IDresponse header.subject, the caller's canonical subject. A refused request's record also carriesemail, looked up in the directory.modelandprovider, once the gateway has resolved them.hook, the hook entry that denied the request.cause, the hook's reason followed by the request text.hook_annotations, such as the entity types the scanner detected or the guardrail categories that fired.
Only denials carry text. A request the scanner redacts, or matches under
log-only or mode: shadow, is forwarded and leaves no failure record. A
redaction that fails refuses the request, and the scanner appends the message it
couldn't redact.
To turn the text on for one screen only, set scanning.denyPromptInMessage or
guardrails.denyPromptInMessage. Each wins over failureLogs.includePrompt in
either direction, so guardrails.denyPromptInMessage: false keeps the
guardrail's text out of the log while the scanner's still goes in.
The gateway drops a cause that looks like a credential and sets
cause_unavailable: true instead.
The shipper examples above forward audit records only, so keep their filters as they are. Read failure records from the gateway pod logs:
kubectl logs -n <NAMESPACE> deploy/<RELEASE_NAME>-ai-gateway-v2 \
| grep -E 'request failed|governance would deny'
To keep them longer, collect the gateway's standard error through a separate pipeline, with access controls and retention suited to raw user content.
Related logs
The Kubernetes API audit log covers control-plane activity such as kubectl
calls and Helm upgrades. Configure it through the API server audit policy.
Operational failure records go to the gateway's standard error, separate
from audit events. A record can name the caller by directory email and, by
default, includes a bounded parsed error message to make failures easier to
diagnose. failureLogs.level sets their severity: warn (the default),
error, or debug. When log retention or access can't accommodate personal
data, set failureLogs.includeMessage to false to drop the message. To add
the text behind a governance denial, see
Log the text behind a denial.
Conversation content reaches no audit event. Screening sends prompt text to
Presidio and AWS Bedrock for analysis. Snapshot capture, configured with the
top-level capture value, records full request and response bodies for testing.
It's unset by default; leave it unset in production.
Next steps
- Detect PCI and PII in prompts to configure the
scanning hook whose decisions appear in
hook_annotations.scanning. - Screen prompts for injection, whose enforced verdicts also end interactions here.
- Collect gateway telemetry to export traces that join these
records on
trace_id.