Vault Leaks Client Token and Token Accessor in Audit Devices
Research is free — Hunters explains how the bug works, the root-cause code pattern, how the fix addresses it, and how to test whether a target is affected, in chat. Investigate & write exploit is a paid run — the engine reads the advisory and fix commits, then builds and validates a working proof-of-concept exploit with reproduction steps.
Affected versions
1.17.3 → fixed in 1.17.5
Details
Vault Community Edition and Vault Enterprise experienced a regression where functionality that HMAC’d sensitive headers in the configured audit device, specifically client tokens and token accessors, was removed. This resulted in the plaintext values of client tokens and token accessors being stored in the audit log. This vulnerability, CVE-2024-8365, was fixed in Vault Community Edition and Vault Enterprise 1.17.5 and Vault Enterprise 1.16.9.
The fix
Release delta 1.17.3 → 1.17.5 (contains the fix)
website/content/docs/concepts/lease-explosions.mdx+0 −375
@@ -1,379 +0,0 @@-layout: docs-page_title: 'Lease Explosions'-description: >--Learn about lease explosions and how you can prevent them.-# Lease Explosions-As your Vault environment scales to meet deployment needs, it is important to avoid over-subscription. A lease explosion can occur when operators reach over-subscription and clients create leases much faster than Vault is set to revoke them. If this continues unchecked, the active node can run out of memory. Once a lease explosion occurs, mitigation is time consuming and resource intensive.-This document shows you how to prevent lease explosions, mitigate when a lease explosion occurs, and clean up your environment after a lease explosion.-Applications and users can overwhelm system resources through consistent and high-volume API requests, resulting in denial-of-service issues in some Vault nodes or even the entire Vault cluster. Review [Vault resource quotas](/vault/docs/concepts/resource-quotas) to learn more about enabling rate-limit quotas and lease-count quotas to protect against requests which could trigger lease explosions.-These are common observations and behaviors operators experience as their Vault deployment matures:-- TTL values for dynamic secret leases or authentication tokens could be too high, resulting in unused leases consuming storage space while waiting to expire.-- Rapid lease count growth disproportionate to the number of clients is a sign of misconfiguration or potential anti-patterns in client usage.-- Lease revocation is failing. This can be caused by failures in an external service in the case of dynamic secrets.-- Valid credentials which have already been leased are not being reused when possible. e.g. a badly behaving app requests new credentials from Vault every time it starts instead of caching ones it previously requested and using them again. This encourages a build up of leases associated with otherwise unused credentials.-- The Vault server is not processing lease revocations as quickly as they're expiring. Usually, this is due to insufficient IOPS for the storage backend.-You can approach lease explosions in three phases:-- Preventing lease explosions-- Mitigating lease explosions-- Cleaning up after lease explosions-## Preventing lease explosions-Prevention is the best tool against lease explosion. The following are three important areas you can focus on to prevent lease explosion in your Vault environment.-Although no technical maximum exists, high lease counts can cause degradation in system performance. We recommend short default time-to-live (TTL) values on tokens and leases to avoid a large backlog of unexpired leases or many simultaneous expirations. Review [Vault lease limits](/vault/docs/internals/limits#lease-limits) to learn more.-### Client best practices-Ensure clients using Vault adhere to best practices for their authentication and secret retrieval, and do not make excessive dynamic secrets requests or service token authentications. Review [Lease Concepts](/vault/docs/concepts/lease) and [Auth Concepts](/vault/docs/concepts/auth) to learn more.-You should avoid these client behavior anti-patterns:-Long TTLs configured, leading to a slow build over-subscription.-Acute aberrant client behavior leading to rapid over-subscription.-A combination of both.-#### AppRole-As Vault matures in your environment, it's important to review and ensure client behavior best practices around machine-based authentication as it can have more impact on lease explosion than human-based authentication typically does.-- [Recommended pattern for Vault AppRole use](/vault/tutorials/recommended-patterns/pattern-approle)-- [How and why to use AppRole correctly in HashiCorp Vault](https://www.hashicorp.com/blog/how-and-why-to-use-approle-correctly-in-hashicorp-vault)-### Monitoring key metrics-Proactive monitoring is key to identifying behavior and usage patterns before they become problematic. Review the following resources for more details:-- [Vault key metrics](/well-architected-framework/reliability/reliability-vault-monitoring-key-metrics)-- [Vault anti-patterns poor metrics](/well-architected-framework/operational-excellence/security-vault-anti-patterns#poor-metrics-or-no-telemetry-data)-### Implementation guardrails-You can choose the appropriate token type for your use case, and use resource quotas as guardrails against lease explosion in your implementation.-#### TTLs-| TTL type | Notes |-| -------- |------ |-| [System-wide maximum TTL](/vault/docs/configuration#default_lease_ttl) and [system-wide default TTL](/vault/docs/configuration#max_lease_ttl) | TTL values which you specify in the Vault server configuration file; they are the last used values by Vault in terms of precedence after mount TTLs and high granularity TTLs |-| [Mount maximum TTL](/vault/api-docs/system/mounts#default_lease_ttl-1) and [mount default TTL](/vault/api-docs/system/mounts#max_lease_ttl-1) | TTL values specified on a per mount instance of auth method or secrets engine. In terms of precedence, these TTL values override system-wide TTLs, but are overridden by highly granular TTLs. |-| Highly granular TTLs, for example: [Database secrets engine role default TTL](/vault/api-docs/secret/databases#default_ttl) and [Database secrets engine role maximum TTL](/vault/api-docs/secret/databases#max_ttl) | These TTLs are specified on a role, group, or user level, and their values override both mount and system-wide TTL values. |-More details are available in the [Token Time-To-Live, periodic tokens, and explicit max TTLs](/vault/docs/concepts/tokens#token-time-to-live-periodic-tokens-and-explicit-max-ttls) and [Lease limits](/vault/docs/internals/limits#lease-limits) documentation.-You should also review the details in the Vault anti-patterns guide: [not adjusting the default lease time](/well-architected-framework/operational-excellence/security-vault-anti-patterns#not-adjusting-the-default-lease-time) for a clear explanation of the issue and solution.-The following are examples for setting default and maximum TTL values using the Vault API and CLI, which you can reference when setting values for your implementation.-<Note>-Adjusting TTL values is not a retroactive operation, and affects just those leases or tokens issued after you make the changes.-</Note>-Update the default TTL to 8 hours and maximum TTL to 12 hours on a username and password auth method user named "alice". The value of `$VAULT_TOKEN` should be that of a token with capabilities to perform the operations.-<Tabs>-<Tab heading="API" group="api">-```shell-session-$ curl \---header "X-Vault-Token: $VAULT_TOKEN" \---request POST \---data '{"token_ttl":"8h","token_max_ttl":"12h"}' \-$VAULT_ADDR/v1/auth/userpass/users/alice-```-This command is not expected to produce output, but you can read the user to confirm the settings.-```shell-session-$ curl \---header "X-Vault-Token: $VAULT_TOKEN" \---request GET \---silent \-$VAULT_ADDR/v1/auth/userpass/users/alice \-| jq-```-Example output:-<CodeBlockConfig hideClipboard>-```json-{-"request_id": "4cfc0293-a3f3-9b3b-b668-82aea63ced91",-"lease_id": "",-"renewable": false,-"lease_duration": 0,-"data": {-"token_bound_cidrs": [],-"token_explicit_max_ttl": 0,-"token_max_ttl": 43200,-"token_no_default_policy": false,-"token_num_uses": 0,-"token_period": 0,-"token_policies": [],-"token_ttl": 28800,-"token_type": "default"-},-"wrap_info": null,-"warnings": null,-"auth": null-}-```-</CodeBlockConfig>-When Alice authenticates with Vault and gets a token, its default TTL value is set to 28800 seconds (8 hours) and the maximum TTL value is 43200 seconds (12 hours).-</Tab>-<Tab heading="CLI" group="cli">-```shell-session-$ VAULT_TOKEN=$VAULT_TOKEN vault write /auth/userpass/users/alice \-token_ttl="8h" token_max_ttl="12h"-```-Example output:-<CodeBlockConfig hideClipboard>-```plaintext-Success! Data written to: auth/userpass/users/alice-```-</CodeBlockConfig>-You can read the user to confirm the settings.-```shell-session-$ VAULT_TOKEN=$VAULT_TOKEN vault read /auth/userpass/users/alice-```-Example output:-<CodeBlockConfig hideClipboard>-```plaintext-Key Value-token_bound_cidrs []-token_explicit_max_ttl 0s-token_max_ttl 12h-token_no_default_policy false-token_num_uses 0-token_period 0s-token_policies []-token_ttl 8h-token_type default-```-</CodeBlockConfig>-When Alice next authenticates with Vault and gets a token, its default TTL value is set to 8 hours and the maximum TTL value is 12 hours.-</Tab>-</Tabs>-#### Resource Quotas-You can use quotas to control Vault resource usage in the form of API rate limiting quotas and [lease count quotas](/vault/tutorials/operations/resource-quotas#lease-count-quotas). For the purposes of this overview, lease count quotas are most relevant as you can cap the maximum number of leases generated on a per-mount basis.-Use this feature for use cases where a hard limit to the number of leases makes sense. Also, be sure to [monitor Vault audit device logs](/vault/tutorials/monitoring/monitor-telemetry-audit-splunk) where Vault emits messages about failures related to exceeding the quota.-The following examples demonstrate creating a lease count quota on an instance of the Approle auth method, for the role named "webapp" to restrict leases to no more than 100. The value of `$VAULT_TOKEN` should be that of a token capable of performing the operations.-<Tabs>-<Tab heading="API" group="api">-1. Create a payload file containing the lease quota parameters.-```shell-session-$ cat > payload.json << EOF-{-"path": "auth/approle",-"role": "webapp",-"max_leases": 100-}-EOF-```-1. Write the webapp-tokens lease count quota.-```shell-session-$ curl \---request POST \---header "X-Vault-Token: $VAULT_TOKEN" \---data @payload.json \-$VAULT_ADDR/v1/sys/quotas/lease-count/webapp-tokens-```-This command is not expected to produce output, but you can read the user to confirm the settings.-1. Confirm settings.-```shell-session-$ curl \---header "X-Vault-Token: $VAULT_TOKEN" \---request GET \---silent \-$VAULT_ADDR/v1/sys/quotas/lease-count/webapp-tokens \-| jq-```-Example output:-<CodeBlockConfig hideClipboard>-```json-{-"request_id": "188e22f1-dc1a-251a-a0a1-005e256fe70f",-"lease_id": "",-"renewable": false,-"lease_duration": 0,-"data": {-"counter": 0,-"inheritable": true,-"max_leases": 100,-"name": "webapp-tokens",-"path": "auth/approle/",-"role": "webapp",-"type": "lease-count"-},-"wrap_info": null,-"warnings": null,-"auth": null-}-```-</CodeBlockConfig>-</Tab>-<Tab heading="CLI" group="cli">-Write the webapp-tokens lease count quota.-```shell-session-$ vault write sys/quotas/lease-count/webapp-tokens \-max_leases=100 \-path="auth/approle" \-role="webapp"-```-Example output:-<CodeBlockConfig hideClipboard>-```plaintext-Success! Data written to: sys/quotas/lease-count/webapp-tokens-```-</CodeBlockConfig>-Confirm the setting.-```shell-session-$ vault read sys/quotas/lease-count/webapp-tokens-```-Example output:-<CodeBlockConfig hideClipboard>-```plaintext-Key Value-counter 0-inheritable true-max_leases 100-name webapp-tokens-path auth/approle/-role webapp-type lease-count-```-</CodeBlockConfig>-</Tab>-</Tabs>-The limit is set to 100 leases for the AppRole auth method role named webapp.-<Note>-Enabling the rate limit audit logging may have an impact on the Vault performance if the volume of rejected requests is large.-</Note>-Review these resources for a deeper dive into controlling Vault resources:-- [Vault resource quotas](/vault/docs/concepts/resource-quotas)-- [Vault Enterprise lease count quotas](/vault/docs/enterprise/lease-count-quotas)-- [Query audit device logs](/vault/tutorials/monitoring/query-audit-device-logs)-#### Token type-In some use cases, batch tokens can be a better fit than service tokens with respect to lease explosion. Review the following resources for help deciding when to use batch tokens and when to use service tokens:-- [Vault service tokens vs batch tokens](/vault/tutorials/tokens/batch-tokens#service-tokens-vs-batch-tokens)-- [Service vs batch token lease handling](/vault/docs/concepts/tokens#service-vs-batch-token-lease-handling)-## Mitigating lease explosions-Ultimately, the number of leases a system can handle is unique to the Vault deployment and environment.-### Increase resources-Increasing available resources in your Vault cluster can help mitigate lease explosion and allow for cluster recovery. Review [hardware sizing](/well-architected-framework/zero-trust-security/raft-reference-architecture#hardware-sizing-for-vault-servers), and focus on increasing available RAM.-#### Within Vault-Use the information from the Implementation guardrails section to adjust TTL values from the default values according to your use case needs.-#### External to Vault-You can use firewalls or load balancers to limit API calls to Vault from aberrant clients.)-[Knowledge base article around load balancing](https://support.hashicorp.com/hc/en-us/articles/14496042865427-Vault-Global-Load-Balancing-Patterns)-[Vault & load balancing](/vault/tutorials/day-one-raft/raft-reference-architecture#load-balancer-recommendations)-## Cleaning up environment after lease explosions-Once the acute event subsides, the Vault active node will continue to purge leases. Sometimes, the explosion is so great, you will need to manually intervene to revoke [leases](/vault/api-docs/system/leases). If you are running a version of Vault prior to 1.13.0, this lease revocation can cause further performance degradation.-Revoking or forcefully revoking leases is potentially a dangerous operation. You should ensure that you have recent valid snapshots of the cluster. Users of Vault versions prior to 1.13.0 on integrated storage must also perform freelist compaction. Vault Enterprise customers should consider proactively contacting the [Customer Support team](https://support.hashicorp.com) for help with this process.-## Additional resources-Proactive monitoring and periodic usage analysis are some of the best practices for Vault operators. Review the following resources for more details.-- [Vault key metrics for common health checks](/well-architected-framework/reliability/reliability-vault-monitoring-key-metrics)-- [Troubleshoot irrevocable leases](/vault/tutorials/monitoring/troubleshoot-irrevocable-leases)-- [Troubleshooting Vault](/vault/tutorials/monitoring/troubleshooting-vault)
audit/entry_formatter.go+52 −50
@@ -15,7 +15,7 @@ import ("github.com/hashicorp/eventlogger""github.com/hashicorp/go-hclog""github.com/hashicorp/go-multierror"-"github.com/hashicorp/vault/helper/namespace"+nshelper "github.com/hashicorp/vault/helper/namespace""github.com/hashicorp/vault/sdk/helper/jsonutil""github.com/hashicorp/vault/sdk/helper/salt""github.com/hashicorp/vault/sdk/logical"@@ -77,7 +77,7 @@ func (*entryFormatter) Type() eventlogger.NodeType {}// Process will attempt to parse the incoming event data into a corresponding-// audit Request/Response which is serialized to JSON/JSONx and stored within the event.+// audit request/response which is serialized to JSON/JSONx and stored within the event.func (f *entryFormatter) Process(ctx context.Context, e *eventlogger.Event) (_ *eventlogger.Event, retErr error) {// Return early if the context was cancelled, eventlogger will not carry on// asking nodes to process, so any sink node in the pipeline won't be called.@@ -93,7 +93,7 @@ func (f *entryFormatter) Process(ctx context.Context, e *eventlogger.Event) (_ *return nil, fmt.Errorf("event is nil: %w", ErrInvalidParameter)}a, ok := e.Payload.(*AuditEvent)+a, ok := e.Payload.(*Event)if !ok {return nil, fmt.Errorf("cannot parse event payload: %w", ErrInvalidParameter)}@@ -137,7 +137,7 @@ func (f *entryFormatter) Process(ctx context.Context, e *eventlogger.Event) (_ *return nil, fmt.Errorf("unable to format %s: %w", a.Subtype, err)}if f.config.requiredFormat == JSONxFormat {+if f.config.requiredFormat == jsonxFormat {var err errorresult, err = jsonx.EncodeJSONBytes(result)if err != nil {@@ -243,45 +243,46 @@ func clone[V any](s V) (V, error) {// newAuth takes a logical.Auth and the number of remaining client token uses// (which should be supplied from the logical.Request's client token), and creates-// an audit Auth.+// an audit auth.// tokenRemainingUses should be the client token remaining uses to include in auth.// This usually can be found in logical.Request.ClientTokenRemainingUses.// NOTE: supplying a nil value for auth will result in a nil return value and// (nil) error. The caller should check the return value before attempting to use it.-func newAuth(auth *logical.Auth, tokenRemainingUses int) (*Auth, error) {-if auth == nil {+// ignore-nil-nil-function-check.+func newAuth(input *logical.Auth, tokenRemainingUses int) (*auth, error) {+if input == nil {return nil, nil}extNSPolicies, err := clone(auth.ExternalNamespacePolicies)+extNSPolicies, err := clone(input.ExternalNamespacePolicies)if err != nil {return nil, fmt.Errorf("unable to clone logical auth: external namespace policies: %w", err)}identityPolicies, err := clone(auth.IdentityPolicies)+identityPolicies, err := clone(input.IdentityPolicies)if err != nil {return nil, fmt.Errorf("unable to clone logical auth: identity policies: %w", err)}metadata, err := clone(auth.Metadata)+metadata, err := clone(input.Metadata)if err != nil {return nil, fmt.Errorf("unable to clone logical auth: metadata: %w", err)}policies, err := clone(auth.Policies)+policies, err := clone(input.Policies)if err != nil {return nil, fmt.Errorf("unable to clone logical auth: policies: %w", err)}var policyResults *PolicyResults-if auth.PolicyResults != nil {-policyResults = &PolicyResults{-Allowed: auth.PolicyResults.Allowed,-GrantingPolicies: make([]PolicyInfo, len(auth.PolicyResults.GrantingPolicies)),+var polRes *policyResults+if input.PolicyResults != nil {+polRes = &policyResults{+Allowed: input.PolicyResults.Allowed,+GrantingPolicies: make([]policyInfo, len(input.PolicyResults.GrantingPolicies)),}for _, p := range auth.PolicyResults.GrantingPolicies {-policyResults.GrantingPolicies = append(policyResults.GrantingPolicies, PolicyInfo{+for _, p := range input.PolicyResults.GrantingPolicies {+polRes.GrantingPolicies = append(polRes.GrantingPolicies, policyInfo{Name: p.Name,NamespaceId: p.NamespaceId,NamespacePath: p.NamespacePath,@@ -290,40 +291,40 @@ func newAuth(auth *logical.Auth, tokenRemainingUses int) (*Auth, error) {}}tokenPolicies, err := clone(auth.TokenPolicies)+tokenPolicies, err := clone(input.TokenPolicies)if err != nil {return nil, fmt.Errorf("unable to clone logical auth: token policies: %w", err)}var tokenIssueTime string-if !auth.IssueTime.IsZero() {-tokenIssueTime = auth.IssueTime.Format(time.RFC3339)+if !input.IssueTime.IsZero() {+tokenIssueTime = input.IssueTime.Format(time.RFC3339)}return &Auth{-Accessor: auth.Accessor,-ClientToken: auth.ClientToken,-DisplayName: auth.DisplayName,-EntityCreated: auth.EntityCreated,-EntityID: auth.EntityID,+return &auth{+Accessor: input.Accessor,+ClientToken: input.ClientToken,+DisplayName: input.DisplayName,+EntityCreated: input.EntityCreated,+EntityID: input.EntityID,ExternalNamespacePolicies: extNSPolicies,IdentityPolicies: identityPolicies,Metadata: metadata,-NoDefaultPolicy: auth.NoDefaultPolicy,-NumUses: auth.NumUses,+NoDefaultPolicy: input.NoDefaultPolicy,+NumUses: input.NumUses,Policies: policies,-PolicyResults: policyResults,+PolicyResults: polRes,RemainingUses: tokenRemainingUses,TokenPolicies: tokenPolicies,TokenIssueTime: tokenIssueTime,-TokenTTL: int64(auth.TTL.Seconds()),-TokenType: auth.TokenType.String(),+TokenTTL: int64(input.TTL.Seconds()),+TokenType: input.TokenType.String(),}, nil}// newRequest takes a logical.Request and namespace.Namespace, transforms and-// aggregates them into an audit Request.-func newRequest(req *logical.Request, ns *namespace.Namespace) (*Request, error) {+// aggregates them into an audit request.+func newRequest(req *logical.Request, ns *nshelper.Namespace) (*request, error) {if req == nil {return nil, fmt.Errorf("request cannot be nil")}@@ -351,7 +352,7 @@ func newRequest(req *logical.Request, ns *namespace.Namespace) (*Request, error)wrapTTL = int(req.WrapInfo.TTL / time.Second)}return &Request{+return &request{ClientCertificateSerialNumber: clientCertSerial,ClientID: req.ClientID,ClientToken: req.ClientToken,@@ -366,7 +367,7 @@ func newRequest(req *logical.Request, ns *namespace.Namespace) (*Request, error)MountRunningSha256: req.MountRunningSha256(),MountRunningVersion: req.MountRunningVersion(),MountType: req.MountType,-Namespace: &Namespace{+Namespace: &namespace{ID: ns.ID,Path: ns.Path,},@@ -382,11 +383,12 @@ func newRequest(req *logical.Request, ns *namespace.Namespace) (*Request, error)}// newResponse takes a logical.Response and logical.Request, transforms and-// aggregates them into an audit Response.+// aggregates them into an audit response.// isElisionRequired is used to indicate that response 'Data' should be elided.// NOTE: supplying a nil value for response will result in a nil return value and// (nil) error. The caller should check the return value before attempting to use it.-func newResponse(resp *logical.Response, req *logical.Request, isElisionRequired bool) (*Response, error) {+// ignore-nil-nil-function-check.+func newResponse(resp *logical.Response, req *logical.Request, isElisionRequired bool) (*response, error) {if resp == nil {return nil, nil}@@ -447,12 +449,12 @@ func newResponse(resp *logical.Response, req *logical.Request, isElisionRequiredreturn nil, fmt.Errorf("unable to clone logical response: headers: %w", err)}var secret *Secret+var s *secretif resp.Secret != nil {-secret = &Secret{LeaseID: resp.Secret.LeaseID}+s = &secret{LeaseID: resp.Secret.LeaseID}}var wrapInfo *ResponseWrapInfo+var wrapInfo *responseWrapInfoif resp.WrapInfo != nil {token := resp.WrapInfo.Tokenif jwtToken := parseVaultTokenFromJWT(token); jwtToken != nil {@@ -460,7 +462,7 @@ func newResponse(resp *logical.Response, req *logical.Request, isElisionRequired}ttl := int(resp.WrapInfo.TTL / time.Second)-wrapInfo = &ResponseWrapInfo{+wrapInfo = &responseWrapInfo{TTL: ttl,Token: token,Accessor: resp.WrapInfo.Accessor,@@ -475,7 +477,7 @@ func newResponse(resp *logical.Response, req *logical.Request, isElisionRequiredreturn nil, fmt.Errorf("unable to clone logical response: warnings: %w", err)}return &Response{+return &response{Auth: auth,Data: data,Headers: headers,@@ -487,15 +489,15 @@ func newResponse(resp *logical.Response, req *logical.Request, isElisionRequiredMountRunningVersion: req.MountRunningVersion(),MountType: req.MountType,Redirect: resp.Redirect,-Secret: secret,+Secret: s,WrapInfo: wrapInfo,Warnings: warnings,}, nil}// createEntry takes the AuditEvent and builds an audit Entry.-// The Entry will be HMAC'd and elided where required.-func (f *entryFormatter) createEntry(ctx context.Context, a *AuditEvent) (*Entry, error) {+// createEntry takes the AuditEvent and builds an audit entry.+// The entry will be HMAC'd and elided where required.+func (f *entryFormatter) createEntry(ctx context.Context, a *Event) (*entry, error) {select {case <-ctx.Done():return nil, ctx.Err()@@ -510,7 +512,7 @@ func (f *entryFormatter) createEntry(ctx context.Context, a *AuditEvent) (*Entryreturn nil, fmt.Errorf("unable to parse request from '%s' audit event: request cannot be nil", a.Subtype)}ns, err := namespace.FromContext(ctx)+ns, err := nshelper.FromContext(ctx)if err != nil {return nil, fmt.Errorf("unable to retrieve namespace from context: %w", err)}@@ -525,7 +527,7 @@ func (f *entryFormatter) createEntry(ctx context.Context, a *AuditEvent) (*Entryreturn nil, fmt.Errorf("cannot convert request: %w", err)}var resp *Response+var resp *responseif a.Subtype == ResponseType {shouldElide := f.config.elideListResponses && req.Operation == logical.ListOperationresp, err = newResponse(data.Response, data.Request, shouldElide)@@ -544,7 +546,7 @@ func (f *entryFormatter) createEntry(ctx context.Context, a *AuditEvent) (*EntryentryType = a.Subtype.String()}entry := &Entry{+entry := &entry{Auth: auth,Error: outerErr,Forwarded: false,
website/content/docs/configuration/prevent-lease-explosions.mdx+105 −0
@@ -0,0 +1,105 @@+---+layout: docs+page_title: Prevent lease explosions+description: >-+Learn how to prevent lease explosions in Vault.+---+# Prevent lease explosions+As your Vault environment scales to meet deployment needs, you run the risk of+lease explosions. Lease explosions can occur when a Vault cluster is+over-subscribed and clients overwhelm system resources with consistent,+high-volume API requests+Unchecked lease explosions create a memory drain on the active node, which can+cascade to other nodes and result in denial-of-service issues for the entire+cluster.+## Look for early warning signs+Cleaning up after a lease explosion is time consuming and resource intensive, so+we strongly recommend monitoring your Vault instance for signals that your+Vault deployment has matured and requires tuning:+Issue | Possible cause+-------------------------------------------------------------------------------- | --------------+Unused leases consume storage space for extended periods while waiting to expire | The TTL values for dynamic secret leases or authentication tokens may be too high+Lease revocation fails frequently | Failures in an external service (e.g., for dynamic secrets)+Build up of leases associated with unused credentials | Clients are not reusing valid, existing leases+Lease revocation is slow | Insufficient IOPS for the storage backend+Rapid lease count growth disproportionate to the number of clients | Misconfiguration or anti-patterns in client usage+## Enforce client best practices+High lease counts can degrade system performance:+- Use the smallest default time-to-live (TTL) possible for tokens and leases to+avoid excessive unexpired lease backlogs and high-volume, simultaneous+expirations.+- Review telemetry for aberrant client behavior that might lead to rapid+over-subscription.+- Limit the number of simultaneous dynamic secret requests and service token+authentication requests.+- Ensure that machine clients adhere to [recommended AppRole patterns](/vault/tutorials/recommended-patterns/pattern-approle).+- Review [AppRole best practices](https://www.hashicorp.com/blog/how-and-why-to-use-approle-correctly-in-hashicorp-vault).+## Set reasonable TTL guardrails+Choose appropriate defaults for your situation and use resource quotas as+guardrails against lease explosion. You can set default and maximum TTLs+globally, in the mount configuration for a specific authN or secrets plugin, and+at the role-level (e.g., database credential roles).+Vault prioritizes TTL values by granularity:+- Global values act as the default.+- Plugin TTL values override global values.+- Role, group, and user level TTL values override plugin and global values.+<Note title="TTL changes are not retroactive">+Leases and tokens keep the TTL value in affect during their creation. When you+adjust TTL values, the new limits only apply to leases and tokens issued after+you deploy the changes.+</Note>+## Monitor key metrics and logs+Proactive monitoring is key to finding problematic behavior and usage patterns+before they escalate:+- Review [key Vault metrics](/well-architected-framework/reliability/reliability-vault-monitoring-key-metrics)+- Understand [metric anti-patterns](/well-architected-framework/operational-excellence/security-vault-anti-patterns#poor-metrics-or-no-telemetry-data)+- Monitor [Vault audit device logs](/vault/tutorials/monitoring/monitor-telemetry-audit-splunk) for quota-related failures.+## Control resource usage with quotas+Use API rate limiting quotas and+[lease count quotas](/vault/tutorials/operations/resource-quotas#lease-count-quotas)+to limit the number of leases generated on a per-mount basis and control+resource consumption for your Vault instance where hard limits makes sense.+## Consider batch tokens+If your environment inherently leads to a large number of lease requests,+consider using batch tokens over service tokens.+The following resources can help you decide if batch tokens are reasonable for+your situation:+- [Vault service tokens vs batch tokens](/vault/tutorials/tokens/batch-tokens#service-tokens-vs-batch-tokens)+- [Service vs batch token lease handling](/vault/docs/concepts/tokens#service-vs-batch-token-lease-handling)+## Next steps+Proactive monitoring and periodic usage analysis can help you identify potential+problems before they escalate.+- Brush up on [general Vault resource quotas](/vault/docs/concepts/resource-quotas) in general.+- Learn about [lease count quotas for Vault Enterprise](/vault/docs/enterprise/lease-count-quotas).+- Learn how to [query audit device logs](/vault/tutorials/monitoring/query-audit-device-logs).+- Review [recommended Vault lease limits](/vault/docs/internals/limits#lease-limits).+- Review [lease anti-patterns](/well-architected-framework/operational-excellence/security-vault-anti-patterns#not-adjusting-the-default-lease-time) for a clear explanation of the issue and solution.
website/content/docs/troubleshoot/generate-root-token.mdx+161 −0
@@ -0,0 +1,161 @@+---+layout: docs+page_title: Regenerate a Vault root token+description: >-+Regenerate a lost or revoked root token.+---+# Regenerate a Vault root token+Your Vault root token is a special token that gives you access to **all** Vault+operations. Best practice is to enable an appropriate authentication method for+Vault admins once the server is running and revoke the root token.+For emergency situations where your require a root token, you can use the+[`operator generate-root`](/vault/docs/commands/operator/generate-root) CLI+command and a one-time password (OTP) or Pretty Good Privacy (PGP) to generate+a new root token.+## Before you start+- **You need your Vault keys**. If you use auto-unseal, you need your+[recovery](/vault/docs/concepts/seal#recovery-key) keys, otherwise you need+your unseal keys.+- **Identify current key holders**. You must distribute the token nonce to your+unseal/recovery key holders during root token generation.+## Step 1: Create a root token nonce+1. Generate a token nonce for your new root token:+<Tabs>+<Tab heading="OTP" group="otp">+**You need the returned OTP value to decode the new root token**.+```shell-session+$ vault operator generate-root -init+A One-Time-Password has been generated for you and is shown in the OTP field.+You will need this value to decode the resulting root token, so keep it safe.+Nonce 15565c79-cc9e-5e64-b986-8506e7bd1918+Started true+Progress 0/1+Complete false+OTP 5JFQaH76Ky2TIuSt4SPvO1CGkx+OTP Length 26+```+</Tab>+<Tab heading="PGP" group="pgp">+Use the `-pgp-key` option to provide a path to your PGP public key or Keybase+username to encrypt the new root token. **You will need the returned PGP+value to decode the new root token**.+```shell-session+$ vault operator generate-root -init -pgp-key=keybase:sethvargo+Nonce e24dec5e-f1ea-2dfe-ecce-604022006976+Started true+Progress 0/5+Complete false+PGP Fingerprint e2f8e2974623ba2a0e933a59c921994f9c27e0ff+```+</Tab>+</Tabs>+1. Distribute the nonce to each of your unseal/recovery key holders.+## Step 2: Establish key quorum with the token nonce+<Highlight title="Use TTY to autocomplete the nonce">+If you use a TTY, the `operator generate-root` command prompts for your key+and automatically completes the nonce value.+</Highlight>+1. Have each unseal/recovery key holder run `operator generator-root` with their+key and the distributed nonce value:+```shell-session+$ echo ${UNSEAL_OR_RECOVERY_KEY} | vault operator generate-root -nonce=${NONCE_VALUE} -+Root generation operation nonce: f67f4da3-4ae4-68fb-4716-91da6b609c3e+Unseal Key (will be hidden):+```+1. Vault returns the new, encoded root token to the user who triggers quorum:+<Tabs>+<Tab heading="OTP" group="otp">+```shell-session+Nonce f67f4da3-4ae4-68fb-4716-91da6b609c3e+Started true+Progress 5/5+Complete true+Encoded Token IxJpyqxn3YafOGhqhvP6cQ==+```+</Tab>+<Tab heading="PGP" group="pgp">+```shell-session+Nonce e24dec5e-f1ea-2dfe-ecce-604022006976+Started true+Progress 1/1+Complete true+PGP Fingerprint e2f8e2974623ba2a0e933a59c921994f9c27e0ff+Encoded Token wcFMA0RVkFtoqzRlARAAI3Ux8kdSpfgXdF9mg...+```+</Tab>+</Tabs>+## Step 3: Decode the new root token+Decode the new root token using OTP or PGP.+<Tabs>+<Tab heading="OTP" group="otp">+Use `operator generate-root` and the OTP value from nonce generation to decode+the new root token:+```shell-session+$ vault operator generate-root \+-decode=${ENCODED_TOKEN} \+-otp=${NONCE_OTP}+hvs.XXXXXXXXXXXXXXXXXXXXXXXX+```+</Tab>+<Tab heading="PGP" group="pgp">+Use your PGP credentials and `gpg` or `keybase` to decrypt the new root token.+**`gpg`**:+```shell-session+$ echo ${ENCODED_TOKEN} | base64 --decode | gpg --decrypt+hvs.XXXXXXXXXXXXXXXXXXXXXXXX+```+**`keybase`**:+```shell-session+$ echo ${ENCODED_TOKEN} | base64 --decode | keybase pgp decrypt+hvs.XXXXXXXXXXXXXXXXXXXXXXXX+```+</Tab>+</Tabs>
audit/hashstructure.go+11 −11
@@ -25,10 +25,10 @@ func hashString(ctx context.Context, salter Salter, data string) (string, error)return salt.GetIdentifiedHMAC(data), nil}// hashAuth uses the Salter to hash the supplied Auth (modifying it).+// hashAuth uses the Salter to hash the supplied auth (modifying it).// hmacAccessor is used to indicate whether the accessor should also be HMAC'd// when present.-func hashAuth(ctx context.Context, salter Salter, auth *Auth, hmacAccessor bool) error {+func hashAuth(ctx context.Context, salter Salter, auth *auth, hmacAccessor bool) error {if auth == nil {return nil}@@ -50,14 +50,14 @@ func hashAuth(ctx context.Context, salter Salter, auth *Auth, hmacAccessor bool)return nil}// hashRequest uses the Salter to hash the supplied Request (modifying it).-// nonHMACDataKeys is used when hashing any 'Data' field within the Request which+// hashRequest uses the Salter to hash the supplied request (modifying it).+// nonHMACDataKeys is used when hashing any 'Data' field within the request which// prevents those specific keys from HMAC'd.// hmacAccessor is used to indicate whether some accessors should also be HMAC'd// when present.-// nonHMACDataKeys is used when hashing any 'Data' field within the Request which+// nonHMACDataKeys is used when hashing any 'Data' field within the request which// prevents those specific keys from HMAC'd.-func hashRequest(ctx context.Context, salter Salter, req *Request, hmacAccessor bool, nonHMACDataKeys []string) error {+func hashRequest(ctx context.Context, salter Salter, req *request, hmacAccessor bool, nonHMACDataKeys []string) error {if req == nil {return nil}@@ -102,13 +102,13 @@ func hashMap(hashFunc hashCallback, data map[string]interface{}, nonHMACDataKeysreturn hashStructure(data, hashFunc, nonHMACDataKeys)}// hashResponse uses the Salter to hash the supplied Response (modifying it).+// hashResponse uses the Salter to hash the supplied response (modifying it).// hmacAccessor is used to indicate whether some accessors should also be HMAC'd// when present.-// nonHMACDataKeys is used when hashing any 'Data' field within the Response which+// nonHMACDataKeys is used when hashing any 'Data' field within the response which// prevents those specific keys from HMAC'd.// See: /vault/docs/audit#eliding-list-response-bodies-func hashResponse(ctx context.Context, salter Salter, resp *Response, hmacAccessor bool, nonHMACDataKeys []string) error {+func hashResponse(ctx context.Context, salter Salter, resp *response, hmacAccessor bool, nonHMACDataKeys []string) error {if resp == nil {return nil}@@ -142,10 +142,10 @@ func hashResponse(ctx context.Context, salter Salter, resp *Response, hmacAccessreturn nil}// hashWrapInfo uses the supplied hashing function to hash ResponseWrapInfo (modifying it).+// hashWrapInfo uses the supplied hashing function to hash responseWrapInfo (modifying it).// hmacAccessor is used to indicate whether some accessors should also be HMAC'd// when present.-func hashWrapInfo(hashFunc hashCallback, wrapInfo *ResponseWrapInfo, hmacAccessor bool) error {+func hashWrapInfo(hashFunc hashCallback, wrapInfo *responseWrapInfo, hmacAccessor bool) error {if wrapInfo == nil {return nil}
website/content/docs/configuration/create-lease-count-quota.mdx+185 −0
@@ -0,0 +1,185 @@+---+layout: docs+page_title: Create a lease count quota+description: >-+Step-by-step instructions for creating lease count quotas for an+authentication plugin+---+# Create a lease count quota+Use lease count quotas to limit the number of leases generated on a per-mount+basis and control resource consumption for your Vault instance where hard+limits makes sense.+## Before you start+- **Confirm you have access to the root or administration namespace for your+Vault instance**. Modifying lease count quotas is a restricted activity.+## Step 1: Determine the appropriate granularity+The granularity of your lease limits can affect the performance of your Vault+cluster. In particular, if your lease limits cause the number of rejected+requests to increase dramatically, the increased audit logging may impact Vault+performance.+Review past system behavior to identify whether the quota limits should be+inheritable or limited to a specific role.+## Step 2: Apply the count quota+<Tabs>+<Tab heading="CLI" group="cli">+Use `vault write` and the `sys/quotas/lease-count/{quota-name}` mount path to+create a new lease count quota:+```shell-session+$ vault write \+sys/quotas/lease-count/<QUOTA_NAME> \+name="<QUOTA_NAME>" \+path="<PLUGIN_MOUNT_PATH>" \+role="<OPTIONAL_AUTHN_ROLE>" \+max_leases=<LEASE_LIMIT>+```+For example, to create a targeted quota limit called **webapp-tokens** on the+`webapp` role for the `approle` plugin at the default mount path:+```shell-session+$ vault write \+sys/quotas/lease-count/webapp-tokens \+name="webapp-tokens" \+path="auth/approle" \+role="webapp" \+max_leases=100+Success! Data written to: sys/quotas/lease-count/webapp-tokens+```+</Tab>+<Tab heading="API" group="api">+1. Create a payload file with your quota settings.+```json+{+"name": "<QUOTA_NAME>",+"path": "<PLUGIN_MOUNT_PATH>",+"role": "<OPTIONAL_AUTHN_ROLE>",+"max_leases": <LEASE_LIMIT>,+}+```+For example, to create a targeted quota limit called **webapp-tokens** on the+`webapp` role for the `approle` plugin at the default mount path:+```json+{+"name": "webapp-tokens",+"path": "auth/approle",+"role": "webapp",+"max_leases": 100,+}+```+1. Call the `/sys/quotas/lease-count/{quota-name}` endpoint to apply the lease+count quota. For example, to apply the `webapp-tokens` quota:+```shell-session+$ curl \+--request POST \+--header "X-Vault-Token: ${VAULT_TOKEN}" \+--data @payload.json \+${VAULT_ADDR}/v1/sys/quotas/lease-count/webapp-tokens+```+<Note title="Silent endpoint">+The `/sys/quotas/lease-count/{quota-name}` endpoint succeeds silently.+</Note>+</Tab>+</Tabs>+## Step 3: Confirm the quota settings+<Tabs>+<Tab heading="CLI" group="cli">+Use `vault read` and the `sys/quotas/lease-count/{quota-name}` mount path to+display the lease count quota details:+```shell-session+$ vault read sys/quotas/lease-count/<QUOTA_NAME>+```+For example, to read the **webapp-tokens** quota details:+```shell-session+$ vault read sys/quotas/lease-count/webapp-tokens+Key Value+--- -----+counter 0+inheritable true+max_leases 100+name webapp-tokens+path auth/approle/+role webapp+type lease-count+```+</Tab>+<Tab heading="API" group="api">+Call the `sys/quotas/lease-count/{quota-name}` endpoint to display the lease+count quota details. For example, to read the **webapp-tokens** quota details:+```shell-session+$ curl \+--header "X-Vault-Token: ${VAULT_TOKEN}" \+--request GET \+--silent \+${VAULT_ADDR}/v1/sys/quotas/lease-count/webapp-tokens | jq+{+"request_id": "188e22f1-dc1a-251a-a0a1-005e256fe70f",+"lease_id": "",+"renewable": false,+"lease_duration": 0,+"data": {+"counter": 0,+"inheritable": false,+"max_leases": 100,+"name": "webapp-tokens",+"path": "auth/approle/",+"role": "webapp",+"type": "lease-count"+},+"wrap_info": null,+"warnings": null,+"auth": null+}+```+</Tab>+</Tabs>+## Next steps+Proactive monitoring and periodic usage analysis can help you identify potential+problems before they escalate.+- Brush up on [general Vault resource quotas](/vault/docs/concepts/resource-quotas) in general.+- Learn about [lease count quotas for Vault Enterprise](/vault/docs/enterprise/lease-count-quotas).+- Learn how to [query audit device logs](/vault/tutorials/monitoring/query-audit-device-logs).+- Review [key Vault metrics for common health checks](/well-architected-framework/reliability/reliability-vault-monitoring-key-metrics).
audit/options_test.go+60 −60
@@ -10,8 +10,8 @@ import ("github.com/stretchr/testify/require")// TestOptions_WithFormat exercises WithFormat Option to ensure it performs as expected.-func TestOptions_WithFormat(t *testing.T) {+// TestOptions_withFormat exercises withFormat option to ensure it performs as expected.+func TestOptions_withFormat(t *testing.T) {t.Parallel()tests := map[string]struct {@@ -38,12 +38,12 @@ func TestOptions_WithFormat(t *testing.T) {"valid-json": {Value: "json",IsErrorExpected: false,-ExpectedValue: JSONFormat,+ExpectedValue: jsonFormat,},"valid-jsonx": {Value: "jsonx",IsErrorExpected: false,-ExpectedValue: JSONxFormat,+ExpectedValue: jsonxFormat,},}@@ -53,7 +53,7 @@ func TestOptions_WithFormat(t *testing.T) {t.Run(name, func(t *testing.T) {t.Parallel()opts := &options{}-applyOption := WithFormat(tc.Value)+applyOption := withFormat(tc.Value)err := applyOption(opts)switch {case tc.IsErrorExpected:@@ -67,8 +67,8 @@ func TestOptions_WithFormat(t *testing.T) {}}// TestOptions_WithSubtype exercises WithSubtype Option to ensure it performs as expected.-func TestOptions_WithSubtype(t *testing.T) {+// TestOptions_withSubtype exercises withSubtype option to ensure it performs as expected.+func TestOptions_withSubtype(t *testing.T) {t.Parallel()tests := map[string]struct {@@ -100,7 +100,7 @@ func TestOptions_WithSubtype(t *testing.T) {t.Run(name, func(t *testing.T) {t.Parallel()opts := &options{}-applyOption := WithSubtype(tc.Value)+applyOption := withSubtype(tc.Value)err := applyOption(opts)switch {case tc.IsErrorExpected:@@ -114,8 +114,8 @@ func TestOptions_WithSubtype(t *testing.T) {}}// TestOptions_WithNow exercises WithNow Option to ensure it performs as expected.-func TestOptions_WithNow(t *testing.T) {+// TestOptions_withNow exercises withNow option to ensure it performs as expected.+func TestOptions_withNow(t *testing.T) {t.Parallel()tests := map[string]struct {@@ -143,7 +143,7 @@ func TestOptions_WithNow(t *testing.T) {t.Parallel()opts := &options{}-applyOption := WithNow(tc.Value)+applyOption := withNow(tc.Value)err := applyOption(opts)switch {case tc.IsErrorExpected:@@ -157,8 +157,8 @@ func TestOptions_WithNow(t *testing.T) {}}// TestOptions_WithID exercises WithID Option to ensure it performs as expected.-func TestOptions_WithID(t *testing.T) {+// TestOptions_withID exercises withID option to ensure it performs as expected.+func TestOptions_withID(t *testing.T) {t.Parallel()tests := map[string]struct {@@ -190,7 +190,7 @@ func TestOptions_WithID(t *testing.T) {t.Run(name, func(t *testing.T) {t.Parallel()opts := &options{}-applyOption := WithID(tc.Value)+applyOption := withID(tc.Value)err := applyOption(opts)switch {case tc.IsErrorExpected:@@ -204,8 +204,8 @@ func TestOptions_WithID(t *testing.T) {}}// TestOptions_WithPrefix exercises WithPrefix Option to ensure it performs as expected.-func TestOptions_WithPrefix(t *testing.T) {+// TestOptions_withPrefix exercises withPrefix option to ensure it performs as expected.+func TestOptions_withPrefix(t *testing.T) {t.Parallel()tests := map[string]struct {@@ -237,7 +237,7 @@ func TestOptions_WithPrefix(t *testing.T) {t.Run(name, func(t *testing.T) {t.Parallel()opts := &options{}-applyOption := WithPrefix(tc.Value)+applyOption := withPrefix(tc.Value)err := applyOption(opts)switch {case tc.IsErrorExpected:@@ -251,8 +251,8 @@ func TestOptions_WithPrefix(t *testing.T) {}}// TestOptions_WithRaw exercises WithRaw Option to ensure it performs as expected.-func TestOptions_WithRaw(t *testing.T) {+// TestOptions_withRaw exercises withRaw option to ensure it performs as expected.+func TestOptions_withRaw(t *testing.T) {t.Parallel()tests := map[string]struct {@@ -275,7 +275,7 @@ func TestOptions_WithRaw(t *testing.T) {t.Run(name, func(t *testing.T) {t.Parallel()opts := &options{}-applyOption := WithRaw(tc.Value)+applyOption := withRaw(tc.Value)err := applyOption(opts)require.NoError(t, err)require.Equal(t, tc.ExpectedValue, opts.withRaw)@@ -283,8 +283,8 @@ func TestOptions_WithRaw(t *testing.T) {}}// TestOptions_WithElision exercises WithElision Option to ensure it performs as expected.-func TestOptions_WithElision(t *testing.T) {+// TestOptions_withElision exercises withElision option to ensure it performs as expected.+func TestOptions_withElision(t *testing.T) {t.Parallel()tests := map[string]struct {@@ -307,7 +307,7 @@ func TestOptions_WithElision(t *testing.T) {t.Run(name, func(t *testing.T) {t.Parallel()opts := &options{}-applyOption := WithElision(tc.Value)+applyOption := withElision(tc.Value)err := applyOption(opts)require.NoError(t, err)require.Equal(t, tc.ExpectedValue, opts.withElision)@@ -315,8 +315,8 @@ func TestOptions_WithElision(t *testing.T) {}}// TestOptions_WithHMACAccessor exercises WithHMACAccessor Option to ensure it performs as expected.-func TestOptions_WithHMACAccessor(t *testing.T) {+// TestOptions_withHMACAccessor exercises withHMACAccessor option to ensure it performs as expected.+func TestOptions_withHMACAccessor(t *testing.T) {t.Parallel()tests := map[string]struct {@@ -339,7 +339,7 @@ func TestOptions_WithHMACAccessor(t *testing.T) {t.Run(name, func(t *testing.T) {t.Parallel()opts := &options{}-applyOption := WithHMACAccessor(tc.Value)+applyOption := withHMACAccessor(tc.Value)err := applyOption(opts)require.NoError(t, err)require.Equal(t, tc.ExpectedValue, opts.withHMACAccessor)@@ -347,8 +347,8 @@ func TestOptions_WithHMACAccessor(t *testing.T) {}}// TestOptions_WithOmitTime exercises WithOmitTime Option to ensure it performs as expected.-func TestOptions_WithOmitTime(t *testing.T) {+// TestOptions_withOmitTime exercises withOmitTime option to ensure it performs as expected.+func TestOptions_withOmitTime(t *testing.T) {t.Parallel()tests := map[string]struct {@@ -371,7 +371,7 @@ func TestOptions_WithOmitTime(t *testing.T) {t.Run(name, func(t *testing.T) {t.Parallel()opts := &options{}-applyOption := WithOmitTime(tc.Value)+applyOption := withOmitTime(tc.Value)err := applyOption(opts)require.NoError(t, err)require.Equal(t, tc.ExpectedValue, opts.withOmitTime)@@ -389,12 +389,12 @@ func TestOptions_Default(t *testing.T) {require.False(t, opts.withNow.IsZero())}// TestOptions_Opts exercises GetOpts with various Option values.+// TestOptions_Opts exercises GetOpts with various option values.func TestOptions_Opts(t *testing.T) {t.Parallel()tests := map[string]struct {-opts []Option+opts []optionIsErrorExpected boolExpectedErrorMessage stringExpectedID string@@ -407,73 +407,73 @@ func TestOptions_Opts(t *testing.T) {opts: nil,IsErrorExpected: false,IsNowExpected: true,-ExpectedFormat: JSONFormat,+ExpectedFormat: jsonFormat,},"empty-options": {-opts: []Option{},+opts: []option{},IsErrorExpected: false,IsNowExpected: true,-ExpectedFormat: JSONFormat,+ExpectedFormat: jsonFormat,},"with-multiple-valid-id": {-opts: []Option{-WithID("qwerty"),-WithID("juan"),+opts: []option{+withID("qwerty"),+withID("juan"),},IsErrorExpected: false,ExpectedID: "juan",IsNowExpected: true,-ExpectedFormat: JSONFormat,+ExpectedFormat: jsonFormat,},"with-multiple-valid-subtype": {-opts: []Option{-WithSubtype("AuditRequest"),-WithSubtype("AuditResponse"),+opts: []option{+withSubtype("AuditRequest"),+withSubtype("AuditResponse"),},IsErrorExpected: false,ExpectedSubtype: ResponseType,IsNowExpected: true,-ExpectedFormat: JSONFormat,+ExpectedFormat: jsonFormat,},"with-multiple-valid-format": {-opts: []Option{-WithFormat("json"),-WithFormat("jsonx"),+opts: []option{+withFormat("json"),+withFormat("jsonx"),},IsErrorExpected: false,-ExpectedFormat: JSONxFormat,+ExpectedFormat: jsonxFormat,IsNowExpected: true,},"with-multiple-valid-now": {-opts: []Option{-WithNow(time.Date(2023, time.July, 4, 12, 3, 0, 0, time.Local)),-WithNow(time.Date(2023, time.July, 4, 13, 3, 0, 0, time.Local)),+opts: []option{+withNow(time.Date(2023, time.July, 4, 12, 3, 0, 0, time.Local)),+withNow(time.Date(2023, time.July, 4, 13, 3, 0, 0, time.Local)),},IsErrorExpected: false,ExpectedNow: time.Date(2023, time.July, 4, 13, 3, 0, 0, time.Local),IsNowExpected: false,-ExpectedFormat: JSONFormat,+ExpectedFormat: jsonFormat,},"with-multiple-valid-then-invalid-now": {-opts: []Option{-WithNow(time.Date(2023, time.July, 4, 12, 3, 0, 0, time.Local)),-WithNow(time.Time{}),+opts: []option{+withNow(time.Date(2023, time.July, 4, 12, 3, 0, 0, time.Local)),+withNow(time.Time{}),},IsErrorExpected: true,ExpectedErrorMessage: "cannot specify 'now' to be the zero time instant",-ExpectedFormat: JSONFormat,+ExpectedFormat: jsonFormat,},"with-multiple-valid-options": {-opts: []Option{-WithID("qwerty"),-WithSubtype("AuditRequest"),-WithFormat("json"),-WithNow(time.Date(2023, time.July, 4, 12, 3, 0, 0, time.Local)),+opts: []option{+withID("qwerty"),+withSubtype("AuditRequest"),+withFormat("json"),+withNow(time.Date(2023, time.July, 4, 12, 3, 0, 0, time.Local)),},IsErrorExpected: false,ExpectedID: "qwerty",ExpectedSubtype: RequestType,-ExpectedFormat: JSONFormat,+ExpectedFormat: jsonFormat,ExpectedNow: time.Date(2023, time.July, 4, 12, 3, 0, 0, time.Local),},}
audit/backend_noop.go+21 −13
@@ -35,14 +35,17 @@ type noopWrapper struct {backend *NoopAudit}// NoopAuditEventListener is a callback used by noopWrapper.Process() to notify-// of each received audit event.-type NoopAuditEventListener func(*AuditEvent)-func (n *NoopAudit) SetListener(listener NoopAuditEventListener) {+// SetListener provides a callback func to the NoopAudit which can be invoked+// during processing of the Event.+//+// Deprecated: SetListener should not be used in new tests.+func (n *NoopAudit) SetListener(listener func(event *Event)) {n.listener = listener}// NoopAudit only exists to allow legacy tests to continue working.+//+// Deprecated: NoopAudit should not be used in new tests.type NoopAudit struct {Config *BackendConfig@@ -68,16 +71,16 @@ type NoopAudit struct {nodeIDList []eventlogger.NodeIDnodeMap map[eventlogger.NodeID]eventlogger.Nodelistener NoopAuditEventListener+listener func(event *Event)}// NoopHeaderFormatter can be used within no-op audit devices to do nothing when+// noopHeaderFormatter can be used within no-op audit devices to do nothing when// it comes to only allow configured headers to appear in the result.// Whatever is passed in will be returned (nil becomes an empty map) in lowercase.-type NoopHeaderFormatter struct{}+type noopHeaderFormatter struct{}// ApplyConfig implements the relevant interface to make NoopHeaderFormatter an HeaderFormatter.-func (f *NoopHeaderFormatter) ApplyConfig(_ context.Context, headers map[string][]string, _ Salter) (result map[string][]string, retErr error) {+// ApplyConfig implements the relevant interface to make noopHeaderFormatter an HeaderFormatter.+func (f *noopHeaderFormatter) ApplyConfig(_ context.Context, headers map[string][]string, _ Salter) (result map[string][]string, retErr error) {if len(headers) < 1 {return map[string][]string{}, nil}@@ -95,6 +98,8 @@ func (f *NoopHeaderFormatter) ApplyConfig(_ context.Context, headers map[string]// NewNoopAudit should be used to create a NoopAudit as it handles creation of a// predictable salt and wraps eventlogger nodes so information can be retrieved on// what they've seen or formatted.+//+// Deprecated: NewNoopAudit only exists to allow legacy tests to continue working.func NewNoopAudit(config *BackendConfig) (*NoopAudit, error) {view := &logical.InmemStorage{}@@ -122,7 +127,7 @@ func NewNoopAudit(config *BackendConfig) (*NoopAudit, error) {nodeMap: make(map[eventlogger.NodeID]eventlogger.Node, 2),}cfg, err := newFormatterConfig(&NoopHeaderFormatter{}, nil)+cfg, err := newFormatterConfig(&noopHeaderFormatter{}, nil)if err != nil {return nil, err}@@ -158,6 +163,8 @@ func NewNoopAudit(config *BackendConfig) (*NoopAudit, error) {// NoopAuditFactory should be used when the test needs a way to access bytes that// have been formatted by the pipeline during audit requests.// The records parameter will be repointed to the one used within the pipeline.+//+// Deprecated: NoopAuditFactory only exists to allow legacy tests to continue working.func NoopAuditFactory(records **[][]byte) Factory {return func(config *BackendConfig, _ HeaderFormatter) (Backend, error) {n, err := NewNoopAudit(config)@@ -184,7 +191,7 @@ func (n *noopWrapper) Process(ctx context.Context, e *eventlogger.Event) (*eventvar err error// We're expecting audit events since this is an audit device.-a, ok := e.Payload.(*AuditEvent)+a, ok := e.Payload.(*Event)if !ok {return nil, errors.New("cannot parse payload as an audit event")}@@ -244,7 +251,7 @@ func (n *noopWrapper) Process(ctx context.Context, e *eventlogger.Event) (*event// formatted headers that would have made it to the logs via the sink node.// They only appear in requests.if a.Subtype == RequestType {-reqEntry := &Entry{}+reqEntry := &entry{}err = json.Unmarshal(b, &reqEntry)if err != nil {return nil, fmt.Errorf("unable to parse formatted audit entry data: %w", err)@@ -336,6 +343,7 @@ func (n *NoopAudit) IsFallback() bool {return false}// Deprecated: TestNoopAudit only exists to allow legacy tests to continue working.func TestNoopAudit(t *testing.T, path string, config map[string]string) *NoopAudit {cfg := &BackendConfig{Config: config,
audit/event.go+17 −17
@@ -23,15 +23,15 @@ const (// Audit formats.const (-JSONFormat format = "json"-JSONxFormat format = "jsonx"+jsonFormat format = "json"+jsonxFormat format = "jsonx")// Check AuditEvent implements the timeProvider at compile time.-var _ timeProvider = (*AuditEvent)(nil)+var _ timeProvider = (*Event)(nil)// AuditEvent is the audit event.-type AuditEvent struct {+// Event is the audit event.+type Event struct {ID string `json:"id"`Version string `json:"version"`Subtype subtype `json:"subtype"` // the subtype of the audit event.@@ -41,14 +41,14 @@ type AuditEvent struct {}// setTimeProvider can be used to set a specific time provider which is used when-// creating an Entry.+// creating an entry.// NOTE: This is primarily used for testing to supply a known time value.-func (a *AuditEvent) setTimeProvider(t timeProvider) {+func (a *Event) setTimeProvider(t timeProvider) {a.prov = t}// timeProvider returns a configured time provider, or the default if not set.-func (a *AuditEvent) timeProvider() timeProvider {+func (a *Event) timeProvider() timeProvider {if a.prov == nil {return a}@@ -62,10 +62,10 @@ type format string// subtype defines the type of audit event.type subtype string// NewEvent should be used to create an audit event. The subtype field is needed+// newEvent should be used to create an audit event. The subtype field is needed// for audit events. It will generate an ID if no ID is supplied. Supported-// options: WithID, WithNow.-func NewEvent(s subtype, opt ...Option) (*AuditEvent, error) {+// options: withID, withNow.+func newEvent(s subtype, opt ...option) (*Event, error) {// Get the default optionsopts, err := getOpts(opt...)if err != nil {@@ -81,7 +81,7 @@ func NewEvent(s subtype, opt ...Option) (*AuditEvent, error) {}}audit := &AuditEvent{+audit := &Event{ID: opts.withID,Timestamp: opts.withNow,Version: version,@@ -95,7 +95,7 @@ func NewEvent(s subtype, opt ...Option) (*AuditEvent, error) {}// validate attempts to ensure the audit event in its present state is valid.-func (a *AuditEvent) validate() error {+func (a *Event) validate() error {if a == nil {return fmt.Errorf("event is nil: %w", ErrInvalidParameter)}@@ -133,7 +133,7 @@ func (t subtype) validate() error {// validate ensures that format is one of the set of allowed event formats.func (f format) validate() error {switch f {-case JSONFormat, JSONxFormat:+case jsonFormat, jsonxFormat:return nildefault:return fmt.Errorf("invalid format %q: %w", f, ErrInvalidParameter)@@ -172,13 +172,13 @@ func (t subtype) String() string {// formattedTime returns the UTC time the AuditEvent was created in the RFC3339Nano// format (which removes trailing zeros from the seconds field).-func (a *AuditEvent) formattedTime() string {+func (a *Event) formattedTime() string {return a.Timestamp.UTC().Format(time.RFC3339Nano)}// IsValidFormat provides a means to validate whether the supplied format is valid.+// isValidFormat provides a means to validate whether the supplied format is valid.// Examples of valid formats are JSON and JSONx.-func IsValidFormat(v string) bool {+func isValidFormat(v string) bool {err := format(strings.TrimSpace(strings.ToLower(v))).validate()return err == nil}
audit/event_test.go+28 −28
@@ -15,7 +15,7 @@ func TestAuditEvent_new(t *testing.T) {t.Parallel()tests := map[string]struct {-Options []Option+Options []optionSubtype subtypeFormat formatIsErrorExpected bool@@ -33,47 +33,47 @@ func TestAuditEvent_new(t *testing.T) {IsErrorExpected: true,ExpectedErrorMessage: "invalid event subtype \"\": invalid internal parameter",},-"empty-Option": {-Options: []Option{},+"empty-option": {+Options: []option{},Subtype: subtype(""),Format: format(""),IsErrorExpected: true,ExpectedErrorMessage: "invalid event subtype \"\": invalid internal parameter",},"bad-id": {-Options: []Option{WithID("")},+Options: []option{withID("")},Subtype: ResponseType,-Format: JSONFormat,+Format: jsonFormat,IsErrorExpected: true,ExpectedErrorMessage: "id cannot be empty",},"good": {-Options: []Option{-WithID("audit_123"),-WithFormat(string(JSONFormat)),-WithSubtype(string(ResponseType)),-WithNow(time.Date(2023, time.July, 4, 12, 3, 0, 0, time.Local)),+Options: []option{+withID("audit_123"),+withFormat(string(jsonFormat)),+withSubtype(string(ResponseType)),+withNow(time.Date(2023, time.July, 4, 12, 3, 0, 0, time.Local)),},Subtype: RequestType,-Format: JSONxFormat,+Format: jsonxFormat,IsErrorExpected: false,ExpectedID: "audit_123",ExpectedTimestamp: time.Date(2023, time.July, 4, 12, 3, 0, 0, time.Local),ExpectedSubtype: RequestType,-ExpectedFormat: JSONxFormat,+ExpectedFormat: jsonxFormat,},"good-no-time": {-Options: []Option{-WithID("audit_123"),-WithFormat(string(JSONFormat)),-WithSubtype(string(ResponseType)),+Options: []option{+withID("audit_123"),+withFormat(string(jsonFormat)),+withSubtype(string(ResponseType)),},Subtype: RequestType,-Format: JSONxFormat,+Format: jsonxFormat,IsErrorExpected: false,ExpectedID: "audit_123",ExpectedSubtype: RequestType,-ExpectedFormat: JSONxFormat,+ExpectedFormat: jsonxFormat,IsNowExpected: true,},}@@ -84,7 +84,7 @@ func TestAuditEvent_new(t *testing.T) {t.Run(name, func(t *testing.T) {t.Parallel()audit, err := NewEvent(tc.Subtype, tc.Options...)+audit, err := newEvent(tc.Subtype, tc.Options...)switch {case tc.IsErrorExpected:require.Error(t, err)@@ -112,7 +112,7 @@ func TestAuditEvent_Validate(t *testing.T) {t.Parallel()tests := map[string]struct {-Value *AuditEvent+Value *EventIsErrorExpected boolExpectedErrorMessage string}{@@ -122,12 +122,12 @@ func TestAuditEvent_Validate(t *testing.T) {ExpectedErrorMessage: "event is nil: invalid internal parameter",},"default": {-Value: &AuditEvent{},+Value: &Event{},IsErrorExpected: true,ExpectedErrorMessage: "missing ID: invalid internal parameter",},"id-empty": {-Value: &AuditEvent{+Value: &Event{ID: "",Version: version,Subtype: RequestType,@@ -138,7 +138,7 @@ func TestAuditEvent_Validate(t *testing.T) {ExpectedErrorMessage: "missing ID: invalid internal parameter",},"version-fiddled": {-Value: &AuditEvent{+Value: &Event{ID: "audit_123",Version: "magic-v2",Subtype: RequestType,@@ -149,7 +149,7 @@ func TestAuditEvent_Validate(t *testing.T) {ExpectedErrorMessage: "event version unsupported: invalid internal parameter",},"subtype-fiddled": {-Value: &AuditEvent{+Value: &Event{ID: "audit_123",Version: version,Subtype: subtype("moon"),@@ -160,7 +160,7 @@ func TestAuditEvent_Validate(t *testing.T) {ExpectedErrorMessage: "invalid event subtype \"moon\": invalid internal parameter",},"default-time": {-Value: &AuditEvent{+Value: &Event{ID: "audit_123",Version: version,Subtype: ResponseType,@@ -171,7 +171,7 @@ func TestAuditEvent_Validate(t *testing.T) {ExpectedErrorMessage: "event timestamp cannot be the zero time instant: invalid internal parameter",},"valid": {-Value: &AuditEvent{+Value: &Event{ID: "audit_123",Version: version,Subtype: ResponseType,@@ -373,7 +373,7 @@ func TestAuditEvent_Subtype_String(t *testing.T) {// method returns the correct format.func TestAuditEvent_formattedTime(t *testing.T) {theTime := time.Date(2024, time.March, 22, 10, 0o0, 5, 10, time.UTC)-a, err := NewEvent(ResponseType, WithNow(theTime))+a, err := newEvent(ResponseType, withNow(theTime))require.NoError(t, err)require.NotNil(t, a)require.Equal(t, "2024-03-22T10:00:05.00000001Z", a.formattedTime())@@ -439,7 +439,7 @@ func TestEvent_IsValidFormat(t *testing.T) {tc := tct.Run(name, func(t *testing.T) {t.Parallel()-res := IsValidFormat(tc.input)+res := isValidFormat(tc.input)require.Equal(t, tc.expected, res)})}
audit/entry_formatter_test.go+52 −52
@@ -15,7 +15,7 @@ import ("github.com/hashicorp/eventlogger""github.com/hashicorp/go-hclog""github.com/hashicorp/go-sockaddr"-"github.com/hashicorp/vault/helper/namespace"+nshelper "github.com/hashicorp/vault/helper/namespace""github.com/hashicorp/vault/helper/testhelpers/corehelpers""github.com/hashicorp/vault/internal/observability/event""github.com/hashicorp/vault/sdk/helper/jsonutil"@@ -132,14 +132,14 @@ func TestNewEntryFormatter(t *testing.T) {Options: map[string]string{"format": "json",},-ExpectedFormat: JSONFormat,+ExpectedFormat: jsonFormat,},"default": {Name: "juan",UseStaticSalt: true,Logger: hclog.NewNullLogger(),IsErrorExpected: false,-ExpectedFormat: JSONFormat,+ExpectedFormat: jsonFormat,},"config-json": {Name: "juan",@@ -149,7 +149,7 @@ func TestNewEntryFormatter(t *testing.T) {"format": "json",},IsErrorExpected: false,-ExpectedFormat: JSONFormat,+ExpectedFormat: jsonFormat,},"config-jsonx": {Name: "juan",@@ -159,7 +159,7 @@ func TestNewEntryFormatter(t *testing.T) {"format": "jsonx",},IsErrorExpected: false,-ExpectedFormat: JSONxFormat,+ExpectedFormat: jsonxFormat,},"config-json-prefix": {Name: "juan",@@ -170,7 +170,7 @@ func TestNewEntryFormatter(t *testing.T) {"format": "json",},IsErrorExpected: false,-ExpectedFormat: JSONFormat,+ExpectedFormat: jsonFormat,ExpectedPrefix: "foo",},"config-jsonx-prefix": {@@ -182,7 +182,7 @@ func TestNewEntryFormatter(t *testing.T) {"format": "jsonx",},IsErrorExpected: false,-ExpectedFormat: JSONxFormat,+ExpectedFormat: jsonxFormat,ExpectedPrefix: "foo",},}@@ -245,7 +245,7 @@ func TestEntryFormatter_Type(t *testing.T) {}// TestEntryFormatter_Process attempts to run the Process method to convert the-// logical.LogInput within an audit event to JSON and JSONx (Entry),+// logical.LogInput within an audit event to JSON and JSONx (entry),func TestEntryFormatter_Process(t *testing.T) {t.Parallel()@@ -261,21 +261,21 @@ func TestEntryFormatter_Process(t *testing.T) {IsErrorExpected: true,ExpectedErrorMessage: "cannot audit a 'request' event with no data: invalid internal parameter",Subtype: RequestType,-RequiredFormat: JSONFormat,+RequiredFormat: jsonFormat,Data: nil,},"json-response-no-data": {IsErrorExpected: true,ExpectedErrorMessage: "cannot audit a 'response' event with no data: invalid internal parameter",Subtype: ResponseType,-RequiredFormat: JSONFormat,+RequiredFormat: jsonFormat,Data: nil,},"json-request-basic-input": {IsErrorExpected: true,ExpectedErrorMessage: "unable to parse request from 'request' audit event: request cannot be nil",Subtype: RequestType,-RequiredFormat: JSONFormat,+RequiredFormat: jsonFormat,Data: &logical.LogInput{Type: "magic"},RootNamespace: true,},@@ -283,34 +283,34 @@ func TestEntryFormatter_Process(t *testing.T) {IsErrorExpected: true,ExpectedErrorMessage: "unable to parse request from 'response' audit event: request cannot be nil",Subtype: ResponseType,-RequiredFormat: JSONFormat,+RequiredFormat: jsonFormat,Data: &logical.LogInput{Type: "magic"},},"json-request-basic-input-and-request-no-ns": {IsErrorExpected: true,ExpectedErrorMessage: "unable to retrieve namespace from context: no namespace",Subtype: RequestType,-RequiredFormat: JSONFormat,+RequiredFormat: jsonFormat,Data: &logical.LogInput{Request: &logical.Request{ID: "123"}},},"json-response-basic-input-and-request-no-ns": {IsErrorExpected: true,ExpectedErrorMessage: "unable to retrieve namespace from context: no namespace",Subtype: ResponseType,-RequiredFormat: JSONFormat,+RequiredFormat: jsonFormat,Data: &logical.LogInput{Request: &logical.Request{ID: "123"}},},"json-request-basic-input-and-request-with-ns": {IsErrorExpected: false,Subtype: RequestType,-RequiredFormat: JSONFormat,+RequiredFormat: jsonFormat,Data: &logical.LogInput{Request: &logical.Request{ID: "123"}},RootNamespace: true,},"json-response-basic-input-and-request-with-ns": {IsErrorExpected: false,Subtype: ResponseType,-RequiredFormat: JSONFormat,+RequiredFormat: jsonFormat,Data: &logical.LogInput{Request: &logical.Request{ID: "123"},Response: &logical.Response{},@@ -321,21 +321,21 @@ func TestEntryFormatter_Process(t *testing.T) {IsErrorExpected: true,ExpectedErrorMessage: "cannot audit a 'request' event with no data: invalid internal parameter",Subtype: RequestType,-RequiredFormat: JSONxFormat,+RequiredFormat: jsonxFormat,Data: nil,},"jsonx-response-no-data": {IsErrorExpected: true,ExpectedErrorMessage: "cannot audit a 'response' event with no data: invalid internal parameter",Subtype: ResponseType,-RequiredFormat: JSONxFormat,+RequiredFormat: jsonxFormat,Data: nil,},"jsonx-request-basic-input": {IsErrorExpected: true,ExpectedErrorMessage: "unable to parse request from 'request' audit event: request cannot be nil",Subtype: RequestType,-RequiredFormat: JSONxFormat,+RequiredFormat: jsonxFormat,Data: &logical.LogInput{Type: "magic"},RootNamespace: true,},@@ -343,7 +343,7 @@ func TestEntryFormatter_Process(t *testing.T) {IsErrorExpected: true,ExpectedErrorMessage: "unable to parse request from 'response' audit event: request cannot be nil",Subtype: ResponseType,-RequiredFormat: JSONxFormat,+RequiredFormat: jsonxFormat,Data: &logical.LogInput{Type: "magic"},RootNamespace: true,},@@ -351,27 +351,27 @@ func TestEntryFormatter_Process(t *testing.T) {IsErrorExpected: true,ExpectedErrorMessage: "unable to retrieve namespace from context: no namespace",Subtype: RequestType,-RequiredFormat: JSONxFormat,+RequiredFormat: jsonxFormat,Data: &logical.LogInput{Request: &logical.Request{ID: "123"}},},"jsonx-response-basic-input-and-request-no-ns": {IsErrorExpected: true,ExpectedErrorMessage: "unable to retrieve namespace from context: no namespace",Subtype: ResponseType,-RequiredFormat: JSONxFormat,+RequiredFormat: jsonxFormat,Data: &logical.LogInput{Request: &logical.Request{ID: "123"}},},"jsonx-request-basic-input-and-request-with-ns": {IsErrorExpected: false,Subtype: RequestType,-RequiredFormat: JSONxFormat,+RequiredFormat: jsonxFormat,Data: &logical.LogInput{Request: &logical.Request{ID: "123"}},RootNamespace: true,},"jsonx-response-basic-input-and-request-with-ns": {IsErrorExpected: false,Subtype: ResponseType,-RequiredFormat: JSONxFormat,+RequiredFormat: jsonxFormat,Data: &logical.LogInput{Request: &logical.Request{ID: "123"},Response: &logical.Response{},@@ -382,7 +382,7 @@ func TestEntryFormatter_Process(t *testing.T) {IsErrorExpected: true,ExpectedErrorMessage: "unable to parse request from 'response' audit event: request cannot be nil",Subtype: ResponseType,-RequiredFormat: JSONxFormat,+RequiredFormat: jsonxFormat,Data: &logical.LogInput{Auth: &logical.Auth{},},@@ -409,7 +409,7 @@ func TestEntryFormatter_Process(t *testing.T) {var ctx context.Contextswitch {case tc.RootNamespace:-ctx = namespace.RootContext(context.Background())+ctx = nshelper.RootContext(context.Background())default:ctx = context.Background()}@@ -461,7 +461,7 @@ func BenchmarkAuditFileSink_Process(b *testing.B) {},}ctx := namespace.RootContext(context.Background())+ctx := nshelper.RootContext(context.Background())// Create the formatter node.cfg, err := newFormatterConfig(&testHeaderFormatter{}, nil)@@ -472,7 +472,7 @@ func BenchmarkAuditFileSink_Process(b *testing.B) {require.NotNil(b, formatter)// Create the sink node.-sink, err := event.NewFileSink("/dev/null", JSONFormat.String())+sink, err := event.NewFileSink("/dev/null", jsonFormat.String())require.NoError(b, err)require.NotNil(b, sink)@@ -551,12 +551,12 @@ func TestEntryFormatter_Process_Request(t *testing.T) {var ctx context.Contextswitch {case tc.RootNamespace:-ctx = namespace.RootContext(context.Background())+ctx = nshelper.RootContext(context.Background())default:ctx = context.Background()}auditEvent, err := NewEvent(RequestType)+auditEvent, err := newEvent(RequestType)auditEvent.setTimeProvider(&testTimeProvider{})require.NoError(t, err)auditEvent.Data = tc.Input@@ -577,18 +577,18 @@ func TestEntryFormatter_Process_Request(t *testing.T) {case tc.ShouldOmitTime:require.NoError(t, err)require.NotNil(t, e2)-b, ok := e2.Format(JSONFormat.String())+b, ok := e2.Format(jsonFormat.String())require.True(t, ok)-var entry *Entry+var entry *entryerr = json.Unmarshal(b, &entry)require.NoError(t, err)require.Zero(t, entry.Time)default:require.NoError(t, err)require.NotNil(t, e2)-b, ok := e2.Format(JSONFormat.String())+b, ok := e2.Format(jsonFormat.String())require.True(t, ok)-var entry *Entry+var entry *entryerr = json.Unmarshal(b, &entry)require.NoError(t, err)require.NotZero(t, entry.Time)@@ -661,12 +661,12 @@ func TestEntryFormatter_Process_ResponseType(t *testing.T) {var ctx context.Contextswitch {case tc.RootNamespace:-ctx = namespace.RootContext(context.Background())+ctx = nshelper.RootContext(context.Background())default:ctx = context.Background()}auditEvent, err := NewEvent(ResponseType)+auditEvent, err := newEvent(ResponseType)auditEvent.setTimeProvider(&testTimeProvider{})require.NoError(t, err)auditEvent.Data = tc.Input@@ -688,18 +688,18 @@ func TestEntryFormatter_Process_ResponseType(t *testing.T) {case tc.ShouldOmitTime:require.NoError(t, err)require.NotNil(t, e2)-b, ok := e2.Format(JSONFormat.String())+b, ok := e2.Format(jsonFormat.String())require.True(t, ok)-var entry *Entry+var entry *entryerr = json.Unmarshal(b, &entry)require.NoError(t, err)require.Zero(t, entry.Time)default:require.NoError(t, err)require.NotNil(t, e2)-b, ok := e2.Format(JSONFormat.String())+b, ok := e2.Format(jsonFormat.String())require.True(t, ok)-var entry *Entry+var entry *entryerr = json.Unmarshal(b, &entry)require.NoError(t, err)require.NotZero(t, entry.Time)@@ -807,7 +807,7 @@ func TestEntryFormatter_Process_JSON(t *testing.T) {// Create an audit event and more generic eventlogger.event to allow us// to process (format).-auditEvent, err := NewEvent(RequestType)+auditEvent, err := newEvent(RequestType)require.NoError(t, err)auditEvent.Data = in@@ -818,10 +818,10 @@ func TestEntryFormatter_Process_JSON(t *testing.T) {Payload: auditEvent,}e2, err := formatter.Process(namespace.RootContext(nil), e)+e2, err := formatter.Process(nshelper.RootContext(nil), e)require.NoErrorf(t, err, "bad: %s\nerr: %s", name, err)jsonBytes, ok := e2.Format(JSONFormat.String())+jsonBytes, ok := e2.Format(jsonFormat.String())require.True(t, ok)require.Positive(t, len(jsonBytes))@@ -829,14 +829,14 @@ func TestEntryFormatter_Process_JSON(t *testing.T) {t.Fatalf("no prefix: %s \n log: %s\nprefix: %s", name, expectedResultStr, tc.Prefix)}expectedJSON := new(Entry)+expectedJSON := new(entry)if err := jsonutil.DecodeJSON([]byte(expectedResultStr), &expectedJSON); err != nil {t.Fatalf("bad json: %s", err)}-expectedJSON.Request.Namespace = &Namespace{ID: "root"}+expectedJSON.Request.Namespace = &namespace{ID: "root"}actualJSON := new(Entry)+actualJSON := new(entry)if err := jsonutil.DecodeJSON(jsonBytes[len(tc.Prefix):], &actualJSON); err != nil {t.Fatalf("bad json: %s", err)}@@ -972,7 +972,7 @@ func TestEntryFormatter_Process_JSONx(t *testing.T) {// Create an audit event and more generic eventlogger.event to allow us// to process (format).-auditEvent, err := NewEvent(RequestType)+auditEvent, err := newEvent(RequestType)require.NoError(t, err)auditEvent.Data = in@@ -983,10 +983,10 @@ func TestEntryFormatter_Process_JSONx(t *testing.T) {Payload: auditEvent,}e2, err := formatter.Process(namespace.RootContext(nil), e)+e2, err := formatter.Process(nshelper.RootContext(nil), e)require.NoErrorf(t, err, "bad: %s\nerr: %s", name, err)jsonxBytes, ok := e2.Format(JSONxFormat.String())+jsonxBytes, ok := e2.Format(jsonxFormat.String())require.True(t, ok)require.Positive(t, len(jsonxBytes))@@ -1071,11 +1071,11 @@ func TestEntryFormatter_ElideListResponses(t *testing.T) {oneInterestingTestCase := tests["Enhanced list (has key_info)"]ss := newStaticSalt(t)-ctx := namespace.RootContext(context.Background())+ctx := nshelper.RootContext(context.Background())var formatter *entryFormattervar err errorformat := func(t *testing.T, config formatterConfig, operation logical.Operation, inputData map[string]any) *Entry {+format := func(t *testing.T, config formatterConfig, operation logical.Operation, inputData map[string]any) *entry {formatter, err = newEntryFormatter("juan", config, ss, hclog.NewNullLogger())require.NoError(t, err)require.NotNil(t, formatter)@@ -1085,7 +1085,7 @@ func TestEntryFormatter_ElideListResponses(t *testing.T) {Response: &logical.Response{Data: inputData},}auditEvent, err := NewEvent(ResponseType)+auditEvent, err := newEvent(ResponseType)require.NoError(t, err)auditEvent.Data = in@@ -1182,7 +1182,7 @@ func TestEntryFormatter_Process_NoMutation(t *testing.T) {… diff truncated
audit/options.go+25 −25
@@ -9,8 +9,8 @@ import ("time")// Option is how options are passed as arguments.-type Option func(*options) error+// option is how options are passed as arguments.+type option func(*options) error// options are used to represent configuration for a audit related nodes.type options struct {@@ -29,15 +29,15 @@ type options struct {func getDefaultOptions() options {return options{withNow: time.Now(),-withFormat: JSONFormat,+withFormat: jsonFormat,withHMACAccessor: true,}}// getOpts applies each supplied Option and returns the fully configured options.-// Each Option is applied in the order it appears in the argument list, so it is-// possible to supply the same Option numerous times and the 'last write wins'.-func getOpts(opt ...Option) (options, error) {+// getOpts applies each supplied option and returns the fully configured options.+// Each option is applied in the order it appears in the argument list, so it is+// possible to supply the same option numerous times and the 'last write wins'.+func getOpts(opt ...option) (options, error) {opts := getDefaultOptions()for _, o := range opt {if o == nil {@@ -50,8 +50,8 @@ func getOpts(opt ...Option) (options, error) {return opts, nil}// WithID provides an optional ID.-func WithID(id string) Option {+// withID provides an optional ID.+func withID(id string) option {return func(o *options) error {var err error@@ -67,8 +67,8 @@ func WithID(id string) Option {}}// WithNow provides an Option to represent 'now'.-func WithNow(now time.Time) Option {+// withNow provides an option to represent 'now'.+func withNow(now time.Time) option {return func(o *options) error {var err error@@ -83,8 +83,8 @@ func WithNow(now time.Time) Option {}}// WithSubtype provides an Option to represent the event subtype.-func WithSubtype(s string) Option {+// withSubtype provides an option to represent the event subtype.+func withSubtype(s string) option {return func(o *options) error {s := strings.TrimSpace(s)if s == "" {@@ -101,8 +101,8 @@ func WithSubtype(s string) Option {}}// WithFormat provides an Option to represent event format.-func WithFormat(f string) Option {+// withFormat provides an option to represent event format.+func withFormat(f string) option {return func(o *options) error {f := strings.TrimSpace(strings.ToLower(f))if f == "" {@@ -121,8 +121,8 @@ func WithFormat(f string) Option {}}// WithPrefix provides an Option to represent a prefix for a file sink.-func WithPrefix(prefix string) Option {+// withPrefix provides an option to represent a prefix for a file sink.+func withPrefix(prefix string) option {return func(o *options) error {o.withPrefix = prefix@@ -130,32 +130,32 @@ func WithPrefix(prefix string) Option {}}// WithRaw provides an Option to represent whether 'raw' is required.-func WithRaw(r bool) Option {+// withRaw provides an option to represent whether 'raw' is required.+func withRaw(r bool) option {return func(o *options) error {o.withRaw = rreturn nil}}// WithElision provides an Option to represent whether elision (...) is required.-func WithElision(e bool) Option {+// withElision provides an option to represent whether elision (...) is required.+func withElision(e bool) option {return func(o *options) error {o.withElision = ereturn nil}}// WithOmitTime provides an Option to represent whether to omit time.-func WithOmitTime(t bool) Option {+// withOmitTime provides an option to represent whether to omit time.+func withOmitTime(t bool) option {return func(o *options) error {o.withOmitTime = treturn nil}}// WithHMACAccessor provides an Option to represent whether an HMAC accessor is applicable.-func WithHMACAccessor(h bool) Option {+// withHMACAccessor provides an option to represent whether an HMAC accessor is applicable.+func withHMACAccessor(h bool) option {return func(o *options) error {o.withHMACAccessor = hreturn nil
enos/enos-scenario-replication.hcl+50 −56
@@ -590,58 +590,6 @@ scenario "replication" {}}step "verify_vault_version" {-description = global.description.verify_vault_version-module = module.vault_verify_version-depends_on = [-step.create_primary_cluster,-step.wait_for_primary_cluster_leader,-]-providers = {-enos = local.enos_provider[matrix.distro]-}-verifies = [-quality.vault_api_sys_version_history_keys,-quality.vault_api_sys_version_history_key_info,-quality.vault_version_build_date,-quality.vault_version_edition,-quality.vault_version_release,-]-variables {-hosts = step.create_primary_cluster_targets.hosts-vault_addr = step.create_primary_cluster.api_addr_localhost-vault_edition = matrix.edition-vault_install_dir = global.vault_install_dir[matrix.artifact_type]-vault_product_version = matrix.artifact_source == "local" ? step.get_local_metadata.version : var.vault_product_version-vault_revision = matrix.artifact_source == "local" ? step.get_local_metadata.revision : var.vault_revision-vault_build_date = matrix.artifact_source == "local" ? step.get_local_metadata.build_date : var.vault_build_date-vault_root_token = step.create_primary_cluster.root_token-}-}-step "verify_ui" {-description = global.description.verify_ui-module = module.vault_verify_ui-depends_on = [-step.create_primary_cluster,-step.wait_for_primary_cluster_leader,-]-providers = {-enos = local.enos_provider[matrix.distro]-}-verifies = quality.vault_ui_assets-variables {-vault_addr = step.create_primary_cluster.api_addr_localhost-hosts = step.create_primary_cluster_targets.hosts-}-}-step "get_primary_cluster_ips" {description = global.description.get_vault_cluster_ip_addressesmodule = module.vault_get_cluster_ips@@ -690,12 +638,57 @@ scenario "replication" {}}step "verify_vault_version" {+description = global.description.verify_vault_version+module = module.vault_verify_version+depends_on = [step.get_primary_cluster_ips]+providers = {+enos = local.enos_provider[matrix.distro]+}+verifies = [+quality.vault_api_sys_version_history_keys,+quality.vault_api_sys_version_history_key_info,+quality.vault_version_build_date,+quality.vault_version_edition,+quality.vault_version_release,+]+variables {+hosts = step.create_primary_cluster_targets.hosts+vault_addr = step.create_primary_cluster.api_addr_localhost+vault_edition = matrix.edition+vault_install_dir = global.vault_install_dir[matrix.artifact_type]+vault_product_version = matrix.artifact_source == "local" ? step.get_local_metadata.version : var.vault_product_version+vault_revision = matrix.artifact_source == "local" ? step.get_local_metadata.revision : var.vault_revision+vault_build_date = matrix.artifact_source == "local" ? step.get_local_metadata.build_date : var.vault_build_date+vault_root_token = step.create_primary_cluster.root_token+}+}+step "verify_ui" {+description = global.description.verify_ui+module = module.vault_verify_ui+depends_on = [step.get_primary_cluster_ips]+providers = {+enos = local.enos_provider[matrix.distro]+}+verifies = quality.vault_ui_assets+variables {+vault_addr = step.create_primary_cluster.api_addr_localhost+hosts = step.create_primary_cluster_targets.hosts+}+}+step "write_test_data_on_primary" {description = global.description.verify_write_test_datamodule = module.vault_verify_write_datadepends_on = [step.get_primary_cluster_ips]providers = {enos = local.enos_provider[matrix.distro]}@@ -724,9 +717,10 @@ scenario "replication" {EOFmodule = module.vault_setup_perf_primarydepends_on = [-step.get_primary_cluster_ips,-step.get_secondary_cluster_ips,-step.write_test_data_on_primary+step.write_test_data_on_primary,+// Do base verification before continuing on to our performance replication verification.+step.verify_vault_version,+step.verify_ui,]providers = {<github-team-secure-vault-core@hashicorp.com>74b6cc799a3266dd007434687097efb6a206c7bd (#28046).../test-run-enos-scenario-matrix.yml | 2 +-enos/enos-dev-scenario-pr-replication.hcl | 12 +-enos/enos-dev-scenario-single-cluster.hcl | 8 +-enos/enos-globals.hcl | 46 ++++--enos/enos-samples-ce-build.hcl | 32 ++--enos/enos-samples-ce-release.hcl | 32 ++--enos/enos-scenario-agent.hcl | 17 +--enos/enos-scenario-autopilot.hcl | 17 +--enos/enos-scenario-proxy.hcl | 17 +--enos/enos-scenario-replication.hcl | 25 ++-enos/enos-scenario-seal-ha.hcl | 19 +--enos/enos-scenario-smoke.hcl | 17 +--enos/enos-scenario-ui.hcl | 6 +-enos/enos-scenario-upgrade.hcl | 17 +--enos/enos-variables.hcl | 12 +-enos/enos.vars.hcl | 4 +-.../build_artifactory_artifact/locals.tf | 8 +-enos/modules/ec2_info/main.tf | 142 ++++++++++++------enos/modules/install_packages/main.tf | 8 +-.../modules/softhsm_create_vault_keys/main.tf | 2 +-.../softhsm_distribute_vault_keys/main.tf | 1 +enos/modules/softhsm_init/main.tf | 1 +enos/modules/softhsm_install/main.tf | 41 ++++-enos/modules/vault_cluster/main.tf | 2 +-.../scripts/maybe-remove-old-unit-file.sh | 2 +-.../scripts/verify-cluster-version.sh | 2 +-26 files changed, 271 insertions(+), 221 deletions(-)
audit/headers.go+14 −14
@@ -42,7 +42,7 @@ func AuditedHeadersKey() string {return AuditedHeadersSubPath + auditedHeadersEntry}type HeaderSettings struct {+type headerSettings struct {// HMAC is used to indicate whether the value of the header should be HMAC'd.HMAC bool `json:"hmac"`}@@ -51,7 +51,7 @@ type HeaderSettings struct {// headers to the audit logs. It uses a BarrierView to persist the settings.type HeadersConfig struct {// headerSettings stores the current headers that should be audited, and their settings.-headerSettings map[string]*HeaderSettings+headerSettings map[string]*headerSettings// view is the barrier view which should be used to access underlying audit header config data.view durableStorer@@ -69,18 +69,18 @@ func NewHeadersConfig(view durableStorer) (*HeadersConfig, error) {// Store the view so that we can reload headers when we 'Invalidate'.return &HeadersConfig{view: view,-headerSettings: make(map[string]*HeaderSettings),+headerSettings: make(map[string]*headerSettings),}, nil}// Header attempts to retrieve a copy of the settings associated with the specified header.// The second boolean return parameter indicates whether the header existed in configuration,// it should be checked as when 'false' the returned settings will have the default values.-func (a *HeadersConfig) Header(name string) (HeaderSettings, bool) {+func (a *HeadersConfig) Header(name string) (headerSettings, bool) {a.RLock()defer a.RUnlock()var s HeaderSettings+var s headerSettingsv, ok := a.headerSettings[strings.ToLower(name)]if ok {@@ -91,16 +91,16 @@ func (a *HeadersConfig) Header(name string) (HeaderSettings, bool) {}// Headers returns all existing headers along with a copy of their current settings.-func (a *HeadersConfig) Headers() map[string]HeaderSettings {+func (a *HeadersConfig) Headers() map[string]headerSettings {a.RLock()defer a.RUnlock()// We know how many entries the map should have.-headers := make(map[string]HeaderSettings, len(a.headerSettings))+headers := make(map[string]headerSettings, len(a.headerSettings))// Clone the headersfor name, setting := range a.headerSettings {-headers[name] = HeaderSettings{HMAC: setting.HMAC}+headers[name] = headerSettings{HMAC: setting.HMAC}}return headers@@ -118,10 +118,10 @@ func (a *HeadersConfig) Add(ctx context.Context, header string, hmac bool) errordefer a.Unlock()if a.headerSettings == nil {-a.headerSettings = make(map[string]*HeaderSettings, 1)+a.headerSettings = make(map[string]*headerSettings, 1)}a.headerSettings[strings.ToLower(header)] = &HeaderSettings{hmac}+a.headerSettings[strings.ToLower(header)] = &headerSettings{hmac}entry, err := logical.StorageEntryJSON(auditedHeadersEntry, a.headerSettings)if err != nil {return fmt.Errorf("failed to persist audited headers config: %w", err)@@ -167,12 +167,12 @@ func (a *HeadersConfig) Remove(ctx context.Context, header string) error {// added to HeadersConfig in order to allow them to appear in audit logs in a raw// format. If the Vault Operator adds their own setting for any of the defaults,// their setting will be honored.-func (a *HeadersConfig) DefaultHeaders() map[string]*HeaderSettings {+func (a *HeadersConfig) DefaultHeaders() map[string]*headerSettings {// Support deprecated 'x-' prefix (https://datatracker.ietf.org/doc/html/rfc6648)const correlationID = "correlation-id"xCorrelationID := fmt.Sprintf("x-%s", correlationID)return map[string]*HeaderSettings{+return map[string]*headerSettings{correlationID: {},xCorrelationID: {},}@@ -192,7 +192,7 @@ func (a *HeadersConfig) Invalidate(ctx context.Context) error {// If we cannot update the stored 'new' headers, we will clear the existing// ones as part of invalidation.-headers := make(map[string]*HeaderSettings)+headers := make(map[string]*headerSettings)if out != nil {err = out.DecodeJSON(&headers)if err != nil {@@ -202,7 +202,7 @@ func (a *HeadersConfig) Invalidate(ctx context.Context) error {// Ensure that we are able to case-sensitively access the headers;// necessary for the upgrade case-lowerHeaders := make(map[string]*HeaderSettings, len(headers))+lowerHeaders := make(map[string]*headerSettings, len(headers))for k, v := range headers {lowerHeaders[strings.ToLower(k)] = v}
website/content/docs/enterprise/replication/index.mdx+22 −8
@@ -24,7 +24,13 @@ applications that need to interoperate.Vault replication addresses both of these needs in providing consistency,scalability, and highly-available disaster recovery.Note: Using replication requires a storage backend that supports transactional updates, such as [Integrated Storage](/vault/docs/concepts/integrated-storage) or Consul.+<Note title="Storage backend requirement">+Using replication requires a storage backend that supports transactional+updates, such as [Integrated Storage](/vault/docs/concepts/integrated-storage)+or Consul.+</Note>## Architecture@@ -143,10 +149,13 @@ replication.$ vault secrets enable -local -path=us_west_data kv-v2```-> **Learn:** Refer to the [Performance Replication with Paths-Filter](/vault/tutorials/enterprise/performance-replication) tutorial for-step-by-step instructions.+<Highlight title="Tutorials">+Refer to the _manage replicated mounts_ section in the [Set up performance+replication](/vault/tutorials/enterprise/performance-replication#manage-replicated-mounts)+tutorial to learn how to specify the mounts to allow or deny data replication.</Highlight>## Disaster recovery (DR) replication@@ -190,7 +199,13 @@ fragments.| Auto | Auto | Unchanged | Receives primary recovery | Seal recovery config and recovery keys replaced with primary's || Auto | Shamir | Receives primary recovery | N/A | Seal config and keys replaced with primary's recovery seal config and keys |Note: Clusters with Shamir seal config do not have separate recovery keys. Auto includes HSM, Cloud KMS, and Transit auto-unseal.+<Note>+Vault clusters configured with+[auto-unseal](/vault/docs/concepts/seal#auto-unseal) have recovery keys instead+of unseal keys.+</Note>### Vault versions@@ -268,13 +283,12 @@ cluster port at a load balancer level.Refer to the following tutorials replication setup and best practices:- [Setting up Performance Replication](/vault/tutorials/enterprise/performance-replication)+- [Set up Performance Replication](/vault/tutorials/enterprise/performance-replication)- [Disaster Recovery Replication Setup](/vault/tutorials/enterprise/disaster-recovery)-- [Performance Replication with Paths Filters](/vault/tutorials/enterprise/performance-replication)- [Monitoring Vault Replication](/vault/tutorials/monitoring/monitor-replication)## APIThe Vault replication component has a full HTTP API. Please see the+The Vault replication component has a full HTTP API. Refer to the[Vault Replication API](/vault/api-docs/system/replication) for moredetails.<github-team-secure-vault-core@hashicorp.com>v0.9.0 into release/1.17.x (#28040)changelog/28016.txt | 3 +++go.mod | 16 ++++++++--------go.sum | 36 ++++++++++++++++++------------------3 files changed, 29 insertions(+), 26 deletions(-)create mode 100644 changelog/28016.txt
More files changed — see the full commit.