Security context
Critical· 9.1GHSA-7cgv-v83v-rr87 CVE-2022-40186CWE-639Published Sep 23, 2022

HashiCorp Vault vulnerable to incorrect metadata access

Research this vulnerability

Research is free — Hunters explains how the bug works, the root-cause code pattern, how the fix addresses it, and how to test whether a target is affected, in chat. Investigate & write exploit is a paid run — the engine reads the advisory and fix commits, then builds and validates a working proof-of-concept exploit with reproduction steps.

Affected versions

1.11.0 → fixed in 1.11.31.10.0 → fixed in 1.10.61.8.0 → fixed in 1.9.9

Details

An issue was discovered in HashiCorp Vault and Vault Enterprise before 1.11.3. A vulnerability in the Identity Engine was found where, in a deployment where an entity has multiple mount accessors with shared alias names, Vault may overwrite metadata to the wrong alias due to an issue with checking the proper alias assigned to an entity. This may allow for unintended access to key/value paths using that metadata in Vault.

The fix

Release delta 1.11.0 → 1.11.3 (contains the fix)

· Jun 20, 2022, 02:48 PM+2386338compare
vault/identity_store_util.go+13 9
@@ -1472,19 +1472,23 @@ func (i *IdentityStore) sanitizeAndUpsertGroup(ctx context.Context, group *ident
}
// Remove duplicate entity IDs and check if all IDs are valid
- group.MemberEntityIDs = strutil.RemoveDuplicates(group.MemberEntityIDs, false)
- for _, entityID := range group.MemberEntityIDs {
- entity, err := i.MemDBEntityByID(entityID, false)
- if err != nil {
- return fmt.Errorf("failed to validate entity ID %q: %w", entityID, err)
- }
- if entity == nil {
- return fmt.Errorf("invalid entity ID %q", entityID)
+ if group.MemberEntityIDs != nil {
+ group.MemberEntityIDs = strutil.RemoveDuplicates(group.MemberEntityIDs, false)
+ for _, entityID := range group.MemberEntityIDs {
+ entity, err := i.MemDBEntityByID(entityID, false)
+ if err != nil {
+ return fmt.Errorf("failed to validate entity ID %q: %w", entityID, err)
+ }
+ if entity == nil {
+ return fmt.Errorf("invalid entity ID %q", entityID)
+ }
}
}
// Remove duplicate policies
- group.Policies = strutil.RemoveDuplicates(group.Policies, false)
+ if group.Policies != nil {
+ group.Policies = strutil.RemoveDuplicates(group.Policies, false)
+ }
txn := i.db.Txn(true)
defer txn.Abort()
<82990506+hc-github-team-secure-vault-core@users.noreply.github.com>
ed52771d63e28f110b03c4ec9da8085fb65cba38 (#16188)
website/content/docs/concepts/policies.mdx | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
website/content/docs/what-is-vault.mdx+35 14
@@ -9,17 +9,36 @@ description: >-
## What is Vault?
-Vault is an identity-based **secrets** and encryption management system. A secret is anything that you want to tightly control access to, such as API encryption keys, passwords, or certificates. Vault provides encryption services that are gated by authentication and authorization methods. Using Vault’s UI, CLI, or HTTP API, access to secrets and other sensitive data can be securely stored and managed, tightly controlled (restricted), and auditable.
+HashiCorp Vault is an identity-based secrets and encryption management system. A _secret_ is anything that you want to tightly control access to, such as API encryption keys, passwords, and certificates. Vault provides encryption services that are gated by authentication and authorization methods. Using Vault’s UI, CLI, or HTTP API, access to secrets and other sensitive data can be securely stored and managed, tightly controlled (restricted), and auditable.
-A modern system requires access to a multitude of secrets: database credentials,
+A modern system requires access to a multitude of secrets, including database credentials,
API keys for external services, credentials for service-oriented architecture
-communication, etc. Understanding who is accessing what secrets is already very
-difficult and platform-specific. Adding on key rolling, secure storage, and
+communication, etc. It can be difficult to understand who is accessing which secrets, especially since this can be platform-specific. Adding on key rolling, secure storage, and
detailed audit logs is almost impossible without a custom solution. This is
where Vault steps in.
-Examples work best to showcase Vault. Please see the
-[use cases](/docs/use-cases).
+Vault validates and authorizes clients (users, machines, apps) before providing them access to secrets or stored sensitive data.
+
+![How Vault Works](/img/how-vault-works.png)
+
+### How does Vault work?
+
+Vault works primarily with tokens and a token is associated to the client's policy. Each policy is path-based and policy rules contrains the actions and accessibility to the paths for each client. With Vault, you can create tokens manually and assign them to your clients, or the clients can log in and obtain a token. The illustration below displays Vault's core workflow.
+
+![Vault Workflow](/img/vault-workflow-diagram1.png)
+
+ The core Vault workflow consists of four stages:
+
+* **Authenticate:** Authentication in Vault is the process by which a client supplies information that Vault uses to determine if they are who they say they are. Once the client is authenticated against an auth method, a token is generated and associated to a policy.
+* **Validation:** Vault validates the client against third-party trusted sources, such as Github, LDAP, AppRole, and more.
+* **Authorize**: A client is matched against the Vault security policy. This policy is a set of rules defining which API endpoints a client has access to with its Vault token. Policies provide a declarative way to grant or forbid access to certain paths and operations in Vault.
+* **Access**: Vault grants access to secrets, keys, and encryption capabilities by issuing a token based on policies associated with the client’s identity. The client can then use their Vault token for future operations.
+
+### Why Vault?
+
+Most enterprises today have credentials sprawled across their organizations. Passwords, API keys, and credentials are stored in plain text, app source code, config files, and other locations. Because these credentials live everywhere, the sprawl can make it difficult and daunting to really know who has access and authorization to what. Having credentials in plain text also increases the potential for malicious attacks, both by internal and external attackers.
+
+Vault was designed with these challenges in mind. Vault takes all of these credentials and centralizes them so that they are defined in one location, which reduces unwanted exposure to credentials. But Vault takes it a few steps further by making sure users, apps, and systems are authenticated and explicitly authorized to access resources, while also providing an audit trail that captures and preserves a history of clients' actions.
The key features of Vault are:
@@ -51,16 +70,18 @@ The key features of Vault are:
Revocation assists in key rolling as well as locking down systems in the
case of an intrusion.
-## What is HCP Vault?
+-> **Tip**: Learn more about Vault [use cases](/docs/use-cases).
+
+### What is HCP Vault?
-HCP Vault is a hosted version of Vault, which is operated by HashiCorp to allow organizations to get up and running quickly. HCP Vault uses the same binary as self-hosted Vault, which means you will have a consistent user experience. You can use the same Vault clients to communicate with HCP Vault as you use to communicate with a self-hosted Vault.
+HashiCorp Cloud Platform (HCP) Vault is a hosted version of Vault, which is operated by HashiCorp to allow organizations to get up and running quickly. HCP Vault uses the same binary as self-hosted Vault, which means you will have a consistent user experience. You can use the same Vault clients to communicate with HCP Vault as you use to communicate with a self-hosted Vault. Refer to the [HCP Vault](https://cloud.hashicorp.com/docs/vault) documentation to learn more.
-~> **Note**: Currently, HCP Vault clusters are located on AWS running in multiple regions across North America, Asia, and Europe. We will support additional cloud providers in the future.
+> **Hands On:** Try the [Get started](https://learn.hashicorp.com/collections/vault/cloud) tutorial on HashiCorp Learn to set up a managed Vault cluster.
-To learn more about HCP Vault, see the [HCP Vault documentation](https://cloud.hashicorp.com/docs/vault). You can also get started with HCP Vault by using the HCP portal to set up your managed Vault cluster. Refer to the [Getting Started with HCP Vault](https://learn.hashicorp.com/collections/vault/cloud) tutorial.
+### Community
-## Next Steps
+We welcome questions, suggestions, and contributions from the community.
-See the page on [Vault use cases](/docs/use-cases) to learn about the multiple ways
-Vault can be used. Then, continue onwards with the [Getting Started](https://learn.hashicorp.com/collections/vault/getting-started) tutorial to use Vault
-to read, write, and create real secrets and see how it works in practice.
+* Ask questions in [HashiCorp Discuss](https://discuss.hashicorp.com/c/vault/30).
+* Read our [contributing guide](https://github.com/hashicorp/tutorials/blob/main/CONTRIBUTING.md).
+* [Submit an issue](https://github.com/hashicorp/vault/issues/new/choose) for bugs and feature requests.
website/content/docs/auth/jwt/oidc-providers/kubernetes.mdx+228 0
@@ -0,0 +1,228 @@
+---
+layout: docs
+page_title: OIDC Provider Setup - Auth Methods - Kubernetes
+description: OIDC provider configuration for Kubernetes
+---
+
+## Kubernetes
+
+Kubernetes can function as an OIDC provider such that Vault can validate its
+service account tokens using JWT/OIDC auth.
+
+-> **Note:** The JWT auth engine does **not** use Kubernetes' `TokenReview` API
+during authentication, and instead uses public key cryptography to verify the
+contents of JWTs. This means tokens that have been revoked by Kubernetes will
+still be considered valid by Vault until their expiry time. To mitigate this
+risk, use short TTLs for service account tokens or use
+[Kubernetes auth](/docs/auth/kubernetes) which _does_ use the `TokenReview` API.
+
+### Using service account issuer discovery
+
+When using service account issuer discovery, you only need to provide the JWT
+auth mount with an OIDC discovery URL, and sometimes a TLS certificate authority
+to trust. This makes it the most straightforward method to configure if your
+Kubernetes cluster meets the requirements.
+
+Kubernetes cluster requirements:
+
+* [`ServiceAccountIssuerDiscovery`][k8s-sa-issuer-discovery] feature enabled.
+ * Present from 1.18, defaults to enabled from 1.20.
+* kube-apiserver's `--service-account-issuer` flag is set to a URL that is
+ reachable from Vault. Public by default for most managed Kubernetes solutions.
+* Must use short-lived service account tokens when logging in.
+ * Tokens mounted into pods default to short-lived from 1.21.
+
+Configuration steps:
+
+1. Ensure OIDC discovery URLs do not require authentication, as detailed
+ [here][k8s-sa-issuer-discovery]:
+
+ ```bash
+ kubectl create clusterrolebinding oidc-reviewer \
+ --clusterrole=system:service-account-issuer-discovery \
+ --group=system:unauthenticated
+ ```
+
+1. Find the issuer URL of the cluster.
+
+ ```bash
+ ISSUER="$(kubectl get --raw /.well-known/openid-configuration | jq -r '.issuer')"
+ ```
+
+1. Enable and configure JWT auth in Vault.
+
+ 1. If Vault is running in Kubernetes:
+
+ ```bash
+ kubectl exec vault-0 -- vault auth enable jwt
+ kubectl exec vault-0 -- vault write auth/jwt/config \
+ oidc_discovery_url=https://kubernetes.default.svc.cluster.local \
+ oidc_discovery_ca_pem=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
+ ```
+
+ 1. Alternatively, if Vault is _not_ running in Kubernetes:
+
+ -> **Note:** When Vault is outside the cluster, the `$ISSUER` endpoint below may
+ or may not be reachable. If not, you can configure JWT auth using
+ [`jwt_validation_pubkeys`](#using-jwt-validation-public-keys) instead.
+
+ ```bash
+ vault auth enable jwt
+ vault write auth/jwt/config oidc_discovery_url="${ISSUER}"
+ ```
+
+1. Configure a role and log in as detailed [below](#creating-a-role-and-logging-in).
+
+[k8s-sa-issuer-discovery]: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#service-account-issuer-discovery
+
+### Using JWT validation public keys
+
+This method can be useful if Kubernetes' API is not reachable from Vault or if
+you would like a single JWT auth mount to service multiple Kubernetes clusters
+by chaining their public signing keys.
+
+Kubernetes cluster requirements:
+
+* [`ServiceAccountIssuerDiscovery`][k8s-sa-issuer-discovery] feature enabled.
+ * Present from 1.18, defaults to enabled from 1.20.
+ * This requirement can be avoided if you can access the Kubernetes master
+ nodes to read the public signing key directly from disk at
+ `/etc/kubernetes/pki/sa.pub`. In this case, you can skip the steps to
+ retrieve and then convert the key as it will already be in PEM format.
+* Must use short-lived service account tokens when logging in.
+ * Tokens mounted into pods default to short-lived from 1.21.
+
+Configuration steps:
+
+1. Fetch the service account signing public key from your cluster's JWKS URI.
+
+ ```bash
+ # Query the jwks_uri specified in /.well-known/openid-configuration
+ kubectl get --raw "$(kubectl get --raw /.well-known/openid-configuration | jq -r '.jwks_uri' | sed -r 's/.*\.[^/]+(.*)/\1/')"
+ ```
+
+1. Convert the keys from JWK format to PEM. You can use a CLI tool or an online
+ converter such as [this one][jwk-to-pem].
+
+1. Configure the JWT auth mount with those public keys.
+
+ ```bash
+ vault write auth/jwt/config \
+ jwt_validation_pubkeys="-----BEGIN PUBLIC KEY-----
+ MIIBIjANBgkqhkiG9...
+ -----END PUBLIC KEY-----","-----BEGIN PUBLIC KEY-----
+ MIIBIjANBgkqhkiG9...
+ -----END PUBLIC KEY-----"
+ ```
+
+1. Configure a role and log in as detailed [below](#creating-a-role-and-logging-in).
+
+[jwk-to-pem]: https://8gwifi.org/jwkconvertfunctions.jsp
+
+### Creating a role and logging in
+
+Once your JWT auth mount is configured, you're ready to configure a role and
+log in. The following assumes you use the projected service account token
+available in all pods by default. See [Specifying TTL and audience](#specifying-ttl-and-audience)
+below if you'd like to control the audience or TTL.
+
+1. Choose any value from the array of default audiences. In these examples,
+ there is only one audience in the `aud` array,
+ `https://kubernetes.default.svc.cluster.local`.
+
+ To find the default audiences, either create a fresh token (requires
+ `kubectl` v1.24.0+):
+
+ ```shell-session
+ $ kubectl create token default | cut -f2 -d. | base64 --decode
+ {"aud":["https://kubernetes.default.svc.cluster.local"], ... "sub":"system:serviceaccount:default:default"}
+ ```
+
+ Or read a token from a running pod's filesystem:
+
+ ```shell-session
+ $ kubectl exec my-pod -- cat /var/run/secrets/kubernetes.io/serviceaccount/token | cut -f2 -d. | base64 --decode
+ {"aud":["https://kubernetes.default.svc.cluster.local"], ... "sub":"system:serviceaccount:default:default"}
+ ```
+
+1. Create a role for JWT auth that the `default` service account from the
+ `default` namespace can use.
+
+ ```bash
+ vault write auth/jwt/role/my-role \
+ role_type="jwt" \
+ bound_audiences="<AUDIENCE-FROM-PREVIOUS-STEP>" \
+ user_claim="sub" \
+ bound_subject="system:serviceaccount:default:default" \
+ policies="default" \
+ ttl="1h"
+ ```
+
+1. Pods or other clients with access to a service account JWT can then log in.
+
+ ```bash
+ vault write auth/jwt/login \
+ role=my-role \
+ jwt=@/var/run/secrets/kubernetes.io/serviceaccount/token
+ # OR equivalent to:
+ curl \
+ --fail \
+ --request POST \
+ --header "X-Vault-Request: true" \
+ --data '{"jwt":"<JWT-TOKEN-HERE>","role":"my-role"}' \
+ "${VAULT_ADDR}/v1/auth/jwt/login"
+ ```
+
+### Specifying TTL and audience
+
+If you would like to specify a custom TTL or audience for service account tokens,
+the following pod spec illustrates a volume mount that overrides the default
+admission injected token. This is especially relevant if you are unable to
+disable the [--service-account-extend-token-expiration][k8s-extended-tokens]
+flag for `kube-apiserver` and want to use short TTLs.
+
+When using the resulting token, you will need to set `bound_audiences=vault`
+when creating roles in Vault's JWT auth mount.
+
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+ name: nginx
+spec:
+ # automountServiceAccountToken is redundant in this example because the
+ # mountPath used overlaps with the default path. The overlap stops the default
+ # admission injected token from being created. You can use this option to
+ # ensure only a single token is mounted if you choose a different mount path.
+ automountServiceAccountToken: false
+ containers:
+ - name: nginx
+ image: nginx
+ volumeMounts:
+ - name: custom-token
+ mountPath: /var/run/secrets/kubernetes.io/serviceaccount
+ volumes:
+ - name: custom-token
+ projected:
+ defaultMode: 420
+ sources:
+ - serviceAccountToken:
+ path: token
+ expirationSeconds: 600 # 10 minutes is the minimum TTL
+ audience: vault # Must match your JWT role's `bound_audiences`
+ # The remaining sources are included to mimic the rest of the default
+ # admission injected volume.
+ - configMap:
+ name: kube-root-ca.crt
+ items:
+ - key: ca.crt
+ path: ca.crt
+ - downwardAPI:
+ items:
+ - fieldRef:
+ apiVersion: v1
+ fieldPath: metadata.namespace
+ path: namespace
+```
+
+[k8s-extended-tokens]: https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/#options
command/agent/config/config_test.go+307 0
@@ -1033,3 +1033,310 @@ func TestLoadConfigFile_EnforceConsistency(t *testing.T) {
t.Fatal(diff)
}
}
+
+func TestLoadConfigFile_Disable_Idle_Conns_All(t *testing.T) {
+ config, err := LoadConfig("./test-fixtures/config-disable-idle-connections-all.hcl")
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ expected := &Config{
+ SharedConfig: &configutil.SharedConfig{
+ PidFile: "./pidfile",
+ },
+ DisableIdleConns: []string{"auto-auth", "caching", "templating"},
+ DisableIdleConnsCaching: true,
+ DisableIdleConnsAutoAuth: true,
+ DisableIdleConnsTemplating: true,
+ AutoAuth: &AutoAuth{
+ Method: &Method{
+ Type: "aws",
+ MountPath: "auth/aws",
+ Namespace: "my-namespace/",
+ Config: map[string]interface{}{
+ "role": "foobar",
+ },
+ },
+ Sinks: []*Sink{
+ {
+ Type: "file",
+ DHType: "curve25519",
+ DHPath: "/tmp/file-foo-dhpath",
+ AAD: "foobar",
+ Config: map[string]interface{}{
+ "path": "/tmp/file-foo",
+ },
+ },
+ },
+ },
+ Vault: &Vault{
+ Address: "http://127.0.0.1:1111",
+ Retry: &Retry{
+ ctconfig.DefaultRetryAttempts,
+ },
+ },
+ }
+
+ config.Prune()
+ if diff := deep.Equal(config, expected); diff != nil {
+ t.Fatal(diff)
+ }
+}
+
+func TestLoadConfigFile_Disable_Idle_Conns_Auto_Auth(t *testing.T) {
+ config, err := LoadConfig("./test-fixtures/config-disable-idle-connections-auto-auth.hcl")
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ expected := &Config{
+ SharedConfig: &configutil.SharedConfig{
+ PidFile: "./pidfile",
+ },
+ DisableIdleConns: []string{"auto-auth"},
+ DisableIdleConnsCaching: false,
+ DisableIdleConnsAutoAuth: true,
+ DisableIdleConnsTemplating: false,
+ AutoAuth: &AutoAuth{
+ Method: &Method{
+ Type: "aws",
+ MountPath: "auth/aws",
+ Namespace: "my-namespace/",
+ Config: map[string]interface{}{
+ "role": "foobar",
+ },
+ },
+ Sinks: []*Sink{
+ {
+ Type: "file",
+ DHType: "curve25519",
+ DHPath: "/tmp/file-foo-dhpath",
+ AAD: "foobar",
+ Config: map[string]interface{}{
+ "path": "/tmp/file-foo",
+ },
+ },
+ },
+ },
+ Vault: &Vault{
+ Address: "http://127.0.0.1:1111",
+ Retry: &Retry{
+ ctconfig.DefaultRetryAttempts,
+ },
+ },
+ }
+
+ config.Prune()
+ if diff := deep.Equal(config, expected); diff != nil {
+ t.Fatal(diff)
+ }
+}
+
+func TestLoadConfigFile_Disable_Idle_Conns_Templating(t *testing.T) {
+ config, err := LoadConfig("./test-fixtures/config-disable-idle-connections-templating.hcl")
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ expected := &Config{
+ SharedConfig: &configutil.SharedConfig{
+ PidFile: "./pidfile",
+ },
+ DisableIdleConns: []string{"templating"},
+ DisableIdleConnsCaching: false,
+ DisableIdleConnsAutoAuth: false,
+ DisableIdleConnsTemplating: true,
+ AutoAuth: &AutoAuth{
+ Method: &Method{
+ Type: "aws",
+ MountPath: "auth/aws",
+ Namespace: "my-namespace/",
+ Config: map[string]interface{}{
+ "role": "foobar",
+ },
+ },
+ Sinks: []*Sink{
+ {
+ Type: "file",
+ DHType: "curve25519",
+ DHPath: "/tmp/file-foo-dhpath",
+ AAD: "foobar",
+ Config: map[string]interface{}{
+ "path": "/tmp/file-foo",
+ },
+ },
+ },
+ },
+ Vault: &Vault{
+ Address: "http://127.0.0.1:1111",
+ Retry: &Retry{
+ ctconfig.DefaultRetryAttempts,
+ },
+ },
+ }
+
+ config.Prune()
+ if diff := deep.Equal(config, expected); diff != nil {
+ t.Fatal(diff)
+ }
+}
+
+func TestLoadConfigFile_Disable_Idle_Conns_Caching(t *testing.T) {
+ config, err := LoadConfig("./test-fixtures/config-disable-idle-connections-caching.hcl")
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ expected := &Config{
+ SharedConfig: &configutil.SharedConfig{
+ PidFile: "./pidfile",
+ },
+ DisableIdleConns: []string{"caching"},
+ DisableIdleConnsCaching: true,
+ DisableIdleConnsAutoAuth: false,
+ DisableIdleConnsTemplating: false,
+ AutoAuth: &AutoAuth{
+ Method: &Method{
+ Type: "aws",
+ MountPath: "auth/aws",
+ Namespace: "my-namespace/",
+ Config: map[string]interface{}{
+ "role": "foobar",
+ },
+ },
+ Sinks: []*Sink{
+ {
+ Type: "file",
+ DHType: "curve25519",
+ DHPath: "/tmp/file-foo-dhpath",
+ AAD: "foobar",
+ Config: map[string]interface{}{
+ "path": "/tmp/file-foo",
+ },
+ },
+ },
+ },
+ Vault: &Vault{
+ Address: "http://127.0.0.1:1111",
+ Retry: &Retry{
+ ctconfig.DefaultRetryAttempts,
+ },
+ },
+ }
+
+ config.Prune()
+ if diff := deep.Equal(config, expected); diff != nil {
+ t.Fatal(diff)
+ }
+}
+
+func TestLoadConfigFile_Disable_Idle_Conns_Empty(t *testing.T) {
+ config, err := LoadConfig("./test-fixtures/config-disable-idle-connections-empty.hcl")
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ expected := &Config{
+ SharedConfig: &configutil.SharedConfig{
+ PidFile: "./pidfile",
+ },
+ DisableIdleConns: []string{},
+ DisableIdleConnsCaching: false,
+ DisableIdleConnsAutoAuth: false,
+ DisableIdleConnsTemplating: false,
+ AutoAuth: &AutoAuth{
+ Method: &Method{
+ Type: "aws",
+ MountPath: "auth/aws",
+ Namespace: "my-namespace/",
+ Config: map[string]interface{}{
+ "role": "foobar",
+ },
+ },
+ Sinks: []*Sink{
+ {
+ Type: "file",
+ DHType: "curve25519",
+ DHPath: "/tmp/file-foo-dhpath",
+ AAD: "foobar",
+ Config: map[string]interface{}{
+ "path": "/tmp/file-foo",
+ },
+ },
+ },
+ },
+ Vault: &Vault{
+ Address: "http://127.0.0.1:1111",
+ Retry: &Retry{
+ ctconfig.DefaultRetryAttempts,
+ },
+ },
+ }
+
+ config.Prune()
+ if diff := deep.Equal(config, expected); diff != nil {
+ t.Fatal(diff)
+ }
+}
+
+func TestLoadConfigFile_Disable_Idle_Conns_Env(t *testing.T) {
+ err := os.Setenv(DisableIdleConnsEnv, "auto-auth,caching,templating")
+ defer os.Unsetenv(DisableIdleConnsEnv)
+
+ if err != nil {
+ t.Fatal(err)
+ }
+ config, err := LoadConfig("./test-fixtures/config-disable-idle-connections-empty.hcl")
+ if err != nil {
+ t.Fatal(err)
+ }
+
+ expected := &Config{
+ SharedConfig: &configutil.SharedConfig{
+ PidFile: "./pidfile",
+ },
+ DisableIdleConns: []string{"auto-auth", "caching", "templating"},
+ DisableIdleConnsCaching: true,
+ DisableIdleConnsAutoAuth: true,
+ DisableIdleConnsTemplating: true,
+ AutoAuth: &AutoAuth{
+ Method: &Method{
+ Type: "aws",
+ MountPath: "auth/aws",
+ Namespace: "my-namespace/",
+ Config: map[string]interface{}{
+ "role": "foobar",
+ },
+ },
+ Sinks: []*Sink{
+ {
+ Type: "file",
+ DHType: "curve25519",
+ DHPath: "/tmp/file-foo-dhpath",
+ AAD: "foobar",
+ Config: map[string]interface{}{
+ "path": "/tmp/file-foo",
+ },
+ },
+ },
+ },
+ Vault: &Vault{
+ Address: "http://127.0.0.1:1111",
+ Retry: &Retry{
+ ctconfig.DefaultRetryAttempts,
+ },
+ },
+ }
+
+ config.Prune()
+ if diff := deep.Equal(config, expected); diff != nil {
+ t.Fatal(diff)
+ }
+}
+
+func TestLoadConfigFile_Bad_Value_Disable_Idle_Conns(t *testing.T) {
+ _, err := LoadConfig("./test-fixtures/bad-config-disable-idle-connections.hcl")
+ if err == nil {
+ t.Fatal("should have error, it didn't")
+ }
+}
website/content/docs/auth/jwt/oidc-providers/azuread.mdx+127 0
@@ -0,0 +1,127 @@
+---
+layout: docs
+page_title: OIDC Provider Setup - Auth Methods - Azure Active Directory
+description: OIDC provider configuration for Azure Active Directory
+---
+
+## Azure Active Directory (AAD)
+
+~> **Note:** Azure Active Directory Applications that have custom signing keys as a result of using
+the [claims-mapping](https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-claims-mapping)
+feature are currently not supported for OIDC authentication.
+
+Reference: [Azure Active Directory v2.0 and the OpenID Connect protocol](https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-protocols-oidc)
+
+1. Choose your Azure tenant.
+
+1. Go to **Azure Active Directory** and
+ [register an application](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app)
+ for Vault.
+
+1. Add Redirect URIs with the "Web" type. You may include two redirect URIs,
+ one for CLI access another one for Vault UI access.
+ - `http://localhost:8250/oidc/callback`
+ - `https://hostname:port_number/ui/vault/auth/oidc/oidc/callback`
+
+1. Record the "Application (client) ID" as you will need it as the `oidc_client_id`.
+
+1. Under **Endpoints**, copy the OpenID Connect metadata document URL, omitting the `/well-known...` portion.
+ - The endpoint URL (`oidc_discovery_url`) will look like: https://login.microsoftonline.com/tenant-guid-dead-beef-aaaa-aaaa/v2.0
+
+1. Under **Certificates & secrets**,
+ [add a client secret](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app#add-a-client-secret)
+ Record the secret's value as you will need it as the `oidc_client_secret` for Vault.
+
+### Connect AD group with Vault external group
+
+Reference: [Azure Active Directory with OIDC Auth Method and External Groups](https://learn.hashicorp.com/tutorials/vault/oidc-auth-azure)
+
+To connect the AD group with a [Vault external groups](/docs/secrets/identity#external-vs-internal-groups),
+you will need
+[Azure AD v2.0 endpoints](https://docs.microsoft.com/en-gb/azure/active-directory/develop/azure-ad-endpoint-comparison).
+You should set up a [Vault policy](https://learn.hashicorp.com/tutorials/vault/policies) for the Azure AD group to use.
+
+1. Go to **Azure Active Directory** and choose your Vault application.
+
+1. Go to **Token configuration** and **Add groups claim**. Select "All" or "SecurityGroup" based on
+ [which groups for a user](https://docs.microsoft.com/en-us/azure/active-directory/hybrid/how-to-connect-fed-group-claims)
+ you want returned in the claim.
+
+1. In Vault, enable the OIDC auth method.
+
+1. Configure the OIDC auth method with the `oidc_client_id` (application ID), `oidc_client_secret`
+ (client secret), and `oidc_discovery_url` (endpoint URL) you recorded from Azure.
+ ```shell
+ vault write auth/oidc/config \
+ oidc_client_id="your_client_id" \
+ oidc_client_secret="your_client_secret" \
+ default_role="your_default_role" \
+ oidc_discovery_url="https://login.microsoftonline.com/tenant_id/v2.0"
+ ```
+
+1. Configure the [OIDC Role](/api-docs/auth/jwt#create-role) with the following:
+ - `user_claim` should be `"sub"` or `"oid"` following the
+ [recommendation](https://docs.microsoft.com/en-us/azure/active-directory/develop/id-tokens#using-claims-to-reliably-identify-a-user-subject-and-object-id)
+ from Azure.
+ - `allowed_redirect_uris` should be the two redirect URIs for Vault CLI and UI access.
+ - `groups_claim` should be set to `"groups"`.
+ - `oidc_scopes` should be set to `"https://graph.microsoft.com/.default"`.
+ ```shell
+ vault write auth/oidc/role/your_default_role \
+ user_claim="sub" \
+ allowed_redirect_uris="http://localhost:8250/oidc/callback,https://online_version_hostname:port_number/ui/vault/auth/oidc/oidc/callback" \
+ groups_claim="groups" \
+ oidc_scopes="https://graph.microsoft.com/.default" \
+ policies=default
+ ```
+
+1. In Vault, create the [external group](/api-docs/secret/identity/group).
+ Record the group ID as you will need it for the group alias.
+
+1. From Vault, retrieve the [OIDC accessor ID](/api-docs/system/auth#list-auth-methods)
+ from the OIDC auth method as you will need it for the group alias's `mount_accessor`.
+
+1. Go to the Azure AD Group you want to attach to Vault's external group. Record the `objectId`
+ as you will need it as the group alias name in Vault.
+
+1. In Vault, create a [group alias](/api-docs/secret/identity/group-alias)
+ for the external group and set the `objectId` as the group alias name.
+ ```shell
+ vault write identity/group-alias \
+ name="your_ad_group_object_id" \
+ mount_accessor="vault_oidc_accessor_id" \
+ canonical_id="vault_external_group_id"
+ ```
+
+### Optional Azure-specific Configuration
+
+If a user is a member of more than 200 groups (directly or indirectly), extra configuration
+is required so that Vault can fetch the groups properly.
+
+- In Azure, under the applications **API Permissions**, grant the following permissions:
+ - Microsoft Graph API permission [Directory.Read.All](https://docs.microsoft.com/en-us/graph/permissions-reference#application-permissions-19)
+
+- In Vault, set `"provider_config"` to Azure.
+ ```shell
+ vault write auth/oidc/config -<<"EOH"
+ {
+ "oidc_client_id": "your_client_id",
+ "oidc_client_secret": "your_client_secret",
+ "default_role": "your_default_role",
+ "oidc_discovery_url": "https://login.microsoftonline.com/tenant_id/v2.0",
+ "provider_config": {
+ "provider": "azure"
+ }
+ }
+ EOH
+ ```
+
+- In Vault, add `"profile"` to `oidc_scopes` so the user's id comes back on the JWT.
+ ```shell
+ vault write auth/oidc/role/your_default_role \
+ user_claim="email" \
+ allowed_redirect_uris="http://localhost:8250/oidc/callback,https://online_version_hostname:port_number/ui/vault/auth/oidc/oidc/callback" \
+ groups_claim="groups" \
+ oidc_scopes="profile" \
+ policies="default"
+ ```
vault/external_tests/identity/identity_test.go+13 1
@@ -628,8 +628,20 @@ func assertMember(t *testing.T, client *api.Client, entityID, groupName, groupID
t.Fatal(err)
}
groupMap := secret.Data
+
+ groupEntityMembers, ok := groupMap["member_entity_ids"].([]interface{})
+ if !ok && expectFound {
+ t.Fatalf("expected member_entity_ids not to be nil")
+ }
+
+ // if type assertion fails and expectFound is false, groupEntityMembers
+ // is nil, then let's just return, nothing to be done!
+ if !ok && !expectFound {
+ return
+ }
+
found := false
- for _, entityIDRaw := range groupMap["member_entity_ids"].([]interface{}) {
+ for _, entityIDRaw := range groupEntityMembers {
if entityIDRaw.(string) == entityID {
found = true
}
website/content/docs/release-notes/1.11.0.mdx+2 2
@@ -16,7 +16,7 @@ We encourage you to upgrade to the latest release to take advantage of the new b
Some of these enhancements and changes in this release include:
-- Vault Consul secrets engine provides a templating policy to allow node and service identities to be set on the Consult token creation
+- Vault Consul secrets engine provides a templating policy to allow node and service identities to be set on the Consul token creation
- Snowflake secrets engine added a key/pair-based authentication
- Vault adds a Kubernetes secrets engine to allow creating dynamic k8s service accounts
- ADP-Transform extends its functionality by adding a convergent tokenization mode and a tokenization lookup
@@ -53,7 +53,7 @@ The KV version 2 secrets engine now includes a set of utilities and enhancements
For more details, refer to the [Version Key/Value Secrets Engine](https://learn.hashicorp.com/tutorials/vault/versioned-kv) tutorial.
-### Support for node identity and service identity for Vault Consult secrets engine
+### Support for node identity and service identity for Vault Consul secrets engine
Within the Consul secrets engine, practitioners writing a Vault role can specify node-identity or service-identity. You can also specify multiples of each identity on a Vault role. For more information, refer to the [Consul Secrets Engine](/docs/secrets/consul) and [Consul Secrets Engine (API)](/api-docs/secret/consul) documentation.
<82990506+hc-github-team-secure-vault-core@users.noreply.github.com>
af52d67dc188688f2cf044e78402993972cdbc18 (#16128)
website/content/api-docs/secret/kubernetes.mdx | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
website/content/docs/auth/jwt/oidc-providers/google.mdx+107 0
@@ -0,0 +1,107 @@
+---
+layout: docs
+page_title: OIDC Provider Setup - Auth Methods - Google
+description: OIDC provider configuration for Google
+---
+
+## Google
+
+Main reference: [Using OAuth 2.0 to Access Google APIs](https://developers.google.com/identity/protocols/OAuth2)
+
+1. Visit the [Google API Console](https://console.developers.google.com).
+1. Create or a select a project.
+1. Create a new credential via Credentials > Create Credentials > OAuth Client ID.
+1. Configure the OAuth Consent Screen. Application Name is required. Save.
+1. Select application type: "Web Application".
+1. Configure Authorized Redirect URIs.
+1. Save client ID and secret.
+
+### Optional Google-specific Configuration
+
+Google-specific configuration is available when using Google as an identity provider from the
+Vault JWT/OIDC auth method. The configuration allows Vault to obtain Google Workspace group membership and
+user information during the JWT/OIDC authentication flow. The group membership obtained from Google Workspace
+may be used for Identity group alias association. The user information obtained from Google Workspace can be
+used to copy claims data into resulting auth token and alias metadata via [claim_mappings](/api-docs/auth/jwt#claim_mappings).
+
+#### Setup
+
+To set up the Google-specific handling, you'll need:
+
+- A Google Workspace account with the [super admin role](https://support.google.com/a/answer/2405986?hl=en)
+ for granting domain-wide delegation API client access.
+- The ability to create a service account in [Google Cloud Platform](https://console.developers.google.com/iam-admin/serviceaccounts).
+- To enable the [Admin SDK API](https://console.developers.google.com/apis/api/admin.googleapis.com/overview).
+- An OAuth 2.0 application with an [external user type](https://support.google.com/cloud/answer/10311615#user-type).
+
+The Google-specific handling that's used to fetch Google Workspace groups and user information in Vault uses
+[Google Workspace Domain-Wide Delegation of Authority](https://developers.google.com/admin-sdk/directory/v1/guides/delegation)
+for authentication and authorization. You need to follow **all steps** in the [guide](https://developers.google.com/admin-sdk/directory/v1/guides/delegation)
+to obtain the key file for a Google service account capable of making requests to the Google Workspace
+[User Accounts](https://developers.google.com/admin-sdk/directory/v1/guides/manage-users) and
+[Groups](https://developers.google.com/admin-sdk/directory/v1/guides/manage-groups) APIs.
+
+In **step 5** within the section titled
+[Delegate domain-wide authority to your service account](https://developers.google.com/admin-sdk/directory/v1/guides/delegation#delegate_domain-wide_authority_to_your_service_account),
+the only OAuth scopes that should be granted are:
+
+- `https://www.googleapis.com/auth/admin.directory.group.readonly`
+- `https://www.googleapis.com/auth/admin.directory.user.readonly`
+
+~> This is an **important security step** in order to give the service account the least set of privileges
+that enable the feature.
+
+The Google service account key file obtained from the steps in the guide must be made available on the
+host that Vault is running on.
+
+#### Configuration
+
+- `provider` `(string: <required>)` - Name of the provider. Must be set to "gsuite".
+- `gsuite_service_account` `(string: <required>)` - Either the path to or the contents of a Google service
+ account key file in JSON format. If given as a file path, it must refer to a file that's readable on
+ the host that Vault is running on. If given directly as JSON contents, the JSON must be properly escaped.
+- `gsuite_admin_impersonate` `(string: <required>)` - Email address of a Google Workspace admin to impersonate.
+- `fetch_groups` `(bool: false)` - If set to true, groups will be fetched from Google Workspace.
+- `fetch_user_info` `(bool: false)` - If set to true, user info will be fetched from Google Workspace using the configured [user_custom_schemas](#user_custom_schemas).
+- `groups_recurse_max_depth` `(int: <optional>)` - Group membership recursion max depth. Defaults to 0, which means don't recurse.
+- `user_custom_schemas` `(string: <optional>)` - Comma-separated list of Google Workspace [custom schemas](https://developers.google.com/admin-sdk/directory/v1/guides/manage-schemas).
+ Values set for Google Workspace users using custom schema fields will be fetched and made available as claims that can be used with [claim_mappings](/api-docs/auth/jwt#claim_mappings). Required if [fetch_user_info](#fetch_user_info) is set to true.
+
+Example configuration:
+
+```
+vault write auth/oidc/config -<<EOF
+{
+ "oidc_discovery_url": "https://accounts.google.com",
+ "oidc_client_id": "your_client_id",
+ "oidc_client_secret": "your_client_secret",
+ "default_role": "your_default_role",
+ "provider_config": {
+ "provider": "gsuite",
+ "gsuite_service_account": "/path/to/service-account.json",
+ "gsuite_admin_impersonate": "admin@gsuitedomain.com",
+ "fetch_groups": true,
+ "fetch_user_info": true,
+ "groups_recurse_max_depth": 5,
+ "user_custom_schemas": "Education,Preferences"
+ }
+}
+EOF
+```
+
+#### Role
+
+The [user_claim](/api-docs/auth/jwt#user_claim) value of the role must be set to
+one of either `sub` or `email` for the Google Workspace group and user information
+queries to succeed.
+
+Example role:
+
+```
+vault write auth/oidc/role/your_default_role \
+ allowed_redirect_uris="http://localhost:8200/ui/vault/auth/oidc/oidc/callback,http://localhost:8250/oidc/callback" \
+ user_claim="sub" \
+ groups_claim="groups" \
+ claim_mappings="/Education/graduation_date"="graduation_date" \
+ claim_mappings="/Preferences/shirt_size"="shirt_size"
+```
builtin/logical/ssh/backend_test.go+4 1
@@ -1480,6 +1480,8 @@ func TestBackend_DefExtTemplatingEnabled(t *testing.T) {
"default_extensions_template": true,
"default_extensions": map[string]interface{}{
"login@foobar.com": "{{identity.entity.aliases." + userpassAccessor + ".name}}",
+ "login@foobar2.com": "{{identity.entity.aliases." + userpassAccessor + ".name}}, " +
+ "{{identity.entity.aliases." + userpassAccessor + ".name}}_foobar",
},
})
if err != nil {
@@ -1505,7 +1507,8 @@ func TestBackend_DefExtTemplatingEnabled(t *testing.T) {
}
defaultExtensionPermissions := map[string]string{
- "login@foobar.com": testUserName,
+ "login@foobar.com": testUserName,
+ "login@foobar2.com": fmt.Sprintf("%s, %s_foobar", testUserName, testUserName),
}
err = validateSSHCertificate(parsedKey.(*ssh.Certificate), sshKeyID, ssh.UserCert, []string{"tuber"}, map[string]string{}, defaultExtensionPermissions, 16*time.Hour)
website/content/docs/platform/github-actions.mdx+46 0
@@ -0,0 +1,46 @@
+---
+layout: docs
+page_title: GitHub Actions
+description: >-
+ GitHub Actions
+---
+
+# GitHub Actions
+
+Workflows in GitHub Actions can make use of secrets stored in Vault by using a
+[`vault-action`](https://github.com/marketplace/actions/vault-secrets) step.
+
+## Example
+
+Here is an example `vault-action` step in a workflow:
+
+```yaml
+jobs:
+ build:
+ # ...
+ steps:
+ # ...
+ - name: Import Secrets
+ uses: hashicorp/vault-action@v2.4.0
+ with:
+ url: https://vault.example.com:8200
+ token: ${{ secrets.VAULT_TOKEN }}
+ caCertificate: ${{ secrets.VAULT_CA_CERT }}
+ secrets: |
+ secret/data/ci/aws accessKey | AWS_ACCESS_KEY_ID ;
+ secret/data/ci/aws secretKey | AWS_SECRET_ACCESS_KEY ;
+ secret/data/ci npm_token
+```
+
+This example will authenticate to Vault instance at `https://vault.example.com:8200` with the GitHub secrets defined in
+`VAULT_TOKEN` and `VAULT_CA_CERT`, and will add environment variables available for next steps in the workflow:
+- The secret at path `secret/data/ci/aws` with the key `accessKey` available in the environment variable `AWS_ACCESS_KEY_ID`
+- The secret at path `secret/data/ci/aws` with the key `secretKey` available in the environment variable `AWS_SECRET_ACCESS_KEY`
+- The secret at path `secret/data/ci` with the key `npm_token` available in the environment variable `NPM_TOKEN`
+
+## Further Information
+
+For more information on using the `vault-action` GitHub Action, visit:
+
+- [`vault-secrets` GitHub action documentation](https://github.com/marketplace/actions/vault-secrets)
+- [Vault GitHub actions tutorial](https://learn.hashicorp.com/tutorials/vault/github-actions)
website/content/docs/platform/k8s/helm/run.mdx+16 5
@@ -252,9 +252,16 @@ server:
GOOGLE_PROJECT: <PROJECT NAME>
GOOGLE_APPLICATION_CREDENTIALS: /vault/userconfig/kms-creds/credentials.json
- extraVolumes:
- - type: 'secret'
- name: 'kms-creds'
+ volumes:
+ - name: userconfig-kms-creds
+ secret:
+ defaultMode: 420
+ secretName: kms-creds
+
+ volumeMounts:
+ - mountPath: /vault/userconfig/kms-creds
+ name: userconfig-kms-creds
+ readOnly: true
ha:
enabled: true
@@ -502,8 +509,12 @@ to the Vault startup command:
```shell-session
$ helm install vault hashicorp/vault \
- --set='server.extraVolumes[0].type=secret' \
- --set='server.extraVolumes[0].name=vault-storage-config' \
+ --set='server.volumes[0].name=userconfig-vault-storage-config' \
+ --set='server.volumes[0].secret.defaultMode=420' \
+ --set='server.volumes[0].secret.secretName=vault-storage-config' \
+ --set='server.volumeMounts[0].mountPath=/vault/userconfig/vault-storage-config' \
+ --set='server.volumeMounts[0].name=userconfig-vault-storage-config' \
+ --set='server.volumeMounts[0].readOnly=true' \
--set='server.extraArgs=-config=/vault/userconfig/vault-storage-config/config.hcl'
```
<82990506+hc-github-team-secure-vault-core@users.noreply.github.com>
befcb2a5eb73c26fc0739372193ad1f93bd19cb0 (#16221)
website/content/docs/concepts/seal.mdx | 54 ++++++++++++++++----------
1 file changed, 34 insertions(+), 20 deletions(-)
website/content/docs/concepts/seal.mdx+34 20
@@ -53,8 +53,8 @@ decrypt the root key.
The unseal process is done by running `vault operator unseal` or via the API.
This process is stateful: each key can be entered via multiple mechanisms
-on multiple computers and it will work. This allows each shard of the master
-key to be on a distinct machine for better security.
+on multiple computers and it will work. This allows each shard of the root key
+to be on a distinct machine for better security.
Once a Vault node is unsealed, it remains unsealed until one of these things happens:
@@ -67,7 +67,7 @@ Once a Vault node is unsealed, it remains unsealed until one of these things hap
-> **Note:** Unsealing makes the process of automating a Vault install
difficult. Automated tools can easily install, configure, and start Vault,
but unsealing it using Shamir is a very manual process. For most users
-AutoUnseal will provide a better experience.
+Auto Unseal will provide a better experience.
## Sealing
@@ -81,7 +81,10 @@ access to the root key shards.
## Auto Unseal
--> **Note:** The Seal Wrap functionality is enabled by default. For this reason, the seal provider (HSM or cloud KMS) must be available throughout Vault's runtime and not just during the unseal process. Refer to the [Seal Wrap](/docs/enterprise/sealwrap) documentation for more information.
+-> **Note:** The Seal Wrap functionality is enabled by default. For this
+reason, the seal provider (HSM or cloud KMS) must be available throughout
+Vault's runtime and not just during the unseal process. Refer to the [Seal
+Wrap](/docs/enterprise/sealwrap) documentation for more information.
Auto Unseal was developed to aid in reducing the operational complexity of
keeping the unseal key secure. This feature delegates the responsibility of
@@ -89,7 +92,7 @@ securing the unseal key from users to a trusted device or service. At startup
Vault will connect to the device or service implementing the seal and ask it
to decrypt the root key Vault read from storage.
-![Auto unseal](/img/vault-auto-unseal.png)
+![Auto Unseal](/img/vault-auto-unseal.png)
There are certain operations in Vault besides unsealing that
require a quorum of users to perform, e.g. generating a root token. When
@@ -101,13 +104,13 @@ Just as the initialization process with a Shamir seal yields unseal keys,
initializing with an Auto Unseal yields recovery keys.
-> **Note:** Recovery keys cannot decrypt the root key, and thus are not
-sufficient to unseal Vault if the AutoUnseal mechanism isn't working. They
+sufficient to unseal Vault if the Auto Unseal mechanism isn't working. They
are purely an authorization mechanism.
It is still possible to seal a Vault node using the API. In this case Vault
-will remain sealed until restarted, or the unseal API is used, which with AutoUnseal
-requires the recovery key fragments instead of the unseal key fragments that
-would be provided with Shamir. The process remains the same.
+will remain sealed until restarted, or the unseal API is used, which with Auto
+Unseal requires the recovery key fragments instead of the unseal key fragments
+that would be provided with Shamir. The process remains the same.
For a list of examples and supported providers, please see the
[seal documentation](/docs/configuration/seal).
@@ -167,7 +170,11 @@ API prefix for this operation is at `/sys/rekey-recovery-key` rather than
## Seal Migration
-The Seal migration process cannot be performed without downtime, and due to the technical underpinnings of the seal implementations, the process requires that you briefly take the whole cluster down. While experiencing some downtime may be unavoidable, we believe that switching seals is a rare event and that the inconvenience of the downtime is an acceptable trade-off.
+The Seal migration process cannot be performed without downtime, and due to the
+technical underpinnings of the seal implementations, the process requires that
+you briefly take the whole cluster down. While experiencing some downtime may
+be unavoidable, we believe that switching seals is a rare event and that the
+inconvenience of the downtime is an acceptable trade-off.
~> **NOTE**: A backup should be taken before starting seal migration in case
something goes wrong.
@@ -177,7 +184,12 @@ available during the migration. For example, migration from Auto Unseal to Shami
seal will require that the service backing the Auto Unseal is accessible during
the migration.
-~> **NOTE**: Seal migration from Auto Unseal to Auto Unseal of the same type is supported since Vault 1.6.0. However, there is a current limitation that prevents migrating from AWSKMS to AWSKMS; all other seal migrations of the same type are supported. Seal migration from One Auto Unseal type (AWS KMS) to different Auto Unseal type (HSM, Azure KMS, etc.) is also supported on older versions as well.
+~> **NOTE**: Seal migration from Auto Unseal to Auto Unseal of the same type is
+supported since Vault 1.6.0. However, there is a current limitation that
+prevents migrating from AWSKMS to AWSKMS; all other seal migrations of the same
+type are supported. Seal migration from One Auto Unseal type (AWS KMS) to
+different Auto Unseal type (HSM, Azure KMS, etc.) is also supported on older
+versions as well.
### Migration post Vault 1.5.1
@@ -191,7 +203,8 @@ any storage backend.
seal block to the configuration.
- If the migration is from Auto seal to Shamir seal, add `disabled = "true"`
to the old seal block.
- - If the migration is from Auto seal to another Auto seal, add `disabled = "true"` to the old seal block and add the desired new Auto seal block.
+ - If the migration is from Auto seal to another Auto seal, add `disabled =
+ "true"` to the old seal block and add the desired new Auto seal block.
Now, bring the standby node back up and run the unseal command on each key, by
supplying the `-migrate` flag.
@@ -215,7 +228,7 @@ any storage backend.
1. The new active node will perform the migration. Monitor the server log in
the active node to witness the completion of the seal migration process.
Wait for a little while for the migration information to replicate to all the
- nodes in case of Integrated Storage. In enterprise Vault, switching a Auto seal
+ nodes in case of Integrated Storage. In enterprise Vault, switching an Auto seal
implies that the seal wrapped storage entries get re-wrapped. Monitor the log
and wait until this process is complete (look for `seal re-wrap completed`).
@@ -247,13 +260,14 @@ keys.
#### Migration From Auto Unseal to Shamir
-To migrate from Auto Unseal to Shamir keys, take your server cluster offline and
-update the [seal configuration](/docs/configuration/seal) and add `disabled = "true"` to the seal block. This allows the migration to use this information to
-decrypt the key but will not unseal Vault. When you bring your server back up,
-run the unseal process with the `-migrate` flag and use the Recovery Keys to
-perform the migration. All unseal commands must specify the `-migrate` flag.
-Once the required threshold of recovery keys are entered, the recovery keys will
-be migrated to be used as unseal keys.
+To migrate from Auto Unseal to Shamir keys, take your server cluster offline
+and update the [seal configuration](/docs/configuration/seal) and add `disabled
+= "true"` to the seal block. This allows the migration to use this information
+to decrypt the key but will not unseal Vault. When you bring your server back
+up, run the unseal process with the `-migrate` flag and use the Recovery Keys
+to perform the migration. All unseal commands must specify the `-migrate` flag.
+Once the required threshold of recovery keys are entered, the recovery keys
+will be migrated to be used as unseal keys.
#### Migration From Auto Unseal to Auto Unseal
<82990506+hc-github-team-secure-vault-core@users.noreply.github.com>
8a49422979620947bde49b49767607b64fb8fde7 (#16233)
changelog/16231.txt | 3 +++
command/agent/config/config.go | 11 ++++++++---
command/agent/config/config_test.go | 4 ++++
.../config/test-fixtures/config-template-full.hcl | 5 +++++
4 files changed, 20 insertions(+), 3 deletions(-)
create mode 100644 changelog/16231.txt
website/content/docs/secrets/kmip.mdx+2 0
@@ -18,6 +18,8 @@ services and applications to perform cryptographic operations without having to
manage cryptographic material, otherwise known as managed objects, by delegating
its storage and lifecycle to a key management server.
+Vault's KMIP secrets engine listens on a separate port from the standard Vault listener. Each Vault server in a Vault cluster configured with a KMIP secrets engine uses the same listener configuration. The KMIP listener defaults to port 5696 and is configurable to alternative ports, for example, if there are multiple KMIP secrets engine mounts configured. KMIP clients connect and authenticate to this KMIP secrets engine listener port using generated TLS certificates. KMIP clients may connect directly to any of the Vault servers on the configured KMIP port. A layer 4 tcp load balancer may be used in front of the Vault server's KMIP ports. The load balancer should support long-lived connections and it may use a round robin routing algorithm as Vault servers will forward to the primary Vault server, if necessary.
+
## KMIP Conformance
Vault implements version 1.4 of the following Key Management Interoperability Protocol Profiles:
<82990506+hc-github-team-secure-vault-core@users.noreply.github.com>
cd3a0b6914e14b49515270659c83bc04a8596d8a (#16247)
website/content/docs/commands/index.mdx | 18 ++++++++++--
website/content/docs/concepts/policies.mdx | 33 +++++++++++++---------
2 files changed, 34 insertions(+), 17 deletions(-)
website/content/docs/commands/index.mdx+15 3
@@ -85,6 +85,18 @@ $ vault write -output-curl-string auth/userpass/users/bob password="long-passwo
curl -X PUT -H "X-Vault-Request: true" -H "X-Vault-Token: $(vault print token)" -d '{"password":"long-password"}' http://127.0.0.1:8200/v1/auth/userpass/users/bob
```
+#### Print Policy Requirements
+
+To view the policy requirements to perform an operation, use the `-output-policy` flag after the subcommand.
+
+```
+$ vault kv put -output-policy kv/secret value=itsasecret
+
+path "kv/data/secret" {
+ capabilities = ["create", "update"]
+}
+```
+
## Command Help
There are two primary ways to get help in Vault: [CLI help (`help`)](#cli-help)
@@ -208,9 +220,9 @@ does not support filenames with `=` in them.
## Mount flag syntax (KV)
-All `kv` commands can alternatively refer to the path to the KV secrets engine using a flag-based syntax like `$ vault kv get -mount=secret password`
-instead of `$ vault kv get secret/password`. The mount flag syntax was created to mitigate confusion caused by the fact that for KV v2 secrets,
-their full path (used in policies and raw API calls) actually contains a nested `/data/` element (e.g. `secret/data/password`) which can be easily overlooked when using
+All `kv` commands can alternatively refer to the path to the KV secrets engine using a flag-based syntax like `$ vault kv get -mount=secret password`
+instead of `$ vault kv get secret/password`. The mount flag syntax was created to mitigate confusion caused by the fact that for KV v2 secrets,
+their full path (used in policies and raw API calls) actually contains a nested `/data/` element (e.g. `secret/data/password`) which can be easily overlooked when using
the above KV v1-like syntax `secret/password`. To avoid this confusion, all KV-specific docs pages will use the `-mount` flag.
## Exit Codes
website/content/docs/auth/jwt/oidc_providers.mdx+32 13
@@ -304,7 +304,7 @@ service account tokens using JWT/OIDC auth.
-> **Note:** The JWT auth engine does **not** use Kubernetes' `TokenReview` API
during authentication, and instead uses public key cryptography to verify the
-contents of JWTs. This means tokens that have been revoked by Kubernetes will
+contents of JWTs. This means tokens that have been revoked by Kubernetes will
still be considered valid by Vault until their expiry time. To mitigate this
risk, use short TTLs for service account tokens or use
[Kubernetes auth](/docs/auth/kubernetes) which _does_ use the `TokenReview` API.
@@ -390,10 +390,7 @@ Configuration steps:
1. Fetch the service account signing public key from your cluster's JWKS URI.
```bash
- # 1. Find the issuer URL of the cluster.
- ISSUER="$(kubectl get --raw /.well-known/openid-configuration | jq -r '.issuer')"
-
- # 2. Query the jwks_uri specified in /.well-known/openid-configuration
+ # Query the jwks_uri specified in /.well-known/openid-configuration
kubectl get --raw "$(kubectl get --raw /.well-known/openid-configuration | jq -r '.jwks_uri' | sed -r 's/.*\.[^/]+(.*)/\1/')"
```
@@ -418,16 +415,36 @@ Configuration steps:
### Creating a role and logging in
Once your JWT auth mount is configured, you're ready to configure a role and
-log in.
+log in. The following assumes you use the projected service account token
+available in all pods by default. See [Specifying TTL and audience](#specifying-ttl-and-audience)
+below if you'd like to control the audience or TTL.
+
+1. Choose any value from the array of default audiences. In these examples,
+ there is only one audience in the `aud` array,
+ `https://kubernetes.default.svc.cluster.local`.
+
+ To find the default audiences, either create a fresh token (requires
+ `kubectl` v1.24.0+):
+
+ ```shell-session
+ $ kubectl create token default | cut -f2 -d. | base64 --decode
+ {"aud":["https://kubernetes.default.svc.cluster.local"], ... "sub":"system:serviceaccount:default:default"}
+ ```
+
+ Or read a token from a running pod's filesystem:
+
+ ```shell-session
+ $ kubectl exec my-pod -- cat /var/run/secrets/kubernetes.io/serviceaccount/token | cut -f2 -d. | base64 --decode
+ {"aud":["https://kubernetes.default.svc.cluster.local"], ... "sub":"system:serviceaccount:default:default"}
+ ```
1. Create a role for JWT auth that the `default` service account from the
- `default` namespace can use. The audience of tokens defaults to the same as
- the issuer, but it is configurable.
+ `default` namespace can use.
```bash
vault write auth/jwt/role/my-role \
role_type="jwt" \
- bound_audiences="${ISSUER}" \
+ bound_audiences="<AUDIENCE-FROM-PREVIOUS-STEP>" \
user_claim="sub" \
bound_subject="system:serviceaccount:default:default" \
policies="default" \
@@ -467,9 +484,9 @@ metadata:
name: nginx
spec:
# automountServiceAccountToken is redundant in this example because the
- # mountPath overlapping with the default path below will already stop the
- # default admission injected token from being created. Use this option if you
- # choose a different mount path.
+ # mountPath used overlaps with the default path. The overlap stops the default
+ # admission injected token from being created. You can use this option to
+ # ensure only a single token is mounted if you choose a different mount path.
automountServiceAccountToken: false
containers:
- name: nginx
@@ -485,7 +502,9 @@ spec:
- serviceAccountToken:
path: token
expirationSeconds: 600 # 10 minutes is the minimum TTL
- audience: vault
+ audience: vault # Must match your JWT role's `bound_audiences`
+ # The remaining sources are included to mimic the rest of the default
+ # admission injected volume.
- configMap:
name: kube-root-ca.crt
items:
<82990506+hc-github-team-secure-vault-core@users.noreply.github.com>
9b186f33ca05c722addda902efb3f1c794052856 (#16105)
website/content/docs/release-notes/1.11.0.mdx | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
More files changed — see the full commit.

Release delta 1.10.0 → 1.10.6 (contains the fix)

· Mar 22, 2022, 12:41 PM+3133864compare
website/content/docs/concepts/client-count/faq.mdx+62 15
@@ -6,32 +6,38 @@ description: An FAQ page to answer the most commonly asked questions about clien
# Frequently Asked Questions (FAQ)
+~> **Note**: Note: Starting in Vault 1.9, Vault changed the non-entity token computation logic to deduplicate non-entity tokens. For non-entity tokens (where there is no entity to which tokens map) Vault uses the contents of the token to generate a unique client identifier, based on the namespace ID and policies. The clientID will prevent the same token from being duplicated in the overall client count. Non-entity token tracking is done on access instead of creation. Since the change was made, Vault 1.10 (via the UI, API, documentation, etc.) refers to these non-entity tokens as non-entity clients, and unique entities as entity clients. To summarize, starting in Vault 1.9, the terms used are: total clients = entity clients + non entity clients. Previously, the terms used were: total clients = unique entities + non-entity tokens.
+
This FAQ section contains frequently asked questions about the client count feature.
- [Q: What is a client?](#q-what-is-a-client)
- [Q: Where can I learn more about Vault clients?](#q-where-can-i-learn-more-about-vault-clients)
-- [Q: What is the difference between a direct entity and a non-entity token?](#q-what-is-the-difference-between-a-direct-entity-and-a-non-entity-token)
+- [Q: What is the difference between a direct entity (entity client) and a non-entity token (non-entity client)?](#q-what-is-the-difference-between-a-direct-entity-entity-client-and-a-non-entity-token-non-entity-client)
- [Q: Which Vault version reflects the most accurate client counts?](#q-which-vault-version-reflects-the-most-accurate-client-counts)
-- [Q: For customers using older versions prior to Vault 1.6, what’s the best way to compute clients?](#q-for-customers-using-older-versions-of-vault-1-6-what-s-the-best-way-to-compute-clients)
+- [Q: For customers using versions of Vault older than 1.6, what’s the best way to compute clients](#q-for-customers-using-versions-of-vault-older-than-1-6-what-s-the-best-way-to-compute-clients)
- [Q: For customers using newer versions than Vault 1.6, what's the best way to compute clients?](#q-for-customers-using-newer-versions-than-vault-1-6-what-s-the-best-way-to-compute-clients)
- [Q: Why do we have two different tools (auditor tool and UI/API) to compute clients? Do we plan to deprecate one in the future?](#q-why-do-we-have-two-different-tools-auditor-tool-and-ui-api-to-compute-clients-do-we-plan-to-deprecate-one-in-the-future)
- [Q: How can I compute KMIP clients for Vault?](#q-how-can-i-compute-kmip-clients-for-vault)
- [Q: Why do the Vault auditor tool and the usage metrics UI show me different results for the total number of clients?](#q-why-do-the-vault-auditor-tool-and-the-usage-metrics-ui-show-me-different-results-for-the-total-number-of-clients)
- [Q: When I upgrade to a version of Vault that's greater than Vault 1.6, will the clients be available for the entire history of the billing period, or only available after the upgrade occurred during the billing period?](#q-when-i-upgrade-to-a-version-of-vault-that-s-greater-vault-1-6-will-the-clients-be-available-for-the-entire-history-of-the-billing-period-or-only-available-after-the-upgrade-occurred-during-the-billing-period)
-- [Q: If I upgrade from Vault 1.8 to 1.9, how will the changes to non-entity token logic and local auth mount made in Vault 1.9 affect the clients created prior to the upgrade?](#q-if-i-upgrade-from-vault-1-8-to-1-9-how-will-the-changes-to-non-entity-token-logic-and-local-auth-mount-made-in-vault-1-9-affect-the-clients-created-prior-to-the-upgrade)
+- [Q: If I upgrade from Vault 1.8 to 1.9+, how will the changes to non-entity token logic and local auth mount made in Vault 1.9 affect the clients created prior to the upgrade?](#q-if-i-upgrade-from-vault-1-8-to-1-9-+-how-will-the-changes-to-non-entity-token-logic-and-local-auth-mount-made-in-vault-1-9-affect-the-clients-created-prior-to-the-upgrade)
- [Q: Post Vault 1.9, will the clientID be viewable via the audit logs when non-entity tokens are used?](#q-post-vault-1-9-will-the-clientid-be-viewable-via-the-audit-logs-when-non-entity-tokens-are-used)
- [Q: What happens if audit logs are unreadable for use by the Vault auditor tool?](#q-what-happens-if-audit-logs-are-unreadable-for-use-by-the-vault-auditor-tool)
- [Q: What does the usage metrics UI look like for Vault 1.9?](#q-what-does-the-usage-metrics-ui-look-like-for-vault-1-9)
-- [Q: In versions prior to Vault 1.9, how do I compute changes to clients month to month from the UI?](#q-in-versions-prior-to-vault-1-9-how-do-i-compute-changes-to-clients-month-to-month-from-the-ui)
+- [Q: What does the usage metrics look like for Vault 1.10?](#q-what-does-the-usage-metrics-look-like-for-vault-1-10)
+- [Q: In versions prior to Vault 1.10, how do I compute changes to clients month to month from the UI?](#q-in-versions-prior-to-vault-1-10-how-do-i-compute-changes-to-clients-month-to-month-from-the-ui)
- [Q: What if I selected an inaccurate billing period via the UI/API?](#q-what-if-i-selected-an-inaccurate-billing-period-via-the-ui-api)
- [Q: What if I want to skip computation of clients for a period of time during the billing period?](#q-what-if-i-want-to-skip-computation-of-clients-for-a-period-of-time-during-the-billing-period)
- [Q: What are the known client count issues in the auditor tool as well as in the UI/API?](#q-what-are-the-known-client-count-issues-in-the-auditor-tool-as-well-as-in-the-ui-api)
-- [Q: Under what conditions can cause the loss of client data?](#q-under-what-conditions-can-cause-the-loss-of-client-data)
+- [Q: What conditions can cause the loss of client data?](#q-what-conditions-can-cause-the-loss-of-client-data)
- [Q: How can I disable the counting of client activity?](http://localhost:3000/docs/concepts/client-count/faq#q-how-can-i-disable-the-counting-of-client-activity)
- [Q: If I request data for January 2021 - December 2021, but April’s data does not exist, what will be included in the total client count result?](#q-if-i-request-data-for-january-2021-december-2021-but-april-s-data-does-not-exist-what-will-be-included-in-the-total-client-count-result)
- [Q: How can I configure the activity for log retention?](#q-how-can-i-configure-the-activity-for-log-retention)
- [Q: Do child namespaces create duplicate tokens?](#q-do-child-namespaces-create-duplicate-tokens)
- [Q: How does the Nomad Vault integration affect client counts?](#q-how-does-the-nomad-vault-integration-affect-client-counts)
+- [Q: Starting in Vault 1.9, Vault does not allow creating two aliases from the same auth mount under a single entity. What changed and how does this impact client counting?](#q-starting-in-vault-1-9-vault-does-not-allow-creating-two-aliases-from-the-same-auth-mount-under-a-single-entity-what-changed-and-how-does-this-impact-client-counting)
+- [Q: How does mount migration impact the client count metric?](#q-how-does-mount-migration-impact-the-client-count-metric)
+- [Q: Vault 1.9 added support for providing custom user filters through the userfilter parameter. How does this affect client counts?](#q-vault-1-9-added-support-for-providing-custom-user-filters-through-the-userfilter-parameter-how-does-this-affect-client-counts)
### Q: What is a client?
@@ -50,7 +56,7 @@ Refer to the table below for documentation resources.
| [Client Count API](https://www.vaultproject.io/api-docs/system/internal-counters#client-count) | Provides information about the client count API endpoints |
| [Vault Auditor Tool](https://learn.hashicorp.com/tutorials/vault/usage-metrics#vault-auditor-tool) | Provides a walkthrough on how to use the vault-auditor tool to extract metrics from the server audit device logs |
-### Q: What is the difference between a direct entity and a non-entity token?
+### Q: What is the difference between a direct entity (entity client) and a non-entity token (non-entity client)?
While the definition of clients appears to be simple on the surface, there are many nuances involved in the computation of clients. As mentioned, clients are unique applications, services, and/or users that authenticate to a Vault cluster. When anything authenticates to Vault, it is associated with a unique identity entity within the [Vault Identity system](/docs/secrets/identity). The name reported to the identity systems by the different types of authentication methods varies, and each entity is created or verified during authorization.
@@ -71,15 +77,21 @@ Although client counts have been available via the usage metrics UI since Vault
- Vault 1.6: Introduction of client counts in the usage metrics UI
- Vault 1.8:
- - Eliminated wrapped tokens and control groups from client count, thereby reducing the non-entity token count. Previously, the creation and usage of control groups and wrapping tokens affected the client count each time the response is read (in the case of a wrapping token) and each time a control group was created (a non-entity token was created)
+ - Eliminated wrapped tokens and control groups from client count, thereby reducing the non-entity token count. Previously, the creation and usage of control groups and wrapping tokens incremented the client count via non-entity tokens, each time a wrapped token and a control group were created.
- Changed the logic of counting of active identity entities on usage instead of at create time, resulting in more accurate client counts
- Vault 1.9:
- - Changed the non-entity token computation logic to deduplicate non-entity tokens, reducing the overall client count. Moving forward, non-entity tokens, where there is no entity to map tokens, Vault will use the contents fo the token to generate a unique client identifier based on the namespace ID and associated policies. The clientID will prevent duplicating the same token in the overall client count when the token is used again during the billing period.
+
+ - Changed the non-entity token computation logic to deduplicate non-entity tokens, reducing the overall client count. Moving forward, non-entity tokens, where there is no entity to map tokens, Vault will use the contents of the token to generate a unique client identifier based on the namespace ID and associated policies. The clientID will prevent duplicating the same token in the overall client count when the token is used again during the billing period.
- Changed the tracking of non-entity tokens to complete on access instead of creation.
- Changed the computation logic to not include root tokens in the client count aggregate.
- Changed the local auth mount computation logic such that local auth mounts count towards clients but not as non-entity tokens. Prior to Vault 1.9, local auth mounts counted towards non-entity tokens. Refer to the [What is a Client?](docs/concepts/client-count) documentation to learn more.
- Added ability to display clients per namespace (top 10, descending order) in the UI and export data for all namespaces. Prior to Vault 1.9, you could not view view the split of clients per namespace on the UI, nor could you export this data via the UI.
- Added ability to display clients earlier than a month (within ten minutes of enabling the feature) in the UI. Prior to Vault 1.9, after enabling the counting of clients, you had to wait for a month to view the client aggregates in the UI.
+ - Changed functionality to disallow creating two aliases from the same auth mount under a single entity. For more information, refer to the question [Starting in Vault 1.9, Vault does not allow creating two aliases from the same auth mount under a single entity. What changed and how does this impact client counting?](#q-starting-in-vault-1-9-vault-does-not-allow-creating-two-aliases-from-the-same-auth-mount-under-a-single-entity-what-changed-and-how-does-this-impact-client-counting)
+
+- Vault 1.10:
+ - Display of clients per auth mount with a namespace in the UI.
+ - Display of clients month to month for a selected billing period via the API.
**Auditor tool**:
@@ -90,7 +102,7 @@ Although client counts have been available via the usage metrics UI since Vault
The latest GA version of the Vault binary contains the most updated version of the client count computation logic. However, it’s important to note that even if one upgrades to the latest version and the billing period falls on either side of the upgrade time, the compute logic may be different across the billing period. For more details, refer to the question [If I migrate from Vault 1.8 to 1.9, how will the changes to non-entity token logic and local auth mount made in Vault 1.9 affect the clients created prior to the migration?](#q-if-i-migrate-from-vault-1-8-to-1-9-how-will-the-changes-to-non-entity-token-logic-and-local-auth-mount-made-in-vault-1-9-affect-the-clients-created-prior-to-the-migration).
-### Q: For customers using older versions of Vault 1.6, what’s the best way to compute clients?
+### Q: For customers using versions of Vault older than 1.6, what’s the best way to compute clients?
The Vault [auditor tool](https://learn.hashicorp.com/tutorials/vault/usage-metrics#vault-auditor-tool) was built to compute clients for Vault versions older than Vault 1.6. It has been tested for versions 1.3 to 1.5 but should work for earlier versions, such as Vault 1.0, although not officially tested. To use the Vault [auditor tool](https://learn.hashicorp.com/tutorials/vault/usage-metrics#vault-auditor-tool), customers should have audit logs for the billing period for computed client counts. You can also set a specific date range in the [auditor tool](https://learn.hashicorp.com/tutorials/vault/usage-metrics#vault-auditor-tool).
@@ -106,7 +118,7 @@ Not all customers may be on a version greater than Vault version 1.6 that levera
### Q: How can I compute KMIP clients for Vault?
-As of Vault 1.9, KMIP clients are not available via the usage metrics UI or the client count API; they are provided via the [auditor tool](https://learn.hashicorp.com/tutorials/vault/usage-metrics#vault-auditor-tool). To learn more, refer to the [Vault Usage Metrics](https://learn.hashicorp.com/tutorials/vault/usage-metrics#vault-auditor-tool) documentation.
+As of Vault 1.10, KMIP clients are not available via the usage metrics UI or the client count API; they are provided via the [auditor tool](https://learn.hashicorp.com/tutorials/vault/usage-metrics#vault-auditor-tool). To learn more, refer to the [Vault Usage Metrics](https://learn.hashicorp.com/tutorials/vault/usage-metrics#vault-auditor-tool) documentation.
### Q: Why do the Vault auditor tool and the usage metrics UI show me different results for the total number of clients?
@@ -125,7 +137,7 @@ For newer versions of Vault 1.8, the API/UI for client counts was updated to ref
The client counts will only be available after the upgrade occurs. For the complete billing period data, it’s preferable to refer to the [auditor tool](https://learn.hashicorp.com/tutorials/vault/usage-metrics#vault-auditor-tool). However, keep in mind that since Vault 1.8, we made improvements to the client count API/UI that may cause mismatched results from the [auditor tool](https://learn.hashicorp.com/tutorials/vault/usage-metrics#vault-auditor-tool). For more details, refer to the question [If I migrate from Vault 1.8 to 1.9, how will the changes to non-entity token logic and local auth mount made in Vault 1.9 affect the clients created prior to the migration?](/docs/concepts/client-count/faq#q-if-i-migrate-from-vault-1-8-to-1-9-how-will-the-changes-to-non-entity-token-logic-and-local-auth-mount-made-in-vault-1-9-affect-the-clients-created-prior-to-the-migration).
A workaround is to leverage the results from the UI/API (if on a newer version greater than Vault 1.8) instead of the [auditor tool](https://learn.hashicorp.com/tutorials/vault/usage-metrics#vault-auditor-tool), and extrapolate the clients for the available period to the billing period.
-### Q: If I upgrade from Vault 1.8 to 1.9, how will the changes to non-entity token logic and local auth mount made in Vault 1.9 affect the clients created prior to the upgrade?
+### Q: If I upgrade from Vault 1.8 to 1.9+, how will the changes to non-entity token logic and local auth mount made in Vault 1.9 affect the clients created prior to the upgrade?
If you have a non-entity token for a fragment pre-Vault 1.9 version and then use the same token post-Vault 1.9 version, it will be counted again. However, for post-upgrade, the token will have an ID associated with it. From there, the subsequent uses of the token will not be counted again, as the token is tracked with the unique clientID. Hence, only for the period post the upgrade, the new deduplication logic for non-entity tokens are accounted for.
@@ -145,7 +157,15 @@ In Vault 1.9, the client count dashboard provides two separate tabs: the **Curre
![Vault Client Count](/img/client-counts.jpg)
-### Q: In versions prior to Vault 1.9, how do I compute changes to clients month to month from the UI?
+### Q: What does the usage metrics look like for Vault 1.10?
+
+In Vault 1.10, the client count dashboard is broken down into tabs, similar to Vault 1.19- the current month and the monthly history. On top of the namespace attribution provided in Vault 1.9 (see [What does the usage metrics UI look like for Vault 1.9?](#q-what-does-the-usage-metrics-ui-look-like-for-vault-1-9) for further information), the UI also contains attribution of clients per auth mount.
+
+![Vault Client Count](/img/vault-usage-metrics-1-10.png)
+
+The Vault 1.10 UI does not include montly attribution of clients, although the API for Vault 1.10 supports the same.
+
+### Q: In versions prior to Vault 1.10, how do I compute changes to clients month to month from the UI?
To perform this calculation, you must know the billing period. For the sake of this example, assume your billing period starts on January 1st and ends on December 31st:
@@ -189,9 +209,10 @@ Known issues for both tools include the following:
**UI/API**:
- Via the UI/API, the billing period cannot be computed for start and end dates that fall in the middle of a month. For example, if the billing period starts on March 15th and ends on March 14th within the subsequent year, the tool can only compute the billing period assuming March 1 is the start date or April 1 is the start date, but not using the March 15th start date
-- As of Vault 1.9, KMIP clients are not provided by the API/CLI. We have plans to add this to a future version
+- As of Vault 1.10, KMIP clients are not provided by the API/CLI. We have plans to add this to a future version
+- As of Vault 1.10, the data on the current month tab does not take the billing period into account. This means that it may include clients that have already been previously counted. However, the monthly history tab does take the billing period into account.
-### Q: Under what conditions can cause the loss of client data?
+### Q: What conditions can cause the loss of client data?
The activity log (component within Vault responsible for computing clients) is tracked on standby nodes and periodically transmitted to the active node over gRPC. The transmission is triggered when the information on the standby node reaches a maximum size of 8KB or 10 minutes has elapsed.
@@ -218,4 +239,30 @@ However, creating a new token across a parent/child namespace boundary could res
### Q: How does the Nomad Vault integration affect client counts?
The [Nomad Vault integration](https://www.nomadproject.io/docs/integrations/vault-integration#token-role-based-integration) uses [token roles](https://www.nomadproject.io/docs/integrations/vault-integration#vault-token-role-configuration). A single token role creates tokens for many Nomad jobs. If no [explicit identity aliases](/api-docs/auth/token#entity_alias) are provided (which is not currently supported in the integration), this would create a non-entity token for every running instance of a Nomad job.
-Prior to Vault 1.9, the Nomad Vault integration caused duplicate clients, resulting in an elevated client count. Post 1.9, with the introduction of the deduplication logic, the number of clients created by the integration is reduced. For more information on improvements made to client count in Vault 1.9, refer to the question [Which version of Vault reflects the most accurate count of clients with Vault?](#q-which-vault-version-reflects-the-most-accurate-client-counts).
+Prior to Vault 1.9, the Nomad Vault integration caused duplicate clients, resulting in an elevated client count. Post Vault 1.9, with the introduction of the deduplication logic, the number of clients created by the integration is reduced. For more information on improvements made to client count in Vault 1.9, refer to the question [Which version of Vault reflects the most accurate count of clients with Vault?](#q-which-vault-version-reflects-the-most-accurate-client-counts).
+
+### Q: Starting in Vault 1.9, Vault does not allow creating two aliases from the same auth mount under a single entity. What changed and how does this impact client counting?
+
+Prior to 1.9, customers could create more than one alias from the same auth mount under a single entity. However, we made a fix in Vault 1.9 to prevent this from occuring as a remediation for a security issue.
+
+This could potentially impact the number of clients generated for customers. For example, pre Vault 1.9, a customer (using K8s, with many namespaces and services accounts in the cluster) could perform entity management by mapping multiple service accounts under different namespaces under a single entity. By default, each service account will create a unique entity. Post Vault 1.9, they can only map a single K8s cluster to a single K8s auth mount.
+
+To provide further clarity, post Vault 1.9, if there was an auth mount per namespace, the customer could create an alias for a service account in namespace-1 and an alias for a service account in namespace-2 onto the same entity without issues. However, within namespace-1, which has a single auth mount, if they have a service account-1 and service account-2, both accounts cannot be aliased to a single client.
+
+### Q: How does mount migration impact the client count metric?
+
+In Vault 1.10, we made improvements to the [`sys/remount`](/api-docs/system/remount) API endpoint to simplify the complexities of moving data, such as secret engine and authentication method configuration from one mount to another, within a namespace or across namespaces. This can help with restructuring namespaces and mounts for various reasons, including [migrating mounts](/docs/concepts/mount-migration) from root to other namespaces when transitioning to using namespaces for the first time. To learn more, refer to the [Mount Move](https://learn.hashicorp.com/tutorials/vault/mount-move) tutorial.
+
+When migrating mounts, any aliases that refer to users on the auth mount could now point to an invalid mount when an auth mount is moved. Pointing to an invalid mount may not be the case for every instance; a remount within a namespace will end in the aliases pointing to a valid mount. Still, a remount across namespaces will always result in the aliases pointing to an invalid mount. In the latter case, the Vault operator should find and remove those aliases from the source namespace, and create equivalent aliases and entities for the new mount in the target namespace. If new entities and aliases aren’t created in the target namespace, Vault will dynamically generate them upon login operations.
+
+When migrating mounts within a namespace, client counts are not impacted.
+
+- For each existing alias on the source mount and its corresponding entity, if the new namespace has an entity corresponding to it, then you can assign a corresponding alias for the mount and the client count will not be impacted. For example, say I am a user `abc` on namespace `ns1/` and namespace `ns2/`, and I have an alias on the source mount in `ns1/`. After the move operation, I just need an alias added to the `abc` entity in `ns2/` for the target mount. This keeps the client counts the same.
+- For the example above, say I do not have an entity `abc` in `ns2/`. This will need to be created in order to make an alias associating it with the target mount. This increases the client count by 1.
+- Following the above example, let's say after the move operation, the entity `abc` in `ns1/` has no other aliases associated with it, i.e., the alias for the source mount was its only alias. At this point, we can clean up and remove the `abc` entity in `ns1/`. This will decrease the client count by 1, as the old entity may have already been used during the billing period.
+
+### Q: Vault 1.9 added support for providing custom user filters through the userfilter parameter. How does this affect client counts?
+
+Vault 1.9 added support for providing custom user filters through the [userfilter](/api-docs/auth/ldap#userfilter) parameter. This addition changed the way that entity alias gets mapped to an entity. Prior to Vault 1.9, alias names were always based on the [login username](/api-docs/auth/ldap#username-3) (which in turn is based on the value of the [userattr](/api-docs/auth/ldap#userattr)). In Vault 1.9, alias names no longer always map to the login username. Instead, the mapping depends on other config values as well, such as [updomain](/api-docs/auth/ldap#upndomain), [binddn](/api-docs/auth/ldap#binddn), [discoverydn](/api-docs/auth/ldap#discoverdn), and userattr.
+
+Vault 1.10 re-introduces the option to force the alias name to map to the login username with the optional parameter username_as_alias. Users that have the LDAP auth method enabled prior to Vault 1.9 may want to consider setting this to true to revert back to the old behavior. Otherwise, depending on the other aforementioned config values, logins may generate a new and different entity for an existing user that already had an entity associated in Vault. This in turn affects client counts since there may be more than one entity tied to this user. The username_as_alias flag will also be made available in subsequent Vault 1.8.1x and Vault 1.9.x releases to allow for this to be set prior to a Vault 1.10 upgrade.
website/content/docs/secrets/identity/oidc-provider.mdx+128 96
@@ -7,130 +7,162 @@ description: >-
# OIDC Identity Provider
-~> **Note:** This feature is currently a ***Tech Preview*** and not recommended
-for deployment in production.
-
-Vault as an OIDC identity provider allows clients speaking the OIDC protocol to
-take advantage of Vault's various authentication methods and source of
-identity. Clients can configure their authentication logic to talk to Vault.
-Once enabled, Vault will act as the bridge to identity providers via its
-existing authentication methods. Clients will also obtain identity information
-for their end-users by leveraging custom templating of Vault identity
-information. For more information on the configuration resources and OIDC endpoints,
+Vault is an OpenID Connect ([OIDC](https://openid.net/specs/openid-connect-core-1_0.html))
+identity provider. This enables client applications that speak the OIDC protocol to leverage
+Vault's source of [identity](/docs/concepts/identity) and wide range of [authentication methods](/docs/auth)
+when authenticating end-users. Client applications can configure their authentication logic
+to talk to Vault. Once enabled, Vault will act as the bridge to other identity providers via
+its existing authentication methods. Client applications can also obtain identity information
+for their end-users by leveraging custom templating of Vault identity information.
+
+-> **Note**: For more detailed information on the configuration resources and OIDC endpoints,
please visit the [OIDC provider](/docs/concepts/oidc-provider) concepts page.
-The Vault OIDC provider feature currently only supports the
-[authorization code flow](https://openid.net/specs/openid-connect-core-1_0.html#CodeFlowAuth).
-
-## OIDC Provider Configuration
+## Setup
The Vault OIDC provider system is built on top of the identity secrets engine.
This secrets engine is mounted by default and cannot be disabled or moved.
-Most secrets engines must be configured in advance before they can perform
-their functions. These steps are usually completed by an operator or
-configuration management tool.
+Each Vault namespace has a default OIDC [provider](/docs/concepts/oidc-provider#providers)
+and [key](/docs/concepts/oidc-provider#key). This built-in configuration enables client
+applications to begin using Vault as a source of identity with minimal configuration. For
+details on the built-in configuration and advanced options, see the [OIDC provider](/docs/concepts/oidc-provider)
+concepts page.
-1. Create a key that will be used to sign/verify ID tokens:
- ```text
- $ vault write identity/oidc/key/my-key \
- allowed_client_ids="xxAQWBYzD2WXsB8GiZqwq4jsUwfG0hJV" \
- verification_ttl="1h" \
- rotation_period="1h" \
- algorithm="RS256"
- Success! Data written to: identity/oidc/key/my-key
- ```
+The following steps show a minimal configuration that allows a client application to use
+Vault as an OIDC provider.
-1. Create an assignment. This specifies which Vault entities and groups are
-authorized to use a specific OIDC client for authentication flows:
+1. Enable a Vault auth method:
```text
- $ vault write identity/oidc/assignment/my-assignment \
- group_ids="b6ea7804-acbd-e866-7c51-0896456bd4bb" \
- entity_ids="aa786a7a-da2f-dca7-3680-0710771cca51"
- Success! Data written to: identity/oidc/assignment/my-assignment
+ $ vault auth enable userpass
+ Success! Enabled userpass auth method at: userpass/
```
-1. Create the 'user' custom scope:
+ Any Vault auth method may be used within the OIDC flow. For simplicity, enable the
+ `userpass` auth method.
+
+2. Create a user:
```text
- $ TOKEN_TEMPLATE=$(cat << EOF
- {
- "username": {{identity.entity.aliases.$MOUNT_ACCESSOR.name}},
- "contact": {
- "email": {{identity.entity.metadata.email}},
- "phone_number": {{identity.entity.metadata.phone_number}}
- },
- "groups": {{identity.entity.groups.names}}
- }
- EOF
- )
- $ vault write identity/oidc/scope/user \
- description="Scope for user metadata" \
- template="$(echo $TOKEN_TEMPLATE | base64 -)"
- Success! Data written to: identity/oidc/scope/user
+ $ vault write auth/userpass/users/end-user password="securepassword"
+ Success! Data written to: auth/userpass/users/end-user
```
-1. Create an OIDC client:
+ This user will authenticate to Vault through a client application, otherwise known as
+ an OIDC [relying party](https://openid.net/specs/openid-connect-core-1_0.html#Terminology).
+
+3. Create a client application:
```text
$ vault write identity/oidc/client/my-webapp \
- redirect_uris="http://127.0.0.1:8251/callback,http://127.0.0.1:8500/ui/oidc/callback" \
- assignments="my-assignment" \
- key="my-key" \
- id_token_ttl="30m" \
- access_token_ttl="1h"
+ redirect_uris="https://localhost:9702/auth/oidc-callback" \
+ assignments="allow_all"
Success! Data written to: identity/oidc/client/my-webapp
```
-1. Create an OIDC provider:
+ This operation creates a client application which can be used to configure an OIDC
+ relying party. See the [client applications](/docs/concepts/oidc-provider#client-applications)
+ section for details on different client types, including `confidential` and `public` clients.
- ```text
- $ vault write identity/oidc/provider/my-provider \
- allowed_client_ids="xxAQWBYzD2WXsB8GiZqwq4jsUwfG0hJV" \
- scopes_supported="user"
- Success! Data written to: identity/oidc/provider/my-provider
- ```
+ The `assignments` parameter limits the Vault entities and groups that are allowed to
+ authenticate through the client application. By default, no Vault entities are allowed.
+ To allow all Vault entities to authenticate, the built-in [allow_all](/docs/concepts/oidc-provider#assignments)
+ assignment is provided.
-1. Query the OIDC provider configuration:
+5. Read client credentials:
```text
- $ curl -s http://127.0.0.1:8200/v1/identity/oidc/provider/my-provider/.well-known/openid-configuration
- {
- "issuer": "http://127.0.0.1:8200/v1/identity/oidc/provider/my-provider",
- "jwks_uri": "http://127.0.0.1:8200/v1/identity/oidc/provider/my-provider/.well-known/keys",
- "authorization_endpoint": "http://127.0.0.1:8200/ui/vault/identity/oidc/provider/my-provider/authorize",
- "token_endpoint": "http://127.0.0.1:8200/v1/identity/oidc/provider/my-provider/token",
- "userinfo_endpoint": "http://127.0.0.1:8200/v1/identity/oidc/provider/my-provider/userinfo",
- "request_uri_parameter_supported": false,
- "id_token_signing_alg_values_supported": [
- "RS256",
- "RS384",
- "RS512",
- "ES256",
- "ES384",
- "ES512",
- "EdDSA"
- ],
- "response_types_supported": [
- "code"
- ],
- "scopes_supported": [
- "user",
- "openid"
- ],
- "subject_types_supported": [
- "public"
- ],
- "grant_types_supported": [
- "authorization_code"
- ],
- "token_endpoint_auth_methods_supported": [
- "client_secret_basic"
- ]
- }
+ $ vault read identity/oidc/client/my-webapp
+
+ Key Value
+ --- -----
+ access_token_ttl 24h
+ assignments [allow_all]
+ client_id GSDTnn3KaOrLpNlVGlYLS9TVsZgOTweO
+ client_secret hvo_secret_gBKHcTP58C4aq7FqPWsuqKgpiiegd7ahpifGae9WGkHRCwFEJTZA9KGdNVpzE0r8
+ client_type confidential
+ id_token_ttl 24h
+ key default
+ redirect_uris [https://localhost:9702/auth/oidc-callback]
```
+ The `client_id` and `client_secret` are the client application's credentials. These
+ values are typically required when configuring an OIDC relying party.
+
+6. Read OIDC discovery configuration:
+
+ ```text
+ $ curl -s http://127.0.0.1:8200/v1/identity/oidc/provider/default/.well-known/openid-configuration
+ {
+ "issuer": "http://127.0.0.1:8200/v1/identity/oidc/provider/default",
+ "jwks_uri": "http://127.0.0.1:8200/v1/identity/oidc/provider/default/.well-known/keys",
+ "authorization_endpoint": "http://127.0.0.1:8200/ui/vault/identity/oidc/provider/default/authorize",
+ "token_endpoint": "http://127.0.0.1:8200/v1/identity/oidc/provider/default/token",
+ "userinfo_endpoint": "http://127.0.0.1:8200/v1/identity/oidc/provider/default/userinfo",
+ "request_uri_parameter_supported": false,
+ "id_token_signing_alg_values_supported": [
+ "RS256",
+ "RS384",
+ "RS512",
+ "ES256",
+ "ES384",
+ "ES512",
+ "EdDSA"
+ ],
+ "response_types_supported": [
+ "code"
+ ],
+ "scopes_supported": [
+ "openid"
+ ],
+ "subject_types_supported": [
+ "public"
+ ],
+ "grant_types_supported": [
+ "authorization_code"
+ ],
+ "token_endpoint_auth_methods_supported": [
+ "none",
+ "client_secret_basic"
+ ]
+ }
+ ```
+
+ Each Vault OIDC provider publishes [discovery metadata](https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderMetadata).
+ The `issuer` value is typically required when configuring an OIDC relying party.
+
+## Usage
+
+After configuring a Vault auth method and client application, the following details can
+be used to configure an OIDC relying party to delegate end-user authentication to Vault.
+
+- `client_id` - The ID of the client application
+- `client_secret` - The secret of the client application
+- `issuer` - The issuer of the Vault OIDC provider
+
+A number of HashiCorp products provide OIDC authentication methods. This means that they
+can leverage Vault as a source of identity using the OIDC protocol. See the following links
+for details on configuring OIDC authentication for other HashiCorp products:
+
+- [Boundary](https://learn.hashicorp.com/tutorials/boundary/oidc-auth)
+- [Consul](https://www.consul.io/docs/security/acl/auth-methods/oidc)
+- [Waypoint](https://www.waypointproject.io/docs/server/auth/oidc)
+
+Otherwise, refer to the documentation of the specific OIDC relying party for usage details.
+
+## Supported Flows
+
+The Vault OIDC provider feature currently supports the following authentication flow:
+
+- [Authorization Code Flow](https://openid.net/specs/openid-connect-core-1_0.html#CodeFlowAuth).
+
+## Learn
+
+Refer to the [Vault as an OIDC Identity Provider](https://learn.hashicorp.com/tutorials/vault/oidc-identity-provider)
+guide for an advanced tutorial on configuring HashiCorp [Boundary](https://www.boundaryproject.io/)
+to leverage Vault as a source of identity using the OIDC protocol.
+
## API
The Vault OIDC provider feature has a full HTTP API. Please see the
actions-packaging-linux@v1 (#14642)
.github/workflows/build.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
website/content/docs/release-notes/1.10.mdx+151 0
@@ -0,0 +1,151 @@
+---
+layout: docs
+page_title: 1.10
+description: |-
+ This page contains release notes for Vault 1.10
+---
+
+# Vault 1.10 Release notes
+
+**Software Release date:** Mar 23, 2022
+
+**Summary:** Vault version 1.10 offers features and enhancements that improve the user experience while closing the loop on key issues previously encountered by our customers. We are providing a summary of these improvements in these release notes.
+
+We encourage you to upgrade to the latest release to take advantage of the new benefits that we are providing. Additionally, with this latest release, we offer solutions to critical feature gaps that have been identified previously. For further information on product improvements, including a comprehensive list of bug fixes, please refer to the [Changelog](https://github.com/hashicorp/vault/blob/main/CHANGELOG.md) within the Vault 1.10 release.
+
+Some of these enhancements and changes in this release include:
+
+- Ability to view client counts per auth and changes to clients over months, therefore, providing more granular visibility into clients.
+- Extended the `sys/remount` API endpoint to support moving secrets engines and auth method mounts from one location to another, within a namespace or across namespaces.
+- Improved security posture that includes MFA on login for Vault OSS customers.
+- Ability to implicitely achieve consistency via tokens.
+- Support of PKCE on Vault’s OIDC auth method with Telemetry support for the Vault Agent.
+- Improvement of key areas and parity to support using Terraform Provider with Vault.
+
+## New Features
+
+This section describes the new features introduced as part of Vault 1.10
+
+### Multi-Factor Authentication (MFA) for Vault OSS
+
+Vault has had support for the [Step-up Enterprise MFA](docs/enterprise/mfa) as part of its Enterprise edition. The Step-up Enterprise MFA allows having an MFA on login, or for step-up access to sensitive resources in Vault.
+
+With Vault 1.10, MFA as part of [login](/docs/auth.login-mfa) is now supported for Vault OSS. This demonstrates HashiCorp’s thought leadership in security and its continued endeavor to enable all Vault users to employ strong security policies with Vault.
+
+~> **Note:** The Legacy MFA in Vault OSS is a [deprecated](https://www.vaultproject.io/docs/deprecation) feature and will be removed in Vault 1.11.
+
+Refer to the [Login MFA FAQ](/auth/login-mfa/faq) to understand the various MFA workflows that are supported in Vault 1.10.
+
+### Vault OIDC provider with PKCE support
+
+Vault’s support to act as an OIDC provider is now generally available. Furthermore, Vault’s OIDC provider functionality can now support PKCE for authorization code flow as well. Thanks to all the excellent community feedback received, we have simplified the user experience around configuration of OIDC provider functionality.
+
+### Caching support for Vault Lambda Extension
+
+With 0.6.0, Vault Lambda Extension supports [caching](https://github.com/hashicorp/vault-lambda-extension#caching) in the local proxy server to avoid proxying every request to enable setting expiry time and invalidate cache, as needed.
+
+### Terraform Provider for Vault
+
+We have introduced three new resources to enable configuration of the [KMIP secrets engine](https://registry.terraform.io/providers/hashicorp/vault/latest/docs/resources/kmip_secret_backend) using the Terraform Provider for Vault. In addition, frequent releases on the Terraform Provider for Vault have been incorporating the ability to configure newer resources and data sources. Please read the [documentation](https://registry.terraform.io/providers/hashicorp/vault/latest/docs) for more details.
+
+### KV Secrets Engine v2 patch operations
+
+We now support an additional method for managing [KV v2 secrets](/api-docs/secret/kv/kv-v2) to maintain least privilege security in certain types of automated environments. This feature creates a new PATCH capability that enables partial updates to KV v2 secrets without requiring the READ privilege to the entire endpoint for an entity.
+
+### DB2 Dynamic Secrets support
+
+Vault operators can leverage the openldap secrets engine to manage credentials for IBM DB2 and the LDAP security plugin for Db2. This allows Db2 to offload authentication and authorization to the LDAP security plugin and allows Vault to manage static credentials or even generate dynamic users. For more details, refer to the For more details, refer to the [IBM Db2 Credentials Management](https://learn.hashicorp.com/tutorials/vault/ibm-db2-openldap) tutorial.
+
+### Temporal Transit Key rotation
+
+Proper key management includes occasionally rotating encryption keys to reduce the risks of a nonce reuse and opportunities for keys to be compromised. Previously, there was no automated way to rotate keys that is native to Vault. Now, we have provided a new configuration element on transit keys and tokenization transform configurations where a time interval triggers the keys to automatically rotate after the interval has lapsed.
+
+### PKI HSM Forwarding
+
+To address security and compliance needs, customers may require that keys be either created or stored within Hardware Security Models (HSMs). Vault 1.10 introduces an accommodation for this requirement with regards to the PKI Secrets Engine. We now support offloading selected PKI operations to HSMs, in particular allowing customers to both generate new PKI key pairs and sign/verify some certificate workflows. All of these operations are conducted in a way that never allows the private key material to leave the secure confines of the HSM itself.
+
+### AWS and AKV KMS Forwarding
+
+The work done above to support HSM-backed PKI operations inspired us to consider what other key possession paradigms we could support. This led us to extend the implementation to support Cloud Key Management Systems in addition to HSMs. In Vault 1.10, users may generate new PKI pairs and perform sign/verify certificate workflows, all with those keys never leaving the cloud KMS itself. Vault 1.10 provides support for AWS Key Management Service and Azure Key Vault Key Management Service.
+
+### Server Side Consisten Tokens
+
+Vault’s [eventual consistency](/docs/enterprise/consistency) model precludes read-after-write guarantees when clients interact with performance standbys or performance replication clusters. The [Client Controlled Consistency](/docs/enterprise/consistency#vault-1-7-mitigations) mitigations supported with Vault 1.7 provide ways to achieve consistency through client modifications or by using the agent for proxied requests, which is not possible in all cases. The Server Side Consistent Tokens feature provides an implicit way to achieve consistency by embedding the minimum Write-Ahead-Log state information in the Service tokens returned from logins or token-create requests. This feature introduces changes in the token format and the new tokesn will be the default tokens starting in Vault 1.10. Vault 1.10 is backwards compatible with old tokens.
+
+See [Replication](/docs/configuration/replication), [Vault Eventual Consistency](/docs/enterprise/consistency), [Upgrade to 1.10](/docs/upgrading/upgrade-to-1.10.x) and [Service Side Consistent Token FAQ](/docs/faq/ssct) to understand the various consistency options available with Vault 1.10 and the considerations to be aware of prior to selecting an option for your use case.
+
+## Vault Agent Features
+
+### Support for Telemetry
+
+Starting with Vault 1.10, the Vault Agent supports a new metrics endpoint and [Telemetry](/docs/agent#telemetry-stanza) metrics around run time, authentication success, authentication failures, cache hits, cache misses, proxy succes, and proxy client errors. This Vault Agent Telemetry should greatly help with the retrieval of key operational insights for Vault Agent deployments.
+
+### User-assigned managed identities for auto auth in Azure
+
+With this [enhancement](/docs/agent/autoauth/methods/azure), users can specify user-assigned managed identities via the `object_id` and `client_id` when configuring Vault agent auto-auth for Azure. This enables users that have more than one user-assigned managed identity associated with their VM to specify which one they'd like to use when authenticating via the Vault's Azure auth method. Note that providing these parameters is an "exclusive or" operation.
+
+### Quit API endpoint with config
+
+Previously, for instances where the Agent is a sidecar in a Kubernetes job and the job hangs, you must either use `shareProcessNamespace: true` for the container so that the process kill signals can be sent, or avoid the sidecar container entirely and solely rely on an init container. With this [enhancement](/docs/agent#quit), we have added support for a Quit API endpoint to automatically shut down the Vault Agent, therefore eliminating the need to perform the workarounds.
+
+## Other Features and Enhancements
+
+This section describes other features and enhancements introduced as part of the Vault 1.10 release.
+
+### Client Count improvements
+
+We have introduced auth mount-based attribution of clients to help better understand where clients are being used within a cluster. This is available via UI and API. This is an enhancement on top of the namespace attribution capability we introduced in Vault 1.9.
+
+We have also introduced the ability to view changes to clients month over month via the client count API, and made other UI enhancements. Refer to [What is a Client?](/docs/concepts/client-count) and [Client Count FAQ](/docs/concepts/client-count/faq) for more details.
+
+### Mount Migration
+
+We have made improvements to the `sys/remount` API endpoint to simplify the complexities of moving data, such as secret engine and authentication method configuration from one mount to another, within a namespace or across namespaces. This can help with restructuring namespaces and mounts for various reasons, including migrating mounts from root to other namespaces when transitioning to using namespaces for the first time. For step-by-step instructions, refer to the [Mount Move](https://learn.hashicorp.com/tutorials/vault/mount-move) tutorial.
+
+### Scaling External Database plugins
+
+Database plugins can now implement [plugin multiplexing](/docs/internals/plugins#plugin-development) which allows a single plugin process to be used for multiple database connections. Database plugin multiplexing will be enabled on the Oracle Database plugin starting in v0.6.0. We will extend this functionality to additional database plugins in subsequent releases.
+
+Any external database plugins that want to adopt multiplexing support will have to update their main.go call from [dbplugin.Serve()](https://github.com/hashicorp/vault/blob/sdk/v0.4.1/sdk/database/dbplugin/v5/plugin_server.go#L13) to [dbplugin.ServeMultiplex()](https://github.com/hashicorp/vault/blob/sdk/v0.4.1/sdk/database/dbplugin/v5/plugin_server.go#L42). Multiplexable database plugins are compatible with older versions of Vault down to Vault 1.6. Refer to this [Oracle Database PR](https://github.com/hashicorp/vault-plugin-database-oracle/pull/74) as an example of the upgrade process.
+
+### Consul Secrets Engine enhancements
+
+Consul has supported [namespace](https://www.consul.io/docs/enterprise/namespaces), [admin partitions](https://www.consul.io/docs/enterprise/admin-partitions) and [ACL roles](https://www.consul.io/commands/acl/role) for some time now. In this release we have added enhancements to the Consul Secrets engine to support [namespace]() awareness and add admin partition and role support for Consul ACL tokens. This significantly simplifies the integrations for customers who want to achieve a zero trust security posture with both Vault and Consul.
+
+### Using sessionStorage instead of localStorage for the Vault UI
+
+Prior to Vault 1.10, the Vault UI used localStorage to store authentication information. The data in localStorage was persisted in browsers and removed only on demand. Now, we have switched the Vault UI to use sessionStorage instead, which ensures that the authentication information is stored in the current browser tab alone, thereby improving security.
+
+### Advanced I/O Handling for Transform FPE
+
+The Transform Secrets Engine allows users to securely encrypt data while providing control over the output format. In Vault 1.9, we introduced [additional format fields](/docs/release-notes/1.9.0#advanced-i-o-handling-for-tranform-fpe-adp-transform) on the templates used for this workflow. In Vault 1.10, we have now added those two new fields, `encode_format` and `decode_format`, to the Create Template page on the UI under Advanced Templating.
+
+## Breaking changes
+
+The following section details breaking changes introduced in Vault 1.10.
+
+### LDAP auth method entity alias mapping
+
+In Vault 1.9, we added support to provide custom user filters through the [userfilter](/api-docs/auth/ldap#userfilter) parameter. This support changed the way that entity alias was mapped to an entity. Prior to Vault 1.9, alias names were always based on the [login username](/api-docs/auth/ldap#username-3) (which in turn is based on the value of the [userattr](/api-docs/auth/ldap#userattr)). In Vault 1.9, alias names no longer mapped to the login username. Instead, the mapping depends on other config values as well, such as [updomain](/api-docs/auth/ldap#upndomain), [binddn](/api-docs/auth/ldap#binddn), [discoverydn](/api-docs/auth/ldap#discoverdn), and [userattr](/api-docs/auth/ldap#userattr).
+
+With Vault 1.10, we re-introduced the option to force the alias name to map to the login username with the optional parameter username_as_alias. Users that have the LDAP auth method enabled prior to Vault 1.9 may want to consider setting this to true to revert back to the old behavior. Otherwise, depending on the other aforementioned config values, logins may generate a new and different entity for an existing user with a previous entity associated in Vault. This in turn affects client counts since there may be more than one entity tied to this user. The username_as_alias flag was also made available in subsequent Vault 1.8.x and Vault 1.9.x releases to allow for this to be set prior to a Vault 1.10 upgrade.
+
+## Known issues
+
+### Single Vault follower restart causes election even with established quorum
+
+We now support Server Side Consistent Tokens (See [Replication](/docs/configuration/replication), [Vault Eventual Consistency](/docs/enterprise/consistency), and [Upgrade to 1.10](/docs/upgrading/upgrade-to-1.10.x).), which introduces a new token format that can only be used on nodes of 1.10 or higher version. This new format is enabled by default upon upgrading to the new version. Old format tokens can be read by Vault 1.10, but the new format Vault 1.10 tokens cannot be read by older Vault versions.
+
+For more details, see the [Server Side Consistent Tokens FAQ](/docs/faq/ssct).
+
+Since service tokens are always created on the leader, as long as the leader is not upgraded before performance standbys, service tokens will be of the old format and still be usable during the upgrade process. However, the usual upgrade process we recommend can't be relied upon to always upgrade the leader last. Due to this known [issue](https://github.com/hashicorp/vault/issues/14153), a Vault cluster using Integrated Storage may result in a leader not being upgraded last, and this can trigger a re-election. This re-election can cause the upgraded node to become the leader, resulting in the newly created tokens on the leader to be unusable on nodes that have not yet been upgraded. Note that this issue does not impact Vault OSS users.
+
+We will have a fix for this issue in Vault 1.10.1. Until this issue is fixed, you may be at risk of having performance standbys unable to service requests until all nodes are upgraded. We recommended that you plan for a maintenance window to upgrade.
+
+### Limited policy shows unhelpful message in UI after mounting a secret engine
+
+When a user has a policy that allows creating a secret engine but not reading it, after successful creation, the user sees a message `n is undefined` instead of a permissions error. We will have a fix for this issue in an upcoming minor release.
+
+## Feature Deprecations and EOL
+
+Please refer to the [Deprecation Plans and Notice](/docs/deprecation) page for up-to-date information on feature deprecations and plans. An [Feature Deprecation FAQ](/deprecation/faq) page is also available to address questions concerning decisions made about Vault feature deprecations.
website/content/docs/concepts/oidc-provider.mdx+73 21
@@ -7,24 +7,25 @@ description: >-
# OIDC Provider
-~> **Note:** This feature is currently a ***Tech Preview*** and not recommended for deployment in production.
-
-
-This document describes how Vault can be an **OpenID Connect (OIDC) identity provider** by enabling applications to leverage Vault as a source of identity using the OIDC protocol.
-
-This feature allows clients speaking the OIDC protocol to take advantage of Vault's various authentication methods and source of identity. Clients can configure their authentication logic to talk to Vault. Once enabled, Vault will act as the bridge to identity providers via its existing authentication methods. Clients will also obtain identity information for their end-users by leveraging custom templating of Vault identity information.
-
-Vault as an OIDC provider allows mutual Vault and Boundary customers to leverage Vault's identity system to delegate authentication and authorization to Vault. Vault, therefore, acts as an identity provider for Boundary. Other HashiCorp products such as Consul can also leverage Vault's identity system and provide delegated authentication and authorization to its users. Having Vault as an OIDC provider allows a single sign-on experience to their end-users for organizations that want to leverage Vault as an identity provider.
+This document provides conceptual information about the Vault **OpenID Connect (OIDC) identity
+provider** feature. This feature enables client applications that speak the OIDC protocol to
+leverage Vault's source of [identity](/docs/concepts/identity) and wide range of [authentication methods](/docs/auth)
+when authenticating end-users. For more information about the usage of Vault's OIDC provider,
+refer to the [OIDC identity provider](/docs/secrets/identity/oidc-provider) documentation.
## Configuration Options
The next few sections of the document provide implementation details for each resource that permits Vault configuration as an OIDC identity provider.
-### Providers
+### OIDC Providers
+
+Each Vault namespace will contain a built-in provider resource named `default`. The `default`
+provider will allow all client applications within the namespace to use it for OIDC flows.
+The `default` provider can be modified but not deleted.
-A Vault namespace may contain several provider resources. Each configured provider will publish the APIs listed within the OIDC flow. The APIs will be served via backend path-based routing on Vault's listen [address](/docs/configuration/listener/tcp#address).
+Additionally, a Vault namespace may contain several provider resources. Each configured provider will publish the APIs listed within the [OIDC flow](/docs/concepts/oidc-provider#oidc-flow) section. The APIs will be served via backend path-based routing on Vault's listen [address](/docs/configuration/listener/tcp#address).
-A provider must have the following configuration parameters:
+A provider has the following configuration parameters:
* **Issuer URL**: used in the `iss` claim of ID tokens
* **Allowed client IDs**: limits which clients can access the provider
@@ -32,7 +33,9 @@ A provider must have the following configuration parameters:
The issuer URL parameter is necessary for the validation of ID tokens by clients. If an URL parameter is not provided explicitly, it will default to a URL with Vault's [api_addr](/docs/configuration#api_addr) as the `scheme://host:port` component and `/v1/:namespace/identity/oidc/provider/:name` as the path component. This means tokens issued by a provider in a specified Vault cluster must be validated within that same cluster. If the issuer URL is provided explicitly, it must point to a Vault instance that is network-reachable by clients for ID token validation.
-The allowed client IDs parameter utilizes the list of client IDs that have been generated by Vault as a part of client registration. By default, all clients will be *disallowed*. Providing an asterisk(*) as the parameter value will allow all clients to use the provider. The scopes parameter employs a list of references to named scope resources. The values provided are discoverable by the `scopes_supported` key in the OIDC discovery document of the provider. By default, a provider will have the `openid` scope available. See the scopes section below for more details on the `openid` scope.
+The allowed client IDs parameter utilizes the list of client IDs that have been generated by Vault as a part of client registration. By default, all clients will be *disallowed*. Providing an asterisk(*) as the parameter value will allow all clients to use the provider.
+
+The scopes parameter employs a list of references to named scope resources. The values provided are discoverable by the `scopes_supported` key in the OIDC discovery document of the provider. By default, a provider will have the `openid` scope available. See the scopes section below for more details on the `openid` scope.
### Scopes
@@ -85,7 +88,7 @@ Several named scopes can be made available on an individual provider. Note that
The `openid` scope is a unique case scope that may not be modified or deleted. The scope will exist in Vault and supported by each provider by default. The scope represents the minimum set of claims required by the OIDC specification for inclusion in ID tokens. As such, templates may not contain top-level keys that overwrite the claims populated by the openid scope.
-The following defines the claims key and value mapping for the openid scope:
+The following defines the claims key and value mapping for the `openid` scope:
* `iss`- configured issuer of the provider
* `sub`- unique entity ID of the Vault user
@@ -93,31 +96,76 @@ The following defines the claims key and value mapping for the openid scope:
* `iat`- time of token issue
* `exp`- time of token issue + ID token TTL
-### Client registration
+### Client Applications
-A client resource allows the relying party to [dynamically register](https://openid.net/specs/openid-connect-registration-1_0.html) by providing metadata about itself to Vault.
+A client resource represents an application that wants to delegate end-user authentication
+to Vault using the OIDC protocol. The information provided by a client resource can be used
+to configure an OIDC [relying party](https://openid.net/specs/openid-connect-core-1_0.html#Terminology).
-The client must have the following configuration parameters:
+A client has the following configuration parameters:
* **Redirect URIs**: limits the valid redirect URIs in an authentication request
-* **Assignments**: determines who can authenticate with the client
+* **Assignments**: determine who can authenticate with the client
* **Key**: used to sign the ID tokens
* **ID token TTL**: specifies the time-to-live for ID tokens
-* **Access token TTL**: establishes the time-to-live for access tokens
+* **Access token TTL**: specifies the time-to-live for access tokens
+* **Client type**: determines the client's ability to maintain confidentiality of credentials
-A `client_id` and `client_secret` are generated and returned after a successful client registration. Their values are strings using the base62 character set. The `client_id` will have 32 characters, and the `client_secret` will have a prefix of `hvo_secret`. The `client_id` uniquely identifies the client. The `client_secret` will be used to authenticate to the token endpoint as described in [client authentication](https://openid.net/specs/openid-connect-core-1_0.html#ClientAuthentication).
+The `key` parameter is optional. The key will be used to sign ID tokens for the client.
+It cannot be modified after creation. If not supplied, defaults to the built-in
+[default key](/docs/concepts/oidc-provider#keys).
-The `key` parameter is required. The user must create a `key` as a required parameter of the client configuration.
+A `client_id` is generated and returned after a successful client registration. The
+`client_id` uniquely identifies the client. Its value will be a string with 32 random
+characters from the base62 character set.
~> **Note**: At least one of the redirect URIs of a client must exactly match the `redirect_uri` parameter used in an authentication request initiated by the client.
+#### Client Types
+
+A client resource has a `client_type` parameter which specifies the OAuth 2.0
+[client type](https://datatracker.ietf.org/doc/html/rfc6749#section-2.1) based on
+its ability to maintain confidentiality of credentials. The following sections detail
+the differences between confidential and public clients in Vault.
+
+##### Confidential
+
+Confidential clients are capable of maintaining the confidentiality of their credentials.
+Confidential clients have a `client_secret`. The `client_secret` will have a prefix of
+`hvo_secret` followed by 64 random characters in the base62 character set.
+
+Confidential clients may use Proof Key for Code Exchange ([PKCE](https://datatracker.ietf.org/doc/html/rfc7636))
+during the authorization code flow.
+
+Confidential clients must authenticate to the token endpoint using the
+`client_secret_basic` [client authentication method](https://openid.net/specs/openid-connect-core-1_0.html#ClientAuthentication).
+
+##### Public
+
+Public clients are not capable of maintaining the confidentiality of their credentials.
+As such, public clients do not have a `client_secret`.
+
+Public clients must use Proof Key for Code Exchange ([PKCE](https://datatracker.ietf.org/doc/html/rfc7636))
+during the authorization code flow.
+
+Public clients use the `none` [client authentication method](https://openid.net/specs/openid-connect-core-1_0.html#ClientAuthentication).
+
### Assignments
Assignment resources are referenced by clients via the `assignments` parameter. This parameter limits the set of Vault users allowed to authenticate. The assignments of an associated client are validated during the authentication request, ensuring that the Vault identity associated with the request is a member of the assignment's entities or groups.
+Each Vault namespace will contain a built-in assignment resource named `allow_all`. The
+`allow_all` assignment allows all Vault entities to authenticate through a client. The
+`allow_all` assignment cannot be modified or deleted.
+
### Keys
-Key resources are referenced by clients via the key parameter. This parameter specifies the key that will be used to sign ID tokens for the client. See existing [documentation](/api-docs/secret/identity/tokens#create-a-named-key) for details on keyring management, supported signing algorithms, rotation periods, and verification TTLs. Currently, a key referenced by a client cannot be changed.
+Key resources are referenced by clients via the `key` parameter. This parameter specifies the key that will be used to sign ID tokens for the client. See existing [documentation](/api-docs/secret/identity/tokens#create-a-named-key) for details on keyring management, supported signing algorithms, rotation periods, and verification TTLs. Currently, a key referenced by a client cannot be changed.
+
+Each Vault namespace will contain a built-in key resource named `default`. Clients that don't
+specify the `key` parameter at creation time will use the `default` key. The `default` key
+will use the `RS256` signing algorithm, allow all client IDs, and have rotation and verification
+TTLs of `24h`. The `default` key can be modified but not deleted.
## OIDC flow
@@ -127,6 +175,10 @@ The following sections provide implementation details for the OIDC compliant API
Vault OIDC providers enable registered clients to authenticate and obtain identity information (or "claims") for their end-users. They do this by providing the APIs and behavior required to satisfy the OIDC specification for the [authorization code flow](https://openid.net/specs/openid-connect-core-1_0.html#CodeFlowAuth). All clients are treated as first-party. This means that end-users will not be required to provide consent to the provider as detailed in section [3.1.2.4](https://openid.net/specs/openid-connect-core-1_0.html#Consent) of the OIDC specification. The provider will release information to clients as long as the end-user has ACL access to the provider and their identity has been authorized via an assignment.
+Vault OIDC providers implement Proof Key for Code Exchange ([PKCE](https://datatracker.ietf.org/doc/html/rfc7636))
+to mitigate authorization code interception attacks. PKCE is required for `public` client types
+and optional for `confidential` client types.
+
### OpenID configuration
Each provider offers an unauthenticated endpoint that facilitates OIDC Discovery. All required metadata listed in [OpenID Provider Metadata](https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderMetadata) is included in the discovery document. Additionally, the recommended `userinfo_endpoint` and `scopes_supported` metadata are included.
website/content/docs/secrets/databases/custom.mdx+78 17
@@ -3,7 +3,7 @@ layout: docs
page_title: Custom - Database - Secrets Engines
description: |-
The database secrets engine allows new functionality to be added through a
- plugin interface without needing to modify vault's core code. This allows you
+ plugin interface without needing to modify Vault's core code. This allows you
write your own code to generate credentials in any database you wish. It also
allows databases that require dynamically linked libraries to be used as
plugins while keeping Vault itself statically linked.
@@ -19,18 +19,25 @@ for more details.
~> **Advanced topic!** Plugin development is a highly advanced topic in Vault,
and is not required knowledge for day-to-day usage. If you don't plan on writing
-any plugins, we recommend not reading this section of the documentation.
+any plugins, feel free to skip this section of the documentation.
The database secrets engine allows new functionality to be added through a
-plugin interface without needing to modify vault's core code. This allows you
+plugin interface without needing to modify Vault's core code. This allows you
write your own code to generate credentials in any database you wish. It also
allows databases that require dynamically linked libraries to be used as plugins
while keeping Vault itself statically linked.
-Please read the [Plugins internals](/docs/internals/plugins) docs for more
+Please read the [Plugins internals](/docs/plugins) docs for more
information about the plugin system before getting started building your
Database plugin.
+Database plugins can be made to implement
+[plugin multiplexing](/docs/plugins/plugin-architecture#plugin-multiplexing)
+which allows a single plugin process to be used for multiple database
+connections. To enable multiplexing, the plugin must be compiled with the
+`ServeMultiplex` function call from Vault's `dbplugin` package.
+
+
## Plugin Interface
All plugins for the database secrets engine must implement the same interface. This interface
@@ -100,12 +107,19 @@ the configuration is valid and able to connect to the database in question. If t
false, no connection should be made during the `Initialize` call, but subsequent calls to the
other functions will need to open a connection.
-## Serving your plugin
+## Serving A Plugin
+
+### Serving A Plugin with Multiplexing
+
+~> Plugin multiplexing requires `github.com/hashicorp/vault/sdk v0.4.0` or above.
+
+The plugin runs as a separate binary outside of Vault, so the plugin itself
+will need a `main` function. Use the `ServeMultiplex` function within
+`sdk/database/dbplugin/v5` to serve your multiplexed plugin. You will also need
+to pass some TLS configuration information that Vault uses when initializing
+the plugin.
-The plugin runs as a separate binary outside of Vault, so the plugin itself will need a `main`
-function. Use the `Serve` function within `sdk/database/dbplugin/v5` to serve your plugin. You
-will also need to pass some TLS configuration information that Vault uses when initializing the
-plugin. Below is an example setup:
+Below is an example setup:
```go
package main
@@ -128,12 +142,7 @@ func main() {
}
func Run() error {
- dbType, err := New()
- if err != nil {
- return err
- }
-
- dbplugin.Serve(dbType.(dbplugin.Database))
+ dbplugin.ServeMultiplex(dbType.(dbplugin.New))
return nil
}
@@ -172,6 +181,29 @@ func (db *MyDatabase) secretValues() map[string]string {
Replacing `MyDatabase` with the actual implementation of your database plugin.
+### Serving A Plugin without Multiplexing
+
+Serving a plugin without multiplexing requires calling the `Serve` function
+from `sdk/database/dbplugin/v5` to serve your plugin. You will also need to
+pass some TLS configuration information that Vault uses when initializing the
+plugin.
+
+The setup is exactly the same as the multiplexed case above, except for the
+`Run` function:
+
+```go
+func Run() error {
+ dbType, err := New()
+ if err != nil {
+ return err
+ }
+
+ dbplugin.Serve(dbType.(dbplugin.Database))
+
+ return nil
+}
+```
+
## Running your plugin
The above main package, once built, will supply you with a binary of your
@@ -179,7 +211,7 @@ plugin. We also recommend if you are planning on distributing your plugin to
build with [gox](https://github.com/mitchellh/gox) for cross platform builds.
To use your plugin with the database secrets engine you need to place the binary in the
-plugin directory as specified in the [plugin internals](/docs/internals/plugins) docs.
+plugin directory as specified in the [plugin internals](/docs/plugins) docs.
You should now be able to register your plugin into the vault catalog. To do
this your token will need sudo permissions.
@@ -200,7 +232,36 @@ $ vault write database/config/mydatabase \
myplugins_connection_details="..."
```
-## Upgrading database plugins
+## Upgrading database plugins to leverage plugin multiplexing
+
+### Background
+
+Scaling many external plugins can become resource intensive. To address
+performance problems with scaling external plugins, database plugins can be
+made to implement [plugin multiplexing](/docs/plugins/plugin-architecture#plugin-multiplexing)
+which allows a single plugin process to be used for multiple database
+connections. To enable multiplexing, the plugin must be compiled with the
+`ServeMultiplex` function call from Vault's `dbplugin` package.
+
+### Upgrading your database plugin to leverage plugin multiplexing
+
+There is only one step required to upgrade from a non-multiplexed to a
+multiplexed database plugin: Change the `Serve` function call to `ServeMultiplex`.
+
+This will run the RPC server for the plugin just as before. However, the
+`ServeMultiplex` function takes the factory function directly as its argument.
+This factory function is a function that returns an object that implements the
+[`dbplugin.Database` interface](/docs/secrets/databases/custom#plugin-interface).
+
+### When should plugin multiplexing be avoided?
+
+Some use cases that should avoid plugin multiplexing might include:
+
+* Plugin process level separation is required
+* Avoiding restart across all mounts/database connections for a plugin type on
+ crashes or plugin reload calls
+
+## Upgrading database plugins to the V5 interface
### Background
website/content/docs/plugins/plugin-architecture.mdx+190 0
@@ -0,0 +1,190 @@
+---
+layout: docs
+page_title: External Plugin Architecture
+description: Learn about Vault's plugin architecture.
+---
+
+# External Plugin Architecture
+
+Vault's external plugins are completely separate, standalone applications that Vault
+executes and communicates with over RPC. This means the plugin process does not
+share the same memory space as Vault and therefore can only access the
+interfaces and arguments given to it. This also means a crash in a plugin can not
+crash the entirety of Vault.
+
+It is possible to enable a custom plugin with a name that's identical to a
+built-in plugin. In such a situation, Vault will always choose the custom plugin
+when enabling it.
+
+## External Plugin Lifecycle
+
+Vault external plugins are long-running processes that remain running once they are
+spawned by Vault, the parent process. Plugin processes can be started by Vault's
+active node and performance standby nodes. Additionally, there are cases where
+plugin processes may be terminated by Vault. These cases include, but are not
+limited to:
+
+- Vault active node step-down
+- Vault barrier seal
+- Vault graceful shutdown
+- Disabling a Secrets Engine or Auth method that uses external plugins
+- Database configured connection deletion
+- Database configured connection update
+- Database configured connection reset request
+- Database root credentials rotation
+- WAL Rollback from a previously failed root credentials rotation operation
+
+The lifecycle of plugin processes are managed automatically by Vault.
+Termination of these processes are typical in certain scenarios, such as the
+ones listed above. Vault will start plugin processes when needed, typically by
+lazily loading the plugin when a request that requires the plugin is received by
+Vault. A plugin process may be started or terminated through other internal
+processes within Vault as well. Since Vault manages and tracks the lifecycle of
+its plugins, these processes should not be terminated by anything other than
+Vault.
+
+### External Plugin Scaling Characteristics
+
+External plugins are able to leverage [Performance Standbys](/docs/enterprise/performance-standby)
+without any explicit action by a plugin author. The default behavior of Vault
+Enterprise is to attempt to handle all requests, including requests to plugins,
+on performance standbys. If the plugin request makes any attempt to modify
+storage, the request will receive a readonly error, and the request routing
+code will then forward the request to the active node. In other words, plugins
+can scale horizontally on Vault Enterprise without any effort on the plugin
+author's part.
+
+## Plugin Communication
+
+Vault creates a mutually authenticated TLS connection for communication with
+the plugin's RPC server. Database secrets engines make use of the AutoMTLS
+feature of [go-plugin](https://www.github.com/hashicorp/go-plugin) which will
+automatically negotiate mTLS for transport authentication. For all other
+plugins, Vault passes a [wrapping token](/docs/concepts/response-wrapping) to
+the plugin process' environment. This token is single use and has a short TTL.
+Once unwrapped, it provides the plugin with a uniquely generated TLS
+certificate and private key for it to use to talk to the original Vault
+process.
+
+The [`api_addr`](/docs/configuration#api_addr) must be set in order for the
+plugin process to establish communication with the Vault server during mount
+time. If the storage backend has HA enabled and supports automatic host address
+detection (e.g. Consul), Vault will automatically attempt to determine the
+`api_addr` as well.
+
+~> Note: Prior to Vault version 1.9.2, reading the original connection's TLS
+connection state is not supported in plugins.
+
+## Plugin Registration
+
+An important consideration of Vault's plugin system is to ensure the plugin
+invoked by Vault is authentic and maintains integrity. There are two components
+that a Vault operator needs to configure before external plugins can be run- the
+plugin directory and the plugin catalog entry.
+
+### Plugin Directory
+
+The plugin directory is a configuration option of Vault and can be specified in
+the [configuration file](/docs/configuration).
+This setting specifies a directory in which all plugin binaries must live;
+_this value cannot be a symbolic link_. A plugin
+cannot be added to Vault unless it exists in the plugin directory. There is no
+default for this configuration option, and if it is not set, plugins cannot be
+added to Vault.
+
+~> Warning: A Vault operator should take caution and lock down the permissions on
+this directory to ensure a plugin cannot be modified by an unauthorized user
+between the time of the SHA check and the time of plugin execution.
+
+### Plugin Catalog
+
+The plugin catalog is Vault's list of approved plugins. The catalog is stored in
+Vault's barrier and can only be updated by a Vault user with sudo permissions.
+Upon adding a new plugin, the plugin name, SHA256 sum of the executable, and the
+command that should be used to run the plugin must be provided. The catalog will
+ensure the executable referenced in the command exists in the plugin
+directory. When added to the catalog, the plugin is not automatically executed,
+but becomes visible to backends and can be executed by them. For more
+information on the plugin catalog please see the [Plugin Catalog API
+docs](/api-docs/system/plugins-catalog).
+
+An example of plugin registration in current versions of Vault:
+
+```shell-session
+$ vault plugin register -sha256=<SHA256 Hex value of the plugin binary> \
+ secret \ # type
+ myplugin-database-plugin
+
+Success! Registered plugin: myplugin-database-plugin
+```
+
+Vault versions prior to v0.10.4 lacked the `vault plugin` operator and the
+registration step for them is:
+
+```shell-session
+$ vault write sys/plugins/catalog/database/myplugin-database-plugin \
+ sha256=<SHA256 Hex value of the plugin binary> \
+ command="myplugin"
+
+Success! Data written to: sys/plugins/catalog/database/myplugin-database-plugin
+```
+
+### Plugin Execution
+
+When a backend wants to run a plugin, it first looks up the plugin, by name, in
+the catalog. It then checks the executable's SHA256 sum against the one
+configured in the plugin catalog. Finally Vault runs the command configured in
+the catalog, sending along the JWT formatted response wrapping token and mlock
+settings. Like Vault, plugins support [the use of mlock when available](/docs/configuration#disable_mlock).
+
+~> Note: If Vault is configured with `mlock` enabled, then the Vault executable
+and each plugin executable in your [plugins directory](/docs/plugins/plugin-architecture#plugin-directory)
+must be given the ability to use the `mlock` syscall.
+
+### Plugin Upgrades
+
+External plugins may be updated by registering and reloading them. More details
+on the upgrade procedure can be found in
+[Upgrading Vault Plugins](/docs/upgrading/plugins).
+
+## Plugin Multiplexing
+
+Database plugins can be made to implement plugin multiplexing,
+allowing a single plugin process to be used for multiple database
+connections. This single process, per database plugin, will be multiplexed
+across all Vault namespaces for mounts of this type. Multiplexing a plugin
+does not affect the current behavior of existing plugins.
+
+To enable multiplexing, the plugin must be compiled with the `ServeMultiplex`
+function call from Vault's `dbplugin` package. At this time, there is no
+opt-out capability for plugins that implement multiplexing. To use a
+non-multiplexed plugin, run an older version of the plugin, i.e., the
+plugin calls the `dbplugin.Serve` function. More details
+on implementing plugin multiplexing can be found in
+[Upgrading Vault Plugins](/docs/secrets/databases/custom#serving-a-plugin-with-multiplexing).
+
+## Troubleshooting
+
+### Unrecognized remote plugin message
+
+If the following error is encountered when enabling a plugin secret engine or
+auth method:
+
+<CodeBlockConfig hideClipboard>
+
+```sh
+Unrecognized remote plugin message:
+
+This usually means that the plugin is either invalid or simply
+needs to be recompiled to support the latest protocol.
+```
+
+</CodeBlockConfig>
+
+Verify whether the Vault process has `mlock` enabled, and if so, run the
+following command against the plugin binary:
+
+```shell-session
+$ sudo setcap cap_ipc_lock=+ep <plugin-binary>
+```
+
website/content/docs/upgrading/upgrade-to-1.10.x.mdx+44 13
@@ -19,7 +19,7 @@ explicitly specify the `algorithm_signer=ssh-rsa` for RSA keys if they used
the implicit (empty) default, but newly created roles will use the new default
value (preferring a literal `default` which presently uses `rsa-sha2-256`).
-### Etcd v2 API no longer supported
+## Etcd v2 API no longer supported
Support for the Etcd v2 API is removed in Vault 1.10. The Etcd v2 API
was deprecated with the release of [Etcd v3.5](https://etcd.io/blog/2021/announcing-etcd-3.5/),
@@ -32,9 +32,9 @@ All storage migrations should have
[backups](/docs/concepts/storage#backing-up-vault-s-persisted-data)
taken prior to migration.
-### OTP Generation Process
+## OTP Generation Process
-Customers passing in OTPs during the the process of generating root tokens must modify
+Customers passing in OTPs during the process of generating root tokens must modify
the OTP generation to include an additional 2 characters before upgrading so that the
OTP can be xor-ed with the encoded root token. This change was implemented as a result
of the change in the prefix from hvs. to s. for service tokens.
@@ -62,17 +62,48 @@ OIDC provider system to reduce configuration steps and enhance usability.
The following built-in resources are included in each Vault namespace starting with Vault
1.10:
- - A "default" OIDC provider that's usable by all client applications
- - A "default" key for signing and verification of ID tokens
- - An "allow_all" assignment which authorizes all Vault entities to authenticate via a
+ - A `default` OIDC provider that's usable by all client applications
+ - A `default` key for signing and verification of ID tokens
+ - An `allow_all` assignment which authorizes all Vault entities to authenticate via a
client application
If you created an [OIDC provider](/api-docs/secret/identity/oidc-provider#create-or-update-a-provider)
-with the name "default", [key](/api-docs/secret/identity/tokens#create-a-named-key) with the
-name "default", or [assignment](/api-docs/secret/identity/oidc-provider#create-or-update-an-assignment)
-with the name "allow_all" using the Vault 1.9 tech preview, the installation of these built-in
-resources will be skipped. We _strongly_ recommend that you delete any resources that have
-naming collisions before upgrading to Vault 1.10. Failing to delete resources with naming
-collisions could result unexpected default behavior. Additionally, we recommend reading the
-corresponding details in the OIDC provider [concepts](/docs/concepts/oidc-provider) document
+with the name `default`, [key](/api-docs/secret/identity/tokens#create-a-named-key) with the
+name `default`, or [assignment](/api-docs/secret/identity/oidc-provider#create-or-update-an-assignment)
+with the name `allow_all` using the Vault 1.9 tech preview, the installation of these built-in
+resources will be skipped. We _strongly recommend_ that you delete any existing resources
+that have naming collisions before upgrading to Vault 1.10. Failing to delete resources with
+naming collisions could result unexpected default behavior. Additionally, we recommend reading
+the corresponding details in the OIDC provider [concepts](/docs/concepts/oidc-provider) document
to understand how the built-in resources are used in the system.
+
+## Known Issues
+
+### Single Vault follower restart causes election even with established quorum
+
+We now support Server Side Consistent Tokens (See [Replication](/docs/configuration/replication) and [Vault Eventual Consistency](/docs/enterprise/consistency)), which introduces a new token format that can only be used on nodes of 1.10 or higher version. This new format is enabled by default upon upgrading to the new version. Old format tokens can be read by Vault 1.10, but the new format Vault 1.10 tokens cannot be read by older Vault versions.
+
+For more details, see the [Server Side Consistent Tokens FAQ](/docs/faq/ssct).
+
+Since service tokens are always created on the leader, as long as the leader is not upgraded before performance standbys, service tokens will be of the old format and still be usable during the upgrade process. However, the usual upgrade process we recommend can't be relied upon to always upgrade the leader last. Due to this known [issue](https://github.com/hashicorp/vault/issues/14153), a Vault cluster using Integrated Storage may result in a leader not being upgraded last, and this can trigger a re-election. This re-election can cause the upgraded node to become the leader, resulting in the newly created tokens on the leader to be unusable on nodes that have not yet been upgraded. Note that this issue does not impact Vault OSS users.
+
+We will have a fix for this issue in Vault 1.10.1. Until this issue is fixed, you may be at risk of having performance standbys unable to service requests until all nodes are upgraded. We recommended that you plan for a maintenance window to upgrade.
+
+### Limited policy shows unhelpful message in UI after mounting a secret engine
+
+When a user has a policy that allows creating a secret engine but not reading it, after successful creation, the user sees a message n is undefined instead of a permissions error. We will have a fix for this issue in an upcoming minor release.
+
+### Adding/Modifying Duo MFA method for Enterprise MFA triggers a panic error
+
+When adding or modifying a Duo MFA method for step-up Enterprise MFA using the `sys/mfa/method/duo` endpoint, a panic gets triggered due to a missing schema field. We will have a fix for this in Vault 1.10.1. Until this issue is fixed, avoid making any changes to your Duo configuration if you are upgrading Vault to v1.10.0.
+
+### Sign in to UI using OIDC auth method results in an error
+
+Signing in to the Vault UI using an OIDC auth mount listed in the "tabs" of the form will result
+in the following error: "Authentication failed: role with oidc role_type is not allowed".
+The auth mounts listed in the "tabs" of the form are those that have [listing_visibility](/api-docs/system/auth#listing_visibility-1)
+set to `unauth`.
+
+There is a workaround for this error that will allow you to sign in to Vault using the OIDC
+auth method. Select the "Other" tab instead of selecting the specific OIDC auth mount tab.
+From there, select "OIDC" from the "Method" select box and proceed to sign in to Vault.
<82990506+hc-github-team-secure-vault-core@users.noreply.github.com>
release/1.10.x (#14715)
changelog/14670.txt | 3 +++
command/policy_write.go | 11 ++++++++---
vault/logical_system.go | 8 +++++++-
3 files changed, 18 insertions(+), 4 deletions(-)
create mode 100644 changelog/14670.txt
website/content/docs/upgrading/plugins.mdx+180 12
@@ -6,11 +6,11 @@ description: These are general upgrade instructions for Vault plugins.
# Upgrading Vault Plugins
-The following procedure details steps for upgrading a plugin that has already
-been registered to the catalog on a running server. This procedure is applicable
-to secret, auth, and database plugins.
+## External Plugin Upgrade Procedure
-## Upgrade Procedure
+The following procedure details steps for upgrading an external plugin that has
+been registered to the catalog on a running server. This procedure is
+applicable to secret engines, auth methods, and database plugins.
Vault executes plugin binaries when they are configured and roles are established
around them. The binary cannot be modified or replaced while running, so
@@ -21,16 +21,41 @@ Instead, you can restart or reload a plugin with the
`sys/plugins/reload/backend` [API][plugin_reload_api]. Follow these steps to
replace or upgrade a Vault plugin binary:
-1. Register plugin_v1 to the catalog
-2. Mount the plugin backend
-3. Register plugin_v2 to the catalog under the same plugin name, but with
- updated command to run plugin_v2 and updated sha256 of plugin_v2
-4. Trigger a plugin reload with `sys/plugins/reload/backend` to reload all
+1. [Register][plugin_registration] version 1 of `my-db-plugin` to the catalog
+
+```shell-session
+$ vault plugin register -sha256=<SHA256 Hex value of the plugin binary> \
+ database \ # type
+ my-db-plugin
+```
+
+2. [Mount][plugin_management] the plugin backend
+
+```shell-session
+$ vault secrets enable database
+```
+
+3. Register version 2 of `my-db-plugin` to the catalog under the same plugin
+ name, but with updated command to run version 2 of `my-db-plugin` and updated
+ sha256 of the new binary
+
+```shell-session
+$ vault plugin register -sha256=<SHA256 Hex value of the plugin binary> \
+ database \ # type
+ my-db-plugin
+```
+
+4. Trigger a [plugin reload][/docs/commands/plugin/reload] to reload all
mounted backends using that plugin or a subset of the mounts using that plugin
with either the `plugin` or `mounts` parameter respectively.
-Until step 4, the mount will still use plugin_v1, and when the reload is
-triggered, Vault will kill plugin_v1’s process and start a plugin_v2 process.
+```shell-session
+$ vault plugin reload -plugin my-db-plugin
+```
+
+Until step 4, the mount will still use version 1 of `my-db-plugin`, and when
+the reload is triggered, Vault will kill `my-db-plugin`’s process and start the
+new plugin process for `my-db-plugin` version 2.
-> **Important:** Plugin reload of a new plugin binary must be
performed on each Vault instance. Performing a plugin upgrade on a single
@@ -38,4 +63,147 @@ instance or through a load balancer can result in mismatched
plugin binaries within a cluster. On a replicated cluster this may be accomplished
by setting the 'scope' parameter of the reload to 'global'.
-[plugin_reload_api]: /api/system/plugins-reload-backend
+## Overriding Built-in Plugins
+
+### Background
+
+Vault's auth methods and secrets engines are structured as plugins, but this
+design is not obvious since many of them are built into Vault.
+
+You can see them with the Vault plugin list command, for example, the list of
+Secrets engines:
+
+```shell-session
+$ vault plugin list secret
+Plugins
+-------
+ad
+alicloud
+aws
+azure
+cassandra
+consul
+gcp
+gcpkms
+kv
+mongodb
+mongodbatlas
+mssql
+mysql
+nomad
+openldap
+pki
+postgresql
+rabbitmq
+ssh
+terraform
+totp
+transit
+```
+
+This will list all Secrets engines, internal (built-in) or external. To find
+out if a plugin is built-in, we can query its info:
+
+```shell-session
+$ vault plugin info secret azure
+Key Value
+--- -----
+args []
+builtin true
+command n/a
+name azure
+sha256 n/a
+```
+
+Because these built-in engines are plugins, they can be overridden. This can be
+a useful way to leverage features or bug fixes in plugins that are newer than
+the version of Vault you're using, without updating or even restarting Vault,
+and while retaining the data for your existing mount.
+
+Assume you have a new version of Azure Secrets and the binary is called
+"azure_new". The binary needs to be in the [plugin directory](/docs/plugins/plugin-architecture#plugin-directory)
+and can then be registered as either a distinct plugin, or overriding the
+current one.
+
+~> **Important:** do not disable (`vault secrets disable ...`) any mount that has
+data you're interested in; that would erase storage. For the in-place update,
+register a new plugin atop the built-in one and leave any mounts alone.
+
+### Procedure for Overriding Built-in Plugins
+
+The syntax is the same as an external plugin, with the difference being you
+name it the same as a built-in:
+
+```shell-session
+$ vault plugin register \
+ -sha256=<SHA256 Hex value of the plugin binary> \
+ -command=azure_new \
+ secret \
+ azure
+```
+
+"-command=azure_new" is the name of the binary, "secret" is the plugin type,
+and "azure" is the name of the built-in plugin that we're overriding. We can
+verify that the override is in place:
+
+```shell-session
+$ vault plugin info secret azure
+Key Value
+--- -----
+args []
+builtin false
+command azure_new
+name azure
+sha256 f6f6ec45d37484c257aa9ff80444b9f244aaef1c650edf8a42a2a1d3f00db2c5
+```
+
+At this point we've overridden the built-in, but it is not yet actively
+handling requests. For that we run:
+
+```shell-session
+$ vault plugin reload -plugin=azure
+```
+
+### Procedure for Reverting After Overriding A Built-in Plugin
+
+To revert the override, first deregister the plugin:
+
+```shell-session
+$ vault plugin deregister secret azure
+```
+
+Next, verify the override has been reverted and we are now using the built-in
+plugin:
+
+```shell-session
+$ vault plugin info secret azure
+Key Value
+--- -----
+args []
+builtin true
+command n/a
+name azure
+sha256 n/a
+```
+
+Finally, reload the plugin:
+
+```shell-session
+$ vault plugin reload -plugin=azure
+```
+
+### Caveats to Overriding Built-in Plugins
+
+* As mentioned earlier, disabling existing mounts will wipe the existing data.
+* This type of upgrade affects all uses of the plugin. So if you have 5
+ different Azure Secrets mounts, they'll all change after the replacement. If
+ you don't want that, you'll need to register the plugin under a different name
+ and start with a fresh mount.
+* In most cases, data upgrade and downgrade is not an issue. If the "new" version
+ introduces new data and you downgrade, the "old" version will ignore the
+ extraneous data. In some cases upgrading changes existing data in non-backwards
+ compatible ways, so it is good to check whether this is an issue.
+
+[plugin_reload_api]: /api-docs/system/plugins-reload-backend
+[plugin_registration]: /docs/plugins/plugin-architecture#plugin-registration
+[plugin_management]: /docs/plugins/plugin-management#enabling-disabling-external-plugins
website/content/docs/plugin.mdx+0 52
@@ -1,54 +0,0 @@
-layout: docs
-page_title: Custom Plugin Backends
-description: >-
- Plugin backends are mountable backends that are implemented using Vault's
- plugin system.
-
-# Custom Plugin Backends
-
-Plugin backends are the components in Vault that can be implemented separately from Vault's
-builtin backends. These backends can be either authentication or secrets engines.
-
-The [`api_addr`][api_addr] must be set in order for the plugin process to establish
-communication with the Vault server during mount time. If the storage backend
-has HA enabled and supports automatic host address detection (e.g. Consul),
-Vault will automatically attempt to determine the `api_addr` as well.
-
-Detailed information regarding the plugin system can be found in the
-[internals documentation](/docs/internals/plugins).
-
-# Enabling/Disabling Plugin Backends
-
-Before a plugin backend can be mounted, it needs to be registered via the
-[plugin catalog](/docs/internals/plugins#plugin-catalog). After
-the plugin is registered, it can be mounted by specifying the registered plugin name:
-
-```shell-session
-$ vault secrets enable -path=my-secrets passthrough-plugin
-Success! Enabled the passthrough-plugin secrets engine at: my-secrets/
-```
-
-Listing secrets engines will display secrets engines that are mounted as
-plugins:
-
-```shell-session
-$ vault secrets list
-Path Type Accessor Plugin Default TTL Max TTL Force No Cache Replication Behavior Description
-my-secrets/ plugin plugin_deb84140 passthrough-plugin system system false replicated
-```
-
-Disabling a plugin backend is the identical to disabling internal secrets engines:
-
-```shell-session
-$ vault secrets disable my-secrets
-```
-
-# Upgrading Plugins
-
-Upgrade instructions can be found in the [Upgrading Plugins - Guides][upgrading_plugins]
-page.
-
-[api_addr]: /docs/configuration#api_addr
-[upgrading_plugins]: /docs/upgrading/plugins
physical/cockroachdb/cockroachdb_ha.go+201 0
@@ -0,0 +1,201 @@
+package cockroachdb
+
+import (
+ "database/sql"
+ "fmt"
+ "sync"
+ "time"
+
+ "github.com/hashicorp/go-uuid"
+ "github.com/hashicorp/vault/sdk/physical"
+)
+
+const (
+ // The lock TTL matches the default that Consul API uses, 15 seconds.
+ // Used as part of SQL commands to set/extend lock expiry time relative to
+ // database clock.
+ CockroachDBLockTTLSeconds = 15
+
+ // The amount of time to wait between the lock renewals
+ CockroachDBLockRenewInterval = 5 * time.Second
+
+ // CockroachDBLockRetryInterval is the amount of time to wait
+ // if a lock fails before trying again.
+ CockroachDBLockRetryInterval = time.Second
+)
+
+// Verify backend satisfies the correct interfaces.
+var (
+ _ physical.HABackend = (*CockroachDBBackend)(nil)
+ _ physical.Lock = (*CockroachDBLock)(nil)
+)
+
+type CockroachDBLock struct {
+ backend *CockroachDBBackend
+ key string
+ value string
+ identity string
+ lock sync.Mutex
+
+ renewTicker *time.Ticker
+
+ // ttlSeconds is how long a lock is valid for.
+ ttlSeconds int
+
+ // renewInterval is how much time to wait between lock renewals. must be << ttl.
+ renewInterval time.Duration
+
+ // retryInterval is how much time to wait between attempts to grab the lock.
+ retryInterval time.Duration
+}
+
+func (c *CockroachDBBackend) HAEnabled() bool {
+ return c.haEnabled
+}
+
+func (c *CockroachDBBackend) LockWith(key, value string) (physical.Lock, error) {
+ identity, err := uuid.GenerateUUID()
+ if err != nil {
+ return nil, err
+ }
+ return &CockroachDBLock{
+ backend: c,
+ key: key,
+ value: value,
+ identity: identity,
+ ttlSeconds: CockroachDBLockTTLSeconds,
+ renewInterval: CockroachDBLockRenewInterval,
+ retryInterval: CockroachDBLockRetryInterval,
+ }, nil
+}
+
+// Lock tries to acquire the lock by repeatedly trying to create a record in the
+// CockroachDB table. It will block until either the stop channel is closed or
+// the lock could be acquired successfully. The returned channel will be closed
+// once the lock in the CockroachDB table cannot be renewed, either due to an
+// error speaking to CockroachDB or because someone else has taken it.
+func (l *CockroachDBLock) Lock(stopCh <-chan struct{}) (<-chan struct{}, error) {
+ l.lock.Lock()
+ defer l.lock.Unlock()
+
+ var (
+ success = make(chan struct{})
+ errors = make(chan error, 1)
+ leader = make(chan struct{})
+ )
+ go l.tryToLock(stopCh, success, errors)
+
+ select {
+ case <-success:
+ // After acquiring it successfully, we must renew the lock periodically.
+ l.renewTicker = time.NewTicker(l.renewInterval)
+ go l.periodicallyRenewLock(leader)
+ case err := <-errors:
+ return nil, err
+ case <-stopCh:
+ return nil, nil
+ }
+
+ return leader, nil
+}
+
+// Unlock releases the lock by deleting the lock record from the
+// CockroachDB table.
+func (l *CockroachDBLock) Unlock() error {
+ c := l.backend
+ c.permitPool.Acquire()
+ defer c.permitPool.Release()
+
+ if l.renewTicker != nil {
+ l.renewTicker.Stop()
+ }
+
+ _, err := c.haStatements["delete"].Exec(l.key)
+ return err
+}
+
+// Value checks whether or not the lock is held by any instance of CockroachDBLock,
+// including this one, and returns the current value.
+func (l *CockroachDBLock) Value() (bool, string, error) {
+ c := l.backend
+ c.permitPool.Acquire()
+ defer c.permitPool.Release()
+ var result string
+ err := c.haStatements["get"].QueryRow(l.key).Scan(&result)
+
+ switch err {
+ case nil:
+ return true, result, nil
+ case sql.ErrNoRows:
+ return false, "", nil
+ default:
+ return false, "", err
+
+ }
+}
+
+// tryToLock tries to create a new item in CockroachDB every `retryInterval`.
+// As long as the item cannot be created (because it already exists), it will
+// be retried. If the operation fails due to an error, it is sent to the errors
+// channel. When the lock could be acquired successfully, the success channel
+// is closed.
+func (l *CockroachDBLock) tryToLock(stop <-chan struct{}, success chan struct{}, errors chan error) {
+ ticker := time.NewTicker(l.retryInterval)
+ defer ticker.Stop()
+
+ for {
+ select {
+ case <-stop:
+ return
+ case <-ticker.C:
+ gotlock, err := l.writeItem()
+ switch {
+ case err != nil:
+ // Send to the error channel and don't block if full.
+ select {
+ case errors <- err:
+ default:
+ }
+ return
+ case gotlock:
+ close(success)
+ return
+ }
+ }
+ }
+}
+
+func (l *CockroachDBLock) periodicallyRenewLock(done chan struct{}) {
+ for range l.renewTicker.C {
+ gotlock, err := l.writeItem()
+ if err != nil || !gotlock {
+ close(done)
+ l.renewTicker.Stop()
+ return
+ }
+ }
+}
+
+// Attempts to put/update the CockroachDB item using condition expressions to
+// evaluate the TTL. Returns true if the lock was obtained, false if not.
+// If false error may be nil or non-nil: nil indicates simply that someone
+// else has the lock, whereas non-nil means that something unexpected happened.
+func (l *CockroachDBLock) writeItem() (bool, error) {
+ c := l.backend
+ c.permitPool.Acquire()
+ defer c.permitPool.Release()
+
+ sqlResult, err := c.haStatements["upsert"].Exec(l.identity, l.key, l.value, l.ttlSeconds)
+ if err != nil {
+ return false, err
+ }
+ if sqlResult == nil {
+ return false, fmt.Errorf("empty SQL response received")
+ }
+
+ ar, err := sqlResult.RowsAffected()
+ if err != nil {
+ return false, err
+ }
+ return ar == 1, nil
+}
website/content/docs/plugins/plugin-management.mdx+69 0
@@ -0,0 +1,69 @@
+---
+layout: docs
+page_title: Plugin Management
+description: >-
+ External Plugins are mountable backends that are implemented using Vault's
+ plugin system.
+---
+
+# Plugin Management
+
+External plugins are the components in Vault that can be implemented separately
+from Vault's built-in plugins. These plugins can be either authentication or
+secrets engines.
+
+The [`api_addr`][api_addr] must be set in order for the plugin process to
+establish communication with the Vault server during mount time. If the storage
+backend has HA enabled and supports automatic host address detection (e.g.
+Consul), Vault will automatically attempt to determine the `api_addr` as well.
+
+Detailed information regarding the plugin system can be found in the
+[internals documentation](/docs/plugins).
+
+## Registering External Plugins
+
+Before an external plugin can be mounted, it needs to be
+[registered](/docs/plugins/plugin-architecture#plugin-registration) in the
+plugin catalog to ensure the plugin invoked by Vault is authentic and maintains
+integrity:
+
+```shell-session
+$ vault plugin register -sha256=<SHA256 Hex value of the plugin binary> \
+ secret \ # type
+ my-secrets
+
+Success! Registered plugin: myplugin-database-plugin
+```
+
+## Enabling/Disabling External Plugins
+
+After the plugin is registered, it can be mounted by specifying the registered
+plugin name:
+
+```shell-session
+$ vault secrets enable -path=my-secrets passthrough-plugin
+Success! Enabled the passthrough-plugin secrets engine at: my-secrets/
+```
+
+Listing secrets engines will display secrets engines that are mounted as
+plugins:
+
+```shell-session
+$ vault secrets list
+Path Type Accessor Plugin Default TTL Max TTL Force No Cache Replication Behavior Description
+my-secrets/ plugin plugin_deb84140 passthrough-plugin system system false replicated
+```
+
+Disabling an external plugins is identical to disabling a built-in plugin:
+
+```shell-session
+$ vault secrets disable my-secrets
+```
+
+## Upgrading Plugins
+
+Upgrade instructions can be found in the [Upgrading Plugins - Guides][upgrading_plugins]
+page.
+
+[api_addr]: /docs/configuration#api_addr
+[upgrading_plugins]: /docs/upgrading/plugins
website/content/docs/concepts/client-count/faq.mdx+5 5
@@ -14,7 +14,7 @@ This FAQ section contains frequently asked questions about the client count feat
- [Q: Where can I learn more about Vault clients?](#q-where-can-i-learn-more-about-vault-clients)
- [Q: What is the difference between a direct entity (entity client) and a non-entity token (non-entity client)?](#q-what-is-the-difference-between-a-direct-entity-entity-client-and-a-non-entity-token-non-entity-client)
- [Q: Which Vault version reflects the most accurate client counts?](#q-which-vault-version-reflects-the-most-accurate-client-counts)
-- [Q: For customers using versions of Vault older than 1.6, what’s the best way to compute clients](#q-for-customers-using-versions-of-vault-older-than-1-6-what-s-the-best-way-to-compute-clients)
+- [Q: For customers using versions of Vault older than Vault 1.6, what’s the best way to compute clients](#q-for-customers-using-versions-of-vault-older-than-1-6-what-s-the-best-way-to-compute-clients)
- [Q: For customers using newer versions than Vault 1.6, what's the best way to compute clients?](#q-for-customers-using-newer-versions-than-vault-1-6-what-s-the-best-way-to-compute-clients)
- [Q: Why do we have two different tools (auditor tool and UI/API) to compute clients? Do we plan to deprecate one in the future?](#q-why-do-we-have-two-different-tools-auditor-tool-and-ui-api-to-compute-clients-do-we-plan-to-deprecate-one-in-the-future)
- [Q: How can I compute KMIP clients for Vault?](#q-how-can-i-compute-kmip-clients-for-vault)
@@ -25,7 +25,7 @@ This FAQ section contains frequently asked questions about the client count feat
- [Q: What happens if audit logs are unreadable for use by the Vault auditor tool?](#q-what-happens-if-audit-logs-are-unreadable-for-use-by-the-vault-auditor-tool)
- [Q: What does the usage metrics UI look like for Vault 1.9?](#q-what-does-the-usage-metrics-ui-look-like-for-vault-1-9)
- [Q: What does the usage metrics look like for Vault 1.10?](#q-what-does-the-usage-metrics-look-like-for-vault-1-10)
-- [Q: In versions prior to Vault 1.10, how do I compute changes to clients month to month from the UI?](#q-in-versions-prior-to-vault-1-10-how-do-i-compute-changes-to-clients-month-to-month-from-the-ui)
+- [Q: In older Vault versions including Vault 1.10, how do I compute changes to clients month to month from the UI?](#q-in-older-vault-versions-including-vault-1-10-how-do-i-compute-changes-to-clients-month-to-month-from-the-ui)
- [Q: What if I selected an inaccurate billing period via the UI/API?](#q-what-if-i-selected-an-inaccurate-billing-period-via-the-ui-api)
- [Q: What if I want to skip computation of clients for a period of time during the billing period?](#q-what-if-i-want-to-skip-computation-of-clients-for-a-period-of-time-during-the-billing-period)
- [Q: What are the known client count issues in the auditor tool as well as in the UI/API?](#q-what-are-the-known-client-count-issues-in-the-auditor-tool-as-well-as-in-the-ui-api)
@@ -35,7 +35,7 @@ This FAQ section contains frequently asked questions about the client count feat
- [Q: How can I configure the activity for log retention?](#q-how-can-i-configure-the-activity-for-log-retention)
- [Q: Do child namespaces create duplicate tokens?](#q-do-child-namespaces-create-duplicate-tokens)
- [Q: How does the Nomad Vault integration affect client counts?](#q-how-does-the-nomad-vault-integration-affect-client-counts)
-- [Q: Starting in Vault 1.9, Vault does not allow creating two aliases from the same auth mount under a single entity. What changed and how does this impact client counting?](#q-starting-in-vault-1-9-vault-does-not-allow-creating-two-aliases-from-the-same-auth-mount-under-a-single-entity-what-changed-and-how-does-this-impact-client-counting)
+- [Q: Starting in Vault 1.7, Vault does not allow creating two aliases from the same auth mount under a single entity. What changed and how does this impact client counting?](#q-starting-in-vault-1-7-vault-does-not-allow-creating-two-aliases-from-the-same-auth-mount-under-a-single-entity-what-changed-and-how-does-this-impact-client-counting)
- [Q: How does mount migration impact the client count metric?](#q-how-does-mount-migration-impact-the-client-count-metric)
- [Q: Vault 1.9 added support for providing custom user filters through the userfilter parameter. How does this affect client counts?](#q-vault-1-9-added-support-for-providing-custom-user-filters-through-the-userfilter-parameter-how-does-this-affect-client-counts)
@@ -165,7 +165,7 @@ In Vault 1.10, the client count dashboard is broken down into tabs, similar to V
The Vault 1.10 UI does not include montly attribution of clients, although the API for Vault 1.10 supports the same.
-### Q: In versions prior to Vault 1.10, how do I compute changes to clients month to month from the UI?
+### Q: In older Vault versions including Vault 1.10, how do I compute changes to clients month to month from the UI?
To perform this calculation, you must know the billing period. For the sake of this example, assume your billing period starts on January 1st and ends on December 31st:
@@ -241,7 +241,7 @@ However, creating a new token across a parent/child namespace boundary could res
The [Nomad Vault integration](https://www.nomadproject.io/docs/integrations/vault-integration#token-role-based-integration) uses [token roles](https://www.nomadproject.io/docs/integrations/vault-integration#vault-token-role-configuration). A single token role creates tokens for many Nomad jobs. If no [explicit identity aliases](/api-docs/auth/token#entity_alias) are provided (which is not currently supported in the integration), this would create a non-entity token for every running instance of a Nomad job.
Prior to Vault 1.9, the Nomad Vault integration caused duplicate clients, resulting in an elevated client count. Post Vault 1.9, with the introduction of the deduplication logic, the number of clients created by the integration is reduced. For more information on improvements made to client count in Vault 1.9, refer to the question [Which version of Vault reflects the most accurate count of clients with Vault?](#q-which-vault-version-reflects-the-most-accurate-client-counts).
-### Q: Starting in Vault 1.9, Vault does not allow creating two aliases from the same auth mount under a single entity. What changed and how does this impact client counting?
+### Q: Starting in Vault 1.7, Vault does not allow creating two aliases from the same auth mount under a single entity. What changed and how does this impact client counting?
Prior to 1.9, customers could create more than one alias from the same auth mount under a single entity. However, we made a fix in Vault 1.9 to prevent this from occuring as a remediation for a security issue.
<82990506+hc-github-team-secure-vault-core@users.noreply.github.com>
OIDC auth method (#14708) (#14713)
.../docs/upgrading/upgrade-to-1.10.x.mdx | 57 ++++++++++++++-----
1 file changed, 44 insertions(+), 13 deletions(-)
website/content/docs/platform/aws/lambda-extension-cache.mdx+304 0
@@ -0,0 +1,304 @@
+---
+layout: docs
+page_title: Vault Lambda Extension Caching
+description: >-
+ Supports caching to the local proxy server for the Vault Lambda Extension.
+---
+
+# Vault Lambda Extension
+
+AWS Lambda lets you run code without provisioning and managing servers.
+You can use the [quick-start](https://github.com/hashicorp/vault-lambda-extension/tree/0af1a648bfa4b9f37a04dd4311d8355f5c3902c3/quick-start) directory which has an end-to-end example if you would like to try out the extension from scratch.
+
+~> **Note**: If you decide to create one from scratch, be aware that this will create real infrastructure with an associated cost as per AWS' pricing.
+
+## Usage
+
+To use the extension, include the following ARN as a layer in your Lambda function:
+
+```text
+arn:aws:lambda:us-east-1:634166935893:layer:vault-lambda-extension:11
+```
+
+Where region may be any of `af-south-1`, `ap-east-1`, `ap-northeast-1`,
+`ap-northeast-2`, `ap-northeast-3`, `ap-south-1`, `ap-southeast-1`,
+`ap-southeast-2`, `ca-central-1`, `eu-central-1`, `eu-north-1`, `eu-south-1`,
+`eu-west-1`, `eu-west-2`, `eu-west-3`, `me-south-1`, `sa-east-1`, `us-east-1`,
+`us-east-2`, `us-west-1`, `us-west-2`.
+
+The extension authenticates with Vault using [AWS IAM auth](/docs/auth/aws),
+and all configuration is supplied via environment variables. There are two methods
+to read secrets, which can both be used side-by-side:
+
+- **Recommended**: Make unauthenticated requests to the extension's local proxy
+ server at `http://127.0.0.1:8200`, which will add an authentication header and
+ proxy to the configured `VAULT_ADDR`. Responses from Vault are returned without
+ modification.
+- Configure environment variables such as `VAULT_SECRET_PATH` for the extension
+ to read a secret and write it to disk.
+
+### Adding the extension to your existing Lambda and Vault infrastructure
+
+#### Requirements
+
+- ARN of the role your Lambda runs as
+- An instance of Vault accessible from AWS Lambda
+- An authenticated `vault` client
+- A secret in Vault that you want your Lambda to access, and a policy giving read access to it
+- Your Lambda function must use one of the [supported runtimes][lambda-supported-runtimes] for extensions
+
+#### Step 1. Configure Vault
+
+Enable the aws auth method.
+
+```shell-session
+$ vault auth enable aws
+```
+
+Configure the AWS client to use the default options.
+
+```shell-session
+$ vault write -force auth/aws/config/client
+```
+
+Create a role prefixed with the AWS environment name.
+
+```shell-session
+$ vault write auth/aws/role/vault-lambda-role \
+ auth_type=iam \
+ bound_iam_principal_arn="${YOUR_ARN}" \
+ policies="${YOUR_POLICY}" \
+ ttl=1h
+```
+
+#### Step 2. Option a) Install the extension for Lambda functions packaged in zip archives
+
+If you deploy your Lambda function as a zip file, you can add the extension
+to your Lambda layers using the console or [cli](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html#configuration-layers-using):
+
+```text
+arn:aws:lambda:<your-region>:634166935893:layer:vault-lambda-extension:11
+```
+
+#### Step 2. Option b) Install the extension for Lambda functions packaged in container images
+
+Alternatively, if you deploy your Lambda function as a container image, simply
+place the built binary in the `/opt/extensions` directory of your image.
+
+Fetch the binary from
+[releases.hashicorp.com](https://releases.hashicorp.com/vault-lambda-extension/).
+The following command requires cURL.
+
+```shell-session
+$ curl --silent https://releases.hashicorp.com/vault-lambda-extension/0.5.0/vault-lambda-extension_0.5.0_linux_amd64.zip \
+ --output vault-lambda-extension.zip
+```
+
+Unzip the donwloaded binary.
+
+```shell-session
+$ unzip vault-lambda-extension.zip
+```
+
+Optionally, you can verify the integrity of the downloaded zip using the release
+archive checksum verification instructions
+[here](https://www.hashicorp.com/security).
+
+Or to build the binary from source. This requires Golang installed. Run from the root of this repository.
+
+```shell-session
+$ GOOS=linux GOARCH=amd64 go build -o vault-lambda-extension main.go
+```
+
+#### Step 3. Configure vault-lambda-extension
+
+Configure the extension using [Lambda environment
+variables](https://docs.aws.amazon.com/lambda/latest/dg/configuration-envvars.html):
+
+Set the Vault API address.
+
+```shell-session
+$ VAULT_ADDR=http://vault.example.com:8200
+```
+
+Set the AWS IAM auth mount point (i.e. the path segment after `auth/` from above).
+
+```shell-session
+$ VAULT_AUTH_PROVIDER=aws
+```
+
+Set the Vault role to authenticate as. Must be configured for the ARN of your
+Lambda's role.
+
+```shell-session
+$ VAULT_AUTH_ROLE=vault-lambda-role
+```
+
+The path to a secret in Vault. Can be static or dynamic. Unless
+VAULT_SECRET_FILE is specified, JSON response will be written to
+`/tmp/vault/secret.json`.
+
+```shell-session
+$ VAULT_SECRET_PATH=secret/lambda-app/token
+```
+
+If everything is correctly set up, your Lambda function can then read secret
+material from `/tmp/vault/secret.json`. The exact contents of the JSON object
+will depend on the secret read, but its schema is the [Secret struct](https://github.com/hashicorp/vault/blob/api/v1.0.4/api/secret.go#L15)
+from the Vault API module.
+
+Alternatively, you can send normal Vault API requests over HTTP to the local
+proxy at `http://127.0.0.1:8200`, and the extension will add authentication
+before forwarding the request. Vault responses will be returned unmodified.
+Although local communication is over plain HTTP, the proxy server will use TLS
+to communicate with Vault if configured to do so as detailed below.
+
+## Configuration
+
+The extension is configured via [Lambda environment variables](https://docs.aws.amazon.com/lambda/latest/dg/configuration-envvars.html).
+Most of the [Vault CLI client's environment variables](/docs/commands#environment-variables) are available,
+as well as some additional variables to configure auth, which secret(s) to read and
+where to write secrets.
+
+| Environment variable | Description | Required | Example value |
+| --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | --------------------------- |
+| `VLE_VAULT_ADDR` | Vault address to connect to. Takes precedence over `VAULT_ADDR` so that clients of the proxy server can be configured using the standard `VAULT_ADDR` | No | `https://x.x.x.x:8200` |
+| `VAULT_ADDR` | Vault address to connect to if `VLE_VAULT_ADDR` is not set. Required if `VLE_VAULT_ADDR` is not set | No | `https://x.x.x.x:8200` |
+| `VAULT_AUTH_PROVIDER` | Name of the configured AWS IAM auth route on Vault | Yes | `aws` |
+| `VAULT_AUTH_ROLE` | Vault role to authenticate as | Yes | `lambda-app` |
+| `VAULT_IAM_SERVER_ID` | Value to pass to the Vault server via the [`X-Vault-AWS-IAM-Server-ID` HTTP Header for AWS Authentication](/api-docs/auth/aws#iam_server_id_header_value) | No | `vault.example.com` |
+| `VAULT_SECRET_PATH` | Secret path to read, written to `/tmp/vault/secret.json` unless `VAULT_SECRET_FILE` is specified | No | `database/creds/lambda-app` |
+| `VAULT_SECRET_FILE` | Path to write the JSON response for `VAULT_SECRET_PATH` | No | `/tmp/db.json` |
+| `VAULT_SECRET_PATH_FOO` | Additional secret path to read, where FOO can be any name, as long as a matching `VAULT_SECRET_FILE_FOO` is specified | No | `secret/lambda-app/token` |
+| `VAULT_SECRET_FILE_FOO` | Must exist for any correspondingly named `VAULT_SECRET_PATH_FOO`. Name has no further effect beyond matching to the correct path variable | No | `/tmp/token` |
+| `VAULT_TOKEN_EXPIRY_GRACE_PERIOD` | Period at the end of the proxy server's auth token TTL where it will consider the token expired and attempt to re-authenticate to Vault. Must have a unit and be parseable by `time.Duration`. Defaults to 10s. | No | `1m` |
+| `VAULT_STS_ENDPOINT_REGION` | The region of the STS regional endpoint to authenticate with. If the AWS IAM auth mount specified uses a regional STS endpoint, then this needs to match the region of that endpoint. Defaults to using the global endpoint, or the region the Lambda resides in if `AWS_STS_REGIONAL_ENDPOINTS` is set to `regional` | No | `eu-west-1` |
+
+The remaining environment variables are not required, and function exactly as
+described in the [Vault Commands (CLI)](/docs/commands#environment-variables) documentation. However,
+note that `VAULT_CLIENT_TIMEOUT` cannot extend the timeout beyond the 10s
+initialization timeout imposed by the Extensions API when writing files to disk.
+
+| Environment variable | Description | Required | Example value |
+| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | ------------------- |
+| `VAULT_CACERT` | Path to a PEM-encoded CA certificate _file_ on the local disk | No | `/tmp/ca.crt` |
+| `VAULT_CAPATH` | Path to a _directory_ of PEM-encoded CA certificate files on the local disk | No | `/tmp/certs` |
+| `VAULT_CLIENT_CERT` | Path to a PEM-encoded client certificate on the local disk | No | `/tmp/client.crt` |
+| `VAULT_CLIENT_KEY` | Path to an unencrypted, PEM-encoded private key on disk which corresponds to the matching client certificate | No | `/tmp/client.key` |
+| `VAULT_CLIENT_TIMEOUT` | Timeout for Vault requests. Default value is 60s. Ignored by proxy server. **Any value over 10s will exceed the Extensions API timeout and therefore have no effect** | No | `5s` |
+| `VAULT_MAX_RETRIES` | Maximum number of retries on `5xx` error codes. Defaults to 2. Ignored by proxy server | No | `2` |
+| `VAULT_SKIP_VERIFY` | Do not verify Vault's presented certificate before communicating with it. Setting this variable is not recommended and voids Vault's [security model](/docs/internals/security) | No | `true` |
+| `VAULT_TLS_SERVER_NAME` | Name to use as the SNI host when connecting via TLS | No | `vault.example.com` |
+| `VAULT_RATE_LIMIT` | Only applies to a single invocation of the extension. See [Vault Commands (CLI)](/docs/commands#environment-variables) documentation for details. Ignored by proxy server | No | `10` |
+| `VAULT_NAMESPACE` | The namespace to use for pre-configured secrets. Ignored by proxy server | No | `education` |
+| `VAULT_DEFAULT_CACHE_TTL` | The time to live configuration (aka, TTL) of the cache used by proxy server. Must have a unit and be parsable as a time.Duration. Required for caching to be enabled. | No | `15m` |
+| `VAULT_DEFAULT_CACHE_ENABLED` | Enable caching for all requests, without needing to set the X-Vault-Cache-Control header for each request. Must be set to a boolean value. | No | `true` |
+
+### AWS STS client configuration
+
+In addition to Vault configuration, you can configure certain aspects of the STS
+client the extension uses through the usual AWS environment variables. For example,
+if your Vault instance's IAM auth is configured to use regional STS endpoints:
+
+```shell-session
+$ vault write auth/aws/config/client \
+ sts_endpoint="https://sts.eu-west-1.amazonaws.com" \
+ sts_region="eu-west-1"
+```
+
+Then you may need to configure the extension's STS client to also use the regional
+STS endpoint by setting `AWS_STS_REGIONAL_ENDPOINTS=regional`, because both the AWS Golang
+SDK and Vault IAM auth method default to using the global endpoint in many regions.
+See documentation on [`sts_regional_endpoints`](https://docs.aws.amazon.com/credref/latest/refdocs/setting-global-sts_regional_endpoints.html) for more information.
+
+### Caching
+
+Caching can be configured for the extension's local proxy server so that it does
+not forward every HTTP request to Vault. The main consideration behind caching
+design is to make caching an explicit opt-in at the request level, so that it is
+only enabled for scenarios where caching makes sense without negative impact in
+others. To turn on caching, set the environment variable
+`VAULT_DEFAULT_CACHE_TTL` to a valid value that is parsable as a time.Duration
+in Go, for example, "15m", "1h", "2m3s" or "1h2m3s", depending on application
+needs. An invalid or negative value will be treated the same as a missing value,
+in which case, caching will not be set up and enabled.
+
+Then requests with HTTP method of "GET", and the HTTP header
+`X-Vault-Cache-Control: cache` will be returned directly from the cache if
+there's a cache hit. On a cache miss the request will be forwarded to Vault and
+the response returned and cached. If the header is set to
+`X-Vault-Cache-Control: recache`, the cache lookup will be skipped, and the
+request will be forwarded to Vault and the response returned and cached.
+Currently, the cache key is a hash of the request URL path, headers, body, and
+token.
+
+Caching may also be enabled for all requests by setting the environment variable
+`VAULT_DEFAULT_CACHE_ENABLE` to `true`. Then all requests will be fetched and/or
+cached as though the header `X-Vault-Cache-Control: cache` was present. Setting
+the header to `nocache` on a request will opt-out of caching entirely in this
+configuration. Setting the header to `recache` will skip the cache lookup and
+return and cache the response from Vault as described previously.
+
+## Limitations
+
+Secrets written to disk or returned from the proxy server will not be automatically
+refreshed when they expire. This is particularly important if you configure the
+extension to write secrets to disk, because the extension will only write to disk
+once per execution environment, rather than once per function invocation. If you
+use [provisioned concurrency](https://docs.aws.amazon.com/lambda/latest/dg/configuration-concurrency.html#configuration-concurrency-provisioned) or if your Lambda
+is invoked often enough that execution contexts live beyond the lifetime of the
+secret, then secrets on disk are likely to become invalid.
+
+In line with [Lambda best practices](https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html), we recommend avoiding
+writing secrets to disk where possible, and exclusively consuming secrets via
+the proxy server. However, the proxy server will still not perform any additional
+processing with returned secrets such as automatic lease renewal. The proxy server's
+own Vault auth token is the only thing that gets automatically refreshed. It will
+synchronously refresh its own token before proxying requests if the token is
+expired (including a grace window), and it will attempt to renew its token if the
+token is nearly expired but renewable.
+
+## Performance impact
+
+AWS Lambda pricing is based on [number of invocations, time of execution and memory
+used](https://aws.amazon.com/lambda/pricing/). The following table details some approximate performance
+related statistics to help assess the cost impact of this extension. Note that AWS
+Lambda allocates [CPU power in proportion to memory](https://docs.aws.amazon.com/lambda/latest/dg/configuration-memory.html) so results
+will vary widely. These benchmarks were run with the minimum 128MB of memory allocated
+so aim to give an approximate baseline.
+
+| Metric | Value | Description | Derivation |
+| -------------- | ---------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- |
+| Layer size | 8.5MB | The size of the unpacked extension binary | `ls -la` |
+| Init latency | 8.5ms (standard deviation 2.4ms) + one network round trip to authenticate to Vault | Extension initialization time in a new execution environment. Authentication round trip time will be highly deployment-dependent | Instrumented in code |
+| Invoke latency | <1ms | The base processing time for each function invocation, assuming no calls to the proxy server | Instrumented in code |
+| Memory impact | 12MB | The marginal impact on "Max Memory Used" when running the extension | As reported by Lambda when running Hello World function with and without extension |
+
+## Uploading to your own AWS account and region
+
+If you would like to upload the extension as a Lambda layer in your own AWS
+account and region, you can do the following:
+
+```shell-session
+$ curl --silent https://releases.hashicorp.com/vault-lambda-extension/0.5.0/vault-lambda-extension_0.5.0_linux_amd64.zip \
+ --output vault-lambda-extension.zip
+```
+
+Set your target AWS region.
+
+```shell-session
+$ export REGION="YOUR REGION HERE"
+```
+
+Upload the extension as a Lambda layer.
+
+```shell-session
+$ aws lambda publish-layer-version \
+ --layer-name vault-lambda-extension \
+ --zip-file "fileb://vault-lambda-extension.zip" \
+ --region "${REGION}"
+```
+
+## Learn
+
+For step-by-step instructions, refer to the [Vault AWS Lambda Extension](https://learn.hashicorp.com/tutorials/vault/aws-lambda) tutorial for details on how to create an AWS Lambda function and use the Vault Lambda Extension to authenticate with Vault.
physical/cockroachdb/cockroachdb.go+66 17
@@ -28,18 +28,23 @@ var (
)
const (
- defaultTableName = "vault_kv_store"
+ defaultTableName = "vault_kv_store"
+ defaultHATableName = "vault_ha_locks"
)
// CockroachDBBackend Backend is a physical backend that stores data
// within a CockroachDB database.
type CockroachDBBackend struct {
- table string
- client *sql.DB
- rawStatements map[string]string
- statements map[string]*sql.Stmt
- logger log.Logger
- permitPool *physical.PermitPool
+ table string
+ haTable string
+ client *sql.DB
+ rawStatements map[string]string
+ statements map[string]*sql.Stmt
+ rawHAStatements map[string]string
+ haStatements map[string]*sql.Stmt
+ logger log.Logger
+ permitPool *physical.PermitPool
+ haEnabled bool
}
// NewCockroachDBBackend constructs a CockroachDB backend using the given
@@ -51,6 +56,8 @@ func NewCockroachDBBackend(conf map[string]string, logger log.Logger) (physical.
return nil, fmt.Errorf("missing connection_url")
}
+ haEnabled := conf["ha_enabled"] == "true"
+
dbTable := conf["table"]
if dbTable == "" {
dbTable = defaultTableName
@@ -61,6 +68,16 @@ func NewCockroachDBBackend(conf map[string]string, logger log.Logger) (physical.
return nil, fmt.Errorf("invalid table: %w", err)
}
+ dbHATable, ok := conf["ha_table"]
+ if !ok {
+ dbHATable = defaultHATableName
+ }
+
+ err = validateDBTable(dbHATable)
+ if err != nil {
+ return nil, fmt.Errorf("invalid HA table: %w", err)
+ }
+
maxParStr, ok := conf["max_parallel"]
var maxParInt int
if ok {
@@ -79,17 +96,30 @@ func NewCockroachDBBackend(conf map[string]string, logger log.Logger) (physical.
return nil, fmt.Errorf("failed to connect to cockroachdb: %w", err)
}
- // Create the required table if it doesn't exists.
+ // Create the required tables if they don't exist.
createQuery := "CREATE TABLE IF NOT EXISTS " + dbTable +
" (path STRING, value BYTES, PRIMARY KEY (path))"
if _, err := db.Exec(createQuery); err != nil {
- return nil, fmt.Errorf("failed to create mysql table: %w", err)
+ return nil, fmt.Errorf("failed to create CockroachDB table: %w", err)
+ }
+ if haEnabled {
+ createHATableQuery := "CREATE TABLE IF NOT EXISTS " + dbHATable +
+ "(ha_key TEXT NOT NULL, " +
+ " ha_identity TEXT NOT NULL, " +
+ " ha_value TEXT, " +
+ " valid_until TIMESTAMP WITH TIME ZONE NOT NULL, " +
+ " CONSTRAINT ha_key PRIMARY KEY (ha_key) " +
+ ");"
+ if _, err := db.Exec(createHATableQuery); err != nil {
+ return nil, fmt.Errorf("failed to create CockroachDB HA table: %w", err)
+ }
}
// Setup the backend
c := &CockroachDBBackend{
- table: dbTable,
- client: db,
+ table: dbTable,
+ haTable: dbHATable,
+ client: db,
rawStatements: map[string]string{
"put": "INSERT INTO " + dbTable + " VALUES($1, $2)" +
" ON CONFLICT (path) DO " +
@@ -99,26 +129,45 @@ func NewCockroachDBBackend(conf map[string]string, logger log.Logger) (physical.
"list": "SELECT path FROM " + dbTable + " WHERE path LIKE $1",
},
statements: make(map[string]*sql.Stmt),
- logger: logger,
- permitPool: physical.NewPermitPool(maxParInt),
+ rawHAStatements: map[string]string{
+ "get": "SELECT ha_value FROM " + dbHATable + " WHERE NOW() <= valid_until AND ha_key = $1",
+ "upsert": "INSERT INTO " + dbHATable + " as t (ha_identity, ha_key, ha_value, valid_until)" +
+ " VALUES ($1, $2, $3, NOW() + $4) " +
+ " ON CONFLICT (ha_key) DO " +
+ " UPDATE SET (ha_identity, ha_key, ha_value, valid_until) = ($1, $2, $3, NOW() + $4) " +
+ " WHERE (t.valid_until < NOW() AND t.ha_key = $2) OR " +
+ " (t.ha_identity = $1 AND t.ha_key = $2) ",
+ "delete": "DELETE FROM " + dbHATable + " WHERE ha_key = $1",
+ },
+ haStatements: make(map[string]*sql.Stmt),
+ logger: logger,
+ permitPool: physical.NewPermitPool(maxParInt),
+ haEnabled: haEnabled,
}
// Prepare all the statements required
for name, query := range c.rawStatements {
- if err := c.prepare(name, query); err != nil {
+ if err := c.prepare(c.statements, name, query); err != nil {
return nil, err
}
}
+ if haEnabled {
+ for name, query := range c.rawHAStatements {
+ if err := c.prepare(c.haStatements, name, query); err != nil {
+ return nil, err
+ }
+ }
+ }
return c, nil
}
-// prepare is a helper to prepare a query for future execution
-func (c *CockroachDBBackend) prepare(name, query string) error {
+// prepare is a helper to prepare a query for future execution.
+func (c *CockroachDBBackend) prepare(statementMap map[string]*sql.Stmt, name, query string) error {
stmt, err := c.client.Prepare(query)
if err != nil {
return fmt.Errorf("failed to prepare %q: %w", name, err)
}
- c.statements[name] = stmt
+ statementMap[name] = stmt
return nil
}
website/content/docs/upgrading/upgrade-to-1.10.x.mdx+45 0
@@ -31,3 +31,48 @@ Vault storage to an Etcd v3 cluster prior to upgrading to Vault 1.10.
All storage migrations should have
[backups](/docs/concepts/storage#backing-up-vault-s-persisted-data)
taken prior to migration.
+
+### OTP Generation Process
+
+Customers passing in OTPs during the the process of generating root tokens must modify
+the OTP generation to include an additional 2 characters before upgrading so that the
+OTP can be xor-ed with the encoded root token. This change was implemented as a result
+of the change in the prefix from hvs. to s. for service tokens.
+
+## Token Format Change
+
+Token prefixes were updated to be more easily identifiable.
+
+- Service tokens previously started with s. now start with hvs.
+- Batch tokens previously started with b. now start with hvb.
+- Recovery tokens previously started with r. now start with hvr.
+
+Additionally, non-root service tokens are now longer than before. Previously, service tokens
+were 26 characters; they now have a minimum of 95 characters. However, existing tokens will
+still work.
+
+Refer to the [Server Side Consistent Token FAQ](/docs/faq/ssct) for details.
+
+## OIDC Provider Built-in Resources
+
+In Vault 1.9, the [OIDC identity provider](/docs/secrets/identity/oidc-provider) feature
+was released as a tech preview. In Vault 1.10, built-in resources were introduced to the
+OIDC provider system to reduce configuration steps and enhance usability.
+
+The following built-in resources are included in each Vault namespace starting with Vault
+1.10:
+
+ - A "default" OIDC provider that's usable by all client applications
+ - A "default" key for signing and verification of ID tokens
+ - An "allow_all" assignment which authorizes all Vault entities to authenticate via a
+ client application
+
+If you created an [OIDC provider](/api-docs/secret/identity/oidc-provider#create-or-update-a-provider)
+with the name "default", [key](/api-docs/secret/identity/tokens#create-a-named-key) with the
+name "default", or [assignment](/api-docs/secret/identity/oidc-provider#create-or-update-an-assignment)
+with the name "allow_all" using the Vault 1.9 tech preview, the installation of these built-in
+resources will be skipped. We _strongly_ recommend that you delete any resources that have
+naming collisions before upgrading to Vault 1.10. Failing to delete resources with naming
+collisions could result unexpected default behavior. Additionally, we recommend reading the
+corresponding details in the OIDC provider [concepts](/docs/concepts/oidc-provider) document
+to understand how the built-in resources are used in the system.
sdk/version/version_base.go | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
More files changed — see the full commit.

Release delta 1.8.0 → 1.9.9 (contains the fix)

· Jul 14, 2021, 11:38 AM+8402171compare
CHANGELOG.md+50 15
@@ -1,34 +1,51 @@
-## 1.8.0-rc1
-### June 16th, 2021
+## 1.8.0-rc2
+### July 15th, 2021
CHANGES:
-* core: License/EULA changes that ensure the presence of a valid HashiCorp license to start Vault. More information is available in the [Vault License FAQ](https://www.vaultproject.io/docs/enterprise/license/faqs)
+* agent: Errors in the template engine will no longer cause agent to exit unless
+explicitly defined to do so. A new configuration parameter,
+`exit_on_retry_failure`, within the new top-level stanza, `template_config`, can
+be set to `true` in order to cause agent to exit. Note that for agent to exit if
+`template.error_on_missing_key` is set to `true`, `exit_on_retry_failure` must
+be also set to `true`. Otherwise, the template engine will log an error but then
+restart its internal runner. [[GH-11775](https://github.com/hashicorp/vault/pull/11775)]
+* agent: Update to use IAM Service Account Credentials endpoint for signing JWTs
+when using GCP Auto-Auth method [[GH-11473](https://github.com/hashicorp/vault/pull/11473)]
* go: Update to Go 1.16.5 [[GH-11802](https://github.com/hashicorp/vault/pull/11802)]
FEATURES:
-* **MySQL Database UI**: The UI now supports adding and editing MySQL connections in the database secret engine [[GH-11532 | MySQL Database UI](https://github.com/hashicorp/vault/pull/11532 | MySQL Database UI)]
+* **MySQL Database UI**: The UI now supports adding and editing MySQL connections in the database secret engine [[GH-11532](https://github.com/hashicorp/vault/pull/11532)]
* cli/api: Add lease lookup command [[GH-11129](https://github.com/hashicorp/vault/pull/11129)]
* core: Add controlled capabilities to control group policy stanza
-* core: Add a darwin/arm64 binary release supporting the Apple M1 CPU
+* secret/rabbitmq: Add ability to customize dynamic usernames [[GH-11899](https://github.com/hashicorp/vault/pull/11899)]
+* secrets/database/elasticsearch: Add ability to customize dynamic usernames [[GH-11957](https://github.com/hashicorp/vault/pull/11957)]
+* secrets/database/influxdb: Add ability to customize dynamic usernames [[GH-11796](https://github.com/hashicorp/vault/pull/11796)]
+* secrets/database/mongodbatlas: Add ability to customize dynamic usernames [[GH-11956](https://github.com/hashicorp/vault/pull/11956)]
+* secrets/database/redshift: Add ability to customize dynamic usernames [[GH-12016](https://github.com/hashicorp/vault/pull/12016)]
+* secrets/database/snowflake: Add ability to customize dynamic usernames [[GH-11997](https://github.com/hashicorp/vault/pull/11997)]
+* secrets/gcp: Adds ability to use existing service accounts for generation of service account keys and access tokens. [[GH-12023](https://github.com/hashicorp/vault/pull/12023)]
+* secrets/keymgmt (enterprise): Adds general availability for distributing and managing keys in AWS KMS. [[GH-11958](https://github.com/hashicorp/vault/pull/11958)]
* ssh: add support for templated values in SSH CA DefaultExtensions [[GH-11495](https://github.com/hashicorp/vault/pull/11495)]
* ui: Add database secret engine support for MSSQL [[GH-11231](https://github.com/hashicorp/vault/pull/11231)]
IMPROVEMENTS:
-* agent: Update to use IAM Service Account Credentials endpoint for signing JWTs
-when using GCP Auto-Auth method [[GH-11473](https://github.com/hashicorp/vault/pull/11473)]
+* agent/template: Added static_secret_render_interval to specify how often to fetch non-leased secrets [[GH-11934](https://github.com/hashicorp/vault/pull/11934)]
+* agent: Allow Agent auto auth to read symlinked JWT files [[GH-11502](https://github.com/hashicorp/vault/pull/11502)]
* api: Allow a leveled logger to be provided to `api.Client` through `SetLogger`. [[GH-11696](https://github.com/hashicorp/vault/pull/11696)]
* auth/aws: Underlying error included in validation failure message. [[GH-11638](https://github.com/hashicorp/vault/pull/11638)]
+* core: Add `prefix_filter` to telemetry config [[GH-12025](https://github.com/hashicorp/vault/pull/12025)]
* core: Add a small (<1s) exponential backoff to failed TCP listener Accept failures. [[GH-11588](https://github.com/hashicorp/vault/pull/11588)]
* core: Add metrics for standby node forwarding. [[GH-11366](https://github.com/hashicorp/vault/pull/11366)]
-* core: Add metrics to report if a node is a perf standby, if a node is a dr
-secondary or primary, and if a node is a perf secondary or primary. Also allow
-DR secondaries to serve metrics requests when using unauthenticated_metrics_access. [[GH-1844](https://github.com/hashicorp/vault/pull/1844)]
* core: Send notifications to systemd on start, stop, and configuration reload. [[GH-11517](https://github.com/hashicorp/vault/pull/11517)]
* core: add irrevocable lease list and count apis [[GH-11607](https://github.com/hashicorp/vault/pull/11607)]
* core: allow arbitrary length stack traces upon receiving SIGUSR2 (was 32MB) [[GH-11364](https://github.com/hashicorp/vault/pull/11364)]
+* db/cassandra: Added tls_server_name to specify server name for TLS validation [[GH-11820](https://github.com/hashicorp/vault/pull/11820)]
+* plugins/ad: Added rotate-role endpoint for manual service account password rotations [[GH-11942](https://github.com/hashicorp/vault/pull/11942)]
+* raft: Improve raft batch size selection [[GH-11907](https://github.com/hashicorp/vault/pull/11907)]
+* raft: change freelist type to map and set nofreelistsync to true [[GH-11895](https://github.com/hashicorp/vault/pull/11895)]
* replication (enterprise): The log shipper is now memory
as well as length bound, and length and size can be
separately configured.
@@ -36,20 +53,31 @@ separately configured.
* secrets/database/mongodb: Add ability to customize `SocketTimeout`, `ConnectTimeout`, and `ServerSelectionTimeout` [[GH-11600](https://github.com/hashicorp/vault/pull/11600)]
* secrets/database/mongodb: Increased throughput by allowing for multiple request threads to simultaneously update users in MongoDB [[GH-11600](https://github.com/hashicorp/vault/pull/11600)]
* storage/raft: Support autopilot for HA only raft storage. [[GH-11260](https://github.com/hashicorp/vault/pull/11260)]
+* ui: Add Validation to KV secret engine [[GH-11785](https://github.com/hashicorp/vault/pull/11785)]
* ui: Add push notification message when selecting okta auth. [[GH-11442](https://github.com/hashicorp/vault/pull/11442)]
* ui: Add regex validation to Transform Template pattern input [[GH-11586](https://github.com/hashicorp/vault/pull/11586)]
* ui: Add specific error message if unseal fails due to license [[GH-11705](https://github.com/hashicorp/vault/pull/11705)]
+* ui: Add validation support for open api form fields [[GH-11963](https://github.com/hashicorp/vault/pull/11963)]
+* ui: Added auth method descriptions to UI login page [[GH-11795](https://github.com/hashicorp/vault/pull/11795)]
* ui: JSON fields on database can be cleared on edit [[GH-11708](https://github.com/hashicorp/vault/pull/11708)]
* ui: Obscure secret values on input and displayOnly fields like certificates. [[GH-11284](https://github.com/hashicorp/vault/pull/11284)]
* ui: Redesign of KV 2 Delete toolbar. [[GH-11530](https://github.com/hashicorp/vault/pull/11530)]
* ui: Replace tool partials with components. [[GH-11672](https://github.com/hashicorp/vault/pull/11672)]
+* ui: Show description on secret engine list [[GH-11995](https://github.com/hashicorp/vault/pull/11995)]
* ui: Update ember to latest LTS and upgrade UI dependencies [[GH-11447](https://github.com/hashicorp/vault/pull/11447)]
* ui: Update partials to components [[GH-11680](https://github.com/hashicorp/vault/pull/11680)]
* ui: Updated ivy code mirror component for consistency [[GH-11500](https://github.com/hashicorp/vault/pull/11500)]
* ui: Updated search select component styling [[GH-11360](https://github.com/hashicorp/vault/pull/11360)]
+* ui: add transform secrets engine to features list [[GH-12003](https://github.com/hashicorp/vault/pull/12003)]
+* ui: add validations for duplicate path kv engine [[GH-11878](https://github.com/hashicorp/vault/pull/11878)]
* ui: show site-wide banners for license warnings if applicable [[GH-11759](https://github.com/hashicorp/vault/pull/11759)]
* ui: update license page with relevant autoload info [[GH-11778](https://github.com/hashicorp/vault/pull/11778)]
+DEPRECATIONS:
+
+* secrets/gcp: Deprecated the `/gcp/token/:roleset` and `/gcp/key/:roleset` paths for generating secrets for rolesets.
+Use `/gcp/roleset/:roleset/token` and `/gcp/roleset/:roleset/key` instead. [[GH-12023](https://github.com/hashicorp/vault/pull/12023)]
+
BUG FIXES:
* activity: Omit wrapping tokens and control groups from client counts [[GH-11826](https://github.com/hashicorp/vault/pull/11826)]
@@ -58,32 +86,37 @@ information from the auto-auth config map on renewals or retries. [[GH-11576](ht
* agent/template: fix command shell quoting issue [[GH-11838](https://github.com/hashicorp/vault/pull/11838)]
* agent: Fixed agent templating to use configured tls servername values [[GH-11288](https://github.com/hashicorp/vault/pull/11288)]
* agent: fix timestamp format in log messages from the templating engine [[GH-11838](https://github.com/hashicorp/vault/pull/11838)]
+* auth/approle: fixing dereference of nil pointer [[GH-11864](https://github.com/hashicorp/vault/pull/11864)]
* auth/jwt: Updates the [hashicorp/cap](https://github.com/hashicorp/cap) library to `v0.1.0` to
bring in a verification key caching fix. [[GH-11784](https://github.com/hashicorp/vault/pull/11784)]
+* auth/ldap: Fix a bug where the LDAP auth method does not return the request_timeout configuration parameter on config read. [[GH-11975](https://github.com/hashicorp/vault/pull/11975)]
+* cli: Add support for response wrapping in `vault list` and `vault kv list` with output format other than `table`. [[GH-12031](https://github.com/hashicorp/vault/pull/12031)]
+* cli: vault delete and vault kv delete should support the same output options (e.g. -format) as vault write. [[GH-11992](https://github.com/hashicorp/vault/pull/11992)]
* core (enterprise): Fix orphan return value from auth methods executed on performance standby nodes.
* core (enterprise): Fix plugins mounted in namespaces being unable to use password policies [[GH-11596](https://github.com/hashicorp/vault/pull/11596)]
-* core (enterprise): serialize access to HSM entropy generation to avoid errors in concurrent key generation.
+* core/metrics: Add generic KV mount support for vault.kv.secret.count telemetry metric [[GH-12020](https://github.com/hashicorp/vault/pull/12020)]
* core: Fix cleanup of storage entries from cubbyholes within namespaces. [[GH-11408](https://github.com/hashicorp/vault/pull/11408)]
* core: Fix edge cases in the configuration endpoint for barrier key autorotation. [[GH-11541](https://github.com/hashicorp/vault/pull/11541)]
* core: Fix goroutine leak when updating rate limit quota [[GH-11371](https://github.com/hashicorp/vault/pull/11371)]
* core: Fix race that allowed remounting on path used by another mount [[GH-11453](https://github.com/hashicorp/vault/pull/11453)]
* core: Fix storage entry leak when revoking leases created with non-orphan batch tokens. [[GH-11377](https://github.com/hashicorp/vault/pull/11377)]
+* core: Fixed double counting of http requests after operator stepdown [[GH-11970](https://github.com/hashicorp/vault/pull/11970)]
* core: correct logic for renewal of leases nearing their expiration time. [[GH-11650](https://github.com/hashicorp/vault/pull/11650)]
* identity: Use correct mount accessor when refreshing external group memberships. [[GH-11506](https://github.com/hashicorp/vault/pull/11506)]
+* mongo-db: default username template now strips invalid '.' characters [[GH-11872](https://github.com/hashicorp/vault/pull/11872)]
* pki: Only remove revoked entry for certificates during tidy if they are past their NotAfter value [[GH-11367](https://github.com/hashicorp/vault/pull/11367)]
-* replication: Fix panic trying to update walState during identity group invalidation. [[GH-1865](https://github.com/hashicorp/vault/pull/1865)]
-* replication: Fix: mounts created within a namespace that was part of an Allow
-filtering rule would not appear on performance secondary if created after rule
-was defined. [[GH-1807](https://github.com/hashicorp/vault/pull/1807)]
* secret/pki: use case insensitive domain name comparison as per RFC1035 section 2.3.3
* secret: fix the bug where transit encrypt batch doesn't work with key_version [[GH-11628](https://github.com/hashicorp/vault/pull/11628)]
+* secrets/ad: Forward all creds requests to active node [[GH-76](https://github.com/hashicorp/vault-plugin-secrets-ad/pull/76)] [[GH-11836](https://github.com/hashicorp/vault/pull/11836)]
* secrets/database/cassandra: Fixed issue where hostnames were not being validated when using TLS [[GH-11365](https://github.com/hashicorp/vault/pull/11365)]
+* secrets/database/cassandra: Fixed issue where the PEM parsing logic of `pem_bundle` and `pem_json` didn't work for CA-only configurations [[GH-11861](https://github.com/hashicorp/vault/pull/11861)]
* secrets/database/cassandra: Updated default statement for password rotation to allow for special characters. This applies to root and static credentials. [[GH-11262](https://github.com/hashicorp/vault/pull/11262)]
* secrets/database: Fix marshalling to allow providing numeric arguments to external database plugins. [[GH-11451](https://github.com/hashicorp/vault/pull/11451)]
* secrets/database: Fixed minor race condition when rotate-root is called [[GH-11600](https://github.com/hashicorp/vault/pull/11600)]
* secrets/database: Fixes issue for V4 database interface where `SetCredentials` wasn't falling back to using `RotateRootCredentials` if `SetCredentials` is `Unimplemented` [[GH-11585](https://github.com/hashicorp/vault/pull/11585)]
* storage/dynamodb: Handle throttled batch write requests by retrying, without which writes could be lost. [[GH-10181](https://github.com/hashicorp/vault/pull/10181)]
* storage/raft: Support cluster address change for nodes in a cluster managed by autopilot [[GH-11247](https://github.com/hashicorp/vault/pull/11247)]
+* storage/raft: Tweak creation of vault.db file [[GH-12034](https://github.com/hashicorp/vault/pull/12034)]
* storage/raft: leader_tls_servername wasn't used unless leader_ca_cert_file and/or mTLS were configured. [[GH-11252](https://github.com/hashicorp/vault/pull/11252)]
* tokenutil: Perform the num uses check before token type. [[GH-11647](https://github.com/hashicorp/vault/pull/11647)]
* transform (enterprise): Fix an issue with malformed transform configuration
@@ -97,7 +130,9 @@ storage when upgrading from 1.5 to 1.6. See Upgrade Notes for 1.6.x.
* ui: Fix status menu no showing on login [[GH-11213](https://github.com/hashicorp/vault/pull/11213)]
* ui: Fix text link URL on database roles list [[GH-11597](https://github.com/hashicorp/vault/pull/11597)]
* ui: Fixed and updated lease renewal picker [[GH-11256](https://github.com/hashicorp/vault/pull/11256)]
+* ui: fix control group access for database credential [[GH-12024](https://github.com/hashicorp/vault/pull/12024)]
* ui: fix issue where select-one option was not showing in secrets database role creation [[GH-11294](https://github.com/hashicorp/vault/pull/11294)]
+* ui: fix oidc login with Safari [[GH-11884](https://github.com/hashicorp/vault/pull/11884)]
## 1.7.3
### June 16th, 2021
website/content/api-docs/secret/kv/kv-v2.mdx | 29 ++++++++++----------
1 file changed, 15 insertions(+), 14 deletions(-)
website/content/docs/secrets/identity.mdx+11 11
@@ -65,7 +65,7 @@ applicable to the token through its identity will happen at request time. This
also adds enormous flexibility to control the behavior of already issued
tokens.
-Its important to note that the policies on the entity are only a means to grant
+It is important to note that the policies on the entity are only a means to grant
_additional_ capabilities and not a replacement for the policies on the token.
To know the full set of capabilities of the token with an associated entity
identifier, the policies on the token should be taken into account.
@@ -128,7 +128,7 @@ token using a token role with a configured list of `allowed_entity_aliases`.
### Identity Auditing
-If the token used to make API calls have an associated entity identifier, it
+If the token used to make API calls has an associated entity identifier, it
will be audit logged as well. This leaves a trail of actions performed by
specific users.
@@ -136,10 +136,10 @@ specific users.
In version 0.9, Vault identity has support for groups. A group can contain
multiple entities as its members. A group can also have subgroups. Policies set
-on the group is granted to all members of the group. During request time, when
+on the group are granted to all members of the group. During request time, when
the token's entity ID is being evaluated for the policies that it has access
-to; along with the policies on the entity itself, policies that are inherited
-due to group memberships are also granted.
+to, policies that are inherited due to group memberships are granted along
+with the policies on the entity itself.
### Group Hierarchical Permissions
@@ -154,11 +154,11 @@ to policies on both GroupA and GroupB.
By default, the groups created in identity store are called the internal
groups. The membership management of these groups should be carried out
manually. A group can also be created as an external group. In this case, the
-entity membership in the group is managed semi-automatically. External group
+entity membership in the group is managed semi-automatically. An external group
serves as a mapping to a group that is outside of the identity store. External
groups can have one (and only one) alias. This alias should map to a notion of
-group that is outside of the identity store. For example, groups in LDAP, and
-teams in GitHub. A username in LDAP, belonging to a group in LDAP, can get its
+a group that is outside of the identity store. For example, groups in LDAP and
+teams in GitHub. A username in LDAP belonging to a group in LDAP can get its
entity ID added as a member of a group in Vault automatically during _logins_
and _token renewals_. This works only if the group in Vault is an external
group and has an alias that maps to the group in LDAP. If the user is removed
@@ -174,7 +174,7 @@ tokens are signed JWTs following the [OIDC ID
token](https://openid.net/specs/openid-connect-core-1_0.html#IDToken) structure.
The public keys used to authenticate the tokens are published by Vault on an
unauthenticated endpoint following OIDC discovery and JWKS conventions, which
-should be a directly usable by JWT/OIDC libraries. An introspection endpoint is
+should be directly usable by JWT/OIDC libraries. An introspection endpoint is
also provided by Vault for token verification.
### Roles and Keys
@@ -194,7 +194,7 @@ may refer to the same key). It is not possible to generate an unsigned ID token.
A named key is a public/private key pair generated by Vault. The private key is
used to sign the identity tokens, and the public key is used by clients to
-verify the signature. Key are regularly rotated, whereby a new key pair is
+verify the signature. Keys are regularly rotated, whereby a new key pair is
generated and the previous _public_ key is retained for a limited time for
verification purposes.
@@ -202,7 +202,7 @@ A named key's configuration specifies a rotation period, a verification ttl,
signing algorithm and allowed client IDs. Rotation period specifies the
frequency at which a new signing key is generated and the private portion of the
previous signing key is deleted. Verification ttl is the time a public key is
-retained for verification, after being rotated. By default, keys are rotated
+retained for verification after being rotated. By default, keys are rotated
every 24 hours, and continue to be available for verification for 24 hours after
their rotation.
api/response.go | 18 +++++++++++++++---
changelog/12061.txt | 3 +++
http/handler.go | 5 ++++-
sdk/logical/response_util.go | 10 ++++++++++
vault/logical_system.go | 7 ++++++-
5 files changed, 38 insertions(+), 5 deletions(-)
create mode 100644 changelog/12061.txt
website/content/api-docs/secret/kv/kv-v2.mdx+15 14
@@ -84,15 +84,17 @@ $ curl \
{
"data": {
"cas_required": false,
- "max_versions": 0,
- "delete_version_after": "3h25m19s"
+ "delete_version_after": "3h25m19s",
+ "max_versions": 0
}
}
```
## Read Secret Version
-This endpoint retrieves the secret at the specified location.
+This endpoint retrieves the secret at the specified location. The metadata returned
+here ( `created_time, destroy,` and `version`) is version specific. It should not be
+confused with the response from the [metadata endpoint](/api/secret/kv/kv-v2#read-secret-metadata).
| Method | Path |
| :----- | :------------------------------------------- |
@@ -146,11 +148,10 @@ have an ACL policy granting the `update` capability.
- `options` `(Map: <optional>)` – An object that holds option settings.
- - `cas` `(int: <optional>)` - Set the "cas" value to use a Check-And-Set
- operation. If not set the write will be allowed. If set to 0 a write will
- only be allowed if the key doesn’t exist. If the index is non-zero the
- write will only be allowed if the key’s current version matches the
- version specified in the cas parameter.
+- `cas` `(int: <optional>)` - This flag is required if cas_required is set
+ to true on either the secret or the engine's config. In order for a write
+ to be successful, cas must be set to the current version of the secret.
+ If cas is set to 0, the write will only be allowed if the key doesn't exist.
- `data` `(Map: <required>)` – The contents of the data map will be stored and
returned on read.
@@ -367,7 +368,7 @@ entries.
## Read Secret Metadata
This endpoint retrieves the metadata and versions for the secret at the
-specified path.
+specified path. Metadata is version-agnostic.
| Method | Path |
| :----- | :----------------------- |
@@ -391,8 +392,10 @@ $ curl \
```json
{
"data": {
+ "cas_required": false,
"created_time": "2018-03-22T02:24:06.945319214Z",
"current_version": 3,
+ "delete_version_after": "3h25m19s",
"max_versions": 0,
"oldest_version": 0,
"updated_time": "2018-03-22T02:36:43.986212308Z",
@@ -417,12 +420,10 @@ $ curl \
}
```
-## Update Metadata
+## Create/Update Metadata
-This endpoint creates a new version of a secret at the specified location. If
-the value does not yet exist, the calling token must have an ACL policy granting
-the `create` capability. If the value already exists, the calling token must
-have an ACL policy granting the `update` capability.
+This endpoint creates or updates the metadata of a secret at the specified location.
+It does not create a new version.
| Method | Path |
| :----- | :----------------------- |
(#12094)
ui/scripts/start-vault.js | 5 +++--
ui/tests/acceptance/secrets/backend/kv/secret-test.js | 2 +-
2 files changed, 4 insertions(+), 3 deletions(-)
vault/diagnose/os_common.go+4 4
@@ -31,11 +31,11 @@ partLoop:
Warn(ctx, fmt.Sprintf("Could not obtain partition usage for %s: %v.", partition.Mountpoint, err))
} else {
if usage.UsedPercent > 95 {
- SpotWarn(ctx, testName, fmt.Sprintf(partition.Mountpoint+" is %d percent full.", usage.UsedPercent))
- Advise(ctx, "It is recommended to have more than five percent of the partition free.")
+ SpotWarn(ctx, testName, fmt.Sprintf(partition.Mountpoint+" is %d percent full.", usage.UsedPercent),
+ Advice("It is recommended to have more than five percent of the partition free."))
} else if usage.Free < 2<<30 {
- SpotWarn(ctx, testName, partition.Mountpoint+" has %d bytes full.")
- Advise(ctx, "It is recommended to have at least 1 GB of space free per partition.")
+ SpotWarn(ctx, testName, partition.Mountpoint+" has %d bytes full.",
+ Advice("It is recommended to have at least 1 GB of space free per partition."))
} else {
SpotOk(ctx, testName, partition.Mountpoint+" usage ok.")
}
.circleci/config.yml+118 37
@@ -21,6 +21,9 @@ jobs:
- restore_cache:
key: package-2ffbc24c482a71c7f16a87ad5cc6c01038544fe3-{{checksum ".buildcache/cache-keys/package-2ffbc24c482a71c7f16a87ad5cc6c01038544fe3"}}
name: Restore package cache
+ - restore_cache:
+ key: package-2ee966c5768e83a92093c0bc6a2fc6042afe4839-{{checksum ".buildcache/cache-keys/package-2ee966c5768e83a92093c0bc6a2fc6042afe4839"}}
+ name: Restore package cache
- restore_cache:
key: package-61f4e059780eac8772bed584d10749860f2fdce1-{{checksum ".buildcache/cache-keys/package-61f4e059780eac8772bed584d10749860f2fdce1"}}
name: Restore package cache
@@ -73,20 +76,20 @@ jobs:
command: ls -lahR .buildcache
name: List Build Cache
- run:
- command: cp packages*.lock/pkgs.yml lockfile-7d1b28ede60990fa.yml
+ command: cp packages*.lock/pkgs.yml lockfile-e0096d0cd2ca1c5a.yml
name: Update Lockfile Name
- run:
- command: tar -czf packages-7d1b28ede60990fa.tar.gz .buildcache/packages lockfile-7d1b28ede60990fa.yml
+ command: tar -czf packages-e0096d0cd2ca1c5a.tar.gz .buildcache/packages lockfile-e0096d0cd2ca1c5a.yml
name: Create Raw Package Tarball
- run:
- command: tar -czf meta-7d1b28ede60990fa.tar.gz .buildcache/packages/store/*.json lockfile-7d1b28ede60990fa.yml
+ command: tar -czf meta-e0096d0cd2ca1c5a.tar.gz .buildcache/packages/store/*.json lockfile-e0096d0cd2ca1c5a.yml
name: Create Metadata Tarball
- store_artifacts:
- path: lockfile-7d1b28ede60990fa.yml
+ path: lockfile-e0096d0cd2ca1c5a.yml
- store_artifacts:
- path: packages-7d1b28ede60990fa.tar.gz
+ path: packages-e0096d0cd2ca1c5a.tar.gz
- store_artifacts:
- path: meta-7d1b28ede60990fa.tar.gz
+ path: meta-e0096d0cd2ca1c5a.tar.gz
- store_artifacts:
path: .buildcache/packages
environment:
@@ -108,7 +111,7 @@ jobs:
version: 19.03.12
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -150,7 +153,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -171,6 +174,84 @@ jobs:
name: Save package cache
paths:
- .buildcache/packages/store
+ darwin_arm64_package:
+ docker:
+ - image: docker.mirror.hashicorp.services/circleci/buildpack-deps
+ shell: /usr/bin/env bash -euo pipefail -c
+ environment:
+ - AUTO_INSTALL_TOOLS: 'YES'
+ - BUILDKIT_PROGRESS: plain
+ - PRODUCT_REVISION: ''
+ - PACKAGE_SPEC_ID: 2ee966c5768e83a92093c0bc6a2fc6042afe4839
+ steps:
+ - setup_remote_docker:
+ docker_layer_caching: false
+ version: 19.03.12
+ - add_ssh_keys:
+ fingerprints:
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
+ - checkout:
+ path: .
+ - run:
+ command: make -C packages*.lock write-package-cache-key
+ name: Write package cache key
+ - restore_cache:
+ key: package-2ee966c5768e83a92093c0bc6a2fc6042afe4839-{{checksum ".buildcache/cache-keys/package-2ee966c5768e83a92093c0bc6a2fc6042afe4839"}}
+ name: Restore package cache
+ - run:
+ command: |2
+
+ if ! { PKG=$(find .buildcache/packages/store -maxdepth 1 -mindepth 1 -name '*.zip' 2> /dev/null) && [ -n "$PKG" ]; }; then
+ echo "No package found, continuing with build."
+ exit 0
+ fi
+ echo "Package already cached, skipping build."
+ circleci-agent step halt
+ name: Check cache status
+ - run:
+ command: make -C packages*.lock write-builder-cache-keys
+ name: Write builder layer cache keys
+ - restore_cache:
+ key: copy-source_01bb_{{checksum ".buildcache/cache-keys/copy-source-01bb587fbaa40eee3270a2dfa7865a8e37dde482"}}
+ keys:
+ - go-modules_87d5_{{checksum ".buildcache/cache-keys/go-modules-87d5fe370bb634974f7b55a067206409c85d5947"}}
+ - build-static-assets_c9eb_{{checksum ".buildcache/cache-keys/build-static-assets-c9eb755a774c85539457676828158a137604f20d"}}
+ - build-ui_f5d8_{{checksum ".buildcache/cache-keys/build-ui-f5d8c1975b103bbe13e2841b5e8a5d1a11c96e78"}}
+ - ui-dependencies_ac8b_{{checksum ".buildcache/cache-keys/ui-dependencies-ac8be120c6d5a16da43fee57c2cecb19a70f8098"}}
+ - install-yarn_3ec0_{{checksum ".buildcache/cache-keys/install-yarn-3ec09455a50e67ce0e6b9f03e2cefa3e97333a5b"}}
+ - set-workdir_3310_{{checksum ".buildcache/cache-keys/set-workdir-331006d1434fd1975dad2affbf71fdbf845d22d6"}}
+ - install-go-tools_dcaa_{{checksum ".buildcache/cache-keys/install-go-tools-dcaa9bb2de49ba79e84aa2ec9e02018c05e62950"}}
+ - install-go_d552_{{checksum ".buildcache/cache-keys/install-go-d55278f9cd49b917d025adaeed3032cc8b0dc7d7"}}
+ - base_c6cd_{{checksum ".buildcache/cache-keys/base-c6cdf1b224722d2520e082320f2a71875913247c"}}
+ name: 'Restore Builder Image Cache: copy-source'
+ - run:
+ command: make -C packages*.lock load-builder-cache
+ name: Load whatever builder cache we have (if any) into the Docker daemon
+ no_output_timeout: 30m
+ - run:
+ command: |2-
+
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
+ eval "$(ssh-agent -s)"
+ if [ -f "$KEYFILE" ]; then
+ ssh-add "$KEYFILE"
+ else
+ echo "==> INFO: SSH key for github.com not found"
+ echo " Attempts to access private repositories from within"
+ echo " the build will fail, e.g. for private go modules, or"
+ echo " attempts to directly clone private repositories."
+ fi
+
+ make -C packages*.lock package
+ name: Compile Package
+ - run:
+ command: ls -lahR .buildcache/packages
+ name: List packages
+ - save_cache:
+ key: package-2ee966c5768e83a92093c0bc6a2fc6042afe4839-{{checksum ".buildcache/cache-keys/package-2ee966c5768e83a92093c0bc6a2fc6042afe4839"}}
+ name: Save package cache
+ paths:
+ - .buildcache/packages/store
windows_386_package:
docker:
- image: docker.mirror.hashicorp.services/circleci/buildpack-deps
@@ -186,7 +267,7 @@ jobs:
version: 19.03.12
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -228,7 +309,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -278,7 +359,7 @@ jobs:
steps:
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -303,7 +384,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -319,7 +400,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -335,7 +416,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -351,7 +432,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -367,7 +448,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -383,7 +464,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -399,7 +480,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -415,7 +496,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -431,7 +512,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -447,7 +528,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -463,7 +544,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -500,7 +581,7 @@ jobs:
version: 19.03.12
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -542,7 +623,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -631,7 +712,7 @@ jobs:
version: 19.03.12
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -673,7 +754,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -792,7 +873,7 @@ jobs:
version: 19.03.12
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -834,7 +915,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -1052,7 +1133,7 @@ jobs:
version: 19.03.12
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -1094,7 +1175,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -1302,7 +1383,7 @@ jobs:
version: 19.03.12
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -1344,7 +1425,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -1380,7 +1461,7 @@ jobs:
version: 19.03.12
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -1422,7 +1503,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -1458,7 +1539,7 @@ jobs:
version: 19.03.12
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -1500,7 +1581,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -1559,7 +1640,7 @@ jobs:
version: 19.03.12
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
… diff truncated
.circleci/config/@build-release.yml+113 37
@@ -23,7 +23,7 @@ jobs:
steps:
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -48,7 +48,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -64,7 +64,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -80,7 +80,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -96,7 +96,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -112,7 +112,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -128,7 +128,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -144,7 +144,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -160,7 +160,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -176,7 +176,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -192,7 +192,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -208,7 +208,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -240,6 +240,9 @@ jobs:
- restore_cache:
key: package-2ffbc24c482a71c7f16a87ad5cc6c01038544fe3-{{checksum ".buildcache/cache-keys/package-2ffbc24c482a71c7f16a87ad5cc6c01038544fe3"}}
name: Restore package cache
+ - restore_cache:
+ key: package-2ee966c5768e83a92093c0bc6a2fc6042afe4839-{{checksum ".buildcache/cache-keys/package-2ee966c5768e83a92093c0bc6a2fc6042afe4839"}}
+ name: Restore package cache
- restore_cache:
key: package-61f4e059780eac8772bed584d10749860f2fdce1-{{checksum ".buildcache/cache-keys/package-61f4e059780eac8772bed584d10749860f2fdce1"}}
name: Restore package cache
@@ -292,20 +295,20 @@ jobs:
command: ls -lahR .buildcache
name: List Build Cache
- run:
- command: cp packages*.lock/pkgs.yml lockfile-7d1b28ede60990fa.yml
+ command: cp packages*.lock/pkgs.yml lockfile-e0096d0cd2ca1c5a.yml
name: Update Lockfile Name
- run:
- command: tar -czf packages-7d1b28ede60990fa.tar.gz .buildcache/packages lockfile-7d1b28ede60990fa.yml
+ command: tar -czf packages-e0096d0cd2ca1c5a.tar.gz .buildcache/packages lockfile-e0096d0cd2ca1c5a.yml
name: Create Raw Package Tarball
- run:
- command: tar -czf meta-7d1b28ede60990fa.tar.gz .buildcache/packages/store/*.json lockfile-7d1b28ede60990fa.yml
+ command: tar -czf meta-e0096d0cd2ca1c5a.tar.gz .buildcache/packages/store/*.json lockfile-e0096d0cd2ca1c5a.yml
name: Create Metadata Tarball
- store_artifacts:
- path: lockfile-7d1b28ede60990fa.yml
+ path: lockfile-e0096d0cd2ca1c5a.yml
- store_artifacts:
- path: packages-7d1b28ede60990fa.tar.gz
+ path: packages-e0096d0cd2ca1c5a.tar.gz
- store_artifacts:
- path: meta-7d1b28ede60990fa.tar.gz
+ path: meta-e0096d0cd2ca1c5a.tar.gz
- store_artifacts:
path: .buildcache/packages
darwin_amd64_package:
@@ -316,7 +319,7 @@ jobs:
docker_layer_caching: false
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -358,7 +361,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -381,6 +384,79 @@ jobs:
name: Save package cache
environment:
PACKAGE_SPEC_ID: 2ffbc24c482a71c7f16a87ad5cc6c01038544fe3
+ darwin_arm64_package:
+ executor: builder
+ steps:
+ - setup_remote_docker:
+ version: 19.03.12
+ docker_layer_caching: false
+ - add_ssh_keys:
+ fingerprints:
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
+ - checkout:
+ path: .
+ - run:
+ command: make -C packages*.lock write-package-cache-key
+ name: Write package cache key
+ - restore_cache:
+ key: package-2ee966c5768e83a92093c0bc6a2fc6042afe4839-{{checksum ".buildcache/cache-keys/package-2ee966c5768e83a92093c0bc6a2fc6042afe4839"}}
+ name: Restore package cache
+ - run:
+ command: |2
+
+ if ! { PKG=$(find .buildcache/packages/store -maxdepth 1 -mindepth 1 -name '*.zip' 2> /dev/null) && [ -n "$PKG" ]; }; then
+ echo "No package found, continuing with build."
+ exit 0
+ fi
+ echo "Package already cached, skipping build."
+ circleci-agent step halt
+ name: Check cache status
+ - run:
+ command: make -C packages*.lock write-builder-cache-keys
+ name: Write builder layer cache keys
+ - restore_cache:
+ key: copy-source_01bb_{{checksum ".buildcache/cache-keys/copy-source-01bb587fbaa40eee3270a2dfa7865a8e37dde482"}}
+ keys:
+ - go-modules_87d5_{{checksum ".buildcache/cache-keys/go-modules-87d5fe370bb634974f7b55a067206409c85d5947"}}
+ - build-static-assets_c9eb_{{checksum ".buildcache/cache-keys/build-static-assets-c9eb755a774c85539457676828158a137604f20d"}}
+ - build-ui_f5d8_{{checksum ".buildcache/cache-keys/build-ui-f5d8c1975b103bbe13e2841b5e8a5d1a11c96e78"}}
+ - ui-dependencies_ac8b_{{checksum ".buildcache/cache-keys/ui-dependencies-ac8be120c6d5a16da43fee57c2cecb19a70f8098"}}
+ - install-yarn_3ec0_{{checksum ".buildcache/cache-keys/install-yarn-3ec09455a50e67ce0e6b9f03e2cefa3e97333a5b"}}
+ - set-workdir_3310_{{checksum ".buildcache/cache-keys/set-workdir-331006d1434fd1975dad2affbf71fdbf845d22d6"}}
+ - install-go-tools_dcaa_{{checksum ".buildcache/cache-keys/install-go-tools-dcaa9bb2de49ba79e84aa2ec9e02018c05e62950"}}
+ - install-go_d552_{{checksum ".buildcache/cache-keys/install-go-d55278f9cd49b917d025adaeed3032cc8b0dc7d7"}}
+ - base_c6cd_{{checksum ".buildcache/cache-keys/base-c6cdf1b224722d2520e082320f2a71875913247c"}}
+ name: 'Restore Builder Image Cache: copy-source'
+ - run:
+ command: make -C packages*.lock load-builder-cache
+ name: Load whatever builder cache we have (if any) into the Docker daemon
+ no_output_timeout: 30m
+ - run:
+ command: |2-
+
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
+ eval "$(ssh-agent -s)"
+ if [ -f "$KEYFILE" ]; then
+ ssh-add "$KEYFILE"
+ else
+ echo "==> INFO: SSH key for github.com not found"
+ echo " Attempts to access private repositories from within"
+ echo " the build will fail, e.g. for private go modules, or"
+ echo " attempts to directly clone private repositories."
+ fi
+
+ make -C packages*.lock package
+ name: Compile Package
+ - run:
+ command: ls -lahR .buildcache/packages
+ name: List packages
+ - save_cache:
+ paths:
+ - .buildcache/packages/store
+ key: package-2ee966c5768e83a92093c0bc6a2fc6042afe4839-{{checksum ".buildcache/cache-keys/package-2ee966c5768e83a92093c0bc6a2fc6042afe4839"}}
+ name: Save package cache
+ environment:
+ PACKAGE_SPEC_ID: 2ee966c5768e83a92093c0bc6a2fc6042afe4839
freebsd_386_package:
executor: builder
steps:
@@ -389,7 +465,7 @@ jobs:
docker_layer_caching: false
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -431,7 +507,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -462,7 +538,7 @@ jobs:
docker_layer_caching: false
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -504,7 +580,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -535,7 +611,7 @@ jobs:
docker_layer_caching: false
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -577,7 +653,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -608,7 +684,7 @@ jobs:
docker_layer_caching: false
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -650,7 +726,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -681,7 +757,7 @@ jobs:
docker_layer_caching: false
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -723,7 +799,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -754,7 +830,7 @@ jobs:
docker_layer_caching: false
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -796,7 +872,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -827,7 +903,7 @@ jobs:
docker_layer_caching: false
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -869,7 +945,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -900,7 +976,7 @@ jobs:
docker_layer_caching: false
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -942,7 +1018,7 @@ jobs:
- run:
command: |2-
- KEYFILE="$HOME/.ssh/id_rsa_0e0377f4e2c356c2536a03e131912f06"
+ KEYFILE="$HOME/.ssh/id_rsa_c6969882dc046c39ddac8305e3151c98"
eval "$(ssh-agent -s)"
if [ -f "$KEYFILE" ]; then
ssh-add "$KEYFILE"
@@ -973,7 +1049,7 @@ jobs:
docker_layer_caching: false
- add_ssh_keys:
fingerprints:
- - 0e:03:77:f4:e2:c3:56:c2:53:6a:03:e1:31:91:2f:06
+ - c6:96:98:82:dc:04:6c:39:dd:ac:83:05:e3:15:1c:98
- checkout:
path: .
- run:
@@ -1015,7 +1091,7 @@ jobs:
- run:
command: |2-
… diff truncated
website/content/docs/enterprise/license/faq.mdx+1 1
@@ -6,7 +6,7 @@ description: An overview of license.
# Frequently Asked Questions (FAQ)
-This FAQ section is for the license changes introduced in Vault Enterprise 1.8. We will no longer support the old license system (`vault write sys/license`), and autoload will be the only way to manage your license.
+This FAQ section is for the license changes introduced in Vault Enterprise 1.8.
- [Q: When will these licensing changes be released for Vault?](#q-when-will-these-licensing-changes-be-released-for-vault)
- [Q: Will these license changes impact HCP Vault?](#q-will-these-license-changes-impact-hcp-vault)
.../content/docs/commands/operator/step-down.mdx | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
sdk/helper/strutil/strutil.go+19 323
@@ -1,480 +1,94 @@
+// DEPRECATED: this has been moved to go-secure-stdlib and will be removed
package strutil
import (
- "encoding/base64"
- "encoding/json"
- "fmt"
- "sort"
- "strings"
-
- "github.com/hashicorp/errwrap"
- glob "github.com/ryanuber/go-glob"
+ extstrutil "github.com/hashicorp/go-secure-stdlib/strutil"
)
-// StrListContainsGlob looks for a string in a list of strings and allows
-// globs.
func StrListContainsGlob(haystack []string, needle string) bool {
- for _, item := range haystack {
- if glob.Glob(item, needle) {
- return true
- }
- }
- return false
+ return extstrutil.StrListContainsGlob(haystack, needle)
}
-// StrListContains looks for a string in a list of strings.
func StrListContains(haystack []string, needle string) bool {
- for _, item := range haystack {
- if item == needle {
- return true
- }
- }
- return false
+ return extstrutil.StrListContains(haystack, needle)
}
-// StrListContainsCaseInsensitive looks for a string in a list of strings.
func StrListContainsCaseInsensitive(haystack []string, needle string) bool {
- for _, item := range haystack {
- if strings.EqualFold(item, needle) {
- return true
- }
- }
- return false
+ return extstrutil.StrListContainsCaseInsensitive(haystack, needle)
}
-// StrListSubset checks if a given list is a subset
-// of another set
func StrListSubset(super, sub []string) bool {
- for _, item := range sub {
- if !StrListContains(super, item) {
- return false
- }
- }
- return true
+ return extstrutil.StrListSubset(super, sub)
}
-// ParseDedupAndSortStrings parses a comma separated list of strings
-// into a slice of strings. The return slice will be sorted and will
-// not contain duplicate or empty items.
func ParseDedupAndSortStrings(input string, sep string) []string {
- input = strings.TrimSpace(input)
- parsed := []string{}
- if input == "" {
- // Don't return nil
- return parsed
- }
- return RemoveDuplicates(strings.Split(input, sep), false)
+ return extstrutil.ParseDedupAndSortStrings(input, sep)
}
-// ParseDedupLowercaseAndSortStrings parses a comma separated list of
-// strings into a slice of strings. The return slice will be sorted and
-// will not contain duplicate or empty items. The values will be converted
-// to lower case.
func ParseDedupLowercaseAndSortStrings(input string, sep string) []string {
- input = strings.TrimSpace(input)
- parsed := []string{}
- if input == "" {
- // Don't return nil
- return parsed
- }
- return RemoveDuplicates(strings.Split(input, sep), true)
+ return extstrutil.ParseDedupLowercaseAndSortStrings(input, sep)
}
-// ParseKeyValues parses a comma separated list of `<key>=<value>` tuples
-// into a map[string]string.
func ParseKeyValues(input string, out map[string]string, sep string) error {
- if out == nil {
- return fmt.Errorf("'out is nil")
- }
-
- keyValues := ParseDedupLowercaseAndSortStrings(input, sep)
- if len(keyValues) == 0 {
- return nil
- }
-
- for _, keyValue := range keyValues {
- shards := strings.Split(keyValue, "=")
- if len(shards) != 2 {
- return fmt.Errorf("invalid <key,value> format")
- }
-
- key := strings.TrimSpace(shards[0])
- value := strings.TrimSpace(shards[1])
- if key == "" || value == "" {
- return fmt.Errorf("invalid <key,value> pair: key: %q value: %q", key, value)
- }
- out[key] = value
- }
- return nil
+ return extstrutil.ParseKeyValues(input, out, sep)
}
-// ParseArbitraryKeyValues parses arbitrary <key,value> tuples. The input
-// can be one of the following:
-// * JSON string
-// * Base64 encoded JSON string
-// * Comma separated list of `<key>=<value>` pairs
-// * Base64 encoded string containing comma separated list of
-// `<key>=<value>` pairs
-//
-// Input will be parsed into the output parameter, which should
-// be a non-nil map[string]string.
func ParseArbitraryKeyValues(input string, out map[string]string, sep string) error {
- input = strings.TrimSpace(input)
- if input == "" {
- return nil
- }
- if out == nil {
- return fmt.Errorf("'out' is nil")
- }
-
- // Try to base64 decode the input. If successful, consider the decoded
- // value as input.
- inputBytes, err := base64.StdEncoding.DecodeString(input)
- if err == nil {
- input = string(inputBytes)
- }
-
- // Try to JSON unmarshal the input. If successful, consider that the
- // metadata was supplied as JSON input.
- err = json.Unmarshal([]byte(input), &out)
- if err != nil {
- // If JSON unmarshalling fails, consider that the input was
- // supplied as a comma separated string of 'key=value' pairs.
- if err = ParseKeyValues(input, out, sep); err != nil {
- return errwrap.Wrapf("failed to parse the input: {{err}}", err)
- }
- }
-
- // Validate the parsed input
- for key, value := range out {
- if key != "" && value == "" {
- return fmt.Errorf("invalid value for key %q", key)
- }
- }
-
- return nil
+ return extstrutil.ParseArbitraryKeyValues(input, out, sep)
}
-// ParseStringSlice parses a `sep`-separated list of strings into a
-// []string with surrounding whitespace removed.
-//
-// The output will always be a valid slice but may be of length zero.
func ParseStringSlice(input string, sep string) []string {
- input = strings.TrimSpace(input)
- if input == "" {
- return []string{}
- }
-
- splitStr := strings.Split(input, sep)
- ret := make([]string, len(splitStr))
- for i, val := range splitStr {
- ret[i] = strings.TrimSpace(val)
- }
-
- return ret
+ return extstrutil.ParseStringSlice(input, sep)
}
-// ParseArbitraryStringSlice parses arbitrary string slice. The input
-// can be one of the following:
-// * JSON string
-// * Base64 encoded JSON string
-// * `sep` separated list of values
-// * Base64-encoded string containing a `sep` separated list of values
-//
-// Note that the separator is ignored if the input is found to already be in a
-// structured format (e.g., JSON)
-//
-// The output will always be a valid slice but may be of length zero.
func ParseArbitraryStringSlice(input string, sep string) []string {
- input = strings.TrimSpace(input)
- if input == "" {
- return []string{}
- }
-
- // Try to base64 decode the input. If successful, consider the decoded
- // value as input.
- inputBytes, err := base64.StdEncoding.DecodeString(input)
- if err == nil {
- input = string(inputBytes)
- }
-
- ret := []string{}
-
- // Try to JSON unmarshal the input. If successful, consider that the
- // metadata was supplied as JSON input.
- err = json.Unmarshal([]byte(input), &ret)
- if err != nil {
- // If JSON unmarshalling fails, consider that the input was
- // supplied as a separated string of values.
- return ParseStringSlice(input, sep)
- }
-
- if ret == nil {
- return []string{}
- }
-
- return ret
+ return extstrutil.ParseArbitraryStringSlice(input, sep)
}
-// TrimStrings takes a slice of strings and returns a slice of strings
-// with trimmed spaces
func TrimStrings(items []string) []string {
- ret := make([]string, len(items))
- for i, item := range items {
- ret[i] = strings.TrimSpace(item)
- }
- return ret
+ return extstrutil.TrimStrings(items)
}
-// RemoveDuplicates removes duplicate and empty elements from a slice of
-// strings. This also may convert the items in the slice to lower case and
-// returns a sorted slice.
func RemoveDuplicates(items []string, lowercase bool) []string {
- itemsMap := map[string]bool{}
- for _, item := range items {
- item = strings.TrimSpace(item)
- if lowercase {
- item = strings.ToLower(item)
- }
- if item == "" {
- continue
- }
- itemsMap[item] = true
- }
- items = make([]string, 0, len(itemsMap))
- for item := range itemsMap {
- items = append(items, item)
- }
- sort.Strings(items)
- return items
+ return extstrutil.RemoveDuplicates(items, lowercase)
}
-// RemoveDuplicatesStable removes duplicate and empty elements from a slice of
-// strings, preserving order (and case) of the original slice.
-// In all cases, strings are compared after trimming whitespace
-// If caseInsensitive, strings will be compared after ToLower()
func RemoveDuplicatesStable(items []string, caseInsensitive bool) []string {
- itemsMap := make(map[string]bool, len(items))
- deduplicated := make([]string, 0, len(items))
-
- for _, item := range items {
- key := strings.TrimSpace(item)
- if caseInsensitive {
- key = strings.ToLower(key)
- }
- if key == "" || itemsMap[key] {
- continue
- }
- itemsMap[key] = true
- deduplicated = append(deduplicated, item)
- }
- return deduplicated
+ return extstrutil.RemoveDuplicatesStable(items, caseInsensitive)
}
-// RemoveEmpty removes empty elements from a slice of
-// strings
func RemoveEmpty(items []string) []string {
- if len(items) == 0 {
- return items
- }
- itemsSlice := make([]string, 0, len(items))
- for _, item := range items {
- if item == "" {
- continue
- }
- itemsSlice = append(itemsSlice, item)
- }
- return itemsSlice
+ return extstrutil.RemoveEmpty(items)
}
-// EquivalentSlices checks whether the given string sets are equivalent, as in,
-// they contain the same values.
func EquivalentSlices(a, b []string) bool {
- if a == nil && b == nil {
- return true
- }
-
- if a == nil || b == nil {
- return false
- }
-
- // First we'll build maps to ensure unique values
- mapA := map[string]bool{}
- mapB := map[string]bool{}
- for _, keyA := range a {
- mapA[keyA] = true
- }
- for _, keyB := range b {
- mapB[keyB] = true
- }
-
- // Now we'll build our checking slices
- var sortedA, sortedB []string
- for keyA := range mapA {
- sortedA = append(sortedA, keyA)
- }
- for keyB := range mapB {
- sortedB = append(sortedB, keyB)
- }
- sort.Strings(sortedA)
- sort.Strings(sortedB)
-
- // Finally, compare
- if len(sortedA) != len(sortedB) {
- return false
- }
-
- for i := range sortedA {
- if sortedA[i] != sortedB[i] {
- return false
- }
- }
-
- return true
+ return extstrutil.EquivalentSlices(a, b)
}
-// EqualStringMaps tests whether two map[string]string objects are equal.
-// Equal means both maps have the same sets of keys and values. This function
-// is 6-10x faster than a call to reflect.DeepEqual().
func EqualStringMaps(a, b map[string]string) bool {
- if len(a) != len(b) {
- return false
- }
-
- for k := range a {
- v, ok := b[k]
- if !ok || a[k] != v {
- return false
- }
- }
-
- return true
+ return extstrutil.EqualStringMaps(a, b)
}
-// StrListDelete removes the first occurrence of the given item from the slice
-// of strings if the item exists.
func StrListDelete(s []string, d string) []string {
- if s == nil {
- return s
- }
-
- for index, element := range s {
- if element == d {
- return append(s[:index], s[index+1:]...)
- }
- }
-
- return s
+ return extstrutil.StrListDelete(s, d)
}
-// GlobbedStringsMatch compares item to val with support for a leading and/or
-// trailing wildcard '*' in item.
func GlobbedStringsMatch(item, val string) bool {
- if len(item) < 2 {
- return val == item
- }
-
- hasPrefix := strings.HasPrefix(item, "*")
- hasSuffix := strings.HasSuffix(item, "*")
-
- if hasPrefix && hasSuffix {
… diff truncated
builtin/credential/aws/path_config_identity.go+1 1
@@ -4,9 +4,9 @@ import (
"context"
"fmt"
+ "github.com/hashicorp/go-secure-stdlib/strutil"
"github.com/hashicorp/vault/sdk/framework"
"github.com/hashicorp/vault/sdk/helper/authmetadata"
- "github.com/hashicorp/vault/sdk/helper/strutil"
"github.com/hashicorp/vault/sdk/logical"
)
changelog/11887.txt+3 0
@@ -0,0 +1,3 @@
+```release-note:bug
+secret/totp: pad input key to ensure length is a multiple of 8
+```
(#12071)
.circleci/config.yml | 183 ++++++++++++++++++++--------
.circleci/config/@build-release.yml | 178 +++++++++++++++++++--------
changelog/12071.txt | 3 +
packages-oss.lock/Makefile | 2 -
packages-oss.lock/pkgs.yml | 72 ++++++++++-
packages-oss.yml | 1 +
6 files changed, 337 insertions(+), 102 deletions(-)
create mode 100644 changelog/12071.txt
website/content/docs/commands/operator/step-down.mdx+8 5
@@ -8,11 +8,14 @@ description: |-
# operator step-down
-The `operator step-down` forces the Vault server at the given address to step
-down from active duty. While the affected node will have a delay before
-attempting to acquire the leader lock again, if no other Vault nodes acquire the
-lock beforehand, it is possible for the same node to re-acquire the lock and
-become active again.
+The `operator step-down` forces the active Vault node within an [HA cluster](/docs/concepts/ha)
+to step down from active duty. When executed against a non-active node, i.e. a
+standby or [performance standby](/docs/enterprise/performance-standby) node,
+the request will be forwarded to the active node. While the
+affected node will have a delay before attempting to acquire
+the leader lock again, if no other Vault nodes acquire the
+lock beforehand, it is possible for the same node to re-acquire
+the lock and become active again.
## Examples
fixes (#12065)
website/content/docs/secrets/identity.mdx | 22 +++++++++++-----------
1 file changed, 11 insertions(+), 11 deletions(-)
changelog/12061.txt+3 0
@@ -0,0 +1,3 @@
+```release-note:bug
+core (enterprise): namespace header included in responses, Go client uses it when displaying error messages
+```
ui/lib/core/addon/components/linked-block.js+22 0
@@ -3,6 +3,28 @@ import Component from '@ember/component';
import hbs from 'htmlbars-inline-precompile';
import { encodePath } from 'vault/utils/path-encoding-helpers';
+/**
+ * @module LinkedBlock
+ * LinkedBlock components are linkable divs that yield any content nested within them. They are often used in list views such as when listing the secret engines.
+ *
+ * @example
+ * ```js
+ * <LinkedBlock
+ * @params={{array 'vault.cluster.secrets.backend.show 'my-secret-path'}}
+ * @queryParams={{hash version=1}}
+ * @class="list-item-row"
+ * data-test-list-item-link
+ * >
+ * // Use any wrapped content here
+ * </LinkedBlock>
+ * ```
+ *
+ * @param {Array} params=null - These are values sent to the router's transitionTo method. First item is route, second is the optional path.
+ * @param {Object} [queryParams=null] - queryParams can be passed via this property. It needs to be an object.
+ * @param {String} [linkPrefix=null] - Overwrite the params with custom route. See KMIP.
+ * @param {Boolean} [encode=false] - Encode the path.
+ */
+
let LinkedBlockComponent = Component.extend({
router: service(),
sdk/helper/base62/base62.go+4 40
@@ -1,52 +1,16 @@
-// Package base62 provides utilities for working with base62 strings.
-// base62 strings will only contain characters: 0-9, a-z, A-Z
+// DEPRECATED: this has been moved to go-secure-stdlib and will be removed
package base62
import (
- "crypto/rand"
"io"
- uuid "github.com/hashicorp/go-uuid"
+ extbase62 "github.com/hashicorp/go-secure-stdlib/base62"
)
-const (
- charset = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
- csLen = byte(len(charset))
-)
-
-// Random generates a random string using base-62 characters.
-// Resulting entropy is ~5.95 bits/character.
func Random(length int) (string, error) {
- return RandomWithReader(length, rand.Reader)
+ return extbase62.Random(length)
}
-// RandomWithReader generates a random string using base-62 characters and a given reader.
-// Resulting entropy is ~5.95 bits/character.
func RandomWithReader(length int, reader io.Reader) (string, error) {
- if length == 0 {
- return "", nil
- }
- output := make([]byte, 0, length)
-
- // Request a bit more than length to reduce the chance
- // of needing more than one batch of random bytes
- batchSize := length + length/4
-
- for {
- buf, err := uuid.GenerateRandomBytesWithReader(batchSize, reader)
- if err != nil {
- return "", err
- }
-
- for _, b := range buf {
- // Avoid bias by using a value range that's a multiple of 62
- if b < (csLen * 4) {
- output = append(output, charset[b%csLen])
-
- if len(output) == length {
- return string(output), nil
- }
- }
- }
- }
+ return extbase62.RandomWithReader(length, reader)
}
sdk/helper/strutil/strutil_test.go+0 399
@@ -1,666 +0,0 @@
-package strutil
-
-import (
- "encoding/base64"
- "encoding/json"
- "reflect"
- "testing"
-)
-
-func TestStrUtil_StrListDelete(t *testing.T) {
- output := StrListDelete([]string{"item1", "item2", "item3"}, "item1")
- if StrListContains(output, "item1") {
- t.Fatal("bad: 'item1' should not have been present")
- }
-
- output = StrListDelete([]string{"item1", "item2", "item3"}, "item2")
- if StrListContains(output, "item2") {
- t.Fatal("bad: 'item2' should not have been present")
- }
-
- output = StrListDelete([]string{"item1", "item2", "item3"}, "item3")
- if StrListContains(output, "item3") {
- t.Fatal("bad: 'item3' should not have been present")
- }
-
- output = StrListDelete([]string{"item1", "item1", "item3"}, "item1")
- if !StrListContains(output, "item1") {
- t.Fatal("bad: 'item1' should have been present")
- }
-
- output = StrListDelete(output, "item1")
- if StrListContains(output, "item1") {
- t.Fatal("bad: 'item1' should not have been present")
- }
-
- output = StrListDelete(output, "random")
- if len(output) != 1 {
- t.Fatalf("bad: expected: 1, actual: %d", len(output))
- }
-
- output = StrListDelete(output, "item3")
- if StrListContains(output, "item3") {
- t.Fatal("bad: 'item3' should not have been present")
- }
-}
-
-func TestStrutil_EquivalentSlices(t *testing.T) {
- slice1 := []string{"test2", "test1", "test3"}
- slice2 := []string{"test3", "test2", "test1"}
- if !EquivalentSlices(slice1, slice2) {
- t.Fatalf("bad: expected a match")
- }
-
- slice2 = append(slice2, "test4")
- if EquivalentSlices(slice1, slice2) {
- t.Fatalf("bad: expected a mismatch")
- }
-}
-
-func TestStrutil_ListContainsGlob(t *testing.T) {
- haystack := []string{
- "dev",
- "ops*",
- "root/*",
- "*-dev",
- "_*_",
- }
- if StrListContainsGlob(haystack, "tubez") {
- t.Fatalf("Value shouldn't exist")
- }
- if !StrListContainsGlob(haystack, "root/test") {
- t.Fatalf("Value should exist")
- }
- if !StrListContainsGlob(haystack, "ops_test") {
- t.Fatalf("Value should exist")
- }
- if !StrListContainsGlob(haystack, "ops") {
- t.Fatalf("Value should exist")
- }
- if !StrListContainsGlob(haystack, "dev") {
- t.Fatalf("Value should exist")
- }
- if !StrListContainsGlob(haystack, "test-dev") {
- t.Fatalf("Value should exist")
- }
- if !StrListContainsGlob(haystack, "_test_") {
- t.Fatalf("Value should exist")
- }
-}
-
-func TestStrutil_ListContains(t *testing.T) {
- haystack := []string{
- "dev",
- "ops",
- "prod",
- "root",
- }
- if StrListContains(haystack, "tubez") {
- t.Fatalf("Bad")
- }
- if !StrListContains(haystack, "root") {
- t.Fatalf("Bad")
- }
-}
-
-func TestStrutil_ListSubset(t *testing.T) {
- parent := []string{
- "dev",
- "ops",
- "prod",
- "root",
- }
- child := []string{
- "prod",
- "ops",
- }
- if !StrListSubset(parent, child) {
- t.Fatalf("Bad")
- }
- if !StrListSubset(parent, parent) {
- t.Fatalf("Bad")
- }
- if !StrListSubset(child, child) {
- t.Fatalf("Bad")
- }
- if !StrListSubset(child, nil) {
- t.Fatalf("Bad")
- }
- if StrListSubset(child, parent) {
- t.Fatalf("Bad")
- }
- if StrListSubset(nil, child) {
- t.Fatalf("Bad")
- }
-}
-
-func TestStrutil_ParseKeyValues(t *testing.T) {
- actual := make(map[string]string)
- expected := map[string]string{
- "key1": "value1",
- "key2": "value2",
- }
- var input string
- var err error
-
- input = "key1=value1,key2=value2"
- err = ParseKeyValues(input, actual, ",")
- if err != nil {
- t.Fatal(err)
- }
- if !reflect.DeepEqual(expected, actual) {
- t.Fatalf("bad: expected: %#v\nactual: %#v", expected, actual)
- }
- for k := range actual {
- delete(actual, k)
- }
-
- input = "key1 = value1, key2 = value2"
- err = ParseKeyValues(input, actual, ",")
- if err != nil {
- t.Fatal(err)
- }
- if !reflect.DeepEqual(expected, actual) {
- t.Fatalf("bad: expected: %#v\nactual: %#v", expected, actual)
- }
- for k := range actual {
- delete(actual, k)
- }
-
- input = "key1 = value1, key2 = "
- err = ParseKeyValues(input, actual, ",")
- if err == nil {
- t.Fatalf("expected an error")
- }
- for k := range actual {
- delete(actual, k)
- }
-
- input = "key1 = value1, = value2 "
- err = ParseKeyValues(input, actual, ",")
- if err == nil {
- t.Fatalf("expected an error")
- }
- for k := range actual {
- delete(actual, k)
- }
-
- input = "key1"
- err = ParseKeyValues(input, actual, ",")
- if err == nil {
- t.Fatalf("expected an error")
- }
-}
-
-func TestStrutil_ParseArbitraryKeyValues(t *testing.T) {
- actual := make(map[string]string)
- expected := map[string]string{
- "key1": "value1",
- "key2": "value2",
- }
- var input string
- var err error
-
- // Test <key>=<value> as comma separated string
- input = "key1=value1,key2=value2"
- err = ParseArbitraryKeyValues(input, actual, ",")
- if err != nil {
- t.Fatal(err)
- }
- if !reflect.DeepEqual(expected, actual) {
- t.Fatalf("bad: expected: %#v\nactual: %#v", expected, actual)
- }
- for k := range actual {
- delete(actual, k)
- }
-
- // Test <key>=<value> as base64 encoded comma separated string
- input = base64.StdEncoding.EncodeToString([]byte(input))
- err = ParseArbitraryKeyValues(input, actual, ",")
- if err != nil {
- t.Fatal(err)
- }
- if !reflect.DeepEqual(expected, actual) {
- t.Fatalf("bad: expected: %#v\nactual: %#v", expected, actual)
- }
- for k := range actual {
- delete(actual, k)
- }
-
- // Test JSON encoded <key>=<value> tuples
- input = `{"key1":"value1", "key2":"value2"}`
- err = ParseArbitraryKeyValues(input, actual, ",")
- if err != nil {
- t.Fatal(err)
- }
- if !reflect.DeepEqual(expected, actual) {
- t.Fatalf("bad: expected: %#v\nactual: %#v", expected, actual)
- }
- for k := range actual {
- delete(actual, k)
- }
-
- // Test base64 encoded JSON string of <key>=<value> tuples
- input = base64.StdEncoding.EncodeToString([]byte(input))
- err = ParseArbitraryKeyValues(input, actual, ",")
- if err != nil {
- t.Fatal(err)
- }
- if !reflect.DeepEqual(expected, actual) {
- t.Fatalf("bad: expected: %#v\nactual: %#v", expected, actual)
- }
- for k := range actual {
- delete(actual, k)
- }
-}
-
-func TestStrutil_ParseArbitraryStringSlice(t *testing.T) {
- input := `CREATE ROLE "{{name}}" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';GRANT "foo-role" TO "{{name}}";ALTER ROLE "{{name}}" SET search_path = foo;GRANT CONNECT ON DATABASE "postgres" TO "{{name}}";`
-
- jsonExpected := []string{
- `DO $$
-BEGIN
- IF NOT EXISTS (SELECT * FROM pg_catalog.pg_roles WHERE rolname='foo-role') THEN
- CREATE ROLE "foo-role";
- CREATE SCHEMA IF NOT EXISTS foo AUTHORIZATION "foo-role";
- ALTER ROLE "foo-role" SET search_path = foo;
- GRANT TEMPORARY ON DATABASE "postgres" TO "foo-role";
- GRANT ALL PRIVILEGES ON SCHEMA foo TO "foo-role";
- GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA foo TO "foo-role";
- GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA foo TO "foo-role";
- GRANT ALL PRIVILEGES ON ALL FUNCTIONS IN SCHEMA foo TO "foo-role";
- END IF;
-END
-$$`,
- `CREATE ROLE "{{name}}" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'`,
- `GRANT "foo-role" TO "{{name}}"`,
- `ALTER ROLE "{{name}}" SET search_path = foo`,
- `GRANT CONNECT ON DATABASE "postgres" TO "{{name}}"`,
- ``,
- }
-
- nonJSONExpected := jsonExpected[1:]
-
- var actual []string
- var inputB64 string
- var err error
-
- // Test non-JSON string
- actual = ParseArbitraryStringSlice(input, ";")
- if !reflect.DeepEqual(nonJSONExpected, actual) {
- t.Fatalf("bad: expected:\n%#v\nactual:\n%#v", nonJSONExpected, actual)
- }
-
- // Test base64-encoded non-JSON string
- inputB64 = base64.StdEncoding.EncodeToString([]byte(input))
- actual = ParseArbitraryStringSlice(inputB64, ";")
- if !reflect.DeepEqual(nonJSONExpected, actual) {
- t.Fatalf("bad: expected:\n%#v\nactual:\n%#v", nonJSONExpected, actual)
- }
-
- // Test JSON encoded
- inputJSON, err := json.Marshal(jsonExpected)
- if err != nil {
- t.Fatal(err)
- }
-
- actual = ParseArbitraryStringSlice(string(inputJSON), ";")
- if !reflect.DeepEqual(jsonExpected, actual) {
- t.Fatalf("bad: expected:\n%#v\nactual:\n%#v", string(inputJSON), actual)
- }
-
- // Test base64 encoded JSON string of <key>=<value> tuples
- inputB64 = base64.StdEncoding.EncodeToString(inputJSON)
- actual = ParseArbitraryStringSlice(inputB64, ";")
- if !reflect.DeepEqual(jsonExpected, actual) {
- t.Fatalf("bad: expected:\n%#v\nactual:\n%#v", jsonExpected, actual)
- }
-}
-
-func TestGlobbedStringsMatch(t *testing.T) {
- type tCase struct {
- item string
- val string
- expect bool
- }
-
- tCases := []tCase{
- {"", "", true},
- {"*", "*", true},
- {"**", "**", true},
- {"*t", "t", true},
- {"*t", "test", true},
- {"t*", "test", true},
- {"*test", "test", true},
- {"*test", "a test", true},
- {"test", "a test", false},
- {"*test", "tests", false},
- {"test*", "test", true},
- {"test*", "testsss", true},
- {"test**", "testsss", false},
- {"test**", "test*", true},
- {"**test", "*test", true},
- {"TEST", "test", false},
- {"test", "test", true},
- }
-
- for _, tc := range tCases {
- actual := GlobbedStringsMatch(tc.item, tc.val)
-
- if actual != tc.expect {
- t.Fatalf("Bad testcase %#v, expected %t, got %t", tc, tc.expect, actual)
- }
- }
-}
-
-func TestTrimStrings(t *testing.T) {
- input := []string{"abc", "123", "abcd ", "123 "}
- expected := []string{"abc", "123", "abcd", "123"}
- actual := TrimStrings(input)
- if !reflect.DeepEqual(expected, actual) {
- t.Fatalf("Bad TrimStrings: expected:%#v, got:%#v", expected, actual)
- }
-}
-
-func TestRemoveEmpty(t *testing.T) {
- input := []string{"abc", "", "abc", ""}
- expected := []string{"abc", "abc"}
- actual := RemoveEmpty(input)
- if !reflect.DeepEqual(expected, actual) {
- t.Fatalf("Bad TrimStrings: expected:%#v, got:%#v", expected, actual)
- }
-
- input = []string{""}
- expected = []string{}
- actual = RemoveEmpty(input)
- if !reflect.DeepEqual(expected, actual) {
- t.Fatalf("Bad TrimStrings: expected:%#v, got:%#v", expected, actual)
- }
-}
-
-func TestStrutil_AppendIfMissing(t *testing.T) {
- keys := []string{}
-
- keys = AppendIfMissing(keys, "foo")
-
- if len(keys) != 1 {
- t.Fatalf("expected slice to be length of 1: %v", keys)
- }
- if keys[0] != "foo" {
- t.Fatalf("expected slice to contain key 'foo': %v", keys)
- }
-
- keys = AppendIfMissing(keys, "bar")
-
- if len(keys) != 2 {
- t.Fatalf("expected slice to be length of 2: %v", keys)
- }
- if keys[0] != "foo" {
- t.Fatalf("expected slice to contain key 'foo': %v", keys)
… diff truncated
More files changed — see the full commit.

References