Security context
High· 8.1GHSA-r3w7-mfpm-c2vw CVE-2024-2048CWE-295Published Mar 4, 2024

Incorrect TLS certificate auth method in Vault

Research this vulnerability

Research is free — Hunters explains how the bug works, the root-cause code pattern, how the fix addresses it, and how to test whether a target is affected, in chat. Investigate & write exploit is a paid run — the engine reads the advisory and fix commits, then builds and validates a working proof-of-concept exploit with reproduction steps.

Affected versions

1.15.0 → fixed in 1.15.50 → fixed in 1.14.10

Details

Vault and Vault Enterprise (“Vault”) TLS certificate auth method did not correctly validate client certificates when configured with a non-CA certificate as trusted certificate. In this configuration, an attacker may be able to craft a malicious certificate that could be used to bypass authentication. Fixed in Vault 1.15.5 and 1.14.10.

The fix

Release delta 1.15.0 → 1.15.5 (contains the fix)

· Sep 26, 2023, 06:18 PM+48691042compare
website/content/api-docs/auth/saml.mdx+399 0
@@ -0,0 +1,440 @@
+---
+layout: api
+page_title: SAML - Auth Methods - HTTP API
+description: |-
+ This is the API documentation for the Vault SAML auth method.
+---
+
+# SAML auth method (API)
+
+<EnterpriseAlert />
+
+This is the API documentation for the Vault SAML auth method. To learn more about the
+usage and operation, see the [Vault SAML auth method documentation](/vault/docs/auth/saml).
+
+This documentation assumes the SAML auth method is mounted at the `/auth/saml` path in
+Vault. Since it is possible to enable auth methods at any location, please update your
+API calls accordingly.
+
+## Create or update configuration
+
+Configures the auth method with a SAML identity provider.
+
+| Method | Path |
+|:-----------| :------------------ |
+| `POST/PUT` | `/auth/saml/config` |
+
+### Parameters
+
+- `idp_metadata_url` `(string, <required>)` - The metadata URL of the identity provider.
+ Mutually exclusive with `idp_sso_url`, `idp_issuer` and `idp_cert`. Must be a
+ well-formatted URL.
+- `idp_sso_url` `(string, <required if idp_metadata_url is not set>)` - The SSO URL of the
+ identity provider. Mutually exclusive with `idp_metadata_url`. Must be a
+ well-formatted URL.
+- `idp_entity_id` `(string, <required if idp_metadata_url is not set>)` - The entity ID of
+ the identity provider. Mutually exclusive with `idp_metadata_url`.
+- `idp_cert` `(string, <required if idp_metadata_url is not set>)` - The PEM-encoded
+ certificate of the identity provider used to verify response and assertion signatures.
+ Mutually exclusive with `idp_metadata_url`.
+- `entity_id` `(string, <required>)` - The entity ID of the SAML authentication
+ service provider. Must match entity ID configured for the application in the
+ SAML identity provider.
+- `acs_urls` `(list, <required>)` - The well-formated URLs of your Assertion
+ Consumer Service (ACS) that should receive a response from the identity
+ provider. Vault returns a security warning if any of the given URLs lack TLS
+ protection.
+- `default_role` `(string, <optional>)` - The role to use if no role is provided during login.
+ If not set, a role is required during login.
+
+### Sample payload
+
+```json
+{
+ "acs_urls": "https://my.vault/v1/auth/saml/callback",
+ "default_role": "admin",
+ "entity_id": "https://my.vault/v1/auth/saml",
+ "idp_metadata_url": "https://company.okta.com/app/abc123eb9xnIfzlaf697/sso/saml/metadata"
+}
+```
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --request PUT \
+ --data @payload.json \
+ http://127.0.0.1:8200/v1/auth/saml/config
+```
+
+## Read configuration
+
+Reads the auth method configuration.
+
+| Method | Path |
+| :------ | :------------------ |
+| `GET` | `/auth/saml/config` |
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --request GET \
+ --data @payload.json \
+ http://127.0.0.1:8200/v1/auth/saml/config
+```
+
+### Sample response
+
+```json
+{
+ "request_id": "09c907d2-2dbe-8a5c-ca97-fad83195738b",
+ "lease_id": "",
+ "lease_duration": 0,
+ "renewable": false,
+ "data": {
+ "acs_urls": [
+ "https://my.vault/v1/auth/saml/callback"
+ ],
+ "default_role": "admin",
+ "entity_id": "https://my.vault/v1/auth/saml",
+ "idp_metadata_url": "https://company.okta.com/app/abc123eb9xnIfzlaf697/sso/saml/metadata"
+ },
+ "warnings": null
+}
+```
+
+## Create or update role
+
+Configures a role in the auth method. Roles define specific constraints required for
+authentication and properties of resulting Vault tokens.
+
+| Method | Path |
+|:-----------|:-------------------------|
+| `POST/PUT` | `/auth/saml/role/:name` |
+
+### Parameters
+
+- `name` `(string: <required>)` - URL parameter that provides the name of the role to create.
+- `bound_subjects` `(string: <optional>)` - The subject being asserted for SAML
+ authentication. One of the provided values must match the subject returned in
+ the SAML assertion from the identity provider.
+- `bound_subjects_type` `(string: <optional>)` - The type of matching assertion to perform
+ on `bound_subjects`. If `string`, requires a direct string match. If `glob`, allows for
+ wildcard matching using the `*` character.
+- `bound_attributes` `(map: <optional>)` - Mapping of attribute names to values that are
+ expected to exist in the SAML assertion. The expected value may be a single string or a
+ comma-separated list of strings. The user will be authenticated if the SAML attributes
+ match at least one of the expected values.
+- `bound_attributes_type` `(string: "string")` - The type of matching assertion to perform
+ on the key-value pairs provided by `bound_attributes`. If set to `string`, a direct string
+ match is required. If set to `glob`, allows for wildcard matching using the `*` character.
+- `groups_attribute` `(string: <optional>)` - The attribute to use to identify the set of
+ groups to which the user belongs. This will be used as the names for the Identity group
+ aliases created due to a successful login.
+
+@include 'tokenfields.mdx'
+
+### Sample payload
+
+```json
+{
+ "bound_attributes": "group=admin",
+ "bound_subjects": "*@hashicorp.com",
+ "bound_subjects_type": "glob",
+ "token_policies": "writer",
+ "ttl": "1h"
+}
+```
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --request PUT \
+ --data @payload.json \
+ http://127.0.0.1:8200/v1/auth/saml/role/admin
+```
+
+## Read role
+
+Reads a configured role.
+
+| Method | Path |
+| :----- |:------------------------|
+| `GET` | `/auth/saml/role/:name` |
+
+### Parameters
+
+- `name` `(string: <required>)` - URL parameter that provides the name of the
+ role to read.
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --request GET \
+ http://127.0.0.1:8200/v1/auth/saml/role/admin
+```
+
+### Sample response
+
+```json
+{
+ "request_id": "3148ca9a-286e-a0a4-5a4b-31b6abb63d37",
+ "lease_id": "",
+ "lease_duration": 0,
+ "renewable": false,
+ "data": {
+ "bound_attributes": {
+ "group": [
+ "admin"
+ ]
+ },
+ "bound_attributes_type": "string",
+ "bound_subjects": [
+ "*@hashicorp.com"
+ ],
+ "bound_subjects_type": "glob",
+ "groups_attribute": "",
+ "token_bound_cidrs": [],
+ "token_explicit_max_ttl": 0,
+ "token_max_ttl": 0,
+ "token_no_default_policy": false,
+ "token_num_uses": 0,
+ "token_period": 0,
+ "token_policies": [
+ "writer"
+ ],
+ "token_ttl": 0,
+ "token_type": "default"
+ },
+ "warnings": null
+}
+```
+
+## List roles
+
+Lists all the configured roles.
+
+| Method | Path |
+| :----- | :---------------- |
+| `LIST` | `/auth/saml/role` |
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --request GET \
+ http://127.0.0.1:8200/v1/auth/saml/role?list=true
+```
+
+### Sample response
+
+```json
+[
+ "admin",
+ "operations"
+]
+```
+
+## Delete Role
+
+Deletes a configured role.
+
+| Method | Path |
+| :------- | :---------------------- |
+| `DELETE` | `/auth/saml/role/:name` |
+
+### Parameters
+
+- `name` `(string: <required>)` - URL parameter that provides the name of the
+ role to delete.
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --request DELETE \
+ http://127.0.0.1:8200/v1/auth/saml/role/admin
+```
+
+## Obtain SSO service URL
+
+Starts a login flow by providing a SAML Single Sign-On (SSO) Service URL for the
+configured identity provider. The returned `token_poll_id` can be used to obtain
+the Vault token after the user is authenticated with the identity provider and the
+SAML response has passed validation.
+
+<Note title="Unauthenticated">
+A Vault token is not required to interact with this API.
+</Note>
+
+| Method | Path |
+|:--------|:-----------------------------|
+| `POST` | `/auth/saml/sso_service_url` |
+
+### Parameters
+
+- `role` `(string, <optional>)` - The role name to use for the login flow.
+ Defaults to the role configured with `default_role`.
+- `client_challenge` `(string, <required>)` - The client challenge value. Must be the
+ output of a base64-encoded, sha256 digest of the `client_verifier` eventually provided
+ to the [Token API](/vault/api-docs/auth/saml#obtain-vault-token). Must be at least 44
+ bytes in length.
+- `client_type` `(string, <required>)` - The type of the requesting client. The response
+ from the Assertion Consumer Service [Callback API](/vault/api-docs/auth/saml#assertion-consumer-service-callback)
+ will differ based on the provided type. If `cli`, an HTML success page will be returned
+ in the response. If `browser`, a blank HTML page will be returned in the response.
+- `acs_url` `(string, <required>)` - The URL where the identity provider will send its
+ SAML response. Must be in the set of configured [`acs_urls`](/vault/api-docs/auth/saml#acs_urls).
+
+### Sample payload
+
+```json
+{
+ "acs_url": "https://my.vault/v1/auth/saml/callback",
+ "client_challenge": "Z6+7owP80d1aHTha1kdixtT99JkvmG4TPSgbvDwZ70A=",
+ "client_type": "cli",
+ "role": "admin"
+}
+```
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --request POST \
+ --data @payload.json \
+ http://127.0.0.1:8200/v1/auth/saml/sso_service_url
+```
+
+### Sample response
+
+```json
+{
+ "sso_service_url": "https://example.okta.com/app/abc123eb9xnIfzlaf697/id/sso/saml?RelayState=...&SAMLRequest=...",
+ "token_poll_id": "ee442348-159b-df10-4c59-63050069df4d"
+}
+```
+
+## Assertion consumer service callback
+
+The assertion consumer service URL of the auth method. Completes the round trip from
+the identity provider and performs validations on the SAML response.
+
+<Note title="Unauthenticated">
+A Vault token is not required to interact with this API.
+</Note>
+
+| Method | Path |
+|:--------|:----------------------|
+| `POST` | `/auth/saml/callback` |
+
+### Parameters
+
+- `RelayState` `(string, <required>)` - The relay state from the original SAML
+ authentication request that was returned by the identity provider.
+- `SAMLResponse` `(string, <required>)` - The signed SAML response from the identity
+ provider.
+
+### Sample payload
+
+```json
+{
+ "RelayState": "0afe62a9-7b83-a182-0650-c749badfb900",
+ "SAMLResponse": "..."
+}
+```
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --request POST \
+ --data @payload.json \
+ http://127.0.0.1:8200/v1/auth/saml/callback
+```
+
+## Obtain vault token
+
+The token endpoint completes the login flow by returning a Vault token.
+
+<Note title="Unauthenticated">
+A Vault token is not required to interact with this API.
+</Note>
+
+| Method | Path |
+|:--------|:-------------------|
+| `POST` | `/auth/saml/token` |
+
+### Parameters
+
+- `client_verifier` `(string, <required>)` - The value which produced the `client_challenge`
+ provided to the [SSO Service URL API](/vault/api-docs/auth/saml#obtain-sso-service-url)
+ at the start of the authentication flow. Its base64-encoded, sha256 digest must match the
+ `client_challenge` value.
+- `token_poll_id` `(string, <required>)` - The `token_poll_id` value returned from the
+ [SSO Service URL API](/vault/api-docs/auth/saml#obtain-sso-service-url) at the start of
+ the authentication flow.
+
+### Sample payload
+
+```json
+{
+ "client_verifier": "59634224-5869-6002-e0b1-35370b8f6b82",
+ "token_poll_id": "ee442348-159b-df10-4c59-63050069df4d"
+}
+```
+
+### Sample request
+
… diff truncated
website/content/docs/auth/saml.mdx+181 0
@@ -0,0 +1,181 @@
+---
+layout: docs
+page_title: SAML - Auth Methods
+description: >-
+ The "saml" auth method allows users to authenticate with Vault using their
+ identity in a SAML identity provider.
+---
+
+# SAML auth method
+
+<EnterpriseAlert />
+
+The `saml` auth method allows users to authentication with Vault using their identity
+within a [SAML V2.0](https://saml.xml.org/saml-specifications) identity provider.
+Authentication is suited for human users by requiring interaction with a web browser.
+
+## Authentication
+
+<Tabs>
+
+<Tab heading="Vault CLI">
+
+The CLI login defaults to the `/saml` path. If this auth method was enabled at a
+different path, specify `-path=/my-path` in the CLI.
+
+```shell-session
+$ vault login -method=saml role=admin
+
+Complete the login via your SAML provider. Launching browser to:
+
+ https://company.okta.com/app/vault/abc123eb9xnIfzlaf697/sso/saml?SAMLRequest=fJI9b9swEIZ3%2FwqBu0SJ%2FpBDRAZce4iBtDViN0MX40Sda...
+```
+
+The CLI opens the default browser to the generated URL where users must authenticate
+with the configured SAML identity provider. The URL may be manually entered into the
+browser if it cannot be automatically opened.
+
+The CLI login behavior may be customized with the following optional parameters:
+
+- `skip_browser` (default: `false`): If set to `true`, automatic launching of the default
+ browser will be skipped. The SAML identity provider URL must be manually entered in a
+ browser to complete the authentication flow.
+- `abort_on_error` (default: `false`): If set to `true`, the CLI returns an error and
+ exits with a non-zero value if it cannot launch the default browser.
+
+</Tab>
+
+<Tab heading="Vault UI">
+
+1. Select "SAML" from the "Method" select box.
+1. Enter a role name for the "Role" field or leave blank to use
+ the [default role](/vault/api-docs/auth/saml#default_role).
+1. Press **Sign in with SAML Provider** and complete the authentication with the
+ configured SAML identity provider.
+
+</Tab>
+
+</Tabs>
+
+## Configuration
+
+Auth methods must be configured in advance before users or machines can
+authenticate. These steps are usually completed by an operator or configuration
+management tool.
+
+1. Enable the SAML authentication method with the `auth enable` CLI command:
+
+ ```shell-session
+ $ vault auth enable saml
+ ```
+
+1. Use the `/config` endpoint to save the configuration of your SAML identity provider and
+ set the default role. You can configure the trust relationship with the SAML Identity
+ Provider by either providing a URL for its Metadata document:
+
+ ```shell-session
+ $ vault write auth/saml/config \
+ default_role=admin \
+ idp_metadata_url=https://company.okta.com/app/abc123eb9xnIfzlaf697/sso/saml/metadata \
+ entity_id="https://my.vault/v1/auth/saml" \
+ acs_urls="https://my.vault/v1/auth/saml/callback"
+ ```
+
+ or by setting the configuration Metadata manually:
+
+ ```shell-session
+ $ vault write auth/saml/config \
+ default_role=admin \
+ idp_sso_url=https://company.okta.com/app/abc123eb9xnIfzlaf697/sso/saml \
+ idp_entity_id=https://www.okta.com/abc123eb9xnIfzlaf697 \
+ idp_cert=@path/to/cert.pem \
+ entity_id="https://my.vault/v1/auth/saml" \
+ acs_urls="https://my.vault/v1/auth/saml/callback"
+ ```
+
+1. Create a named role:
+
+ ```shell-session
+ $ vault write auth/saml/role/admin \
+ bound_subjects="*@hashicorp.com" \
+ bound_subjects_type="glob" \
+ token_policies="writer" \
+ bound_attributes=group="admin" \
+ ttl=1h
+ ```
+
+ This role authorizes users that have a subject with an `@hashicorp.com` suffix and
+ are in the `admin` group to authenticate. It also gives the resulting Vault token a
+ time-to-live of 1 hour and the `writer` policy.
+
+Refer to the SAML [API documentation](/vault/api-docs/auth/saml) for a
+complete list of configuration options.
+
+### Assertion consumer service URLs
+
+The [`acs_urls`](/vault/api-docs/auth/saml#acs_urls) configuration parameter determines
+where the SAML response will be sent after users authenticate with the configured SAML
+identity provider in their browser.
+
+The values provided to Vault must:
+
+- Match or be a subset of the configured values for the SAML application within the
+ configured identity provider.
+- Be directed to the auth method's [assertion consumer service
+ callback](/vault/api-docs/auth/saml#assertion-consumer-service-callback) API.
+
+<Note>
+ It is highly recommended and enforced by some identity providers to TLS-protect the
+ assertion consumer service URLs. A warning will be returned from Vault if any of the
+ configured assertion consumer service URLs are not protected by TLS.
+</Note>
+
+#### Configuration for replication
+
+To support a single auth method mount being used across Vault [replication](/vault/docs/enterprise/replication)
+clusters, `acs_urls` supports configuration of multiple values. For example, to support
+SAML authentication on a primary and secondary Vault cluster, the following `acs_urls`
+configuration could be given:
+
+ ```shell-session
+ $ vault write auth/saml/config \
+ acs_urls="https://primary.vault/v1/auth/saml/callback,https://secondary.vault/v1/auth/saml/callback"
+ ```
+
+The Vault UI and CLI will automatically request the proper assertion consumer service URL
+for the cluster they're configured to communicate with. This means that the entirety of the
+authentication flow will stay within the targeted cluster.
+
+#### Configuration for namespaces
+
+The SAML auth method can be used within Vault [namespaces](/vault/docs/enterprise/namespaces).
+The assertion consumer service URLs configured in both Vault and the identity provider must
+include the namespace path segment.
+
+The following table provides assertion consumer service URLs given different namespace paths:
+
+| Namespace path | Assertion consumer service URL |
+|-----------------|-------------------------------------------------------|
+| `admin/` | `https://my.vault/v1/admin/auth/saml/callback` |
+| `org/security/` | `https://my.vault/v1/org/security/auth/saml/callback` |
+
+### Bound attributes
+
+Once the user has been authenticated the authorization flow will validate
+that both the [`bound_subjects`](/vault/api-docs/auth/saml#bound_subjects) and
+[`bound_attributes`](/vault/api-docs/auth/saml#bound_attributes) match expected
+values configured for the role. This can be used to restrict access to Vault for
+a subset of users in the SAML identity provider.
+
+For example, a role with `bound_subjects=*@hashicorp.com` and
+`bound_attributes=groups=support,engineering` will only authorize users whose subject has
+an `@hashicorp.com` suffix and that are in either the `support` or `engineering` group.
+
+Whether it should be an exact match or interpret `*` as a wildcard can be
+controlled by the [`bound_subjects_type`](/vault/api-docs/auth/saml#bound_subjects_type) and
+[`bound_attributes_type`](/vault/api-docs/auth/saml#bound_attributes_type) parameters.
+
+## API
+
+The SAML authentication plugin has a full HTTP API. Refer to the
+[SAML API documentation](/vault/api-docs/auth/saml) for more details.
command/agentproxyshared/auth/azure/azure_test.go+96 0
@@ -0,0 +1,96 @@
+// Copyright (c) HashiCorp, Inc.
+// SPDX-License-Identifier: BUSL-1.1
+
+package azure
+
+import (
+ "testing"
+
+ "github.com/hashicorp/go-hclog"
+ "github.com/hashicorp/vault/command/agentproxyshared/auth"
+)
+
+// TestAzureAuthMethod tests that NewAzureAuthMethod succeeds
+// with valid config.
+func TestAzureAuthMethod(t *testing.T) {
+ t.Parallel()
+ config := &auth.AuthConfig{
+ Logger: hclog.NewNullLogger(),
+ MountPath: "auth-test",
+ Config: map[string]interface{}{
+ "resource": "test",
+ "client_id": "test",
+ "role": "test",
+ "scope": "test",
+ "authenticate_from_environment": true,
+ },
+ }
+
+ _, err := NewAzureAuthMethod(config)
+ if err != nil {
+ t.Fatal(err)
+ }
+}
+
+// TestAzureAuthMethod_StringAuthFromEnvironment tests that NewAzureAuthMethod succeeds
+// with valid config, where authenticate_from_environment is a string literal.
+func TestAzureAuthMethod_StringAuthFromEnvironment(t *testing.T) {
+ t.Parallel()
+ config := &auth.AuthConfig{
+ Logger: hclog.NewNullLogger(),
+ MountPath: "auth-test",
+ Config: map[string]interface{}{
+ "resource": "test",
+ "client_id": "test",
+ "role": "test",
+ "scope": "test",
+ "authenticate_from_environment": "true",
+ },
+ }
+
+ _, err := NewAzureAuthMethod(config)
+ if err != nil {
+ t.Fatal(err)
+ }
+}
+
+// TestAzureAuthMethod_BadConfig tests that NewAzureAuthMethod fails with
+// an invalid config.
+func TestAzureAuthMethod_BadConfig(t *testing.T) {
+ t.Parallel()
+ config := &auth.AuthConfig{
+ Logger: hclog.NewNullLogger(),
+ MountPath: "auth-test",
+ Config: map[string]interface{}{
+ "bad_value": "abc",
+ },
+ }
+
+ _, err := NewAzureAuthMethod(config)
+ if err == nil {
+ t.Fatal("Expected error, got none.")
+ }
+}
+
+// TestAzureAuthMethod_BadAuthFromEnvironment tests that NewAzureAuthMethod fails
+// with otherwise valid config, but where authenticate_from_environment is
+// an invalid string literal.
+func TestAzureAuthMethod_BadAuthFromEnvironment(t *testing.T) {
+ t.Parallel()
+ config := &auth.AuthConfig{
+ Logger: hclog.NewNullLogger(),
+ MountPath: "auth-test",
+ Config: map[string]interface{}{
+ "resource": "test",
+ "client_id": "test",
+ "role": "test",
+ "scope": "test",
+ "authenticate_from_environment": "bad_value",
+ },
+ }
+
+ _, err := NewAzureAuthMethod(config)
+ if err == nil {
+ t.Fatal("Expected error, got none.")
+ }
+}
<82990506+hc-github-team-secure-vault-core@users.noreply.github.com>
03732eb158357765906281f4dbc780e242678072 (#23313)
.gitignore | 3 +
.../content/api-docs/system/secrets-sync.mdx | 538 ++++++++++++++++++
website/content/docs/sync/awssm.mdx | 147 +++++
website/content/docs/sync/azurekv.mdx | 132 +++++
website/content/docs/sync/gcpsm.mdx | 249 ++++++++
website/content/docs/sync/github.mdx | 129 +++++
website/content/docs/sync/index.mdx | 57 ++
website/content/docs/sync/vercelproject.mdx | 132 +++++
website/data/api-docs-nav-data.json | 9 +
website/data/docs-nav-data.json | 39 ++
10 files changed, 1435 insertions(+)
create mode 100644 website/content/api-docs/system/secrets-sync.mdx
create mode 100644 website/content/docs/sync/awssm.mdx
create mode 100644 website/content/docs/sync/azurekv.mdx
create mode 100644 website/content/docs/sync/gcpsm.mdx
create mode 100644 website/content/docs/sync/github.mdx
create mode 100644 website/content/docs/sync/index.mdx
create mode 100644 website/content/docs/sync/vercelproject.mdx
website/content/api-docs/system/secrets-sync.mdx+399 0
@@ -0,0 +1,538 @@
+---
+layout: api
+page_title: /sys/sync - HTTP API
+description: The `/sys/sync` endpoints are used to configure destinations and associate secrets to sync with these destinations.
+---
+
+# `/sys/sync`
+
+The `/sys/sync` endpoints are used to configure destinations and associate secrets to sync with these destinations.
+
+Each destination type has its own endpoint for creation & update operations, but share the same endpoints for read &
+delete operations.
+
+## List destinations
+
+This endpoint lists all configured sync destination names regrouped by destination type.
+
+| Method | Path |
+|:-------|:-------------------------|
+| `LIST` | `/sys/sync/destinations` |
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --request LIST
+ http://127.0.0.1:8200/v1/sys/sync/destinations
+```
+
+### Sample response
+
+```json
+{
+ "request_id": "uuid",
+ "lease_id": "",
+ "renewable": false,
+ "lease_duration": 0,
+ "data": {
+ "key_info": {
+ "aws-sm": [
+ "my-dest-1"
+ ],
+ "gh": [
+ "my-dest-1"
+ ]
+ },
+ "keys": [
+ "aws-sm",
+ "gh"
+ ]
+ },
+ "wrap_info": null,
+ "warnings": null,
+ "auth": null
+}
+```
+
+## Read destination
+
+This endpoint retrieves information about the destination of a given type and name. Sensitive information from the
+connection details are obfuscated.
+
+| Method | Path |
+|:-------|:-------------------------------------|
+| `GET` | `/sys/sync/destinations/:type/:name` |
+
+### Parameters
+
+- `type` `(string: <required>)` - Specifies the destination type. This is specified as part of the URL.
+
+- `name` `(string: <required>)` - Specifies the name for this destination. This is specified as part of the URL.
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --reuquest GET
+ http://127.0.0.1:8200/v1/sys/sync/destinations/aws-sm/my-store-1
+```
+
+### Sample response
+
+```json
+{
+ "request_id": "uuid",
+ "lease_id": "",
+ "renewable": false,
+ "lease_duration": 0,
+ "data": {
+ "connection_details": {
+ "access_key_id": "*****",
+ "secret_access_key": "*****",
+ "region": "us-west-1"
+ },
+ "name": "my-store-1",
+ "type": "aws-sm"
+ },
+ "wrap_info": null,
+ "warnings": null,
+ "auth": null
+}
+```
+
+## Delete destination
+
+This endpoint deletes information about the destination of a given type and name if it exists. Destinations still managing
+associations cannot be deleted.
+
+| Method | Path |
+|:---------|:-------------------------------------|
+| `DELETE` | `/sys/sync/destinations/:type/:name` |
+
+### Parameters
+
+- `type` `(string: <required>)` - Specifies the destination type. This is specified as part of the URL.
+
+- `name` `(string: <required>)` - Specifies the name for this destination. This is specified as part of the URL.
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --request DELETE
+ http://127.0.0.1:8200/v1/sys/sync/destinations/aws-sm/my-store-1
+```
+
+## Create|Update AWS Secrets Manager destination
+
+This endpoint creates a destination to synchronize secrets with the AWS Secrets manager.
+
+| Method | Path |
+|:-------|:--------------------------------------|
+| `POST` | `/sys/sync/destinations/aws-sm/:name` |
+
+### Parameters
+
+- `name` `(string: <required>)` - Specifies the name for this destination. This is specified as part of the URL.
+
+- `access_key_id` `(string: "")` - Access key id to authenticate against the AWS secrets manager. If omitted, authentication
+fallbacks on the AWS credentials provider chain and tries to infer authentication from the environment.
+
+- `secret_access_key` `(string: "")` - Secret access key to authenticate against the AWS secrets manager. If omitted,
+authentication fallbacks on the AWS credentials provider chain and tries to infer authentication from the environment.
+
+- `region` `(string: "")` - Region where to manage the secrets manager entries. If omitted, configuration fallbacks on
+the AWS credentials provider chain and tries to infer region from the environment.
+
+### Sample payload
+```json
+{
+ "access_key_id": "AKI***",
+ "secret_access_key": "ktri****",
+ "region": "us-west-1"
+}
+```
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --request POST
+ --data @payload.json
+ http://127.0.0.1:8200/v1/sys/sync/destinations/aws-sm/my-store-1
+```
+
+### Sample response
+
+```json
+{
+ "request_id": "uuid",
+ "lease_id": "",
+ "renewable": false,
+ "lease_duration": 0,
+ "data": {
+ "connection_details": {
+ "access_key_id": "*****",
+ "secret_access_key": "*****",
+ "region": "us-west-1"
+ },
+ "name": "my-store-1",
+ "type": "aws-sm"
+ },
+ "wrap_info": null,
+ "warnings": null,
+ "auth": null
+}
+```
+
+## Create|Update Azure Key Vault destination
+
+This endpoint creates a destination to synchronize secrets with an Azure Key Vault instance.
+
+| Method | Path |
+|:-------|:----------------------------------------|
+| `POST` | `/sys/sync/destinations/azure-kv/:name` |
+
+### Parameters
+
+- `name` `(string: <required>)` - Specifies the name for this destination. This is specified as part of the URL.
+
+- `key_vault_uri` `(string: <required>)` - URI of an existing Azure Key Vault instance.
+
+- `client_id` `(string: <required>)` - Client ID of an Azure app registration.
+
+- `client_secret` `(string: <required>)` - Client secret of an Azure app registration.
+
+- `tenant_id` `(string: <required>)` - ID of the target Azure tenant.
+
+- `cloud` `(string: "cloud")` - Specifies a cloud for the client. The default is Azure Public Cloud.
+
+
+### Sample payload
+```json
+{
+ "key_vault_uri": "https://keyvault-1234abcd.vault.azure.net",
+ "subscription_id": "uuid",
+ "tenant_id": "uuid",
+ "client_id": "uuid",
+ "client_secret": "90y8Q***"
+}
+```
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --request POST
+ --data @payload.json
+ http://127.0.0.1:8200/v1/sys/sync/destinations/aws-sm/my-store-1
+```
+
+## Create|Update GCP Secret Manager destination
+
+This endpoint creates a destination to synchronize secrets with the GCP Secret Manager.
+
+| Method | Path |
+|:-------|:--------------------------------------|
+| `POST` | `/sys/sync/destinations/gcp-sm/:name` |
+
+### Parameters
+
+- `name` `(string: <required>)` - Specifies the name for this destination. This is specified as part of the URL.
+
+- `credentials` `(string: <required>)` - JSON credentials (either file contents or '@path/to/file')
+See docs for [alternative ways](/vault/docs/secrets/gcp#authentication) to pass in to this parameter
+
+### Sample payload
+```json
+{
+ "credentials": "<JSON string>"
+}
+```
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --request POST
+ --data @payload.json
+ http://127.0.0.1:8200/v1/sys/sync/destinations/gcp-sm/my-store-1
+```
+
+## Create|Update GitHub Repository Action destination
+
+This endpoint creates a destination to synchronize action secrets with a GitHub repository.
+
+| Method | Path |
+|:-------|:----------------------------------|
+| `POST` | `/sys/sync/destinations/gh/:name` |
+
+### Parameters
+
+- `name` `(string: <required>)` - Specifies the name for this destination. This is specified as part of the URL.
+
+- `access_token` `(string: <required>)` - Fine-grained or personal access token.
+
+- `repository_owner` `(string: <required>)` - GitHub organization or username that owns the repository. For example, if a repository is located at https://github.com/hashicorp/vault.git the owner is hashicorp.
+
+- `repisitory_name` `(string: <required>)` - Name of the repository. For example, if a repository is located at https://github.com/hashicorp/vault.git the name is vault.
+
+### Sample payload
+```json
+{
+ "access_token": "github_pat_12345",
+ "repository_owner": "my-organization-or-username",
+ "repository_name": "my-repository"
+}
+```
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --request POST
+ --data @payload.json
+ http://127.0.0.1:8200/v1/sys/sync/destinations/gh/my-store-1
+```
+
+## Create|Update Vercel Project destination
+
+This endpoint creates a destination to synchronize secrets with the GCP Secret Manager.
+
+| Method | Path |
+|:-------|:----------------------------------------------|
+| `POST` | `/sys/sync/destinations/vercel-project/:name` |
+
+### Parameters
+
+- `name` `(string: <required>)` - Specifies the name for this destination. This is specified as part of the URL.
+
+- `access_token` `(string: <required>)` - Vercel API access token with the permissions to manage environment variables.
+
+- `project_id` `(string: <required>)` - Project ID where to manage environment variables.
+
+- `team_id` `(string: "")` - Team ID the project belongs to. Optional.
+
+- `deployment_environments` `(string: <required>)` - Deployment environments where the environment variables are available. Accepts 'development', 'preview' & 'production'.
+
+### Sample payload
+```json
+{
+ "access_token": "<token>>",
+ "project_id": "prj_12345",
+ "deployment_environments": ["development", "preview", "production"]
+}
+```
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --request POST
+ --data @payload.json
+ http://127.0.0.1:8200/v1/sys/sync/destinations/vercel-project/my-store-1
+```
+
+
+## Read Associations
+
+This endpoint returns all existing associations for a given destination. An association references the mount via its accessor.
+Associations also contain the latest sync status for the secret they represent.
+
+<Note>
+
+ In the event a synchronisation operation does not succeed, the sync status will indicate the cause
+ of the error and is a useful tool when troubleshooting.
+
+</Note>
+
+| Method | Path |
+|:-------|:--------------------------------------------------|
+| `GET` | `/sys/sync/destinations/:type/:name/associations` |
+
+### Parameters
+
+- `type` `(string: <required>)` - Specifies the destination type. This is specified as part of the URL.
+
+- `name` `(string: <required>)` - Specifies the name for this destination. This is specified as part of the URL.
+
+### Sample request
+
+```shell-session
+$ curl \
+ --header "X-Vault-Token: ..." \
+ --request GET
+ --data @payload.json
+ http://127.0.0.1:8200/v1/sys/sync/destinations/aws-sm/my-store-1/associations
+```
+
+### Sample response
+
+```json
+{
+ "request_id": "uuid",
+ "lease_id": "",
+ "renewable": false,
+ "lease_duration": 0,
+ "data": {
+ "associated_secrets": {
+ "kv_eb4acbae/my-secret-1": {
+ "accessor": "kv_eb4acbae",
+ "secret_name": "my-secret-1",
+ "sync_status": "SYNCED",
+ "updated_at": "2023-09-20T10:51:53.961861096-04:00"
+ }
+ },
+ "store_name": "my-store-1",
+ "store_type": "aws-sm"
+ },
+ "wrap_info": null,
+ "warnings": null,
… diff truncated
enos/enos-scenario-agent.hcl+248 24
@@ -5,8 +5,12 @@ scenario "agent" {
matrix {
arch = ["amd64", "arm64"]
artifact_source = ["local", "crt", "artifactory"]
+ artifact_type = ["bundle", "package"]
+ backend = ["consul", "raft"]
+ consul_version = ["1.12.9", "1.13.9", "1.14.9", "1.15.5", "1.16.1"]
distro = ["ubuntu", "rhel"]
- edition = ["oss", "ent", "ent.fips1402", "ent.hsm", "ent.hsm.fips1402"]
+ edition = ["ce", "ent", "ent.fips1402", "ent.hsm", "ent.hsm.fips1402"]
+ seal = ["awskms", "shamir"]
# Our local builder always creates bundles
exclude {
@@ -30,12 +34,18 @@ scenario "agent" {
]
locals {
- bundle_path = matrix.artifact_source != "artifactory" ? abspath(var.vault_artifact_path) : null
+ artifact_path = matrix.artifact_source != "artifactory" ? abspath(var.vault_artifact_path) : null
enos_provider = {
rhel = provider.enos.rhel
ubuntu = provider.enos.ubuntu
}
- install_artifactory_artifact = local.bundle_path == null
+ manage_service = matrix.artifact_type == "bundle"
+ vault_install_dir = matrix.artifact_type == "bundle" ? var.vault_install_dir : global.vault_install_dir_packages[matrix.distro]
+ }
+
+ step "get_local_metadata" {
+ skip_step = matrix.artifact_source != "local"
+ module = module.get_local_metadata
}
step "build_vault" {
@@ -43,7 +53,7 @@ scenario "agent" {
variables {
build_tags = var.vault_local_build_tags != null ? var.vault_local_build_tags : global.build_tags[matrix.edition]
- bundle_path = local.bundle_path
+ artifact_path = local.artifact_path
goarch = matrix.arch
goos = "linux"
artifactory_host = matrix.artifact_source == "artifactory" ? var.artifactory_host : null
@@ -52,7 +62,7 @@ scenario "agent" {
artifactory_token = matrix.artifact_source == "artifactory" ? var.artifactory_token : null
arch = matrix.artifact_source == "artifactory" ? matrix.arch : null
product_version = var.vault_product_version
- artifact_type = matrix.artifact_source == "artifactory" ? var.vault_artifact_type : null
+ artifact_type = matrix.artifact_type
distro = matrix.artifact_source == "artifactory" ? matrix.distro : null
edition = matrix.artifact_source == "artifactory" ? matrix.edition : null
revision = var.vault_revision
@@ -71,8 +81,19 @@ scenario "agent" {
}
}
- step "read_license" {
- skip_step = matrix.edition == "oss"
+ // This step reads the contents of the backend license if we're using a Consul backend and
+ // the edition is "ent".
+ step "read_backend_license" {
+ skip_step = matrix.backend == "raft" || var.backend_edition == "ce"
+ module = module.read_license
+
+ variables {
+ file_name = global.backend_license_path
+ }
+ }
+
+ step "read_vault_license" {
+ skip_step = matrix.edition == "ce"
module = module.read_license
variables {
@@ -97,9 +118,49 @@ scenario "agent" {
}
}
+ step "create_vault_cluster_backend_targets" {
+ module = matrix.backend == "consul" ? module.target_ec2_instances : module.target_ec2_shim
+ depends_on = [step.create_vpc]
+
+ providers = {
+ enos = provider.enos.ubuntu
+ }
+
+ variables {
+ ami_id = step.ec2_info.ami_ids["arm64"]["ubuntu"]["22.04"]
+ awskms_unseal_key_arn = step.create_vpc.kms_key_arn
+ cluster_tag_key = global.backend_tag_key
+ common_tags = global.tags
+ vpc_id = step.create_vpc.vpc_id
+ }
+ }
+
+ step "create_backend_cluster" {
+ module = "backend_${matrix.backend}"
+ depends_on = [
+ step.create_vault_cluster_backend_targets
+ ]
+
+ providers = {
+ enos = provider.enos.ubuntu
+ }
+
+ variables {
+ cluster_name = step.create_vault_cluster_backend_targets.cluster_name
+ cluster_tag_key = global.backend_tag_key
+ license = (matrix.backend == "consul" && var.backend_edition == "ent") ? step.read_backend_license.license : null
+ release = {
+ edition = var.backend_edition
+ version = matrix.consul_version
+ }
+ target_hosts = step.create_vault_cluster_backend_targets.hosts
+ }
+ }
+
step "create_vault_cluster" {
module = module.vault_cluster
depends_on = [
+ step.create_backend_cluster,
step.build_vault,
step.create_vault_cluster_targets
]
@@ -109,17 +170,42 @@ scenario "agent" {
}
variables {
- artifactory_release = matrix.artifact_source == "artifactory" ? step.build_vault.vault_artifactory_release : null
- awskms_unseal_key_arn = step.create_vpc.kms_key_arn
- cluster_name = step.create_vault_cluster_targets.cluster_name
- enable_audit_devices = var.vault_enable_audit_devices
- install_dir = var.vault_install_dir
- license = matrix.edition != "oss" ? step.read_license.license : null
- local_artifact_path = local.bundle_path
- packages = concat(global.packages, global.distro_packages[matrix.distro])
- storage_backend = "raft"
- target_hosts = step.create_vault_cluster_targets.hosts
- unseal_method = "shamir"
+ artifactory_release = matrix.artifact_source == "artifactory" ? step.build_vault.vault_artifactory_release : null
+ awskms_unseal_key_arn = step.create_vpc.kms_key_arn
+ backend_cluster_name = step.create_vault_cluster_backend_targets.cluster_name
+ backend_cluster_tag_key = global.backend_tag_key
+ cluster_name = step.create_vault_cluster_targets.cluster_name
+ consul_license = (matrix.backend == "consul" && var.backend_edition == "ent") ? step.read_backend_license.license : null
+ consul_release = matrix.backend == "consul" ? {
+ edition = var.backend_edition
+ version = matrix.consul_version
+ } : null
+ enable_audit_devices = var.vault_enable_audit_devices
+ install_dir = local.vault_install_dir
+ license = matrix.edition != "ce" ? step.read_vault_license.license : null
+ local_artifact_path = local.artifact_path
+ manage_service = local.manage_service
+ packages = concat(global.packages, global.distro_packages[matrix.distro])
+ storage_backend = matrix.backend
+ target_hosts = step.create_vault_cluster_targets.hosts
+ unseal_method = matrix.seal
+ }
+ }
+
+ // Wait for our cluster to elect a leader
+ step "wait_for_leader" {
+ module = module.vault_wait_for_leader
+ depends_on = [step.create_vault_cluster]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ timeout = 120 # seconds
+ vault_hosts = step.create_vault_cluster_targets.hosts
+ vault_install_dir = local.vault_install_dir
+ vault_root_token = step.create_vault_cluster.root_token
}
}
@@ -128,6 +214,7 @@ scenario "agent" {
depends_on = [
step.build_vault,
step.create_vault_cluster,
+ step.wait_for_leader,
]
providers = {
@@ -135,6 +222,7 @@ scenario "agent" {
}
variables {
+ vault_install_dir = local.vault_install_dir
vault_instances = step.create_vault_cluster_targets.hosts
vault_root_token = step.create_vault_cluster.root_token
vault_agent_template_destination = "/tmp/agent_output.txt"
@@ -147,6 +235,7 @@ scenario "agent" {
depends_on = [
step.create_vault_cluster,
step.start_vault_agent,
+ step.wait_for_leader,
]
providers = {
@@ -160,7 +249,147 @@ scenario "agent" {
}
}
- output "awkms_unseal_key_arn" {
+ step "get_vault_cluster_ips" {
+ module = module.vault_get_cluster_ips
+ depends_on = [step.wait_for_leader]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ vault_hosts = step.create_vault_cluster_targets.hosts
+ vault_install_dir = local.vault_install_dir
+ vault_root_token = step.create_vault_cluster.root_token
+ }
+ }
+
+ step "verify_vault_version" {
+ module = module.vault_verify_version
+ depends_on = [step.wait_for_leader]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ vault_instances = step.create_vault_cluster_targets.hosts
+ vault_edition = matrix.edition
+ vault_install_dir = local.vault_install_dir
+ vault_product_version = matrix.artifact_source == "local" ? step.get_local_metadata.version : var.vault_product_version
+ vault_revision = matrix.artifact_source == "local" ? step.get_local_metadata.revision : var.vault_revision
+ vault_build_date = matrix.artifact_source == "local" ? step.get_local_metadata.build_date : var.vault_build_date
+ vault_root_token = step.create_vault_cluster.root_token
+ }
+ }
+
+ step "verify_vault_unsealed" {
+ module = module.vault_verify_unsealed
+ depends_on = [step.wait_for_leader]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ vault_install_dir = local.vault_install_dir
+ vault_instances = step.create_vault_cluster_targets.hosts
+ }
+ }
+
+ step "verify_write_test_data" {
+ module = module.vault_verify_write_data
+ depends_on = [
+ step.create_vault_cluster,
+ step.get_vault_cluster_ips
+ ]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ leader_public_ip = step.get_vault_cluster_ips.leader_public_ip
+ leader_private_ip = step.get_vault_cluster_ips.leader_private_ip
+ vault_instances = step.create_vault_cluster_targets.hosts
+ vault_install_dir = local.vault_install_dir
+ vault_root_token = step.create_vault_cluster.root_token
+ }
+ }
+
+ step "verify_raft_auto_join_voter" {
+ skip_step = matrix.backend != "raft"
+ module = module.vault_verify_raft_auto_join_voter
+ depends_on = [
+ step.create_vault_cluster,
+ step.get_vault_cluster_ips
+ ]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ vault_install_dir = local.vault_install_dir
+ vault_instances = step.create_vault_cluster_targets.hosts
+ vault_root_token = step.create_vault_cluster.root_token
+ }
+ }
+
+ step "verify_replication" {
+ module = module.vault_verify_replication
+ depends_on = [
+ step.create_vault_cluster,
+ step.get_vault_cluster_ips
+ ]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ vault_edition = matrix.edition
+ vault_install_dir = local.vault_install_dir
+ vault_instances = step.create_vault_cluster_targets.hosts
+ }
+ }
+
+ step "verify_read_test_data" {
+ module = module.vault_verify_read_data
+ depends_on = [
+ step.verify_write_test_data,
+ step.verify_replication
+ ]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ node_public_ips = step.get_vault_cluster_ips.follower_public_ips
+ vault_install_dir = local.vault_install_dir
+ }
+ }
+
+ step "verify_ui" {
+ module = module.vault_verify_ui
+ depends_on = [step.create_vault_cluster]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ vault_instances = step.create_vault_cluster_targets.hosts
+ }
+ }
+
+ output "audit_device_file_path" {
+ description = "The file path for the file audit device, if enabled"
+ value = step.create_vault_cluster.audit_device_file_path
+ }
+
+ output "awskms_unseal_key_arn" {
description = "The Vault cluster KMS key arn"
value = step.create_vpc.kms_key_arn
}
@@ -214,9 +443,4 @@ scenario "agent" {
description = "The Vault cluster unseal keys hex"
value = step.create_vault_cluster.unseal_keys_hex
}
-
- output "vault_audit_device_file_path" {
- description = "The file path for the file audit device, if enabled"
- value = step.create_vault_cluster.audit_device_file_path
- }
}
enos/enos-scenario-proxy.hcl+249 20
@@ -5,8 +5,24 @@ scenario "proxy" {
matrix {
arch = ["amd64", "arm64"]
artifact_source = ["local", "crt", "artifactory"]
+ artifact_type = ["bundle", "package"]
+ backend = ["consul", "raft"]
+ consul_version = ["1.12.9", "1.13.9", "1.14.9", "1.15.5", "1.16.1"]
distro = ["ubuntu", "rhel"]
- edition = ["oss", "ent", "ent.fips1402", "ent.hsm", "ent.hsm.fips1402"]
+ edition = ["ce", "ent", "ent.fips1402", "ent.hsm", "ent.hsm.fips1402"]
+ seal = ["awskms", "shamir"]
+
+ # Our local builder always creates bundles
+ exclude {
+ artifact_source = ["local"]
+ artifact_type = ["package"]
+ }
+
+ # HSM and FIPS 140-2 are only supported on amd64
+ exclude {
+ arch = ["arm64"]
+ edition = ["ent.fips1402", "ent.hsm", "ent.hsm.fips1402"]
+ }
}
terraform_cli = terraform_cli.default
@@ -18,11 +34,13 @@ scenario "proxy" {
]
locals {
- bundle_path = matrix.artifact_source != "artifactory" ? abspath(var.vault_artifact_path) : null
+ artifact_path = matrix.artifact_source != "artifactory" ? abspath(var.vault_artifact_path) : null
enos_provider = {
rhel = provider.enos.rhel
ubuntu = provider.enos.ubuntu
}
+ manage_service = matrix.artifact_type == "bundle"
+ vault_install_dir = matrix.artifact_type == "bundle" ? var.vault_install_dir : global.vault_install_dir_packages[matrix.distro]
}
step "get_local_metadata" {
@@ -35,7 +53,7 @@ scenario "proxy" {
variables {
build_tags = var.vault_local_build_tags != null ? var.vault_local_build_tags : global.build_tags[matrix.edition]
- bundle_path = local.bundle_path
+ artifact_path = local.artifact_path
goarch = matrix.arch
goos = "linux"
artifactory_host = matrix.artifact_source == "artifactory" ? var.artifactory_host : null
@@ -44,7 +62,7 @@ scenario "proxy" {
artifactory_token = matrix.artifact_source == "artifactory" ? var.artifactory_token : null
arch = matrix.artifact_source == "artifactory" ? matrix.arch : null
product_version = var.vault_product_version
- artifact_type = matrix.artifact_source == "artifactory" ? var.vault_artifact_type : null
+ artifact_type = matrix.artifact_type
distro = matrix.artifact_source == "artifactory" ? matrix.distro : null
edition = matrix.artifact_source == "artifactory" ? matrix.edition : null
revision = var.vault_revision
@@ -63,8 +81,19 @@ scenario "proxy" {
}
}
- step "read_license" {
- skip_step = matrix.edition == "oss"
+ // This step reads the contents of the backend license if we're using a Consul backend and
+ // the edition is "ent".
+ step "read_backend_license" {
+ skip_step = matrix.backend == "raft" || var.backend_edition == "ce"
+ module = module.read_license
+
+ variables {
+ file_name = global.backend_license_path
+ }
+ }
+
+ step "read_vault_license" {
+ skip_step = matrix.edition == "ce"
module = module.read_license
variables {
@@ -89,9 +118,49 @@ scenario "proxy" {
}
}
+ step "create_vault_cluster_backend_targets" {
+ module = matrix.backend == "consul" ? module.target_ec2_instances : module.target_ec2_shim
+ depends_on = [step.create_vpc]
+
+ providers = {
+ enos = provider.enos.ubuntu
+ }
+
+ variables {
+ ami_id = step.ec2_info.ami_ids["arm64"]["ubuntu"]["22.04"]
+ awskms_unseal_key_arn = step.create_vpc.kms_key_arn
+ cluster_tag_key = global.backend_tag_key
+ common_tags = global.tags
+ vpc_id = step.create_vpc.vpc_id
+ }
+ }
+
+ step "create_backend_cluster" {
+ module = "backend_${matrix.backend}"
+ depends_on = [
+ step.create_vault_cluster_backend_targets
+ ]
+
+ providers = {
+ enos = provider.enos.ubuntu
+ }
+
+ variables {
+ cluster_name = step.create_vault_cluster_backend_targets.cluster_name
+ cluster_tag_key = global.backend_tag_key
+ license = (matrix.backend == "consul" && var.backend_edition == "ent") ? step.read_backend_license.license : null
+ release = {
+ edition = var.backend_edition
+ version = matrix.consul_version
+ }
+ target_hosts = step.create_vault_cluster_backend_targets.hosts
+ }
+ }
+
step "create_vault_cluster" {
module = module.vault_cluster
depends_on = [
+ step.create_backend_cluster,
step.build_vault,
step.create_vault_cluster_targets
]
@@ -101,17 +170,42 @@ scenario "proxy" {
}
variables {
- artifactory_release = matrix.artifact_source == "artifactory" ? step.build_vault.vault_artifactory_release : null
- awskms_unseal_key_arn = step.create_vpc.kms_key_arn
- cluster_name = step.create_vault_cluster_targets.cluster_name
- enable_audit_devices = var.vault_enable_audit_devices
- install_dir = var.vault_install_dir
- license = matrix.edition != "oss" ? step.read_license.license : null
- local_artifact_path = local.bundle_path
- packages = concat(global.packages, global.distro_packages[matrix.distro])
- storage_backend = "raft"
- target_hosts = step.create_vault_cluster_targets.hosts
- unseal_method = "shamir"
+ artifactory_release = matrix.artifact_source == "artifactory" ? step.build_vault.vault_artifactory_release : null
+ awskms_unseal_key_arn = step.create_vpc.kms_key_arn
+ backend_cluster_name = step.create_vault_cluster_backend_targets.cluster_name
+ backend_cluster_tag_key = global.backend_tag_key
+ cluster_name = step.create_vault_cluster_targets.cluster_name
+ consul_license = (matrix.backend == "consul" && var.backend_edition == "ent") ? step.read_backend_license.license : null
+ consul_release = matrix.backend == "consul" ? {
+ edition = var.backend_edition
+ version = matrix.consul_version
+ } : null
+ enable_audit_devices = var.vault_enable_audit_devices
+ install_dir = local.vault_install_dir
+ license = matrix.edition != "ce" ? step.read_vault_license.license : null
+ local_artifact_path = local.artifact_path
+ manage_service = local.manage_service
+ packages = concat(global.packages, global.distro_packages[matrix.distro])
+ storage_backend = matrix.backend
+ target_hosts = step.create_vault_cluster_targets.hosts
+ unseal_method = matrix.seal
+ }
+ }
+
+ // Wait for our cluster to elect a leader
+ step "wait_for_leader" {
+ module = module.vault_wait_for_leader
+ depends_on = [step.create_vault_cluster]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ timeout = 120 # seconds
+ vault_hosts = step.create_vault_cluster_targets.hosts
+ vault_install_dir = local.vault_install_dir
+ vault_root_token = step.create_vault_cluster.root_token
}
}
@@ -127,12 +221,147 @@ scenario "proxy" {
}
variables {
- vault_instances = step.create_vault_cluster_targets.hosts
- vault_root_token = step.create_vault_cluster.root_token
+ vault_install_dir = local.vault_install_dir
+ vault_instances = step.create_vault_cluster_targets.hosts
+ vault_root_token = step.create_vault_cluster.root_token
+ }
+ }
+
+ step "get_vault_cluster_ips" {
+ module = module.vault_get_cluster_ips
+ depends_on = [step.wait_for_leader]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ vault_hosts = step.create_vault_cluster_targets.hosts
+ vault_install_dir = local.vault_install_dir
+ vault_root_token = step.create_vault_cluster.root_token
}
}
- output "awkms_unseal_key_arn" {
+ step "verify_vault_version" {
+ module = module.vault_verify_version
+ depends_on = [step.create_vault_cluster]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ vault_instances = step.create_vault_cluster_targets.hosts
+ vault_edition = matrix.edition
+ vault_install_dir = local.vault_install_dir
+ vault_product_version = matrix.artifact_source == "local" ? step.get_local_metadata.version : var.vault_product_version
+ vault_revision = matrix.artifact_source == "local" ? step.get_local_metadata.revision : var.vault_revision
+ vault_build_date = matrix.artifact_source == "local" ? step.get_local_metadata.build_date : var.vault_build_date
+ vault_root_token = step.create_vault_cluster.root_token
+ }
+ }
+
+ step "verify_vault_unsealed" {
+ module = module.vault_verify_unsealed
+ depends_on = [step.create_vault_cluster]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ vault_install_dir = local.vault_install_dir
+ vault_instances = step.create_vault_cluster_targets.hosts
+ }
+ }
+
+ step "verify_write_test_data" {
+ module = module.vault_verify_write_data
+ depends_on = [
+ step.create_vault_cluster,
+ step.get_vault_cluster_ips
+ ]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ leader_public_ip = step.get_vault_cluster_ips.leader_public_ip
+ leader_private_ip = step.get_vault_cluster_ips.leader_private_ip
+ vault_instances = step.create_vault_cluster_targets.hosts
+ vault_install_dir = local.vault_install_dir
+ vault_root_token = step.create_vault_cluster.root_token
+ }
+ }
+
+ step "verify_raft_auto_join_voter" {
+ skip_step = matrix.backend != "raft"
+ module = module.vault_verify_raft_auto_join_voter
+ depends_on = [step.create_vault_cluster]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ vault_install_dir = local.vault_install_dir
+ vault_instances = step.create_vault_cluster_targets.hosts
+ vault_root_token = step.create_vault_cluster.root_token
+ }
+ }
+
+ step "verify_replication" {
+ module = module.vault_verify_replication
+ depends_on = [step.create_vault_cluster]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ vault_edition = matrix.edition
+ vault_install_dir = local.vault_install_dir
+ vault_instances = step.create_vault_cluster_targets.hosts
+ }
+ }
+
+ step "verify_read_test_data" {
+ module = module.vault_verify_read_data
+ depends_on = [
+ step.verify_write_test_data,
+ step.verify_replication
+ ]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ node_public_ips = step.get_vault_cluster_ips.follower_public_ips
+ vault_install_dir = local.vault_install_dir
+ }
+ }
+
+ step "verify_ui" {
+ module = module.vault_verify_ui
+ depends_on = [step.create_vault_cluster]
+
+ providers = {
+ enos = local.enos_provider[matrix.distro]
+ }
+
+ variables {
+ vault_instances = step.create_vault_cluster_targets.hosts
+ }
+ }
+
+ output "audit_device_file_path" {
+ description = "The file path for the file audit device, if enabled"
+ value = step.create_vault_cluster.audit_device_file_path
+ }
+
+ output "awskms_unseal_key_arn" {
description = "The Vault cluster KMS key arn"
value = step.create_vpc.kms_key_arn
}
website/content/docs/sync/gcpsm.mdx+249 0
@@ -0,0 +1,249 @@
+---
+layout: docs
+page_title: Google Cloud Platform Secret Manager - Secrets Sync Destination
+description: The Google Cloud Platform Secret Manager destination syncs secrets from Vault to GCP.
+---
+
+# Google Cloud Platform Secret Manager
+
+The Google Cloud Platform (GCP) Secret Manager sync destination allows Vault to safely synchronize secrets to your GCP projects.
+This is a low footprint option that enables your applications to benefit from Vault-managed secrets without requiring them
+to connect directly with Vault. This guide walks you through the configuration process.
+
+Prerequisites:
+* Ability to read or create KVv2 secrets
+* Ability to create GCP Service Account credentials with access to the Secret Manager
+* Ability to create sync destinations and associations on your Vault server
+
+## Setup
+
+1. If you do not already have a Service Account, navigate to the IAM & Admin page in the Google Cloud console to
+ [create a new Service Account](https://cloud.google.com/iam/docs/service-accounts-create) with the
+ [necessary permissions](/vault/docs/sync/gcpsm#permissions). [Instructions](/vault/docs/sync/gcpsm#provision-service-account)
+ to provision this Service Account via Terraform can be found below.
+
+1. Configure a sync destination with the Service Account JSON credentials created in the previous step. See docs for
+ [alternative ways](/vault/docs/secrets/gcp#authentication) to pass in the `credentials` parameter.
+
+ ```shell-session
+ $ vault write sys/sync/destinations/gcp-sm/my-dest \
+ credentials='@path/to/credentials.json'
+ ```
+
+ **Output:**
+
+ <CodeBlockConfig hideClipboard>
+
+ ```plaintext
+ Key Value
+ --- -----
+ connection_details map[credentials:*****]
+ name my-dest
+ type gcp-sm
+ ```
+
+ </CodeBlockConfig>
+
+## Usage
+
+1. If you do not already have a KVv2 secret to sync, mount a new KVv2 secrets engine.
+
+ ```shell-session
+ $ vault secrets enable -path=my-kv kv-v2
+ ```
+
+ **Output**:
+
+ <CodeBlockConfig hideClipboard>
+
+ ```plaintext
+ Success! Enabled the kv-v2 secrets engine at: my-kv/
+ ```
+
+ </CodeBlockConfig>
+
+1. Create secrets you wish to sync with a target GCP Secret Manager.
+
+ ```shell-session
+ $ vault kv put -mount=my-kv my-secret foo='bar'
+ ```
+
+ **Output**:
+
+ <CodeBlockConfig hideClipboard>
+
+ ```plaintext
+ ==== Secret Path ====
+ my-kv/data/my-secret
+
+ ======= Metadata =======
+ Key Value
+ --- -----
+ created_time <timestamp>
+ custom_metadata <nil>
+ deletion_time n/a
+ destroyed false
+ version 1
+ ```
+
+ </CodeBlockConfig>
+
+1. Create an association between the destination and a secret to synchronize.
+
+ ```shell-session
+ $ vault write sys/sync/destinations/gcp-sm/my-dest/associations/set \
+ mount='my-kv' \
+ secret_name='my-secret'
+ ```
+
+ **Output:**
+
+ <CodeBlockConfig hideClipboard>
+
+ ```plaintext
+ Key Value
+ --- -----
+ associated_secrets map[kv_1234/my-secret:map[accessor:kv_1234 secret_name:my-secret sync_status:SYNCED updated_at:<timestamp>]]
+ store_name my-dest
+ store_type gcp-sm
+ ```
+
+ </CodeBlockConfig>
+
+1. Navigate to the [Secret Manager](https://console.cloud.google.com/security/secret-manager) in the Google Cloud console
+ to confirm your secret was successfully created in your GCP project.
+
+Moving forward, any modification on the Vault secret will be propagated in near real time to its GCP Secret Manager
+counterpart. Creating a new secret version in Vault will create a new version in GCP Secret Manager. Deleting the secret
+or the association in Vault will delete the secret in your GCP project as well.
+
+## Permissions
+
+The credentials given to Vault must have the following permissions to synchronize secrets:
+
+```shell-session
+secretmanager.secrets.create
+secretmanager.secrets.delete
+secretmanager.secrets.get
+secretmanager.secrets.list
+secretmanager.secrets.update
+secretmanager.versions.access
+secretmanager.versions.add
+secretmanager.versions.destroy
+secretmanager.versions.get
+secretmanager.versions.list
+```
+
+## Provision service account
+
+Vault needs to be configured with credentials to establish a trust relationship with your GCP project so it can manage
+Secret Manager secrets on your behalf. The IAM & Admin page in the Google Cloud console can be used to
+[create a new Service Account](https://cloud.google.com/iam/docs/service-accounts-create) with access to the Secret Manager.
+
+You can equally use the [Terraform Google provider](https://registry.terraform.io/providers/hashicorp/google/latest/docs#authentication-and-configuration)
+to provision a GCP Service Account with the appropriate policies.
+
+1. Copy-paste this HCL snippet into a `secrets-sync-setup.tf` file.
+
+ ```hcl
+ provider "google" {
+ // See https://registry.terraform.io/providers/hashicorp/google/latest/docs#authentication-and-configuration to setup the Google Provider
+ // for options on how to configure this provider. The following parameters or environment
+ // variables are typically used.
+
+ // Parameters
+ // region = "" (Optional)
+ // project = ""
+ // credentials = ""
+
+ // Environment Variables
+ // GOOGLE_REGION (optional)
+ // GOOGLE_PROJECT
+ // GOOGLE_CREDENTIALS (The path to a service account key file with the
+ // "Service Account Admin", "Service Account Key
+ // Admin", and "Security Admin" roles attached)
+ }
+
+ data "google_client_config" "config" {}
+
+ resource "google_service_account" "vault_secrets_sync_account" {
+ account_id = "gcp-sm-vault-secrets-sync"
+ description = "service account for Vault Secrets Sync feature"
+ }
+
+ // Production environments should use a more restricted role.
+ // The built-in secret manager admin role is used as an example for simplicity.
+ data "google_iam_policy" "vault_secrets_sync_iam_policy" {
+ binding {
+ role = "roles/secretmanager.admin"
+ members = [
+ google_service_account.vault_secrets_sync_account.email,
+ ]
+ }
+ }
+
+ resource "google_project_iam_member" "vault_secrets_sync_iam_member" {
+ project = data.google_client_config.config.project
+ role = "roles/secretmanager.admin"
+ member = google_service_account.vault_secrets_sync_account.member
+ }
+
+ resource "google_service_account_key" "vault_secrets_sync_account_key" {
+ service_account_id = google_service_account.vault_secrets_sync_account.name
+ public_key_type = "TYPE_X509_PEM_FILE"
+ }
+
+ resource "local_file" "vault_secrets_sync_credentials_file" {
+ content = base64decode(google_service_account_key.vault_secrets_sync_account_key.private_key)
+ filename = "gcp-sm-sync-service-account-credentials.json"
+ }
+
+ output "vault_secrets_sync_credentials_file_path" {
+ value = abspath("${path.module}/${local_file.sync_service_account_credentials_file.filename}")
+ }
+ ```
+
+1. Execute a plan to validate the Terraform Google provider is properly configured.
+
+ ```shell-session
+ $ terraform init && terraform plan
+ ```
+
+ **Output:**
+
+ <CodeBlockConfig hideClipboard>
+
+ ```plaintext
+ (...)
+ Plan: 4 to add, 0 to change, 0 to destroy.
+ ```
+
+ </CodeBlockConfig>
+
+1. Execute an apply to provision the Service Account.
+
+ ```shell-session
+ $ terraform apply
+ ```
+
+ **Output:**
+
+ <CodeBlockConfig hideClipboard>
+
+ ```plaintext
+ (...)
+ Apply complete! Resources: 4 added, 0 changed, 0 destroyed.
+
+ Outputs:
+
+ sync_service_account_credentials_file = "/path/to/credentials/file/gcp-sm-sync-service-account-credentials.json"
+ ```
+
+ </CodeBlockConfig>
+
+The generated Service Account credentials file can then be used to configure the Vault GCP Secret Manager destination
+following the [setup](/vault/docs/sync/gcpsm#setup) steps.
+
+## API
+
+Please see the [secrets sync API](/vault/api-docs/system/secrets-sync) for more details.
website/content/docs/plugins/containerized-plugins.mdx+105 0
@@ -0,0 +1,105 @@
+---
+layout: docs
+page_title: Containerized plugins
+description: External Vault plugins can be run in containers.
+---
+
+# Containerized plugins
+
+<Note title="Beta feature">
+ Beta functionality is stable but possibly incomplete and subject to change.
+</Note>
+
+<Note title="Limited OS support">
+ Support for the 'container` runtime is currently limited to Linux.
+</Note>
+
+By default, external plugins run as subprocesses that share Vault's user and
+environment variables. Administrators managing Vault instances on Linux can
+choose to run external plugins in containers. Running plugins in containers
+increases the isolation between plugins, and between plugins and Vault.
+
+## System requirements
+
+- **Your Vault instance must be running on Linux**.
+
+- **Your environment must provide Vault local access to the Docker Engine API**.
+ Vault uses the [Docker SDK](https://pkg.go.dev/github.com/docker/docker) to
+ manage containerized plugins.
+
+- **You must have a valid container runtime installed**. We recommend
+ [installing gVisor](https://gvisor.dev/docs/user_guide/install/) for your
+ container runtime as Vault specifies the `runsc` runtime by default.
+
+- **You must have all your plugin container images pulled and available locally**.
+ Vault does not currently support pulling images as part of the plugin
+ registration process.
+
+## Plugin requirements
+
+All plugins have the following basic requirements to be containerized:
+
+- **Your plugin must be built with at least v1.5.0 of the HashiCorp
+ [`go-plugin`](https://github.com/hashicorp/go-plugin) library**.
+
+- **The image entrypoint should run the plugin binary**.
+
+Some configurations have additional requirements for the container image, listed
+in [supported configurations](#supported-configurations).
+
+## Supported configurations
+
+Vault's containerized plugins are compatible with a variety of configurations.
+In particular, it has been tested with the following:
+
+- Docker and Podman.
+- Default and rootless container engine.
+- OCI runtimes runsc and runc.
+- Plugin container images with root and non-root users.
+- [Mlock](/vault/docs/configuration#disable_mlock) disabled or enabled.
+
+Not all combinations work and some have additional requirements, listed below.
+If you use a configuration that matches multiple headings, you should combine
+the requirements from each matching heading.
+
+### Rootless installation with non-root container user
+
+Not currently supported. We are hoping to provide support in future.
+
+### runsc runtime
+
+- You must pass an additional `--host-uds=all` flag to the `runsc` runtime.
+
+### Rootless installation with `runsc`
+
+- Does not currently support cgroup limits.
+- You must pass an additional `--ignore-cgroups` flag to the `runsc` runtime.
+
+### Non-root container user with mlock enabled
+
+- You must set the IPC_LOCK capability on the plugin binary.
+
+### Rootless container engine with mlock enabled
+
+- You must set the IPC_LOCK capability on the container engine's binary.
+- You do not need to set the IPC_LOCK capability if running with Docker and runsc.
+ The `runsc` runtime supports mlock syscalls in rootless Docker without needing
+ IPC_LOCK itself.
+
+## Container lifecycle and metadata
+
+Like any other external plugin, Vault will automatically manage the lifecycle
+of plugin containers. If they are killed out of band, Vault will restart them
+before servicing any requests that need to be handled by them. Vault will also
+[multiplex](/vault/docs/plugins/plugin-architecture#plugin-multiplexing) multiple
+mounts to be serviced by the same container if the plugin supports multiplexing.
+
+Vault labels each plugin container with a standard set of metadata to help
+identify the owner of the container, including the cluster ID, Vault's own
+process ID, and the plugin's name, type, and version.
+
+## Plugin runtimes
+
+Users who require more control over plugin containers can use the "plugin
+runtime" APIs for finer grained settings. See the CLI documentation for
+[`vault plugin runtime`](/vault/docs/commands/plugin/runtime) for more details.
website/content/docs/sync/azurekv.mdx+132 0
@@ -0,0 +1,132 @@
+---
+layout: docs
+page_title: Azure Key Vault - Secrets Sync Destination
+description: The Azure Key Vault destination syncs secrets from Vault to Azure.
+---
+
+# Azure Key Vault
+
+The Azure Key Vault destination enables Vault to sync and unsync secrets of your choosing into
+an external Azure account. When configured, Vault will actively maintain the state of each externally-synced
+secret in realtime. This includes sending new secrets, updating existing secret values, and removing
+secrets when they either get dissociated from the destination or deleted from Vault.
+
+Prerequisites:
+* Ability to read or create KVv2 secrets
+* Ability to create Azure AD user credentials with access to an Azure Key Vault
+* Ability to create sync destinations and associations on your Vault server
+
+## Setup
+
+1. If you do not already have an Azure Key Vault instance, navigate to the Azure Portal to create a new
+ [Key Vault](https://learn.microsoft.com/en-us/azure/key-vault/general/quick-create-portal).
+
+1. A service principal with a client id and client secret will be needed to configure Azure Key Vault as a
+ sync destination. This [guide](https://learn.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal)
+ will walk you through creating the service principal.
+
+1. Once the service principal is created, the next step is to
+ [grant the service principal](https://learn.microsoft.com/en-us/azure/key-vault/general/rbac-guide?tabs=azure-cli)
+ access to Azure Key Vault. We recommend using the "Key Vault Secrets Officer" built-in role,
+ which gives sufficient access to manage secrets.
+
+1. Configure a sync destination with the service principal credentials and Key Vault URI created in the previous steps.
+
+ ```shell-session
+ $ vault write sys/sync/stores/azure-kv/my-azure-1 \
+ key_vault_uri="$KEY_VAULT_URI" \
+ client_id="$CLIENT_ID" \
+ client_secret="$CLIENT_SECRET" \
+ tenant_id="$TENANT_ID"
+ ```
+
+ **Output:**
+
+ <CodeBlockConfig hideClipboard>
+
+ ```plaintext
+ Key Value
+ --- -----
+ connection_details map[client_id:123 client_secret:***** key_vault_uri:***** tenant_id:123]
+ name my-azure-1
+ type azure-kv
+ ```
+
+ </CodeBlockConfig>
+
+## Usage
+
+1. If you do not already have a KVv2 secret to sync, mount a new KVv2 secrets engine.
+
+ ```shell-session
+ $ vault secrets enable -path='my-kv' kv-v2
+ ```
+
+ **Output:**
+
+ <CodeBlockConfig hideClipboard>
+
+ ```plaintext
+ Success! Enabled the kv-v2 secrets engine at: my-kv/
+ ```
+
+ </CodeBlockConfig>
+
+1. Create secrets you wish to sync with a target Azure Key Vault.
+
+ ```shell-session
+ $ vault kv put -mount='my-kv' my-secret foo='bar'
+ ```
+
+ **Output:**
+
+ <CodeBlockConfig hideClipboard>
+
+ ```plaintext
+ ==== Secret Path ====
+ my-kv/data/my-secret
+
+ ======= Metadata =======
+ Key Value
+ --- -----
+ created_time 2023-09-19T13:17:23.395109Z
+ custom_metadata <nil>
+ deletion_time n/a
+ destroyed false
+ version 1
+ ```
+
+ </CodeBlockConfig>
+
+1. Create an association between the destination and a secret to synchronize.
+
+ ```shell-session
+ $ vault write sys/sync/destinations/azure-kv/my-azure-1/associations/set \
+ mount='my-kv' \
+ secret_name='my-secret'
+ ```
+
+ **Output:**
+
+ <CodeBlockConfig hideClipboard>
+
+ ```plaintext
+ Key Value
+ --- -----
+ associated_secrets map[kv_7532a8b4/my-secret:map[accessor:kv_7532a8b4 secret_name:my-secret sync_status:SYNCED updated_at:2023-09-21T13:53:24.839885-07:00]]
+ store_name my-azure-1
+ store_type azure-kv
+ ```
+
+ </CodeBlockConfig>
+
+1. Navigate to [Azure Key Vault](https://portal.azure.com/#view/HubsExtension/BrowseResource/resourceType/Microsoft.KeyVault%2Fvaults)
+ in the Azure portal to confirm your secret was successfully created.
+
+Moving forward, any modification on the Vault secret will be propagated in near real time to its Azure Key Vault
+counterpart. Creating a new secret version in Vault will create a new version in Azure Key Vault. Deleting the secret
+or the association in Vault will delete the secret in your Azure Key Vault as well.
+
+## API
+
+Please see the [secrets sync API](/vault/api-docs/system/secrets-sync) for more details.
enos/modules/vault_verify_write_data/scripts/smoke-enable-secrets-kv.sh+12 8
@@ -5,7 +5,7 @@
set -e
-function retry {
+retry() {
local retries=$1
shift
local count=0
@@ -24,11 +24,15 @@ function retry {
return 0
}
-function fail {
- echo "$1" 1>&2
- exit 1
+fail() {
+ echo "$1" 1>&2
+ exit 1
}
+[[ -z "$VAULT_ADDR" ]] && fail "VAULT_ADDR env variable has not been set"
+[[ -z "$VAULT_INSTALL_DIR" ]] && fail "VAULT_INSTALL_DIR env variable has not been set"
+[[ -z "$VAULT_TOKEN" ]] && fail "VAULT_TOKEN env variable has not been set"
+
binpath=${VAULT_INSTALL_DIR}/vault
test -x "$binpath" || fail "unable to locate vault binary at $binpath"
@@ -36,16 +40,16 @@ test -x "$binpath" || fail "unable to locate vault binary at $binpath"
retry 5 "$binpath" status > /dev/null 2>&1
# Create user policy
-retry 5 $binpath policy write reguser -<<EOF
+retry 5 "$binpath" policy write reguser -<<EOF
path "*" {
capabilities = ["read", "list"]
}
EOF
# Enable the userpass auth method
-retry 5 $binpath auth enable userpass > /dev/null 2>&1
+retry 5 "$binpath" auth enable userpass > /dev/null 2>&1
# Create new user and attach reguser policy
-retry 5 $binpath write auth/userpass/users/testuser password="passuser1" policies="reguser"
+retry 5 "$binpath" write auth/userpass/users/testuser password="passuser1" policies="reguser"
-retry 5 $binpath secrets enable -path="secret" kv
+retry 5 "$binpath" secrets enable -path="secret" kv
website/content/docs/sync/vercelproject.mdx+132 0
@@ -0,0 +1,132 @@
+---
+layout: docs
+page_title: Vercel Project - Secrets Sync Destination
+description: The Vercel Project destination syncs secrets from Vault to Vercel.
+---
+
+# Vercel Project environment variables
+
+The Vercel Project sync destination allows Vault to safely synchronize secrets as Vercel environment variables.
+This is a low footprint option that enables your applications to benefit from Vault-managed secrets without requiring them
+to connect directly with Vault. This guide walks you through the configuration process.
+
+Prerequisites:
+* Ability to read or create KVv2 secrets
+* Ability to create Vercel tokens with access to modify project environment variables
+* Ability to create sync destinations and associations on your Vault server
+
+## Setup
+
+1. If you do not already have a Vercel token, navigate [your account settings](https://vercel.com/account/tokens) to
+ generate credentials with the necessary permissions to manage your project's environment variables.
+
+1. Next you need to locate your project ID. It can be found under the `Settings` tab in your project's overview page.
+
+1. Configure a sync destination with the access token and project ID obtained in the previous steps.
+
+ ```shell-session
+ $ vault write sys/sync/destinations/vercel-project/my-dest \
+ access_token="$TOKEN" \
+ project_id="$PROJECT_ID" \
+ deployment_environments=development \
+ deployment_environments=preview \
+ deployment_environments=production
+ ```
+
+ **Output:**
+
+ <CodeBlockConfig hideClipboard>
+
+ ```plaintext
+ Key Value
+ --- -----
+ connection_details map[access_token:***** deployment_environments:[development preview production] project_id:<project-id>]
+ name my-dest
+ type vercel-project
+ ```
+
+ </CodeBlockConfig>
+
+## Usage
+
+1. If you do not already have a KVv2 secret to sync, mount a new KVv2 secrets engine.
+
+ ```shell-session
+ $ vault secrets enable -path='my-kv' kv-v2
+ ```
+
+ **Output:**
+
+ <CodeBlockConfig hideClipboard>
+
+ ```plaintext
+ Success! Enabled the kv-v2 secrets engine at: my-kv/
+ ```
+
+ </CodeBlockConfig>
+
+1. Create secrets you wish to sync with a target Vercel project.
+
+ ```shell-session
+ $ vault kv put -mount='my-kv' my-secret foo='bar'
+ ```
+
+ **Output:**
+
+ <CodeBlockConfig hideClipboard>
+
+ ```plaintext
+ ==== Secret Path ====
+ my-kv/data/my-secret
+
+ ======= Metadata =======
+ Key Value
+ --- -----
+ created_time <timestamp>
+ custom_metadata <nil>
+ deletion_time n/a
+ destroyed false
+ version 1
+ ```
+
+ </CodeBlockConfig>
+
+1. Create an association between the destination and a secret to synchronize.
+
+ ```shell-session
+ $ vault write sys/sync/destinations/vercel-project/my-dest/associations/set \
+ mount='my-kv' \
+ secret_name='my-secret'
+ ```
+
+ **Output:**
+
+ <CodeBlockConfig hideClipboard>
+
+ ```plaintext
+ Key Value
+ --- -----
+ associated_secrets map[kv_1234/my-secret:map[accessor:kv_1234 secret_name:my-secret sync_status:SYNCED updated_at:<timestamp>]]
+ store_name my-dest
+ store_type vercel-project
+ ```
+
+ </CodeBlockConfig>
+
+1. Navigate to your project's settings under the `Environment Variables` section to confirm your secret was successfully
+ created in your Vercel project.
+
+Moving forward, any modification on the Vault secret will be propagated in near real time to its Vercel environment variable
+counterpart. Creating a new secret version in Vault will overwrite the value in your Vercel Project. Deleting the secret
+or the association in Vault will delete the secret on Vercel as well.
+
+<Note>
+
+Vercel Project environment variables only support single value secrets, so KVv2 secrets from Vault will be stored as a JSON string.
+In the example above, the value for secret "my-secret" will be synced to Vercel as the JSON string `{"foo":"bar"}`.
+
+</Note>
+
+## API
+
+Please see the [secrets sync API](/vault/api-docs/system/secrets-sync) for more details.
enos/modules/vault_agent/scripts/set-up-approle-and-agent.sh+14 14
@@ -5,7 +5,7 @@
set -e
-binpath=${vault_install_dir}/vault
+binpath=${VAULT_INSTALL_DIR}/vault
fail() {
echo "$1" 1>&2
@@ -15,14 +15,14 @@ fail() {
test -x "$binpath" || fail "unable to locate vault binary at $binpath"
export VAULT_ADDR='http://127.0.0.1:8200'
-export VAULT_TOKEN='${vault_token}'
+[[ -z "$VAULT_TOKEN" ]] && fail "VAULT_TOKEN env variable has not been set"
# If approle was already enabled, disable it as we're about to re-enable it (the || true is so we don't fail if it doesn't already exist)
$binpath auth disable approle || true
-approle_create_status=$($binpath auth enable approle)
+$binpath auth enable approle
-approle_status=$($binpath write auth/approle/role/agent-role secret_id_ttl=700h token_num_uses=1000 token_ttl=600h token_max_ttl=700h secret_id_num_uses=1000)
+$binpath write auth/approle/role/agent-role secret_id_ttl=700h token_num_uses=1000 token_ttl=600h token_max_ttl=700h secret_id_num_uses=1000
ROLEID=$($binpath read --format=json auth/approle/role/agent-role/role-id | jq -r '.data.role_id')
@@ -36,8 +36,8 @@ if [[ "$SECRETID" == '' ]]; then
fail "expected SECRETID to be nonempty, but it is empty"
fi
-echo $ROLEID > /tmp/role-id
-echo $SECRETID > /tmp/secret-id
+echo "$ROLEID" > /tmp/role-id
+echo "$SECRETID" > /tmp/secret-id
cat > /tmp/vault-agent.hcl <<- EOM
pid_file = "/tmp/pidfile"
@@ -51,18 +51,18 @@ vault {
}
cache {
- enforce_consistency = "always"
- use_auto_auth_token = true
+ enforce_consistency = "always"
+ use_auto_auth_token = true
}
listener "tcp" {
- address = "127.0.0.1:8100"
- tls_disable = true
+ address = "127.0.0.1:8100"
+ tls_disable = true
}
template {
- destination = "${vault_agent_template_destination}"
- contents = "${vault_agent_template_contents}"
+ destination = "${VAULT_AGENT_TEMPLATE_DESTINATION}"
+ contents = "${VAULT_AGENT_TEMPLATE_CONTENTS}"
exec {
command = "pkill -F /tmp/pidfile"
}
@@ -72,7 +72,7 @@ auto_auth {
method {
type = "approle"
config = {
- role_id_file_path = "/tmp/role-id"
+ role_id_file_path = "/tmp/role-id"
secret_id_file_path = "/tmp/secret-id"
}
}
@@ -89,7 +89,7 @@ EOM
pkill -F /tmp/pidfile || true
# If the template file already exists, remove it
-rm ${vault_agent_template_destination} || true
+rm "${VAULT_AGENT_TEMPLATE_DESTINATION}" || true
# Run agent (it will kill itself when it finishes rendering the template)
$binpath agent -config=/tmp/vault-agent.hcl > /tmp/agent-logs.txt 2>&1
command/agentproxyshared/auth/azure/azure.go+6 3
@@ -10,6 +10,8 @@ import (
"io"
"net/http"
+ "github.com/hashicorp/go-secure-stdlib/parseutil"
+
policy "github.com/Azure/azure-sdk-for-go/sdk/azcore/policy"
az "github.com/Azure/azure-sdk-for-go/sdk/azidentity"
cleanhttp "github.com/hashicorp/go-cleanhttp"
@@ -101,10 +103,11 @@ func NewAzureAuthMethod(conf *auth.AuthConfig) (auth.AuthMethod, error) {
authenticateFromEnvironmentRaw, ok := conf.Config["authenticate_from_environment"]
if ok {
- a.authenticateFromEnvironment, ok = authenticateFromEnvironmentRaw.(bool)
- if !ok {
- return nil, errors.New("could not convert 'authenticate_from_environment' config value to bool")
+ authenticateFromEnvironment, err := parseutil.ParseBool(authenticateFromEnvironmentRaw)
+ if err != nil {
+ return nil, fmt.Errorf("could not convert 'authenticate_from_environment' config value to bool: %w", err)
}
+ a.authenticateFromEnvironment = authenticateFromEnvironment
}
switch {
website/content/docs/commands/plugin/runtime/list.mdx+40 0
@@ -0,0 +1,40 @@
+---
+layout: docs
+page_title: plugin runtime list - Command
+description: The "plugin runtime list" command lists all available plugin runtimes in the plugin runtime catalog.
+---
+
+# plugin list
+
+List all plugin runtimes currently registered with Vault. Returns all the
+available plugin runtimes or an error if the set of registered runtimes is empty.
+Vault considers any registered plugin runtime "available", regardless of whether
+it is currently in use.
+
+## Examples
+
+List all available plugin runtimes in the catalog.
+
+```shell-session
+$ vault plugin runtime list
+
+Name Type OCI Runtime Parent Cgroup CPU Nanos Memory Bytes
+---- ---- ----------- ------------- --------- ------------
+runc container runc n/a 0 0
+```
+
+## Usage
+
+The following flags are available in addition to the [standard set of
+flags](/vault/docs/commands) included on all commands.
+
+### Output options
+
+- `-format` `(string: "table")` - Print the output for the current command in
+ the given format. Valid formats are `table`, `json`, or `yaml`. Use the
+ `VAULT_FORMAT` environment variable to set your output preferences globally.
+
+### Command options
+
+- `-type` `(string: "")` - Plugin runtime type. Vault currently only supports
+ `container` runtime type.
enos/modules/vault_cluster/main.tf+67 61
@@ -109,9 +109,11 @@ resource "enos_remote_exec" "install_packages" {
if length(var.packages) > 0
}
- content = templatefile("${path.module}/templates/install-packages.sh", {
- packages = join(" ", var.packages)
- })
+ environment = {
+ PACKAGES = join(" ", var.packages)
+ }
+
+ scripts = [abspath("${path.module}/scripts/install-packages.sh")]
transport = {
ssh = {
@@ -271,59 +273,6 @@ resource "enos_vault_unseal" "leader" {
}
}
-# We need to ensure that the directory used for audit logs is present and accessible to the vault
-# user on all nodes, since logging will only happen on the leader.
-resource "enos_remote_exec" "create_audit_log_dir" {
- depends_on = [
- enos_bundle_install.vault,
- enos_vault_unseal.leader,
- ]
- for_each = toset([
- for idx, host in toset(local.instances) : idx
- if var.enable_audit_devices
- ])
-
- environment = {
- LOG_FILE_PATH = local.audit_device_file_path
- SERVICE_USER = local.vault_service_user
- }
-
- scripts = [abspath("${path.module}/scripts/create_audit_log_dir.sh")]
-
- transport = {
- ssh = {
- host = var.target_hosts[each.value].public_ip
- }
- }
-}
-
-resource "enos_remote_exec" "enable_audit_devices" {
- depends_on = [
- enos_remote_exec.create_audit_log_dir,
- enos_vault_unseal.leader,
- ]
- for_each = toset([
- for idx in local.leader : idx
- if local.enable_audit_devices
- ])
-
- environment = {
- VAULT_TOKEN = enos_vault_init.leader[each.key].root_token
- VAULT_ADDR = "http://127.0.0.1:8200"
- VAULT_BIN_PATH = local.bin_path
- LOG_FILE_PATH = local.audit_device_file_path
- SERVICE_USER = local.vault_service_user
- }
-
- scripts = [abspath("${path.module}/scripts/enable_audit_logging.sh")]
-
- transport = {
- ssh = {
- host = var.target_hosts[each.key].public_ip
- }
- }
-}
-
resource "enos_vault_unseal" "followers" {
depends_on = [
enos_vault_init.leader,
@@ -387,11 +336,42 @@ resource "enos_remote_exec" "vault_write_license" {
enos_vault_unseal.maybe_force_unseal,
]
- content = templatefile("${path.module}/templates/vault-write-license.sh", {
- bin_path = local.bin_path,
- root_token = coalesce(var.root_token, try(enos_vault_init.leader[0].root_token, null), "none")
- license = coalesce(var.license, "none")
- })
+ environment = {
+ BIN_PATH = local.bin_path,
+ LICENSE = coalesce(var.license, "none")
+ VAULT_TOKEN = coalesce(var.root_token, try(enos_vault_init.leader[0].root_token, null), "none")
+ }
+
+ scripts = [abspath("${path.module}/scripts/vault-write-license.sh")]
+
+ transport = {
+ ssh = {
+ host = var.target_hosts[each.value].public_ip
+ }
+ }
+}
+
+# We need to ensure that the directory used for audit logs is present and accessible to the vault
+# user on all nodes, since logging will only happen on the leader.
+resource "enos_remote_exec" "create_audit_log_dir" {
+ depends_on = [
+ enos_vault_start.leader,
+ enos_vault_start.followers,
+ enos_vault_unseal.leader,
+ enos_vault_unseal.followers,
+ enos_vault_unseal.maybe_force_unseal,
+ ]
+ for_each = toset([
+ for idx, host in toset(local.instances) : idx
+ if var.enable_audit_devices
+ ])
+
+ environment = {
+ LOG_FILE_PATH = local.audit_device_file_path
+ SERVICE_USER = local.vault_service_user
+ }
+
+ scripts = [abspath("${path.module}/scripts/create_audit_log_dir.sh")]
transport = {
ssh = {
@@ -400,6 +380,32 @@ resource "enos_remote_exec" "vault_write_license" {
}
}
+resource "enos_remote_exec" "enable_audit_devices" {
+ depends_on = [
+ enos_remote_exec.create_audit_log_dir,
+ ]
+ for_each = toset([
+ for idx in local.leader : idx
+ if local.enable_audit_devices
+ ])
+
+ environment = {
+ VAULT_TOKEN = enos_vault_init.leader[each.key].root_token
+ VAULT_ADDR = "http://127.0.0.1:8200"
+ VAULT_BIN_PATH = local.bin_path
+ LOG_FILE_PATH = local.audit_device_file_path
+ SERVICE_USER = local.vault_service_user
+ }
+
+ scripts = [abspath("${path.module}/scripts/enable_audit_logging.sh")]
+
+ transport = {
+ ssh = {
+ host = var.target_hosts[each.key].public_ip
+ }
+ }
+}
+
resource "enos_local_exec" "wait_for_install_packages" {
depends_on = [
enos_remote_exec.install_packages,
More files changed — see the full commit.

References