Security context
Medium· 5.3GHSA-6239-28c2-9mrm CVE-2021-38554CWE-212Published Aug 30, 2021

Improper Removal of Sensitive Information Before Storage or Transfer in HashiCorp Vault

Research this vulnerability

Research is free — Hunters explains how the bug works, the root-cause code pattern, how the fix addresses it, and how to test whether a target is affected, in chat. Investigate & write exploit is a paid run — the engine reads the advisory and fix commits, then builds and validates a working proof-of-concept exploit with reproduction steps.

Affected versions

0 → fixed in 1.6.61.7.0 → fixed in 1.7.4

Details

HashiCorp Vault and Vault Enterprise’s UI erroneously cached and exposed user-viewed secrets between sessions in a single shared browser. Fixed in 1.8.0 and pending 1.7.4 / 1.6.6 releases.

The fix

Release delta 1.7.0 → 1.7.4 (contains the fix)

· Mar 24, 2021, 09:05 PM+28451197compare
vendor/github.com/gocql/gocql/doc.go+315 2
@@ -4,6 +4,319 @@
// Package gocql implements a fast and robust Cassandra driver for the
// Go programming language.
+//
+// Connecting to the cluster
+//
+// Pass a list of initial node IP addresses to NewCluster to create a new cluster configuration:
+//
+// cluster := gocql.NewCluster("192.168.1.1", "192.168.1.2", "192.168.1.3")
+//
+// Port can be specified as part of the address, the above is equivalent to:
+//
+// cluster := gocql.NewCluster("192.168.1.1:9042", "192.168.1.2:9042", "192.168.1.3:9042")
+//
+// It is recommended to use the value set in the Cassandra config for broadcast_address or listen_address,
+// an IP address not a domain name. This is because events from Cassandra will use the configured IP
+// address, which is used to index connected hosts. If the domain name specified resolves to more than 1 IP address
+// then the driver may connect multiple times to the same host, and will not mark the node being down or up from events.
+//
+// Then you can customize more options (see ClusterConfig):
+//
+// cluster.Keyspace = "example"
+// cluster.Consistency = gocql.Quorum
+// cluster.ProtoVersion = 4
+//
+// The driver tries to automatically detect the protocol version to use if not set, but you might want to set the
+// protocol version explicitly, as it's not defined which version will be used in certain situations (for example
+// during upgrade of the cluster when some of the nodes support different set of protocol versions than other nodes).
+//
+// When ready, create a session from the configuration. Don't forget to Close the session once you are done with it:
+//
+// session, err := cluster.CreateSession()
+// if err != nil {
+// return err
+// }
+// defer session.Close()
+//
+// Authentication
+//
+// CQL protocol uses a SASL-based authentication mechanism and so consists of an exchange of server challenges and
+// client response pairs. The details of the exchanged messages depend on the authenticator used.
+//
+// To use authentication, set ClusterConfig.Authenticator or ClusterConfig.AuthProvider.
+//
+// PasswordAuthenticator is provided to use for username/password authentication:
+//
+// cluster := gocql.NewCluster("192.168.1.1", "192.168.1.2", "192.168.1.3")
+// cluster.Authenticator = gocql.PasswordAuthenticator{
+// Username: "user",
+// Password: "password"
+// }
+// session, err := cluster.CreateSession()
+// if err != nil {
+// return err
+// }
+// defer session.Close()
+//
+// Transport layer security
+//
+// It is possible to secure traffic between the client and server with TLS.
+//
+// To use TLS, set the ClusterConfig.SslOpts field. SslOptions embeds *tls.Config so you can set that directly.
+// There are also helpers to load keys/certificates from files.
+//
+// Warning: Due to historical reasons, the SslOptions is insecure by default, so you need to set EnableHostVerification
+// to true if no Config is set. Most users should set SslOptions.Config to a *tls.Config.
+// SslOptions and Config.InsecureSkipVerify interact as follows:
+//
+// Config.InsecureSkipVerify | EnableHostVerification | Result
+// Config is nil | false | do not verify host
+// Config is nil | true | verify host
+// false | false | verify host
+// true | false | do not verify host
+// false | true | verify host
+// true | true | verify host
+//
+// For example:
+//
+// cluster := gocql.NewCluster("192.168.1.1", "192.168.1.2", "192.168.1.3")
+// cluster.SslOpts = &gocql.SslOptions{
+// EnableHostVerification: true,
+// }
+// session, err := cluster.CreateSession()
+// if err != nil {
+// return err
+// }
+// defer session.Close()
+//
+// Executing queries
+//
+// Create queries with Session.Query. Query values must not be reused between different executions and must not be
+// modified after starting execution of the query.
+//
+// To execute a query without reading results, use Query.Exec:
+//
+// err := session.Query(`INSERT INTO tweet (timeline, id, text) VALUES (?, ?, ?)`,
+// "me", gocql.TimeUUID(), "hello world").WithContext(ctx).Exec()
+//
+// Single row can be read by calling Query.Scan:
+//
+// err := session.Query(`SELECT id, text FROM tweet WHERE timeline = ? LIMIT 1`,
+// "me").WithContext(ctx).Consistency(gocql.One).Scan(&id, &text)
+//
+// Multiple rows can be read using Iter.Scanner:
+//
+// scanner := session.Query(`SELECT id, text FROM tweet WHERE timeline = ?`,
+// "me").WithContext(ctx).Iter().Scanner()
+// for scanner.Next() {
+// var (
+// id gocql.UUID
+// text string
+// )
+// err = scanner.Scan(&id, &text)
+// if err != nil {
+// log.Fatal(err)
+// }
+// fmt.Println("Tweet:", id, text)
+// }
+// // scanner.Err() closes the iterator, so scanner nor iter should be used afterwards.
+// if err := scanner.Err(); err != nil {
+// log.Fatal(err)
+// }
+//
+// See Example for complete example.
+//
+// Prepared statements
+//
+// The driver automatically prepares DML queries (SELECT/INSERT/UPDATE/DELETE/BATCH statements) and maintains a cache
+// of prepared statements.
+// CQL protocol does not support preparing other query types.
+//
+// When using CQL protocol >= 4, it is possible to use gocql.UnsetValue as the bound value of a column.
+// This will cause the database to ignore writing the column.
+// The main advantage is the ability to keep the same prepared statement even when you don't
+// want to update some fields, where before you needed to make another prepared statement.
+//
+// Executing multiple queries concurrently
+//
+// Session is safe to use from multiple goroutines, so to execute multiple concurrent queries, just execute them
+// from several worker goroutines. Gocql provides synchronously-looking API (as recommended for Go APIs) and the queries
+// are executed asynchronously at the protocol level.
+//
+// results := make(chan error, 2)
+// go func() {
+// results <- session.Query(`INSERT INTO tweet (timeline, id, text) VALUES (?, ?, ?)`,
+// "me", gocql.TimeUUID(), "hello world 1").Exec()
+// }()
+// go func() {
+// results <- session.Query(`INSERT INTO tweet (timeline, id, text) VALUES (?, ?, ?)`,
+// "me", gocql.TimeUUID(), "hello world 2").Exec()
+// }()
+//
+// Nulls
+//
+// Null values are are unmarshalled as zero value of the type. If you need to distinguish for example between text
+// column being null and empty string, you can unmarshal into *string variable instead of string.
+//
+// var text *string
+// err := scanner.Scan(&text)
+// if err != nil {
+// // handle error
+// }
+// if text != nil {
+// // not null
+// }
+// else {
+// // null
+// }
+//
+// See Example_nulls for full example.
+//
+// Reusing slices
+//
+// The driver reuses backing memory of slices when unmarshalling. This is an optimization so that a buffer does not
+// need to be allocated for every processed row. However, you need to be careful when storing the slices to other
+// memory structures.
+//
+// scanner := session.Query(`SELECT myints FROM table WHERE pk = ?`, "key").WithContext(ctx).Iter().Scanner()
+// var myInts []int
+// for scanner.Next() {
+// // This scan reuses backing store of myInts for each row.
+// err = scanner.Scan(&myInts)
+// if err != nil {
+// log.Fatal(err)
+// }
+// }
+//
+// When you want to save the data for later use, pass a new slice every time. A common pattern is to declare the
+// slice variable within the scanner loop:
+//
+// scanner := session.Query(`SELECT myints FROM table WHERE pk = ?`, "key").WithContext(ctx).Iter().Scanner()
+// for scanner.Next() {
+// var myInts []int
+// // This scan always gets pointer to fresh myInts slice, so does not reuse memory.
+// err = scanner.Scan(&myInts)
+// if err != nil {
+// log.Fatal(err)
+// }
+// }
+//
+// Paging
+//
+// The driver supports paging of results with automatic prefetch, see ClusterConfig.PageSize, Session.SetPrefetch,
+// Query.PageSize, and Query.Prefetch.
+//
+// It is also possible to control the paging manually with Query.PageState (this disables automatic prefetch).
+// Manual paging is useful if you want to store the page state externally, for example in a URL to allow users
+// browse pages in a result. You might want to sign/encrypt the paging state when exposing it externally since
+// it contains data from primary keys.
+//
+// Paging state is specific to the CQL protocol version and the exact query used. It is meant as opaque state that
+// should not be modified. If you send paging state from different query or protocol version, then the behaviour
+// is not defined (you might get unexpected results or an error from the server). For example, do not send paging state
+// returned by node using protocol version 3 to a node using protocol version 4. Also, when using protocol version 4,
+// paging state between Cassandra 2.2 and 3.0 is incompatible (https://issues.apache.org/jira/browse/CASSANDRA-10880).
+//
+// The driver does not check whether the paging state is from the same protocol version/statement.
+// You might want to validate yourself as this could be a problem if you store paging state externally.
+// For example, if you store paging state in a URL, the URLs might become broken when you upgrade your cluster.
+//
+// Call Query.PageState(nil) to fetch just the first page of the query results. Pass the page state returned by
+// Iter.PageState to Query.PageState of a subsequent query to get the next page. If the length of slice returned
+// by Iter.PageState is zero, there are no more pages available (or an error occurred).
+//
+// Using too low values of PageSize will negatively affect performance, a value below 100 is probably too low.
+// While Cassandra returns exactly PageSize items (except for last page) in a page currently, the protocol authors
+// explicitly reserved the right to return smaller or larger amount of items in a page for performance reasons, so don't
+// rely on the page having the exact count of items.
+//
+// See Example_paging for an example of manual paging.
+//
+// Dynamic list of columns
+//
+// There are certain situations when you don't know the list of columns in advance, mainly when the query is supplied
+// by the user. Iter.Columns, Iter.RowData, Iter.MapScan and Iter.SliceMap can be used to handle this case.
+//
+// See Example_dynamicColumns.
+//
+// Batches
+//
+// The CQL protocol supports sending batches of DML statements (INSERT/UPDATE/DELETE) and so does gocql.
+// Use Session.NewBatch to create a new batch and then fill-in details of individual queries.
+// Then execute the batch with Session.ExecuteBatch.
+//
+// Logged batches ensure atomicity, either all or none of the operations in the batch will succeed, but they have
+// overhead to ensure this property.
+// Unlogged batches don't have the overhead of logged batches, but don't guarantee atomicity.
+// Updates of counters are handled specially by Cassandra so batches of counter updates have to use CounterBatch type.
+// A counter batch can only contain statements to update counters.
+//
+// For unlogged batches it is recommended to send only single-partition batches (i.e. all statements in the batch should
+// involve only a single partition).
+// Multi-partition batch needs to be split by the coordinator node and re-sent to
+// correct nodes.
+// With single-partition batches you can send the batch directly to the node for the partition without incurring the
+// additional network hop.
+//
+// It is also possible to pass entire BEGIN BATCH .. APPLY BATCH statement to Query.Exec.
+// There are differences how those are executed.
+// BEGIN BATCH statement passed to Query.Exec is prepared as a whole in a single statement.
+// Session.ExecuteBatch prepares individual statements in the batch.
+// If you have variable-length batches using the same statement, using Session.ExecuteBatch is more efficient.
+//
+// See Example_batch for an example.
+//
+// Lightweight transactions
+//
+// Query.ScanCAS or Query.MapScanCAS can be used to execute a single-statement lightweight transaction (an
+// INSERT/UPDATE .. IF statement) and reading its result. See example for Query.MapScanCAS.
+//
+// Multiple-statement lightweight transactions can be executed as a logged batch that contains at least one conditional
+// statement. All the conditions must return true for the batch to be applied. You can use Session.ExecuteBatchCAS and
+// Session.MapExecuteBatchCAS when executing the batch to learn about the result of the LWT. See example for
+// Session.MapExecuteBatchCAS.
+//
+// Retries and speculative execution
+//
+// Queries can be marked as idempotent. Marking the query as idempotent tells the driver that the query can be executed
+// multiple times without affecting its result. Non-idempotent queries are not eligible for retrying nor speculative
+// execution.
+//
+// Idempotent queries are retried in case of errors based on the configured RetryPolicy.
+//
+// Queries can be retried even before they fail by setting a SpeculativeExecutionPolicy. The policy can
+// cause the driver to retry on a different node if the query is taking longer than a specified delay even before the
+// driver receives an error or timeout from the server. When a query is speculatively executed, the original execution
+// is still executing. The two parallel executions of the query race to return a result, the first received result will
+// be returned.
+//
+// User-defined types
+//
+// UDTs can be mapped (un)marshaled from/to map[string]interface{} a Go struct (or a type implementing
+// UDTUnmarshaler, UDTMarshaler, Unmarshaler or Marshaler interfaces).
+//
+// For structs, cql tag can be used to specify the CQL field name to be mapped to a struct field:
+//
+// type MyUDT struct {
+// FieldA int32 `cql:"a"`
+// FieldB string `cql:"b"`
+// }
+//
+// See Example_userDefinedTypesMap, Example_userDefinedTypesStruct, ExampleUDTMarshaler, ExampleUDTUnmarshaler.
+//
+// Metrics and tracing
+//
+// It is possible to provide observer implementations that could be used to gather metrics:
+//
+// - QueryObserver for monitoring individual queries.
+// - BatchObserver for monitoring batch queries.
+// - ConnectObserver for monitoring new connections from the driver to the database.
+// - FrameHeaderObserver for monitoring individual protocol frames.
+//
+// CQL protocol also supports tracing of queries. When enabled, the database will write information about
+// internal events that happened during execution of the query. You can use Query.Trace to request tracing and receive
+// the session ID that the database used to store the trace information in system_traces.sessions and
+// system_traces.events tables. NewTraceWriter returns an implementation of Tracer that writes the events to a writer.
+// Gathering trace information might be essential for debugging and optimizing queries, but writing traces has overhead,
+// so this feature should not be used on production systems with very high load unless you know what you are doing.
package gocql // import "github.com/gocql/gocql"
-
-// TODO(tux21b): write more docs.
CHANGELOG.md+15 3
@@ -1,15 +1,21 @@
-## 1.7.0-rc1
-### 10 March 2021
+## 1.7.0
+### 24 March 2021
CHANGES:
-* go: Update go version to 1.15.8 [[GH-11060](https://github.com/hashicorp/vault/pull/11060)]
+* go: Update Go version to 1.15.10 [[GH-11173](https://github.com/hashicorp/vault/pull/11173)]
FEATURES:
* **Aerospike Storage Backend**: Add support for using Aerospike as a storage backend [[GH-10131](https://github.com/hashicorp/vault/pull/10131)]
+* **Autopilot for Integrated Storage**: A set of features has been added to allow for automatic operator-friendly management of Vault servers. This is only applicable when integrated storage is in use.
+ * **Dead Server Cleanup**: Dead servers will periodically be cleaned up and removed from the Raft peer set, to prevent them from interfering with the quorum size and leader elections.
+ * **Server Health Checking**: An API has been added to track the state of servers, including their health.
+ * **New Server Stabilization**: When a new server is added to the cluster, there will be a waiting period where it must be healthy and stable for a certain amount of time before being promoted to a full, voting member.
+* **Tokenization Secrets Engine (Enterprise)**: The Tokenization Secrets Engine is now generally available. We have added support for MySQL, key rotation, and snapshot/restore.
* agent: Support for persisting the agent cache to disk [[GH-10938](https://github.com/hashicorp/vault/pull/10938)]
* auth/jwt: Adds `max_age` role parameter and `auth_time` claim validation. [[GH-10919](https://github.com/hashicorp/vault/pull/10919)]
+* core (enterprise): X-Vault-Index and related headers can be used by clients to manage eventual consistency.
* kmip (enterprise): Use entropy augmentation to generate kmip certificates
* sdk: Private key generation in the certutil package now allows custom io.Readers to be used. [[GH-10653](https://github.com/hashicorp/vault/pull/10653)]
* secrets/aws: add IAM tagging support for iam_user roles [[GH-10953](https://github.com/hashicorp/vault/pull/10953)]
@@ -19,6 +25,9 @@ FEATURES:
* secrets/database/mssql: Add ability to customize dynamic usernames [[GH-10767](https://github.com/hashicorp/vault/pull/10767)]
* secrets/database/mysql: Add ability to customize dynamic usernames [[GH-10834](https://github.com/hashicorp/vault/pull/10834)]
* secrets/database/postgresql: Add ability to customize dynamic usernames [[GH-10766](https://github.com/hashicorp/vault/pull/10766)]
+* secrets/db/snowflake: Added support for Snowflake to the Database Secret Engine [[GH-10603](https://github.com/hashicorp/vault/pull/10603)]
+* secrets/keymgmt (enterprise): Adds beta support for distributing and managing keys in AWS KMS.
+* secrets/keymgmt (enterprise): Adds general availability for distributing and managing keys in Azure Key Vault.
* secrets/openldap: Added dynamic roles to OpenLDAP similar to the combined database engine [[GH-10996](https://github.com/hashicorp/vault/pull/10996)]
* secrets/terraform: New secret engine for managing Terraform Cloud API tokens [[GH-10931](https://github.com/hashicorp/vault/pull/10931)]
* ui: Adds check for feature flag on application, and updates namespace toolbar on login if present [[GH-10588](https://github.com/hashicorp/vault/pull/10588)]
@@ -51,6 +60,8 @@ IMPROVEMENTS:
* storage/raft (enterprise): Listing of peers is now allowed on DR secondary
cluster nodes, as an update operation that takes in DR operation token for
authenticating the request.
+* transform (enterprise): Improve FPE transformation performance
+* transform (enterprise): Use transactions with batch tokenization operations for improved performance
* ui: Clarify language on usage metrics page empty state [[GH-10951](https://github.com/hashicorp/vault/pull/10951)]
* ui: Customize MongoDB input fields on Database Secrets Engine [[GH-10949](https://github.com/hashicorp/vault/pull/10949)]
* ui: Upgrade Ember-cli from 3.8 to 3.22. [[GH-9972](https://github.com/hashicorp/vault/pull/9972)]
@@ -108,6 +119,7 @@ the given key will be used to encrypt the snapshot using AWS KMS.
* transform (enterprise): Fix transform configuration not handling `stores` parameter on the legacy path
* transform (enterprise): Make expiration timestamps human readable
* transform (enterprise): Return false for invalid tokens on the validate endpoint rather than returning an HTTP error
+* ui: Add role from database connection automatically populates the database for new role [[GH-11119](https://github.com/hashicorp/vault/pull/11119)]
* ui: Fix bug in Transform secret engine when a new role is added and then removed from a transformation [[GH-10417](https://github.com/hashicorp/vault/pull/10417)]
* ui: Fix bug that double encodes secret route when there are spaces in the path and makes you unable to view the version history. [[GH-10596](https://github.com/hashicorp/vault/pull/10596)]
* ui: Fix expected response from feature-flags endpoint [[GH-10684](https://github.com/hashicorp/vault/pull/10684)]
changelog/11213.txt | 3 +++
ui/app/services/namespace.js | 6 +++++-
2 files changed, 8 insertions(+), 1 deletion(-)
create mode 100644 changelog/11213.txt
plugins/database/cassandra/test-fixtures/with_tls/cassandra.yaml+399 0
@@ -0,0 +1,1279 @@
+# Cassandra storage config YAML
+
+# NOTE:
+# See http://wiki.apache.org/cassandra/StorageConfiguration for
+# full explanations of configuration directives
+# /NOTE
+
+# The name of the cluster. This is mainly used to prevent machines in
+# one logical cluster from joining another.
+cluster_name: 'Test Cluster'
+
+# This defines the number of tokens randomly assigned to this node on the ring
+# The more tokens, relative to other nodes, the larger the proportion of data
+# that this node will store. You probably want all nodes to have the same number
+# of tokens assuming they have equal hardware capability.
+#
+# If you leave this unspecified, Cassandra will use the default of 1 token for legacy compatibility,
+# and will use the initial_token as described below.
+#
+# Specifying initial_token will override this setting on the node's initial start,
+# on subsequent starts, this setting will apply even if initial token is set.
+#
+# If you already have a cluster with 1 token per node, and wish to migrate to
+# multiple tokens per node, see http://wiki.apache.org/cassandra/Operations
+num_tokens: 256
+
+# Triggers automatic allocation of num_tokens tokens for this node. The allocation
+# algorithm attempts to choose tokens in a way that optimizes replicated load over
+# the nodes in the datacenter for the replication strategy used by the specified
+# keyspace.
+#
+# The load assigned to each node will be close to proportional to its number of
+# vnodes.
+#
+# Only supported with the Murmur3Partitioner.
+# allocate_tokens_for_keyspace: KEYSPACE
+
+# initial_token allows you to specify tokens manually. While you can use it with
+# vnodes (num_tokens > 1, above) -- in which case you should provide a
+# comma-separated list -- it's primarily used when adding nodes to legacy clusters
+# that do not have vnodes enabled.
+# initial_token:
+
+# See http://wiki.apache.org/cassandra/HintedHandoff
+# May either be "true" or "false" to enable globally
+hinted_handoff_enabled: true
+
+# When hinted_handoff_enabled is true, a black list of data centers that will not
+# perform hinted handoff
+# hinted_handoff_disabled_datacenters:
+# - DC1
+# - DC2
+
+# this defines the maximum amount of time a dead host will have hints
+# generated. After it has been dead this long, new hints for it will not be
+# created until it has been seen alive and gone down again.
+max_hint_window_in_ms: 10800000 # 3 hours
+
+# Maximum throttle in KBs per second, per delivery thread. This will be
+# reduced proportionally to the number of nodes in the cluster. (If there
+# are two nodes in the cluster, each delivery thread will use the maximum
+# rate; if there are three, each will throttle to half of the maximum,
+# since we expect two nodes to be delivering hints simultaneously.)
+hinted_handoff_throttle_in_kb: 1024
+
+# Number of threads with which to deliver hints;
+# Consider increasing this number when you have multi-dc deployments, since
+# cross-dc handoff tends to be slower
+max_hints_delivery_threads: 2
+
+# Directory where Cassandra should store hints.
+# If not set, the default directory is $CASSANDRA_HOME/data/hints.
+# hints_directory: /var/lib/cassandra/hints
+
+# How often hints should be flushed from the internal buffers to disk.
+# Will *not* trigger fsync.
+hints_flush_period_in_ms: 10000
+
+# Maximum size for a single hints file, in megabytes.
+max_hints_file_size_in_mb: 128
+
+# Compression to apply to the hint files. If omitted, hints files
+# will be written uncompressed. LZ4, Snappy, and Deflate compressors
+# are supported.
+#hints_compression:
+# - class_name: LZ4Compressor
+# parameters:
+# -
+
+# Maximum throttle in KBs per second, total. This will be
+# reduced proportionally to the number of nodes in the cluster.
+batchlog_replay_throttle_in_kb: 1024
+
+# Authentication backend, implementing IAuthenticator; used to identify users
+# Out of the box, Cassandra provides org.apache.cassandra.auth.{AllowAllAuthenticator,
+# PasswordAuthenticator}.
+#
+# - AllowAllAuthenticator performs no checks - set it to disable authentication.
+# - PasswordAuthenticator relies on username/password pairs to authenticate
+# users. It keeps usernames and hashed passwords in system_auth.roles table.
+# Please increase system_auth keyspace replication factor if you use this authenticator.
+# If using PasswordAuthenticator, CassandraRoleManager must also be used (see below)
+authenticator: PasswordAuthenticator
+
+# Authorization backend, implementing IAuthorizer; used to limit access/provide permissions
+# Out of the box, Cassandra provides org.apache.cassandra.auth.{AllowAllAuthorizer,
+# CassandraAuthorizer}.
+#
+# - AllowAllAuthorizer allows any action to any user - set it to disable authorization.
+# - CassandraAuthorizer stores permissions in system_auth.role_permissions table. Please
+# increase system_auth keyspace replication factor if you use this authorizer.
+authorizer: CassandraAuthorizer
+
+# Part of the Authentication & Authorization backend, implementing IRoleManager; used
+# to maintain grants and memberships between roles.
+# Out of the box, Cassandra provides org.apache.cassandra.auth.CassandraRoleManager,
+# which stores role information in the system_auth keyspace. Most functions of the
+# IRoleManager require an authenticated login, so unless the configured IAuthenticator
+# actually implements authentication, most of this functionality will be unavailable.
+#
+# - CassandraRoleManager stores role data in the system_auth keyspace. Please
+# increase system_auth keyspace replication factor if you use this role manager.
+role_manager: CassandraRoleManager
+
+# Validity period for roles cache (fetching granted roles can be an expensive
+# operation depending on the role manager, CassandraRoleManager is one example)
+# Granted roles are cached for authenticated sessions in AuthenticatedUser and
+# after the period specified here, become eligible for (async) reload.
+# Defaults to 2000, set to 0 to disable caching entirely.
+# Will be disabled automatically for AllowAllAuthenticator.
+roles_validity_in_ms: 2000
+
+# Refresh interval for roles cache (if enabled).
+# After this interval, cache entries become eligible for refresh. Upon next
+# access, an async reload is scheduled and the old value returned until it
+# completes. If roles_validity_in_ms is non-zero, then this must be
+# also.
+# Defaults to the same value as roles_validity_in_ms.
+# roles_update_interval_in_ms: 2000
+
+# Validity period for permissions cache (fetching permissions can be an
+# expensive operation depending on the authorizer, CassandraAuthorizer is
+# one example). Defaults to 2000, set to 0 to disable.
+# Will be disabled automatically for AllowAllAuthorizer.
+permissions_validity_in_ms: 2000
+
+# Refresh interval for permissions cache (if enabled).
+# After this interval, cache entries become eligible for refresh. Upon next
+# access, an async reload is scheduled and the old value returned until it
+# completes. If permissions_validity_in_ms is non-zero, then this must be
+# also.
+# Defaults to the same value as permissions_validity_in_ms.
+# permissions_update_interval_in_ms: 2000
+
+# Validity period for credentials cache. This cache is tightly coupled to
+# the provided PasswordAuthenticator implementation of IAuthenticator. If
+# another IAuthenticator implementation is configured, this cache will not
+# be automatically used and so the following settings will have no effect.
+# Please note, credentials are cached in their encrypted form, so while
+# activating this cache may reduce the number of queries made to the
+# underlying table, it may not bring a significant reduction in the
+# latency of individual authentication attempts.
+# Defaults to 2000, set to 0 to disable credentials caching.
+credentials_validity_in_ms: 2000
+
+# Refresh interval for credentials cache (if enabled).
+# After this interval, cache entries become eligible for refresh. Upon next
+# access, an async reload is scheduled and the old value returned until it
+# completes. If credentials_validity_in_ms is non-zero, then this must be
+# also.
+# Defaults to the same value as credentials_validity_in_ms.
+# credentials_update_interval_in_ms: 2000
+
+# The partitioner is responsible for distributing groups of rows (by
+# partition key) across nodes in the cluster. You should leave this
+# alone for new clusters. The partitioner can NOT be changed without
+# reloading all data, so when upgrading you should set this to the
+# same partitioner you were already using.
+#
+# Besides Murmur3Partitioner, partitioners included for backwards
+# compatibility include RandomPartitioner, ByteOrderedPartitioner, and
+# OrderPreservingPartitioner.
+#
+partitioner: org.apache.cassandra.dht.Murmur3Partitioner
+
+# Directories where Cassandra should store data on disk. Cassandra
+# will spread data evenly across them, subject to the granularity of
+# the configured compaction strategy.
+# If not set, the default directory is $CASSANDRA_HOME/data/data.
+# data_file_directories:
+# - /var/lib/cassandra/data
+
+# commit log. when running on magnetic HDD, this should be a
+# separate spindle than the data directories.
+# If not set, the default directory is $CASSANDRA_HOME/data/commitlog.
+# commitlog_directory: /var/lib/cassandra/commitlog
+
+# Enable / disable CDC functionality on a per-node basis. This modifies the logic used
+# for write path allocation rejection (standard: never reject. cdc: reject Mutation
+# containing a CDC-enabled table if at space limit in cdc_raw_directory).
+cdc_enabled: false
+
+# CommitLogSegments are moved to this directory on flush if cdc_enabled: true and the
+# segment contains mutations for a CDC-enabled table. This should be placed on a
+# separate spindle than the data directories. If not set, the default directory is
+# $CASSANDRA_HOME/data/cdc_raw.
+# cdc_raw_directory: /var/lib/cassandra/cdc_raw
+
+# Policy for data disk failures:
+#
+# die
+# shut down gossip and client transports and kill the JVM for any fs errors or
+# single-sstable errors, so the node can be replaced.
+#
+# stop_paranoid
+# shut down gossip and client transports even for single-sstable errors,
+# kill the JVM for errors during startup.
+#
+# stop
+# shut down gossip and client transports, leaving the node effectively dead, but
+# can still be inspected via JMX, kill the JVM for errors during startup.
+#
+# best_effort
+# stop using the failed disk and respond to requests based on
+# remaining available sstables. This means you WILL see obsolete
+# data at CL.ONE!
+#
+# ignore
+# ignore fatal errors and let requests fail, as in pre-1.2 Cassandra
+disk_failure_policy: stop
+
+# Policy for commit disk failures:
+#
+# die
+# shut down gossip and Thrift and kill the JVM, so the node can be replaced.
+#
+# stop
+# shut down gossip and Thrift, leaving the node effectively dead, but
+# can still be inspected via JMX.
+#
+# stop_commit
+# shutdown the commit log, letting writes collect but
+# continuing to service reads, as in pre-2.0.5 Cassandra
+#
+# ignore
+# ignore fatal errors and let the batches fail
+commit_failure_policy: stop
+
+# Maximum size of the native protocol prepared statement cache
+#
+# Valid values are either "auto" (omitting the value) or a value greater 0.
+#
+# Note that specifying a too large value will result in long running GCs and possbily
+# out-of-memory errors. Keep the value at a small fraction of the heap.
+#
+# If you constantly see "prepared statements discarded in the last minute because
+# cache limit reached" messages, the first step is to investigate the root cause
+# of these messages and check whether prepared statements are used correctly -
+# i.e. use bind markers for variable parts.
+#
+# Do only change the default value, if you really have more prepared statements than
+# fit in the cache. In most cases it is not neccessary to change this value.
+# Constantly re-preparing statements is a performance penalty.
+#
+# Default value ("auto") is 1/256th of the heap or 10MB, whichever is greater
+prepared_statements_cache_size_mb:
+
+# Maximum size of the Thrift prepared statement cache
+#
+# If you do not use Thrift at all, it is safe to leave this value at "auto".
+#
+# See description of 'prepared_statements_cache_size_mb' above for more information.
+#
+# Default value ("auto") is 1/256th of the heap or 10MB, whichever is greater
+thrift_prepared_statements_cache_size_mb:
+
+# Maximum size of the key cache in memory.
+#
+# Each key cache hit saves 1 seek and each row cache hit saves 2 seeks at the
+# minimum, sometimes more. The key cache is fairly tiny for the amount of
+# time it saves, so it's worthwhile to use it at large numbers.
+# The row cache saves even more time, but must contain the entire row,
+# so it is extremely space-intensive. It's best to only use the
+# row cache if you have hot rows or static rows.
+#
+# NOTE: if you reduce the size, you may not get you hottest keys loaded on startup.
+#
+# Default value is empty to make it "auto" (min(5% of Heap (in MB), 100MB)). Set to 0 to disable key cache.
+key_cache_size_in_mb:
+
+# Duration in seconds after which Cassandra should
+# save the key cache. Caches are saved to saved_caches_directory as
+# specified in this configuration file.
+#
+# Saved caches greatly improve cold-start speeds, and is relatively cheap in
+# terms of I/O for the key cache. Row cache saving is much more expensive and
+# has limited use.
+#
+# Default is 14400 or 4 hours.
+key_cache_save_period: 14400
+
+# Number of keys from the key cache to save
+# Disabled by default, meaning all keys are going to be saved
+# key_cache_keys_to_save: 100
+
+# Row cache implementation class name. Available implementations:
+#
+# org.apache.cassandra.cache.OHCProvider
+# Fully off-heap row cache implementation (default).
+#
+# org.apache.cassandra.cache.SerializingCacheProvider
+# This is the row cache implementation availabile
+# in previous releases of Cassandra.
+# row_cache_class_name: org.apache.cassandra.cache.OHCProvider
+
+# Maximum size of the row cache in memory.
+# Please note that OHC cache implementation requires some additional off-heap memory to manage
+# the map structures and some in-flight memory during operations before/after cache entries can be
+# accounted against the cache capacity. This overhead is usually small compared to the whole capacity.
+# Do not specify more memory that the system can afford in the worst usual situation and leave some
+# headroom for OS block level cache. Do never allow your system to swap.
+#
+# Default value is 0, to disable row caching.
+row_cache_size_in_mb: 0
+
+# Duration in seconds after which Cassandra should save the row cache.
+# Caches are saved to saved_caches_directory as specified in this configuration file.
+#
+# Saved caches greatly improve cold-start speeds, and is relatively cheap in
+# terms of I/O for the key cache. Row cache saving is much more expensive and
+# has limited use.
+#
+# Default is 0 to disable saving the row cache.
+row_cache_save_period: 0
+
+# Number of keys from the row cache to save.
+# Specify 0 (which is the default), meaning all keys are going to be saved
+# row_cache_keys_to_save: 100
+
+# Maximum size of the counter cache in memory.
+#
+# Counter cache helps to reduce counter locks' contention for hot counter cells.
+# In case of RF = 1 a counter cache hit will cause Cassandra to skip the read before
+# write entirely. With RF > 1 a counter cache hit will still help to reduce the duration
+# of the lock hold, helping with hot counter cell updates, but will not allow skipping
+# the read entirely. Only the local (clock, count) tuple of a counter cell is kept
+# in memory, not the whole counter, so it's relatively cheap.
+#
+# NOTE: if you reduce the size, you may not get you hottest keys loaded on startup.
+#
+# Default value is empty to make it "auto" (min(2.5% of Heap (in MB), 50MB)). Set to 0 to disable counter cache.
+# NOTE: if you perform counter deletes and rely on low gcgs, you should disable the counter cache.
+counter_cache_size_in_mb:
+
+# Duration in seconds after which Cassandra should
+# save the counter cache (keys only). Caches are saved to saved_caches_directory as
+# specified in this configuration file.
+#
+# Default is 7200 or 2 hours.
+counter_cache_save_period: 7200
+
+# Number of keys from the counter cache to save
+# Disabled by default, meaning all keys are going to be saved
+# counter_cache_keys_to_save: 100
+
+# saved caches
+# If not set, the default directory is $CASSANDRA_HOME/data/saved_caches.
+# saved_caches_directory: /var/lib/cassandra/saved_caches
+
+# commitlog_sync may be either "periodic" or "batch."
+#
+# When in batch mode, Cassandra won't ack writes until the commit log
+# has been fsynced to disk. It will wait
+# commitlog_sync_batch_window_in_ms milliseconds between fsyncs.
+# This window should be kept short because the writer threads will
+# be unable to do extra work while waiting. (You may need to increase
+# concurrent_writes for the same reason.)
+#
+# commitlog_sync: batch
+# commitlog_sync_batch_window_in_ms: 2
+#
+# the other option is "periodic" where writes may be acked immediately
+# and the CommitLog is simply synced every commitlog_sync_period_in_ms
+# milliseconds.
+commitlog_sync: periodic
+commitlog_sync_period_in_ms: 10000
+
+# The size of the individual commitlog file segments. A commitlog
+# segment may be archived, deleted, or recycled once all the data
+# in it (potentially from each columnfamily in the system) has been
+# flushed to sstables.
+#
+# The default size is 32, which is almost always fine, but if you are
+# archiving commitlog segments (see commitlog_archiving.properties),
+# then you probably want a finer granularity of archiving; 8 or 16 MB
+# is reasonable.
+# Max mutation size is also configurable via max_mutation_size_in_kb setting in
+# cassandra.yaml. The default is half the size commitlog_segment_size_in_mb * 1024.
+# This should be positive and less than 2048.
… diff truncated
website/content/api-docs/secret/pki.mdx+15 13
@@ -1573,9 +1573,11 @@ expiration time.
- `tidy_cert_store` `(bool: false)` Specifies whether to tidy up the certificate
store.
-- `tidy_revoked_certs` `(bool: false)` Set to true to expire all revoked and
- expired certificates, removing them both from the CRL and from storage. The
- CRL will be rotated if this causes any values to be removed.
+- `tidy_revoked_certs` `(bool: false)` Set to true to remove all invalid and
+ expired certificates from storage. A revoked storage entry is considered
+ invalid if the entry is empty, or the value within the entry is empty. If a
+ certificate is removed due to expiry, the entry will also be removed from the
+ CRL, and the CRL will be rotated.
- `safety_buffer` `(string: "")` Specifies A duration (given as an integer
number of seconds or a string; defaults to `72h`) used as a safety buffer to
@@ -1605,29 +1607,29 @@ $ curl \
# Cluster Scalability
-Most non-introspection operations in the PKI secrets engine require a write to
-storage, and so are forwarded to the cluster's active node for execution.
-This table outlines which operations can be executed on performance standbys
+Most non-introspection operations in the PKI secrets engine require a write to
+storage, and so are forwarded to the cluster's active node for execution.
+This table outlines which operations can be executed on performance standbys
and thus scale horizontally.
| Path | Operations |
-| --------------------------- | ------------------- |
-| ca[/pem] | Read |
-| cert/<em>serial-number</em> | Read |
+| --------------------------- | ------------------- |
+| ca[/pem] | Read |
+| cert/<em>serial-number</em> | Read |
| cert/ca_chain | Read |
| config/crl | Read |
-| certs | List |
+| certs | List |
| ca_chain | Read |
| crl[/pem] | Read |
-| crl/pem | Read |
+| crl/pem | Read |
| issue | Update <sup>*</sup> |
| revoked/* | Read, List |
| sign | Update <sup>*</sup> |
| sign-verbatim | Update <sup>*</sup> |
\* Only if the corresponding role has `no_store` set to true and `generate_lease`
-set to false. If `generate_lease` is true the lease creation will be forwarded to
-the active node; if `no_store` is false the entire request will be forwarded to
+set to false. If `generate_lease` is true the lease creation will be forwarded to
+the active node; if `no_store` is false the entire request will be forwarded to
the active node.
.circleci/config.yml | 448 +--
.circleci/config/@build-release.yml | 436 +--
.circleci/config/executors/@executors.yml | 8 +-
changelog/11395.txt | 3 +
packages-oss.lock/layers/layers.mk | 664 ++---
packages-oss.lock/pkgs.yml | 3122 ++++++++++-----------
packages-oss.yml | 2 +-
scripts/docker/Dockerfile | 2 +-
8 files changed, 2344 insertions(+), 2341 deletions(-)
create mode 100644 changelog/11395.txt
plugins/database/cassandra/cassandra_test.go+18 15
@@ -3,7 +3,6 @@ package cassandra
import (
"context"
"reflect"
- "strings"
"testing"
"time"
@@ -17,14 +16,16 @@ import (
)
func getCassandra(t *testing.T, protocolVersion interface{}) (*Cassandra, func()) {
- cleanup, connURL := cassandra.PrepareTestContainer(t, "latest")
- pieces := strings.Split(connURL, ":")
+ host, cleanup := cassandra.PrepareTestContainer(t,
+ cassandra.Version("latest"),
+ cassandra.CopyFromTo(insecureFileMounts),
+ )
db := new()
initReq := dbplugin.InitializeRequest{
Config: map[string]interface{}{
- "hosts": connURL,
- "port": pieces[1],
+ "hosts": host.ConnectionURL(),
+ "port": host.Port,
"username": "cassandra",
"password": "cassandra",
"protocol_version": protocolVersion,
@@ -34,8 +35,8 @@ func getCassandra(t *testing.T, protocolVersion interface{}) (*Cassandra, func()
}
expectedConfig := map[string]interface{}{
- "hosts": connURL,
- "port": pieces[1],
+ "hosts": host.ConnectionURL(),
+ "port": host.Port,
"username": "cassandra",
"password": "cassandra",
"protocol_version": protocolVersion,
@@ -53,7 +54,7 @@ func getCassandra(t *testing.T, protocolVersion interface{}) (*Cassandra, func()
return db, cleanup
}
-func TestCassandra_Initialize(t *testing.T) {
+func TestInitialize(t *testing.T) {
db, cleanup := getCassandra(t, 4)
defer cleanup()
@@ -66,7 +67,7 @@ func TestCassandra_Initialize(t *testing.T) {
defer cleanup()
}
-func TestCassandra_CreateUser(t *testing.T) {
+func TestCreateUser(t *testing.T) {
type testCase struct {
// Config will have the hosts & port added to it during the test
config map[string]interface{}
@@ -126,15 +127,17 @@ func TestCassandra_CreateUser(t *testing.T) {
for name, test := range tests {
t.Run(name, func(t *testing.T) {
- cleanup, connURL := cassandra.PrepareTestContainer(t, "latest")
- pieces := strings.Split(connURL, ":")
+ host, cleanup := cassandra.PrepareTestContainer(t,
+ cassandra.Version("latest"),
+ cassandra.CopyFromTo(insecureFileMounts),
+ )
defer cleanup()
db := new()
config := test.config
- config["hosts"] = connURL
- config["port"] = pieces[1]
+ config["hosts"] = host.ConnectionURL()
+ config["port"] = host.Port
initReq := dbplugin.InitializeRequest{
Config: config,
@@ -162,7 +165,7 @@ func TestCassandra_CreateUser(t *testing.T) {
}
}
-func TestMyCassandra_UpdateUserPassword(t *testing.T) {
+func TestUpdateUserPassword(t *testing.T) {
db, cleanup := getCassandra(t, 4)
defer cleanup()
@@ -198,7 +201,7 @@ func TestMyCassandra_UpdateUserPassword(t *testing.T) {
assertCreds(t, db.Hosts, db.Port, createResp.Username, newPassword, 5*time.Second)
}
-func TestCassandra_DeleteUser(t *testing.T) {
+func TestDeleteUser(t *testing.T) {
db, cleanup := getCassandra(t, 4)
defer cleanup()
vault/logical_system_raft.go+9 9
@@ -402,8 +402,8 @@ func (b *SystemBackend) handleStorageRaftSnapshotRead() framework.OperationFunc
func (b *SystemBackend) handleStorageRaftAutopilotState() framework.OperationFunc {
return func(ctx context.Context, req *logical.Request, d *framework.FieldData) (*logical.Response, error) {
- raftBackend, ok := b.Core.underlyingPhysical.(*raft.RaftBackend)
- if !ok {
+ raftBackend := b.Core.getRaftBackend()
+ if raftBackend == nil {
return logical.ErrorResponse("raft storage is not in use"), logical.ErrInvalidRequest
}
@@ -431,12 +431,12 @@ func (b *SystemBackend) handleStorageRaftAutopilotState() framework.OperationFun
func (b *SystemBackend) handleStorageRaftAutopilotConfigRead() framework.OperationFunc {
return func(ctx context.Context, req *logical.Request, d *framework.FieldData) (*logical.Response, error) {
- raftStorage, ok := b.Core.underlyingPhysical.(*raft.RaftBackend)
- if !ok {
+ raftBackend := b.Core.getRaftBackend()
+ if raftBackend == nil {
return logical.ErrorResponse("raft storage is not in use"), logical.ErrInvalidRequest
}
- config := raftStorage.AutopilotConfig()
+ config := raftBackend.AutopilotConfig()
if config == nil {
return nil, nil
}
@@ -456,8 +456,8 @@ func (b *SystemBackend) handleStorageRaftAutopilotConfigRead() framework.Operati
func (b *SystemBackend) handleStorageRaftAutopilotConfigUpdate() framework.OperationFunc {
return func(ctx context.Context, req *logical.Request, d *framework.FieldData) (*logical.Response, error) {
- raftStorage, ok := b.Core.underlyingPhysical.(*raft.RaftBackend)
- if !ok {
+ raftBackend := b.Core.getRaftBackend()
+ if raftBackend == nil {
return logical.ErrorResponse("raft storage is not in use"), logical.ErrInvalidRequest
}
@@ -506,7 +506,7 @@ func (b *SystemBackend) handleStorageRaftAutopilotConfigUpdate() framework.Opera
persist = true
}
- effectiveConf := raftStorage.AutopilotConfig()
+ effectiveConf := raftBackend.AutopilotConfig()
effectiveConf.Merge(config)
if effectiveConf.CleanupDeadServers && effectiveConf.MinQuorum < 3 {
@@ -525,7 +525,7 @@ func (b *SystemBackend) handleStorageRaftAutopilotConfigUpdate() framework.Opera
}
// Set the effectiveConfig
- raftStorage.SetAutopilotConfig(effectiveConf)
+ raftBackend.SetAutopilotConfig(effectiveConf)
return nil, nil
}
vendor/go.uber.org/goleak/options.go+164 0
@@ -0,0 +1,164 @@
+// Copyright (c) 2017 Uber Technologies, Inc.
+//
+// Permission is hereby granted, free of charge, to any person obtaining a copy
+// of this software and associated documentation files (the "Software"), to deal
+// in the Software without restriction, including without limitation the rights
+// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+// copies of the Software, and to permit persons to whom the Software is
+// furnished to do so, subject to the following conditions:
+//
+// The above copyright notice and this permission notice shall be included in
+// all copies or substantial portions of the Software.
+//
+// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+// THE SOFTWARE.
+
+package goleak
+
+import (
+ "strings"
+ "time"
+
+ "go.uber.org/goleak/internal/stack"
+)
+
+// Option lets users specify custom verifications.
+type Option interface {
+ apply(*opts)
+}
+
+// We retry up to 20 times if we can't find the goroutine that
+// we are looking for. In between each attempt, we will sleep for
+// a short while to let any running goroutines complete.
+const _defaultRetries = 20
+
+type opts struct {
+ filters []func(stack.Stack) bool
+ maxRetries int
+ maxSleep time.Duration
+}
+
+// optionFunc lets us easily write options without a custom type.
+type optionFunc func(*opts)
+
+func (f optionFunc) apply(opts *opts) { f(opts) }
+
+// IgnoreTopFunction ignores any goroutines where the specified function
+// is at the top of the stack. The function name should be fully qualified,
+// e.g., go.uber.org/goleak.IgnoreTopFunction
+func IgnoreTopFunction(f string) Option {
+ return addFilter(func(s stack.Stack) bool {
+ return s.FirstFunction() == f
+ })
+}
+
+// IgnoreCurrent records all current goroutines when the option is created, and ignores
+// them in any future Find/Verify calls.
+func IgnoreCurrent() Option {
+ excludeIDSet := map[int]bool{}
+ for _, s := range stack.All() {
+ excludeIDSet[s.ID()] = true
+ }
+ return addFilter(func(s stack.Stack) bool {
+ return excludeIDSet[s.ID()]
+ })
+}
+
+func maxSleep(d time.Duration) Option {
+ return optionFunc(func(opts *opts) {
+ opts.maxSleep = d
+ })
+}
+
+func addFilter(f func(stack.Stack) bool) Option {
+ return optionFunc(func(opts *opts) {
+ opts.filters = append(opts.filters, f)
+ })
+}
+
+func buildOpts(options ...Option) *opts {
+ opts := &opts{
+ maxRetries: _defaultRetries,
+ maxSleep: 100 * time.Millisecond,
+ }
+ opts.filters = append(opts.filters,
+ isTestStack,
+ isSyscallStack,
+ isStdLibStack,
+ isTraceStack,
+ )
+ for _, option := range options {
+ option.apply(opts)
+ }
+ return opts
+}
+
+func (vo *opts) filter(s stack.Stack) bool {
+ for _, filter := range vo.filters {
+ if filter(s) {
+ return true
+ }
+ }
+ return false
+}
+
+func (vo *opts) retry(i int) bool {
+ if i >= vo.maxRetries {
+ return false
+ }
+
+ d := time.Duration(int(time.Microsecond) << uint(i))
+ if d > vo.maxSleep {
+ d = vo.maxSleep
+ }
+ time.Sleep(d)
+ return true
+}
+
+// isTestStack is a default filter installed to automatically skip goroutines
+// that the testing package runs while the user's tests are running.
+func isTestStack(s stack.Stack) bool {
+ // Until go1.7, the main goroutine ran RunTests, which started
+ // the test in a separate goroutine and waited for that test goroutine
+ // to end by waiting on a channel.
+ // Since go1.7, a separate goroutine is started to wait for signals.
+ // T.Parallel is for parallel tests, which are blocked until all serial
+ // tests have run with T.Parallel at the top of the stack.
+ switch s.FirstFunction() {
+ case "testing.RunTests", "testing.(*T).Run", "testing.(*T).Parallel":
+ // In pre1.7 and post-1.7, background goroutines started by the testing
+ // package are blocked waiting on a channel.
+ return strings.HasPrefix(s.State(), "chan receive")
+ }
+ return false
+}
+
+func isSyscallStack(s stack.Stack) bool {
+ // Typically runs in the background when code uses CGo:
+ // https://github.com/golang/go/issues/16714
+ return s.FirstFunction() == "runtime.goexit" && strings.HasPrefix(s.State(), "syscall")
+}
+
+func isStdLibStack(s stack.Stack) bool {
+ // Importing os/signal starts a background goroutine.
+ // The name of the function at the top has changed between versions.
+ if f := s.FirstFunction(); f == "os/signal.signal_recv" || f == "os/signal.loop" {
+ return true
+ }
+
+ // Using signal.Notify will start a runtime goroutine.
+ return strings.Contains(s.Full(), "runtime.ensureSigM")
+}
+
+func isTraceStack(s stack.Stack) bool {
+ if f := s.FirstFunction(); f != "runtime.goparkunlock" {
+ return false
+ }
+
+ return strings.Contains(s.Full(), "runtime.ReadTrace")
+}
vendor/github.com/hashicorp/vault-plugin-auth-jwt/path_config.go+18 1
@@ -160,6 +160,23 @@ func (b *jwtAuthBackend) pathConfigRead(ctx context.Context, req *logical.Reques
return nil, nil
}
+ provider, err := NewProviderConfig(ctx, config, ProviderMap())
+ if err != nil {
+ return nil, err
+ }
+
+ // Omit sensitive keys from provider-specific config
+ providerConfig := make(map[string]interface{})
+ if provider != nil {
+ for k, v := range config.ProviderConfig {
+ providerConfig[k] = v
+ }
+
+ for _, k := range provider.SensitiveKeys() {
+ delete(providerConfig, k)
+ }
+ }
+
resp := &logical.Response{
Data: map[string]interface{}{
"oidc_discovery_url": config.OIDCDiscoveryURL,
@@ -173,7 +190,7 @@ func (b *jwtAuthBackend) pathConfigRead(ctx context.Context, req *logical.Reques
"jwks_url": config.JWKSURL,
"jwks_ca_pem": config.JWKSCAPEM,
"bound_issuer": config.BoundIssuer,
- "provider_config": config.ProviderConfig,
+ "provider_config": providerConfig,
"namespace_in_state": config.NamespaceInState,
},
}
vendor/github.com/hashicorp/vault-plugin-auth-jwt/provider_gsuite.go+45 34
@@ -6,26 +6,28 @@ import (
"errors"
"fmt"
"io/ioutil"
+ "os"
"github.com/mitchellh/mapstructure"
"golang.org/x/oauth2"
"golang.org/x/oauth2/google"
- "golang.org/x/oauth2/jwt"
admin "google.golang.org/api/admin/directory/v1"
"google.golang.org/api/option"
)
// GSuiteProvider provides G Suite-specific configuration and behavior.
type GSuiteProvider struct {
- config GSuiteProviderConfig // Configuration for the provider
- jwtConfig *jwt.Config // Google JWT configuration
- adminSvc *admin.Service // Google admin service
+ // Configuration for the provider
+ config GSuiteProviderConfig
+
+ // Google admin service
+ adminSvc *admin.Service
}
// GSuiteProviderConfig represents the configuration for a GSuiteProvider.
type GSuiteProviderConfig struct {
- // Path to a Google service account key file. Required.
- ServiceAccountFilePath string `mapstructure:"gsuite_service_account"`
+ // The path to or contents of a Google service account key file. Required.
+ ServiceAccount string `mapstructure:"gsuite_service_account"`
// Email address of a G Suite admin to impersonate. Required.
AdminImpersonateEmail string `mapstructure:"gsuite_admin_impersonate"`
@@ -41,9 +43,6 @@ type GSuiteProviderConfig struct {
// Comma-separated list of G Suite custom schemas to fetch as claims.
UserCustomSchemas string `mapstructure:"user_custom_schemas"`
-
- // JSON contents of a Google service account key file.
- serviceAccountKeyJSON []byte
}
// Initialize initializes the GSuiteProvider by validating and creating configuration.
@@ -54,23 +53,10 @@ func (g *GSuiteProvider) Initialize(ctx context.Context, jc *jwtConfig) error {
return err
}
- // Read the Google service account key file
- keyJSON, err := ioutil.ReadFile(config.ServiceAccountFilePath)
- if err != nil {
- return err
- }
- config.serviceAccountKeyJSON = keyJSON
-
- return g.initialize(ctx, config)
-}
-
-func (g *GSuiteProvider) initialize(ctx context.Context, config GSuiteProviderConfig) error {
- var err error
-
// Validate configuration
- if config.ServiceAccountFilePath == "" {
- return errors.New("'gsuite_service_account' must be set to the file path for a " +
- "service account key")
+ if config.ServiceAccount == "" {
+ return errors.New("'gsuite_service_account' must be either the path to or contents of " +
+ "a JSON service account key file")
}
if config.AdminImpersonateEmail == "" {
return errors.New("'gsuite_admin_impersonate' must be set to an email address of a " +
@@ -80,28 +66,47 @@ func (g *GSuiteProvider) initialize(ctx context.Context, config GSuiteProviderCo
return errors.New("'gsuite_recurse_max_depth' must be a positive integer")
}
- // Create the google JWT config from the service account key file
- if g.jwtConfig, err = google.JWTConfigFromJSON(config.serviceAccountKeyJSON,
- admin.AdminDirectoryGroupReadonlyScope, admin.AdminDirectoryUserReadonlyScope); err != nil {
- return err
+ // A file path or JSON string may be provided for the service account parameter.
+ // Check to see if a file exists at the given path, and if so, read its contents.
+ // Otherwise, assume the service account has been provided as a JSON string.
+ var err error
+ keyJSON := []byte(config.ServiceAccount)
+ if fileExists(config.ServiceAccount) {
+ keyJSON, err = ioutil.ReadFile(config.ServiceAccount)
+ if err != nil {
+ return err
+ }
}
- // Set the subject to impersonate and config
- g.jwtConfig.Subject = config.AdminImpersonateEmail
- g.config = config
+ // Set the requested scopes
+ scopes := []string{
+ admin.AdminDirectoryGroupReadonlyScope,
+ admin.AdminDirectoryUserReadonlyScope,
+ }
+
+ // Create the google JWT config from the service account
+ jwtConfig, err := google.JWTConfigFromJSON(keyJSON, scopes...)
+ if err != nil {
+ return fmt.Errorf("error parsing service account JSON: %w", err)
+ }
+
+ // Set the subject to impersonate
+ jwtConfig.Subject = config.AdminImpersonateEmail
// Create a new admin service for requests to Google admin APIs
- g.adminSvc, err = admin.NewService(ctx, option.WithHTTPClient(g.jwtConfig.Client(ctx)))
+ svc, err := admin.NewService(ctx, option.WithHTTPClient(jwtConfig.Client(ctx)))
if err != nil {
return err
}
+ g.adminSvc = svc
+ g.config = config
return nil
}
// SensitiveKeys returns keys that should be redacted when reading the config of this provider
func (g *GSuiteProvider) SensitiveKeys() []string {
- return []string{}
+ return []string{"gsuite_service_account"}
}
// FetchGroups fetches and returns groups from G Suite.
@@ -214,3 +219,9 @@ func (g *GSuiteProvider) getUserClaim(b *jwtAuthBackend, allClaims map[string]in
return userClaim, nil
}
+
+// fileExists returns true if a file exists at the given path.
+func fileExists(path string) bool {
+ fi, err := os.Stat(path)
+ return err == nil && fi != nil && !fi.IsDir()
+}
vendor/github.com/gocql/gocql/README.md+3 68
@@ -19,8 +19,8 @@ The following matrix shows the versions of Go and Cassandra that are tested with
Go/Cassandra | 2.1.x | 2.2.x | 3.x.x
-------------| -------| ------| ---------
-1.13 | yes | yes | yes
-1.14 | yes | yes | yes
+1.15 | yes | yes | yes
+1.16 | yes | yes | yes
Gocql has been tested in production against many different versions of Cassandra. Due to limits in our CI setup we only test against the latest 3 major releases, which coincide with the official support from the Apache project.
@@ -114,73 +114,7 @@ statement.
Example
-------
-```go
-/* Before you execute the program, Launch `cqlsh` and execute:
-create keyspace example with replication = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };
-create table example.tweet(timeline text, id UUID, text text, PRIMARY KEY(id));
-create index on example.tweet(timeline);
-*/
-package main
-
-import (
- "fmt"
- "log"
-
- "github.com/gocql/gocql"
-)
-
-func main() {
- // connect to the cluster
- cluster := gocql.NewCluster("192.168.1.1", "192.168.1.2", "192.168.1.3")
- cluster.Keyspace = "example"
- cluster.Consistency = gocql.Quorum
- session, _ := cluster.CreateSession()
- defer session.Close()
-
- // insert a tweet
- if err := session.Query(`INSERT INTO tweet (timeline, id, text) VALUES (?, ?, ?)`,
- "me", gocql.TimeUUID(), "hello world").Exec(); err != nil {
- log.Fatal(err)
- }
-
- var id gocql.UUID
- var text string
-
- /* Search for a specific set of records whose 'timeline' column matches
- * the value 'me'. The secondary index that we created earlier will be
- * used for optimizing the search */
- if err := session.Query(`SELECT id, text FROM tweet WHERE timeline = ? LIMIT 1`,
- "me").Consistency(gocql.One).Scan(&id, &text); err != nil {
- log.Fatal(err)
- }
- fmt.Println("Tweet:", id, text)
-
- // list all tweets
- iter := session.Query(`SELECT id, text FROM tweet WHERE timeline = ?`, "me").Iter()
- for iter.Scan(&id, &text) {
- fmt.Println("Tweet:", id, text)
- }
- if err := iter.Close(); err != nil {
- log.Fatal(err)
- }
-}
-```
-
-
-Authentication
-
-```go
-cluster := gocql.NewCluster("192.168.1.1", "192.168.1.2", "192.168.1.3")
-cluster.Authenticator = gocql.PasswordAuthenticator{
- Username: "user",
- Password: "password"
-}
-cluster.Keyspace = "example"
-cluster.Consistency = gocql.Quorum
-session, _ := cluster.CreateSession()
-defer session.Close()
-```
+See [package documentation](https://pkg.go.dev/github.com/gocql/gocql#pkg-examples).
Data Binding
------------
ui/tests/acceptance/secrets/backend/kv/secret-test.js+3 0
@@ -65,6 +65,7 @@ module('Acceptance | secrets/secret/create', function(hooks) {
await mountSecrets
.next()
.path(enginePath)
+ .toggleOptions()
.version(1)
.submit();
await listPage.create();
@@ -83,6 +84,7 @@ module('Acceptance | secrets/secret/create', function(hooks) {
await mountSecrets
.next()
.path(enginePath)
+ .toggleOptions()
.version(1)
.submit();
await listPage.create();
@@ -137,6 +139,7 @@ module('Acceptance | secrets/secret/create', function(hooks) {
await mountSecrets
.next()
.path(enginePath)
+ .toggleOptions()
.version(1)
.submit();
await listPage.create();
used with mTLS and/or an explicit CA cert. (#11252) (#11281)
vault/raft.go | 6 ++++++
1 file changed, 6 insertions(+)
changelog/10181.txt+3 0
@@ -0,0 +1,3 @@
+```release-note:bug
+storage/dynamodb: Handle throttled batch write requests by retrying, without which writes could be lost.
+```
sdk/logical/storage_view.go+1 3
@@ -42,9 +42,7 @@ func (s *StorageView) Get(ctx context.Context, key string) (*StorageEntry, error
if entry == nil {
return nil, nil
}
- if entry != nil {
- entry.Key = s.TruncateKey(entry.Key)
- }
+ entry.Key = s.TruncateKey(entry.Key)
return &StorageEntry{
Key: entry.Key,
vendor/github.com/hashicorp/vault/sdk/logical/storage_view.go+1 3
@@ -42,9 +42,7 @@ func (s *StorageView) Get(ctx context.Context, key string) (*StorageEntry, error
if entry == nil {
return nil, nil
}
- if entry != nil {
- entry.Key = s.TruncateKey(entry.Key)
- }
+ entry.Key = s.TruncateKey(entry.Key)
return &StorageEntry{
Key: entry.Key,
changelog/11294.txt+3 0
@@ -0,0 +1,3 @@
+```release-note:bug
+ui: fix issue where select-one option was not showing in secrets database role creation
+```
More files changed — see the full commit.

References