* upgrade go-jose library to v3
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
* chore: fix unnecessary import alias
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
* upgrade go-jose library to v2 in vault
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
---------
Signed-off-by: Mark Sagi-Kazar <mark.sagikazar@gmail.com>
* Add current_billing_period activity endpoint param
This commit introduces a new parameter: `current_billing_period`, which
can be used in lieu of `start_time` and `end_time` options.
GET ... /sys/internal/counters/activity?current_billing_period=true now
results in a response which contains the full billing period
information.
* changelog
* Update internal counters docs
Note the three overlapping scenarios discussed in the comments. In the
future, when this interface is more broadly supported, we should likely
add the interface directly to SystemView and implement it over the GRPC
interface, removing this nasty layering of already complex SystemView
implementations.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* including path parameters into stub Paths for enterprise-only endpoints
* Set Required to true for path parameters in enterprise-only path stubs
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* properly format go code
* re-adding initialization of Fields and Operations fields in the stubbed Path struct
---------
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* change testing password policy to be deterministic
* fix panic
* test password against rules
* improve error message
* make test password gen more random
* fix check on test password length
* Add stub ACME billing interfaces
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add initial implementation of client count
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly attribute to mount, namespace
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor adding entities of custom types
This begins to add custom types of events; presently these are counted
as non-entity tokens, but prefixed with a custom ClientID prefix.
In the future, this will be the basis for counting these events
separately (into separate buckets and separate storage segments).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor creation of ACME mounts
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test case for billing
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Better support managed key system view casting
Without an additional parameter, SystemView could be of a different
internal implementation type that cannot be directly casted to in OSS.
Use a separate parameter for the managed key system view to use instead.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor creation of mounts for enterprise
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Validate mounts in ACME billing tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use a hopefully unique separator for encoded identifiers
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use mount accesor, not path
Co-authored-by: miagilepner <mia.epner@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Rename AddEventToFragment->AddActivityToFragment
Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: miagilepner <mia.epner@hashicorp.com>
Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>
* Move seal barrier type field from Access to autoSeal struct.
Remove method Access.SetType(), which was only being used by a single test, and
which can use the name option of NewTestSeal() to specify the type.
* Change method signatures of Access to match those of Wrapper.
* Turn seal.Access struct into an interface.
* Tweak Access implementation.
Change `access` struct to have a field of type wrapping.Wrapper, rather than
extending it.
* Add method Seal.GetShamirWrapper().
Add method Seal.GetShamirWrapper() for use by code that need to perform
Shamir-specific operations.
When executing multi-stage, multi-namespace tests, stopping the ticker
multiple times (via closing the StopTicker channel) results in a panic.
Store whether or not we've stopped it once, and do not close it again.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* define ent paths in OSS codebase with common handler
* fixup! define ent paths in OSS codebase with common handler
* add missing path
* retain existing behaviour for replication/status path
* remove commented out path
The extra core setup is no longer needed in Vault Enterprise, and the
licensing code here has no effect here or in Vault Enterprise.
I pulled this commit into Vault Enterprise and it still compiled fine,
and all tests pass. (Though a few functions can be deleted there as
well after this is merged.)
When testing the Rollback Manager's one-time invocation in Enterprise,
it was noticed that due to the channel being closed, we'd always hit
this case and thus spam logs rather quickly with this message.
Switch to a boolean flip to log this once, as it is not executed in
parallel and thus doesn't need a sync.Once.
This only affected anyone calling the test core's
StopAutomaticRollbacks() helper.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* ensure we supply the node type when it's for a voter
* bumped autopilot version back to v0.2.0 and ran go mod tidy
* changed condition in knownservers and added some comments
* Export GetRaftBackend
* Updated tests for autopilot (related to dead server cleanup)
* Export Raft NewDelegate
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
When testing the rollback mechanism, there's two categories of tests
typically written:
1. Ones in which the rollback manager is entirely left alone, which
usually are a bit slower and less predictable. However, it is still
sufficient in many scenarios.
2. Ones in which the rollback manager is explicitly probed by tests
and "stepped" to achieve the next rollback.
Here, without a mechanism to fully disable the rollback manager's
periodic ticker (without affecting its ability to work!) we'll continue
to see races of the sort:
> --- FAIL: TestRevocationQueue (50.95s)
> panic: sync: WaitGroup is reused before previous Wait has returned [recovered]
> panic: sync: WaitGroup is reused before previous Wait has returned
This allows us to disable the ticker, returning control to the test
suite entirely.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* responses for rotate endpoints
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* added changelog
* add test for rotate config
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* update to use newer function
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* use new func
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
---------
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* added response struct for version-history
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add response struct for leader
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add response struct for ha-status
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add response struct for host-info
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add response struct for in-flight-req
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* added changelog
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* make fmt
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
---------
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* add response structures for /sys/wrapping endpoints
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* added changelog
* dynamic tests should be nil
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
---------
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add struct for /sys/tools/hash
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* added responses for /sys/tools paths
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add changelog
* verify respose structure for hash
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* verify respose structure for hash/random
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* use newer testing funct
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* use new test method
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
---------
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* fix data race on plugin reload
* add changelog
* add comment for posterity
* revert comment and return assignment in router.go
* rework plugin continue on error tests to use compilePlugin
* fix race condition on route entry
* add test for plugin reload and rollback race detection
* add go doc for test
* replace use of os.Unsetenv in test with t.Setenv and remove t.Parallel from test that rely on env being modified.
* experiment with using fromJSON function
* revert previous experiment
* including double quotes in the output value for the string ubuntu-latest
* use go run to launch gofumpt
* Fix cubbyhole and revocation for legacy service tokens
Legacy service tokens generated in Vault 1.10+ with env var
VAULT_DISABLE_SERVER_SIDE_CONSISTENT_TOKENS=true are not assigned
a cubbyhole ID. The implication is that cubbyhole/ cannot be
used, nor can the tokens be revoked.
This commit assigns a cubbyhole ID to these tokens and adds
a new test case to see that cubbyhole and revocation works correctly.
* add changelog
* add godoc to test cases
* added responses for sys/internal/ui/mounts
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* responses for internal paths
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* added changelog
* add schema validation for internal/ui/mounts
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add counters test
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* update test to use new method
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* use new method in TestSystemBackend_InternalUIMounts
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* :rage4: fixed test, diff between core.HandleRequest and backend.HandleRequest
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* test feature flags
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
---------
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
For plugin tests, we copy the test binary. On macOS, if the
destination binary already exists, then copying over it will result
in an invalid signature.
The easiest workaround is to delete the file before copying.
* added in the missing test cases to validate response structures
* added changelog file
* remove unneeded changelog file
* removed comment to update when indentity/entity is implemented
---------
Co-authored-by: lursu <leland.ursu@hashicorp.com>
The [WebSockets spec](https://www.rfc-editor.org/rfc/rfc6455) states
that text messages must be valid UTF-8 encoded strings, which protobuf
messages virtually never are. This now correctly sends the protobuf events
as binary messages.
We change the format to correspond to CloudEvents, as originally intended,
and remove a redundant timestamp and newline.
We also bump the eventlogger to fix a race condition that this code triggers.
* test/plugin: test external db plugin
* use test helper to get cluster and plugins
* create test helper to create a vault admin user
* add step to revoke lease
* make tests parallel and add reload test
* use more descriptive name for test group; check response
* add capabilities
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* added change log
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add test
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* use nil for dynamic fields
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
---------
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* added responses to /sys/auth/.../tune
* add response structure for auth/...
* added changelog
* Update vault/logical_system_paths.go
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* its TypeString
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* use nil for dynamic fields
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* test auth endpoint schema
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* kicking off ci
---------
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* added response objects to all of the endpoints laid out by the ticket linked
* added changelog file and updated based on review
* added the required bool to the correct fields
* Update vault/logical_system_paths.go
Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Update vault/logical_system_paths.go
Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Update vault/logical_system_paths.go
Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Update vault/logical_system_paths.go
Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Update vault/logical_system_paths.go
Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Update vault/logical_system_paths.go
Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Update vault/logical_system_paths.go
Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* updated based on review
* Update vault/logical_system_paths.go
Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Update vault/logical_system_paths.go
Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* updated based on review and added test cases for validating response structures
* fix copy pasta issues breaking tests
* Update vault/logical_system_paths.go
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* fix test failures
* fixed issue with refrencing the wrong req var name
* fixed another test case and double checked the rest
* updated based on review
* updated in all locations
* Update vault/logical_system_paths.go
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* Update vault/logical_system_paths.go
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* fixed my brain fart
* Update vault/logical_system_paths.go
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* address fmt error
---------
Co-authored-by: lursu <leland.ursu@hashicorp.com>
Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* add RequestResponseCallback to core/options
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* pass in router and apply function on requests
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add callback
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* cleanup
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* Update vault/core.go
* bad typo...
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* use pvt interface, can't downcast to child struct
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* finer grained errors
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* trim path for backend
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* remove entire mount point instead of just the first part of url
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Update vault/testing.go
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* add doc string
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* update docstring
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* reformat
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* added changelog
---------
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
When subscriptions are too slow to accept messages on their channels,
then we should remove them from the fanout so that we don't have
dead subscriptions using up resources.
In addition, when a subscription is explicitly canceled, we should
also clean up after it remove the corresponding pipeline.
We also add a new metrics, `events.subscriptions`, to keep track
of the number of active subscriptions. This also helps us test.
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
This checks the request against the `read` permission for
`sys/events/subscribe/{eventType}` on the initial subscribe.
Future work includes moving this to its own verb (`subscribe`)
and periodically rechecking the request.
Tested locally by minting a token with the wrong permissions
and verifying that they are rejected as expected, and that
they work if the policy is adjusted to `sys/event/subscribe/*`
(or the specific topic name) with `read` permissions.
I had to change the `core.checkToken()` to be publicly accessible,
as it seems like the easiest way to check the token on the
`logical.Request` against all relevant policies, but without
going into all of the complex logic further in `handleLogical()`.
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Also updates the event receieved to include a timestamp.
Websockets support both JSON and protobuf binary formats.
This can be used by either `wscat` or the new
`vault events subscribe`:
e.g.,
```sh
$ wscat -H "X-Vault-Token: $(vault print token)" --connect ws://127.0.0.1:8200/v1/sys/events/subscribe/abc?json=true
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```
and
```sh
$ vault events subscribe abc
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
* Add a stronger warning about the usage of recovery keys
* Update website/content/docs/concepts/seal.mdx
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Keep the mitigation text in the warning box
---------
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* test/plugin: refactor compilePlugin for reuse
- move compilePlugin to helper package
- make NewTestCluster use compilePlugin
* do not overwrite plugin directory in CoreConfig if set
* fix getting plugin directory path for go build
This isn't perfect for sure, but it's solidifying and becoming a useful
base to work off.
This routes events sent from auth and secrets plugins to the main
`EventBus` in the Vault Core. Events sent from plugins are automatically
tagged with the namespace and plugin information associated with them.
* Revert "Don't execute the seal recovery tests on ENT. (#18841)"
This reverts commit 990d3bacc2.
* Revert "Add the ability to unseal using recovery keys via an explicit seal option. (#18683)"
This reverts commit 2ffe49aab0.
* Move some test helper stuff from the vault package to a new helper/testhelpers/corehelpers package. Consolidate on a single "noop audit" implementation.
* Regexp metacharacter `.` should be escaped when used literally
The paths including `/.well-known/` in the Vault API could currently
technically be invoked with any random character in place of the dot.
* Replace implementation of OpenAPI path translator with regexp AST-based one
* Add changelog
* Typo fix from PR review - thanks!
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* Add comment based on review feedback
* Change style of error handling as suggested in code review
* Make a further tweak to the handling of the error case
* Add more tests, testing cases which fail with the previous implementation
* Resolve issue with a test, and improve comment
---------
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* The verify-sign command in it's cleanest existing form.
* Working state
* Updates to proper verification syntax
Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
* make fmt
* Git CI caught some stuff.
* Base functionality.
* make fmt; changelog
* pki issue command.
* Make fmt. Changelog.
* Error Handling Is Almost A Tutorial
* What I thought empty issuers response fix would be.
* Some tests
* PR-review updates.
* make fmt.
* Fix null response data for listing empty issuers causing a crash.
* Update command/pki_list_children_command.go
Fix double specifier
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Add test for pki_list_children.
* Fix tests.
* Update descriptions for correctness based on PR reviews.
* make fmt.
* Updates based on PR feedback.
* Allow multiple arguements (space separated)
* Remove bad merge-thing.
* White-space hell fix change.
* Tests, and return information for issue ca
* Fix make fmt error introduced here: https://github.com/hashicorp/vault/pull/18876
* Update command/pki_issue_intermediate.go
Puncutation.
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove smart quotes for standard quotes.
* More information as part of the help text.
* Better help text.
* Add missing "/" into error message.
---------
Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Add timeout functionality to inmem
* Update vault/cluster/inmem_layer.go
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Add comment about forceTimeout
* Add comment about time
---------
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* wip
* wip
* Got it 'working', but not happy about cleanliness yet
* Switch to a dedicated defaultSeal with recovery keys
This is simpler than trying to hijack SealAccess as before. Instead, if the operator
has requested recovery unseal mode (via a flag in the seal stanza), we new up a shamir
seal with the recovery unseal key path instead of the auto seal. Then everything proceeds
as if you had a shamir seal to begin with.
* Handle recovery rekeying
* changelog
* Revert go.mod redirect
* revert multi-blob info
* Dumb nil unmarshal target
* More comments
* Update vault/seal.go
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Update changelog/18683.txt
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* pr feedback
* Fix recovery rekey, which needs to fetch root keys and restore them under the new recovery split
* Better comment on recovery seal during adjustSealMigration
* Make it possible to migrate from an auto-seal in recovery mode to shamir
* Fix sealMigrated to account for a recovery seal
* comments
* Update changelog/18683.txt
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Address PR feedback
* Refactor duplicated migration code into helpers, using UnsealRecoveryKey/RecoveryKey where appropriate
* Don't shortcut the reast of seal migration
* get rid of redundant transit server cleanup
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* named MFA method configurations
* fix a test
* CL
* fix an issue with same config name different ID and add a test
* feedback
* feedback on test
* consistent use of passcode for all MFA methods (#18611)
* make use of passcode factor consistent for all MFA types
* improved type for MFA factors
* add method name to login CLI
* minor refactoring
* only accept MFA method name with its namespace path in the login request MFA header
* fix a bug
* fixing an ErrorOrNil return value
* more informative error message
* Apply suggestions from code review
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* feedback
* test refactor a bit
* adding godoc for a test
* feedback
* remove sanitize method name
* guard a possbile nil ref
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* return error for meta auth and mount listing if sealed
* some logging changes
* some more logging changes
* add panic recovery
* use ErrInternalError
* Add global, cross-cluster revocation queue to PKI
This adds a global, cross-cluster replicated revocation queue, allowing
operators to revoke certificates by serial number across any cluster. We
don't support revoking with private key (PoP) in the initial
implementation.
In particular, building on the PBPWF work, we add a special storage
location for handling non-local revocations which gets replicated up to
the active, primary cluster node and back down to all secondary PR
clusters. These then check the pending revocation entry and revoke the
serial locally if it exists, writing a cross-cluster confirmation entry.
Listing capabilities are present under pki/certs/revocation-queue,
allowing operators to see which certs are present. However, a future
improvement to the tidy subsystem will allow automatic cleanup of stale
entries.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow tidying revocation queue entries
No manual operator control of revocation queue entries are allowed.
However, entries are stored with their request time, allowing tidy to,
after a suitable safety buffer, remove these unconfirmed and presumably
invalid requests.
Notably, when a cluster goes offline, it will be unable to process
cross-cluster revocations for certificates it holds. If tidy runs,
potentially valid revocations may be removed. However, it is up to the
administrator to ensure the tidy window is sufficiently long that any
required maintenance is done (or, prior to maintenance when an issue is
first noticed, tidy is temporarily disabled).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Only allow enabling global revocation queue on Vault Enterprise
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use a locking queue to handle revocation requests
This queue attempts to guarantee that PKI's invalidateFunc won't have
to wait long to execute: by locking only around access to the queue
proper, and internally using a list, we minimize the time spent locked,
waiting for queue accesses.
Previously, we held a lock during tidy and processing that would've
prevented us from processing invalidateFunc calls.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* use_global_queue->cross_cluster_revocation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Grab revocation storage lock when processing queue
We need to grab the storage lock as we'll actively be revoking new
certificates in the revocation queue. This ensures nobody else is
competing for storage access, across periodic funcs, new revocations,
and tidy operations.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix expected tidy status test
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow probing RollbackManager directly in tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Address review feedback on revocationQueue
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add more cancel checks, fix starting manual tidy
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add WriteForwardedStorage to sdk's plugin, logical in OSS
This should allow backends to specify paths to forward write
(storage.Put(...) and storage.Delete(...)) operations for.
Notably, these semantics are subject to change and shouldn't yet be
relied on.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Collect paths for write forwarding in OSS
This adds a path manager to Core, allowing tracking across all Vault
versions of paths which could use write forwarding if available. In
particular, even on OSS offerings, we'll need to template {{clusterId}}
into the paths, in the event of later upgrading to Enterprise. If we
didn't, we'd end up writing paths which will no longer be accessible
post-migration, due to write forwarding now replacing the sentinel with
the actual cluster identifier.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add forwarded writer implementation to OSS
Here, for paths given to us, we determine if we need to do cluster
translation and perform local writing. This is the OSS variant.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Wire up mount-specific request forwarding in OSS
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Clarify that state lock needs to be held to call HAState in OSS
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Move cluster sentinel constant to sdk/logical
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Expose ClusterID to Plugins via SystemView
This will let plugins learn what the Cluster's ID is, without having to
resort to hacks like writing a random string to its cluster-prefixed
namespace and then reading it once it has replicated.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add GRPC ClusterID implementation
For any external plugins which wish to use it.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* added audit-hash operations
* more audit paths
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* added audit fields
* add changelog file
* dynamic fields should be nil
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* start to add test helper
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add tests for /sys/audit openapi paths
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <anton.averchenkov@hashicorp.com>
For example, using:
```sh
vault server -dev -experiment events.beta1
```
Tested by checking that the events were enabled and disabled
when the `-experiment events.beta1` flag was present and absent.
Also added a small fix to pass the `hclog.Logger` in now so that
the logging hierarchy and levels are respected.
* OpenAPI `generic_mount_paths` follow-up
An incremental improvement within larger context discussed in #18560.
* Following the revert in #18617, re-introduce the change from
`{mountPath}` to `{<path-of-mount>_mount_path}`; this is needed, as
otherwise paths from multiple plugins would clash - e.g. almost every
auth method would provide a conflicting definition for
`auth/{mountPath}/login`, and the last one written into the map would
win.
* Move the half of the functionality that was in `sdk/framework/` to
`vault/logical_system.go` with the rest; this is needed, as
`sdk/framework/` gets compiled in to externally built plugins, and
therefore there may be version skew between it and the Vault main
code. Implementing the `generic_mount_paths` feature entirely on one
side of this boundary frees us from problems caused by this.
* Update the special exception that recognizes `system` and `identity`
as singleton mounts to also include the other two singleton mounts,
`cubbyhole` and `auth/token`.
* Include a comment that documents to restricted circumstances in which
the `generic_mount_paths` option makes sense to use:
// Note that for this to actually be useful, you have to be using it with
// a Vault instance in which you have mounted one of each secrets engine
// and auth method of types you are interested in, at paths which identify
// their type, and for the KV secrets engine you will probably want to
// mount separate kv-v1 and kv-v2 mounts to include the documentation for
// each of those APIs.
* Fix tests
Also remove comment "// TODO update after kv repo update" which was
added 4 years ago in #5687 - the implied update has not happened.
* Add changelog
* Update 18663.txt
