derivepassphrase.exporter ¶

Foreign configuration exporter for derivepassphrase.

NotAVaultConfigError ¶

NotAVaultConfigError(
    path: str | bytes | PathLike, format: str | None = None
)

Bases: ValueError

The path does not hold a format-type vault configuration.

ExportVaultConfigDataFunction ¶

Bases: Protocol

Typing protocol for vault config data export handlers.

call ¶

__call__(
    path: str | bytes | PathLike | None = None,
    key: str | Buffer | None = None,
    *,
    format: str
) -> Any

Export the full vault-native configuration stored in path.

Parameters:

Name	Type	Description	Default
`path`	`str \| bytes \| PathLike \| None`	The path to the vault configuration file or directory. If not given, then query `get_vault_path` for the correct value.	`None`
`key`	`str \| Buffer \| None`	Encryption key/password for the configuration file or directory, usually the username, or passed via the `VAULT_KEY` environment variable. If not given, then query `get_vault_key` for the value.	`None`
`format`	`str`	The format to attempt parsing as. Must be `v0.2`, `v0.3` or `storeroom`.	required

Returns:

Type	Description
`Any`	The vault configuration, as recorded in the configuration file. This may or may not be a valid configuration according to `vault` or `derivepassphrase`.

Raises:

Type	Description
`IsADirectoryError`	The requested format requires a configuration file, but `path` points to a directory instead.
`NotADirectoryError`	The requested format requires a configuration directory, but `path` points to something else instead.
`OSError`	There was an OS error while accessing the configuration file/directory.
`RuntimeError`	Something went wrong during data collection, e.g. we encountered unsupported or corrupted data in the configuration file/directory.
`JSONDecodeError`	An internal JSON data structure failed to parse from disk. The configuration file/directory is probably corrupted.
`NotAVaultConfigError`	The file/directory contents are not in the claimed configuration format.
`ValueError`	The requested format is invalid.
`ModuleNotFoundError`	The requested format requires support code, which failed to load because of missing Python libraries.

get_vault_key ¶

get_vault_key() -> bytes

Automatically determine the vault(1) master key/password.

Query the VAULT_KEY, LOGNAME, USER and USERNAME environment variables, in that order. This is the same algorithm that vault uses.

Returns:

Type	Description
`bytes`	The master key/password. This is generally used as input to a key-derivation function to determine the actual encryption and signing keys for the vault configuration.

Raises:

Type	Description
`KeyError`	We cannot find any of the named environment variables. Please set `VAULT_KEY` manually to the desired value.

get_vault_path ¶

get_vault_path() -> Path

Automatically determine the vault(1) configuration path.

Query the VAULT_PATH environment variable, or default to ~/.vault. This is the same algorithm that vault uses. If not absolute, then VAULT_PATH is relative to the home directory.

Returns:

Type	Description
`Path`	The vault configuration path. Depending on the vault version, this may be a file or a directory.

Raises:

Type	Description
`RuntimeError`	We cannot determine the home directory. Please set `HOME` manually to the correct value.

find_vault_config_data_handlers ¶

find_vault_config_data_handlers() -> None

Find all export handlers for vault config data.

(This function is idempotent.)

Raises:

Type	Description
`ModuleNotFoundError`	A required module was not found.

export_vault_config_data ¶

export_vault_config_data(
    path: str | bytes | PathLike | None = None,
    key: str | Buffer | None = None,
    *,
    format: str
) -> Any

Export the full vault-native configuration stored in path.

See ExportVaultConfigDataFunction for an explanation of the call signature, and the exceptions to expect.

derivepassphrase.exporter.storeroom ¶

Exporter for the vault “storeroom” configuration format.

The “storeroom” format is the experimental format used in alpha and beta versions of vault beyond v0.3.0. The configuration is stored as a separate directory, which acts like a hash table (i.e. has named slots) and provides an impure quasi-filesystem interface. Each hash table entry is separately encrypted and authenticated. James Coglan designed this format to avoid concurrent write issues when updating or synchronizing the vault configuration with e.g. a cloud service.

The public interface is the export_storeroom_data function. Multiple non-public functions are additionally documented here for didactical and educational reasons, but they are not part of the module API, are subject to change without notice (including removal), and should not be used or relied on.

export_storeroom_data ¶

export_storeroom_data(
    path: str | bytes | PathLike | None = None,
    key: str | Buffer | None = None,
    *,
    format: str = "storeroom"
) -> dict[str, Any]

Export the full configuration stored in the storeroom.

See exporter.ExportVaultConfigDataFunction for an explanation of the call signature, and the exceptions to expect.

Other Parameters:

Name	Type	Description
`format`	`str`	The only supported format is `storeroom`.

_derive_master_keys_keys ¶

_derive_master_keys_keys(
    password: str | Buffer, iterations: int
) -> StoreroomKeyPair

Derive encryption and signing keys for the master keys data.

The master password is run through a key derivation function to obtain a 64-byte string, which is then split to yield two 32-byte keys. The key derivation function is PBKDF2, using HMAC-SHA1 and salted with the storeroom master keys UUID.

Parameters:

Name	Type	Description	Default
`password`	`str \| Buffer`	A master password for the storeroom instance. Usually read from the `VAULT_KEY` environment variable, otherwise defaults to the username.	required
`iterations`	`int`	A count of rounds for the underlying key derivation function. Usually stored as a setting next to the encrypted master keys data.	required

Returns:

Type	Description
`StoreroomKeyPair`	A 2-tuple of keys, the encryption key and the signing key, to decrypt and verify the master keys data with.

Warning

Non-public function, provided for didactical and educational purposes only. Subject to change without notice, including removal.

_decrypt_master_keys_data ¶

_decrypt_master_keys_data(
    data: Buffer, keys: StoreroomKeyPair
) -> StoreroomMasterKeys

Decrypt the master keys data.

The master keys data contains:

a 16-byte IV,
a 96-byte AES256-CBC-encrypted payload, plus 16 further bytes of PKCS7 padding, and
a 32-byte MAC of the preceding 128 bytes.

The decrypted payload itself consists of three 32-byte keys: the hashing, encryption and signing keys, in that order.

The encrypted payload is encrypted with the encryption key, and the MAC is created based on the signing key. As per standard cryptographic procedure, the MAC can be verified before attempting to decrypt the payload.

Because the payload size is both fixed and a multiple of the cipher blocksize, in this case, the PKCS7 padding always is b'\x10' * 16.

Parameters:

Name	Type	Description	Default
`data`	`Buffer`	The encrypted master keys data.	required
`keys`	`StoreroomKeyPair`	The encryption and signing keys for the master keys data. These should have previously been derived via the `_derive_master_keys_keys` function.	required

Returns:

Type	Description
`StoreroomMasterKeys`	The master encryption, signing and hashing keys.

Raises:

Type	Description
`InvalidSignature`	The data does not contain a valid signature under the given key.
`ValueError`	The format is invalid, in a non-cryptographic way. (For example, it contains an unsupported version marker, or unexpected extra contents, or invalid padding.)

Warning

Non-public function, provided for didactical and educational purposes only. Subject to change without notice, including removal.

_decrypt_session_keys ¶

_decrypt_session_keys(
    data: Buffer, master_keys: StoreroomMasterKeys
) -> StoreroomKeyPair

Decrypt the bucket item’s session keys.

The bucket item’s session keys are single-use keys for encrypting and signing a single item in the storage bucket. The encrypted session key data consists of:

a 16-byte IV,
a 64-byte AES256-CBC-encrypted payload, plus 16 further bytes of PKCS7 padding, and
a 32-byte MAC of the preceding 96 bytes.

The encrypted payload is encrypted with the master encryption key, and the MAC is created with the master signing key. As per standard cryptographic procedure, the MAC can be verified before attempting to decrypt the payload.

Because the payload size is both fixed and a multiple of the cipher blocksize, in this case, the PKCS7 padding always is b'\x10' * 16.

Parameters:

Name	Type	Description	Default
`data`	`Buffer`	The encrypted bucket item session key data.	required
`master_keys`	`StoreroomMasterKeys`	The master keys. Presumably these have previously been obtained via the `_decrypt_master_keys_data` function.	required

Returns:

Type	Description
`StoreroomKeyPair`	The bucket item’s encryption and signing keys.

Raises:

Type	Description
`InvalidSignature`	The data does not contain a valid signature under the given key.
`ValueError`	The format is invalid, in a non-cryptographic way. (For example, it contains an unsupported version marker, or unexpected extra contents, or invalid padding.)

Warning

Non-public function, provided for didactical and educational purposes only. Subject to change without notice, including removal.

_decrypt_contents ¶

_decrypt_contents(
    data: Buffer, session_keys: StoreroomKeyPair
) -> Buffer

Decrypt the bucket item’s contents.

The data consists of:

a 16-byte IV,
a variable-sized AES256-CBC-encrypted payload (using PKCS7 padding on the inside), and
a 32-byte MAC of the preceding bytes.

The encrypted payload is encrypted with the bucket item’s session encryption key, and the MAC is created with the bucket item’s session signing key. As per standard cryptographic procedure, the MAC can be verified before attempting to decrypt the payload.

Parameters:

Name	Type	Description	Default
`data`	`Buffer`	The encrypted bucket item payload data.	required
`session_keys`	`StoreroomKeyPair`	The bucket item’s session keys. Presumably these have previously been obtained via the `_decrypt_session_keys` function.	required

Returns:

Type	Description
`Buffer`	The bucket item’s payload.

Raises:

Type	Description
`InvalidSignature`	The data does not contain a valid signature under the given key.
`ValueError`	The format is invalid, in a non-cryptographic way. (For example, it contains an unsupported version marker, or unexpected extra contents, or invalid padding.)

Warning

Non-public function, provided for didactical and educational purposes only. Subject to change without notice, including removal.

_decrypt_bucket_item ¶

_decrypt_bucket_item(
    bucket_item: Buffer, master_keys: StoreroomMasterKeys
) -> Buffer

Decrypt a bucket item.

Parameters:

Name	Type	Description	Default
`bucket_item`	`Buffer`	The encrypted bucket item.	required
`master_keys`	`StoreroomMasterKeys`	The master keys. Presumably these have previously been obtained via the `_decrypt_master_keys_data` function.	required

Returns:

Type	Description
`Buffer`	The decrypted bucket item.

Raises:

Type	Description
`InvalidSignature`	The data does not contain a valid signature under the given key.
`ValueError`	The format is invalid, in a non-cryptographic way. (For example, it contains an unsupported version marker, or unexpected extra contents, or invalid padding.)

Warning

Non-public function, provided for didactical and educational purposes only. Subject to change without notice, including removal.

_decrypt_bucket_file ¶

_decrypt_bucket_file(
    filename: str | bytes | PathLike,
    master_keys: StoreroomMasterKeys,
    *,
    root_dir: str | bytes | PathLike = "."
) -> Iterator[Buffer]

Decrypt a complete bucket.

Parameters:

Name	Type	Description	Default
`filename`	`str \| bytes \| PathLike`	The bucket file’s filename.	required
`master_keys`	`StoreroomMasterKeys`	The master keys. Presumably these have previously been obtained via the `_decrypt_master_keys_data` function.	required
`root_dir`	`str \| bytes \| PathLike`	The root directory of the data store. The filename is interpreted relatively to this directory.	`'.'`

Yields:

Type	Description
`Buffer`	A decrypted bucket item.

Raises:

Type	Description
`InvalidSignature`	The data does not contain a valid signature under the given key.
`ValueError`	The format is invalid, in a non-cryptographic way. (For example, it contains an unsupported version marker, or unexpected extra contents, or invalid padding.)

Warning

Non-public function, provided for didactical and educational purposes only. Subject to change without notice, including removal.

derivepassphrase.exporter.vault_native ¶

Exporter for the vault native configuration format (v0.2 or v0.3).

The vault native formats are the configuration formats used by vault v0.2 and v0.3. The configuration is stored as a single encrypted file, which is encrypted and authenticated. v0.2 and v0.3 differ in some details concerning key derivation and expected format of internal structures, so they are not compatible. v0.2 additionally contains cryptographic weaknesses (API misuse of a key derivation function, and a low-entropy method of generating initialization vectors for CBC block encryption mode) and should thus be avoided if possible.

The public interface is the export_vault_native_data function. Multiple non-public classes are additionally documented here for didactical and educational reasons, but they are not part of the module API, are subject to change without notice (including removal), and should not be used or relied on.

VaultNativeConfigParser ¶

VaultNativeConfigParser(
    contents: Buffer, password: str | Buffer
)

Bases: ABC

A base parser for vault’s native configuration format.

Certain details are specific to the respective vault versions, and are abstracted out. This class by itself is not instantiable because of this.

Parameters:

Name	Type	Description	Default
`contents`	`Buffer`	The binary contents of the encrypted configuration file. Note: On disk, these are usually stored in base64-encoded form, not in the “raw” form as needed here.	required
`password`	`str \| Buffer`	The vault master key/master passphrase the file is encrypted with. Must be non-empty. See `exporter.get_vault_key` for details. If this is a text string, then the UTF-8 encoding of the string is used as the binary password.	required

Raises:

Type	Description
`ValueError`	The password must not be empty.

Warning

Non-public class, provided for didactical and educational purposes only. Subject to change without notice, including removal.

call ¶

__call__() -> Any

Return the decrypted and parsed vault configuration.

Raises:

Type	Description
`InvalidSignature`	The encrypted configuration does not contain a valid signature.
`ValueError`	The format is invalid, in a non-cryptographic way. (For example, it contains an unsupported version marker, or unexpected extra contents, or invalid padding.)

_pbkdf2 `staticmethod` ¶

_pbkdf2(
    password: str | Buffer, key_size: int, iterations: int
) -> bytes

Generate a key from a password.

Uses PBKDF2 with HMAC-SHA1, with vault.Vault.UUID as a fixed salt value.

Parameters:

Name	Type	Description	Default
`password`	`str \| Buffer`	The password from which to derive the key.	required
`key_size`	`int`	The size of the output string. The effective key size (in bytes) is thus half of this output string size.	required
`iterations`	`int`	The PBKDF2 iteration count.	required

Returns:

Type	Description
`bytes`	The PBKDF2-derived key, encoded as a lowercase ASCII hexadecimal string.

Insecure use of cryptography

This function is insecure because it uses a fixed salt value, which is not secure against rainbow tables. It is further difficult to use because the effective key size is only half as large as the “size” parameter (output string size). Finally, though the use of SHA-1 in HMAC per se is not known to be insecure, SHA-1 is known not to be collision-resistant.

_parse_contents ¶

_parse_contents() -> None

Parse the contents into IV, payload and MAC.

This operates on, and sets, multiple internal attributes of the parser.

Raises:

Type	Description
`ValueError`	The configuration file contents are clearly truncated.

_derive_keys ¶

_derive_keys() -> None

Derive the signing and encryption keys.

This is a bookkeeping method. The actual work is done in _generate_keys.

_generate_keys `abstractmethod` ¶

_generate_keys() -> None

Derive the signing and encryption keys, and set the key sizes.

Subclasses must override this, as the derivation system is version-specific. The default implementation raises an error.

Raises:

Type	Description
`AssertionError`	There is no default implementation.

_check_signature ¶

_check_signature() -> None

Check for a valid MAC on the encrypted vault configuration.

The MAC uses HMAC-SHA1, and thus is 32 bytes long, before encoding.

Raises:

Type	Description
`ValueError`	The MAC is invalid.

_hmac_input `abstractmethod` ¶

_hmac_input() -> bytes

Return the input the MAC is supposed to verify.

Subclasses must override this, as the MAC-attested data is version-specific. The default implementation raises an error.

Raises:

Type	Description
`AssertionError`	There is no default implementation.

_decrypt_payload ¶

_decrypt_payload() -> Any

Return the decrypted vault configuration.

Requires _parse_contents and _derive_keys to have run, and relies on _check_signature for tampering detection.

_make_decryptor `abstractmethod` ¶

_make_decryptor() -> CipherContext

Return the cipher context object used for decryption.

Subclasses must override this, as the cipher setup is version-specific. The default implementation raises an error.

Raises:

Type	Description
`AssertionError`	There is no default implementation.

VaultNativeV03ConfigParser ¶

VaultNativeV03ConfigParser(*args: Any, **kwargs: Any)

Bases: VaultNativeConfigParser

A parser for vault’s native configuration format (v0.3).

This is the modern, pre-storeroom configuration format.

Warning

Non-public class, provided for didactical and educational purposes only. Subject to change without notice, including removal.

KEY_SIZE `class-attribute` `instance-attribute` ¶

KEY_SIZE = 32

Key size for both the encryption and the signing key, including the encoding as a hexadecimal string. (The effective cryptographic strength is half of this value.)

_generate_keys ¶

_generate_keys() -> None

Derive the signing and encryption keys, and set the key sizes.

Version 0.3 vault configurations use a constant key size; see KEY_SIZE. The encryption and signing keys differ in how many rounds of PBKDF2 they use (100 and 200, respectively).

Insecure use of cryptography

This function makes use of the insecure function VaultNativeConfigParser._pbkdf2, without any attempts at mitigating its insecurity. It further uses _pbkdf2 with the low iteration count of 100 and 200 rounds, which is drastically insufficient to defend against password guessing attacks using GPUs or ASICs. We provide this function for the purpose of interoperability with existing vault installations. Do not rely on this system to keep your vault configuration secure against access by even moderately determined attackers!

_hmac_input ¶

_hmac_input() -> bytes

Return the input the MAC is supposed to verify.

This includes hexadecimal encoding of the message payload.

_make_decryptor ¶

_make_decryptor() -> CipherContext

Return the cipher context object used for decryption.

This is a standard AES256-CBC cipher context using the previously derived encryption key and the IV declared in the (MAC-verified) message payload.

VaultNativeV02ConfigParser ¶

VaultNativeV02ConfigParser(*args: Any, **kwargs: Any)

Bases: VaultNativeConfigParser

A parser for vault’s native configuration format (v0.2).

This is the classic configuration format. Compared to v0.3, it contains an (accidental) API misuse for the generation of the master keys, a low-entropy method of generating initialization vectors for the AES-CBC encryption step, and extra layers of base64 encoding. Because of these significantly weakened confidentiality guarantees, v0.2 configurations should be upgraded to at least v0.3 as soon as possible.

Warning

Non-public class, provided for didactical and educational purposes only. Subject to change without notice, including removal.

_parse_contents ¶

_parse_contents() -> None

Parse the contents into IV, payload and MAC.

Like the base class implementation, this operates on, and sets, multiple internal attributes of the parser. In version 0.2 vault configurations, the payload is encoded in base64 and the message tag (MAC) is encoded in hexadecimal, so unlike the base class implementation, we additionally decode the payload and the MAC.

Raises:

Type	Description
`ValueError`	The configuration file contents are clearly truncated, or the payload or the message tag cannot be decoded properly.

_generate_keys ¶

_generate_keys() -> None

Derive the signing and encryption keys, and set the key sizes.

Version 0.2 vault configurations use 8-byte encryption keys and 16-byte signing keys, including the hexadecimal encoding. They both use 16 rounds of PBKDF2. This is due to an oversight in vault, where the author mistakenly supplied the intended iteration count as the key size, and the key size as the iteration count.

Insecure use of cryptography

This function makes use of the insecure function VaultNativeConfigParser._pbkdf2, without any attempts at mitigating its insecurity. It further uses _pbkdf2 with the low iteration count of 16 rounds, which is drastically insufficient to defend against password guessing attacks using GPUs or ASICs, and generates the encryption key as a truncation of the signing key. We provide this function for the purpose of interoperability with existing vault installations. Do not rely on this system to keep your vault configuration secure against access by even moderately determined attackers!

_hmac_input ¶

_hmac_input() -> bytes

Return the input the MAC is supposed to verify.

This includes hexadecimal encoding of the message payload.

_evp_bytestokey_md5_one_iteration_no_salt `staticmethod` ¶

_evp_bytestokey_md5_one_iteration_no_salt(
    data: bytes, key_size: int, iv_size: int
) -> tuple[bytes, bytes]

Reimplement OpenSSL’s EVP_BytesToKey with fixed parameters.

EVP_BytesToKey in general is a key derivation function, i.e., a function that derives key material from an input byte string. EVP_BytesToKey conceptually splits the derived key material into an encryption key and an initialization vector (IV).

Algorithm description

EVP_BytesToKey takes an input byte string, two output size (encryption key size and IV size), a message digest function, a salt value and an iteration count. The derived key material is calculated in blocks, each of which is the output of (iterated application of) the message digest function. The input to the message digest function is the concatenation of the previous block (if any) with the input byte string and the salt value (if any):

data = block_input = b''.join([previous_block, input_string, salt])
for i in range(iteration_count):
    data = message_digest(data)
block = data

We use as many blocks as are necessary to cover the total output byte string size. The first few bytes (dictated by the encryption key size) form the encryption key, the other bytes (dictated by the IV size) form the IV.

We implement exactly the subset of EVP_BytesToKey that the Node.js crypto library (v21 series and older) uses in its implementation of crypto.createCipher("aes256", password). Specifically, the message digest function is fixed to MD5, the salt is always empty, and the iteration count is fixed at one.

Returns:

Type	Description
`tuple[bytes, bytes]`	A 2-tuple containing the derived encryption key and the derived initialization vector.

Insecure use of cryptography

This function reimplements the OpenSSL function EVP_BytesToKey, which generates cryptographically weak keys, without any attempts at mitigating its insecurity. We provide this function for the purpose of interoperability with existing vault installations. Do not rely on this system to keep your vault configuration secure against access by even moderately determined attackers!

_make_decryptor ¶

_make_decryptor() -> CipherContext

Return the cipher context object used for decryption.

This is a standard AES256-CBC cipher context. The encryption key and the IV are derived via the OpenSSL EVP_BytesToKey function (using MD5, no salt, and one iteration). This is what the Node.js crypto library (v21 series and older) used in its implementation of crypto.createCipher("aes256", password).

Insecure use of cryptography

This function makes use of (an implementation of) the OpenSSL function EVP_BytesToKey, which generates cryptographically weak keys, without any attempts at mitigating its insecurity. We provide this function for the purpose of interoperability with existing vault installations. Do not rely on this system to keep your vault configuration secure against access by even moderately determined attackers!

export_vault_native_data ¶

export_vault_native_data(
    path: str | bytes | PathLike | None = None,
    key: str | Buffer | None = None,
    *,
    format: str
) -> Any

Export the full configuration stored in vault native format.

See exporter.ExportVaultConfigDataFunction for an explanation of the call signature, and the exceptions to expect.

Other Parameters:

Name	Type	Description
`format`	`str`	The only supported formats are `v0.2` and `v0.3`.

derivepassphrase.exporter ¶

NotAVaultConfigError ¶

ExportVaultConfigDataFunction ¶

__call__ ¶

get_vault_key ¶

get_vault_path ¶

find_vault_config_data_handlers ¶

export_vault_config_data ¶

derivepassphrase.exporter.storeroom ¶

export_storeroom_data ¶

_derive_master_keys_keys ¶

_decrypt_master_keys_data ¶

_decrypt_session_keys ¶

_decrypt_contents ¶

_decrypt_bucket_item ¶

_decrypt_bucket_file ¶

derivepassphrase.exporter.vault_native ¶

VaultNativeConfigParser ¶

__call__ ¶

_pbkdf2 staticmethod ¶

_parse_contents ¶

_derive_keys ¶

_generate_keys abstractmethod ¶

_check_signature ¶

_hmac_input abstractmethod ¶

_decrypt_payload ¶

_make_decryptor abstractmethod ¶

VaultNativeV03ConfigParser ¶

KEY_SIZE class-attribute instance-attribute ¶

_generate_keys ¶

_hmac_input ¶

_make_decryptor ¶

VaultNativeV02ConfigParser ¶

_parse_contents ¶

_generate_keys ¶

_hmac_input ¶

_evp_bytestokey_md5_one_iteration_no_salt staticmethod ¶

_make_decryptor ¶

export_vault_native_data ¶

call ¶

call ¶

_pbkdf2 `staticmethod` ¶

_generate_keys `abstractmethod` ¶

_hmac_input `abstractmethod` ¶

_make_decryptor `abstractmethod` ¶

KEY_SIZE `class-attribute` `instance-attribute` ¶

_evp_bytestokey_md5_one_iteration_no_salt `staticmethod` ¶