vinegar.data_source.sqlite

Data source backed by an SQLite database.

The data source provided by this module uses an instance of vinegar.utils.sqlite_store.DateStore for retrieving configuration data. This makes it most suitable for scenarios, where separate configuration values have to be managed for each individual systems.

Due to the nature of the backing data store, concurrent updates of configuration data are safely possible. This data source does not employ any caching techniques because these techniques would collide with the goal of providing safe, immediate updates in an environment where multiple processes might access the database.

The data source provided by this module has a close() method that can be used to close the data store backing the source. In a regular application, where a data source is created once and then used until the application is shutdown, this typically is not necessary, so that method is mainly intended for environments where a new data source instance is created frequently (e.g. for tests).

Using find_system

Due to the lack of a cache, every call to find_system or get_data will result in a lookup in the database and consequently access to the file system. In order to limit the amount of lookups, the find_system function can be disabled when it is not needed by setting find_system_enabled configuration option to False (see below).

When find_system is enabled, it will only return a system ID if that system is a unique match, meaning that this is the only system in the data store that has the specified value for the specified key. If multiple matching systems are found, None is returned.

Specifying a key prefix

When the key_prefix configuration option is used, find_system only looks for matching system when the key_prefix matches the specified key and removes the prefix from the specified key before looking it up in the database.

For example, if key_prefix is set to abc:def and find_system is called for the key abc:def:ghi, the key ghi is used for the lookup in the database.

The key_prefix option is also used by the get_data option. The actual data retrieved from the database is wrapped in dict instances so that the keys from the database effectively have the specified prefix.

For example, if key_prefix is set to abc:def and the data provided by the database is {'123': '456', '789': 'abc'}, the data returned by get_data is {'abc': {'def': {'123': '456', '789': 'abc'}}}.

Configuration options

This data source has several configuration options that can be used to control its behavior. Of all these options, only the db_file option must be specified. All other options have sensible default values.

db_file:

Path to the SQLite database file (as a str). This path is passed on to the backing DataStore.

find_system_enabled:

If True (the default) the find_system function is enabled. If set to False it is disabled completely. This makes sense for performance reasons if one knows that this data source will not make any meaningful contribution to a lookup through find_system.

key_prefix:

Prefix to be used for data item keys (a str). If this is the empty (the default), the keys from the database are used as is, both by find_system and get_data. If it is set to a non-empty strings, the data is returned wrapped in a dict for each component in the key_prefix (the : character serves as the component separator). This can be useful to put the data from the database in a sub-structure of the hierarchy forming the data tree, as the backing data store only uses a flat key structure.

class vinegar.data_source.sqlite.SQLiteSource(config: Mapping[Any, Any])

Data source that reads data from a SQLite database.

This data source uses a vinegar.utils.sqlite_store.DataStore as its backend for retrieving data.

For information about the configuration options supported by this data source, please refer to the module documentation.

close()

Close the data store backing this data source. All operations that involve the data store will fail by raising an exception after calling this method.

For most applications, where data sources are long-lived, relying on Python’s garbage collection is fine for closing the underlying data store. However, if an application rapidly creates and discards data source instances (e.g. for automated tests), closing the data source explicitly can be beneficial because it helps to release resources early on.

find_system(lookup_key: str, lookup_value: str) → str | None

Find a system given the specified key and value.

If no system can be found, the data source returns None.

Parameters:

• lookup_key – key for which to look. The interpretation of the key is up to the data source. Some data sources might use a flat structure, while others might support hierarchical data-structures. In the latter case, the use of the colon (:) as a hierarchy separator in the key is encouraged, but not required.

• lookup_value – value for which to look. The interpreation of the value is up to the data source.

Returns:

system identifier or None if no system could be identified using the specified key and value.

get_data(system_id: str, preceding_data: Mapping[Any, Any], preceding_data_version: str) → Tuple[Mapping[Any, Any], str]

Return data associated with the specified system.

If the data source does not have any information associated with the specified system ID, it should return an empty dictionary.

The return value of this method is in fact a tuple of the configuration data and a version string. The version string can be used by the calling code to decide whether the data has changed and thus caches have to be discarded. For example, the results of rendering a template might be cached and the cached version might be used as long as the version string returned by this method does not change. This means that implementations have to be careful to never return the same version string when the data for a system has changed. The vinegar.utils.version provides utility functions for generating version strings in a way that makes accidental collisions unlikely.

Please note that it is not the job of a data source to merge the preceding_data with the data provided by itself. The calling code takes care of this. Code wanting to use multiple data sources in a chain can use the get_composite_data_source function.

Implementations are encouraged to use caching to improve performance when this method is repeatedly called for the same systems.

Parameters:

• system_id – ID of the system for which data is requested.

• preceding_data – Data provided by the data source(s) that come earlier in the chain. This may be empty if there are no preceding data sources or if they did not provide any data for the system.

• preceding_data_version – Version of the preceding_data. This is an arbitrary string (typically a hash) that can be used to detect when the data provided by the preceding sources has changed.

Returns:

tuple where the first element is the data associated with the specified system and the second element is a version string that changes whenver the returned data changes (for the same system).

vinegar.data_source.sqlite.get_instance(config: Mapping[Any, Any]) → SQLiteSource

Create a SQLite data source.

For information about the configuration options supported by that source, please refer to the module documentation.

Parameters:

config – configuration for the data source.

Returns:

SQLite data source using the specified configuration.