vinegar.request_handler.file¶

Request handler that serves files from the file systems.

The files can either be served as is or they can be processed through a template engine. The handler can either work with a single file or with a whole directory of files.

Optionally, this handler can supply system data from a DataSource when rendering templates. In this case, the request path has to contain a piece of information that allows for identifying the system. This can either be the system ID, or a value that allows for looking up the system ID through the data source’s find_system method. This is configured through the lookup_key configuration option.

The request handler classes provided by this module implement the DataSourceAware interface in order to get access to the data source. When using the request handlers in a container context, the container will usually take care of injecting the data source. When using the classes directly, the using code has to take care of that. However, a DataSource is only needed when using the lookup_key configuration option. If that option is empty (the default), no data source is needed and it is not even used if it is available.

This module provides request handlers for both HTTP and TFTP. These two handlers are almost identical in their behavior with only a few minor differences:

The options related to the Content-Type header are only available in the HTTP version. TFTP does not have any equivalent of the Content-Type header in HTTP.
The TFTP version of the handler allows request paths that do not start with a forward slash (/). In this case, the forward slash is added automatically. The HTTP version of the handler, in contrast, does not do anything like this because request paths not starting with a forward slash are rejected by the HTTP server component.

By default, this request handler does not process files as templates and treats them as binary files. However, when enabling templating through the template option, files are read as text files and decoded as UTF-8. The result of the rendering process of the template engine is then encoded as UTF-8 before sending it to the client.

This means that the default value of the content_type option (only available in the HTTP version of the handler) depends on whether a template engine is used or not.

Request path matching¶

The request paths for which a handler is used are configured through its request_path option. The configured request path must always start with a / and must not end with a / (the exception being the case where the whole path just consists of a single /).

When operating in file mode (if the file configuration option is used), the request path must be an exact match. For example, the configured request path /my/file matches the actual request path /my/file (and my/file for TFTP), but it does not match /my/file/ or /my/file/abc. In file mode, the special request path / can only be used for the HTTP version of the handler, not the TFTP one. The reason is that TFTP has no notion of index files and due to a missing / being added automatically, such a configuration would actually match requests with an empty path.

When operating in directory mode (if the root_dir configuration option is used), the configured request path specifies the prefix that must match. Extra portions of the actual request path are treated as the path of the requested file relative to the root_dir. For example, if request_path is set to /myprefix and root_dir is set to /path/on/fs, a request specifying the path /myprefix/some/file would result in the file /path/on/fs/some/file being served.

A request handler operating in directory mode will catch all requests that start with the specified prefix, even if the file does not actually exist. This means that a request handler serving a more specific path prefix has to come earlier in the chain of request handlers or it will never be asked whether it can handle a request.

When using the lookup_key configuration option, the configured request path has to contain exactly one placeholder. The default placeholder string is ..., but this can be changed through the lookup_value_placeholder configuration option.

When such a configuration is used, the placeholder marks a portion of the request path that might be different from request to request. For example, a configured request_path of /prefix/file-... will match an actual request path of /prefix/file-abc, /prefix/file-def, etc. The placeholder will not match an empty string, so the request path will not match /prefix/file-.

The value that replaces the placeholder in the actual request path is transformed according to the configuration set through the lookup_value_transform configuration option and then used to find a system ID through DataSource.find_system. This is most useful when also using a template engine (see Using templates). If no matching system can be found, the request file is treated as not existing (unless lookup_no_result_action is set to continue).

Placeholders do not necessarily have to appear at the end of a request path. For example, a request_path of /prefix/.../suffix` will match /prefix/abc/suffix

Placeholders can be used in file mode as well as in directory mode. Like request paths that do not use place holders, the extra portion of the path will be used as the path of the file inside the directory, when operating in directory mode.

Using templates¶

Optionally, files served by the request handlers can be treated as templates. This behavior is enabled by setting the template configuration option to the name of one of the template engines supported by get_template_engine. One good choice might be the jinja template engine.

If the lookup_key configuration option is also set, the request handler provides two context objects to the template engine: The id object contains the system ID (as a str) and the data object contains the data that has been returned from the data source’s get_data method. The data object is passed as a SmartLookupOrderedDict to make it easier to get nested values.

The data object is not available if the data source’s get_data method raised an exception. Usually, this will cause the template not even to be rendered, but this can be overridden through the data_source_error_action configuration option.

If the data source’s find_system method returns None and lookup_no_result_action is set to continue, neither id nor data are available. The same applies if find_system raises an exception and lookup_no_result_action is set to continue and data_source_error_action is set to ignore or warn.

Configuration example¶

In order to understand how the various configuration options work together, we look at an example configuration (expressed in YAML):

lookup_key: 'net:mac_addr'
lookup_no_result_action: continue
lookup_value_transform:
    - mac_address.normalize
request_path: '/prefix/...'
root_dir: '/path/to/the/directory'
template: jinja

We also assume that there is a file /path/to/the/directory/myconf.txt with the following content:

{%- if id is not defined -%}
This content is for systems which we do not know.
{%- else -%}
This system has the ID {{ id }} and the MAC address
{{ data.get('net:mac_addr') }}.
{%- endif -%}

When a request for /prefix/02-03-04-05-06-0a/myconf.txt arrives, the MAC address in the URL will be extracted and normalized (the - characters will be replaced by : and letters will be converted to upper case). The resulting value (02:03:04:05:06:0A) will be used as a lookup value together and passed to the data source’s find_system method along with the lookup key (net:mac_addr). Assuming that the data source can resolve this key-value combination, the returned system ID is used in a call to get_data and both the system ID and the data are passed to the template engine when rendering myconf.txt.

As we have set the lookup_no_result_action to continue a MAC address that is not known by the data source will not result in a “not found” error (status code 404 in the case of HTTP). Instead, the template is rendered without a system ID and system data, so that some default content is returned.

Configuration options¶

The file request handlers have several configuration options that can be used to control their behavior. Most of them apply both the HTTP and TFTP handler, but some are specific to only one of the two.

For a more detailed discussion about the options controlling request path matching, please refer to Request path matching. More information about the templating mechanism can be found in Using templates and a example making use of some of the options an be found in Configuration example.

The common options are:

file (mandatory): Path to the file that is served by the request handler. Either this option or the root_dir option must be set. Both options cannot be used together. If this option is set, the request handler only answers requests that specify the exact request_path and it does not serve any other files.
request_path (mandatory): request path for which this request handler should be used. The specified path must start with a / and it must not end with a / (unless the / is the only character in the whole path). When operating in directory mode, the value of this option is treated as a prefix. If the lookup_key option is set, the specified path must contain a placeholder (which is configured through the lookup_value_placeholder option). Please refer to Request path matching for a more thorough discussion of request path matching.
root_dir (mandatory): Path to the directory that contains the files that are served by the request handler. Either this option or the file option must be set. Both options cannot be used together. If this option is set, the request handler answers all requests that specify a path that starts with the request_path. The parts of the request path after that prefix are interpreted as the path of the file to be served, relative to the root_dir.
data_source_error_action (optional): Action to be taken if the data source’s find_system or get_data method raises an exception. If set to error (the default), the request handler lets this exception bubble up, typically resulting in the exception being logged and an error message being returned to the client. If set to ignore the exception is caught and process continues without the data from the data source (depending on the lookup_no_result_action if find_system raised the exception). If set to warn, the same actions as for the the ignore action are taken, but the exception is also logged.
lookup_key (optional): Name of the lookup key that shall be used when calling find_system. If None or empty (the default), no system-specifc data is retrieved and the request_path is not expected to contain a placeholder. If set to the special value :system_id:, the value extracted from the request path is not passed to find_system, but used as a system ID directly.
lookup_no_result_action (optional): Action to be taken if no system ID can be determined. This can happen if find_system returns None or if it raises an exception and data_source_error_action is set to ignore or warn. If set to continue the code proceeds without information about the system, so neither the id nor the data object are available when rendering the template. If set to not_found (the default), a “not found” error (error code 404 in the case of HTTP) os returned to the client.
lookup_value_placeholder (optional): Placeholder that is used for the lookup value in the request_path option. The default placeholder is .... This option has no effect unless lookup_key is set.
lookup_value_transform (optional): Transformations to be applied to the lookup value. This is a list of transformations that are applied to the value extracted from the request path and being passed to find_system or being used as a system ID directly. This list is passed to vinegar.transform.get_transformation_chain. If this list is empty (the default), no transformations are applied and the string extracted from the request path is used as is.
template (optional): name of the template engine (as a str) that shall be used for rending the files. This name is passed to get_template_engine in order to retrieve the template engine. If empty or set to None (the default), templating is disabled.
template_config (optional): configuration for the template engine. The default is an empty dictionary ({}). This configuration is passed on to the template engine as is.

The configuration options specific to the HTTP version of the request handler are:

content_type (optional): Value to use for the Content-Type header that is sent to the client. If None or empty (the default), the content_type is guessed based on the template option. If no template engine is used, the value application/octet-stream is used. If a template engine is used, the value text/plain; charset=UTF-8 is used. When specifying a different type, the value should include a charset specification for text types. The output of a template engine is always encoded as UTF-8, so typically this charset should be specified.
content_type_map (optional): Dictionary of filenames and file extensions to Content-Type specifications. This option is similar to the content_type option, but it allows for using different values for different files served by the same handler. When serving a file, the handler first looks for a key that exactly matches the filename (without any preceding path components). If no such key is found, it looks for a key that matches the file extension. If no such key is found either, it uses the value of the content_type option. For example, when serving the file /path/to/my.html, the handler first looks for my.html and, if that key does not exist, tries .html. This option can only be used in directory mode. In file mode, using this option does not make sense because there only is a single file and thus using the content_type option is simpler.

class vinegar.request_handler.file.FileRequestHandlerBase(config: Mapping[Any, Any])¶

Base class for the HttpFileRequestHandler and TftpFileRequestHandler.

This class implements most of the functionality of both handlers. The separate classes only exist due to the slightly different APIs for the two handler types.

For information about the configuration options supported by this request handler, please refer to the module documentation.

set_data_source(data_source: vinegar.data_source.DataSource) → None¶

Set the data source to be used by this component.

Calling inject_data_source might be preferable to calling this method directly.

In general, code using this method should not assume that this method is thread safe, even if the object that implements it is considered safe in general. This means that this method should typically be called only once, directly after creating an object.

Parameters: data_source – data source to be injected.

class vinegar.request_handler.file.HttpFileRequestHandler(config: Mapping[Any, Any])¶

HTTP request handler that serves files from the file system.

For information about the configuration options supported by this request handler, please refer to the module documentation.

handle(filename: str, method: str, headers: http.client.HTTPMessage, body: io.BufferedIOBase, client_address: Tuple, context: Any) → Tuple[http.HTTPStatus, Mapping[str, str], io.BufferedIOBase]¶

Handle the request.

This method returns a tuple of three items. The first item is the HTTP status code, the second item are the headers that shall be sent to the client, and the third is a file-like object from which the data for the requested file can be read. The returned file-like object must supply its data in binary form.

Parameters

filename – filename that has been requested by the client. The filename includes the full request path including the query string (if present) and has not been URL decoded.
method – the HTTP method used for the request (e.g. “GET”).
headers – HTTP headers provided by the client.
body – file-like object that provides the request body sent by the client. This file-like object returns bytes when reading.
client_address – client address. The structure of the tuple depends on the address family in use, but typically the first element is the client’s host address and the second element is the client’s port number.
context – context object that was returned by prepare_context.

Returns

tuple of the HTTP status code, the response headers, and a file-like object that provides the data that is transferred to the client. The file-like object must provide binary data.

class vinegar.request_handler.file.TftpFileRequestHandler(config: Mapping[Any, Any])¶

TFTP request handler that serves files from the file system.

For information about the configuration options supported by this request handler, please refer to the module documentation.

can_handle(filename: str, context: Any) → bool¶

Tell whether the request can be handled by this request handler.

Returns True if the request can be handled and False if it cannot be handled and the next request handler should be tried.

Parameters

filename – filename that has been requested by the client.
context – context object that was returned by prepare_context.

Returns

True if this request handler can handle the specified request, False if the request should be deferred to the next handler.

handle(filename: str, client_address: Tuple, context: Any) → io.BufferedIOBase¶

Handle the request. This method returns a file-like object from which the data for the requested file can be read. The returned file-like object must supply its data in binary form.

If the request handler detects that it actually cannot send data to the client (e.g. because the client lacks the required permissions), it should signal that by raising a TftpError.

Parameters

filename – filename that has been requested by the client.
client_address – client address. The structure of the tuple depends on the address family in use, but typically the first element is the client’s host address and the second element is the client’s port number.
context – context object that was returned by prepare_context.

Returns

file-like object that provides the data that is transferred to the client. The file-like object must provide binary data.

prepare_context(filename: str) → Any¶

Prepare a context object for use by can_handle and handle. This method is called for each request before calling can_handle.

This is useful when both function need to do some processing on the filename or client address. This processing can be implemented in prepare_context and passed to the two other methods through the context so that it does not have to be done twice.

The return value of this method is passed to can_handle and handle. The default implementation simply returns None.

Parameters: filename – filename that has been requested by the client.
Returns: context object that is passed to can_handle and handle.

vinegar.request_handler.file.get_instance_http(config: Mapping[Any, Any]) → vinegar.request_handler.file.HttpFileRequestHandler¶

Create a HTTP request handler serving files.

If the request handler needs a data source (if its lookup_key configuration option is used), the data source has to be set by calling the set_data_source method of the returned object.

Parameters: config – configuration for this request handler. Please refer to the module documentation for a list of supported options.
Returns: HTTP request handler serving files from the file system.

vinegar.request_handler.file.get_instance_tftp(config: Mapping[Any, Any]) → vinegar.request_handler.file.HttpFileRequestHandler¶

Create a TFTP request handler serving files.

If the request handler needs a data source (if its lookup_key configuration option is used), the data source has to be set by calling the set_data_source method of the returned object.

Parameters: config – configuration for this request handler. Please refer to the module documentation for a list of supported options.
Returns: TFTP request handler serving files from the file system.