vinegar.request_handler.file

Request handler that serves files from the file systems.

The files can either be served as is or they can be processed through a template engine. The handler can either work with a single file or with a whole directory of files.

Optionally, this handler can supply system data from a DataSource when rendering templates. In this case, the request path has to contain a piece of information that allows for identifying the system. This can either be the system ID, or a value that allows for looking up the system ID through the data source’s find_system method. This is configured through the lookup_key configuration option.

The request handler classes provided by this module implement the DataSourceAware interface in order to get access to the data source. When using the request handlers in a container context, the container will usually take care of injecting the data source. When using the classes directly, the using code has to take care of that. However, a DataSource is only needed when using the lookup_key configuration option. If that option is empty (the default), no data source is needed and it is not even used if it is available.

This module provides request handlers for both HTTP and TFTP. These two handlers are almost identical in their behavior with only a few minor differences:

The options related to the Content-Type header are only available in the HTTP version. TFTP does not have any equivalent of the Content-Type header in HTTP.
The TFTP version of the handler allows request paths that do not start with a forward slash (/). In this case, the forward slash is added automatically. The HTTP version of the handler, in contrast, does not do anything like this because request paths not starting with a forward slash are rejected by the HTTP server component.

By default, this request handler does not process files as templates and treats them as binary files. However, when enabling templating through the template option, files are read as text files and decoded as UTF-8. The result of the rendering process of the template engine is then encoded as UTF-8 before sending it to the client.

This means that the default value of the content_type option (only available in the HTTP version of the handler) depends on whether a template engine is used or not.

Request path matching

The request paths for which a handler is used are configured through its request_path option. The configured request path must always start with a / and must not end with a / (the exception being the case where the whole path just consists of a single /).

When operating in file mode (if the file configuration option is used), the request path must be an exact match. For example, the configured request path /my/file matches the actual request path /my/file (and my/file for TFTP), but it does not match /my/file/ or /my/file/abc. In file mode, the special request path / can only be used for the HTTP version of the handler, not the TFTP one. The reason is that TFTP has no notion of index files and due to a missing / being added automatically, such a configuration would actually match requests with an empty path.

When operating in directory mode (if the root_dir configuration option is used), the configured request path specifies the prefix that must match. Extra portions of the actual request path are treated as the path of the requested file relative to the root_dir. For example, if request_path is set to /myprefix and root_dir is set to /path/on/fs, a request specifying the path /myprefix/some/file would result in the file /path/on/fs/some/file being served.

A request handler operating in directory mode will catch all requests that start with the specified prefix, even if the file does not actually exist. This means that a request handler serving a more specific path prefix has to come earlier in the chain of request handlers or it will never be asked whether it can handle a request.

When using the lookup_key configuration option, the configured request path has to contain exactly one placeholder. The default placeholder string is ..., but this can be changed through the lookup_value_placeholder configuration option.

When such a configuration is used, the placeholder marks a portion of the request path that might be different from request to request. For example, a configured request_path of /prefix/file-... will match an actual request path of /prefix/file-abc, /prefix/file-def, etc. The placeholder will not match an empty string, so the request path will not match /prefix/file-.

The value that replaces the placeholder in the actual request path is transformed according to the configuration set through the lookup_value_transform configuration option and then used to find a system ID through DataSource.find_system. This is most useful when also using a template engine (see Using templates). If no matching system can be found, the request file is treated as not existing (unless lookup_no_result_action is set to continue).

Placeholders do not necessarily have to appear at the end of a request path. For example, a request_path of /prefix/.../suffix` will match /prefix/abc/suffix

Placeholders can be used in file mode as well as in directory mode. Like request paths that do not use place holders, the extra portion of the path will be used as the path of the file inside the directory, when operating in directory mode.

Using templates

Optionally, files served by the request handlers can be treated as templates. This behavior is enabled by setting the template configuration option to the name of one of the template engines supported by get_template_engine. One good choice might be the jinja template engine.

A context object with the name request_info is passed to the template engine. For more information about this object, please refer to Request information.

If the lookup_key configuration option is also set, the request handler additionally provides two context objects to the template engine: The id object contains the system ID (as a str). The data object contains the data that has been returned from the data source’s get_data method. The data object is passed as a SmartLookupDict to make it easier to get nested values.

The data object is not available if the data source’s get_data method raised an exception. Usually, this will cause the template not even to be rendered, but this can be overridden through the data_source_error_action configuration option.

If the data source’s find_system method returns None and lookup_no_result_action is set to continue, neither id nor data are available. The same applies if find_system raises an exception and lookup_no_result_action is set to continue and data_source_error_action is set to ignore or warn.

Request information

The request_info object is a dict, which contains the following keys. The headers and method keys are only present for requests received via HTTP. Additional keys might be added in future versions.

client_address

The value for this key is a two-tuple, where the first element is the IP address of the requesting client (as a str) and the second element is the port number used by the client (as an int).

headers

The HTTP request headers as an instance of http.client.HTTPMessage.

This key is only available for HTTP requests.

method

The HTTP request method (e.g. GET, POST, etc.).

This key is only available for HTTP requests.

server_address

The value for this key is a two-tuple, where the first element is the IP address on which the server received the request (as a str) and the second element is the port number on which the server received the requeset (as an int).

Please note that for TFTP requests, the server might not be able to determine the address on which a request was received (due to limitations on some platforms). In this case, this will instead be the IP address to which the server’s socket has been bound.

uri

The full request URI specified by the client in undecoded form. For a TFTP request, this typically is the filename. For an HTTP request, this might also include a query string.

Access restrictions

By default, this request handler will allow any client to retrieve files rendered for any system. This might not be desirable if the files contain confidential information (e.g. password), in particular when the network in which the Vinegar server is running cannot be deemed secure.

Therefore, it is possible to limit access to the data for each system to specific clients. This is achieved through the client_address_key configuration option. This option specifies a key into the system data. The key can consist of multiple components separated by colons (:) to point into a nested dictionary.

For example, suppose that client_address_key is set to net:ip_addr. If the handler gets a request for the system ID myid, it will ask the data source for the system data for this system by calling data_source.get_data('myid', {}, '').

If the system data returned for this system does not contain a value for the specified key, the request is denied with HTTP status code 403 (forbidden).

For this example, let us assume that the data source returns the following data for the system myid (expressed as YAML):

net:
    ip_addr: 192.0.2.1

In this case, the request handler will only allow the request, if it comes from the IP address 192.0.2.1. In all other cases, it will reject the request.

Instead of a single IP address, the system data may also contain a list or set of IP addresses. For example, if get_data returned the following system data

net:
    ip_addr:
        - 192.0.2.1
        - 2001:db8::1

the request would be allowed if it came from 192.0.2.1 or 2001:db8::1.

In addition to single IP addresses, IP subnets using CIDR notation are allowed as well. For example,

net:
    ip_addr:
        - 192.0.2.0/24
        - 2001:db8::/64

will allow access for all clients from the IP subnets 192.0.2.0/24 and 2001:db8::/64.

As an alternative to the client_address_key option, the client_address_list option may be specified. This option takes a list of IP addresses or IP subnets and allows requests from clients that match one of these entries. This can be useful in order to allow full access from certain administrative clients.

When both client_address_key and client_address_list are specified, they are combined, meaning that a request is allowed when it either matches one of the entries from client_address_list or the entries from the data tree for the client_address_key.

When the client_address_key option is used, the lookup_key option must be set as well and this request handler needs a data source in order to get information about the system. This request handler implements the DataSourceAware interface, so when it is used inside a container, that container will typically take care of setting the data source. If instantiated directly, the data source has to be set explicitly by calling the handler’s set_data_source method.

Configuration example

In order to understand how the various configuration options work together, we look at an example configuration (expressed in YAML):

lookup_key: 'net:mac_addr'
lookup_no_result_action: continue
lookup_value_transform:
    - mac_address.normalize
request_path: '/prefix/...'
root_dir: '/path/to/the/directory'
template: jinja

We also assume that there is a file /path/to/the/directory/myconf.txt with the following content:

{%- if id is not defined -%}
This content is for systems which we do not know.
{%- else -%}
This system has the ID {{ id }} and the MAC address
{{ data.get('net:mac_addr') }}.
{%- endif -%}

When a request for /prefix/02-03-04-05-06-0a/myconf.txt arrives, the MAC address in the URL will be extracted and normalized (the - characters will be replaced by : and letters will be converted to upper case). The resulting value (02:03:04:05:06:0A) will be used as a lookup value together and passed to the data source’s find_system method along with the lookup key (net:mac_addr). Assuming that the data source can resolve this key-value combination, the returned system ID is used in a call to get_data and both the system ID and the data are passed to the template engine when rendering myconf.txt.

As we have set the lookup_no_result_action to continue a MAC address that is not known by the data source will not result in a “not found” error (status code 404 in the case of HTTP). Instead, the template is rendered without a system ID and system data, so that some default content is returned.

Configuration options

The file request handlers have several configuration options that can be used to control their behavior. Most of them apply both the HTTP and TFTP handler, but some are specific to only one of the two.

For a more detailed discussion about the options controlling request path matching, please refer to Request path matching. More information about the templating mechanism can be found in Using templates and a example making use of some of the options an be found in Configuration example.

The common options are:

file (mandatory):: Path to the file that is served by the request handler. Either this option or the root_dir option must be set. Both options cannot be used together. If this option is set, the request handler only answers requests that specify the exact request_path and it does not serve any other files.
request_path (mandatory):: request path for which this request handler should be used. The specified path must start with a / and it must not end with a / (unless the / is the only character in the whole path). When operating in directory mode, the value of this option is treated as a prefix. If the lookup_key option is set, the specified path must contain a placeholder (which is configured through the lookup_value_placeholder option). Please refer to Request path matching for a more thorough discussion of request path matching.
root_dir (mandatory):: Path to the directory that contains the files that are served by the request handler. Either this option or the file option must be set. Both options cannot be used together. If this option is set, the request handler answers all requests that specify a path that starts with the request_path. The parts of the request path after that prefix are interpreted as the path of the file to be served, relative to the root_dir.
client_address_key (optional):: Key into the system data that points to the place in the data where the allowed client address or addresses are stored. If this option is not set (the default), each client can access this handler for arbitrary system IDs and thus potentially retrieve data associated with these systems. When this is not desired, this option can be used to limit the allowe client (IP) addresses for each system. The key can point into a nested dictionary, using the colon (:) to separate key components for the various levels. The value can be a string (matching exactly one IP address or IP subnet) or a list or set of IP addresses or IP subnets (matching any of the addresses or subnets in the list or set). If this option is set, lookup_key must be set as well. Please refer to Access restrictions for a more detailed discussion of how this option can be used.
client_address_list (optional):: List of IP addresses or IP subnets from which requests are allowed. Please refer to Access restrictions for a more detailed discussion of how this option can be used.
data_source_error_action (optional):: Action to be taken if the data source’s find_system or get_data method raises an exception. If set to error (the default), the request handler lets this exception bubble up, typically resulting in the exception being logged and an error message being returned to the client. If set to ignore the exception is caught and process continues without the data from the data source (depending on the lookup_no_result_action if find_system raised the exception). If set to warn, the same actions as for the the ignore action are taken, but the exception is also logged.
file_suffix (optional):: File name suffix that is added to the requested file name when operating in directory mode. For example, when this is set to .xyz and the file name in the request URI is abc.txt, the handler looks for the file abc.txt.xyz on the file system. By default, this option is set to the empty string, so that the file name on the file system must be the same one as in the request. Setting this option to a non-empty value is particularly useful when using a template engine, so that the files on the file system can use a suffix appropriate for this template engine, helping editors with using the correct type of syntax highlighting. This option can only be used in combination with the root_dir option, not with the file option.
lookup_key (optional):: Name of the lookup key that shall be used when calling find_system. If None or empty (the default), no system-specifc data is retrieved and the request_path is not expected to contain a placeholder. If set to the special value :system_id:, the value extracted from the request path is not passed to find_system, but used as a system ID directly.
lookup_no_result_action (optional):: Action to be taken if no system ID can be determined. This can happen if find_system returns None or if it raises an exception and data_source_error_action is set to ignore or warn. If set to continue the code proceeds without information about the system, so neither the id nor the data object are available when rendering the template. If set to not_found (the default), a “not found” error (error code 404 in the case of HTTP) os returned to the client.
lookup_value_placeholder (optional):: Placeholder that is used for the lookup value in the request_path option. The default placeholder is .... This option has no effect unless lookup_key is set.
lookup_value_transform (optional):: Transformations to be applied to the lookup value. This is a list of transformations that are applied to the value extracted from the request path and being passed to find_system or being used as a system ID directly. This list is passed to vinegar.transform.get_transformation_chain. If this list is empty (the default), no transformations are applied and the string extracted from the request path is used as is.
template (optional):: name of the template engine (as a str) that shall be used for rending the files. This name is passed to get_template_engine in order to retrieve the template engine. If empty or set to None (the default), templating is disabled.
template_config (optional):: configuration for the template engine. The default is an empty dictionary ({}). This configuration is passed on to the template engine as is.

The configuration options specific to the HTTP version of the request handler are:

content_type (optional):: Value to use for the Content-Type header that is sent to the client. If None or empty (the default), the content_type is guessed based on the template option. If no template engine is used, the value application/octet-stream is used. If a template engine is used, the value text/plain; charset=UTF-8 is used. When specifying a different type, the value should include a charset specification for text types. The output of a template engine is always encoded as UTF-8, so typically this charset should be specified.
content_type_map (optional):: Dictionary of filenames and file extensions to Content-Type specifications. This option is similar to the content_type option, but it allows for using different values for different files served by the same handler. When serving a file, the handler first looks for a key that exactly matches the filename (without any preceding path components). If no such key is found, it looks for a key that matches the file extension. If no such key is found either, it uses the value of the content_type option. For example, when serving the file /path/to/my.html, the handler first looks for my.html and, if that key does not exist, tries .html. This option can only be used in directory mode. In file mode, using this option does not make sense because there only is a single file and thus using the content_type option is simpler.

class vinegar.request_handler.file.HttpFileRequestHandler(config: Mapping[Any, Any])

HTTP request handler that serves files from the file system.

For information about the configuration options supported by this request handler, please refer to the module documentation.

can_handle(uri: str, context: Any) → bool

Tell whether the request can be handled by this request handler.

Returns True if the request can be handled and False if it cannot be handled and the next request handler should be tried.

Parameters:

uri – URI that has been requested by the client. The URI includes the full request path including the query string (if present) and has not been URL decoded.
context – context object that was returned by prepare_context.

Returns:

True if this request handler can handle the specified request, False if the request should be deferred to the next handler.

handle(request_info: HttpRequestInfo, body: BufferedIOBase, context: Any) → Tuple[HTTPStatus, Mapping[str, str] | None, BufferedIOBase | None]

Handle the request.

This method returns a tuple of three items. The first item is the HTTP status code, the second item are the headers that shall be sent to the client, and the third is a file-like object from which the data for the requested file can be read. The returned file-like object must supply its data in binary form.

Parameters:

request_info – information about the request that shall be handled.
body – file-like object that provides the request body sent by the client. This file-like object returns bytes when reading.
context – context object that was returned by prepare_context.

Returns:

tuple of the HTTP status code, the response headers, and a file-like object that provides the data that is transferred to the client. The file-like object must provide binary data. The response headers may be None, which has the same effect as supplying an empty dict (no headers are added to the response). The file-like object may also be None, which means that the body of the response is empty. Typically, this is only useful when indicating an error (status code >= 400). In this case, a body with an appropriate error message for the status code is generated by the server.

prepare_context(uri: str) → Any

Prepare a context object for use by can_handle and handle. This method is called for each request before calling can_handle.

This is useful when both function need to do some processing on the URI. This processing can be implemented in prepare_context and passed to the two other methods through the context so that it does not have to be done twice.

The return value of this method is passed to can_handle and handle. The default implementation simply returns None.

Parameters:: uri – URI that has been requested by the client. The URI includes the full request path including the query string (if present) and has not been URL decoded.
Returns:: context object that is passed to can_handle and handle.

class vinegar.request_handler.file.TftpFileRequestHandler(config: Mapping[Any, Any])

TFTP request handler that serves files from the file system.

For information about the configuration options supported by this request handler, please refer to the module documentation.

can_handle(filename: str, context: Any) → bool

Tell whether the request can be handled by this request handler.

Returns True if the request can be handled and False if it cannot be handled and the next request handler should be tried.

Parameters:

filename – filename that has been requested by the client.
context – context object that was returned by prepare_context.

Returns:

True if this request handler can handle the specified request, False if the request should be deferred to the next handler.

handle(filename: str, client_address: Tuple[str, int] | Tuple[str, int, int, int], server_address: Tuple[str, int] | Tuple[str, int, int, int], context: Any) → BufferedIOBase

Handle the request. This method returns a file-like object from which the data for the requested file can be read. The returned file-like object must supply its data in binary form.

If the request handler detects that it actually cannot send data to the client (e.g. because the client lacks the required permissions), it should signal that by raising a TftpError.

Parameters:

filename – filename that has been requested by the client.
client_address – client address. The structure of the tuple depends on the address family in use, but typically the first element is the client’s host address and the second element is the client’s port number.
server_address – server address. The structure of the tuple depends on the address family in use, but typically the first element is the server’s host address and the second element is the server’s port number. Please note that determining the actual address specified by the client might not be possible if the sever binds to all interfaces (::) and the platform does not support the IPV6_PKTINFO socket option. In this case, this will be the address to which the server is bound and not the address on which the request was actually received.
context – context object that was returned by prepare_context.

Returns:

file-like object that provides the data that is transferred to the client. The file-like object must provide binary data.

prepare_context(filename: str) → Any

Prepare a context object for use by can_handle and handle. This method is called for each request before calling can_handle.

This is useful when both function need to do some processing on the filename or client address. This processing can be implemented in prepare_context and passed to the two other methods through the context so that it does not have to be done twice.

The return value of this method is passed to can_handle and handle. The default implementation simply returns None.

Parameters:: filename – filename that has been requested by the client.
Returns:: context object that is passed to can_handle and handle.

vinegar.request_handler.file.get_instance_http(config: Mapping[Any, Any]) → HttpFileRequestHandler

Create a HTTP request handler serving files.

If the request handler needs a data source (if its lookup_key configuration option is used), the data source has to be set by calling the set_data_source method of the returned object.

Parameters:: config – configuration for this request handler. Please refer to the module documentation for a list of supported options.
Returns:: HTTP request handler serving files from the file system.

vinegar.request_handler.file.get_instance_tftp(config: Mapping[Any, Any]) → TftpFileRequestHandler

Create a TFTP request handler serving files.

If the request handler needs a data source (if its lookup_key configuration option is used), the data source has to be set by calling the set_data_source method of the returned object.

Parameters:: config – configuration for this request handler. Please refer to the module documentation for a list of supported options.
Returns:: TFTP request handler serving files from the file system.