API reference

taskhawk.listen_for_messages(priority: taskhawk.models.Priority, num_messages: int = 1, visibility_timeout_s: int = None, loop_count: int = None, shutdown_event: threading.Event = None) → None

Starts a taskhawk listener for message types provided and calls the task function with given args and kwargs.

If the task function accepts a param called metadata, it’ll be passed in with a dict containing the metadata fields: id, timestamp, version, receipt.

The message is explicitly deleted only if task function ran successfully. In case of an exception the message is kept on queue and processed again. If the task function keeps failing, SQS dead letter queue mechanism kicks in and the message is moved to the dead-letter queue.

This function is blocking by default. It may be run for specific number of loops by passing loop_count. It may also be stopped by passing a shut down event object which can be set to stop the function.

Parameters:

• priority – The priority queue to listen to
• num_messages – Maximum number of messages to fetch in one SQS API call. Defaults to 1
• visibility_timeout_s – The number of seconds the message should remain invisible to other queue readers. Defaults to None, which is queue default
• loop_count – How many times to fetch messages from SQS. Default to None, which means loop forever.
• shutdown_event – An event to signal that the process should shut down. This prevents more messages from being de-queued and function exits after the current messages have been processed.

taskhawk.process_messages_for_lambda_consumer(lambda_event: dict) → None

Process messages for a Taskhawk consumer Lambda app, and calls the task function with given args and kwargs

If the task function accepts a param called metadata, it’ll be passed in with a dict containing the metadata fields: id, timestamp, version, receipt.

In case of an exception, the message is kept on Lambda’s retry queue and processed again a fixed number of times. If the task function keeps failing, Lambda dead letter queue mechanism kicks in and the message is moved to the dead-letter queue.

taskhawk.task(*args, priority: taskhawk.models.Priority = <Priority.default: 1>, name: Optional[str] = None) → Callable

Decorator for taskhawk task functions. Any function may be converted into a task by adding this decorator as such:

@taskhawk.task
def send_email(to: str, subject: str, from: str = None) -> None:
    ...

Additional methods available on tasks are described by taskhawk.Task class

class taskhawk.Task(fn: Callable, priority: taskhawk.models.Priority, name: str)

Represents a Taskhawk task. This class provides methods to dispatch tasks asynchronously, You can also chain customizations such as:

send_email.with_headers(request_id='1234')\
          .with_priority(taskhawk.Priority.high)\
          .dispatch('example@email.com')

These customizations may also be saved and re-used multiple times

send_email_high_priority = send_email.with_priority(taskhawk.Priority.high)

send_email_high_priority.dispatch('example@email.com')

send_email_high_priority.with_headers(request_id='1234')\
                        .dispatch('example@email.com')

dispatch(*args, **kwargs) → None

Dispatch task for async execution

Parameters:

• args – arguments to pass to the task
• kwargs – keyword args to pass to the task

with_headers(**headers) → taskhawk.task_manager.AsyncInvocation

Create a task invocation that uses custom headers

Parameters:headers – Arbitrary headers Returns:an invocation that uses custom headers

with_priority(priority: taskhawk.models.Priority) → taskhawk.task_manager.AsyncInvocation

Create a task invocation with custom priority

Parameters:priority – Custom priority to attach to this invocation Returns:an invocation that uses custom priority

class taskhawk.Priority

Priority of a task. This may be used to differentiate batch jobs from other tasks for example.

High and low priority queues provide independent scaling knobs for your use-case.

default = 1

This is the default priority of a task if nothing is specified. In most cases, using just the default queue should work fine.

high = 2

low = 3

bulk = 4

Bulk queue will typically have different monitoring, and may be used for bulk jobs, such as sending push notifications to all users. This allows you to effectively throttle the tasks.

taskhawk.requeue_dead_letter(priority: taskhawk.models.Priority, num_messages: int = 10, visibility_timeout: int = None) → None

Re-queues everything in the Taskhawk DLQ back into the Taskhawk queue.

Parameters:

• priority – The priority queue to listen to
• num_messages – Maximum number of messages to fetch in one SQS call. Defaults to 10.
• visibility_timeout – The number of seconds the message should remain invisible to other queue readers. Defaults to None, which is queue default

taskhawk.extend_visibility_timeout(priority: taskhawk.models.Priority, receipt: str, visibility_timeout_s: int) → None

Extends visibility timeout of a message on a given priority queue for long running tasks.

Exceptions

class taskhawk.RetryException

Special exception that does not log an exception when it is received. This is a retryable error.

class taskhawk.LoggingException(message, extra=None)

An exception that allows passing additional logging info. extra must be a dict that will be passed to logging.exception and can be used by a logging adaptor etc.

class taskhawk.IgnoreException

Indicates that this task should be ignored.

class taskhawk.ValidationError

Message failed JSON schema validation

class taskhawk.ConfigurationError

There was some problem with settings

class taskhawk.TaskNotFound

No task was found that can handle the given message. Ensure that you call taskhawk.RegisterTask.