flexmeasures.cli.jobs
CLI commands for controlling jobs
Functions
- flexmeasures.cli.jobs.get_exception_handler(queue_name: str) Callable
Look up the exception handler to use for jobs on a given queue.
Queues without a handler of their own fall back to the generic handler.
- Parameters:
queue_name – the name of a single queue, e.g. ‘forecasting’.
- Returns:
an RQ exception handler.
- flexmeasures.cli.jobs.handle_worker_exception(job: Job, exc_type: Type[Exception], exc_value: Exception, traceback: TracebackType) None
Just a fallback, usually we would use the per-queue handler.
- flexmeasures.cli.jobs.parse_queue_list(queue_names_str: str) list[Queue]
Parse a | separated string of queue names against the app.queues dict.
The app.queues dict is expected to have queue names as keys, and rq.Queue objects as values.
- Parameters:
queue_names_str – a string with queue names separated by the | character
- Returns:
a list of Queue objects.
- flexmeasures.cli.jobs.release_database_connections_before_forking()
Keep forked job processes from sharing the worker’s database connections.
RQ’s Worker forks a process for every job, and a fork copies the parent’s connection pool, sockets included. A job would then talk to Postgres over the same session as the worker and as every job before it, so a job exiting mid-transaction would make the next job’s first query fail (#2806). Therefore, the worker gives up its own connections before it starts, and every forked process starts from an empty pool, leaving the parent’s sockets alone. See https://docs.sqlalchemy.org/en/20/core/pooling.html#using-connection-pools-with-multiprocessing-or-os-fork.