Metadata-Version: 2.4
Name: thingsboard-common
Version: 1.0.0
Summary: ThingsBoard common package
Home-page: https://gitlab.com/things-board
Author: HES-SO Valais, School of Engineering, Sion
Author-email: thomas.sterren@hes-so.ch
License-Expression: MIT
Project-URL: Bug Tracker, https://gitlab.com/things-board/thingsboard-common/-/issues
Project-URL: Source Code, https://gitlab.com/things-board/thingsboard-common
Project-URL: Documentation, https://gitlab.com/things-board/thingsboard-common#thingsboard-common
Keywords: ThingsBoard common library IoT cloud connector core utils
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: Information Technology
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Programming Language :: Python :: 3.13
Classifier: Operating System :: OS Independent
Requires-Python: >=3.13
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: configobj
Requires-Dist: urllib3
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: home-page
Dynamic: keywords
Dynamic: license-file
Dynamic: project-url
Dynamic: requires-dist
Dynamic: summary

# ThingsBoard Common

> Note: 
> 
> This is not an official project of the [
ThingsBoard](https://thingsboard.io/) authors! Please refer to [ThingsBoard - Open-source IoT Platform](https://github.com/thingsboard) for official ThingsBoard projects!

This project serves as a toolkit allowing other projects to use the same packages/modules as codebase.

These projects are using the [ThingsBoard](https://thingsboard.io/) platform and extend it with additional functionality.


## What it is for

`thingsboard.common` holds what the ThingsBoard applications of this family (endpoint, monitor, ...) share. The
core of it is how to run a worker thread that keeps going when the world around it fails:

| You want                                                    | Use             |
|-------------------------------------------------------------|-----------------|
| A thread that repeats a step, retries it when it fails, and stops cleanly | `Threaded`      |
| Growing, jittered delays between attempts of anything       | `Backoff`       |
| To know which threads are dead or stuck                     | `ThreadMonitor` |

Besides, `thingsboard.common.utils` has small helpers (strings, paths, timestamps, resources). It needs Python 3.13 or
later and ships type hints (`py.typed`). It has no dependency on an MQTT client.

The snippets below are excerpts. The complete scripts are in [examples/](examples), and the tests run them:
`PYTHONPATH=src python examples/threaded_retry.py`.

## Threaded: a worker thread that retries

Subclass `Threaded` and implement `_step()`. The thread calls it again and again, with no pause of its own: `_step()`
blocks, or calls `_thread_sleep_interval()` to sleep one control interval.

```python
from thingsboard.common.core import Backoff, Threaded

class Connector(Threaded):
    def __init__(self):
        # Retry after 0.1 s, 0.2 s, ... at most 1 s (the defaults are 2 s up to 60 s)
        super().__init__(control_interval_in_seconds=0.1, backoff=Backoff(initial=0.1, maximum=1.0))

    def _step(self):                        # Raise to tell it failed
        connect_and_work()
        self._thread_sleep_interval()       # Ends at once on stop_thread()

    def _is_fatal(self, exc):               # Retrying cannot help: the thread ends, see last_error
        return isinstance(exc, PermissionError)

connector = Connector()
connector.setup_thread()                    # Creates the thread (and registers it, see ThreadMonitor)
connector.start_thread()
...
connector.stop_thread()                     # Ends the wait or sleep at once, then joins the thread
```

Full example: [examples/threaded_retry.py](examples/threaded_retry.py).

**Retry mode** (`retry_on_error = True`, the default). If `_step()` raises an `Exception`, the thread:

1. ends if `_is_fatal(exc)` says so, with the exception in `last_error` (and logged);
2. else logs it, calls `_on_failure(exc, delay)` (e.g. to release what the failed step left open), waits `delay`
   seconds and calls `_step()` again.

The delay comes from the `Backoff` and grows while the failures last. The wait is cut short by `stop_thread()`, by
`retry_now()` (also if the step is still running, for a request to retry) and by `wakeup_thread()` (only if the thread
is waiting).

**What is logged.** The first failure of an outage is logged with its traceback, then at most one per
`failure_log_interval` seconds (default 60); the others go to debug level. Override `_describe_failure(exc, delay)` to
change level, message or traceback, e.g. to log an outage that is expected as a warning without a traceback.

**Hooks and knobs:** `_step()`, `_is_fatal()`, `_on_failure()`, `_describe_failure()`, `retry_on_error`,
`failure_log_interval`, `backoff`, `last_error`, `thread_alive`, `retry_now()`, `wakeup_thread()`.

**Stopping.** `stop_thread()` is safe to call at any time (never started, already stopped, died, or from the thread
itself). A thread that is blocked in a step and does not leave in time is retired: it ends on its own as soon as it
gets back, and changes nothing of the object, even if the thread was started again meanwhile.

**Not retrying.** Set `retry_on_error = False` in the subclass to have a failure end the thread (with the error in
`last_error`). A subclass may also override `_run()` with a loop of its own, then `_step()`, retry and backoff are not
used. Its thread still beats when it calls `_thread_sleep_interval()`, but the `ThreadMonitor` then checks it for `DEAD`
only, as there is no telling how often it beats; register it with your own `stalled_after` to change that.

## Backoff: delays between attempts

`Threaded` uses it in retry mode. Use it directly for a loop of your own:

```python
from thingsboard.common.core import Backoff

backoff = Backoff(initial=2.0, maximum=60.0, factor=2.0, jitter=0.25, stable_after=10.0)

backoff.record_success()                # whenever an attempt went well
delay = backoff.next_delay()            # whenever one failed
backoff.wait(delay)                     # blocks until the delay is over, or interrupt() / skip_next_wait()
```

The delay grows by `factor` after each failure up to `maximum`, and restarts at `initial` once things went well for
`stable_after` seconds: a flapping connection keeps its long delay, one that was good for a while gets a quick retry.
The jitter only shortens the delay, so `maximum` always holds. Full example, on a clock of its own so that nothing has to
wait: [examples/backoff.py](examples/backoff.py).

## ThreadMonitor: which threads are not live, and why

`ThreadMonitor` is an ordinary object (no singleton, no class state), so several may coexist. It is passive: call
`check()` regularly, e.g. from the main loop of your application, and decide what to do about a fault (restart, exit,
...). The monitor does neither.

```python
from thingsboard.common.core import Fault, ThreadMonitor

monitor = ThreadMonitor()
poller = Poller(thread_monitor=monitor)     # A Threaded registers its thread in setup_thread()
poller.setup_thread()
poller.start_thread()

for fault in monitor.check():               # Every thread that is not live, [] if all are
    print(fault.name, fault.reason, fault.since, fault.last_error)
    if fault.reason is Fault.DEAD:          # The thread ended, last_error tells why
        poller.start_thread()               # Restarting registers the new thread with a fresh heartbeat
    # Fault.STALLED: alive, but no cycle completed for too long (e.g. blocked in a call)

monitor.is_live()                           # True if check() finds nothing
```

Full example, with a thread that stalls and one that dies: [examples/thread_monitor.py](examples/thread_monitor.py).

- A thread is `DEAD` if it ended and `STALLED` if it is alive but has not beaten its `Heartbeat` for `stalled_after`
  seconds. `since` is the time (on the monitor's clock, `time.monotonic` by default) of its last sign of life.
- `Threaded` beats after each `_step()`, failed or not, and in each `_thread_sleep_interval()`. Its default
  `stalled_after` is 10 control intervals plus the longest backoff delay, as it does not beat while it waits to retry.
  A step that blocks for longer than that makes the thread look stalled.
- Any other thread registers itself and beats on its own:

  ```python
  heartbeat = monitor.register(thread, name='reader', stalled_after=30.0)    # stalled_after=None: dead-only
  ...
  heartbeat.beat()                                                           # once per completed cycle
  monitor.deregister(thread)
  ```

- `check()` changes nothing of the threads. The monitor logs once when a thread stops being live and once when it is
  live again, not on every `check()`. A restarted thread takes over the state of the one it replaces (same thread
  registered again, or same name after `deregister()`): a thread that was logged as dead is logged as live again.

## On failure: what the library does, and what it leaves to you

| The library does                                                    | You do                                                        |
|---------------------------------------------------------------------|---------------------------------------------------------------|
| Catches the exception of a step, logs it (throttled), waits and retries with growing delays | Decide what is fatal (`_is_fatal`), what to release (`_on_failure`), how loud to be (`_describe_failure`) |
| Ends the thread on a fatal error, or in `retry_on_error = False`, keeps the error in `last_error` and logs it | Restart it, or exit: call `start_thread()` again |
| Keeps heartbeats and tells, on `check()`, which threads are dead or stalled, and logs each change once | Call `check()` regularly and act on the faults |
| Stops threads quickly and retires one that does not leave in time   | Make steps that block return when `_thread_should_run` is false |
| Never kills a thread, never restarts one                            | Everything else: reconnecting, giving up, alerting           |

## Migrating to 1.0.0

Breaking changes, for the changelog:

- **Retry mode is the default.** A `Threaded` that implements `_step()` retries a failing step with backoff instead of
  ending. Set `retry_on_error = False` in the subclass to keep the old behavior (a failure ends the thread). A subclass
  that overrides `_run()` is not affected.
- **`ThreadMonitor` is an instance, not a singleton.** Gone: `ThreadMonitor.instance()`, the class-level `register()`,
  `deregister()` and `threadList`, the `log_name_prefix` argument. Create one with `ThreadMonitor()` and pass it to
  what is to be monitored (`Threaded(thread_monitor=monitor)`, or `monitor.register(thread)`).
- **`check_threads()` is now `check()`.** It returns a list of `LivenessFault` (empty if all threads are live) instead
  of a `bool`, and finds stalled threads as well as dead ones. `is_live()` gives the old yes/no. `register()` takes a
  `threading.Thread` (not a `Threaded`) and returns a `Heartbeat` (not `True`); `deregister()` returns `None` (not `True`).
- **A thread that crashes is logged by the class's logger and ends at once.** Before, the error went to the root logger
  and the thread lingered for 5 s before it ended. The error is now also in `last_error`, and the `ThreadMonitor`
  reports the thread as dead without that delay.
- **The `tb-mqtt-client` requirement is gone.** An application that imports it must depend on it itself.
- **Python 3.13 or later is required** (it was 3.9).

Also new, not breaking: `Backoff`, `RetryLoop`, `LogThrottle`, `last_error`, `retry_now()`, `thread_alive`, type hints
and `py.typed`, and a `stop_thread()` that is safe to call at any time.
