Metadata-Version: 2.4
Name: datahub-api-connector
Version: 1.4
Summary: Package to help connect to Verity's Data Hub API
Author-email: Verity <support@verity.global>
Maintainer-email: Verity <support@verity.global>
License-Expression: MIT
Project-URL: Homepage, https://github.com/opinum/datahub-api-connector
Project-URL: Bug Tracker, https://github.com/opinum/datahub-api-connector/issues
Keywords: datahub,Data Hub,Verity,Opinum,API,Keycloak
Classifier: Development Status :: 5 - Production/Stable
Classifier: Programming Language :: Python
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests
Requires-Dist: requests_oauthlib
Requires-Dist: oauthlib
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Dynamic: license-file

This package simplifies the calls to Data Hub's API.

No magic. You need to follow the [Data Hub Swagger Documentation](https://api.opinum.com) for correct formatting of your requests

Be aware that this is a new package, certainly missing many features. Feel free to contribute.

You first need to create an instance of the ApiConnector class with following parameters:

*ATTENTION:* this package can only be used from Data Hub version 7.0 (July 1, 2025) onward, as it uses the authentication to the the new technology (Keycloak).
For previous versions, the opinum-api-connector package must be used (https://github.com/opinum/opinum-api-connector) instead.

*VERSION 1.4*
Retry release.
* Server-side failures are now retried instead of being raised on the first attempt. Until now only connection errors and read timeouts were, while `raise_for_status()` turns a 500 into an HTTPError that no clause caught, so a single transient failure aborted the call. See the new _retry_on_status_ and _retry_unsafe_methods_ parameters.
* The wait between two attempts now doubles each time, capped at 60 seconds, with a small jitter so that the threads sharing an instance stop retrying in lockstep. A _Retry-After_ response header is honoured.
* Retry and failure logs now name the method, the URL and the beginning of the response body, where the API states the actual cause. It was discarded before.
* The module no longer calls `logging.basicConfig()` nor changes the root logger level, which reset the logging an application had already configured for itself.

*VERSION 1.3*
Sturdiness and correctness release.
* The token lifetime now follows the real expiry returned by the authentication server (with a 120 seconds safety margin) instead of a fixed 3 minutes limit, and is renewed with its refresh token when possible.
* All calls now go through a single persistent connection pool (see the new _pool_size_ parameter) instead of opening a new connection each time, which exhausted the available sockets on long multi-threaded runs.
* Fixed a retried call sending a doubly encoded body (rejected with a 400), and a token renewed between two attempts not being used. Read timeouts are now retried as well, and the last failed attempt no longer waits before raising.
* Fixed _operation_timeout_sec_ being ignored on push_data when _operation_id_ was given too, and file names containing special characters on send_file_to_storage.
* multi_thread_request_on_path now really uses threads (it raised a pickling error before) and splits the calls on _max_parameter_entities_ as documented, instead of on _max_futures_. Expect different call sizes than in 1.2.
* ApiConnector can be closed, and used as a context manager, to release its connections.

*VERSION 1.2*
Fixed operation_id on PUSH calls, and added operation_timeout_sec. Also added the possibility to request a count of items on some calls with the IncludeItemsCount optional parameter.

*VERSION 1.1*
Improved sturdiness. Added thread lock on token requests, and a default timeout of 10 seconds on all requests (incliding token)

> _environment_
> > a dictionary of environment variables
> >
> > if `None`, ApiConnector uses your environment variables (_os.environ_)
> >
> > Mandatory environment variables are:
> >
> > * _DATAHUB_USERNAME_: the Data Hub user. <br>
> > TAKE CARE: if this user has access to multiple tenants and if you do not specify a tenant id,
> > ApiConnector will use the last tenant used.
> > * _DATAHUB_PASSWORD_: the password for the user
> > * _DATAHUB_CLIENT_ID_: the client id for accessing the API
> > * _DATAHUB_CLIENT_SECRET_ the corresponding secret
> > 
> > Optional environment variables are:
> >
> > * _DATAHUB_API_URL_: another API URL than the Europe SaaS one (https://api.opinum.com)
> > * _DATAHUB_AUTH_URL_: another authentication URL than the Europe SaaS one (https://auth.opinum.com)
> > * _DATAHUB_PUSH_URL_: another push URL than the Europe SaaS one (https://push.opinum.com)
> > * _DATAHUB_SCOPE_: the scope of you session (default: "_datahub-api_")<br>
> > if you want to push data, the scope should be "_datahub-api push-data_"

> _account_id_
> > one of the tenant ids available for the Data Hub user (default: `None`)

> _retries_when_connection_failure_
> > number of extra attempts when no 200 or 204 return code (default: 0, maximum: 5)
> >
> > the wait between two attempts doubles each time (_seconds_between_retries_, then
> > twice that, and so on, capped at 60 seconds) and carries a small random jitter,
> > so that several threads sharing the instance do not all retry at the very same
> > moment. A _Retry-After_ response header takes precedence over that wait.

> _request_timeout_
> > timeout value in seconds on all requests (including fetch token) (default: 10)

> _retry_on_status_
> > HTTP statuses retried instead of being raised straight away
> > (default: 429, 500, 502, 503, 504)
> >
> > 501 is absent on purpose, and so are the 4xx other than 429: they would fail
> > identically on a second attempt. Pass `None` to disable status retries.

> _retry_unsafe_methods_
> > also apply _retry_on_status_ to post, put, patch and delete (default: `False`)
> >
> > by default only get is retried on those statuses. Replaying a call that changes
> > something, when the server may have applied it before failing, would duplicate
> > the change. Set this to `True` when your write calls are safe to replay, or when
> > the call only reads despite being a post (the query-by-body `POST /data` is one).

> _log_level_
> > sets log level for the module (default: INFO)
> >
> > only this package's logger is touched; the root logger of the calling
> > application is left as that application configured it

> _pool_size_
> > size of the connection pool shared by all calls of the instance (default: 32)
> >
> > it must be at least as large as the number of threads sharing the instance
> > (the _workers_ parameter of multi_thread_request_on_path, default: 16)

Once you have your ApiConnector instance, you may use the class methods

* get
* post
* patch
* put
* delete
* send_file_to_storage

All keyword arguments will be converted to path parameters in the API call with one important exception,
the _data_ keyword referring to the body of your call.

There are two other class methods for data pushing because we have another API for this

* push_data
* push_dataframe_data

There is a little bit of magic with the method multi_thread_request_on_path that splits a list of parameters
Allowing to make parallel calls.
