Metadata-Version: 2.4
Name: shareddata
Version: 6.86.2
Summary: Memory-mapped record and time-series store with an HTTP API and a distributed scheduler
Home-page: https://shareddata.net
Author: shareddata
Author-email: admin@shareddata.net
License: MIT
Project-URL: Homepage, https://shareddata.net
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.11, <3.13
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: ipykernel<8,>=7.4
Requires-Dist: python-json-logger<5,>=4.2
Requires-Dist: python-dotenv<2,>=1.2.4
Requires-Dist: pandas<3,>=2.2
Requires-Dist: scipy<1.18,>=1.14
Requires-Dist: numpy<1.27,>=1.26.4
Requires-Dist: numba==0.59.1
Requires-Dist: XlsxWriter<4,>=3.2.9
Requires-Dist: openpyxl<4,>=3.1.5
Requires-Dist: tqdm<5,>=4.70.1
Requires-Dist: cffi<3,>=2.1
Requires-Dist: tzlocal<6,>=5.4.4
Requires-Dist: websockets<18,>=17.1
Requires-Dist: lz4<5,>=4.4.5
Requires-Dist: flask<4,>=3.1.3
Requires-Dist: waitress<4,>=3.0.2
Requires-Dist: requests<3,>=2.34.2
Requires-Dist: flasgger==0.9.7.1
Requires-Dist: pymongo<5,>=4.18.2
Requires-Dist: setuptools>=84
Requires-Dist: filelock<5,>=4.0.9
Requires-Dist: gunicorn<27,>=26.2
Requires-Dist: aiohttp<4,>=3.14.3
Requires-Dist: boto3<2,>=1.43
Requires-Dist: confluent-kafka<3,>=2.15.1
Requires-Dist: aiokafka<1,>=0.14
Requires-Dist: redis<9,>=8.1
Requires-Dist: Authlib<2,>=1.8
Requires-Dist: PyJWT<3,>=2.8
Requires-Dist: argon2-cffi<26,>=25.1
Requires-Dist: flask-jwt-extended<5,>=4.6
Requires-Dist: cryptography<51,>=50.0.2
Requires-Dist: segno<2,>=1.6
Requires-Dist: psutil<8,>=5.9
Requires-Dist: mcp>=1.27
Requires-Dist: httpx>=0.27
Provides-Extra: test
Requires-Dist: pytest<10,>=8.3.4; extra == "test"
Requires-Dist: pytest-asyncio<2,>=0.24; extra == "test"
Requires-Dist: playwright<2,>=1.63; extra == "test"
Provides-Extra: s3
Provides-Extra: kafka
Provides-Extra: redis
Provides-Extra: all
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license
Dynamic: license-file
Dynamic: project-url
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# SharedData

SharedData is an open-source memory-mapped store for records and time series —
in the workload category of tick stores such as kdb+ — plus the scheduler and
worker fleet that produce the data. Local processes map the same files through
the operating system's page cache; indexed writes coordinate through a
per-table process lock. Remote clients use an authenticated HTTP API; shared
memory does not span hosts.

Website and manual: <https://shareddata.net>

## Install

Python 3.11 or 3.12:

```sh
python -m pip install shareddata
```

The [quick start](https://shareddata.net/docs/QUICK_START/) uses illustrative
records in isolated storage. See [configuration](https://shareddata.net/docs/CONFIGURATION/)
for public environment names and [Windows limitations](https://shareddata.net/docs/WINDOWS/)
for native requirements and rewrite behavior.

## A record in one minute

Save as `example.py` after setting up isolated storage in the
[quick start](https://shareddata.net/docs/QUICK_START/), then run `python example.py`:

```python
from SharedData.SharedData import SharedData
import pandas as pd

store = SharedData(__file__, user="example")
index = pd.MultiIndex.from_tuples(
    [(pd.Timestamp("2026-01-02"), "EXAMPLE")],
    names=["date", "symbol"],
)
rows = pd.DataFrame({"close": [100.0]}, index=index)
records = store.table(
    "MarketData", "D1", "DEMO", "PRICES",
    value=rows,
)
print(records.loc["2026-01-02", "EXAMPLE"])
```

## Containers and execution

- [Tables](https://shareddata.net/docs/core/table/): structured NumPy records and primary-key lookup.
- [Time series](https://shareddata.net/docs/core/timeseries/): timestamped float matrices.
- Collections, Redis caches, Kafka streams and optional S3 integration.
- [HTTP API](https://shareddata.net/docs/guide/swagger/): authenticated remote access.
- [Measured latency](https://shareddata.net/docs/performance/): what the containers cost in practice.

## A store and the fleet that fills it

A routine is any program the Scheduler owns — a feed collector, a parser, a
consolidation job, an API server. A schedule lists the routines that should
stay alive or run at their times. Batch routines wait for their batch dependencies
to complete; realtime routines wait for their realtime dependencies to be running.
Dependencies between the two kinds are not enforced at runtime. Commands travel
through a per-host worker pool that validates the requested source revision
before every launch; routines report heartbeats and lifecycle messages through
the logging plane, and the Scheduler writes queryable state into the store.

- [Orchestration](https://shareddata.net/docs/orchestration/): the routine model, the Scheduler's tick and the worker pool.
- [Use cases](https://shareddata.net/docs/use-cases/): the workloads a production deployment runs, generalized.
- [Schedules](https://shareddata.net/docs/scheduler-store/) and [batch jobs](https://shareddata.net/docs/job-store/): where the state lives.

## Data by symbol

The [semantic data API](https://shareddata.net/docs/data-api/) resolves a
symbol or entity to a dataset and includes provenance. It is a convenience,
not the fast path: producers keep using direct container interfaces for
performance-critical work.

## Client integration

[Client integration](https://shareddata.net/docs/ecosystem/) covers
authenticated API access, the MCP server for compatible agent clients, and
worker access.

## Development and license

Submit changes through the repository's review process. Read the
[development guide](https://shareddata.net/docs/DEVELOPMENT/) and run the
checks relevant to your change; [building and publishing](https://shareddata.net/docs/guide/building/)
explains how the public site is produced and audited. SharedData is released
under the MIT license; see `LICENSE` in the source distribution.
