Metadata-Version: 2.4
Name: mash-core
Version: 0.9.0
Summary: Standardized data and RDF made practical
Keywords: RDF,DCAT,standards,serialization,deserialization
Author: Natalie Jakobsen
Author-email: Natalie Jakobsen <natalie@jkbn.no>
License-Expression: MIT
License-File: LICENSE
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: License :: OSI Approved :: MIT License
Requires-Dist: ipykernel>=6.29.5
Requires-Dist: pytest>=8.3.5
Requires-Dist: uvicorn>=0.34.0
Requires-Dist: maplib>=0.20.25
Requires-Dist: polars>=0.20.13
Requires-Dist: pyarrow==23.0.1
Requires-Dist: fastapi[standard]>=0.115
Requires-Dist: pyproj>=3.7.1
Requires-Dist: platformdirs>=4.4.0
Requires-Dist: pyoxigraph>=0.5.2
Requires-Dist: colorama>=0.4.6
Requires-Dist: pydantic-settings>=2.12.0
Requires-Dist: cachetools>=6.2.2
Requires-Dist: azure-storage-blob>=12.27.1
Requires-Dist: azure-identity>=1.25.1
Requires-Dist: smart-open[azure]>=7.6.0
Requires-Dist: linkml>=1.11.1
Requires-Dist: jinja2>=3.1.6
Requires-Dist: pathvalidate>=3.3.1
Maintainer: Natalie Jakobsen
Maintainer-email: Natalie Jakobsen <natalie@jkbn.no>
Requires-Python: >=3.13
Description-Content-Type: text/markdown

# Mash

Mash attempts to solve the mess that is working with data spread between systems and people by making it easy to store your data alongside rich metadata. This brings with it organization, searchability, traceability and many other useful tings.

In practice mash allows you to:

- Search your data by any criteria from metadata
- See where your data originated and its history
- Validate that your data meets the criteria for a specified usage
- Connect to remote data sources while maintaining all of the above

#### Batteries-included

Mash comes pre-loaded with useful tools for working with RDF and grid models. The most helpful things for getting started are the profiles, standards and organizations found in the `grid` module. These can be used to search, validate and describe our data.

#### Standards-based

All models used in mash are based on widely-implemented standard vocabularies like `dcat`, `prof` and `prov-o`, and can be serialized into RDF following those standards.

#### Further documentation

Some deeper documentation can be found here:

- [Introduction to Mash](docs/intro.md)
- [Datasets and distributions](docs/datasets.md)
- [Profiles](docs/profiles.md)
- [Developer guidelines](docs/development.md)

### Examples of usage

(more incoming!)

#### Define new data using datasets and distributions

```Python
import mash

from mash import models
from mash.core.profiles import MetadataProfile
from mash.core.utilities.organizations import make_organization


dist_id = "http://example.com/example#metadata-plaintext"
dataset = models.Dataset(
    identifier="http://example.com/example#metadata",
    title="Example metadata",
    description="Example dataset containing metadata with little",
    conforms_to=[MetadataProfile],  # This is a lie
    information_owner=make_organization("ElBits AS", "931264079"),
    distributions=[models.Distribution(identifier=dist_id, mime_type="text/plain")],
)

mash.save(dataset)
data = b"this is some text"
mash.write_data(data, dist_id)


print(mash.read_data(dist_id).decode())
```


#### Write a SPARQL query towards a graph

```Python
import mash

rdf_dataset = mash.fetch("urn:uuid:5393e12b-96c3-4fab-8b98-d4ba2e940b94")
m = mash.model_from_dataset(rdf_dataset)
res = m.query("""
    PREFIX cim: <http://iec.ch/TC57/CIM100#>
    SELECT (COUNT(?s) as ?substation_count) WHERE {
        ?s a cim:Substation
    }
""")
print(res)
```

## Mash ORM

Mash features a simple RDF ORM which allows you to extract data from RDF graphs into Pydantic BaseModel classes, and to serialize BaseModel classes into RDF graphs. Using the SemanticModel class and Predicate annotation you can define your own rich metadata classes - either based on existing standards or your own models.

#### A basic example

```Python
from typing import Annotated, ClassVar
from mash import models, serialize, Predicate as P


class Person(models.SemanticModel):
    _type: ClassVar = "foaf:Person"
    name: Annotated[str, P("foaf:name")]
    mail: Annotated[str | None, P("foaf:mbox")] = None
    friends: Annotated[list["Person"], P("mash:friends")] = []


kristoffer_robin = Person(identifier="kr", name="Kristoffer Robin")
ole_brumm = Person(identifier="ob", name="Ole Brumm", mail="ole.brum@100meter.skogen", friends=[kristoffer_robin])
turtle_text = serialize(ole_brumm)
print(turtle_text)
```

```
<urn:mash:kr>    a foaf:Person ;
    foaf:name "Kristoffer Robin" .

<urn:mash:ob>    a foaf:Person ;
    mash:friends <urn:mash:kr> ;
    foaf:mbox "ole.brum@100meter.skogen" ;
    foaf:name "Ole Brumm" .
```

#### Making data easier to parse with ReverseRelations

Normally in RDF some relations might go in directions that are not pracitical. If you want everything attached to one class rather than to point in all directions from different classes, you can achieve this with ReverseRelations.

Notice how in the previous example `Substation` has no references to `VoltageLevel` or `SubstationPart` since those actually contain the reference to `Substation`.

```Python
from typing import Annotated, ClassVar

import mash

from mash import Predicate as P, ReverseRelation
from mash.grid import get_latest_nemo
from tests.grid.profiles.nemo import IdentifiedObject, VoltageLevel, SubstationPart, NemoProfile


class SubstationEmbed(IdentifiedObject):
    _type: ClassVar = "cim:Substation"
    substation_kind: Annotated[list[str], P("elb:Substation.substationKind")]
    substation_parts: Annotated[list[SubstationPart], ReverseRelation("elb:SubstationPart.Substation")]
    voltage_levels: Annotated[list[VoltageLevel], ReverseRelation("cim:VoltageLevel.Substation")]


latest_nemo = get_latest_nemo()
m = mash.model_from_dataset(latest_nemo)
with NemoProfile:
    substations = mash.instantiate(m, model=SubstationEmbed)
print(substations[0:2])
```

### Use namespaces and models from a profile

You might have conflicting namespaces (links between a prefix like `cim` and its full URI like `http://iec.ch/TC57/CIM100`) in different contexts, and you might have different models that represent a resource with the same type.

Using profiles you can group models and namespaces, and tell mash when to use them. This avoids conflicts and confusion in the code.

Looking at a part of the example above, you will see the context manager that activates a profile (`with NemoProfile:`). Within this block mash will use the namespaces placed on Nemo, and will recognize Nemo models when relevant.

In this particular example the namespaces are important, since the `grid` module and the `NemoProfile` have different URI's for the prefix `elb`.

```Python
latest_nemo = get_latest_nemo()
m = mash.model_from_dataset(latest_nemo)
with NemoProfile:
    substations = mash.instantiate_all(m, model=SubstationEmbed)
```

## Data catalog

To open a visual catalog of the data mash has access to, simply run `uv run poe catalog` and a page will open in your browser.
