Metadata-Version: 2.4
Name: openapi2rdf
Version: 1.0.0
Summary: Generates OWL, SHACL shapes and SKOS concepts from OpenAPI documents, with the schema mapping of jsonschema2rdf
License-Expression: EUPL-1.2
License-File: LICENSE
Author: Mathias Vanden Auweele
Author-email: mathias@matdata.eu
Requires-Python: >=3.12
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: File Formats
Requires-Dist: jsonschema2rdf (>=1.1.0,<2)
Requires-Dist: pyyaml (>=6.0)
Requires-Dist: xsd2rdf (>=2.1.0,<3)
Project-URL: Changelog, https://gitlab.com/era-europa-eu/private/OPDU/SOT/era-ontology/telematics/openapi2rdf/-/blob/main/CHANGELOG.md
Project-URL: Homepage, https://gitlab.com/era-europa-eu/private/OPDU/SOT/era-ontology/telematics/openapi2rdf
Project-URL: Repository, https://gitlab.com/era-europa-eu/private/OPDU/SOT/era-ontology/telematics/openapi2rdf
Description-Content-Type: text/markdown

# openapi2rdf

[![License](https://img.shields.io/badge/license-EUPL_1.2-blue)](http://data.europa.eu/eli/dec_impl/2017/863/oj)

Converts the schemas of an OpenAPI document into SHACL shapes, an OWL ontology
and SKOS concept schemes, with the mapping model of
[xsd2rdf](https://gitlab.com/era-europa-eu/public/interoperable-data-programme/era-ontology/xsd2rdf).

The schemas of an OpenAPI document (`components/schemas`) are JSON Schema:
OpenAPI 3.0 uses a dialect of draft 4/5, OpenAPI 3.1 JSON Schema 2020-12. So
openapi2rdf is jsonschema2rdf (its converter is a subclass of jsonschema2rdf's)
with what OpenAPI adds: where the named schemas are, `nullable`,
`x-extensible-enum`, `discriminator`, names for the inline schemas of the
operations, and the ontology metadata from `info`. Everything else, and every
option, is that of jsonschema2rdf and so of xsd2rdf: the same shape IRIs,
labels, constraints and SKOS schemes, the same `-k` naming policy and the same
output files.

## Installation

Python 3.12 or later.

```bash
pip install openapi2rdf
```

From source:

```bash
python -m pip install poetry
poetry install
```

openapi2rdf depends on `jsonschema2rdf>=1.1.0,<2`, `xsd2rdf>=2.1.0,<3` and
PyYAML, all installed from PyPI.

## Usage

```bash
openapi2rdf convert path/to/openapi.yaml -o out
```

writes, as xsd2rdf does:

- `openapi.yaml.shape.ttl`: the SHACL shapes
- `openapi.yaml.owl.ttl`: the OWL ontology
- `openapi.yaml.skos.<Name>.ttl`: one SKOS concept scheme per enum
- `openapi.yaml.provenance.ttl`: with `--source-provenance` only

The document may be YAML or JSON. A folder converts each of its `*.yaml`,
`*.yml` and `*.json` files on its own. A file that is not an OpenAPI document
is converted as a JSON Schema.

```bash
openapi2rdf validate path/to/openapi.yaml data.ttl -o out
```

validates RDF data against the converted shapes (the `validate` of xsd2rdf).

From Python:

```python
from openapi2rdf import convert

result = convert("openapi.yaml", use_namespace="http://data.europa.eu/949/", property_kind_qualified_iris=True)
result.write("out")
```

## Options

The options and flags are those of jsonschema2rdf (see its README), which are
those of xsd2rdf where the setting is the same: `-u/--namespace`, `-k`,
`--functional-properties` (off by default), `-n`, `-p`, `-a`,
`--source-provenance`, `--no-pruning`, `--license`, `--date`, `-s`, `--strict`,
`-d`, `-q`. `--license IRI` sets the licence of the output, the object of
`cc:license` on each SKOS concept scheme; the default is the EUPL 1.2 that
xsd2rdf writes (`http://data.europa.eu/eli/dec_impl/2017/863/oj`), and xsd2rdf
puts no licence on the ontology header. An OpenAPI document often states its
own licence in `info.license` (OSDM: Apache 2.0):

```bash
openapi2rdf convert OSDM-online-api.yml --license https://www.apache.org/licenses/LICENSE-2.0
```

The default namespace is `http://example.com/` (an OpenAPI document has no
`$id`).

## Mapping rules

The schema mapping is that of jsonschema2rdf: an object schema is a node shape
and a class, a property a property shape and an OWL property, an enum a SKOS
concept scheme, and so on; see the mapping rules in the jsonschema2rdf README.
OpenAPI adds these rules. Each has a fixture in
[`tests/fixtures`](tests/fixtures/README.md) with its golden output and
instance data that must (not) conform.

### The document

| OpenAPI | Result |
|---|---|
| `components/schemas/<Name>` | a named schema `<Name>`, converted whether an operation uses it or not |
| `definitions/<Name>` (Swagger 2.0) | a named schema `<Name>` |
| `$ref: '#/components/schemas/<Name>'` | a reference to that named schema (a class, a scheme or a value space, as in jsonschema2rdf) |
| `$ref` into another file | that file is loaded; its component schemas are converted when the document uses them (`--no-pruning`: always) |
| inline object, enum or array schema of an operation's request body | named `<OperationId>Request` |
| inline schema of a response | named `<OperationId>Response<status>` (`SearchOffersResponse200`) |
| a body with several media types | named once, after the first |
| an operation without `operationId`, parameters, headers | not converted |
| the root of the document | not a schema |
| `info.title`, `info.description`, `info.version` | `dct:title`, `dct:description` and `owl:versionInfo` of the ontology; `rdfs:label` `Ontology generated from <title> OpenAPI` |

### OpenAPI keywords

| OpenAPI | SHACL / OWL / SKOS |
|---|---|
| `nullable: true` (OpenAPI 3.0) | null is no value in RDF: a required nullable property gets no `sh:minCount`; also next to `allOf: [$ref]` |
| `enum: [A, B]` | a closed list: a concept scheme, and the property takes concepts of that scheme only (`sh:class skos:Concept`, `sh:nodeKind sh:IRI` and a `skos:inScheme` constraint, as in jsonschema2rdf) |
| `x-extensible-enum: [A, B]` | an open list: a concept scheme with the known codes, as an `enum`, but a code that is not listed is valid too. The property shape keeps `sh:nodeKind sh:IRI` and links the scheme with `rdfs:seeAlso` (an annotation SHACL does not check); it has no `sh:class skos:Concept`, no `sh:in` and no `skos:inScheme` constraint. The OWL property keeps `rdfs:range skos:Concept` |
| an enum value written as `{CODE: description}` | concept with notation `CODE` and `skos:definition` the description |
| an enum value written as an object with `code` (or `value`, `id`, `notation`) | concept with that notation; `description` (or `definition`, `note`, `comment`) is its `skos:definition` |
| `discriminator.propertyName` (Swagger 2.0: `discriminator: name`) | the property says which subtype an object is, which is `rdf:type` in RDF: it gets no property shape, in the base schema nor in the schemas that extend it |
| a subtype: `allOf: [{$ref: Base}, {...}]` | `rdfs:subClassOf <Base>`, `sh:node` of the base shape (jsonschema2rdf) |
| `oneOf` of subtypes | `sh:xone` of their shapes (jsonschema2rdf) |
| `discriminator.mapping` | not used: the class of a value is its `rdf:type` |
| `format: int32`, `int64`, `float`, `double`, `byte` | `xsd:int`, `xsd:long`, `xsd:float`, `xsd:double`, `xsd:base64Binary` (jsonschema2rdf) |
| `deprecated: true` on a property, also next to a `$ref` | `owl:deprecated true` on the property shape, and on the OWL property when every property shape with its IRI is deprecated (jsonschema2rdf); the shape still validates |
| `deprecated: true` on a component schema | `owl:deprecated true` on the class and its node shape (jsonschema2rdf) |
| `deprecated: true` on an operation | ignored, as operations are |
| `readOnly`, `writeOnly`, `example`, `externalDocs`, `xml` | ignored |

An `x-extensible-enum` stays a SKOS concept scheme because its codes are
concepts like those of an `enum`, and data uses the same concept IRIs for both
(`<ns>concepts/FareType_ADMISSION`). What changes is the property shape: a new
code (`<ns>concepts/FareType_SUPPLEMENT`) is in no scheme and has no
`rdf:type skos:Concept` in the data, so `sh:class skos:Concept` would reject it
as the scheme check would; only the kind of value (an IRI) is checked.
`rdfs:seeAlso` keeps the link from the property shape to the known codes
without validating; `dct:conformsTo` would claim more than holds. The OWL range
`skos:Concept` is no constraint (OWL infers, it does not reject).

## Provenance

With `--source-provenance`, `<document>.provenance.ttl` links each shape,
class, property and concept scheme to the schema it was made from with
`dct:source`: the file and a JSON Pointer into it, for example
`OSDM-online-api.yml#/components/schemas/Offer/properties/offerId`, and each
property shape to its JSON name (`jsonschema2rdf:jsonName`), as in
jsonschema2rdf.

## Tests

```bash
poetry install
poetry run pytest
```

- `tests/test_fixtures.py`: the [fixture suite](tests/fixtures/README.md), one
  folder per OpenAPI rule. After an intended output change:
  `poetry run pytest tests/test_fixtures.py --update-golden`, then review the diff.
  Regenerate the catalogue with `python tests/fixture_support.py`.
- `tests/test_golden.py`: the real OSDM OpenAPI document of the ERA telematics
  ontology, see [tests/golden](tests/golden/README.md).

The fixtures are also published as a documentation site on GitLab Pages,
built by the `pages` job on `main`: one page per case with the OpenAPI source,
the golden output, the instance data and UML-like class diagrams of the source,
the SHACL shapes and the OWL ontology, plus the mapping rules above and a
summary of the OSDM golden test. The site is at
[https://era-europa-eu.gitlab.io/private/OPDU/SOT/era-ontology/telematics/openapi2rdf/](https://era-europa-eu.gitlab.io/private/OPDU/SOT/era-ontology/telematics/openapi2rdf/)
(Deploy → Pages in GitLab shows the address if the project uses a unique Pages
domain; the site is visible to whoever can see the project). To build it
locally (Mermaid then loads from a CDN):

```bash
poetry run python docs/build_site.py public
```

## Releasing

Releases are published to [PyPI](https://pypi.org/project/openapi2rdf/) from GitLab CI.

1. Update `CHANGELOG.md`: move the changes to a new `[X.Y.Z] - YYYY-MM-DD`
   section and merge it to `main`.
2. In GitLab, create a release (Deploy → Releases → New release) with a new
   tag `vX.Y.Z` created from `main`. The release notes can go in its
   description.
3. The tag pipeline runs lint and the tests, sets the package version from the
   tag (`v1.2.3` gives `1.2.3`), builds the wheel and the sdist, and uploads
   them to PyPI with trusted publishing: no PyPI password or token is stored in
   GitLab. Only tags of the form `vX.Y.Z` (or `X.Y.Z`) build and publish.

openapi2rdf depends on jsonschema2rdf from PyPI: release jsonschema2rdf first
when openapi2rdf needs a new version of it (for 1.0.0, jsonschema2rdf 1.1.0 must
be on PyPI before openapi2rdf 1.0.0 is tagged).

The PyPI project has a GitLab trusted publisher with these settings (before
the first release, when the project does not exist on PyPI yet, add it as a
pending publisher under Your account → Publishing):

| Setting | Value |
|---|---|
| Namespace | `era-europa-eu/private/OPDU/SOT/era-ontology/telematics` |
| Project name | `openapi2rdf` |
| Top-level pipeline file path | `.gitlab-ci.yml` |
| Environment name | (empty) |

## License

[EUPL 1.2](http://data.europa.eu/eli/dec_impl/2017/863/oj)


