Metadata-Version: 2.4
Name: rdf-fingerprinter
Version: 3.6.1
Summary: Find out kind of data shapes your RDF dataset instantiates.
License: Apache-2.0
License-File: LICENSE
Keywords: rdf,application profile,data shape,statistics,fingerprint,sparql,linked-data
Author: Eugeniu Costetchi
Author-email: costezki.eugen@gmail.com
Maintainer: Eugeniu Costetchi
Maintainer-email: costezki.eugen@gmail.com
Requires-Python: >=3.12,<4.0
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Environment :: Console
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Utilities
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: Science/Research
Classifier: Natural Language :: English
Requires-Dist: click (>=8.5.0,<9)
Requires-Dist: eds4jinja2 (>=2.1,<3)
Requires-Dist: pydantic (>=2.10,<3)
Requires-Dist: rdflib (>=7.0,<8)
Project-URL: Documentation, https://meaningfy-ws.github.io/rdf-fingerprinter/
Project-URL: Homepage, https://github.com/meaningfy-ws/rdf-fingerprinter
Description-Content-Type: text/markdown

# RDF fingerprinter

**Understand** the structure of your RDF data at a glance using automatically built **application profiles** and spot differences between dataset structures. 

An [_application profile_](https://en.wikipedia.org/wiki/Application_profile), in this context, is the set of [_data shapes_](https://www.w3.org/2014/data-shapes/wiki/Main_Page) designed for a particular purpose acting as constraints on how the data are instantiated and so can be used to validate the data.

_Fingerprinting_ is the action of generating, or rather, guessing, the application profile applied to a particular dataset. This is an inductive process of reconstructing the data shape for each class instantiated in the dataset. 

[![ci](https://github.com/meaningfy-ws/rdf-fingerprinter/actions/workflows/ci.yml/badge.svg)](https://github.com/meaningfy-ws/rdf-fingerprinter/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/meaningfy-ws/rdf-fingerprinter/branch/master/graph/badge.svg)](https://codecov.io/gh/meaningfy-ws/rdf-fingerprinter)
[![docs](https://img.shields.io/badge/docs-GitHub%20Pages-blue)](https://meaningfy-ws.github.io/rdf-fingerprinter/)

![PyPI](https://img.shields.io/pypi/v/rdf-fingerprinter?color=teal&label=version)
![PyPI - Status](https://img.shields.io/pypi/status/rdf-fingerprinter)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/rdf-fingerprinter)
![PyPI - License](https://img.shields.io/pypi/l/rdf-fingerprinter?color=green)
![PyPI - Wheel](https://img.shields.io/pypi/wheel/rdf-fingerprinter)

**Documentation:** https://meaningfy-ws.github.io/rdf-fingerprinter/

# Installation
RDF fingerprinter may be installed with pip as follows. 
 
```
pip install rdf-fingerprinter
```
Python 3.12 or later is required (tested on 3.12, 3.13 and 3.14).

**Upgrading from 0.3 or 1.x?** The library was rewritten in 2.0 and has grown since; read the
breaking changes per release in the [CHANGELOG](CHANGELOG.md) and the
[Getting Started](https://meaningfy-ws.github.io/rdf-fingerprinter/rdf-fingerprinter/getting-started.html)
upgrade notes.

# What it does

- **Profiles any SPARQL 1.1 endpoint, read-only**, class by class with bounded queries, and says how
  precise every number is (exact, or first N of M when a class is too big).
- **Writes** an HTML report and a saved `profile.json`, from which SHACL shapes, enriched ShExC,
  PlantUML diagrams and a pattern schema for language models are generated without querying again.
- **Scans for modelling pitfalls** (`--pitfalls`, 28 SPARQL checks, most from the OOPS! catalogue)
  and fails a CI job on them (`--fail-on`).
- **Recognises reifications** (`rdf:Statement`, singleton properties, `owl:Axiom`, PROV, Wikibase,
  RDF 1.2 triple terms, …) and shows the relations they stand for.

How it compares with SHACL Play, sheXer, void-generator, QSE and others:
[Comparison](https://meaningfy-ws.github.io/rdf-fingerprinter/rdf-fingerprinter/comparison.html).

# Usage

The easiest way to build a fingerprint of a SPARQL endpoint is by calling the fingeprinting CLI command and write the report in an output folder.
 
```shell script
fingerprint -e http://my.sparql.endpoint.com -o my/output/folder
``` 

To use the fingerprinter programmatically, see the
[API Reference](https://meaningfy-ws.github.io/rdf-fingerprinter/rdf-fingerprinter/api-reference.html). 

## Schemas for LLMs

To give a language model the shape of your data before it writes SPARQL, render the saved profile as
a **pattern schema**: five files per graph (`prefixes-*.md`, `patterns-*.md`, `patterns-*.csv`,
`glossary-*.md`, `glossary-*.csv`) with compact `(:Class)-property-> (:Target)` lines, statistics, and
labels, alt labels, definitions and notes. Or as **enriched ShExC** (the same content as ShEx shapes):

```shell script
fingerprint-gen-pattern my/output/folder/profile.json -o llm/ --ontology vocab.ttl --max-classes 30
fingerprint-gen-shex    my/output/folder/profile.json -o llm/ --min-instances 10
```

Give a model that writes SPARQL `patterns-*.md` and `prefixes-*.md`; filter the CSV files with grep or
DuckDB. On the test data, the patterns Markdown of a graph is about five times smaller than its SHACL
shapes (2,081 against 10,950 characters).

Leave out whole namespaces, in every output (report, SHACL, ShExC, PlantUML, pattern schema),
by prefix, prefix glob or IRI stem; repeat the option to combine them, and use
`--include-namespace` to bring one back:

```shell script
fingerprint-gen-pattern profile.json -o llm/ --exclude-namespace relationid --exclude-namespace 'prv*'
fingerprint-gen-doc     profile.json -o out/ --exclude-namespace 'https://my.brainframe.com/#/knowledge/'
```

The report keeps every namespace in its table, marks the excluded ones, and has a switch per
namespace that hides its classes and property rows on the page. See
[Namespace filters](https://meaningfy-ws.github.io/rdf-fingerprinter/rdf-fingerprinter/representations.html#filters)
and
[Schemas for LLMs](https://meaningfy-ws.github.io/rdf-fingerprinter/rdf-fingerprinter/llm-schema.html).

# Development

Requires Poetry, Docker and (for `make check-spec`) Node.js.

```shell script
make install                    # Poetry env with the test and lint groups
make check-all                  # lint, types, layers, OpenSpec, parallel tests with coverage floors — the gate CI runs
make check-release              # check-all + release tests + random properties + e2e; before a release
make oxigraph-load-test-data    # in-memory Oxigraph on :7878 with the test graphs
make test-e2e                   # end-to-end tests against it (local only)
```

See [Development](https://meaningfy-ws.github.io/rdf-fingerprinter/rdf-fingerprinter/development.html).

# Contributing
You are more than welcome to help expand and mature this project. We adhere to [Apache code of conduct](https://www.apache.org/foundation/policies/conduct), please follow it in all your interactions on the project.   

When contributing to this repository, please first discuss the change you wish to make via issue, email, or any other method with the maintainers of this repository before making a change.

## Licence 
This project is licensed under [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).
