Metadata-Version: 2.5
Name: sentimentdemo
Version: 0.3.3
Summary: A simple sentiment analysis application, published solely as an artifact for the purpose of demonstrating and testing a software bill of materials. Not recommended for any text classification tasks.
Project-URL: homepage, https://github.com/bact/sentimentdemo
Project-URL: source, https://github.com/bact/sentimentdemo.git
Project-URL: download, https://github.com/bact/sentimentdemo/releases
Project-URL: releasenotes, https://github.com/bact/sentimentdemo/releases
Project-URL: documentation, https://github.com/bact/sentimentdemo#readme
Project-URL: issues, https://github.com/bact/sentimentdemo/issues
Author-email: Arthit Suriyawongkul <suriyawa@tcd.ie>
License-Expression: CC0-1.0
License-File: LICENSE
Keywords: ai,aibom,artificial intelligence,bom,machine learning,natural language processing,nlp,sbom,sentiment analysis,software bill of materials,spdx
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: <3.12,>=3.10
Requires-Dist: fasttext-community==0.11.7
Requires-Dist: newmm-tokenizer==0.2.2
Requires-Dist: numpy==1.26.4
Requires-Dist: pitloom>=0.15.0
Requires-Dist: th-simple-preprocessor==0.10.1
Provides-Extra: dev
Requires-Dist: mypy>=2.2.0; extra == 'dev'
Requires-Dist: pylint>=4.0.6; extra == 'dev'
Requires-Dist: pytest>=9.1.1; extra == 'dev'
Provides-Extra: eval
Requires-Dist: scikit-learn==1.6.1; extra == 'eval'
Description-Content-Type: text/markdown

---
SPDX-FileContributor: Arthit Suriyawongkul
SPDX-FileCopyrightText: 2024-present Arthit Suriyawongkul <suriyawa@tcd.ie>
SPDX-FileType: DOCUMENTATION
SPDX-License-Identifier: CC0-1.0
---

# Sentiment Demo: A Simple AI Application and its AI BOM Example

[![PyPI - Version](https://img.shields.io/pypi/v/sentimentdemo)](https://pypi.org/project/sentimentdemo/)
![GitHub License](https://img.shields.io/github/license/bact/sentimentdemo)
[![DOI](https://img.shields.io/badge/doi-10.5281%2Fzenodo.14055332-blue)](https://doi.org/10.5281/zenodo.14055332)

A simple text classification application, published solely to demonstrate
[a software bill of materials (SBOM)](https://en.wikipedia.org/wiki/Software_supply_chain)
in [SPDX 3.0 format](https://spdx.dev/use/specifications/).

**The main content of the package are software bill of materials at
[bom.spdx3.json](./bom.spdx3.json) (written by hand) and
[bom-generated.spdx3.json](./bom-generated.spdx3.json) (Pitloom-generated).**
Other files are given just to complete the illustration.

*Not recommended for actual text classification tasks.*

Updates:

- July 2026: Version 0.3.1 - [SBOM fragments](./fragments/) that were generated
  during the runs of evaluate/predict/preprocess/train.py are merged into the
  main SBOM -- shipped with the Python wheel.
  This gives more data/model provenance.
- March 2026: Adopted as a development reference for the **[Pitloom]**
  SBOM generator.
- May 2025: Added to the **[SPDX Usage Examples][spdx-examples]** repository
  as [ai/example02].

SBOM demonstration design goals:

- **Comprehensible:** Small enough for a human to understand easily.
- **Informative:** Elaborate enough to showcase the use of various information
  fields within an SBOM.
- **Testable:** Designed to facilitate testing and evaluation against specific
  use case requirements.

For more information about implementing AI BOM using SPDX specification,
see *Karen Bennet, Gopi Krishnan Rajbahadur, Arthit Suriyawongkul,
and Kate Stewart,
[“Implementing AI Bill of Materials (AI BOM) with SPDX 3.0: A Comprehensive Guide to Creating AI and
Dataset Bill of Materials”](https://www.linuxfoundation.org/research/ai-bom),
The Linux Foundation, October 2024*.

[spdx-examples]: https://github.com/spdx/spdx-examples
[ai/example02]: https://github.com/spdx/spdx-examples/tree/master/ai/example02
[Pitloom]: https://github.com/bact/pitloom

## Content

```text
.
├── LICENSE                    License information
├── README.md                  This README file
├── bom-generated.spdx3.json   SBOM in SPDX 3 format (generated)
├── bom.spdx3.json             SBOM in SPDX 3 format (handwritten)
├── data                       Dataset, preprocessed and tokenized
│   ├── test.txt               Testing data
│   ├── train.txt              Training data
│   └── valid.txt              Validation data
├── rawdata                    Raw dataset, before preprocessing
│   ├── test                   Testing data
│   │   ├── neg.txt            Testing samples for label "neg" (negative)
│   │   ├── neu.txt            Testing samples for label "neu" (neutral)
│   │   ├── pos.txt            Testing samples for label "pos" (positive)
│   │   └── q.txt              Testing samples for label "q" (question)
│   ├── train                  Training data
│   │   └── ...
│   └── valid                  Validation data
│       └── ...
├── src
│   ├── evaluate.py            Script to evaluate prediction performance
│   ├── model.bin              Sentiment analysis model
│   ├── predict.py             Script to predict a label of a text
│   ├── preprocess.py          Script to prepare training data
│   └── train.py               Script to build a model
└── techdocs                   Technical documentation
    ├── dataprepare.md         Data preparation
    └── instructions.md        Instruction for use
```

[![A diagram showing relationships between elements in the Sentiment Demo package.](./bom.spdx3.png "A diagram showing relationships between elements in the Sentiment Demo package.")](./bom.spdx3.png)

## Usage

See [instruction for use](./techdocs/instructions.md) for how to use the
application.

## Data preparation

See [data preparation](./techdocs/dataprepare.md).

## Generating the SBOM

Run the following commands to generate SBOM fragments at each stage of
the AI development lifecycle.
Finally, use Pitloom to merge these fragments with the core project SBOM into
a single, comprehensive file.

### SBOM fragment 1: Preprocess the data

```shell
python src/sentimentdemo/preprocess.py rawdata data
```

This processes the raw data into the `data/` directory
and generates an SBOM fragment at `fragments/preprocess.spdx3.json`
(as specified by `loom.run` in `preprocess.py`).

### SBOM fragment 2: Train the model

```shell
AUTOTUNE_DURATION=5 python src/sentimentdemo/train.py data/train.txt data/valid.txt src/sentimentdemo/model.bin
```

This step outputs the training fragment to `fragments/train.spdx3.json`.

> Note: We use AUTOTUNE_DURATION=5 to speed up the training process.
> Since the goal of this demo is to illustrate SBOM generation rather than
> produce a highly performant model, a short duration is sufficient.

### SBOM fragment 3: Run predictions

```shell
python src/sentimentdemo/predict.py "test"
```

This step outputs the prediction fragment to `fragments/predict.spdx3.json`.

> Production note: Generating a fragment on every prediction may introduce
> unnecessary overhead. You should typically enable this only during testing
> and packaging, and disable it before deploying to production.

### SBOM fragment 4: Evaluate model

```shell
python src/sentimentdemo/evaluate.py src/sentimentdemo/model.bin data/test.txt
```

This step outputs the evaluation fragment to `fragments/evaluate.spdx3.json`.

### Merge and generate the final SBOM

```shell
pitloom project . --pretty --output bom-generated.spdx3.json
```

Pitloom collects all the generated fragments in the `fragments/` directory
and merges them. The final, consolidated SBOM is saved to
`bom-generated.spdx3.json` in your current working directory.

## Notes

- Development is in the `main` branch.
- The diagram is generated from a PlantUML file:
  [bom.spdx.puml](./bom.spdx3.puml).
  The PlantUML file is generated by
  [spdx3ToGraph](https://github.com/maxhbr/spdx3ToGraph).
  To brevity, spdxIds and long strings are shortened by the
  [shortenid.sh](./tools/shortenid.sh) script in tools/, and all but one
  hyperparameter have been manually removed.
- The energy used by the computer during model training is tracked by
  [energy-tracker](https://github.com/rdegges/energy-tracker).
  It measures how much energy the computer uses during the training.
  This means the actual energy used for training the model might be a bit less
  than the reported amount.
- Both handwritten SBOM and generated SBOM are validated by
  [spdx3-validate](https://github.com/JPEWdev/spdx3-validate),
  which use the JSON Schema from
  <https://spdx.org/schema/3.0.1/spdx-json-schema.json>
  and the SHACL from
  <https://spdx.org/rdf/3.0.1/spdx-model.ttl>.
- Next steps:
  - Add external dependency relationships
    (e.g. `dependsOn`, `hasProvidedDependency`)
  - Using information requirements and obligations in the EU AI Act as a
    target, labeling all relevant properties and relationships with
    corresponding difficulty levels and support levels, based on the
    [BOM Maturity Model](https://scvs.owasp.org/bom-maturity-model/difficulty-levels/).

## Licenses

Apart from the data and components listed in the table below, the code and
content in this repository are dedicated to the public domain under the terms
of Creative Commons Zero ("CC0") 1.0 Universal, which have no copyright and
related or neighboring rights worldwide to the extent allowed by law.

| Component | Name | License | Notes |
| --------- | ---- | ------- | ----- |
| Training data | [Wisesight Sentiment Corpus](https://github.com/PyThaiNLP/wisesight-sentiment) | CC0-1.0 | Samples from the corpus are in `rawdata/`. Preprocessed data is in `data/`. See [data preparation](./techdocs/dataprepare.md) for details. |
| Text preprocessor | [th-simple-preprocessor](https://pypi.org/project/th-simple-preprocessor/) | Apache-2.0 | |
| Word tokenizer | [newmm-tokenizer](https://pypi.org/project/newmm-tokenizer/) | Apache-2.0 | Inherited the license from [PyThaiNLP](https://pypi.org/project/pythainlp/). |
| Text classifier | [fastText](https://fasttext.cc/) | MIT | Use [fasttext-community](https://pypi.org/project/fasttext-community/), which is a community-maintained fork. |
| Array package | [NumPy](https://pypi.org/project/numpy/) | BSD-3-Clause AND 0BSD AND MIT AND Zlib AND CC0-1.0 | |

The specific version information can be found in [pyproject.toml](./pyproject.toml).

## Citation

If you use this software, including its software bill of materials (SBOM),
please cite it as follows:

> Suriyawongkul, Arthit. “Sentiment Demo: A Simple AI Application and Its AI BOM Example”. Zenodo, 8 November 2024. <https://doi.org/10.5281/zenodo.14055332>.

BibTeX:

```bibtex
@software{Suriyawongkul_Sentiment_Demo_A_2024,
    author = {Suriyawongkul, Arthit},
    doi = {10.5281/zenodo.14055332},
    license = {CC0-1.0},
    month = nov,
    title = {{Sentiment Demo: A Simple AI Application and its AI BOM Example}},
    url = {https://github.com/bact/sentimentdemo/},
    version = {0.1},
    year = {2024}
}
```
