Metadata-Version: 2.4
Name: wordtangible
Version: 0.7.0
Summary: Python library for word concreteness and sensory-profile analysis.
License: MIT
License-File: LICENSE
Keywords: nlp,linguistics,concreteness,imageability
Author: Jason Robison
Author-email: jrrobison@protonmail.com
Requires-Python: >=3.10,<4.0
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Dist: nltk (>=3.9.1,<4.0.0)
Project-URL: Homepage, https://github.com/jrrobison1/wordtangible
Project-URL: Repository, https://github.com/jrrobison1/wordtangible
Description-Content-Type: text/markdown

# WordTangible
[![PyPI - Version](https://img.shields.io/pypi/v/wordtangible?link=https%3A%2F%2Fpypi.org%2Fproject%2Fwordtangible%2F)](https://pypi.org/project/wordtangible/) [![DOI](https://zenodo.org/badge/844322355.svg)](https://doi.org/10.5281/zenodo.22635419)

WordTangible is a simple Python library for analyzing the concreteness and sensory profile of words and text. This can be useful for various natural language processing tasks, readability analysis, stylometry, and linguistic research.

Data is pulled from:
- The Brysbaert dataset (Brysbaert et al., 2014)[1]
- The Muraki multiword-expression dataset (Muraki et al., 2023)[2]
- The Glasgow dataset (Scott et al., 2019)[3]
- The MRC Psycholinguistic Database (Coltheart, 1981; machine-usable
  dictionary: Wilson, 1988)[4][5]
- The Lancaster Sensorimotor Norms (Lynott et al., 2020)[7] — for the
  sensory-modality functions

The default rating is a quality-ordered fallback on a 1-5 scale (5 = most
concrete): Brysbaert's raw value when a word is in Brysbaert (the largest
single-word source, natively 1-5), otherwise Muraki (multiword
expressions, same scale and lab lineage), otherwise Glasgow, otherwise
MRC, the latter two linearly rescaled to 1-5. 

The sources are deliberately *not* averaged — their normalized distributions 
have systematically different means, so a linear-rescale average would 
skew multi-source words rather than reduce noise. This way every value is 
a real published rating from a single identifiable study (and ~99% of words 
return Brysbaert's exact published value). The raw per-source ratings, an 
MRC-free variant, and a mean of the available sources are all accessible via 
the `source` parameter (see below).

## Features

- Get concreteness ratings for individual words
- Choose the ratings source: the default fallback, any single dataset
  un-normalized (`brysbaert`, `muraki`, `glasgow`, `mrc`), an MRC-free
  variant (`open`) for commercial use, or a normalized `mean`
- Rated multiword expressions match as units, longest first: "baseball
  bat" and "piece of cake" get their own ratings (2,896 Brysbaert
  compounds + 62,889 Muraki expressions, idioms included) instead of
  blending their constituent words; disable with
  `match_expressions=False` if idiom ratings firing on literal uses
  ("he ate a piece of cake") is a concern
- Unrated inflected forms score as their WordNet lemma ("whales" as
  "whale", "replied" as "reply") — worth ~10 points of token coverage on
  typical fiction; disable with `lemma_fallback=False` for values
  strictly comparable to the published norms
- Calculate average concreteness for a given text
- Compute the ratio of concrete to abstract words in a text (with optional
  add-k smoothing to keep it finite and stable on short texts)
- Report rating coverage — how much of a text the concreteness mean
  actually rests on
- Sensory-modality analysis via the Lancaster Sensorimotor Norms:
  per-word perceptual profiles across six modalities (visual, auditory,
  haptic, interoceptive, gustatory, olfactory), a word's dominant
  modality, and text-level sensory profiles — how visual vs auditory vs
  tactile a text is

## Installation

You can install WordTangible using pip:

```bash
pip install wordtangible
```

## Usage

Here are some basic examples of how to use WordTangible:

```python
from wordtangible import word_concreteness, avg_text_concreteness, concrete_abstract_ratio

# Get concreteness rating for a single word
print(word_concreteness("apple"))  # Output: 5.0 (highly concrete)

# Calculate average concreteness of a text
text = "The abstract concept of love is as tangible as the apple in your hand."
print(avg_text_concreteness(text))  # Output: ~2.9 (mix of concrete and abstract)

# Get the ratio of concrete to abstract words
print(concrete_abstract_ratio(text))  # Output: ~1.0 (balanced concrete and abstract words)

# Smoothed ratio: finite even when a text has no very-abstract words,
# and steadier on short texts (add-k smoothing; default k=0 keeps the
# classic behavior, including float('inf') for the no-abstract case)
print(concrete_abstract_ratio(text, smoothing=1))

# How much of the text the concreteness mean rests on (0.0-1.0):
# a mean at 0.85 coverage is trustworthy; the same mean at 0.12
# (jargon, dialect, names) is noise
from wordtangible import concreteness_coverage
print(concreteness_coverage(text))


```

### Choosing a ratings source

Every function accepts a `source` parameter:

```python
word_concreteness("apple")               # 5.0   — default fallback, 1-5 scale
word_concreteness("apple", "brysbaert")  # 5.0   — raw Brysbaert, 1-5 scale
word_concreteness("apple", "glasgow")    # 6.824 — raw Glasgow CNC, 1-7 scale
word_concreteness("apple", "mrc")        # 620   — raw MRC CNC, 100-700 scale
word_concreteness("apple", "open")       # 5.0   — like default, but never MRC
word_concreteness("apple", "mean")       # 4.78  — mean of available sources, rescaled to 1-5
word_concreteness("piece of cake")       # 2.8   — Muraki multiword expressions, idioms included

avg_text_concreteness(text, source="open")

# Get the concreteness score and average concreteness for text using exactly Brysbaert and nothing else:
word_concreteness("apple", source="brysbaert", lemma_fallback=False)
avg_text_concreteness(
    text,
    include_stopwords=True,
    source="brysbaert",
    lemma_fallback=False,
)
```

- `default` — Brysbaert, else Muraki, else Glasgow, else MRC (the
  latter two rescaled to 1-5).
- `brysbaert` / `muraki` / `glasgow` / `mrc` — one dataset's raw, un-normalized
  values on its native scale; `None` for words it doesn't rate. Useful
  for comparing directly against the published norms. If you pass these
  to `concrete_abstract_ratio`, adjust its thresholds to the source's
  scale.
- `open` — Brysbaert, else Muraki, else Glasgow: excludes the MRC database, whose
  terms are "for research purposes" (see licensing below), making this
  the right choice for commercial products.
- `mean` — the mean of whichever of the four rate the word, each
  linearly rescaled to 1-5. Beware: linear rescaling doesn't fully align
  the scales (the sources' normalized means differ systematically), so
  these values aren't comparable to any single set of published norms.

### Sensory-modality analysis

Concreteness says *how much* a word is tied to experience; the
sensorimotor functions say through which senses:

```python
from wordtangible import (
    word_sensorimotor, dominant_modality,
    avg_text_sensorimotor, modality_profile,
)

word_sensorimotor("thunder")
# {"visual": 0.53, "auditory": 3.71, ..., "dominant": "auditory",
#  "max_perceptual": 3.71, "minkowski3_perceptual": ..., "exclusivity": ...}

dominant_modality("perfume")   # "olfactory"
dominant_modality("velvet")    # "haptic"
dominant_modality("nausea")    # "interoceptive"

# Mean 0-5 perceptual strength per modality across a text
avg_text_sensorimotor(text)
# {"visual": 2.69, "auditory": 1.88, "haptic": 0.98, ...}

# Composition: share of rated words dominated by each modality
modality_profile(text)
# {"visual": 0.63, "interoceptive": 0.18, "auditory": 0.17, ...}
```

These functions accept the same `include_stopwords`, `lemma_fallback`,
and `match_expressions` parameters as the concreteness functions (there
is no `source` — the Lancaster norms are the single source). As a
taste of what this measures: an 1887 cookbook profiles ~4x higher on
gustatory strength than five other reference texts, Austen leads on
interoception (inner states), and Darwin is the most visually dominant.

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

## License

The WordTangible **code** is licensed under the MIT License - see the
[LICENSE](LICENSE) file for details.

### Data sources & licensing

The bundled ratings file (`wordtangible/resources/concreteness_ratings.csv`)
is derived from third-party datasets, and the MIT license above does **not**
apply to that data. Each source has its own terms:

- **Glasgow Norms** (Scott et al., 2019): published open access under a
  [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/)
  license — redistribution of derived data is permitted with attribution,
  which the citation below provides.
- **Brysbaert concreteness norms** (Brysbaert et al., 2014): made freely
  available by the authors (via the *Behavior Research Methods* supplement
  and the Ghent CRR lab) and widely redistributed in research software;
  no formal license accompanies the data, so provenance and citation are
  provided here as is standard practice.
- **Muraki multiword-expression norms** (Muraki et al., 2023): distributed
  by the authors via the paper's OSF repository (osf.io/ksypa), same
  research-community terms as the Brysbaert norms.
- **Lancaster Sensorimotor Norms** (Lynott et al., 2020): published open
  access and distributed via the project's OSF repository (osf.io/7emr6)
  under a [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/)
  license — redistribution of derived data is permitted with
  attribution, which the citation below provides.
- **MRC Psycholinguistic Database** (Coltheart, 1981; Wilson, 1988): the
  database's distribution terms state that it is available **for research
  purposes**. The default ratings fall back to MRC-derived values for the
  few hundred words none of the other sources rate. If you intend to
  use WordTangible in a commercial (non-research) product, pass
  `source="open"` — it draws only on Brysbaert, Muraki, and Glasgow —
  or verify the MRC terms for your use case.

The bundled CSV keeps each source's raw rating in its own column, and
`scripts/build_ratings.py` regenerates it from the original datasets
(downloaded on demand; the raw files are not stored in this repository).

The lemma fallback additionally uses **WordNet** (Miller, 1995), which is
not bundled either: NLTK downloads it on first use, under the permissive
[Princeton WordNet license](https://wordnet.princeton.edu/license-and-commercial-use).

This section documents provenance in good faith and is not legal advice.

## Citing WordTangible

If you use WordTangible in research, please cite both the tool and the
rating datasets your results rest on (for concreteness: all four under
the default source; Brysbaert, Muraki, and Glasgow if you use
`source="open"`; the single dataset if you use a raw source — and the
Lancaster Sensorimotor Norms[7] if you use the sensory-modality
functions). GitHub's "Cite this repository"
button generates a citation from [CITATION.cff](CITATION.cff), or use:

> Robison, J. (2026). *WordTangible* (Version 0.7.0) [Computer software].
> https://github.com/jrrobison1/wordtangible

```bibtex
@software{robison_wordtangible,
  author  = {Robison, Jason},
  title   = {WordTangible},
  version = {0.7.0},
  year    = {2026},
  url     = {https://github.com/jrrobison1/wordtangible}
}
```

If your analysis uses the default lemma fallback (`lemma_fallback=True`),
consider also citing WordNet[6], which provides the lemmatization.

Dataset citations are given in full in the [References](#references)
below.

## References
[1] Brysbaert, M., Warriner, A. B., & Kuperman, V. (2014). Concreteness ratings for 40 thousand generally known English word lemmas. *Behavior Research Methods, 46*(3), 904-911. https://doi.org/10.3758/s13428-013-0403-5

[2] Muraki, E. J., Abdalla, S., Brysbaert, M., & Pexman, P. M. (2023). Concreteness ratings for 62,000 English multiword expressions. *Behavior Research Methods, 55*(5), 2522-2531. https://doi.org/10.3758/s13428-022-01912-6

[3] Scott, G. G., Keitel, A., Becirspahic, M., Yao, B., & Sereno, S. C. (2019). The Glasgow Norms: Ratings of 5,500 words on nine scales. *Behavior Research Methods, 51*(3), 1258-1270. https://doi.org/10.3758/s13428-018-1099-3

[4] Coltheart, M. (1981). The MRC psycholinguistic database. *The Quarterly Journal of Experimental Psychology Section A, 33*(4), 497-505. https://doi.org/10.1080/14640748108400805

[5] Wilson, M. (1988). MRC Psycholinguistic Database: Machine-usable dictionary, version 2.00. *Behavior Research Methods, Instruments, & Computers, 20*(1), 6-10. https://doi.org/10.3758/BF03202594

[6] Miller, G. A. (1995). WordNet: A lexical database for English. *Communications of the ACM, 38*(11), 39-41. https://doi.org/10.1145/219717.219748
<<<<<<< HEAD
=======

[7] Lynott, D., Connell, L., Brysbaert, M., Brand, J., & Carney, J. (2020). The Lancaster Sensorimotor Norms: Multidimensional measures of perceptual and action strength for 40,000 English words. *Behavior Research Methods, 52*(3), 1271-1291. https://doi.org/10.3758/s13428-019-01316-z
>>>>>>> 720131b (Add sensorimotor norms)

