Metadata-Version: 2.1
Name: hugging-quik
Version: 0.0.1
Summary: helper package to make loading/unloading huggingface models quik-er
Home-page: https://github.com/donchesworth/hugging-quik
Author: Don Chesworth
Author-email: donald.chesworth@gmail.com
License: BSD
Platform: UNKNOWN
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3.8
Description-Content-Type: text/markdown
Requires-Dist: torch (>=1.7.0)
Requires-Dist: pandas (>=1.1)
Requires-Dist: transformers (==4.16.2)
Requires-Dist: sentencepiece
Requires-Dist: pytest
Requires-Dist: pytest-cov
Requires-Dist: codecov
Requires-Dist: numpy (>=1.21.0)
Requires-Dist: bump2version

# hugging-quik
A helper package to make loading/unloading huggingface models quik-er

This was originally part of [pytorch-quik](https://github.com/donchesworth/pytorch-quik), but its requirements are large, and hugging-quik is a simple way to pull and use huggingface.co pretrained models and tokenizers.


hugging-quik does the following:
    - get_tokenizer: pull a tokenizer from the huggingface suite of models
    - save_tokenizer: save the tokenizer to disk
    - get_encodings: use the tokenizer to get text encodings
    - get_pretrained_model: pull a model from the huggingface suite
    - save_pretrained_model: save the model to disk


