Metadata-Version: 2.1
Name: caikit-nlp-client
Version: 0.0.2
Summary: caikit-nlp client
Author-email: Vaibhav Jain <vajain@redhat.com>, Laurentiu Bradin <laurentiu.bradin@ibm.com>, Daniele Trifirò <dtrifiro@redhat.com>
License: Apache License 2.0
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: grpcio-reflection==1.59.2
Requires-Dist: requests==2.31.0
Provides-Extra: dev
Requires-Dist: ruff==0.1.5; extra == "dev"
Requires-Dist: grpcio-tools==1.59.2; extra == "dev"
Requires-Dist: caikit_nlp_client[types]; extra == "dev"
Provides-Extra: types
Requires-Dist: mypy==1.7.0; extra == "types"
Requires-Dist: types-requests==2.31.0.10; extra == "types"
Requires-Dist: types-protobuf==4.24.0.4; extra == "types"
Provides-Extra: tests
Requires-Dist: pytest; extra == "tests"
Requires-Dist: pytest-cov; extra == "tests"
Requires-Dist: pytest-docker; extra == "tests"
Requires-Dist: pytest-mock; extra == "tests"
Requires-Dist: coverage[toml]; extra == "tests"
Requires-Dist: grpcio-health-checking; extra == "tests"
Requires-Dist: caikit[runtime-grpc,runtime-http]<0.25.0,>=0.23.2; extra == "tests"

# caikit-nlp-client

A client library for [`caikit-nlp`](https://github.com/caikit/caikit-nlp)

## Installation

Install from [PyPi](https://pypi.org/project/caikit-nlp-client/)

```bash
pip install caikit-nlp-client
```

## Usage

A few examples follow, see [`example.py`](/examples/example.py)

### http

To use the http protocol

```python
from caikit_nlp_client import HttpClient

host = "localhost"
port = 8080
model_name = "flan-t5-small-caikit"
http_client = HttpClient(f"http://{host}:{port}")

text = http_client.generate_text(model_name, "What is the boiling point of Nitrogen?")
```

### gRPC

To use the gRPC protocol

```python
from caikit_nlp_client import GrpcClient

host = "localhost"
port = 8085
model_name = "flan-t5-small-caikit"
grpc_client = GrpcClient(f"http://{host}:{port}", insecure=True)
# or, with https:
grpc_client = GrpcClient(f"https://{host}:{port}")

text = grpc_client.generate_text(model_name, "What is the boiling point of Nitrogen?")
```

### Self-signed certificates

To use a self signed certificate, assuming we have a certificate authority cert `ca.pem`

```python
http_client = HttpClient(f"https://{host}:{http_port}", ca_cert_path="ca.pem")

with open("ca.pem", "rb") as fh:
    ca_cert = fh.read()
grpc_client = GrpcClient(host, grpc_port, ca_cert=ca_cert)
```

### mTLS

Assuming we have a `client.pem` and `client-key.pem` certificate files, and we require `ca.pem` to validate the server certificate:

```python
# http
http_client = HttpClient(
    f"https://{host}:{http_port}",
    ca_cert_path="ca.pem",
    client_cert_path="client.pem",
    client_key_path="client-key.pem"
)

# grpc
with open("ca.pem", "rb") as fh:
    ca_cert = fh.read()
with open("client.pem", "rb") as fh:
    client_cert = fh.read()
with open("client-key.pem", "rb") as fh:
    client_key = fh.read()

grpc_client = GrpcClient(
    host,
    port,
    ca_cert=ca_cert,
    client_key=client_key,
    client_cert
)
```

## Contributing

Set up [`pre-commit`](https://pre-commit.com) for linting/style/misc fixes:

```bash
pip install pre-commit
pre-commit install
```

This project uses [`nox`](https://github.com/wntrblm/nox) to manage test automation:

```bash
pip install nox
nox --list  # list available sessions
nox --python 3.10 -s tests # run tests session for a specific python version
```

### Testing against a real caikit+tgis stack

Tests are run against a mocked instance of a TGIS backend by default. To test against a real
caikit+tgis stack, it is sufficient to run using the `--real-caikit` flag when running `pytest`:

```bash
nox -s tests -- --real-caikit tests
```

_Notes:_

- The required images (`caikit-tgis-serving`, `text-generation-inference`), so it could take a while for tests to start while
  compose is pulling the required images, it may seem like the tests are hanging.
- This uses a real model ([google/flan-t5-small](https://huggingface.co/google/flan-t5-small)), which will be downloaded
  to `tests/fixtures/resources/flan-t5-small-caikit` and is around 300MB in size.
