Metadata-Version: 2.4
Name: furhat-face-recognition
Version: 0.0.2
Summary: Face recognition and identity recognition system for Furhat robots
Author-email: Iro Tochukwu Samuel <irotochukwusamuel@gmail.com>
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: facenet-pytorch==2.5.3
Requires-Dist: furhat_realtime_api==0.1.3
Requires-Dist: einops==0.8.2
Requires-Dist: huggingface_hub==1.23.0
Requires-Dist: numpy
Requires-Dist: opencv-python
Requires-Dist: pillow
Requires-Dist: scikit-learn
Requires-Dist: scipy
Requires-Dist: timm==1.0.27
Requires-Dist: torch
Requires-Dist: torchvision
Requires-Dist: tqdm
Requires-Dist: yacs==0.1.8
Requires-Dist: python-dotenv
Provides-Extra: dev
Requires-Dist: ipython; extra == "dev"
Requires-Dist: ipykernel; extra == "dev"
Requires-Dist: ipywidgets; extra == "dev"
Requires-Dist: jupyterlab; extra == "dev"
Requires-Dist: jupyter_server; extra == "dev"
Requires-Dist: nbconvert; extra == "dev"
Requires-Dist: nbformat; extra == "dev"
Requires-Dist: seaborn; extra == "dev"
Requires-Dist: matplotlib; extra == "dev"
Dynamic: license-file

# (Face Identification + Vision Events)

This project provides:
- Face identification pipelines (InceptionResnetV1 and ViT-based)
- Training/evaluation/tuning scripts
- A real-time `Vision` interface (`interaction.vision.Vision`) that emits user identity events

## 1) Setup (minimal)

### Prerequisites
- Python 3.12
- Conda (recommended) or pip

### Option A: Conda (recommended)
```bash
conda env create -f environment.yml
conda activate dissertation
```

### Option B: pip
```bash
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

## 2) Core commands

Run from repo root:

```bash
python core/models/inception_resnet.py
python core/models/vision_transformer.py
python notebooks/eval.py
python notebooks/vit_eval.py
python core/inference/parameter_tune/resnet_tune.py
python core/inference/parameter_tune/vit_tune.py
```

## 3) Using `Vision` from another project (submodule usage)

If this repo is added as a submodule, import and start `Vision` like this:

```python
import asyncio
from IPython.display import display
from PIL import Image
from recognition.interaction.vision import Vision


async def handle_vision_event(event_data: dict):
    """Callback triggered whenever user state changes."""
    if not event_data:
        return

    event_type = event_data.get("event")
    name = event_data.get("name", "unknown")
    role = event_data.get("role")

    if event_type == "user_changed":
        print(f"[VISION EVENT] User Changed -> Name: {name} | Role: {role}")

        if name == "unknown":
            await system.say(
                "Hello there! I don't believe we've met yet. Could you tell me your name?"
            )
            response = await system.listen(timeout=6.0)
            if response:
                await system.say(f"Nice to meet you, {response}!")

        elif role == "student":
            await system.set_robot_role_mode("student")
            await system.say(
                f"Welcome back, {name}! "
                f"I am ready as your personal development tutor today. "
                f"What would you like to work on?"
            )

        elif role == "staff":
            await system.set_robot_role_mode("staff")
            await system.say(
                f"Hello {name}. "
                f"Staff assistance mode activated. "
                f"How can I assist you today?"
            )

    elif event_type == "user_left":
        print(f"[VISION EVENT] User Left -> Name: {name}")
        if name and name != "unknown":
            await system.say(f"Goodbye {name}, have a nice day!")


system = Vision(host="127.0.0.1", callback=handle_vision_event, use_webcam=True)

# In Jupyter/async environments, create a task instead of blocking.
monitoring_task = asyncio.create_task(system.start_monitoring())

await asyncio.sleep(3)
if system.processed_image:
    display(Image.open(system.processed_image))
```

### `Vision` constructor
- `Vision(host="localhost", callback=None, use_webcam=False)`

### Event payloads
- `user_changed`: includes `name`, `role`, `score`, `previous_user`
- `user_left`: includes `name`, `role`

## 4) Add this repo as a Git submodule

From your parent project root:

```bash
git submodule add <YOUR_REPO_URL> external/dissertation-codebase
git commit -m "Add dissertation codebase submodule"
```

Clone later with submodules:

```bash
git clone --recurse-submodules <PARENT_PROJECT_URL>
```

If already cloned without submodules:

```bash
git submodule update --init --recursive
```

Use it in Python (example):

```python
import sys
from pathlib import Path

sys.path.append(str(Path("external/dissertation-codebase").resolve()))
from recognition.interaction.vision import Vision
```

## 5) Notes
- SLURM shell scripts (`train_*.sh`, `eval_*.sh`, `tune_*.sh`) are cluster-oriented and may need local path/env updates.
- Ensure your dataset paths and model checkpoints in `recognition/config.py` match your machine.
