Metadata-Version: 2.5
Name: schift-ko-pii
Version: 0.3.2
Summary: Korean PII detection — 34.1M dual-path LoRA model (person + address + organization)
Project-URL: Homepage, https://github.com/schift-io/schift-ko-pii
Project-URL: Documentation, https://schift.io/docs/pii
Project-URL: Model Card, https://huggingface.co/schift-io/schift-ko-pii-v6
Project-URL: Bug Tracker, https://github.com/schift-io/schift-ko-pii/issues
Author-email: "Schift Inc." <oss@schift.io>
License: Schift License v2.0
        
        Copyright (c) 2026 Schift, Inc. (Room821 Co., Ltd.)
        
        Based on the Apache License, Version 2.0, with the following additional terms.
        
        ------------------------------------------------------------------------
        
        ADDITIONAL TERMS — Revenue Threshold
        
        The rights granted under this License are subject to the following condition:
        
          If the Legal Entity exercising rights under this License (or any entity
          that controls, is controlled by, or is under common control with such
          entity) has annual gross revenue exceeding ten million United States
          dollars (USD $10,000,000) in its most recently completed fiscal year,
          the rights granted under Sections 2 and 3 below do not apply to
          commercial use of the Work or Derivative Works. Such entities must
          obtain a separate commercial license from the Licensor.
        
          "Commercial use" means use of the Work or Derivative Works, directly or
          indirectly, for or in connection with revenue-generating activities,
          including but not limited to: offering the Work as part of a paid
          product or service, using the Work to process data in a commercial
          pipeline, or embedding the Work in commercial software.
        
          Research, education, evaluation, personal projects, and use by
          non-profit organizations are always permitted regardless of revenue.
        
        ------------------------------------------------------------------------
        
        Apache License
                                   Version 2.0, January 2004
                                http://www.apache.org/licenses/
        
           TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
        
           1. Definitions.
        
              "License" shall mean the terms and conditions for use, reproduction,
              and distribution as defined by Sections 1 through 9 of this document,
              together with the Additional Terms above.
        
              "Licensor" shall mean Schift Inc.
        
              "Legal Entity" shall mean the union of the acting entity and all
              other entities that control, are controlled by, or are under common
              control with that entity. For the purposes of this definition,
              "control" means (i) the power, direct or indirect, to cause the
              direction or management of such entity, whether by contract or
              otherwise, or (ii) ownership of fifty percent (50%) or more of the
              outstanding shares, or (iii) beneficial ownership of such entity.
        
              "You" (or "Your") shall mean an individual or Legal Entity
              exercising permissions granted by this License.
        
              "Source" form shall mean the preferred form for making modifications,
              including but not limited to software source code, documentation
              source, and configuration files.
        
              "Object" form shall mean any form resulting from mechanical
              transformation or translation of a Source form, including but
              not limited to compiled object code, generated documentation,
              and conversions to other media types.
        
              "Work" shall mean the work of authorship, whether in Source or
              Object form, made available under the License, as indicated by a
              copyright notice that is included in or attached to the work.
        
              "Derivative Works" shall mean any work, whether in Source or Object
              form, that is based on (or derived from) the Work and for which the
              editorial revisions, annotations, elaborations, or other modifications
              represent, as a whole, an original work of authorship.
        
              "Contribution" shall mean any work of authorship, including
              the original version of the Work and any modifications or additions
              to that Work or Derivative Works thereof, that is intentionally
              submitted to Licensor for inclusion in the Work by the copyright owner
              or by an individual or Legal Entity authorized to submit on behalf of
              the copyright owner.
        
              "Contributor" shall mean Licensor and any individual or Legal Entity
              on behalf of whom a Contribution has been received by Licensor and
              subsequently incorporated within the Work.
        
           2. Grant of Copyright License. Subject to the terms and conditions of
              this License (including the Additional Terms), each Contributor hereby
              grants to You a perpetual, worldwide, non-exclusive, no-charge,
              royalty-free, irrevocable copyright license to reproduce, prepare
              Derivative Works of, publicly display, publicly perform, sublicense,
              and distribute the Work and such Derivative Works in Source or Object
              form.
        
           3. Grant of Patent License. Subject to the terms and conditions of
              this License (including the Additional Terms), each Contributor hereby
              grants to You a perpetual, worldwide, non-exclusive, no-charge,
              royalty-free, irrevocable (except as stated in this section) patent
              license to make, have made, use, offer to sell, sell, import, and
              otherwise transfer the Work, where such license applies only to those
              patent claims licensable by such Contributor that are necessarily
              infringed by their Contribution(s) alone or by combination of their
              Contribution(s) with the Work to which such Contribution(s) was
              submitted. If You institute patent litigation against any entity
              alleging that the Work or a Contribution incorporated within the Work
              constitutes direct or contributory patent infringement, then any
              patent licenses granted to You under this License for that Work shall
              terminate as of the date such litigation is filed.
        
           4. Redistribution. You may reproduce and distribute copies of the
              Work or Derivative Works thereof in any medium, with or without
              modifications, and in Source or Object form, provided that You
              meet the following conditions:
        
              (a) You must give any other recipients of the Work or
                  Derivative Works a copy of this License; and
        
              (b) You must cause any modified files to carry prominent notices
                  stating that You changed the files; and
        
              (c) You must retain, in the Source form of any Derivative Works
                  that You distribute, all copyright, patent, trademark, and
                  attribution notices from the Source form of the Work; and
        
              (d) If the Work includes a "NOTICE" text file as part of its
                  distribution, then any Derivative Works that You distribute must
                  include a readable copy of the attribution notices contained
                  within such NOTICE file.
        
           5. Submission of Contributions. Unless You explicitly state otherwise,
              any Contribution intentionally submitted for inclusion in the Work
              by You to the Licensor shall be under the terms and conditions of
              this License, without any additional terms or conditions.
        
           6. Trademarks. This License does not grant permission to use the trade
              names, trademarks, service marks, or product names of the Licensor,
              except as required for reasonable and customary use in describing the
              origin of the Work and reproducing the content of the NOTICE file.
        
           7. Disclaimer of Warranty. Unless required by applicable law or
              agreed to in writing, Licensor provides the Work (and each
              Contributor provides its Contributions) on an "AS IS" BASIS,
              WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
              implied, including, without limitation, any warranties or conditions
              of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
              PARTICULAR PURPOSE. You are solely responsible for determining the
              appropriateness of using or redistributing the Work and assume any
              risks associated with Your exercise of permissions under this License.
        
           8. Limitation of Liability. In no event and under no legal theory,
              whether in tort (including negligence), contract, or otherwise,
              unless required by applicable law or agreed to in writing, shall
              any Contributor be liable to You for damages, including any direct,
              indirect, special, incidental, or consequential damages of any
              character arising as a result of this License or out of the use or
              inability to use the Work.
        
           9. Accepting Warranty or Additional Liability. While redistributing
              the Work or Derivative Works thereof, You may choose to offer,
              and charge a fee for, acceptance of support, warranty, indemnity,
              or other liability obligations and/or rights consistent with this
              License. However, in accepting such obligations, You may act only
              on Your own behalf and on Your sole responsibility, not on behalf
              of any other Contributor.
        
           END OF TERMS AND CONDITIONS
License-File: LICENSE
Keywords: korean,ner,onnx,pii,privacy,token-classification
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Text Processing :: Linguistic
Requires-Python: >=3.10
Requires-Dist: huggingface-hub>=0.20.0
Requires-Dist: safetensors>=0.4.0
Requires-Dist: torch>=2.0.0
Requires-Dist: transformers>=4.40.0
Description-Content-Type: text/markdown

---
language: ko
license: other
license_name: schift-2.0
license_link: LICENSE
library_name: transformers
pipeline_tag: token-classification
tags:
  - pii
  - korean
  - ner
  - privacy
  - token-classification
datasets:
  - custom
model-index:
  - name: schift-ko-pii-v4
    results:
      - task:
          type: token-classification
          name: Korean PII Detection
        metrics:
          - type: f1
            value: 0.863
            name: F1 (raw, no postprocess)
          - type: f1
            value: 0.968
            name: F1 (with postprocess)
---

# schift-ko-pii-v4

**111M parameter Korean PII detector built on `klue/roberta-base`.**

Fine-tuned with LoRA on Korean legal, financial, and administrative texts. The
active service artifact is the v4 ONNX package.

## Benchmark status

The active release remains v4. The dated report records the v5 candidate
comparison and decision: `docs/2026-08-18-pii-v5-latency-cost.md`.

## Release decision (2026-08-19)

**HOLD — v4 remains active; v5 is a measured candidate, not the active release.**
The v5 candidate scored F1 0.629 raw and 0.929 with postprocessing on the local
93-case suite, and F1 0.8906 on the 100-case hard holdout. Its measured CPU
normal-call latency was 13.74x v4 on average, so the quality result does not yet
justify the latency/cold-start cost. The active Cloud Run service and the checked-in
dist package still reference v4. No v5 release pointer or deployment is implied by
this README.

The complete evidence, exact commands, artifact commit, package metadata, and
service revision are recorded in
`../docs/2026-08-18-pii-v5-latency-cost.md`.

## Quick start

```bash
pip install schift-ko-pii
```

```python
from schift_ko_pii import detect

# Raw model output
spans = detect("피고 김민수의 전화번호는 010-1234-5678이다.")
# [
#   {"start": 3, "end": 6, "label": "private_person", "score": 0.99, "text": "김민수"},
#   {"start": 14, "end": 27, "label": "private_phone", "score": 0.98, "text": "010-1234-5678"},
# ]

# With postprocessing (regex validation + structured-ID rules, higher precision)
spans = detect("주민번호 850205-1234567을 확인.", postprocess=True)
```

### Postprocessing

The `postprocess=True` flag applies Korean-specific rules:

- Regex validation for structured IDs (resident numbers, passports, vehicle plates, bank accounts)
- Luhn/checksum verification where applicable
- Context-aware span merging
- False-positive suppression for legal case numbers and statute references

This raises strong-category F1 from 0.893 to 0.968.

### API (free)

For production use without managing model files:

```python
from schift import Schift

client = Schift(api_key="...")  # free at schift.io
result = client.pii.redact("김민수의 전화번호는 010-1234-5678입니다.")
# Postprocessing is always enabled on the API.
```

## Labels

| Label | Description | Examples |
|---|---|---|
| `private_person` | Person names (Korean, Hanja, foreign) | 김민수, 황보영희, Lee Jenny |
| `private_phone` | Phone numbers | 010-1234-5678, 02-1234-5678 |
| `private_email` | Email addresses | user@example.com |
| `private_address` | Street/postal addresses | 서울특별시 강남구 테헤란로 521 |
| `private_date` | Dates | 2024년 3월 15일, 2024-03-15 |
| `private_url` | URLs, SNS links, IP addresses | instagram.com/user, 192.168.1.1 |
| `account_number` | Structured IDs: resident/business/bank/passport/vehicle | 850205-1234567, M12345678 |
| `secret` | Secrets, API keys, passwords | |

## Benchmark

The benchmark suite is included in this repository under `benchmark/`.

```bash
# Benchmark this model
python benchmark/run_benchmark.py

# With postprocess
python benchmark/run_benchmark.py --postprocess

# Compare any HuggingFace model
python benchmark/run_benchmark.py --hf-model LiquidAI/LFM2.5-Encoder-350M-PII-Detector
```

93 hand-crafted cases covering Korean legal, investigative, and administrative
text. Categories: person (standard/rare/short/foreign/hanja), phone, email,
address (urban/rural), dates, URLs, resident IDs, business IDs, bank accounts,
passports, vehicle plates, IP addresses, plus negative cases (legal case
numbers, statute references, Hanja legal terms).

## Model details

- **Base model**: `klue/roberta-base` (111M params)
- **Training**: LoRA adapter on Korean legal/financial/admin examples
- **Format**: v4 ONNX service artifact / safetensors release source
- **Inference**: `transformers` pipeline, CPU or GPU
- **Max length**: 512 tokens
- **Tagging scheme**: BIES (Begin/Inside/End/Single)

## License

[Schift License v2.0](LICENSE) — Apache 2.0 base with a revenue threshold.
Free for everyone under $10M annual revenue. Research, education, and non-profit
use always permitted. Companies above the threshold: contact hello@schift.io.

## Citation

```bibtex
@software{schift_ko_pii_2026,
  author = {Schift Inc.},
  title = {schift-ko-pii: Korean PII Detection Model},
  year = {2026},
  url = {https://huggingface.co/schift-io/schift-ko-pii-v4},
}
```
