Metadata-Version: 2.4
Name: swe-internship-scraper
Version: 1.0.0rc4
Summary: Scrape official ATS job boards for software engineering internships.
Author: Furkan Candar
License-Expression: MIT
Project-URL: Homepage, https://github.com/FurkanSource/swe-internship-scraper
Project-URL: Documentation, https://github.com/FurkanSource/swe-internship-scraper#readme
Project-URL: Issues, https://github.com/FurkanSource/swe-internship-scraper/issues
Project-URL: Changelog, https://github.com/FurkanSource/swe-internship-scraper/blob/main/CHANGELOG.md
Keywords: internships,jobs,scraper,ats,greenhouse,lever,ashby,workday,smartrecruiters,oracle
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests<3,>=2.32
Provides-Extra: browser
Requires-Dist: scrapling[fetchers]==0.4.15; extra == "browser"
Dynamic: license-file

# SWE Internship Scraper

[![CI](https://github.com/FurkanSource/swe-internship-scraper/actions/workflows/ci.yml/badge.svg)](https://github.com/FurkanSource/swe-internship-scraper/actions/workflows/ci.yml)
[![Provider health](https://github.com/FurkanSource/swe-internship-scraper/actions/workflows/provider-health.yml/badge.svg)](https://github.com/FurkanSource/swe-internship-scraper/actions/workflows/provider-health.yml)
[![PyPI](https://img.shields.io/pypi/v/swe-internship-scraper.svg)](https://pypi.org/project/swe-internship-scraper/)

A command-line tool for finding software engineering internships on official public job boards.

## Quick start

**You do not need to clone or download this repository to use the scraper.** Install
the package with Python 3.10–3.14, then run a small sample scan. The CSV file is
written to the folder where you run the command.

Windows PowerShell:

```powershell
py -m pip install --pre swe-internship-scraper
py -m swe_scraper scan --quick --output internships.csv
```

macOS or Linux:

```sh
python3 -m venv .venv
.venv/bin/python -m pip install --pre swe-internship-scraper
.venv/bin/python -m swe_scraper scan --quick --output internships.csv
```

Open `internships.csv` in a spreadsheet. `--quick` checks the small set of monitored
boards; it is a sample, not the full catalog. Omit `--quick` for the default
priority boards, or use `--target-set all` for the entire catalog. The `--pre`
install flag is needed while the first release is a release candidate; it can be
removed for stable `1.0.0`.

### Know what your scan covers

| Mode | Configured employer boards |
| --- | ---: |
| `--quick` | 18 monitored sample boards |
| Default | 72 priority boards |
| `--target-set all` | 1,292 boards across six providers |

These are catalog counts, not guarantees that every board is available or has
internships. Matching uses English job titles; generic program names can be
missed. Employer coverage is uneven, and a remote label does not establish
eligibility to work from every country. Location filters use text matching.

Scans report failed boards. JSON retains failure details; CSV contains only job
rows. For automation, use `--strict`: exit 1 means partial results were saved;
exit 2 means provider errors left no matching jobs, or configuration was invalid.
A complete scan with zero matches exits 0. See the
[command guide](docs/command-line.md) and [release roadmap](docs/release-roadmap.md).

Want to run the source code you downloaded from GitHub? Follow the
[source checkout guide](docs/getting-started.md). It has exact Windows and
macOS/Linux commands, without requiring PowerShell activation.

## What you install

`swe-internship-scraper` is the main package. It includes the command and six
providers: Greenhouse, Lever, Ashby, Workday, SmartRecruiters, and Oracle.
The separate `swe-scraper-icims` package is an **optional, experimental** plugin
for public iCIMS portals. Most users only need the main package. See the
[plugin guide](docs/icims-plugin.md) if you have an iCIMS portal to scan.

The repository contains scraper code and a public employer catalog. It does not
include an application tracker, personal data, or applicant ranking rules.

## Commands

```powershell
py -m swe_scraper scan --output internships.csv
py -m swe_scraper scan --location "New York" --output nyc-internships.csv
py -m swe_scraper scan --target-set all --output all-internships.csv
py -m swe_scraper providers
py -m swe_scraper --help
```

The installed `swe-scraper` command is equivalent to `py -m swe_scraper` on
Windows or `python3 -m swe_scraper` in a Python environment on macOS/Linux.
For JSON output, use a `.json` filename. See the
[full command guide](docs/command-line.md) for custom targets, watching, health
checks, and deduplication reports.

## Output and deduplication

Schema v2 keeps the familiar primary job fields and adds:

- `sources`: every provider, source ID, and direct application URL represented by the record;
- `merge_evidence`: the rule and confidence for each merged source.

Exact canonical URLs and provider/source identities have confidence `1.0`. Semantic merging requires canonical company, equivalent title including season and year, and equivalent location. Uncertain pairs remain separate and can be written to the optional deduplication audit.

Primary record selection uses completeness and stable lexical tie breakers, so thread completion order does not change output. See [the schema reference](docs/output-schema.md) and [the v2 migration guide](docs/schema-v2-migration.md).

## Development

Read the [project guide](PROJECT_GUIDE.md) for contribution checks, support,
security, conduct and compatibility, and the [provider guide](docs/provider-development.md)
for adapter changes.

## License

MIT
