Metadata-Version: 2.4
Name: swe-internship-scraper
Version: 1.0.0rc6
Summary: Scrape official ATS job boards for software engineering internships.
Author: Furkan Candar
License-Expression: MIT
Project-URL: Homepage, https://github.com/FurkanSource/swe-internship-scraper
Project-URL: Documentation, https://github.com/FurkanSource/swe-internship-scraper#readme
Project-URL: Issues, https://github.com/FurkanSource/swe-internship-scraper/issues
Project-URL: Changelog, https://github.com/FurkanSource/swe-internship-scraper/blob/main/CHANGELOG.md
Keywords: internships,jobs,scraper,ats,greenhouse,lever,ashby,workday,smartrecruiters,oracle
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests<3,>=2.32
Provides-Extra: browser
Requires-Dist: scrapling[fetchers]==0.4.15; extra == "browser"
Dynamic: license-file

# SWE Internship Scraper

[![CI](https://github.com/FurkanSource/swe-internship-scraper/actions/workflows/ci.yml/badge.svg)](https://github.com/FurkanSource/swe-internship-scraper/actions/workflows/ci.yml)
[![Provider health](https://github.com/FurkanSource/swe-internship-scraper/actions/workflows/provider-health.yml/badge.svg)](https://github.com/FurkanSource/swe-internship-scraper/actions/workflows/provider-health.yml)
[![PyPI](https://img.shields.io/pypi/v/swe-internship-scraper.svg)](https://pypi.org/project/swe-internship-scraper/)

A command-line tool for finding software engineering internships on official public job boards.

## Quick start

**You do not need to clone or download this repository to use the scraper.** Install
the package with Python 3.10–3.14, then run a small sample scan. The CSV file is
written to the folder where you run the command.

Windows PowerShell:

```powershell
py -m pip install --pre swe-internship-scraper
py -m swe_scraper scan --quick --output internships.csv
```

macOS or Linux:

```sh
python3 -m venv .venv
.venv/bin/python -m pip install --pre swe-internship-scraper
.venv/bin/python -m swe_scraper scan --quick --output internships.csv
```

Open `internships.csv` in a spreadsheet. `--quick` checks the small set of monitored
boards; it is a sample, not the full catalog. Omit `--quick` for the default
priority boards, or use `--target-set all` for the entire catalog. The `--pre`
install flag is needed while the first release is a release candidate; it can be
removed for stable `1.0.0`.

### Know what your scan covers

| Mode | Configured employer boards |
| --- | ---: |
| `--quick` | 18 monitored sample boards |
| Default | 72 priority boards |
| `--target-set all` | 1,261 boards across six providers |

These are catalog counts, not guarantees that every board is available or has
internships. Matching uses English job titles; generic program names can be
missed. Employer coverage is uneven, and a remote label does not establish
eligibility to work from every country. Location filters use text matching.

Scans retain validated records from incomplete boards by default. Use
`--complete-boards-only` to discard those boards. JSON retains failure details;
CSV contains only job rows. Every scan reports incomplete coverage: exit 1 means
results were saved despite provider failures;
exit 2 means provider errors left no matching jobs, or configuration was invalid.
A complete scan with zero matches exits 0. See the
[command guide](docs/command-line.md) and [release roadmap](docs/release-roadmap.md).

Want to run the source code you downloaded from GitHub? Follow the
[source checkout guide](docs/getting-started.md). It has exact Windows and
macOS/Linux commands, without requiring PowerShell activation.

## What you install

`swe-internship-scraper` is the main package. It includes the command and six
providers: Greenhouse, Lever, Ashby, Workday, SmartRecruiters, and Oracle.
The separate `swe-scraper-icims` package is an **optional, experimental** plugin
for public iCIMS portals. Most users only need the main package. See the
[plugin guide](docs/icims-plugin.md) if you have an iCIMS portal to scan.

The repository contains scraper code and a public employer catalog. It does not
include an application tracker, personal data, or applicant ranking rules.

## Commands

```powershell
py -m swe_scraper scan --output internships.csv
py -m swe_scraper scan --location "New York" --output nyc-internships.csv
py -m swe_scraper scan --target-set all --output all-internships.csv
py -m swe_scraper providers
py -m swe_scraper --help
```

The installed `swe-scraper` command is equivalent to `py -m swe_scraper` on
Windows or `python3 -m swe_scraper` in a Python environment on macOS/Linux.
For JSON output, use a `.json` filename. See the
[full command guide](docs/command-line.md) for custom targets, watching, health
checks, and deduplication reports.

## Output and deduplication

Schema v2 keeps the familiar primary job fields and adds:

- `sources`: every provider, board namespace, source ID, and direct application URL represented by the record;
- `merge_evidence`: the rule and confidence for each merged source.

Automatic merging requires an exact canonical URL or the same provider, explicit board namespace, and case-sensitive source ID (confidence `1.0`). Similar company, title, or location alone never merges separate requisitions. Similarity candidates remain separate and can be written to the optional deduplication audit. Older exports without a source namespace still support exact URL matching.

Primary record selection uses completeness and stable lexical tie breakers, so thread completion order does not change output. See [the schema reference](docs/output-schema.md) and [the v2 migration guide](docs/schema-v2-migration.md).

## Development

Read the [project guide](PROJECT_GUIDE.md) for contribution checks, support,
security, conduct and compatibility, and the [provider guide](docs/provider-development.md)
for adapter changes.

## License

MIT
