Metadata-Version: 2.5
Name: splitpilot
Version: 0.1.1
Summary: An explainable toolkit for designing and validating machine learning evaluation splits.
Project-URL: Homepage, https://github.com/krishgupta129/splitpilot
Project-URL: Repository, https://github.com/krishgupta129/splitpilot
Project-URL: Issues, https://github.com/krishgupta129/splitpilot/issues
Author: Krish Gupta
License: MIT
License-File: LICENSE
Keywords: cross-validation,data-leakage,data-science,dataset,machine-learning,mlops,train-test-split
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Requires-Python: >=3.10
Requires-Dist: pandas>=2.0
Requires-Dist: scikit-learn>=1.4
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == 'dev'
Description-Content-Type: text/markdown

# SplitPilot

**An explainable toolkit for designing and validating machine-learning
evaluation splits**

SplitPilot is a Python package for choosing, executing, and validating
train/test splitting strategies based on the structural characteristics
of a dataset.

Instead of treating `train_test_split()` as a universal solution,
SplitPilot considers signals such as repeated entities, temporal
information, and grouping structure to recommend a more appropriate
evaluation strategy.

> **Project status:** Early development / alpha\
> **License:** MIT\
> **Python:** 3.10+

------------------------------------------------------------------------

## Table of Contents

1.  [Project Overview](#1-project-overview)
2.  [The Problem](#2-the-problem)
3.  [Project Objective](#3-project-objective)
4.  [Core Idea](#4-core-idea)
5.  [How SplitPilot Works](#5-how-splitpilot-works)
6.  [Supported Split Strategies](#6-supported-split-strategies)
7.  [Why Split Strategy Matters](#7-why-split-strategy-matters)
8.  [Architecture](#8-architecture)
9.  [Project Structure](#9-project-structure)
10. [Installation](#10-installation)
11. [Quick Start](#11-quick-start)
12. [Recommendation Workflow](#12-recommendation-workflow)
13. [SplitResult](#13-splitresult)
14. [Validation and Leakage Checks](#14-validation-and-leakage-checks)
15. [Testing](#15-testing)
16. [Design Decisions](#16-design-decisions)
17. [Current Limitations](#17-current-limitations)
18. [Roadmap](#18-roadmap)
19. [Example Use Cases](#19-example-use-cases)
20. [Frequently Asked Questions](#20-frequently-asked-questions)
21. [Interview Questions and
    Answers](#21-interview-questions-and-answers)
22. [Contributing](#22-contributing)
23. [License](#23-license)

------------------------------------------------------------------------

# 1. Project Overview

SplitPilot addresses an important but frequently overlooked part of
machine-learning experimentation:

> **How should a dataset be divided so that model evaluation reflects
> the way the model will actually be used?**

A random split is convenient, but convenience does not guarantee a valid
evaluation.

For example, if a dataset contains multiple observations for the same
customer, randomly distributing rows can place the same customer in both
training and testing data. The model may then benefit from information
about an entity during training that also appears in the test set.

Likewise, if the data represents events over time, randomly mixing older
and newer observations can allow future information to influence
evaluation of the past.

SplitPilot therefore treats dataset splitting as an **evaluation-design
problem**, not merely a preprocessing operation.

------------------------------------------------------------------------

# 2. The Problem

A conventional workflow often looks like this:

``` text
Raw Dataset
    |
    v
train_test_split()
    |
    v
Train Model
    |
    v
Evaluate Model
```

The problem is that the default random split may not respect the
structure of the data.

Consider a customer transaction dataset:

  Customer   Date       Transaction   Churn
  ---------- -------- ------------- -------
  A          Jan 01             100       0
  A          Jan 15             150       0
  A          Feb 20             120       1
  B          Jan 03              80       0
  B          Mar 02             210       1

A random row-level split could place observations from customer A into
both training and test sets.

That produces an evaluation question that may not match the intended
real-world question.

Instead of:

> "Can the model generalize to unseen customers or future observations?"

the experiment may effectively ask:

> "Can the model predict another observation from an entity it has
> already seen?"

These are different evaluation problems.

------------------------------------------------------------------------

# 3. Project Objective

The primary objective of SplitPilot is to make dataset splitting more
deliberate and explainable.

The package is designed around three principles:

### 3.1 Inspect dataset structure

Identify useful structural signals such as:

-   repeated entity identifiers
-   temporal columns
-   potential grouping columns
-   target structure

### 3.2 Recommend an evaluation strategy

Use those signals to recommend an appropriate splitting approach.

### 3.3 Execute and validate the split

Produce train/test datasets while enforcing the selected structural
constraints.

The intended workflow is:

``` mermaid
flowchart TD
    A[Dataset] --> B[Inspect Structure]
    B --> C[Detect Repeated Entities]
    B --> D[Detect Temporal Information]
    C --> E[Recommendation]
    D --> E
    E --> F[Select Split Strategy]
    F --> G[Execute Split]
    G --> H[Validate Evaluation Boundaries]
    H --> I[Train and Evaluate Model]
```

------------------------------------------------------------------------

# 4. Core Idea

SplitPilot separates two related but different responsibilities:

  Responsibility       Purpose
  -------------------- ---------------------------------------------------
  **Recommendation**   Decide which splitting strategy is appropriate
  **Splitting**        Actually construct train/test datasets
  **Validation**       Check whether important boundaries were respected
  **Explanation**      Tell the user why a strategy was recommended

This distinction is important because a splitting library should not
simply return arrays. It should help the user understand **why the split
exists in its particular form**.

------------------------------------------------------------------------

# 5. How SplitPilot Works

At a high level:

``` mermaid
flowchart LR
    A[DataFrame] --> B[Profiler]
    B --> C[Recommender]
    C --> D[Recommendation]
    D --> E[DatasetSplitter]
    E --> F[SplitResult]
    F --> G[Validation]
```

The current package is organized around several components:

  Component                Role
  ------------------------ ------------------------------------------------
  `Profiler`               Examines dataset characteristics
  `Recommender`            Produces a split recommendation
  `DatasetSplitter`        Executes the requested split
  `SplitResult`            Provides a structured result
  `Recommendation` model   Represents recommendation information
  `Pilot`                  Provides the higher-level user-facing workflow

The exact implementation is intentionally modular so that recommendation
logic and splitting logic can evolve independently.

------------------------------------------------------------------------

# 6. Supported Split Strategies

SplitPilot currently supports the following strategies.

## 6.1 Random Split

The conventional row-level train/test split.

``` text
Rows
 |
 +-- Training
 |
 +-- Testing
```

Conceptually:

``` python
train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42
)
```

### Appropriate when

-   observations can reasonably be treated as independent
-   there is no important entity boundary
-   there is no meaningful temporal ordering
-   random sampling represents the intended deployment scenario

### Risk

If repeated entities or temporal dependencies exist, a random split can
create leakage or an overly optimistic evaluation.

------------------------------------------------------------------------

## 6.2 Group Split

Group splitting keeps all observations belonging to the same group on
one side of the split.

Example:

``` text
Customer A ───────────────> TRAIN
Customer B ───────────────> TRAIN
Customer C ───────────────> TEST
Customer D ───────────────> TEST
```

For customer-level data, the group column might be:

``` python
group_column="customer_id"
```

The important property is:

``` text
TRAIN groups ∩ TEST groups = ∅
```

### Appropriate when

-   observations belong to entities
-   the model should generalize to unseen entities
-   repeated observations exist for the same entity

------------------------------------------------------------------------

## 6.3 Group-Stratified Split

Group-stratified splitting attempts to maintain a target-related
distribution while still separating groups.

The implementation creates a group-level target summary and uses that
summary to form strata before splitting groups.

Conceptually:

``` text
Individual observations
        |
        v
Aggregate by group
        |
        v
Group-level target distribution
        |
        v
Stratified group split
        |
        +-------- TRAIN GROUPS
        |
        +-------- TEST GROUPS
```

### Appropriate when

-   groups must remain isolated
-   target distribution matters
-   the dataset contains enough groups to support stratification

### Important consideration

The current implementation uses a median-based binary stratum derived
from the group-level target mean. This is a practical initial approach,
not a universal stratification algorithm.

------------------------------------------------------------------------

## 6.4 Time Split

Time splitting respects chronological ordering.

``` text
PAST ------------------------------------> FUTURE

|---------------- TRAIN -----------------|--- TEST ---|
```

The implementation:

1.  converts the specified time column to datetime
2.  rejects invalid dates
3.  sorts observations chronologically
4.  creates the train/test boundary according to `test_size`

Example:

``` python
strategy="time"
time_column="transaction_date"
```

### Appropriate when

-   future predictions are the real deployment scenario
-   historical data is used to predict later observations
-   temporal ordering contains meaningful information

### Core principle

The test set should represent a later period than the training set.

------------------------------------------------------------------------

## 6.5 Group-Time Split

Group-time splitting combines entity separation with chronological
ordering.

This strategy is useful when both conditions matter:

1.  groups should not appear in both train and test
2.  the split should respect time

Conceptually:

``` text
                         TIME
              -------------------------------->

TRAIN GROUPS     |-----------------------------|

TEST GROUPS                                   |--------|
                 ^                             ^
                 |                             |
            train boundary               future test period
```

The current implementation requires a boundary that satisfies both group
isolation and strict chronology. It checks that the resulting train
observations occur before the test observations and that no group is
present on both sides.

If the dataset's group/time structure makes those constraints impossible
to satisfy, the strategy raises a clear error rather than silently
producing an invalid evaluation split.

This strategy was specifically tested for:

-   no group overlap
-   chronological ordering
-   leakage prevention

------------------------------------------------------------------------

# 7. Why Split Strategy Matters

A model score is only meaningful relative to the evaluation design that
produced it.

Suppose two experiments report:

  Experiment   Split      Accuracy
  ------------ -------- ----------
  A            Random          94%
  B            Group           82%

It would be incorrect to immediately conclude that experiment A produced
a better model.

The two experiments may be answering different questions.

A useful mental model is:

``` text
Model Performance
       |
       v
Evaluation Protocol
       |
       v
Split Strategy
       |
       v
Assumptions About Real-World Data
```

Therefore, SplitPilot focuses on making the evaluation protocol
explicit.

------------------------------------------------------------------------

# 8. Architecture

The current source architecture follows a `src` layout:

``` text
splitpilot/
|
+-- src/
|   |
|   +-- splitpilot/
|       |
|       +-- __init__.py
|       |
|       +-- core/
|       |   +-- __init__.py
|       |   +-- pilot.py
|       |   +-- profiler.py
|       |   +-- recommender.py
|       |   +-- splitter.py
|       |
|       +-- models/
|           +-- __init__.py
|           +-- recommendation.py
|
+-- tests/
|   +-- test_splitter.py
|
+-- CHANGELOG.md
+-- LICENSE
+-- README.md
+-- pyproject.toml
```

The `src/splitpilot` directory is the actual Python package. The outer
`splitpilot` directory is the project/repository root.

This structure helps keep package source code separate from
project-level files.

------------------------------------------------------------------------

# 9. Project Structure

  -------------------------------------------------------------------------------
  Path                                        Purpose
  ------------------------------------------- -----------------------------------
  `src/splitpilot/core/pilot.py`              High-level package workflow

  `src/splitpilot/core/profiler.py`           Dataset structure analysis

  `src/splitpilot/core/recommender.py`        Strategy recommendation

  `src/splitpilot/core/splitter.py`           Train/test split execution

  `src/splitpilot/core/validator.py`          Split-quality validation and
                                              leakage checks

  `src/splitpilot/models/recommendation.py`   Recommendation data model

  `tests/test_integration.py`                 Integration tests

  `tests/test_pilot.py`                       High-level Pilot tests

  `tests/test_splitter.py`                    Splitter test suite

  `pyproject.toml`                            Package metadata and build
                                              configuration

  `README.md`                                 Project documentation

  `CHANGELOG.md`                              Version/change history

  `LICENSE`                                   MIT license
  -------------------------------------------------------------------------------

------------------------------------------------------------------------

# 10. Installation

## From the source repository

Clone the repository and install the package in editable mode:

``` bash
git clone https://github.com/krishgupta129/splitpilot.git
cd splitpilot
pip install -e .
```

For development dependencies:

``` bash
pip install -e ".[dev]"
```

The development extra currently includes `pytest`.

## Dependencies

The package currently depends on:

-   Python 3.10+
-   pandas
-   scikit-learn

------------------------------------------------------------------------

------------------------------------------------------------------------

## Public API

The main public objects are available directly from `splitpilot`:

``` python
from splitpilot import (
    Pilot,
    DatasetSplitter,
    SplitResult,
    SplitValidator,
    ValidationReport,
)
```

For most users, `Pilot` is the recommended high-level entry point.

# 11. Quick Start

A high-level workflow can be used through `Pilot`.

``` python
import pandas as pd
from splitpilot import Pilot

df = pd.read_excel("customer_churn.xlsx")

pilot = Pilot(
    df,
    target="churn"
)

recommendation = pilot.recommend()

print(recommendation)

result = pilot.split()

print(result.X_train.shape)
print(result.X_test.shape)
print(result.strategy)

report = pilot.validation()
print(report.summary())
```

A recommendation may identify a strategy such as:

``` text
SplitRecommendation(
    strategy='group_time',
    ...
    group_column='customer_id',
    time_column='transaction_date'
)
```

The exact recommendation depends on the structure of the supplied
dataset.

------------------------------------------------------------------------

# 12. Recommendation Workflow

The recommended workflow is:

``` mermaid
sequenceDiagram
    participant U as User
    participant P as Pilot
    participant F as Profiler
    participant R as Recommender
    participant S as Splitter

    U->>P: Provide DataFrame and target
    P->>F: Inspect dataset
    F-->>P: Structural signals
    P->>R: Evaluate signals
    R-->>P: SplitRecommendation
    U->>P: Request split
    P->>S: Execute recommended strategy
    S-->>P: SplitResult
    P-->>U: Train/test data
```

This workflow separates **decision-making** from **execution**.

That makes the package easier to test and makes recommendations easier
to explain during experimentation.

------------------------------------------------------------------------

# 13. SplitResult

The splitter returns a `SplitResult` dataclass containing:

  Attribute    Type             Meaning
  ------------ ---------------- -------------------
  `X_train`    `pd.DataFrame`   Training features
  `X_test`     `pd.DataFrame`   Testing features
  `y_train`    `pd.Series`      Training target
  `y_test`     `pd.Series`      Testing target
  `strategy`   `str`            Strategy used

Example:

``` python
result = pilot.split()

print(result.X_train)
print(result.X_test)
print(result.y_train)
print(result.y_test)
print(result.strategy)
```

The result also supports tuple-style unpacking:

``` python
X_train, X_test, y_train, y_test = result
```

This provides a familiar interface while retaining metadata through the
structured result object.

------------------------------------------------------------------------

# 14. Validation and Leakage Checks

A useful split should be validated rather than trusted simply because
the function executed successfully.

For grouped data, an important validation is:

``` python
train_ids = set(result.X_train["customer_id"])
test_ids = set(result.X_test["customer_id"])

print(train_ids.intersection(test_ids))
```

Expected result:

``` text
set()
```

This verifies that no customer appears in both partitions.

For temporal splits, inspect the date ranges:

``` python
print(result.X_train["transaction_date"].min())
print(result.X_train["transaction_date"].max())

print(result.X_test["transaction_date"].min())
print(result.X_test["transaction_date"].max())
```

The expected relationship is:

``` text
max(train_time) <= min(test_time)
```

when the evaluation design requires a strictly chronological boundary.

### Validation philosophy

SplitPilot treats successful execution and valid evaluation as related
but distinct concepts:

``` text
Function runs successfully
          |
          v
Correct shapes?
          |
          v
Correct group boundaries?
          |
          v
Correct temporal ordering?
          |
          v
Acceptable evaluation design
```

------------------------------------------------------------------------

# 15. Testing

The splitter has a dedicated `pytest` test suite.

The current test run reports:

``` text
26 passed
```

The tested areas include:

  Test                             Purpose
  -------------------------------- -----------------------------------------
  Random split shapes              Checks expected train/test dimensions
  Group split overlap              Verifies group separation
  Group-stratified split overlap   Verifies group separation
  Time split chronology            Verifies temporal ordering
  Group-time split overlap         Verifies group separation
  Group-time chronology            Verifies temporal ordering
  Group-time leakage prevention    Verifies structural isolation
  Missing target                   Validates error handling
  Invalid test size                Validates parameter checking
  Missing group column             Validates group configuration
  Missing time column              Validates temporal configuration
  Unknown strategy                 Validates unsupported strategy handling

Run the full test suite with:

``` bash
pytest
```

The current project test suite contains **26 tests**, covering the
splitter, high-level `Pilot` API, validation/error handling, and
integration behavior.

A successful run currently reports:

``` text
26 passed
```

------------------------------------------------------------------------

# 16. Design Decisions

## 16.1 Why use a recommendation layer?

A user should not need to manually remember every condition under which
a random split becomes questionable.

The recommendation layer converts structural observations into an
explicit suggestion.

This also makes the reasoning inspectable:

``` text
Dataset
  |
  +-- repeated customer_id
  |
  +-- transaction_date detected
  |
  v
Recommendation:
group_time
```

------------------------------------------------------------------------

## 16.2 Why return a dataclass?

Returning a raw tuple is familiar, but it loses useful metadata.

A `SplitResult` provides:

-   named outputs
-   explicit strategy information
-   easier debugging
-   clearer documentation
-   compatibility with tuple-style unpacking

------------------------------------------------------------------------

## 16.3 Why validate columns before splitting?

Failing early is preferable to allowing a cryptic downstream error.

For example:

``` python
if target not in df.columns:
    raise ValueError(...)
```

The same principle is applied to group and time columns.

------------------------------------------------------------------------

## 16.4 Why validate temporal values?

A column may exist while containing invalid dates.

Therefore, the time-based implementation converts values with:

``` python
pd.to_datetime(..., errors="coerce")
```

and raises an explicit error if invalid values are detected.

This prevents silent corruption of the chronological split.

------------------------------------------------------------------------

# 17. Current Limitations

SplitPilot is intentionally an early-stage project. Its current behavior
should therefore be understood in that context.

### 17.1 Recommendation heuristics are not universal truth

A recommendation is a structured suggestion based on detected dataset
characteristics. It does not replace domain knowledge.

### 17.2 Group-stratification is currently simplified

The current implementation creates a group-level target summary and uses
a median-based binary stratum.

This is useful for the initial implementation, but more sophisticated
target-distribution strategies may be appropriate for some datasets.

### 17.3 Group-time behavior depends on the data structure

Combining grouping and chronology is inherently more complicated than a
conventional random split.

The implementation currently uses ordered groups derived after sorting
by the supplied time column. Different real-world longitudinal
structures may require more specialized strategies.

### 17.4 Small datasets can impose practical constraints

Strategies involving groups or stratification require enough groups and
sufficiently varied target information to construct meaningful
partitions.

### 17.5 Recommendation does not guarantee absence of every form of leakage

SplitPilot focuses on structural split boundaries. Feature engineering,
preprocessing, target construction, duplicated information, and other
parts of an ML pipeline can also introduce leakage.

The split is one part of a broader evaluation protocol.

------------------------------------------------------------------------

# 18. Roadmap

Potential future development areas include:

  Area                    Possible improvement
  ----------------------- ----------------------------------------------------
  Recommendation engine   More robust dataset-structure heuristics
  Group stratification    More flexible target-distribution handling
  Temporal splitting      More sophisticated longitudinal strategies
  Validation              Dedicated automated leakage diagnostics
  Diagnostics             Human-readable validation reports
  API                     More configurable splitting policies
  Testing                 Broader edge-case and property-based testing
  Documentation           More real-world datasets and case studies
  Packaging               Continued PyPI release improvements
  CI                      Automated testing across supported Python versions

The roadmap is intentionally open. Future features should be driven by
real evaluation problems rather than adding complexity for its own sake.

------------------------------------------------------------------------

# 19. Example Use Cases

## 19.1 Customer churn

**Dataset structure**

-   multiple transactions per customer
-   transaction timestamps
-   customer-level prediction target

Potential recommendation:

``` text
group_time
```

because both customer boundaries and temporal ordering may matter.

------------------------------------------------------------------------

## 19.2 Medical records

If several records belong to the same patient, a random row-level split
may allow the same patient to appear in both partitions.

Potential strategy:

``` text
group
```

with:

``` python
group_column="patient_id"
```

The appropriate choice ultimately depends on the intended deployment
scenario.

------------------------------------------------------------------------

## 19.3 Financial forecasting

When historical observations are used to predict future outcomes:

``` text
time
```

may be more appropriate than a random split.

Example:

``` python
time_column="date"
```

------------------------------------------------------------------------

## 19.4 Independent tabular observations

For genuinely independent observations where no meaningful entity or
temporal boundary exists:

``` text
random
```

may be appropriate.

The key is not to avoid random splitting. The key is to avoid using it
automatically when the dataset structure says otherwise.

------------------------------------------------------------------------

# 20. Frequently Asked Questions

## What is SplitPilot?

SplitPilot is an explainable Python toolkit for designing and validating
machine-learning evaluation splits.

Its goal is to help users select a train/test strategy based on dataset
structure rather than defaulting to a random split.

------------------------------------------------------------------------

## Is SplitPilot a machine-learning library?

No.

SplitPilot does not train models or replace libraries such as
scikit-learn.

Its focus is the **evaluation-split layer** of the machine-learning
workflow.

------------------------------------------------------------------------

## Does SplitPilot replace `train_test_split()`?

No.

Random splitting is still a valid strategy when the dataset assumptions
support it.

SplitPilot provides additional strategies and a recommendation layer for
situations where ordinary random sampling may not represent the intended
evaluation scenario.

------------------------------------------------------------------------

## What problem does SplitPilot solve?

It addresses structural problems in train/test evaluation, particularly:

-   repeated entities
-   group overlap
-   temporal ordering
-   potential evaluation leakage caused by inappropriate partitioning

------------------------------------------------------------------------

## What is data leakage in this context?

Data leakage occurs when information that should not be available to the
model during evaluation influences the training process.

A common structural example is placing observations from the same entity
into both training and testing partitions when the intended task is
generalization to unseen entities.

SplitPilot reduces this particular class of risk by enforcing
appropriate group or temporal boundaries.

------------------------------------------------------------------------

## Why not always use random splitting?

Because rows are not always independent.

If ten rows represent ten measurements from one customer, randomly
splitting those rows is different from splitting ten independent
customers.

The correct split depends on the question the model is supposed to
answer.

------------------------------------------------------------------------

## What is the difference between group and group-time splitting?

### Group split

Separates entities:

``` text
TRAIN: Customer A, B, C
TEST:  Customer D, E
```

### Group-time split

Attempts to respect both entity boundaries and temporal ordering:

``` text
TRAIN: earlier groups
TEST:  later groups
```

The latter is useful when both repeated entities and time are important
to the evaluation design.

------------------------------------------------------------------------

## Does group splitting guarantee no leakage?

No.

It guarantees the specific group-boundary constraint implemented by the
splitter.

Other forms of leakage can still come from:

-   feature engineering
-   preprocessing
-   target-derived features
-   duplicated records
-   improperly constructed labels
-   external information

Therefore, group isolation should be considered one leakage-control
mechanism rather than a complete leakage detector.

------------------------------------------------------------------------

## Does a recommended strategy have to be followed?

No.

The recommendation is guidance.

A domain expert may deliberately choose another strategy if the actual
deployment scenario calls for it.

The important part is that the decision should be intentional and
explainable.

------------------------------------------------------------------------

## Why is explainability important for a data-splitting library?

Because evaluation methodology directly affects model metrics.

If a model achieves 95% accuracy, the next question should be:

> "95% under what evaluation protocol?"

A recommendation that explains its reasoning makes that protocol easier
to understand, reproduce, and discuss.

------------------------------------------------------------------------

## Why use a `src` directory?

The project uses the standard `src` layout:

``` text
project/
└── src/
    └── splitpilot/
```

The repository root contains project-level files, while `src/splitpilot`
contains the actual Python package.

This creates a clear separation between source code and repository
tooling.

------------------------------------------------------------------------

## What does `SplitResult` provide?

`SplitResult` stores:

``` text
X_train
X_test
y_train
y_test
strategy
```

It therefore provides both the split datasets and information about how
the split was generated.

------------------------------------------------------------------------

## Can `SplitResult` still be unpacked like a tuple?

Yes.

The implementation provides `__iter__`, so this is supported:

``` python
X_train, X_test, y_train, y_test = result
```

------------------------------------------------------------------------

## How does SplitPilot handle invalid dates?

The time-based splitter converts the specified time column to datetime.

Invalid values are detected and result in a `ValueError` rather than
silently producing an invalid chronological split.

------------------------------------------------------------------------

## Is SplitPilot production-ready?

Not yet.

The project is currently in early development / alpha.

The current implementation is suitable for development, experimentation,
learning, and continued package engineering, but users should review its
assumptions before relying on it for critical production evaluation
pipelines.

------------------------------------------------------------------------

# 21. Interview Questions and Answers

This section is intentionally written as an interview-preparation guide.
The goal is to explain the project from both a software-engineering and
machine-learning perspective.

------------------------------------------------------------------------

## Q1. Explain your project in one minute.

**Answer:**

SplitPilot is a Python toolkit I built to make machine-learning
train/test splitting more deliberate and explainable.

The basic problem is that random splitting is not always appropriate. If
a dataset contains repeated entities, such as multiple transactions from
the same customer, random row-level splitting can place the same
customer in both training and testing data. Similarly, time-dependent
data should generally respect chronological ordering.

SplitPilot profiles the dataset, recommends a strategy such as random,
group, group-stratified, time, or group-time splitting, and then
executes that strategy while exposing the resulting train/test
partitions through a structured `SplitResult`.

The main idea is that dataset splitting is part of evaluation design,
not just a preprocessing command.

------------------------------------------------------------------------

## Q2. Why did you build SplitPilot?

**Answer:**

I wanted to address a practical machine-learning problem that is easy to
overlook.

A model can have an impressive evaluation score while the evaluation
protocol itself is unrealistic.

Instead of building another model-training wrapper, I focused on the
data-splitting stage because the split determines what kind of
generalization is actually being measured.

------------------------------------------------------------------------

## Q3. Why is `train_test_split()` sometimes insufficient?

**Answer:**

`train_test_split()` performs a random partition by default. That is
appropriate when observations can reasonably be treated as independent
and identically distributed for the intended evaluation.

However, many real datasets contain structure.

For example, multiple rows may belong to the same customer, patient,
device, or account. Other datasets have a meaningful temporal order.

Randomly mixing those observations can produce an evaluation that does
not represent the deployment scenario.

------------------------------------------------------------------------

## Q4. What is group leakage?

**Answer:**

Group leakage in this context occurs when the same entity appears in
both training and testing partitions even though the intended evaluation
requires unseen entities.

For example, if customer A has ten transactions and random splitting
places six transactions in training and four in testing, the model has
already seen customer A during training.

A group split prevents this by ensuring:

``` text
TRAIN groups ∩ TEST groups = ∅
```

------------------------------------------------------------------------

## Q5. What is temporal leakage?

**Answer:**

Temporal leakage occurs when information from the future influences an
evaluation that is supposed to represent prediction of future
observations.

For a forecasting-style problem, training should generally use
historical information and testing should represent later observations.

A time split therefore establishes a chronological boundary rather than
randomly mixing all observations.

------------------------------------------------------------------------

## Q6. Why did you implement both group and time strategies?

**Answer:**

They address different structural assumptions.

A group split answers an entity-generalization question:

> Can the model generalize to groups it has not seen?

A time split answers a temporal-generalization question:

> Can the model use historical data to predict later observations?

Some datasets require both constraints, which motivated the group-time
strategy.

------------------------------------------------------------------------

## Q7. Why is group-time splitting difficult?

**Answer:**

Because two constraints must be satisfied simultaneously.

The split should respect group boundaries while also maintaining
temporal ordering.

A naive implementation can accidentally solve one problem while
violating the other.

That is why I added explicit tests for group overlap, chronology, and
leakage prevention rather than checking only whether the function
returned without an exception.

------------------------------------------------------------------------

## Q8. How did you validate your group-time implementation?

**Answer:**

I wrote tests covering three important properties:

1.  No group appears in both train and test.
2.  The split respects chronological ordering.
3.  The resulting partition prevents the intended structural leakage.

The current splitter test suite contains 12 tests, and the latest run
completed with all 12 passing.

------------------------------------------------------------------------

## Q9. Why return a dataclass instead of a tuple?

**Answer:**

A tuple is convenient, but it does not communicate what each value
represents and does not provide room for metadata.

`SplitResult` gives named fields:

``` text
X_train
X_test
y_train
y_test
strategy
```

At the same time, I preserved tuple-style unpacking through `__iter__`.

So the API provides structure without sacrificing familiarity.

------------------------------------------------------------------------

## Q10. Why does SplitPilot have a profiler and recommender?

**Answer:**

I wanted to separate observation from decision.

The profiler identifies characteristics of the dataset, such as repeated
entities and temporal information.

The recommender uses those signals to produce a strategy recommendation.

That separation makes the architecture easier to test and allows the
recommendation logic to evolve independently from the actual splitting
implementation.

------------------------------------------------------------------------

## Q11. Is the recommendation always correct?

**Answer:**

No.

It is a recommendation based on detectable structural signals, not a
replacement for domain knowledge.

For example, detecting a customer identifier does not automatically tell
us whether the real deployment scenario involves unseen customers.

The user still needs to understand the prediction task and choose an
evaluation protocol accordingly.

------------------------------------------------------------------------

## Q12. Why did you use pandas and scikit-learn?

**Answer:**

Pandas is the natural interface for tabular dataset manipulation, and
scikit-learn already provides reliable primitives for several splitting
operations.

Instead of reimplementing every low-level operation, SplitPilot builds
an explainable layer around those primitives and adds strategies that
enforce the structural constraints required by the project.

------------------------------------------------------------------------

## Q13. How does the time splitter work?

**Answer:**

It first validates the time column, converts it to datetime, checks for
invalid values, sorts the dataset chronologically, and then calculates a
train/test boundary using the requested test size.

The important property is that the temporal ordering is established
before the partition is created.

------------------------------------------------------------------------

## Q14. Why do you validate missing columns yourself?

**Answer:**

Explicit validation produces clearer errors.

For example, if the target column is missing, I raise:

``` text
Target column '...' was not found in the dataset.
```

This is more useful to the user than allowing a later pandas or
scikit-learn operation to fail with a less contextual error.

------------------------------------------------------------------------

## Q15. What happens if an invalid strategy is provided?

**Answer:**

The splitter raises a `ValueError` identifying the unknown strategy.

This prevents silent fallback to another splitting method.

------------------------------------------------------------------------

## Q16. What was one challenge you encountered while developing the project?

**Answer:**

One important challenge was making group-time splitting satisfy both
entity and temporal constraints.

An implementation can appear reasonable while still producing a
misleading evaluation boundary.

I therefore treated the behavior as something to test explicitly rather
than assuming that a successful function call meant the strategy was
correct.

------------------------------------------------------------------------

## Q17. How would you improve SplitPilot?

**Answer:**

I would improve it in several stages.

First, I would strengthen the recommendation heuristics.

Second, I would make group-stratification more flexible because the
current median-based binary grouping is intentionally simple.

Third, I would add stronger automated leakage diagnostics.

Finally, I would expand testing across more edge cases and Python
versions and add continuous integration.

------------------------------------------------------------------------

## Q18. What would you do if a user disagrees with the recommendation?

**Answer:**

The recommendation should be treated as guidance rather than an enforced
decision.

I would ask what the intended deployment scenario is and determine
whether the recommendation matches that scenario.

For example, repeated customer records could justify group splitting in
one project but not necessarily in another if the model is specifically
intended to personalize predictions for existing customers.

------------------------------------------------------------------------

## Q19. What makes this project different from simply writing a wrapper around scikit-learn?

**Answer:**

The core idea is not merely wrapping an existing function.

The project separates:

``` text
Dataset analysis
       ↓
Strategy recommendation
       ↓
Split execution
       ↓
Validation
```

The emphasis is on making evaluation design explainable and structurally
aware.

The package therefore treats splitting as a decision problem rather than
only an API call.

------------------------------------------------------------------------

## Q20. What is the biggest limitation of the current project?

**Answer:**

The recommendation system is heuristic and the project is still in
alpha.

It can identify useful structural signals, but it cannot understand
every domain-specific reason why a particular evaluation protocol may be
appropriate.

Also, split-level controls cannot eliminate every form of data leakage.

------------------------------------------------------------------------

## Q21. If the model gets 99% accuracy after using SplitPilot, does that mean the model is good?

**Answer:**

Not necessarily.

SplitPilot can help make the evaluation split more appropriate, but
model quality still depends on the entire experimental pipeline.

I would investigate:

-   the deployment scenario
-   target construction
-   feature leakage
-   preprocessing
-   duplicates
-   class imbalance
-   appropriate evaluation metrics
-   temporal and group boundaries

The split is an important part of evaluation validity, but it is not the
entire evaluation protocol.

------------------------------------------------------------------------

## Q22. How would you explain the project to a non-technical interviewer?

**Answer:**

Imagine testing a student using questions from a chapter they already
memorized.

The score might be high, but it would not tell us how well they handle
new material.

Machine-learning datasets can have a similar problem.

If the same customer or future information appears on both sides of the
evaluation, the score may not represent the real problem.

SplitPilot helps design the test so that the evaluation better matches
the question we actually want to answer.

------------------------------------------------------------------------

## Q23. What software-engineering concepts does this project demonstrate?

**Answer:**

The project demonstrates:

-   Python package structure
-   modular architecture
-   dataclasses
-   input validation
-   exception handling
-   unit testing with pytest
-   dependency management
-   Git version control
-   semantic project organization
-   documentation
-   package metadata through `pyproject.toml`

It also demonstrates translating machine-learning methodology into
reusable software.

------------------------------------------------------------------------

## Q24. What would you say if an interviewer asks whether this is "just a small utility"?

**Answer:**

I would agree that the current implementation is relatively focused, but
the problem it addresses is fundamental to machine-learning
experimentation.

The value is not in the number of lines of code. It is in formalizing an
evaluation decision that is often handled informally.

The project also gives me a foundation for extending the system into
automated split diagnostics, leakage analysis, richer recommendations,
and reproducible evaluation reports.

------------------------------------------------------------------------

## Q25. What did you learn from building SplitPilot?

**Answer:**

The main lesson was that machine-learning engineering is not only about
model algorithms.

The reliability of an experiment also depends on how the data is
prepared and evaluated.

Building SplitPilot made me think more carefully about assumptions
behind train/test splitting, software interfaces, validation, testing,
package structure, and how to communicate technical decisions clearly.

------------------------------------------------------------------------

# 22. Contributing

Contributions are welcome.

A typical development workflow is:

``` bash
git clone https://github.com/krishgupta129/splitpilot.git
cd splitpilot
pip install -e ".[dev]"
pytest
```

Before opening a pull request:

1.  Add or update tests for behavioral changes.
2.  Run the test suite.
3.  Update documentation where necessary.
4.  Keep changes focused.
5.  Explain the reasoning behind non-obvious implementation decisions.

------------------------------------------------------------------------

# 23. License

SplitPilot is distributed under the MIT License.

See [`LICENSE`](LICENSE) for the full license text.

------------------------------------------------------------------------

## Project Philosophy

SplitPilot is built around a simple principle:

> **A model evaluation is only as meaningful as the assumptions behind
> its evaluation protocol.**

A train/test split is not merely a line of preprocessing code.

It is a statement about what the model is expected to generalize to:

``` text
Random split
    → independent observations

Group split
    → unseen entities

Time split
    → future observations

Group-time split
    → future observations from structurally isolated groups
```

SplitPilot aims to make those assumptions visible, testable, and
reusable.
