Metadata-Version: 2.4
Name: pytest_harness
Version: 0.1.6
Summary: Easy-to-use pytest workflow orchestrator with isolated test-file execution, combined coverage, dashboard summaries, and optional per-test-file logs.
Author-email: "R. Wasson" <r.wasson.dev@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/rwasson/pytest_harness
Project-URL: Repository, https://github.com/rwasson/pytest_harness
Project-URL: Issues, https://github.com/rwasson/pytest_harness/issues
Keywords: pytest,testing,coverage,test runner,python,IDE
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.13
Classifier: Operating System :: OS Independent
Classifier: Framework :: Pytest
Classifier: Topic :: Software Development :: Testing
Classifier: Typing :: Typed
Requires-Python: >=3.13
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: pytest>=9.0.0
Requires-Dist: pytest-cov>=5.0.0
Requires-Dist: pytest-json-report
Requires-Dist: pytest-rerunfailures<17,>=16
Requires-Dist: logduo>=0.1.5
Provides-Extra: dev
Requires-Dist: build; extra == "dev"
Requires-Dist: twine; extra == "dev"
Requires-Dist: ruff>=0.15.0; extra == "dev"
Requires-Dist: mypy<3,>=1.10; extra == "dev"
Requires-Dist: vulture; extra == "dev"
Requires-Dist: pre-commit>=3.7.0; extra == "dev"
Dynamic: license-file

pytest_harness
==============

An easy-to-use Python testing workflow orchestrator built around pytest.

Set the project paths and call `pytest_harness()` in a small runner script.
pytest_harness automatically manages isolated test execution, combined coverage,
dashboard output, summary logging, and optional per-test-file logs.

The runner script can be launched with one click from an IDE or run like any
other Python script from Terminal.


Why use pytest_harness?
-----------------------
- no command-line flags, `pyproject.toml` coverage settings, or logging setup required
- test files run independently; if one test file crashes or fails during import,
  collection, or execution, pytest_harness continues to the next test file
- compact color-coded dashboard and plain-text log for the complete run:
    - summary of test results 
    - list of files with import or test collection errors
    - combined coverage
    - optional coverage for each source file
    - output directory and names of created log files
    - exit code
- optional per-test-file logs containing pytest results and output from `print()` statements
    - support visual inspection of test and function outputs
    - display missing (uncovered) source lines
    - can serve as lightweight documentation of tested behavior and passing status
- detailed docstring with complete argument list:  

        from pytest_harness import pytest_harness

        help(pytest_harness)


pytest_harness is a workflow orchestrator, not a pytest plugin.


Tips
----
- pytest_harness_runner:
  - Place pytest_harness_runner.py in your project's test directory.
  - Place test_helper files in a subdirectory inside your project's test directory.
  - pytest_harness uses Logduo internally for its own console output and test-run
    logs. Do not add separate logging setup for the runner or for
    pytest_harness-generated test-file logs.
  - Give your runner script a name that does not start with
    `test_` or end with `_test.py`. PyCharm may treat those names as pytest test
    files rather than as executable runner scripts (right-click won't work).

- test files:
  - Test file names should match `test_*.py`.
  - Test functions may configure and inspect Logduo logging via Duo() when logging behavior
    is part of what the test verifies (see example test below).
  - `print()` statements inside test functions are captured in the test file's individual log and 
     can be used to verify the tested code is executing as intended.
  - `print()` statements outside test functions are not included in the console,
    the summary log, or the test file's individual log.
  - Use pytest's `tmp_path` fixture when tests create temporary files or directories 
    (see example test below).
  - Keep test functions independent and do not rely on test function execution order.
  - Recommendations: 
    - Give test functions descriptive names that include a numeric component
    (e.g., `test_01_verify_add_calculation_and_output_to_console_and_log`).
    - Ordered numeric components make it easier to find and edit flagged tests.

  
Example runner
--------------
    """
    pytest_harness_runner.py

    Run from PyCharm: Right-click on `pytest_harness_runner.py` and select Run.
    Run from Terminal (in tests' directory): `python pytest_harness_runner.py`
    """

    from pathlib import Path

    from pytest_harness import pytest_harness

    # --- Path settings ---
    PROJECT_ROOT = Path(__file__).resolve().parents[1]


    def main() -> None:
        pytest_harness(
            test_file_dir=PROJECT_ROOT / "tests",                 
            log_dir=PROJECT_ROOT / "tests" / "logs",              
            tested_code_dir=PROJECT_ROOT / "src" / "my_project",  
            log_keep=3,                                           
        )


    if __name__ == "__main__":
        main()


* IMPORTANT: `tested_code_dir` identifies the dedicated code directory used
for test imports and coverage analysis.

For example, if `tested_code_dir = PROJECT_ROOT / "src" / "sample_package"`
and the `sample_package` directory contains `calculator.py` (and the recommended empty `__init__.py`),
then the function `add` can be imported from `test_calculator.py` as shown below. 


Example test file (excerpt)
---------------------------
    """
    test_calculator.py
    """
    from pathlib import Path

    import pytest
    from logduo import Duo

    from sample_package.calculator import add

    def test_01_verify_add_calculation_and_output_to_console_and_log(
        tmp_path: Path,
        capsys: pytest.CaptureFixture[str],
    ) -> None:

        result = add(4, 7)
        assert result == 11

        message = f"add(4, 7) returned {result:g}"

        log_dir_path = str(tmp_path / "logs")

        # pytest manages and later removes the temporary directory log_dir_path.
        # Duo() creates an isolated Logduo logger for this test.
        log = Duo()
        log.configure(log_dir_path=log_dir_path)

        log(message)
 
        log_file_path = log.main_log_file_path
        assert log_file_path is not None
    
        log.close()
    
        captured = capsys.readouterr()
        console_output = captured.out + captured.err
        log_content = log_file_path.read_text(encoding="utf-8")
    
        assert message in console_output
        assert message in log_content

        print("test_01_verify_add_calculation_and_output_to_console_and_log")
        print(f"Passed add() assert statement: confirmed add(4, 7) returns expected result = {result}.")
        print("Passed output assert statements: confirmed that expected messages are in console and log.")
        print("\n--- CAPTURED CONSOLE OUTPUT FOR VISUAL INSPECTION ---")
        print(console_output.rstrip())
        print("\n--- LOG CONTENT FOR VISUAL INSPECTION ---")
        print(log_content.rstrip())
        print("")
        print("--- End of test_01 ---")


Example console output
----------------------
- Console output is color-coded (logs are plain text).

    Logging started:  2026-08-03 16:41:48
    Running script:   pytest_harness_runner.py
    pruned run directories: 1 (keep=3)  

    Running 5 test files: ..... done

    Pytest Harness Summary
    ═════════════════════════════════════
    
    Test file summary
    -------------------------------------
    Source files covered:         2
    Test files run:               5
    Test files passed all tests:  2
    
    Test files not processed (check imports) (1):
        test_import_error.py
    
    Test files with no collected tests (1):
        test_empty.py
    
    
    Test function summary
    -------------------------------------
        Passed:     22
        Failed:      1
        Error:       1
        XPassed:     1
        Skipped:     1
        XFailed:     1
    
    Flagged test functions (in 1 test file):
        test_flagged.py
            Failed (1):
                test_03_intentional_fail
            Error (1):
                test_06_intentional_error_during_setup
            XPassed (1):
                test_05_intentional_unexpected_pass
    
    
    Total coverage
    -------------------------------------
    Statements:   96%
    Branches:    100%
    Total:        97%


    Source file  Executed/    Source
    coverage     statements   file
    -----------  -----------  -----------
    95%          21/22        calculator.py
    100%         2/2          __init__.py
    ──────────────────────────────────────────────────────
    Logging ended:    2026-08-03 16:41:48 (duration 00 sec)
    Script path:
        /Users/<my_name>/basic_project/tests/pytest_harness_runner.py
    Output directory:
        /Users/<my_name>/basic_project/tests/logs/pytest_harness_runner/run_2026_08_03__16_41_48
    Log-generated files in output directory:
        pytest_harness_runner.log
        test_calculator.log
        test_empty.log
        test_flagged.log
        test_import_error.log
        test_validation.log
    
    Process finished with exit code 1


Example individual log file (excerpt from test_calculator.log)
--------------------------------------------------------------
    ───────────────────────────────────────────────────────
    test_calculator.log
    Logging started:  2026-08-04 10:03:16
    Generated by:     pytest_harness_runner.py
    ───────────────────────────────────────────────────────

    test_01_verify_add_calculation_and_output_to_console_and_log
    Passed add() assert statement: confirmed add(4, 7) returns expected result = 11.
    Passed output assert statements: confirmed that expected messages are in console and log.
    
    --- CAPTURED CONSOLE OUTPUT FOR VISUAL INSPECTION ---
    Logging started:  2026-08-04 10:27:54
    | INFO     | add(4, 7) returned 11
    ───────────────────────────────────────────────────────
    Logging ended:    2026-08-04 10:27:54 (duration 00 sec)
    Output directory:
        /private/var/folders/_3/4hr4l8_j0xgg2qp127f3bs180000gn/T/pytest-of-<my_name>/pytest-663/
            test_01_verify_add_calculation0/logs/session/run_2026_08_04__10_27_54
    Log-generated files in output directory:
        config_table.txt
        session.log
    
    --- LOG CONTENT FOR VISUAL INSPECTION ---
    ───────────────────────────────────────────────────────
    session.log
    Logging started:  2026-08-04 10:27:54
    ───────────────────────────────────────────────────────
    
    10:27:54.919 | INFO     | add(4, 7) returned 11
    ───────────────────────────────────────────────────────
    Logging ended:    2026-08-04 10:27:54 (duration 00 sec)
    Output directory:
        /private/var/folders/_3/4hr4l8_j0xgg2qp127f3bs180000gn/T/pytest-of-<my_name>/pytest-663/
            test_01_verify_add_calculation0/logs/session/run_2026_08_04__10_27_54
    Log-generated files in output directory:
        config_table.txt
        session.log
    
    --- End of test_01 ---
    ...

    ==================================== PASSES ====================================
    ================================ tests coverage ================================
    _______________ coverage: platform darwin, python 3.13.5-final-0 _______________
    
    Name                               Stmts   Miss Branch BrPart   Cover   Missing
    -------------------------------------------------------------------------------
    src/sample_package/__init__.py         2      0      0      0 100.00%
    src/sample_package/calculator.py      22     10      6      2  50.00%   21, 50, 68, 86-96
    -------------------------------------------------------------------------------
    TOTAL                                 24     10      6      2  53.33%
    ============================= slowest 10 durations =============================
    0.05s call     examples/basic_project/tests/test_calculator.py::test_01_verify_add_calculation_and_output_to_console_and_log
    
    (9 durations < 0.005s hidden.  Use -vv to show these durations.)

    =========================== short test summary info ============================
    PASSED tests/test_calculator.py::test_01_verify_add_calculation_and_output_to_console_and_log
    ...

    pytest exit code: 0
    duration: 0.42 seconds
    ───────────────────────────────────────────────────────
    Logging ended:    2026-08-04 10:18:32
    Script path:
        /Users/<my_name>/basic_project/tests/pytest_harness_runner.py
    Log file path:
        /Users/<my_name>/basic_project/tests/logs/pytest_harness_runner/
            run_2026_08_04__10_18_31/test_calculator.log


- The individual test-file log also includes detailed tracebacks for failed tests.
- The coverage of 53.33% applies to this test file only. Other test files boosted total coverage to 97%.
- The `Missing` source lines can help target code that needs additional tests.
- A complete basic project is available in the GitHub repository under

    `examples/basic_project/`.


Arguments
---------
- **`test_file_dir`**: `pathlib.Path`  
   Existing directory containing pytest test files.

- **`log_dir`**: `pathlib.Path`  
    Directory where pytest_harness creates run logs. The directory is created
    if it does not already exist.

- **`tested_code_dir`**: `pathlib.Path`  
    Existing dedicated directory containing the code targeted by the tests and
    measured for coverage. Its parent is used for imports in each pytest
    subprocess.

- **`include_list`**: `list[str | pathlib.Path] | None`  
    Optional test files or directories to run. Default is None, which discovers
    all matching test files under `test_file_dir`.

- **`exclude_list`**: `list[str | pathlib.Path] | None`  
    Optional test files or directories to exclude after discovery or inclusion.
    Default is None.

- **`individual_logs`**: `bool`  
    Write one detailed log for each selected test file. Default is True.

- **`coverage_warning_threshold`**: `float | None`  
    Warn when total coverage is below this percentage. This does not affect the
    process exit code. Default is 85.0. Use 0 or None to disable the warning.

- **`show_source_file_coverage`**: `bool`  
    Display the source-file coverage table. Default is True.

- **`show_skipped_and_xfailed`**: `bool`  
    Include Skipped and XFailed outcomes in the flagged-test section. Default
    is False.

- **`log_keep`**: `int | None`  
    Number of recent marked run directories to retain. Default is None, which
    disables pruning.

- **`console_theme`**: `str`  
    Console color theme: `"dark"` or `"light"`. Default is `"dark"`.

- **`console_wrap_width`**: `int`  
    Console wrapping width. Must be at least 80. Default is 150.

- **`debug_pytest_harness`**: `bool`  
    Display additional internal diagnostics when a test-file subprocess cannot
    be processed. Default is False.

For complete validation rules and documentation:

    from pytest_harness import pytest_harness

    help(pytest_harness)


Exit Codes
----------
pytest_harness exits with:

    0    the complete selected test run succeeded
    1    the complete selected test run did not succeed

The run exits with code 1 when any of the following occurs:
- one or more test functions Failed
- one or more test functions produced Error
- one or more tests XPassed
- one or more selected test files could not be processed
- one or more selected test files collected no tests

Note:
- Skipped and XFailed outcomes do not by themselves cause exit code 1.
- They also do not trigger tests being listed under `Flagged test functions` 
  unless `show_skipped_and_xfailed=True` (default = False)
