Metadata-Version: 2.4
Name: searchdiff
Version: 0.1.0
Summary: Did the change work? Weekly blocks, cliffs, and difference-in-differences for Google Search Console data, from a CSV or the API.
Author: Tsuruta Lab
License: MIT
Project-URL: Homepage, https://github.com/tsurutanmen/searchdiff
Project-URL: Issues, https://github.com/tsurutanmen/searchdiff/issues
Keywords: search console,seo,difference in differences,ctr,analytics
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Internet :: WWW/HTTP :: Site Management
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: api
Requires-Dist: google-auth>=2.0; extra == "api"
Requires-Dist: requests>=2.25; extra == "api"
Provides-Extra: test
Requires-Dist: pytest>=7; extra == "test"
Dynamic: license-file

# searchdiff

Did the change work?

Search Console answers two questions badly on its own. "When did traffic drop, and how far back is
it?" gets a daily chart with the weekday wobble in it. "We rewrote the titles, did it help?" gets a
before/after on the changed pages, which contains the season, the algorithm update, and the change,
all mixed together. `searchdiff` gives the first question weekly blocks from a fixed anchor, and the
second a control group and a difference in differences.

```
pip install git+https://github.com/tsurutanmen/searchdiff                 # CSV only, no dependencies
pip install "git+https://github.com/tsurutanmen/searchdiff#egg=searchdiff[api]"   # plus the API
```

## Weekly blocks

```
$ searchdiff weeks daily.csv --anchor 2026-07-18

week                    days  clicks   /day     impr    ctr   pos  vs prev  of peak
2026-07-18..07-24     7    1486  212.3    28420   5.2%   6.6      +2%      95%
2026-07-25..07-31     7    1487  212.4    28605   5.2%   7.0      +0%      95%
2026-08-01..08-07     7    1562  223.1    28365   5.5%   6.1      +5%     100%  peak
2026-08-08..08-14     7    1000  142.9    17816   5.6%   6.1     -36%      64%  CLIFF
2026-08-15..08-21     7    1031  147.3    18516   5.6%   5.9      +3%      66%
2026-08-22..08-28     7    1197  171.0    20313   5.9%   6.2     +16%      77%
2026-08-29..08-31     3     483  161.0     8482   5.7%   7.6      -6%      72%
last block is partial; its /day figure is comparable, its totals are not
note: cliff in the block starting 2026-08-08 (-36%); latest block is at 72% of peak
```

`daily.csv` here is the `Dates` export from the Performance report, unchanged. Any CSV with
`date,clicks,impressions[,position]` works, with or without a `page` column. Blocks are counted
from the anchor in both directions, so the same anchor gives the same blocks next month.

## Did the change work

```
$ searchdiff effect pages.csv --change 2026-08-04 --treated "re:/p/0[0-4]$" --window 28

change on 2026-08-04, 28 days before and after

            pages            clicks/day              impr/day                 ctr        position
treated         5     30.25 ->     21.50     564.4 ->     409.5    5.36% ->    5.25%    7.2 ->    8.0
control        35    182.57 ->    143.82    3487.6 ->    2512.7    5.23% ->    5.72%    7.7 ->    7.4

difference in differences (treated change minus control change):
  ctr           -0.60 points   (treated -0.11, control +0.49)
  clicks/day     -7.7 %        (treated -28.9%, control -21.2%)
  impr/day       +0.5 %        (treated -27.4%, control -28.0%)
  position      +0.95 ranks    (treated +0.72, control -0.23; negative is better)
```

The treated pages lost 29% of their clicks after the change. So did everything else: a traffic cliff
hit the whole site four days later. The difference in differences is within a CTR point of zero. The
rewrite did nothing, and a before/after on the five pages alone would have called it a disaster.

The same data with pages that really did get a lift:

```
  ctr           +3.41 points   (treated +3.33, control -0.09)
  clicks/day    +49.0 %        (treated +20.4%, control -28.6%)
```

`pages.csv` needs `date,page,clicks,impressions[,position]` rows, which the API gives and the UI does
not. `--treated` and `--control` take a substring, a glob, `re:` a regex, or `@file` with one URL per
line. The control defaults to every page not treated; a control of the same kind as the treated pages
is better. `--gap N` skips the first N days after the change for edits that take time to be re-indexed.

The report adds notes when the window runs past the data, when the treated group has too few clicks
for anything to be distinguishable from noise, and when treated and control moved together.

## Fetching from the API

```
searchdiff fetch --site https://example.com/ --key service-account.json \
    --start 2026-06-01 --end 2026-08-31 --out pages.csv --coverage
```

The service account's e-mail has to be added as a user of the property. `--coverage` prints how much
of the clicks the query dimension shows. Search Console hides rare queries; on a site of ours the
query-level sum was 39% of the total while the page-level sum matched it. Analyse by page unless the
words are the point. Paging past 25,000 rows is handled.

## Python

```python
import datetime as dt, searchdiff as sd

rows = sd.load_csv("pages.csv")
print(sd.weekly(rows, anchor=dt.date(2026, 7, 18)))
rep = sd.effect(rows, dt.date(2026, 8, 4), sd.matcher("re:/blog/"), window=28)
rep.did          # {'ctr': points, 'clicks': relative, 'impressions': relative, 'position': ranks}
rep.to_dict()
```

## Where this came from

A title rewrite on a set of comparison pages, measured before/after, looked like nothing: 3.38% to
3.37%. A second rewrite on another set, the same. Both times the tempting reading was "wait longer".
With the untouched pages as control the answer was the same and available on day 28, and the
proposal to rewrite 475 more titles was dropped. The weekly-block view is how a 40% cliff and its
slow return were tracked without re-reading a daily chart every session. The tool packages those
two habits.

## Claude Code skill

`skill/searchdiff/SKILL.md` teaches Claude Code to reach for the control group instead of the
before/after, and how to read the two tables. Install by copying the folder:

```
cp -r skill/searchdiff ~/.claude/skills/searchdiff
```

## License

MIT.
