|
| 1 | +# Contributing |
| 2 | + |
| 3 | +Bug reports and pull requests are welcome on GitHub. Run commands from the repository root. |
| 4 | + |
| 5 | +## Development setup |
| 6 | + |
| 7 | +Use Python 3.13 to work on the package and its documentation. The SDK CI also tests Python 3.8 through 3.14. Create a virtual environment and install the package with its test and documentation dependencies: |
| 8 | + |
| 9 | +```sh |
| 10 | +uv venv --python 3.13 |
| 11 | +uv pip install -e '.[test,docs]' |
| 12 | +``` |
| 13 | + |
| 14 | +The editable install uses the Python files in your checkout, so source changes take effect without reinstalling the package. |
| 15 | + |
| 16 | +Activate the environment on macOS or Linux: |
| 17 | + |
| 18 | +```sh |
| 19 | +source .venv/bin/activate |
| 20 | +``` |
| 21 | + |
| 22 | +On Windows PowerShell: |
| 23 | + |
| 24 | +```powershell |
| 25 | +.\.venv\Scripts\Activate.ps1 |
| 26 | +``` |
| 27 | + |
| 28 | +The commands below use `python` from that environment. |
| 29 | + |
| 30 | +## Testing |
| 31 | + |
| 32 | +### Checks that need no API key |
| 33 | + |
| 34 | +These tests cover response parsing, image uploads with simulated responses, timeouts, exceptions, documentation syntax, and the example runner: |
| 35 | + |
| 36 | +```sh |
| 37 | +python -m pytest tests/test_output_formats.py tests/test_image_upload.py tests/test_timeout.py tests/test_exceptions.py tests/test_docs_examples.py tests/test_docs_example_runner.py -k 'not live' -q |
| 38 | +``` |
| 39 | + |
| 40 | +The file list limits this command to tests that make no external requests. Applying `-k 'not live'` to the entire suite is not enough to run offline: some SDK integration tests do not have `live` in their names. |
| 41 | + |
| 42 | +### SDK integration tests |
| 43 | + |
| 44 | +Set `API_KEY` in your shell or editor's run configuration before running integration tests. Documentation examples also accept `SERPAPI_KEY`, but the shared SDK fixtures require `API_KEY`. If you already have `SERPAPI_KEY` set, copy it to `API_KEY` in the same shell: |
| 45 | + |
| 46 | +```sh |
| 47 | +export API_KEY="$SERPAPI_KEY" |
| 48 | +``` |
| 49 | + |
| 50 | +In PowerShell, use `$env:API_KEY = $env:SERPAPI_KEY`. Keep real keys out of source files and commits. |
| 51 | + |
| 52 | +To match the test selection in the SDK CI workflow: |
| 53 | + |
| 54 | +```sh |
| 55 | +python -m pytest tests --ignore-glob='tests/test_docs_*.py' -k 'not example' -q |
| 56 | +``` |
| 57 | + |
| 58 | +This includes live account, location, search, and pagination checks. To run every discovered test, including the standalone engine examples and documentation examples: |
| 59 | + |
| 60 | +```sh |
| 61 | +python -m pytest -q |
| 62 | +``` |
| 63 | + |
| 64 | +Both commands make real SerpApi requests. The full suite can use more searches than a test run limited to the files you changed. PR CI also runs the standalone engine examples on Python 3.14. Documentation tests run only in the package and documentation release workflows, on `master` or release tags. |
| 65 | + |
| 66 | +### Testing documentation examples |
| 67 | + |
| 68 | +The tests discover Python code blocks in the README and all Markdown pages under `docs/`. They run each page as a separate script against SerpApi, including the multiprocessing example's worker processes. The Lens upload examples use the repository's PNG icon as `image.png`. Each page has a five-minute limit, and HTTP errors or JSON responses containing an `error` fail the check even if the example catches the exception. |
| 69 | + |
| 70 | +With `SERPAPI_KEY` or `API_KEY` set in your environment, run: |
| 71 | + |
| 72 | +```sh |
| 73 | +python -m pytest tests/test_docs_examples.py --require-docs-key -q |
| 74 | +``` |
| 75 | + |
| 76 | +The `--require-docs-key` option fails if neither key is set. Without that option, local runs skip live tests when no key is available. To check syntax and the test runner without making API calls: |
| 77 | + |
| 78 | +```sh |
| 79 | +python -m pytest tests/test_docs_examples.py tests/test_docs_example_runner.py -k 'not live' -q |
| 80 | +``` |
| 81 | + |
| 82 | +Code blocks marked with `docs-test: skip` are checked for syntax but not executed. These cover the old `google-search-results` package and examples that require private proxy or certificate settings, or disable TLS verification. Each marker includes its reason. |
| 83 | + |
| 84 | +### Testing documentation publishing |
| 85 | + |
| 86 | +Run the publishing tests locally with: |
| 87 | + |
| 88 | +```sh |
| 89 | +python -m pytest tests/test_docs_publishing.py -q |
| 90 | +``` |
| 91 | + |
| 92 | +These tests use simulated RTD responses to check outgoing HTTP requests, CI event and commit checks, version selection, polling, and cleanup after failures. They need no API keys and do not publish documentation. In CI, they run only in the Documentation workflow on Python 3.13, before the documentation build and publication. The SDK and PyPI release workflows exclude this file. RTD checks the actual checkout with `scripts.check_docs_revision` before and after building, without rerunning the publishing tests. |
| 93 | + |
| 94 | +### Running a single test or example |
| 95 | + |
| 96 | +Run one test file while working on that part of the package: |
| 97 | + |
| 98 | +```sh |
| 99 | +python -m pytest tests/test_output_formats.py -q |
| 100 | +``` |
| 101 | + |
| 102 | +To list documentation test IDs without running their examples: |
| 103 | + |
| 104 | +```sh |
| 105 | +python -m pytest tests/test_docs_examples.py --collect-only -q |
| 106 | +``` |
| 107 | + |
| 108 | +For example, run only the Google Lens upload and search page with: |
| 109 | + |
| 110 | +```sh |
| 111 | +python -m pytest 'tests/test_docs_examples.py::test_documentation_examples_live[docs/examples/google-lens-image-upload.md]' --require-docs-key -q |
| 112 | +``` |
| 113 | + |
| 114 | +Use `-x -vv` in place of `-q` to stop at the first failure and show more detail. A syntax failure points to a page and block number. A live failure can come from the example code, an invalid key, exhausted quota, or an upstream API error; inspect the reported engine and error before changing the example. |
| 115 | + |
| 116 | +When adding a documentation example, use a fenced block labelled `python`. Blocks on the same page execute in order and share variables. Avoid hard-coded dates that expire. Use a `docs-test: skip` marker with a reason only when a block cannot run in the test environment, such as one requiring a user's proxy or certificate. |
| 117 | + |
| 118 | +## Building the documentation locally |
| 119 | + |
| 120 | +The documentation uses Sphinx with MyST for Markdown pages and the Read the Docs theme. The development setup above includes its dependencies. Build and serve the HTML site with: |
| 121 | + |
| 122 | +```sh |
| 123 | +python -m sphinx -M html docs docs/_build -W --keep-going |
| 124 | +python -m http.server 8000 --bind 127.0.0.1 --directory docs/_build/html |
| 125 | +``` |
| 126 | + |
| 127 | +Open [the local documentation](http://127.0.0.1:8000). After editing a page, rerun the Sphinx build command and refresh your browser. Check the affected page, its code blocks, and the sidebar links. Press Ctrl+C to stop the server. |
| 128 | + |
| 129 | +Sphinx discovers public APIs with autodoc and reads their docstrings without running the search examples. Sidebar order comes from the toctrees in `docs/index.md`, so documentation filenames do not need numeric prefixes. The `-W` option makes warnings fail the build. |
| 130 | + |
| 131 | +CI also builds EPUB. Check it locally with: |
| 132 | + |
| 133 | +```sh |
| 134 | +python -m sphinx -M epub docs docs/_build -W --keep-going |
| 135 | +``` |
| 136 | + |
| 137 | +## Building the package |
| 138 | + |
| 139 | +Build the source distribution and wheel with: |
| 140 | + |
| 141 | +```sh |
| 142 | +uv build |
| 143 | +``` |
| 144 | + |
| 145 | +The files are written to `dist/`. Documentation sources, this guide, and the logo assets are included in the source distribution (`.tar.gz`) so contributors can build the docs from a source release. The wheel contains only the `serpapi` library and its package metadata. Generated docs and documentation dependencies are not part of a normal installation. |
| 146 | + |
| 147 | +## Documentation publishing |
| 148 | + |
| 149 | +The [documentation workflow](.github/workflows/docs.yml) runs the live examples before building HTML and EPUB on pushes to `master`, release tags, and manual runs on either ref. It does not run on pull requests. |
| 150 | + |
| 151 | +After the live examples and documentation build pass, the same workflow publishes to Read the Docs for `master` and `v*` release tags. PR runs never publish. The publishing job uses the `docs` GitHub environment and its `RTD_API_TOKEN` secret. The SerpApi key stays in GitHub as the existing `API_KEY` repository or organization secret. |
| 152 | + |
| 153 | +The workflow syncs RTD versions, activates the requested version if needed, and waits for the build to finish. `latest` tracks `master`. A release tag has its own version and also updates `stable` when RTD identifies it as the highest stable release. RTD still builds the site from the repository using [.readthedocs.yaml](.readthedocs.yaml); it does not receive the HTML artifact from GitHub. Documentation publishing runs independently of the PyPI release workflow. |
| 154 | + |
| 155 | +Before requesting a build, the workflow creates a temporary RTD environment variable named `DOCS_CI_REVISION`. It contains the tested commit, the permitted versions, and an expiration time. RTD checks this record against its checkout before and after the Sphinx build. A missing, expired, or different revision stops publication. RTD needs no SerpApi key. The workflow removes the temporary record after publishing, including when a build fails. Publishing jobs run one at a time so they cannot overwrite each other's revision record. |
| 156 | + |
| 157 | +### Maintainer setup |
| 158 | + |
| 159 | +Maintainers can configure the existing RTD project and GitHub environment with these steps: |
| 160 | + |
| 161 | +1. In RTD **Settings**, set **Connected repository** to **No connected repository** and keep **Repository URL** set to `https://github.com/serpapi/serpapi-python.git`. Set **Default branch** to `master` and the configuration file path to `.readthedocs.yaml`. The public repository URL lets RTD clone the source without receiving GitHub push events through the GitHub App. |
| 162 | +2. Under RTD **Integrations**, remove incoming GitHub webhook integrations for this project. If an older RTD webhook is also listed in the GitHub repository's **Settings > Webhooks**, disable or remove that webhook. Do not remove integrations for other projects. |
| 163 | +3. Under RTD **Automation Rules**, remove rules that activate new versions or change the default version. The workflow handles release activation. Under **Settings > Pull request builds**, turn off **Build pull requests for this project**. GitHub Actions runs the existing SDK and engine example tests on PRs. |
| 164 | +4. Under RTD **Environment Variables**, remove `API_KEY` or `SERPAPI_KEY` if you added either for docs tests. Do not add the RTD API token here. The workflow manages `DOCS_CI_REVISION` automatically. |
| 165 | +5. Keep `latest` active in **Versions** and use it as the default documentation version during this migration. Existing release tags contain their original docs and build configuration. After the first release containing these changes builds successfully, you can choose `stable` as the default version. |
| 166 | +6. Create an RTD API token in your [RTD profile settings](https://app.readthedocs.org/accounts/tokens/), using an account that maintains the `serpapi-python` project. In GitHub, open the repository's **Settings > Environments**, create an environment named `docs`, and add an environment secret named `RTD_API_TOKEN` with that value. Under **Deployment branches and tags**, select **Selected branches and tags** and add a Branch rule for `master` and a Tag rule for `v*`. Leave required reviewers and wait timers disabled if publishing should run without a manual approval. |
| 167 | +7. Merge the changes to `master`. In GitHub **Actions > Documentation**, follow **Live documentation examples**, **Build Sphinx documentation**, and **Publish Read the Docs**. The publishing log links to the RTD build. To retry publishing, run the Documentation workflow on `master` or the intended release tag. A manual run on another branch skips the documentation jobs. |
| 168 | + |
| 169 | +Use GitHub Actions to request builds after this setup. A manual RTD build has no CI revision record and will fail the revision check. If a branch or tag moves between testing and the RTD checkout, rerun the workflow for its current commit. Pushing a `v*` tag also starts the PyPI release workflow, so use an actual package release to test release documentation. |
| 170 | + |
| 171 | +See the [RTD build API](https://docs.readthedocs.com/platform/stable/api/v3.html#build-triggering), [Git integration settings](https://docs.readthedocs.com/platform/stable/reference/git-integration.html), and [GitHub environment settings](https://docs.github.com/en/actions/how-tos/deploy/configure-and-manage-deployments/manage-environments) for the platform setup details. |
| 172 | + |
| 173 | +## Publishing a new release |
| 174 | + |
| 175 | +1. Update the version in `serpapi/__version__.py`. |
| 176 | +2. Push a tag for that version. The release pipeline runs automatically: |
| 177 | + ```sh |
| 178 | + git tag v1.2.3 |
| 179 | + git push origin v1.2.3 |
| 180 | + ``` |
| 181 | + This triggers the [release workflow](.github/workflows/release.yml), which tests, builds, and publishes to PyPI, then smoke-tests the published package. |
| 182 | + |
| 183 | +> **Required secret:** `API_KEY` (used by the live documentation examples and the published-package smoke test). |
0 commit comments