Skip to content

Use a separate working directory per connection (fixes #163) - #187

Open
NicklasBlidmo wants to merge 1 commit into
MerginMaps:masterfrom
NicklasBlidmo:fix-163-workdir-per-connection
Open

NicklasBlidmo wants to merge 1 commit into
MerginMaps:masterfrom
NicklasBlidmo:fix-163-workdir-per-connection

Conversation

@NicklasBlidmo

Copy link
Copy Markdown

Problem

All connections to the same Mergin Maps project share one local checkout (<working_dir>/). When a project has several GeoPackages — which is what the QGIS plugin produces when creating a project from an existing QGIS project — the first connection that sees a new server version calls mc.pull_project(), which updates every GeoPackage in the checkout, but pull() only computes and applies the changeset for that connection's own sync_file. The remaining connections then find server_version == local_version, log "No changes on Mergin Maps" and return. Their changes never reach the database, and nothing is logged as an error.

On our eight-GeoPackage project the polygon layer silently stopped updating within an hour of a fresh init while the point layer (first in the config) kept working. The same shared checkout also breaks init after a restart: each connection "corrects" the checkout to its own base-schema version in turn, and the next connection fails with "base schema is not synchronized with source GPKG".

Fix

_get_work_dir(conn_cfg) returns <working_dir>// and replaces the four places (init, pull, push, status) that derived work_dir from the project name alone. The modified schema name is unique per connection and stable if connections are reordered.

Each connection now has its own checkout. The cost is one download of the project per connection.

Compatibility

Existing installations re-download the checkout on the next start: init sees the schemas, finds no working directory at the new path, and downloads the version recorded in the base schema comment, then pulls forward. Database schemas are not touched. The old shared checkout directory can be deleted.

Testing

Running in production since 2026-09-16 on a project with eight GeoPackages and an active field team (inserts, updates and deletes across all layers). Before the change, two of eight layers had diverged from the GeoPackages within an hour of a clean init; after the change, all eight layers have stayed in sync, verified by comparing feature counts per table in each GeoPackage against the modified schema every morning.

black -l 120 --check dbsync.py passes.

All connections to the same Mergin Maps project shared one local checkout
(<working_dir>/<project>). The first connection to see a new server version
pulled it - updating every GeoPackage in the checkout - but only computed and
applied the changeset for its own sync file. The other connections then found
local_version == server_version and returned without applying anything, so
their changes silently never reached the database.

Each connection now uses <working_dir>/<project>/<modified schema>. Existing
installations re-download the checkout on the next start (init downloads the
version recorded in the base schema and pulls forward from there); database
schemas are unaffected.

(cherry picked from commit 7bdc180a079b91c9ae5ed0a39946d1cd41d68222)
Comment thread docs/using.md

A project with several GeoPackage files needs one connection per file (see the `connections` list in the
configuration). Each connection keeps its own local checkout of the project in
`<working_dir>/<project name>/<modified schema>`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed your changes — good job.

We're suggesting a different convention for the folder name. For backward compatibility with previous versions of db-sync, we're suggesting using - between the project name and the modified schema:

<working-dir>/<project name>-<modified schema>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants