deferred-sync

Contents

1. Overview
2. Features
3. Supported Operating Systems
4. Installation
5. Configuration
6. Policy
7. Usage Example
8. Directory Structure
9. Contribution
10. License

Overview

deferred-sync is a backup and synchronization tool designed to periodically copy and version-control critical files while enabling remote backup capabilities. It is particularly useful for:

• Protecting configuration files and home directory data that are not managed by a version control system.
• Performing incremental backups for repository servers, databases, and file storage systems.
• Synchronizing backups across multiple VPS or cloud environments.

The tool is implemented as a shell script framework with plugin support, allowing users to extend its functionality for various tasks, such as system upgrades and database dumps.

Features

• Incremental backup with versioning
• Remote synchronization via rsync and ssh
• Highly extensible plugin system
• Support for automatic execution via cron
• Configurable exclusion of files and directories

Supported Operating Systems

deferred-sync is designed to run on the following UNIX-like operating systems:

• Red Hat Enterprise Linux 5 and later
• CentOS 5 and later
• Scientific Linux 5 and later
• Debian GNU/Linux 5 and later
• Ubuntu 8.04 LTS and later
• Solaris 10 and later
• Mac OS X 10.5 and later

Some plugins may not be compatible with Solaris and macOS.

Installation

For a standard system-wide installation, run the provided install.sh script without arguments:

./install.sh

This installs under /opt/deferred-sync and additionally sets up:

• /etc/cron.daily/deferred-sync (skipped if /etc/cron.d/deferred-sync already exists)
• /etc/logrotate.d/deferred-sync
• the default backup directories /home/backup and /home/remote

These steps run only on Linux, and only when no [target_path] is given. Passing an explicit [target_path] is treated as a custom installation: the components are deployed to that path, but cron, logrotate, and backup directory setup are skipped. Give [target_path] as an absolute path, since a relative path is rejected as an unknown option.

./install.sh /opt/deferred-sync   # deploys components only, no cron or logrotate setup

Specifying nosudo, --no-sudo, or -n runs the installer without sudo and skips the recursive ownership change of the target directory. If you wish to install in your home directory, run:

./install.sh ~/local/deferred-sync nosudo

You can optionally add the --link flag to create symlinks in /etc/cron.config/ and /etc/cron.exec/:

./install.sh --link

The /etc/cron.exec/deferred-sync symlink always points at /opt/deferred-sync/exec/deferred-sync, so --link is intended for the default installation path.

If you want to specify an exact execution time, instead of relying on cron.daily, you can manually configure cron.d using the sample file provided in cron/cron.d/deferred-sync.

After installation, edit the configuration file to customize its behavior.

Uninstallation

To uninstall a system-wide installation (default /opt/deferred-sync), run:

sudo ./install.sh --uninstall

This will remove all files installed by deferred-sync except the log directory (/var/log/deferred-sync).

For safety, --uninstall removes only /opt/deferred-sync as the installation target. Custom installation targets are not removed automatically.

Configuration

The main configuration file is config/sync.conf. It defines all parameters required for operation, including:

• DRY_RUN - Enables dry-run mode if set to true.
• EXCLUDEFILE - Specifies files or patterns to be excluded.
• JOBLOG - Defines the log file location.
• STARTSCRIPT - A script to run before the synchronization process.
• ENDSCRIPT - A script to run after the synchronization process.
• ADMIN_MAIL_ADDRESS - Email to receive job completion notifications.
• LOAD_PLUGINS_ALL - If true, all plugins will be loaded automatically.
• PLUGINS - List of plugins to load selectively.

Optional System-Wide Configuration Symlinks

If you pass the --link option during installation, deferred-sync will automatically create symlinks:

• /etc/cron.config/sync.conf → /etc/opt/deferred-sync/sync.conf
• /etc/cron.config/exclude.conf → /etc/opt/deferred-sync/exclude.conf
• /etc/cron.exec/deferred-sync → /opt/deferred-sync/exec/deferred-sync

This is useful when integrating with a centralized cron execution and configuration framework.

Policy

deferred-sync adheres to a strict, POSIX-compliant policy for error handling, return codes, and plugin design. It is stated in doc/POLICY.md, which is where these rules are maintained.

What matters before writing or enabling a plugin:

• A plugin is sourced, not executed. It returns instead of exiting, keeps a cd inside a subshell, and owns the variable names it sets. See The Contract Between the Core and a Plugin.
• Nothing aborts the job. lib/load reports a failing plugin as [WARN], keeps the first nonzero status, and runs the rest. See Warn and Continue.
• Return codes are 0 success, 1 command failure or resource missing, 2 network unreachable, 3 local prerequisite missing, with the two documented wrappers propagating an external status. See Return Codes.
• A missing prerequisite is skipped, never created, so that a failed mount cannot become a backup written to the wrong disk. See Safety.
• Log output uses [INFO], [WARN], and [ERROR], and stamps each phase with the time, because the log is read hours after the run. See Logging.

Usage Example

Set up cron to execute deferred-sync periodically. This ensures that all protected files and directories are backed up regularly.

Example: Daily Backup and Remote Sync

• Primary environment (Data Center):

  • Backs up critical files daily
  • Synchronizes them to a remote server

• Remote Backup Server (Different Location):

  • Stores historical versions of backups
  • Allows recovery in case of failures

+----------------------+
|  Production Server  |  (Data Center)
+----------------------+
           |
           | cron executes deferred-sync daily
           |
+----------------------+
|  Backup Server      |  (Remote Location)
+----------------------+

Directory Structure

This section describes the main directories of the repository and what each one is for. It is not a complete file listing: only the entries worth knowing about before configuring a run or writing a plugin are shown.

.
├── exec/
│   └── deferred-sync         Main execution script. The entry point cron invokes.
├── config/                   Deployed to /etc/opt/deferred-sync/ and edited there.
│   ├── sync.conf             All settings for a run (see Configuration).
│   └── exclude.conf          Patterns excluded from the backup.
├── lib/                      Everything the main script sources at run time.
│   ├── load                  Plugin loader. Runs each plugin and applies warn-and-continue.
│   ├── before                Default STARTSCRIPT, run before synchronization.
│   ├── after                 Default ENDSCRIPT, run after synchronization.
│   └── plugins/              One file per task, run in filename order.
│       ├── 09_show_version
│       ├── 10_get_resources
│       ├── 11_server_alive_check
│       ├── 15_get_hardware_info
│       ├── 20_system_upgrade
│       ├── 25_ubuntu_kernel_upgrade
│       ├── 30_dump_mysql
│       ├── 31_dump_postgresql
│       ├── 32_dump_mongodb
│       ├── 35_dump_svn
│       ├── 70_incremental_backup
│       ├── 80_backup_to_remote
│       └── 85_get_remote_dir
├── install.sh                Installer and uninstaller.
├── cron/                     Scheduling and log rotation samples, installed on Linux.
│   ├── deferred-sync         Placed in /etc/cron.daily/.
│   ├── cron.d/               Sample for a fixed execution time, for /etc/cron.d/.
│   └── logrotate.d/          Log rotation config, for /etc/logrotate.d/.
└── doc/
    ├── POLICY.md             Implementation policy for this repository.
    ├── VERSIONS              Version history of the repository.
    ├── LICENSE.md            License notice.
    ├── COPYING               GPL version 3 text.
    └── COPYING.LESSER        LGPL version 3 text.

The split between exec/, config/ and lib/ is what the installer deploys: exec/ is the one thing cron calls, config/ is the only part meant to be edited on a host, and lib/ is the code that sync.conf selects between. A change in behavior is normally a change in config/, not in the other two.

lib/plugins/ is where the work actually happens, and each file is one task. Plugins run in filename order; the numeric prefix controls that order and may be omitted in PLUGINS, since each entry is matched against the end of the plugin filename (for example, get_resources matches 10_get_resources). Adding a task means adding a file here, named so that it sorts into the right place, and following the plugin contract.

Contribution

We welcome contributions! Here's how you can help:

1. Fork the repository.
2. Add or improve a feature, or fix an issue.
3. Submit a pull request with clear documentation and changes.

Please ensure your code is well-structured and documented, and follow doc/POLICY.md. Everything under lib/ is sourced by a root shell that cron starts unattended, so that document asks new code there to destroy nothing it was not asked to touch, to let the job continue when it fails, and to leave a log that answers the question the next morning.

License

This repository is dual licensed under the GPL version 3 or the LGPL version 3, at your option. For full details, please refer to the LICENSE file. See also COPYING and COPYING.LESSER for the complete license texts.

Thank you for using and contributing to this repository!