BorgBackup – Deduplicating Backup Program with Compression and Encryption

Complete Guide to BorgBackup: Enterprise-Grade Backup for Everyone

BorgBackup, commonly known as Borg, has established itself as one of the most powerful and efficient backup solutions available in the open-source ecosystem. This command-line backup tool excels at creating space-efficient, encrypted backups through its innovative deduplication algorithm. Originally forked from Attic backup, Borg has evolved into a mature, production-ready solution trusted by individuals and organizations worldwide for protecting critical data.

The efficiency of BorgBackup stems from its content-defined chunking algorithm, which divides files into variable-size blocks based on content rather than fixed positions. This approach enables detecting identical data across files and backup runs, storing each unique chunk only once regardless of how many times it appears. Combined with optional compression and client-side encryption, Borg provides a complete backup solution that minimizes storage requirements while maximizing security.

Key Features and Benefits

Deduplication represents Borg’s most significant advantage. When backing up similar data—whether multiple versions of files across backups or duplicate content across different directories—Borg identifies and stores identical chunks only once. This typically results in backup repositories that are dramatically smaller than the source data, especially for incremental backups over time.

Encryption ensures that your backup data remains confidential even if stored on untrusted servers. Borg uses authenticated encryption (AES-256-CTR + HMAC-SHA256 or ChaCha20-Poly1305), protecting data against both unauthorized access and tampering. The encryption key never leaves your control, making Borg suitable for backing up to cloud storage or remote servers managed by third parties.

Compression further reduces storage requirements, with multiple algorithms available to balance compression ratio against speed. The auto option intelligently selects compression based on file content, applying aggressive compression to text while avoiding compressing already-compressed media files.

Installing BorgBackup

BorgBackup is available through package managers on most platforms, with installation straightforward across Linux distributions, macOS, and Windows (via WSL).

Linux Installation

# Ubuntu/Debian
sudo apt update
sudo apt install borgbackup

# Fedora
sudo dnf install borgbackup

# Arch Linux
sudo pacman -S borg

# openSUSE
sudo zypper install borgbackup

# CentOS/RHEL (via EPEL)
sudo yum install epel-release
sudo yum install borgbackup

# Using pip (latest version)
pip install borgbackup

# Verify installation
borg --version

# Install with optional dependencies
pip install borgbackup[fuse]  # For mounting backups

macOS Installation

# Homebrew installation
brew install borgbackup

# With FUSE support for mounting
brew install macfuse
brew install borgbackup-fuse

# Verify installation
borg --version

Windows Installation (via WSL)

# In WSL Ubuntu
sudo apt update
sudo apt install borgbackup

# Native Windows alternative
# Use Borg's Windows binaries from releases
# Or run via Docker

Repository Initialization

Before creating backups, you must initialize a Borg repository. The repository stores all backup data, including encrypted chunks and metadata.

# Initialize local repository with encryption
borg init --encryption=repokey /path/to/backup/repo

# Initialize with keyfile encryption (key stored separately)
borg init --encryption=keyfile /path/to/backup/repo

# Initialize without encryption (not recommended)
borg init --encryption=none /path/to/backup/repo

# Initialize remote repository via SSH
borg init --encryption=repokey user@server:/path/to/repo

# Initialize with specific encryption algorithm
borg init --encryption=repokey-blake2 /path/to/backup/repo

# Encryption options:
# repokey: Key stored in repository, encrypted with passphrase
# keyfile: Key stored in ~/.config/borg/keys/, encrypted with passphrase
# repokey-blake2/keyfile-blake2: Uses BLAKE2b instead of SHA256
# authenticated: Authentication only, no encryption
# none: No encryption or authentication

# Export encryption key (important for recovery!)
borg key export /path/to/repo /path/to/backup/key.txt

# Import key to new system
borg key import /path/to/repo /path/to/backup/key.txt

Creating Backups

Backups are created using the borg create command, which accepts numerous options for filtering files, controlling compression, and managing backup behavior.

# Basic backup command
borg create /path/to/repo::backup-{now} /path/to/data

# Backup with automatic timestamp naming
borg create /path/to/repo::backup-{now:%Y-%m-%d_%H:%M:%S} ~/Documents

# Backup multiple directories
borg create /path/to/repo::archive-name \
    ~/Documents \
    ~/Projects \
    ~/.config

# Backup with compression
borg create --compression lz4 /path/to/repo::archive ~/data
borg create --compression zstd /path/to/repo::archive ~/data
borg create --compression zstd,10 /path/to/repo::archive ~/data
borg create --compression auto,lzma,6 /path/to/repo::archive ~/data

# Backup with progress display
borg create --progress --stats /path/to/repo::archive ~/data

# Backup with exclusions
borg create --exclude '*.tmp' \
    --exclude 'node_modules' \
    --exclude '__pycache__' \
    --exclude '.cache' \
    /path/to/repo::archive ~/Projects

# Backup using exclude file
borg create --exclude-from /path/to/exclude.txt \
    /path/to/repo::archive ~/data

# Backup to remote repository
borg create user@server:/path/to/repo::archive ~/Documents

# One-file-system (don't cross mount points)
borg create --one-file-system /path/to/repo::root-backup /

# Show files as they're backed up
borg create --list /path/to/repo::archive ~/data

Managing Archives

Archives within a repository can be listed, inspected, renamed, and deleted using various Borg commands.

# List all archives in repository
borg list /path/to/repo

# List archives with details
borg list --format '{archive} {time} {size}' /path/to/repo

# List contents of specific archive
borg list /path/to/repo::archive-name

# List with pattern matching
borg list --glob-archives 'backup-2024-*' /path/to/repo

# Get archive information
borg info /path/to/repo::archive-name

# Get repository information
borg info /path/to/repo

# Rename archive
borg rename /path/to/repo::old-name new-name

# Delete specific archive
borg delete /path/to/repo::archive-name

# Delete multiple archives matching pattern
borg delete --glob-archives 'backup-2024-01-*' /path/to/repo

# Delete with dry run (preview)
borg delete --dry-run --list /path/to/repo::archive-name

Restoring Data

Borg provides multiple methods for restoring data, from extracting entire archives to retrieving individual files or mounting archives as filesystems.

# Extract entire archive to current directory
borg extract /path/to/repo::archive-name

# Extract to specific directory
cd /restore/location
borg extract /path/to/repo::archive-name

# Extract specific files/directories
borg extract /path/to/repo::archive-name path/to/file.txt
borg extract /path/to/repo::archive-name home/user/Documents

# Extract with pattern matching
borg extract --pattern '+ home/user/*.txt' \
    --pattern '- *' \
    /path/to/repo::archive-name

# Extract with progress
borg extract --progress /path/to/repo::archive-name

# List files without extracting
borg extract --list --dry-run /path/to/repo::archive-name

# Mount archive as filesystem
mkdir /mnt/borg
borg mount /path/to/repo::archive-name /mnt/borg
# Browse files at /mnt/borg
ls /mnt/borg

# Mount entire repository (all archives)
borg mount /path/to/repo /mnt/borg

# Unmount when done
borg umount /mnt/borg

# Export archive to tar
borg export-tar /path/to/repo::archive-name backup.tar
borg export-tar /path/to/repo::archive-name - | gzip > backup.tar.gz

Pruning and Retention Policies

The prune command implements retention policies, automatically removing old archives while keeping specified numbers of recent backups.

# Prune archives keeping specific retention
borg prune \
    --keep-hourly 24 \
    --keep-daily 7 \
    --keep-weekly 4 \
    --keep-monthly 6 \
    --keep-yearly 2 \
    /path/to/repo

# Prune with archive prefix matching
borg prune --glob-archives 'laptop-*' \
    --keep-daily 7 \
    --keep-weekly 4 \
    /path/to/repo

# Prune with dry run to preview
borg prune --dry-run --list \
    --keep-daily 7 \
    /path/to/repo

# Prune with verbose output
borg prune --verbose --list \
    --keep-daily 7 \
    --keep-weekly 4 \
    /path/to/repo

# Prune and reclaim space
borg prune --keep-daily 7 /path/to/repo
borg compact /path/to/repo

# Retention options:
# --keep-within: Keep all within time interval (e.g., 30d)
# --keep-last: Keep last N archives
# --keep-minutely: Keep N per minute
# --keep-hourly: Keep N per hour
# --keep-daily: Keep N per day
# --keep-weekly: Keep N per week
# --keep-monthly: Keep N per month
# --keep-yearly: Keep N per year

Automation Scripts

Automating backups ensures consistent protection without manual intervention. These scripts demonstrate production-ready backup automation patterns.

#!/bin/bash
# borg-backup.sh - Automated backup script

# Configuration
export BORG_REPO='/path/to/backup/repo'
export BORG_PASSPHRASE='your-secure-passphrase'

# Alternatively, use passcommand for security
# export BORG_PASSCOMMAND='cat /path/to/secure/passfile'
# export BORG_PASSCOMMAND='pass show borg-backup'

# Backup source directories
BACKUP_PATHS="/home/user/Documents /home/user/Projects /etc"

# Exclusions
EXCLUDE_PATTERNS=(
    "*.tmp"
    "*.cache"
    "**/node_modules"
    "**/__pycache__"
    "**/.git/objects"
    "**/Cache"
    "**/cache"
)

# Build exclude arguments
EXCLUDES=""
for pattern in "${EXCLUDE_PATTERNS[@]}"; do
    EXCLUDES="$EXCLUDES --exclude '$pattern'"
done

# Archive name with timestamp
ARCHIVE="backup-{hostname}-{now:%Y-%m-%d_%H:%M:%S}"

# Create backup
echo "Starting backup..."
eval borg create \
    --verbose \
    --filter AME \
    --stats \
    --compression auto,zstd,6 \
    --exclude-caches \
    $EXCLUDES \
    "::$ARCHIVE" \
    $BACKUP_PATHS

backup_exit=$?

# Prune old backups
echo "Pruning old archives..."
borg prune \
    --list \
    --glob-archives 'backup-{hostname}-*' \
    --keep-hourly 24 \
    --keep-daily 7 \
    --keep-weekly 4 \
    --keep-monthly 6 \
    --keep-yearly 2

prune_exit=$?

# Compact repository
echo "Compacting repository..."
borg compact

compact_exit=$?

# Report status
global_exit=$(( backup_exit > prune_exit ? backup_exit : prune_exit ))
global_exit=$(( compact_exit > global_exit ? compact_exit : global_exit ))

if [ $global_exit -eq 0 ]; then
    echo "Backup completed successfully"
elif [ $global_exit -eq 1 ]; then
    echo "Backup completed with warnings"
else
    echo "Backup failed with errors"
fi

exit $global_exit

Remote Backups via SSH

Borg natively supports backing up to remote servers via SSH, enabling offsite backup strategies.

# Initialize remote repository
borg init --encryption=repokey user@backup-server:/backup/repo

# Create backup to remote
borg create user@backup-server:/backup/repo::archive-name ~/data

# Use specific SSH key
export BORG_RSH='ssh -i /path/to/private_key'
borg create user@server:/backup/repo::archive ~/data

# Configure SSH in ~/.ssh/config for convenience
# Host backup-server
#     HostName backup.example.com
#     User borguser
#     IdentityFile ~/.ssh/backup_key
#     Port 22

# Then use:
borg create backup-server:/backup/repo::archive ~/data

# Limit bandwidth
export BORG_RSH='ssh -o "IPQoS=throughput"'
borg create --remote-ratelimit 5000 server:/repo::archive ~/data

# Check repository on remote
borg check user@server:/backup/repo

BorgMatic Integration

Borgmatic provides a configuration-file-based wrapper around Borg, simplifying backup automation and management.

# Install borgmatic
pip install borgmatic

# Generate configuration
borgmatic config generate --destination /etc/borgmatic/config.yaml

# Example borgmatic configuration
# /etc/borgmatic/config.yaml
location:
    source_directories:
        - /home
        - /etc
        - /var/log
    repositories:
        - /backup/borg-repo
        - user@server:/backup/repo
    exclude_patterns:
        - '*.tmp'
        - '**/cache'
        - '**/node_modules'

storage:
    encryption_passphrase: "your-passphrase"
    compression: auto,zstd,6
    archive_name_format: '{hostname}-{now}'

retention:
    keep_hourly: 24
    keep_daily: 7
    keep_weekly: 4
    keep_monthly: 6
    keep_yearly: 2

consistency:
    checks:
        - repository
        - archives

hooks:
    before_backup:
        - echo "Starting backup"
    after_backup:
        - echo "Backup complete"
    on_error:
        - echo "Backup failed"

# Run borgmatic
borgmatic --verbosity 1

# Create backup only
borgmatic create --verbosity 2

# List archives
borgmatic list

# Check repository
borgmatic check

# Prune old archives
borgmatic prune

# Initialize new repository
borgmatic init --encryption repokey

Integrity Checking

Regular integrity checks ensure backup data remains valid and recoverable.

# Check repository and archives
borg check /path/to/repo

# Repository check only (faster)
borg check --repository-only /path/to/repo

# Archives check only
borg check --archives-only /path/to/repo

# Verify data integrity (slower but thorough)
borg check --verify-data /path/to/repo

# Check specific archive
borg check --first 1 /path/to/repo

# Repair repository (if corruption detected)
borg check --repair /path/to/repo

# Schedule regular checks in cron
# 0 3 * * 0 borg check /path/to/repo >> /var/log/borg-check.log 2>&1

Security Best Practices

Implementing security best practices ensures your backups remain protected and recoverable.

# Store passphrase securely (not in plain scripts)
# Option 1: Use password manager
export BORG_PASSCOMMAND='pass show backup/borg'

# Option 2: Restricted file permissions
echo "passphrase" > ~/.borg-passphrase
chmod 600 ~/.borg-passphrase
export BORG_PASSPHRASE=$(cat ~/.borg-passphrase)

# Export and backup encryption key
borg key export /path/to/repo /secure/location/borg.key
# Store key separately from backups!

# Append-only mode for remote repository (ransomware protection)
# On server, in ~/.ssh/authorized_keys:
# command="borg serve --append-only --restrict-to-path /backup/repo" ssh-rsa AAA...

# Test recovery periodically
borg extract --dry-run /path/to/repo::latest

# Monitor backup age
find /backup/repo -name 'index.*' -mtime +1 -exec echo "Warning: No recent backup" \;

Performance Optimization

Tuning Borg for optimal performance involves balancing compression, network usage, and system resources.

# Use fast compression for initial backup
borg create --compression lz4 /path/to/repo::archive ~/data

# Higher compression for archival
borg create --compression zstd,15 /path/to/repo::archive ~/data

# Parallel chunk hashing (faster on multi-core)
borg create --chunker-params 19,23,21,4095 /path/to/repo::archive ~/data

# Limit I/O impact
ionice -c 3 nice -n 19 borg create /path/to/repo::archive ~/data

# Use checkpoint for large backups
borg create --checkpoint-interval 600 /path/to/repo::archive ~/data

# Resume interrupted backup
borg create /path/to/repo::archive ~/data  # Continues from checkpoint

Conclusion

BorgBackup delivers enterprise-grade backup capabilities in an accessible, open-source package. Its unique combination of deduplication, encryption, and compression creates space-efficient backups that protect your data against loss while maintaining confidentiality. Whether backing up personal files to an external drive or implementing organization-wide backup infrastructure, Borg provides the tools and flexibility needed for reliable data protection.