Back Up and Restore Limelight Data

If you are running your own Limelight installation, this guide will provide direction on how to back up and restore your data. This guide will assume you have followed our Installing Limelight tutorial and are running Limelight using Docker and Docker Compose. Even if this is not the case, this guide should provide sufficient detail for you to back up and restore your data.

Back Up Data

Most data in Limelight are stored in MySQL, a relational database management engine. Spectra that are uploaded are stored as processed files on disk. Follow the sections below to back up each of these sets of data.

Note

Backing up will work best if no data imports are currently running.

Back Up MySQL Data

Important

The Limelight application must be running to back up MySQL in this way. Review how to start and stop Limelight.

To back up a current snapshot of your data, use the mysqldump command. This can be done via Docker by typing the following into your Linux terminal (including Windows users running Docker on WSL2 as per our Installing Docker tutorial).

sudo docker exec limelight-mysql sh -c 'exec mysqldump --opt --max_allowed_packet=200M --single-transaction -u root -p"MYSQL_ROOT_PASSWORD" limelight' | pigz --stdout > ~/limelight-backup.sql.gz

If you are not running MySQL in Docker, the command would be:

mysqldump --opt --max_allowed_packet=200M --single-transaction -u root -p"MYSQL_ROOT_PASSWORD" limelight | pigz --stdout > ~/limelight-backup.sql.gz

Note

MYSQL_ROOT_PASSWORD should be replaced with the actual root password for your MySQL installation. If you followed our Installing Limelight tutorial, this will be in your .env file as described on Step 4.

Note

If the pigz command is not found, gzip can be substituted into the command.

This will create a file in your home directory named limelight-backup.sql.gz that contains a compressed backup of the database.

To output this file with the name limelight-backup-20210501.sql.gz and to a different location, perhaps D:\limelight-backups\ on a Windows machine, ensure that directory exists then run:

sudo docker exec limelight-mysql sh -c 'exec mysqldump --opt --max_allowed_packet=200M --single-transaction -u root -p"MYSQL_ROOT_PASSWORD" limelight' | pigz --stdout > /mnt/d/limelight-backups/limelight-backup-20210501.sql.gz

For extensive documentation of mysqldump see the official MySQL website

Back Up Spectra Data

The spectra data are files on disk, and so can be easily backed up using standard archiving and compression programs, like tar, gzip, or zip. To do this the files must first be located.

Docker-Managed Data Location

If Docker is managing where files are store (the default behavior in our tutorial), type the following into your Linux terminal (including Windows users running Docker on WSL2 as per our Installing Docker tutorial).

sudo docker inspect limelight_spectr_storage

The output should be similar to:

[
    {
        "CreatedAt": "2021-12-09T17:14:15-08:00",
        "Driver": "local",
        "Labels": {
            "com.docker.compose.project": "limelight",
            "com.docker.compose.version": "1.29.1",
            "com.docker.compose.volume": "spectr_storage"
        },
        "Mountpoint": "/var/lib/docker/volumes/limelight_spectr_storage/_data",
        "Name": "limelight_spectr_storage",
        "Options": null,
        "Scope": "local"
    }
]

The directory is the one labeled with Mountpoint. In this case, the data directory is: /var/lib/docker/volumes/limelight_spectr_storage/_data.

Customized Data Location

If you have customized the data storage locations, as per our Customize Data Locations tutorial, this will be the directory you have assigned to the SPECTR_STORAGE_DIRECTORY in your .env file. For example, if the line in your .env file reads:

SPECTR_STORAGE_DIRECTORY=/data/limelight-data/spectr-storage

Your directory is /data/limelight-data/spectr-storage.

Back Up the Data

Using the directory you found above, type the following into a Linux terminal to back up the spectra data:

# replace DIRECTORY_PATH with the directory you found above
sudo tar -C DIRECTORY_PATH -czf /path/to/limelight-spectra-backup.tgz .

# for example:
sudo tar -C /var/lib/docker/volumes/limelight_spectr_storage/_data -czf ~/limelight-spectra-backup.tgz .

This will create a file named limelight-spectra-backup.tgz in your home directory that contains the spectra data. The ~/limelight-spectra-backup.tgz part of the command may be replaced with any directory or filename you prefer to use for your backup. For example:

# for example:
sudo tar -C /var/lib/docker/volumes/limelight_spectr_storage/_data -czf /mnt/d/limelight-backups/limelight-spectra-20210501.tgz .

Would save the backup in the limelight-backups directory of the D:\ drive with the filename limelight-spectra-20210501.tgz.

Note

If you are on a different operating system or want to back up the data in a different way, any method for backing up the above directory should suffice.

Restore Data

Follow the guides below to restore your data from a backup.

Restore MySQL Data

Important

Restoring data will replace your existing data! You will lose any data created after the time of the backup file you are restoring.

Important

The Limelight application must be running to restore MySQL data in this way. Review how to start and stop Limelight.

To restore a Limelight MySQL backup, type the following into your Linux terminal (including Windows users running Docker on WSL2 as per our Installing Docker tutorial).

zcat /path/to/limelight-backup.sql.gz | sudo docker exec -i limelight-mysql sh -c 'exec mysql -u root -p"MYSQL_ROOT_PASSWORD" limelight'

If you are not using Docker to run MySQL, the command would be:

zcat /path/to/limelight-backup.sql.gz | mysql -u root -p"MYSQL_ROOT_PASSWORD" limelight

For example, if the backup file is named limelight-backup-20210501.sql.gz and is in D:\limelight-backups\ on a Windows machine, then run:

zcat /mnt/d/limelight-backups/limelight-backup-20210501.sql.gz | sudo docker exec -i limelight-mysql sh -c 'exec mysql -u root -p"MYSQL_ROOT_PASSWORD" limelight'

Note

MYSQL_ROOT_PASSWORD should be replaced with the actual root password for your MySQL installation. If you followed our Installing Limelight tutorial, this will be in your .env file as described on Step 4.

This will replace your Limelight MySQL database with the data in the backup file.

You should now stop and start the Limelight app. Review how to start and stop Limelight.

Restore Spectra Data

Important

Shut down the Limelight app before restoring spectra data! Review how to start and stop Limelight.

To restore spectra data, determine the directory in which the spectra data should be found by following the directions above.

Type the following into a Linux terminal to restore a spectra data backup:

sudo tar -xzf /path/to/backup.tgz -C /path/to/spectra/data

/path/to/backup.tgz is the location of your backup file made with the instructions above. /path/to/spectra/data is the location where the spectra data should go, determined by following the instructions above.

For example, to restore a backup file named limelight-spectra-20210501.tgz found in D:\limelight-backups to the directory /var/lib/docker/volumes/limelight_spectr_storage/_data type the following:

sudo tar -xzf /mnt/d/limelight-backups/limelight-spectra-20210501.tgz -C /var/lib/docker/volumes/limelight_spectr_storage/_data

You can now start Limelight. Review how to start and stop Limelight.

Getting Help

If you have any questions, bug reports, or feature requests, please contact us!