Why setup a private mirror?
Setting up a private mirror is usually a good idea as soon as you have more than a handful of servers using Freexian’s package repositories:
- it lets you grant access to additional servers without having to go through Freexian
- it ensures that you have a local copy of the repository and that you can continue as usual even if the Freexian server is unreachable for some reason
- it avoids overloading the Freexian server
Step by step explanation to setup a private mirror
The explanations are tailored for Debian systems (version 10 or newer). Adapt them accordingly if you use some other operating system. You must have the rsync username and password given to you by Freexian. The rsync access is provided only to customers with a Pro or Business subscription.
Create a dedicated user
We recommend that you run the mirror under a dedicated user. It makes it easier to setup SSH push mirroring later on and is part of good security practice to isolate external interactions as much as possible.
For the purpose of this explanation, we will call the dedicated user mirror.
Let’s create the user with adduser and fill in the various fields:
$ sudo adduser --disabled-password mirror
Adding user 'mirror' ...
[...]
Is the information correct? [Y/n]
Installing the mirroring tool
We use the ftpsync tool developed by Debian to mirror APT Debian
repositories. You can install it with apt install ftpsync or install
it manually from this
archive
by following the instructions in the provided README.
Configuring the mirroring tool
In the dedicated user’s home directory, you will create the
~/.config/ftpsync/ftpsync-php.freexian.com.conf configuration file as
well as a log directory. We will configure the mirror so that it stores
the files in /srv/mirrors/php.freexian.com.
Note: you must change the value assigned to RSYNC_USER and RSYNC_PASSWORD to match the credentials that you have been given by Freexian.
# sudo mkdir -p /srv/mirrors/php.freexian.com
$ sudo chown mirror:mirror /srv/mirrors/php.freexian.com
$ sudo su - mirror
mirror$ mkdir -p .config/ftpsync log
mirror$ export RSYNC_USER=megacorp
mirror$ export RSYNC_PASSWORD=sekret1
mirror$ cat >.config/ftpsync/ftpsync-php.freexian.com.conf <<END
TO="/srv/mirrors/php.freexian.com/"
BASEDIR="$HOME"
LOGDIR="$HOME/log"
RSYNC_HOST="php.freexian.com"
RSYNC_PATH="php"
RSYNC_USER="$RSYNC_USER"
RSYNC_PASSWORD="$RSYNC_PASSWORD"
END
mirror$
Running the mirror regularly
Running the mirror with cron
To keep your mirror up-to-date, you have to run ftpsync sync:archive:php.freexian.com while being the mirror user.
We recommend that you run it from cron four times a day with a cron entry like this one (assuming that the system time is in UTC):
mirror$ (crontab -l; echo "17 2-23/6 * * * ftpsync sync:archive:php.freexian.com") | crontab -
The freexian package repository is updated at 0h, 6h, 12h, 18h (all times in UTC) and you want to avoid updating your local mirror at those times. Thus the suggestion is to use 2h, 8h, 14h, 20h. You are also invited to tweak the precise minute (17 in the example above) to a random value between 0 and 59 to spread the load so that not all servers connect at the same time.
Running the mirror with SSH push triggers
If you want your mirror to update as soon as we have updated our package repository, then you can configure your mirror to run on Freexian’s request by setting up what we call “SSH push triggers”.
This works by adding a specific entry in the ~/.ssh/authorized_keys file
of the “mirror” user. The key registered is owned by Freexian and will only
be able to run the ftpsync script which starts the mirroring process.
You can setup the file with those commands (to be run with the user dedicated to the mirror):
mirror$ cd $HOME
mirror$ mkdir -p .ssh
mirror$ chmod 700 .ssh
mirror$ wget http://php.freexian.com/public/authorized_keys -O .ssh/authorized_keys
[...]
mirror$ chmod 644 .ssh/authorized_keys
Note that if you have manually installed ftpsync and if it’s not
available in the PATH, then you have to edit the .ssh/authorized_keys
file to indicate the full path in the command parameter.
Once everything is setup, you should ask Freexian to add your mirror to the list of mirrors that we are triggering. Send us a mail at support@freexian.com with the following information:
- the DNS name of the server hosting the mirror
- the username of the account hosting the mirror on your server
- the SSH port if it’s different from the default port (22)
If you restrict access to your SSH port, then you should tweak your firewall to allow php.freexian.com to connect to the SSH port.