Ubuntu 16.04 Create LAN Repository with Reprepro
Unless you work in a homogeneously Linux shop, you’re unlikely to even think about setting up your own local repository where your internal servers can grab .deb packages from. The package manager apt-get searches for, downloads, and installs the package for you. A repository in Linux terms is simply the collection of packages, but almost nothing in the *nix world is as simple as it seems. Especially when you have the sometimes unwanted personality trait called perfectionist. A repository can be viewed as the same concept as a caching proxy server, except it is for application packages that you would like to maintain and make available inside your internal network. You can make it public as well, but my article will focus on the use case of creating an internal repository for Ubuntu 16.04 (codename Xenial).
Whenever I determine a task somewhat hard to accomplish due to a lack of resources through search engines like Google, I like to document my work and post it as a blog on my website. Sometimes I even use my own website as reference material because I after a while of not using some knowledge, it inevitably gets tossed into the garbage. How long does it take you to forget material after not using it? Or maybe it’s the early onset of Alzheimer’s
Prerequisites:
- Two Ubuntu 16.04 LTS virtual machines
- Patience
By the end of the guide you will have:
- Prepared and published a repository signing key
- Set up a repository with Reprepro, the repository manager
- Made the repository accessible to internal servers with NGINX
- Added the repository on another server
- Tested pulling and installing a package from your own local repo
First, we need a valid package signing key. This step is crucial for a secure repository, since we will be digitally signing all the packages. Package signing gives the downloader confidence that the source can be trusted.
In this section, you will generate an encrypted master public key and a signing sub-key by following these steps:
- Generate a Master Key
- Generate a Subkey for Package Signing
- Detach Master Key from Subkey
Let’s make the master key. This key should be kept safe and secure since this is what people will be trusting.
Before we begin, let’s install rng-tools though apt-get:
apt-get install rng-tools
GPG requires random data, called entropy, to generate keys. Entropy is normally generated over time by the Linux kernel and stored in a pool. However, on cloud servers, the kernel may have trouble generating the amount of entropy required by GPG. To help the kernel, we install the rngd program (found in the rng-tools package). This program will ask the host server for entropy. Once retrieved, rngd will add the data to the entropy pool to be used by other applications like GPG.
If you get a message like this:
Trying to create /dev/hwrng device inode... Starting Hardware RNG entropy gatherer daemon: (failed). invoke-rc.d: initscript rng-tools, action "start" failed. Start the rngd daemon manually with: rngd -r /dev/urandom
By default rngd looks for a special device to retrieve entropy from /dev/hwrng. Some Virtual machines do not have this device. To compensate we use the pseudo random device /dev/urandom by specifying the -r directive. For more information, you can check out the rng-tools wiki page.
Now that we have a pool of entropy, we can generate the master key. Do this by invoking the command gpg. You will see a prompt similar to the following:
gpg --gen-key gpg (GnuPG) 1.4.16; Copyright (C) 2013 Free Software Foundation, Inc. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.
Please select what kind of key you want: (1) RSA and RSA (default) (2) DSA and Elgamal (3) DSA (sign only) (4) RSA (sign only) Your selection? 1
Specify the first option, “RSA and RSA (default)” 1, in the prompt. Selecting this will have gpg generate first a signing key, then a encryption subkey (both using the RSA algorithm). We don’t need an encryption key for this tutorial, but as a great person once said, “why not?” There is no disadvantage in having both, and you may use the key for encryption in the future.
Hit Enter and you’ll be prompted for a keysize:
RSA keys may be between 1024 and 4096 bits long. What keysize do you want? (2048) 4096
The key size correlates directly to how secure you want your master key to be. The higher the bit size, the more secure key. The Debian project recommends using 4096 bits for any signing key, so I would specify 4096 here. For the next 2-5 years the default bit size 2048 is sufficient if you’d rather use that. A size of 1024 is uncomfortably close to being unsafe and should not be used.
Press Enter for the expire prompt.
Please specify how long the key should be valid. 0 = key does not expire <n> = key expires in n days <n>w = key expires in n weeks <n>m = key expires in n months <n>y = key expires in n years Key is valid for? (0) 0
Master keys do not normally have expiration dates, but set this value as long as you expect to use this key. If you only plan to use this repository for only the next 6 months you can specify 6m. 0 will make it valid forever.
Hit Enter, then y. You will be prompted to generate a user ID. This information will be used by others and yourself to identify this key – so use real information!
You need a user ID to identify your key; the software constructs the user ID
from the Real Name, Comment and Email Address in this form:
"First Last (First Last) <firstlast@company.com>" Real name: Travis Runyard Email address: travis.runyard@example.com Comment: You selected this USER-ID: "Travis Runyard <travis.runyard@example.com>" Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o
If the information is correct, hit o and Enter. We need to add a password to ensure that only you have access to this key. Make sure to memorize this password since there is no way to recover a gpg key password (a good thing).
You need a Passphrase to protect your secret key. Enter passphrase: (hidden) Repeat passphrase: (hidden)
Now for some magic (math) to happen. This might take a little while, so sit back or get a cup of your favorite drink.
We need to generate a lot of random bytes. It is a good idea to perform some other action (type on the keyboard, move the mouse, utilize the disks) during the prime generation; this gives the random number generator a better chance to gain enough entropy. Not enough random bytes available. Please do some other work to give the OS a chance to collect more entropy! (Need 300 more bytes) +++++ ................+++++ We need to generate a lot of random bytes. It is a good idea to perform some other action (type on the keyboard, move the mouse, utilize the disks) during the prime generation; this gives the random number generator a better chance to gain enough entropy. ..+++++ +++++ gpg: /root/.gnupg/trustdb.gpg: trustdb created gpg: key 10E6133F marked as ultimately trusted public and secret key created and signed. gpg: checking the trustdb gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u pub 4096R/10E6133F 2014-08-16 Key fingerprint = 1CD3 22ED 54B8 694A 0975 7164 6C1D 28A0 10E6 133F uid Travis Runyard <travis.runyard@example.com> sub 4096R/7B34E07C 2014-08-16
Now we have a master key. The output shows that we created a master key for signing (`0E6133F on the pub line above). Your key will have different IDs. Make note of your signing key’s ID (the example uses 10E6133F). We’ll need that information in the next steps when creating another subkey for signing.
Generate a Subkey for Package Signing
Now we’ll create a second signing key so that we don’t need the master key on this server. Think of the master key as the root authority that gives authority to subkeys. If a user trusts the master key, trust in a subkey is implied.
In the terminal execute:
gpg --edit-key 10E6133F
Replace the example ID with your key’s ID. This command enters us into the GPG environment. Here we can edit our new key and add a subkey. You’ll see the following output:
gpg (GnuPG) 1.4.16; Copyright (C) 2013 Free Software Foundation, Inc. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Secret key is available. pub 4096R/10E6133F created: 2014-08-16 expires: never usage: SC trust: ultimate validity: ultimate sub 4096R/7B34E07C created: 2014-08-16 expires: never usage: E [ultimate] (1). Travis Runyard <travis.runyard@example.com> gpg>
At the prompt, type addkey:
addkey
Press Enter. GPG will prompt for your password. Enter the password that you used to encrypt this key.
Key is protected. You need a passphrase to unlock the secret key for user: "Travis Runyard <travis.runyard@example.com>" 4096-bit RSA key, ID 10E6133F, created 2014-08-16 gpg: gpg-agent is not available in this session Enter passphrase: <hidden>
You will see the following prompt for key type.
Please select what kind of key you want: (3) DSA (sign only) (4) RSA (sign only) (5) Elgamal (encrypt only) (6) RSA (encrypt only) Your selection? 4
We want to create a signing subkey, so select “RSA (sign only)” 4. RSA is faster for the client, while DSA is faster for the server. We’re picking RSA in this case because, for every signature that we make on a package, possibly hundreds of clients will need to verify it. The two types are equally secure.
Again we are prompted for a key size.
RSA keys may be between 1024 and 4096 bits long. What keysize do you want? (2048) 4096
This tutorial uses 4096 for increased security.
Please specify how long the key should be valid. 0 = key does not expire <n> = key expires in n days <n>w = key expires in n weeks <n>m = key expires in n months <n>y = key expires in n years Key is valid for? (0) 1y
We already have a master key, so the expiration time for the subkey is less important. One year is a good time frame.
Hit Enter, and then type y (yes) twice for the next two prompts. Some math will generate another key.
We need to generate a lot of random bytes. It is a good idea to perform some other action (type on the keyboard, move the mouse, utilize the disks) during the prime generation; this gives the random number generator a better chance to gain enough entropy. ............+++++ .+++++ pub 4096R/10E6133F created: 2014-08-16 expires: never usage: SC trust: ultimate validity: ultimate sub 4096R/7B34E07C created: 2014-08-16 expires: never usage: E sub 4096R/A72DB3EF created: 2014-08-16 expires: 2015-08-16 usage: S [ultimate] (1). Travis Runyard <travis.runyard@example.com> gpg>
Type save at the prompt.
save
In the output above, the SC from our master key tells us that the key is only for signing and certification. The E means the key may only be used for encryption. Our signing key can be correctly seen with only the S.
Note your new signing key’s ID (the example shows A72DB3EF on the second sub line above). Your key’s ID will be different.
Enter save to return to the terminal and to save your new key.
save
Detach Master Key From Subkey
The point of creating the subkey is so we don’t need the master key on our server, which makes it more secure. Now we’ll detach our master key from our subkey. We will need to export the master key and subkey, then delete the keys from GPG’s storage, then re-import just the subkey.
First let’s use the –export-secret-key and –export commands to export the whole key. Remember to use your master key’s ID!
gpg --export-secret-key 10E6133F > private.key gpg --export 10E6133F >> private.key
By default –export-secret-key and –export will print the key to our console, so instead we pipe the output to a new file (private.key). Make sure to specify your own master key ID, as noted above.
Important: Make a copy of the private.key file somewhere safe (not on the server). Possible locations are on a floppy disk or USB drive. This file contains your private key, your public key, your encryption subkey, and your signing subkey.
After you have backed up this file to a safe location, delete the file:
Back up the private.key file before running this:
rm private.key
Now export your public key and your subkey. Make sure to change the IDs to match the master key and the second subkey that you generated (don’t use the first subkey).
gpg --export 10E6133F > public.key gpg --export-secret-subkeys A72DB3EF > signing.key
Now that we have a backup of our keys we can remove our master key from our server.
gpg --delete-secret-key 10E6133F
Re-import only our signing subkey.
gpg --import public.key signing.key
Check that we no longer have our master key on our server:
gpg --list-secret-keyssec# 4096R/10E6133F 2014-08-16 uid Travis Runyard <travis.runyard@example.com> ssb 4096R/7B34E07C 2014-08-16 ssb 4096R/A72DB3EF 2014-08-16
Notice the # after sec. This means our master key is not installed. The server contains only our signing subkey.
Clean up your keys:
rm public.key signing.key
The last thing you need to do is publish your signing key.
gpg --keyserver keyserver.ubuntu.com --send-key 10E6133F
This command publishes your key to a public storehouse of keys – in this case Ubuntu’s own key server. This allows others to download your key and easily verify your packages.
Set Up a Repository Using Reprepro
Now let’s get to the point of this tutorial: creating an apt-get repository. Apt-get repositories are not the easiest things to manage. Thankfully R. Bernhard created Reprepro, who used to “produce, manage and sync a local repository of Debian packages” (also known as Mirrorer). Reprepro is under the GNU licence and completely open source.
Install and Configure Reprepro
Reprepro can be installed from the default Ubuntu repositories.
apt-get update apt-get install reprepro
Configuration for Reprepro is repository-specific, meaning you can have different configurations if you make multiple repositories. Let’s first make a home for our repository.
Make a dedicated folder for this repository and move to it
mkdir -p /var/repositories/ cd /var/repositories/
Create the configuration directory.
mkdir conf cd conf/
Create two empty config files (options and distributions).
touch options distributions
Open up the options file in your favorite text editor (vi is installed by default but to get full functionality install vim).
vi options
This file contains options for Reprepro and will be read every time Reprepro runs. There are several options that you can specify here. See the manual for the other options.
In your text editor add the following by pressing the letter i
ask-passphrase
Press escape then type :wq then hit enter. This will save our changes and return to the console.
The ask-passphrase directive tells Reprepro to request a GPG password when signing. If we don’t add this to the options Reprepro will die if our key is encrypted (it is).
Open the distributions file.
vi distributions
This file has four required directives. Add these to the file:
Codename: xenial Components: main Architectures: i386 amd64 SignWith: A72DB3EF
The Codename directive directly relates to the code name of the released Debian distributions and is required. This is the code name for the distribution that will be downloading packages, and doesn’t necessarily have to match the distribution of this server. For example, the Ubuntu 16.04 LTS release is called xenial, Ubuntu 14.04 LTS is called trusty, Ubuntu 12.04 LTS is called precise, and Debian 7.6 is known as wheezy. This repository is for Ubuntu 16.04 LTS so xenial should be set here.
The Components field is required. This is only a simple repository so set main here. There are other namespaces such as “non−free” or “contrib” – refer to apt-get for proper naming schemes.
Architectures is another required field. This field lists binary architectures within this repository separated by spaces. This repository will be hosting packages for 32-bit and 64-bit servers, so i386 amd64 is set here. Add or remove architectures as you need them.
To specify how other computers will verify our packages we use the SignWith directive. This is an optional directive, but required for signing. The signing key earlier in this example had the ID A72DB3EF, so that is set here. Change this field to match the subkey’s ID that you generated.
Save and exit from the file by pressing escape then :wq and Enter.
You have now set up the required structure for Reprepro.
Add a Package(s) with Reprepro
First let’s change our directory to a temporary location.
mkdir -p /tmp/debs
cd /tmp/debs
We need some example packages to work with – wget them with:
wget https://github.com/Silvenga/examples/raw/master/example-helloworld_1.0.0.0_amd64.deb wget https://github.com/Silvenga/examples/raw/master/example-helloworld_1.0.0.0_i386.deb
These packages contain a simple bash script to prove the functionality of our repository. If you would prefer to simply move sync/copy all the .deb’s already cached on your computer use this command:
rsync -av /var/cache/apt/archives/ /tmp/debs/
Running the program ls should give us this layout:
example-helloworld_1.0.0.0_amd64.deb example-helloworld_1.0.0.0_i386.deb
We now have two example packages. One for 32-bit (i386) computers, another for 64-bit (amd64) computers. You can add them both to our repository with:
reprepro -b /var/repositories includedeb xenial example-helloworld_1.0.0.0_*
The -b argument specifies the “(b)ase” directory for the repository. The includedeb command requires two arguments – < distribution code name > and < file path(s) >. Reprepro will prompt for our subkey passcode twice. If you want to specify all deb files in the directory, simply use *.deb.
Exporting indices... C3D099E3A72DB3EF Travis Runyard <travis.runyard@example.com> needs a passphrase Please enter passphrase: < hidden > C3D099E3A72DB3EF Travis Runyard <travis.runyard@example.com> needs a passphrase Please enter passphrase: < hidden > Success!
Listing and Deleting
We can list the managed packages with the list command followed by the codename. For example:
reprepro -b /var/repositories/ list xenial xenial|main|i386: example-helloworld 1.0.0.0 xenial|main|amd64: example-helloworld 1.0.0.0
To delete a package, use the remove command. The remove command requires the codename of the package, and the package name. For example:
reprepro -b /var/repositories/ remove xenial example-helloworld
Make the Repository Public
We now have a local package repository with a couple of packages. Next, we’ll install Nginx as a web server to make this repository available to your internal network.
Install Nginx
apt-get update apt-get install nginx
Nginx comes installed with a default example configuration. Make a copy of the file in case you want to look at it at another time.
mv /etc/nginx/sites-available/default /etc/nginx/sites-available/default.bak touch /etc/nginx/sites-available/default
Now that we have an empty configuration file, we can start configuring our Nginx server to host our new repository.
Open the configuration file with your favorite text editor.
vi /etc/nginx/sites-available/default
And add the following configuration directives:
server {
listen 80 default_server;
listen [::]:80 default_server;
## Let your repository be the root directory
root /var/repositories;
## Always good to log
access_log /var/log/nginx/repo.access.log;
error_log /var/log/nginx/repo.error.log;
location / {
index index.php index.html;
autoindex on;
}
## Prevent access to Reprepro's files
location ~ /(db|conf) {
deny all;
return 404;
}
}
Nginx has some pretty sane defaults. All we needed to configure was the root directory, while denying access to Reprepro’s files. See the in-line comments for more details.
Restart the Nginx service to load these new configurations.
service nginx restart
Or you can simply reload the configuration without restarting the daemon.
nginx –s reload
Your public Ubuntu repository is ready to use!
You’ll need your Virtual machine’s IP address to let users know the location of the repository. If you don’t know your Virtual machine’s public address you can find it with ifconfig.
ifconfig eth0 eth0 Link encap:Ethernet HWaddr 04:01:23:f9:0e:01 inet addr:192.168.1.100 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::601:23ff:fef9:e01/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:16555 errors:0 dropped:0 overruns:0 frame:0 TX packets:16815 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:7788170 (7.7 MB) TX bytes:3058446 (3.0 MB)
In the above example, the server’s address is 192.168.1.100. Yours will be different.
With your Reprepro server’s IP address, you can now add this repository to any other appropriate server.
Install a Package from Our New Repository
If you haven’t already, spin up another Virtual machine with Ubuntu 16.04 LTS, so that you can do a test installation from your new repository.
On the new server, download your public key to verify the packages from your repository. Recall that you published your key to keyserver.ubuntu.com.
This is done with the apt-key command.
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 10E6133F
This command downloads the specified key and adds the key to the apt-get database. The adv command tells apt-key to use GPG to download the key. The other two arguments are passed directly to GPG. Since you uploaded your key to “keyserver.ubuntu.com” use the –keyserver keyserver.ubuntu.com directive to retrived the key from the same location. The –recv-keys <key ID> directive specifies the exact key to add.
Now add the repository’s address for apt-get to find. You’ll need your repository server’s IP address from the previous step. This is easily done with the add-apt-repository program.
add-apt-repository "deb http://192.168.1.100/ xenial main"
Note the string that we give add-apt-repository. Most Debian repositories can be added with the following general format:
deb (repository location) (current distribution code name) (the components name)
The repository location should be set to the location of your server. We have an HTTP server so the protocol is http://. The example’s location was 192.168.1.100. Our server’s code name is xenial. This is a simple repository, so we called the component “main”. You can also simply append the text inside quotations at the end of the /etc/apt/sources.list file
After we add the repository, make sure to run an apt-get update. This command will check all the known repositories for updates and changes (including the one you just made).
apt-get update
After updating apt-get, you can now install the example package from your repository. Use the apt-get command normally.
apt-get install example-helloworld
If everything is successful you can now execute example-helloworld and see:
Hello, World! This package was successfully installed! Congratulations! You have just installed a package from the repository that you created!
To remove the example package, run this command:
apt-get remove example-helloworld
This removes the example package that you just installed.
