Centralising your tools in a custom repository – Part 2

During this blog post I will talk about creating your own custom package repository. However, before you can setup a repository, you need packages. This previous post talks about setting up your custom packages.

Chapter 2: creating your custom repository

So you have packed all your tools into neat packages and you can install them on all your systems. All fine and good, but you have to copy all files continuously to all your systems. It would be so much nicer if you could have your packages installed the same way as your Linux packages. Off course, you cannot add your own packages to redhat’s or CentOS’s own repository, but you can create very easily your own repository.

Afterward the only thing you have to do is;

yum install changecase

And when you have an update or bugfix, you can just issue an

yum update

on all systems and your software will automatically update.

Setting up a repository:

Setting up a repository is easy. You only need to have a web server running (I use apache, but anyone will do). All packages must be inside a directory on the web server and every YUM package managers can use your repository.

For this example, we assume you have a working webserver, which hosts the website rpm.noxlogic.nl and this points to the /wwwroot/rpm.noxlogic.nl on your local web server. We also use CentOS 5 as the current OS you need, but it’s pretty easy to support multiple OS’es. Just make sure your packages work on the target OS and add them to the correct directory.

cd /wwwroot/rpm.noxlogic.nl
mkdir centos/5/noxlogic/{SRPMS,x86_64,noarch,i386}
cp ~/buildroot/SRPMS/* centos/5/noxlogic/SRPMS
cp ~/buildroot/x86_64/* centos/5/noxlogic/x86_64
cp ~/buildroot/i386/* centos/5/noxlogic/i386
cp ~/buildroot/noarch/* centos/5/noxlogic/noarch

This creates all the necessary directories. We have 4 kind of packages: the SRPMS, where you find all the source RPM packages. The noarch, in which are the packages which are platform independent, the x86_64 are 64bit packages for intel systems and i386 are the (oldish) 32bit versions. Unless you have binary tools, you probably have most packages inside the noarch directory (php-scripts, bash-script, python etc).

Now that all packages are copied to the correct places, we can create the actual repository. To be precise: it will create a “repodata” directory inside each architecture directory with file information about all the packages found.

createrepo -v /wwwroot/rpm.noxlogic.nl/centos/5/noxlogic/{SRPMS,x86_64,noarch,i386}

From now, we have a fully functional repository which you can use. Whoohoo!

Using your repository

To let your system use the new repository, you must tell it to use it by adding a configuration file.

Create the file /etc/yum.repos.d/noxlogic.repo and add the following:

[noxlogic]
name = NoxLogic repository
baseurl = http://rpm.noxlogic.nl/centos/5/noxlogic/$basearch
enabled = 1
gpgcheck = 1
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-noxlogic

[noxlogic_noarch]
name = NoxLogic repository - noarch
baseurl = http://rpm.noxlogic.nl/centos/5/noxlogic/noarch
enabled = 1
gpgcheck = 1
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-noxlogic

[noxlogic_source]
name = NoxLogic repository - source
baseurl = http://rpm.noxlogic.nl/centos/5/noxlogic/SRPMS
enabled = 1
gpgcheck = 1
gpgkey = file:///etc/pki/rpm-gpg/RPM-GPG-KEY-noxlogic

Off course you must change the URL and the company name to your specific needs. This files tells you to use both the “noarch” and the source rpms’s of your custom repository. All files are checked against a GPG to ensure that you actually download the package you intended to. The $basearch variable is the architecture of your server. So when you are running on a x86_64 system, yum will only use the 64bit packages, when on a i386, it will only use the 32bit versions. We specify the noarch and source rpms separately.

Since all files are checked against a GPG key, we must download this key onto your system. For now, you can put the GPG key (which you have created in the first chapter to sign your packages with) in the root directory of your web server so we can download it by issuing:

mkdir -p /etc/pki/rpm-gpg
cd /etc/pki/rpm-gpg
wget http://rpm.noxlogic.nl/RPM-GPG-KEY-noxlogic

When yum downloads a package, it will check the signature (unless gpgcheck is set to 0 in the repo-file above).

Setting up a repository install package

By now you have seen that adding the repository requires a bit of work. Off course, this can be automated. We could create a package with two files (the /etc/yum.repos.d/noxlogic.repo  and the /etc/pki/rpm-gpg/RPM-GPG-KEY-noxlogic file).

Now the only thing we need to do is download and install this file, and the repository is ready for use. Large custom repositories (like rpmforge) use the same method. Now, enabling the repository is simply running:

wget http://rpm.noxlogic.nl/repository.rpm
rpm -ihv repository.rpm

Conclusion

Setting up the repository is actually the easiest part to do. Create your directories, copy all your packages and issuing a “createrepo” is enough to get you going. In the next chapter we will talk about automating these steps by linking your subversion repository to this process, so a new release inside your subversion automatically creates a new RPM.

When you get the hang of creating your packages, using a custom repository is ideally for large scale deployment. You don’t have to worry about having the latest release of your tools since a simple “yum update” will update to the latest release from your repository. Bug fixes to critical tools are deployed in a heartbeat and you might even have custom made php-packages or “dummy”-packages which only job it is to install other packages. For instance: you can create a lamp-packages, which depends on apache, mysql, php, php-mysql, and all other tools you need on your lamp servers.  Instead of using deployment-scripts, you can easily create packages for your software that do the deployment and let the system worry about up- or downgrades. Does your tool need php 5.3 or higher? Just add the dependency and yum will automatically update php if you like. The possibilities are endless!

In the last chapter I will talk about connecting SVN to your repository and about multi-distro repositories. Stay tuned!