Ubuntu Tutorials

I’ve written a detailed article on the topic of creating your own Ubuntu repository mirror using the Apt-Mirror utility. This article also includes leveraging an Apt-Cacher, for consolidating and saving bandwidth between multiple machines. We regularly make use of both of these utilities to maintain our Ubuntu and Debian installations in my workplace. We have maintained an internal repository to support our developers (all running Ubuntu!) for over a year. I also make use of an Apt-Cacher at home, which is a great boost to my limited bandwidth.

If you’d like to read my latest writeup on leveraging these two tools, see my article at Packt Publishing.

I’m sure that I’ve mentioned a number of times that the bandwidth speeds at my house are pretty horrible. It is very frustrating to have such a limited pipe considering the amount of work I do that requires bandwidth. Based on this limitation I regularly come up with ways to conserve and cache. One of these methods (which I have blogged about in the past) is apt-cacher.

Apt-Cacher is a method by which you can centrally cache and share packages already stored within your network. It also allows the option of caching any downloaded packages in the future. This allows me to download a package once and transparently share it with an unlimited number of machines within my network. This way I only use my Lanwidth (LAN-bandwidth), and not my limited WAN connection.

Apt-Cacher can be installed and configured network wide in five simple steps. These steps are:

• Install
• Autostart daemon
• Configure ACL
• Import current packages
• Configure Clients

To be honest, the first four steps are finished in just a few minutes. The last step of configuring your clients can take longer. The amount of time depends on how many clients you have.

Install

To install apt-cacher simply install the package (or click the link below):

sudo aptitude install apt-cacher

Autostart daemon

If you want the apt-cacher daemon to autostart at boot you’ll need to change a single value in the config:

sudo sed -i.orig 's/AUTOSTART=0/AUTOSTART=1/' /etc/default/apt-cacher

sudo /etc/init.d/apt-cacher restart

Configure ACL

This step is optional. By default apt-cacher will allow access to any IP. If you would like to limit access to your cache from a specific subnet or other specific addresses you’ll need to change the values for allowed_hosts and denied_hosts in the /etc/apt-cacher/apt-cacher.conf.

Import current packages

You’ll likely want to import the current packages that you have in your machine’s package cache. This can be done using the command:

sudo /usr/share/apt-cacher/apt-cacher-import.pl -s /var/cache/apt/archives/

At this point this machine will have the ability to share any packages that it has previously downloaded as well as any package that it (or any client) will download in the future. The only requirement to begin using this cache throughout your network is configuring the clients to use it.

Configure Clients

To configure your clients to use your cache you can simply add a line to your apt.conf file, telling the client to use the cache server. To do this add the following line to your /etc/apt/apt.conf file, replacing the IP with your own:

Acquire::http::Proxy "http://192.168.0.30:3142"

You’re now done. As long as your apt-cacher is accessible your client will look there for cached packages.

If anyone knows of a method to allow for multiple proxy entries I would be very interested. For example, if this were configured on a laptop and the apt-cacher were not accessible, I would like the client to transparently try the next entry in a list. This is something available in many other package managers, I would hope Debian based distributions would support the same.

With only four days to go before Ubuntu 8.10 is released we should start preparing.  Those of us that know we are going to be upgrading, or would like to seed torrents, should do what we can ahead of time to help conserve bandwidth.  Here are some things you can do to be prepared come release day:

1. Use jigdo to download your image(s) now, and simply do an incremental download on release day.
2. Make sure your internal mirrors are up-to-date.
3. Use apt-cacher if you’ll be using network-upgrade to update multiple machines.
4. Volunteer your internal mirror to your local release party.
5. Use a local or regional mirror and take the load off the main sites.

Ubuntu 8.04 “Hardy Heron” is released!  Definitely cause for celebration with many, many new features and improvements!  I figure a hot search item for the next little bit will be regarding how to upgrade.  I thought I’d share a collection of links that will hopefully cover enough of the bases to cover everyone.  Remember, the interweb tubes are going to be fairly clogged today so don’t be surprised if core Ubuntu sites take forever to load or don’t load at all.  I notice the Ubuntu Wiki is already overloaded this morning.  I’m hoping I’ve covered enough of the bases with recent posts to get everyone covered.

I’ve also linked posts from previous releases.  The steps are generally similar enough to give you a good idea.  Feel free to use them as a reference but not pure-gospel.

Upgrade Ubuntu 6.06 or 7.10 to Ubuntu 8.04

Upgrade 7.10 “Gutsy” to Ubuntu 8.04 “Hardy” (ubuntu-tutorials)

Upgrade 6.06 “Dapper” to Ubuntu 8.04 “Hardy” (ubuntu-tutorials)

Upgrade from Ubuntu Release Candidate to Ubuntu 8.04 Final (ubuntu-tutorials)

Upgrade Ubuntu Images to Final with Jigdo (ubuntu-tutorials)

Upgrade Ubuntu 6.06 “Dapper” Server to Ubuntu 8.04 “Hardy” Server (ubuntu-tutorials)

Upgrade Ubuntu 7.10 (Gutsy Gibbon) to Ubuntu 8.04 LTS (Hardy Heron) Beta (ubuntu geek)

Ubuntu Community Wiki Upgrade Notes (community wiki)

Previous Release Steps

How To Upgrade to Ubuntu 7.10 (ubuntu-tutorials)

Upgrade Ubuntu 7.04 Server to Ubuntu 7.10 (ubuntu-tutorials)

Upgrade Problems?  Try A Different Repository Mirror (ubuntu-tutorials)

Caching Upgrade for Multiple Machines with Apt-Cacher

If you plan on upgrading multiple machines on a single network using the above options please, for your sake and for the sake of the interweb tubes, set up something like Apt-Cacher to make your upgrades more bandwidth efficient!  Basically, if you’ve got two or three machines you plan on upgrading, Apt-Cacher will pull down all the required packages for the first machine & then cache them for the remaining machines.  The second and third (and fourth.. and fifth..) will upgrade tremendously faster and you’ll help unclog the interweb tubes for everyone else

Over the holidays I was visiting my parents and helping my brother with his Ubuntu system. One of the biggest drawbacks of heading up that way for a visit is leaving the fiber-optic comforts of home and heading up to DSL villa. I swear its like rewinding a decade and hearing the hellish dial-up noises all over again!

In any event, I was helping my brother update his Ubuntu machine and was getting tired of seeing the estimated download time hover around 18hrs! It was about that time that I got to thinking… “my laptop is up to date. There should be a way for him to simply download the updates from my machine over the LAN.” After some Google magic I came across Apt-Cacher and a cure was found!

“Apt-cacher performs caching of .deb packages which have been downloaded by local users. When a package is requested, the cache checks whether it already has the latest version, in which case it sends the package to the user immediately. If not, it downloads the package while streaming it to the user at the same time. A local copy is then kept for use by other users.”

Apt-Cacher will allow you to centrally cache any updates or installed packages on a central machine and help you save bandwidth. No matter if you have two Ubuntu machines or twenty, as far as your bandwidth is concerned your ISP will only think you have one! Below are steps for setting up Apt-Cacher.

First, of course, we’ll need to install it (it only needs to be installed on one central machine).

sudo aptitude install apt-cacher

To test that it is installed correctly you can try visiting the address below:

http://[localhost]:3142/ or http://[LOCAL.IP]:3142/

If you see the Apt-Cacher page the server end of things are setup. There are a few other things you can tweak, and you’ll probably want to take a look at a few configuration options such as allowed_hosts and denied_hosts. You can edit these settings in the /etc/apt-cacher/apt-cacher.conf

allowed_hosts=192.168.0.0/24 (to allow all local machines)

denied_hosts=

You also need to configure Apt-Cacher to “go live” so you’ll need to change one more option in another file. Simply change the AUTOSTART within /etc/default/apt-cacher from 0 to 1.

sudo vim /etc/default/apt-cacher

AUTOSTART=0

to

AUTOSTART=1

And one last option to run on the server machine is to import any existing .deb packages on that machine into the Apt-Cache database. Use the following command to import your current cache:

sudo /usr/share/apt-cacher/apt-cacher-import.pl /var/cache/apt/archives

After you’ve made any of these changes to the apt-cacher.conf file you’ll need to restart Apt-Cacher by using:

sudo /etc/init.d/apt-cacher restart

The next thing to do is simply point your other local machines to this central machine instead of the central repositories. You’ll want to first backup your original sources.list using the command:

sudo cp /etc/apt/sources.list /etc/apt/sources.list-backup

And then simply edit your the file to look something more like:

deb http://[LOCAL.IP]:3142/archive.ubuntu.com/ubuntu/ edgy main restricted
deb http://[LOCAL.IP]:3142/archive.ubuntu.com/ubuntu/ edgy universe multiverse
deb http://[LOCAL.IP]:3142/archive.ubuntu.com/ubuntu/ edgy-updates main restricted
deb http://[LOCAL.IP]:3142/archive.ubuntu.com/ubuntu/ edgy-updates universe multiverse
deb http://[LOCAL.IP]:3142/archive.ubuntu.com/ubuntu/ edgy-security main restricted
deb http://[LOCAL.IP]:3142/archive.ubuntu.com/ubuntu/ edgy-security universe multiverse
You’ll notice that I’ve simply prefixed archive.ubuntu.com with the local network address of the machine running Apt-Cacher and specified the default port. This is a broken down version of a sources.list for the sake of space. From there on everything should run the same as normal.

Now you can give things a spin by checking for updates as normal.

sudo aptitude update && sudo aptitude upgrade

Everything should appear as normal. Remember you can also continue to use this to install individual packages on each machine. Not all the machines need to carry the same either. Whatever is requested from any machine will be checked on the Apt-Cacher server and retrieved if it isn’t yet available.

For more information on using Apt-Cacher you might be interested in these pages:

How to setup a repository cache with Apt-Cacher

Nick Andrew’s Apt-Cacher Page

Debian Universe Apt-Cacher Information