I was discussing the creation and maintenance of an Ubuntu repository mirror in IRC today and realized that I haven’t blogged about how to do that. Its funny sometimes how topics pop up, and something I’ve been using for months never made it onto the blog. In any event, below I’ll outline how to create your own Ubuntu repository mirror–great for consolidating and saving bandwidth for multiple machines!
Creating the Mirror
To create an Ubuntu repository mirror we’ll use the apt-mirror tool. This can be installed by issuing the following command or clicking the link:
sudo aptitude install apt-mirror
Once this package is installed you’ll need to configure a few items:
- Where will your mirror be stored?
- How aggressive will you be about downloading?
- What do you want to mirror?
To edit these values you’ll want to edit the /etc/apt/mirror.list file. I’ve displayed mine below, and broken it down into the three main sections.
Where Will Your Mirror Be Stored?
I’ve configured my apt-mirror to be stored on an external USB drive. This is defined via the line:
set base_path /media/gurudisk/UBUNTU/MIRROR
This overrides the default value, which stores the data in /var/spool/apt.
How Aggressive Will You Be About Downloading?
This title may be a bit misleading but this is the way I think about it. Technically its the number of download threads you want to use. The higher the number, the more threads, the faster its done. This will depend on your bandwidth connection though. On my home DSL I can’t put this number any higher than 2. In faster locations you’ll be able to put the number much higher. Your milage may vary.
set nthreads 2
What Do You Want To Mirror?
Now, the most important part, is what do you actually want to mirror? The syntax for this section is pretty simple. It looks just like a normal sources.list entry, but for each address you want to mirror. Here is an example of my entry for Ubuntu 8.04, i386:
# Ubuntu 8.04 "Hardy" i386
deb-i386 http://archive.ubuntu.com/ubuntu hardy main restricted universe multiverse
deb-i386 http://archive.ubuntu.com/ubuntu hardy-updates main restricted universe multiverse
deb-i386 http://archive.ubuntu.com/ubuntu hardy-backports main restricted universe multiverse
deb-i386 http://archive.ubuntu.com/ubuntu hardy-security main restricted universe multiverse
deb-i386 http://archive.ubuntu.com/ubuntu hardy-proposed main restricted universe multiverse
If you want amd64 bit mirrored you might add entries like this:
# Ubuntu 8.04 "Hardy" amd64
deb-amd64 http://archive.ubuntu.com/ubuntu hardy main restricted universe multiverse
deb-amd64 http://archive.ubuntu.com/ubuntu hardy-updates main restricted universe multiverse
deb-amd64 http://archive.ubuntu.com/ubuntu hardy-backports main restricted universe multiverse
deb-amd64 http://archive.ubuntu.com/ubuntu hardy-security main restricted universe multiverse
deb-amd64 http://archive.ubuntu.com/ubuntu hardy-proposed main restricted universe multiverse
Notice that I’m mirroring main, restricted, universe and multiverse plus the security, proposed, backports and updates sections. This gives me a complete mirror of everything that would normally be available. If you don’t want/need proposed or backports, etc you can simply remove those lines.
If you want to be able to use your mirror to support network based installations you’ll also need to add the following (depending on your architecture):
deb-amd64 http://archive.ubuntu.com/ubuntu hardy main main/debian-installer
deb-amd64 http://archive.ubuntu.com/ubuntu hardy restricted restricted/debian-installer
deb-amd64 http://archive.ubuntu.com/ubuntu hardy universe universe/debian-installer
deb-amd64 http://archive.ubuntu.com/ubuntu hardy multiverse multiverse/debian-installer
or
deb-i386 http://archive.ubuntu.com/ubuntu hardy main main/debian-installer
deb-i386 http://archive.ubuntu.com/ubuntu hardy restricted restricted/debian-installer
deb-i386 http://archive.ubuntu.com/ubuntu hardy universe universe/debian-installer
deb-i386 http://archive.ubuntu.com/ubuntu hardy multiverse multiverse/debian-installer
Basically just add the sources.list entry to this file for anything you want to mirror. I also mirror Medibuntu and a few PPA locations. It sure makes for faster installation when I am able to use the LAN as compared to the public tubes.
To keep your mirror updated activate the cron entry in /etc/cron.d/apt-mirror. By default it will try to update your mirror at 4:00am, and it is disabled. To activate it simply uncomment the line and change the 4 to the preferred hour of your choice. Enjoy!
Update: My mirror, including main, restricted, universe, multiverse, -proposed, -backports, -security, Medibuntu and the network installation sections takes up 37G. I do mirror i386 and amd64, so if you just need one architecture you could half-that.
Nice tutorial. However, note that the default ubuntu config file is /etc/apt/mirror.list, so you should run the update:
apt-mirror -c /etc/apt/mirror.list
The config file name in the original article was wrongly specified as apt-mirror.
Ian.
Whoops… made a mistake myself:) The correct command is:
apt-mirror /etc/apt/mirror.list
Ian.
So after creating the repository mirror how does one access it from the client computers?
-Thank You
@Nicco – simply update the clients to point to something like:
deb http://ip.address.of.mirror/ubuntu hardy main restricted, etc..
It’ll require that the mirror machine has some web server installed (apache, lightty, etc). Otherwise its the same format as normal, just using your local IP vs the public address.
Thank You!
Hi Christer,
I’m a little unclear about the syntax in the mirror.list file.
Is it necessary to list the d-i packages as:
deb-i386 http://archive.ubuntu.com/ubuntu hardy main main/debian-installer
?
Can the debian-installer branches be listed as:
deb-i386 http://archive.ubuntu.com/ubuntu hardy main/debian-installer
Or just incorporated into the first set?
Also, I’ve found that mirroring the d-i packages under updates and security breaks my network install: ie. placing:
deb http://archive.ubuntu.com/ubuntu hardy-security main/debian-installer
Resulted in a broken install. I had to clean and and purge the mirror. Any idea why this is the case?
Thanks for the tutorial.
-DP
I am doing your steps into Hardy Heron but I can´t do than it works. I give me an error from ../bin/apt-mirror
I am not sure what can be but is difficult to me find the solve.
Thxs
I found a “low-tech” solution for when setting a local mirror is too much for a few computers:
http://nxadm.wordpress.com/2008/10/31/home-network-low-tech-trick-for-a-very-fast-ubuntu-release-upgrade/
(BTW: I set up your solution at work and I am the hero 🙂 )
I have configured a machine with the mirror but do I need to update the mirror list and remove the http entries and simply just add the ip address of the server that mirrors all the updates?
Hi!
If the amount of memory allotted on a particular server is small (less than the 37GB or 18gB you mentioned), what are the directories that can be excluded without causing too much of a flutter?
Hi,
I am new to the forum. Please can you help me with a apt-mirror issue.
I have three repository's but I no longer need one of them. How do I remove it without re-downloading the once I want?
Regards.
If you simply remove the configuration lines regarding the repositories you no longer want you should be set. It should not affect the repositories or packages that you want to keep, and should also autoclean the removed repository contents.
My portable programm can make Ubuntu repository mirror, calculate total downloading size, remove old packages from repository.
Picture:
http://mozgoed-mgoy.narod.ru/ubuntu/screen.jpg
Programm link:
http://mozgoed-mgoy.narod.ru/ubuntu/ubuntu-repository.zip
Required:
Windows 2000, Windows XP – .Net Framework 2.0
Windows Vista, Windows 7 – Nothing
Ubuntu 10.10 – Mono project packages. Programm can be executed from terminal “mono ubuntu-repository.exe”
Let’s work with visual tools! =)