Ubuntu Server and WordPress in 15 Minutes Flat

Probably the most common command used to keep an eye on log files is tail. Here are a few examples:

tail -f /var/log/syslog

tail -n25 /var/log/mail.log

tail -F /var/log/lighttpd/error.log

You’ll notice that each command above used a different option. Let me explain what each does.

-f, –follow : output appended data as the file grows

-n, –lines: output the last N lines, instead of the last 10; or use +N to output lines starting with the Nth

-F: same as –follow –retry

–retry: keep trying to open a file even if it is inaccessible when tail starts or if it becomes inaccessible later

Also note that ctrl-c will stop the -f or –follow options.

One thing that I’ve learned in all my troubleshooting and administration is that the system almost always tells you what the problem is. The trick is to simply know where to look. /var/log/messages, /var/log/syslog and other files in the /var/log/ directory will almost surely have the answer.

Apt-Cacher is a method by which you can centrally cache and share packages already stored within your network. It also allows the option of caching any downloaded packages in the future. This allows me to download a package once and transparently share it with an unlimited number of machines within my network. This way I only use my Lanwidth (LAN-bandwidth), and not my limited WAN connection.

Apt-Cacher can be installed and configured network wide in five simple steps. These steps are:

• Install
• Autostart daemon
• Configure ACL
• Import current packages
• Configure Clients

To be honest, the first four steps are finished in just a few minutes. The last step of configuring your clients can take longer. The amount of time depends on how many clients you have.

Install

To install apt-cacher simply install the package (or click the link below):

sudo aptitude install apt-cacher

Autostart daemon

If you want the apt-cacher daemon to autostart at boot you’ll need to change a single value in the config:

sudo sed -i.orig 's/AUTOSTART=0/AUTOSTART=1/' /etc/default/apt-cacher

sudo /etc/init.d/apt-cacher restart

Configure ACL

This step is optional. By default apt-cacher will allow access to any IP. If you would like to limit access to your cache from a specific subnet or other specific addresses you’ll need to change the values for allowed_hosts and denied_hosts in the /etc/apt-cacher/apt-cacher.conf.

Import current packages

You’ll likely want to import the current packages that you have in your machine’s package cache. This can be done using the command:

sudo /usr/share/apt-cacher/apt-cacher-import.pl -s /var/cache/apt/archives/

At this point this machine will have the ability to share any packages that it has previously downloaded as well as any package that it (or any client) will download in the future. The only requirement to begin using this cache throughout your network is configuring the clients to use it.

Configure Clients

To configure your clients to use your cache you can simply add a line to your apt.conf file, telling the client to use the cache server. To do this add the following line to your /etc/apt/apt.conf file, replacing the IP with your own:

Acquire::http::Proxy "http://192.168.0.30:3142"

You’re now done. As long as your apt-cacher is accessible your client will look there for cached packages.

If anyone knows of a method to allow for multiple proxy entries I would be very interested. For example, if this were configured on a laptop and the apt-cacher were not accessible, I would like the client to transparently try the next entry in a list. This is something available in many other package managers, I would hope Debian based distributions would support the same.

Configuring named

The named man page (man named) lists two options for supporting IPv4 or IPv6.  Each of these commands are mutually exclusive, meaning using one of the options will not allow you to use the other.  Either IPv4 or iPv6, not both.  Now the default is to use both, so if you want to continue supporting lookups on IPv4 and IPv6 there is nothing more you need to do.  If you want to *only* use one or the other you can use the -4 or -6 options in the configuration.

IPv4 only (/etc/default/bind9):

# run resolvconf?
RESOLVCONF=yes
# startup options for the server
OPTIONS="-4 -u bind"

IPv6 only (/etc/default/bind9):

# run resolvconf?
RESOLVCONF=yes
# startup options for the server
OPTIONS="-6 -u bind"

Once you have updated this file and defined the option you want, you’ll simply need to restart the BIND service and it’ll start listening on or or the other but, again, not both.

sudo /etc/init.d/bind9 restart

My BIND installation is now listening on only IPv4 and I have yet to see the same slowdown or amount of log output that I used to.  I guess, when we start using IPv6 one of these days I’ll need to change it, but I don’t have a lot of faith in that happening anytime soon.

#!/bin/bash

yum check-update | mail -s “Weekly Errata Report for $(hostname)” admin@domain.tld

After dropping this little one-liner into /etc/cron.weekly/ we’re now notified of available package updates on a weekly basis, and our machines are keeping up to date much better!

I got to thinking this evening about how to achieve the same results on my Ubuntu and Debian servers.  There does not seem to be an equivalent command to ‘yum check-update‘, but there appears to be a similar solution.

#!/bin/bash

(apt-get update && apt-get -s safe-upgrade && apt-get -s full-upgrade) | mail -s “Weekly Errata Report for $(hostname)” admin@domain.tld

The apt-get equivalent command above is a bit more verbose but it basically does the same thing.  note: the parens are required so that the collective output of the three commands will be piped to the mail utility.

Does anyone else have any suggestions an how to accomplish this with standard Ubuntu/Debian tools?

Install The Package

sudo aptitude install logwatch

Configuration

The default configuration can be found in /usr/share/logwatch/default.conf/logwatch.conf.  Take a look around the file, but the main thing you might want to update:

MailTo = root updates to MailTo = user.name@domain.tld

I have my systems email me daily at my gmail address, so each morning I get an overview of the previous days logs.

If you have problems with the mail actually getting out you might want to check out my post regarding Postfix smtp via Gmail.

If you happen to be one that missed his post recently on the planet, or if you’re just finding this via Google, Listadmin is a tool to help you bulk moderate mailman lists. If you are the admin or moderator on more than one mailman list do yourself a favor and keep reading. If you have no idea what I’m talking about, well, thanks for the visit anyway.

Installation

Installing Listadmin is just as simple as installing anything else really. Use the following command (or click the link using apturl):

sudo aptitude install listadmin

Configuration

Before we can begin moderating our lists we’ll need to tell Listadmin which lists to moderate. This is done by creating a file in your home directory ~/.listadmin.ini

With your favorite editor create the file and follow the below example, replacing with your own information:

default skip
log ~/.listadmin.log

# this example uses the same password
# for the users, process and admin lists
adminurl http://{domain}/mailman/admindb/{list}
password mypassword
users@lists.example.org
process@lists.example.org
admin@lists.example.org


# this example uses different passwords
# for the supporters and staff lists
adminurl https://{domain}/mailman/admindb/{list}


password "myotherpassword"
supporters@lists.example.com


password "mythirdpassword"
staff@lists.example.com

As you can see the configuration is pretty simple. The address to the list, the password needed and the lists you want to moderate.

If the lists use the same password you can list them under a single password block.

If the lists use different passwords you can list the by individual password and address.

Once you’ve replaced the details with your own information there is one more step before we run the utility. Considering this file holds sensitive passwords you’ll want to limit access to the file.

chmod go-r .listadmin.ini

Go ahead now and launch the tool and moderate your lists:

listadmin

I’ve been using this now for only two days or so but I’m so thankful I found it. Instead of manually moderating a large number of lists I simply run this tool once a day, clean out the garbage and get back to real work.

So far I am really happy with AWStats. It appears to be running properly and pulling in a lot more data than statcounter ever did. It actually is showing me that I had much more traffic than I thought. I had mentioned 5,000 the other day, which I have long since surpassed based on AWStats output.

I’d like to share the steps I took for installing and configuring AWStats on my Ubuntu Server.

Installation

Ubuntu has the AWStats package available in the repositories, which we can install with:

sudo aptitude install awstats

This will install the basic files, but there is still a bit of configuring to do, so we’ll dive into that next.

Auto Configuration

From the tutorials that I found elsewhere in my searching there is a awstats_configure.pl file that will try to configure it for you. I did not use this personally, but if you’d like to try it you can run:

sudo perl /usr/share/doc/awstats/examples/awstats_configure.pl

The rest of this tutorial will discuss manual configuration, but if anyone can offer feedback concerning the configure script I’m sure many would be interested.

Manual Configuration

I configured my system manually, which I will outline below. The only requirements here are that you have access to the apache2 logs, or that you have custom log locations for each of your virtual domains (if used). For more information on custom log locations for virtual domains see my previous post, Configuring Virtual Hosting on Ubuntu with Apache2.

The first step is creating an awstats.conf file for your domain. This can be done by moving or copying the /etc/awstats.conf, and giving it a more unique name:

sudo cp /etc/awstats/awstats.conf /etc/awstats/awstats.domain.tld.conf

I created a unique file, using the syntax awstats + domain.tld + conf for each of the domains hosted on my server. Each of these also has their own unique log file as well.

We’ll then edit our new /etc/awstats with custom values for that domain. The main points you’ll want to look for within this file:

• LogFile=”/path/to/your/domain/access.log”
• LogFormat=1 (this will give you more detailed stats)
• SiteDomain=”domain.tld”
• HostAliases=”www.domain.tld localhost 127.0.0.1″

Once you’ve made these changes you’ll want to build your initial statistics, which will be generated from the current logs on your machine. We can do this using:

sudo /usr/lib/cgi-bin/awstats.pl -config=domain.tld -update

What this will do is scan the /etc/awstats folder for anything of the pattern awstats + domain.tld + conf, reading that config to generate its output. You should see some output here, and depending on the size of your logs it’ll take anywhere from a few seconds to a few minutes or hours. Each time it is run after that will be minimal, as it only updates the information, but the initial generation can take some time.

Configure Apache to Display AWStats

At this point our statistics should be generated (if not, go back and double check you haven’t missed anything!), but we need a way to see them. We’ll need to configure Apache2 to show us these stats. The way I did this was by using an Include in my apache2.conf, instead of cluttering up the default config file. This is generally my preferred method.

Apache2.conf already has a line near the botton Include /etc/apache2/conf.d/, which will read anything in there as additional data. What I did was create a new file in the /etc/apache2/conf.d/ directory called awstats, and filled it with the following content:

Alias /awstatsclasses "/usr/share/awstats/lib/"
Alias /awstats-icon/ "/usr/share/awstats/icon/"
Alias /awstatscss "/usr/share/doc/awstats/examples/css"
ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
ScriptAlias /awstats/ /usr/lib/cgi-bin/
Options ExecCGI -MultiViews +SymLinksIfOwnerMatch

This is basically creating some access aliases, and defining the cgi-bin paths, etc. Once this is saved you should be able to restart Apache2 and we’ll should be able to access our stats. Restart Apache2 using:

sudo /etc/init.d/apache2 restart

You should now be able to access your statistics using:

http://domain.tld/awstats/awstats.pl

Assuming you didn’t get any errors during your stats generation, and Apache2 didn’t complain when you restarted the service, you should see statistics at this point.

Continually Updating Your Stats

The last thing you’ll probably want to do is update your statistics via cron. This will allow you to have your site statistics updated on a regular basis, not requiring intervention on your part. What I have done is added a line to my /etc/crontab file telling AWStats to update every ten minutes. I have seen minimal system load even when updating a dozen sites on that interval. To update every ten minutes we’ll add the following line:

*/10 * * * * root /usr/lib/cgi-bin/awstats.pl -config=domain.tld -update >/dev/null

Repeat this line, updating the domain.tld value for any site you want continually updated.

Securing Statistics

If you’d like to make your statistics private you might be interested in one of my previous posts, Limiting Access To Websites & Directories with .htaccess.