Working with log files in the /var/log/ directory Part 1

The point of this tutorial is to give a better understanding of how to look at the log files in your /var/log directory. Using the built-in command line tools available, if you are not using some special purpose monitoring suit. This tool is in the intermediate level, all though those with lost of *nix experiance will find certain sections tedious as I describe what I typed in detail, for novice first time users. This is not how ever a tutorial on vim or any of the other tools.

I will be looking at apache logs and dmesg or syslog files. But with a good understanding of these tools you can adapt them to other log files.

The basic command you should be familiar with are:
vim a text editor
cat a tool to out put the contents of a text file. 
sed a stream editor
grep a text search tool
cut a tool to cut words or characters from lines of text
head a tool to out put the first n lines of a text file. 
tail a tool to out put the last n lines of a text file.
less a tool to out put the contents of a text file.
sort tool to sort lines of text
uniq tool to find unique lines of from sorted text
tee tool to out put text passing through a pipe
wc count the lines, characters and bytes in file. 

Some other tools that would be worth looking at are:
ssh
bash especially how redirects and pipes work. "> < | ".

At some point you need to go and read the man pages on all those command, and applications.

Part 1 Using Vim

Why use vim, because you can use some very power full text search and replace filters to get some useful info from you log files. The other tools allow you to do the same thing in a much more automated way.

Counting occurrences of a particular line or error message. One of the most useful things you can do is be able to get some stats like a counts of a particular error message.

In vim we use a very similar syntax to what we use in sed.
First we open the log file in a read only mode. You could use view, but in some environments view is compiled with out certain options. Which makes using the visual selection commands impossible.

vim -R /var/log/syslog

Next I looked for some text that I want to get a count of. "local kernel:" seamed to stand out. I entered the visual mode using the "v" key and selected the text I want to search for "local kernel". I yanked the text with "y", which is similar to copying. Next I stared the global search tool by typing in ":g/" now I want to past the text I just copied so I use `ctlr+r+"` to past the text I just copied into the command.

g/local kernel:/

After I pressed enter, global brings up a list of all occurrences of the provided text "local kernel:". It was a long list. To see how long that list is I can use the substitute command. Global allows you to add the substitute command to the end of your search. So I entered the command ":" mode again and pressed the up arrow to get to the previous command and add "s///n" at the end. Which will give me a count for my search string.

g/local kernel:/s///n 
47918 substitutions on 47918 lines 

Now that I know how many times that lines with local kernel is found in the file, I want to remove those lines to see what else might be needing my attention. Again I enter command mode ":" and press the up arrow. I then change my global command to look some thing link this.

:g/local kernel:/d

W10: Warning: Changing a readonly file
47918 fewer lines

That warning about being read only can be ignored, because that is how I want this process to work. I don't want to remove vital information from a production log file. It would probably be a good Idea to copy these files some where else before doing what I am doing but with some 4 GB monster log files this may not be an option. Repeat this process until you are left with and empty log file or unique strings. 

This is process is useful to see how many times certain errors are cropping up. Some times it is easier to focus on fixing on problem that  accounts for 90% of the errors. But using vim in this way gets tiresome very quickly. The faster and more automated way of doing this is to use your command line and eventually to script the processes. 

Part 2 Using just the command line

The tool that makes command line processing of log files to get stats about the number of error messages is uniq. The problem with most log files is that unique on every line because of date information and process information.  So we need a way to get around this problem.

For starters I generate some 404 page not found errors in my apache log file.


 wc /var/log/apache2/error.log
  28  386 3421 /var/log/apache2/error.log

 wc /var/log/apache2/error.log
  81 1075 8863 /var/log/apache2/error.log

wc gives me some stats on the file like how many lines are in it how many charters and lastly how big the file is in bytes. So when I try to run the file through uniq it comes out  the same size. 

uniq /var/log/apache2/error.log | wc
     81    1075    8863

uniq needs the files sorted inorder to be able to give us the count of duplicate lines in a file or to produce a list of only those lines that are uniq. So just buy sorting the file I am already removing some of the duplicates. Now if I want to get some stats on the duplicates I can by telling "uniq -c" to count lines and "uniq -d" to only show duplicates. Read the man page for more info. 

sort /var/log/apache2/error.log | uniq | wc

     76    1010    8358

sort /var/log/apache2/error.log | uniq -dc | wc
      5      70     545


sort /var/log/apache2/error.log | uniq -dc 
      2 [Tue May 15 09:54:10 2012] [error] [client 127.0.0.1] File does not exist: /var/www/html/favicon.ico
      2 [Tue May 15 09:54:13 2012] [error] [client 127.0.0.1] File does not exist: /var/www/html/favicon.ico
      2 [Tue May 15 09:54:14 2012] [error] [client 127.0.0.1] File does not exist: /var/www/html/favicon.ico
      2 [Tue May 15 09:54:23 2012] [error] [client 127.0.0.1] File does not exist: /var/www/html/favicon.ico
      2 [Tue May 15 09:54:27 2012] [error] [client 127.0.0.1] File does not exist: /var/www/html/favicon.ico

As  you can see favicon.ico is not found, but because of the date it is not producing an accurate count of that "File does not exist error". I need some way to remove that information from file before sending it to uniq.

The first and simplest way to do that is with the "cut" command.  Using the magic of pipes I am going to use head and cut together to so I don't get my terminal scrolling while working out what parameters cut needs. "head -n 2" tells head to only show the first 2 lines.  `cut -d" " -f2-` tells cut to use the delimiter of a single " " space and to cut from the 2nd space "-" "-f2-" to the end of the line.


head -n 2 /var/log/apache2/error.log 
[Mon May 14 09:36:07 2012] [error] python_init: Python version mismatch, expected '2.7.2+', found '2.7.3'.
[Mon May 14 09:36:07 2012] [error] python_init: Python executable found '/usr/bin/python'.


head -n 2 /var/log/apache2/error.log | cut -d" " -f2-
May 14 09:36:07 2012] [error] python_init: Python version mismatch, expected '2.7.2+', found '2.7.3'.
May 14 09:36:07 2012] [error] python_init: Python executable found '/usr/bin/python'.


head -n 2 /var/log/apache2/error.log | cut -d" " -f5-
2012] [error] python_init: Python version mismatch, expected '2.7.2+', found '2.7.3'.
2012] [error] python_init: Python executable found '/usr/bin/python'.


head -n 2 /var/log/apache2/error.log | cut -d" " -f6-
[error] python_init: Python version mismatch, expected '2.7.2+', found '2.7.3'.
[error] python_init: Python executable found '/usr/bin/python'.

Now there is none of the date string left in the file. I can sort it and try uniq again. The only problem is that even though I have the counts of each error, it is not showing with the most common first or last. I could also remove those lines that only occur once using the "-d" duplicates only option on uniq.



cut -d" " -f6- /var/log/apache2/error.log | sort | uniq -c | wc
     14     130    1273


cut -d" " -f6- /var/log/apache2/error.log | sort | uniq -c
      6 [error] [client 127.0.0.1] File does not exist: /usr/share/doc/apache2-doc/manual/en/index.html21
      5 [error] [client 127.0.0.1] File does not exist: /var/www/html/conf/12
     29 [error] [client 127.0.0.1] File does not exist: /var/www/html/favicon.ico
      6 [error] [client 127.0.0.1] File does not exist: /var/www/html/isos.html12
      7 [error] [client 127.0.0.1] File does not exist: /var/www/html/tmp/12
      4 [error] python_init: Python executable found '/usr/bin/python'.
      4 [error] python_init: Python path being used '/usr/lib/python2.7/:/usr/lib/python2.7/plat-linux2:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload'.
      4 [error] python_init: Python version mismatch, expected '2.7.2+', found '2.7.3'.
      4 [notice] Apache/2.2.22 (Ubuntu) PHP/5.3.10-1ubuntu3.1 with Suhosin-Patch mod_python/3.3.1 Python/2.7.3 configured -- resuming normal operations
      2 [notice] caught SIGTERM, shutting down
      1 [notice] Graceful restart requested, doing restart
      4 [notice] mod_python: Creating 8 session mutexes based on 150 max processes and 0 max threads.
      4 [notice] mod_python: using mutex_directory /tmp 



cut -d" " -f6- /var/log/apache2/error.log | sort | uniq -dc | sort -n
      2 [notice] caught SIGTERM, shutting down
      4 [error] python_init: Python executable found '/usr/bin/python'.
      4 [error] python_init: Python path being used '/usr/lib/python2.7/:/usr/lib/python2.7/plat-linux2:/usr/lib/python2.7/lib-tk:/usr/lib/python2.7/lib-old:/usr/lib/python2.7/lib-dynload'.
      4 [error] python_init: Python version mismatch, expected '2.7.2+', found '2.7.3'.
      4 [notice] Apache/2.2.22 (Ubuntu) PHP/5.3.10-1ubuntu3.1 with Suhosin-Patch mod_python/3.3.1 Python/2.7.3 configured -- resuming normal operations
      4 [notice] mod_python: Creating 8 session mutexes based on 150 max processes and 0 max threads.
      4 [notice] mod_python: using mutex_directory /tmp 
      5 [error] [client 127.0.0.1] File does not exist: /var/www/html/conf/12
      6 [error] [client 127.0.0.1] File does not exist: /usr/share/doc/apache2-doc/manual/en/index.html21
      6 [error] [client 127.0.0.1] File does not exist: /var/www/html/isos.html12
      7 [error] [client 127.0.0.1] File does not exist: /var/www/html/tmp/12
     29 [error] [client 127.0.0.1] File does not exist: /var/www/html/favicon.ico

That is now very useful. I can still take it further buy removing the information I don't want more selectively using sed. Sed is a very advanced tool, it will take a while to get the hang of it, as learning regular expressions is learning a new language. There are lots of really good tutorials on sed that can be found through google. 

In my case I want to remove every thing inside brackets, as that information is not really useful to me. The first example only removes the first set of brackets, the second removes all brackets. The difference is the "g" at the end of the sed regex. Which tell sed to go global and use the whole line. 

sed -e "s/\[[^][]*\]//" /var/log/apache2/error.log | sort | uniq -dc | sort -n | tail -n 3

      6  [error] [client 127.0.0.1] File does not exist: /var/www/html/isos.html12
      7  [error] [client 127.0.0.1] File does not exist: /var/www/html/tmp/12
     29  [error] [client 127.0.0.1] File does not exist: /var/www/html/favicon.ico

sed -e "s/\[[^][]*\]//g" /var/log/apache2/error.log | sort | uniq -dc | sort -n | tail -n 3

      6    File does not exist: /var/www/html/isos.html12

      7    File does not exist: /var/www/html/tmp/12

     29    File does not exist: /var/www/html/favicon.ico

One thing worth noting here is that some messages in my error log are error level notice and other are error level error. Reading the Apache documentation there are a couple more like waning, and critical.
If I want to only see stats for messages with are of the level error I need to reduce the input I am sending to the whole long pipe. I will use grep  to only process those lines in the file that are containg some special text, like "[notice]" or "[error]". The problem is that with regular expressions "[]" are special characters, so I need to escape them using the "\"


grep "\[notice\]" /var/log/apache2/error.log | sed -e "s/\[[^][]*\]//g" - | sort | uniq -dc | sort -n | tail -n 3
      4   Apache/2.2.22 (Ubuntu) PHP/5.3.10-1ubuntu3.1 with Suhosin-Patch mod_python/3.3.1 Python/2.7.3 configured -- resuming normal operations
      4   mod_python: Creating 8 session mutexes based on 150 max processes and 0 max threads.
      4   mod_python: using mutex_directory /tmp 

grep "\[error\]" /var/log/apache2/error.log | sed -e "s/\[[^][]*\]//g" - | sort | uniq -dc | sort -n | tail -n 3
      6    File does not exist: /var/www/html/isos.html12
      7    File does not exist: /var/www/html/tmp/12
     29    File does not exist: /var/www/html/favicon.ico

The only thing to be a ware of the more complex the pipe become the cpu intensive the work to be done. As your text files get larger into the Giga byte range you may want to be selective in what you do and when. 

Part 3 working with scripts and over a network. 

Part 3 will deal with using these commands in scripts and over ssh to get some information on remote servers.

creating a custom session for gnome to only run awn

This how to is reproduced in full cause it is so use full, follow the links to go to the original. 

Run AWN / Dock in Natty Narwhal, instead of Gnome Panel or Unity Launcher/Panel

If you want to use Avant Window Navigator (AWN) or another dock instead of the default panels/launcher in Natty Narwhal, there are multiple approaches to achieve that.

The best way is to add a custom session option, because it would leave the default session options in their original state, so that you could easily use them as well, and it wouldn't get reset by an upgrade of the respective packages.

Installation

If you don't have AWN installed yet, run these commands in the Terminal, this will add its own PPA to your sources to ensure you always have the most recent version of it:

sudo add-apt-repository ppa:awn-testing/ppa
sudo apt-get update
sudo apt-get install avant-window-navigator-trunk

If you already installed AWN through the official repos, remove it before:

sudo apt-get purge avant-window-navigator
sudo apt-get autoremove

Once it is installed, you will find it under "Applications > Accessories" in classic Gnome, or by typing its name into the Unity Dash.

Sessions

Notice that the 'required components' that were specified in Gconf in previous releases of Ubuntu are now specified by the session settings in "/usr/share/gnome-session/sessions".

Modify the red marked entries to use another dock than AWN.

Custom Session

1. Create the .desktop file:

gksudo gedit /usr/share/xsessions/dock.desktop

[Desktop Entry]
Name=AWN
Comment=This session logs you into GNOME with the dock of your choice
Exec=gnome-session --session=dock
TryExec=gnome-session
Icon=
Type=Application
X-Ubuntu-Gettext-Domain=gnome-session-2.0

Note: What you put as the value for "Name=" here, will be shown as the name of the new session option.

2. Create the session settings:

gksudo gedit /usr/share/gnome-session/sessions/dock.session

[GNOME Session]
Name=Dock
Required=windowmanager;panel;filemanager;
Required-windowmanager=gnome-wm
Required-panel=avant-window-navigator
Required-filemanager=nautilus
DefaultApps=gnome-settings-daemon;

Classic Gnome Session, "Ubuntu Classic"

Modify "/usr/share/gnome-session/sessions/classic-gnome.session":

gksudo gedit /usr/share/gnome-session/sessions/classic-gnome.session

[GNOME Session]
Name=Classic GNOME
Required=windowmanager;panel;filemanager;
Required-windowmanager=gnome-wm
Required-panel=avant-window-navigator
Required-filemanager=nautilus
DefaultApps=gnome-settings-daemon;
IsRunnableHelper=/usr/lib/nux/unity_support_test --compiz
FallbackSessionsID=GNOME2d
GNOME2d=2d-gnome

Unity Session, "Ubuntu"

1. Disable the "Ubuntu Unity Plugin" in "CompizConfig Settings Manager".

2a. Add "avant-window-navigator" to "Startup Applications"

2b. or modify "/usr/share/gnome-session/sessions/ubuntu.session":

gksudo gedit /usr/share/gnome-session/sessions/ubuntu.session

[GNOME Session]
Name=Ubuntu
Required=windowmanager;panel;filemanager;
Required-windowmanager=compiz
Required-panel=avant-window-navigator
Required-filemanager=nautilus
DefaultApps=gnome-settings-daemon;
IsRunnableHelper=/usr/lib/nux/unity_support_test
FallbackSessionsID=FallbackUnity2d;FallbackClassicGnome
FallbackUnity2d=2d-ubuntu
FallbackClassicGnome=classic-gnome
FallbackClassicGnomeMessage=It seems that you do not have the hardware required to run Unity. Please choose Ubuntu Classic at the login screen and you will be using the traditional environment.

Tips

You should place launchers on your desktop in case you need to restart the one, AWN or another dock:

avant-window-navigator

or you want to run the other, Gnome Panel:

gnome-panel

If your dock is also capable to host the Notification Area, like AWN does, then remove those from your Gnome Panel. Otherwise, the respective AWN applet gets killed each time you start Gnome Panel.

I myself find it very handy to have the ability to fire up Gnome Panel occasionally. Mostly for UF support reasons, otherwise not really.

Run Dialog (Alt+F2)

Since the usual Run Dialog's are either provided by Gnome Panel or the Unity Dash, Alt+F2 will stop working when you run only AWN.

AWN's Cairo Menu applet includes a "Launch..." menu item, but those needs to be 'filled' with a replacement for the Run Dialog.

XFRun4

The best replacement in terms of style and integration compared to Gnome Panel's Run Dialog is XFRun4. It's included by the package "xfce4-utils", which is provided by the official repos. To avoid the installation of unnecessary additional packages, if you don't have XFCE installed parallelly, use the "--no-install-recommends" option of 'apt-get':

sudo apt-get install --no-install-recommends xfce4-utils

After the installation, you can either run it through the mentioned "Launch..." menu item in Cairo Menu, or create a keyboard shortcut in CCSM for the command "xfrun4". If you choose the set the key combination to Alt+F2, you will get a conflict dialog, then choose "Disable Run Dialog" (that option is included by the "Gnome Compatibility" plugin).

Synapse

Another great alternative to Gnome Panel's Run Dialog and Unity's Dash is Synapse. It can not only run commands, but also installed applications, which you would otherwise have to run through the menu of classic Gnome, Cairo Menu of AWN, or the Dash of Unity. And similar to the latter makes it use of the Zeitgeist engine to also give you quick access to any kind of recently accessed stuff.

See this guide about further information and how to install it:

http://www.tuxgarage.com/2011/05/synapse-alternative-to-run-dialog-dash.html

Troubleshooting

If Gnome Panel is still getting run at login, alongside to your dock or instead of it:

- Remove the content of "~/.config/gnome-session/saved-session":

rm -f ~/.config/gnome-session/saved-session/*

- Remove the now obsolete Gconf keys:

gconftool-2 --unset /desktop/gnome/session/required_components_list
gconftool-2 --recursive-unset /desktop/gnome/session/required_components

- Check if Gnome Panel is listed/enabled in "Startup Applications", and if so, disable it.

If your dock doesn't get run at login:

- Remove "panel" from the line "Required=" in the concerning session settings:

Required=windowmanager;filemanager;

- Add the command for your dock to "Startup Applications":

avant-window-navigator

List of all the drivers on your Linux installation.

cd /lib/modules/2.6.18-128.1.OpenVA.2.0.1039/kernel/drivers/net

ls -1 *.ko | xargs -I{} modinfo {} | grep -e "description"

description:    3Com 3c59x/3c9xx ethernet driver 

description:    RealTek RTL-8139C+ series 10/100 PCI Ethernet driver

description:    RealTek RTL-8139 Fast Ethernet driver

description:    AceNIC/3C985/GA620 Gigabit Ethernet driver

description:    AMD8111 based 10/100 Ethernet Controller. Driver Version 3.0.3

description:    Broadcom 4400 10/100 PCI ethernet driver

.....

ls -1 *.ko | xargs -I{} modinfo {} | grep -e "description\|alias"

description:    3Com 3c59x/3c9xx ethernet driver 

alias:          pci:v000010B7d00009210sv*sd*bc*sc*i*

alias:          pci:v000010B7d00009056sv*sd*bc*sc*i*

alias:          pci:v000010B7d00001202sv*sd*bc*sc*i*

alias:          pci:v000010B7d00001201sv*sd*bc*sc*i*

alias:          pci:v000010B7d00009201sv*sd*bc*sc*i*

alias:          pci:v000010B7d00004500sv*sd*bc*sc*i*

alias:          pci:v000010B7d00006564sv*sd*bc*sc*i*

alias:          pci:v000010B7d00006562sv*sd*bc*sc*i*

alias:          pci:v000010B7d00006560sv*sd*bc*sc*i*

alias:          pci:v000010B7d00005257sv*sd*bc*sc*i*

alias:          pci:v000010B7d00005157sv*sd*bc*sc*i*

alias:          pci:v000010B7d00005057sv*sd*bc*sc*i*

alias:          pci:v000010B7d00005B57sv*sd*bc*sc*i*

....

This is an other way to do it for all your kernel modules 

modprobe -l | sed -e "s:\(/.*/\)::" - | sed -e "s/\.ko//" - | xargs -i{} modinfo {} | grep -e "description"

description:    3Com 3C359 Velocity XL Token Ring Adapter Driver 

description:    3Com 3c574 series PCMCIA ethernet driver

description:    3Com 3c589 series PCMCIA ethernet driver

description:    3Com 3c59x/3c9xx ethernet driver 

description:    3ware 9000 Storage Controller Linux Driver

description:    3ware Storage Controller Linux Driver

description:    RealTek RTL-8139C+ series 10/100 PCI Ethernet driver

description:    RealTek RTL-8139 Fast Ethernet driver

description:    Dell PERC2, 2/Si, 3/Si, 3/Di, Adaptec Advanced Raid Products, HP NetRAID-4M, IBM ServeRAID & ICP SCSI driver

description:    Abit uGuru Sensor device

 .....

What are you going to do with all that.  I don't know. 

-- 

Aaron Nel 

https://plus.google.com/u/0/117484568487416046905/posts

http://www.facebook.com/nelaaro

disabling FSPPS/2 Sentelic FingerSensingPad track pad.

Linux help.

I created a script to disable the FSPPS/2 Sentelic FingerSensingPad track pad on my gigabyte Q1585n note book. The Sentelic FigerSensingPad is two sensitive, and I have a mouse. When I type I want to disable the track pad. The note book does not have a hardware button combination to disable the track pad.

So I wrote this script to switch states. Every time you run it it changes it from enabled to disabled. Run it again and it switches from disabled to enabled. I tested it on my Ubuntu 11.04 and my friends fedora 14.  It should work on any Linux machine with xinput installed.

here is a link to a zipped file I uploaded to google docs.
tracpad.sh.tgz


to extract the archive

tar -xf ./tracpad.sh.tgz

to make it executable

chmod a+x ./tracpad.sh

to run it
./tracpad.sh


Here is the contents of the script. I use xinput. If you just want to use xinput  to disable the touch pad or any thing else handled buy xinput read its man page.


#!/bin/bash
#
# Author: Aaron Nel
# Created: 23 12 2011

# This script disables and enables the track pad
# every time it is run it switches the state of the track pad
# Specfically the "FSPPS/2 Sentelic FingerSensingPad"

#test to see what state the tracpad is in
padE=`xinput --list-props "FSPPS/2 Sentelic FingerSensingPad" | grep "Device Enabled" | cut -f3`
#echo $padE

case $padE in
    0)
        xinput set-prop "FSPPS/2 Sentelic FingerSensingPad" "Device Enabled" 1
        echo "device enabled"
        ;;
    1)
        xinput set-prop "FSPPS/2 Sentelic FingerSensingPad" "Device Enabled" 0
        echo "device disabled"
        ;;
    *)
        echo "Error:"
        echo "Device in an unknowen state"
        echo "$padE"
        ;;

esac

If you are looking for more info or other solutions  check out these links.

http://www.google.co.za/search?gcx=c&sourceid=chrome&client=ubuntu&channel=cs&ie=UTF-8&q=FSPPS%2F2+Sentelic+FingerSensingPad

How to configure the touch pad, to make it less sensitive and other nice tweaks.
http://ubuntuforums.org/showthread.php?t=1502560

The souceforge driver project page.
http://sourceforge.net/projects/fsp-lnxdrv/

TuxGarage: Tutorials, News, Troubleshooting: Run AWN / Dock in Natty Narwhal, instead of Gnome ...

This how to is probably the best most versatile and useful way to get AWN running in Ubuntu Linux. It adds a session option at the log in screen. So you don't touch the default setup. It is also a look into a very powerful world. Creating your own custom sessions.

TuxGarage: Tutorials, News, Troubleshooting: Run AWN / Dock in Natty Narwhal, instead of Gnome ...: If you want to use Avant Window Navigator (AWN) or another dock instead of the default panels/launcher in Natty Narwhal, there are multiple ...

Reading the documentation in /usr/share/doc especially the .gz compressed files

Have you ever wanted to read the the documentation in the /usr/share/doc directory. If you install apache it makes it nice and easy to read all the documentation that comes with most Linux packages.
First install Apache2

$ sudo apt-get install apache2 apache2-doc

apache2-doc is the special case here. It allows you to browse your documentation /usr/share/doc/ through your web browser. from

http://localhost/doc/

This does not work out right though. you need to change the configuration of apache to get it to decompress and show the *.gz files as plain text.
I posted on Stackoverflow to get a way to use the apache to show the content of *.gz documents in the /usr/share/doc/ directory. This is what was posted as possible solution
http://stackoverflow.com/questions/5242106/apache-gz-gzip-content-handler-for-linux-documentation-usr-share-doc-and-local

Here are those instructions in brief. It tells apache how to deal with .gz files to process them as plain text and send them to the browser as plain text.

 sudo a2enmod headers
 sudo a2enmod deflate
 gksu gedit /etc/apache2/sites-enabled/000-default

Go to the bottom of the file and find the section with Alias /doc/ "user/share/doc/" and change it to look like this.

Alias /doc/ "/usr/share/doc/"
<Directory "/usr/local/doc">
    Options Indexes MultiViews FollowSymLinks
    AllowOverride None
    Order deny,allow
    Deny from all
    Allow from 127.0.0.0/255.0.0.0 ::1/128

    AddEncoding gzip gz
    <FilesMatch "\.gz$">
      ForceType text/plain
      Header set Content-Encoding: gzip
    </FilesMatch>
</Directory>   

 sudo apache2ctl restart

Now you are done and can read your *nix documentation that comes with most packages.

stackexchange

I use the internet, and Google mostly to answer questions I have. Some times when I get an answer I have been looking for I post about it on my blog / social media. I recently found my favorite sight for interacting with like minded individuals. You would have come across stack exchange on some of your Google searches as well.

The community is not made up of normal's, but exceptional people. Who are very knowledgeable in the areas they contribute. The technical areas of programming, Linux, security and and language.  As a introvert I find my self clinking from question to the next to read all the different and interesting responses.
I spent some time answering questions as well.

how to copy an ubuntu install from one laptop to another http://askubuntu.com/questions/62340/how-to-copy-an-ubuntu-install-from-one-laptop-to-another/62383#62383

how are typical users expected to read the documentation in usr share doc http://askubuntu.com/questions/24072/how-are-typical-users-expected-to-read-the-documentation-in-usr-share-doc/29768#29768 with my answer http://askubuntu.com/a/29768/10998.

why do computers work in binary http://stackoverflow.com/questions/5165013/why-do-computers-work-in-binary/5167023#5167023

I highly recommend, using your Google account to create an account on stackexchange. To ask your technical questions here.

Trying to get Firefox to accept my proxy setting in ubuntu 10.04. I don't know what else I could do.

Z formal systems specification tool and CZT

I recently as part of a systems engineering course had to write a few system specification documents using the Z language. I now had to find a nice way to do this. The Community Z Tools at source forge provided the best and simplest way for me to get things done. They have integrated their tools in to jedit so that it would be very simple for them to get things working.

The instructions at source forge are not exactly straight forward. It seamed to me that I need to download the source and compile from source this is not the case. Just download the czt binary file form source forge and extract. The follow the instructions at http://czt.sourceforge.net/jedit/index.html ignoring the step to download and install the source. Every thing is include in the czt download.

The jedit tool also seams to be a quite useful tool worth a look.

Jawug, wug, and wireless equipment

I recently decide to get involved with the South African Wireless Users Group. So I had to decide on what equipment to uses where and how I was going to connect. It was a really good learning experience. I now know a bit more about the wireless networks, the equipment and types of connections possible.

For equipment I found that Uniterm Direct had the best selection and devices. I will probably end up with an RB433, or RB411 device. The Mikrotik devices have a better routing software and are designed for extream conditions like out doors, or in relativly high heat. These mikrotik devices also allow for power of ethernet, so you can put in them in some really strange places, not have to worry to much about how to power them.

Most standarde wireless routres could do the same thing and a lot of people would use the linksys WRT54g series routers. Those are designed for indoor use and don't do well in your roof or out side in an antenna enclosure.

The only thing I was not sure about was signal quality, strength and my ability to connect to the high sites a round my house. The following articles helped me understand what was going on.
http://wireless.gumph.org/articles/longrangelink.html
and http://www.cisco.com/en/US/tech/tk722/tk809/technologies_tech_note09186a00801c12c2.shtml
and http://www.universalweb.com/CustomCablesUSA/Technical.html. This last one is particularly good at explaining the differences between the different low loss co-axial cables when using antenna.

Ubuntu Network manager, static interface setup, multiple profiles, no internet, no dns

Recently I decided to experiment with the multiple profiles feature of the network manager provide by Ubuntu 8.04. I set up my lan network card with a static ip address so that I could manager port forwarding on my router specifically for my laptop.

I still wanted to be able to switch to the roaming mode provided by the network manager if I went to university or some where else. So I decide to save the set up in the network manager as a separate profile. I then saved other profile for the roaming modes on my wireless lan, and my cable lan interfaces.

After a restart I found that I could no longer get out on the internet after start up. I was not sure what the problem was. I could ping my router but nothing beyond my router. In checking my network manager set up I could see that where I used to have my router as a dns, I now had nothing.

So I add some dns servers saved the settings under my static profile and applied these setting buy clicking the green tick. Now every thing was working again. I could connect to the internet web pages would now load in firefox.

Again after a restart I found I had the same problem so I check my network manger and found that the settings were still there but were not applied. So I again choose the correct profile and applied the settings. This behaviour of being not being set up after a restart continued. It gets a little irritating having to go into network manager and apply setting after ever restart.

After reading around in fourums I found about about resov.conf resolvconf /etc/resolv.conf, /etc/resolvconf/interface-order and /etc/resolvconf/resolv.conf.d/. I could see that in interface-order and ifconfig my cabled lan card was begin configured correctly at start. In the resolv.conf file no dns servers were configured at start up, having no dns server to resolve requests out to the internet I could not browse the web or send emails or any thing you would need to do.

To solve the problem of not being set up at start up open the /etc/resolveconf/resolv.conf.d/base as root in a text editor and add a dns server like your router or some other dns server. Read the man resolvconf to find out why. Redundancy here is a good thing, have it point to more then one dns server.

Simply place entries like "nameserver 127.0.0.1" or "nameserver #your router#". Obviously 127.0.0.1 is localhost and probably wont work for the average set up. If your not sure what dns servers to use, ping a top level domain like www.com or www.co.za and get their ip address. Do a traceroute and look for domain names with .dns in them. You can see and example of how network manager saves one of these file in /etc/resolvconf/resolv.conf.d/original.

linux the cisco vpn and monash south africa

I have spent quite a bit of time trying different options for the vpn connection to Monash South Africa. The open source vpnc dose not work because when importing the cisco profile it warns that IP tunnelling across TCP has been enabled. Which may cause undesired resualts, in this case it simply dose not work.

The only option left for linux is the cisco client which has not been updated since 2006. Major problem. Thus we now have to go to projects.tuxx-home.at. Here is the only way to get support for the cisco linux vpn client. There are instructions for intallation, patching and support forums. Every thing a person would need. I have used them on a number of different kernel builds, 2.6.18 -- 2.6.24. They all worked.

The vpn client on the web site dose not, but you still need to down load it to get the correct profile configuration file. It is called sa.pcf. You will need to copy that file to the cisco /etc/opt/cisco-vpnclient/Profiles/. Then all you need to do to start it is type "sudo vpnclient connect sa".
Enter your user name and password and every thing should work.

I have found some small issues with the gnome network manager and haveing both my lan network card and wireless network card enabled. So depending on how I am connecting to the internet I disable which ever one I am not using. Type "ifconfig" to find out which device to disable. Then type "ifcofig eth#" where # is the device number, some thing like 0 or 1.

Here is some more info on getting the cisco vpn working www.blog.arun-prabha.com/2008/05/01/cisco-vpn-installation-issue-with-ubuntu-804-hardy-heron/.

Problems in Ubuntu 8.04 LTS

I recently upgraded to Ubuntu 8.04 LTS. I found that after using it for about 20 minuites their were certain things I could do that would totally crash the whole system. Later I found that it was a bug in the released kernel, check https://bugs.launchpad.net/ubuntu/+source/linux/+bug/188226.

It has to do with the cpu schedualer (Completely Fair Schedualer), not being configured properly for the desktop system. Their is already a way to fix it, but in requires over 100 mb of downloads to fix the problem. This will be a serrious problem to people who pay exhorbitant rates for internet access.

The basic solution is to update your system to an the latest development release kernel. From 2.6.24-16-generic t0 2.6.24-17-generic. Caution this 2.6.24-17-generic kernel
is a developmental release used for testing changes and bug fixes, it is not the same as the release version kernel. This new kernel will only be released 2008-07-01 it the next version of hardy, ubuntu-8.04.1.

Any way I upgraded to ensure that I have a Linux desktop that dose not stall any time I open two many windows or try to do thing as a different users, ie every time i use the sudo command to do some thing cpu intensive.

Fix
Using synaptic
Open Synaptic > Settings > Repositories
Go to the updates tab and select Pre-released updates ( hardy proposed)
Close the window and reload the package information.
Either navigate through the Base System Section or search to find
linux-image-2.6.24-17-generic
linux-ubuntu-modules-2.6.24-17-generic
Select both packages.
If you are using an nvidia graphics card you will need to also up grade it. Serach for
nvidia-glx-new - we want the version that matches our kernel, 169.12+2.6.24.12-17.36 instead of the old one 169.12+2.6.24.12-16.36 notice that the 12-16 / 12- 17 referes to the version of kernel supported. This should automatically include the
linux-restricted-modules-2.6.24-17-generic. Which include the binary / compiled modules necessary for the nvidia driver to work.
This should be all the updates that you need to fix the problem with out a major difficulty.
Next click apply and wait.
When it finished be sure to disable the Pre-released updates ( hardy proposed)
Synaptic > Settings > Repositories
Go to the updates Tab and select Pre-released updates ( hardy proposed)
And reload the package lists.

That should fix the problem, all this is described in the bug report check https://bugs.launchpad.net/ubuntu/+source/linux/+bug/188226. It also includes instructions on how to compile the kernel form source with the correct options. For more advanced users. This would probable save people form having to download a whole new kernel, and graphics card drivers.

mips and spim materials to help you get started.

Here is a link to a full text book on the mips processor.
http://www.comms.scitech.susx.ac.uk/fft/programming/R4400_Uman_book_Ed2.pdf
You will not need to read the whole text book, but it will come in very hand if you are struggling with MIPS. Choose what you read out of this text book.

I found it at the following site http://2020ok.com/. Here you may find some other useful books on a range of other computer related topics.

Also check out http://en.wikipedia.org/wiki/MIPS_architecture to get a better understanding of MIPS.

The official web site for spim is one of the best places to go and has a lot of extra resources, make user you visit this web site. http://pages.cs.wisc.edu/~larus/spim.html.At the bottom of the page is a section on additional materials.

Lastly here is a link to a document I got of the net and cant remember where. I found it gave a good overview of all the very basics and is an excellent reference when writing simple MIPS programs. http://www.scribd.com/doc/2576056/SPIM

ffmpeg

I spent close to 2 days trying to get around a problem with ffmpeg
Input #0, flv, from 'tmp.flv':
Duration: 01:43:15.8, start: 0.000000, bitrate: 64 kb/s
Stream #0.0: Video: flv, yuv420p, 320x240, 29.97 tb(r)
Stream #0.1: Audio: mp3, 22050 Hz, mono, 64 kb/s
Output #0, mp4, to 'tmp.mp4':
Stream #0.0: Video: mpeg4, yuv420p, 320x240, q=2-31, 200 kb/s, 29.97 tb(c)
Stream #0.1: Audio: 0x0000, 22050 Hz, mono, 64 kb/s
Stream mapping:
Stream #0.0 -> #0.0
Stream #0.1 -> #0.1
Unsupported codec for output stream #0.1

Which is basically saying that it dose not know how to take the audio input codec mp3 and produce and audio out put codec.

I tried to used the -acodec option to force output to mp3. Which as far as I understand I don't need to do as mp4 has both an mp4 video codec and audio codec.

The only site I found useful information on all in one place was http://howto-pages.org/ffmpeg/.
It describes how to compile from source and enable the different lib's required. I still found this site much more helpfull then the mailing lists that keep popping up in google searches.

This can be used as a work around for the mp3 encoding problem which I found at the ubuntu forums
ffmpeg -i inputfile.flv -acodec copy -vcodec mpeg4 outputfile.mp4

Whats important here is the -acodec copy, I dont know what the quality of the output will be like but at least it dose not bomb out in an error.

I found the quality to be good, not great just good. I lost some video information. The sound was great, no distortion or delay. The file size shrank close to 20%. I think that -sameq option could improve things but my file sise more then doubles. Which gets me great out put. I am going to experiment to try and get a better result.

Another day, another cable

Another day, another cable

This article provides some interesting incites to why we pay so much for so little in South Africa when it come to the internet.

vlc segmentation fault

I recently had problems with vlc media player. It would crash unexpectedly when running. It would not start when run from the Applications > Sound & Video > VLC. Occasionally it would start up but it would look terrible, or start two player interfaces. I then turned to the console using vlc, or wxvlc to get it to run. Some times it would come out with two player interface's, and whole string of exceptions.

Solution

in your home directory /home/yourhome/ their is a config directory for vlc called .vlc /home/yourhome/.vlc go and delete it completely. VLC automatically creates it when it is frist started. This will reset all the settings that could cause this type of problem.
rm /home/[you home dir]/.vlc

Immediately after doing that I tryied vlc from the application menu and it worked. I expect that I am not the only one to have this problem. I hope it helps some one else out there.

Bauer-Power: Information is Power!

Bauer-Power: Information is Power!

A really good video on how data recovery works. It was associated with 2 links to sites that have more info. http://myharddrivedied.com/presentations.html and web.forensicspeak.com. If any one is interested in finding out more about recovering data on a hard drive I recommend these sites. If you are interested in taking apart a hard drive go to the video above.

I also recomend going to http://www.cgsecurity.org/ it explains to linux tools for data recovery. I have used the one and it worked for me. I was able to recover from bad partion information. It has a number of links as well to site that could prove to be useful.

Coding Horror: Gigabyte: Decimal vs. Binary

Coding Horror: Gigabyte: Decimal vs. Binary

This is an in depth look at why a 250 gb hard drive is a lot less then the advertised size.

Apache2 Documentation apache2-doc manuall problems

For those of us who want to find out how to fix the problem of apache2-doc at http://localhost/manual/ giving us junk like this

URI: index.html.de Content-Language: de Content-type: text/html; charset=ISO-8859-1 URI: index.html.en Content-Language: en Content-type: text/html; charset=ISO-8859-1 URI: index.html.es Content-Language: es Content-type: text/html; charset=ISO-8859-1 URI: index.html.fr Content-Language: fr Content-type: text/html; charset=ISO-8859-1 URI: index.html.ja.euc-jp Content-Language: ja Content-type: text/html; charset=EUC-JP URI: index.html.ko.euc-kr Content-Language: ko Content-type: text/html; charset=EUC-KR URI: index.html.pt-br Content-Language: pt-br Content-type: text/html; charset=ISO-8859-1.

Which is a type map, which is meant to allow for multiple languages in the documentation. It dose not work so to fix it try the following at this link.Apache2 Documentation doesn't work - Ubuntu Forums.

I changed these instructions a bit by
sudo mkdir typemaps
and
sudo mv *.html to ./typemaps/
which moved the all the *.html type map pages to the typemaps dir. so if I need to restore them I can. you could also just delete them with
sudo rm -f *.html
but then you have no way of retrieving them. This post dose not contain any documentation on modifying the /etc/apache2/apache2.conf file.

You may have to repeat the above instructions in some of the /usr/share/doc/apache2-doc/manual/ sub directories, like: developer, faq, howto, misc, mod, rewrites, programs, platform, ssl, and vhosts. As the documentation links to them as well.