Setup your own deb repository

It is difficult to setup a Debian repository that is signed and can handle multiple versions of any single package. I went through many different packages and tutorials in how to setup my own Debian repository and it was a pain to find a method that works. My two requirements might be specific to my needs but they give the user the ability to do these two things.

The signed part allows automated scripts to upgrade a package as it allows the following.

sudo apt-get -y install <package>

The ability to handle multiple versions of the same package would allow the user to install which ever version of the package with the following command example

sudo apt-get -y install <package>=<version>

Generate a gpg key

We need a gpg key to sign our packages and repository. So let’s generate one before setting up freight.

gpg –gen-key

Select option 4 RSA (sign only)

Fill out all the information that it asks.

Note: Be sure to add an email. It will be needed to configure freight.

One thing that will happen most likely is that there is not enough entropy to generate the key and it waits for entropy. I found a good way to generate some entropy is to run stress. Open another terminal to install and run stress while leaving the gen key running.

sudo apt-get install stress

stress –hdd 8 –io 8

If you want to watch the available entropy get generated, open another terminal and watch with this command

watch cat /proc/sys/kernel/random/entropy_avail

Installing Freight

The package that I found to handle both my requirements was freight. The instructions are there, but I will document what I did just to be complete.

I installed via apt-get and so I needed to add the third party source list before installing

echo “deb http://packages.rcrowley.org $(lsb_release -sc) main” | sudo tee /etc/apt/sources.list.d/rcrowley.list

sudo wget -O /etc/apt/trusted.gpg.d/rcrowley.gpg http://packages.rcrowley.org/keyring.gpg

sudo apt-get update

sudo apt-get -y install freight

Configuring Freight

Copy the example conf

sudo cp /etc/freight.conf.example /etc/freight.conf

Edit the freight.conf and add

GPG=”emailfromgpgkey@mydomain.com”

Add deb packages

Take a deb file you already created and added to the repository. The apt/squeeze, etc arguments are the different distros where the deb should be published.

freight add foobar_1.2.3-1_all.deb apt/squeeze apt/lucid apt/natty

Build the cache

freight cache

Setting up Nginx

Now we need to serve out the repository over http. You can use any web server to do this. I chose Nginx and here is the setup procedure.

sudo apt-get install nginx

Setup hosting file

cd /etc/nginx/sites-available

sudo vi mydomain.com

Put something like this into the site configuration file

server {
listen 80;
server_name mydomain.com;
access_log /var/log/nginx/mydoamin.com.access.log;
error_log /var/log/nginx/mydomain.com.error.log;

location / {
alias /var/cache/freight/;
}
}

Enable the new site

cd /etc/nginx/sites-enabled

sudo ln -s /etc/nginx/sites-available/mydomain.com mydomain.com

Restart Nginx

sudo service nginx restart

Consume packages

Now to install the packages from your new repository on a Debian machine, add the source list and the key

echo “deb http://mydomain.com $(lsb_release -sc) main” | sudo tee /etc/apt/sources.list.d/mydomain.list

sudo wget -O /etc/apt/trusted.gpg.d/mydomain.gpg http://mydomain.com/keyring.gpg

Now you can do the normal apt-get procedure to install a package

sudo apt-get update
sudo apt-get -y install foobar

Google Chrome Voice Search Extension Memory Leak

Screenshot from 2013-12-11 16:57:26This was observed on Ubuntu 12.04 where just opening Chrome without going to any pages would see memory start to get eaten up. The cpu usage would go way up as well. The culprit was the Google Voice Search Hotword Extension. If you go to upper right three horizontal bars -> Tools -> Extensions and disable that extension, everything is solved.

Ubuntu 12.04 LTS Beta Is Out

There are supposed to be many changes in the upcoming release of Ubuntu desktop. One notable UI change is HUD (Head-Up Display) where you can type a menu command to get to an application’s functionality without navigation through the menu system. It is useful for users who are experienced with an application and know what they want to get to without menu, sub-menu, sub-sub-menu, etc.

I will need to do a VirtualBox install to play around soon.

http://www.pcworld.com/businesscenter/article/251367/ubuntu_linux_1204_oneups_windows_and_mac_shuttleworth_says.html

Replacing OpenGeo Suites on Windows with GeoServer on Linux

After our initial installation of OpenGeo Suites for WMS service on Windows, we encountered some stability issues that could not be resolved. We could have hired some high priced consultants to debug the issue but we opted for a lighter weight infrastructure.

Our initial system specs:

  • Windows Server 2008
  • 4GB ram
  • Quad core
  • Open Geo Suites running as Windows application

Our slim and fast system specs:

  • Ubuntu 10.04
  • 512MB ram
  • Quad core
  • GeoServer running as java process

So far so good. All layers are loading super fast. We will see how the stability issue is resolved. At least we are using a lot few resources now which makes scaling out to a cluster of GeoServers much more affordable.

Scaling down from MS to open source

We just moved a service that was running on the Microsoft stack (Windows Server 2008, .NET, MSSql Server, IIS) to an open source stack (Ubuntu Server, php, Codeigniter framework, apache2, postgres).

We now run the service on a cloud server with dual cpu and 256MB ram (Yes, that’s megabytes).

The old server ran on dual core with 2GB of ram.

A four letter word can ruin your day

halt + Amazon EC2 + Instance Store = A bad day

I did a stupid thing while on vacation. Decided to clone an EC2 server and executed halt from commandline without checking to see if the server was using Instance Store. Whoops, Amazon showed the server as terminating…terminated… ahhhhh.

After having to rebuild the server while on vacation, I have learned a valuable lesson.

Fail2ban does not start after reboot

This problem was identified with the following configuration:
Ubuntu 8.04
fail2ban

Problem: After fail2ban install, everything works fine, but after reboot fail2ban does not start. Manual /etc/init.d/fail2ban restart fails also.

Cause: Fail2ban looks for fail2ban.sock in
/var/run/fail2ban/
During reboot, that directory is removed. Fail2ban assumes it is there and fails on restart.

Solution: Make sure the directory exists during start of fail2ban. Edit the init.d for fail2ban to fix this.
sudo vi /etc/init.d/fail2ban
Find the do_start option.

do_start()
{
# Return
# 0 if daemon has been started
# 1 if daemon was already running
# 2 if daemon could not be started
do_status && return 1

if [ -e "$SOCKFILE" ]; then
log_failure_msg "Socket file $SOCKFILE is present"
[ "$1" = "force-start" ] \
&& log_success_msg "Starting anyway as requested" \
|| return 2
DAEMON_ARGS="$DAEMON_ARGS -x"
fi

start-stop-daemon --start --quiet --chuid root --exec $DAEMON -- \
$DAEMON_ARGS start > /dev/null\
|| return 2

return 0
}

Add the following after the if statement.

# Assure that /var/run/fail2ban exists
[ -d /var/run/fail2ban ] || mkdir -p /var/run/fail2ban


Finally, it should look like this

do_start()
{
# Return
# 0 if daemon has been started
# 1 if daemon was already running
# 2 if daemon could not be started
do_status && return 1

if [ -e "$SOCKFILE" ]; then
log_failure_msg "Socket file $SOCKFILE is present"
[ "$1" = "force-start" ] \
&& log_success_msg "Starting anyway as requested" \
|| return 2
DAEMON_ARGS="$DAEMON_ARGS -x"
fi

# Assure that /var/run/fail2ban exists
[ -d /var/run/fail2ban ] || mkdir -p /var/run/fail2ban

start-stop-daemon --start --quiet --chuid root --exec $DAEMON -- \
$DAEMON_ARGS start > /dev/null\
|| return 2

return 0
}

Now restart and it should work.

sudo /etc/init.d/fail2ban restart

Just for kicks, see how /var/run has the newly created fail2ban directory.

sudo ls /var/run