The solution was to setup the bash script to echo the data it was collecting (for MRTG purposes), to go out as a UDP broadcast packet to the network.

It was annoying to setup, because current versions of netcat differ to those I was using originally, and eventually found the need for a timeout wait on the netcat command.

It works great tho, and spawned the next idea of an Applet to monitor the Aurora output, the CC128 consumption, and anything else I want to spit out on any part of our local network.

It’d work great as a file copy mechanism too – base64 encode each file, and send that way. Each machine on a network typically receives broadcast packets, and so it’s no issue to spread the same data to all machines at the same time (instead of sending it to one machine at a time).

I like the idea of it – and am amazed I didn’t come across that before. It gives me the data on as many machines as I care to have listening for it – and is relatively reliable (if I wanted guaranteed, I’d not be using wireless..).

I’m actually referring to Linux and my continual abuse of root for tasks that don’t belong to root.

Recently, at work I’ve been working on a few minor tasks, however, because the system is locked down, I’ve ran into issues – for example, recently, I hit an issue with trying to copy files from one directory to another. They are owned by seperate users, and so I couldn’t read from one to the other.

On my own setup, I’d have already been root, because that’s how I’ve done things for the last several years. This abuse of the root login has the consequence that sure, I can piece software together, and get things to work, but take away the root login, and all of a sudden the security that was always bypassed gets in the way.

I’m progressively avoiding the root login altogether, and using one of my own, it’s easier to avoid since root is no longer allowed to login, but it’s still the permissions issues that will get in the way doing their job.

I’ve previously abused cron for much of my tasks, but a few weeks back when I wrote mysql_monitor, I wrote that and it is started using an init script, instead of cron.

I’ll keep at it, and hopefully it’s not so much of a huge issue to pick up on the issues that were always avoided using root.

I’m also lazy though, so will likely resort to using root when something gets difficult for the normal user. One thing I did learn is that the execute permission, is needed for other users to list a directory, nuts to that (execute a directory.. wtf?)

The idea being that the inverter I have can output RS485 which gives us details about how much it’s putting out, the temperature, and we can then measure efficiency and make any further adjustments to it to improve efficiency.

I’m not installing anything yet, as I’m still waiting on the gross meter to be installed (been a bit of time actually, getting impatient). The company contracted to do it claimed 2 weeks last week, so hopefully in the next 7 days we are grid connected.

Back to the serial idea though, I can take RS485, use an RS232 converter and that gives us the ability to get the data to the PC. A wireless transmitter and receiver is what would be needed, and Bluetooth being the most common, used in industrial applications for monitoring / controlling various machinery.

I’ve found a few candidates in my search yesterday:

This PS110W Wi-Fi Serial Device Server – 1 Port offers much of what is needed, including RJ45 if we later abandon wireless. Not sure on price though.

I then looked into the LTC100 which seems a better device. Pricing internationally puts it at about $50 AUD, but I’ve asked RF modules if they can provide a price and then I might get it via them.

The advantages of the LTC100 are that it can be port powered, which means I can supply it 9V via the serial port. Getting the data from it should be relatively easy, I’d need a bluetooth adapter, paired to it, and then using the SPP (Serial Port Profile) to create a virtual serial port. Attach that to my Virtual Server 2005 or VirtualBox virtual machine, and then probe the serial port in the virtual machine for the data.

Then, place that data into RRD Graphs and use that for comparison of day and seasonal changes, and to detect performance issues. Think MRTG Threshold alerts – “Your Solar System is performing worse than a drained AA battery”.

The hardware isn’t going to be IP65 rated like the inverter is, so I’ll pick up a IP65 enclosure from Jaycar, use that to hide the Cat6 cabling from the inverter, the bluetooth adapter. Whilst I’m getting the enclosure, I’ll get a battery terminal so that the wires don’t get damaged from replacing the battery.

I’ll have to figure out how much power it uses, considering the continual probes – although.. it won’t be continual if we shut it down after 9pm and wake it up around 5am, so that’ll save some of the battery.

I’m half tempted to get a Linutop or similar low powered PC to replace the current server, but the offerings out there so far are from $400 to $700 for something very low powered (1Ghz). Due to the virtual machine, it’d need two to be effective – which makes a new server all of a sudden worth while (extra computing power for the same price).

I could ignore the statements that fail – but that could be potentially dangerous – leaving with no slave backup at all.

The next best option to ensure the database is up to date is simple, monitor mysql for slave errors, which is what I do now – on my home server, I have it querying the 4 services, and checking if the two “Yes” options are ever “No”.

I created the script to run in bash, I usually heavily use PHP for all sorts of tasks (yep, many years ago, I did a PHP script for simple ping monitoring, and just stuck with it).

Using bash doesn’t require as many resources (not that PHP is resource intensive).

The script is probably pretty simple, but it was a bit of a learning curve for me (with my minimal bash experience).

/usr/sbin/mysql_monitor:

#!/bin/bash
EMAIL=”admin@email.address”
HOSTS=”192.168.x.x 192.168.x.x x.x.x.x x.x.x.x”
while :
do
for host in $HOSTS;
do
result=`mysql -h $host -u root –password=x –execute=”SHOW SLAVE STATUS\G” 2>&1`
slave_result=`printf “%s\n” “$result” | grep -i running`
connect_result=`printf “%s\n” “$result” | grep -i Error`
if [[ $slave_result == *No* ]]
then
echo “$host – $result” | mail -s “SLAVE OUT OF SYNC”  “$EMAIL”
fi

if [[ $connect_result == *connect* ]]
then
echo “$host – $result” | mail -s “MySQL Connection Issue” “$EMAIL”
fi
done
sleep 1800
done

HOSTS is a variable which contains the IPs of the hosts to be monitored.  EMAIL is self explanatory.

The script sleeps for 30 minutes so that it’s not hammering, yet will check more often then my random every so often, sometimes weeks between follow up.

It’s added bonus is it’ll also tell me if a MySQL instance isn’t running by chance to ensure I am alerted to that.

I created an init script too to make sure it runs on startup on my home CentOS box.

/etc/init.d/mysql_monitor:

#!/bin/sh
# chkconfig: 2345 95 20
# description: MySQL Monitor
# Monitors MySQL Slave Status
# processname: mysql_monitor
APP=/usr/sbin/mysql_monitor
PIDFILE=/tmp/mysql_monitor.pid
case “$1″ in
start)
echo -n “Starting $APP: ”
$APP & pid=$!
echo $pid > $PIDFILE
echo “Done.”
;;
stop)
echo -n “Stopping $APP: ”
PID=`cat $PIDFILE`
kill -9 $PID
rm -f $PIDFILE
echo “Done.”
;;
restart)
echo -n “Restarting $APP: ”
PID=`cat $PIDFILE`
kill -9 $PID
rm -f $PIDFILE
sleep 1
$APP & pid=$!
echo $pid > $PIDFILE
echo “Done.”
;;
esac
exit 0

I’ll have to make more of my monitoring tools run using init scripts and rely less on cron.

After testing Linphone, Wengo, and several others, Twinkle seemed to be the one which seemed to be pretty robust.

I like to have music going when I’m not on the phone, but that proves to be quiet the distraction during a voice call, as well, I ran into problems with having the headphones in, and the speakers turned on at the same time (microphone feedback, for example).

Twinkle has the ability to run scripts on call incoming, call answered, call hang up, call missed, etc.  I put that to good use and created some scripts.

callincoming.sh

#!/bin/bash
/usr/bin/amixer set “Front” 100%
/usr/bin/amixer set “Side” 100%
/usr/bin/amixer set “Master” 75%

That script would turn the main speakers up, and the headphones (which I’ve got into the “side” speaker plug in the back), to 100% – so we can hear the ringing if I’m away from the keyboard.

The Master is also turned up to 75% which is where I find it’s ‘loud enough’ to be heard, but not deafening that the kids would get significantly disturbed if they were having a day time sleep.

The script can be expanded to kill the radio, or rhythmbox. I’d love to be able to stop the music streaming online, but I can’t get application control that the Sound Preferences offers just yet.

callanswered.sh

#!/bin/bash
/usr/bin/amixer set “Front” 0%
/usr/bin/amixer set “Side” 100%
/usr/bin/amixer set “Mic” 48%
/usr/bin/amixer set “Mic Boost” 0%
/usr/bin/amixer set “Front Mic” 0%
/usr/bin/amixer set “Master” 60%
/usr/bin/amixer set “Input Source”,0 “Mic”

What this script does is turn the “Front” speakers (my primary ones) off – so the caller / callee doesn’t hear themselves. Turns the headphones up to 100% (it’s actually got a control on the cord to the PC, that I turn it down and leave it). Turns the mic to 48% – this is a good level that doesn’t give the kids in the background into the call, but allows good level of voice, without echo.

Mic Boost – this was awful, picture a poorly mixed MP3, like 64k, turned right up, with emphasis on bass, it ‘crackles’ and ‘distorts’ heavily.  So, I kill that because I believe amixer resets after a reboot and setting it down is desired.

Front Mic is not used, but gets turned up to 100% for some reason, so I leave that off! Finally, I set the Input Source to “Mic” to ensure it doesn’t capture from the radio card for example.

callend.sh

#!/bin/bash
/usr/bin/amixer set “Front” 100%
/usr/bin/amixer set “Mic” 0%
/usr/bin/amixer set “Side” 0%
/usr/bin/amixer set “Front Mic” 0%
/usr/bin/amixer set “Mic Boost” 0%
/usr/bin/amixer set “Master” 75%

This script resets the music back to a usable level, kills the mic to stop that awful feedback through the speakers. As the others, can be expanded to start the radio, or rhythmbox for example.

This ability is also useful if we later want to expand the sound to other sound sources – like to the TV for MP3s in the lounge room!

After the scripts are created, chmod them a+x so that they are executable.

Adding the scripts to Twinkle was as easy as opening User Settings, going to “Scripts” and pointing “Outgoing Call, Outgoing Call Answered, Incoming Call Answered” to “callanswered.sh”. Set Incoming Call to “callincoming.sh”, and the call ended options and call failed options to callended.sh.

I was testing an upgrade for MySQL, from 5.0 to 5.1, to gain Row Based Replication instead of Statement based replication (as statements are a cause of SBR breaking, and it doesn’t proceed til I tell it to).

So, in my usual lazy, just do it, manner, installed MySQL Server 5.1, pointed the datadir at the old so that I’d have my data pretty quickly up and running.

And the installation worked, until I started MySQL.

I use hMailserver for email on my box, this uses MySQL for it’s settings and database storage (which is good actually, a config file is inflexible for multi domain environments).

The database used by hMailserver is InnoDB, most of my needs are well serviced with MyISAM.

In my rush, I didn’t do a backup. I didn’t copy the data folders, just went right over it, started, and got an error:

100610 22:15:32  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files…
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer…
100610 22:15:32  InnoDB: Starting log scan based on checkpoint at
InnoDB: log sequence number 0 1204538731.
InnoDB: Doing recovery: scanned up to log sequence number 0 1204538731
InnoDB: Page directory corruption: supremum not pointed to

It was shut down normally – using the Windows Service, the issue is that the MySQL database won’t work with the log files from the previous database for some weird reason.

I’m still yet to determine if any recovery can be done on the log files, making them usable again, however I have doubts.

I reinstalled my mailserver and did a quick setup on it today to get us operational again. And learnt a lesson – always backup.

Actually, I should have had a database backup of recent, I don’t. I have no database backup – I only have the datafiles as they are.

That instance of MySQL is rarely used, it’s most usage would be the mailserver, followed shortly by the replication I get it to do from the server in Newcastle. It is my home windows test server, I also have a linux one that I planned similar for – except that does much, much more and data loss there would very much annoy me.

So, I ask myself – why is there no backup of it then? Because, there isn’t.

I MUST use this weekend (which thankfully is a long one), to:
A) Figure out if that log is recoverable, and move back to it, then onward to MySQL 5.1 using a mysqldump.
B) Upgrade To MySQL 5.1 for the windows and Linux box here, and the box in Newcastle.
C) Setup replication using Row Based Replication for those databases I have that need it.
D) Design a regular backup script which backs up the databases locally, something like weekly (With email reporting).
E) Update regularly. 5.0 is really old now.

Reconsider using hMailserver is also on the cards, it’s not it’s fault, in fact, it has the ability to use it’s own seperate instance of MySQL (and does by default, and that’s what I currently have in Newcastle).  But, I want the ability to access that data – later for user purposes, greylisting statistics and so forth.

UPDATE: I found the issue! It turns out the ibdata1 file I needed, was in C:\MySQL Datafiles. In the Program Files folder was a 10MB ibdata1 file, which is why the error messages were occurring.

The Database came right back to life the very moment I shut it down. Copied the file from C:\MySQL Datafiles to the version 5.0 program files folder, and started the version 5.0.45 MySQL server with that file, and the log files intact.

Remember to remove the innodb_force_recovery option if it was added, and it can help to change default-storage-engine=INNODB to default-storage-engine=MyISAM – so that the server starts with MyISAM instead of INNODB.

This will not work if you don’t have consistent logfiles and data files – but otherwise, it’s a save!

Now, the plan is to dump all that data using mysqldump, and then go forth and better my lazy processes for database server management.

My colocation box is a HP DL360, G3, so it’s not exactly bleeding edge, but it’s a fantastic server, does it’s job and does it reliably.

Last night, we had a planned outage for just a short time to move the box to a different power situation, this was done and the box came back online.

It dropped off 6.30pm, came back 6.45pm. Then, 7.15pm there was notices coming through – I’ve began to ignore the warnings for a while now, so I failed there but it wasn’t too bad, because..

.. My colo provider, Inticon detected the port was flapping 20 minutes later, and went back to have a look, at 8pm he had advised that the issue was with the processor fan assembly, only one fan was spinning – strange, it was OK for a while before hand.

He went out and about to find a replacement part, and tried to find alternative measures, but came up empty. Considering the nature of the part – it’s pretty server specific, so it was understandable that we had no alternatives.

I then began hunting around for the part – found one, $185 online. I thought, wow, this must be a pretty premium part..

The contact at the colo provider went even further, and actually advised me there was a box on eBay for $250 – no kidding.

I looked at it, considered it’s marginally more expensive, but gives us a lot more hardware to play with as well. I contacted the eBay seller, asked if he could accommodate pickup for tomorrow – he was OK with it, fantastic I thought..

So, I head to bed – we can’t pick it up at night, it’s just not THAT important (and I had started work 8am, so by that point I was very tired). I think about this for a while in bed, then somehow drift off to sleep.

One of the kids is sick, so they crawl into bed beside me and annoy me, so I get up at 6am, head to sleep on the lounge, peaceful.

The next morning, I get up, sort out a pending PC related issue for somebody (it was arranged already), then the seller was interstate for the morning, not back to 1pm, so that was as good as we could get.

He then contacted us, advising he wouldn’t be back til 3.30pm – considering we were getting ready to leave around then, that’d work out perfectly. We got near the address in the southern Sydney area, but he calls and advises 4.30 – he was delayed. No problem, can’t do much if he isn’t there..

So, 4.30pm, he arrives, we have a look at the box, it works, he upgrades the CPU and RAM and throws in a 15K 36GB SCSI drive – fantastic I thought, more spare parts. The box worked, so in the boot with it, time to hit the highway for Newcastle.

We hit Newcastle at 7pm (after litterally, racing down the freeway to Newcastle), and the colo provider contact was already there – fantastic.

I take the server into the colo room, upgrade the box with what good parts I could – extra 1GB of RAM, 3.06 to 3.2Ghz Xeon, swapped over the fan assembly.. Power it on, it takes some time to boot, unusual, it eventually does boot.

Then, I enter into windows, get a ping going, get task manager up – make sure the virtual machine kicks over, it does. Box stays up stable.

Pack up the changed parts into the ‘new’ box, and get the colo provider to hold those incase something else fails down the track – pretty much a fully working machine with exception the front panel – which shouldn’t fail again.

Then, head back home approximately 30 minutes later, by this time the kid who is sick complains about pain, ends up needing a toilet break, all sorted.

Hit the highway to go back home, arrive at 9.30pm.

Wait for the colo provider contact to remove some trickery done to assist with the downtime, and I modify the SMTP route I put in place to redirect the affected domains to my router.

The outcome – all mail leaves, some spam shows, delete this, and I update the database, all in sync.. fantastic..

All up, nearly a full day, 460KM’s travelled, $14 spent on food at Aldi in Sydney, $22 spent at Maccas in Newcastle, $250 spent on a pretty darn good deal (compared to $185 for the processor fan assembly) – and we are all back and operational.

Hardware failures are REALLY annoying – especially when you can’t just call HP and say fix it in 4 hours.. – but we did pretty good for a 23 hour restore time (and still managed to keep some essential services up anyway).

And the dedication of Inticon is outstanding.

Had this been Servers Australia, I have no idea where we’d be. Part of me says ‘online sooner, cause they have HP DL360′s (or had them)’, the other says ‘getting Jared to get off his arse and take some damned action’. I’m leaning towards the latter, it’s just typical of previous experiences.

Note to self: work harder on high availability.

It has many different methods of operation, and can be used in different ways, such as a simple proxy, where you set it up, only allow your traffic, and set your browser to use it.

A reverse proxy, so that it caches traffic for busy websites and therefore removes load.

A transparent proxy, so that users don’t have a definite idea they are being proxied, and this is the method I used – purely so we didn’t have to set it up on the client machine, and that’d just be boring anyway.

So, I setup the access list in my wonderful Cisco 871. The access list is set such that any traffic that is not web traffic, goes out directly, the PCs that matter are also not routed to the access list, and every other machine is.

Then, I setup squid, I configured it as a transparent proxy, and then enabled the rewrite_program option, then I setup the rewrite script to rewrite the urls for youtube.com to xtube.com – so, any request for youtube, results in xtube – a more explicit knock off of youtube.

Unfortunately, the desired effect of having the traffic decrease was not the case, they indeed viewed the xtube traffic instead.

So, the next step (a later on thing), will be to setup a virtual host in apache on that server, mirror the youtube website content, and then replace the area where a video would normally appear with a singular cached video. I’ll have to find one worthy of such a task.

Then, any requests for youtube videos shall be met with a repeat video. Or, we could confuse things, and swap the code in the youtube video page, with one from another site, so when they next seek out the latest Black Eyed Pea (s)hit video on YouTube, they are met with a very, very different black, eyed, pea.

Since the change from Windows to Ubuntu, my email backup has been pretty lax – as in, it’s not a job for cron. I’ll have to work on this soon.

I find the subject to be somewhat true. I’ll say something, but I may not necessarily do that. Maybe it’s because that sense that “I can probably fix whatever it is” that promotes it..

I was having problems, randomly, my system would choose to have a ‘read only filesystem’ error appear, and all the labels on the windows would lockup, and the graphics would randomly offset somewhere in the order of + / – 150px, making using them difficult.

I tried to troubleshoot this, and despite the very obvious error message, I ignored that and went off suspecting gdm was the cause.

At some point between there and last week, I found the issue may have been in my face all along – I was forced a fsck by Ubuntu’s startup and this revealed many errors in the file system. So, I piped the output to ‘yes’, and let it do its thing.

After doing this, and switching to GNOME via KDM, I haven’t had the issue. I do suspect the issue as being the file system errors, as the system boots with ‘remount-ro’ – which explains would explain the problem.

Back to the spam problem, I got blocked once, dang virus. Anyway, unblocked, I thought nothing of it until recently, where I came across some more reports – I looked at the date / time, and sure enough, they were recent. I then looked at the router to see where port 25 clients came from and found the server IP there – that was OK, it runs an SMTP server I thought, then I considered they were client connections.

Remote desktopped, and I found IE opened with spam pages – the calling card of a virus. Checked task manager, ended an imposter process, and now running an online virus scan.

There’s a few things to consider here, in order of importance:
1. This would never happen if it was a linux system.
2. The system running a mail server goes a long way to promoting the ignorance of port 25 sessions.

However, being a ‘server’ there shouldn’t be many client connections, so now, I’ve changed the monitoring slightly – I now get alerts when a SMTP threshold is exceeded. One might consider installing an AV, but that just adds to load, the likelihood of a virus is actually, zero in normal conditions. Having users susceptible to viruses (gullible is the word), is not a normal situation here.

So, back on goes the firewall, a bash script is in place to grep the output of expect’s telnet checks, and email the domain about any large numbers of SMTP client sessions. I had checked ntop for the availability of this – it doesn’t have it. Cisco’s ‘kron’ doesn’t seem capable of sending the output out to a URL (I was initially going to write a PHP script to count the sessions and create an email).

Netflow is feeding ntop, but I couldn’t identify any quick way to get those sessions, so the ‘least’ intense method of checking is in fact, telnet, expect, and grep.

You can resume spamming now, apologies for the misconfiguration.

(Comments are fixed now).

For years I was glued to it, and couldn’t find enough drive to get rid of a really bad problem of mine. But, just as last year was coming to an end, I managed to get rid of the bad rubbish.

I finally, after years of contemplating it, made the change from a Windows Desktop to a Linux based Desktop. I loaded up the mirror site, found the ISO, downloaded it, burnt it, and installed it. Ubuntu for x64.

I was concerned mainly about my RAID setup (2 x 250GB configured as 500GB). I am capable of recovering data off a partition, or off a drive in many (not all) cases where there’s been no to low physical damage. Spreading the data across two drives, isn’t very good for a recovery aspect – if one fails, the data could well be gone!

So, I reinstalled Windows, as I was going to anyway to drop the RAID. After windows was installed, I also thought about the second drive and concluded I should place Linux on the other drive.

I didn’t place much thought into software replacements for my regular applications, such as Outlook, Office, and Dreamweaver. I also placed little consideration into hardware compatibility. I figured – if it doesn’t work, I’ll use Windows some more and work Ubuntu out later – at the very least, I’d have the fall back environment for work.

Much to my extreme amazement – Ubuntu was really quick to install, around 15 minutes to XP’s 1 hour. I then tested my hardware – video is a key aspect, as I use a dual head video card, and have 2x 19″ monitors attached to this. It wasn’t difficult to set that up at all. Then, sound – can’t really listen to anything or do anything without sound – so I tested – it works!

I also have a TV / FM Tuner – I figured, if anything was going to be difficult for it, this WILL be it.. I fired it up, configured TV Time, and … it worked! FM Tuner – “gnomeradio”, works.

Compatibility is vastly improved from the years ago when I was attempting Red Hat 3! Now, that was an awful experience. Ubuntu have really done a great job at making Desktop Linux absolutely user friendly.

Software replacements – I couldn’t port my PST files from Outlook, and since Email is a key function, I investigated an alternative. It just so happens the good folk behind Mozilla Thunderbird import emails automatically – and I did exactly this. Now, I run Thunderbird, works MUCH (i can’t stress this enough) faster than Outlook.

I have all my rules established, and where Outlook failed ‘randomly’, Thunderbird is very efficient at filtering the thousands of emails a month automatically.

There is only one ‘minor’ issue with, I suspect, Thunderbird, which relates to the mouse no longer able to click anything when a certain activity occurs (not sure yet), but it’s survivable.

I then looked for an alternative FTP program, and an alternative PHP editor. PHP editor came quickly – gPHPEdit, a nice light replacement for dreamweaver – it’s not as integrated ‘site’ based as it is, but I don’t really mind this anyway.

Browser is of course, Firefox. And this works really well.

Currently, I’m using aMSN for MSN messenging. It’s also not without it’s setbacks (unable to change sounds for messages and online / offline) but, again, I can adapt to it.

The system itself is great – performs fine. I’ll admit – there’s an issue with the TV Tuner card, when I try and place the system in a ‘suspend’ state – the system comes back, but a repeat flashing ‘shutdown’ applet window appears.

I’ve managed to isolate this, with help from the Ubuntu forums to my TV Tuner card, because it doesn’t occur when unloading the kernel module, but I can recover the system and work around this for the time being by unloading and reloading the radio module.

I have had to go back to Windows TWICE after the installation. The first, to export emails from Outlook to Thunderbird for use in Thunderbird on Ubuntu. The second, was to recompile a VB application for someone. The Windows install is still not completed, and I’m not sure I will spend the somewhat day and a bit to fully restore all applications. Ubuntu is ready for me now, and has taken very little time to get running.

I also recompiled my own kernel to use the performance enhancements available to Core2Duo (in an effort to see if the suspend issue was going to be easily resolved).

Windows isn’t all bad though – it’s got support for some games, but I’m hardly a hard core gamer. I found new Ubuntu games, and now play more Poker and BZFlag (great game).

The initial impressions before install were ‘expect a challenge’ – after install, I found it’s actually relatively easy for MOST, if not ALL users to move to. If I were Microsoft, I would be scared – the OS, after the short install, comes with everything you need (even an Office application). There’s really not much that Windows does, that Ubuntu can’t do as easy, if not easier, and the mountain of entertaining games (I’m really tired of Solitaire) makes the switch just that much more worth it.

Back to Ubuntu 8.10.
Enjoy!

I use a realtime implementation to allow for easy addition of users and to have the data I want in MySQL (makes it easier to monitor activity in CDR, etc). The problem recently was as a result of an upgrade, there appears to be changing to Asterisk’s debug log. Previously, Asterisk debug output would end up in the full log.

This made it easy to review the traces of an issue being experienced, and follow them back to the source – the log would simply contain the full sequence of issues leading up to the ‘issue’, and what happened after.

The newer version of asterisk however doesn’t enable debugging unless the startup option ‘-d’ is specified. safe_asterisk restarts asterisk in the event it bombs out – and one has to admit – crashing doesn’t happen often with asterisk, it’s ‘reliable’.

So, whilst I was reading the log, I was seeing messages such as:
” res_config_mysql.c: MySQL RealTime: Failed to query database. Check debug for more info.”

That’s a nice warning, it tells me to find the source of the issue, check into asterisk debug log. So we open the seperate file appointed to debug (it removes clutter), and it’s empty.

So, I look into the asterisk configuration file – is debug enabled? Of course it is. Spelt correctly? Yep. Where’s the debug – how can you debug a bug without debug?

I check Asterisk ‘logger show channels’ – this is described to show logging channels.

Sure enough, there is a line that states my ‘debug’ log is of “File” type, and is “Enabled” and is configured to be “Debug”. Asterisk isn’t doing anything wrong there.

Then I look into ‘help’ – core set debug. Could it really be, asterisk requires the debug option to be set? It does. You can set it there, or alternatively, you can have it enabled on startup in using safe_asterisk.

In the safe_asterisk script (nano -w /usr/sbin/safe_asterisk) – find “ASTARGS” and edit it to include a “-d” – this enables asterisk debugging information to a log file, by default, asterisk doesn’t log debug information.

This is confusing. The problem turns out that real-time now has an additional field required – ‘regserver’, adding that field seems to have caused asterisk to stop spewing the error message.

The issue may be ‘better’ managed by asterisk real time mysql having a seperate SQL based log for any errors it experiences. In a production system, you might not have so many errors.

Although, another point to note is the mysql realtime module is very chatty:
 res_config_mysql.c: MySQL RealTime: Connection okay.
res_config_mysql.c: MySQL RealTime: Retrieve SQL: SELECT * FROM sip_buddies WHERE name = ”
res_config_mysql.c: MySQL RealTime: Connection okay.
res_config_mysql.c: MySQL RealTime: Retrieve SQL: SELECT * FROM sip_buddies WHERE name = ” AND host = ‘dynamic’
res_config_mysql.c: MySQL RealTime: Connection okay.
This query might return no results, so it begs to question why is it even executed?

Enjoy!

Previously, you had to turn it off for peers stuck behind NAT, this is resolved in newer versions of Asterisk which attempt to determine if a peer is behind NAT, or if the peer is connectable.

The method is simple, for the peers affected or unsure, set the canreinvite option to ‘nonat’ – this will cause the audio to be connected directly to the peer if possible, otherwise, it will packet2packet bridge the audio.

It is important to have the right mix of codecs on the server and specified in the ATA settings also – without it, you’ll just have a mess to sort out.

NAT is a big problem for VoIP – nearly every gateway / router will implement NAT differently from what I have been told. The BEST configuration is to have the ports forwarded. This isn’t hard for many people – it’s just a matter of finding what your router calls it, and then forward the ports. Once the ports are forwarded, there should be only codec mismatches to worry about – and of course, if your codecs are specified right – well, there is no problem – the audio should be connected directly, rather than bridged.

Asterisk 1.4 and 1.6 have had many advancements to SIP and RTP implementation. T.38 support is at a strong point now – not completely tested by me, however with T.38 support, it greatly reduces the errors you may experience trying to transmit fax over ulaw / alaw.

Check out the UPGRADE.txt for Asterisk 1.6 – the list is massive and impressive.
Enjoy!

One of the key features I took advantage of is the context option for each SIP peer. It is possible to exclude one group of users from calling another group by using contexts.

Let’s say I want user 300 to never be able to call two users, 400 and 420. 420 and 400 can call each other, and call 300. User 300 is simply someone who will get bullied by 400 & 420, but won’t be able to get back at them.

This is accomplished easily through the use of extensions and SIP peer contexts.

I set the context for 300 to ‘victim’. I set the context for 400 and 420 to ‘attack’.

Then, in extensions.conf I create the appropriate configurations.

[victim]
exten => _310X.,1,Dial(SIP/${EXTEN})
exten => _310X.,n,Hangup

[attack]
exten => _4X.,1,Dial(SIP/${EXTEN})
exten => _4X.,n,Hangup
exten => _3X.,n,Dial(SIP/${EXTEN})
exten => _3X.,n,Hangup

The sip configuration for each is similar to the below:
[310]
type=friend
username=310
secret=yamomma
allow=all
context=victim

[400]
type=friend
username=400
secret=md5
allow=all
context=attack

[420]
type=friend
username=420
secret=secret
allow=all
context=attack

Try as 310 might, there is no way they can call via the system to 400, however, 400 and 420 can ring 310 as often as they wish.

You can of course add the 4X route to the extensions, and playback a message to the user.

This method can be used to restrict calls to certain areas, such as 1900, or international numbers. It can also be used as a call control method, in the event you don’t want a specific extension calling someone.

It is important to have a peer and user definition. After upgrading to Asterisk 1.6.0 on my test server, I found my test development server (used for simulation) running Trixbox 1.1 (yep, ancient crap), would not accept calls.

I upgraded to 1.4.22 on the test development server, and found the same issue. This was resolved by changing the sip definitions used for interserver communication to be ‘friend’ type, rather than ‘user’ type that appears to have been automatically created by Trixbox – no longer used.

Oh well, back to enjoying a long weekend here in NSW. Another day for me to experiment some more and try and push forward with the updates planned for OzVoIPStatus!

Speaking of OzVoIPStatus – one of the areas I’m having difficulties enhancing is the method in which providers may specify plan data – some have included calls, others have none. Some have unlimited calls, and others have untimed call rates. It’s not so much a difficulty designing this – it’s more in the way of presentation on the page.

Enjoy!

The fix was to simply run resize2fs with it mounted in read/write mode, I kept assuming it’d need them not mounted, so was trying to resize it, when it wasn’t existing.

I got some sweet speeds on the Exetel mirror today too, leeching CentOS 5 ISO (650MB) in just 12Mins  (around 998KB/sec).

The line drop out situation isn’t improving however, and I’m not certain I want to drop the speed profile down to the ADSL2 or ADSL1 mode for the sake of making the connection stay up more. A drop out every 24 hours or so is OK, sure, its better to have none, but, well, that’s not gonna happen sometime soon, here’s why.

A news article today reveals Telstra technicians have resorted to taping cables together with plastic bags to keep them dry, due to staff shortages resulting from the sacking of 12,000 Telstra employees (says the union).

So, what will happen when the plastic bags cop a good dose of lightning, and end up welded to the copper wire (or welding copper wires together). Will Telstra accept responsibility for the crossed wires and pay compensation due to breach of privacy on some mass scale (all those copper wires being welded together by lightning would result in much a case of people listening to other peoples conversations).

Who knows…  What will be the result of all this plastic bag repair work? What happens when the destruction comes too much to bear and the copper cables start requiring full replacement?

Or, is this some elaborate plan to derail the PSTN network to make FTTN a near requirement?

I found the news story amazing, at the same time disturbing, shouldn’t they have their cabling licenses removed for doing unsatisfactory work (surely, plastic bag repairs aren’t AUSTEL certified).

It’s a good thing we don’t live in Sydney though, where most of this seems to be the problem!

As per usual though, it pays to keep a second service (mobile phone) on standby incase of phone line issues, etc.

The Virtual Server issue being fixed meant that I could proceed to getting the next task on the list done,  updating it, which went through nicely, and allowed me to do something else, get asterisk running on it – yes, probably heaps quicker to setup a second VM, but I wanted one machine for many reasons, server load is one.

Our juice bill came today, and it was a good result, with the bill dropping on average 4KWH/day.

We just removed our off peak hot water systems equivilent load from the bill, and after todays measurement, looks like it could be dipping down some more with the new washing machine in place (larger washer, but less loads needed to meet demand).

I’m happy with the results, and hopefully we can see a natural progression in the drop for another bill, before we might either dip to the bottom, or find new ways to reduce consumption even further.

It’s good at least, that we have removed the equivilent of a hot water system from our bill. For the savings, we could be running 2 split system air conditioners (or, keep the savings).

Enjoy!

Not considering the consequences of doing so, I figure it’ll be able to be resized at a later date (or, dynamically expand), but unless you leave the maximum size at some huge amount (even if you dont have it), you do have to create another disk and migrate your data over to it.

Luckily, dd is our friend, and can migrate bit by bit, byte by byte from one virtual disk to another.

I tried to find a way to resize the volume so that the drive is able to be dynamically expanded using LVM (the installation was done a long time ago, I think with a CentOS server CD), the installation chose LVM by default, which I figure is OK – so long as I get a command line, what do I care.

Unfortuantely, that’s a bad move, LVM is great for managing volumes (like Windows’ Disk Manager), but when faced with something such as a resize task, you make the job exponentially more difficult by using LVM.

Simply, unless LVM has already told Linux about the disks, you can’t execute resize2fs to resize the drive to the new size!

So, the task that remains is to figure out how we can get that LVM drive in a rescue type mode, so we can execute resize2fs, and get moving on with the original task, updating the server and then installing more software.

Note to self: Never, never setup a VM with a hard drive limit. It’s a pain in the butt.

Anyway, I refer to it as “Lucky” volume management, because LVM has many features, including expanding over two partitions (yep), so whilst I was trying to find a way to simply resize the /dev/hda2 partition, it made it all so much easier by simply adding /dev/hda3 to the volume group, and then expanding the logical volume over the top of it as well (feels good having an imaginary 100GB drive).

If only the resize process was a little easier.

It should be fine, tomorrow, outside of the ‘uncounted period’ we should be able to source a CentOS ISO, boot that into rescue mode and the latest versions should have support for the LVM, otherwise, we are going to have to find another way.

Yes, I realise it might have been easier to simply move the larger folders onto a second virtual hard drive and worked with that, but that runs counter to my plans, eventually I’m going to migrate that server, so don’t want to be finding myself missing a second virtual disk or something else.

Enjoy!

Computers have become very much so beefed up that the grunt they have spends a lot of time idle, and only peak loads when you are doing intensive tasks, even then, they generally keep within 50% of the processors full capacity.

Game play will use more of that processor usage, but that doesn’t really come into question in the server environment.

So, just about many of the servers out there are very capable of having additional virtual servers added to them, and with the price of disk space falling, virtualization technologies are very much viable.

I run Virtual Server here at my home, and it hosts my linux test environment, this is to save me booting between OS and also allows me to have computing resources available to me simultaneously.

So, I don’t need a second server running, I don’t need to pay for extra hardware, power, or anything like that, when I want to run the linux machine (and we use it often enough anyway).

One common issue with Virtual Server Linux Virtual Machines is the clock being out of sync and audio style resources skipping ahead or behind in the established virtual machine.

The fix to that is a combination of setting ‘clock=pit’ as a boot parameter, and also installing virtual machine additions.

If you run a CentOS distro though, installing Virtual Machine additions as recommended doesn’t work as expected.

The reason for this is, the ./install.sh script looks for a string in the /etc/redhat-release file.

The string it is looking for is the third one. So, on Red Hat (where CentOS is derived) it would be “release” - Red Hat Release.

On CentOS however, it is “final” – CentOS release final.

The fix to this? Couldn’t be simpler.

nano into /etc/redhat-release, and between CentOS, add a space, so it might read Cent OS release.

Make sure you have the kernel sources (yum install kernel-devel), and that will allow the virtual machine additions to install, and once installed, leave as is, or just change it back.

Use Virtual Machines, there really isn’t much going against them, they save power, they allow you to have the extra resource available (ie. Linux system) as well as your Windows machine.

And, Virtual Server is one of Microsoft’s good releases, it’s a great setup for virtualization, nothing silly like VMWare, just straight forward virtual machines.

Enjoy!

And, it happened. I set out to complete something, and for the most part, it is completed.

I started today from yesterday, working on fattening a linux partition.

The process was pretty much where my thoughts were yesterday.

dd the drive (so dd /dev/hda to /dev/hdb), then remove /dev/hda, leaving only /dev/hdb (as /dev/hda).

I played around with fdisk and parted to resize the partition, but had no joy.

I googled at this point really quickly, and came across a blog post which suggested using resize2fs, to get the volume resized.

The problem for me was the virtual disk, which has a capacity limit beyond the bounds of actual disk space.

So, I resorted to trying to specify a limit, but gave up waiting after a while, and killed the command.

Rebooted it (hey, it’s a virtual box, worst case scenario, insert rescue disk, salvage any valued data, and insert rescue ISO).

Then, as the system came back, it complained about fsck (heh, that looks like an interesting command, considering the damage it is capable of), and so, I ran it. Answered yes to everything (like I care what inodes are worthy of bashing), so I then got back in the system.

And, ran a ‘df’, and what do you know, it’s gone and given us stacks of space. Exactly what I wanted.

Now, you know why I had to do this? Of course not.
I had to issue a yum upgrade to upgrade software on the box.
The yum upgrade seemed to suggest it had to have 30GB of disk space for 200Mb of downloaded packages.

You and I, and the world knows that it’s full of ‘it suggesting it can pull 30GB out of 200Mb.
But, none the less, I perservered and we came to a success!

Then, I got on with the rest of what I wanted to do yesterday. Install various software services, and get the system up.
Oh, and Elastix developers, I commend you for your creativity, but, alas, poor managed dependancies are enough  to drive me nuts. No longer shall I bother with any of the Trixbox, Elastix, FreePBX style distros. Plain and simple waste of time.

You want the job done, format, install raw CentOS. Get the packages from digium. Smack those on (that’s right, compile the suckers), and configure it manually.

It works as expected, it does all that you tell it to. And it doesn’t screw around with kernel dependancies, installing the xen kernel, or oslec modules, or zaptel where its not wanted.

It gives you exactly what you tell it to, without all the stuff around. You want a system that works, you definitely want to get rid of the slapped together distros (no offence to the Elastix crew, they are leaps ahead of Trixbox!), and do the configuration directly.

Anyway, moving on, the rest of the day was filled with migrating, DNS checking, and so forth.

Now, I just need to set about finding myself a few larger SCSI drives! Any offers? 73GB+, leave a comment.

Enjoy!

The first is the right way, the second is the wrong way, and the third, is perhaps a combination of both in a more interesting combination.

When the right way doesn’t (or can’t) work, and the wrong way is not an option, you then need to figure out a way of making the right way work, or the wrong way right.

I’m such a sucker for this kind of entertainment at times.

I’ve just spent the better part of today reorganising a linux filesystem, and using virtual drives to get space, in a space poor environment, I came across an issue.

I need to move the entire OS off one drive, onto another, due to partition size constraints.

How do I best do this, when the virtual drive cannot be resized?

Well, my solution thus far has been to add a second drive, which is capable of large growth, and then, dd all the files, and swap out the old for the new.

I originally planned on just moving /, and that’d work, but, I’ll admit, I’m not as proficient in the setup part of linux, to move partitions like that.

Being the sucker for an entertaining scenario such as that though, I tried to do it anyway.

So, I added the additional drive. That worked fine. Mounted it. Worked fine. Partitioned it. Yep, that worked fine.

Executed:”dd if=/dev/hda2 of=/dev/hdb1″, being carbon copy all of the partition of /dev/hda2, to /dev/hdb1.

It finished successfully. Sweet I thought. I thought that out, guessed and got it right so far.

Now, we have the next problem. Switching the OS to the new root and booting from it.

So I figure, ok, I can just open /etc/fstab, and change LABEL=/, to /dev/hdb1, and point that as the / partition.

Apparently, the kernel panic afterwards says no.

So, I go back to the ISO, boot into rescue, and change back to the old line, reboot, still no joy.

Hmm k, now we have a problem. So, I resolved it temporarily by removing the second virtual drive from the system.

Now, I must figure out a solution. At a guess, it’s still finding /dev/hda2, and sees it labelled as / and is doing some kinky stuff with it.

I probably need to find a way to Label /dev/hdb1 as “/”, so LABEL=/ works, then, find a non-destructive way of hiding /dev/hda2, so that we can go forward in the other much larger task.

Or, maybe it is more wise, to instead, dd /dev/hda /dev/hdb, thus replicating the full system.

I did have a concern about hitting /dev/random, and /dev/null, but figure, perhaps it won’t hit those and we won’t have a problem (and we didn’t).

Task for tomorrow will be to figure out how to do this copy successfully, otherwise, it’s off to find some DOS based clone software and use that to do the job.

Though, I should learn how to do it with dd, can’t be that difficult!

Enjoy!

Specifically what I want to do is upgrade the physical disks on my RAID array.

Unfortunately, I can only have 2 physical disks, and both are used by my RAID setup, which mirrors data.

So I sorted through some Google posts on RAID. RAIDing through all the various RAID information, I found a post that was concentrating on a larger array, and one which wasn’t a system drive.

But it was in some sense, useful. The suggestions they had were, adding the larger drives, and letting them rebuild, do it for set A, then set B.

Once you have that rebuild, keep the process moving with other disks.

That doesn’t work 100%, because when you have a RAID disk, the RAID controller configures the logical drive sizing.

In the case of non system drives, you would upgrade the drive to the larger disk, the RAID controller should hopefully already use the full extent of the Physical drive, and extend the logical drive (or do this yourself), then, you would just use your operating system to extend the partition to span the full drive.

In the case of system drives though, that are hosting the OS as well.
You are essentially required to reinstall.
At least, I thought so.

I imagine you could instead, do a GHOST between drives. Split your RAID out of mirroring, go into single mode.
Once in single mode, add your second SCSI disk.
With the SCSI disk in place, ghost the partition over.
Use partition tools to resize the partition outside of the operating system.

Viola, one parent drive done.

Plug that in, remove the other drive.

Add your second mirror drive.

Configure your controller to enter into mirroring mode again.

Start a rebuild of the second drive in your array.

And that should be it.
Keep the other two drives as backups of the data at that point.

All sounds good in theory right? Low downtime (the copy operation might take a bit), and safe enough to avoid any reinstall issues.

Enjoy!

I got thinking.

Do I code simply because its ‘something to do’? Nah. Not really. There’s plenty of other stuff I could do. Locally, I could do more and earn more, surely.

So it’s not “something to do”.

Maybe it’s because I love a good challenge? Yeh.
I’m generally a problem solver. I love finding solutions to problems. When in the right state of mind, I like to find a solution to something complex.

I love data mining as well. It’s fun playing with data. I like to match columns and rows of data, and work on ways to automate data matching.

And the echo to that statement, I can hear an echo from my partner saying “You sure do!”.

So, that describes me to some extent. I’m a problem solver. I like to find problems, and solve them.
I can sit on an issue for hours upon hours and have fun reaching a solution that meets all requirements to solving a problem.

And I think thats a quality that is somewhat required to do what I do. Solve problems.

I’m not completely lazy in solving problems that I might get enjoyment out of solving. It’s something that I find fun, I enjoy, and probably the reason I do code.

I code to solve problems. I code to create, and implement problem solving stategies.

The downside to it? First there must be a problem to solve to get my attention. Otherwise I don’t generally take much interest in what I am doing.

I find the most fun in solving a complex issue, or implementing something that solves a problem for someone else. Most of what I do is web based, and not a lot centers on PC based issues, but I do not say that, without first realising the numerous amounts of software and PC issues that I also solve.

I solve dial up issues for people with dial up modems occassionally.
I troubleshoot broadband connection issues for broadband connections occasionally.

I solve issues with Linux for a few people I know.

I solve problems with Excel related VB code. I solve PHP issues. I solve ASP issues. I solve Windows issues.

There’s a few requirements to solving a problem, for example, depending on the extent of the issue, or the complexity of it, I might want to see it in front of me to solve it. It could be that it’s unfamiliar and in seeing the problem, I would be best placed to solve it.

Or, it could be asking a question on a forum, sure I’ll read it. If its not descriptive enough, or provokes enough thought in my mind about an issue to spark a solution, I probably can’t solve an issue.

I’m curious, what other industries would use a problem solver?

There’s plenty out there, I’m sure, such as Police Officers and Lawyers, and Fire crew, and well, to a much lesser extent, our government (they still do solve SOME problems, right?).

But on a serious note, there’s something driving me to code over being a police officer, lawyer, fireman, or politician.

It must be something specific about coding that drives me to enjoy solving problems! The answer to that isn’t immediately known, and I can’t seem to consider why I enjoy solving problems, with PCs. Maybe that’s just my specialty, when you consider that I enjoy solving problems, and I have HEAPS of experience with computers.

Anyway, my point is, I enjoy solving problems, and I enjoy working with data, so out of everything I could do on a PC, the best is working with code and data to solve a problem.

Enjoy!

You can’t generally do that when swapping between say an Intel 865PE chipset, and an Intel P35 chipset, simply because Windows will vomit so badly, you’ll be mopping the floors of the mess for weeks or even months after.

Once its tried to change over as a result of detecting a hardware change (ie change in Chipset) windows will literally end up in a blue screen crash fest.

Luckily for me I don’t get caught up in that trap either.

I generally take the working windows drive out, and any drives containing important data. Get the system running on a new HDD. Insert the old HDD afterwards, and then copy data freely about until you have the desired effect.

Next, if there is no room (or it seems wasteful), you generally take the drive out and that’s it really.

Unless of course, you come to the situation I now have.
We have in the server:
2 x 80GB drives – bought em years ago, off eBay.
1 x 160GB SATA drive.
1 x 250GB SATA drive.
1 x 320GB IDE drive.
1 x 120GB IDE drive.

The 320GB was out of my partners system, which we bought because she had a failed 120GB years back, and I still haven’t got around to doing the head swapover on that drive. We’d love to recover the data though.

Anyway, the 320GB is an IDE drive, and her system is SATA (which I planned for, I purchased a 250GB SATA to replace it).

So, now out of the above list, it’s pretty obvious I am in a position to perhaps do a merge of some of those drives onto the 320GB IDE, and therefore realise some power savings (costs more to have 3 drives running than just one bigger drive).

The problem of course, comes down to adjusting the server, and the network to suit the modified conditions. For example, the second 80GB drive serves as my internal updating server (BIG PLUS, seriously, if you upgrade machines, or change machines, and still use XP for some machines, that’s a BIG saving, it updates them without the need to go online and fetch updates, therefore saving your net connection).

So, to get rid of two or three drives (and make room so I can put the 250GB that was in it, back in it), I have to take all the data off the update services drive, dump it in a temporary folder, do the same for the 120GB drive, which has all the installers for all my applications and one of my longer running clients data on it, and that brings it to 200GB, leaving 120GB of room, so maybe consider taking out the 160GB SATA, though I am not in a hurry to do that drive just yet.

The server here has access to around 1010GB of data, of that, it’s used much less than half.

Might be time to rip some drives out and chuck em on eBay! The problem of course is trying to manage the data movement without upsetting things too much, and getting the drive letters back in the right place. Nothing too bad.

In other news, I went online to Jaycar just recently and decided I might just see how much juice some of the house sucks down the power lines, at least we can then decide where we want to apply even more downward pressure on our power consumption.

They only cost $39.95, and from what I have heard are pretty good devices, measuring power intake and reporting on usage statistics, in fact, I heard you can even calculate the bill for that particular power point.

Seriously though, I support timed based billing and not the use of Off Peak metering, since you aren’t allowed to have power points designated as Off Peak (obviously due to the problems associated with the supply nature).

Some of the meters available can even determine Time Of Use, which is a great feature, so that they can hopefully start time of use billing. Burn the power at 6pm and pay the standard rates, burn it at 1am and give a cheaper rate (since the network is cheapest in the earlier hours of the morning).

Anyway, the new system for my partner is also up and running nice and smoothly, still in that process known as “Software Installation” – where you repeatedly find programs you are missing and reinstall them, and to cap that off you have the settings to adjust, such as wallpaper, screen savers, and program specific settings (IE Home Page, etc).

All a royal pain, and its probably why many would instead setup an Active Directory server so that profiles are portable! Makes sense. But then, the data is on the server and not the workstation, so that raises more issues.

It was actually a smooth day with the transfer, and my little one was a perfect assistant to the task, helping wherever he could (you’d be amazed at just how smart a young child can be!!!).

I also got another practice driving test in today, and it was better considerably, with just 2 fail items (so a 50% improvement). The drive itself was smooth, and the only real issues focused on those pesky things they call reverse parallel parks, and my way of taking turns (too fast, so I’m crossing white lines). All seems likely however that we can improve again to top that and come out with a great score equal to a pass and no fails.

Interesting, I’ve avoided the Telstra supplied Broadband Services suck issue for a bit now. Maybe because they are priced to suck, and that’s already a well known fact? Maybe we are just tired of Telstra’s greed, yet also tired of searching for an alternative solution..

Enjoy!

The particular provider issue wasn’t determined until just recently, but has been fixed.

The reason for the delay? Well, I was investigating that issue and the time it took to isolate and correct it, was the time that should have gone into finishing off a last few lines of code, and then taking the site offline so I could integrate it all.

The changes are all backend, but they will affect the front end to some extent as a direct result.

I also took the time today to do a few other things, like watch a movie – nothing entertaining, was “Santa Clause 3″, pretty plain really, none the less, it was a break away.

Then, I got my night driving section ticked off in my log book in a nice long drive towards Budgewoi and back – it’s more of a highway stretch of road, so wasn’t town style night driving, and therefore made a little more complex, than that experienced in the suburban streets I’d rarely find myself on at a night.

So, therefore, it wasn’t completely error free, with a few instances exceeding the speed limit (it’s a 90K street, I kept dropping back to 80K- my limited limit, but continually found myself over the limit a few times), and a little distraction caused me to nearly touch the kerb, pulled away and corrected though.

But indeed, it is made more complex in darker highway conditions, when compared to that of regular streets, such as the well lit up Wyong Road.

Aside from that, I was pretty much error free, mainly due to the time, and the stretch of road being a straight ahead drive really, so no real blind spots to check, no turns to make, no giving way, no stop signs, no entering traffic, … none of the standard traffic conditions you’d be exposed to on a general mix of road conditions.

But, it got the night drive ticked off, so that works for me.

This week is an exciting week! I have a few things I am taking care of, such as the finalisation of a released website, the new system for my partner, getting my internal server back online, hopefully finding time to finish off what I was doing today, a few other projects and some more excitement.

All in the lead up to the most exciting weekend you’ll find in around 3 years, the federal election on Saturday, which really is looking for a Labor win, and my thoughts recently suggest voting Labor, because the way Howard is going against drugs, our crime rate is likely to increase.

He plans to withhold Centrelink funds from those who are convicted of drug offences and only allow vital purchasing.

The problem with that is a drug addict or not, they’ll get the drugs. They’ll resort to lives of crimes, crimes that will harm innocent members of the public.

Better having them shooting themselves up, then causing more work for our police indirectly, by withholding Centrelink funds.

It’s a disaster if they take the drugs away from them, because, they’ll steal to get the drugs. Drug dealing and drug usage affect the selected individuals only (generally), and with the supply being taken away, they’ll simply find a new way of getting their supply. Crime will become the answer. And instead of select individuals being affected, members of the public will also suffer.

I agree with his stance on intervening on Child Abuse in NT, fantastic move to clean the area up of Child Abuse. Children were affected.

This move however? Seems to be more about winning votes, by being seen as doing something about drugs.

Indirectly, creating more problems.

A vote for Rudd will see some good dollars spent over the nation, that’s probably what’s going to win me over at this rate, though I can see Labor State governments have bad debts, so perhaps we can instead later, put Costello back in if they do push the nation into a defecit, and the Liberals will again save the nation.

Better having a debt than having a nation of crime, pushed by poorly thought out drug policies.

A whirlpool poll reveals a 77% Labor win, when asked “What sort of Government will Australia have” – with notes that state not necessarily who you will vote for.

77% believe we will have a Labor government. Just 18% believe it will be Coalition.

Of course, that’s only 174 users, and the polls in main stream generic media suggest that its more close, and might even see a Coalition win.. It has happened, it can happen.

Here’s hoping that not necessarily the Coalition or Labor, but “The Best Party” wins.

Enjoy!

Essentially that was just my older 2 year old machine, which unfortunately, had an accident. It wasn’t the servers fault however.
Unfortunately, a Pop Top bottle filled with a bit of Sprite came into contact with the Zalman fan after being knocked on to the case, laying on its side, resting on the two servers that superseded it.

Unfortunately for that motherboard, it didn’t seem to want to come back on after suffering numerous injuries as a result of the soft drink being quickly propelled around the case by the Zalman fan (which I just put into that case not so long ago, upgrading my own fan due to copper fin ‘bendage’).

Anyway, so the time has come to solve two problems, one, my partners thirst for more computational power to process tasks faster, and the newly invented problem, my server suffering from liquid near the brain (CPU).

The good news for me is my habit of buying parts nearly identical for my machine and my partners, and the end result of my older machine becoming the new server means I don’t have to do any formatting or reinstall.

It’s simply take her high spec P4 3.0Ghz 531 Processor, 4GB of RAM, and the same model motherboard, and place that in the server case, plug the HDDs back in the same order, and fire them up, and we should be good to go again.

Unfortunately however, for me and my server (and the projects I maintain on it), they have to sit back a bit whilst we hack data from the Linux Virtual Hard Drive (we use Linux Virtual Machines over Virtual Server, on a Windows Server 2003 base), from the hard drive on that machine, so that I can continue finishing some updates to OzVoIPStatus, or, wait until next weekend, when the hardware for my partners machine, is to arrive (they were supposed to be here Friday, but the distributors can’t tell the difference between Sydney and Melbourne – strange, I can, one has a highly problematic public transport network, and its not Melbourne).

Anyway, due to the delay from Melbourne, I have to wait until at least Monday to get the gear, which means probably taking Monday (which I would usually use to put towards focusing on websites of clients) to build the server and get it back online, just so I can get somewhere, otherwise the whole week will be nearly spent doing little at all in the way of development (lack of test area).

The new gear will see a performance improvement for my partner, who long has been stifled by the Hyper Threading artifical dual CPU technoloy, and will almost definitely love the ultimate responsiveness of a dual core machine!

I was planning the upgrade, but not so soon, after Christmas.

The next item on her upgrade list is two LCD monitors to help trim down the fat on the power bill her two CRTs generate.
Although trimming fat shouldn’t be something we have to do, instead, power solutions should be worked on that are friendly to the environment and reduce costs to consumers. Such examples exist in GeoThermal power. Truly amazing idea (figures quoted as 5c a KWH, compared to the ~12c KWH we pay now (and the 15c KWH after XX kWH).

My plan to release a key update to OzVoIPStatus is likely to go ahead, with thanks to Virtual PC 2007, which works just like Virtual Server, except its desktop oriented, which will do now for the time being.

My updates aren’t much on the web side, and more in the back, though some of the changes I imagine will be visible to viewers, as this essentially will fix a bug with outages being logged every 60 seconds due to the more passive nature of the tests currently being done.

It sort of sucks to lose a machine that isn’t so hardly worked yet! Many of my machines get around 5 years or so before I take them out, simply due to aged hardware, leaky capacitors, warping due to CPU heat, etc. But then, it was originally my workstation, and I do work them pretty hard.

So, tomorrow, hopefully, assuming I complete it, we’ll have a OzVoIPStatus fix up that solves outages being logged incorrectly, and can (but won’t for a little while) provide data back out as to what was found at the cause of each outage.

Enjoy!

1. A new machine

I’ve got myself my new machine up and running in the last 2 weeks, and I’m still loving it!
It wasn’t expensive like the those who buy Idiots By Me (IBM) or Dill (Dell), but it wasn’t cheap like my previous new builds that come in a little more less than 1K. The better side of this is it’s future proof, and is a great purchase.

2. Another server

No, I’m not replacing the OzVoIPStatus server, which has been running fantastically fine since it’s BIOS upgrade which cured a memory incompatibility issue causing linux crashes several months ago. In fact, I looked the other day and it had 40 something days on it. Rock solid.

This server is for a different operation, and will give me much more control over some problems that have been happening, and at least allow us to pinpoint and solve these more readily.

3. A car

I’m somewhat lazy when it comes to getting my P’s license, mainly because I don’t really go anywhere that often, generally just the regular food shop and that’s roughly it. Not that we don’t enjoy our home lifestyle however.

Anyway, the goal with the car is obviously to become less of a big spender on public transport, and instead drive myself, and it also opens a few more oppourtunities for us, and so will be a worthwhile purchase.

It’s such a tricky task entering a private used car buyer market though, buying from a dealer you are sure its not hot, but on the other hand, you can’t be sure its mechanically sound either, as there are a lot of poor dealers being exposed.

So, buying privately you can at least ask questions and measure the nerves of the seller to see if he is obviously sure of himself and the sale.

Ask for obviously papers to prove ownership, check license and then go do a REVS check to be 100% certain. Can’t go too wrong.

Another issue with private buys though is arranging a mechanic to check the car (or an NRMA check), from what I understand the wait times for that is something like 10 days. That’s a fair wait to ask for someone keen (or not) on a sale.

All that shopping however, is very worthwhile and money well spent, compared to spending money on Telstra services!

Enjoy!

This machine was loaded with a fair bit of software, and I don’t think the user was keen on dropping all the software to save on space, and I imagine rightly so too. You install items of software because you use (or plan to) utilise them in many ways.

And this user is very remote, as remote as remote gets, the other side of the country, in a regional area, running on a BigPond satelitte connection, and pays more than $600 for it.

So, what I had before me was basically some way of guiding a user through resizing his hard drive using partitioning tools (hey, he knew what a drive and a partition were, and what RAM is!, can’t go too wrong!).

The solution was that I basically was guiding the user through, and the user did everything to the letter.

Resize the last partition down 10GB to make an extra 10GB free at the end of the drive.
With that same partition, move it to the end of the disk, placing the 10GB of “space” at the end of the partition we wanted to resize.
Resize the middle partition an additional 10GB to swallow the space in the middle, and viola, resize complete.

So the theory goes.

Partition Commander was the software we were using, and it finished the 1st resize step just fine.
At the 2nd step, where you would wait for a partition to be moved on the disk to make the free space in the right position available, the system locked up.

Now, I know some IDE actions can produce that same issue, been there, done that, many times, but the user checked the HDD light and he concluded the system had locked up.

Great, just what I needed with the many hours of time that was spare that I would have been able to use … differently.

So, I was somewhat stuck on how to resolve this.

My immediate reaction was a rather annoyed ‘ohhhhhh!!! ‘, realising that the user’s emails and data was on this partition. I wasn’t going to panic or give up on it, so in the hours that followed, were many checks of software items to find something that would do the job of:

1. Getting the system booting again (apparently, Partition Commander writes an MBR and then replaces it, I believe, when it does any actions, so that if it fails, it can recover in some way), so the system wouldn’t boot when trying to get back into Windows.

2. Get that partition assigned as an NTFS partition and recognised the data on that drive, so he can function as normal.

In the 6 hours that followed this, I was very much annoyed.

I tried Partition Magic, Smart FDISK, EasyRecover, Seatools, anything I could throw at this in search of an option to assign the partition it’s System ID of 07.

We ended up getting Winternals to recognise the missing volume, but.. but..

It didn’t have any valid destination to copy to, A: ? For 20GB ? yeh right. C: (ram disk ? well, if he had 20GB of RAM, and a dang good UPS, maybe), R: (another ram disk, not likely).

So, that was put back out as useless at recovering the stuff up caused by Partition Commander, which locked the machine during a partition move and caused the Partition Table to be in a state of confusion over whether it is or is not a partition, and whether the data is recognisable.

But, we made a breakthrough, through all the testing we did to try and get a boot, or a recovered partition, we got the drive to boot back into Windows.

And I knew just the tool for the job at that point.
Nothing beats Runtime Software’s GetDataBack for NTFS.

This item of software was shown to me a few years back when I had a drive that just was being a pain, and it didn’t get much to recover, but it did recover data then.

So, with just a screwed partition table, but a drive that has 2 fresh formatted partitions at hand as well, the solution was obvious, recover data off the hidden partition, and place the recovered data on the formatted drive, sure beats rewriting a partition table and risking losing data.

But, the whole process could almost certainly be done completely different.

We went in ready to resize partitions, when it should have been that we used ghost to ghost the partition to the new drive, and then deleted the other partition and resized the original partition and recreated the second.

That would have saved.. well, many hours.

I have used Partition Commander for a while and it normally performs well, but I doubt I have plans to do any resizing of drives with it again soon, I think I would rather use the ghost method, and a larger HDD (they are cheap anyway).

Enjoy!

A domain I have (no, its not this one, and its not the other popular one), has been getting spam sent to it at a not so low rate.

The domain doesn’t have a link in the search engines, and hasn’t been published anywhere.

But the spammers still have a go don’t they.

Anyway, the catch all was doing just that, catching it all. So the spammers were having a party sending spam to a catch all address that is unmonitored really, unless I manually decide to go and check it for anything lost (rarely ever).

So I investigated a few options. And decided I would see if I could sort out another issue that was bugging me recently, relaying mail via the ISP. A pain when you want mail to originate from your own server.

The solutions were all centered around ISPConfig, however, the setup I run doesn’t use MySQL. So, I tried to skim those for anything obvious that was stopping me relaying email directly.

I didn’t want to use mynetworks, as this isn’t ideal for the situation, I wanted something specific that would do the trick.

And then it came to me.

The spam emails are all just guesses at domains and don’t even come close to the right domain that I am running on.

The addresses they came from all were from seemingly existent domains.

So I added a regexp to kill all the email to the address they were attacking at, and viola. Bye spam. I also implemented a DNSBL, however, I remain under testing on this one because, just like Telstra with a national telecommunications network, you can’t trust one person to provide you with a service consistently and with good service.

The mail has been flowing through nicely, and it seems to have reduced the issues to start with, of spam, at the same time, allow me the access for relaying messages directly through the server, without turning it into an open mail relay, and doesn’t use Sorbs for their blocking (all dynamic IPs? Come on!), and seems to handle all email thus far rather nicely.

I’ll keep monitoring the logs though and if anything seems out of whack, will be able to revert the changes or update them.

SMTP servers are absolutely not the easiest to configure. They aren’t out of this world, but the particular one I was configuring this time was a little more difficult and the information, and the logging messages weren’t very helpful!

But through those hours of chopping into it, i’ve got the setup that is great, I get emails in and out, the spam is down to.. well, assumably 0.

Enjoy!

I’ve had to move between servers for a little while now because of a few issues that were causing software issues.

Those issues have been focused on and solved, however, each time I move server, and begin reconfiguring, I always consider each one could be done a little differently.

Each different software item we run could be configured a little differently for a better result.

It’s seriously an artform, the art of configuring a linux server from scratch.

Of course, I didn’t sit there and manually reconfigure the box each time, that’d be a big waste. Instead, we use the power of scp to copy data over, have all the processes on the server tested and running, implement the configuration files and add the data and let it run, after some testing of course.

Now, each time I do this, not everything works 100%, for example, email might act a little funny, or we’ll lose the ability to receive or make calls.

Actually, just while on that note, I’m “VERY” tired of the inconsistent programming by a particular coder, I won’t name them here, but it’s just “silly”.

For example, the first update, they used a variable like “get-this”, they then changed it in the next version, assumably to make it shorter, to “get”, and this time we fetch the next update, and it’s changed back to “get-this”.

Very much annoying, as if you use a direct call to that, you end up having more issues, like fixing the issue.

I wrote about this a while back, basically, the person should be using an alias to the function, and not simply “renaming”, other coders use the same, and it’s just inconsiderate to be changing it so much.

Anyway, with the move near complete again (and a nice 9days uptime in the data centre! ), we are on top of it all and screaming along to finding my weekend time free again for OzVoIPStatus maintenance.

I originally planned this weekend for it, however, moving from a underpowered loan server to my beast was much, much more of a priority, and moving a dynamic site like OzVoIPStatus does take a fair bit of thought. It’s not a simple one line command, though, I am sure I could make it so, it takes a fair bit of thought into what part of it slots into what area of move, and basically everything is live before data is even migrated to ensure that we always have the providers monitored, one server is on top of it, the other is getting data and staying up to date, and keeping on top of it.

Eventually the two are split, and we see if the configuration was successful.

Assuming all goes well for OzVoIPStatus, we move on to setting up mail for it, and test that its successful, and eventually we land with the setup configured.

It’s certainly not a walk in the park, and takes a fair bit of time, and in nearly every case, I am sure I could have configured it all differently and got a different result!

As I mentioned earlier, there’s gotta be some sort of artform to it. You really have to find the time each time to manually configure and compile software and optimise it to your requirements, and adjust configuration settings to tweak them to the peak.

That’s not a quick task, and I guess the only thing that can be said is.. Thanks a heap for yum and the open source gurus that will happily compile these packages and a configuration file that makes it simple to configure and tweak each option to your needs.

Trixbox and systems like that just basically have a script that runs (and breaks in several scenarios) after install and writes out configuration the way you want. It’s just another compilation of packages really.

Trixbox could have been better done in my opinion too.

It would have been far easier to not rely on a “trixboxload” folder, and instead, use a repo, then it’s a simple yum install trixbox to get it and all its dependancies. For a CD install, the same thing, just a repo that points at the cd .

The script they use is generally buggy from my testing, breaking for various reasons, like yum not installed (the first case ), all could be easily checked for and error handled (but its not).

And the other let down for Trixbox: It installs a dang IRCd for no apparent (or related) reason, therefore opening the system to a security exploit, and besides, it takes up resources, needlessly.

Back to the topic: Building a linux server, building a distro, it’s all a very unique artform to get a server from command prompt to your chosen ideal configuration, very fun, if you can find the time.

Enjoy!

The final part is to get ChilliSpot running on the Tomato router, which includes modifying its firewall to block packets.

The first step tonight is to modify the firewall script. Tomato is a bit different in how the firewall is setup.
Open the file in the network share, etc/firewall.openwrt with your favourite editor.

We only need to make a few changes to this file for tomato to work.

The file starts with defining the WANIF. This is generally needing to be changed to:

WANIF=`nvram get wan_ifname`

The next two lines:
WLANIF=”eth1″
LANIF=”br0″
.. are fine.

The definition of IPTABLES is correct, as “/usr/sbin/iptables”

Delete the next 4 lines, which are:
for T in filter nat mangle ; do
$IPTABLES -t $T -F
$IPTABLES -t $T -X
done

The next lines change the behaviour of the firewall, let’s not confuse anyone, and leave them alone.

The next lines are all OK, to the point where we get to the end, which is Enable Nat on the output device.
It’s generally already enabled, so delete that line from the file, save the file.

Now, with that done, we are all set to set to start the application on the router.

We start with:
/cifs1/usr/sbin/chilli –conf=/cifs1/etc/chilli.conf

And, unfortunately, with debug on, we get:

ChilliSpot version 1.0RC3 started.
chillispot[5490]: tun.c: 673: 2 (No such file or directory) open() failed
chillispot[5490]: chilli.c: 3437: Failed to create tun

What is the issue here is, we need to get the tun.o module loaded for this kernel. I’ve tried loading one from a different kernel, in the hour I’ve taken since starting this post, to now, to find out, and unfortunately, we’ve hit a deadend.

We can alternatively, recompile the firmware with mknod support, and that should set us on the right track to running ChilliSpot on Tomato.

Either way, that’s a very time consuming operation. If anyone has a tun.o module for the 2.4.20 kernel, I’m keen to get my hands on it and give that a go.

I’ve also asked Tomato’s developer if he was able to provide some input.

The task of recompiling the firmware involves downloading the Tomato Source, done.
Downloading a 180MB Linksys Source file, not done.
And having a feel around to find out what we need to fix to make ChilliSpot run on Tomato’s OS.

Hopefully we get some more information on this soon, and I can finish this off.

Believe it or not, the total time this blog post took was 2 hours, and around 90 minutes of that was spent researching and finding a tun.o module.

You can always look at Open WRT, but you lose the decent QoS that Tomato offers.

Maybe a merge of the both would be ideal? Not sure what we are up for, will certainly look when I get time to open this up again!

Enjoy!

What we need to do now is look at the configuration, and make it suit our requirements.

Open your network share, and edit the file ‘chilli.conf’ in your favourite (or not) editor. Chilli.conf is located in the /etc/ folder.

Find “radiusserver1″, edit this to the LAN IP of your Linux server running FreeRadius.

Make the same edit to radiusserver2.

Find radiussecret, set it to the value of the secret you specified in /etc/raddb/clients.conf yesterday, we chose “mypass”.

Find dhcpif, change it to “eth1″ (should already be set to this).

We haven’t yet created a webpage to handle requests as yet, either. Chillispot (the regular package), comes with a script called “hotspotlogin”, download the regular chillispot Source Code, from chillispot.org, open the file with WinRAR, and inside, find the “doc/” folder, which is inside the chillispot-1.1.0 folder, inside the archive. Extract the “hotspotlogin.cgi” file to a place where you can get to it.

Rename hotspotlogin.cgi to “hotspotlogin”.

This is relatively simple to implement on most apache servers, you do also require SSL for this to function. The parameter in the configuration file has https://radius.chillispot.org/hotspotlogin, we want to edit this to be the IP address of the radius server, eg.

https://192.168.1.103/hotspotlogin

Find “uamsecret”, and set a secret, remember this, it has to be specified in the “hotspotlogin” file.

This guide is going to assume you already have Apache setup with SSL, and perl, and configured correctly. If you don’t, you should refer to Google for assistance, it’s relatively simple, but can be a little confusing for some.

You can place the hotspotlogin file in any folder on your webserver you like, but it should be in the location that would match the uamserver we specified in chilli.conf. Essentially, you should try and access the uamserver URL in your browser. If you see a Page Cannot Be Found error, you haven’t placed it in the right location, or your server isn’t configured correctly.

For most, the location needed is /var/www/html/.

Permissions on the “hotspotlogin” file should be set to read and execute for the apache user on your system.

The next part of this, is to edit the hotspotlogin file, with the uamsecret, and other settings.

Open the hotspotlogin file with your favourite editor, and locate the “$uamsecret” line, remove the # at the front of this line, and set the secret to whatever you set the uamsecret to in chillispot.

Uncomment the next line also, which is the “#$userpassword” line.

With those lines setup, we now only have edit the iptables script to work with Tomato, and ensure we don’t destroy its fantastic QoS changes in the process.

I am yet to dig this up, so that will come another time. To start chillispot tho (without the blocking of internet access that it does), we can simply telnet into the router, and start chilli, but that would be pretty pointless right now.

Again, I remind all this information is provided on an all care, 0 responsibility basis. Don’t come crying to me if it doesn’t work for you, I very well might have not tested this information myself, and it would serve you right for taking information off the internet, instead of investigating what you are doing. Of course, the information provided isn’t intended to do harm (and of course isn’t likely to cause harm), so use it at your own risk, if in doubt, hire a professional. This should be a little more scary compared to the last one .

Enjoy.