Lazy Hacker Babble

Just some random babblings from a lazy hacker…

Archive for the 'Software' Category

Restoring My Linux drive

Posted by hsin on 19th June 2010

Last week, while I was using my Linux machine the windows manager crashed. Menu bar disappeared, programs shut down, kaput… When I rebooted, it told me during the file system check that something was wrong and that was it. No more booting.

Fedora is a little annoying because during boot up it doesn’t show you the start up message unless you hit F8 although the message didn’t really help since it said that it couldn’t find the UUID of the drive which can be misleading. Most posts on it talks about how UUID mismatches between your /etc/fstab and the actual drive’s UUID causes the message. For me, that wasn’t the case since what happened was that Linux was unable to find the drive, but the message along with “Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK” in the dmesg log did help me figure out that the problem was with a particular drive (I have two in the linux box).

My first guess was that that partition table was corrupted, but I couldn’t use any standard utility to fix it since Linux couldn’t mount the drive at all. I feared that it was a hardware failure and I’d have lost all my data (I had an older backup, but prefer to rescue the most recent data if I could). I take out the drive and replaced it with a new drive, booted it up with a rescue disk and recreated the partitions that was on the drive. Initially, the system didn’t see the drive and for a moment I thought, “Oh no, don’t tell me the controller board is fried ’cause that means having to buy a new motherboard.” Calmness prevailed and I realized that I put in a new drive and my motherboard most likely didn’t support 3 Gb/sec SATA drive. I put in the jumper that put the drive at 1.5Gb/sec and the system recognized the new drive. That allowed me to boot up the machine again.

Next thing was to see if I can rescue the data on the drive. I put the old drive into an external USB enclosure and plugged it into the Linux box. No dice, Linux still couldn’t do anything with it. I then plugged it into a macbook that had macfuse and support for ext2/ext3 file system and the data partitions came up! Now I wanted to copy over the data as quickly as I can in case the drive really fails, but OSX was pretty crappy at it. It would stop or complain and couldn’t get through the copying for various reasons including that it isn’t a case sensitive file system and so when it runs into errors it would abort the whole copy.

Reformatted my portable backup drive to be case sensitive and used rsync instead. Once I copied all the files over, I plugged the USB drive into my linux box to restore the files. Of course, Fedora boots into the GUI interface and I needed to copy the files over first so that the GUI can boot in correctly. Fedora won’t let you boot into the GUI as root so you’ll have to be at the machine and using CTRL-ALT-2, go into the console and log in.

I was then able to copy over all my files and get back to work!

Posted in Linux | No Comments »

Setting up GIT for home network.

Posted by hsin on 8th April 2010

I have a few different computers that I use at home and I wanted to set up GIT that I can access my code from any of them. It took me a bit because I was too used to working with a client-server model where I designate one machine as the “server” that holds the repository and everything else was a client. Instead, GIT seems to operate more like a merge tool and every local copy is its own “master.” Once I understood that, it turns out that setting up GIT is very simple and just needs GIT itself and SSH.

Let’s start with 2 machines. The first one, “remote”, is where you want to store the code and could be where other users will also pull copy of the code from. The second one, “local”, is the machine where you want to check out the code and make your edits.

On “remote” we want to store the repository in /usr/local/src:

cd /usr/local/src
git –bare init
 

On “local” we want to work on the code in ~/workspace and we already have some files to populate the repository with so we first create a local git repository:

cd ~/workspace
git init
git add .
git status
git commit
 

Now let’s push this code to the remote server:

git push ssh://<username>@<hostname>/usr/local/src
 

Done! Now we have both servers with the files. Files can be edited locally, commit locally and periodically synced/pushed to the remote server.

Now, on another computer, we want to “check out” the source code to work on it:

mkdir ~/workspace
cd ~/workspace
git clone ssh://<username>@<name or remote>/usr/local/src
 

Some things to make it easier to work with:

1. Use ssh keys so that you don’t have to put in your password each time you push to the remote host.

Posted in Linux, Programming | No Comments »

Chrome OSX with Bookmark Manager

Posted by hsin on 15th January 2010

The latest dev channel build of Google Chrome now have a basic bookmark manager and cookie manager.

With a bookmark manager, even though it is very basic, Chrome for OS X has the basic set of features that I feel makes up a browser and might can make Chrome my default browser on OS X soon.

Posted in Google, OSX, Software | No Comments »

2009 OS of the Year… LINUX!

Posted by hsin on 1st January 2010

Looking back on my computer usage this past year, I realized that I’ve stopped using Windows. For the first time, I don’t even have a Windows computer at work. Having mostly used DOS/Windows (usually along side a linux server) for so long I didn’t expect that my usage to stop so quickly, but before I knew it I was doing everything I needed to do on another OS. One of the key changes that I made that contributed to being able to stop using Windows is that I stopped PC gaming. However, I would have expected that OSX would have been the primary OS following Windows (especially since I replaced my main Windows machines with a MBP), but the work horse OS that I used the most turned out to be Linux. Where consumer apps on Linux used to lag behind Windows and OSX, developers are now building them for Linux including media players (VLC) and programs like Picasa and Handbrake with the arcane interfaces that are often associated with the Unix world. I believe having these types of user accessible applications will drive Linux more then just having windows/osx-like window managers.

Posted in Linux, OSX, Software, Windows | No Comments »

Restoring RPM and YUM on Fedora after an accidental yum remove rpm

Posted by hsin on 28th December 2009

By default there is nothing that prevents an user from accidentally removing both yum and rpm from a Fedora system and once removed it is very painful to try to put them back. If only yum was removed it would be a little easier to restore. Yum can be reinstalled through rpm using the yum-*.rpm package downloaded from Fedora or the install disc. Without RPM, it’s like a chicken-and-the-egg problem. The packages from Fedora and install discs (including the rpm package itself) are .rpm files and needs rpm to install.

Today I found myself in the situation where both YUM and RPM got removed by accident. I was following Fedora’s instructions on finding and removing orphaned packages after upgrading and had yum remove some Fedora 11 packages that was left over. Being distracted by other things at the time, I noticed too late that removing those F11 orphaned packages also led yum to remove a bunch of F12 packages including rpm and yum.

So, what does one do in this situation besides “backup and reinstall? which I wanted to avoid if at all possible. It took me awhile to figure out but I was get rpm back on my system and restore it to its earlier state. Here are the steps I took.

My first inclination was to try to compile RPM. However, this was a FAIL since it required additional packages that I didn’t have installed and I didn’t have a package management system… Instead, I downloaded the rpm-*.rpm package from Fedora and was hoping to extract the files so I can use them to reinstall rpm and yum. To extract files out of an RPM package requires… rpm! Oh, the irony. There is a rpm2cpio program that will do that but that is available in . . . the rpm-build-*.rpm package! Fortunately, there are a lot of scripts available that will do it including some shell scripts. The Fedora RPM docs included some scripts and I downloaded the bash script version but it didn’t work. After a little more digging, I found out that Fedora changed the compression scheme it uses in F12 from gzip to xz, so I modified the shell script as follows:

#!/bin/sh

pkg=$1
if [ "$pkg" = "" -o ! -e "$pkg" ]; then
   echo "no package supplied" 1>&2
   exit 1
fi
leadsize=96
o=`expr $leadsize + 8`
set `od -j $o -N 8 -t u1 $pkg`
il=`expr 256 \* \( 256 \* \( 256 \* $2 + $3 \) + $4 \) + $5`
dl=`expr 256 \* \( 256 \* \( 256 \* $6 + $7 \) + $8 \) + $9`
# echo "sig il: $il dl: $dl"
sigsize=`expr 8 + 16 \* $il + $dl`
o=`expr $o + $sigsize + \( 8 – \( $sigsize \% 8 \) \) \% 8 + 8`
set `od -j $o -N 8 -t u1 $pkg`
il=`expr 256 \* \( 256 \* \( 256 \* $2 + $3 \) + $4 \) + $5`
dl=`expr 256 \* \( 256 \* \( 256 \* $6 + $7 \) + $8 \) + $9`
# echo "hdr il: $il dl: $dl"
hdrsize=`expr 8 + 16 \* $il + $dl`
o=`expr $o + $hdrsize`
dd if=$pkg ibs=$o skip=1 2>/dev/null | xz -d
 

With this script, I extracted the rpm (rpm, rpm-build, rpm-libs, rpm-devel, rpm-python) files to a temp directory and install the files manually:

rpm2cpio.sh rpm-*.rpm | cpio -d -i
tar cf – ./usr ./etc ./bin ./var | (cd /; tar xvf -)
 

This installs the files onto the system but doesn’t update the rpm databases, so I did a ‘rpm -ivh rpm*.rpm’ in addition to the dependent packages they needed.

With RPM installed, I was able to install YUM . I had kept a list of all the files that got removed earlier so I had yum reinstall all those packages and my system is now restored.

Posted in Linux | No Comments »

Upgrading to Fedora 12 (part 2)

Posted by hsin on 27th December 2009

It took awhile but Fedora upgraded itself and booted in to the GUI without a hitch. Everything seems to be running even with both the LCD TV and monitor hooked up (past upgrades would forget which was the primary display). Then I noticed that there was no audio. My first thought was that somehow Fedora 12 didn’t configure the audio correctly (uh oh) which wouldn’t be the first time that a piece of hardware worked in a previous version but not the upgraded version. Fortunately, I thought to just double check the volume setting and noticed that the upgrade set the audio-out to MUTE for some reason. Unchecking that re-enabled the sound.

Posted in Linux | No Comments »

Upgrading to Fedora 12 (part 1)

Posted by hsin on 27th December 2009

I decided to take the time off from the holidays to upgrade from Fedora 11 to Fedora 12. My past few upgrades of Fedora have all hit some sort of snag some of which I felt were real obstacles that the typical computer user would find it very difficult to overcome.

For this upgrade, I was ambitious and decided to try the live upgrade feature that was introduced in Fedora 10. By running “preupgrade”, Fedora will download all the packages needed to upgrade to the new version in the background so the system can still be used. When everything is downloaded, the system will reboot, install the packages and then the system will be updated. That’s the theory at least. This method of upgrading should be faster since it only downloads the packages needed instead of every package. Given my complaint from the last post where I downloaded the entire DVD iso then having to download nearly another DVD worth of packages on the first ‘yum update’, this method would address that since it will bypass the first part.

The instructions for using using Preupgrade is on the Fedora wiki. Essentially, it is suppose to just be running the command ‘preupgrade’ and everything else is taken care of automatically. That was kind of accurate. I ran the command and immediately ran into the most common bug with F12 upgrade. I used method 1 to work around it, but I didn’t have any extra kernels so it was the tunefs that got me the extra space I needed. Seriously does the Fedora team really think that consumers would be comfortable having to do this in order to upgrade an OS? What Fedora did very well is that it warned me about the problem and none of it effected the current OS and as it promised, I was able to continue to use the system.

Okay, with that out of the way, it completed the download and it was now time to reboot. This is the stage where the system is not able to be used as it install the downloaded packages. However, at reboot, the system just hanged with the monitor showing “GRUB” and a blinking cursor… The upgrade corrupted GRUB and in order to fix it would require re-installing GRUB. The existing linux system is still there unharmed, the boot process just never made it that far. The irony of this bug is that “preupgrade” was meant to avoid having to create a bootable Fedora disc, but a bootable disc is needed to reinstall GRUB. I dug out my Fedora 11 install disc and booted into rescue mode. The following will install GRUB back to the system:

chroot /mnt/sysimage
grub-install /dev/sda  # change /dev/sda to the drive you have your /boot
 

Rebooting after this, the system booted back into Fedora 11…?!? I guess I was able to confirm that so far my F11 instance is still completed unharmed, but this is suppose to be an automated upgrade to F12. I looked at the grub.conf file and it was suppose to give me the option to do a F12 Upgrade. So I rebooted again and made sure that I selected that option in GRUB before it booted back to F11.

Now the system is installing the package so let’s see how it goes. I took a bigger chance this time by keeping both my LCD TV and USB drive connected. In the past, I already had to disconnection in order to upgrade successfully, so let’s see what happen this time.

Posted in Linux | No Comments »

Upgraded to Thunderbird 3

Posted by hsin on 22nd December 2009

With most of the attention on the Google Chrome browser release for OSX and Linux, the release of Thunderbird 3 got overshadowed. While some feel that stand-alone email readers are becoming obsolete, I still believe that a dedicate email client is very useful especially for power users. That said, the convenience and advancements of browser-based email such as GMail have come such a long way that we’re not far from the time (if we’re not already there) when email clients compliments the web version rather then the other way around.

Upgrading from TB2 to TB3 was painless. Just install the application and it handled all the upgrades. It asks whether you want to switch to using some of the new interface features or stick with the previous ones and then it will start to create the search index for all your email. Search was definitely one of the weaknesses of previous TBs especially when you compare it to Gmail’s search, but that seems to be one of the areas that TB3 really tries to improve on in this release. I think TB3 also took the right steps in keeping itself relevant by integrating better with web-based mail system. The one shortcoming is that it still doesn’t integrate directly with online contacts that you might have with Yahoo or Google. TB3′s interface is also a little more Firefox-like. Desktop clients have some advantages of running on the OS so I still feel that they should continue to use that advantage. It isn’t a radical change, though, so it might be that good balance between the bringing users some familiarity with browsers while still exposing email features.

I need to play with it more but so far I tip my hat to the team for this nice release.

Posted in Software | No Comments »

One script with different script names.

Posted by hsin on 29th November 2009

Sometimes I want to have my bash script behave differently based on how I call it without having to set up different parameter inputs. For example, I might have a script that can sync a file from a local server to one of two remote servers. I can make my script to take in parameters so I can call it using ‘syncfiles -h serverA; syncfiles -h serverB’, but sometimes I forget what my params are and it’s easier to remember ‘syncToA’ or ‘syncToB’.

To do this is pretty simple, yet I always have to look it up each time so I’m writing this as note to myself.

#!/bin/sh

echo $0

if echo $0 | grep "syncToA" > /dev/null
then
   # do stuff
elif echo$0 | grep "syncToB" > /dev/null
then
   # do stuff
fi
 

Next, create symlinks to the original source files with the names you chose (i.e. ‘symlink syncToA syncfiles’).

Posted in Linux, OSX, Programming | No Comments »

GNU Screen and Bash

Posted by hsin on 29th November 2009

As pretty as GUIs are, sometimes it is most efficient to work in a text console which is why I find myself using GNU Screen. However, Screen’s command starts with CTRL-A which is also Bash’s move to start of line command. To have it work, use ‘ctrl-a a’.

Also, screen sets the value of TERM to be screen so if you have any settings based on the TERM value, update it appropriately.

One thing about using screen is that you lose your terminal’s scroll buffer so as line scroll past your view you can’t just use the scrollbar to move up to see what you missed. Instead, screen has it’s own scroll buffer that you can use. You can set the buffer size using ‘defscrollback #’ (where # is the number of lines) in your .screenrc or in the screen command line (ctrl-a). You can switch to the buffer with ‘ctrl-a [‘ (and exit buffer mode with ESC). Navigation follows normal VI keys.

Finally, I noticed that hitting the delete key didn’t send backspace as expected with using the OSX terminal. To correct that, add the following to your .screenrc:

termcapinfo xterm-color kD=\E[3~
 

This tip came from here.

Posted in Linux, OSX, Programming, Software | No Comments »