For the first time in a long time, I spent my weekend not working on something directly related to my job. I decided that it’s time that I really got myself to learn VI. I’ve been using the basics of VI forever and when it came to serious code editing in a terminal, I tend to fall back to EMACS. In a GUI environment, my favorite editor is Visual Slickedit but right now I don’t have it for OSX and our servers don’t have it or EMACS installed.

So the first thing I did was try to make VIM a comfortable environment for me to work in. The default black text on white background didn’t work for me so I changed the terminal to black-on-white. Of course, this re-introduced another of those annoyance that always got under my skin. Who the hell chose a dark blue font color for directories?!? Who can actually read that without going blind after 2 minutes? So, of course, that has to change… Out comes the editor and changing the LS_COLORS environment variable…. but wait… OSX doesn’t use that name. It uses LSCOLORS instead… Those wacky BSD guys. Okay, no problem. Let’s see export LSCOLORS=’di=…’…
Uh, wait, that doesn’t work ’cause that just makes things too easy to understand. Instead, how about:

export LSCOLORS=’fxFxcxdxbxegedabagacad’

Uh…yeah… that’s intuitive. Assembly programmers, I respect. Whoever came up with this is an idiot.

My linux version is a bit more customized:

export LS_COLORS="no=00:fi=00:di=36:ln=01;36:pi=40;33:so=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=01;32:*.cmd=01;32:*.exe=01;32:*.com=
01;32:*.btm=01;32:*.bat=01;32:*.sh=01;32:*.csh=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;31:
*.bz2=01;31:*.bz=01;31:*.tz=01;31:*.rpm=01;31:*.cpio=01;31:*.jpg=01;35:*.gif=01;35:*.bmp=01;35:*.xbm=01;35:*.xpm=01;35:*.png=01;35:*.tif=01;35:"

Okay, now it was time to get to know VIM and all its goodies such code folding, color syntax, debugging, intellisense (or it’s new name: omnicomplete), etc. Being the lazy hacker that I am, I first looked around to see what other people already did so I can borrow their stuff. I came across Andrei Zmievski’s presentation and he included his VIM scripts which did pretty much everything I wanted. Sweet!

So, putting on my RHEL5 workstation and OSX machine had no problems. The problem is that I’m also doing a lot of work on RHEL4 machine which only has VIM 6.3 and a lot of the plug-ins don’t work. I had to download the source from vim.org and compile it myself. Since I wanted the ability to work with xdebug, I had to tell the build to include those features. After compressing the source, you can run:

./configure –help

to see all the different options. Basically, I needed to do this:

./configure –enable-pythoninterp –with-python-config-dir=/usr/lib/python2.3/config

(Look at the output from config to make sure it found the python config. If not, download and install the python_dev package.)

Then run make and it compiled.

In the end, it was fun to learn something new. I learned a lot more about VIM such as color schemes, plug-ins, etc., and now I have a comfortable environment to work in even without EMACS.

cat filestodelete.txt | while read line; do rm ${line}; done

for s in `cat server.list`; do ssh $s uptime; done;

Silicon Valley is very much about coming up with new things and creating new markets often through technology. This is a different challenge then trying to break into a existing market through an evolution of market. One of the things that the makers must demonstrate is that the new technology can work and demonstrate it’s usefulness. To do that means the 1.0 release must be stable and “just works”.

I often see teams trying to build the “perfect” product with the “perfect” interface as part of version 1.0. Engineering attention is taken away from the product technology and stability to focus on the UI. Everyone wants to have the best product out there, but in a new market nobody knows what “perfect” or “best” means since nobody has experience with it. It’s just better to make sure that what is built is built solidly and be flexible about making user experience improvements iteratively (which is easier to do in software then hardware).

Personally, I believe that User Experience and Design (UED) plays an important in product development. As Apple has shown, a good design can help take a product to the next level. However, I also feel that too much focus on UED can sometimes hurt innovation (non-UED type innovation) of a new product type. Apple didn’t invent the mp3 player. It allowed the mp3 market to develop and once the world has come to accept the need for mp3 players, Apple build a product with a design and interface that dazzled and became the leader.

So folks, if you’re trying to enter an existing market, take the time to examine the competition, don’t make the same mistake and improve on what is out there. If you’re trying to create a new market, put your resources into making the first launch stable and useful so you can grow the market.

I’m continuing to get more familiar with working with a Macbook Pro and am starting to customize it to suit my working style. The first thing was to get some parity with what I was used to in Windows. At work, I have two machines (Macbook Pro notebook and a Linux workstation) and two monitors (24″ LCD and 17″ LCD). I previously had a Windows notebook instead of the Macbook and had configured my system to be a three monitor system. The Windows notebook was on the far left and connected into a docking station. I used the notebook’s LCD for Thunderbird, IM buddies, etc.

To the right of the notebook, I had the large monitor which is the extension of the Windows desktop and what I’m staring at most of the day (when I’m at my desk that is). Next to large monitor is the 17″ monitor that is connected to my Linux work station.

There is only a single keyboard and mouse, and both are connected to the Linux workstation. By running a VNC Server on the Windows machine and a program call x2vnc on the Linux machine, I can move the mouse/keyboard cursor across all three monitors as if they were one and even copy-and-paste between UNIX-Windows apps. The only limitation is that Linux apps can’t be moved off the the 17″ screen and Windows apps can’t be on the 17″ screen, but effectively my one keyboard and mouse controls both computers.

Being so used to VNC, I tried to do the same thing with OSX. Imagine the above system except with the Macbook instead of the Windows notebook. At first I tried using Apple’s Remote Desktop which is suppose to be basically a VNC Server. The problem is that x2vnc or any other vnc client I tried could connect to OSX. I then downloaded a VNC Server for OSX, but that didn’t recognize the dual monitor display of the notebook+external monitor.

Fortunately, the solution was the open-source Synergy tool which does much of the same thing as x2vnc and has both a server and client for Linux and OSX. Basically, on the Linux workstation, I run the Synergy “server”. On the Macbook, I run the Synergy “client” and now I can do what exactly the same thing as before with one mouse and keyboard.

Installing Synergy was a snap. On Linux, use whatever your app management tool might be (i.e. “yum install synergy”) and on OSX use Macports (i.e. sudo port install synergy). The Synergy site has instruction on what the configuration file should be and then it’s just a matter of running the programs on each machine.

With my physical environment ready, I started to tackle trying to get a good ToDo app for OSX. For me, a todo utility must be easy to access. I’m too lazy to have to move my hands from the keyboard to the mouse. ^^; I also want the data to be presented simply and non-obtrusively. It’s surprisingly hard to find this combo. Even on Windows, I didn’t have a great solution: I used Google Desktop’s todo widget which required that I use the mouse to select the widget before I can type into it, but the look was simple and basic.

I tried a few Mac Todo lists and was going to go with DoIt since it had Quicksilver integration that allowed me to write items to the list with keyboard strokes… almost. The problem was that DoIt also requires you select a category from a drop down list before it adds the todo item, and for the life of me I couldn’t figure out how to select the category without using the mouse.

I finally came across a mixed low-tech/high-tech solution. Quicksilver has an action called “Append to text file”. Using a simple text file (i.e. todo.txt), I created a trigger and keyboard shortcut that when hit, let’s me type a todo item that gets appended directly into the text file. To see my changes, I used a very neat little app called Geektool that display a transparent window containing the text output of anything log file, script output or image.

One limitation of this setup was that I used Geektool’s file type to handle the display of the todo file. This only works when the file is appended to at the end like typical log files. This means that if you edit the file with another application, the change doesn’t show up. The solution is to not use “file” but use a Geektool “command” with a refresh timer. By using the command, “curl ” every 30 seconds, it’ll regularly update the window with whatever is in the text file. Not as good as detecting a change automatically, but works well.

I’ve looked at Eclipse on-and-off since the very beginning, but I was never compelled to use it for a variety of reasons. The last time I made a attempt to use Eclipse was a couple of years ago and it wouldn’t start up for me for reasons I was never able to determine, so I just left it at that.

I decided to take another look at Eclipse again because 1) I’ve been hearing a lot of good things about it so I figure a lot of issues have been resolved, 2) I don’t have any software for the Mac so the free open-source nature of Eclipse is a plus, and 3) I’m thinking I’m going to be doing some Java coding.

I went to the Eclipse site and downloaded the the J2EE package. The different packages just means they come with different sets of plug-ins. You can always download the most basic package and install each plug-in your want manually. I didn’t want the hassle of doing something and then finding out in the middle of it that I was missing a plug-in so I grabbed the full J2EE version.

Installing and running it was as simple of uncompressing the archive and clicking on the Eclipse icon. While Java has made a lot of progress addressing performance issues especially on the server side, I wanted to see how it does on a desktop application that is stopped and started frequently. While I didn’t do any real benchmarking, the IDE started up faster then I remember it is still no speed demon. Once started, though, the performance seems pretty good.

Eclipse has it’s own set of terminology like perspectives, Team, etc. which takes a bit getting used to, but the features you’d expect in a modern IDE is all there such as code completion, syntax highlighting, code navigation, etc. One thing I really like is that it checks your code for you even as you type. When you open up a project, you can immediately see which directory/file/line has potential errors through the explorer pane. You can do a lot more with templates code assists that will help with some coding tasks.

Before I did too much with Eclipse, I found out about PDT (PHP Developer Tool) plug-in. This adds PHP support to Eclipse and was a project that originated from the makers of the Zend IDE. Since my current projects tends to be PHP-based, it peaked my interests to see if I can use this IDE to handle my projects.

Installing the plug-in is super easy with Eclipse now. Just go to the plug-in manager and add the PDT repository to it’s list and tell it to install. A few clicks later, I had PDT in Eclipse that recognizes PHP files and my project directory. Unfortunately, I haven’t had much more time to get beyond that. I still want to test the debugging tool, etc. but I’ll have to do that later.

I feel like I’m behind at work recently so I didn’t want to start the new year at the office spending time configuring the new notebook. The baby was nice to me tonight and went to bed early so it allowed me to have the chance to get my development environment on the new Macbook Pro with OSX Tiger configured. Thanks to M in Ohio who saved me hours of time by pointing me to the right solution.

The nice thing about the OSX is that it is build on top of UNIX which is a developer’s OS. You’ll find that most open source tools are available and if not then getting it compiled from source is usually possible. I used compile everything from source including the kernel, but now that I am a lazy old man, I tend to prefer using some sort of package management solution that has the dependencies resolved. For Linux, I like the yum package management tool that sits on top of RPM. It beats the hell out of downloading the source and figuring out all the dependencies.

M pointed me to MacPorts as a similar repository/package management solution for OSX open source packages. It’s really easy to use. Simply download the install package from the site, run it and you’re ready to go. I did ran into one bug where it didn’t create a .profile file for me so the path to “port” couldn’t be found. I’m not sure why, but a quick look on the site and I knew which directories to add to my $PATH variable.

export PATH=/opt/local/bin:/opt/local/sbin:$PATH

Don’t forget to “source ~/.profile” to pick up the change.

With MacPorts installed, it was only a matter of grabbing the programs I needed for a web development environment: Apache2, MySQL, PostgreSQL, PHP5.

Installing Apache2:

• sudo port install apache2
• copy /opt/local/apache2/conf/httpd.conf.sample to /opt/local/apache2/conf/httpd.conf
• sudo /opt/local/apache2/bin/apachectl start
• point your browser to http://localhost to see that everything is running.

Installing MySQL and PostgreSQL:

• sudo port install mysql5 +server
  (at the end of the install, it tells you the command you should run to initialize the mysql db)

• sudo port install postgresql82 +server

Installing PHP5 with support for MySQL, PostgreSQL:

• sudo /opt/local/apache2/bin/apachectl stop
• sudo port install php5 +apache2 +mysql5 +postgresql +sqlite +pear
• sudo cp /opt/local/etc/php.ini-dist /opt/local/etc/php.ini
• sudo vi /opt/local/apache2/conf/httpd.conf
  • add “Include conf/extras-conf/*.conf” to /opt/local/apache2/conf/httpd.conf
• sudo /opt/local/apache2/bin/apachectl start

Installing Eclipse IDE:

• Download the IDE from Eclipse.
• unarchive it to whichever location you want to run it from (I put it in /Applications/eclipse).

With that, the an Unix development environment capable of doing web development with PHP, MySQL and/or PostgreSQL as well as Java/C/C++ development is all ready!

For a little more detail about the set up process, I stumbled on this nice post.

One of the things I did when I joined Yahoo was that I put all my Java books into boxes and took them out to the garage. Java wasn’t my favorite language, but I didn’t dislike it either. I can appreciate some of the advantages that it brought to the table and how it simplified some of the grunt work that is required when using another language.

Thus, it’s been a few years since I really did anything serious with Java and I decided it’s about time to refresh myself as well as finding out what all has been changed. The newer things I’ve been hearing are Spring and Hibernate while a lot of the things I was using before like Tomcat are still around.

I picked up the book, “Beginning Pojos” by Brian Sam-Bodden to give me an introduction on some of these topics. It’s been a decent read so far. I think the next thing I study more on is GWT.

Today, I fired up the web browser and typed in ‘http://www.apress.com‘ to check out what new computer titles has come out and I realized that I used to go to O’Reilley instead. In fact, it’s been awhile since I’ve gone to O’Reilley other then to read some of their tech articles. O’Reilley used to be THE publisher of practical technical computer books related to development, UNIX and open source. They published the Perl book among other must have references that were on the desks of every programmer I know.

Now, though, their titles seems to cater to a more mass-market crowd and publish a lot of quick-reference guide. If I want to learn about something, I no longer think about O’Reilley. My last few purchases ( Building and Extending GAIM, The Definitive Guide to GCC, Foundations of GTK+ Development, Bginning PHP and PostgreSQL 8 ) have been from Apress who seems to have replaced O’Reilley in the area that O’Reilley used to dominate. I still have my old O’Reilley books which I reference frequently (Programming Perl, Essential System Administration, UNIX Power Tools, etc.), but now I have to do some research on a new O’Reilley title before I purchase one.

Shard, or sharding, has become the new buzz word lately. The term seemed to have been popularized by Google and made more prominent by Flickr and Digg who credits it to helping them scale up performance. Not being familiar with the term, I started to do some research around the web to try to understand what this new technology is.

My initial investigation made me more confused. Based on description by Digg, it sounded like all they are doing is traditional data partitioning (vertical or horizontal, I forgot) or maybe database clustering to distribute load.

When I look at the comments made by Google they also only spoke about the physical layout of their database, but on closer examination it revealed that they work more with Hybernate’s sharding mechanism, which is a software ORM (object relational mapping) solution, along with their database architecture. ORM is not new either. Essentially, it means to create an object for accessing data that abstract it from the underlying database architecture. What Hybernate takes one step further is to allow multiple data sources and still create one data object to be used.

In the end, it seems like sharding is a new term to describe the use of ORM with database partitioning together.

Here is a tutorial for writing a sliding divs like you see on Digg or many other sites. It was written in response to another tutorial on implementing a similar behavior.

Both articles are worth reading as it spells out clearly how to do the effect. The version on firblitz takes a more object oriented approach. In both examples, the initial call is from an inline Javascript (a href=”javascript:…”). There is nothing wrong with doing this, but to have an even cleaner separation between Javascript and the HTML presentation, it’s best to not have inline Javascript at all. Instead, use the event model to attach the function to the event so that that the HTML is completely clean.

Start with the following addition to firblitz’s javascript code.

function toggle() {
	var sd = document.getElementById("slidediv");
	if (sd.style.display == 'none') {
		Slide('slidediv').down();
	} else {
		Slide('slidediv').up();
	}
}
function init() {
   // attach event listener to objects
   var slidediv = document.getElementById("mydiv");
   slidediv.addEventListener("click", toggle, true);
}
window.onload = init;

And the body of the html would just be:

<div id="mydiv">clickme</div>
<div id="slidediv" style="height100px;display:none;overflow:hidden">Hello world!</div>

To take it one step further, take out the inline styles. However, this introduced one problem. Javascripts is unable to access the style value directly if it is not an inline style. This is because the element’s style is not the style of the CSS object that is assigned through the #id. Thus, you have to manually assign the styles to the element. This is the end result:

function toggle() {
	var sd = document.getElementById("slidediv");
	var styles = getComputedStyle(sd, '');
	sd.style.display = styles.display;
	sd.style.height = styles.height;
	if (sd.style.display == 'none') {
		Slide('slidediv').down();
	} else {
		Slide('slidediv').up();
	}
}
function init() {
   // attach event listener to objects
   var slidediv = document.getElementById("mydiv");
   slidediv.addEventListener("click", toggle, true);
}

And the body of the html is a clean:

<div id="mydiv">clickme</div>
<div id="slidediv">Hello world!</div>