I’ve decided to jot down some notes on some of the data tools installed on my laptop. A friend asked about my setup recently, so this will be a nice description I can refer him too as well.
cygwin – if you don’t know Unix you’re truly disadvantaged if you work with data. I love ‘nix, and can’t live without it.
gawk – you may laugh, but I use awk every day to hack data, it’s lightweight, fast and simple to use. Be sure it is installed with cygwin. Grab xgawk too, which is awk for xml files which makes for a great fast parser as well as doing other very handy things.
puttycyg – if you use putty, you absolutely must install puttcyg, it makes cygwin feel real. Honestly, without this I couldn’t use cygwin.
SQLyog – A terrific user interface for MySql. If you use Oracle, think Toad. Again, this is essential kit.
MySQL v5.0.67 – A great lightweight database I use for hacking data every day, super handy. Essential. Note I use this version for a very specific reason, the explanation for this is below.
RMySQL – R bindings for MySQL. Allows you to pass data back and forth between R & MySQL very quickly, plus you can execute SQL statements and stored procedures from R. Note – you need to be running a supported version of MySQL, like version 5.0.67 which you can get here.
R statistical software – I’m still learning R but finding it extremely powerful as a general purpose data tool. It’s graphical capabilities especially are extremely useful.
RExcel. This is really fantastic. It allows you data to pass between R and Excel very quickly, and to control R from Excel. Using this as a bridge to R, I’ve also been using this to integrate MySql and Excel too, which is awesome.
As an aside, RExcel comes with a (D)Com server for R, so if you’re a com whiz you can use this to integrate your other windows projects with R too.
Graphviz – a declarative graph drawing language and visualization tool. I find this extremely useful to visualize data and to produce architectural drawings.
bytefreq 