Author: lhl

SXSW Music 2011 Playlist

Posted on by lhl

I haven’t been very diligent listening to the SXSW torrents this year, but my schedule is already plenty packed: my current SCHED. (My SXSWi is much less structured; primarily planning on catching up with old friends).

Anyway, before things get officially crazy this week (I head into Austin on Thursday), I made a little share a quick and dirty playlist of some of the bands I’m most excited about catching live:

Bonus Track: Not playing at SXSW, but I just stumbled on a great track that Quantic spun at his set last year:
Nicolas Jaar – Mi Mujer (Wolf + Lamb 2010) by Clown and Sunset

New Code: autotunnel

Posted on by lhl

It’s been years since I wrote my original post on using SSH tunnels for SOCKS proxying, but since I’ve been meaning to dig into launchd more, I figured it was time for an upgrade.

Way too many hours later, I present autotunnel. It’s a couple configs/scripts that work together to automatically (re)create your SSH tunnels when you’re set to a Network Location that requests a SOCKS proxy. It’s now a one step process (change your Network Location in the Apple menu) to get secure.

(I also found an app that’s quite similar, sheepsafe. It’s a Ruby-based daemon that does automatic location switching based on “trusted” networks. It switches post-network connection (WatchPaths on SystemConfiguration) but I don’t know if that’s a practical concern or not.)

Old Code: pystatsd-flickr

Posted on by lhl

I was digging out some old code and realized that it might be useful for some, so threw it on github. Nothing special, just a lightweight python port of some of the old Flickr statsd that I wrote… almost two years ago. (time flies!)

If you’re looking to run something fancier in production, it looks like Etsy has some great stuff going on (see also).

Anyway, if you just want a quick graph and/or don’t want to setup Graphite, the code I’ve put up should get you generating an RRD pretty quickly.

Installing Windows 7 on a Mac Pro via USB Key

Posted on by lhl

First of all, holy shit this was a miserable experience. The tl;dr version is that on a 2008 Mac Pro (2 x 3.2GHz Quad-core) running a freshly installed OS X 10.6.6 with rEFIt 0.14 I couldn’t successfully install a Windows 7 x64 ISO from USB.

To successfully install Windows 7, I ended up driving out to buy some DVD-Rs and burning the ISO in a Win7 VM (VMWare Fusion 3.1.12) installed from the ISO via Boot Camp. If you are trying to install on a machine without a DVD drive, I suggest either simply using a VM or if you must Boot Camp, acquiring an external drive.

Additional caveats: once I was able to boot was that the Windows 7 installed did not recognize the BOOTCAMP NTFS partition, and the after formatting, threw an apparently common Error 0x80300024 on install (the solution for this was to shut down and pull out my 3 other drives out so the Windows installer didn’t get confused).

At the end of the ordeal, I ended up with a working Windows 7 install that used 28.4GB of space. o_O

Disabling components actually only increased usage. There is a tool called DISM (must be run as admin) that can manage packages, but appears to treat almost the entire Windows 7 install as one big package. The only way to strip Windows 7 down seems to be using a tool like RT Seven Lite to create an ISO that removes the packages entirely beforehand.

Note: if you need to zap the BOOTCAMP volume during the Windows 7 install, you’ll need to grab the Boot Camp drivers from the Snow Leopard CD or from BitTorrent (Apple doesn’t offer 3.0 drivers standalone, and 3.1 and 3.2 require 3.0 to be installed) and run those manually. (You may need to do this anyway as apparently Boot Camp 3.0 may fail by trying to installing the x86 instead of the x64 drivers).

In case you didn’t get it by now, this experience has been a pretty convincing argument for continuing to stay away from Windows as much as possible, certainly on Macs where the unholiness combination of EFI, Boot Camp, and Windows just makes a complete mess.

For your edification/amusement/horror, additional things that did not work on my Mac when trying to install from USB:

dd
The recommendations to use dd if=$iso of=/dev/$usbkey -bs=1m is a total fail as it’ll create a UDF image on the USB key, which apparently the Mac’s EFI won’t detect
Windows 7 USB/DVD Download Tool
I had an older Windows XP partition, so I was able to try Microsoft’s ISO tool. This created an image that was bootable in the sense that rEFIt could see and select it, but subsequently did not show up as a boot device (DOS prompt for inserting a bootable disk)
UNetbootin
Another try once I started down the WinXP path, I tried using UNetbootin to make the USB stick bootable, but to no more success. Same result as the Microsoft ISO tool.
WinToFlash
Next into my increasingly desperate descent into madness was to use an app that could write Vista/7 boot procedures using EFI. Unfortunately, these resulted in an “Error: Unsupported while loading bootmgr.efi” message when booting in rEFIt
Windows 7 in VMWare Fusion + DISKPART
Finally, I decided to install the ISO in VMWare Fusion on OS X, and then follow this method with DISKPART (you can’t do it in XP because it’s version of DISKPART doesn’t recognize removeable drives as disks). This also failed spectacularly (also, trying bootsect), at which point I threw in the towel on USB booting.

UPDATE: Here’s a method of getting BootCamp Assistant to create a bootable USB Windows installer that might work, although I haven’t tested it at all and it’s probably also dependent on how old your Mac is if it does work…

UPDATE 2: On a recent PC, I was able to make a bootable install disk via Linux which may be helpful, but may not be applicable w/ old Macs w/ 32-bit UEFI.

Personal Data Storage 2011

Posted on by lhl

The last time I spent much time on personal data storage was over 2 years ago. Since then, all my NASes and most of my disks have actually been themselves in storage.

While I’ve been nomadic, I’ve mostly been haphazardly filling up 2TB 3.5″ external disks and various 2.5″ portable disks. Lots of these are dupes however, so now that I’m a bit more settled, I’ll be trying to bring these in line.

To that end, I’m building a new storage filer, and potentially will build a second as a backup if it works well. I’ve decided to take the ZFS plunge so I’ll probably be running FreeNAS which has ZFS support and is USB stick friendly. Here’s what I picked up last night:

  • Patroit Xporter XT Boost 16GB Flash Drive – the 8GB is potentially faster, but price/MB is better on the 16GB. (the XT Rage is a bit faster than the Boost, but the casing can block other USB ports). There may be a 5th SATA port I can use with the included HD, but I’ll probably pull that and boot off of USB anyway, save some power.
  • HP ProLiant N36L MicroServer – this seems to be the best solution right now for a build-your-own NAS. It’s <$400, it's much faster than an Atom solution with about the same power consumption; supports 4 drives, sadly no hotswap. It’s actually about $50 cheaper at Newegg, but if you’re in CA, tax and shipping make it a wash w/ Amazon
  • 4 x WD 2TB Caviar Green WD20EARS drives – these drives are cool, quiet, and fast enough. I’ve been using them for years and haven’t had any problems. There was just a crazy $20/drive rebate that apparently I just missed, but jeez, $85 for 2TB? Just remind yourself how much that would have cost even 5 years ago.
  • 4GB 1333MHz DDR3 Non-ECC CL9 DIMM (Kit of 2) – HP thoughtfully includes 1GB of RAM with their MicroServer. Like the 160/250GB hard drive… it’s marginally useful. The server maxes out at 8GB, but 4GB for $44 is a pretty reasonable price. ZFS apparently really likes memory for cache, but lets get real. We’re not doing much mission critical work here.

I’ll post an update once everything’s up and running, including final kill-a-watt numbers.

Cloning Macs via the Command Line

Posted on by lhl

Recently, I’ve been doing a lot of cloning/imaging of machines. At a certain point, Carbon Copy Cloner and Disk Utility (or any GUI tool), doesn’t cut it. CCC of course started out as a wrapper for asr and there are guides for using asr for disk imaging online.

I’ve written a slick set of scripts that are streamlined for my purposes (automatically finding the mountpoints for sources and targets etc), but there’s probably some general stuff worth sharing…

Before cloning, I have a script that ‘preps’ the machine. This includes running periodic [daily|weekly|monthly] and clearing the caches:

sudo rm -rf /Library/Caches/*
sudo rm -rf /System/Library/Caches/*
sudo rm -rf /private/var/root/Library/Caches/*
sudo rm -rf ~/Library/Caches/*

Making the “master” image is pretty straightforward:

diskutil unmount "$volume"
time hdiutil create "$dmg" -format UDZO -nocrossdev -srcdevice "$device"
time asr imagescan --source "$dmg"

(I use a combination of time and overall timing recorders with date +%s epoch math to get proper performance numbers)

And cloning is even simpler:

time asr restore --source "$1" --target "$device" --erase --noprompt

Using --erase is essential to doing block-level writes and is a magnitude faster and more reliable.

Lastly, and what probably makes this writeup somewhat useful, I have some additional code that renames the hard drive and the machine name:

diskutil rename "$device" "$newname"
perl -p -i -e "s/$oldname/$newname/" "/Volumes/$newname/Library/Preferences/SystemConfiguration/preferences.plist"

The last step is preferable to using scutil because you can do it without rebooting, and it’ll already be renamed before there are any conflicts jumping on the network (which leads to the dreaded “(2)” problem).

With these scripts, I build a fully production ready clone in about 10 minutes flat.

Missing at this point:

  • Screensaver preferences don’t clone for some reason, I probably need to find a similar plist setting and rewrite that
  • I want a keyboard that just has a “T” key.
  • I probably should build a launchd script to run a “first bootup” script for anything else that needs to happen.

Making sure WordPress is up-to-date

Posted on by lhl

For anyone that has to manage many (any?) WordPress instances, keeping it up to date can be a real PITA. (I don’t expect WP to ever have an SSH vs FTP based auto-update).

The best way to make life a little easier is to move over to using svn tagged versions. Then you can simply switch over the next time a security vulnerability is patched with a simple ‘svn’ switch.

Here’s some simple bash-ness that’ll help you know when to upgrade. In *theory* you could run this on cron and have it auto-update, but I’m assuming that if there’s a DB schema update required, you’ll need to watch over it, so probably not something you want to run on production without some verification:


installed_version=`svn info $yourwp | grep URL | cut -f 2 -d' '`
current_version=`/usr/bin/lynx --dump --nonumbers http://core.svn.wordpress.org/tags/ | tail -n 2 | head -n 1`

if [ installed_version != current_version ]; then
  # this is where you could get fancy and do an svn switch and update
  echo `date` | mail -s "WP is out of date!" yourmail@example.com
fi

Checkvist is my Task List

Posted on by lhl

Like most geeks, over the years I’ve gone through a lot of a lot of task management tools (most successfully, OmniOutliner and TVO), only to go back to using text files (which unfortunately, tended to accumulate on multiple computers at multiple locations, I still have a few megs of files tucked all over the place). Lately I’ve been using a lot of Evernote, and as one of the few things that synced flawlessly (although Dropbox has really changed the game there), I used that for about a year or so as my main task “manager”, and used FogBugz for development-related stuff.

After the latest round being really unhappy w/ my setup, I took another look around what kind of online/shared options were available and stumbled onto Checkvist early last year. Although there are others that do more (Toodledo, and of course, Remember the Milk), Checkvist was the one that stuck the best, mostly because of its simplicity and speed. Also, probably the biggest factor (and a big contributing factor to the speed aspect) is that it can be almost entirely keyboard driven. It’s a moded editor with tons of chained shortcut keys. You can see the appeal to a longtime vim user. It also has decent multi-user sharing baked in from the start, which is nice for small teams (although there are a few things missing that have been stumbling blocks for adoption by my co-workers).

That being said, despite a few things that originally seemed like dealbreakers (lack of mobile app, for one), over the past few months, Checkvist has managed to take over as my primarily task manager. I attribute it primarily to the parts that it gets right: keyboard driven UI, search, list switching, expansion among them, that are just unmatched in anything else I’ve tried.

Here, BTW is my ongoing Checkvist Evaluation/Improvement List:

Checkvist’s closest peers are probably Todoist, which has a few nice touches, but seems to be inferior to Checkvist in just about every way, and Workflowy, which is conceptually very interesting (dynamic scoping/zooming, keyboard everything, and while slicker, is too limited to be really useful for me (even less metadata, no sharing). Also, I suspect that the lack of modalness actually makes the keyboard nav a lot more complex than it otherwise would need to be…

My Mac Apps

Posted on by lhl

NOTE: I’m keeping a more up-to-date (and prettier version) of this list on Bagcheck.

One thing I’ve noticed having done a couple of reinstalls lately is that I’m using a lot less apps on my computer. I figure now (that I’m procrastinating) is as good a time as any other to write things down in case anyone finds it useful:

  • Google Chrome, Firefox 4 and Plainview
  • SIMBL and TerminalColours – pretty much the first thing I install, a must for seeting what you’re doing on a dark terminal background (I bump up green, blue, red, magenta and cyan). I’ve linked to the Network graph for TerminalColours, as the head seems to be ever-shifting
  • XCode
  • OpenTerminal – incredibly useful; I change the prefs to always open a new window and to open the selected folder
  • ClipMenu – the best clipboard history app I’ve found. I don’t use it’s snippets much, just increase the buffer to 50, increase the autosave frequency and show the last 10 inline
  • MacPorts – while others might have moved to Homebrew, MacPorts hasn’t failed me yet and I’ve invested the time to learning variants, upgrades, cleaning and activations. As long as it keeps working, I’ll keep using it. The first thing I’ll usually do is install python27 and python_select to python27. I finally learned my lesson not to use Apple’s built-ins. I also use MacPorts’ vim (+python27), curl (+ssl), and git-core.
  • Thunderbird – one day my Mail.app stopped working w/ GMail’s IMAP. I switching to browser-only, but that didn’t work well, especially w/ 5 GMail/GApps accounts. I’ve switched before, but after a year w/ Thunderbird 3, it looks like it’s taken for good. I have a few useful extensions, but by far the most useful is keyconfig, which I primarily to map the ‘y’ key to archive. I mostly use the widescreen view w/ Group By reverse chronological sorting in the unified inbox.

    • Alfred – I finally switched off of Quicksilver a few months ago. Now, there’s still some development going on, and it mostly works, but it felt like the right time to move to something a bit more vital. After trying Quick Search Box, I settled on Alfred mostly because I wanted a quick way to access individual preference panes. My launcher needs have always been pretty basic, I just wanted something that works and doesn’t crash much.

  • iStats Menu $ – For a long time I ran MenuMeters, which is great and does everything that I want. I mostly switched because iStats Menu has a nicer calendar than MagiCal, on par with the features I used from MenuCalendarClock (I can’t recommend the latter, as I registered, and shortly after a new version came out that required re-registering). Paying another $20 was a bit much for me.
  • Adium – the only feature I wish it had was cloud-based log syncing… hmm, I wonder if saving to Dropbox is a good option… I’ve use the Domo-kun notification icon for years, although the Totoro is pretty tempting. I make a lot of customizations to my Adium, but by far the most importan are deleting Growl notifications, turning off sounds, disabling the menu icon, and hiding messages and lists when backgrounded – basically, anything that would actually interrupt my work (I have my dock hidden by default, otherwise I would disable that stuff as well).
  • VLC – must have for MKVs and the like (I’ve used Perian in the past, just couldn’t be bothered to reinstall it)
  • Cog – I loathe iTunes, and Cog is the best lightweight music player I’ve found. The official release hasn’t been updated in a while. I use the latest nightly without problems. Vox is a slicker alternative, but doesn’t have playlist management or scrobbling (it does!). (Cog does via the Last.fm Scrobbler)
  • DTerm – I’ve yet to really start using it, but the copy results to clipboard seems really nice.
  • Alarm Clock – I found this a few years ago, and it does everything I want/need in an alarm clock.
  • Dropbox $ – I pay for the Pro account and use it everywhere. Here’s an interesting Quora question on why Dropbox is so much more popular than competitors. The simple answer is that it just works.
  • Carbon Copy Cloner – Its read-only image writing seems to work much better than SuperDuper‘s for some reason. With it, I can image a machine (11GB uncompressed) in about 5 minutes.
  • TeamViewer
  • 1password $ – this works I think as well as can be expected. LastPass is probably its only real competition. One slick thing is that if you’re syncing with Dropbox, the 1password’s license file will be automatically synced.
  • Evernote $ – It’s not perfect, but it does that one thing well that no one else did (sync!) With 2.0 (folders!), I find myself less angry at it now at least.
  • Gabble – a far better Yammer app than the AIR one
  • Sequel Pro – as much as I live in mysqlclient, Sequel Pro does some really convenient things (duplicating rows for example) and is a lot faster than MySQL Workbench.
  • TinkerTool – I only use this app to move the Dock (I run mine hidden on the top left side w/ only running apps), although I include it in the list because it’s does other neat stuff as well.

OK, not the shortest list, but compared to the number of apps I had installed before…