Going Mobile with the iPhone

Ever since I got the iPhone sometime last year (okay, I admit, it was June 29th, at 6:25pm), more and more of the tools I use every day have moved to the web.

столовеEmail:

I have migrated virtually all of my email to five separate Gmail accounts (3 of which are Google Apps domains): private/personal (family and friends only), public/personal (mailing lists subscriptions, website registrations, etc.), Alienwebshop business, Broad business and an account just for old domains and addresses that are now 99.9% spam. The move to Gmail was made possible when Google finally released the IMAP feature last year, which made it easy to import messages from all of my old accounts.

The benefits of moving to Gmail (from IMAP servers that I managed myself) have been huge!

First, of course, has been the spam filtering, which is amazingly accurate. Then there’s the search capabilities, which are an enormous improvement over my previous mail client, Apple’s Mail.app, particularly when you consider that I have literally hundreds of thousands of messages going back over 14 years! That, coupled with the custom filters and colored labels makes it really easy to find whatever message(s) you might be looking for with Gmail.

I can still access everything with Mail.app just like I always have, but I hardly ever do except to move messages between accounts. I prefer the rent a car bulgariakeyboard shortcuts that the Gmail interface offers, as well as the search, threading and support for multiple (colored!) labels. Even though I mostly use Mail.app from the phone, I’m getting more and more comfortable with the new Gmail iPhone app, which has several features the built in mail client lacks.

Calendars:

I also started using Google Calendar for all my calendaring needs sometime last year and have totally fallen in love with it!

The best part about it is the sharing. Bailey and I have several shared calendars together. I have calendars for the work softball team and for a music mix club that are shared with everyone involved in each group. I can let some people just see the events (or just when I’m busy) and let other’s actually add or modify events.

With the custom colors you can set for each calendar and the bars in month view for events that last several days, Google Calendar is great to look at. On the iPhone, Google Calendar is much prettier than the built in calendar, though both will let you create new events and neither allows you to modify them.

In order to get all the events into the iPhone calendar I simply subscribe to each of the calendars through Apple’s iCal. Then, all of my Google Calendars are available in iTunes for syncing to the phone. This is a one-way sync from Google to iCal and doesn’t let you create events on the computer or the phone. All events must be created from Google Calendar. To get around this, I created a shared calendar in iCal and subscribed to it from Google Calendar, but I’ve never had to use it.

Todos:

Another important tool that I use every day is a todo list. Unfortunately, neither Apple nor Google offers an application for managing a todo list. Thank goodness for Remember the Milk! RTM is a web-based task list application that employs tags and tag clouds, as well as several great search features and keyboard shortcuts for most of its functions. It also provides offline access with Google Gears.

Integration with Google Calendar adds a little RTM drop-down menu on each day showing which events are due. A mach zehnder modulatorFirefox plugin for integration with Gmail provides an RTM menu to the right of your inbox, making it feel like it’s all one application. Integration with Twitter adds the ability to create new events and receive notifications from any Twitter device. Of course, RTM also provides a fabulous iPhone application, which provides most of the functionality of the standard web app as well as a great look and feel.

More:

The list goes on. I’ve been an avid user of Google Reader as a newsreader for a couple of years, now available in a new iPhone version. I’ve recently begun the process of migrating all of my Word, Excel and PowerPoint documents to Google Docs, also available on the iPhone. I’ve also began managing my finances (checking/credit card accounts) with Wesabe, which again, has a custom app for the iPhone.

I’ve also recently started using a MacBook Air, so it’s nice to be able to access everything from that without having to worry about where that file I was working on is stored or which computer has my most up-to-date Quicken file. The more I take advantage of these online applications, the less work it is for me to manage and the less it matters where I am when I’m trying to be productive.

Before I got the iPhone I hardly did anything with my phone other than make and receive phone calls. Also before the iPhone, most of the applications I used every day ran locally on my computer and that required me to always have it with me to be productive. Now it doesn’t matter where I am. I can be productive from anywhere. I’ve finally gone mobile!

Storage

Storage has always been a big concern of mine. My mother always taught us to keep everything forever, so despite of my repeated efforts to trim down the cruft, I’ve managed to accumulate quite a few things over the years. Once you have a lot of stuff, where to put it becomes the ever-important question. Granted, an ever better question might be whether you really need to keep it or not, but some of us have trouble answering that one. So we stock up on folders and drawers, baskets, bins, boxes and buckets, label it all and pack it away in our shelves, closets, trunks, basements (or perhaps a $147 per month storage unit if your landlord forbids access to the basement) and sheds.

For those that put some thought into it, there’s a science to where everything goes. Simply, it’s based on having the things you need the most frequently or with the most urgency ready at hand (tier 1), and having things that are just nice-to-haves accessible with an amount of effort that is reasonable for however often or under whichever circumstances you want to be able to access them (tiers 2 and 3). For example, you might keep your flashlight in the hallway closet and keep the rest of your camping equipment at the storage unit, knowing that you’ll probably have a lot more advance notice about needing your canteen while the need for the flashlight might just creep up on you, unannounced.

This problem extends into the digital world more and more. As drives get bigger and bigger every year, the volume of our data grows even faster and becomes ever more difficult to manage. For example, last year we got a fancy digital video camera that we’ve managed to use quite a bit, to the tune of nearly 700 movie clips comprising well over 100 gigabytes of raw footage. Because neither of us has enough space on our computers, we have to get external FireWire drives to store the stuff and since hard drives are unreliable and we don’t want to lose everything in some horrible mishap, we have to back it all up. I have burned nearly 30 DVDs of all of our footage from the last year! Not only is that time consuming, but now we have to find some place to keep all those discs. If we store them at home, then both the original and the copy could be lost in the same fire, for example. Storage unit? Ugh.

At the Broad Institute, where I lead the server/storage team, we have this same problem only our problem is several orders of magnitude larger. Much of the work we do involves running genetic material through fancy instruments that apply some chemistry to the samples to generate high-resolution images that are processed and evaluated for the greater good of mankind (curing cancer and the like). Because we’re constantly refining our processes and because some of the samples are in limited supply and can’t be reproduced, we have to keep much of this raw data around forever in case we want to re-process it. Much like my mother, our scientists have a “let’s just keep it forever” perspective on their data.

When I started at the Broad more than four years ago, this storage problem was measured in hundreds of gigabytes and terabytes. As current sequencing technologies are scaled up and new technologies emerge, the volume of data we generate increases exponentially. Today we measure our storage problem in hundreds of terabytes and petabytes. We have a single application that generates upwards of 50 terabytes a week! It is becoming a common occurrence to receive a ticket with a request like “I’ve run out of space for my research data, can I please have another 10 terabytes today?” Keeping up with the ever-growing demand for storage is currently our biggest challenge with cooling/power/space a close second.

In order to stay on top of new storage technologies, we are constantly involved in extensive product evaluations. Over the last 18 months, I have tried or purchased many of the most advanced storage technologies on the market today. Here are some examples:

• Acopia ARX6000 – Adaptive Resource Switch (File Virtualization)
• ADIC Scalar i2000 – Intelligent Enterprise Tape Library
• IBM GPFS – Parallel File System on IBM servers & storage
• IBM TS3500 – Enterprise Tape Library
• IBRIX – Parallel File System on Dell servers & EMC CLARiiON storage
• EMC Celerra Multi-Path File System (MPFSI) – High-performance File System on EMC CLARiiON storage
• EMC Rainfinity – Global File Virtualization
• Isilon IQ 9000 – Clustered Storage System
• NetApp FAS6000 – Net Attached Storage (NAS: CIFS/NFS/iSCSI)
• Sun x4500 Thumper - Data Server

In addition, I’ve been in talks with 3par, Agámi and BlueArc, as well as one company that doesn’t even have a real website or any customers, but has a very interesting product. On Monday, I will be speaking with Amazon about their S3 simple storage service. It’s a constant effort to know what’s out there and to get enough understanding and hands on experience with the technology to really know what makes sense for our present or future needs.

This problem is not going away. It was sometime in this year that we officially entered the “petabyte club,” with it’s rapidly growing membership, and it won’t be long before we are dealing in 10s of petabytes. Data is our product and it’s very valuable to us, sometimes priceless. Storing and protecting that data is my job and is something I take very seriously.

Now if I could only find that flashlight.

Social Networks

Social networking is (in relative terms) the new killer application of the Internet. This is apparent in the placement of MySpace in Alexa’s top 500 most accessed sites in the United States. In spite of the fact that MySpace could indeed be the ugliest website ever, it still manages to draw more traffic than heavy hitters like eBay, Craig’s List, Wikipedia and Amazon.

Admittedly, that particular statistic may not be as important since last month when Nielsen/NetRatings pronounced total time spent, not number of page views, as the best measure for online engagement. Whichever rating you use, bringing people together is clearly a service people want and they’ll take it any way they can get it.

My new favorite and the recent (since last September when they opened their doors to anyone with an email address) master of the social networking space is Facebook. Not only is the layout clean and easy on the eyes, but it has a highly-extensible platform that makes it relatively easy (and potentially lucrative) to build exciting, new applications for the millions of Facebook users to enjoy.

One of the best things about Facebook is the way that you add external content into your profile, making it the central place you’d point people to in order to see all your stuff at once. Below are some of the other social networking sites I currently have integrated into my Facebook profile.

• Twitter is a “micro-blogging” site that allows users to post short updates (140 characters or less) to whoever chooses to follow them. Users subscribe to each other’s feeds (or not) and post about what they’re doing right now, where they are or whatever is on their minds. Some users may follow hundreds (or thousands) of other “tweeters,” including friends, bloggers, news feeds, or any application someone can think of that might make good use of the architecture. (my Twitter)
• Jaiku is a site similar to Twitter that gives a user the additional ability to add RSS feeds his presence, making it yet another way to consolidate external content into one place. Both sites provide widgets that users can put in their blog or online profiles to display their recent activity. (my Jaiku)
• Last.fm is a social networking site built around music. An easily installed application updates the site with each song played in your iTunes, live, keeping friends and others abreast of your listening habits. You can connect with people based on your musical tastes and test your compatibility with the “Taste-o-meter.” (my Last.fm)
• del.icio.us is a bookmark networking site. Users post their bookmarks, along with options tags and a comment, making them available for all to see. It is possible to make your entries private (not shared) so it can be used strictly for storing and retrieving your favorite bookmarks. However, with sharing comes the social networking aspect. You can see how many people have linked to the same things as you and you can see what else people are linking to. You can subscribe to your friends’ links and keep up on what they’re interested in. (my del.icio.us).
• Digg is another bookmark networking site, focused specifically on news articles, videos and podcasts. Users post their favorite media content along with tags and comments and everyone can see what everyone else is digging. Hot topics can generate comment threads with hundreds of responses or more. As is also true for posting to del.icio.us, many blogs and news sites will include “digg this” links, allowing people to easily digg whichever article they’re currently reading. (my Digg)
• Stylefeeder is yet another bookmark networking site, but built around a person’s own individual style. Users still post bookmarks with tags and a comment, but they include an image with each link and can also specify a gender relevancy (male/female/neutral). (my Stylefeed)
• Upcoming is a Yahoo! site built around events. You post events (with tags and comments) for everyone to see. Anyone can set themselves as “watching” or “attending” an event, so you can see who else might be going (or who might be convinced). You can keep up with your friends’ events and they can keep up with yours. (my Upcoming)

So in spite of all these wonderful sites out there and all the fascinating options they provide to their users, many of my readers only actually see the content as it is presented in my Facebook profile. As Facebook continues to add features and developers continue to build new applications, more and more folks will be focusing on that platform as the focus of their online presence. Look for Facebook to continue to blow up in the coming months.

Also look for future posts about social networking on this site as it is has been a major focus of mine for as long as I can remember and it’s time I started writing more about it.

About This Mac

I’ve replaced my primary computer at home with the following:

The 2.33GHz/3GB served me well for some time, until the CD/DVD drive stopped reading any discs. The extra 1GB in the new machine is sweet (and I’m sure the extra 70 MHz is going to be really cool too).

Harmony 676

A few weeks back, Logitech held a 100 Remotes A Day Giveaway where 700 lucky winners would receive a new Harmony 676 Universal Programmable Remote ($199.95 msrp). To enter, I filled out a little web questionnaire about my home entertainment system and how many remotes there are and whether or not anyone can seem to use it for anything without my assistance. Not long after I was happy to receive the report that I was indeed a winner!

The first thing you notice about the Harmony 676 is that in order to configure it, you don’t mess around with reading a bunch of annoying codes that you have to cycle through and try before you find the on that controls your device. Instead, you plug it into your computer via the included USB cable and program it by directing your web browser to HarmonyRemote.com. Once you setup an account and enter the manufacturer and model number of all your devices and tell it how you want them to interact with one another, it updates the remote automatically and you’re ready to go.

For me, the setup phase took some time because of the number of devices I’m working with. After gathering all the information and figuring out how I wanted it to work, the process of entering it and updating the device wasn’t too tedious.

I configured it to control the following devices:

• Sony television
• Sony amplifier
• Sony stereo receiver
• Sony dual cassette deck
• Sony 200-disc CD jukebox
• Sony VHS/DVD combo
• Sony PlayStation 2
• Motorola digital cable box
• TiVo Series 2
• Turtle Beach Audiotron

The Harmony 676 is also a pleasure to use. The way I have it configured, I hit the “Watch TV” button and it turns on the TV, sets it to Video 1 (the TiVo), turns on the amplifier and sets it to Video 1. When I use the volume buttons, they control the volume on my amplifier. When I hit pause it pauses the TiVo. The off button sends the off signal to all the devices.

In addition, the Harmony 676 also allows you to set a list of favorite channels for which it will download the schedules and allow you to scroll through a “now playing” list of shows that are currently airing on your favorite channels. If you click on one it will go to that channel, in my case, via the TiVo.

The remote is physically designed similarly to the TiVo Series 2 remote. Though it lacks some of the more tactile features like the giant pause button, it has the same hourglass figure and a similar layout. I found the buttons didn’t feel quite as responsive as I’m used to with the TiVo Remote, but that may also be just it still needs to be broken in. Either way, for now I’m down from nine to just two remotes and loving it!

things google knows about me

I decided to go through all the hits returned by a Google query for my name that actually reference me (and not some other joker named the same) to see what all is out there. Below is a summary of what the first bunch of real hits will tell you:

• I run this here website (and have my resume online)
• I have hacked my Tivo to do some interesting (though not necessarily original) things
• I am a fan of Mr. Show and also A Christmas Story
• I was involved with the whole “Kendall Square Turkey” thing and was apprehended for taking photographs on government property
• I once worked at Cloud 9
• I co-own an “ISP” in Somerville and am involved with BBLISA
• I went to Oberlin and was active in the Computer Science department
• I run Livephish.org
• I send email with pine
• I am an OpenSRS domain reseller
• I am (or have at one time been) a Saab owner
• I was at one point “way into” the OpenMind Common Sense server
• The Wall Street Journal said that I am “not a shadowy stalker” in an article about the Tivo titled “Oh no! My TiVo thinks I’m gay”
• I am a Panix user and my birthday is the last day of November
• I once used BSDi
• I was in some way involved with the “Heaven’s Gate” thing
• I have contributed sample listings to the Top Sample Lists
• I am a FreeBSD user
• I am a Quaker
• I am a TWiki user
• I am a fan of Liz Phair
• I am a fan of DJ Shadow
• I am active in the Phish online community
• I have used the Amanda backup system
• I am interested in hip-hop culture
• Carl Erikson calls me his friend.

It seems like a lot, but it’s really not that bad. It could be worse. Besides, all that stuff in Google and Googlism still only knows this about me.

itunes catalog

Since I started importing all of my music into iTunes, one thing I’ve missed is all the album artwork that I used to look at back when I listened to CDs. However, iTunes has the ability to store cover art in the ID3 tags of each music file. So, I decided to take that on as another project. Thanks to Clutter, FetchArt, Slothdog’s Amazing CD / Album Cover Finder, Amazon and Google, I’ve managed to find and import cover art for nearly everything in my iTunes Library.

Now what do I do with it? Well, for a while now I’ve had my iChat set up to display what I’m listening to right now. So anyone else using iChat can see the title, artist and album of whatever is currently playing in my iTunes. Thanks to the CocoaObjects.com: iChat Script Collection, I was able to just drop an AppleScript into a folder on my Mac to make iChatStatus update my buddy icon with the album artwork of the current track. I am loving this! I’ve never been that into buddy icons for the most part, but this is really great.

Another thing I’ve wanted to do is publish a page of all of the music in my iTunes Library, like I currently do with my CD collection. Enter iTunesCatalog. This $10 application will read in your iTunes Library.xml file and create an HTML catalog of all of your music, with album covers and just about any other information you desire. I still haven’t got it working exactly the way I’d like. I emailed Kavasoft about some of the things I’d like to do and they said that the next version of the software should do everything, including allowing me to completely customize the layout of the catalog as well as having full control over the content. You can see a current copy of my iTunesCatalog here. It’s pretty large because of all the album covers, so be patient while it loads.

Another neat thing about iTunesCatalog is that it will update MusicMobs with your listening habits so that it can recommend other music that you’d like. I’m hadn’t been using MusicMobs before this, but since the software made it so easy, I figured I’d give it a try. You can see my page here.

So what’s next? I’m not quite sure yet. I have been thinking about writing some scripts to upload the information about what I’m listening to to this site with album artwork and all. I may also do something that changes the background image on my desktop to the current album cover. But the real project is still the ripping. I estimate that I have about 250-300 CDs left to import. So far, I’ve completed everything up to and including Béla Fleck and the Flecktones, as well as scattered items throughout the rest of the alphabet. There is still much to do. I am hoping to finish by the end of the year. Wish me luck!

cdw | macwarehouse

I’ve recently managed to fill up the 200GB hard drive that I purchased not too long ago. Most of what’s on there is live music I’ve recently accumulated that I need to dump to CD or DVD. So I figured, why don’t I buy a spindle of DVD-Rs and start burning? That hasn’t been as easy to do as I had originally expected.

First of all, I have a 12″ PowerBook that I purchased right when they were released in January of 2003. The SuperDrive I have burns at 2x. Of course, the SuperDrive has been updated twice in the year and a half since I got mine, so all the current ones burn at 4x or 8x. Apple no longer carries the 2x DVD-Rs. A note on their online store explains “A 2x SuperDrive writes to 4x media at 1x, so to obtain the highest performance from your 2x SuperDrive, we recommend that you continue using 2x DVD-R media just as you do today.” Of course, since they don’t carry it, that means I have to find another place to get it.

So I figured I’d try an old favorite of mine, MacWarehouse. Surprise, surprise, they no longer exist. Now they’re cdw | macwarehouse. So I decided to place an order. This was on the 10th. I called yesterday to find out why I hadn’t received it yet and they told me that someone (my “sales rep?”) canceled the order, but nobody knows why. My sales rep isn’t in. I explained to the customer service rep that this was my first purchase and how it wasn’t going well. I told her “I expect that you’ll be reordering the item for me and shipping it to me overnight at no additional cost to me.” She took care of me.

Today, I got my DVD-Rs! So, it all worked out. I still don’t know why my original order is canceled, but when I told them the issue and what they needed to do to fix it, they came through. Now its time to get burning!

powerbook repair #2

After getting my PowerBook back from its last repair trip and finding that they did literally nothing to address the key issue, that my hard drive was extremely flaky and needed to be replaced, I spent another week of my own time trying to gather evidence for the second attempt. I backed up and formatted my hard drive a number of times. I ran tools like Norton Utilities and DiskWarrior repeatedly, which found more issues each time I ran them. I attempted to reinstall the operating system, which failed due to disk errors.

So, I finally sucked it up and sent it in once again. I was sure to express my concerns to the Genius Bar staff person, so as to ensure that things would be properly handled this time around. I am pleased to announce that they took care of everything! They replaced the hard drive and the ribbon cable that connects it to the rest of the computer. They also replaced the rubber foot which went missing during the computer’s last trip to Texas. So far as I can tell, things are finally in working order.

This is huge. I have been without a properly working computer since returning from North Carolina in September. It’s been nearly a month of dealing with stupid computer hassles. Much of the time my computer was with me I spent trying to get it working again instead of using it for what it was intended. Many things have fallen off because of this. The lack of posts on this site is only one of those things. So, now I’m trying to get back on top again. Of course, it is not going to be easy considering how preoccupied I’ve been and will continue to be with the move. But at least it’s one less difficulty to overcome.

powerbook repair #1

Earlier this week, I sent my 12″ PowerBook in for it’s first repair. Upon returning from North Carolina a few weeks ago, I started to notice lots of problems with my laptop. It was hanging in weird places and having trouble reading and writing files. I tried a number of things, including repairing it with DiskWarrior, reinstalling the operating system, etc. Each time, things seemed to be working, and then I’d get to a particular place on the hard drive and it was start having problems again.

After settling for booting to an external disk for a week or two, I finally decided to part with the machine for a few days to get it fixed. I took it to the Apple Store and explained the situation. Since I was sending it in anyway, I figured I’d ask them to fix the darkened area in the bottom right hand corner of the screen and replace one of the missing rubber feet.

During the last few days, I’ve been running off a 700MHz iBook that I borrowed from work. I’ve been booting it up to an external FireWire drive that I also borrowed from work which contains the newest version of my laptop hard drive data. Once thing I noticed was that the 700MHz G3 seemed to be working faster than I remembered my 866Mhz G4 working. Since the data on the drive is the same, the only real difference is the hardware. Obviously, the processor should be faster, as well as most of the hardware in the machine. The hard drive seems like a very likely reason for that slowness (and the frequent hangs I was experiencing in TraktorFS).

When I went to pick up the machine from the Apple Store this morning, what I found was very disappointing. They replaced the screen, which is great, but they didn’t do anything with the hard drive. They said they ran some tests and it checked out fine. Apparently, it didn’t matter that I had reinstalled the OS twice in the last week and formatted the hard drive more than once, but continued to see problems. They didn’t see any problems during the hour or two that they ran tests, so they decided that I must be crazy. Oh, and they replaced the missing rubber foot, but it came back with an entirely different rubber foot missing. WTF!?

So, I’m once again worn out by the process of trying to get my computer fixed. I’m probably going to deal with it for another week or two before I suck it up and send it in again. What a pain in the butt. I mean, I guess it’s possible that I am crazy and things will start working perfectly now, but how likely is that? Not very. When are these guys gonna get their shit together?