Socrata

Happy Memorial Day. A genuine and sincere thank you to all who gave their lives in defense of freedom.

If you have some time today, I encourage you to research the roots of the song “Battle Hymn of the Republic.” Originally written in 1855 by William Steff as a campfire song, this patriotic anthem has motivated many – soldiers in both the union and confederate armies, abolitionists, evangelicals, modern day soldiers and politicians. The lyrics have changed from time to time, with the current version most widely heard today having been written as a poem in 1861 by abolitionist Julia Howe.

Spend some time today poking around the web. Read the articles you find. Listen to the many different renditions. Think about the men and women it inspired. Let it inspire you. Remember those who were inspired by it in defense of the freedom we enjoy.

You may have seen my post that a couple of weeks ago I decided to try twitter. Feel free to follow me at www.twitter.com/kmerritt. My first 10 days or so have been mostly positive, other than the annoyance of the service being down fairly often and the mental distraction of the engineer in me wanting to solve their scaling problems.

Today it was revealed that twitter raised $15M in venture capital. Congratulations to them. Hopefully now they can rebuild their messaging architecture by hiring some people who can solve these kinds of scale problems. It’s a very solvable problem technically. My concern is the time it will take relative to their exponential growth. It’s a real world race condition. Can their current network survive for as long as it takes to build a more scalable one?

In all candor, they should have worked harder to not disclose their funding. Of course the funding itself is going to drive usage at an even faster rate. Why do I have that position? There’s no doubt in my mind that twitter hit a network effect inflection point recently. By that I mean at some point within the last 60 days or so, fence sitters like me observed that the twitter network itself was large enough and growing fast enough that the quality of the network was improving simply by the presence of new people. Linked In hit a similar inflection point 18 months or so ago. I think the only thing holding a lot of people back from using twitter was company viability. Raising money isn’t a business model, but it does buy twitter some time and I think more people will try it now.

There isn’t a requirement to disclose a financing. There’s a very minor risk to investors to not file with the SEC. It only comes into play if you try to go public and have unaccredited outside investors. That seems unlikely assuming twitter has had decent legal counsel. Under the circumstances, I think twitter might have been better served to keep the financing quiet and focus 100% of their energy on stabilizing the platform in part by not exposing it to undue stress. They could always spend 90 or 120 days, fix up their infrastructure, and then come out with a much more meaningful announcement proclaiming both network reliability and a significant financing.

One of blist’s most powerful features is Discovery, which lets you create a new blist by using someone else’s blist as a starting point. I have a guest post over on FlowingData today, describing what Discovery is, and the thinking behind making everyone’s blists discoverable by others.

Check out the post and while you’re there, stick around and check out some of the other great posts. If you love data and the analysis thereof, you’ll love this blog.

Congratulations to Tony Wright, co-founder of RescueTime for being my first follower on Twitter. I know Tony and know he’s here in Seattle, so Tony come on by to pick out your blist t-shirt. We’ve got long sleeve or short and two versions – hi fidelity color logo or monochromatic white logo on black fabric for those special occasions.

For those who don’t know, RescueTime is a Y Combinator startup offering “ridiculously easy time management & analytics.” I encourage you to check them out.

I contemplate a lot of things before jumping in. Contrary to the perceptions and stereotypes of entrepreneurs, I’m not all that impulsive.

I’ve thought about Twitter for a while and honestly can’t decide if it’s going to be a gargantuan time drain or if it’s going to be a way to communicate effectively. When I can’t figure something out, I run an experiment. So that’s how I’m going to treat Twitter for now. Unlike blogging, to which I am 100% committed, please treat my current Twitter status as experimental. You’ll be the first to know when my status changes.

So if you’re interested in micro updates on what’s going on at blist, in the Seattle tech sector or with Kevin Merritt, follow me on Twitter. The first person to follow me on Twitter gets a free blist t-shirt. (Sorry but blist employees and members of my family may follow me, but you are ineligible for the free shirt). For those who don’t know, we hardly ever give away our t-shirts – we like them being scarce and we’ve heard great stories of people asking “where did you get that blist t-shirt?”

You might wonder whom I’ve followed on Twitter so far. Well, I won’t give you the full scoop, but I will share that the first person I followed is Loic le Meur, the CEO of Seesmic.

Last night Paul & I went to a talk at Google’s Fremont (Seattle) campus on the future of cloud computing. It was a great talk about Google’s core technology assets – GFS, MapReduce and BigTable – and the open source re-implementations of those same technologies – Hadoop and Hbase. Aaron Kimball, UW computer science grad student and founder of Spinnaker Labs, gave a great presentation that was well attended by 100 or so folks who are interested in Internet scale computing.

It was a good, high level overview, which stimulated some good questions and discussion. Among the more interesting things I noted last night include:

*) The optimal profile for a server in a compute farm doesn’t require a lot of top speed CPUs nor should it be packed with RAM. The bottleneck is getting the data from the disk. A 2.4 Ghz 1U server with 4 CPU cores, 4-8 GB of RAM and 2 SATA disk drives (as fast and big as possible) is best.

*) There was some discussion about the performance delta between Hadoop and Google’s technologies. A great point was made that in the long term grand scheme of things, the delta is irrelevant. MapReduce represents a paradigm shift as signficant as the shift to client/server programming 2 decades ago. This approach is likely to be the norm for batch processing of very large data sets for 20 or 30 years to come.

*) The time and overhead of starting up a MapReduce job means that it really is inappropriate for processing datasets under 20 GB.

*) Cycles and bytes, not hardware, are the new commodity.

*) As more technology companies like Amazon and Google provide temporal, on demand access to large compute farms it has the effect of democratizing distributed computing.

*) Hadoop has stabilized significantly over the last year. Hbase needs another 6 months to reach the same level of maturity and stability.

*) There was some minor disagreement about whether virtualization is a foundational prerequisite for cloud computing. In an Amazon Web Services model where different external customers are commissioning and decommissioning servers often, virtualization is mandatory. In the case of internally consumed Internet scale compute farms like the ones Yahoo and Microsoft use exclusively for their own needs, virtualization isn’t a prerequisite.

*) Aaron characterized how SmugMug is building their business on Amazon Web Services. He made an interesting observation that SmugMug has effectively become a value added reseller of S3.

Thanks to Aaron for conducting this talk and to Google for hosting it. As you would expect for Google, there was a great spread of appetizers, beer and wine for the event. It was personally surprising and rewarding to see Aaron highlight blist in one his slides. Aaron was making a case for the web replacing the desktop and pointed to GMail, facebook, Google Apps, meebo, flickr and blist as examples. Hey, I can’t complain about keeping company with this group of innovators.