Socrata

Early to mid last century switchboards were manually operated by telephone operators, most of whom were women. In the late 1960′s Lily Tomlin endearingly characterized the role as Ernestine the telephone operator on Laugh-In. Advances in technology democratized the act of dialing a phone number and now we all do it ourselves.

In the 1960′s and 1970′s many corporations had pools of typists. If you needed to send someone a letter, you submitted your longhand request to the typing pool. Mini-computer based systems from Wang and Xerox replaced the typewriters with centralized word processing capabilities, but you still submitted your letter longhand to the pool. While these word processing systems were big advances over typewriters, they were still too expensive and too complex for the mainstream users. Remember, most typists had to be retrained on how to compose SGML in order to send special formatting instructions to the printer. Then in the 80′s, personal computers were adopted, followed at first by markup-based word processing software from Wordstar, then by WYSIWYG software from WordPerfect. These advances democratized the act of writing letters and we now take it for granted that we can write letters ourselves with Microsoft Word.

Number crunching followed a similar course. Accountants used paper spreadsheets to crunch numbers. If you needed some analysis, you asked the accounting group to develop paper models for you. Then in the 70′s McCormack & Dodge created electronic spreadsheet software for IBM mainframes. In the early 80′s software pioneers Dan Bricklin and Bob Frankston created the PC-based electronic spreadsheet VisiCalc, which later was supplanted by Lotus 123, which later was toppled by Microsoft Excel. Electronic spreadsheet software democratized the act of making financial decisions based on analysis of numerical data.

Art departments used to create foils for presentations. Making presentations has been democratized and we now take for granted that we can use PowerPoint to make presentations ourselves.

Engineering and architecture departments used to draw schematics and floor plans for us. Now with programs like Visio, we take for granted that we can create our own floor plans and technical drawings.

You could even argue that TypePad and WordPress have democratized creating simple web pages.

Why hasn’t working with data been democratized? In an odd paradox, IBM, Oracle, Microsoft and a handful of other companies have made powerful relational databases both ubiquitously available to enterprises yet out of reach to mainstream audiences. While advancements in technology empowered people to solve their own problems in all of the aforementioned areas, current database technology has failed to democratize the act of organizing and analyzing data. Instead of extinguishing a legacy profession, databases are so hard they’ve actually spawned a new profession – database administrators.

Why isn’t organizing and analyzing data as easy as creating an electronic document, spreadsheet or presentation? Stay tuned. Soon it will be. That’s exactly what we’re aiming to do with blist.

I’ve argued for a long time that world’s most ubiquitous database is Microsoft Excel. After all, Excel supports a large number of columns, a practically limitless number of rows, auto-filtering and sorting. So what’s wrong with Excel as a database? Here are 10 shortfalls to get the conversation started:

1) It’s cumbersome to store elemental data that might have more than one value per row. For example, let’s say you are using Excel to keep track of applicants for a sales position. You want to keep track of each applicant’s phone number. Most people have two or three phones now. How do you organize that in Excel?

2) It’s way too hard to create a pick list and use that pick list to populate cells. Maybe you want a column called stage in your applicant tracking system – Resume Received;Phone Screen;Interview;Extend Offer; etc. This is possible in Excel, but it’s unbelievably hard. Excel’s help describes how to do this in 9 steps. 3 of the steps are so hard, they hyperlink to another help page.

3) It’s not very visual. Just plain, old boring numbers and text. When I define a range as a date range, why doesn’t Excel embed a calendar control so I can pick a date from the calendar? When I try to insert a photo, why does it feel like it overlays my entire spreadsheet instead of inserting the picture into a cell? Why can’t I simply say that I want all the cells in column to have a checkbox?

4) It’s entombed on my PC. Yes, I can email to you. But now we each have separate copies. What if you update yours and I update mine? How do we reconcile those changes?

5) It’s not multi-user. What if you and I want to work on the database together at the same time?

6) I can’t create meaningful queries. What if I’m a VC and I want to construct a query like "Show me all startups who’ve been into the office for a meeting, were founded less than 12 months ago, are in Seattle and are not yet VC backed." If you have 20 or 30 rows in your Excel database, you can just eyeball it. But what if your deal pipeline tracker has 400 or 500 rows? How do you do that query in Excel (the answer is you don’t. It’s amazing how many VCs I know who are building this pipeline management system in house).

7) I can’t easily save and load multiple views of the same data. What if I have a list of 5,000 sales leads and I want to save some predefined views:

7.1) Leads rated "A" and not contacted in the last 45 days

7.2) Leads in the northwest (southwest, northeast, southeast)

7.3) Leads with an opportunity size > $50,000 and a probability of close > "Good"

7.4) Leads where stage changed to "Lost" within the last 30 days

7.5) Leads where stage equals "New" and Date Created is less than 7 days and source is "website" How do you save these views without bifurcating the raw data?

I can’t easily let someone casually subscribe to updates. If I’m the sales manager, wouldn’t I want to subscribe (via email or RSS) to the view in 7.4 above? Wouldn’t I want to know that as it happens? How do you do that in Excel?

9) What if I want a data entry form on my website to insert rows into my database in Excel? How do I do that, especially when my Excel database is entombed on my PC and my PC is turned off?

10) How do I relate two Excel spreadsheets to each other? If I’m trying to create a project list and I want to keep track of which tasks are assigned to which people resources, shouldn’t I be able to embed part of my people sheet into my project sheet? Of course you should, but making that happen in Excel is incredibly hard.

What we’re building at blist addresses all 10 of these Excel pain points very nicely. It’s as familiar as Excel but designed to be a collaborative, visual database. It’s the world’s easiest database. Have you tested the limits of Excel as a database? We’d love to hear your frustrations and encourage you to sign up for our beta.

Excel is two-dimensional. Columns run horizontally. Rows span vertically. Initially we think of data as having two dimensions as well. In fact, two key terms in database lexicon are rows and columns, just like in Excel. Here’s an example of what an Excel database of some fictitious people might look like:

In reality, however, data usually has three dimensions. Rows describe items. Columns are singular attributes of those items. Lists are plural attributes of those items. Observe how adding a column for the names of each person’s children necessarily morphs the structure into a third dimension. Here’s what that Excel database might look like with a list in a cell:

The way I’ve drawn that table is OK. The problem is that I had to grab each row and adjust its height manually.  Another option is I could have selected all the rows and navigated to Format –> Row –> Height and set the height for all rows to some arbitrary height. The problem with that, of course, is that the appropriate height differs from row to row. Jacob has three kids, Paul has two and Kim has one.

Maybe a better approach is for the software to allow you to enter a list of items in each cell, but be smart about how it shows you that data. Tighten it up. Maybe decorate it with a little triangle to suggest there’s more here than you see, but when you hover your mouse over the cell, show you all the values. Maybe it would like something like this:

What do you think?

Just a quick post to congratulate the team on a great sprint. It was our best to date. At blist, we use a development methodology called Scrum. It’s designed to be lightweight and agile. Within the framework of Scrum we run two-week sprints. That’s a short development cycle. On the starting Monday of each sprint we have a kickoff meeting where we review a prioritized product backlog of features and the engineering team assigns cost estimates for each feature. Then we draw a line where the total estimate equals the total available man power. Everything above the line is what we hope to accomplish in the sprint.

Early in the development of blist we’d be lucky to penetrate 50% into the backlog in any particular sprint. I think we finished this sprint with something like 93% of the work items being completed. Hoowah! Great work, team! Our burn down rate was linear through the entire sprint as well. The burn down rate shows on a daily basis how many hours of estimated work remain relative to how many man hours are left in the sprint. Here’s what a sample burndown chart looks like:

We’re still not revealing too much about what it means when we say we’re building the world’s easiest database, but a few features completed in this sprint include:

* Lenses
* Simple search
* Multi-sided columns
* Preferences

I’m excited for Monday’s sprint kickoff meeting to see what we’re going to bite off in the next two weeks.

Today AdventNet, developer of the Zoho suite of lightweight office applications, launched Zoho DB. They already had a lightweight forms building, database application called Zoho Creator. Zoho DB is more powerful, especially considering that it understands SQL queries. Dabble DB and QuickBase are two other notable and well regarded web-based databases.

Throw the blist hat into the ring too. When we launch early next year we’ll compete with QuickBase, Dabble DB and now Zoho DB as well.

What strikes me most about the current offerings in the space is that they seem to have been designed for folks who love databases and there’s absolutely nothing wrong with that. While these solutions offer what might be thought of as an online version of Access, it’s important to observe that Excel is the most ubiquitous desktop database because Access is too hard for most people.

At blist our core design premise is that people want data, not databases. People want to see a product that moves us away from SQL, not towards it. We think you want a database that just gets out of the way and lets you create and share your data, connecting with others who are also passionate about whatever information it is that you’re organizing.

blist. Create, Organize. Share. Connect

A few weeks ago I received a phone call from a guy in Miami. Here’s how the conversation started:

- My phone rings.

- me “Hello, this is Kevin”

- Miami guy “Kevin Merritt?”

- me “Yep, Kevin Merritt”

- Miami guy “The same Kevin Merritt who once knew a guy named Neal Kaskel?”

- me “Um, uh, yeah” (this is weird, I’m thinking)

- Miami guy “Eureka! I’ve been hunting you down for weeks”

- me “uh oh”

- Miami guy “Are you the guy who made some software called Prospector?”

- me “Well, I led a team that built it, but yes Prospector was my baby”

- Miami guy “Great. I want to buy it. Can you sell it it to me?”

That odd conversation went on like that for another 20 minutes. Miami Guy wouldn’t take no for an answer, even when I told him that Smith Barney now owns Prospector and it’s not for sale as far as I know, not to mention 10 years out of date. He had to have Prospector.

Prospector is an application we designed and built between 1997 and 1999, when I ran IT for a 400-person investment bank. Smith Barney bought the company in 2001, in part because they thought Prospector was pretty cool and could use it in other parts of the company.

On the surface, Prospector was a multi-dimensional customer relationship management (CRM) system. It reflected our role as an infomediary (information intermediary) doing mergers & acquisitions (M&A). We didn’t buy a company and then sell it to a bigger company. We introduced sellers to buyers and then facilitated a deal being consummated.

Miami Guy recently started his own M&A firm. News traveled 3,000 miles from California to Miami. He heard that Prospector is unequivocally what you need to run an M&A firm.

The amazing thing about Prospector was that to its users it was a CRM system, but to its developers it was a platform for modeling, organizing, manipulating and analyzing data. No one on the dev team knew anything about CRM or investment banking, yet the people who used it thought it was the best CRM system you could ever invent for doing M&A. We didn’t create a malleable platform because we were smart. Rather, we built a platform because we really didn’t know what the app needed to do. We did what seemed natural – we delegated functional specification to business people.

Bringing this post back into context and 2007, blist has its roots in Prospector. We’re developing an innovative platform for modeling, organizing, manipulating and analyzing data. Want to track applicants or sales leads? blist will be good for that. Want to share your favorite recipes with friends? blist will be good for that too. Want to create your own VC done deals database? Yep. blist will be able to do that too. As importantly, you won’t need a DBA or even anyone in IT to get the job done. blist is for you, whether you’re from Miami or not.

Today’s Wall Street Journal has an article by Jeannette Borzo titled “Do-It-Yourself Software.” The subtitle is “New tools let businesspeople avoid the IT department and create their own computer applications.” The article describes some of the pros and cons of using tools from statups like Coghead selling do-it-yourself application builders. There are other startups in this space, including Wufoo and LongJump.

The subtitle of the article highlights the perception that there’s an ongoing tension between department managers and IT departments. That topic provides an opportunity to share a little more about our blist product vision. As I’ve shared previously in this blog I spent six years as the CIO of an investment bank (question for LazyWeb: should blog posts be totally self-contained or can an assumption be made that the reader has read all prior posts?). I had both a reasonably good sized budget and development staff. Department managers would call me from time to time and ask for a meeting to discuss a custom development need they had. I’d listen to the manager describe her problem. I’d ask how she was solving the problem today. Often it was a collection of manual processes and/or Excel spreadsheets.

More often than I’d like, I’d encourage the department manager to keep limping along with the home grown system. Why? That’s always the question. The honest, matter-of-fact answer is that good CIOs are good in part because they can discern what’s opportunistic and strategic from what’s not strategic. Invest in what provides competitive advantage and outsource or ignore the rest.

How do good CIOs feel turning department managers away empty handed? The good ones feel awful. Good CIOs want to solve problems. They want happy constituents. They want to empower people to do their best work. Today’s WSJ article suggests that these new do-it-yourself application builders can help you avoid IT. Actually, if they work and they’re secure and relatively open, IT will be their biggest fans. If they work, nobody’s going to have to sneak them in at all.

And that’s where we bring this post back to the blist product vision. blist is in large part a response to the disappointment I felt as a CIO whenever I turned a department manager away empty handed. We’re building a platform that empowers department managers to easily solve their own problems, strategic or otherwise. But we’re doing so not in spite of IT, but from the premise that this is the do-all platform that IT would build if they had the bandwidth in the first place. Isn’t the best technology the solution that just gets out of the way and lets people get their job done without friction? That’s what we’re building at blist.

It’s a win-win. The department manager gets the perfect solution – designed for her and by her. The CIO gets what he wants – a happy department manager.

Most of us are list makers. Shopping lists. To-do lists. Wish lists. Honey do’s.

Back in the 90′s I managed a team of network administrators and network engineers. They were good at what they did, but not so good at remembering what to do and in keeping me up to date on their progress. I did what any technically inclined CIO would do – I whipped out Excel and created a template for a status reporting tool. But this was no ordinary status report. It had six or seven columns and about 4,000 lines of VBA code I wrote over a few evenings. That VBA code created a floating “Status Report” menu, brought in some cool status icons (Done, Not Started Yet, Blocked, Won’t Do, On Track, Slipping, etc.) and linked in hidden worksheets containing departments, employees, network engineers, cost centers, etc. to feed combo boxes in the main status report sheet.

My network engineers found that using this tool wasn’t too burdensome and even kind of fun. It was a really easy, visual tool for quickly updating status. They would send their sheets to me at the end of each week and I had some tools to filter them quickly to look for tasks that were blocking or slipping. It really helped keep the team cranking and me in the know.

As useful as the tool was, there were problems with this approach to solving problems:

* Not everyone can write VBA code
* The solution was very specific
* Collating the sheets from the engineers was painful
* Adding new tasks was a manual chore

Excel alone didn’t solve my problem. It was the combination of Excel with VBA that worked for me because my background prepared me to code. The universe of people with problems is much bigger than the universe of people who can code.

We’re creating blist to solve this and many other kinds of problems in an easy, powerful, flexible and generalized way. When blist launches, anyone can easily recreate my status report tool without any coding, without having to create status icons, without having to link data from other sheets, without having to collate and reconcile changes.

The blist vision is a culmination of a dozen real pain points and real solutions for all kinds of real people. We’ll be telling more of these stories in the future.