Self serve business intelligence dreams of letting everyone whip up any report or analysis they want. The reality is that its often not the report that’s the problem- it the underlying data and model.

So the idea of self serve business intelligence is a wonderful idea- the problem is that its not all about pretty graphs and fancy web user interfaces. You need to somehow design the data model so that every possible report that users dream up is possible- or move so much data modeling functionality into the report writer that it ends up looking more like a data transformation tool, yet is easy to use. There are “new” techniques that are getting lots of discussion- columnar databases are one, and certainly they provide interesting techniques, but only IF you’ve got meaningful data.

Here’s something to try- call up your favorite Business Intelligence vendor, and ask for a demo. It will be wonderful, it will be clear, easy, and simple- and it will be done on a set of data that was made by someone who knew exactly what the demo was going to be.

Real world data is messy, and it often does not follow simple rules. As a result, it takes significant work to build analysis- and to have a system that is ready for ANY analysis that any user might think up at the moment is non-trivial.

But its a worthy pursuit. And I think there are three key fronts in this battle;

1. Reporting and Analysis front end tools
2. Data transformation and integration tools
3. Fixing the source systems.

All the big players have integrated suites of products that perform the functions of the first two categories- IBM/Cognos, Oracle, SAP.

There are also lots of very interesting new tools in the first category- Tableau is one that gets a lot of buzz. Of course, the king of graphing and dashboarding is still the spreadsheet, and Excel has the crown.

There are lots of tools for the IT department in the second category. Datamartist is a unique new tool for the end user in this area- a self-serve desktop tool for data transformation. It is a tool that allows users to quickly transform data sets to experiment and create new analysis that can then be queried and viewed using the tools in the first category.

But in the end, I think the number one limiter on achieving the dream of self-serve business intelligence will be getting a handle on the quality of the data in the source systems. Garbage in means garbage out- and the last two layers shouldn’t have to tie themselves in knots trying to fix issues that are generated in the transactional systems.

But they do tie themselves in knots- and it is those knots that stop users from more freely and accessing their data using an intuitive and almost brainstorming approach- which in the end is the goal of self serve.

Unless the underlying data in the source systems is quality controlled, and designed to capture the information that is critical for analysts, then the Business Intelligence layer will have to work too hard to “fix” the data, keeping a large IT team busy writing code, and there’s nothing self serve about that.

In the end, the data quality issue is often what makes data warehouses and data marts so expensive to build- and drives users to spreadsheets or even databases (microsoft access is a common one). This is what I call “self serve data transformation”- and this is what Datamartist does- if you’re frustrated with the access you have to your data- download it and give it a try.