Home » Business Intelligence

How to Rescue your Data Warehouse Initiative

4 May 2007 One Comment

Years back, in an Oracle user conference, I saw this very pithy slide:

 
Let’s admit it. Most of us who have been in this field for a while have had the sad experience of seeing the latter outcome some time during our careers. Failed projects are harsh realities that most organizations face. We, professionals, learn from these failed initiatives, and apply the knowledge to detect problems early and remedy them effectively.

To understand if your DW/BI initiative is failing, we need to understand what constitutes success. As I noted in my earlier post, An Agile Road to a Successful Data Warehouse, David Wells and James Thomann of TDWI describe three “types” of success and their definitions as they relate to data warehousing – economic success, political success and technical success.

Most businesses have moved away from the idea of a large unified data warehouse effort that will involve hundreds of internal resources and consultants and take two years to deliver the first cut of data into the warehouse. In today’s world, you know you have a problem with your initiative if, after a year, you are not seeing some of the benefits you expected. Use this free spreadsheet tool from Nucleus Research to measure ROI for your BI investment.

Signs of a faltering DW/BI initiative may surface in the early stages and these should be taken seriously. The first is missed deadlines and cost overruns for individual projects. It is not unusual for these projects to run two or even three times over budget. Overruns usually result from inflexible business processes that create needless complexity in reports or ETL forcing compromises in design. Costs for such efforts can easily trump whatever benefits they might produce. In a recent project, my client wanted to see costs, revenue and expenses each aggregated to a different grain in one single report, rather than following the much more common practice of drilling down to details from consistent higher level aggregates, all at the same uniform grain. The resultant effort was inordinately expensive to produce a single, unique report. The second warning sign - complaints by employees about poor usability or unmet performance promises may point to serious underlying design problems.

Unfortunately, in many cases, companies tend to just live with such inadequate data warehouses for years on end or, leave them largely unused. A few scrap them altogether. It is not uncommon to find a data warehouse sourcing data from a previously built data warehouse which now merely serves as a repository of data, rather than a central place for analytics. With persistence and patience, such projects can be turned around. The key to uncovering the root cause of the problem is to reassess and reaffirm the business goals of the system and the organizational support it receives. Technical shortcomings can usually be overcome much easier than some of the political hurdles within the company.

Turn-around Team

There are several steps to effectively troubleshoot and recover a stuttering DW/BI initiative. The first step is to assemble a team that can lead the turnaround. This team, led by the chief business sponsor, should comprise of key business partners who have in-depth knowledge of the several business functions the data warehouse aims to benefit. And of course, the team should have competent members of the technical staff, who have the expertise to make different design choices to align the data warehouse to the business goals.

The Rescue Approach

Funding: It is important to have a clear, separate allocation of funds for the attempt to revamp the BI initiative. If you have been able to detect signs of misalignment of the DW/BI initiative in its early stages, then, the chances are you will be able to secure adequate funding for this effort.

Business alignment: Once this team is assembled and empowered to make the bold decisions needed to resurrect the DW/BI initiative, the first task is to reestablish clear business objectives for the initiative. The scope should be tightened to make sure that the most critical gaps are addressed and prioritized and needless features eliminated. Having too broad a scope often leads to a development effort that takes a life of its own and loses touch with the very problem it was meant to solve.

Political alignment: The success of the BI initiative depends on its users. It is important to connect to the very people who will be using the BI system to solve many of their business problems. Without understanding their issues of usability and without granting some political concessions or incentives to secure their buy-in for the initiative, any attempt to revamp the effort will fail. If the culture and practices of the company pose a significant barrier to achieving these objectives, scale back aspirations and build the organizational capacity needed to achieve early results, rather than adopting a strong-arm technique to force adoption of the BI application. The key to success is to identify the most critical users of the BI infrastructure and secure cooperation and collaboration of these users.

Technical realignment: The technical infrastructure, the architecture of the data warehouse, BI metadata and even ETL programs may need radical changes to align these to established best practices. Priorities should be established to deliver analytics that have the greatest business impact, and, to reiterate the point again, the technology architecture should align very closely with the business objectives established by the turn around team.

Monitoring: Once the solution blueprint and implementation roadmap has been adopted, the turnaround team must monitor progress against its reaffirmed or revised goals. As I have written earlier, an agile approach, with frequent releases where the users remain engaged and interested has the greatest chance of success. Nevertheless, it is important to measure this success against internal benchmarks. Make sure project deadlines are met and costs remain under budget and the team should remain alert to any recurrence of the original signs of trouble, such as complaints from users. I believe that a well-executed turnaround ought to yield early wins and measurable progress on most goals within 4 to 6 months, given proper funding and resources.

Ralph Kimball and Margy Ross, in this article - Data Warehouse Check-Ups prescribe periodic assessments of your data warehouse to avoid these common disorders of inconsistent business sponsorship, poor data quality, scant acceptance of the DW/BI environment by the business community, conflicting political priorities of “doing it right” vs. “doing it fast”, and problems with the design or ETL infrastructure. Good reading.

One Comment »

  • John D said:

    Awesome read. What is perhaps most remarkable is how few managers are actually aware or worse, care, about how far off course a EDW project goes, or what the end users think about the DW. Maybe because they are internal consumers?

Leave your response!

Add your comment below, or trackback from your own site. You can also subscribe to these comments via RSS.

Be nice. Keep it clean. Stay on topic. No spam.

You can use these tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

This is a Gravatar-enabled weblog. To get your own globally-recognized-avatar, please register at Gravatar.