I have earlier talked about the merits of an agile, customer-focused approach to a data warehouse greatly enhances the chances of success for the project. In this post, I will outline a plan to staff such a project and how to get things started.

Christening

Hopefully, before embarking on a data warehouse initiative, you have done ample due diligence to form the basis and business justification for such a project. You have probably identified the key benefits you want to derive and the most important business questions you want to answer. You should have also set a not-to-exceed limit of the overall project cost. If you adopt the approach of most product companies, the very next thing you need to do is to establish the identity of the project. Most product companies thrive on acronyms – CDW for Customer Data Warehouse, SEW (Strategic Enterprise Warehouse) and the like. One of my clients, while building an enterprise scale data warehouse, created a theme or analogy of constructing an actual house (the “Great Room” at the centre of the house being the executive dashboard where all key information entities “meet”), and adopted a logo of a construction helmet as well that appeared in all presentations, deliverables, coffee mugs, etc. Such “christening” of the project helps to assimilate the project team to the same mission. It also serves as an introduction to the larger business or user community in the organization and delivers the not-too-subtle message to the business cohorts that they are, in part responsible for the care of the initiative in its infancy. The collaboration with all these business users is where the roots of success of this initiative lie.

Staffing the project

A data warehouse / BI project requires a number of different roles and skills from IS as well as the user communities. It is helpful to remember, that there is hardly a project where you will be able to find a “specialist” for every role of the project. Most projects I have worked have people with multi-faceted skills who can fill more than one role. A word of caution: try to avoid people who claim to have been skilled at every aspect of the project. Such people are less likely to be well versed in best practices in all the different areas and are more likely to cause irreparable damage if entrusted with a key area in which they are deficient, but have masked it well. At one of my previous projects, we had a “Superman Architect” who was not a believer of surrogate keys in the data warehouse. When the warehouse needed to accommodate an additional data source, it faced a tremendous development effort to move from having natural keys to surrogates.  In the real world, you will find people with excellent skills in just two or three areas who can fill those specialist roles.

Let us now examine the different roles needed for a data warehouse / BI project:

Project Manager: The Project Manager needs to be an experienced person, who believes that the project will only be judged by the addition to business value and can maintain the business focus at all times. The other equally important skill should be the ability to deftly navigate and leverage the political environment of the organization. This job is not for one who does not thrive amongst political heavyweights within the organization. Lastly, the project manager should understand the “schools” of data warehousing. The Inmon graduate will scoff at the Kimball’s Bus architecture with dimensional models and conformed dimensions while the Kimball graduate will never quite take to the 3NF enterprise model of the Corporate Information Factory. One company, having spent a good eighteen months building a dimensional data warehouse, brought in a consultant, indoctrinated to the Inmon approach, to evaluate the state of the warehouse. After spending a few weeks looking at the pure dimensional design without getting into the details of those models, this highly paid consultant served up a scathing report advising the client to pull the plug from the entire effort.

Business Partner(s): These are the identified “super users” in each of the business areas the data warehouse will address. Although the data warehouse team will gather requirements, this business user is ideally an expert in most of the business functions. Examples of such knowledge experts are easy to find in smaller organizations, but much harder in a large global organization. However, having such experts accessible to the team greatly reduces the potential of a gap between what IS develops and what was originally intended.

Core Team of Peers:

The core team has the actual responsibility for the design and development of the data warehouse. In my experience, the most successful data warehouse projects had agile teams where each role had specific responsibilities and none was more important than another. This mindset places emphasis on an environment of mutual respect, a necessary ingredient for any team to operate efficiently. The result is enhanced group dynamics and increased accountability of team members. To be successful with this team of peers, all roles must have ownership of the product’s quality, must act as user advocates, and must understand the business problem they are trying to solve. Unrestricted communication between the Project Manager, Business partner and between the roles in the core team is critical for the success of this mindset and, of the project itself. These are the roles a typical data warehouse / BI project usually sees:

Business Analyst: The business analyst is the equivalent of the product manager in product companies like Oracle or SAP. The BA works with the business stakeholders to understand their needs and goals and translate those into scenarios and quality of service requirements that the development team will use to build the data warehouse and BI infrastructure.

Technical Architect: The architect is responsible for designing the foundations of the data warehouse and BI application. This includes defining the dimensional data model as well as the structure of the BI metadata. The architect’s goal should always be to ensure a solid integrated data architecture and, at the same time, reduce the complexity of the user by dividing the metadata into clean and simple partitions (cubes, business areas, folders, universes etc). This is extremely important because it dictates how the system will be used. The architect should be cognizant of other very important considerations of reliability and maintainability, performance and security standards, and whether the architecture can be evolved easily in the face of changing requirements.

Developers: ETL Developers typically have intimate knowledge of the source systems. They are also expected to have expertise in the ETL tool or programming language. They help build and/or supervise extraction routines and, prepare the warehouse for deployment. BI Developers are responsible for creating canned BI reports and dashboards. They also help users write ad-hoc queries. They are typically experts in the BI tool of choice and have a keen interest and understanding of the business functions.

The other key roles in a successful project are: Testers, whose goal is to discover and communicate problems about the ETL routines or reports that could adversely impact the overall delivery or business value; DBAs to help tune the database for optimal performance and maintain the data warehouse environments; and Trainers, who educate the users on the content and structure of the data, availability and usability of reports and cubes.

Consultants: Most data warehouse / BI projects have a need to fill one or more of the roles I have just described with consultants. However, given that, in real life, data warehousing efforts tend to last a long time, organizations need to employ consultants judiciously. It needs to be clearly understood while a budget for consulting is set up, whether a consultant is being hired for his or her special expertise or whether it is to augment your staff.

In the interest of full disclosure, please visit the About Me page in this blog to learn about my background and the services I provide.