Data Warehousing Appliances – Concept and Benefits

Data warehouse appliance consists of an integrated set of servers, storage, operating system, DBMS and software specifically pre-installed and pre-optimized for data warehousing (DW). Alternatively, the term can also apply to similar software-only system — purportedly very easy to install on specific recommended hardware configurations. DW appliances provide solutions for the mid-to-large volume data warehouse market, offering low-cost performance most commonly on data volumes in the terabyte to petabyte range. (Ref:Wikipedia) Data warehouse appliances are expandable, affordable and uniquely suited to the ever-growing needs of users, providing orders of magnitude performance improvements in terms of speed and sophistication of data analysis.

The current BI infrastructure is a patchwork of hardware, software and storage that is growing ever more complex. In the absence of significant architectural improvements, the traditional answer to the growing BI problem is to continue to add more hardware. This basic patchwork paradigm has not changed during the past 10 years. These types of systems require data and user applications to be continuously tuned and optimized. Terabyte-scale databases that continue to grow steadily put tremendous strain on these systems. Even in cases where the user base and data set are relatively stable, current BI systems often fail to meet their basic goal of delivering vital business information in order to make timely decisions. From an administration standpoint, this current patchwork of solutions is extremely difficult and time-consuming to manage and maintain. From the business-user point of view, it is frustrating and does not provide the agility and performance the users are looking for. These strains occur because vendors have upgraded these systems incrementally over the years rather than change the underlying architecture to address the unique requirements of today’s terabyte-scale databases.

Importance of Data warehousing Appliances .The data warehouse appliance is designed specifically for the streaming workload of BI and is built using commodity components. It architecturally integrates hardware, DBMS and storage into one opaque device and combines the best elements of SMP and massively parallel processing approaches into one that allows a query to be processed in the most optimized way possible. A data warehouse appliance is architected to remove all the bottlenecks to data flow so that the only remaining limit is the disk speed – a data-flow architecture where data moves at streaming speeds. Through standard interfaces, a data warehouse appliance is fully compatible with existing BI applications, tools and data. It has an extremely low total cost of ownership and is very simple to use.

The benefits: More reporting and analytical capabilities If a data warehouse appliance executes queries, it should be able to handle a bigger query workload, which means more queries, and more complex queries. Unplanned queries might even become an option. In addition, those queries that were once forbidden (too much resource consumption), can now be executed. This means that the organization can run more reports, it can do more complex data analysis, it can work with bigger datasets, and it can analyze more detailed data. In short, a data warehouse appliance offers more reporting and analytical capabilities, and that might improve the quality of the decision-making process.
Cost reductions: A data warehouse appliance requires a minimal amount of tuning and optimization of the server, of the database server, and of the database design. The data warehouse appliance should be able to run most queries very fast. In fact, this is what a number of organizations experienced when they migrated their data warehouses, implemented with a classic database server that they had been optimizing for years, to a data warehouse appliance. Most of them experienced a performance improvement without any tuning and optimization.

Less time has to be spent on tuning and optimization of the server, the database server, and the database design. The value of this should not be underestimated. A classic database server requires a lot of optimization and tuning before an acceptable performance is achieved. It has to be invested a lot of time spent on discussing where to create indexes, how to partition tables, what the values of the tablespaces parameter should be, how to set the bufferpool parameters, and so on. And don’t forget all those hours discussing whether to denormalize the tables, or change from a star to snowflake design or vice versaSo, the data warehouse appliance could reduce costs considerably. And these costs can seriously outweigh the costs of buying a data warehouse appliance.

Flexibility: An additional advantage is that if less tuning and optimization is needed, it will be easier to implement new user requests. With other database servers, a new query might lead to quite a number of technical changes, such as creating and dropping indexes, repartitioning tables, changing bufferpool parameters, and so on.

The success of decision making in a company relies on business intelligence. BI, in turn, relies on the underlying database architecture. Current database architectures are patchwork systems, built in pieces and not optimized for delivering timely results. The maturity and stability of the relational database, paired with the power of commodity components, allows for a breaking down of the database system. A new generation of data warehouse appliances holds promise for companies that depend on business intelligence.

References : Advantages of DWA ,Rick van der Lans, BeyeNetWork

Teradata documentation,

TwinFin Netezza documentation

Article by Foster D Hinshaw, Information Management Magazine Sept 2004


One thought on “Data Warehousing Appliances – Concept and Benefits

Comment please...

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s