The ETL Layer


Bizcovery Foundation features an adaptive layer for robust data extraction, transformation and load, or ETL, capabilities, including data cleansing and data mapping. Through the robust connectivity layer, adapters to e-business, financial and customer data may be "melded" to achieve a complete business view of an entire enterprise.

Powerful ADP (Analytic Design Platform)

Building a complex analytic application that can provide business view from all angles for various subjects requires

·         pulling data from multiple data sources

·         building complicated data transformations for both at the column level and at the table level

·         team work for co-development

·         debugging for high level issues and issues at deep sub-query and transformation level

·         provide tracing for where data comes from and where data flows to

·         version control and comparison for different revisions of applications

·         code review and performance tuning

·         deployment to heterogeneous environment to support multi-tier Dev/QA/Production IT infrastructures

·         faster time-to-market with incremental development and deliveries to allow narrowing the gap between technical deliveries and business requirements

Bizcovery Foundation ADP is designed to solve all above mentioned issues and is built on top of a very rich metadata layer.

Adapters. The adaptive ETL layer allows integration with key enterprise, e-business systems, and customer data sources. Bizcovery Foundation may be integrated with e-commerce systems such as those offered by BroadVision, InterWorld, Vignette, Open Market and Microsoft Site Server; customer interaction systems such as those offered by Oracle, Siebel, PeopleSoft, Clarify, Onyx, Saratoga, Scopus, Aurum, and Pivotal; enterprise resource planning applications such as those offered by SAP, Oracle, PeopleSoft, Baan and JD Edwards; custom, legacy and homegrown applications and systems; external demographic data providers such as Acxiom and Dun & Bradstreet; and leading enterprise data warehouses such as those offered by Oracle, Sybase, Informix, IBM and NCR. Bizcovery also provides the ability to integrate data from legacy systems and custom systems including flat files and spreadsheets.

Library of ETL Transformations. The Bizcovery ETL solution incorporates Java classes to transform data and automate processes. Because transformations are not coded as stored procedures, which are specific to particular databases, the ETL solution may be used with any database. Thus, the Java based transformations allow for a completely open ETL solution.

Extraction and Pooling of Data. Bizcovery ETL transformations are not only used to extract data from operational databases. The transformations are also used to "pool" data from these systems in a staging area (staging tables) located in the application data store.

Historical Reporting. Bizcovery can incorporate historical data tables in the application data store in order to capture historical trends and to allow for historical reporting of information. These tables pool historical information from the various operational systems in order to give users a complete view of the customer life cycle.

Incorporating Real Time Data for Real Time Decision. In Bizcovery, because transformations are not coded as stored procedures, decisions and alerts based on both real time and historical data are possible.

Data Cleansing and Management. As the transformations extract data they also serve an important data-cleansing role. Bizcovery eases the reformatting and cleansing work for the data extracted from operational systems, so that the data may be properly analyzed in the Bizcovery system. In addition, semantic changes are also made to data columns. For example, a column in a sales system named "NL-Sales" would be automatically translated to a more easily understandable title, "Actual-Sales".

Notification on Content Changes. In a lot of BI/Analytic applications, business rules may need to be modified if the data content in certain OLTP tables varies. Using transformations provided in Bizcovery’s ETL engine, combination of set operations and E-Mail notification can act as an agent to detect content changes in OLTP tables.

Bulk and Incremental Loads. Bizcovery provides the ability to perform bulk and incremental data loads. Bulk loads are commonly performed for the first extraction of data from operational systems. After the first bulk load extraction is performed, incremental loads may be performed. Incremental loads only extract new data and are therefore faster and more efficient. Bulk load and incremental load functionality provided by Bizcovery ensures efficiency of the on-going data extraction process.

Scheduling and Triggering Extraction. The Bizcovery integrated management environment allows customers to schedule ETL operational system extractions on a weekly, daily, or hourly basis. Actions performed by users in the Bizcovery system may also be directed to trigger a data extraction. The Bizcovery ETL transformations handle all of this functionality.

Bizcovery's unique and powerful adaptive ETL provides businesses with the flexibility to integrate other data sources and systems as businesses change.


The ETL Layer Open Data Store Application Server Integrated Management