Data Virtualization

Report
On Demand Data Integration
with Data Virtualization
David Besemer, CTO
Agenda
State of Enterprise Information
The Case for Data Virtualization
How Data Virtualization Works
Data Virtualization Adoption Patterns
2
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
The State Of Enterprise Information
More demanding business users
 Competition drives faster time-to-information
 Younger staff want more “do-it-yourself”
 “IT’s challenges are not my problem.”
Information overload
 Exponential data volume growth
 Omnipresent delivery
“Over the top” IT complexity
 New sources, uses, and enabling technology
 Layered on byzantine IT infrastructures
3
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Data Management Trends
Changing role of the Data Warehouse
 Data warehouse no longer viewed as only focal point for
all data integration
Lower latencies required
 Information needs moving toward real time
Rising “fit-for-purpose” storage and processing
 Appliances, MPP, NoSQL
Data Quality being addressed at every layer
 Source, Consolidation, Virtual, and Visualization
Clouds are approaching…
 Most enterprises looking to leverage cloud computing
4
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Agenda
State of Enterprise Information
The Case for Data Virtualization
How Data Virtualization Works
Data Virtualization Adoption Patterns
5
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
The Challenge
Business
Solutions
Constant
Change
BI, CPM, and
Reporting
Portals and
Dashboards
Custom and
Composite Apps
SOA
Initiatives
Data
Integration
Challenge
Source
Data
Siloed
& Rigid
Files
6
Big Data
Packaged
Applications
RDBMS
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Web
Services
Traditional Physical Data Consolidation
Business
Solutions
BI, CPM, and
Reporting
Physical
Intermediate
Stores & ETL
Middleware
Portals and
Dashboards
Physical
Data Marts
Custom and
Composite Apps
Enterprise Data
Warehouse
SOA
Initiatives
Physical
Operational
Data Stores
Source
Data
Files
7
Big Data
Packaged
Applications
RDBMS
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Web
Services
Traditional Physical Data Consolidation
Physical consolidation
Business
SolutionsForces the business to wait
CPM,solutions
and
Portals and
longerBI,Reporting
for
Dashboards
More silos & complexity
Slows future IT progress
Custom and
Composite Apps
SOA
Initiatives
Wait, wait, wait!
Batch integration
Uncontrolled data replication
PhysicalDelay real-time information
Intermediate
Stores & ETL
On-Demand
Data
Physical
Customer X
Enterprise Data
Data
Marts
Middleware Invoice
Warehouse
UNPAID
Source
Data
Batch Data
Files
8
Customer X
Invoice
PAID
IN FULL
Big Data
Packaged
Applications
 Reduced data quality
 Significant hidden costs
$$$$ $ $ $$ $
$ $ $ $ $$ $
Physical
Operational
Data Stores
RDBMS
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Web
Services
Data Integration Architectures and Patterns:
Build a Portfolio to Address the Range of Needs
BI Tools/Apps.
Master Data Mgmt.
Common Design, Admin., Governance
Physical
Movement and
Consolidation
(e.g., ETL)
Operational Apps.
Abstraction/Virtual
Consolidation
(Data Federation)
Interenterprise
Change-Capture
and Propagation
(Replication or
Messaging)
Common Metadata (Location, Format, Structure, Quality, Meaning)
Common Connectivity (Full range of source/target types)
Leading organizations support multiple styles of data
integration and delivery to address a range of business
requirements — breadth enables leverage and agility.
How Data Virtualization Differs
Physical Movement
and Consolidation
(ETL, CDC)
Abstraction / Virtual
Consolidation
(Data Federation)
Synchronization
and Propagation
(Messaging)
Middleware
ETL
CDC
Data Virtualization
EAI / ESB
Purpose
DB  DB
DB  DB
DB  Application
Application  Application
Attribute
Scheduled
Event
Driven
On Demand
Event
Driven
10
©
Software,
Inc. /Inc.
Composite
Proprietary
and Confidential
© 2010
2011Composite
Composite
Software,
/ Composite
Proprietary
and Confidential
Traditional Physical Data Consolidation
BI, CPM, and
Reporting
Physical
Intermediate
Stores & ETL
Middleware
Physical
Data Marts
Files
11
Portals and
Dashboards
Big Data
Custom and
Composite Apps
Enterprise Data
Warehouse
Packaged
Applications
SOA
Initiatives
Physical
Operational
Data Stores
RDBMS
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Web
Services
Data Virtualization Increases Agility
BI, CPM, and
Reporting
Portals and
Dashboards
Data
Virtualization
Virtual
Data Marts
Physical
Intermediate
Stores & ETL
Middleware
Physical
Data Marts
Files
12
Big Data
Custom and
Composite Apps
Enterprise
Search
Virtual
Operational
Data Stores
Enterprise Data
Warehouse
Packaged
Applications
Physical
Operational
Data Stores
RDBMS
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Web
Services
Shared Data Services & Relational Views Further
Extend Flexibility and Agility
BI, CPM, and
Reporting
Portals and
Dashboards
Custom and
Composite Apps
SOA
Initiatives
Virtual Data Layer
Composite
Information
Server
Virtual
Data Marts
Physical
Intermediate
Stores & ETL
Middleware
Physical
Data Marts
Files
13
Big Data
Web Data Services
& Relational Views
Virtual
Operational
Data Stores
Enterprise Data
Warehouse
Physical
Operational
Data Stores
Packaged
Applications
RDBMS
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Web
Services
A Complete Data Integration Architecture
BI, CPM, and
Reporting
Portals and
Dashboards
Custom and
Composite Apps
SOA
Initiatives
Virtual Data Layer
Virtual
Data Marts
Virtual
Operational
Data Stores
Shareable Data Services
& Relational Views
Physical Data Consolidation Layer
Physical
Data Marts
Files
14
Big Data
Enterprise Data
Warehouse
Packaged
Applications
Physical
Operational
Data Stores
RDBMS
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Web
Services
Forrester Data Management Reference Architecture
15
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Agenda
State of Enterprise Information
The Case for Data Virtualization
How Data Virtualization Works
Data Virtualization Adoption Patterns
16
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
How Data Virtualization Works – Example Scenario
1) I need to build an
application that
looks like this…
2) The view or data
service needs to
look like this…
3) And the data
comes from these
sources, in these
formats…
17
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Data Discovery and Design
Design Steps
1.
2.
3.
4.
Discover data and relationships
Model individual view/service
Validate view/service
Composite Information Server
Modify as required
Discovery
Benefits
Studio
Faster time to solution
Easy to learn and use
Extensible / reusable objects
18
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Data Virtualization Production
Production Steps
1. Application invokes
request
2. Optimized query (single
statement) executes
3. Deliver data in proper form
Composite Information Server
Optimizer
Benefits
Up-to-the-minute data
High performance
Less replication required
19
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Data Virtualization Production with Caching
Production Steps
1. Cache essential data
2. Application invokes request
3. Optimized query (leveraging
cached data) executes
4. Deliver data in proper form
Composite Information Server
Optimizer
Benefits
Removes network
constraints
7-24 availability
Optimal performance
20
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Cache
Agenda
State of Enterprise Information
The Case for Data Virtualization
How Data Virtualization Works
Data Virtualization Adoption Patterns
21
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Data Virtualization Adoption Patterns
Data
Federation
DW Extension
Data
Virtualization
Layer
Big Data
Integration
Cloud Data
Integration
22
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Data Federation for Business Intelligence
Project Manager
“My application
requires data from
multiple
incompatible
sources.”
23
Data
Federation
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Data Warehouse Extension for 360o View
Data Warehouse
Owner
“My data
warehouse does
not contain all the
data required for
the reports we
need to build.”
24
Data Warehouse
Extension
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Data Virtualization Layer for Business & IT Agility
IT Director
“How do I build an
agile data layer for
easy data access
and delivery.”
25
Data
Virtualization
Layer
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Cloud Data Integration for IT Extensibility
CIO
“I need to integrate
data between onpremise systems
and applications
running in the
cloud.”
26
Cloud Data
Integration
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Big Data for Analytics
Business Analyst
“More and more of
my data now lives in
MPP and Hadoop
sources. How do I
combine big data
with traditional data
for analysis?
27
Big Data
Integration
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Data Virtualization Adoption Patterns
Data
Federation





DW Extension
Data
Virtualization
Layer
Semantic Abstraction
Federated Query
Loose Coupling
Caching
Location Independence
=
Data
Virtualization
Big Data
Integration
Cloud Data
Integration
28
© 2011 Composite Software, Inc. / Composite Proprietary and Confidential
Composite Software Contact
For more information please contact:
Pamela Sotnick
Director, Federal Accounts
Mobile 240.460.9566
[email protected]
Katy Mann
Director, Federal Accounts
Mobile 301.452.7042
[email protected]
David Besemer
CTO
[email protected]
29
Questions

similar documents