cassandra

Report
Cassandra in xPatterns
Seattle Java User’s Group
May 2014
1
Atigeo Confidential
Agenda
• xPatterns Architecture
• Export to NoSql API & GUI (Demo)
• xPatterns dashboard application (Demo)
• xPatterns monitoring and instrumentation (Demo)
• Data model optimization
• Publishing from HDFS/Hive/Shark to Cassandra
• Generated REST API’s

Instrumentation

Throttling & auto-retries
• Geo-Replication

Cross-data-center replication, encryption & failover
• Lessons Learned since 0.6 till 2.0.6
2
Atigeo Confidential
3
Atigeo Confidential
4
Atigeo Confidential
Export to NoSql API
•
Datasets in the warehouse need to be exposed to high-throughput low-latency real-time
APIs. Each application requires extra processing performed on top of the core datasets,
hence additional transformations are executed for building data marts inside the
warehouse
•
Exporter tool builds the efficient data model and runs an export of data from a Shark/Hive
table to a Cassandra Column Family, through a custom Spark job with configurable
throughput (configurable Spark processors against a Cassandra ring) (instrumentation
dashboard embedded, logs, progress and instrumentation events pushed though SSE)
•
Data Modeling is driven by the read access patterns provided by an application engineer
building dashboards and visualizations: lookup key, columns (record fields to read), paging,
sorting, filtering
•
The end result of a job run is a REST API endpoint (instrumented, monitored, resilient, georeplicated) that uses the underlying generated Cassandra data model and fuels the data in
the dashboards
•
Configuration API provided for creating export jobs and executing them (ad-hoc or
scheduled).
5
Atigeo Confidential
6
Atigeo Confidential
Mesos/Spark cluster
7
Atigeo Confidential
Cassandra multi DC ring – write latency
8
Atigeo Confidential
Nagios monitoring
9
Atigeo Confidential
10
Atigeo Confidential
Referral Provider Network
•
One of the many applications that we built for our largest healthcare customers using
the xPatterns APIs and tools on the new upgraded infrastructure: ELT Pipeline, Jaws,
Export to NoSql API. The dashboard for the RPN application was built using D3.js and
angular against the generic api published by the export tool.
•
The application allows for building a graph of downstream and upstream referred and
referring providers, grouped by specialty, with computed aggregates like patient counts,
claim counts and total charged amounts. RPN is used for both fraud detection and for
aiding a clinic buying decision, by following the busiest graph paths.
•
The dataset behind the app consists of 8 billion medical records, from which we
extracted 1.7 million providers (Shark warehouse) and built 53 million relationships in
the graph (persisted in Cassandra)
•
While we demo the graph building we will also look at the Graphite instrumentation
dashboard for analyzing the runtime performance of the geo-replicated Cassandra read
operations (latency in the 20-50ms range)
11
Atigeo Confidential
12
Atigeo Confidential
Graphite – Cassandra multi DC ring
13
Atigeo Confidential
VPC-to-VPC IPSEC Tunnel
14
Atigeo Confidential
Lessons learned 0.6 - 2.0.6
• NTP: synchronize ALL clocks (servers and clients)
• Reduce the number of CFs (avoid OOM)
• Rows not too skinny and not too wide (avoid OOM)
o Less memory pressure during high-throughput writes
o Reduced network I/O, less rows, more column slices
o Key cache & bloom filter index size affects perf
o Efficient compaction, avoid hot spots
• Custom serialization and dynamic columns for maximum perf gain
• Do not drop CFs before emptying them (truncate/compact first)
• Monitoring, instrumentation, automatic restarts
• ConsistencyLevel: ONE is best … for our use cases
• Key cache, Snappy compression, vnodes
15
Atigeo Confidential
Q&A
Oh btw … we’re hiring!
16
Atigeo Confidential
© 2013 Atigeo, LLC. All rights reserved. Atigeo and the xPatterns logo are trademarks of Atigeo. The information herein is for informational purposes only and represents the current view of Atigeo as of the date of this
presentation. Because Atigeo must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Atigeo, and Atigeo cannot guarantee the accuracy of any information provided
after the date of this presentation. ATIGEO MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
18
Atigeo Confidential

similar documents