It’s been a year since Gartner removed ‘big data analytics’ from its Hype Cycle. The reason was simple, as stated by the CEO Betsy Burton of Gartner Research, “Cloud, big data, etc. are few technologies which have moved pretty fast to the Plateau of Productivity. CIOs and CEOs now understand the uses and strategies which work and may not work with these…”
We now know that infrastructure plays a critical role in the successful integration of big data analytics into the organization’s strategic performance. The data analytics infrastructure, for any organisation, must support the following keys to success in the big data space:
- Faster Time to Value: Your big data analytics system must offer insights on collected data in minimal time, even real time if possible
- Capture to the Omni-Channel Data: You need your analytics suite to provide you insights from multiple sources fluently; i.e. Hadoop data, sensor and device data, social media and even transaction data all at one place
- Real Time Insights/Predictive Analytics Availability: Insights from the big data analytics system should be available across domains within the organisation to the right people, at the right time, for better decisions at every level
Well, if you are thinking of setting up the data analytics solution on premise, you could be in for a shock. Setup and maintenance of on-premise data analytics infrastructure with all the aforementioned features has prohibitive setup and maintenance cost. For many organizations, security challenges pose a bigger threat than the cost of the setup.
Figure 1: On Premise vs. Cloud Data Analytics, Break-up of Costs Involved
Because of these issues, businesses are increasingly choosing cloud over on-premise data analytics infrastructure. At least the revenue trends clearly indicate it, “In 2015, the market for on-premise data analytics grew only 1.2% while the revenues of public cloud services grew 38.5% marking the start of the cloud era.”
What is driving this change?
There are many reasons apart from the cost, although most alternatively lead to cost increments:
“Expectations are now turning to the cloud as an alternative deployment option, because of its flexibility, agility and operational pricing models.”
- Scalability
- Collaboration and Agility
- Reliability
- Maintenance and Security
Expectations are now turning to the cloud as an alternative deployment option, because of its flexibility, agility and operational pricing models.”
Gartner
Scalability
When you are planning for a system that can capture multi-channel (or omni-channel) data, scalability should be your major concern. Just to build the perspective, by the end of 2014 Facebook was collecting approximately 600 TB of user data per day. The 5000 sensors installed on one Pratt & Whitney’s Geared Turbo Fan (GTF) engine generate 10 GB data per second, round it to a 12-hour flight of a twin engine aircraft it will be approximately 844 TB.
This is just one perspective, for example, Facebook is one social channel, while the sensors are another isolated channel. Imagine collecting data from multiple such channels at one place.
Will it be possible to continuously scale up the data capacity? You will also need to continuously upgrade computation capability.
Scalability is something where cloud excels. In fact, the cloud offers the best alternative for organizations with uncertain computing and storage demand pattern for their data.
Collaboration & Agility
Collaboration and Multi-tenancy are one of the important features of cloud based solutions. It implies that the even a single deployment caters to multiple geographies, languages, price points, charging mechanisms, time zones, and currencies, etc.
Implementation of cloud based analytics solution by ABN AMRO bank has channelized an active route for business through big data analytics practices. The multi-tenancy factor has helped in accelerated development for more than 4,000 engineers and IT professionals.
Matrimony.com which is into the business of matchmaking has cited the true benefits of big data analytics in collaborating the data. By capturing the customer data from multiple channels including emails, SMS’s, banner, telesales and retail centres, the firm uses cloud-based big data analytics solutions for driving personalized marketing campaigns to match potential partners faster and attract more subscribers.
The insights from big data on collaboration, benefits all levels of the organization. Senior executives gain tremendous insight into the way the organization is working while accessing to data as it changes over time.
Employees are benefitted by having better access to data insights enabling better time allocation. This has led to a more productive conversation between managers and employees about the struggle to stay focused on priorities. Thus, the cloud offers reduced latency over on-premise systems for offshore users.
Reliability & Maintenance
The reliability and maintenance offered by cloud-based big data solutions have led to its tremendous use in the e-commerce world today. Owing to the data analytics solution provided, Flipkart analyzes 25 million rows of inventory data every day. Similarly, Snapdeal and HomeShop18 have claimed to generate about 30-40% of their orders due to the reliability on the big data tools.
Cloud-based data warehouses are almost always available. For example, two of the world’s largest cloud service providers (Amazon and Google) offer a minimum up-time of more than 99.95%. Amazon for EC2 Servers and Google for Cloud Storage and BigQuery.
Security
When Matt Reidy, director of IT operations at SnagAJob.com, embarked on a goal to move from a 75 percent virtualized environment to a 100 percent virtualized, ‘security’ was a major concern. However, the company successfully achieved a 100 percent virtualized and secure environment, and while doing so, he cited that “Cloud security is a shared responsibility between the provider and the client”.
SaaS (Software as a Service) providers, who supply analytics and storage software to the cloud vendors, keep the software updated. For them, it is a continuous fight with the hackers, and since their entire business and brand value depends on the safety of the storage, they put a lot of effort on maintaining the security.
In fact, most of the recent data breaches in U.S., Bankrate reviewed, occurred on the on-site facilities rather than cloud-systems.
Often it is hard for firms to keep their private data isolated, especially when they are large, and their decision centres located in different geographies. Also, legacy tech which has not been updated for a while is more prone to a successful hack attack.
Cost
Implementation of the big data analytics solution by Kerala Water Authority (KWA) has completely changed the way in which water distribution of Thiruvananthapuram was managed. The analytics solution has revolutionized the way of tracking water meters across the city’s consumption, thereby reducing billing anomalies and improving revenue collection by more than 10%.
The cost factor is perhaps the greatest contributor to the growth of cloud usage for data analytics. There is zero money spent on infrastructure, manpower or the software development.
Thus, you only pay a subscription fee based on the capacity utilization for storage and processor.
Customers can spend up to four times the cost of their software license per year to own and manage their applications. – Gartner
Cloud providers also have multiple pricing models, which you can negotiate along with the SLA. There are many static or dynamic pricing models to choose from:
- Pay as you Go
- Subscription Based
- Pay for Resources
- Value Based Pricing
- Cost based Pricing
- Hybrid Pricing
Conclusion
The cloud infrastructure serves the business proactivity better than the physical products and hardware. Best solutions often come in the package. For example, cloud data analytics is not an isolated or ‘one true solution’ for data warehousing and analytics. Instead, the organizations must figure out the ways to maximize value and balance between security, cost, and efficiency of their data analytics systems.
Every organization is bound to get the different set of benefits from such systems. The solution for you may include a mix of on-premise, private cloud, and public cloud systems. Getting on the right track with respect to a well-planned transition holds the key.
