Data warehouses (DWHs) are widely recognized as essential components of business intelligence and analytics operations. But the question of whether the optimal deployment route is in the cloud or on-premise remains hotly debated.
Like so many things, the truth is that there is no one-size-fits-all solution. Every business is different, and there are advantages and disadvantages in both approaches. On the one hand the cloud offers scalability and low entry cost advantages. On the other, there’s the security and flexibility that only an on-prem solution can offer.
Should you deploy your data warehouse in the cloud or maintain it on-premise? Here’s our take on the factors to consider as you make this decision.
The great advantage of taking the cloud route over the on-prem solution is that scaling up can be accomplished easily and effortlessly.
Scaling up on-prem systems is a time-consuming and resource-intensive task, as it usually entails purchasing and installing new hardware. Conversely, data held in the cloud can be scaled up or down instantly and with virtually no hassle.
As with any other SaaS offering, cloud-based data warehousing offers sizable cost benefits by eliminating heavy upfront costs. No hardware, server rooms, IT-related staffing issues, or operational costs to maintain your DWH.
It’s no wonder that more and more enterprises are moving their DWHs to the cloud. In fact, a survey of 1,060 IT professionals for the T2016 Cloud Computing Trends Report revealed that 95% of respondents are using some form of cloud infrastructure this year, an increase over the 93% reported in 2015.
In the long run, though, if the scale of data, processing, and the transfer rate of data becomes very large, on-premise DWH is likely to offer cost advantages.
The “cloud” is a virtual entity, but in reality it’s implemented based on servers that are based in specific geographic locations. Often the ‘Cloud DWH’ will offer you a service that includes multiple locations for redundancy and improved performance.
Sometimes the distance the data has to travel from ‘the cloud’ to the client takes time that has an unacceptable impact on the business. In such cases, on-premise DWH might be the better solution because latency and speed can be managed better locally.
On the other hand, if the business needs to serve multiple locations around the world and provide fast turnaround, the cloud is specifically designed to meet these needs: it can easily and quickly serve multiple geographic locations with replication capabilities between them.
Comparisons of speed performance between the cloud and on-premise solutions are based on measurements in milliseconds. If the speeds required by your business are measured in seconds, then both options can provide excellent performance.
With a cloud-based DWH, it's easy to connect to other cloud services. There are many services that make it easier to digest the data, store it in file systems, and access it.
For example, cloud ETL tools allow you to integrate a huge variety of data sources based on ready-made “connectors” and transform and manipulate the data easily for analytics.
An on-premise DWH enables the organization to have absolute control over security, how and when applications interact with each other, and other connectivity or access issues. In sectors where these kinds of restrictions are critical, such as banking or government, on-premise DWH is the more common choice.
Adoption of cloud-based solutions of all kinds has been hampered by fears about reliability. A study by IDG Connect found that 57% of industry leaders believed that concerns about reliability and availability of network bandwidth are a barrier to adoption.
However, the advantage of cloud-based DWHs is that they’re always available – depending on the provider and their SLA.
For example, Amazon promises a minimum uptime of 99.95% availability for their EC2 DWH service, and Google a monthly uptime percentage of 99.9% for Cloud Storage and BigQuery. Both Google and Amazon, as well as other cloud DWH providers, replicate your data across multiple clusters to ensure maximum reliability.
For on-prem data warehouses, reliability is a function of the quality of hardware and staff at your disposal. The maintenance, management, and customization of an on-prem DWH is entirely in your hands.
As with connectivity, security is one of the biggest challenges that cloud service providers have to face when pitching to their prospective clients.
There is still a lot of concern over the security of data stored in the cloud rather than on-prem. That said, some CIOs are beginning to see cloud computing as a more secure environment than hosting on local machines.
Their reasoning is that because cloud data warehouse providers’ entire business model relies on data security and encryption, they may be better at it than you are. These companies invest heavily in security technology and dedicate entire departments to the protection of your data.
For example, Google BigQuery and Amazon Redshift both have swaths of security features to guarantee the safety of your data at every points in its journey.
On the other hand, if your business requires very specific security measures, that can not be met by the security offering of the cloud supplier, than you might need to go for a tailor made on-prem solution.
Most companies will benefit greatly by deploying a cloud-based data warehouse, as it is cost-effective, quick to set up, instantly scalable, accessible, easy to use, and secure. Delegating the maintenance and management of a data warehouse to a third party will free up valuable time and resources that can be used for analytics or other activities critical to your business.
Still, companies that require total control, flexibility, accessibility, and predictability might find that an on-prem solution is a better fit for their needs.
If you are still unsure which option is the best fit for your needs, you could also opt for a hybrid approach, storing your data in an on-prem data center and using the cloud for data processing and analytics. Alternatively, store your data in a cloud data warehouse and perform analytics on-prem.
Learn more about how your business could benefit from Xplenty’s simplified data integration service.