You’ve implemented a cloud data warehouse. Good. Employees are adding data and sharing it. Great. Your cloud DW vendor sends you a monthly bill and it’s higher than you budgeted. Uh oh. Welcome to the cloud data warehouse paradox. Balancing the dual priorities of encouraging data sharing and containing cloud data warehouse costs isn’t easy. Over-emphasizing one leads to sub-optimizing the other. Or does it? 

Many organizations have implemented the practice of “Cloud FinOps” to manage and reduce their cloud costs. This includes the costs of cloud data warehouses. Reporting to finance and sitting within operations, these teams are tasked with keeping cloud spending costs under control. Initially, many have good results. Identifying the low-hanging fruit can result in 30% cost reduction, according to Forrester Research. Post migration cloud data warehouse costs are easy to identify and reduce. Temp files sitting around. Copies of instances. Wrong-sized dev environments. Some shutdown actions are easy by identifying old or ownerless cloud instances. Those are zero-conflict situations. But recent instances with valid owners are not so easy to shut down. And right-size actions are always met with some resistance. Why? Conflict. Each of those scenarios is based on you asking people to change and give-up something. The next level of cloud data warehouse efficiency is more difficult to realize. 

When it comes to FinOps, data-centric clouds are more challenging than other types of cloud instances. Cloud data warehouses are the prime culprits. Why? People hoard data. They may not use it, but they think it’s valuable. So they hold onto it for as long as they can. A copy of data might have a few manual modifications made for one analytic use case. That’s all the justification a data hoarder needs to claim they still need that file. Who’s to say what’s valid and what’s redundant? 

That’s where data-centric FinOps comes in. Managing cloud data warehouse costs comes down to striking a balance between the needs of business users and data owners vs. financial objectives. There are three essential characteristics of a cloud data warehouse FinOps team that produce ongoing efficiencies. 

1 – Holistic Understanding of Data, Analytics and Business Functions

The cloud dw FinOps team needs to understand data. Specifically, where it comes from and what its intended uses are. You have to be a data-person to manage data requests. Expertise in cloud data warehouses, usage of data warehouses and marts, temporary files, ad-hoc sandboxes, data lineage and processing, and temporal right-sizing for various environments (production, dev, QA). Temporal right-sizing requires expertise in data patterns. Data surges. Inquiry spikes. Is that a run-away query or a valid one-off analysis? When should it be performed? The data expertise required for cost reduction is multi-faceted.

The team also needs expertise in analytics to determine how much data is required to build and operate advanced analytic and AI models, and for how long that data is needed. And of course, that same team needs expertise in business functions. What are data and analytics used for? What’s driving the request and how long will it be needed? Are business users reading reports and benefiting from the analysis? Ultimately this team needs to make a business judgment between the potential benefit of the data usage and its cost. 

2 – Neutrality

The data team needs to be neutral for many reasons. The first is quite obvious – to make good decisions from a balanced perspective. The second is less obvious, but more important. Buy-in. If the decision-making is perceived by data owners and users to be tough-but-fair, then it is much more likely that they will abide by the decision. You can’t have finance alone dictating costs and constantly asking for reductions without data and business context. You can’t have data-leaders prioritizing requests as they too-often will favor data usage vs. cost reduction. You need neutrality as well as the strong perception of neutrality in order to have an effective cloud data warehouse FinOps program.

3 – Drive Adoption via Change Management

Cloud Data Warehouse FinOps is a change management program! Your cloud DW FinOps team needs to not only make decisions, but help users adopt them. This isn’t just a decision-making body. It’s a change management team. That is where a lot of programs stumble. They think it is as easy as setting rules, publishing them, making decisions, and policing them. It isn’t. Data owners and users will figure out the rules, and if they don’t agree, they will figure out ways around them. There are parallels to other data-centric practices. Take Data Governance, for example. Once upon a time, data governance was seen as a compliance exercise. Rules for use would be written down, published to a shared directory, and everyone would read them and abide by them. Did they? Of course not. Data governance didn’t evolve and become more effective until the people leading it evolved from “people who say no” to “people who understand what the business is trying to do and try to find a way for data to make it possible.” The evolution of cloud dw FinOps will follow the same trend. Why not skip the learning curve and just start out with the principle that “Cloud Data Warehouse FinOps is first and foremost a CHANGE MANAGEMENT EXERCISE.”

The Cloud Data Warehouse Paradox doesn’t need to be a blocker for you. It isn’t really a paradox. It’s only a problem if you take a simplistic approach to cloud data warehouse FinOps. But if you take a more evolved approach, one that creates a team with holistic expertise, neutrality, and a change management mindset, you will solve the paradox.