This is part two of the Cloud Reference Architecture (CRA) blog series and here I am going to guide you on how to achieve cloud financial governance in the cloud. [Check out part 1 – Cloud Reference Architecture or CRA – Foundation].
Establishing financial government in the cloud is the basis of establishing a cloud reference architecture as it helps you manage cost of your cloud spending and plan your budget by establishing accountability and cost monitoring measures. Read more in this blog series.
Note: if you want to learn more about the cloud reference architecture, cloud security and cloud migration, then make sure to check my published book here.
In fact, all what is discussed in this blog series is just scratching the surface of what I am covering in the Cloud Migration Handbook. The book covers other major topics for architects and security professionals such as:
- Chapter 1: Practical Foundations for Cloud Computing.
- Chapter 2: Types of cloud migration.
- Chapter 3: Cloud Governance.
- Chapter 4: Cloud Reference Architecture (CRA).
- Chapter 5: Security in the Cloud.
Previous Talk: Cloud Reference Architecture (CRA)
In a previous blog post, we’ve introduced the concept of the cloud reference architecture(CRA) as defined in ISO/IEC 17789 standard, and why you should consider having one. The end result of a cloud reference architecture is to achieve a balance between cloud agility from one side, and security and governance from the other side.
Simply put, the Cloud Reference Architecture (CRA) helps organizations address the need for detailed, modular and current architecture guidance for building solutions in the cloud.
We’ve also covered that the Cloud Reference Architecture (CRA) serves as a collection of design guidance and design patterns to support structured approach to deploy services and applications in the cloud. This means that every workload is deployed with security, governance and compliance in mind from day one.
To accomplish this, we need to define the components of the cloud reference architecture (we call it an Enterprise Scaffold) that we will use to build secure, compliant and flexible framework that developers can build application on top with agility and speed of delivery in mind.
We also covered the the foundation layers of the enterprise scaffold that contains key components including networking, security, cost management, deployment essentials and more.
In this blog post, we are going to start unfolding another layer of the cloud reference architecture which is the cloud financial governance layer and how this can help you better manage your cost, plan your budget, and establish financial accountability in the cloud, and why all this matters to you.
Defining an Enterprise Structure
So what does enterprise structure really means? And how is it that spending time planning head can save you a lot of time and effort later on, and how this would help you achieve that balance between agility from one side and security and governance from the other side. In fact, doing it right from the beginning can save you a lot of work later and gives you visibility across all your deployed resources.
Let me ask you one thing, if you are going to move an application to the cloud, what would be the first thing you would do? Of course, we all hear the term lift and shift, just create couple of VMs in the cloud, and put your workloads there, and then worry about governance, cost management security later on. Well, you can do that, but it is very hard to establish security and governance after you have your workloads deployed in the cloud.
The right way is to spend some time imagining how your workloads will be deployed, how cost is going to be managed, how to plan for hybrid connectivity and how security is going to be one of the main deciding factors for your design.
That’s what the enterprise structure is all about. It helps you plan a head how you are going to establish financial accountability in the cloud so that you can track back every cloud workload to a cost center. It helps you answer key questions like how you are going to group similar resources in logical groups to facilitate better access controls and policy enforcement, the same way you group similar objects in Active Directory into organizational unit to facilitate applying group policies and delegate access.
It also helps you enforce isolation so that each business application is hosted in a separate management space, and how to manage your cloud spending and identify opportunities for IT cost savings. If you acknowledge these benefits, then you might be wondering how you are going to achieve all that?
The answer is simple to acknowledge but needs a lot work. At the foundation of the enterprise structure are two hierarchies. First, an Enterprise Hierarchy that reflects your cost model across corporate departments. And a Management Hierarchy that helps you group subscriptions for better granular access control management and policy enforcement.
Now, don’t worry if this sounds confusing at first, in fact, I will spend a considerable amount of time explaining to you why this is so important to do, and then how you can do it properly, so here we go.
The Need For Cloud Financial Governance
First, let’s talk about the first hierarchy, the Enterprise Hierarchy. When you start planning your enterprise hierarchy, think of cost and billing. Of course, there is a cost when you deploy and consume resources and services in the cloud, it is not free. It is usually based on consumption, the more you consume, the more you pay.
And since there is a cost involved and a money to spend, we want to reduce the surprise factor and have sort of cloud financial governance in place before deploying cloud workloads. Now, let’s talk why this is important and what might go wrong without having a cloud financial governance.
I will start by shocking you with some facts and share with you some studies, because I believe those might have the biggest effect on introducing the urgency I want you to have when you think about all this.
First, lets look at this study. With a growing number of enterprises making the move from on-premise infrastructure to on-demand cloud services, there has been a major shift from CapEx to OpEx spending.
According to Computer Economics, 65% of organizations are increasing IT operational spending, while IT capital expenditures are at a 5-year low of 18%, dropping from 23% in 2014.
I know, people are moving more to the cloud, so what? Well, this transition entails big process and organizational changes and new best practices. We’ve reached an inflection point where new methods are needed to understand, control, and manage IT costs.
For example, budgeting is no longer a one-time operational process you complete annually. Instead, your spending must be monitored and controlled on an ongoing basis due to the dynamic nature of the public cloud.
How infrastructure is procured has radically changed too. It’s now decentralized. You could agree with me that today, any employee can spin up cloud resources in seconds. This means that yesterday’s solutions for the control and predictability of infrastructure expenditures don’t work well in this new era of cloud services.
In fact, his is interesting, the agility of the cloud means things are changing. From one side, the cloud provides you with a platform for more dynamic decision-making, accelerated innovation, and a lot other benefits. But cloud also requires vigilance and real-time monitoring.
Now here is another interesting fact. A recent Google study on cloud financial governance among IT and Finance professionals found that lack of predictability is the single greatest cloud cost management pain point.
So things in the cloud are not predictable, and you might agree with me on that. I once had a subscription in Azure that I anticipate to cost me something but end up costing me a lot more.
In fact, respondents to the google study, cited the need for simplified billing and more accurate and predictable budget forecasting. They welcomed guidance on how to collaboratively work with cloud resource owners to find cost efficiencies and implement cost optimization.
Simply put, there is a need to predict cost and manage it to eliminate the factor of surprise.
By 2020, Gartner also predicts that organizations that lack cost optimization processes will average 40% overspend in public cloud. This is huge. 40% more spending because there is no cost management practices, or because you are still using the old way of managing cost that you use for your on-premises infrastructure, which obviously does not work well with the cloud.
Now to help you understand what cloud financial governance means, let’s look at this quote from Joe Davilia, and this guys is the CIO Advisory Director of KPMG
He said that “companies are spending, on average, 36% more on cloud than they actually need to” he continues “With such ease of access to the cloud allowing anyone in the business to buy cloud solutions on demand, coupled with an organization often using pre-cloud era policies, it’s no mystery why these issues exist” he explains. “This is where discipline and strong partnering intersect.”
So what you actually need is a way to manage cost for your cloud spending. If you get a bill with 15000$ for example, and you planned to spend only 5000$, then won’t you want to know why you are spending more than predicted?
The next thing to consider carefully here is the need to Understand who is spending what and stay on budget by accurately allocating your cloud spend across business units, products, cost centers and roles. This in return gives finance the ability to charge-back and operations the ability to optimize resources.
And finally, you need to think of cost optimization to quickly identify unused resources that should be shut down, underutilized resources that should be rightsized and instances for which Reserved Instances would have better rates.
So basically this is saying “I want to know for every cloud workload I have, who created that resource, which team or business unit owns that resources and then go talk to these guys and try to optimize spending“.
Who Is Responsible of Cloud Spending?
The question now that you should be asking as you know how things can go wrong with cloud spending is “who is responsible for cloud spending in your organization?” If your get a very expensive bill from your cloud provider that you definitely didn’t predict or budget for, then who is ultimately responsible for that spending in your organization?
The responsibility for preventing budget overruns and eliminating unauthorized use of cloud resources requires close collaboration between IT and Finance. A Google study on cloud financial governance revealed that IT teams play a key role in the selection of cloud service providers but that Finance also provides important inputs during the process, with the ultimate decision made by C-level executives.
After the decision is made, Finance plays an important oversight role in reporting and forecasting spend and budgeting. The oversight roles for both IT and Finance personnel require real-time monitoring and reporting of usage and costs to support real-time decisions.
So this means that IT has a role and finance has a different but related role in managing cloud cost, and they both should work together to manage and enforce cloud financial governance.
In fact, when IT and Finance professionals were asked what capabilities were the most sought after for cloud cost management, their top four answers were identical:
- Keeping costs within the budget
- Identity opportunities for IT cost saving
- Forecast IT spend
- Keep IT costs consistent and limit surprises
But the reality is that achieving all that is not easy and requires planning and careful considerations before deploying cloud workloads.
This is why in a recent Gartner survey “90% of the respondents agree that public cloud will save money for their organizations, However, few companies have implemented financial management processes for public cloud, Therefore, few have any idea whether they are achieving their goals or possibly spending even more than running their own data centers.
Cloud Financial Governance
Now that you understand why it is important to have some sort of structure before moving to the cloud to manage and predict IT cost, let’s talk about how to achieve financial governance by establishing an enterprise hierarchy. This is where you need to pay attention and think about each concept I am going to talk about in the remaining of this blog post.
Controlling Who Can Use And Manage Cloud Resources
I will start by addressing a simple question here, since cloud workloads are not free, then who is allowed in your organization to create cloud workloads, and start costing your company more money? You should really give this considerable thought before you start your cloud migration journey. And what you need is set of controls and policies.
Controlling who can use and manage cloud resources with granular policies and permissions helps your organization reduce risk and keep cloud costs in check. Today’s cloud financial governance strategy should consider the use of least privilege policy, which allows only authorized users to provision resources, and believe me, not all organizations are aware of who could create cloud resources among their employees, which a big issue to address.
This level of control helps enforce usage and management restrictions across your organization, with enforcement down to a specific application or workload. These permissions can also take into account regulatory requirements unique to your organization or industry. So for example, if your data should not leave Europe, then you want to make sure that people who are authorized to create cloud resources are aware of such requirement.
But not all environments are the same. You might have a production environments in the cloud, and other testing environments and each might require different policy enforcement. So by using policies and permissions, access to production environments can be restricted to only a few trusted users, and a less stringent financial governance strategy might be appropriate in the case of dev/test environments. Here in the dev and testing environment , you might choose to relax access controls, but enforce low usage quotas tied to budgets to quickly terminate resource usage when budget totals are reached.
Resource Governance And Consumption Rate
Now that you have appropriate controls on who can create cloud resources, you can move to the next step, which is controlling the rate at which cloud services can be consumed with resource quota policies that helps preventing unforeseen spikes in usage.
For example, admins can set a quota for maximum concurrent compute usage, triggering alerts or service throttling when the quota is reached. Other quotas might be set for total daily use or per-user use.
So if John is allowed to create resources for a business application, now your job is to control his spending with quotas and alerts to govern his usage of cloud resources. This reminds me of the concept of Trust but verify.
Spending Controls
But it might not be enough to control the consumption rate, you should also have monitors and create alerts when a certain dollar amount of spending is reached.
Monitoring your cloud spend is understanding how much money is being spent in the cloud, where the dollars are going and how they change over time
You can use a lot of tools and services to do that. Here you can say that John is allowed to do anything in this subscription as long as he does not spend more than 1000$ and then you would create alerts when usage is approaching that 1000$ threshold.
Sometimes, the use of resource tagging can help tracking the cost. If you have a tag that identifies the cost center for each resource that gets deployed in the cloud, you can then track the cost of all resources that are tagged with a specific cost center and manage cost that way.
Enterprise Hierarchy
So far, we’ve talked about some important measures like controlling which authorized users can create cloud resources, how to control their consumption and monitor their spending when it approaches a preset threshold.
The next thing you want to do is driving organizational accountability. Driving financial accountability is the process of assigning each dollar spent to the appropriate business function, this can be a cost center like a business unit, a manager or even a specific application.
In simple words, when you provision cloud resources, you need to ask yourself, which department or cost center in your organization is going to pay for these resources. Moreover, who can provision resources for each cost center so that you don’t end up with many resources that will cost you a lot of money, without properly tracking who create these resources and who will pay for them.
Does this resource belong to the marketing department or the HR department? If this resource is a virtual machine that serves the HR application, then you need a way to associate this resource to a cost center, the HR department in this case. At the end, you can have a view of all resources and who is going to pay for them, and by that you can establish accountability.
That’s why we need a concept of delegation and a hierarchy of responsibilities to establish financial accountability in the cloud which we will cover in details in the next blog post.
Featured Posts
You Can Also Become Microsoft MVP
How To Start Your Own Blog – Microsoft MVP Story
Cloud Reference Architecture CRA P1 – Foundation
Azure Bastion Step-by Step Guide
Azure advanced threat protection lateral movement
Get my latest book about Cloud Migration
This book covers a practical approach for adopting and migrating on premises systems and applications to the Public Cloud. Based on a clear migration master plan, it helps companies and enterprises to be prepared for Cloud computing, what and how to successfully migrate or deploy systems on Cloud, preparing your IT organization with a sound Cloud Governance model, Security in the Cloud and how to reach the benefits of Cloud computing by automation and optimizing your cost and workloads..
Get the book here and learn more.
Subscribe to my YouTube Channel
In my YouTube channel, I post videos about cloud security and Microsoft MVPs story to help people understand cloud and cybersecurity in simplified and professional way.
