The intersections between Observability, FinOps, and Sustainable IT
Introduction
This piece will cover new practices and exciting trends that will change and augment IT infrastructure.
A common question in the developer community is, why should I use observability? And yes, the most frequent reply is to monitor your system’s infrastructure; that way, your team can see the visibility and stay ahead of changes. Why these responses are correct understanding how your business goals correlate with your cloud infrastructure is a crucial reason to observe your system, especially for the Saas and e-commerce industry.
A closer look at FinOps and Sustainable highlights the value in investing in observability as intersections between these three trends allow organizations to stay ahead of the trend while meeting the demands of both a developing sustainable world and customer demand.
According to Gartner’s Top 10 Strategic Trends, it is highlighted in the image above. The growth of digital transformation and the surge for new IT innovation has led to IT companies’ migration to the cloud. At the same time, this brings about the desired results. Moving from on-premises infrastructure to cloud computing offers significant energy savings. However, while organizations worldwide are adopting cloud computing, few are aware of its environmental impact. A study carried out by Sustainable; IT found that only 19% measure the cloud’s ecological impact. Due to the ease of getting more cloud resources, the waste grew when the companies began to pursue Performance, Quality, and innovation.
Performance and Quality requirements
More than ever, the end-user experience requires an ever-increasing performance and availability; no one can wait 0.1 seconds to open an application to finish a transaction.
On the Netflix series “The Playlist” about the Spotify story, they told about Daniel Ek’s requirement to have the music start instantly; he did accept nothing less them it; 0.001 was insufficient.
The demand for availability is the same; 10 minutes with an unavailable application can lose millions of dollars, so 99.999% uptime is the ‘minimum’ requirement.
The usage of new technologies and cloud-native applications is growing because it can support developers to reach new requirements. Still, on the other hand, the complexity of managing a distributed microservice application across a hybrid, multi-cloud, and heterogeneous infrastructure also grows significantly.
It brings us to a scenario where the Finance teams prioritize the business values, reducing costs, growing the business, and satisfying the regulations (including sustainability). The application teams prioritize new functions, innovations, fast delivery, and high-quality servers. The SRE should support the innovation but also aim to avoid negative impacts on performance, availability, and user experience.
Costs requirements
Mainly after the COVID pandemic, Cloud costs are growing significantly, and the waste remains high. As soon as the companies move workloads to Cloud, they transform the cost model from CapEx to the OpEx model, which is good in theory. Still, another factor started to emerge, it is straightforward to create new components in the Cloud, it is pretty quick and easy to request a new server, and because of that, the companies started to lose control of these costs.
According to Forbes, 30% 30% of cloud spending is waste. Still, the problem is to define where these wastes are because enterprises want to continue increasing the usage of new cloud services to allow them to grow their business.
Accelerating FinOps and Sustainable IT article says that electricity represents about 70% of the Data Center operating costs
Capgemini Sustainable IT Report has detailed information about the costs and wastes.
Sustainable Requirements
We know sustainability is not something new; we see discussions everywhere; UN (United Nations) always discusses it on different forums, creating the Kyoto Protocol and Paris Agreement to reduce the carbon footprint. In addition, other actions are happening, such as creating governmental regulations in many counties where the companies should meet.
Very soon, the expectation is that the technology could be responsible for consuming 20% of the total electricity in the world.
IT companies should also contribute to all those actions once they are responsible for s significant percentage of the carbon footprint generated daily. But unfortunately, only 18% of companies nowadays have a comprehensive sustainable IT strategy.
You can also learn more in this excellent article Stacey M Gifford created: A brief history of sustainability across IT.
Sustainable IT can be explained as all initiatives used to reduce the environmental impacts caused by IT resources. But why are companies interested in it? It generates benefits for the organization, complies with government regulations, improves the company’s reputation, and reduces costs.
Gartner also put Sustainable IT as one of the Top 10 Strategic Technologies trends for 2023 precisely because of the reasons I mentioned before.
But who is responsible for sustainability when a company runs its applications on the Cloud? Sustainability is a shared responsibility where the cloud providers are responsible for the cloud infrastructure, using modern IT resources with less electricity consumption (cooling systems, supply chain, water consumption reduction); and the Cloud customers are responsible for the best way to use Cloud resources evaluating the workload in a holistic way covering the full-stack.
The full-stack evaluation is the better strategy for organizations to have elastic and sustainable operations.
The solutions
To address the three primary factors, Your team can focus on Implementing the emerging practices, such as observability, FinOps, and SRE Methodology.
You can align your FinOps and sustainable IT to address the three major factors. I’ll discuss this in detail here.
FinOps
FinOps is an evolving cloud financial management discipline and cultural practice that enables organizations to get maximum business value by helping engineering, finance, technology, and business teams to collaborate on data-driven spending decisions. — FinOps Org
In other words, FinOps adds finance and cost control to the operations, not just to reduce the costs but to use the budget efficiently to promote innovation and improve quality, speed, and, why not, sustainability (GreenOps).
As you can see in the image, all the Personas should be involved in the entire process, financial data should be available for everybody in real-time, and everybody should drive decisions based on the business objectives.
Scenarios you can use FinOps for better decisions:
- Your entire platform is slow; you saw on your Observability platforms that some applications should be improved to solve the latency problems; at the same time, your team workload is high due to new capabilities you committed to delivering this month that will bring more customers to the platform, based on the financial reports, the FinOps team can decide if is better to use the budget to increase the Cloud resources to fix the latency problem and deliver the new functions, or is better to redirect the team efforts to resolve the application latency problems, changing the code and postpone the new capabilities.
- You have a workload to run on the Cloud once a week, you know the workload behavior, and you have more than one Cloud provider; one Cloud provider charges by GB / Month, and the other Cloud Provider charge by egress traffic. Based on your Finance reports and the workload behavior, you can decide where to send the specific workload to reduce the costs.
FinOps mutually support Sustainable IT and vice-versa; both can bring together cost optimization and carbon footprint reduction. We can have scenarios where we can reduce the costs but not precisely reduce the carbon footprint (For example, using a different Cloud Provider for a specific workload). Still, in general, the cost reduction will reduce the waste, and reducing the waste will reduce the Carbon Footprint.
Conclusion
The image above shows us some essential requirements, such as Increase Demand, High User Expectations, Quick Innovation, Easy getting more IT Resources, and reliability requirements.
All of them together bring us some crucial problems we want to address, such as the increase in Cloud waste, the increase in costs, and the increase in carbon footprint generated by IT.
Observability will bring visibility related to application performance and metrics, a correlation between components, identification of code issues that consume resources, and so on.
SRE Practices will support the teams on what to do with Observability’s data, fix the application problems permanently, and make decisions balancing innovation and reliability.
FinOps will bring finance information into the game, allowing the team to make decisions with technical data, business goals, and costs.
Sustainable IT will also bring sustainability data to the whole team, allowing them to make decisions, think about the impacts generated by the solutions, and satisfy sustainability regulations.
I hope this article helps you understand and use those concepts to support company targets.
Tiago Dias Generoso is a Distinguished IT Architect | Senior SRE | Master Inventor based in Pocos de Caldas, Brazil. The above article is personal and does not necessarily represent the employer’s positions, strategies or opinions.
