Do you want to improve the time-to-market and financial supervision of data projects ? The DataOps and FinOps review make this possible! Two quick and iterative methods that make it possible to optimise projects, and which contribute to the continuous improvement of data valuation.
DataOps: a DevOps specific to data projects
In 2009, Belgian IT specialist Patrick Debois devised a new approach aimed at reconciling two professions with sometimes conflicting challenges: the developer (Dev) on the one hand, and the systems and architecture administrator (Ops) on the other.
Called DevOps, this new approach has introduced two major innovations.
- Taking into account deployment constraints from the programming phase.
- The generalisation of agile methods to the entire IT project.
DataOps: a triple challenge
However, IT projects are no longer the same as ten years ago. More complex and longer, data projects are also characterised by a low transformation rate (only 53% of POCs are deployed in production). In this context, the DataOps method addresses three challenges.
The human challenge
By promoting communication and cooperation between teams, Data Ops makes it possible to reconcile sometimes divergent interests between business lines, technical profiles and IT and finance managers.
The methodological challenge
At present, this aspect is still too often overlooked in data projects. For example, we do not ask enough questions upstream about the best way to associate data managers (data engineers, data scientists, data stewards) with data consumers (business analysts, business lines, etc.).
The technological challenge
Integration, data science, data processing… For all these aspects, the tools available are abundant. Choosing the right solutions – and, above all, making them work together – is a real battleground for teams.
DataOps: the process of a project
Like DevOps, DataOps aims to automate design and deployment to production. It all starts with the design and packaging of the pipeline. Then the teams work on deployment and orchestration. Then, a monitoring phase leads to deployment to the integration environment. This transition from one environment to another is based on a validation workflow that promotes communication and collaboration between the teams. Once a step is validated, a new staging cycle is opened for the acceptance phase, again with dedicated steps for packaging, deployment, orchestration and monitoring.
FinOps to optimise the budget of a data project
The FinOps is the alter-ego of DataOps in terms of financial supervision. This method is particularly relevant for projects that rely on the cloud rather than an on-premise solution.
Concretely, the FinOps consists of establishing a dialogue with financiers at the start of the project. The idea is to define storage, compute and processing charges, which are among the most important costs when deployed on the cloud. From there, a cycle begins. The teams control consumption and seek to optimise it continuously. For example, we can choose to resize the capacity of servers, compute, or data acquisition processes.
Like DataOps, FinOps draws its philosophy from agile methods, with a focus on meeting business expectations (here, financiers). But it also allows architects and developers to make the right choices when it comes to solutions. In other words, select the ones that will be the least costly and most relevant to the customer. Ultimately, FinOps enables the IT and Finance departments to better manage their budgets and organise their showbacks and chargebacks.
