Cloud cost & compute resource optimisation (FinOps)

Cloud and AI bills are variable, opaque and growing; engineers provision capacity, finance pays for it later. FinOps tooling combines normalised billing data, utilisation telemetry, forecasting, anomaly detection and solvers to allocate costs, right-size workloads, schedule non-production environments and choose the right mix of on-demand, spot and committed capacity. Increasingly an LLM layer answers cost questions in natural language and drafts or executes the fix.

Problem & value

• Today: invoices arrive weeks after usage; tags are incomplete; commitments (savings plans, reservations) are bought by spreadsheet; over-provisioned VMs, idle GPUs, orphaned storage and forgotten dev environments run for months. Finance cannot link spend to product or customer.
• AI value: anomaly detection catches runaway spend within hours instead of at month end; forecasting makes budgets and commitments less guesswork; solvers pick the cheapest feasible capacity mix; LLM assistants explain bills and turn recommendations into pull requests or tickets.
• Evidence: the FinOps Foundation's State of FinOps 2026 survey (1,192 respondents) reports that FinOps scope is expanding beyond public cloud: 98% of respondents manage AI spend, 90% SaaS and 48% data-centre spend [1]. Exact savings vary widely by maturity; vendors quote their own figures, so measure against your own baseline.

Variants

• SMB: native cloud tools (budgets, cost anomaly detection, advisor recommendations) plus one SaaS or a managed-service partner; focus on schedules, right-sizing and a simple commitment.
• Enterprise / platform teams: FOCUS-normalised multi-cloud data lake, showback/chargeback, Kubernetes cost allocation, commitment portfolio management, FinOps-as-code in CI/CD.
• AI/GPU FinOps: token and GPU-hour cost per model, feature and customer; batch vs. real-time routing, model caching, cheaper-model routing (see CODE-023 for kernel-level efficiency).
• Carbon-aware (GreenOps): shift flexible workloads to regions/hours with lower grid carbon intensity; report cost and emissions together.
• Public sector: budget transparency and procurement-compliant reporting on sovereign/EU clouds.
• Private use: not relevant beyond hobby-cloud budgets and alerts.

How it works

1. billing export + usage metrics + tags/labels
2. normalise (FOCUS schema) and enrich (ownership, product, environment)
3. allocate (tags, Kubernetes namespaces, shared-cost rules)
4. showback/chargeback
5. forecast (time series per service/team) + anomaly detection
6. optimise (right-sizing, scheduling, spot mix, commitment portfolio via solver/LP)
7. recommend / open ticket or pull request (human approval)
8. optional auto-apply within guardrails
9. measure realised savings and unit cost

• Forecasting: gradient boosting or time-series foundation models on daily cost per service/account, with calendar and release features.
• Anomaly detection: seasonal baselines or isolation-forest style models on cost per dimension; LLM summarises the probable driver ("new NAT gateway traffic in project X after deploy Y").
• Optimisation: right-sizing from percentile utilisation; commitment sizing as an optimisation problem under demand uncertainty; Kubernetes bin-packing and node consolidation; spot-interruption risk models. Reinforcement-learning autoscalers exist but rule-plus-forecast approaches dominate.
• Human-in-the-loop: production changes and commitment purchases require owner approval; automation is limited to non-production, schedules and reversible actions.

Data preparation

Data needed: billing and usage exports from the cloud providers, resource tags, Kubernetes and GPU utilisation metrics, commitment and reservation inventories, and the mapping from resources to teams and products. It lives in the provider consoles, billing exports in object storage, monitoring systems, the CMDB or spreadsheet inventories, and finance's ERP cost centres. Typical gaps: missing or inconsistent tags, shared accounts without owners, multi-cloud bills in different formats, no link between cloud cost and product revenue, and idle resources nobody claims.

Preparation steps: 1. Enable detailed billing exports and keep at least 13 months of history. 2. Normalise formats to the FOCUS specification for multi-cloud data. 3. Enforce a tagging standard (team, product, environment) and back-fill untagged spend with rules (PREP-010). 4. Agree cost-centre mapping and metric definitions with finance (PREP-012, PREP-004). 5. Connect utilisation metrics for right-sizing.

Tools: provider billing exports, SQL or DuckDB, dbt, OpenCost. Rough share of effort: 15-25 percent; tag hygiene is the biggest lever. No personal data is involved.

Implementation paths

Which path for whom

• SMB: A, because the cloud providers' native cost tools and mature FinOps products (e.g. Vantage, CloudZero, nOps) give anomaly alerts and right-sizing at low cost without a project.
• Enterprise: C, because a shared internal platform with OpenCost, Kubecost or FOCUS-based pipelines serves many teams, avoids per-spend pricing and keeps billing data in-house.
• Developer: C, because open-source tooling (OpenCost, Infracost, Prometheus) is simple to run on HW1 and fits infrastructure-as-code workflows.

Path A — Buy

• Native cloud tooling: AWS Cost Explorer/Compute Optimizer/Cost Anomaly Detection, Azure Cost Management and Advisor, Google Cloud Recommender and FinOps hub.
• FinOps platforms: Flexera, Apptio Cloudability (IBM), CloudZero, Vantage, Ternary, Finout, nOps, ProsperOps (commitment automation), Spot by NetApp, Kion; Cast AI and PerfectScale (Kubernetes). Most are US-based; check EU data residency and DPA, or use an EU managed-FinOps partner.
• Choose A when: spend is mostly on AWS/Azure/GCP, the team is small and tags are reasonable.

Path B — Build on cloud APIs

• Pull billing exports (CUR/Parquet, Azure exports, BigQuery export) into a warehouse, convert to FOCUS, build dashboards (Grafana, Metabase, Power BI) and alerts.
• Add forecasting/anomaly models with managed ML services in EU regions, and an LLM (Claude, Gemini, GPT, Mistral — EU region/provider) behind a text-to-SQL or agent interface for questions like "why did storage cost rise in September?". Keep the LLM read-only on the cost warehouse.
• Orchestrate with n8n, Step Functions or Airflow; push recommendations as Jira tickets or Terraform pull requests.

Path C — Self-host (open source)

• OpenCost (CNCF) for Kubernetes cost allocation, Karpenter and KEDA for node and workload scaling, Kubernetes VPA/Goldilocks for request sizing, Cloud Custodian for policy and schedule automation, Prometheus + Grafana, Apache Superset/DuckDB on FOCUS data.
• Time-series models: Prophet, statsforecast, Chronos/TimesFM-style foundation models; solvers: OR-Tools, HiGHS, PuLP for commitment and bin-packing.
• Optional local LLM (Qwen, Llama, Mistral class) for natural-language cost queries.
• Hardware: HW1 (CPU server) is sufficient; HW2 only if a local LLM is added.

Path D — Custom training

Worthwhile only at very large scale: custom demand and spot-interruption models, RL schedulers for GPU clusters, or fine-tuned small models for ticket generation. Needs 12+ months of granular usage history; most organisations get better returns from tagging discipline and Path A/B.

State of the art (2026)

• FOCUS (FinOps Open Cost and Usage Specification) is the common billing schema across clouds, SaaS and increasingly AI providers, making multi-vendor allocation and benchmarking feasible [2].
• FinOps scope expanded from cloud to SaaS, data centre and AI/GPU spend; "unit economics per token / per customer" is the new headline metric [1].
• Kubernetes cost tooling is mature: OpenCost standard allocation, Karpenter-style just-in-time node provisioning and consolidation, in-place pod resizing in recent Kubernetes versions [3][4].
• Agentic FinOps: LLM agents investigate anomalies by querying billing and deployment data and open remediation changes; vendors ship MCP servers for cost data. Reliability depends on guardrails and a clean allocation model.
• Limits and failure modes: poor tagging dominates accuracy; right-sizing on short histories causes performance incidents; commitment purchases are financially binding, so forecast error turns into locked-in waste; LLM answers can misattribute costs; vendor savings claims are rarely audited; optimisation tools themselves add cost.

Shared resources & combinations

• Same stack (time-series + anomaly-detection + optimization-solver): OPT-005 inventory optimisation, OPT-002 production scheduling, OPT-007 building energy (energy and cost schedules), PRED-016 cash-flow forecasting.
• Anomaly tooling: ANOM-030 business KPI anomalies and ANOM-006 ledger anomalies share methods.
• LLM/agent layer: AGENT-002 back-office automation; AGENT-006 finance close consumes accrual and allocation outputs; CODE-023 reduces compute cost at kernel level.
• Data: FOCUS warehouse is reusable for ESG/carbon reporting.

Effort, cost & KPIs

• Path A: days to weeks; SaaS often priced as a percentage of managed spend or per resource, so check that pricing does not scale perversely with savings.
• Path B: 1-3 months for a warehouse, dashboards, alerts and an assistant.
• Path C: 2-4 months plus platform-team time; no licence cost.
• KPIs: tagged/allocated spend %, forecast accuracy (MAPE), anomaly time-to-detect, commitment coverage and utilisation, idle/waste %, realised savings vs. baseline, unit cost (per customer, transaction, 1k tokens), GPU utilisation, gCO2e per workload.

People & change

Involve management for savings targets and accountability, the IT lead or platform team as process owner, finance for budgets and commitments, and IT security for the access rights of any tool that can change resources. Engineering teams are the end users; they must own their cost. Works council: normally no, because the data concerns resources, not people; check only if costs are reported per named employee or developer, which would raise §87(1) no. 6 BetrVG, and inform the council under §90(1) no. 3 when planning automated scheduling that changes working conditions. Data protection: usually no personal data, so no DPIA; the DPO only needs to be informed if user-level logs are used.

AI literacy: support engineers and finance staff with a short session on how forecasts and anomaly scores come about and when to distrust them (AI Act Art. 4). Start with showback (reporting) before chargeback, and with recommendations rather than automatic changes. Pilot with one team or one cloud account. Typical concerns: "automation will switch off my service" and "this is surveillance of my spending". Address them with approval steps for production changes, an opt-out tag, transparent rules and a rollback path.

See GUIDE-011, GUIDE-013 and GUIDE-009.

Risks, regulation & compliance

• EU AI Act: minimal risk; no Annex III use. AI literacy (Art. 4) applies to staff using the tools. Agents that control production infrastructure are an operational risk, not a high-risk category, unless embedded in critical-infrastructure safety components [5].
• GDPR: billing and utilisation data is mostly non-personal; resource tags, user names, IP addresses and logs can identify employees or customers; apply DPA, EU region and data minimisation before sending any data to external LLMs.
• Other EU rules: DORA requires financial entities to manage ICT third-party and concentration risk, which affects automated multi-cloud shifts; the Energy Efficiency Directive and CSRD reporting drive emissions reporting for data-centre workloads.
• Risks: automation causing outages (mitigate with approvals, canaries, rollback), vendor lock-in from commitments, shadow data copies of billing data, employee-surveillance perception when cost is attributed per developer (use team-level showback).

Getting started

1. Turn on billing exports, enforce a minimal tag set and measure the share of unallocated spend.
2. Enable native anomaly detection and budgets; add schedules for non-production environments.
3. Review right-sizing and idle-resource recommendations weekly with owning teams.
4. Normalise data to FOCUS, publish showback dashboards, then size commitments conservatively.
5. Add AI/GPU cost tracking per model and feature; pilot an LLM assistant read-only.

Sources

1. FinOps Foundation — State of FinOps (annual report) — https://www.finops.org/insights/state-of-finops/ ; 2026 results press release — https://www.linuxfoundation.org/press/state-of-finops-survey-ai-value-and-skills-top-priorities-as-finops-matures-across-technology-value-98-manage-ai-90-saas-64-licensing-48-data-center-1
2. FinOps Foundation — FOCUS specification — https://focus.finops.org/
3. OpenCost — CNCF project documentation — https://opencost.io/docs/
4. Karpenter — Kubernetes node autoscaler documentation — https://karpenter.sh/docs/
5. Regulation (EU) 2024/1689 (AI Act) — https://eur-lex.europa.eu/eli/reg/2024/1689/oj
6. Cloud Custodian — https://cloudcustodian.io/docs/
7. FinOps Foundation — Framework — https://www.finops.org/framework/