Architecture Reference

Terraform Data Flow

End-to-end walkthrough of how Terraform processes configuration, builds a dependency graph, plans changes, and applies them via provider plugins.

Repo: hashicorp/terraform Commit: a2ad11c Language: Go

Complete Data Flow Pipeline

┌─────────────────────────────────────────────────────────────────────────────┐
│  CLI Entry  ·  main.go:realMain()  →  initCommands()  →  cli.CLI.Run()      │
└──────────────────────────────────────┬──────────────────────────────────────┘
                                       │
          ┌────────────────────────────┼───────────────────────────┐
          │                            │                           │
          ▼                            ▼                           ▼
┌──────────────────┐        ┌──────────────────┐       ┌──────────────────┐
│  terraform init  │        │  terraform plan  │       │  terraform apply │
└────────┬─────────┘        └────────┬─────────┘       └────────┬─────────┘
         │                           │                           │
         ▼                           ▼                           ▼

── INIT ──────────────────────────────────────────────────────────────────────

  InitCommand.Run()
    │
    ├─► installModules()
    │     └─► ModuleInstaller.InstallModules()
    │           ├── Walk module calls in *configs.Module
    │           ├── Resolve versions from registry
    │           ├── Download & cache to .terraform/modules/
    │           └── Write .terraform/modules/modules.json
    │
    ├─► getProviders()
    │     ├── getproviders.Source.AvailableVersions()
    │     ├── Resolve constraints → Lock file
    │     ├── Download binaries to .terraform/providers/
    │     └── Verify checksums
    │
    └─► depsfile.Locks  →  .terraform.lock.hcl

── CONFIG LOAD ───────────────────────────────────────────────────────────────

  configload.Loader.LoadConfig(dir)
    │
    ├─► Parse *.tf files with HCL2 parser
    ├─► Build *configs.Module  (resources, vars, outputs, providers…)
    └─► configs.BuildConfig(rootMod, walker, loader)
          ├── Recursively walks ModuleCalls
          ├── Loads each child *configs.Module
          └─► *configs.Config  (tree of modules)

── PLAN ──────────────────────────────────────────────────────────────────────

  PlanCommand.Run()
    │
    ├─► PrepareBackend()  →  backend.Backend.StateMgr()  →  prior *states.State
    ├─► OperationRequest()
    └─► backend.RunOperation()
          └─► Context.Plan(config, prevRunState, opts)
                │
                ├─► planWalk()
                │     ├── BasicGraphBuilder.Build()
                │     │     └── GraphTransformers (ResourceTransformer,
                │     │         ReferenceTransformer, ProviderTransformer…)
                │     │         →  *terraform.Graph (DAG)
                │     │
                │     ├── Walk — Refresh Phase
                │     │     └── NodeAbstractResourceInstance.Execute()
                │     │           └─► providers.Interface.ReadResource()
                │     │
                │     └── Walk — Plan Phase
                │           └── NodePlannableResourceInstance.Execute()
                │                 └─► providers.Interface.PlanResourceChange()
                │                       →  plans.ResourceInstanceChange
                │
                └─► *plans.Plan  →  planfile.Writer  →  .tfplan (ZIP)

── APPLY ─────────────────────────────────────────────────────────────────────

  ApplyCommand.Run()
    │
    ├─► PrepareBackend()
    ├─► OperationRequest()
    └─► backend.RunOperation()
          └─► Context.Apply(plan, config, opts)
                │
                ├─► applyGraph()
                │     └── Build apply-specific DAG
                │
                ├── Walk — Apply Phase
                │     └── NodeApplyableResourceInstance.Execute()
                │           └─► providers.Interface.ApplyResourceChange()
                │                 →  updated resource object (cty.Value)
                │
                ├─► states.SyncState.SetResourceInstanceCurrent()
                │     └── Thread-safe state mutation
                │
                └─► statemgr.Full.WriteState(*states.State)
                      └── Backend persists state to local file / remote API

Three-Phase Model

Init — Dependency Resolution

terraform init prepares the working directory. It resolves and installs external modules and provider plugins, writes a lock file, and leaves the workspace ready for subsequent commands. No infrastructure is touched.

Output artifacts: .terraform/modules/, .terraform/providers/, .terraform.lock.hcl

Plan — Declarative Diff

terraform plan loads the HCL configuration into a *configs.Config tree, then builds a dependency graph. Two graph walks occur:

Refresh: calls ReadResource on each provider to reconcile stored state against real infrastructure.
Plan: calls PlanResourceChange to diff desired vs. actual, producing a list of ResourceInstanceChange objects.

The resulting *plans.Plan is serialized to a binary plan file for auditability.

Key invariant: The plan is fully deterministic and serializable — apply reads the same plan object without re-evaluating configuration.

Apply — State Mutation

terraform apply loads the plan, builds an apply-specific graph, and walks it in dependency order. Each resource node calls ApplyResourceChange on its provider, receives the resulting state, and writes it into states.SyncState. The final state is persisted via the backend.

Concurrency: the graph walk executes up to 10 nodes in parallel (controlled by -parallelism flag). SyncState uses an sync.RWMutex for safe concurrent writes.

Key Entry Points

Operation	Command Entry	Core Engine	Key Type
Init	`command/init.go:50` ↗ GitHub	`initwd/module_install.go:35` ↗ GitHub	`ModuleInstaller`
Config Load	`configs/config_build.go:30` ↗ GitHub		`*configs.Config`
Plan	`command/plan.go:22` ↗ GitHub	`terraform/context_plan.go:180` ↗ GitHub	`*plans.Plan`
Apply	`command/apply.go:27` ↗ GitHub	`terraform/context_apply.go:81` ↗ GitHub	`*states.State`
Providers	`providers/provider.go:17` ↗ GitHub		`providers.Interface`
State	`states/state.go:27` ↗ GitHub		`*states.State`

Component Deep-Dives

⚙️

Init

Module installation, provider download, lock file generation

📄

Configuration

HCL parsing, module tree, Config & Module structs

📊

Plan + Graph

DAG construction, graph transformers, refresh & plan walks

🚀

Apply

Apply graph walk, provider calls, state mutation & persistence

🔌

Providers

Plugin subprocess model, gRPC, providers.Interface

💾

State

State structure, SyncState, backend persistence

Architectural Patterns

1 · Graph-Based Execution

All three phases (refresh, plan, apply) use a directed acyclic graph (DAG). BasicGraphBuilder accepts a slice of GraphTransformer steps that progressively add nodes and edges. The final graph is topologically sorted and walked concurrently up to the parallelism limit.

2 · Interface-Driven Design

Three core interfaces decouple the engine from implementations: Backend (state storage), providers.Interface (cloud APIs), and EvalContext (expression evaluation environment). This makes unit-testing the core engine straightforward.

3 · Separation of Plan and Apply

The plan is a serializable, inspectable artifact. Apply reads it without re-evaluating HCL. This enables the review-before-apply workflow and means remote execution environments (Terraform Cloud) can apply plans generated elsewhere.

4 · cty Value System

All values flowing through the evaluator, provider calls, and state are typed using cty.Value from the go-cty library. It supports unknown values (for speculative planning), null, and sensitive-marked values without special-casing.

5 · Thread-Safe State

states.SyncState wraps *states.State with a sync.RWMutex. The apply graph walk runs nodes in parallel; all state writes go through SyncState methods to prevent races.

6 · Provider-as-Process

Providers are separate executables communicating over gRPC via stdin/stdout. plugin.GRPCProvider implements providers.Interface by translating Go method calls into protobuf RPC calls. Terraform manages the provider process lifecycle (start, keep-alive, shutdown).