diff --git a/.gitignore b/.gitignore index 8ce01c8..27a72fa 100644 --- a/.gitignore +++ b/.gitignore @@ -50,3 +50,10 @@ openai.token # Local persistent E2E runtime variables (canonical for this troubleshooting phase) src/config/e2e.tfvars src/config/e2e-bootstrap.override.tfvars + +# Cached OPNsense API key created by the firewall bootstrap +.firewall-api-credentials.json +src/.firewall-api-credentials.json + +# Ignore QEMU disk images (firewall) +*.qcow2 diff --git a/docs/architecture.md b/docs/architecture.md index 4c77e08..b7b76cb 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -33,6 +33,7 @@ All modules live under `src/modules/`. The root `src/main.tf` calls them in depe | `devops` | `src/modules/devops/` | Optional DevOps project with managed Git instance | | `landing-zone` | `src/modules/landing-zone/` | Per-workload project: network, RBAC, Secrets Manager, object storage, DNS child zone, routing | | `sandboxes` | `src/modules/sandboxes/` | Lightweight sandbox projects for experimentation | +| `firewall-config` | `src/modules/firewall-config/` | Optional: rules, NAT and internet hardening pushed to the OPNsense appliance through its API | ### Governance @@ -103,9 +104,12 @@ When `connectivity.firewall` is set, a VM running OPNsense (provided as a `.qcow | `vtnet0` (WAN) | `wan_network`: attached to the WAN routing table | Outbound internet egress, assigned a static public IP | | `vtnet1` (LAN) | `lan_network`: a dedicated private subnet | Internal next-hop for all corporate landing zone traffic | -The firewall's LAN IP is exported as `firewall_next_hop_ip` and passed to every corporate landing zone so they can point their default route at it. +The firewall's LAN IP is exported as `firewall_next_hop_ip` and passed to every corporate landing zone so they can point their default route at it. Everything the platform sends outwards therefore crosses the appliance. -Source: `src/modules/connectivity/` +Both interfaces are DHCP clients and pick up the fixed addresses STACKIT assigns them. The image boots with almost no policy: only the two default `allow LAN to any` rules (IPv4 and IPv6), no gateways, no static routes, no IPsec, and outbound NAT on `automatic`. It filters nothing until a policy is pushed. + +> [!WARNING] +> After the first apply the web GUI answers on the public IP to the whole internet, and it still carries the `root` password shipped in the image, which is identical on every copy. Port 443 is open, 80 and 22 are not, and there is no security group in front of the WAN interface. Pushing the policy closes it, so do not leave a deployment sitting between the two applies — see [Configure OPNsense firewall](getting-started.md#configure-opnsense-firewall). Binding the GUI to the LAN interface only (**System → Settings → Administration → Listen Interfaces**) is the alternative that does not depend on the ruleset. ### Landing Zone @@ -219,6 +223,39 @@ Traffic flow for a corporate landing zone (firewall flavor): East-west traffic between corporate LZs stays within the Network Area and can be permitted or denied by firewall policies. +## Site-to-Site VPN (optional) + +Any hub-spoke flavor can terminate an IPsec VPN in the connectivity project, connecting the Network Area to on-premises or another cloud. Enable it with the `connectivity.vpn` block — see the commented example in `src/config/hub-and-spoke.tfvars`. + +The gateway is highly available: it runs two tunnels in separate availability zones, each with its own public IP. Because both sides need the other's address, roll it out in two applies — provision the gateway with `connections = {}`, read `tofu output connectivity_vpn_public_ips`, configure the remote peer, then add the connection. + +Pre-shared keys are kept out of the config object in the separate `vpn_pre_shared_keys` variable, so the tunnel topology stays committable: + +```bash +export TF_VAR_vpn_pre_shared_keys='{"onprem"={"tunnel1"="<20+ chars>","tunnel2"="<20+ chars>"}}' +``` + +Routes to the remote prefixes are distributed through the Network Area automatically, so every corporate landing zone reaches the remote site without per-spoke configuration. + +### What goes through the firewall + +In the firewall flavor, two of the four traffic directions run through the appliance cleanly. Neither VPN direction can be filtered with the managed gateway: one breaks if you try, the other is not steerable at all. + +| Direction | Through the firewall? | Mechanism | +|---|---|---| +| LZ → Internet | Yes, by default | Default route of the landing zone routing table points at the firewall LAN address. Needs an `outbound_nat` entry too, otherwise traffic reaches the appliance and stops there. | +| LZ → LZ | Yes, by default | `system_routes = false` suppresses project-to-project routes, so spoke-to-spoke falls to the default route. Needs a static route back into the Network Area, otherwise the traffic leaves through WAN and the egress NAT rewrites the source address. | +| LZ → on-premises (VPN) | **No** | The VPN prefix beats `0.0.0.0/0`, so it bypasses the appliance. | +| on-premises → LZ (VPN) | **No** | Delivered straight to the spoke through the STACKIT-managed routing table. Not steerable. | + +Inbound VPN traffic bypasses the appliance because `static_routes` of a VPN connection are distributed as *dynamic* routes across the Network Area, and a remote prefix is always more specific than `0.0.0.0/0`. The VPN gateway itself forwards through a STACKIT-managed routing table that cannot be modified or reassigned. + +> [!WARNING] +> Setting `dynamic_routes = false` does not fix the outbound direction, it breaks it. Outbound then goes through the firewall while inbound still bypasses it, and the stateful filter drops the half-flow it sees. + +If VPN traffic has to be inspected, terminate the tunnel on the OPNsense appliance itself instead of using the managed gateway — it is already the default route for every corporate landing zone, so both directions stay symmetric. The trade-off is losing the managed gateway's active-active HA. + + ## Resource Naming All resources follow a consistent convention driven by `company_code`: diff --git a/docs/getting-started.md b/docs/getting-started.md index 66ea0e6..3f06884 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -27,6 +27,10 @@ Three ready-to-use configurations are provided in `src/config/`: Choose the flavour that matches your requirements and adjust the corresponding `.tfvars` file before deployment (step 7). At a minimum, update `owner_email`, `organization_id`, `company_name`, and `company_code`. +The firewall flavour takes one extra step: the appliance boots unconfigured and its policy is pushed in a second apply, from the `firewall_config` block that ships commented out in the same `.tfvars` file. Until then it filters nothing and its web GUI is reachable from the internet — see [Configure OPNsense firewall](#configure-opnsense-firewall). + +Both hub-spoke flavours can additionally terminate a site-to-site IPsec VPN in the hub. It is disabled by default — see the commented `connectivity.vpn` block in the `.tfvars` file and [Site-to-Site VPN](architecture.md#site-to-site-vpn-optional). If you deploy the firewall flavour, read [what traffic the firewall actually sees](architecture.md#what-goes-through-the-firewall) before relying on it for VPN inspection. + > [!NOTE] > This single-root-module approach works well for smaller environments. At larger scale — typically beyond 10 landing zones — you may encounter STACKIT API rate limits during applies and slower plan/refresh cycles due to a growing state file. Tools like [Terragrunt](https://terragrunt.gruntwork.io/), [Terramate](https://terramate.io/), or [Spacelift](https://spacelift.io/) can help by splitting landing zones into isolated state files and orchestrating root module calls with proper concurrency controls. If you are planning a larger enterprise deployment, reach out to [STACKIT](https://stackit.de) or a partner offering a verified landing zone solution via the [STACKIT Marketplace](https://marketplace.stackit.cloud/de/catalog?marketplaceFilters=industries:Service%20%26%20IT%20Provider,deliveryMethod:PROFESSIONAL_SERVICE,categories:DevOps). @@ -124,6 +128,14 @@ tofu init ### 9. Deploy the landing zone +If you enabled the `connectivity.vpn` block, export the pre-shared keys as an environment variable first. They are kept out of the `tfvars` on purpose: + +```bash +export TF_VAR_vpn_pre_shared_keys='{"onprem"={"tunnel1"="<20+ chars>","tunnel2"="<20+ chars>"}}' +``` + +In any case run opentofu to deploy the infrastructure: + ```bash tofu apply ``` @@ -221,11 +233,93 @@ stackit project delete --project-id ## Post-Deployment (Optional) +### Configure OPNsense firewall + +Step 9 boots the appliance but leaves it unconfigured: it filters nothing and its web GUI answers on the public IP. Pushing a policy is a **second apply**, because the API key is derived from the appliance login and a provider configuration has to resolve before any resource is planned, so a key created during an apply cannot configure the provider in that same run. + +What the policy does, and which traffic directions it can actually see, is in [What goes through the firewall](architecture.md#what-goes-through-the-firewall). + +> [!NOTE] +> The policy is pushed with the [`browningluke/opnsense`](https://registry.terraform.io/providers/browningluke/opnsense/latest/docs) provider. It maps OPNsense's expanded read format back to what was written, so in-place updates and drift detection work — the generic `Mastercard/restapi` provider cannot do this and makes objects effectively write-once. It is community maintained and its author advises against production use, so weigh that before relying on it. + +**1. Decide which firewall interface OpenTofu should talk to** + +*Public: OpenTofu runs outside the Network Area* — a workstation, or a CI runner on the public internet. This is the normal case for a first deployment, because nothing is inside the area yet. Traffic goes over the appliance's public address: + +```bash +tofu output connectivity_firewall_public_ip # the endpoint +curl -s https://ifconfig.me # your own public address +``` + +*Private: OpenTofu runs inside the Network Area* — a CI runner in a landing zone, a jumphost, or an established site-to-site VPN. Nothing to look up: the LAN address is the default, and your source address is already covered by `10.0.0.0/16`. + +**2. Enable the policy** + +Uncomment the `firewall_config` block in `config/hub-and-spoke-firewall.tfvars`. It comes with default rules: internet egress with NAT, spoke-to-spoke limited to HTTPS and ICMP, and the two floating rules that take the web GUI off the internet. + +Two entries need your values, and what you put there follows from step 1: + +```hcl +# Example for public + +endpoint = "https://" +fw_management = { + content = ["10.0.0.0/16", "/32"] +} + + +# Example for private + +# omit endpoint +fw_management = { + content = ["10.0.0.0/16"] +} +``` + +> [!WARNING] +> Whatever runs OpenTofu must be covered by the `fw_management` alias before this apply. Otherwise the same run that closes the GUI cuts off its own path to the API, and recovery goes through the serial console (`stackit server console --project-id `). +> +> A dynamic home or office address will eventually change and lock out a later run. + +**3. Bootstrap the API key, then push the policy** + +```bash +tofu apply -var firewall_bootstrap=true && tofu apply -var firewall_bootstrap=true +``` + +The first pass derives the API key (waiting for the appliance to finish booting, which takes a few minutes) and caches it in `src/.firewall-api-credentials.json`, gitignored. The second stores it in the management Secrets Manager and pushes aliases, static routes, rules and NAT. Two passes are unavoidable: the provider needs the key at plan time, and a value created during an apply cannot configure a provider in that same run, which is why they chain safely as one command. + +The appliance login it uses comes from `firewall_admin_password`, which defaults to the `root` password baked into the STACKIT OPNsense image, so nothing has to be exported for a fresh deployment. + +> [!IMPORTANT] +> That default is baked into the image, and between the two applies the web GUI is reachable from the internet. Change the password on the appliance (**System → Access → Users**). The password is only used to derive the API key. Once the key exists, it is no longer read at all. + +**4. Drop the bootstrap variable and delete the cache file** + +From here on every apply is a plain `tofu apply` with no variables: + +```bash +rm src/.firewall-api-credentials.json +tofu apply +``` + +**Rotating the key** + +Re-running the bootstrap revokes every existing API key on the appliance and mints a fresh one, which immediately invalidates the copy in the Secrets Manager. The follow-up is therefore not optional: bump `firewall_api_secret_version` by one, so the new key is written through. Forgetting the bump fails loudly with a 401 instead of silently keeping a dead key. + +```bash +rm -f src/.firewall-api-credentials.json +tofu apply -var firewall_bootstrap=true # mints the new key +# bump firewall_api_secret_version in your tfvars, then +tofu apply -var firewall_bootstrap=true # writes it to the Secrets Manager +rm src/.firewall-api-credentials.json +``` + ### DNS automation for Gateway API resources For Gateway API resources (for example Envoy Gateway with `Gateway` + `HTTPRoute`), use DNS records directly via `stackit_dns_record_set` until native provider support for `extensions.dns.gatewayApi` is available. -For the existing sample content in this repository (`landing_zone_sample_gateway` + `landing_zone_sample_http_route` in `src/namespace-service.tf`), the DNS record is created automatically based on the Envoy Gateway LoadBalancer endpoint discovered via `kubernetes_resources`. +For the existing sample content in this repository (`landing_zone_sample_gateway` + `landing_zone_sample_http_route` in `src/_landing-zone-kubernetes.tf`), the DNS record is created automatically based on the Envoy Gateway LoadBalancer endpoint discovered via `kubernetes_resources`. Implementation pattern: @@ -260,8 +354,4 @@ resource "stackit_dns_record_set" "landing_zone_sample_gateway" { } ``` -This ensures a stable, Terraform-managed DNS path without external scripts until provider-native `gatewayApi` DNS extension support is available. - -### Configure OPNsense firewall - -If you deployed the Hub-Spoke + Firewall flavour, configure the OPNsense. Guidance will be available soon in the STACKIT docs. +This ensures a stable, Terraform-managed DNS path without external scripts until provider-native `gatewayApi` DNS extension support is available. \ No newline at end of file diff --git a/mise.lock b/mise.lock index 5a4e0e7..b172536 100644 --- a/mise.lock +++ b/mise.lock @@ -34,6 +34,11 @@ checksum = "sha256:f2a44a0ff3393c18e76159b5e0fe3a55042f17bda90900a282b3d2f69100f url = "https://github.com/stackitcloud/stackit-cli/releases/download/v0.67.0/stackit-cli_0.67.0_darwin_amd64.tar.gz" url_api = "https://api.github.com/repos/stackitcloud/stackit-cli/releases/assets/483324208" +[tools."github:stackitcloud/stackit-cli"."platforms.windows-arm64"] +checksum = "sha256:d0c937bb48183df19a9bbb68f7512215cc924b4dc1f6e930397392881aadae92" +url = "https://github.com/stackitcloud/stackit-cli/releases/download/v0.67.0/stackit-cli_0.67.0_windows_arm64.zip" +url_api = "https://api.github.com/repos/stackitcloud/stackit-cli/releases/assets/483324155" + [tools."github:stackitcloud/stackit-cli"."platforms.windows-x64"] checksum = "sha256:7d9f2eabb6d0e9b7e8238fbe56508dcc36264184301694c7e0bd4ab34aebaba7" url = "https://github.com/stackitcloud/stackit-cli/releases/download/v0.67.0/stackit-cli_0.67.0_windows_amd64.zip" @@ -67,6 +72,10 @@ url = "https://github.com/opentofu/opentofu/releases/download/v1.12.5/tofu_1.12. checksum = "sha256:1012d8f3d4567bcbcd1f2c7d766feca39a30bced32fb8be47e1887fbbee2456d" url = "https://github.com/opentofu/opentofu/releases/download/v1.12.5/tofu_1.12.5_darwin_amd64.tar.gz" +[tools.opentofu."platforms.windows-arm64"] +checksum = "sha256:af11850b496f3720e0184084c56d8b43aa74ea92d2338978bf368d70c96473f1" +url = "https://github.com/opentofu/opentofu/releases/download/v1.12.5/tofu_1.12.5_windows_amd64.tar.gz" + [tools.opentofu."platforms.windows-x64"] checksum = "sha256:af11850b496f3720e0184084c56d8b43aa74ea92d2338978bf368d70c96473f1" url = "https://github.com/opentofu/opentofu/releases/download/v1.12.5/tofu_1.12.5_windows_amd64.tar.gz" diff --git a/src/_firewall-bootstrap.tf b/src/_firewall-bootstrap.tf new file mode 100644 index 0000000..459d6a7 --- /dev/null +++ b/src/_firewall-bootstrap.tf @@ -0,0 +1,97 @@ +######################## +## FIREWALL BOOTSTRAP ## +######################## + +locals { + firewall_endpoint = try( + coalesce( + try(var.firewall_config.endpoint, null), + "https://${coalesce( + var.connectivity.firewall.lan_ip, + cidrhost(var.connectivity.firewall.lan_network_range, 4) + )}" + ), + "https://127.0.0.1" + ) + firewall_bootstrapped_credentials = try(jsondecode(file("${path.root}/.firewall-api-credentials.json")), null) + + firewall_secretsmanager_credentials = try({ + api_key = ephemeral.vault_kv_secret_v2.firewall_api[0].data.api_key + api_secret = ephemeral.vault_kv_secret_v2.firewall_api[0].data.api_secret + }, null) + + firewall_api_credentials = coalesce( + var.firewall_api_credentials, + local.firewall_bootstrapped_credentials, + local.firewall_secretsmanager_credentials, + { api_key = "unset", api_secret = "unset" }, + ) + + firewall_config_enabled = var.firewall_config != null && ( + var.firewall_api_credentials != null || + !var.firewall_bootstrap || + local.firewall_bootstrapped_credentials != null + ) +} + +############################################ +## FIREWALL API KEY SECRET - FIRST APPLY ## +############################################ + +resource "terraform_data" "firewall_api_bootstrap" { + count = ( + try(var.connectivity.firewall, null) != null && + var.firewall_bootstrap && + var.firewall_api_credentials == null && + local.firewall_bootstrapped_credentials == null + ) ? 1 : 0 + + triggers_replace = [ + local.firewall_endpoint, + try(module.connectivity[0].firewall_public_ip, ""), + ] + + provisioner "local-exec" { + command = "'${path.module}/modules/firewall-config/scripts/bootstrap-api-key.sh' '${local.firewall_endpoint}' '${var.firewall_admin_username}'" + interpreter = ["/usr/bin/env", "bash", "-c"] + + environment = { + OPNSENSE_PASSWORD = var.firewall_admin_password + OUTPUT_FILE = "${path.root}/.firewall-api-credentials.json" + } + } +} + +resource "vault_kv_secret_v2" "firewall_api_credentials" { + count = ( + try(var.connectivity.firewall, null) != null && ( + local.firewall_bootstrapped_credentials != null || + (var.firewall_config != null && !var.firewall_bootstrap) + ) + ) ? 1 : 0 + + mount = module.management.secretsmanager_instance_id + name = "firewall_api_${replace(var.company_code, "-", "_")}_pltfm_hub_prod" + delete_all_versions = true + data_json_wo = jsonencode( + local.firewall_bootstrapped_credentials != null + ? local.firewall_bootstrapped_credentials + : local.firewall_secretsmanager_credentials + ) + data_json_wo_version = var.firewall_api_secret_version +} + +############################################# +## FIREWALL API KEY SECRET - SECOND APPLY ## +############################################# + +ephemeral "vault_kv_secret_v2" "firewall_api" { + count = ( + try(var.connectivity.firewall, null) != null && + var.firewall_config != null && + !var.firewall_bootstrap + ) ? 1 : 0 + + mount = module.management.secretsmanager_instance_id + name = "firewall_api_${replace(var.company_code, "-", "_")}_pltfm_hub_prod" +} diff --git a/src/namespace-service.tf b/src/_landing-zone-kubernetes.tf similarity index 88% rename from src/namespace-service.tf rename to src/_landing-zone-kubernetes.tf index 7fcffc3..36e744b 100644 --- a/src/namespace-service.tf +++ b/src/_landing-zone-kubernetes.tf @@ -1,6 +1,15 @@ -############################# -## LANDING ZONE NAMESPACES ## -############################# +################################ +## LANDING ZONE ON KUBERNETES ## +################################ + +# The Kubernetes side of a landing zone: every entry in landing_zone_namespace_services +# gets a tenant slice of the shared platform cluster — its own namespace, a scoped service +# account with a Role limited to that namespace, and optionally a Kyverno policy that +# blocks direct Secret management so credentials have to come through the Secrets Manager. +# +# The sample workload behind sample_load is demo material, not part of the landing zone +# contract: a pod, a Gateway API route and a DNS record that together prove the path from +# the internet to a namespace works. Drop it once real workloads move in. locals { secrets_enforcement_default_exempt_principals = [ @@ -10,7 +19,7 @@ locals { landing_zone_namespace_services = { for key, value in var.landing_zone_namespace_services : key => { - demo_enabled = value.demo_enabled + demo_enabled = value.demo_enabled demo_metrics_ingestion = { enabled = value.demo_metrics_ingestion.enabled target_urls = value.demo_metrics_ingestion.target_urls @@ -46,25 +55,25 @@ locals { landing_zone_namespace_demo_services = { for key, value in local.landing_zone_namespace_services : key => { - namespace = value.namespace - use_secretsmanager = value.use_secretsmanager - landing_zone_project_id = module.landing_zone[key].project_id - secretsmanager_instance_id = module.landing_zone[key].secretsmanager_instance_id - observability_instance_id = module.landing_zone[key].observability_instance_id - observability_grafana_url = module.landing_zone[key].observability_grafana_url - observability_admin_user = module.landing_zone[key].observability_grafana_admin_user - demo_metrics_ingestion_enabled = value.demo_metrics_ingestion.enabled - demo_metrics_ingestion_target_urls = length(value.demo_metrics_ingestion.target_urls) > 0 ? value.demo_metrics_ingestion.target_urls : ( + namespace = value.namespace + use_secretsmanager = value.use_secretsmanager + landing_zone_project_id = module.landing_zone[key].project_id + secretsmanager_instance_id = module.landing_zone[key].secretsmanager_instance_id + observability_instance_id = module.landing_zone[key].observability_instance_id + observability_grafana_url = module.landing_zone[key].observability_grafana_url + observability_admin_user = module.landing_zone[key].observability_grafana_admin_user + demo_metrics_ingestion_enabled = value.demo_metrics_ingestion.enabled + demo_metrics_ingestion_target_urls = length(value.demo_metrics_ingestion.target_urls) > 0 ? value.demo_metrics_ingestion.target_urls : ( value.sample_load.enabled && value.dns_fqdn != null ? ["${value.dns_fqdn}:80"] : [] ) demo_metrics_ingestion_scheme = value.demo_metrics_ingestion.scheme demo_metrics_ingestion_metrics_path = value.demo_metrics_ingestion.metrics_path demo_metrics_ingestion_scrape_interval = value.demo_metrics_ingestion.scrape_interval demo_metrics_ingestion_scrape_timeout = value.demo_metrics_ingestion.scrape_timeout - platform_project_id = local.platform_kubernetes_cluster_key != null ? module.platform_kubernetes[local.platform_kubernetes_cluster_key].project_id : null - platform_observability_instance_id = local.platform_kubernetes_cluster_key != null ? module.platform_kubernetes[local.platform_kubernetes_cluster_key].observability_instance_id : null - platform_observability_targets_url = local.platform_kubernetes_cluster_key != null ? module.platform_kubernetes[local.platform_kubernetes_cluster_key].observability_targets_url : null - dns_zone_name = module.landing_zone[key].dns_zone_dns_name + platform_project_id = local.platform_kubernetes_cluster_key != null ? module.platform_kubernetes[local.platform_kubernetes_cluster_key].project_id : null + platform_observability_instance_id = local.platform_kubernetes_cluster_key != null ? module.platform_kubernetes[local.platform_kubernetes_cluster_key].observability_instance_id : null + platform_observability_targets_url = local.platform_kubernetes_cluster_key != null ? module.platform_kubernetes[local.platform_kubernetes_cluster_key].observability_targets_url : null + dns_zone_name = module.landing_zone[key].dns_zone_dns_name } if value.demo_enabled } @@ -405,14 +414,14 @@ resource "kubernetes_pod_v1" "landing_zone_sample_load" { namespace = each.value.metadata[0].namespace labels = { - "app.kubernetes.io/name" = "sample-load" + "app.kubernetes.io/name" = "sample-load" "stackit.cloud/landing-zone" = each.key "stackit.cloud/sample-load" = "true" } } spec { - restart_policy = "Never" + restart_policy = "Never" enable_service_links = false container { @@ -459,7 +468,7 @@ resource "kubernetes_service_v1" "landing_zone_sample_load" { namespace = each.value.metadata[0].namespace labels = { - "app.kubernetes.io/name" = "sample-load" + "app.kubernetes.io/name" = "sample-load" "stackit.cloud/landing-zone" = each.key "stackit.cloud/sample-load" = "true" } @@ -500,7 +509,7 @@ resource "kubernetes_manifest" "landing_zone_sample_gateway" { "external-dns.alpha.kubernetes.io/hostname" = each.value.dns_fqdn } labels = { - "app.kubernetes.io/name" = "sample-load" + "app.kubernetes.io/name" = "sample-load" "stackit.cloud/landing-zone" = each.key "stackit.cloud/sample-load" = "true" } @@ -549,7 +558,7 @@ resource "kubernetes_manifest" "landing_zone_sample_http_route" { name = "${kubernetes_service_v1.landing_zone_sample_load[each.key].metadata[0].name}-route" namespace = kubernetes_namespace_v1.landing_zone[each.key].metadata[0].name labels = { - "app.kubernetes.io/name" = "sample-load" + "app.kubernetes.io/name" = "sample-load" "stackit.cloud/landing-zone" = each.key "stackit.cloud/sample-load" = "true" } diff --git a/src/config/hub-and-spoke-firewall.tfvars b/src/config/hub-and-spoke-firewall.tfvars index 592040a..4d576a9 100644 --- a/src/config/hub-and-spoke-firewall.tfvars +++ b/src/config/hub-and-spoke-firewall.tfvars @@ -76,16 +76,229 @@ connectivity = { default_prefix_length = 25 } - # Delete the variable to skip firewall deployment (network area and routing still created) + # Delete the variable to skip firewall deployment firewall = { zone = "eu01-m" flavor = "c1.2" name = "opnsense-26.1" - lan_network_range = "10.0.0.0/28" - wan_network_range = "10.0.0.16/28" + lan_network_range = "10.0.2.0/28" + wan_network_range = "10.0.2.16/28" } + + # Optional: site-to-site IPsec VPN terminating in the hub. Uncomment to enable + # + # Pre-shared keys need to be supplied separately: + # export TF_VAR_vpn_pre_shared_keys='{"onprem"={"tunnel1"="<20+ chars>","tunnel2"="<20+ chars>"}}' + # + # vpn = { + # plan_id = "p100" # p100 = 1 connection, p500 = 3, p1000 = 5 + # routing_type = "ROUTE_BASED" # or POLICY_BASED + # + # availability_zones = { + # tunnel1 = "eu01-1" + # tunnel2 = "eu01-2" + # } + # + # connections = { + # "onprem" = { + # # Remote prefixes reachable through the tunnel, installed as routes in the network area + # static_routes = ["192.0.2.0/24"] + # + # # Point each tunnel at a different remote endpoint if the peer is redundant, + # # otherwise use the same address twice. + # tunnel1 = { remote_address = "198.51.100.10" } + # tunnel2 = { remote_address = "203.0.113.10" } + # } + # } + # } } +##################### +## FIREWALL POLICY ## +##################### + +# Rules, routes and NAT pushed to the OPNsense appliance through its API. +# +# The firewall needs to exist before configuring it. Bootstrapping is described here: +# docs/getting-started.md#configure-opnsense-firewall +# +# firewall_config = { +# endpoint = "https://198.51.100.20" # check bootstrap docs +# +# aliases = { +# # Everything routed inside the network area: all landing zones plus the appliance's +# # own LAN and WAN prefixes. Keep in sync with connectivity.network_area.ranges. +# network_area = { +# description = "All prefixes routed inside the STACKIT network area" +# content = ["10.0.0.0/16"] +# } +# +# # Sources allowed to reach the web GUI and the API. The whole network area is the +# # loosest setting that still keeps the GUI off the public internet. Narrow it to the +# # subnet the OpenTofu runner and the operators sit in once that is known. +# fw_management = { +# description = "May administer the firewall" +# content = ["10.0.0.0/16", "203.0.113.10/32"] # check bootstrap docs +# } +# +# # Resolvers handed to the landing zones through network_area.default_nameservers. +# # The landing zones reach them through this appliance, so they need an explicit rule +# # as soon as the blanket internet rule is switched off. +# platform_dns = { +# type = "host" +# description = "Resolvers the landing zones are pointed at" +# content = ["1.1.1.1", "1.0.0.1"] +# } +# +# # Example of a domain based allow rule. A host alias accepts FQDNs and OPNsense re-resolves them on a timer +# ubuntu_update_servers = { +# type = "host" +# description = "Canonical archive, security and changelog mirrors" +# content = [ +# "archive.ubuntu.com", +# "security.ubuntu.com", +# "changelogs.ubuntu.com", +# "esm.ubuntu.com", +# ] +# } +# +# ubuntu_update_ports = { +# type = "port" +# description = "Ports the Ubuntu mirrors are served on" +# content = ["80", "443"] +# } +# } +# +# # Both appliance interfaces are DHCP clients on a /28 and the default route leaves +# # through WAN, so without this the landing zone prefixes are only reachable the long way +# # round: spoke bound traffic would be sent to the WAN gateway and re-enter the network +# # area from the outside. That also drags LZ-to-LZ traffic through the WAN interface, +# # where the egress NAT rule would rewrite it and hide the real source address. +# routes = { +# network-area-via-lan = { +# network = "10.0.0.0/16" +# gateway = "LAN_DHCP" +# description = "Landing zones sit behind the LAN interface" +# } +# } +# +# rules = { +# allow-webgui-from-management = { +# sequence = 10 +# action = "pass" +# interfaces = [] # floating +# protocol = "TCP" +# source_net = "fw_management" +# destination_net = "(self)" +# destination_port = "443" +# } +# +# block-webgui-from-everywhere-else = { +# sequence = 20 +# action = "block" +# interfaces = [] # floating +# protocol = "TCP" +# source_net = "any" +# destination_net = "(self)" +# destination_port = "443" +# log = true +# } + +# allow-lz-to-dns = { +# sequence = 100 +# action = "pass" +# interfaces = ["lan"] +# protocol = "TCP/UDP" +# source_net = "network_area" +# destination_net = "platform_dns" +# destination_port = "53" +# } +# +# allow-lz-to-lz-https = { +# sequence = 110 +# action = "pass" +# interfaces = ["lan"] +# protocol = "TCP" +# source_net = "network_area" +# destination_net = "network_area" +# destination_port = "443" +# } +# +# allow-lz-to-lz-icmp = { +# sequence = 120 +# action = "pass" +# interfaces = ["lan"] +# protocol = "ICMP" +# source_net = "network_area" +# destination_net = "network_area" +# } +# +# block-lz-to-lz = { +# sequence = 130 +# action = "block" +# interfaces = ["lan"] +# protocol = "any" +# source_net = "network_area" +# destination_net = "network_area" +# log = true +# } +# +# allow-lz-to-ubuntu-updates = { +# sequence = 160 +# action = "pass" +# interfaces = ["lan"] +# protocol = "TCP" +# source_net = "network_area" +# destination_net = "ubuntu_update_servers" +# destination_port = "ubuntu_update_ports" +# } +# +# allow-lz-to-internet = { +# sequence = 200 +# action = "pass" +# interfaces = ["lan"] +# protocol = "any" +# source_net = "network_area" +# destination_net = "any" +# description = "Blanket egress. Disable this to switch to default-deny." +# } +# } +# +# # Without an entry the landing zones reach nothing, since their default route ends here. +# # target_ip takes an address, an alias, or ip. These are applied regardless of the +# # appliance's Outbound NAT mode, which the API cannot change and which stays automatic. +# outbound_nat = { +# no-nat-inside-network-area = { +# sequence = 10 +# interface = "wan" +# source_net = "network_area" +# destination_net = "network_area" +# disable_nat = true +# } +# +# # The public IP is bound to the WAN interface address, so egress has to be translated +# # to exactly that address for the platform to reach the internet at all. +# network-area-egress = { +# sequence = 100 +# interface = "wan" +# source_net = "network_area" +# target_ip = "wanip" +# } +# } +# +# # Inbound NAT. Only rewrites the destination — unlike the GUI the provider adds no +# # filter rule, so each forward needs its own pass rule above. +# port_forwards = { +# # web-to-dmz = { +# # source_net = "198.51.100.0/24" +# # destination_net = "wanip" +# # destination_port = "443" +# # target_ip = "10.0.0.20" +# # target_port = "8443" # omit to keep the destination port +# # } +# } +# } + ############ ## DEVOPS ## ############ diff --git a/src/config/hub-and-spoke.tfvars b/src/config/hub-and-spoke.tfvars index fc4852d..4cb2c1e 100644 --- a/src/config/hub-and-spoke.tfvars +++ b/src/config/hub-and-spoke.tfvars @@ -75,6 +75,39 @@ connectivity = { max_prefix_length = 28 default_prefix_length = 25 } + + # Optional: site-to-site IPsec VPN terminating in the hub, bridging the network area to + # on-premises or another cloud. Uncomment to enable — it provisions a billed VPN gateway. + # + # Roll out in two steps, because each side needs the other's public IP: + # 1. Apply with connections = {} to provision the gateway. + # 2. Read `tofu output connectivity_vpn_public_ips` and configure the remote peer. + # 3. Fill in remote_address below and apply again. + # + # Pre-shared keys are deliberately not stored here. Supply them separately: + # export TF_VAR_vpn_pre_shared_keys='{"onprem"={"tunnel1"="<20+ chars>","tunnel2"="<20+ chars>"}}' + # + # vpn = { + # plan_id = "p100" # p100 = 1 connection, p500 = 3, p1000 = 5 + # routing_type = "ROUTE_BASED" # or POLICY_BASED + # + # availability_zones = { + # tunnel1 = "eu01-1" + # tunnel2 = "eu01-2" + # } + # + # connections = { + # "onprem" = { + # # Remote prefixes reachable through the tunnel, installed as routes in the network area + # static_routes = ["192.0.2.0/24"] + # + # # Point each tunnel at a different remote endpoint if the peer is redundant, + # # otherwise use the same address twice. + # tunnel1 = { remote_address = "198.51.100.10" } + # tunnel2 = { remote_address = "203.0.113.10" } + # } + # } + # } } ############ diff --git a/src/main.tf b/src/main.tf index 5dff7c7..a352926 100644 --- a/src/main.tf +++ b/src/main.tf @@ -47,6 +47,25 @@ module "connectivity" { dns_zones = var.connectivity.dns_zones network_area = var.connectivity.network_area firewall = var.connectivity.firewall + vpn = var.connectivity.vpn + vpn_pre_shared_keys = var.vpn_pre_shared_keys +} + +##################### +## FIREWALL POLICY ## +##################### + +module "firewall_config" { + source = "./modules/firewall-config" + count = local.firewall_config_enabled ? 1 : 0 + + aliases = var.firewall_config.aliases + routes = var.firewall_config.routes + rules = var.firewall_config.rules + outbound_nat = var.firewall_config.outbound_nat + port_forwards = var.firewall_config.port_forwards + + depends_on = [terraform_data.firewall_api_bootstrap, module.connectivity] } ############ diff --git a/src/modules/connectivity/7-vpn.tf b/src/modules/connectivity/7-vpn.tf new file mode 100644 index 0000000..fec2cc1 --- /dev/null +++ b/src/modules/connectivity/7-vpn.tf @@ -0,0 +1,89 @@ +######### +## VPN ## +######### + +resource "stackit_vpn_gateway" "this" { + count = var.vpn != null ? 1 : 0 + + project_id = stackit_resourcemanager_project.this.project_id + display_name = var.vpn.display_name != null ? var.vpn.display_name : "${var.naming_pattern}-vpn" + plan_id = var.vpn.plan_id + routing_type = var.vpn.routing_type + labels = length(var.labels) > 0 ? var.labels : null # provider bug: empty map becomes null after apply + + availability_zones = { + tunnel1 = var.vpn.availability_zones.tunnel1 + tunnel2 = var.vpn.availability_zones.tunnel2 + } +} + +# The gateway public IPs are only exposed through the status endpoint, not on the gateway +# resource itself. They are required to configure the remote peer, so surface them as outputs. +data "stackit_vpn_gateway_status" "this" { + count = var.vpn != null ? 1 : 0 + + project_id = stackit_resourcemanager_project.this.project_id + gateway_id = stackit_vpn_gateway.this[0].gateway_id +} + +################# +## CONNECTIONS ## +################# + +resource "stackit_vpn_connection" "this" { + for_each = var.vpn != null ? var.vpn.connections : {} + + project_id = stackit_resourcemanager_project.this.project_id + gateway_id = stackit_vpn_gateway.this[0].gateway_id + display_name = each.value.display_name != null ? each.value.display_name : each.key + enabled = each.value.enabled + labels = length(var.labels) > 0 ? var.labels : null # provider bug: empty map becomes null after apply + + local_subnets = each.value.local_subnets + remote_subnets = each.value.remote_subnets + static_routes = each.value.static_routes + + tunnel1 = { + remote_address = each.value.tunnel1.remote_address + pre_shared_key = var.vpn_pre_shared_keys[each.key].tunnel1 + peering = each.value.tunnel1.peering + + phase1 = { + encryption_algorithms = each.value.tunnel1.phase1.encryption_algorithms + integrity_algorithms = each.value.tunnel1.phase1.integrity_algorithms + dh_groups = each.value.tunnel1.phase1.dh_groups + rekey_time = each.value.tunnel1.phase1.rekey_time + } + + phase2 = { + encryption_algorithms = each.value.tunnel1.phase2.encryption_algorithms + integrity_algorithms = each.value.tunnel1.phase2.integrity_algorithms + dh_groups = each.value.tunnel1.phase2.dh_groups + rekey_time = each.value.tunnel1.phase2.rekey_time + dpd_action = each.value.tunnel1.phase2.dpd_action + start_action = each.value.tunnel1.phase2.start_action + } + } + + tunnel2 = { + remote_address = each.value.tunnel2.remote_address + pre_shared_key = var.vpn_pre_shared_keys[each.key].tunnel2 + peering = each.value.tunnel2.peering + + phase1 = { + encryption_algorithms = each.value.tunnel2.phase1.encryption_algorithms + integrity_algorithms = each.value.tunnel2.phase1.integrity_algorithms + dh_groups = each.value.tunnel2.phase1.dh_groups + rekey_time = each.value.tunnel2.phase1.rekey_time + } + + phase2 = { + encryption_algorithms = each.value.tunnel2.phase2.encryption_algorithms + integrity_algorithms = each.value.tunnel2.phase2.integrity_algorithms + dh_groups = each.value.tunnel2.phase2.dh_groups + rekey_time = each.value.tunnel2.phase2.rekey_time + dpd_action = each.value.tunnel2.phase2.dpd_action + start_action = each.value.tunnel2.phase2.start_action + } + } +} diff --git a/src/modules/connectivity/README.md b/src/modules/connectivity/README.md index 0b8bcc6..ba3b56a 100644 --- a/src/modules/connectivity/README.md +++ b/src/modules/connectivity/README.md @@ -4,15 +4,15 @@ | Name | Version | |------|---------| | [terraform](#requirement\_terraform) | >= 1.10 | -| [stackit](#requirement\_stackit) | >=0.93.0 | -| [time](#requirement\_time) | >= 0.13.0 | +| [stackit](#requirement\_stackit) | 0.101.0 | +| [time](#requirement\_time) | ~> 0.14.0 | ## Providers | Name | Version | |------|---------| | [stackit](#provider\_stackit) | 0.93.0 | -| [time](#provider\_time) | >= 0.13.0 | +| [time](#provider\_time) | ~> 0.14.0 | ## Modules @@ -22,23 +22,26 @@ No modules. | Name | Type | |------|------| -| [stackit_authorization_project_role_assignment.this](https://registry.terraform.io/providers/stackitcloud/stackit/latest/docs/resources/authorization_project_role_assignment) | resource | -| [stackit_dns_zone.this](https://registry.terraform.io/providers/stackitcloud/stackit/latest/docs/resources/dns_zone) | resource | -| [stackit_image.firewall](https://registry.terraform.io/providers/stackitcloud/stackit/latest/docs/resources/image) | resource | -| [stackit_network.lan](https://registry.terraform.io/providers/stackitcloud/stackit/latest/docs/resources/network) | resource | -| [stackit_network.wan](https://registry.terraform.io/providers/stackitcloud/stackit/latest/docs/resources/network) | resource | -| [stackit_network_area.this](https://registry.terraform.io/providers/stackitcloud/stackit/latest/docs/resources/network_area) | resource | -| [stackit_network_area_region.this](https://registry.terraform.io/providers/stackitcloud/stackit/latest/docs/resources/network_area_region) | resource | -| [stackit_network_interface.lan](https://registry.terraform.io/providers/stackitcloud/stackit/latest/docs/resources/network_interface) | resource | -| [stackit_network_interface.wan](https://registry.terraform.io/providers/stackitcloud/stackit/latest/docs/resources/network_interface) | resource | -| [stackit_public_ip.wan-ip](https://registry.terraform.io/providers/stackitcloud/stackit/latest/docs/resources/public_ip) | resource | -| [stackit_resourcemanager_project.this](https://registry.terraform.io/providers/stackitcloud/stackit/latest/docs/resources/resourcemanager_project) | resource | -| [stackit_routing_table.wan](https://registry.terraform.io/providers/stackitcloud/stackit/latest/docs/resources/routing_table) | resource | -| [stackit_routing_table_route.wan](https://registry.terraform.io/providers/stackitcloud/stackit/latest/docs/resources/routing_table_route) | resource | -| [stackit_server.firewall](https://registry.terraform.io/providers/stackitcloud/stackit/latest/docs/resources/server) | resource | -| [stackit_volume.firewall](https://registry.terraform.io/providers/stackitcloud/stackit/latest/docs/resources/volume) | resource | +| [stackit_authorization_project_role_assignment.this](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/authorization_project_role_assignment) | resource | +| [stackit_dns_zone.this](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/dns_zone) | resource | +| [stackit_image.firewall](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/image) | resource | +| [stackit_network.lan](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/network) | resource | +| [stackit_network.wan](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/network) | resource | +| [stackit_network_area.this](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/network_area) | resource | +| [stackit_network_area_region.this](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/network_area_region) | resource | +| [stackit_network_interface.lan](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/network_interface) | resource | +| [stackit_network_interface.wan](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/network_interface) | resource | +| [stackit_public_ip.wan-ip](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/public_ip) | resource | +| [stackit_resourcemanager_project.this](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/resourcemanager_project) | resource | +| [stackit_routing_table.wan](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/routing_table) | resource | +| [stackit_routing_table_route.wan](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/routing_table_route) | resource | +| [stackit_server.firewall](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/server) | resource | +| [stackit_volume.firewall](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/volume) | resource | +| [stackit_vpn_connection.this](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/vpn_connection) | resource | +| [stackit_vpn_gateway.this](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/resources/vpn_gateway) | resource | | [time_sleep.wait_before_network_area_region_destroy](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) | resource | | [time_sleep.wait_for_network_area](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) | resource | +| [stackit_vpn_gateway_status.this](https://registry.terraform.io/providers/stackitcloud/stackit/0.101.0/docs/data-sources/vpn_gateway_status) | data source | ## Inputs @@ -55,6 +58,8 @@ No modules. | [parent\_container\_id](#input\_parent\_container\_id) | Parent container ID (folder or organization) where the project will be created. | `string` | n/a | yes | | [project\_name](#input\_project\_name) | Name of the STACKIT project to create. Falls back to naming\_pattern if not set. | `string` | `null` | no | | [role\_assignments](#input\_role\_assignments) | List of role assignments for the project. Subject can be a user email or service account email. | 
list(object({
    role    = string
    subject = string
  }))
| `[]` | no | +| [vpn](#input\_vpn) | IPsec VPN gateway for the hub, attached to the network area through the connectivity project. Set to null to skip. The gateway is HA: it terminates two tunnels in separate availability zones, each with its own public IP. Connections are created in a second apply once the remote peer addresses are known. Supports POLICY\_BASED and ROUTE\_BASED routing. | 
object({
    display_name = optional(string, null)
    plan_id      = optional(string, "p100")
    routing_type = optional(string, "ROUTE_BASED")
    availability_zones = object({
      tunnel1 = string
      tunnel2 = string
    })
    connections = optional(map(object({
      display_name   = optional(string, null)
      enabled        = optional(bool, true)
      local_subnets  = optional(list(string), null)
      remote_subnets = optional(list(string), null)
      static_routes  = optional(list(string), null)
      tunnel1 = object({
        remote_address = string
        peering = optional(object({
          local_address  = string
          remote_address = string
        }), null)
        phase1 = optional(object({
          encryption_algorithms = optional(list(string), ["aes256"])
          integrity_algorithms  = optional(list(string), ["sha2_384"])
          dh_groups             = optional(list(string), ["ecp384"])
          rekey_time            = optional(number, null)
        }), {})
        phase2 = optional(object({
          encryption_algorithms = optional(list(string), ["aes256"])
          integrity_algorithms  = optional(list(string), ["sha2_384"])
          dh_groups             = optional(list(string), ["ecp384"])
          rekey_time            = optional(number, null)
          dpd_action            = optional(string, null)
          start_action          = optional(string, null)
        }), {})
      })
      tunnel2 = object({
        remote_address = string
        peering = optional(object({
          local_address  = string
          remote_address = string
        }), null)
        phase1 = optional(object({
          encryption_algorithms = optional(list(string), ["aes256"])
          integrity_algorithms  = optional(list(string), ["sha2_384"])
          dh_groups             = optional(list(string), ["ecp384"])
          rekey_time            = optional(number, null)
        }), {})
        phase2 = optional(object({
          encryption_algorithms = optional(list(string), ["aes256"])
          integrity_algorithms  = optional(list(string), ["sha2_384"])
          dh_groups             = optional(list(string), ["ecp384"])
          rekey_time            = optional(number, null)
          dpd_action            = optional(string, null)
          start_action          = optional(string, null)
        }), {})
      })
    })), {})
  })
| `null` | no | +| [vpn\_pre\_shared\_keys](#input\_vpn\_pre\_shared\_keys) | Pre-shared keys per VPN connection key, one per tunnel. Kept separate from var.vpn so the connection topology stays committable; supply through TF\_VAR\_vpn\_pre\_shared\_keys or a gitignored tfvars file. Minimum 20 characters. | 
map(object({
    tunnel1 = string
    tunnel2 = string
  }))
| `{}` | no | ## Outputs @@ -68,4 +73,8 @@ No modules. | [project\_container\_id](#output\_project\_container\_id) | The container ID of the created STACKIT project. | | [project\_id](#output\_project\_id) | The project ID of the created STACKIT project. | | [project\_name](#output\_project\_name) | The name of the created STACKIT project. | +| [vpn\_connection\_ids](#output\_vpn\_connection\_ids) | Map of VPN connection keys to their connection IDs. | +| [vpn\_gateway\_id](#output\_vpn\_gateway\_id) | The ID of the VPN gateway in the hub. | +| [vpn\_internal\_next\_hop\_ips](#output\_vpn\_internal\_next\_hop\_ips) | Map of VPN tunnel names to their network area side IP. Ping targets to verify a tunnel carries traffic into the SNA. | +| [vpn\_public\_ips](#output\_vpn\_public\_ips) | Map of VPN tunnel names to their public IP. These are the addresses the remote peer has to be configured against. | \ No newline at end of file diff --git a/src/modules/connectivity/outputs.tf b/src/modules/connectivity/outputs.tf index 14c3e4b..6534ee5 100644 --- a/src/modules/connectivity/outputs.tf +++ b/src/modules/connectivity/outputs.tf @@ -37,3 +37,23 @@ output "project_name" { description = "The name of the created STACKIT project." value = stackit_resourcemanager_project.this.name } + +output "vpn_connection_ids" { + description = "Map of VPN connection keys to their connection IDs." + value = { for k, c in stackit_vpn_connection.this : k => c.connection_id } +} + +output "vpn_gateway_id" { + description = "The ID of the VPN gateway in the hub." + value = try(stackit_vpn_gateway.this[0].gateway_id, null) +} + +output "vpn_internal_next_hop_ips" { + description = "Map of VPN tunnel names to their network area side IP. Ping targets to verify a tunnel carries traffic into the SNA." + value = try({ for t in data.stackit_vpn_gateway_status.this[0].tunnels : t.name => t.internal_next_hop_ip }, {}) +} + +output "vpn_public_ips" { + description = "Map of VPN tunnel names to their public IP. These are the addresses the remote peer has to be configured against." + value = try({ for t in data.stackit_vpn_gateway_status.this[0].tunnels : t.name => t.public_ip }, {}) +} diff --git a/src/modules/connectivity/terraform.tf b/src/modules/connectivity/terraform.tf index f79ffe1..5f5a4ee 100644 --- a/src/modules/connectivity/terraform.tf +++ b/src/modules/connectivity/terraform.tf @@ -4,7 +4,7 @@ terraform { required_providers { stackit = { source = "stackitcloud/stackit" - version = "0.101.0" + version = "0.104.0" } time = { source = "hashicorp/time" diff --git a/src/modules/connectivity/variables.tf b/src/modules/connectivity/variables.tf index 15ba03b..297ab25 100644 --- a/src/modules/connectivity/variables.tf +++ b/src/modules/connectivity/variables.tf @@ -90,4 +90,121 @@ variable "role_assignments" { })) description = "List of role assignments for the project. Subject can be a user email or service account email." default = [] -} \ No newline at end of file +} + +variable "vpn" { + type = object({ + display_name = optional(string, null) + plan_id = optional(string, "p100") + routing_type = optional(string, "ROUTE_BASED") + availability_zones = object({ + tunnel1 = string + tunnel2 = string + }) + connections = optional(map(object({ + display_name = optional(string, null) + enabled = optional(bool, true) + local_subnets = optional(list(string), null) + remote_subnets = optional(list(string), null) + static_routes = optional(list(string), null) + tunnel1 = object({ + remote_address = string + peering = optional(object({ + local_address = string + remote_address = string + }), null) + phase1 = optional(object({ + encryption_algorithms = optional(list(string), ["aes256"]) + integrity_algorithms = optional(list(string), ["sha2_384"]) + dh_groups = optional(list(string), ["ecp384"]) + rekey_time = optional(number, null) + }), {}) + phase2 = optional(object({ + encryption_algorithms = optional(list(string), ["aes256"]) + integrity_algorithms = optional(list(string), ["sha2_384"]) + dh_groups = optional(list(string), ["ecp384"]) + rekey_time = optional(number, null) + dpd_action = optional(string, null) + start_action = optional(string, null) + }), {}) + }) + tunnel2 = object({ + remote_address = string + peering = optional(object({ + local_address = string + remote_address = string + }), null) + phase1 = optional(object({ + encryption_algorithms = optional(list(string), ["aes256"]) + integrity_algorithms = optional(list(string), ["sha2_384"]) + dh_groups = optional(list(string), ["ecp384"]) + rekey_time = optional(number, null) + }), {}) + phase2 = optional(object({ + encryption_algorithms = optional(list(string), ["aes256"]) + integrity_algorithms = optional(list(string), ["sha2_384"]) + dh_groups = optional(list(string), ["ecp384"]) + rekey_time = optional(number, null) + dpd_action = optional(string, null) + start_action = optional(string, null) + }), {}) + }) + })), {}) + }) + # BGP_ROUTE_BASED is intentionally unsupported: it needs the gateway-level bgp attribute, and + # stackitcloud/stackit (through 0.104.0) cannot convert an unknown value for that nested object, + # which breaks `tofu validate` in CI. Re-add the bgp attribute once the provider handles it. + description = "IPsec VPN gateway for the hub, attached to the network area through the connectivity project. Set to null to skip. The gateway is HA: it terminates two tunnels in separate availability zones, each with its own public IP. Connections are created in a second apply once the remote peer addresses are known. Supports POLICY_BASED and ROUTE_BASED routing." + default = null + + validation { + condition = var.vpn == null || contains(["POLICY_BASED", "ROUTE_BASED"], try(var.vpn.routing_type, "")) + error_message = "vpn.routing_type must be one of POLICY_BASED, ROUTE_BASED. BGP_ROUTE_BASED is not supported by this module yet." + } + + validation { + condition = var.vpn == null || can(regex("^p[0-9]+$", var.vpn.plan_id)) + error_message = "vpn.plan_id must be a STACKIT VPN plan identifier (e.g. p100, p500, p1000). List plans with: stackit curl https://vpn.api.eu01.stackit.cloud/v1beta1/regions/eu01/plans" + } + + # Route-based gateways steer traffic over a virtual tunnel interface, so the remote prefixes + # have to be installed as static routes. Policy-based gateways derive them from the SA selectors. + validation { + condition = var.vpn == null || var.vpn.routing_type != "ROUTE_BASED" || alltrue([ + for c in values(var.vpn.connections) : c.static_routes != null && length(coalesce(c.static_routes, [])) > 0 + ]) + error_message = "vpn.connections[*].static_routes is mandatory when vpn.routing_type is ROUTE_BASED." + } + + validation { + condition = var.vpn == null || var.vpn.routing_type != "POLICY_BASED" || alltrue([ + for c in values(var.vpn.connections) : + length(coalesce(c.local_subnets, [])) > 0 && length(coalesce(c.remote_subnets, [])) > 0 + ]) + error_message = "vpn.connections[*].local_subnets and remote_subnets are mandatory when vpn.routing_type is POLICY_BASED." + } +} + +variable "vpn_pre_shared_keys" { + type = map(object({ + tunnel1 = string + tunnel2 = string + })) + description = "Pre-shared keys per VPN connection key, one per tunnel. Kept separate from var.vpn so the connection topology stays committable; supply through TF_VAR_vpn_pre_shared_keys or a gitignored tfvars file. Minimum 20 characters." + default = {} + sensitive = true + + validation { + condition = var.vpn == null || alltrue([ + for k in keys(var.vpn.connections) : contains(keys(var.vpn_pre_shared_keys), k) + ]) + error_message = "Every key in vpn.connections needs a matching entry in vpn_pre_shared_keys." + } + + validation { + condition = alltrue([ + for psk in values(var.vpn_pre_shared_keys) : length(psk.tunnel1) >= 20 && length(psk.tunnel2) >= 20 + ]) + error_message = "Pre-shared keys must be at least 20 characters long." + } +} diff --git a/src/modules/debug-bastion/terraform.tf b/src/modules/debug-bastion/terraform.tf index 2a3fcb9..24b6681 100644 --- a/src/modules/debug-bastion/terraform.tf +++ b/src/modules/debug-bastion/terraform.tf @@ -2,7 +2,7 @@ terraform { required_providers { stackit = { source = "stackitcloud/stackit" - version = "0.101.0" + version = "0.104.0" } } } diff --git a/src/modules/devops/terraform.tf b/src/modules/devops/terraform.tf index 0ea12d2..0626f40 100644 --- a/src/modules/devops/terraform.tf +++ b/src/modules/devops/terraform.tf @@ -4,7 +4,7 @@ terraform { required_providers { stackit = { source = "stackitcloud/stackit" - version = "0.101.0" + version = "0.104.0" } } } \ No newline at end of file diff --git a/src/modules/firewall-config/1-aliases.tf b/src/modules/firewall-config/1-aliases.tf new file mode 100644 index 0000000..ad74856 --- /dev/null +++ b/src/modules/firewall-config/1-aliases.tf @@ -0,0 +1,22 @@ +########################## +## CATEGORY AND ALIASES ## +########################## + +resource "opnsense_firewall_category" "this" { + name = var.category_name +} + +resource "opnsense_firewall_alias" "this" { + for_each = var.aliases + + name = each.key + type = each.value.type + enabled = each.value.enabled + description = each.value.description != null ? each.value.description : each.key + content = each.value.content + stats = each.value.stats + categories = [opnsense_firewall_category.this.id] + + # Only read for urltable aliases, where OPNsense requires it. null keeps the provider default. + update_freq = each.value.update_freq +} diff --git a/src/modules/firewall-config/2-rules.tf b/src/modules/firewall-config/2-rules.tf new file mode 100644 index 0000000..94835c4 --- /dev/null +++ b/src/modules/firewall-config/2-rules.tf @@ -0,0 +1,39 @@ +########### +## RULES ## +########### + +resource "opnsense_firewall_filter" "this" { + for_each = var.rules + + enabled = each.value.enabled + sequence = each.value.sequence + description = each.value.description != null ? each.value.description : each.key + categories = [opnsense_firewall_category.this.id] + + interface = { + interface = each.value.interfaces + } + + filter = { + action = each.value.action + direction = each.value.direction + protocol = each.value.protocol + ip_protocol = each.value.ip_protocol + quick = each.value.quick + log = each.value.log + + source = { + net = each.value.source_net + port = each.value.source_port != null ? each.value.source_port : "" + invert = each.value.source_invert + } + + destination = { + net = each.value.destination_net + port = each.value.destination_port != null ? each.value.destination_port : "" + invert = each.value.destination_invert + } + } + + depends_on = [opnsense_firewall_alias.this] +} diff --git a/src/modules/firewall-config/3-nat.tf b/src/modules/firewall-config/3-nat.tf new file mode 100644 index 0000000..f1f3dd8 --- /dev/null +++ b/src/modules/firewall-config/3-nat.tf @@ -0,0 +1,65 @@ +######### +## NAT ## +######### + +resource "opnsense_firewall_nat" "this" { + for_each = var.outbound_nat + + enabled = each.value.enabled + sequence = each.value.sequence + interface = each.value.interface + protocol = each.value.protocol + ip_protocol = each.value.ip_protocol + disable_nat = each.value.disable_nat + log = each.value.log + # OPNsense only accepts alphanumerics, spaces and dots in a NAT description, while + # the map keys are kebab-case, so fall back to a sanitised form of the key. + description = each.value.description != null ? each.value.description : replace(each.key, "/[^a-zA-Z0-9. ]/", " ") + + source = { + net = each.value.source_net + } + + destination = { + net = each.value.destination_net + } + + target = { + ip = each.value.target_ip + } + + depends_on = [opnsense_firewall_alias.this] +} + +################### +## PORT FORWARDS ## +################### + +resource "opnsense_firewall_nat_port_forward" "this" { + for_each = var.port_forwards + + enabled = each.value.enabled + sequence = each.value.sequence + interface = toset(each.value.interfaces) + protocol = each.value.protocol + ip_protocol = each.value.ip_protocol + log = each.value.log + nat_reflection = each.value.nat_reflection + description = each.value.description != null ? each.value.description : each.key + + source = { + net = each.value.source_net + } + + destination = { + net = each.value.destination_net + port = each.value.destination_port + } + + target = { + ip = each.value.target_ip + port = each.value.target_port != null ? each.value.target_port : each.value.destination_port + } + + depends_on = [opnsense_firewall_alias.this] +} diff --git a/src/modules/firewall-config/4-routes.tf b/src/modules/firewall-config/4-routes.tf new file mode 100644 index 0000000..2620c46 --- /dev/null +++ b/src/modules/firewall-config/4-routes.tf @@ -0,0 +1,12 @@ +############ +## ROUTES ## +############ + +resource "opnsense_route" "this" { + for_each = var.routes + + enabled = each.value.enabled + network = each.value.network + gateway = each.value.gateway + description = each.value.description != null ? each.value.description : each.key +} diff --git a/src/modules/firewall-config/README.md b/src/modules/firewall-config/README.md new file mode 100644 index 0000000..f2d1577 --- /dev/null +++ b/src/modules/firewall-config/README.md @@ -0,0 +1,215 @@ +# firewall-config + +Pushes firewall rules, NAT and internet-facing hardening to an OPNsense appliance through +its API, using the [`browningluke/opnsense`](https://registry.terraform.io/providers/browningluke/opnsense/latest/docs) +provider. + +The appliance itself is deployed by the `connectivity` module (`connectivity.firewall`); +this module only configures it. Wire it up through the root `firewall_config` variable +rather than calling it directly — see `src/config/hub-and-spoke-firewall.tfvars`. + +## What it manages + +- **Aliases** — named network, host, port or FQDN groups referenced from rules and NAT entries. +- **Static routes** — teaches the appliance which prefixes sit behind which gateway. +- **Filter rules** — pass, block or reject, per interface or floating. +- **Outbound NAT** — landing zones egress behind the firewall's public address. +- **Port forwards** — inbound exposure from the internet, guarded (see below). + +The module contains no policy of its own. Every rule comes from the caller, so the +effective configuration is always readable in one place instead of being split between +tfvars and module internals. + +## Rule ordering + +Everything written here lands in OPNsense's *automation* ruleset, which is evaluated +ahead of the rules configured under **Firewall → Rules**. The full order is: + +1. system rules (anti-lockout, DHCP, loopback) +2. automation **and** manual floating rules — priority `200000` +3. automation **and** manual interface *group* rules — `300000` +4. **automation** single interface rules — `400000` +5. **manual** single interface rules + +Two consequences for this topology: + +- The image's `Default allow LAN to any rule` is a manual interface rule, so it sits at + step 5. A `block` rule written here on `lan` is reached first and wins, which is what + makes a default-deny policy on the LAN side possible without touching the image. +- Within a step the lowest `sequence` wins, and rules default to `quick = true`, so the + first match decides. Leave gaps between sequence numbers. + +Setting `interfaces = []` produces a floating rule (step 2), the only way to get ahead of +a floating rule that the image already ships. + +## Routes + +Both appliance interfaces are DHCP clients on a `/28`, so out of the box OPNsense only +knows those two prefixes and a default route through WAN. The landing zones are further +inside the network area and reachable through the LAN gateway, which has to be stated: + +```hcl +routes = { + network-area-via-lan = { + network = "10.0.0.0/16" + gateway = "LAN_DHCP" + } +} +``` + +Without it, spoke-bound traffic leaves through WAN and re-enters the network area from the +outside. That still reaches the spokes, because the WAN routing table carries system +routes, but it also drags landing-zone-to-landing-zone traffic across the WAN interface +where the egress NAT rule rewrites the source address — so the receiving spoke sees the +firewall instead of the sending spoke, and source based rules stop meaning anything. + +`gateway` must name a gateway that already exists on the appliance. The STACKIT image +ships `LAN_DHCP` and `WAN_DHCP`, with `WAN_DHCP` marked as the default. + +## Domain based rules + +An alias with `type = "host"` accepts FQDNs, and the appliance re-resolves them on a +timer, so a rule pointing at it follows the addresses as they change. `type = "urltable"` +fetches a list of prefixes from a URL instead and requires `update_freq`, in days. + +This is still packet filtering against resolved addresses, not name based filtering. A +client that resolves the same name to an address the appliance has not seen yet is +dropped, and anything else sharing an address that *was* resolved is let through. For a +hard guarantee, point at a mirror with a stable address or put a proxy in the path. + +## Closing the web GUI to the internet + +The STACKIT image ships a **floating** rule named `Initial webUI Access Rule` that passes +TCP/443 from any source to the firewall itself. After a plain deployment the web GUI +answers on the public IP. + +Overriding it takes two rules, and both have to be floating as well: OPNsense evaluates +floating rules (priority group 200000) before interface-bound ones (400000), so a rule +attached to WAN never gets the chance to match first. `src/config/hub-and-spoke-firewall.tfvars` +carries the worked example: + +| Sequence | Rule | Effect | +| --- | --- | --- | +| 10 | pass TCP → `(self):443` from a management alias | keeps administration working | +| 20 | block TCP → `(self):443` from any, floating | closes the GUI for everyone else | + +Verified against a live appliance: an allowed source got `HTTP 200` while an internet +source that was not on the list was dropped. + +> [!WARNING] +> The management alias has to contain whatever runs OpenTofu before the block rule is +> applied. Otherwise the apply that closes the GUI also cuts off the connection needed +> for the next run, and recovery goes through the serial console. + +> [!NOTE] +> A "block RFC1918 on WAN" rule is a common hardening reflex but does not fit this +> topology unchecked: the appliance's WAN interface sits on a network area prefix +> (`wan_network_range`), not on an internet edge, so such a rule can drop legitimate +> internal traffic. Test it before adopting it. + +## NAT + +OPNsense splits NAT into three sections. The module covers the first two: + +| OPNsense GUI | NAT direction | Module variable | +| --- | --- | --- | +| Firewall → NAT → Outbound | source NAT | `outbound_nat` | +| Firewall → NAT → Port Forward | destination NAT, i.e. inbound | `port_forwards` | +| Firewall → NAT → One-to-One | bidirectional 1:1 | not exposed | + +Without at least one `outbound_nat` entry the landing zones have no egress identity, +because their default route ends at this appliance. Translate to `wanip`: the STACKIT +public IP is bound to the WAN interface address, so any other source address is not +translated at the network edge and never reaches the internet. + +The image leaves **Firewall → NAT → Outbound** on `automatic`, and the API cannot change +that mode. It does not have to: rules created here are automation rules, which are applied +irrespective of the mode and match ahead of anything under `Firewall → NAT`. The automatic +rules the appliance generates only cover its own directly connected `/28`s, so they never +collide with landing zone traffic. + +`disable_nat = true` marks traffic as untranslated *and stops outbound NAT processing for +it*, so a no-NAT entry only works if its `sequence` is lower than the entry it has to beat. + +> [!IMPORTANT] +> A port forward only rewrites the destination address. The GUI offers to create the +> matching filter rule alongside it; the provider has no such option, so every forward +> needs its own pass rule in `rules` permitting the translated traffic on the WAN +> interface. Without it the packet is translated and then dropped. + +Every entry in `port_forwards` opens a path from the internet, so the root variable +rejects an enabled forward whose `source_net` is still `any`. Narrow it, or disable the +entry. + +NAT descriptions accept only alphanumerics, spaces and dots. Kebab-case map keys are +sanitised automatically when no explicit `description` is given. + +## Reachability + +The API lives on the appliance's web GUI on the LAN address inside the network area. +Whatever runs OpenTofu has to reach it from there — a bastion in the hub, a landing zone, +or the site-to-site VPN. A runner on the public internet only works while the GUI is +still exposed, which is exactly what the block rule removes. + +## Provider choice + +The generic [`Mastercard/restapi`](https://registry.terraform.io/providers/Mastercard/restapi/latest/docs) +provider also drives this API, but OPNsense answers reads in an expanded form (every +select field becomes an options object with `selected` flags) that never matches what was +written. Suppressing the resulting permanent diff with `ignore_all_server_changes` also +suppresses diffs coming from the configuration, which makes objects write-once — measured: +adding a CIDR to an alias reported `No changes` while the appliance kept the old content. + +`browningluke/opnsense` maps the two forms internally, so in-place updates and drift +detection work. It is community maintained and its author advises against production use. +If an endpoint it does not cover is needed, the generic provider remains a viable +escape hatch for that specific object. + + +## Requirements + +| Name | Version | +| ---- | ------- | +| [terraform](#requirement\_terraform) | >= 1.10 | +| [opnsense](#requirement\_opnsense) | 0.24.0 | + +## Providers + +| Name | Version | +| ---- | ------- | +| [opnsense](#provider\_opnsense) | 0.24.0 | + +## Modules + +No modules. + +## Resources + +| Name | Type | +| ---- | ---- | +| [opnsense_firewall_alias.this](https://registry.terraform.io/providers/browningluke/opnsense/0.24.0/docs/resources/firewall_alias) | resource | +| [opnsense_firewall_category.this](https://registry.terraform.io/providers/browningluke/opnsense/0.24.0/docs/resources/firewall_category) | resource | +| [opnsense_firewall_filter.this](https://registry.terraform.io/providers/browningluke/opnsense/0.24.0/docs/resources/firewall_filter) | resource | +| [opnsense_firewall_nat.this](https://registry.terraform.io/providers/browningluke/opnsense/0.24.0/docs/resources/firewall_nat) | resource | +| [opnsense_firewall_nat_port_forward.this](https://registry.terraform.io/providers/browningluke/opnsense/0.24.0/docs/resources/firewall_nat_port_forward) | resource | +| [opnsense_route.this](https://registry.terraform.io/providers/browningluke/opnsense/0.24.0/docs/resources/route) | resource | + +## Inputs + +| Name | Description | Type | Default | Required | +| ---- | ----------- | ---- | ------- | :------: | +| [aliases](#input\_aliases) | Firewall aliases keyed by alias name. Reference an alias from a rule, route or NAT entry by using its key wherever a network is expected. type = "host" also accepts FQDNs, which the appliance re-resolves periodically. | 
map(object({
    type        = optional(string, "network")
    enabled     = optional(bool, true)
    description = optional(string, null)
    content     = optional(list(string), [])
    update_freq = optional(number, null)
    stats       = optional(bool, false)
  }))
| `{}` | no | +| [category\_name](#input\_category\_name) | OPNsense category every object created by this module is tagged with, so managed objects are recognisable in the GUI. | `string` | `"landing-zone"` | no | +| [outbound\_nat](#input\_outbound\_nat) | Additional outbound NAT rules keyed by name. target\_ip accepts an address, an alias, or ip such as wanip. | 
map(object({
    sequence        = optional(number, 200)
    enabled         = optional(bool, true)
    interface       = optional(string, "wan")
    protocol        = optional(string, "any")
    ip_protocol     = optional(string, "inet")
    source_net      = optional(string, "any")
    destination_net = optional(string, "any")
    target_ip       = optional(string, "wanip")
    disable_nat     = optional(bool, false)
    log             = optional(bool, false)
    description     = optional(string, null)
  }))
| `{}` | no | +| [port\_forwards](#input\_port\_forwards) | Inbound port forwards from the internet keyed by name. Every entry punches a hole through the WAN, so keep the list short and set source\_net where possible. | 
map(object({
    sequence         = optional(number, 100)
    enabled          = optional(bool, true)
    interfaces       = optional(list(string), ["wan"])
    protocol         = optional(string, "TCP")
    ip_protocol      = optional(string, "inet")
    source_net       = optional(string, "any")
    destination_net  = optional(string, "wanip")
    destination_port = string
    target_ip        = string
    target_port      = optional(string, null)
    nat_reflection   = optional(string, "default")
    log              = optional(bool, true)
    description      = optional(string, null)
  }))
| `{}` | no | +| [routes](#input\_routes) | Static routes keyed by name. gateway must name a gateway that exists on the appliance; the STACKIT image ships LAN\_DHCP and WAN\_DHCP. | 
map(object({
    enabled     = optional(bool, true)
    network     = string
    gateway     = string
    description = optional(string, null)
  }))
| `{}` | no | +| [rules](#input\_rules) | Firewall filter rules keyed by name. An empty interfaces list creates a floating rule, which OPNsense evaluates before interface-bound rules. | 
map(object({
    sequence           = optional(number, 100)
    enabled            = optional(bool, true)
    action             = optional(string, "pass")
    direction          = optional(string, "in")
    interfaces         = optional(list(string), ["lan"])
    protocol           = optional(string, "any")
    ip_protocol        = optional(string, "inet")
    quick              = optional(bool, true)
    source_net         = optional(string, "any")
    source_port        = optional(string, null)
    source_invert      = optional(bool, false)
    destination_net    = optional(string, "any")
    destination_port   = optional(string, null)
    destination_invert = optional(bool, false)
    log                = optional(bool, false)
    description        = optional(string, null)
  }))
| `{}` | no | + +## Outputs + +| Name | Description | +| ---- | ----------- | +| [category](#output\_category) | Name and UUID of the category every object created by this module is tagged with. Filtering on the name in the OPNsense GUI shows exactly what OpenTofu manages and nothing else. | +| [managed\_objects](#output\_managed\_objects) | Names of the objects this module holds on the appliance, by kind. Read from the resources rather than the inputs, so it reflects what was actually pushed. | +| [rule\_evaluation\_order](#output\_rule\_evaluation\_order) | Filter rules in the order the appliance evaluates them. Every rule defaults to quick, so the first match decides and this is the list to read when a rule does not behave as expected. Derived from the inputs, so it is reviewable in a plan before anything is pushed. | + diff --git a/src/modules/firewall-config/outputs.tf b/src/modules/firewall-config/outputs.tf new file mode 100644 index 0000000..2779c14 --- /dev/null +++ b/src/modules/firewall-config/outputs.tf @@ -0,0 +1,52 @@ +locals { + # OPNsense evaluates floating rules first (priority group 200000), then interface group + # rules (300000), then single interface rules (400000), and within a group by ascending + # sequence. Encoding group and sequence into one sortable string reproduces that order, + # so the key is what decides which rule a packet hits first. + rule_evaluation_order = [ + for sort_key in sort([ + for k, r in var.rules : format( + "%d|%06d|%s", + length(r.interfaces) == 0 ? 2 : (length(r.interfaces) > 1 ? 3 : 4), + r.sequence, + k + ) + ]) : split("|", sort_key)[2] + ] +} + +output "category" { + description = "Name and UUID of the category every object created by this module is tagged with. Filtering on the name in the OPNsense GUI shows exactly what OpenTofu manages and nothing else." + value = { + name = opnsense_firewall_category.this.name + id = opnsense_firewall_category.this.id + } +} + +output "managed_objects" { + description = "Names of the objects this module holds on the appliance, by kind. Read from the resources rather than the inputs, so it reflects what was actually pushed." + value = { + aliases = sort(keys(opnsense_firewall_alias.this)) + routes = sort(keys(opnsense_route.this)) + rules = sort(keys(opnsense_firewall_filter.this)) + outbound_nat = sort(keys(opnsense_firewall_nat.this)) + port_forwards = sort(keys(opnsense_firewall_nat_port_forward.this)) + } +} + +output "rule_evaluation_order" { + description = "Filter rules in the order the appliance evaluates them. Every rule defaults to quick, so the first match decides and this is the list to read when a rule does not behave as expected. Derived from the inputs, so it is reviewable in a plan before anything is pushed." + value = [ + for k in local.rule_evaluation_order : { + rule = k + scope = length(var.rules[k].interfaces) == 0 ? "floating" : join(",", var.rules[k].interfaces) + sequence = var.rules[k].sequence + enabled = var.rules[k].enabled + action = var.rules[k].action + protocol = var.rules[k].protocol + source = var.rules[k].source_net + target = var.rules[k].destination_net + dest_port = var.rules[k].destination_port != null ? var.rules[k].destination_port : "any" + } + ] +} diff --git a/src/modules/firewall-config/scripts/bootstrap-api-key.sh b/src/modules/firewall-config/scripts/bootstrap-api-key.sh new file mode 100644 index 0000000..3146016 --- /dev/null +++ b/src/modules/firewall-config/scripts/bootstrap-api-key.sh @@ -0,0 +1,107 @@ +#!/usr/bin/env bash +# +# Derives an OPNsense API key/secret from the appliance login. +# +# The REST API rejects the login password for basic auth (401), but it does accept a +# web GUI session cookie. With that session plus the rotating X-CSRFToken header, the +# key/secret pair can be created through /api/auth/user/addApiKey — no manual GUI step. +# +# Any previously issued API keys of the user are removed first, so a re-run is a real +# rotation: the old key stops working the moment the new one exists. Without that, a +# stale key kept in a secret store would silently stay valid forever. +# +# Usage: +# OPNSENSE_PASSWORD='...' ./bootstrap-api-key.sh https://10.0.2.4 [username] +# +# Output: {"api_key":"...","api_secret":"..."} — written atomically to OUTPUT_FILE if +# that variable is set, to stdout otherwise. The atomic write matters when Terraform +# invokes this: a shell redirect would leave an empty file behind on failure, which the +# root module would then read as "no credentials" while half a bootstrap has happened. +# +# The root module invokes this automatically, so there is normally nothing to run by +# hand. It stays callable on its own for recovery or for seeding a secret store. + +set -euo pipefail + +ENDPOINT="${1:?usage: bootstrap-api-key.sh [username]}" +USERNAME="${2:-root}" +PASSWORD="${OPNSENSE_PASSWORD:?set OPNSENSE_PASSWORD}" + +JAR="$(mktemp)" +trap 'rm -f "$JAR"' EXIT + +curl_opts=(--silent --show-error --max-time 30 --insecure) + +# 0) Wait for the appliance. Terraform starts this the moment the VM resource exists, +# but OPNsense needs a few minutes to boot before the web server answers. +deadline=$((SECONDS + 600)) +until login_page="$(curl --silent --max-time 10 --insecure -c "$JAR" "$ENDPOINT/")" \ + && grep -q '= deadline)); then + echo "appliance at $ENDPOINT did not serve a login page within 10 minutes" >&2 + exit 1 + fi + sleep 15 +done + +# 1) The login form's CSRF field has a randomly generated name and value. +hidden="$(grep -oE '&2 + exit 1 +fi + +api() { # api [curl args...] + local path="$1"; shift + curl "${curl_opts[@]}" -b "$JAR" -c "$JAR" \ + -X POST -H 'Content-Type: application/json' -H "X-CSRFToken: $token" "$@" \ + "$ENDPOINT$path" +} + +# 4) Revoke every existing key of the user. The id is the key base64-encoded, and the +# endpoint only matches it with the padding stripped — verified against a live +# appliance, where the padded form answers "not found". +api /api/auth/user/search_api_key -d '{}' \ + | python3 -c 'import json,sys; [print(r["id"].rstrip("=")) for r in json.load(sys.stdin).get("rows",[])]' \ + | while read -r id; do + api "/api/auth/user/del_api_key/$id" -d '{}' >/dev/null \ + || echo "warning: could not revoke old API key $id" >&2 + done + +# 5) Create the key pair. +response="$(api "/api/auth/user/addApiKey/$USERNAME")" + +key="$(python3 -c 'import json,sys; print(json.load(sys.stdin).get("key",""))' <<<"$response")" +secret="$(python3 -c 'import json,sys; print(json.load(sys.stdin).get("secret",""))' <<<"$response")" + +if [[ -z "$key" || -z "$secret" ]]; then + echo "API key creation failed: $response" >&2 + exit 1 +fi + +# JSON so Terraform can consume the file directly with jsondecode(). +result="$(printf '{"api_key":"%s","api_secret":"%s"}\n' "$key" "$secret")" + +if [[ -n "${OUTPUT_FILE:-}" ]]; then + tmp="$(mktemp "${OUTPUT_FILE}.XXXX")" + printf '%s\n' "$result" > "$tmp" + mv "$tmp" "$OUTPUT_FILE" +else + printf '%s\n' "$result" +fi diff --git a/src/modules/firewall-config/terraform.tf b/src/modules/firewall-config/terraform.tf new file mode 100644 index 0000000..5dc64d1 --- /dev/null +++ b/src/modules/firewall-config/terraform.tf @@ -0,0 +1,10 @@ +terraform { + required_version = ">= 1.10" + + required_providers { + opnsense = { + source = "browningluke/opnsense" + version = "0.24.0" + } + } +} diff --git a/src/modules/firewall-config/variables.tf b/src/modules/firewall-config/variables.tf new file mode 100644 index 0000000..848777b --- /dev/null +++ b/src/modules/firewall-config/variables.tf @@ -0,0 +1,122 @@ +variable "category_name" { + type = string + description = "OPNsense category every object created by this module is tagged with, so managed objects are recognisable in the GUI." + default = "landing-zone" +} + +variable "aliases" { + type = map(object({ + type = optional(string, "network") + enabled = optional(bool, true) + description = optional(string, null) + content = optional(list(string), []) + update_freq = optional(number, null) + stats = optional(bool, false) + })) + description = "Firewall aliases keyed by alias name. Reference an alias from a rule, route or NAT entry by using its key wherever a network is expected. type = \"host\" also accepts FQDNs, which the appliance re-resolves periodically." + default = {} + + validation { + condition = alltrue([ + for a in values(var.aliases) : + contains(["host", "network", "port", "url", "urltable", "urljson", "geoip", "asn", "networkgroup", "mac", "external"], a.type) + ]) + error_message = "aliases[*].type must be one of host, network, port, url, urltable, urljson, geoip, asn, networkgroup, mac, external." + } + + # OPNsense refuses a urltable without a refresh interval, and the API error it returns + # for it does not name the field. + validation { + condition = alltrue([ + for a in values(var.aliases) : a.type != "urltable" || a.update_freq != null + ]) + error_message = "aliases[*].update_freq is required when type is urltable. It is expressed in days, so 1.25 means every 30 hours." + } +} + +variable "routes" { + type = map(object({ + enabled = optional(bool, true) + network = string + gateway = string + description = optional(string, null) + })) + description = "Static routes keyed by name. gateway must name a gateway that exists on the appliance; the STACKIT image ships LAN_DHCP and WAN_DHCP." + default = {} +} + +variable "outbound_nat" { + type = map(object({ + sequence = optional(number, 200) + enabled = optional(bool, true) + interface = optional(string, "wan") + protocol = optional(string, "any") + ip_protocol = optional(string, "inet") + source_net = optional(string, "any") + destination_net = optional(string, "any") + target_ip = optional(string, "wanip") + disable_nat = optional(bool, false) + log = optional(bool, false) + description = optional(string, null) + })) + description = "Additional outbound NAT rules keyed by name. target_ip accepts an address, an alias, or ip such as wanip." + default = {} +} + +variable "port_forwards" { + type = map(object({ + sequence = optional(number, 100) + enabled = optional(bool, true) + interfaces = optional(list(string), ["wan"]) + protocol = optional(string, "TCP") + ip_protocol = optional(string, "inet") + source_net = optional(string, "any") + destination_net = optional(string, "wanip") + destination_port = string + target_ip = string + target_port = optional(string, null) + nat_reflection = optional(string, "default") + log = optional(bool, true) + description = optional(string, null) + })) + description = "Inbound port forwards from the internet keyed by name. Every entry punches a hole through the WAN, so keep the list short and set source_net where possible." + default = {} +} + +variable "rules" { + type = map(object({ + sequence = optional(number, 100) + enabled = optional(bool, true) + action = optional(string, "pass") + direction = optional(string, "in") + interfaces = optional(list(string), ["lan"]) + protocol = optional(string, "any") + ip_protocol = optional(string, "inet") + quick = optional(bool, true) + source_net = optional(string, "any") + source_port = optional(string, null) + source_invert = optional(bool, false) + destination_net = optional(string, "any") + destination_port = optional(string, null) + destination_invert = optional(bool, false) + log = optional(bool, false) + description = optional(string, null) + })) + description = "Firewall filter rules keyed by name. An empty interfaces list creates a floating rule, which OPNsense evaluates before interface-bound rules." + default = {} + + validation { + condition = alltrue([ + for r in values(var.rules) : contains(["pass", "block", "reject"], r.action) + ]) + error_message = "rules[*].action must be one of pass, block, reject." + } + + validation { + condition = alltrue([ + for r in values(var.rules) : contains(["in", "out"], r.direction) + ]) + error_message = "rules[*].direction must be in or out." + } +} + diff --git a/src/modules/governance/terraform.tf b/src/modules/governance/terraform.tf index f79ffe1..5f5a4ee 100644 --- a/src/modules/governance/terraform.tf +++ b/src/modules/governance/terraform.tf @@ -4,7 +4,7 @@ terraform { required_providers { stackit = { source = "stackitcloud/stackit" - version = "0.101.0" + version = "0.104.0" } time = { source = "hashicorp/time" diff --git a/src/modules/landing-zone/outputs.tf b/src/modules/landing-zone/outputs.tf index 721a597..ce79f72 100644 --- a/src/modules/landing-zone/outputs.tf +++ b/src/modules/landing-zone/outputs.tf @@ -15,12 +15,12 @@ output "project_name" { output "dns_zone_dns_name" { description = "The DNS name of the landing zone's child DNS zone." - value = var.dns_zone_name != null ? stackit_dns_zone.this[0].dns_name : null + value = try(stackit_dns_zone.this[0].dns_name, null) } output "dns_zone_id" { description = "The ID of the landing zone's child DNS zone." - value = var.dns_zone_name != null ? stackit_dns_zone.this[0].zone_id : null + value = try(stackit_dns_zone.this[0].zone_id, null) } output "connected_network_area_id" { diff --git a/src/modules/landing-zone/terraform.tf b/src/modules/landing-zone/terraform.tf index 8f082ce..dbad570 100644 --- a/src/modules/landing-zone/terraform.tf +++ b/src/modules/landing-zone/terraform.tf @@ -4,7 +4,7 @@ terraform { required_providers { stackit = { source = "stackitcloud/stackit" - version = "0.101.0" + version = "0.104.0" } time = { source = "hashicorp/time" diff --git a/src/modules/management/outputs.tf b/src/modules/management/outputs.tf index 345a4c1..4675743 100644 --- a/src/modules/management/outputs.tf +++ b/src/modules/management/outputs.tf @@ -32,4 +32,8 @@ output "secretsmanager_password" { output "bucket_name_tfstate" { description = "The name of the tfstate object storage bucket." value = stackit_objectstorage_bucket.tfstate.name -} \ No newline at end of file +} +output "secretsmanager_instance_id" { + description = "The ID of the Secrets Manager instance, usable as the vault mount." + value = stackit_secretsmanager_instance.this.instance_id +} diff --git a/src/modules/management/terraform.tf b/src/modules/management/terraform.tf index f87acca..4512af9 100644 --- a/src/modules/management/terraform.tf +++ b/src/modules/management/terraform.tf @@ -4,7 +4,7 @@ terraform { required_providers { stackit = { source = "stackitcloud/stackit" - version = "0.101.0" + version = "0.104.0" } time = { source = "hashicorp/time" @@ -12,7 +12,7 @@ terraform { } vault = { source = "hashicorp/vault" - version = "~> 5.9.0" + version = "5.10.1" } } } \ No newline at end of file diff --git a/src/modules/namespace-service-demo/main.tf b/src/modules/namespace-service-demo/main.tf index 1011abe..65b82a2 100644 --- a/src/modules/namespace-service-demo/main.tf +++ b/src/modules/namespace-service-demo/main.tf @@ -2,11 +2,11 @@ terraform { required_providers { stackit = { source = "stackitcloud/stackit" - version = "~> 0.99.0" + version = "0.104.0" } grafana = { source = "grafana/grafana" - version = "~> 3.0" + version = "4.41.0" } } } @@ -19,7 +19,7 @@ locals { services_with_observability = { for key, value in var.services : key => value - if ( + if( try(value.observability_grafana_url, null) != null && try(value.observability_admin_user, null) != null && try(value.observability_instance_id, null) != null && @@ -137,8 +137,8 @@ resource "grafana_dashboard" "namespace_overview" { for_each = local.services_with_observability - folder = grafana_folder.stackit_managed[0].uid - overwrite = true + folder = grafana_folder.stackit_managed[0].uid + overwrite = true config_json = templatefile("${path.module}/dashboards/namespace-overview.json.tmpl", { dashboard_uid = local.dashboard_uid[each.key] dashboard_title = "${each.key} Namespace Overview" diff --git a/src/modules/namespace-service-demo/outputs.tf b/src/modules/namespace-service-demo/outputs.tf index 2310555..338e879 100644 --- a/src/modules/namespace-service-demo/outputs.tf +++ b/src/modules/namespace-service-demo/outputs.tf @@ -4,11 +4,11 @@ output "samples" { for key, value in var.services : key => { dashboard_folder_uid = try(grafana_folder.stackit_managed[0].uid, null) dashboard_folder_name = try(grafana_folder.stackit_managed[0].title, null) - namespace = value.namespace - external_secret_user = try(stackit_secretsmanager_user.external_secret_demo[key].username, null) - dashboard_uid = try(local.dashboard_uid[key], null) - dashboard_url = try("${value.observability_grafana_url}/d/${local.dashboard_uid[key]}", null) - dashboard_api_url = try("${value.observability_grafana_url}/api/dashboards/uid/${local.dashboard_uid[key]}", null) + namespace = value.namespace + external_secret_user = try(stackit_secretsmanager_user.external_secret_demo[key].username, null) + dashboard_uid = try(local.dashboard_uid[key], null) + dashboard_url = try("${value.observability_grafana_url}/d/${local.dashboard_uid[key]}", null) + dashboard_api_url = try("${value.observability_grafana_url}/api/dashboards/uid/${local.dashboard_uid[key]}", null) } } } diff --git a/src/modules/namespace-service-demo/variables.tf b/src/modules/namespace-service-demo/variables.tf index 5cb54b6..a272656 100644 --- a/src/modules/namespace-service-demo/variables.tf +++ b/src/modules/namespace-service-demo/variables.tf @@ -1,22 +1,22 @@ variable "services" { type = map(object({ - namespace = string - use_secretsmanager = bool - landing_zone_project_id = string - secretsmanager_instance_id = string - observability_instance_id = optional(string) - observability_grafana_url = optional(string) - observability_admin_user = optional(string) + namespace = string + use_secretsmanager = bool + landing_zone_project_id = string + secretsmanager_instance_id = string + observability_instance_id = optional(string) + observability_grafana_url = optional(string) + observability_admin_user = optional(string) demo_metrics_ingestion_enabled = optional(bool, false) demo_metrics_ingestion_target_urls = optional(list(string), []) demo_metrics_ingestion_scheme = optional(string, "https") demo_metrics_ingestion_metrics_path = optional(string, "/") demo_metrics_ingestion_scrape_interval = optional(string, "60s") demo_metrics_ingestion_scrape_timeout = optional(string, "30s") - platform_project_id = optional(string) - platform_observability_instance_id = optional(string) - platform_observability_targets_url = optional(string) - dns_zone_name = optional(string) + platform_project_id = optional(string) + platform_observability_instance_id = optional(string) + platform_observability_targets_url = optional(string) + dns_zone_name = optional(string) })) description = "Enabled namespace-service demo configurations keyed by landing-zone key." default = {} diff --git a/src/modules/platform-kubernetes/terraform.tf b/src/modules/platform-kubernetes/terraform.tf index df4a55d..79dd445 100644 --- a/src/modules/platform-kubernetes/terraform.tf +++ b/src/modules/platform-kubernetes/terraform.tf @@ -4,7 +4,7 @@ terraform { required_providers { stackit = { source = "stackitcloud/stackit" - version = "0.101.0" + version = "0.104.0" } time = { source = "hashicorp/time" diff --git a/src/modules/sandboxes/terraform.tf b/src/modules/sandboxes/terraform.tf index 0ea12d2..0626f40 100644 --- a/src/modules/sandboxes/terraform.tf +++ b/src/modules/sandboxes/terraform.tf @@ -4,7 +4,7 @@ terraform { required_providers { stackit = { source = "stackitcloud/stackit" - version = "0.101.0" + version = "0.104.0" } } } \ No newline at end of file diff --git a/src/outputs.tf b/src/outputs.tf index 92704d9..0355026 100644 --- a/src/outputs.tf +++ b/src/outputs.tf @@ -37,6 +37,31 @@ output "connectivity_firewall_public_ip" { value = try(module.connectivity[0].firewall_public_ip, null) } +output "connectivity_vpn_gateway_id" { + description = "The ID of the hub VPN gateway." + value = try(module.connectivity[0].vpn_gateway_id, null) +} + +output "connectivity_vpn_public_ips" { + description = "Public IPs of the hub VPN gateway tunnels. Configure the remote peer against these." + value = try(module.connectivity[0].vpn_public_ips, {}) +} + +output "connectivity_vpn_internal_next_hop_ips" { + description = "Network area side IPs of the hub VPN gateway tunnels." + value = try(module.connectivity[0].vpn_internal_next_hop_ips, {}) +} + +output "firewall_admin_url" { + description = "Where to reach the OPNsense web GUI. Only reachable from inside the network area once the policy blocks the WAN." + value = try(var.connectivity.firewall, null) != null ? local.firewall_endpoint : null +} + +output "connectivity_vpn_connection_ids" { + description = "Map of hub VPN connection keys to their connection IDs." + value = try(module.connectivity[0].vpn_connection_ids, {}) +} + output "platform_kubernetes_projects" { description = "Map of platform Kubernetes projects and cluster metadata per key." value = { diff --git a/src/providers.tf b/src/providers.tf index ff591f2..acb35a1 100644 --- a/src/providers.tf +++ b/src/providers.tf @@ -66,4 +66,11 @@ provider "vault" { } } -provider "time" {} \ No newline at end of file +provider "time" {} + +provider "opnsense" { + uri = local.firewall_endpoint + api_key = local.firewall_api_credentials.api_key + api_secret = local.firewall_api_credentials.api_secret + allow_insecure = try(var.firewall_config.insecure, true) +} \ No newline at end of file diff --git a/src/terraform.tf b/src/terraform.tf index 6125857..dc2915a 100644 --- a/src/terraform.tf +++ b/src/terraform.tf @@ -1,5 +1,5 @@ terraform { - required_version = ">= 1.10, < 2.0" + required_version = ">= 1.11" required_providers { stackit = { @@ -8,15 +8,15 @@ terraform { } kubernetes = { source = "hashicorp/kubernetes" - version = "~> 3.0" + version = "3.2.1" } helm = { source = "hashicorp/helm" - version = "~> 3.0" + version = "3.2.0" } time = { source = "hashicorp/time" - version = "~> 0.14.0" + version = "0.14.0" } vault = { source = "hashicorp/vault" @@ -24,7 +24,11 @@ terraform { } grafana = { source = "grafana/grafana" - version = "~> 3.0" + version = "4.41.0" + } + opnsense = { + source = "browningluke/opnsense" + version = "0.24.0" } } } \ No newline at end of file diff --git a/src/variables.tf b/src/variables.tf index e1633e0..98f5484 100644 --- a/src/variables.tf +++ b/src/variables.tf @@ -261,9 +261,204 @@ variable "connectivity" { lan_ip = optional(string, null) wan_ip = optional(string, null) }), null) + vpn = optional(object({ + display_name = optional(string, null) + plan_id = optional(string, "p100") + routing_type = optional(string, "ROUTE_BASED") + availability_zones = object({ + tunnel1 = string + tunnel2 = string + }) + connections = optional(map(object({ + display_name = optional(string, null) + enabled = optional(bool, true) + local_subnets = optional(list(string), null) + remote_subnets = optional(list(string), null) + static_routes = optional(list(string), null) + tunnel1 = object({ + remote_address = string + peering = optional(object({ + local_address = string + remote_address = string + }), null) + phase1 = optional(object({ + encryption_algorithms = optional(list(string), ["aes256"]) + integrity_algorithms = optional(list(string), ["sha2_384"]) + dh_groups = optional(list(string), ["ecp384"]) + rekey_time = optional(number, null) + }), {}) + phase2 = optional(object({ + encryption_algorithms = optional(list(string), ["aes256"]) + integrity_algorithms = optional(list(string), ["sha2_384"]) + dh_groups = optional(list(string), ["ecp384"]) + rekey_time = optional(number, null) + dpd_action = optional(string, null) + start_action = optional(string, null) + }), {}) + }) + tunnel2 = object({ + remote_address = string + peering = optional(object({ + local_address = string + remote_address = string + }), null) + phase1 = optional(object({ + encryption_algorithms = optional(list(string), ["aes256"]) + integrity_algorithms = optional(list(string), ["sha2_384"]) + dh_groups = optional(list(string), ["ecp384"]) + rekey_time = optional(number, null) + }), {}) + phase2 = optional(object({ + encryption_algorithms = optional(list(string), ["aes256"]) + integrity_algorithms = optional(list(string), ["sha2_384"]) + dh_groups = optional(list(string), ["ecp384"]) + rekey_time = optional(number, null) + dpd_action = optional(string, null) + start_action = optional(string, null) + }), {}) + }) + })), {}) + }), null) + }) + description = "Connectivity configuration including DNS zones, network area, firewall, and VPN. Set firewall/network_area/vpn to null to skip deployment." + default = null +} + +variable "vpn_pre_shared_keys" { + type = map(object({ + tunnel1 = string + tunnel2 = string + })) + description = "Pre-shared keys per connectivity.vpn.connections key, one per tunnel. Kept out of the connectivity object so the topology stays committable; supply through TF_VAR_vpn_pre_shared_keys or a gitignored tfvars file. Minimum 20 characters." + default = {} + sensitive = true +} + +###################### +## FIREWALL CONFIG ## +###################### + +variable "firewall_config" { + type = object({ + # Defaults to https://, derived from connectivity.firewall. + endpoint = optional(string, null) + insecure = optional(bool, true) + + + aliases = optional(map(object({ + type = optional(string, "network") + enabled = optional(bool, true) + description = optional(string, null) + content = optional(list(string), []) + update_freq = optional(number, null) + stats = optional(bool, false) + })), {}) + + routes = optional(map(object({ + enabled = optional(bool, true) + network = string + gateway = string + description = optional(string, null) + })), {}) + + rules = optional(map(object({ + sequence = optional(number, 100) + enabled = optional(bool, true) + action = optional(string, "pass") + direction = optional(string, "in") + interfaces = optional(list(string), ["lan"]) + protocol = optional(string, "any") + ip_protocol = optional(string, "inet") + quick = optional(bool, true) + source_net = optional(string, "any") + source_port = optional(string, null) + source_invert = optional(bool, false) + destination_net = optional(string, "any") + destination_port = optional(string, null) + destination_invert = optional(bool, false) + log = optional(bool, false) + description = optional(string, null) + })), {}) + + outbound_nat = optional(map(object({ + sequence = optional(number, 200) + enabled = optional(bool, true) + interface = optional(string, "wan") + protocol = optional(string, "any") + ip_protocol = optional(string, "inet") + source_net = optional(string, "any") + destination_net = optional(string, "any") + target_ip = optional(string, "wanip") + disable_nat = optional(bool, false) + log = optional(bool, false) + description = optional(string, null) + })), {}) + + port_forwards = optional(map(object({ + sequence = optional(number, 100) + enabled = optional(bool, true) + interfaces = optional(list(string), ["wan"]) + protocol = optional(string, "TCP") + ip_protocol = optional(string, "inet") + source_net = optional(string, "any") + destination_net = optional(string, "wanip") + destination_port = string + target_ip = string + target_port = optional(string, null) + nat_reflection = optional(string, "default") + log = optional(bool, true) + description = optional(string, null) + })), {}) }) - description = "Connectivity configuration including DNS zones, network area, and firewall. Set firewall/network_area to null to skip deployment." + description = "Policy pushed to the OPNsense appliance through its API. Requires connectivity.firewall to be deployed and firewall_api_credentials to be set. Set to null to leave the appliance untouched." default = null + + validation { + condition = var.firewall_config == null || try(var.connectivity.firewall, null) != null + error_message = "firewall_config requires connectivity.firewall to be set, there is no appliance to push a policy to otherwise." + } + + validation { + condition = var.firewall_config == null || alltrue([ + for p in values(var.firewall_config.port_forwards) : p.source_net != "any" || !p.enabled + ]) + error_message = "port_forwards[*].source_net must be narrowed from \"any\", or the entry disabled. An unrestricted forward exposes the target to the whole internet." + } +} + +variable "firewall_admin_username" { + type = string + description = "Login used to derive the API key from the appliance. The STACKIT OPNsense image ships with root." + default = "root" +} + +variable "firewall_admin_password" { + type = string + description = "Password of firewall_admin_username, used once to derive the API key. Defaults to the password baked into the STACKIT OPNsense image. Override through TF_VAR_firewall_admin_password after rotating it, or set to null and supply firewall_api_credentials instead." + default = "STACKIT123!" + sensitive = true +} + +variable "firewall_bootstrap" { + type = bool + description = "Pass -var firewall_bootstrap=true on the two greenfield applies that create the OPNsense API key. It suppresses the Secrets Manager read, which would otherwise fail at plan time because the secret does not exist yet. Never set it on an established deployment: without the bootstrap cache file it plans the destruction of the policy and the stored key. The steady state needs no variable at all." + default = false +} + +variable "firewall_api_secret_version" { + type = number + description = "Write version of the OPNsense API key in the Secrets Manager. The key is only transmitted when this number changes, so bump it by one after a re-bootstrap has produced a fresh key, otherwise the Secrets Manager keeps serving the old one." + default = 1 +} + +variable "firewall_api_credentials" { + type = object({ + api_key = string + api_secret = string + }) + description = "OPNsense API key and secret. Bootstrap them once from the appliance login with src/modules/firewall-config/scripts/bootstrap-api-key.sh, then supply through TF_VAR_firewall_api_credentials or a gitignored tfvars file. Kept out of firewall_config so the policy stays committable." + default = null + sensitive = true } ############### @@ -315,7 +510,7 @@ variable "landing_zones" { variable "landing_zone_namespace_services" { type = map(object({ - demo_enabled = optional(bool, false) + demo_enabled = optional(bool, false) demo_metrics_ingestion = optional(object({ enabled = optional(bool, false) target_urls = optional(list(string), [])