Automatically determine values for Terraform Azure Private endpoint module - module

I need a help with a terraform module that I've created. It works perfectly, but I need to add some automation.
I created a module which creates multiple private endpoints, but I always need to put the variable values in manually.
This is the module:
resource "azurerm_private_endpoint" "endpoint" {
for_each = try({ for endpoint in var.endpoints : endpoint.name => endpoint }, toset([]))
name = each.key
location = var.location
resource_group_name = var.resource_group_name
subnet_id = each.value.subnet_id
dynamic "private_service_connection" {
for_each = each.value.private_service_connection
content {
name = each.key
private_connection_resource_id = private_service_connection.value.private_connection_resource_id
is_manual_connection = false
subresource_names = var.subresource_name ### see values on : https://learn.microsoft.com/fr-fr/azure/private-link/private-endpoint-overview#private-link-resource
}
}
lifecycle {
ignore_changes = [
private_dns_zone_group
]
}
tags = var.tags
}
I need to have:
1 - for the private endpoint name : I need it to be automatically provided: "pendp-(the subresource_name value in lower cases- my resource_name =>(mysql server for example))"
2 - for the private connection name: I need the values to be automatically: "connection-(the subresource_name value in lower cases- my ressource_name =>(mysql server for exemple))"
3 - some automation to detect automatically the subresource_name ( if I create a private endpoint for a blob or for a mariadb or for a mysqlserver, the module should detected it.
terraform version:
terraform {
required_version = "~> 1"
required_providers {
azurerm = "~> 3.0"
}
}

The easiest way to combine values automatically would be to use the Terraform string join() function to join multiple strings together. For lower case strings, you can use the lower() function.
Some examples:
name = join("-", ["pandp", lower(var.subresource_name)])
...
name = join("-", ["connection", lower(var.subresource_name), lower(each.key)])
For your third rule, you want to use a conditional expression to determine if it's a blob, or mariadb, or mysqlserver.
In this example, we set an example_name local with a value some-blob-value if var.subresource_name contains a string that starts with "blob", and set it to something-else if the condition is false:
locals {
example_name = startswith(lower(var.subresource_name), "blob") ? "some-blob-value" : "something-else"
}
There are many options available for doing a conditional on if a value is passed to what you expect and then determine a result based on that value. What exactly you want isn't clear in the question, but hopefully this will get you pointed in the right direction.
Terraform even has several helper functions that might help you if you only need part of a string, such as startswith(), endswith(), or contains() depending on your needs.

Related

Terraform for_each on custom registry

We have a custom VPC registry built within our organization. The objective is to enable user of registry to be able to create VPC, create multiple public and private subnets for multiple availability zones within VPC.
I have a for_each variable set at
variable az_sub {list(object(
az = string,
public_cidr_block = list,
private_cidr_block =list
))}
Variable has a value of
[{az='us-east-1a",
public_cidr_block =[list of cidr],
private_cidr_block= [list of cidr]
},
az='us-east-1b",
public_cidr_block =[list of cidr],
private_cidr_block= [list of cidr} ]
When I set for_each on this within registry,
module "az"{
source="./modules/az"
vpc_id = module.vpc.vpc_id
for_each = toset(keys({for i,v in var.az_sub: i => v}))
availability_zone = var.az_sub[each.value]["az"]
public_cidr_block = var.az_sub[each.value]["public_cidr_block"]
private_cidr_block =var.az_sub[each.value]["private_cidr_block"]
}
I get unsupported attribute error on terraform/modules/modReg/output.tf in output public_subnet_ids
module.az is object with 2 attributes
output.tf has the outputs defined.
The az variables.tf has all three variables defined.
Also Note, if I replace for_each with below, do not get errors, but objects are successfully created on aws
availability_zone="us-east-1a"
public_cidr_block = ["10.97.224.0/22"]
private_cidr_block =["10.97.228.0/22"]
So doubt its issue with az module
Note, if I use only 1 az zone on the variable value for list(object) I the error changes to
module.az is object with 1 attribute "us-east-1"
output "private_subnet_ids" {
description = "List of private subnet IDs"
value = values(module.az).*.private_subnet_ids
}
For anyone having same issue, this resolved the problem. I had used list(object and not map(object. If map(object was used, then output.tf would need to have map values.
If you use for_each to send values to the module, the outputs would need to be defined as shown above.

Terraform - Conditional expression for a AWS role

I have the following in my module to create a role:
resource "aws_iam_role" "default" {
name = var.name
assume_role_policy = var.assume_role_policy
permissions_boundary = var.account_id != "" ? var.permissions_boundary : "arn:aws:iam::${data.aws_caller_identity.default.account_id}:policy/BoundedPermissionsPolicy"
}
Problem - I want to be able to set the permissions boundary argument to use the account ID if present and if not specified then make use of var.permissions_boundary (arn:aws:iam::${data.aws_caller_identity.default.account_id}:policy/BoundedPermissionsPolicy).
The code above does not work when I try to use the account ID.

Invalid character error while running terraform init, terraform plan or apply

I'm running Terraform using VScode editor which uses PowerShell as the default shell and getting the same error when I try to validate it or to run terraform init/plan/apply through VScode, external PowerShell or CMD.
The code was running without any issues until I added Virtual Machine creation code. I have clubbed the variables.tf, terraform.tfvars and the main Terraform code below.
terraform.tfvars
web_server_location = "West US 2"
resource_prefix = "web-server"
web_server_address_space = "1.0.0.0/22"
web_server_address_prefix = "1.0.1.0/24"
Environment = "Test"
variables.tf
variable "web_server_location" {
type = string
}
variable "resource_prefix" {
type = string
}
variable "web_server_address_space" {
type = string
}
#variable for network range
variable "web_server_address_prefix" {
type = string
}
#variable for Environment
variable "Environment" {
type = string
}
terraform_example.tf
# Configure the Azure Provider
provider "azurerm" {
# whilst the `version` attribute is optional, we recommend pinning to a given version of the Provider
version = "=2.0.0"
features {}
}
# Create a resource group
resource "azurerm_resource_group" "example_rg" {
name = "${var.resource_prefix}-RG"
location = var.web_server_location
}
# Create a virtual network within the resource group
resource "azurerm_virtual_network" "example_vnet" {
name = "${var.resource_prefix}-vnet"
resource_group_name = azurerm_resource_group.example_rg.name
location = var.web_server_location
address_space = [var.web_server_address_space]
}
# Create a subnet within the virtual network
resource "azurerm_subnet" "example_subnet" {
name = "${var.resource_prefix}-subnet"
resource_group_name = azurerm_resource_group.example_rg.name
virtual_network_name = azurerm_virtual_network.example_vnet.name
address_prefix = var.web_server_address_prefix
}
# Create a Network Interface
resource "azurerm_network_interface" "example_nic" {
name = "${var.resource_prefix}-NIC"
location = azurerm_resource_group.example_rg.location
resource_group_name = azurerm_resource_group.example_rg.name
ip_configuration {
name = "internal"
subnet_id = azurerm_subnet.example_subnet.id
private_ip_address_allocation = "Dynamic"
public_ip_address_id = azurerm_public_ip.example_public_ip.id
}
}
# Create a Public IP
resource "azurerm_public_ip" "example_public_ip" {
name = "${var.resource_prefix}-PublicIP"
location = azurerm_resource_group.example_rg.location
resource_group_name = azurerm_resource_group.example_rg.name
allocation_method = var.Environment == "Test" ? "Static" : "Dynamic"
tags = {
environment = "Test"
}
}
# Creating resource NSG
resource "azurerm_network_security_group" "example_nsg" {
name = "${var.resource_prefix}-NSG"
location = azurerm_resource_group.example_rg.location
resource_group_name = azurerm_resource_group.example_rg.name
# Security rule can also be defined with resource azurerm_network_security_rule, here just defining it inline.
security_rule {
name = "RDPInbound"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_range = "3389"
source_address_prefix = "*"
destination_address_prefix = "*"
}
tags = {
environment = "Test"
}
}
# NIC and NSG association
resource "azurerm_network_interface_security_group_association" "example_nsg_association" {
network_interface_id = azurerm_network_interface.example_nic.id
network_security_group_id = azurerm_network_security_group.example_nsg.id
}
# Creating Windows Virtual Machine
resource "azurerm_virtual_machine" "example_windows_vm" {
name = "${var.resource_prefix}-VM"
location = azurerm_resource_group.example_rg.location
resource_group_name = azurerm_resource_group.example_rg.name
network_interface_ids = [azurerm_network_interface.example_nic.id]
vm_size = "Standard_B1s"
delete_os_disk_on_termination = true
storage_image_reference {
publisher = "MicrosoftWindowsServer"
offer = "WindowsServerSemiAnnual"
sku  = "Datacenter-Core-1709-smalldisk"
version = "latest"
}
storage_os_disk  {
name = "myosdisk1"
caching  = "ReadWrite"
create_option = "FromImage"
storage_account_type = "Standard_LRS"
}
os_profile {
computer_name = "hostname"
admin_username = "adminuser"
admin_password = "Password1234!"
}
os_profile_windows_config {
disable_password_authentication = false
}
tags = {
environment = "Test"
}
}
Error:
PS C:\Users\e5605266\Documents\MyFiles\Devops\Terraform> terraform init
There are some problems with the configuration, described below.
The Terraform configuration must be valid before initialization so that
Terraform can determine which modules and providers need to be installed.
Error: Invalid character
on terraform_example.tf line 89, in resource "azurerm_virtual_machine" "example_windows_vm":
89: location = azurerm_resource_group.example_rg.location
This character is not used within the language.
Error: Invalid expression
on terraform_example.tf line 89, in resource "azurerm_virtual_machine" "example_windows_vm":
89: location = azurerm_resource_group.example_rg.location
Expected the start of an expression, but found an invalid expression token.
Error: Argument or block definition required
on terraform_example.tf line 90, in resource "azurerm_virtual_machine" "example_windows_vm":
90: resource_group_name = azurerm_resource_group.example_rg.name
An argument or block definition is required here. To set an argument, use the
equals sign "=" to introduce the argument value.
Error: Invalid character
on terraform_example.tf line 90, in resource "azurerm_virtual_machine" "example_windows_vm":
90: resource_group_name = azurerm_resource_group.example_rg.name
This character is not used within the language.
*
I've encountered this problem myself in several different contexts, and it does have a common solution which is no fun at all: manually typing the code back in...
This resource block seems to be where it runs into problems:
resource "azurerm_virtual_machine" "example_windows_vm" {
name = "${var.resource_prefix}-VM"
location = azurerm_resource_group.example_rg.location
resource_group_name = azurerm_resource_group.example_rg.name
network_interface_ids = [azurerm_network_interface.example_nic.id]
vm_size = "Standard_B1s"
delete_os_disk_on_termination = true
storage_image_reference {
publisher = "MicrosoftWindowsServer"
offer = "WindowsServerSemiAnnual"
sku  = "Datacenter-Core-1709-smalldisk"
version = "latest"
}
storage_os_disk  {
name = "myosdisk1"
caching  = "ReadWrite"
create_option = "FromImage"
storage_account_type = "Standard_LRS"
}
os_profile {
computer_name = "hostname"
admin_username = "adminuser"
admin_password = "Password1234!"
}
os_profile_windows_config {
disable_password_authentication = false
}
tags = {
environment = "Test"
}
}
Try copying that back into your editor as is. I cannot see any problematic characters in it, and ironically StackOverflow may have done you a solid and filtered them out. Literally copy/pasting it over the existing block may remedy the situation.
I have seen Terraform examples online with stylish double quotes (which aren't ASCII double quotes and won't work) many times. That may be what you are seeing.
Beyond that, you'd need to push your code to GitHub or similar so I can see the raw bytes for myself.
In the off-chance this helps someone who runs into this error and comes across it on Google, I just thought I would post my situation and how I fixed it.
I have an old demo Terraform infrastructure that I revisited after months and, long story short, I issued this command two days ago and forgot about it:
terraform plan -out=plan.tf
This creates a zip archive of the plan. Upon coming back two days later and running a terraform init, my terminal scrolled garbage and "This character is not used within the language." for about 7 seconds. Due to the .tf extension, terraform was looking at the zip data and promptly pooping its pants.
Through moving individual tf files to a temp directory and checking their validity with terraform init, I found the culprit, deleted it, and functionality was restored.
Be careful when exporting your plan files, folks!
I ran into the same problem and found this page.
I solved the issue and decided to post here.
I opened my plan file in Notepad++ and selected View-Show all symbols.
I removed all the TAB characters and replaced them with spaces.
In my case, the problem was fully resolved by this.
In my case, when I ran into the same problem ("This character is not used within the language"), I found the encoding of the files was UTF-16 (it was a generated file from PS). Changing the file encoding to UTF-8 (as mentioned in this question) solved the issue.
I found I got this most often when I go from Windows to linux. The *.tf file does not like the windows TABs and Line Breaks.
I tried to some of the same tools I use when I have this problem with *.sh, but so far I've resorted to manually cleaning up the lines I've seen in there error.
In my case, the .tf file was generated by the following command terraform show -no-color > my_problematic.tf, and this file's encoding is in "UTF-16 LE BOM", converting it to UTF-8 fixed my issue.

Terraform 0.12: Output list of buckets, use as input for another module and iterate

I'm using Tf 0.12. I have an s3 module that outputs a list of buckets, that I would like to use as an input for a cloudfront module that I've got.
The problem I'm facing is that when I do terraform plan/apply I get the following error count.index is 0 |var.redirect-buckets is tuple with 1 element
I've tried all kinds of splats moving the count.index call around to no avail. My sample code is below.
module.s3
resource "aws_s3_bucket" "redirect" {
count = length(var.redirects)
bucket = element(var.redirects, count.index)
}
mdoule.s3.output
output "redirect-buckets" {
value = [aws_s3_bucket.redirect.*]
}
module.cdn.variables
...
variable "redirect-buckets" {
description = "Redirect buckets"
default = []
}
....
The error is thrown down here
module.cdn
resource "aws_cloudfront_distribution" "redirect" {
count = length(var.redirect-buckets)
default_cache_behavior {
// Line below throws the error, one amongst many
target_origin_id = "cloudfront-distribution-origin-${var.redirect-buckets[count.index]}.s3.amazonaws.com"
....
//Another error throwing line
target_origin_id = "cloudfront-distribution-origin-${var.redirect-buckets[count.index]}.s3.amazonaws.com"
Any help is greatly appreciated.
module.s3
resource "aws_s3_bucket" "redirects" {
for_each = var.redirects
bucket = each.value
}
Your variable definition for redirects needs to change to something like this:
variable "redirects" {
type = map(string)
}
module.s3.output:
output "redirect_buckets" {
value = aws_s3_bucket.redirects
}
module.cdn
resource "aws_cloudfront_distribution" "redirects" {
for_each = var.redirect_buckets
default_cache_behavior {
target_origin_id = "cloudfront-distribution-origin-${each.value.id}.s3.amazonaws.com"
}
Your variable definition for redirect-buckets needs to change to something like this (note underscores, using skewercase is going to behave strangely in some cases, not worth it):
variable "redirect_buckets" {
type = map(object(
{
id = string
}
))
}
root module
module "s3" {
source = "../s3" // or whatever the path is
redirects = {
site1 = "some-bucket-name"
site2 = "some-other-bucket"
}
}
module "cdn" {
source = "../cdn" // or whatever the path is
redirects_buckets = module.s3.redirect_buckets
}
From an example perspective, this is interesting, but you don't need to use outputs from S3 here since you could just hand the cdn module the same map of redirects and use for_each on those.
There is a tool called Terragrunt which wraps Terraform and supports dependencies.
https://terragrunt.gruntwork.io/docs/features/execute-terraform-commands-on-multiple-modules-at-once/#dependencies-between-modules

Declaring list and map variable values in terraform.tfvars

I recently moved over the values for my variables within my terraform code to terraform.tfvars. I am now getting an error that is due to how I am declaring my list and map variables. The code where I am getting the error is replicated below:
image_id = var.web_amis[var.region]
this is how I have these variables specified in terraform.tfvars:
web_amis = ["ami-0dacb0c129b49f529", "ami-00068cd7555f543d5", ]
this is the error code I am getting:
Error: Invalid index
on autoscaling.tf line 3, in resource "aws_launch_configuration" "web_lc":
3: image_id = var.web_amis[var.region]
|----------------
| var.region is "us-east-2"
| var.web_amis is tuple with 2 elements
The given key does not identify an element in this collection value: a number
is required.
You're trying to access a list element with a non index key instead of by position.
What you probably want instead is to have your web_amis variable be a map that is keyed by the region name:
main.tf
variable "region" {}
variable "web_amis" {}
resource "foo_bar" "baz" {
# ...
image_id = var.web_amis[var.region]
}
terraform.tfvars
web_amis = {
us-east-2 = "ami-0dacb0c129b49f529"
us-west-2 = "ami-00068cd7555f543d5"
}
But, this is a very old school and inelegant way of doing things with Terraform nowadays. Instead you could use the aws_ami data source to look up the AMI for the region based on filters such as tags or the name of the AMI.
A basic example is given in the aws_instance resource documentation:
provider "aws" {
region = "us-west-2"
}
data "aws_ami" "ubuntu" {
most_recent = true
filter {
name = "name"
values = ["ubuntu/images/hvm-ssd/ubuntu-trusty-14.04-amd64-server-*"]
}
filter {
name = "virtualization-type"
values = ["hvm"]
}
owners = ["099720109477"] # Canonical
}
resource "aws_instance" "web" {
ami = data.aws_ami.ubuntu.id
instance_type = "t2.micro"
tags = {
Name = "HelloWorld"
}
}