Using GitHub Actions with Terraform on GCP

6 min readApr 19, 2023

Automating infrastructure with Terraform helps a lot when you have multi environments and want to replicate the infrastructure fast in the cloud or on-prem. In this tutorial, I show how to integrate GitHub Actions with Terraform to manage Google Cloud Platform services.

Creating GitHub Workflow

To create a workflow, access the Actions tab on the GitHub repository and search for Terraform. The card Terraform By HashCorp will be shown.

Click the button Configure of Terraform By HashiCorp and a new window with a template for integration will appear.

The initial workflow file will look like this:

The step Terraform Format can be removed. Sometimes this step just fails and it’s painful making it work.

At the end of the file, in the if condition, the file checks github.ref == 'refs/heads/"main"' . Remove double quotes about the main branch like this github.ref == 'refs/heads/main' to enable the changes being applied by CI/CD.

Accept this initial template and click start commit.

Configuring the Google Cloud Platform

Create Service Account

For GitHub Terraform Workflow interact with Google Cloud, It’s necessary to have a Google service account. Let’s create one in the IAM panel.

Fill out the form with a service account name and click continue. In the role of the service account you should give only the access needed for services you will create, but if you are in a running give an Editor access.

Generate JSON Key for Service Account

In the list table of services accounts, access the option Manage Keys in the column Actions. Then create a JSON key for your Service Account.

The browser will ask to download a JSON file. Download It to use as a service account integration on GitHub.

Create GOOGLE_CREDENTIALS secret on GitHub

The next step is using the JSON credentials key from the service account and creating a secret called GOOGLE_CREDENTIALS in your GitHub project.

Create Google Cloud Storage Bucket

Terraform uses a file to control the state of changes applied. Create a bucket on Google Cloud Storage to store this file. It can be a standard single-region bucket.

Managing GitHub Workflow

Working directory

Now It’s time to edit your workflow.yml. By default, GitHub workflow looks for tf files in the root project directory. If you want to organize it in a separated folder, it’s needed to add a working-directory configuration.

jobs:
  terraform:
    name: 'Terraform'
    runs-on: ubuntu-latest
    environment: production

   defaults:
      run:
        shell: bash
        #Inform a working directory if .tf files are not in root folder
        working-directory: ./terraform

Using GOOGLE_CREDENTIALS in workflow

The GOOGLE_CREDENTIALS secret created will be used as env in the workflow to create resources on GCP. Put it on each step where have a Terraform action.

    - name: Terraform Init
      run: terraform init
      env:
        GOOGLE_CREDENTIALS: ${{ secrets.GOOGLE_CREDENTIALS }} 


    # Generates an execution plan for Terraform
    - name: Terraform Plan
      run: terraform plan -input=false
      env:
        GOOGLE_CREDENTIALS: ${{ secrets.GOOGLE_CREDENTIALS }}

    - name: Terraform Apply
      if: github.ref == 'refs/heads/main' && github.event_name == 'push'
      run: terraform apply -auto-approve -input=false
      env:
        GOOGLE_CREDENTIALS: ${{ secrets.GOOGLE_CREDENTIALS }}

Configuring terraform

Time to create terraform files. Terraform files have .tf extension. Create main.tf informing backend and provider and variables.tf to help you control input variables.

Terraform variables

In this example variables.tf file contains the variables project, region and data-project. We can create any variable to help better organize the project.

Variables can be declared by giving a value manually hardcoded:

variable "region" {
  type    = string
  default = "us-central1"
}

Or by integrating secrets or repository variables on GitHub project with Terraform workflow. For example, having these variables declared on GitHub project

It’s possible to use them as values for Terraform variables. By default, the GitHub Workflow looks for a .tfvars file to know about values to fill de variables. So let’s create this step on workflow.

    steps:
    # Checkout the repository to the GitHub Actions runner
    - name: Checkout
      uses: actions/checkout@v3

    # Install the latest version of Terraform CLI and configure the Terraform CLI configuration file with a Terraform Cloud user API token
    - name: Setup Terraform
      uses: hashicorp/setup-terraform@v1

    - name: Setup terraform variables
      id: vars
      run: |-
        cat > pipeline.auto.tfvars <<EOF
        region="${{ vars.GCP_REGION }}" 
        project="${{ vars.GCP_PROJECT }}" 
        data-project="${{ vars.GCP_DATA_PROJECT }}"
        EOF

Now those Terraform variables region, project and data-project will receive values direct from GitHub Action variables.

Your final GitHub workflow should look like this.

Creating Terraform Resources

Google Cloud services like a bucket in Google Cloud Storage, Cloud Functions, Dataproc Workflow, BigQuery datasets and tables are created as resources with Terraform.

A good practice when creating resources is separating them into files that represent the group like bucket.tf, dataproc.tf, bigquery.tf…

Terraform documentation is really good. Wherever you want to create on Google Cloud Platform there’s an example on docs. To conclude the tutorial here’s an example of how to create a bucket with Terraform.

TIP: In this example, the bucket name has the data-project variable concatenated. This approach can be used specially on resources that are available in a global infrastructure and can’t have a repeated name. Then avoiding having issues related to the name of resources.

When push commit new files, the GitHub workflow will be triggered and Terraform actions will be played.

If the workflow runs with success the resource will be created.

PROBLEM: Can’t push to GitHub “Refusing to allow an OAuth App to create or update workflow without workflow scope”

If you have this message while trying git push your workflow updates, one solution is to generate a token in your GitHub account. For this, you can navigate to https://github.com/settings/tokens/new, and create a token with permission for workflow and repo.

Then click Generate token button and copy your token. In your machine open the Windows Credential Manager and edit the github.com credential updating your password with the token generated previously.