We use an internally hosted gitlab instance, and are trying out the Gitlab app integrations. I am struggling to convert the current CI/CD work flow for Infracost, using docker image and terraform plan to the gitlab infracost app. Our CI/CD is built on an automated continuous deployment. We have review branches and main/master branches, production is deployed from a git tag on the main/master branch. Any change, human or bot (bots run every 6 hours to look for patches and updates) creates a merge request and review branch, branch names control the CI/CD jobs that run as defined in the gitlab-ci.yaml. A review branch kicks off a review pipeline, where all CI/CD security and QA checks are done, this is where we have located infracost, sit well with the guardrails, and cost checking and now the new tags checking. Assuming the CI/CD pipeline passed, any bot created merge request are automatically merged with the main/master branch and the merge request automatically closed (unless a gitlab project is set up to require a human merge review) as merge request are relative short lived we copy the merge request comments to the gitlab wiki as well to keep a easley viable record of the last pipeline run. Human created merge requests are merged, when the pipeline passes and any changes tested, but the same workflow happens once merged. When the merge is done, it starts a main/master branch CI/CD pipeline, it will run with minimal jobs mostly those required by Gitalb to add security checks and reports to the Gitlab dash boards. It also runs a semantic release job which does a semantic git tag on the repo. The semantic git tag starts a CI/CD pipeline for the tagged release to production. In this workflow we only have one merge request, the terraform code is written so that CI jobs can run on review or production variables are controlled by variables in the gitlab-ci.yaml file. Infracost Create Terraform Review Test Plan Json: # Create a terraform plan of review changes for Costs variables: TF_VAR_Gitlab_Environment: test TF_STATE_FILE_NAME: test TF_VAR_Docker_Image_version: test Infracost Create Terraform Production Plan Json: # Create a terraform plan of production changes for Costs variables: TF_VAR_Gitlab_Environment: prod TF_STATE_FILE_NAME: deployed TF_VAR_Docker_Image_version: latest `````` In the review pipeline we can run two jobs one to create the production plan and one to create the review plan, then run these through the Infracost docker image, to get two merge request comments on for production and one for review. needs: ["Infracost Create Terraform Review Test Plan Json"] image: # Always use the latest 0.10.x version to pick up bug fixes and new resources. # See https://www.infracost.io/docs/integrations/cicd/#docker-images for other options name: infracost/infracost:ci-0.10 entrypoint: [""] variables: Environment: test TF_STATE_FILE_NAME: test INFRACOST_VCS_BASE_BRANCH: $CI_DEFAULT_BRANCH INFRACOST_VCS_BRANCH: $CI_COMMIT_REF_NAME script: - echo "OPEN MERGE REQUEST ID:" "${CI_OPEN_MERGE_REQUESTS##*\!}" - | if [ -n "${CI_OPEN_MERGE_REQUESTS##*\!}" ]; then export INFRACOST_VCS_PULL_REQUEST_TITLE=$(curl --silent --request GET --header "PRIVATE-TOKEN:${AUTOMATION_ADMIN_TOKEN}" "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/merge_requests/${CI_OPEN_MERGE_REQUESTS##*\!}" | jq -r '.title') export INFRACOST_VCS_PULL_REQUEST_URL="${CI_PROJECT_URL}/-/merge_requests/${CI_OPEN_MERGE_REQUESTS##*\!}" echo "INFRACOST_VCS_PULL_REQUEST_TITLE:" "$INFRACOST_VCS_PULL_REQUEST_TITLE" echo "INFRACOST_VCS_PULL_REQUEST_URL:" "$INFRACOST_VCS_PULL_REQUEST_URL" fi - infracost diff --path=makefile-tokens-dir/${TF_STATE_FILE_NAME}-tf-plan.json --format=json --out-file=makefile-tokens-dir/${Environment}-infracost-diff.json - infracost comment gitlab --path=makefile-tokens-dir/${Environment}-infracost-diff.json --repo=$CI_PROJECT_PATH --merge-request=1 --gitlab-server-url=$CI_SERVER_URL --gitlab-token=$AUTOMATION_ADMIN_TOKEN --behavior=update --dry-run > ./wiki/${Environment}-infracost-diff.md - infracost breakdown --path=makefile-tokens-dir/${TF_STATE_FILE_NAME}-tf-plan.json --format=html --out-file=./wiki/${Environment}-infracost-breakdown.html - | if [ -n "${CI_OPEN_MERGE_REQUESTS##*\!}" ]; then infracost comment gitlab --path=makefile-tokens-dir/${Environment}-infracost-diff.json --repo=$CI_PROJECT_PATH --merge-request="${CI_OPEN_MERGE_REQUESTS##*\!}" --gitlab-server-url=$CI_SERVER_URL --gitlab-token=$AUTOMATION_ADMIN_TOKEN --tag=$Environment --behavior=update fi This job creates the merge request comment for review and creates a file that is then uploaded to the Gitlab wiki as documentation and a similar one runs for production. Is it possible to overlay the infracost external app job with stages and Gitlab environment variables ? How would we set this up using the infracost config file ? Can we create an infracost config yaml template and generate an infracost yaml config on the fly for each repo using variables from the gitlab-ci yaml file. Once the app has written to the merge request we can extract the notes via api call and write them to the wiki.

Hi Joe, I’ve got a few questions to make sure I understand correctly what you want to do 1. When a user/bot opens a MR request to master you create two infracost comments on the MR. One for the test env and one for the prod env? 2. Following on from the above. How do these two environments differ and how is this configured in terraform? i.e. do you pass a different terraform tfvar file/workspace? 3. Infracost Gitlab app only has the ability to post one comment right now. Following the logic above, we’d normally suggest users aggregate the Infracost output into one single comment. With this approach you’d have a table with all your projects shown. e.g. “test-some-app”, “prod-some-app”. Would this approach work for you, or do you require the separate comments to post to different sections in your wiki? 4. It is not possible to define stages in Infracost Gitlab app, but with the CI/CD pipeline you’ve described I don’t believe it’s necessary to. We should hopefully be able to get you a working config file/template that will provide the correct configurations for your setup.

In answer to question 1) yes (in some case there may only be one env, due to licencing or as no prodction instance has been completed) 2) The difference between Environments is set in the gitlab-ci.yaml file as terraform Environment variables. Or in some case the terraform main.tf if very static, the terraform code then dose a selection using a Gitlab variable set in the job. Due to some use case we have two main terraform Env we look for that are configure at a CI job stage

TF_VAR_Gitlab_Environment: prod or TF_VAR_Gitlab_Environment: test
      TF_VAR_Greenpeace_Environment: test or TF_VAR_Greenpeace_Environment: prod

3) One comment as documentation is fine.

ok so in this case it should be quite easy to define a config file with the required variables. You would just specify these environment variables in the

env

key of the project, e.g;

version: 0.1
projects:
  - path: some/path/to/terraformcode
    name: dev
    terraform_vars:
      Gitlab_Environment: prod
      Greenpeace_Environment: test

I’m not sure if the

Gitlab_Environment

is actually used in the terraform code, if not just ommit it. You would then specify all your terraform projects by defining them in the config file. However, I’d recommend using our config template syntax as this will avoid duplication. For example you could do things like:

version: 0.1

projects:
{{- $envs := list "prod" "test"}}
{{- range $envs }}
      - path: someapp
        name: {{ . }}-someapp
        terraform_vars:
          Greenpeace_Environment: {{ . }}
{{- end }}

It’s a little bit tricky to help further without knowing your project structure. And this is probably easier to go over on a call. We can get all the config template and stuff ironed out in that. Would a call tomorrow work?

yes i free most of the morning