Title
#general
Ali (Infracost)

Ali (Infracost)

09/15/2022, 10:41 PM
Hey Guy, the config-file can be used against either Terraform dirs or plan JSON files. The error you see is because you’re passing in
--path
too, so it should work fine if you remove that, like this:
infracost breakdown \
                                  --format=json \
                                  --config-file /atlantis-data/repos/b2c2/infrastructure/$PULL_NUM/default/infracost-generated.yml
                                  --log-level=info \
                                  --out-file=$INFRACOST_OUTPUT
If you’re running
infracost diff --config-file
AND the config-file has paths to plan JSON files (which it should given that you’re using Atlantis with $SHOWFILE), then it should work as normal. (if you’re passing in Terraform directories, you can use option 2 mentioned in the docs, but that doesn’t apply to you)
g

Gurpal Singh

09/15/2022, 10:43 PM
cheers for the reply
10:43 PM
what's the significance of removing
--path=$SHOWFILE
?
10:44 PM
which path gets planned then?
10:44 PM
i definitely don't want all paths in the config to get planned, if that's what you mean
Ali (Infracost)

Ali (Infracost)

09/15/2022, 10:44 PM
Whatever paths are defined inside the config-file are used
g

Gurpal Singh

09/15/2022, 10:44 PM
ah, ok, i had a fundemental misunderstanding of what the config file was then
10:45 PM
i thought the config file helps
infracost
understand which variables to set depending on what paths were being parsed to it
Ali (Infracost)

Ali (Infracost)

09/15/2022, 10:45 PM
Config-file is a way to run the CLI over multiple paths. Maybe I need to explain that better in our docs 😅
10:45 PM
each path has associated vars too, so a config-file can be used to define the vars for each path
g

Gurpal Singh

09/15/2022, 10:45 PM
yes. thats why i did it
10:46 PM
because i need a different session token per directory
Ali (Infracost)

Ali (Infracost)

09/15/2022, 10:46 PM
cool, so why don’t you want all the paths to be evaluated by the CLI on each run?
10:46 PM
i’m happy to jump on a quick zoom call if you prefer that, it might be faster
g

Gurpal Singh

09/15/2022, 10:47 PM
because i didn't realise the config file was a replacement for --path
10:47 PM
hmm
10:47 PM
do you have examples of this working in a terragrunt repo that deploys over multiple accounts?
Ali (Infracost)

Ali (Infracost)

09/15/2022, 10:48 PM
I assume the examples in here weren’t usefu? specifically that last tab
10:49 PM
your
path:
values would be plan JSON files though, not directories
g

Gurpal Singh

09/15/2022, 10:53 PM
ok, so then tell me if i understand correctly. in order to get a diff of costs commented in my PR, based on actual usage fetched from cloud watch, i need to do the following: generate a config file pointing to the jsons for each dir being changed.
path: aws/accounts/connectivity/eu-west-1/ec2/plan.json
    env:
      AWS_ACCESS_KEY_ID: xxxxx
      AWS_SECRET_ACCESS_KEY: xx+xx/xx
      AWS_SESSION_TOKEN: xxxxx

 path: aws/accounts/connectivity/eu-west-1/eks/plan.json
    env:
      AWS_ACCESS_KEY_ID: xxxxx
      AWS_SECRET_ACCESS_KEY: xx+xx/xx
      AWS_SESSION_TOKEN: xxxxx
and then use a command like this?
infracost breakdown \
                                  --format=json \
                                  --config-file /atlantis-data/repos/org/infrastructure/$PULL_NUM/default/infracost-generated.yml
                                  --log-level=info \
                                  --out-file=$INFRACOST_OUTPUT
Ali (Infracost)

Ali (Infracost)

09/15/2022, 10:56 PM
Yep, almost, to fetch the usage from cloudwatch, you’d need this:
infracost breakdown \
  --format=json \
  --config-file /atlantis-data/repos/b2c2/infrastructure/$PULL_NUM/default/infracost-generated.yml
  --log-level=info \
  --out-file=$INFRACOST_OUTPUT \
  --sync-usage-file --usage-file /tmp/ignore.yml
The last line is what tells the CLI to fetch the usage data and use a tmp file as you don’t care about storing that usage or checking it into your repo, you just want the CLI output to include the usage data. This workaround is mentioned here in the docs too.
10:57 PM
sorry wait, I got that wrong… one min
11:00 PM
The usage-file is also defined in the config-file on a per-path basis, so it would look like this:
# infracost-config.yml file:
version: 0.1
projects:
  - path: aws/accounts/connectivity/eu-west-1/ec2/plan.json
    usage_file: /tmp/infracost-usage-1.yml
    env:
      AWS_ACCESS_KEY_ID: xxxxx
      AWS_SECRET_ACCESS_KEY: xx+xx/xx
      AWS_SESSION_TOKEN: xxxxx

  - path: aws/accounts/connectivity/eu-west-1/eks/plan.json
    usage_file: /tmp/infracost-usage-2.yml
    env:
      AWS_ACCESS_KEY_ID: xxxxx
      AWS_SECRET_ACCESS_KEY: xx+xx/xx
      AWS_SESSION_TOKEN: xxxxx
infracost breakdown \
  --format=json \
  --config-file /atlantis-data/repos/org/infrastructure/$PULL_NUM/default/infracost-generated.yml
  --log-level=info \
  --out-file=$INFRACOST_OUTPUT \
  --sync-usage-file
g

Gurpal Singh

09/15/2022, 11:17 PM
I'm actually thinking I will need to generate the config for only the directory that has had changes made.
11:17 PM
I'm not sure how I'd do that
Ali (Infracost)

Ali (Infracost)

09/15/2022, 11:22 PM
I don’t really understand your setup so it’s hard to advise but the majority of infracost users run infracost across the whole repo (or all plan JSON files they have) as otherwise the cost comment shows a % increase/decrease that is not inclusive of the total repo costs.
g

Gurpal Singh

09/16/2022, 10:20 AM
that's interesting. we use Atlantis which shows you planned changes, are you saying everyone who runs the atlantis integration, runs infracost on the entire repo? even though they may only be changing a small part of their infrastructure? i.e
dev/eks/
10:20 AM
i thought it would make more sense only to say the cost differences for the changes you are applying
10:20 AM
which is why you pass infracost the plan file.
10:20 AM
what am i missing?
Ali (Infracost)

Ali (Infracost)

09/16/2022, 8:49 PM
For Atlantis when using the $SHOWFILE, you’re right that it’s better to just run Infracost against that file as that’s the only changed plan. But since you’re auto-generating a config-file, I’m not sure how you’ll get “only changed plan files” - that’s why I wondered if it’s easier to run Infracost against all plan files. Anyhow, you know your infra setup best so whatever works for you is ok
g

Gurpal Singh

09/16/2022, 8:50 PM
i managed to figure this out
8:50 PM
#!/usr/bin/env bash
set -e
export HOME=/tmp

process(){
  case $1 in

    "prod")
      accountId=x
      ;;

    "nonprod")
      accountId=y
      ;;

    "shared")
      accountId=z
      ;;

    "connectivity" )
      accountId=w
      ;;

    *)
      exit 1
      ;;
  esac

  credentials="$(aws sts assume-role --role-arn=arn:aws:iam::${accountId}:role/infracost --role-session-name session | jq -r .Credentials)"
  AccessKeyId="$(echo "${credentials}" | jq -r .AccessKeyId)"
  SecretAccessKey="$(echo "${credentials}" | jq -r .SecretAccessKey)"
  SessionToken="$(echo "${credentials}" | jq -r .SessionToken)"

  echo -e "version: 0.1\n\nprojects:" > "${CONFIG_DIR}"/infracost-generated.yml
  echo -e "  - path: $JSONFILE" >> "${CONFIG_DIR}"/infracost-generated.yml
  echo -e "    usage_file: usage.yml" >> "${CONFIG_DIR}"/infracost-generated.yml
  echo -e "    env:" >> "${CONFIG_DIR}"/infracost-generated.yml
  echo -e "      AWS_ACCESS_KEY_ID: $AccessKeyId" >> "${CONFIG_DIR}"/infracost-generated.yml
  echo -e "      AWS_SECRET_ACCESS_KEY: $SecretAccessKey" >> "${CONFIG_DIR}"/infracost-generated.yml
  echo -e "      AWS_SESSION_TOKEN: $SessionToken\n" >> "${CONFIG_DIR}"/infracost-generated.yml
}

CONFIG_DIR=$(dirname "$PLANFILE")
JSONFILE="${CONFIG_DIR}/default.json"

ACCOUNT=$(echo "${PLANFILE}" | cut -d/ -f10 )

process $ACCOUNT
Ali (Infracost)

Ali (Infracost)

09/16/2022, 8:51 PM
That’s some sweet bash right there, thanks for sharing, the rest of the community might find it useful
g

Gurpal Singh

09/16/2022, 8:51 PM
yeah, i'm gonna write a medium post soon.
8:51 PM
i'll send it when it's done
8:52 PM
i hate using bash, i'm sure there's a cleaner way i could have done it. but it's the first thing that came to my head
Ali (Infracost)

Ali (Infracost)

09/16/2022, 8:53 PM
hahaha, all of our CI/CD integrations were purely bash before we made
infracost comment
Great! Feel free to share the link on the general chat, I can also include it in the monthly blog+newsletter.