https://infracost.io logo
Powered by
Title
w

white-airport-8778

09/15/2022, 10:41 PM
Hey Guy, the config-file can be used against either Terraform dirs or plan JSON files. The error you see is because you’re passing in 
--path
too, so it should work fine if you remove that, like this: 
infracost breakdown \
                                  --format=json \
                                  --config-file /atlantis-data/repos/b2c2/infrastructure/$PULL_NUM/default/infracost-generated.yml
                                  --log-level=info \
                                  --out-file=$INFRACOST_OUTPUT
If you’re running 
infracost diff --config-file
AND the config-file has paths to plan JSON files (which it should given that you’re using Atlantis with $SHOWFILE), then it should work as normal. (if you’re passing in Terraform directories, you can use option 2 mentioned in the docs, but that doesn’t apply to you)
w

worried-soccer-67640

09/15/2022, 10:43 PM
cheers for the reply
what's the significance of removing 
--path=$SHOWFILE
?
which path gets planned then?
i definitely don't want all paths in the config to get planned, if that's what you mean
w

white-airport-8778

09/15/2022, 10:44 PM
Whatever paths are defined inside the config-file are used
w

worried-soccer-67640

09/15/2022, 10:44 PM
ah, ok, i had a fundemental misunderstanding of what the config file was then
i thought the config file helps 
infracost
understand which variables to set depending on what paths were being parsed to it
w

white-airport-8778

09/15/2022, 10:45 PM
Config-file is a way to run the CLI over multiple paths. Maybe I need to explain that better in our docs 😅
each path has associated vars too, so a config-file can be used to define the vars for each path
w

worried-soccer-67640

09/15/2022, 10:45 PM
yes. thats why i did it
because i need a different session token per directory
w

white-airport-8778

09/15/2022, 10:46 PM
cool, so why don’t you want all the paths to be evaluated by the CLI on each run?
i’m happy to jump on a quick zoom call if you prefer that, it might be faster
w

worried-soccer-67640

09/15/2022, 10:47 PM
because i didn't realise the config file was a replacement for --path
hmm
do you have examples of this working in a terragrunt repo that deploys over multiple accounts?
w

white-airport-8778

09/15/2022, 10:48 PM
I assume the examples in here weren’t usefu? specifically that last tab
your 
path:
values would be plan JSON files though, not directories
w

worried-soccer-67640

09/15/2022, 10:53 PM
ok, so then tell me if i understand correctly. in order to get a diff of costs commented in my PR, based on actual usage fetched from cloud watch, i need to do the following: generate a config file pointing to the jsons for each dir being changed. 
path: aws/accounts/connectivity/eu-west-1/ec2/plan.json
    env:
      AWS_ACCESS_KEY_ID: xxxxx
      AWS_SECRET_ACCESS_KEY: xx+xx/xx
      AWS_SESSION_TOKEN: xxxxx

 path: aws/accounts/connectivity/eu-west-1/eks/plan.json
    env:
      AWS_ACCESS_KEY_ID: xxxxx
      AWS_SECRET_ACCESS_KEY: xx+xx/xx
      AWS_SESSION_TOKEN: xxxxx
and then use a command like this? 
infracost breakdown \
                                  --format=json \
                                  --config-file /atlantis-data/repos/org/infrastructure/$PULL_NUM/default/infracost-generated.yml
                                  --log-level=info \
                                  --out-file=$INFRACOST_OUTPUT
w

white-airport-8778

09/15/2022, 10:56 PM
Yep, almost, to fetch the usage from cloudwatch, you’d need this: 
infracost breakdown \
  --format=json \
  --config-file /atlantis-data/repos/b2c2/infrastructure/$PULL_NUM/default/infracost-generated.yml
  --log-level=info \
  --out-file=$INFRACOST_OUTPUT \
  --sync-usage-file --usage-file /tmp/ignore.yml
The last line is what tells the CLI to fetch the usage data and use a tmp file as you don’t care about storing that usage or checking it into your repo, you just want the CLI output to include the usage data. This workaround is mentioned here in the docs too.
sorry wait, I got that wrong… one min
The usage-file is also defined in the config-file on a per-path basis, so it would look like this: 
# infracost-config.yml file:
version: 0.1
projects:
  - path: aws/accounts/connectivity/eu-west-1/ec2/plan.json
    usage_file: /tmp/infracost-usage-1.yml
    env:
      AWS_ACCESS_KEY_ID: xxxxx
      AWS_SECRET_ACCESS_KEY: xx+xx/xx
      AWS_SESSION_TOKEN: xxxxx

  - path: aws/accounts/connectivity/eu-west-1/eks/plan.json
    usage_file: /tmp/infracost-usage-2.yml
    env:
      AWS_ACCESS_KEY_ID: xxxxx
      AWS_SECRET_ACCESS_KEY: xx+xx/xx
      AWS_SESSION_TOKEN: xxxxx
infracost breakdown \
  --format=json \
  --config-file /atlantis-data/repos/org/infrastructure/$PULL_NUM/default/infracost-generated.yml
  --log-level=info \
  --out-file=$INFRACOST_OUTPUT \
  --sync-usage-file
w

worried-soccer-67640

09/15/2022, 11:17 PM
I'm actually thinking I will need to generate the config for only the directory that has had changes made.
I'm not sure how I'd do that
w

white-airport-8778

09/15/2022, 11:22 PM
I don’t really understand your setup so it’s hard to advise but the majority of infracost users run infracost across the whole repo (or all plan JSON files they have) as otherwise the cost comment shows a % increase/decrease that is not inclusive of the total repo costs.
w

worried-soccer-67640

09/16/2022, 10:20 AM
that's interesting. we use Atlantis which shows you planned changes, are you saying everyone who runs the atlantis integration, runs infracost on the entire repo? even though they may only be changing a small part of their infrastructure? i.e 
dev/eks/
i thought it would make more sense only to say the cost differences for the changes you are applying
which is why you pass infracost the plan file.
what am i missing?
w

white-airport-8778

09/16/2022, 8:49 PM
For Atlantis when using the $SHOWFILE, you’re right that it’s better to just run Infracost against that file as that’s the only changed plan. But since you’re auto-generating a config-file, I’m not sure how you’ll get “only changed plan files” - that’s why I wondered if it’s easier to run Infracost against all plan files. Anyhow, you know your infra setup best so whatever works for you is ok
w

worried-soccer-67640

09/16/2022, 8:50 PM
i managed to figure this out
#!/usr/bin/env bash
set -e
export HOME=/tmp

process(){
  case $1 in

    "prod")
      accountId=x
      ;;

    "nonprod")
      accountId=y
      ;;

    "shared")
      accountId=z
      ;;

    "connectivity" )
      accountId=w
      ;;

    *)
      exit 1
      ;;
  esac

  credentials="$(aws sts assume-role --role-arn=arn:aws:iam::${accountId}:role/infracost --role-session-name session | jq -r .Credentials)"
  AccessKeyId="$(echo "${credentials}" | jq -r .AccessKeyId)"
  SecretAccessKey="$(echo "${credentials}" | jq -r .SecretAccessKey)"
  SessionToken="$(echo "${credentials}" | jq -r .SessionToken)"

  echo -e "version: 0.1\n\nprojects:" > "${CONFIG_DIR}"/infracost-generated.yml
  echo -e "  - path: $JSONFILE" >> "${CONFIG_DIR}"/infracost-generated.yml
  echo -e "    usage_file: usage.yml" >> "${CONFIG_DIR}"/infracost-generated.yml
  echo -e "    env:" >> "${CONFIG_DIR}"/infracost-generated.yml
  echo -e "      AWS_ACCESS_KEY_ID: $AccessKeyId" >> "${CONFIG_DIR}"/infracost-generated.yml
  echo -e "      AWS_SECRET_ACCESS_KEY: $SecretAccessKey" >> "${CONFIG_DIR}"/infracost-generated.yml
  echo -e "      AWS_SESSION_TOKEN: $SessionToken\n" >> "${CONFIG_DIR}"/infracost-generated.yml
}

CONFIG_DIR=$(dirname "$PLANFILE")
JSONFILE="${CONFIG_DIR}/default.json"

ACCOUNT=$(echo "${PLANFILE}" | cut -d/ -f10 )

process $ACCOUNT
w

white-airport-8778

09/16/2022, 8:51 PM
That’s some sweet bash right there, thanks for sharing, the rest of the community might find it useful
w

worried-soccer-67640

09/16/2022, 8:51 PM
yeah, i'm gonna write a medium post soon.
i'll send it when it's done
i hate using bash, i'm sure there's a cleaner way i could have done it. but it's the first thing that came to my head
w

white-airport-8778

09/16/2022, 8:53 PM
hahaha, all of our CI/CD integrations were purely bash before we made 
infracost comment
Great! Feel free to share the link on the general chat, I can also include it in the monthly blog+newsletter.
#generalJoin Slack
Powered by