Title
#general
p

Pancho Daskalov

10/05/2022, 7:22 PM
Hey everyone! Iโ€™ve been trying to setup Infracost in CircleCI pipeline and managed to
generate an Infracost JSON file as the baseline
+ 
generate a diff by comparing the latest code change with the baseline
and the Infracost PR comments gets published as expected, however for some reason the
infracost
estimate remains:
unchanged
every time and it does not seem to update ๐Ÿ“‰ and ๐Ÿ“ˆ emojis as changes are pushed --- What Iโ€™ve checked so far:
[x] CircleCI Box: /tmp dir structure

.
โ”œโ”€โ”€ base
โ”œโ”€โ”€ circle-agent-runner.pid
โ”œโ”€โ”€ circleci-1594698576
โ”œโ”€โ”€ circleci-ts.sock
โ”œโ”€โ”€ infracost.json
โ””โ”€โ”€ infracost-base.json

[x] ran cost estimate breakdown locally 
infracost breakdown --path .

It produced all the cloud resources that got detected while on the PR it showed only the ones that got changed --- on local and on the PR the overall cost however matches as expected! 

[?] as I have CircleCI orb [circleci/terraform@3.1.0] for terraform, did terraform show -json plan.out > infracost-base.json so I can persist it and attach it within the Infracost job but somehow got lost on the way and not sure if that is needed as Infracost job itself already produced the terraform plan ...
Any advise or direction on how to continue will be greatly appreciated!
Vadim (Infracost)

Vadim (Infracost)

10/05/2022, 7:27 PM
Hello! Can you share the comment command you're using?
p

Pancho Daskalov

10/05/2022, 7:29 PM
sure! --- formatting does not seem well, though!
- run:
          name: Post Infracost comment
          command: |
              # Extract the PR number from the PR URL
              PULL_REQUEST_NUMBER=${CIRCLE_PULL_REQUEST##*/}
              infracost comment github --path=/tmp/infracost.json \
                                       --repo=$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME \
                                       --pull-request=$PULL_REQUEST_NUMBER \
                                       --github-token=$GITHUB_TOKEN \
                                       --behavior=update
Vadim (Infracost)

Vadim (Infracost)

10/05/2022, 7:30 PM
W/r/t the terraform plan creation: is there a reason you're generating a plan? Infracost parses HCL without TF binary, and it's much quicker, so if you don't need the plan for Infracost, you can skip it ๐Ÿ™‚
7:31 PM
The comment command looks okay, may I also ask how you're generating the diff? Do you follow our examples maybe?
p

Pancho Daskalov

10/05/2022, 7:32 PM
sure! And yes followed the example as it is available from here: link
- run:
          name: Generate Infracost cost estimate baseline
          command: |
              infracost diff --path=${TF_ROOT} \
                            --format=json \
                            --compare-to=/tmp/infracost-base.json \
                            --out-file=/tmp/infracost.json
7:33 PM
My initial thought was I need to generate terraform plan from the
circleci orb
which I am using and use it so I can attach it to the infracost job --- then found out that is not needed as infracost does it already anyway
Vadim (Infracost)

Vadim (Infracost)

10/05/2022, 7:36 PM
Ah, I think I see the problem in our docs: the idea is to generate the baseline against your main/master branch. So when you diff it with your feature branch, there, well, will be a diff ๐Ÿ˜„ It seems like the instructions have an error on this line: https://github.com/infracost/infracost-circleci/blob/master/README.md?plain=1#L80
7:36 PM
Instead of
TF_ROOT
it should be
/tmp/base
where we clone the main branch on line https://github.com/infracost/infracost-circleci/blob/master/README.md?plain=1#L62
7:37 PM
Could you try to change it and re-run?
p

Pancho Daskalov

10/05/2022, 7:37 PM
sure! Will do it now! ๐Ÿ™‚
7:45 PM
Just to make sure, I got this right!
jobs:
  infracost:
    docker:
      - image: infracost/infracost:ci-0.10
    environment:
      TF_ROOT: /tmp/base
      BASE_BRANCH: poc/setup-infracost
changed
TF_ROOT
to
/tmp/base
and for
BASE_BRANCH
I am still using the feat/ branch --- in this case
poc/setup-infracost
Vadim (Infracost)

Vadim (Infracost)

10/05/2022, 7:46 PM
I think
BASE_BRANCH
should be the branch you want to compare with
7:46 PM
It can be main/master or maybe another branch, but not the same as your feature branch, otherwise they will have identical TF code, right?
7:47 PM
Oh, one more thing, sorry, missed it
7:48 PM
Yeah, sorry, I'm slow ๐Ÿ˜„
7:49 PM
TF_ROOT: /tmp/base
- TF_ROOT is the main path of your code from feature branch, in the example it is
terraform
7:49 PM
Let me explain the flow ๐Ÿ™‚
7:51 PM
1. We clone your feature branch to
TF_ROOT
- it is our main place to run Infracost 2. We clone the master branch to
/tmp/base
- it is our baseline place 3. We run
infracost breakdown --path /tmp/base --out-file /tmp/infracost-base.json --format json
to generate the baseline Infracost data 4. Now we need to run
infracost diff --path TF_ROOT --compare-to /tmp/infracost-base.json
command (with other flags) - this way we compare baseline with your feature branch changes
7:52 PM
So in your case you would change only one line - the one with
infracost breakdown
command
7:53 PM
Let me push the fix to our example, hopefully it would make sense visually ๐Ÿ™‚
p

Pancho Daskalov

10/05/2022, 7:59 PM
Explaining the whole flow has been very useful! Appreciate it very much! ๐Ÿ™
Vadim (Infracost)

Vadim (Infracost)

10/05/2022, 7:59 PM
Let's see if this works ๐Ÿ™‚
p

Pancho Daskalov

10/05/2022, 8:00 PM
testing it now!
Vadim (Infracost)

Vadim (Infracost)

10/05/2022, 8:04 PM
Awesome! ๐Ÿ˜›artyparrot: Glad it worked! ๐Ÿ™‚
p

Pancho Daskalov

10/05/2022, 8:04 PM
Iโ€™ve spent quite a lot of time trying to figure it out and couldnโ€™t so that has been amazing! ๐Ÿ™‚
Vadim (Infracost)

Vadim (Infracost)

10/05/2022, 8:04 PM
To be fair, it's our example had an error, so you were not wrong ๐Ÿ˜„
8:04 PM
Sorry about that ๐Ÿ˜„
8:05 PM
Thank you for your patience! We fixed the issue, hopefully other CircleCI users will have a smoother experience ๐Ÿ™‚
p

Pancho Daskalov

10/05/2022, 8:06 PM
All good! I guess you hear that a lot of time, but you guys are making a difference and this tools brings a lot of value! ๐Ÿ™‚ If there is anything I can help out with or contribute with my humble experience I will be more than happy to take on anything you got on your backlog!
Vadim (Infracost)

Vadim (Infracost)

10/05/2022, 8:07 PM
Oh, thank you for the kind words! This means a lot to us :infraheart:
8:08 PM
If you're willing to contribute, we have some issues that are good to get familiar with the Infracost code: https://github.com/infracost/infracost/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22.
8:10 PM
Look around and if you find anything you like, just leave a comment that you'd like to work on it, and we'll gladly assign you ๐Ÿ™‚
8:11 PM
Thank you for using Infracost and for your interest! ๐Ÿ™‚
8:15 PM
Please forgive me for a shameless plug: we are also working on our SaaS with the goal to help your team and organization to tackle the cloud costs even better than CLI alone. https://dashboard.infracost.io/
p

Pancho Daskalov

10/06/2022, 11:38 AM
will sure check it out!
1:03 PM
Hey @Vadim (Infracost) have been using Infracost for sometime now and it works great. There is something that I noticed when you try to POST Infracost on Github - I get following
422 Validation Failed
422 Validation Failed [{Resource:IssueComment Field:data Code:unprocessable Message:Body is too long (maximum is 65536 characters)}]
It seems the Terraform plan output is too big to display --- any idea how I can navigate this?
Vadim (Infracost)

Vadim (Infracost)

11/02/2022, 1:05 PM
Hey! Glad to hear you have a nice experience! ๐Ÿ™‚ How do you do that? Using
infracost comment github
or doing a API request directly?
Tim (Infracost)

Tim (Infracost)

11/02/2022, 1:06 PM
And which infracost CLI version are you using? This was supposed to be fixed in v0.10.10
Vadim (Infracost)

Vadim (Infracost)

11/02/2022, 1:07 PM
Yep, that was my next question. Thanks Tim! ๐Ÿ˜„
Tim (Infracost)

Tim (Infracost)

11/02/2022, 1:08 PM
Although looking at that PR we use
GitHubMaxMessageSize = 262144
hmmm
Vadim (Infracost)

Vadim (Infracost)

11/02/2022, 1:09 PM
Yeah, it doesn't translate number of characters == the number of bytes
1:11 PM
Ah, I left a comment about that, in the commit message though, not in code: https://github.com/orgs/community/discussions/27190#discussioncomment-3254953
p

Pancho Daskalov

11/02/2022, 1:12 PM
I am currently using
Infracost v0.10.13
1:12 PM
for the Infracost CLI version
1:19 PM
just to follow up on Vadimโ€™s comment - I am using
infracost comment github
as part of the Post Infracost comment block for CircleCI
Vadim (Infracost)

Vadim (Infracost)

11/02/2022, 1:20 PM
Hm, interesting ๐Ÿค” As Tim mentioned, we fixed this issue and your CLI version should be okay. Let me try to replicate that
p

Pancho Daskalov

11/02/2022, 1:32 PM
could it be that I am still using this docker image
docker:
      - image: infracost/infracost:ci-0.10
1:32 PM
I mean, if a new img is released with an updated tag
Vadim (Infracost)

Vadim (Infracost)

11/02/2022, 1:33 PM
Yeah, this image should have the latest 0.10.x version
1:33 PM
Can you try to pull it just in case?
p

Pancho Daskalov

11/02/2022, 1:33 PM
sure, will try it now
p

Pancho Daskalov

11/02/2022, 1:37 PM
just pulled it
infracost/infracost:ci-0.10
1:44 PM
Iโ€™ve tried with following versions of the docker img
ci-0.10.13
ci-latest
and still getting the error
Vadim (Infracost)

Vadim (Infracost)

11/02/2022, 1:45 PM
I'm still trying to go over the limit. You need a bunch of resources for that ๐Ÿ˜…
1:45 PM
Okay, got the error now
1:59 PM
Hm, okay, seems like the message size is still off, although the truncation works ๐Ÿค”
2:00 PM
We're going to fix that and it should be out with the next release, but until then, you might be blocked ๐Ÿ˜ž
p

Pancho Daskalov

11/02/2022, 2:02 PM
all good, Vadim! That will not stop me from using Infracost as my team has seen immense value up until now โ€” will be around and wait for the new release ๐Ÿ™‚
2:02 PM
any idea, when is it expected?
Vadim (Infracost)

Vadim (Infracost)

11/02/2022, 2:03 PM
Not sure, maybe next week
2:04 PM
By the way, have you tried our new GitHub App integration?
2:04 PM
With it you don't need CircleCI integration, however, it's part of Infracost Cloud plan
p

Pancho Daskalov

11/02/2022, 2:06 PM
I havenโ€™t, I would have given it a try, itโ€™s not RFC compliant though that is why I am stuck with CircleCI --- I mean within the company that I work at
Vadim (Infracost)

Vadim (Infracost)

11/02/2022, 2:07 PM
Oh, I see
p

Pancho Daskalov

11/15/2022, 1:12 PM
Hey @Vadim (Infracost) itโ€™s me again, apologies for bothering! Here is what I am trying to do: I have a CircleCI workflow, which has a terraform orb and Iโ€™ve managed to generate the
Terraform plan JSON
and it produced the file
plan.json
which now I am trying to pass over to the Infracost job --- however I cannot attach it to workspace, because thatโ€™s already taken for the base branch โ€ฆ also I cannot pass it over to
TF_ROOT: terraform/plan.json
โ€™cause it wonโ€™t find it โ€ฆ
- attach_workspace:
          at: /tmp
1:13 PM
any ideas how can I pass the
plan.json
file that got generated to the infracost and get the diff and output the calculated cost?
Vadim (Infracost)

Vadim (Infracost)

11/15/2022, 1:13 PM
Hey! No worries ๐Ÿ™‚
1:14 PM
Can you show me the commands you're using?
p

Pancho Daskalov

11/15/2022, 1:14 PM
sure
Vadim (Infracost)

Vadim (Infracost)

11/15/2022, 1:15 PM
Usually we do something like
infracost breakdown --path=path/to/code --out-file=/tmp/infracost.json --format=json
1:15 PM
So it will save the Infracost JSON to /tmp/
1:15 PM
Then we use
infracost diff --path=path/to/code --compare-to=/tmp/infracost.json
1:15 PM
The
path/to/code
can be a Terraform JSON file too
p

Pancho Daskalov

11/15/2022, 1:17 PM
this is the
terraform
orb I am using
orbs:
  terraform: i circleci/terraform@3.1.0
I have a CircleCI job called:
terraform-build-deploy:
I produce the plan.json with
terraform show -json plan.out > plan.json
Vadim (Infracost)

Vadim (Infracost)

11/15/2022, 1:17 PM
Ah, okay
p

Pancho Daskalov

11/15/2022, 1:18 PM
oh, I see the
path=path/to/code
is where I tries to use the path to where the
plan.json file
is sotred
1:18 PM
but it wonโ€™t find it
Vadim (Infracost)

Vadim (Infracost)

11/15/2022, 1:18 PM
It should be the explicit file in this case
p

Pancho Daskalov

11/15/2022, 1:18 PM
as it attaches the workspace to
/temp
Vadim (Infracost)

Vadim (Infracost)

11/15/2022, 1:18 PM
path/plan.json
p

Pancho Daskalov

11/15/2022, 1:19 PM
explicit file, you mean the one generated by Infracost ,right?
Vadim (Infracost)

Vadim (Infracost)

11/15/2022, 1:19 PM
Sorry, I'll rephrase: when using TF JSON files, the
--path=
should point to the file, not just the directory where it is located.
1:20 PM
I'm not very familiar with CircleCI, is it possible to share the generated plan with other orbs?
1:24 PM
My understanding is that the terraform orb generates the plan and stores it somewhere (like
/temp
), and Infracost job would be able to access this
/temp
so the command would look like
infracost breakdown --path=/temp/plan.json
p

Pancho Daskalov

11/15/2022, 1:24 PM
good question! not sure --- I am familar with using only
workspaces
to share date between jobs
1:25 PM
so if I generate the
plan.json
from the terraform job in
/temp
1:25 PM
Infracost sohuld be able to find it
1:25 PM
right
1:25 PM
theortically
Vadim (Infracost)

Vadim (Infracost)

11/15/2022, 1:25 PM
From this https://circleci.com/docs/workspaces/ looks like that the Infracost jobs need to have something like
attach_workspace
to access the file
p

Pancho Daskalov

11/15/2022, 1:26 PM
will generate the file in
/temp
and then use
- persist_to_workspace:
Vadim (Infracost)

Vadim (Infracost)

11/15/2022, 1:27 PM
Yep, looks like it's required
p

Pancho Daskalov

11/15/2022, 1:27 PM
to store it in
/temp/plan.json
1:27 PM
and see if Infracost will find it
1:27 PM
give me a min and will try it
Vadim (Infracost)

Vadim (Infracost)

11/15/2022, 1:27 PM
Hope this works ๐Ÿ™‚
p

Pancho Daskalov

11/15/2022, 1:46 PM
ah, the
persist_to_workspace
wonโ€™t do it โ€ฆ it does not recognizr that there is such a dir/
The specified paths did not match any files in /tmp
Here is what I am thinking of --- having another
mid
job which carries the output from the
plan.json
and then attaching the workspace to
/tmp
where Infracost is looking for it
1:46 PM
it ainโ€™t very smart, but cannot think of anything else ๐Ÿ˜ž
Vadim (Infracost)

Vadim (Infracost)

11/15/2022, 1:47 PM
Sorry, I haven't used CircleCI, so can't be much help here ๐Ÿ˜’miling_face_with_tear:
1:48 PM
But first step is making it working, then refactor ๐Ÿ™‚
1:49 PM
The example in the docs also shows that "executor" should have the working directory set as
working_directory: /tmp
p

Pancho Daskalov

11/15/2022, 1:52 PM
oh, that is a good point! Will add the
working_directory
and try it out again! Please donโ€™t feel bad, I am very much
greateful
for you taking the time and looking into it as I was stuck and making no progress and ran out of ideas --- hope I am not bothering you and asking always stuff ๐Ÿ™‚
Vadim (Infracost)

Vadim (Infracost)

11/15/2022, 1:52 PM
No worries! We want to make our users happy ๐Ÿ™‚
p

Pancho Daskalov

11/16/2022, 2:34 PM
@Vadim (Infracost) managed to get it working with the
plan.json
that I created! ๐Ÿ™‚๐ŸŽ‰ Do you know by any chance if I can pass two
--path
within the
infracost diff --path ...
2:35 PM
as I have
multiple
---
plan.json
files that I want to pass on and this is a bit tricky โ€ฆ
2:35 PM
Iโ€™ve tried adding second
---path
but it does not seem to be working โ€ฆ
Vadim (Infracost)

Vadim (Infracost)

11/16/2022, 2:37 PM
Good job!
2:38 PM
The diff command expects only one path and one compare-to. So if you have multiples, I guess you'll have to run the diff command N times
2:39 PM
But then the comment/upload commands accept multiple paths to merge all diff info together
p

Pancho Daskalov

11/16/2022, 2:44 PM
Alright! Will give it a try to have the infracost diff replicated and then if the infracost comment supports multiple paths should do the trick! ๐Ÿ˜‰
2:45 PM
The reason why I am doing
this
is because I am expecting several
tfstate
files which I want to merge into one eventually โ€ฆ
Vadim (Infracost)

Vadim (Infracost)

11/16/2022, 2:45 PM
Are these different projects/repos?
p

Pancho Daskalov

11/16/2022, 2:46 PM
It is a
lerna
project with multiple
Cloudfront
distributuons and each Cf distro has
.tf
files associated with
.tf
modules + Lambda etcโ€ฆ
2:46 PM
and once setup, it will produce diff
tfstate
files for diff scenarios
Vadim (Infracost)

Vadim (Infracost)

11/16/2022, 2:47 PM
I wonder what is the baseline then?
p

Pancho Daskalov

11/16/2022, 2:48 PM
good question! I am avoiding it, I mean I donโ€™t setup
baseline
at all --- I am using the
Terraform plan JSON
and then run just
infracost diff --path ./terraform/tfplan.json
2:48 PM
and I am expecting multiple
tfplan.json
files which I somehow want to merge together into one file
Vadim (Infracost)

Vadim (Infracost)

11/16/2022, 2:49 PM
So diff will always show $0 -> $XXX change, right?
p

Pancho Daskalov

11/16/2022, 2:49 PM
tbh donโ€™t know yet, I guess you are right! ๐Ÿ™‚
Vadim (Infracost)

Vadim (Infracost)

11/16/2022, 2:50 PM
Okay, then in this case you don't need the diff command at all
2:50 PM
You can just use
infracost breakdown
and provide several
--path
2:51 PM
It will produce the infracost JSON that you can use with the comment command
2:52 PM
The point of the
diff
command is to have a baseline (for example, the master/main branch), and compare PR branches with the baseline
2:52 PM
If you don't have baseline (aka previous state), then the diff will produce the identical output as breakdown
2:53 PM
Sorry I misunderstood your setup
p

Pancho Daskalov

11/16/2022, 2:53 PM
No, all good! This is a really good point though! ๐Ÿ™‚ Using the
infracost
breakdown allows to have multiple options --- will give it a try now
3:32 PM
tried the
infracost
breakdown and it ran fine as you described it! However, now onto the
Infracost
comment I hit another roadblock ๐Ÿ˜ž
invalid Infracost JSON file version. Supported versions are 0.2 โ‰ค x โ‰ค 0.2
3:33 PM
Vadim (Infracost)

Vadim (Infracost)

11/16/2022, 3:34 PM
The comment command expects Infracost JSON file, the one you generate via
infracost breakdown --path path1 --path path2 --format json --out-file infracost.json
p

Pancho Daskalov

11/16/2022, 3:51 PM
Thanks @Vadim (Infracost) this is how I fixed it
infracost breakdown --path=./terraform \
                                  --format=json \
                                  --out-file=./terraform/tfplan.json
had to pass first the
./terraform
dir and then to
--out-file
I just pointed to the
tfplan.json
file which was expected
3:52 PM
then within the
infracost
comment
infracost comment github --path=./terraform/tfplan.json \
3:52 PM
Vadim (Infracost)

Vadim (Infracost)

11/16/2022, 3:52 PM
Awesome ๐Ÿ™‚ We deliberately call the output file as
infracost.json
to avoid confusion with tfplan files that have completely different format ๐Ÿ™‚
p

Pancho Daskalov

11/16/2022, 3:53 PM
Appreciate the help and sticking through all the Qs! ๐Ÿ™
Vadim (Infracost)

Vadim (Infracost)

11/16/2022, 3:53 PM
Of course ๐Ÿ™‚