https://infracost.io logo
w

witty-book-85526

10/05/2022, 7:22 PM
Hey everyone! Iโ€™ve been trying to setup Infracost in CircleCI pipeline and managed to
Copy code
generate an Infracost JSON file as the baseline
+ 
generate a diff by comparing the latest code change with the baseline
and the Infracost PR comments gets published as expected, however for some reason the
infracost
estimate remains:
unchanged
every time and it does not seem to update ๐Ÿ“‰ and ๐Ÿ“ˆ emojis as changes are pushed --- What Iโ€™ve checked so far:
Copy code
[x] CircleCI Box: /tmp dir structure

.
โ”œโ”€โ”€ base
โ”œโ”€โ”€ circle-agent-runner.pid
โ”œโ”€โ”€ circleci-1594698576
โ”œโ”€โ”€ circleci-ts.sock
โ”œโ”€โ”€ infracost.json
โ””โ”€โ”€ infracost-base.json

[x] ran cost estimate breakdown locally 
infracost breakdown --path .

It produced all the cloud resources that got detected while on the PR it showed only the ones that got changed --- on local and on the PR the overall cost however matches as expected! 

[?] as I have CircleCI orb [circleci/terraform@3.1.0] for terraform, did terraform show -json plan.out > infracost-base.json so I can persist it and attach it within the Infracost job but somehow got lost on the way and not sure if that is needed as Infracost job itself already produced the terraform plan ...
Any advise or direction on how to continue will be greatly appreciated!
b

busy-agent-35515

10/05/2022, 7:27 PM
Hello! Can you share the comment command you're using?
w

witty-book-85526

10/05/2022, 7:29 PM
sure! --- formatting does not seem well, though!
Copy code
- run:
          name: Post Infracost comment
          command: |
              # Extract the PR number from the PR URL
              PULL_REQUEST_NUMBER=${CIRCLE_PULL_REQUEST##*/}
              infracost comment github --path=/tmp/infracost.json \
                                       --repo=$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME \
                                       --pull-request=$PULL_REQUEST_NUMBER \
                                       --github-token=$GITHUB_TOKEN \
                                       --behavior=update
b

busy-agent-35515

10/05/2022, 7:30 PM
W/r/t the terraform plan creation: is there a reason you're generating a plan? Infracost parses HCL without TF binary, and it's much quicker, so if you don't need the plan for Infracost, you can skip it ๐Ÿ™‚
The comment command looks okay, may I also ask how you're generating the diff? Do you follow our examples maybe?
w

witty-book-85526

10/05/2022, 7:32 PM
sure! And yes followed the example as it is available from here: link
Copy code
- run:
          name: Generate Infracost cost estimate baseline
          command: |
              infracost diff --path=${TF_ROOT} \
                            --format=json \
                            --compare-to=/tmp/infracost-base.json \
                            --out-file=/tmp/infracost.json
My initial thought was I need to generate terraform plan from the
circleci orb
which I am using and use it so I can attach it to the infracost job --- then found out that is not needed as infracost does it already anyway
b

busy-agent-35515

10/05/2022, 7:36 PM
Ah, I think I see the problem in our docs: the idea is to generate the baseline against your main/master branch. So when you diff it with your feature branch, there, well, will be a diff ๐Ÿ˜„ It seems like the instructions have an error on this line: https://github.com/infracost/infracost-circleci/blob/master/README.md?plain=1#L80
Instead of
TF_ROOT
it should be
/tmp/base
where we clone the main branch on line https://github.com/infracost/infracost-circleci/blob/master/README.md?plain=1#L62
Could you try to change it and re-run?
w

witty-book-85526

10/05/2022, 7:37 PM
sure! Will do it now! ๐Ÿ™‚
Just to make sure, I got this right!
Copy code
jobs:
  infracost:
    docker:
      - image: infracost/infracost:ci-0.10
    environment:
      TF_ROOT: /tmp/base
      BASE_BRANCH: poc/setup-infracost
changed
TF_ROOT
to
/tmp/base
and for
BASE_BRANCH
I am still using the feat/ branch --- in this case
poc/setup-infracost
b

busy-agent-35515

10/05/2022, 7:46 PM
I think
BASE_BRANCH
should be the branch you want to compare with
It can be main/master or maybe another branch, but not the same as your feature branch, otherwise they will have identical TF code, right?
Oh, one more thing, sorry, missed it
Yeah, sorry, I'm slow ๐Ÿ˜„
TF_ROOT: /tmp/base
- TF_ROOT is the main path of your code from feature branch, in the example it is
terraform
Let me explain the flow ๐Ÿ™‚
1. We clone your feature branch to
TF_ROOT
- it is our main place to run Infracost 2. We clone the master branch to
/tmp/base
- it is our baseline place 3. We run
infracost breakdown --path /tmp/base --out-file /tmp/infracost-base.json --format json
to generate the baseline Infracost data 4. Now we need to run
infracost diff --path TF_ROOT --compare-to /tmp/infracost-base.json
command (with other flags) - this way we compare baseline with your feature branch changes
So in your case you would change only one line - the one with
infracost breakdown
command
Let me push the fix to our example, hopefully it would make sense visually ๐Ÿ™‚
w

witty-book-85526

10/05/2022, 7:59 PM
Explaining the whole flow has been very useful! Appreciate it very much! ๐Ÿ™
b

busy-agent-35515

10/05/2022, 7:59 PM
Let's see if this works ๐Ÿ™‚
w

witty-book-85526

10/05/2022, 8:00 PM
testing it now!
b

busy-agent-35515

10/05/2022, 8:04 PM
Awesome! ๐Ÿฆœ Glad it worked! ๐Ÿ™‚
w

witty-book-85526

10/05/2022, 8:04 PM
Iโ€™ve spent quite a lot of time trying to figure it out and couldnโ€™t so that has been amazing! ๐Ÿ™‚
b

busy-agent-35515

10/05/2022, 8:04 PM
To be fair, it's our example had an error, so you were not wrong ๐Ÿ˜„
Sorry about that ๐Ÿ˜„
Thank you for your patience! We fixed the issue, hopefully other CircleCI users will have a smoother experience ๐Ÿ™‚
w

witty-book-85526

10/05/2022, 8:06 PM
All good! I guess you hear that a lot of time, but you guys are making a difference and this tools brings a lot of value! ๐Ÿ™‚ If there is anything I can help out with or contribute with my humble experience I will be more than happy to take on anything you got on your backlog!
b

busy-agent-35515

10/05/2022, 8:07 PM
Oh, thank you for the kind words! This means a lot to us infraheart
If you're willing to contribute, we have some issues that are good to get familiar with the Infracost code: https://github.com/infracost/infracost/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22.
Look around and if you find anything you like, just leave a comment that you'd like to work on it, and we'll gladly assign you ๐Ÿ™‚
Thank you for using Infracost and for your interest! ๐Ÿ™‚
Please forgive me for a shameless plug: we are also working on our SaaS with the goal to help your team and organization to tackle the cloud costs even better than CLI alone. https://dashboard.infracost.io/
w

witty-book-85526

10/06/2022, 11:38 AM
will sure check it out!
Hey @busy-agent-35515 have been using Infracost for sometime now and it works great. There is something that I noticed when you try to POST Infracost on Github - I get following
422 Validation Failed
Copy code
422 Validation Failed [{Resource:IssueComment Field:data Code:unprocessable Message:Body is too long (maximum is 65536 characters)}]
It seems the Terraform plan output is too big to display --- any idea how I can navigate this?
b

busy-agent-35515

11/02/2022, 1:05 PM
Hey! Glad to hear you have a nice experience! ๐Ÿ™‚ How do you do that? Using
infracost comment github
or doing a API request directly?
c

crooked-daybreak-55253

11/02/2022, 1:06 PM
And which infracost CLI version are you using? This was supposed to be fixed in v0.10.10
b

busy-agent-35515

11/02/2022, 1:07 PM
Yep, that was my next question. Thanks Tim! ๐Ÿ˜„
c

crooked-daybreak-55253

11/02/2022, 1:08 PM
Although looking at that PR we use
GitHubMaxMessageSize = 262144
hmmm
b

busy-agent-35515

11/02/2022, 1:09 PM
Yeah, it doesn't translate number of characters == the number of bytes
Ah, I left a comment about that, in the commit message though, not in code: https://github.com/orgs/community/discussions/27190#discussioncomment-3254953
w

witty-book-85526

11/02/2022, 1:12 PM
I am currently using
Infracost v0.10.13
for the Infracost CLI version
just to follow up on Vadimโ€™s comment - I am using
infracost comment github
as part of the Post Infracost comment block for CircleCI
b

busy-agent-35515

11/02/2022, 1:20 PM
Hm, interesting ๐Ÿค” As Tim mentioned, we fixed this issue and your CLI version should be okay. Let me try to replicate that
w

witty-book-85526

11/02/2022, 1:32 PM
could it be that I am still using this docker image
Copy code
docker:
      - image: infracost/infracost:ci-0.10
I mean, if a new img is released with an updated tag
b

busy-agent-35515

11/02/2022, 1:33 PM
Yeah, this image should have the latest 0.10.x version
Can you try to pull it just in case?
w

witty-book-85526

11/02/2022, 1:33 PM
sure, will try it now
w

witty-book-85526

11/02/2022, 1:37 PM
just pulled it
infracost/infracost:ci-0.10
Iโ€™ve tried with following versions of the docker img
Copy code
ci-0.10.13
ci-latest
and still getting the error
b

busy-agent-35515

11/02/2022, 1:45 PM
I'm still trying to go over the limit. You need a bunch of resources for that ๐Ÿ˜…
Okay, got the error now
Hm, okay, seems like the message size is still off, although the truncation works ๐Ÿค”
We're going to fix that and it should be out with the next release, but until then, you might be blocked ๐Ÿ˜ž
w

witty-book-85526

11/02/2022, 2:02 PM
all good, Vadim! That will not stop me from using Infracost as my team has seen immense value up until now โ€” will be around and wait for the new release ๐Ÿ™‚
any idea, when is it expected?
b

busy-agent-35515

11/02/2022, 2:03 PM
Not sure, maybe next week
By the way, have you tried our new GitHub App integration?
With it you don't need CircleCI integration, however, it's part of Infracost Cloud plan
w

witty-book-85526

11/02/2022, 2:06 PM
I havenโ€™t, I would have given it a try, itโ€™s not RFC compliant though that is why I am stuck with CircleCI --- I mean within the company that I work at
b

busy-agent-35515

11/02/2022, 2:07 PM
Oh, I see
w

witty-book-85526

11/15/2022, 1:12 PM
Hey @busy-agent-35515 itโ€™s me again, apologies for bothering! Here is what I am trying to do: I have a CircleCI workflow, which has a terraform orb and Iโ€™ve managed to generate the
Terraform plan JSON
and it produced the file
plan.json
which now I am trying to pass over to the Infracost job --- however I cannot attach it to workspace, because thatโ€™s already taken for the base branch โ€ฆ also I cannot pass it over to
TF_ROOT: terraform/plan.json
โ€™cause it wonโ€™t find it โ€ฆ
Copy code
- attach_workspace:
          at: /tmp
any ideas how can I pass the
plan.json
file that got generated to the infracost and get the diff and output the calculated cost?
b

busy-agent-35515

11/15/2022, 1:13 PM
Hey! No worries ๐Ÿ™‚
Can you show me the commands you're using?
w

witty-book-85526

11/15/2022, 1:14 PM
sure
b

busy-agent-35515

11/15/2022, 1:15 PM
Usually we do something like
infracost breakdown --path=path/to/code --out-file=/tmp/infracost.json --format=json
So it will save the Infracost JSON to /tmp/
Then we use
infracost diff --path=path/to/code --compare-to=/tmp/infracost.json
The
path/to/code
can be a Terraform JSON file too
w

witty-book-85526

11/15/2022, 1:17 PM
this is the
terraform
orb I am using
Copy code
orbs:
  terraform: i circleci/terraform@3.1.0
I have a CircleCI job called:
terraform-build-deploy:
I produce the plan.json with
Copy code
terraform show -json plan.out > plan.json
b

busy-agent-35515

11/15/2022, 1:17 PM
Ah, okay
w

witty-book-85526

11/15/2022, 1:18 PM
oh, I see the
path=path/to/code
is where I tries to use the path to where the
plan.json file
is sotred
but it wonโ€™t find it
b

busy-agent-35515

11/15/2022, 1:18 PM
It should be the explicit file in this case
w

witty-book-85526

11/15/2022, 1:18 PM
as it attaches the workspace to
/temp
b

busy-agent-35515

11/15/2022, 1:18 PM
path/plan.json
w

witty-book-85526

11/15/2022, 1:19 PM
explicit file, you mean the one generated by Infracost ,right?
b

busy-agent-35515

11/15/2022, 1:19 PM
Sorry, I'll rephrase: when using TF JSON files, the
--path=
should point to the file, not just the directory where it is located.
I'm not very familiar with CircleCI, is it possible to share the generated plan with other orbs?
My understanding is that the terraform orb generates the plan and stores it somewhere (like
/temp
), and Infracost job would be able to access this
/temp
so the command would look like
infracost breakdown --path=/temp/plan.json
w

witty-book-85526

11/15/2022, 1:24 PM
good question! not sure --- I am familar with using only
workspaces
to share date between jobs
so if I generate the
plan.json
from the terraform job in
/temp
Infracost sohuld be able to find it
right
theortically
b

busy-agent-35515

11/15/2022, 1:25 PM
From this https://circleci.com/docs/workspaces/ looks like that the Infracost jobs need to have something like
attach_workspace
to access the file
w

witty-book-85526

11/15/2022, 1:26 PM
will generate the file in
/temp
and then use
Copy code
- persist_to_workspace:
b

busy-agent-35515

11/15/2022, 1:27 PM
Yep, looks like it's required
w

witty-book-85526

11/15/2022, 1:27 PM
to store it in
/temp/plan.json
and see if Infracost will find it
give me a min and will try it
b

busy-agent-35515

11/15/2022, 1:27 PM
Hope this works ๐Ÿ™‚
w

witty-book-85526

11/15/2022, 1:46 PM
ah, the
persist_to_workspace
wonโ€™t do it โ€ฆ it does not recognizr that there is such a dir/
Copy code
The specified paths did not match any files in /tmp
Here is what I am thinking of --- having another
mid
job which carries the output from the
plan.json
and then attaching the workspace to
/tmp
where Infracost is looking for it
it ainโ€™t very smart, but cannot think of anything else ๐Ÿ˜ž
b

busy-agent-35515

11/15/2022, 1:47 PM
Sorry, I haven't used CircleCI, so can't be much help here ๐Ÿฅฒ
But first step is making it working, then refactor ๐Ÿ™‚
The example in the docs also shows that "executor" should have the working directory set as
Copy code
working_directory: /tmp
w

witty-book-85526

11/15/2022, 1:52 PM
oh, that is a good point! Will add the
working_directory
and try it out again! Please donโ€™t feel bad, I am very much
greateful
for you taking the time and looking into it as I was stuck and making no progress and ran out of ideas --- hope I am not bothering you and asking always stuff ๐Ÿ™‚
b

busy-agent-35515

11/15/2022, 1:52 PM
No worries! We want to make our users happy ๐Ÿ™‚
w

witty-book-85526

11/16/2022, 2:34 PM
@busy-agent-35515 managed to get it working with the
plan.json
that I created! ๐Ÿ™‚๐ŸŽ‰ Do you know by any chance if I can pass two
--path
within the
Copy code
infracost diff --path ...
as I have
multiple
---
plan.json
files that I want to pass on and this is a bit tricky โ€ฆ
Iโ€™ve tried adding second
---path
but it does not seem to be working โ€ฆ
b

busy-agent-35515

11/16/2022, 2:37 PM
Good job!
The diff command expects only one path and one compare-to. So if you have multiples, I guess you'll have to run the diff command N times
But then the comment/upload commands accept multiple paths to merge all diff info together
w

witty-book-85526

11/16/2022, 2:44 PM
Alright! Will give it a try to have the infracost diff replicated and then if the infracost comment supports multiple paths should do the trick! ๐Ÿ˜‰
The reason why I am doing
this
is because I am expecting several
tfstate
files which I want to merge into one eventually โ€ฆ
b

busy-agent-35515

11/16/2022, 2:45 PM
Are these different projects/repos?
w

witty-book-85526

11/16/2022, 2:46 PM
It is a
lerna
project with multiple
Cloudfront
distributuons and each Cf distro has
.tf
files associated with
.tf
modules + Lambda etcโ€ฆ
and once setup, it will produce diff
tfstate
files for diff scenarios
b

busy-agent-35515

11/16/2022, 2:47 PM
I wonder what is the baseline then?
w

witty-book-85526

11/16/2022, 2:48 PM
good question! I am avoiding it, I mean I donโ€™t setup
baseline
at all --- I am using the
Terraform plan JSON
and then run just
Copy code
infracost diff --path ./terraform/tfplan.json
and I am expecting multiple
tfplan.json
files which I somehow want to merge together into one file
b

busy-agent-35515

11/16/2022, 2:49 PM
So diff will always show $0 -> $XXX change, right?
w

witty-book-85526

11/16/2022, 2:49 PM
tbh donโ€™t know yet, I guess you are right! ๐Ÿ™‚
b

busy-agent-35515

11/16/2022, 2:50 PM
Okay, then in this case you don't need the diff command at all
You can just use
infracost breakdown
and provide several
--path
It will produce the infracost JSON that you can use with the comment command
The point of the
diff
command is to have a baseline (for example, the master/main branch), and compare PR branches with the baseline
If you don't have baseline (aka previous state), then the diff will produce the identical output as breakdown
Sorry I misunderstood your setup
w

witty-book-85526

11/16/2022, 2:53 PM
No, all good! This is a really good point though! ๐Ÿ™‚ Using the
infracost
breakdown allows to have multiple options --- will give it a try now
tried the
infracost
breakdown and it ran fine as you described it! However, now onto the
Infracost
comment I hit another roadblock ๐Ÿ˜ž
Copy code
invalid Infracost JSON file version. Supported versions are 0.2 โ‰ค x โ‰ค 0.2
message has been deleted
b

busy-agent-35515

11/16/2022, 3:34 PM
The comment command expects Infracost JSON file, the one you generate via
infracost breakdown --path path1 --path path2 --format json --out-file infracost.json
w

witty-book-85526

11/16/2022, 3:51 PM
Thanks @busy-agent-35515 this is how I fixed it
Copy code
infracost breakdown --path=./terraform \
                                  --format=json \
                                  --out-file=./terraform/tfplan.json
had to pass first the
./terraform
dir and then to
--out-file
I just pointed to the
tfplan.json
file which was expected
then within the
infracost
comment
Copy code
infracost comment github --path=./terraform/tfplan.json \
message has been deleted
b

busy-agent-35515

11/16/2022, 3:52 PM
Awesome ๐Ÿ™‚ We deliberately call the output file as
infracost.json
to avoid confusion with tfplan files that have completely different format ๐Ÿ™‚
w

witty-book-85526

11/16/2022, 3:53 PM
Appreciate the help and sticking through all the Qs! ๐Ÿ™
b

busy-agent-35515

11/16/2022, 3:53 PM
Of course ๐Ÿ™‚
@witty-book-85526 Hello! Sorry it's been a month, but we finally released a new CLI version that fixes the long GitHub comments. Hope it'll still be useful for you.
w

witty-book-85526

12/06/2022, 9:06 PM
Hey @busy-agent-35515 thatโ€™s brilliant! ๐ŸŽ‰ ๐Ÿ™ ๐Ÿ’ฏ It will surely come in very handy!
3 Views