Title
#help
g

Gerard Kavanagh

10/14/2022, 9:02 AM
BTW both infracost usage files have different settings for each environment for testing and demo purposes.
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 9:04 AM
hey @Gerard Kavanagh, could you post your full pipeline yaml/configuration so that I can understand all the steps you’re using
g

Gerard Kavanagh

10/14/2022, 9:22 AM
--- steps: - bash: | ssh-agent -a $(SSH_AUTH_SOCK) mkdir -p ~/.ssh echo "$(echo $GIT_SSH_KEY_BASE_64 | base64 -d)" | tr -d '\r' | ssh-add - ssh-keyscan ssh.dev.azure.com
~/.ssh/known_hosts displayName: Add GIT_SSH_KEY # condition: and(succeeded(), eq(variables['Build.Reason'], 'IndividualCI')) condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) env: GIT_SSH_KEY_BASE_64: $(gitSshKeyBase64)

Install the Infracost CLI, see

https://github.com/infracost/infracost-azure-devops#infracostsetup - task: InfracostSetup@1 displayName: Setup Infracost # condition: and(succeeded(), eq(variables['Build.Reason'], 'IndividualCI')) condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) inputs: apiKey: $(infracostApiKey)

Clone the base branch of the pull request (e.g. main/master) into a temp directory.

- bash: | branch=$(System.PullRequest.TargetBranch) branch=${branch#refs/heads/} echo $branch mkdir $(Pipeline.Workspace)/s/$(Build.Repository.Name)/base git clone git@ssh.dev.azure.com:v3/MetroBank/Metro%20Cloud%20Platform/$(Build.Repository.Name) \ --branch=${branch} --single-branch "$(Pipeline.Workspace)/s/$(Build.Repository.Name)/base" displayName: Checkout base branch condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) # condition: and(succeeded(), or( eq(variables['Build.Reason'], 'IndividualCI'), eq(variables['Build.Reason'], 'Manual'))

Generate an Infracost cost estimate baseline from the comparison branch, so that Infracost can compare the cost difference.

- bash: | pwd ls -ltr mkdir $(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports infracost breakdown --path=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/base/ \ --format=json \ --out-file=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost-base.json displayName: Generate Infracost cost estimate baseline workingDirectory: $(Pipeline.Workspace)/s/$(Build.Repository.Name)/base condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) # condition: and(succeeded(), eq(variables['Build.Reason'], 'IndividualCI'))

Generate an Infracost diff and save it to a JSON file.

- bash: | pwd ls -ltr infracost diff --path=$(Pipeline.Workspace)/s/$(Build.Repository.Name) \ --format=json \ --compare-to=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost-base.json \ --out-file=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost.json displayName: Generate Infracost diff workingDirectory: $(Pipeline.Workspace)/s/$(Build.Repository.Name) condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) # condition: and(succeeded(), or( eq(variables['Build.Reason'], 'IndividualCI'), eq(variables['Build.Reason'], 'Manual'))

- publish: '$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports'

displayName: 'Publish reports'

artifact: drop

Posts a comment to the PR using the 'update' behavior.

This creates a single comment and updates it. The "quietest" option.

The other valid behaviors are:

delete-and-new - Delete previous comments and create a new one.

new - Create a new cost estimate comment on every push.

See

https://www.infracost.io/docs/features/cli_commands/#comment-on-pull-requests for other options. - bash: | infracost comment azure-repos --path=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost.json \ --azure-access-token=$(System.AccessToken) \ --pull-request=$(System.PullRequest.PullRequestId) \ --repo-url=$(Build.Repository.Uri) \ --behavior=new displayName: Post Infracost comment condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) # condition: and(succeeded(), or( eq(variables['Build.Reason'], 'IndividualCI'), eq(variables['Build.Reason'], 'Manual'))
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 9:26 AM
Ok two follow-up questions: 1. Do you have this
infracost.yml
file checked into the main branch yet (or is it being introduced in this PR) 2. What is the output you’re getting on the PR, what projects is the
table
showing?
g

Gerard Kavanagh

10/14/2022, 9:33 AM
9:33 AM
We did an ls -ltra on the directory to prove this ourselves and also attached the Pr comments.
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 9:34 AM
so the problem is this:
9:34 AM
in your pipeline you specify
diff
&
breakdown
with
--path
rather than
--config-file
g

Gerard Kavanagh

10/14/2022, 9:35 AM
Sorry I posted the wring branch
9:35 AM
2 secs
9:38 AM
--- steps: - bash: | ssh-agent -a $(SSH_AUTH_SOCK) mkdir -p ~/.ssh echo "$(echo $GIT_SSH_KEY_BASE_64 | base64 -d)" | tr -d '\r' | ssh-add - ssh-keyscan ssh.dev.azure.com
~/.ssh/known_hosts displayName: Add GIT_SSH_KEY # condition: and(succeeded(), eq(variables['Build.Reason'], 'IndividualCI')) condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) env: GIT_SSH_KEY_BASE_64: $(gitSshKeyBase64)

Install the Infracost CLI, see

https://github.com/infracost/infracost-azure-devops#infracostsetup - task: InfracostSetup@1 displayName: Setup Infracost # condition: and(succeeded(), eq(variables['Build.Reason'], 'IndividualCI')) condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) inputs: apiKey: $(infracostApiKey)

Clone the base branch of the pull request (e.g. main/master) into a temp directory.

- bash: | branch=$(System.PullRequest.TargetBranch) branch=${branch#refs/heads/} echo $branch mkdir $(Pipeline.Workspace)/s/$(Build.Repository.Name)/base git clone git@ssh.dev.azure.com:v3/MetroBank/Metro%20Cloud%20Platform/$(Build.Repository.Name) \ --branch=${branch} --single-branch "$(Pipeline.Workspace)/s/$(Build.Repository.Name)/base" displayName: Checkout base branch condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) # condition: and(succeeded(), or( eq(variables['Build.Reason'], 'IndividualCI'), eq(variables['Build.Reason'], 'Manual'))

Generate an Infracost cost estimate baseline from the comparison branch, so that Infracost can compare the cost difference.

- bash: | pwd ls -ltr mkdir $(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports infracost breakdown --config-file $(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost.yml \ --format=json \ --out-file=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost-base.json displayName: Generate Infracost cost estimate baseline workingDirectory: $(Pipeline.Workspace)/s/$(Build.Repository.Name)/base condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) # condition: and(succeeded(), eq(variables['Build.Reason'], 'IndividualCI'))

Generate an Infracost diff and save it to a JSON file.

- bash: | pwd ls -ltr infracost diff --config-file $(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost.yml \ --format=json \ --compare-to=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost-base.json \ --out-file=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost.json displayName: Generate Infracost diff workingDirectory: $(Pipeline.Workspace)/s/$(Build.Repository.Name) condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) # condition: and(succeeded(), or( eq(variables['Build.Reason'], 'IndividualCI'), eq(variables['Build.Reason'], 'Manual')) - publish: '$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports' displayName: 'Publish reports' artifact: drop

Posts a comment to the PR using the 'update' behavior.

This creates a single comment and updates it. The "quietest" option.

The other valid behaviors are:

delete-and-new - Delete previous comments and create a new one.

new - Create a new cost estimate comment on every push.

See

https://www.infracost.io/docs/features/cli_commands/#comment-on-pull-requests for other options. - bash: | infracost comment azure-repos --path=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost.json \ --azure-access-token=$(System.AccessToken) \ --pull-request=$(System.PullRequest.PullRequestId) \ --repo-url=$(Build.Repository.Uri) \ --behavior=update displayName: Post Infracost comment condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) # condition: and(succeeded(), or( eq(variables['Build.Reason'], 'IndividualCI'), eq(variables['Build.Reason'], 'Manual'))
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 9:42 AM
hmm this does look strange
9:42 AM
i’m wondering if it’s getting an old artefact
9:42 AM
what happens when you remove
- publish: '$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports'
        displayName: 'Publish reports'
        artifact: drop
step
g

Gerard Kavanagh

10/14/2022, 9:52 AM
The publish step was just added at the end for us to be able to download the results, all it does is take the json files and publish them to a drop location so we can see them in the azdo dashboard.
9:52 AM
Same results with or without this task
9:53 AM
Another thing we noticed with InfraCost --path setting that does not make sense with config-file
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 9:53 AM
could you send me the
$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost.json
artefact in a private message
g

Gerard Kavanagh

10/14/2022, 9:55 AM
So when we use --path in the command like: infracost breakdown --path=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/base/ \ infracost diff --path=$(Pipeline.Workspace)/s/$(Build.Repository.Name) \ You notice on break down we point to /base which in the temp main branch, then on diff we point to the current branch to we can compare both, right??
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 9:56 AM
yup
g

Gerard Kavanagh

10/14/2022, 9:56 AM
When you change to --config-file, there is no why to have two different paths for each branch so basically scans the same branch twice? --config-file $(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost.yml \
9:56 AM
as the config file just has a field for --path and can't be changed between calls
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 9:57 AM
ah I think I see what’s going wrong with your yaml
g

Gerard Kavanagh

10/14/2022, 9:58 AM
The way we got around this was to switch working directories on the bash task, so we can use the config-file. workingDirectory: $(Pipeline.Workspace)/s/$(Build.Repository.Name)/base workingDirectory: $(Pipeline.Workspace)/s/$(Build.Repository.Name)
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 9:58 AM
yeah, that’s the correct way to do it
g

Gerard Kavanagh

10/14/2022, 9:59 AM
What have you spotted ?
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 10:00 AM
I thought I had but no joy sadly
10:00 AM
can you send me that infracost.json
g

Gerard Kavanagh

10/14/2022, 10:00 AM
sure
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 10:16 AM
hmm @Gerard Kavanagh this does look like a bug our side. It seems that we’re not populating the
past_resources
properly in for this case. I’ve managed to replicate your pr output locally - give me some time to figure out whats going on
g

Gerard Kavanagh

10/14/2022, 10:19 AM
Cool, no worries..
10:20 AM
Also a great idea to update your doc to use workingDirectory when using --config-file as this caught us out for a while. workingDirectory: $(Pipeline.Workspace)/s/$(Build.Repository.Name)/base workingDirectory: $(Pipeline.Workspace)/s/$(Build.Repository.Name) 🙂
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 10:28 AM
hey @Gerard Kavanagh so I think i’ve found the issue (which turns out not to be a bug but more a UX issue). With that PR are you introducing any cost changes?
g

Gerard Kavanagh

10/14/2022, 10:37 AM
In the usage file we have different settings for each env and we also updated the modules to use different ec2 instances so these should show the break down of nonp and prod?
10:39 AM
and we do not see the different envs breakdown in the dashboard
10:40 AM
10:41 AM
Strange, it's like it is not picking up the main resources at all from main.tf
10:41 AM
only finding resources in the usage-file.yml
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 10:44 AM
It looks like it's not being able I pick up the usage file path
10:44 AM
This fails silently so you wouldn't see this
10:45 AM
Can I check that the path structure you've got is correct - where are you usage files in relation to the project
g

Gerard Kavanagh

10/14/2022, 10:45 AM
10:46 AM
this is the ls from the root, they are stored in config/env/...
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 10:47 AM
Ok I've got to go offline for about an hour and ill pick this up when I'm back.
g

Gerard Kavanagh

10/14/2022, 10:47 AM
No worries, thanks for looking into this
10:47 AM
coffee time
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 12:11 PM
hey @Gerard Kavanagh I’m back
12:12 PM
give me a shout when your about and if you’re free to jump on a zoom to help debug this and get you to success
g

Gerard Kavanagh

10/14/2022, 1:00 PM
Free in an hour, sneaked out for a cheaky Friday lunch beer with the laptop.... shhhh 🙂
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 1:01 PM
ha
1:01 PM
I’ve got meetings from 2:30pm (BST) to 4pm. So ill be free after that
g

Gerard Kavanagh

10/14/2022, 2:55 PM
How's you cal for Monday morning ?
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 2:56 PM
very free, ping me an invite to hugo@infracost.io and let’s get to the bottom of it
2:56 PM
first meeting is at 2:30pm (BST)
g

Gerard Kavanagh

10/14/2022, 3:11 PM
Hows 10:30 GMT for youwill.pereira@metrobank.plc.uk gerard.kavanagh@metrobank.plc.uk Could you set it up as we don't have a corp zoom for this? Building a nice new templated terraform model and see InrfaCost fitting in.
Hugo (Infracost)

Hugo (Infracost)

10/14/2022, 3:12 PM
works fine with me - I’ll ping you now
3:15 PM
Sent
g

Gerard Kavanagh

10/14/2022, 3:35 PM
see you Monday. Thanks `Hugo... great supprt.