BTW both infracost usage files have different sett...
# help
g
BTW both infracost usage files have different settings for each environment for testing and demo purposes.
m
hey @glamorous-cat-93012, could you post your full pipeline yaml/configuration so that I can understand all the steps you’re using
g
--- steps: - bash: | ssh-agent -a $(SSH_AUTH_SOCK) mkdir -p ~/.ssh echo "$(echo $GIT_SSH_KEY_BASE_64 | base64 -d)" | tr -d '\r' | ssh-add - ssh-keyscan ssh.dev.azure.com >> ~/.ssh/known_hosts displayName: Add GIT_SSH_KEY # condition: and(succeeded(), eq(variables['Build.Reason'], 'IndividualCI')) condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) env: GIT_SSH_KEY_BASE_64: $(gitSshKeyBase64) # Install the Infracost CLI, see https://github.com/infracost/infracost-azure-devops#infracostsetup - task: InfracostSetup@1 displayName: Setup Infracost # condition: and(succeeded(), eq(variables['Build.Reason'], 'IndividualCI')) condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) inputs: apiKey: $(infracostApiKey) # Clone the base branch of the pull request (e.g. main/master) into a temp directory. - bash: | branch=$(System.PullRequest.TargetBranch) branch=${branch#refs/heads/} echo $branch mkdir $(Pipeline.Workspace)/s/$(Build.Repository.Name)/base git clone git@ssh.dev.azure.com:v3/MetroBank/Metro%20Cloud%20Platform/$(Build.Repository.Name) \ --branch=${branch} --single-branch "$(Pipeline.Workspace)/s/$(Build.Repository.Name)/base" displayName: Checkout base branch condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) # condition: and(succeeded(), or( eq(variables['Build.Reason'], 'IndividualCI'), eq(variables['Build.Reason'], 'Manual')) # Generate an Infracost cost estimate baseline from the comparison branch, so that Infracost can compare the cost difference. - bash: | pwd ls -ltr mkdir $(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports infracost breakdown --path=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/base/ \ --format=json \ --out-file=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost-base.json displayName: Generate Infracost cost estimate baseline workingDirectory: $(Pipeline.Workspace)/s/$(Build.Repository.Name)/base condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) # condition: and(succeeded(), eq(variables['Build.Reason'], 'IndividualCI')) # Generate an Infracost diff and save it to a JSON file. - bash: | pwd ls -ltr infracost diff --path=$(Pipeline.Workspace)/s/$(Build.Repository.Name) \ --format=json \ --compare-to=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost-base.json \ --out-file=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost.json displayName: Generate Infracost diff workingDirectory: $(Pipeline.Workspace)/s/$(Build.Repository.Name) condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) # condition: and(succeeded(), or( eq(variables['Build.Reason'], 'IndividualCI'), eq(variables['Build.Reason'], 'Manual')) # - publish: '$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports' # displayName: 'Publish reports' # artifact: drop # Posts a comment to the PR using the 'update' behavior. # This creates a single comment and updates it. The "quietest" option. # The other valid behaviors are: # delete-and-new - Delete previous comments and create a new one. # new - Create a new cost estimate comment on every push. # See https://www.infracost.io/docs/features/cli_commands/#comment-on-pull-requests for other options. - bash: | infracost comment azure-repos --path=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost.json \ --azure-access-token=$(System.AccessToken) \ --pull-request=$(System.PullRequest.PullRequestId) \ --repo-url=$(Build.Repository.Uri) \ --behavior=new displayName: Post Infracost comment condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) # condition: and(succeeded(), or( eq(variables['Build.Reason'], 'IndividualCI'), eq(variables['Build.Reason'], 'Manual'))
m
Ok two follow-up questions: 1. Do you have this
infracost.yml
file checked into the main branch yet (or is it being introduced in this PR) 2. What is the output you’re getting on the PR, what projects is the
table
showing?
g
message has been deleted
We did an ls -ltra on the directory to prove this ourselves and also attached the Pr comments.
m
so the problem is this:
in your pipeline you specify
diff
&
breakdown
with
--path
rather than
--config-file
g
Sorry I posted the wring branch
2 secs
--- steps: - bash: | ssh-agent -a $(SSH_AUTH_SOCK) mkdir -p ~/.ssh echo "$(echo $GIT_SSH_KEY_BASE_64 | base64 -d)" | tr -d '\r' | ssh-add - ssh-keyscan ssh.dev.azure.com >> ~/.ssh/known_hosts displayName: Add GIT_SSH_KEY # condition: and(succeeded(), eq(variables['Build.Reason'], 'IndividualCI')) condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) env: GIT_SSH_KEY_BASE_64: $(gitSshKeyBase64) # Install the Infracost CLI, see https://github.com/infracost/infracost-azure-devops#infracostsetup - task: InfracostSetup@1 displayName: Setup Infracost # condition: and(succeeded(), eq(variables['Build.Reason'], 'IndividualCI')) condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) inputs: apiKey: $(infracostApiKey) # Clone the base branch of the pull request (e.g. main/master) into a temp directory. - bash: | branch=$(System.PullRequest.TargetBranch) branch=${branch#refs/heads/} echo $branch mkdir $(Pipeline.Workspace)/s/$(Build.Repository.Name)/base git clone git@ssh.dev.azure.com:v3/MetroBank/Metro%20Cloud%20Platform/$(Build.Repository.Name) \ --branch=${branch} --single-branch "$(Pipeline.Workspace)/s/$(Build.Repository.Name)/base" displayName: Checkout base branch condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) # condition: and(succeeded(), or( eq(variables['Build.Reason'], 'IndividualCI'), eq(variables['Build.Reason'], 'Manual')) # Generate an Infracost cost estimate baseline from the comparison branch, so that Infracost can compare the cost difference. - bash: | pwd ls -ltr mkdir $(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports infracost breakdown --config-file $(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost.yml \ --format=json \ --out-file=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost-base.json displayName: Generate Infracost cost estimate baseline workingDirectory: $(Pipeline.Workspace)/s/$(Build.Repository.Name)/base condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) # condition: and(succeeded(), eq(variables['Build.Reason'], 'IndividualCI')) # Generate an Infracost diff and save it to a JSON file. - bash: | pwd ls -ltr infracost diff --config-file $(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost.yml \ --format=json \ --compare-to=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost-base.json \ --out-file=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost.json displayName: Generate Infracost diff workingDirectory: $(Pipeline.Workspace)/s/$(Build.Repository.Name) condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) # condition: and(succeeded(), or( eq(variables['Build.Reason'], 'IndividualCI'), eq(variables['Build.Reason'], 'Manual')) - publish: '$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports' displayName: 'Publish reports' artifact: drop # Posts a comment to the PR using the 'update' behavior. # This creates a single comment and updates it. The "quietest" option. # The other valid behaviors are: # delete-and-new - Delete previous comments and create a new one. # new - Create a new cost estimate comment on every push. # See https://www.infracost.io/docs/features/cli_commands/#comment-on-pull-requests for other options. - bash: | infracost comment azure-repos --path=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost.json \ --azure-access-token=$(System.AccessToken) \ --pull-request=$(System.PullRequest.PullRequestId) \ --repo-url=$(Build.Repository.Uri) \ --behavior=update displayName: Post Infracost comment condition: and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) # condition: and(succeeded(), or( eq(variables['Build.Reason'], 'IndividualCI'), eq(variables['Build.Reason'], 'Manual'))
m
hmm this does look strange
i’m wondering if it’s getting an old artefact
what happens when you remove
Copy code
- publish: '$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports'
        displayName: 'Publish reports'
        artifact: drop
step
g
The publish step was just added at the end for us to be able to download the results, all it does is take the json files and publish them to a drop location so we can see them in the azdo dashboard.
Same results with or without this task
Another thing we noticed with InfraCost --path setting that does not make sense with config-file
m
could you send me the
$(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost-reports/infracost.json
artefact in a private message
g
So when we use --path in the command like: infracost breakdown --path=$(Pipeline.Workspace)/s/$(Build.Repository.Name)/base/ \ infracost diff --path=$(Pipeline.Workspace)/s/$(Build.Repository.Name) \ You notice on break down we point to /base which in the temp main branch, then on diff we point to the current branch to we can compare both, right??
m
yup
g
When you change to --config-file, there is no why to have two different paths for each branch so basically scans the same branch twice? --config-file $(Pipeline.Workspace)/s/$(Build.Repository.Name)/infracost.yml \
as the config file just has a field for --path and can't be changed between calls
m
ah I think I see what’s going wrong with your yaml
g
The way we got around this was to switch working directories on the bash task, so we can use the config-file. workingDirectory: $(Pipeline.Workspace)/s/$(Build.Repository.Name)/base workingDirectory: $(Pipeline.Workspace)/s/$(Build.Repository.Name)
m
yeah, that’s the correct way to do it
g
What have you spotted ?
m
I thought I had but no joy sadly
can you send me that infracost.json
g
sure
m
hmm @glamorous-cat-93012 this does look like a bug our side. It seems that we’re not populating the
past_resources
properly in for this case. I’ve managed to replicate your pr output locally - give me some time to figure out whats going on
g
Cool, no worries..
Also a great idea to update your doc to use workingDirectory when using --config-file as this caught us out for a while. workingDirectory: $(Pipeline.Workspace)/s/$(Build.Repository.Name)/base workingDirectory: $(Pipeline.Workspace)/s/$(Build.Repository.Name) 🙂
m
hey @glamorous-cat-93012 so I think i’ve found the issue (which turns out not to be a bug but more a UX issue). With that PR are you introducing any cost changes?
g
In the usage file we have different settings for each env and we also updated the modules to use different ec2 instances so these should show the break down of nonp and prod?
and we do not see the different envs breakdown in the dashboard
message has been deleted
Strange, it's like it is not picking up the main resources at all from main.tf
only finding resources in the usage-file.yml
m
It looks like it's not being able I pick up the usage file path
This fails silently so you wouldn't see this
Can I check that the path structure you've got is correct - where are you usage files in relation to the project
g
message has been deleted
this is the ls from the root, they are stored in config/env/...
m
Ok I've got to go offline for about an hour and ill pick this up when I'm back.
g
No worries, thanks for looking into this
coffee time
m
hey @glamorous-cat-93012 I’m back
give me a shout when your about and if you’re free to jump on a zoom to help debug this and get you to success
g
Free in an hour, sneaked out for a cheaky Friday lunch beer with the laptop.... shhhh 🙂
m
ha
I’ve got meetings from 2:30pm (BST) to 4pm. So ill be free after that
g
How's you cal for Monday morning ?
m
very free, ping me an invite to hugo@infracost.io and let’s get to the bottom of it
first meeting is at 2:30pm (BST)
g
Hows 10:30 GMT for you will.pereira@metrobank.plc.uk gerard.kavanagh@metrobank.plc.uk Could you set it up as we don't have a corp zoom for this? Building a nice new templated terraform model and see InrfaCost fitting in.
m
works fine with me - I’ll ping you now
Sent
g
see you Monday. Thanks `Hugo... great supprt.