https://infracost.io logo
#general
Title
# general
k

kind-angle-62935

08/05/2022, 7:52 AM
Hello, I have a situation where I'm doing a breakdown on my default branch, then a breakdown on my feature branch, then I'm doing a diff, and I'm getting the error
Invalid --compare-to Infracost JSON, found duplicate project name ...
. What exactly does this mean?
I'm getting that error on the diff
l

little-author-61621

08/05/2022, 8:02 AM
@kind-angle-62935 can you share the commands you are running for breakdown and diff? I'll be able to have more of a look in about an hour.
k

kind-angle-62935

08/05/2022, 8:25 AM
@little-author-61621 Here is where I create the breakdown based on the base branch: https://github.com/terrateamio/action/blob/v1/terrat_runner/workflow_step_infracost_setup_base.py and here is where I do the breakdown on the feature branch and the diff: https://github.com/terrateamio/action/blob/v1/terrat_runner/workflow_step_infracost_setup.py
l

little-author-61621

08/05/2022, 8:56 AM
So this error appears if you have multiple projects in your Infracost JSON that have the same project name. Since Infracost is unable to distinguish between them then when it is doing the diff it can’t tell which one to diff against. So I’m wondering what the config file looks like or the Infracost output JSON that means that there’s two projects with the same name.
k

kind-angle-62935

08/05/2022, 11:17 AM
@little-author-61621 What does "two projects with the same name" mean exactly? Is only the
diff
checking this? Should
breakdown
fail in this case?
l

little-author-61621

08/05/2022, 11:20 AM
It means two projects in the
prev_infracost
JSON file have the same
name
key. The diff is checking this because it assumes unique project names otherwise it doesn’t know which one to diff against. Are you setting project names in the config file when you create it?
k

kind-angle-62935

08/05/2022, 11:20 AM
I am not setting project names
l

little-author-61621

08/05/2022, 11:21 AM
ok, do any of the
dirspaces
have the same path and workspace?
k

kind-angle-62935

08/05/2022, 11:21 AM
I will double check but no, they should not
That tuple should be unique
l

little-author-61621

08/05/2022, 11:23 AM
the name it is showing in the
here
found duplicate project name ...
is the one it’s finding twice.
k

kind-angle-62935

08/05/2022, 11:24 AM
Yes, I'm trying to setup a test for this. But since I'm not specifying a project name, how is infracost assigning a project name that would be duplicated?
l

little-author-61621

08/05/2022, 11:25 AM
Infracost defaults the name based on the path and the workspace. So the name it’s generating should be
<path> (Workspace: <workspace>)
k

kind-angle-62935

08/05/2022, 11:26 AM
Does it not include that if the workspace is
default
?
Does it not include the (workspace) part I mean
l

little-author-61621

08/05/2022, 11:26 AM
correct it just uses the path for that case
k

kind-angle-62935

08/05/2022, 11:26 AM
Thank you. I'll try some things
l

little-author-61621

08/05/2022, 11:27 AM
You can set your own project name by specifying a
name
key for each project in the config file if you want as well.
k

kind-angle-62935

08/05/2022, 12:34 PM
@little-author-61621 How does infracost handle a configuration like:
Copy code
projects:
- path: foo
  terraform_workspace: default
- path: foo/bar
  terraform_workspace: default
l

little-author-61621

08/05/2022, 12:36 PM
Ah that might be causing the problem. If
foo
is a Terraform mono repo then it will create a new project for each project underneath it, and so there might be two created for
foo/bar
k

kind-angle-62935

08/05/2022, 12:37 PM
I think that is what I'm seeing
Is there any way to tell it to not be recursive?
Could I set an exclude path for anything underneath?
l

little-author-61621

08/05/2022, 12:43 PM
There’s
exclude_paths
(https://www.infracost.io/docs/features/config_file/#usage) - does that help?
k

kind-angle-62935

08/05/2022, 12:44 PM
A non-recursive flag would be preferred but I might be able to work with exclude_paths
Would this work?
Copy code
projects:
- path: foo
  terraform_workspace: default
  exclude_paths: ['foo/*/*']
- path: foo/bar
  terraform_workspace: default
  exclude_paths: ['foo/bar/*/*']
l

little-author-61621

08/05/2022, 12:46 PM
It could work - it’s not something I’ve used much, so I can’t say for sure. My bigger question is why do you need to specify all the sub-paths separately?
k

kind-angle-62935

08/05/2022, 12:47 PM
Because it's a monorepo we have precise control over what we consider a terraform dir and what we don't
l

little-author-61621

08/05/2022, 12:48 PM
Okay, so if you pass the parent directory to Infracost it doesn’t properly detect the terraform dirs?
k

kind-angle-62935

08/05/2022, 12:48 PM
We have our own tooling which detects what the terraform dirs are and which ones we want to run various operations on
Hrm, excluse_paths does not seem to work
exclude_paths
@little-author-61621 If i were to manipulate the output JSON to remove dup projects, would
diff
work fine? Or does it depend on the
summary
information in a
breakdown
to be correct?
l

little-author-61621

08/05/2022, 1:48 PM
yeah that could work. Another option could be to set the
name
manually in the config file.
k

kind-angle-62935

08/05/2022, 1:49 PM
Well a question:
breakdown
is producing duplicate entries in the output, but does that mean the summary contains the duplicate values?
Or rather does the summary have the a duplicate sum?
@little-author-61621 I created an Issue as well: https://github.com/infracost/infracost/issues/1952
l

little-author-61621

08/05/2022, 2:58 PM
Sorry for the slow response, thanks for creating an issue. The summaries will probably still contain duplicates if you modify the JSON directly yeah. Just trying to understand a little more. Are there
.tf
files directly in both
/foo
and
/foo/bar
? And are there directories under
/foo
you aren’t wanting to generate Infracost results for?
k

kind-angle-62935

08/05/2022, 2:59 PM
Yes, and possibly yes.
l

little-author-61621

08/05/2022, 3:04 PM
Maybe try:
Copy code
exclude_paths:
      - ./*
This seems to work for me. The exclude_paths is relative to the path parameter.
k

kind-angle-62935

08/05/2022, 3:07 PM
Ah ok I'll try
@little-author-61621 Sorry for the long delay (and I know it's weekend, so no rush getting back to me), but
exclude_paths
did not work either.
Copy code
projects:
- exclude_paths:
  - ./*
  path: non-prod
  terraform_workspace: default
- exclude_paths:
  - ./*
  path: non-prod/us-east-1
  terraform_workspace: default
- exclude_paths:
  - ./*
  path: non-prod/us-east-1/qa
  terraform_workspace: default
- exclude_paths:
  - ./*
  path: non-prod/us-east-1/qa/mysql
  terraform_workspace: default
- exclude_paths:
  - ./*
  path: non-prod/us-east-1/qa/webserver-cluster
  terraform_workspace: default
- exclude_paths:
  - ./*
  path: non-prod/us-east-1/stage
  terraform_workspace: default
- exclude_paths:
  - ./*
  path: non-prod/us-east-1/stage/mysql
  terraform_workspace: default
- exclude_paths:
  - ./*
  path: non-prod/us-east-1/stage/webserver-cluster
  terraform_workspace: default
version: '0.1'
l

little-author-61621

08/06/2022, 9:09 PM
Okay, this is odd, here’s some ideas to debug it further. 1. It would be good to see the output of
tree
on this directory. 2. What is the output of running
infracost breakdown --config-file=infracost.yml --format=json | jq -r '.projects[].name'
3. What is the output of
INFRACOST_PARALLELISM=1 infracost breakdown --config-file=infracost.yml --format=json --log-level=debug 2>&1 | grep 'walking path\|skipping directory'
? 4. If you run with just the first project, i.e.:
Copy code
projects:
  - exclude_paths:
    - ./*
    path: non-prod
    terraform_workspace: default
What is the output of the above commands from 2 & 3? Random idea that might work for just now. Is there any way of flattening the directories yourself before running Infracost? i.e. having
non-prod
,
non-prod-us-east-1
,
non-prod-us-east-1-qa
. I know this isn’t ideal, just an idea for a workaround.
k

kind-angle-62935

08/07/2022, 5:57 AM
@little-author-61621 I edited the JSON file and removed all duplicate path/workspaces, and that seems to be working as expected. It looks like diff recalcualtes the summary values on its own