Title
#general
s

Sylvain Witmeyer

08/15/2022, 9:45 PM
I run infracost in a docker from my laptop in a terragrunt project and it works perfectly. I’m using the env INFRACOST_API_KEY. But when switching to the github actions I faced a bad
Error: Invalid API key
Please check your /home/runner/.config/infracost/credentials.yml file or INFRACOST_API_KEY environment variable.
I replace the API key, thinking I pasted a wrong key or added a whitespace but when I try to debug to see the key
k=$(infracost configure get api_key)
  echo ${k: -2}
  cat /home/runner/.config/infracost/credentials.yml

du
version: "0.1"
api_key: ***

and du are the correct 2 last chars of my key.
Any idea what could be wrong ?
Ali (Infracost)

Ali (Infracost)

08/15/2022, 10:14 PM
In GitHub Actions, you can just set
INFRACOST_API_KEY
as a repo secret with the key, are you using https://github.com/infracost/actions/#quick-start ? that mentions this in step 2 and you can see how it’s used by the “Setup Infracost” action.
s

Sylvain Witmeyer

08/15/2022, 10:32 PM
that’s what I did. And the env is used during the setup
- name: Setup Infracost
        uses: infracost/actions/setup@v2
        # See <https://github.com/infracost/actions/tree/master/setup> for other inputs
        # If you can't use this action, see Docker images in <https://infracost.io/cicd>
        with:
          api-key: ${{ secrets.INFRACOST_API_KEY }}
But this var is just used to configure the cli instead of being used as an env variable
/home/runner/work/_temp/25a24987-d34e-445e-88a8-e429f84786fc/infracost configure set api_key ***
So the setup looks good since my
/home/runner/.config/infracost/credentials.yml
contains the correct key and the version.
Ali (Infracost)

Ali (Infracost)

08/15/2022, 10:35 PM
Does using the env var have the same error with your github actions? I ask as I’m not sure if github actions persist the file between the steps, that’s why in all CI/CD systems we ask users to set the
INFRACOST_API_KEY
env variable (i.e. you don’t need the credentials.yml file at all).
s

Sylvain Witmeyer

08/15/2022, 10:45 PM
I’m gonna try but I was thinking that credentials.yml has precedence over the env var. My cat credentials comes from another step so the file seems persisted. But I agree usig the env would be way easier
10:47 PM
Ali (Infracost)

Ali (Infracost)

08/15/2022, 10:55 PM
yep, in case you need to verify or regenerate the API key, you can do it from the Org Settings page in https://dashboard.infracost.io btw, I created this issue as our error is confusing users in CI/CD.
s

Sylvain Witmeyer

08/15/2022, 10:58 PM
where do you read the api key first when you have a credentials.yml and also an
INFRACOST_API_KEY
?
Ali (Infracost)

Ali (Infracost)

08/15/2022, 10:59 PM
INFRACOST_API_KEY
I’m just tweaking this docs page to clarify that 🙂
s

Sylvain Witmeyer

08/15/2022, 11:01 PM
it works with the env . So you could update the example and remove the set api_key from the action. Also api_key become a optional param now
INFRACOST_ENABLE_CLOUD: true
INFRACOST_API_KEY: ${{ secrets.INFRACOST_API_KEY }}
Ali (Infracost)

Ali (Infracost)

08/15/2022, 11:05 PM
Great! Sorry about the confusion - we’ll make the error clear in that GH issue. I vaguely remember we discussed setting a top-level env var (which is what we do for GitLab and others) vs using a param in the setup action for GitHub and picked the latter, I can’t remember the exact reason just now though. I’ll check with the team…
s

Sylvain Witmeyer

08/15/2022, 11:06 PM
Thanks for your help, let me know if you want me to fix the GH Actions.
Ali (Infracost)

Ali (Infracost)

08/15/2022, 11:07 PM
thanks! will keep you in the loop and reply to this thread once I find out…
s

Sylvain Witmeyer

08/15/2022, 11:08 PM
I don’t know if I’m the first but the GH actions example seems broken. Even if I don’t understand why the credentials weren’t used... Thanks again @Ali (Infracost)
Ali (Infracost)

Ali (Infracost)

08/15/2022, 11:09 PM
What other issues did you run into? We test the examples by copy/pasting them into a brand new simple TF repo with GH Actions to see if it “just works” but there are always edge cases and not-simple TF repos!
s

Sylvain Witmeyer

08/15/2022, 11:10 PM
no other issue, I literally copy paste the example
11:11 PM
sorry, I had to use this action to connect to aws, just after the add ssh key
uses: aws-actions/configure-aws-credentials@v1
Ali (Infracost)

Ali (Infracost)

08/15/2022, 11:12 PM
are you generating a Terraform plan JSON in your GH Action too? (or are you using the default CLI behavior that parses HCL code - that shouldn’t need aws creds unless you’re fetching usage data with
--sync-usage-file
)
s

Sylvain Witmeyer

08/15/2022, 11:13 PM
aws provider is configured with env variable so I had to inject them
11:14 PM
I don’t know what --sync-usage-file does
Ali (Infracost)

Ali (Infracost)

08/15/2022, 11:18 PM
this page explains
--sync-usage-file
- it’s an advanced feature, don’t worry about it initially 🙂 Thanks for sending that gist - it helps a lot. So you’re using the brand new caching example… we need more community millage on it to get rid of the “broken” feeling…@Alistair (Infracost) will probably love to ping you to see if that made your pipeline faster too. He’s in EU timezone so he’ll ping tmrw 🙂
s

Sylvain Witmeyer

08/15/2022, 11:18 PM
worth to mention, it is a terragrunt project
11:27 PM
I wonder why infracost needs credentials to access AWS if you say it can work without ?
Ali (Infracost)

Ali (Infracost)

08/15/2022, 11:28 PM
It doesn’t - that’s why I’m confused, what error do you get when you comment-out the
Configure AWS Credentials
task in the GH action step in your gist?
s

Sylvain Witmeyer

08/15/2022, 11:32 PM
Run infracost breakdown --path=${TF_ROOT} \
  infracost breakdown --path=${TF_ROOT} \
                      --format=json \
                      --out-file=/tmp/infracost-base.json
  shell: /usr/bin/bash -e {0}
  env:
    SSH_AUTH_SOCK: /tmp/ssh_agent.sock
    TF_ROOT: resources/clusters
    INFRACOST_ENABLE_CLOUD: false
    INFRACOST_GITHUB_ACTION: true
    INFRACOST_SKIP_UPDATE_CHECK: true
    INFRACOST_VCS_REPOSITORY_URL: <https://github.com/xxx/infragrunt>
    INFRACOST_LOG_LEVEL: info


Error: Failed to parse the Terragrunt code using the Terragrunt library:
25 errors occurred:
	* Error finding AWS credentials (did you set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables?): NoCredentialProviders: no valid providers in chain. Deprecated.

25 times the same message
Ali (Infracost)

Ali (Infracost)

08/15/2022, 11:35 PM
Interesting, that’s coming from the terragrunt library (the infracost CLI calls out to that to download modules) - I’ll let Alistair reply tmrw
s

Sylvain Witmeyer

08/16/2022, 12:05 AM
For @Alistair (Infracost) you mention the cache earlier. It helps a lot, I just rerun the same branch and got another issue but not related to infracost - more to the cache I think. Looks a lot like this one https://github.com/gruntwork-io/terragrunt/issues/764 I was able to fix it by commenting the .git exclusion
with:
  path: |
    ${{ env.TF_ROOT }}/**/.infracost/terraform_modules/**
#    !${{ env.TF_ROOT }}/**/.infracost/terraform_modules/**/.git/**
Alistair (Infracost)

Alistair (Infracost)

08/16/2022, 7:41 AM
Hi @Sylvain Witmeyer, thanks for reporting and helping debug this! Yeah it looks like both the issues are coming from inside the terragrunt library which we use. Are you able to:1. provide an example of your terragrunt.hcl file? 2. run with
INFRACOST_LOG_LEVEL: debug
- it might help us see what Terragrunt is doing before it fails
s

Sylvain Witmeyer

08/16/2022, 12:19 PM
Hi @Alistair (Infracost) here is a dump of the logs https://gist.github.com/sylwit/ca8f184f6575d9a998d3c5e3e9379cfb I truncated them and the terragrunt.hcl https://gist.github.com/sylwit/a50b92a9597390d795d4253fc6727ade
Alistair (Infracost)

Alistair (Infracost)

08/16/2022, 12:58 PM
Thanks, do you specify the IAM role anywhere in any of the parent terragrunt config files?
s

Sylvain Witmeyer

08/16/2022, 1:00 PM
yes
iam_role = "arn:aws:iam::${local.aws_account_id}:role/Deployer"
on the parent terragrunt.hcl
Alistair (Infracost)

Alistair (Infracost)

08/16/2022, 1:03 PM
Okay, I wonder if it works if you just comment that line out? We will also have a look and see if there’s a fix we can push for it.
s

Sylvain Witmeyer

08/16/2022, 1:20 PM
it works without assuming the role 🙂
Alistair (Infracost)

Alistair (Infracost)

08/16/2022, 1:20 PM
Awesome! That probably means we can fix it by ignoring it
1:21 PM
I’ll have a play around and see if that’s going to work ok
s

Sylvain Witmeyer

08/16/2022, 1:22 PM
but it will fail if you use the sync-usage no ?
Ali (Infracost)

Ali (Infracost)

08/16/2022, 3:15 PM
@Sylvain Witmeyer the sync-usage explicitly requires creds (via env vars) to fetch usage data from AWS but it does not fail, it just skips the resources it cannot fetch and logs a warning. The sync-usage happens after the HCL parsing, so it’s kinda independent.
s

Sylvain Witmeyer

08/16/2022, 3:18 PM
Understand that, I was just asking if you ignore the assumed role in terragrunt then it should break the sync-usage since you won’t have access to every accounts ?
Ali (Infracost)

Ali (Infracost)

08/16/2022, 3:27 PM
hmm I’m not sure if it’ll break as if you do this then the CLI’ll use those env vars to call the AWS APIs, and those should work across accounts with assume role (or maybe I missing something?)
s

Sylvain Witmeyer

08/16/2022, 3:36 PM
the env var works for a single account but with terragrunt you then assume 1 role in each account you want to deploy. That the reason of the iam_role property. AS you said you need to assume a role so if you ignore this, how could it work ?
Ali (Infracost)

Ali (Infracost)

08/16/2022, 3:50 PM
I see your point, I think we need to update this docs section to explain how the CLI uses those env vars when dealing with multi-account infra projects (if at all). I think the only option we have just now for Terragrunt would be using a config-file and defining per-project env vars and usage-files. If you’re keen on the usage data feature, another option we’re exploring is to define creds in Infracost Cloud - happy to chat about this more over a call if you want?
s

Sylvain Witmeyer

08/16/2022, 4:17 PM
I think the simpler would be ignore assume_role in terragrunt unless you use sync_usage. And everything should work as expected
Alistair (Infracost)

Alistair (Infracost)

08/16/2022, 4:40 PM
I’m pushing a change (https://github.com/infracost/infracost/pull/1977) to ignore the IAM roles for Terragrunt, so this should go out in the next release. Having the sync usage functionality automatically pick up creds/profiles/assumed roles from the HCL would be awesome but is a bit more complex since there’s many ways these can be specified in Terraform or Terragrunt.
4:40 PM
For the GitHub action cache issue we could add a comment to that line saying you might want to exclude it for Terragrunt. When I was testing this, the trade-off was that including the .git directory increased the time to store the cache. What do y’all think?
s

Sylvain Witmeyer

08/16/2022, 4:46 PM
so the fix will break the sync-usage if your main role can’t access children accounts ?
Alistair (Infracost)

Alistair (Infracost)

08/16/2022, 4:59 PM
Sync-usage doesn’t work by reading the
iam_role
from the HCL at the moment anyway, it needs those env variables explicitly set. So this shouldn’t change its behavior.
s

Sylvain Witmeyer

08/16/2022, 5:14 PM
ok not sure I understand how it works with multi account but I’ll try that later. About the github action, there seems to have 2 issues. AS you said the cache but also the required api-key param during setup. If you set a global env which is needed you don’t need to do thathttps://github.com/infracost/actions/blob/master/setup/src/index.ts#L148, hence api-key is not required anymore
Ali (Infracost)

Ali (Infracost)

08/16/2022, 6:03 PM
So I just looked into it and unfortunately the assume role stuff doesn’t work with multiple accounts and the sync-usage-file feature. I pushed this docs change. The code that fetches the usage data uses env variables and is independent of the HCL parsing. We want to iterate on the whole usage-based resources area later - happy to ping you when we start exploring it more if you want. Re the setup action and the INFRACOST_API_KEY, I remember why we did it that way: it’s what a lot of the other actions like AWS or Terraform Cloud actions do with api keys, so it made sense to input the param via env vs then-and-there, instead of telling users to set a global env var as that requires explanation on what workflow level to set it at. I pushed this docs change. Most users I’ve seen pass the param, I assume you didn’t do it that way and instead ran the
infracost configure set
command that didn’t work - anyhow, you have it working now so I wouldn’t change it! Thanks for working through all these issues with us - I really appreciate your help.
s

Sylvain Witmeyer

08/16/2022, 6:15 PM
About the action, that’s not totally true. I didn’t ran any command since this is done by the code of the action itself (cf l148). the setup takes the api-key param and run the configure set. There is no way you can disable this. Also as the debug shown, the credentials file was created but not working.
k=$(infracost configure get api_key)
  echo ${k: -2}
  cat /home/runner/.config/infracost/credentials.yml

du <-- 2 last chars of my key
version: "0.1"
api_key: ***
Ali (Infracost)

Ali (Infracost)

08/16/2022, 6:18 PM
Can you jump on a quick screenshare just now? I want to ensure I’m understanding it right as that smells like a bug
s

Sylvain Witmeyer

08/16/2022, 6:18 PM
sure
Alistair (Infracost)

Alistair (Infracost)

08/16/2022, 6:23 PM
Thanks for finding the .git issue as well 🙏 I’ve added an inline comment for it to our example here which should help other users: https://github.com/infracost/actions/blob/master/examples/using-cache/README.md
8:02 PM
Do you have a
.env
or
.env.local
file with
INFRACOST_API_KEY=ico-XXX
?
Ali (Infracost)

Ali (Infracost)

08/16/2022, 8:14 PM
I pushed a docs change for ^
3:32 AM
I updated this GH issue with a summary of what we think might have happened and also 3 things we can do to avoid this pain for future users. We’ll discuss this in our weekly grooming meeting and update the GH issue with next steps