Title
#contributors
b

Balázs Marján

08/12/2022, 2:06 PM
Hi everyone, I’m working with @Hilton Lem on adding IBM Cloud support to Infracost. We are looking at generating usage files for terraform configurations and we have 2 use cases that I want to get your opinion about: 1, The user wants to generate their initial usage file:
sync-usage-file
option helps them out here by providing a usage file with default values(mostly 0s). Do you see a way this could be improved by somehow providing reasonable defaults, something like the contents of the
infracost-usage-example.yml
. Perhaps somehow leveraging that file. Do you see any value here? 2, The user already has their estimate and usage file but they add a new resource that’s type matches an already existing one in the configuration. When using the
sync-usage-file
option, could option leverage the already existing usage estimates for the existing resource type for the same type of resource that was just added? Again, do you think there is any value here? Thanks!
Ali (Infracost)

Ali (Infracost)

08/12/2022, 4:34 PM
Hi @Balázs Marján - thanks for working on IBM Cloud support! Re 1. yep, I’ve heard this from other users before. Specially centralized Infra/DevOps/SRE teams who want to specify defaults for other users. For example, always show the cost of 10GB of storage in an object storage bucket instead of the current default 0 in the usage file. Re 2. that also makes sense, existing usage values defined by a user can be a good baseline for new resources. I wonder if the key missing piece for making 1 and 2 above work nicely is wildcard support in the usage-file. Array wildcards are supported just now:
This works just now:
my_tf_resource.my_group[*]:
  storage_gb: 1000


It would be cool if we can make these work too:
my_tf_resource.my_*
  storage_gb: 1000
my_tf_resource.*
  storage_gb: 1000
What do you think? @Sina Bakhtiari (another community member) opened a PR about wildcard support for modules too.
h

Hilton Lem

08/15/2022, 7:29 PM
Hi @Ali (Infracost) I’m trying to understand how the wildcard support would work for our use case. If the generated usage file triggered from the --sync-usage-file option contained values from https://github.com/IBM-Cloud/infracost/blob/master/infracost-usage-example.yml instead of zero, would that mean that example would have wildcards specified?
Ali (Infracost)

Ali (Infracost)

08/15/2022, 7:43 PM
I suppose there are 2 issues:1. if the usage-file yaml syntax supported wildcards for resources, it would mean I can re-use the same usage-file (with my custom default values) across projects. 2. assuming issue #1 works, how can we generate that file? I was thinking this happens outside of the CLI, as it depends on what values the company wants to set for the users using their custom default values. Or are you thinking that you’d want a way to set the values in https://github.com/IBM-Cloud/infracost/blob/master/infracost-usage-example.yml for all of your users?
h

Hilton Lem

08/15/2022, 7:53 PM
ah.. ok I see now. Your explanation of #1 would be enough for us to generate a usage file that could be used if one wasn’t provided.
8:00 PM
should https://github.com/infracost/infracost/pull/1105/files be updated to handle the additional wild card support? it looks like it hasn’t been looked at for awhile
Ali (Infracost)

Ali (Infracost)

08/15/2022, 8:03 PM
yeah that would be amazing if you have bandwidth, I think this is the missing wildcard use-case we need:
my_tf_resource.* 
  storage_gb: 1000

my_module.my_tf_resource.* 
  storage_gb: 1000
This one seems less important, does it benefit you?
my_tf_resource.my_*
  storage_gb: 1000
(@Alistair (Infracost) any other cases I’m missing?)
h

Hilton Lem

08/15/2022, 8:04 PM
the partial wildcard isn’t something we would use. the first 2 would be good enough
Alistair (Infracost)

Alistair (Infracost)

08/15/2022, 8:19 PM
should https://github.com/infracost/infracost/pull/1105/files be updated to handle the additional wild card support? it looks like it hasn’t been looked at for awhile
I think it can be separate, that handles
<http://module.my|module.my>_module[*].resource_type.resource_name
style wildcards.
h

Hilton Lem

08/17/2022, 2:55 PM
@Ali (Infracost) we’ve started looking at this. we have a lot of modules and would like to have
my_tf_resource.* 
  storage_gb: 1000
be able to apply to any module that has that resource. so the module paths wouldn’t matter, but the current parser looks at string suffixes in the address, and the terraform address would need to be parsed (probably with something like https://github.com/hashicorp/go-terraform-address). Would it be better to support something like
*.my_tf_resource.* 
  storage_gb: 1000
Ali (Infracost)

Ali (Infracost)

08/17/2022, 3:33 PM
@Hilton Lem that makes sense! Would your work mean the following works too?
my_*.my_tf_resource.*
module.my_*.my_tf_resource.*
module.my_module[*].my_tf_resource.*
That would be the most flexible but I understand if you don’t need that as you’re simply defining a global default for a resource type regardless of how it’s used. I think the MVP is just
*.my_tf_resource.*
as a way to set a global default (not even
my_*.my_tf_resource.my_*
).
Alistair (Infracost)

Alistair (Infracost)

08/17/2022, 3:38 PM
Another option could be to add a separate top-level key for specifying defaults based on the resource type. Not sure what the naming should be, but it could look something like this:
version: 0.1
resource_type_usage:
  my_tf_resource_type:
    monthly_requests: 10000 # This sets a default for all my_tf_resource_type resources

resource_usage:
  my_tf_resource_type.specific_resource:
    monthly_requests: 2000 # This overrides the my_tf_resource_type default for this specific resource
Ali (Infracost)

Ali (Infracost)

08/17/2022, 3:52 PM
Assuming @Hilton Lem doesn’t need the 3 additional wildcard options I mentioned above, then your proposal seems cleaner @Alistair (Infracost) as it also makes the precedence clear (“defaults can be overridden”).
4:12 PM
Thinking more about this, I really like Alistair’s solution as it solves one use-case very cleanly: defaults for a resource type - and we’ve heard this from users, e.g. I want to default all S3 buckets to 10GB and 10K requests/month. Using defaults also means those resource types would show in the PR comment summary table too, so the usage-based costs aren’t seen as “skipped” or “free” when someone looks at that table. Wildcards are harder to explain/document for users, specially their precedence.
h

Hilton Lem

08/17/2022, 5:00 PM
I really like that option.
5:01 PM
@Balázs Marján ^^^
b

Balázs Marján

08/18/2022, 4:20 PM
Hello! I opened an issue for this:https://github.com/infracost/infracost/issues/1978 Here is the POC for it:https://github.com/infracost/infracost/pull/1979 See if you like it or find issues with it
Vadim (Infracost)

Vadim (Infracost)

08/18/2022, 4:28 PM
@Balázs Marján Hello! 👋 Thank you! Awesome! @Hugo (Infracost) will take a look soon 👍
Ali (Infracost)

Ali (Infracost)

09/08/2022, 1:35 PM
@Hilton Lem just FYI that we reviewed https://github.com/infracost/infracost/pull/1105 and left a comment