https://infracost.io logo
Join Slack
Powered by
This message was deleted.
# general
b
This message was deleted.
l
Hi @stocky-salesmen-15167, yes it should do - are you seeing an issue?
s
Think so .. I’m using Leapp to get my AWS credentials for the cli and when I run 
infracost breakdown --path . --sync-usage-file --usage-file /tmp/ignore.yml
I see no usage is fetched for the S3 resource which is created inside the module I call in my main.tf.
html output still looks like this:
hmmm … wait, where are the 
…mock
things coming from. Are the tags used to fetch the corresponding metrics from cloudwatch?
l
I think it could be one of two things: 1. The sync usage functionality uses whatever AWS credentials are in the default provider chain, so the same as what the 
aws
cli uses. So you might need to export creds with 
AWS_ACCESS_KEY_ID
and 
AWS_SECRET_ACCESS_KEY
env variables. 2. S3 storage should be synced by default but for S3 requests to appear in CloudWatch you’ll need to set up a metrics filter in AWS.
Are the tags used to fetch the corresponding metrics from cloudwatch?
Those tags we pull from your Terraform code and just display next to the resource here, they’re not used to sync usage - we use the bucket name for syncing.
s
The 
mock
is solved as I forgot to point the TF backend to the TFC workspace, so no workspace variables were fetched. I’ll dig into the metrics filter. Thanks so far.
l
👍 let me know if you get it working. If you run with 
--log-level=debug
there might be more clues.
s
Hmmm … something else is a little bit weird. In the debug log I see a somehow “suffixed” Bucketname, which is not correct and there are some NPE also. Maybe I have a mismatch locally in my TF cli version and the code it is written for. Need to find a different working project and try again.
l
Hmm, could you try generating a Terraform plan JSON and running with 
--path
pointing to that? Maybe it’s not getting the correct bucket name when doing HCL parsing.
s
Let’s see if I can make this happen with “remote” execution in TFC…
@little-author-61621 getting a plan from TFC is nasty and only possible via API, but anyway … got it. And yes, the S3 storage is correctly fetched if I point infracost to the plan.json. The S3 bucketname is also correct now and I think I know what’s the matter: we suffix our buckets always with a 
random_pet
. it seems that infracost’s HCL parsing is not fully working if resource name attributes are using 
random_pets
… that’s probably the NPE I’m seeing
The NPE when using HCL parsing looks like this:
Copy code
time="2022-06-29T13:32:26+02:00" level=debug parser=terraform_hcl msg="could not evaluate value for attr: id err: runtime error: invalid memory address or nil pointer dereference\ngoroutine 70 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\<http://ngithub.com/infracost/infracost/internal/hcl.(*Attribute).value.func1()\n\tgithub.com/infracost/infracost/internal/hcl/attribute.go:71|ngithub.com/infracost/infracost/internal/hcl.(*Attribute).value.func1()\n\tgithub.com/infracost/infracost/internal/hcl/attribute.go:71> +0x79\npanic({0x2995700, 0x4203840})\n\truntime/panic.go:838 +0x207\<http://ngithub.com/zclconf/go-cty/cty.Type.FriendlyName(...)\n\tgithub.com/zclconf/go-cty@v1.10.0/cty/type.go:47\ngithub.com/hashicorp/hcl/v2/hclsyntax.describeConditionalTypeMismatch(|ngithub.com/zclconf/go-cty/cty.Type.FriendlyName(...)\n\tgithub.com/zclconf/go-cty@v1.10.0/cty/type.go:47\ngithub.com/hashicorp/hcl/v2/hclsyntax.describeConditionalTypeMismatch(>{{0x0?, 0x0?}}, {{0x32d5ca0?, 0xc000200255?}})\n\<http://tgithub.com/hashicorp/hcl/v2@v2.12.0/hclsyntax/expression.go:833|tgithub.com/hashicorp/hcl/v2@v2.12.0/hclsyntax/expression.go:833> +0x3ee\<http://ngithub.com/hashicorp/hcl/v2/hclsyntax.(*ConditionalExpr).Value(0xc0001ae2a0|ngithub.com/hashicorp/hcl/v2/hclsyntax.(*ConditionalExpr).Value(0xc0001ae2a0>, 0xc000b271b8)\n\<http://tgithub.com/hashicorp/hcl/v2@v2.12.0/hclsyntax/expression.go:620|tgithub.com/hashicorp/hcl/v2@v2.12.0/hclsyntax/expression.go:620> +0x1306\<http://ngithub.com/infracost/infracost/internal/hcl.(*Attribute).value(0xc000cd8320|ngithub.com/infracost/infracost/internal/hcl.(*Attribute).value(0xc000cd8320>, 0x0)\n\<http://tgithub.com/infracost/infracost/internal/hcl/attribute.go:77|tgithub.com/infracost/infracost/internal/hcl/attribute.go:77> +0xce\<http://ngithub.com/infracost/infracost/internal/hcl.(*Attribute).Value(...)\n\tgithub.com/infracost/infracost/internal/hcl/attribute.go:65\ngithub.com/infracost/infracost/internal/hcl.(*Block).Values(0xc000d2c8e0?)\n\tgithub.com/infracost/infracost/internal/hcl/block.go:740|ngithub.com/infracost/infracost/internal/hcl.(*Attribute).Value(...)\n\tgithub.com/infracost/infracost/internal/hcl/attribute.go:65\ngithub.com/infracost/infracost/internal/hcl.(*Block).Values(0xc000d2c8e0?)\n\tgithub.com/infracost/infracost/internal/hcl/block.go:740> +0xf0\<http://ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).getValuesByBlockType|ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).getValuesByBlockType>(0xc000a48b40, {0x2e86fa4, 0x6})\n\<http://tgithub.com/infracost/infracost/internal/hcl/evaluator.go:469|tgithub.com/infracost/infracost/internal/hcl/evaluator.go:469> +0x285\<http://ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).evaluateStep(0xc000a48b40|ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).evaluateStep(0xc000a48b40>, 0x1)\n\<http://tgithub.com/infracost/infracost/internal/hcl/evaluator.go:192|tgithub.com/infracost/infracost/internal/hcl/evaluator.go:192> +0x14b\<http://ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).evaluate|ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).evaluate>(0xc000a48b40, {0x0?, 0x0?, 0x0?})\n\<http://tgithub.com/infracost/infracost/internal/hcl/evaluator.go:170|tgithub.com/infracost/infracost/internal/hcl/evaluator.go:170> +0x7c\<http://ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).Run(0xc000a48b40)\n\tgithub.com/infracost/infracost/internal/hcl/evaluator.go:141|ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).Run(0xc000a48b40)\n\tgithub.com/infracost/infracost/internal/hcl/evaluator.go:141> +0xc5\<http://ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).evaluateModules(0xc000a46900)\n\tgithub.com/infracost/infracost/internal/hcl/evaluator.go:240|ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).evaluateModules(0xc000a46900)\n\tgithub.com/infracost/infracost/internal/hcl/evaluator.go:240> +0x333\<http://ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).evaluateStep|ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).evaluateStep>(0xc000a46900, 0x0)\n\<http://tgithub.com/infracost/infracost/internal/hcl/evaluator.go:203|tgithub.com/infracost/infracost/internal/hcl/evaluator.go:203> +0x3b3\<http://ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).evaluate|ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).evaluate>(0xc000a46900, {0x0?, 0x0?, 0x0?})\n\<http://tgithub.com/infracost/infracost/internal/hcl/evaluator.go:170|tgithub.com/infracost/infracost/internal/hcl/evaluator.go:170> +0x7c\<http://ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).Run(0xc000a46900)\n\tgithub.com/infracost/infracost/internal/hcl/evaluator.go:145|ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).Run(0xc000a46900)\n\tgithub.com/infracost/infracost/internal/hcl/evaluator.go:145> +0x108\<http://ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).evaluateModules(0xc00095d0e0)\n\tgithub.com/infracost/infracost/internal/hcl/evaluator.go:240|ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).evaluateModules(0xc00095d0e0)\n\tgithub.com/infracost/infracost/internal/hcl/evaluator.go:240> +0x333\<http://ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).evaluateStep|ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).evaluateStep>(0xc00095d0e0, 0x0)\n\<http://tgithub.com/infracost/infracost/internal/hcl/evaluator.go:203|tgithub.com/infracost/infracost/internal/hcl/evaluator.go:203> +0x3b3\<http://ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).evaluate|ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).evaluate>(0xc00095d0e0, {0x0?, 0x0?, 0x0?})\n\<http://tgithub.com/infracost/infracost/internal/hcl/evaluator.go:170|tgithub.com/infracost/infracost/internal/hcl/evaluator.go:170> +0x7c\<http://ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).Run(0xc00095d0e0)\n\tgithub.com/infracost/infracost/internal/hcl/evaluator.go:145|ngithub.com/infracost/infracost/internal/hcl.(*Evaluator).Run(0xc00095d0e0)\n\tgithub.com/infracost/infracost/internal/hcl/evaluator.go:145> +0x108\<http://ngithub.com/infracost/infracost/internal/hcl.(*Parser).ParseDirectory(0xc0003e62c0)\n\tgithub.com/infracost/infracost/internal/hcl/parser.go:329|ngithub.com/infracost/infracost/internal/hcl.(*Parser).ParseDirectory(0xc0003e62c0)\n\tgithub.com/infracost/infracost/internal/hcl/parser.go:329> +0x4b0\<http://ngithub.com/infracost/infracost/internal/providers/terraform.(*HCLProvider).Modules(0xc00045a240)\n\tgithub.com/infracost/infracost/internal/providers/terraform/hcl_provider.go:248|ngithub.com/infracost/infracost/internal/providers/terraform.(*HCLProvider).Modules(0xc00045a240)\n\tgithub.com/infracost/infracost/internal/providers/terraform/hcl_provider.go:248> +0x139\<http://ngithub.com/infracost/infracost/internal/providers/terraform.(*HCLProvider).LoadPlanJSONs(0xc00045a240)\n\tgithub.com/infracost/infracost/internal/providers/terraform/hcl_provider.go:216|ngithub.com/infracost/infracost/internal/providers/terraform.(*HCLProvider).LoadPlanJSONs(0xc00045a240)\n\tgithub.com/infracost/infracost/internal/providers/terraform/hcl_provider.go:216> +0x4a\<http://ngithub.com/infracost/infracost/internal/providers/terraform.(*HCLProvider).LoadResources(0xc0001906c0|ngithub.com/infracost/infracost/internal/providers/terraform.(*HCLProvider).LoadResources(0xc0001906c0>?, 0x4?)\n\<http://tgithub.com/infracost/infracost/internal/providers/terraform/hcl_provider.go:171|tgithub.com/infracost/infracost/internal/providers/terraform/hcl_provider.go:171> +0x35\nmain.(*parallelRunner).runProjectConfig(0xc000371230, 0xc0003517c0)\n\<http://tgithub.com/infracost/infracost/cmd/infracost/run.go:474|tgithub.com/infracost/infracost/cmd/infracost/run.go:474> +0x1269\nmain.(*parallelRunner).run.func1()\n\<http://tgithub.com/infracost/infracost/cmd/infracost/run.go:310|tgithub.com/infracost/infracost/cmd/infracost/run.go:310> +0x1c5\<http://ngolang.org/x/sync/errgroup.(*Group).Go.func1()\n\tgolang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57|ngolang.org/x/sync/errgroup.(*Group).Go.func1()\n\tgolang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57> +0x67\ncreated by <http://golang.org/x/sync/errgroup.(*Group).Go\n\tgolang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:54|golang.org/x/sync/errgroup.(*Group).Go\n\tgolang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:54> +0x8d\n"
Code example looks like:
Copy code
resource "random_pet" "s3" {
}

locals {
  pet = (var.ci_mode == true ? "ci-pet" : random_pet.s3.id)
}

resource "aws_s3_bucket" "public" {
  bucket = "testbucket-${local.pet}"
}
Maybe it’s the 
local.pet
in between?!
l
Hmm… I’ve not seen that panic before. I’ll ping @mysterious-teacher-68276 in case it’s one he’s seen/fixed recently, but if not we should file an issue and look at it,
getting a plan from TFC is nasty and only possible via API, but anyway … got it.
Yes, I know this pain. We have to this like https://gitlab.com/infracost/infracost-gitlab-ci/-/blob/master/examples/plan-json/terraform-cloud-enterprise/README.md#L43-53 in our examples currently.
m
Hey @stocky-salesmen-15167, I’m not entirely following what you’re wanting here. With HCL parsing 
random_pet.s3.id
will evaluate each time as a different 
mock
value, this is where the 
-mock
values are coming from in the dashboard/share link. HCL parsing doesn’t have the idea of state, so random generation will be random each time. So if you’re trying to sync a bucket which relies on a random value this won’t work. Or at least I don’t think it will. The panic debug log looks like a problem generating these mock attributes, but I’m not entirely sure this should matter. Something I’ll fix anyway.
s
@mysterious-teacher-68276 Thanks for the explanation. Sure 🤦‍♂️ HCL has no clue of state and therefore also not on the outcome of random pets. Makes sense.
Sorry, for this little rabbit hole I went into.
l
So if you’re trying to sync a bucket which relies on a random value this won’t work.
Yep, this is something we have some ideas on how to get round - e.g. query buckets that could match, or introduce some tag matching.
The panic debug log looks like a problem generating these mock attributes, but I’m not entirely sure this should matter.
Thanks @mysterious-teacher-68276. @stocky-salesmen-15167 are you able to file an issue for this panic?
s
Should be possible (filing an issue) 😉
🙌 1
l
Sorry, for this little rabbit hole I went into.
I love rabbit holes! 🐰
s
l
Thanks!
6 Views
Open in Slack
PreviousNext
Powered by