https://infracost.io logo
Join Slack
Title
#contributors
s

Sina Bakhtiari

05/17/2021, 12:31 PM
What would be the best way to convert such test case to golden file test? https://github.com/infracost/infracost/blob/master/internal/providers/terraform/aws/db_instance_test.go#L131
Tim (Infracost)

Tim (Infracost)

05/17/2021, 12:41 PM
At the risk of being annoying 😁: What was your initial thought? (I’m just noticing you seen to have a lot of ideas and I love that. The more ideas the better).
😛artyparrot: 1👍 1❤️ 1
12:47 PM
In the spirit of being less annoying: My first reaction would be to “unwind the loop” so you end up with a tf file a resource for each engine:
resource "aws_db_instance" "aurora" { 
  engine = "aurora"
  instance_class = "db.t3.small"
}

resource "aws_db_instance" "aurora-mysql" { 
  engine = "aurora-mysql"
  instance_class = "db.t3.small"
}

...
👍 1
s

Sina Bakhtiari

05/17/2021, 12:49 PM
My initial idea was to have a resource for each of them. I think it won't have a sensible impact on the speed of tests.
👍 1
12:51 PM
Seeing this test reminded me of so many conditions that we are probably not handling correctly in the code. Since we do not have a test for all possible permutation of enum configs. I don't know how but I think we have to think about it some point in the future.
Tim (Infracost)

Tim (Infracost)

05/17/2021, 12:58 PM
You mean how we have a test for “aurora” “db.t3.small”, but not one “aurora” “db.t3.medium”, right? That’s interesting,
12:59 PM
Seems like it would be nice if we somehow at least had visibility into what permutations we’re not testing.
s

Sina Bakhtiari

05/17/2021, 1:51 PM
Sorry for late response, yes that's exactly what I mean. There are two kinds of uncovered cases:1. Simple cases: where there is no special logic involved and we can just be sure about the pricing API coverage 2. Complex cases: These are where we have if, select statements and ... for instance when the field 
A
value is 
X
then we set the query field 
B
to 
Z
. These are where most errors are. I personally can think of at-least 2-3 times that my code had such errors and the reviewer found it.
👍 1
Tim (Infracost)

Tim (Infracost)

05/17/2021, 2:33 PM
What’s a concrete example of #2?
s

Sina Bakhtiari

05/17/2021, 3:07 PM
Take https://github.com/infracost/infracost/blob/master/internal/providers/terraform/google/storage_bucket.go#L48 as an eaxmple: 
resourceGroup
is based on the value of 
storageClass
(there is a probability of typos here). Then if 
resourceGroup
is regional and we have certain 
location
field we have to change the 
resourceGroup
. So it's kind of a nested switch statement.
Tim (Infracost)

Tim (Infracost)

05/17/2021, 3:23 PM
Ok yeah I see what you mean.
Ali (Infracost)

Ali (Infracost)

05/17/2021, 10:12 PM
Does the https://fuzzbuzz.io/what-is-fuzz-testing/
“Coverage Guided Fuzzing” seem interesting? If so, let me know if you’d like me to arrange a quick demo call with them this Wed/Thu, I know the founders…
s

Sina Bakhtiari

05/18/2021, 7:27 PM
That would probably cover some areas but I'm not sure if that could be the best strategy. I think we have to think more about the possible solutions.
View count: 7
Join thread in Slack