https://infracost.io logo
Powered by
p

polite-vr-31842

05/17/2021, 12:31 PM
What would be the best way to convert such test case to golden file test? https://github.com/infracost/infracost/blob/master/internal/providers/terraform/aws/db_instance_test.go#L131
c

crooked-daybreak-55253

05/17/2021, 12:41 PM
At the risk of being annoying 😁: What was your initial thought? (I’m just noticing you seen to have a lot of ideas and I love that. The more ideas the better).
🦜 1
👍 1
❤️ 1
In the spirit of being less annoying: My first reaction would be to “unwind the loop” so you end up with a tf file a resource for each engine:
Copy code
resource "aws_db_instance" "aurora" { 
  engine = "aurora"
  instance_class = "db.t3.small"
}

resource "aws_db_instance" "aurora-mysql" { 
  engine = "aurora-mysql"
  instance_class = "db.t3.small"
}

...
👍 1
p

polite-vr-31842

05/17/2021, 12:49 PM
My initial idea was to have a resource for each of them. I think it won't have a sensible impact on the speed of tests.
👍 1
Seeing this test reminded me of so many conditions that we are probably not handling correctly in the code. Since we do not have a test for all possible permutation of enum configs. I don't know how but I think we have to think about it some point in the future.
c

crooked-daybreak-55253

05/17/2021, 12:58 PM
You mean how we have a test for “aurora” “db.t3.small”, but not one “aurora” “db.t3.medium”, right? That’s interesting,
Seems like it would be nice if we somehow at least had visibility into what permutations we’re not testing.
p

polite-vr-31842

05/17/2021, 1:51 PM
Sorry for late response, yes that's exactly what I mean. There are two kinds of uncovered cases: 1. Simple cases: where there is no special logic involved and we can just be sure about the pricing API coverage 2. Complex cases: These are where we have if, select statements and ... for instance when the field 
A
value is 
X
then we set the query field 
B
to 
Z
. These are where most errors are. I personally can think of at-least 2-3 times that my code had such errors and the reviewer found it.
👍 1
c

crooked-daybreak-55253

05/17/2021, 2:33 PM
What’s a concrete example of #2?
p

polite-vr-31842

05/17/2021, 3:07 PM
Take https://github.com/infracost/infracost/blob/master/internal/providers/terraform/google/storage_bucket.go#L48 as an eaxmple: 
resourceGroup
is based on the value of 
storageClass
(there is a probability of typos here). Then if 
resourceGroup
is regional and we have certain 
location
field we have to change the 
resourceGroup
. So it's kind of a nested switch statement.
c

crooked-daybreak-55253

05/17/2021, 3:23 PM
Ok yeah I see what you mean.
w

white-airport-8778

05/17/2021, 10:12 PM
Does the https://fuzzbuzz.io/what-is-fuzz-testing/ > “Coverage Guided Fuzzing” seem interesting? If so, let me know if you’d like me to arrange a quick demo call with them this Wed/Thu, I know the founders…
p

polite-vr-31842

05/18/2021, 7:27 PM
That would probably cover some areas but I'm not sure if that could be the best strategy. I think we have to think more about the possible solutions.
7 Views
Powered by