Channels
  • s

    Sina Bakhtiari

    1 year ago
    What would be the best way to convert such test case to golden file test? https://github.com/infracost/infracost/blob/master/internal/providers/terraform/aws/db_instance_test.go#L131
  • Tim (Infracost)

    Tim (Infracost)

    1 year ago
    At the risk of being annoying 😁: What was your initial thought? (I’m just noticing you seen to have a lot of ideas and I love that. The more ideas the better).
    In the spirit of being less annoying: My first reaction would be to “unwind the loop” so you end up with a tf file a resource for each engine:
    resource "aws_db_instance" "aurora" { 
      engine = "aurora"
      instance_class = "db.t3.small"
    }
    
    resource "aws_db_instance" "aurora-mysql" { 
      engine = "aurora-mysql"
      instance_class = "db.t3.small"
    }
    
    ...
  • s

    Sina Bakhtiari

    1 year ago
    My initial idea was to have a resource for each of them. I think it won't have a sensible impact on the speed of tests.
    Seeing this test reminded me of so many conditions that we are probably not handling correctly in the code. Since we do not have a test for all possible permutation of enum configs. I don't know how but I think we have to think about it some point in the future.
  • Tim (Infracost)

    Tim (Infracost)

    1 year ago
    You mean how we have a test for “aurora” “db.t3.small”, but not one “aurora” “db.t3.medium”, right? That’s interesting,
    Seems like it would be nice if we somehow at least had visibility into what permutations we’re not testing.
  • s

    Sina Bakhtiari

    1 year ago
    Sorry for late response, yes that's exactly what I mean. There are two kinds of uncovered cases:1. Simple cases: where there is no special logic involved and we can just be sure about the pricing API coverage 2. Complex cases: These are where we have if, select statements and ... for instance when the field
    A
    value is
    X
    then we set the query field
    B
    to
    Z
    . These are where most errors are. I personally can think of at-least 2-3 times that my code had such errors and the reviewer found it.
  • Tim (Infracost)

    Tim (Infracost)

    1 year ago
    What’s a concrete example of #2?
  • s

    Sina Bakhtiari

    1 year ago
    Take https://github.com/infracost/infracost/blob/master/internal/providers/terraform/google/storage_bucket.go#L48 as an eaxmple:
    resourceGroup
    is based on the value of
    storageClass
    (there is a probability of typos here). Then if
    resourceGroup
    is regional and we have certain
    location
    field we have to change the
    resourceGroup
    . So it's kind of a nested switch statement.
  • Tim (Infracost)

    Tim (Infracost)

    1 year ago
    Ok yeah I see what you mean.
  • Ali (Infracost)

    Ali (Infracost)

    1 year ago
    Does the https://fuzzbuzz.io/what-is-fuzz-testing/
    “Coverage Guided Fuzzing” seem interesting? If so, let me know if you’d like me to arrange a quick demo call with them this Wed/Thu, I know the founders…