Skip to content

Checking app code agains thresholds #23

Closed as not planned
Closed as not planned

Description

@jonathansamines

I wonder if it is possible to add thresholds checking to the ember-cli build process.

Activity

  1. RobbieTheWagner commented on Sep 30, 2017

    @RobbieTheWagner
    Collaborator

    @jonathansamines sorry for the delay in responding, but could you please elaborate 😄

  2. jonathansamines commented on Oct 2, 2017

    @jonathansamines
    Author

    @rwwagner90 I mean, something like instanbul check coverage functionality, where if your current coverage doesn't comply with the expected coverage your build breaks :)

  3. RobbieTheWagner commented on Feb 8, 2018

    @RobbieTheWagner
    Collaborator

    @jonathansamines I think we can configure istanbul more directly now, so this might be possible with the latest versions. Do you have any interest in trying to figure it out? I generally just enforce coverage thresholds with Code Climate, Coveralls, etc.

  4. adamjmcgrath commented on Feb 8, 2018

    @adamjmcgrath
    Collaborator

    Not checked which fields of.istanbul.yml would work. You could also do nyc check-coverage --lines 95 --functions 95 --branches 95 in your posttest script.

  5. adamjmcgrath commented on Feb 13, 2018

    @adamjmcgrath
    Collaborator

    Not checked which fields of .istanbul.yml would work.

    Have checked this, the check .istanbul.yml fields don't work - in the long run we should probably fix this, but it wont work in the short run.

    You could also do nyc check-coverage --lines 95 --functions 95 --branches 95 in your posttest script.

    Have tested this, also doesn't work because nyc expects the coverage report to be in a different place/format

    The solution that I've tested and works fine is using the istanbul@1 CLI

    $ npm install --save-dev istanbul
    $ COVERAGE=true ember test
    $ istanbul check-coverage --statements=100 --functions=100 --branches=100 --lines=100 coverage/coverage-final.json

    You probably need to make sure you use the json reporter so that you get a coverage-final.json file in your coverage folder

  6. RobbieTheWagner commented on Apr 4, 2018

    @RobbieTheWagner
    Collaborator

    @adamjmcgrath with the merge of #169 which said that version works with nyc, can we officially recommend nyc as the method for checking thresholds in the README? Perhaps even offer a config to add nyc and check automatically, failing the Ember build if the threshold is not met?

  7. adamjmcgrath commented on Apr 8, 2018

    @adamjmcgrath
    Collaborator

    I haven't had an opportunity to try it yet - perhaps @charlesdemers can confirm?

  8. charlesdemers commented on Apr 8, 2018

    @charlesdemers
    Contributor

    Yep it’s possible to fail the build on Travis with nyc now that #169 is merged, I do it with this config:

    .nycrc

    {
      "all": true,
      "check-coverage": true,
      "per-file": true,
      "temp-directory": "coverage",
      "branches": 60,
      "functions": 60,
      "lines": 60,
      "statements": 60,
      "watermarks": {
        "branches": [70, 80],
        "functions": [70, 80],
        "lines": [70, 80],
        "statements": [70, 80]
      }
    }
    

    in package.json:

    "scripts": {
      "test": "COVERAGE=true ember test",
      "check-coverage": "./node_modules/.bin/nyc check-coverage",
    }
    

    in .travis.yml

    script:
       - npm test
       - npm run check-coverage
    
  9. RobbieTheWagner commented on Apr 9, 2018

    @RobbieTheWagner
    Collaborator

    @adamjmcgrath @charlesdemers would one of you like to document this in the README? I also think it could be nice to add nyc as a dep to this addon, and just build this functionality in. I would like the Ember build to fail, when the coverage is not sufficient, if we can do that. That way, testing locally and on travis are the same.

  10. kategengler commented on Apr 11, 2018

    @kategengler
    Collaborator

    I think leaving it to the user to add in checking coverage is sufficient. They can change their npm test command to both run tests and check coverage, if they want that. I do think documenting that nyc utils can now be used with this addon is something we should do.

  11. RobbieTheWagner commented on Apr 11, 2018

    @RobbieTheWagner
    Collaborator

    Sounds good. @charlesdemers would you mind submitting a PR to document how you use nyc?

  12. charlesdemers commented on Apr 11, 2018

    @charlesdemers
    Contributor

    Sure! I can do it sometime this week :)

  13. RobbieTheWagner commented on Jun 14, 2020

    @RobbieTheWagner
    Collaborator

    This probably shouldn't be closed until the PR is either merged or closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions