Skip to content

Coverage for GTS files is inaccurate #417

Open

Description

@elwayman02

This may or may not be related to #407, except that issue has a fix and I've tested with the fixed version of ember-cli-code-coverage to confirm the issue I'm reporting.

To make sure I had this addon set up correctly, I followed the documented steps for ember-cli-build.js in the readme:

  • sourceMaps are set to inline
  • babel plugin is set up
  • sourcemaps is enabled and set to JS extension
  • ember-template-imports inline_source_map config is enabled

Then, running the tests and checking the coverage output showed that when a component is a GTS file (with a GJS integration test file), the coverage % is lower than when it's converted back to separate TS/HBS files (with a JS integration test file).

One gts file I checked has a coverage output of {"lines":{"total":11,"covered":2,"skipped":0,"pct":18.18}.
The same file converted to separate ts/hbs files now has an output of {"lines":{"total":11,"covered":3,"skipped":0,"pct":27.27}.

Another one:
GTS - {"lines":{"total":5,"covered":1,"skipped":0,"pct":20}
TS - {"lines":{"total":5,"covered":2,"skipped":0,"pct":40}

While GTS files are in fact being output in coverage reports now, the numbers are not accurate, and that's artificially lowering test coverage compared to using classic Ember file formats. This likely also impacts GJS files, though I did not explicitly test it.

Tested in both v2.1.2 and v3.0.0 of ember-cli-code-coverage

Here's a snapshot of some other possibly relevant dependency versions:

  • ember-source: 4.12.3
  • ember-cli: 4.12.2
  • ember-template-imports: 3.4.2
  • ember-cli-babel: 7.26.11
  • typescript: 5.5.3

Activity

  1. elwayman02 commented on Jul 24, 2024

    @elwayman02
    Author

    @vstefanovic97 since you were able to fix the missing coverage in the first place, do you have any insight into why this might be happening?

  2. elwayman02 commented on Jul 24, 2024

    @elwayman02
    Author

    Just to eliminate a theory, I tried keeping the component files as GTS but changing the test files back from GJS to normal JS (using the htmlbars inline precompile import), and the numbers did not change at all. It's only when abandoning GTS for the component that the coverage numbers rise again.

  3. vstefanovic97 commented on Jul 24, 2024

    @vstefanovic97
    Contributor

    @elwayman02 could you maybe setup a repo with a minimal reproduction? Have one ts component and one gts component so I could debug and figure out why the coverages are diferent

  4. elwayman02 commented on Jul 24, 2024

    @elwayman02
    Author

    I'll see if I can do that at some point (don't have time right now unfortunately), but one theory we just came up with is that it's because we're using ember-template-imports v3, whereas v4 has improved source map support. 🤔

  5. vstefanovic97 commented on Jul 24, 2024

    @vstefanovic97
    Contributor

    @elwayman02 can you give hint where is this extra covered line in the ts files?

  6. elwayman02 commented on Jul 24, 2024

    @elwayman02
    Author

    I haven't tested side by side, but the examples I looked at were very basic. For example, one of the components I listed above (5 lines, 40% coverage, 20% coverage if GTS) had 3 actions, a tracked property, and a service injection. Something like this:

    export default class Foo extends Component {
      @service('foo@bar') declare bar: BarService;
    
      @tracked isVisible: boolean = false;
    
      @action showBar(baz: string) {
        this.barService.showBar(baz);
      }
    
      @action makeVisible(): void {
        this.isVisible = true;
      }
    
      @action onShowBar() {
        this.showBar('stuff');
      }
    }
    

    and the template, whether in HBS or format, was basically just a button that triggered makeVisible but was hidden if isVisible, another button that triggered showBar with fn to pass a different string, and a component which was rendered if isVisible and passed onShowBar as an argument.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions