Skip to content

Support Addon Coverage - #31

Merged
rwjblue merged 7 commits into
masterfrom
addon-spike
Sep 14, 2016
Merged

rwjblue merged 7 commits into
masterfrom
addon-spike

Conversation

@rwjblue

@rwjblue rwjblue commented Aug 24, 2016 •

Copy link
Copy Markdown
Collaborator

/cc @EWhite613 @sglanzer

Fixes #14.

@rwjblue

rwjblue commented Aug 24, 2016

Copy link
Copy Markdown
Collaborator Author

I'll squash the commits down before merging (likely into one each for @EWhite613 and myself).

@EWhite613 / @sglanzer: Can y'all test this against some of your addons and confirm things are working properly?

@rwjblue rwjblue mentioned this pull request Aug 24, 2016
Comment thread index.js
this._doesFileExistIsCurrentProjectAddonModule(relativePath)
);

console.log(!fileExists, name, relativePath);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you mean to leave this in?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, good catch!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is still in here; I'm seeing a bunch of console logging right now trying to run things on ember-ajax that match this pattern

@kategengler

Copy link
Copy Markdown
Collaborator

I haven't actually tried it yet, but I'm excited to do so. Thanks for doing this!

@kashiif

kashiif commented Aug 24, 2016

Copy link
Copy Markdown

This is exicitng news :). I'll try to test a couple of our addons. Thanks

try {
return instrumenter.instrumentSync(content, relativePath);
} catch (e) {
console.error('Unable to cover:', relativePath, '. Try setting useBabelInstrumenter to true. \n', e.stack);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice addition!

@sandersky

Copy link
Copy Markdown
Contributor

@rwjblue this PR loos great! Thanks a ton for doing this.

@sglanzer-fire

Copy link
Copy Markdown

@rwjblue We've run it against a fresh addon and one of our engines, we can test against a few other addons tomorrow as well - looking 👍

@rwjblue

rwjblue commented Aug 25, 2016

Copy link
Copy Markdown
Collaborator Author

@sglanzer - Awesome, thanks for testing!

Definitely want as many testers (of various different ember-cli versions if possible) as possible here...

@toranb

toranb commented Aug 25, 2016

Copy link
Copy Markdown

I can't wait to use this :) great work everyone! Will this be something that easily drops into ember observer I assume?

@rwjblue

rwjblue commented Aug 25, 2016

Copy link
Copy Markdown
Collaborator Author

Yep, that's the idea...

@arjansingh

arjansingh commented Aug 25, 2016 •

Copy link
Copy Markdown

👍 Tried it on an addon I have and it "Just Worked (tm)". And more importantly the coverage metrics look accurate.

@RobbieTheWagner

Copy link
Copy Markdown
Collaborator

Seems to work for me! Should I add COVERAGE=true in front of the ember try:one in travis.yml?

@sglanzer-fire

Copy link
Copy Markdown

Checking another addon - is anyone else seeing app and tests directories being included in the results? I tried adding a config/coverage.js with

module.exports = {
  excludes: [
    '*/app/**/*',
    '*/tests/**/*'
  ]
}

but that doesn't appear to have an impact either. Not a huge deal, but would be nice to get those files out.

@RobbieTheWagner

Copy link
Copy Markdown
Collaborator

@sglanzer I am seeing the same issue. Not a huge deal for me, but would be nice to not do coverage on tests and app, only the addon folder.

@rwjblue

rwjblue commented Aug 30, 2016

Copy link
Copy Markdown
Collaborator Author

Hmm. The app/ of the addon should be considered as part of its coverage IMO (it is code provided by the addon after all). Including tests/ seems odd though, we need to dig into that to see what is happening.

@sandersky

Copy link
Copy Markdown
Contributor

I agree with @rwjblue. Addons can provide code in app/ that gets merged into the consumers tree, we still want to ensure that code is covered.

@RobbieTheWagner

Copy link
Copy Markdown
Collaborator

@sandersky @rwjblue that is true about the app folder, but I think the issue is most of the app files end up just being export { default } from 'foo/components/bar/component';. No problem from my end leaving app in, but tests should definitely be removed from coverage.

@sglanzer-fire

Copy link
Copy Markdown

Yeah, I'm on board with keeping app

@rwjblue

rwjblue commented Aug 30, 2016

Copy link
Copy Markdown
Collaborator Author

@sglanzer - Do you think we should just auto-filter tests/ so we can ship this, then circle around and see if we can avoid instrumenting more than we need to during the build?

@sglanzer-fire

Copy link
Copy Markdown

I'm cool with that - I'd prefer to start using a release instead of this fork 👍

@rwjblue

rwjblue commented Aug 30, 2016

Copy link
Copy Markdown
Collaborator Author

Hehe, confirm.

@lukesargeant

Copy link
Copy Markdown

ember-cli: 2.7.0
ember-cli-mocha: 0.10.4
No additional coverage config

Running 'ember test' passes 23 tests
Running 'COVERAGE=true ember test' passes the 23 tests
NPM linking to a local copy of this PR branch then running 'COVERAGE=true ember test' fails a 24th test:

not ok 24 PhantomJS 1.9 - Global error: INVALID_STATE_ERR: DOM Exception 11: An attempt was made to use an object that is not, or is no longer, usable. at http://localhost:7357/4061/tests/index.html?hidepassed, line 48
    ---
        Log: |
            { type: 'error',
              text: 'INVALID_STATE_ERR: DOM Exception 11: An attempt was made to use an object that is not, or is no longer, usable. at http://localhost:7357/4061/tests/index.html?hidepassed, line 48\n' }

@sglanzer-fire

Copy link
Copy Markdown

@lukesargeant Any additional info? Do you get similar results on a newer browser (e.g. Phantom 2.x, Chrome, FireFox)

@lukesargeant

Copy link
Copy Markdown

The failure is in Phantom 1.9.8
Chrome: all 23 passing.
I'll hopefully have more time to look into this later, but for now I've uploaded my WIP addon:

https://github.com/lukesargeant/ember-box-utils

The only method that touches the dom is:

static fromElement(el) {
    if (el === window) {
      return new Box({ top: 0, left: 0, bottom: window.innerHeight, right: window.innerWidth });
    } else {
      const rect = el.getBoundingClientRect();
      return new Box({ top: rect.top, left: rect.left, bottom: rect.bottom, right: rect.right });
    }
  }

I imagine the problem is here, or in some tear down in the tests themselves.
I haven't reviewed the diff of this PR but i'm surprised it would cause/reveal a global error.

@sglanzer-fire

sglanzer-fire commented Sep 1, 2016 •

Copy link
Copy Markdown

Yeah, I'd be surprised by that as well - my only thought is that the instrumentation wrapper (basically a function call that wraps each line to let us know if it was executed) might be exposing an error that was occurring, but hidden otherwise? Slightly grasping at straws for an explanation of this one without doing some real debugging.

It's good to see that Chrome behaves correctly - Phantom 1.x runs an old version of webkit and has given me problems in the past, we avoid it for most of our testing.

@lukesargeant

lukesargeant commented Sep 1, 2016 •

Copy link
Copy Markdown

Good news. Reinstalled everything, re-tested and the issue went away.

[EDIT] The issue was actually experienced on master, NOT on this branch. Apologies. This commit seems to have introduced the issue:

f2ba0e8

[EDIT] Raised a separate issue #32

@sglanzer-fire

Copy link
Copy Markdown

Sexy - that's good news - thanks a ton for the investigation @lukesargeant

@sglanzer-fire

Copy link
Copy Markdown

Any further blockers on merging this? Auto-filtering tests?

@RobbieTheWagner

Copy link
Copy Markdown
Collaborator

+1 would love to switch all my apps and addons to this!

@arjansingh

Copy link
Copy Markdown

👍 ditto

On Wednesday, September 7, 2016, Robert Wagner notifications@github.com
wrote:

+1 would love to switch all my apps and addons to this!

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#31 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABhksW1PTx_EQzzZcShO_iNVR_70utyBks5qnutMgaJpZM4JsgKb
.

Arjan

@sandersky

sandersky commented Sep 7, 2016 •

Copy link
Copy Markdown
Contributor

The only issue I see with all of this work which shouldn't block a merge is the path for files coming from the addon directory are in the coverage as modules/<name-of-addon>/path/to/file instead of addon/path/to/file. This will prevent editor plugins that highlight code coverage and services like Coveralls from properly reporting coverage on these files.

@RobbieTheWagner

Copy link
Copy Markdown
Collaborator

@sandersky doesn't something similar happen with blanket? I know I have to rewrite the names to get accurate reports.

@sglanzer-fire

Copy link
Copy Markdown

Yeah, I didn't dig enough to figure out why the path was modules, we would need to go back further in the build lifecycle to pick up the "real" path - it's a legit point, but as @sandersky says, I think it can be addressed as a separate issue

@lukesargeant

Copy link
Copy Markdown

@sandersky +1 for re-writting module/ to addon/ to facilitate editor plugins. Shouldn't block though since this is still useful, but will be the first new issue to raise :-)

@RobbieTheWagner

Copy link
Copy Markdown
Collaborator

So do we want to try to fix the paths now or are there any issues blocking the merging of this?

@sglanzer-fire

Copy link
Copy Markdown

None that I know of, just waiting for the button to be pushed

@kategengler

Copy link
Copy Markdown
Collaborator

I'm deferring to @rwjblue on this, just to be explicit about it.

On Wednesday, September 14, 2016, Steven Glanzer notifications@github.com
wrote:

None that I know of, just waiting for the button to be pushed

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#31 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAbHOjxpsZUxRzYHMg3yRTwfWvk45d_Uks5qqA1WgaJpZM4JsgKb
.

@rwjblue
rwjblue merged commit e5d3955 into master Sep 14, 2016
@rwjblue

rwjblue commented Sep 14, 2016

Copy link
Copy Markdown
Collaborator Author

Released as 0.3.0

This was referenced Sep 14, 2016
@rwjblue
rwjblue deleted the addon-spike branch August 31, 2020 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.