Skip to content

Commit eeee5dc

Browse files
owenthcareyowenthcarey
committed
test: add e2e-suite covering every public API
1 parent e097c0c commit eeee5dc

168 files changed

Lines changed: 5404 additions & 164 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/e2e.yml‎

Lines changed: 21 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,24 @@ on:
88
workflow_dispatch:
99

1010
jobs:
11+
coverage:
12+
# Runs first; fast static check that gates the long device jobs so
13+
# missing demos / flows fail before the emulator boots.
14+
runs-on: ubuntu-latest
15+
steps:
16+
- name: Checkout
17+
uses: actions/checkout@v4
18+
- name: Set up Python
19+
uses: actions/setup-python@v5
20+
with:
21+
python-version: '3.11'
22+
- name: Check E2E coverage of pythonnative.__all__
23+
run: python scripts/check-e2e-coverage.py
24+
1125
e2e-android:
26+
needs: coverage
1227
runs-on: ubuntu-latest
13-
timeout-minutes: 30
28+
timeout-minutes: 45
1429

1530
steps:
1631
- name: Checkout
@@ -46,12 +61,12 @@ jobs:
4661
with:
4762
api-level: 31
4863
arch: x86_64
49-
script: >-
50-
bash -lc "cd examples/hello-world && pn run android --no-logs && cd ../.. && maestro test tests/e2e/android.yaml"
64+
script: bash -lc "./scripts/run-e2e.sh android"
5165

5266
e2e-ios:
67+
needs: coverage
5368
runs-on: macos-latest
54-
timeout-minutes: 30
69+
timeout-minutes: 45
5570

5671
steps:
5772
- name: Checkout
@@ -71,11 +86,7 @@ jobs:
7186
echo "$HOME/.maestro/bin" >> $GITHUB_PATH
7287
brew tap facebook/fb && brew install idb-companion
7388
74-
- name: Build and launch iOS app
75-
working-directory: examples/hello-world
76-
run: pn run ios --no-logs
89+
- name: Build and run E2E tests
90+
run: ./scripts/run-e2e.sh ios
7791
env:
7892
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
79-
80-
- name: Run E2E tests
81-
run: maestro --platform ios test tests/e2e/ios.yaml

‎CONTRIBUTING.md‎

Lines changed: 32 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -35,10 +35,13 @@ cd examples/hello-world && pn run android
3535
- `src/pythonnative/` – installable library and CLI
3636
- `pythonnative/` – core cross‑platform UI components and utilities
3737
- `cli/` – `pn` command
38-
- `tests/` – unit tests for the library
38+
- `tests/` – unit tests for the library, plus the Maestro E2E suite
39+
- `e2e/` – the comprehensive E2E suite (see [E2E tests](#e2e-tests-maestro) below and `tests/e2e/AGENTS.md`)
3940
- `templates/` – Android/iOS project templates and zips
4041
- `examples/` – runnable example apps
41-
- `hello-world/` – minimal demo app using the library
42+
- `hello-world/` – minimal marketing demo
43+
- `e2e-suite/` – comprehensive feature catalog that drives the Maestro E2E suite
44+
- `scripts/` – helper scripts (`check.sh`, `run-e2e.sh`, `check-e2e-coverage.py`)
4245
- `README.md`, `pyproject.toml` – repo docs and packaging
4346

4447
## Coding guidelines
@@ -269,7 +272,9 @@ fix/cli-regression
269272

270273
### E2E tests (Maestro)
271274

272-
End-to-end tests use [Maestro](https://maestro.dev/) to drive the hello-world example on real emulators and simulators.
275+
End-to-end tests use [Maestro](https://maestro.dev/) to drive the dedicated `examples/e2e-suite` app on real emulators and simulators. That app contains one screen per public symbol in `pythonnative.__all__`; every flow under `tests/e2e/flows/<category>/` exercises one symbol.
276+
277+
The dedicated `examples/hello-world` app is left in place as a small marketing demo; it is **not** the E2E target.
273278

274279
```bash
275280
# Install Maestro (one-time)
@@ -279,21 +284,36 @@ curl -Ls "https://get.maestro.mobile.dev" | bash
279284
brew tap facebook/fb && brew install idb-companion
280285
```
281286

282-
Build and launch the app first, then run the tests:
287+
Build and run everything via the convenience script:
283288

284289
```bash
285-
cd examples/hello-world
286-
287290
# Android (emulator must be running)
288-
pn run android
289-
maestro test ../../tests/e2e/android.yaml
291+
./scripts/run-e2e.sh android
292+
293+
# iOS (simulator must be running)
294+
./scripts/run-e2e.sh ios
295+
```
296+
297+
For tight iteration, run a single category instead of the full pass:
290298

291-
# iOS (simulator must be running; --platform ios needed when an Android emulator is also connected)
292-
pn run ios
293-
maestro --platform ios test ../../tests/e2e/ios.yaml
299+
```bash
300+
./scripts/run-e2e.sh android hooks
301+
./scripts/run-e2e.sh ios components
294302
```
295303

296-
Test flows live in `tests/e2e/flows/` and cover the main screen rendering, counter interaction, and multi-screen navigation. The `e2e.yml` workflow runs these automatically on pushes to `main` and PRs.
304+
Available categories: `components`, `hooks`, `navigation`, `layout`, `styling`, `animations`, `misc`.
305+
306+
A coverage checker, `scripts/check-e2e-coverage.py`, gates CI: every name in `pythonnative.__all__` must be covered by a demo + flow, or listed in `INTENTIONAL_EXEMPTIONS` with a justification.
307+
308+
When you add a new public symbol you must also:
309+
310+
1. Add a demo screen under `examples/e2e-suite/app/screens/<category>/`.
311+
2. Append a `DemoEntry` in `examples/e2e-suite/app/registry.py`.
312+
3. Add a Maestro flow at `tests/e2e/flows/<category>/<name>.yaml`.
313+
4. Append the flow to the top-level `tests/e2e/android.yaml`, `tests/e2e/ios.yaml`, and the matching `tests/e2e/suites/<category>.yaml`.
314+
5. Confirm `python scripts/check-e2e-coverage.py` exits 0.
315+
316+
`tests/e2e/AGENTS.md` is the deeper reference (label conventions, failure triage, naming rules); AI agents should read it before touching the suite. The `e2e.yml` workflow runs the suite automatically on pushes to `main` and PRs.
297317

298318
### CI
299319

‎examples/e2e-suite/.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
build/
2+
__pycache__/
3+
*.pyc
4+
.DS_Store

‎examples/e2e-suite/README.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# PythonNative E2E Suite
2+
3+
A comprehensive demo app that exercises every public feature in `pythonnative`. It is the target of the top-level Maestro E2E suite and doubles as a living reference for the framework's surface area.
4+
5+
Unlike `examples/hello-world`, this app is not a marketing demo: it is structured for automated testing. Each PythonNative feature gets a dedicated screen that:
6+
7+
- Renders a stable, unique title (so Maestro can wait for the screen to appear).
8+
- Exposes interactive controls with stable, unique labels (so Maestro can tap them).
9+
- Prints a "Result:" line that reflects the feature's state (so Maestro can assert behavior, not just rendering).
10+
11+
## Running locally
12+
13+
From the repo root:
14+
15+
```bash
16+
cd examples/e2e-suite
17+
pn run android # or: pn run ios
18+
```
19+
20+
Then, in another shell:
21+
22+
```bash
23+
# Android
24+
maestro test tests/e2e/android.yaml
25+
26+
# iOS (use --platform ios if Android is also connected)
27+
maestro --platform ios test tests/e2e/ios.yaml
28+
```
29+
30+
## Adding a new feature demo
31+
32+
1. Add a screen module under `app/screens/<category>/<feature>.py` exporting a `pn.component`-decorated function.
33+
2. Register it in `app/registry.py` with a unique `id`.
34+
3. Add a Maestro flow at `tests/e2e/flows/<category>/<feature>.yaml` that opens the screen via its registry `id` and asserts the expected behavior.
35+
4. Re-run `scripts/check-e2e-coverage.py` to make sure every public symbol in `pythonnative.__all__` is covered by a flow.
36+
37+
See `tests/e2e/AGENTS.md` for a deeper tour of how AI agents should interact with this suite.

‎examples/e2e-suite/app/__init__.py‎

Whitespace-only changes.

‎examples/e2e-suite/app/main.py‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
"""E2E suite entry point.
2+
3+
Wires every demo screen from :mod:`app.registry` into the root
4+
[`Stack.Navigator`][pythonnative.create_stack_navigator]. The first
5+
route, ``"Home"``, is a categorized list of buttons that opens the
6+
rest of the demos. Each demo screen owns its own back navigation via
7+
[`use_navigation().go_back()`][pythonnative.use_navigation].
8+
9+
The stack-only architecture keeps the navigation surface flat and
10+
predictable for automated tests: every demo is reachable in exactly
11+
one push, and every back press lands the user back on ``"Home"``.
12+
"""
13+
14+
from __future__ import annotations
15+
16+
import pythonnative as pn
17+
from app.registry import DEMOS
18+
from app.screens.category import CategoryListScreen
19+
from app.screens.home import HomeScreen
20+
21+
print("[e2e-suite] main module imported")
22+
23+
Stack = pn.create_stack_navigator()
24+
25+
26+
@pn.component
27+
def App() -> pn.Element:
28+
"""Root component: a Stack with Home, every Category screen, and every demo."""
29+
return pn.NavigationContainer(
30+
Stack.Navigator(
31+
Stack.Screen("Home", component=HomeScreen, options={"title": "PythonNative E2E Suite"}),
32+
Stack.Screen(
33+
"Category",
34+
component=CategoryListScreen,
35+
options={"title": "Category"},
36+
),
37+
*(Stack.Screen(demo.id, component=demo.component, options={"title": demo.title}) for demo in DEMOS),
38+
)
39+
)

0 commit comments

Comments
 (0)