feat(reports): replace mock Reports pages with real backend aggregates
Wire the three admin Reports pages (/admin/student-performance, /admin/stats-corporate, /admin/record) to a new encoach_lms_api/controllers/reports.py that aggregates from encoach.student.attempt. * /api/reports/student-performance: per-student band averages + CEFR, with entity / level / search filters. * /api/reports/stats-corporate: by_module bar, N-month trend line, CEFR distribution pie, entity comparison bar, threshold + entity filters and meta.attempts_considered for UI. * /api/reports/record: paginated attempt history with entity / user / period filters, EX-### exam codes, duration derived from start/end. * /api/reports/filters: shared picker returning only entities / students that actually have attempts. Frontend: new reports.service.ts, all three pages rewritten to hit these endpoints; Recharts graphs now read live data, CSV export added on Student Performance and Record. Seeding: seed_reports.py completes any in_progress attempts and backfills 6 months of historical attempts across 3 entities so the trend / distribution / KPI panels have meaningful data. Idempotent. Tests: test_reports_flows.py (25/25 PASS) covers shape, filters, pagination. Regressions still green: Configuration 24/24, Support 29/29, Training 26/26. Browser-verified on localhost:8080 with admin login — all 4 tabs in Stats Corporate render, Student Performance shows real students + KPIs, Record shows 28 attempts with filter + CSV export working. Docs: new §20 in PROJECT_SUMMARY.md documenting scope, artifacts, seeding, test results, and gotchas (encoach.exam.custom uses `title` not `name`; encoach.entity requires `code`; in_progress attempts are excluded from aggregates). Made-with: Cursor
This commit is contained in:
Yamen Ahmad
@@ -5,6 +5,7 @@
|
||||
> This workspace (`odoo19/`) is a **developer monorepo / working tree only** — it conveniently contains both halves side-by-side for local development and testing. The two split repos above are the **authoritative origins** for each half: every change must be published to them (via `git subtree split + push`) before the team lead can deploy. See **§6 Git Remotes & Repositories** for the exact workflow.
|
||||
|
||||
> **Latest events:**
|
||||
> - **2026-04-19 (reports section):** Built the Reports section end-to-end — the three pages `/admin/student-performance`, `/admin/stats-corporate`, `/admin/record` (previously pure hardcoded-array mocks) are now wired to real aggregated data from `encoach.student.attempt`. New `/api/reports/{student-performance,stats-corporate,record,filters}` controller (`encoach_lms_api/controllers/reports.py`) does the rollups: per-student band averages + CEFR, per-module corporate charts, trend / distribution / entity comparison, and per-user attempt history with search / level / entity / period filters and CSV export. New `seed_reports.py` completes in-progress attempts and backfills six months of historical attempts so the trend chart and KPI cards are meaningful. 25/25 API smoke passing (`test_reports_flows.py`), 24/24 Configuration + 29/29 Support + 26/26 Training regressions still green, all three pages verified live in-browser with 28 real attempts showing across 4 tabs. See §20.
|
||||
> - **2026-04-19 (remote rename):** Aligned local remote names with the new doctrine — `backend-v4 → origin-backend`, `frontend-v4 → origin-frontend`, `origin → mirror-monorepo`. The `v4` branch now tracks `mirror-monorepo/v4`. All publishing commands in §6 updated. No remote URLs changed.
|
||||
> - **2026-04-19 (repos-of-record reorganization):** Declared `encoach_backend_v4` and `encoach_frontend_v4` as the canonical origins for their respective halves; the monorepo `encoach_backend_new_v2/v4` is now a secondary working-tree mirror kept only for history/convenience. §6 rewritten around this model.
|
||||
> - **2026-04-19 (release to VPS repos):** Pushed the accumulated institutional + support + training work. Frontend canonical → `encoach_frontend_v4/main` (`b78124bb..435930a8`, 403 files, clean fast-forward). Backend canonical → `encoach_backend_v4/main` (`74d83af5..6ec68160`, 420 files, clean fast-forward). Monorepo mirror → `mirror-monorepo/v4` (`98b9837a`). `.gitignore` now excludes `pgdata_bak_*/`, `frontend/.vite/`, `frontend/dist/`, `frontend/node_modules/` so the 674 MB of local pgdata backups never leak into a push. See §6.
|
||||
@@ -1215,3 +1216,106 @@ No functional errors in the console — only React Router v7 future-flag warning
|
||||
- Progress rows are keyed to `res.users.id`, not `op.student`. Any authenticated API user (admin, teacher, student) sees only their own completions in the `summary` block, while `learners_count` / `completion_count` on the library item itself aggregate across all users.
|
||||
- The `/api/training/<kind>/<id>/progress` endpoint is **upsert**-like: the first call creates the progress row, subsequent calls update it and bump `review_count`. This lets the frontend treat it as "toggle" without worrying about a missing row.
|
||||
|
||||
## 20. Reports Section (2026-04-19)
|
||||
|
||||
Before this pass, the three pages under the sidebar "Reports" group were **pure mock UIs** — hardcoded JavaScript arrays in each component, non-functional filters, no network requests. They now aggregate real data from the attempt pipeline.
|
||||
|
||||
### 20.1 Scope
|
||||
|
||||
| Page | URL | Before | After |
|
||||
|------|-----|--------|-------|
|
||||
| Student Performance | `/admin/student-performance` | 6 hardcoded students ("Sarah Johnson", "Ahmed Hassan"…) | Per-student band averages from `encoach.student.attempt`, CEFR level derived from the latest reportable attempt, entity + level + search filters, CSV export, three KPI cards |
|
||||
| Stats Corporate | `/admin/stats-corporate` | Hardcoded bar/line/pie data | Four tabs (Overview, Trends, Distribution, Comparison) driven by a single `/api/reports/stats-corporate` rollup — bar chart of module averages (× 10), six-month trend, CEFR distribution pie, and entity comparison bar. Threshold + entity + months filters all hit the backend. |
|
||||
| Record | `/admin/record` | Four hardcoded rows | Paginated per-user attempt history with search / entity / user / period filters, real exam codes (`EX-###`), duration derived from `started_at` → `completed_at`, CSV export |
|
||||
|
||||
All three pages share a single `/api/reports/filters` endpoint that returns the entities and students that actually have attempts, so dropdowns never offer filters that would produce empty results.
|
||||
|
||||
### 20.2 Backend artifacts
|
||||
|
||||
**New controller** — `backend/custom_addons/encoach_lms_api/controllers/reports.py`:
|
||||
|
||||
| Route | Method | Purpose |
|
||||
|-------|--------|---------|
|
||||
| `/api/reports/student-performance` | GET | One row per student. Aggregates per-skill bands across all their reportable attempts, takes the CEFR level from the most recent attempt (or derives it from overall band). Filters: `entity_id`, `level`, `search`, `since`. |
|
||||
| `/api/reports/stats-corporate` | GET | Corporate rollups: `by_module` (4 module averages × 10 for the bar chart), `trend` (last N months, default 6), `distribution` (CEFR bins with palette colors), `comparison` (per-entity average × 10). Filters: `entity_id`, `threshold` (0-100), `months`, `since`. |
|
||||
| `/api/reports/record` | GET | Paginated attempt history. Same shape as `{items, total, page, size}`. Filters: `user_id`, `entity_id`, `period=day\|week\|month`, `status`, `page`, `size`. Emits a normalized `status_label` and an `EX-###` exam code. |
|
||||
| `/api/reports/filters` | GET | Lightweight picker payload: `{entities: [...], students: [...]}` — only students that have at least one attempt, only entities that exist. |
|
||||
|
||||
An attempt is considered **reportable** if `status ∈ {completed, scoring, scored, released}`. In-progress attempts are skipped from aggregates so a student with only a half-finished exam doesn't show up with zeros.
|
||||
|
||||
CEFR labels are normalized to canonical display casing (`Pre-A1 / A1 / … / C2`) via `_normalize_cefr`, and falls back to band-based bucketing via `_cefr_for_band` when `cefr_level` is null on the attempt.
|
||||
|
||||
No new models were needed — the Reports section only reads from:
|
||||
- `encoach.student.attempt` (overall / per-skill bands, cefr_level, entity_id, started/completed timestamps)
|
||||
- `encoach.exam.custom` (exam title — note: this model uses `title`, not `name`, which tripped the first version of the controller)
|
||||
- `encoach.exam.assignment` (optional, to link an attempt back to its assignment when one exists)
|
||||
- `encoach.entity`, `res.users`
|
||||
|
||||
### 20.3 Frontend artifacts
|
||||
|
||||
**New service** — `frontend/src/services/reports.service.ts`:
|
||||
|
||||
```typescript
|
||||
reportsService.studentPerformance({ entity_id, level, search, since })
|
||||
reportsService.statsCorporate({ entity_id, threshold, months, since })
|
||||
reportsService.record({ user_id, entity_id, period, status, page, size })
|
||||
reportsService.filters()
|
||||
```
|
||||
|
||||
Typed interfaces: `StudentPerformanceRow`, `StatsCorporateResponse` (with nested `StatsModuleRow` / `StatsTrendRow` / `StatsDistributionRow` / `StatsComparisonRow`), `RecordRow`, `ReportsFiltersResponse`.
|
||||
|
||||
**Rewritten pages:**
|
||||
- `frontend/src/pages/StudentPerformancePage.tsx` — dropped all hardcoded arrays; added three KPI cards (students tracked, avg overall, top performer level), a search input, Entity select populated from `/api/reports/filters`, Level select (`all / A1 / A2 / B1 / B2 / C1 / C2`), CSV export, loading + empty states. Per-row AI Grade Explainer retained.
|
||||
- `frontend/src/pages/StatsCorporatePage.tsx` — dropped all hardcoded chart data; wired every tab to the `stats-corporate` payload. Added loading + empty states. Threshold buttons (0/50/70/90%) now hit the backend. Comparison tab now renders a real entity bar chart instead of a placeholder panel. AI narrative bubbles receive live data.
|
||||
- `frontend/src/pages/RecordPage.tsx` — dropped all hardcoded records; wired filters (entity, user, period) + CSV export. Status badge variant adapts to `completed / released / scored / in_progress / scoring`.
|
||||
|
||||
### 20.4 Seeding — `seed_reports.py` (idempotent)
|
||||
|
||||
The live database had only 12 attempts, most of them in-progress with no band scores, so the Reports pages would have shown empty charts. The new script:
|
||||
|
||||
1. Ensures three entities exist: `Acme Corp (ACME)`, `Global Ltd (GLOBAL)`, `Tech Co (TECHCO)` — using `code` (required NOT NULL on `encoach.entity`) as the idempotency key.
|
||||
2. Walks every existing `in_progress` attempt and assigns it a plausible score matrix from `BAND_MATRIX`, marks it `completed`, stamps `completed_at = started_at + 120 min`, and parks it in one of the three entities if it had none.
|
||||
3. Creates up to `6 months × 4 students = 24` additional historical attempts (with a random 30% skip to keep the trend line jagged), using a small monthly `delta` so the trend chart rises over time. Each new row is guarded by a 6-day proximity lookup so re-running only fills gaps.
|
||||
4. Commits once at the end and prints the final reportable-attempt count.
|
||||
|
||||
Result after the first run: **28 reportable attempts** spread across 3 entities, 4 students, and 6 months — enough to populate every chart.
|
||||
|
||||
Run with:
|
||||
|
||||
```bash
|
||||
cd /Users/yamenahmad/projects2026/odoo/odoo19/odoo
|
||||
../.conda-envs/odoo19/bin/python odoo-bin shell -c ../odoo.conf --no-http --stop-after-init < ../seed_reports.py
|
||||
```
|
||||
|
||||
### 20.5 Smoke test — 25/25 PASS
|
||||
|
||||
`test_reports_flows.py` exercises all four endpoints:
|
||||
|
||||
| Endpoint | Checks |
|
||||
|----------|--------|
|
||||
| `/api/reports/filters` | GET returns `{entities: [...], students: [...]}` with at least one of each |
|
||||
| `/api/reports/student-performance` | LIST returns rows, row shape complete, `overall` numeric, CEFR level is a known label, `search` filter narrows to the matching student, `level` filter returns only that level, `entity_id` filter narrows rows |
|
||||
| `/api/reports/stats-corporate` | Envelope has `by_module` + `trend` + `distribution` + `comparison` + `meta`; by_module has the 4 IELTS modules; trend length == `months` (default 6, also tested with 3); distribution has per-level color; `threshold=70` drops attempts considered (28 → 14); entity filter narrows comparison; months=3 shortens trend |
|
||||
| `/api/reports/record` | Paginated list, row shape complete, `exam_code` is `EX-###`, `status_label` is titlecased, `user_id` filter scopes to that student only, `entity_id` filter scopes to that entity, `period=month` returns the in-window subset, pagination `size=5 page=1` + `page=2` both work |
|
||||
|
||||
Latest output: `Summary: 25 passed, 0 failed, 25 total`.
|
||||
|
||||
Regressions also re-run on this pass: Configuration `24/24`, Support `29/29`, Training `26/26` — all still green.
|
||||
|
||||
### 20.6 Browser verification
|
||||
|
||||
Logged in as `admin / admin` on `localhost:8080` and walked all three pages:
|
||||
|
||||
- **Student Performance** — KPI cards populated (5 students tracked, 6.3 average band), table shows real students (Sarah Ahmed, Omar Khan, Layla Nasser, TestUser WriteFlow, Admin User), Level `A2` filter narrowed to 2 rows, Entity dropdown showed real entities, CSV export triggered a download. No console errors.
|
||||
- **Stats Corporate** — Heading confirmed `(28 scored attempts)`. Overview bar chart shows 4 modules with values ~64–72. Trends line chart has 6 monthly points Oct–Apr peaking at ~85. Distribution pie: C1 11 / B2 6 / B1 5 / A2 4. Comparison bar: Global Ltd ~81, Acme Corp ~63, Tech Co ~62. AI Summary bubbles quote the real numbers. All filters responsive.
|
||||
- **Record** — Heading `(28 attempts)`, table shows real student names / exam codes (EX-001, EX-005, EX-017…) / dates / scores / statuses. User filter scoped the list to that student; CSV export worked.
|
||||
|
||||
Network tab showed `/api/reports/filters 200`, `/api/reports/record?size=100 200`, `/api/reports/student-performance 200`, `/api/reports/stats-corporate 200` — zero 4xx/5xx.
|
||||
|
||||
### 20.7 Gotchas fixed during this pass
|
||||
|
||||
- **`encoach.exam.custom` uses `title`, not `name`.** The first version of the Record endpoint threw `AttributeError: 'encoach.exam.custom' object has no attribute 'name'`. The controller now uses `getattr(exam, 'title', None) or getattr(exam, 'display_name', '')` so it survives an upstream rename.
|
||||
- **`encoach.entity` has a `NOT NULL` `code` column.** `seed_reports.py` originally tried to create entities with just `{'name': ...}` and hit a `NotNullViolation`. The seeder now passes `(name, code)` and also looks up by `code` when probing for existing rows.
|
||||
- **Both `encoach_exam_template` and `encoach_scoring` declare `encoach.student.attempt`.** The Reports controller only reads fields that exist on both definitions (`listening/reading/writing/speaking/overall_band`, `cefr_level`, `status`, `started_at`, and whichever of `completed_at` / `finished_at` is present — `_attempt_completed_at()` probes both). No field access assumes a specific addon version.
|
||||
- **In-progress attempts are excluded from the aggregates.** If you ever notice a student you expect to see missing from Student Performance or Stats Corporate, check their attempt status — `in_progress` attempts are deliberately skipped. They DO still appear in the Record page (which shows everything) unless you also pass `status` or `period` filters.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user