6712d1d551
- docs/WORK_REPORT.md: recap of the Phase 0/1/2/3 hardening sprint — what shipped per phase, branding update, repo migration to the new full_encoach_platform repo, per-commit file-churn, verification results (tsc, build, Playwright), known gaps, and an operator checklist for production promotion. - docs/USER_MANUAL.md: end-user guide for every role (Student, Teacher, Admin, Corporate, Agent, Developer). Walks through every portal URL, the AI coach / IELTS / adaptive flows, human-in-the-loop exam review, AI prompt editor, feedback triage, GDPR Privacy Center, language + theme toggles, and troubleshooting. Made-with: Cursor
330 lines
17 KiB
Markdown
330 lines
17 KiB
Markdown
# EnCoach — Work Report
|
||
|
||
**Period covered:** Recent hardening & delivery sprint (Phases 0 → 3 of the
|
||
improvement roadmap).
|
||
**Repository of record:** `https://git.albousalh.com/yamen/full_encoach_platform`
|
||
**Primary branch:** `main` (= current `v4` tip, commit `93c530ee`).
|
||
|
||
This report summarizes what was built, fixed, and documented in the last few
|
||
days. It's organized by phase so each block maps 1:1 to the roadmap items and
|
||
the commits that shipped them.
|
||
|
||
---
|
||
|
||
## Table of contents
|
||
|
||
- [Executive summary](#executive-summary)
|
||
- [What shipped — by phase](#what-shipped--by-phase)
|
||
- [Phase 0 — Platform safety & operations](#phase-0--platform-safety--operations)
|
||
- [Phase 1 — Exam correctness & data provenance](#phase-1--exam-correctness--data-provenance)
|
||
- [Phase 2 — Performance & observability](#phase-2--performance--observability)
|
||
- [Phase 3 — Human-in-the-loop, compliance, i18n, a11y, CI](#phase-3--human-in-the-loop-compliance-i18n-a11y-ci)
|
||
- [Branding update](#branding-update)
|
||
- [Repository migration](#repository-migration)
|
||
- [Commits landed in this sprint](#commits-landed-in-this-sprint)
|
||
- [Verification results](#verification-results)
|
||
- [Known gaps & deferred items](#known-gaps--deferred-items)
|
||
- [Operator checklist for production](#operator-checklist-for-production)
|
||
|
||
---
|
||
|
||
## Executive summary
|
||
|
||
| Area | Before | After |
|
||
| -------------------------- | -------------------------------------------- | ----------------------------------------------------------------------- |
|
||
| Duplicate exam models | Two `student.attempt` models in two modules | Single canonical `encoach.student.attempt` in `encoach_scoring` |
|
||
| Auth on AI routes | Public in places | `@jwt_required` on every AI / coach endpoint |
|
||
| Health check | None | `GET /api/health` + `GET /api/health/ready` (DB + LLM probes) |
|
||
| LLM resilience | Hangs possible | `request_timeout=30s` on OpenAI calls |
|
||
| Exam submit | Straight to graded | QualityChecker + IeltsValidator → `pending_review` gate |
|
||
| RAG metadata | Missing entity/course/subject | Full provenance on every embedding + chunking for large passages |
|
||
| Question provenance | Anonymous | `model`, `prompt_hash`, `log_id` stamped on every `encoach.question` |
|
||
| LLM output validation | Raw JSON hope-for-the-best | Schema validation before DB insert |
|
||
| Response envelope | Inconsistent across controllers | `{items,total,page,size}` everywhere |
|
||
| Reports | Python loops over thousands of rows | SQL `read_group` aggregates |
|
||
| Payments | Mocked Paymob | Real Accept flow + HMAC-SHA512 webhook verification |
|
||
| JWT | Single access token only | Refresh tokens + revocation table |
|
||
| Frontend bundle | Monolithic | `React.lazy` + Vite `manualChunks` (radix/charts/icons/forms split) |
|
||
| TypeScript errors | 181 | 0 |
|
||
| i18n | Hard-coded English | `i18next` + en/ar locales + RTL auto-switch + LanguageToggle |
|
||
| Exam review | No human review step | `pending_review → publish` workflow + admin UI |
|
||
| AI prompts | Buried in Python constants | `encoach.ai.prompt` versioned templates + admin editor + render preview |
|
||
| Student feedback on AI | None | Thumbs up/down → `encoach.ai.feedback` + admin triage |
|
||
| GDPR | None | `/api/gdpr/export` + `/api/gdpr/delete` + Privacy Center page |
|
||
| Dark mode | Partial | Full `ThemeProvider` + toggle + chart tokens |
|
||
| Accessibility | Dialog warnings | `DialogDescription` on every `DialogContent` |
|
||
| CI | None | GitHub Actions: frontend (tsc, lint, build, Playwright) + backend Odoo |
|
||
| Documentation | Scattered | §21 Hardening Release in `PROJECT_SUMMARY.md` + 4 ADRs |
|
||
|
||
---
|
||
|
||
## What shipped — by phase
|
||
|
||
### Phase 0 — Platform safety & operations
|
||
|
||
Goal: make the platform *safe to run* before adding features.
|
||
|
||
1. **Merged duplicate exam models.** `encoach_scoring` is now the single home
|
||
for `encoach.student.attempt` and `encoach.student.answer`; the stale
|
||
copies in `encoach_exam_template` were deleted along with the duplicate
|
||
`/api/exam/*` controllers.
|
||
2. **Added `@jwt_required` to every AI / coach route** so no AI endpoint is
|
||
reachable without an authenticated token.
|
||
3. **Added health endpoints**
|
||
- `GET /api/health` — liveness.
|
||
- `GET /api/health/ready` — readiness (DB ping + LLM reachability).
|
||
4. **Hardened OpenAI calls** with `request_timeout=30s` to prevent hung
|
||
workers on flaky provider responses.
|
||
5. **Gated `seed_demo_data.py` raw SQL** behind an explicit
|
||
`ENCOACH_SEED_DEMO=1` env flag so a stray run in production cannot
|
||
clobber real data.
|
||
6. **Fixed `docker-compose.yml`** and shipped `odoo-docker.conf` so the
|
||
container-local run works out of the box.
|
||
7. **Published `/logo.svg`** as a deployable asset (later superseded by the
|
||
project-manager PNG — see [Branding update](#branding-update)).
|
||
8. **Promoted a canonical `cefr_mapper`** to `encoach_ai.services.cefr_mapper`
|
||
(no more two copies drifting).
|
||
9. **JWT cache TTL = 30s** with an invalidation hook on user mutations so
|
||
role/entity changes propagate within half a minute.
|
||
|
||
### Phase 1 — Exam correctness & data provenance
|
||
|
||
Goal: make every AI-generated item *trustworthy and traceable*.
|
||
|
||
1. **QualityChecker + IeltsValidator wired into exam submit.** Submitted
|
||
attempts now pass through a validation layer and are parked in
|
||
`pending_review` if anything fails IELTS rubric / quality rules, instead
|
||
of being scored blind.
|
||
2. **Populated RAG metadata** on every `encoach.vector.embedding` record:
|
||
`course_id`, `subject_id`, `entity_id`, and taxonomy tags. Queries can
|
||
now be scoped per tenant / per skill.
|
||
3. **Chunking pipeline** for content > 2000 chars keeps embeddings within
|
||
model limits and preserves passage locality.
|
||
4. **Provenance fields on `encoach.question`**: `ai_model`, `prompt_hash`,
|
||
`log_id` — so any item can be traced back to the exact prompt + run
|
||
that produced it.
|
||
5. **Schema validation of LLM output** before DB insert (jsonschema-style).
|
||
Malformed JSON no longer reaches storage.
|
||
6. **Response envelope unified** to `{items, total, page, size}` across all
|
||
controllers.
|
||
7. **Approval reject rollback** now uses a PG savepoint so a partial
|
||
failure during rejection leaves no orphaned rows.
|
||
8. **Ticket notifications** fire on status and assignee change.
|
||
9. **`new_project/` retired.** `backend/` is declared canonical;
|
||
`new_project/DEPRECATED.md` carries the redirect notice.
|
||
|
||
### Phase 2 — Performance & observability
|
||
|
||
Goal: make the platform *fast and measurable*.
|
||
|
||
1. **Reports use SQL `read_group`** instead of Python loops. The admin
|
||
Reports pages no longer scan tens of thousands of attempt rows in
|
||
Python.
|
||
2. **`X-Request-ID` middleware + structured JSON logs** so a single
|
||
request can be traced end-to-end across controllers.
|
||
3. **Prometheus-style counters** (basic in-process) exposed through the new
|
||
`encoach_api` `openapi.py` controller, which also exports a live
|
||
OpenAPI spec by scanning `@http.route` decorators at runtime.
|
||
4. **In-process caching** for hot reports and the AI narrative endpoint,
|
||
with a Redis-ready seam.
|
||
5. **Paymob Accept integration** — the full 3-step flow (auth → order →
|
||
payment key) plus an HMAC-SHA512-verified webhook endpoint that
|
||
updates `encoach.paymob.order` atomically. Credentials come from
|
||
`ir.config_parameter`.
|
||
6. **JWT refresh tokens** with a revocation table and auto-refresh on the
|
||
frontend API client.
|
||
7. **Composite DB indexes** on the hottest report / ticket / attempt paths.
|
||
8. **React.lazy + Vite `manualChunks`** split `vendor-radix`,
|
||
`vendor-charts`, `vendor-icons`, `vendor-forms`, `vendor-react`, and
|
||
`vendor-query` into their own bundles.
|
||
9. **Fixed all 181 TypeScript errors.** `npx tsc --noEmit -p
|
||
tsconfig.app.json` now passes cleanly.
|
||
|
||
### Phase 3 — Human-in-the-loop, compliance, i18n, a11y, CI
|
||
|
||
Goal: make the platform *professional and defensible*.
|
||
|
||
1. **i18n.** `i18next` + language detector + en/ar locales + RTL
|
||
auto-switch + `LanguageToggle` component wired into both role and
|
||
admin layouts.
|
||
2. **GDPR compliance.** `/api/gdpr/export` returns a JSON snapshot of the
|
||
caller's personal data. `/api/gdpr/delete` anonymizes PII, nulls
|
||
logs, deactivates the account, creates an
|
||
`encoach.gdpr.erasure.request` tombstone, and refuses admin
|
||
self-erasure. A new `PrivacyCenter` page in the student / teacher /
|
||
admin portals exposes both actions.
|
||
3. **Human-in-the-loop exam review.** `pending_review → publish`
|
||
workflow, backed by a new `review_workflow.py` controller and two new
|
||
admin pages: `ExamReviewQueue` and `ExamReviewDetail`.
|
||
4. **AI prompt management.** New `encoach.ai.prompt` model with
|
||
versioning ("one active row per key"), REST endpoints for list /
|
||
retrieve / create / activate / render-preview, and the
|
||
`AIPromptEditor` admin page.
|
||
5. **Student AI feedback loop.** `encoach.ai.feedback` with upsert
|
||
semantics per `(user, subject_type, subject_id)`, reusable
|
||
`AIFeedbackButtons` component, and the `AIFeedbackTriage` admin page
|
||
with status filters and resolve actions.
|
||
6. **Dark mode** fully wired: `ThemeProvider` from `next-themes`,
|
||
`ThemeToggle`, and chart color tokens that follow the theme.
|
||
7. **Accessibility.** `DialogDescription` added to every
|
||
`DialogContent`, fixing the Radix a11y warning across the app.
|
||
8. **CI scaffolding.** `.github/workflows/ci.yml` runs two jobs on every
|
||
push: *frontend* (tsc → lint → build → Playwright smoke) and
|
||
*backend* (Postgres 16 + `odoo:19 --test-enable --test-tags
|
||
encoach_api`).
|
||
9. **Onboarding consolidated.** Single root `README.md` plus four ADRs:
|
||
- ADR-0001 canonical directory layout
|
||
- ADR-0002 JWT refresh token flow
|
||
- ADR-0003 paginated response envelope
|
||
- ADR-0004 RAG metadata + chunking
|
||
10. **Dead code removed.** `ExamPage` marketing, `Index`, `ProfilePage`
|
||
import, and other legacy routes excised.
|
||
|
||
---
|
||
|
||
## Branding update
|
||
|
||
The project-manager-supplied `EnCoach` logo (icon + wordmark + tagline
|
||
"Unlock your potential with AI powered platform") now appears across:
|
||
|
||
- **Login page** — shown at `h-36 w-auto` so the wordmark and tagline are
|
||
legible.
|
||
- **Student / teacher sidebar** — small rounded icon when collapsed, full
|
||
wordmark at `h-14` when expanded.
|
||
- **Admin sidebar** — same collapsed / expanded behavior.
|
||
- **`og:image` meta** (social cards) — already pointed at
|
||
`/logo-icon.png`, so it picks up the new asset automatically.
|
||
|
||
The duplicated inline `<span>EnCoach</span>` text was removed from every
|
||
sidebar since the wordmark is now baked into the image.
|
||
|
||
Commit: `47d09a3c chore(branding): adopt project-manager EnCoach logo
|
||
across login and sidebars`.
|
||
|
||
---
|
||
|
||
## Repository migration
|
||
|
||
To consolidate the codebase into a single canonical repo for future work,
|
||
the entire monorepo was migrated to:
|
||
|
||
- **`https://git.albousalh.com/yamen/full_encoach_platform`**
|
||
|
||
Migration steps performed:
|
||
|
||
1. Added the new repo as the `origin` remote.
|
||
2. Pushed the current `v4` tip as `main` (default branch).
|
||
3. Pushed all six local branches: `v4`, `v3`, `frontend-v3`,
|
||
`full_stack_dev`, `feature/full-backend-v1`,
|
||
`feature/generation-page-ai-workflows`.
|
||
4. Pushed all tags (none existed locally).
|
||
5. Rebound `v4` upstream to `origin/v4`.
|
||
6. **Removed all seven prior remotes** from `.git/config`:
|
||
`backend-v3`, `frontend`, `frontend-v3`, `ip-origin`,
|
||
`mirror-monorepo`, `origin-backend`, `origin-frontend`.
|
||
|
||
The local checkout now has exactly one remote — `origin` → the new URL —
|
||
so any future `git push` / `git pull` goes there with no further config.
|
||
|
||
---
|
||
|
||
## Commits landed in this sprint
|
||
|
||
```
|
||
93c530ee chore(ci,docs): GitHub Actions, ADRs, README overhaul, §21 Hardening Release
|
||
e70a2854 feat(frontend): Phase 2/3 hardening release
|
||
dcf5ea69 feat(backend): Phase 2/3 hardening release
|
||
47d09a3c chore(branding): adopt project-manager EnCoach logo across login and sidebars
|
||
c016a522 feat(reports): replace mock Reports pages with real backend aggregates
|
||
d940db07 fix(config): align Configuration pages with platform logic
|
||
7f23127e chore(remotes,docs): rename remotes to match source-of-truth doctrine
|
||
7737f6de docs(summary): declare encoach_backend_v4 + encoach_frontend_v4 as repos of record
|
||
```
|
||
|
||
File-churn totals for the four hardening commits:
|
||
|
||
| Commit | Files changed | Lines + | Lines − |
|
||
| ----------------------- | ------------- | ------- | ------- |
|
||
| branding (logo) | 4 | +42 | −18 |
|
||
| backend hardening | 70 | +4 221 | −716 |
|
||
| frontend hardening | 75 | +3 722 | −546 |
|
||
| CI + docs + ADRs | 10 | +740 | −96 |
|
||
|
||
---
|
||
|
||
## Verification results
|
||
|
||
Ran before each commit and again after the final commit:
|
||
|
||
| Check | Result |
|
||
| ------------------------------------------------------ | ----------------------------------------- |
|
||
| `python -m compileall backend/custom_addons/encoach_*` | Pass, no syntax errors |
|
||
| `npx tsc --noEmit -p tsconfig.app.json` | Pass (0 errors, down from 181) |
|
||
| `npm run build` | Pass — 3.63 s, bundles split as expected |
|
||
| `python -m compileall seed_demo_data.py` | Pass |
|
||
| Playwright smoke (`test:e2e`) | 2 / 2 tests pass (login renders, root→/login) |
|
||
|
||
Frontend bundle profile after `manualChunks`:
|
||
|
||
```
|
||
vendor-charts 422.80 kB │ gzip: 112.73 kB
|
||
vendor-radix 318.60 kB │ gzip: 99.50 kB
|
||
index (app shell) 219.02 kB │ gzip: 65.36 kB
|
||
vendor-forms 88.31 kB │ gzip: 24.40 kB
|
||
vendor-icons 53.77 kB │ gzip: 9.83 kB
|
||
vendor-query 41.17 kB │ gzip: 12.23 kB
|
||
vendor-react 23.33 kB │ gzip: 8.60 kB
|
||
```
|
||
|
||
---
|
||
|
||
## Known gaps & deferred items
|
||
|
||
These were intentionally **not** addressed in this sprint to keep scope
|
||
bounded. Each has a clear next owner / next step.
|
||
|
||
- **P1.2 — `ir.rule` entity isolation + drop blanket `sudo()`.** Requires
|
||
a full `sudo()` audit across every module + coordinated frontend
|
||
changes. Deferred to a follow-up ticket.
|
||
- **AIFeedbackButtons inside `AiStudyCoach`.** The coach's response
|
||
currently lacks the `ai_log_id` that the feedback model keys on. The
|
||
reusable component ships, but wiring it into the coach is gated on a
|
||
small service change.
|
||
- **Redis cache backend.** The in-process cache ships; wiring it to Redis
|
||
is a config + deployment-time step and was deferred.
|
||
- **Paymob credentials.** `payment.paymob.api_key`,
|
||
`payment.paymob.integration_id`, `payment.paymob.hmac_secret`, and
|
||
`payment.paymob.iframe_id` must be set in `ir.config_parameter`
|
||
before checkout works. Left unset on purpose — no secrets in git.
|
||
- **Arabic translation coverage.** Core UI (nav, auth, privacy,
|
||
feedback, theme, common) is translated; deeper exam-builder strings
|
||
still fall through to English.
|
||
|
||
---
|
||
|
||
## Operator checklist for production
|
||
|
||
Before promoting to production, an operator should:
|
||
|
||
1. **Set Paymob credentials** in `ir.config_parameter`:
|
||
- `payment.paymob.api_key`
|
||
- `payment.paymob.integration_id`
|
||
- `payment.paymob.hmac_secret`
|
||
- `payment.paymob.iframe_id`
|
||
2. **Set the OpenAI key** in `ir.config_parameter` as `openai.api_key`
|
||
(or the matching env var on the container).
|
||
3. **Run migrations** for the new modules / models:
|
||
`encoach.ai.prompt`, `encoach.ai.feedback`,
|
||
`encoach.gdpr.erasure.request`, `encoach.paymob.order`,
|
||
`encoach.jwt.token` (refresh revocation).
|
||
4. **Verify health** from outside the container:
|
||
`curl https://<host>/api/health` → 200, and
|
||
`curl https://<host>/api/health/ready` → 200 or 503 with structured
|
||
reason.
|
||
5. **Pin the default-branch protection** on
|
||
`git.albousalh.com/yamen/full_encoach_platform` (`main`).
|
||
6. **Schedule the CI workflow** — first run happens automatically on the
|
||
next push.
|
||
7. **Smoke Playwright locally** one more time: `cd frontend && npm run
|
||
test:e2e` (requires browsers via `npm run test:e2e:install`).
|