feat(ai): LangGraph as core runtime + AI Agents/Tools console + full-demo seed
Core AI runtime - New encoach.ai.agent + encoach.ai.tool models with M2M tool binding, graph topology (simple|plan_review_revise|rag|react), model + fallback, temperature, max_tokens, response_format, max_revisions, quality checks and system prompt fields. - services/agent_runtime.py compiles a langgraph.StateGraph per agent and caches the build per (key, write_date). Emits a structured trace (output, tool_calls, retrieval_hits, revisions, quality_issues, ms, model_used, fallback_used) and auto-falls-back on rate-limit/5xx. - services/agent_tools.py registers 11 tool handlers wrapping existing services: resources.search, rubric.fetch, outcomes.fetch, student.profile, quality.cefr_check, quality.ai_detect, quality.content_gate, course_plan.save (mutates), course_plan.save_materials (mutates), scoring.grade_writing, scoring.grade_speaking. - 7 default agents seeded via data/agents_defaults.xml: course_planner, course_week_materials, exam_generator, exercise_generator, lms_tutor, writing_grader, speaking_grader. - Feature flag encoach_ai.use_langgraph_runtime (default True). - encoach_ai_course pipeline now routes through AgentRuntime when on, legacy SDK path kept as fallback. Admin UI - /admin/ai/prompts is now a tabbed Agents | Tools | Prompts console. - AIAgentsPanel: card grid + config dialog (model/temp/graph/tools/ system prompt) + built-in Test Runner showing live trace. - AIToolsPanel: registry table with category badges, mutates flag, schema viewer, edit dialog. - New /api/ai/agents* and /api/ai/tools* controller (list/get/update/ test, list-tools, toggle-tool). - Sidebar label nav.aiPrompts -> nav.aiAgents (AI Agents and Tools). - EN + AR (RTL) translations for ~80 new keys. Smart Wizard pages - /admin/quick-setup hub + CourseWizard, CoursePlanWizard, RubricWizard, ExamStructureWizard step-by-step flows. - /admin/course-plans list + detail pages. - /teacher/quick-setup mirror. Full demo seed + 8-role E2E - seed_full_demo.py adds the 5 missing user_types (approver, corporate, mastercorporate, agent, developer), activates a 2-stage exam-approval workflow with one pending request, creates a GE1-aligned 12-week B1 course plan with 6 detailed Week-1 materials (reading 400w, writing, listening 4-min script, speaking, grammar present simple vs continuous, vocabulary), and inserts sample ai.log + ai.feedback rows. - reset_demo_passwords.py forces every demo login back to canonical passwords (admin123/teacher123/student123/approver123/corporate123/ master123/agent123/dev123). - e2e_full_scenario.py: 46/46 PASS read-only API smoke across all 8 roles, including a live LangGraph round-trip on writing_grader. - e2e_approval_chain.py: 6/6 PASS mutation E2E - approver approves stage 1, admin approves stage 2, linked encoach.exam.custom flips to status=published, verified via psql. Docs - docs/PROJECT_SUMMARY.md updated to 2026-04-25: new Latest events bullets, refreshed credentials table, full sections 22 (LangGraph runtime) and 23 (full demo seed + 8-role E2E). - docs/ENCOACH_FULL_DEMO_QA_REPORT.md added with credentials, per-endpoint PASS/FAIL, mutation chain proof, LangGraph live output. - backend/GE1 Course Outline_ Fall AY25-26.pdf vendored as the reference outline the GE1 plan/materials are aligned to. Dependencies - requirements.txt: langgraph>=0.2.0, langchain-core>=0.3.0. - encoach_ai/__manifest__.py: external_dependencies updated. Made-with: Cursor
This commit is contained in:
Yamen Ahmad
189
docs/ENCOACH_FULL_DEMO_QA_REPORT.md
Normal file
189
docs/ENCOACH_FULL_DEMO_QA_REPORT.md
Normal file
@@ -0,0 +1,189 @@
|
||||
# EnCoach — Demo Seed & Full E2E QA Report
|
||||
|
||||
**Date:** 2026-04-25
|
||||
**Database:** `encoach_v2`
|
||||
**Backend:** `http://localhost:8069`
|
||||
**Frontend:** `http://localhost:5173`
|
||||
**Scope:** Seed every product user-type with believable data and run end-to-end smoke + mutation tests across all 8 roles, including the new LangGraph agent runtime.
|
||||
|
||||
---
|
||||
|
||||
## 1. What was added in this pass
|
||||
|
||||
| Artefact | Path | Purpose |
|
||||
|---|---|---|
|
||||
| Idempotent demo filler | `seed_full_demo.py` | Adds the 5 missing user types, an active 2-stage exam-approval workflow with one pending request, a rich GE1-aligned B1 course plan with full week-1 teaching materials, sample agent telemetry, and AI feedback rows |
|
||||
| Password reset helper | `reset_demo_passwords.py` | Re-applies the canonical demo passwords (idempotent, safe to re-run) |
|
||||
| Read-only role smoke test | `e2e_full_scenario.py` | Logs in as each user type and exercises the API surface they can reach |
|
||||
| Approval-chain mutation test | `e2e_approval_chain.py` | Walks the full happy path: approver approves → admin approves → exam auto-published |
|
||||
|
||||
All four scripts are idempotent — re-running them does not duplicate data, and after the seed creates 0 new records on subsequent runs.
|
||||
|
||||
---
|
||||
|
||||
## 2. Demo accounts (canonical credentials)
|
||||
|
||||
Every user type the platform supports is now represented. All accounts are activated, verified, linked to the `EnCoach Demo Academy` entity, and run inside `encoach_v2`.
|
||||
|
||||
| user_type | Login | Password | Notes |
|
||||
|---|---|---|---|
|
||||
| `admin` | `admin@encoach.test` | `admin123` | Top-level admin; final approver in the demo workflow |
|
||||
| `student` | `sarah@encoach.test` | `student123` | Has 4 exam assignments + course-plan visibility |
|
||||
| `student` | `omar@encoach.test` | `student123` | |
|
||||
| `student` | `layla@encoach.test` | `student123` | |
|
||||
| `teacher` | `khalid@encoach.test` | `teacher123` | Owner / requester of the pending approval |
|
||||
| `teacher` | `fatima@encoach.test` | `teacher123` | |
|
||||
| `teacher` (approver) | `approver@encoach.test` | `approver123` | Stage 1 of the demo approval workflow |
|
||||
| `corporate` | `corporate@encoach.test` | `corporate123` | Hits corporate stats reports |
|
||||
| `mastercorporate` | `master@encoach.test` | `master123` | Multi-entity overview |
|
||||
| `agent` | `agent@encoach.test` | `agent123` | |
|
||||
| `developer` | `dev@encoach.test` | `dev123` | Has AI agents introspection + `/api/metrics` |
|
||||
|
||||
---
|
||||
|
||||
## 3. Demo dataset snapshot (after seed)
|
||||
|
||||
| Entity | Count | Notes |
|
||||
|---|---:|---|
|
||||
| `res.users` (demo) | **11** | Covers all 7 product `user_type`s |
|
||||
| `encoach.entity` | 4 | `EnCoach Demo Academy` is the primary |
|
||||
| `op.course` | 6 | Includes new `GE1-B1` linked to the GE1 plan |
|
||||
| `op.student` | 4 | |
|
||||
| `encoach.rubric` | 4 | Writing + speaking |
|
||||
| `encoach.exam.custom` | 18 | One is `published` after the mutation E2E |
|
||||
| `encoach.exam.assignment` | 12 | |
|
||||
| `encoach.student.attempt` | 32 | |
|
||||
| `encoach.course.plan` | **3** | Includes full GE1 B1 plan |
|
||||
| `encoach.course.plan.week` | **25** | GE1 plan contributes 12 weeks |
|
||||
| `encoach.course.plan.material` | **16** | GE1 week 1: 6 detailed materials |
|
||||
| `encoach.approval.workflow` | 1 active | `Exam Approval Workflow` (id=4, 2 stages) |
|
||||
| `encoach.approval.request` | 2 | One was approved end-to-end during testing |
|
||||
| `encoach.ai.agent` | 7 | LangGraph agents seeded by `agents_defaults.xml` |
|
||||
| `encoach.ai.tool` | 11 | LangGraph tool registry |
|
||||
| `encoach.ai.log` | 2,587 | |
|
||||
| `encoach.ai.feedback` | 3 | New rows from this pass |
|
||||
|
||||
### GE1 course-plan highlight
|
||||
|
||||
A 12-week B1 course plan modelled on the UTAS *General English 1 Fall AY25-26* outline shared by the user — same skills split (10 hrs/wk Reading & Writing + 8 hrs/wk Listening & Speaking), same outcome codes (`RLO1`–`RLO6`, `WLO1`–`WLO3`, `LLO1`–`LLO3`, `SLO1`–`SLO3`, `GLO1`–`GLO2`, `VLO1`), same assessment split (60% CA / 40% FE).
|
||||
**Week 1** has six fully-fleshed materials:
|
||||
|
||||
- **Reading** — 390-word B1 passage *"A Day in Maya's Life"* + 6 comprehension questions
|
||||
- **Writing** — `~150-word weekly-routine` task with 5-step planning checklist
|
||||
- **Listening** — 3-minute monologue *"My week at college"* with 5 comprehension items
|
||||
- **Speaking** — 5-minute pair interview with success criteria
|
||||
- **Grammar** — Present simple vs continuous mini-lesson + 8 controlled-practice items
|
||||
- **Vocabulary** — 24 high-frequency items grouped by daily-routines / family / free-time
|
||||
|
||||
Weeks 2–12 are skeleton rows (date label, unit, focus) ready for the `course_week_materials` agent to fill.
|
||||
|
||||
---
|
||||
|
||||
## 4. Read-only role smoke test (`e2e_full_scenario.py`)
|
||||
|
||||
```
|
||||
Summary: 46 PASS 0 FAIL 0 SKIP
|
||||
|
||||
admin 12 pass 0 fail
|
||||
teacher 9 pass 0 fail
|
||||
approver 4 pass 0 fail
|
||||
student 6 pass 0 fail
|
||||
corporate 4 pass 0 fail
|
||||
mastercorporate 4 pass 0 fail
|
||||
agent 3 pass 0 fail
|
||||
developer 4 pass 0 fail
|
||||
```
|
||||
|
||||
### Highlights per role
|
||||
|
||||
| Role | What was exercised | Result |
|
||||
|---|---|---|
|
||||
| **admin** | login, profile, AI agents list (LangGraph), tool registry, agent detail, **live LangGraph writing-grader run**, AI prompts library, branding, approval workflows + users, user list, student-performance report | All 200 OK; live LangGraph round-trip 3.3 s; band score returned |
|
||||
| **teacher** | login, profile, course-plan list+detail+materials, courses, exam structures, exam schedules, approval-requests | All 200 OK; reads the GE1 plan + week-1 materials |
|
||||
| **approver** | login, profile, approval-request inbox (`?mine=1`), exam-review queue | All 200 OK; inbox has 1 pending |
|
||||
| **student** | login, profile, my-exams (4), my-courses, **AI coach chat (lms_tutor agent)**, coach tip | All 200 OK |
|
||||
| **corporate** | login, profile, corporate stats report, user list | All 200 OK |
|
||||
| **mastercorporate** | login, profile, multi-entity stats, user list | All 200 OK |
|
||||
| **agent** | login, profile, courses | All 200 OK |
|
||||
| **developer** | login, profile, AI agents introspection, `/api/metrics` | All 200 OK |
|
||||
|
||||
---
|
||||
|
||||
## 5. Mutation E2E — full approval chain (`e2e_approval_chain.py`)
|
||||
|
||||
```
|
||||
[1] approver login ✓
|
||||
approver inbox: 1 pending request ✓
|
||||
target: req_id=2, res_model=encoach.exam.custom, res_id=1
|
||||
[2] approver approves stage 1 ✓
|
||||
state remains 'in_progress' (advanced to next stage) ✓
|
||||
[3] admin login + verify request now in admin's inbox ✓
|
||||
[4] admin approves the FINAL stage ✓
|
||||
request state → 'approved' ✓
|
||||
[5] underlying exam auto-published by the controller ✓
|
||||
[6] student my-exams still healthy after publish (4 items) ✓
|
||||
|
||||
✓ Approval chain E2E PASSED
|
||||
```
|
||||
|
||||
### DB-side proof
|
||||
|
||||
```sql
|
||||
-- Exam was auto-published by the controller side-effect
|
||||
SELECT id, title, status FROM encoach_exam_custom WHERE id = 1;
|
||||
id | title | status
|
||||
----+--------------------+-----------
|
||||
1 | IELTS Mock Q2 2026 | published
|
||||
|
||||
-- Both stages of the workflow record the approver and the comment
|
||||
SELECT sequence, approver_id, status, comment FROM encoach_approval_stage WHERE workflow_id = 4 ORDER BY sequence;
|
||||
seq | approver_id | status | comment
|
||||
-----+-------------+----------+---------------------------------------------------------
|
||||
10 | 13 | approved | Looks fine to me — passing to admin for final sign-off.
|
||||
20 | 5 | approved | Approved — publishing exam.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. LangGraph runtime — live verification
|
||||
|
||||
Triggered through the public API by the admin smoke test.
|
||||
|
||||
| Agent | Topology | Round-trip | Outcome |
|
||||
|---|---|---:|---|
|
||||
| `writing_grader` | `simple` | 3.3 s | Returned `overall_band: 5` for a deliberately-flawed sample with `taked → took`. Tool trace empty (correct for `simple`). |
|
||||
| `lms_tutor` (run earlier) | `react` | 13 s | Two real tool calls (`resources.search`, `outcomes.fetch`), then a coherent B1 tip referencing the outcomes registry. |
|
||||
| `course_planner` | `plan_review_revise` | (covered indirectly) | Existing course-plan pipeline now routes through `AgentRuntime` when the feature flag is on. |
|
||||
|
||||
Both LangGraph topologies (single LLM node and multi-tool ReAct loop) are exercised end-to-end on the live system.
|
||||
|
||||
---
|
||||
|
||||
## 7. How to reproduce
|
||||
|
||||
```bash
|
||||
cd /Users/yamenahmad/projects2026/odoo/odoo19
|
||||
|
||||
# 1. Seed (idempotent — safe to re-run)
|
||||
.conda-envs/odoo19/bin/python odoo/odoo-bin shell -c odoo.conf -d encoach_v2 --no-http < seed_full_demo.py
|
||||
|
||||
# 2. Reset all demo passwords (idempotent)
|
||||
.conda-envs/odoo19/bin/python odoo/odoo-bin shell -c odoo.conf -d encoach_v2 --no-http < reset_demo_passwords.py
|
||||
|
||||
# 3. Read-only role smoke test
|
||||
python3 e2e_full_scenario.py
|
||||
|
||||
# 4. Mutation: full approval chain
|
||||
python3 e2e_approval_chain.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Final tally
|
||||
|
||||
- **8 user types** seeded and exercised — admin, teacher, approver, student, corporate, mastercorporate, agent, developer
|
||||
- **46 / 46** read-only smoke calls passed
|
||||
- **6 / 6** mutation steps passed in the approval chain
|
||||
- **2 LangGraph topologies** verified live (`simple` 3.3 s, `react` with real tool calls 13 s)
|
||||
- **DB invariants** confirmed via `psql` — exam auto-publish, both approval stages stored with comments
|
||||
- **Idempotency** confirmed — second run of `seed_full_demo.py` created 0 new rows
|
||||
Reference in New Issue
Block a user