encoach_be_odoo19/ODOO_MIGRATION_SRS_v2.md

# EnCoach Platform -- Odoo 19 Full Migration SRS (v2)

## Software Requirements Specification

**Version:** 2.0
**Date:** March 11, 2026
**Status:** Draft
**Supersedes:** ODOO_MIGRATION_SRS.md (v1)
**Key change from v1:** The ielts-be (FastAPI) microservice is fully eliminated. All AI/ML functionality is absorbed into custom Odoo modules.

---

## Table of Contents

1. [Executive Summary](#1-executive-summary)
2. [Odoo Module Plan](#2-odoo-module-plan)
3. [Data Models](#3-data-models-odoo-models)
4. [REST API Specification](#4-rest-api-specification)
5. [Authentication & Authorization](#5-authentication--authorization)
6. [AI/ML Services Integration](#6-aiml-services-integration)
7. [Payment Integration](#7-payment-integration)
8. [Business Rules & Workflows](#8-business-rules--workflows)
9. [Data Migration Plan](#9-data-migration-plan)
10. [Non-Functional Requirements](#10-non-functional-requirements)

---

## 1. Executive Summary

### 1.1 Platform Overview

EnCoach is an IELTS preparation and English proficiency testing platform serving students, teachers, corporate clients, and administrators. The platform provides:

- Full IELTS exam simulation (Reading, Listening, Writing, Speaking, Level)
- AI-powered grading of writing and speaking responses (OpenAI GPT-4o)
- AI-powered exam content generation (OpenAI GPT-4o)
- AI-generated audio for listening exams (AWS Polly)
- AI-generated video for speaking exams (ELAI)
- Speech-to-text transcription (OpenAI Whisper)
- AI plagiarism/detection for writing (GPTZero)
- Personalized training recommendations (FAISS + sentence-transformers + GPT)
- Multi-tenant entity (organization) management
- Classroom and assignment management
- Subscription-based access with multiple payment providers
- Support ticket system

### 1.2 Migration Scope

**Everything moves to Odoo 19.** Both backend systems are replaced:

1. **ielts-ui API routes** (Node.js, MongoDB, iron-session, Firebase Auth) -- 108 API route files, 22 MongoDB collections
2. **ielts-be** (Python, FastAPI, OpenAI, Whisper, AWS Polly, ELAI, GPTZero, FAISS) -- all AI/ML workloads

**What stays unchanged:**

- ielts-ui browser/frontend code (React components, Zustand stores, pages)
- Strapi CMS (encoachcms)
- Landing page (encoach-landing-page)

### 1.3 Target Architecture

```
ielts-ui (Next.js 14)
  └── Browser UI (unchanged)
        │
        ▼
Odoo 19 (Python + PostgreSQL)
  ├── REST API Controllers (JSON)
  ├── Business Logic (custom modules)
  ├── AI/ML Modules
  │     ├── OpenAI GPT-4o (grading + content generation)
  │     ├── OpenAI Whisper (speech-to-text, local model)
  │     ├── AWS Polly (text-to-speech for listening)
  │     ├── ELAI (AI video for speaking)
  │     ├── GPTZero (AI detection for writing)
  │     └── FAISS + sentence-transformers (training search)
  ├── PostgreSQL (all data)
  ├── Odoo Attachments (file storage)
  └── Payment Providers (Stripe, PayPal, Paymob)
```

There is no separate microservice. Odoo is the sole backend.

---

## 2. Odoo Module Plan

### 2.1 Standard Odoo Modules to Leverage

| Odoo Module | Usage |
|-------------|-------|
| `base` / `res.users` / `res.partner` | User accounts, extended with EnCoach-specific fields |
| `payment` | Payment provider framework for Stripe, PayPal, Paymob |
| `product` | Subscription packages as products |
| `mail` | Email notifications (password reset, verification, invites) |
| `queue_job` (OCA) or `ir.cron` | Background task processing for async AI grading |

### 2.2 Custom Modules to Develop

| Module | Depends On | Complexity | Description |
|--------|------------|------------|-------------|
| `encoach_core` | `base`, `mail` | Medium | User type extensions, entity management, roles, permissions, codes, invites |
| `encoach_exam` | `encoach_core` | High | Exam models for 5 modules with complex nested exercise structures |
| `encoach_classroom` | `encoach_core` | Low | Group/classroom management with participant tracking |
| `encoach_assignment` | `encoach_core`, `encoach_exam` | Medium | Assignment lifecycle (create, start, release, archive) |
| `encoach_stats` | `encoach_core`, `encoach_exam` | Medium | Exam sessions, per-exercise statistics, score tracking |
| `encoach_evaluation` | `encoach_core`, `encoach_exam`, `encoach_ai` | Medium | Async grading records for writing/speaking |
| `encoach_training` | `encoach_core`, `encoach_ai` | Medium | Training content, walkthrough, FAISS semantic search |
| `encoach_subscription` | `encoach_core`, `product`, `payment` | Medium | Packages, discounts, subscription management, Stripe/PayPal/Paymob |
| `encoach_registration` | `encoach_core` | Medium | Multi-path registration (individual, corporate), codes, invites |
| `encoach_ticket` | `encoach_core` | Low | Support tickets |
| `encoach_ai` | `base` | **Very High** | Core AI services: OpenAI client, Whisper, AWS Polly, ELAI, GPTZero, FAISS |
| `encoach_ai_grading` | `encoach_ai`, `encoach_exam` | High | Writing/speaking grading logic, rubrics, prompt templates |
| `encoach_ai_generation` | `encoach_ai`, `encoach_exam` | High | Exam content generation for all 5 modules |
| `encoach_ai_media` | `encoach_ai` | High | Audio (Polly TTS), video (ELAI), transcription (Whisper) |
| `encoach_api` | All above | High | All REST JSON controllers for frontend consumption (~60 endpoints) |

### 2.3 Module Dependency Graph

```
encoach_api
  ├── encoach_core
  │     ├── base / res.users / res.partner
  │     └── mail
  ├── encoach_exam
  │     └── encoach_core
  ├── encoach_classroom
  │     └── encoach_core
  ├── encoach_assignment
  │     ├── encoach_core
  │     └── encoach_exam
  ├── encoach_stats
  │     ├── encoach_core
  │     └── encoach_exam
  ├── encoach_evaluation
  │     ├── encoach_core
  │     ├── encoach_exam
  │     └── encoach_ai
  ├── encoach_training
  │     ├── encoach_core
  │     └── encoach_ai
  ├── encoach_subscription
  │     ├── encoach_core
  │     ├── product
  │     └── payment
  ├── encoach_registration
  │     └── encoach_core
  ├── encoach_ticket
  │     └── encoach_core
  ├── encoach_ai (core AI services)
  │     └── base
  ├── encoach_ai_grading
  │     ├── encoach_ai
  │     └── encoach_exam
  ├── encoach_ai_generation
  │     ├── encoach_ai
  │     └── encoach_exam
  └── encoach_ai_media
        └── encoach_ai
```

---

## 3. Data Models (Odoo Models)

All models from v1 remain unchanged. This section adds the AI/ML-related models that were previously managed by ielts-be.

> **Note:** For the full specification of the following models, refer to Section 3 of the v1 document (`ODOO_MIGRATION_SRS.md`). They are identical:
> `encoach.user`, `encoach.user.entity.rel`, `encoach.entity`, `encoach.role`, `encoach.group`, `encoach.exam`, `encoach.assignment`, `encoach.session`, `encoach.stat`, `encoach.evaluation`, `encoach.package`, `encoach.payment`, `encoach.subscription.payment`, `encoach.ticket`, `encoach.code`, `encoach.invite`, `encoach.permission`, `encoach.discount`, `encoach.walkthrough`, `encoach.approval.workflow`

The following models are new or significantly expanded in v2:

### 3.1 `encoach.training` (expanded)

**Source:** MongoDB `training` collection (previously managed by ielts-be)

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `user_id` | Many2one(`res.users`) | Yes | Student |
| `created_at` | Datetime | Yes | Training generation date |
| `exams` | Text (JSON) | No | JSON array of exam performance summaries: `[{ "id", "date", "performance_comment", "detailed_summary" }]` |
| `tips` | Text (JSON) | No | JSON object with categorized tips from FAISS search |
| `weak_areas` | Text (JSON) | No | JSON array: `[{ "area": "...", "comment": "..." }]` |
| `legacy_id` | Char | No | Original MongoDB ID |

### 3.2 `encoach.training.tip` (new)

**Source:** ielts-be `pathways_2_rw_with_ids.json` tips data

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `tip_id` | Char | Yes | Original tip identifier |
| `category` | Selection | Yes | `ct_focus`, `language_for_writing`, `reading_skill`, `strategy`, `writing_skill`, `word_link`, `word_partners` |
| `content` | Text | Yes | Tip text content |
| `embedding` | Binary | No | Pre-computed embedding vector (float32 array) |

### 3.3 `encoach.ai.job` (new)

Tracks async AI jobs (grading, video generation).

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `job_type` | Selection | Yes | `writing_grading`, `speaking_grading`, `video_generation` |
| `status` | Selection | Yes | `pending`, `in_progress`, `completed`, `error`. Default: `pending` |
| `user_id` | Many2one(`res.users`) | No | Requesting user |
| `evaluation_id` | Many2one(`encoach.evaluation`) | No | Related evaluation record |
| `input_data` | Text (JSON) | No | Serialized input for the AI task |
| `result_data` | Text (JSON) | No | Serialized result from the AI task |
| `error_message` | Text | No | Error details if failed |
| `created_at` | Datetime | Yes | Job creation time |
| `completed_at` | Datetime | No | Job completion time |
| `retry_count` | Integer | No | Number of retries. Default: 0 |

### 3.4 `encoach.elai.avatar` (new)

| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `name` | Char | Yes | Avatar display name |
| `avatar_code` | Char | Yes | ELAI avatar code |
| `avatar_url` | Char | No | Avatar preview image URL |
| `gender` | Selection | No | `male`, `female` |
| `canvas` | Char | No | ELAI canvas ID |
| `voice_id` | Char | No | ELAI voice ID |
| `voice_provider` | Char | No | Voice provider name |

### 3.5 Exam `parts` JSON Structure

Identical to v1. See v1 document Section 3.6.1 for the full JSON schema for Reading, Listening, Writing, Speaking, and Level exam parts.

### 3.6 Evaluation `result` JSON Structure

Identical to v1. See v1 document Section 3.10.1 for the writing and speaking grading result schemas.

---

## 4. REST API Specification

All endpoints from v1 remain unchanged in path and request/response format. The key difference is that endpoints previously marked as "proxy to ielts-be" are now handled directly by Odoo.

> **Note:** For the full specification of all non-AI endpoints, refer to Section 4 of the v1 document. They are identical: Auth (4.1), Users (4.2), Registration/Codes (4.3), Entities (4.12), Groups (4.11), Roles (4.13), Permissions (4.14), Invites (4.7), Codes (4.8), Assignments (4.10), Sessions (4.11), Stats (4.12), Payments (4.15), Packages (4.16), Discounts (4.17), Tickets (4.18), Storage (4.19), Approval Workflows (4.21).

The following endpoints were previously "proxy to ielts-be" and are now handled directly by Odoo:

### 4.1 Exam Generation Endpoints

#### `GET /api/exam/reading/{passage}`

Generate a reading passage using AI.

**Path params:** `passage` = `1` | `2` | `3`

**Query params:** `topic` (optional), `word_count` (optional, default ~500)

**Response (200):**
```json
{
  "title": "The Impact of Urbanization",
  "text": "Full passage text..."
}
```

**Implementation:** Call OpenAI GPT-4o with passage-specific prompt (see Section 6.3.1).

#### `POST /api/exam/reading/`

Generate reading exercises from a passage.

**Request:**
```json
{
  "text": "passage text",
  "exercises": [
    { "type": "multipleChoice", "quantity": 5 },
    { "type": "trueFalse", "quantity": 4 },
    { "type": "fillBlanks", "quantity": 3, "num_random_words": 1, "max_words": 3 }
  ],
  "difficulty": ["B1", "B2"]
}
```

**Response (200):** `{ "exercises": [...] }`

#### `GET /api/exam/listening/{section}`

Generate a listening dialog/monologue.

**Path params:** `section` = `1` | `2` | `3` | `4`

**Query params:** `topic` (optional), `difficulty` (optional)

**Response (200):**
```json
{
  "dialog": {
    "conversation": [
      { "name": "Sarah", "gender": "female", "text": "Hello..." },
      { "name": "James", "gender": "male", "text": "Hi..." }
    ]
  }
}
```

Sections 1 and 3 return `conversation` (dialog). Sections 2 and 4 return `monologue`.

#### `POST /api/exam/listening/media`

Generate MP3 audio from a dialog/monologue using AWS Polly TTS.

**Request:**
```json
{
  "conversation": [
    { "name": "Sarah", "gender": "female", "text": "Hello...", "voice": "Ruth" }
  ]
}
```

Or for monologue:
```json
{
  "monologue": "Full monologue text..."
}
```

**Response:** MP3 binary data (`Content-Type: audio/mpeg`)

**Implementation:** Call AWS Polly with neural engine (see Section 6.4).

#### `POST /api/exam/listening/transcribe`

Transcribe audio using Whisper.

**Request:** Multipart form with `audio` file

**Response (200):** Dialog object (conversation or monologue text)

**Implementation:** Run Whisper model locally, then use GPT-4o to clean overlapping segments (see Section 6.5).

#### `POST /api/exam/listening/instructions`

Generate MP3 for listening instructions text.

**Request:** `{ "text": "instructions text" }`

**Response:** MP3 binary data

#### `POST /api/exam/listening/`

Generate listening exercises from a dialog.

**Request:**
```json
{
  "text": "transcript text",
  "exercises": [
    { "type": "multipleChoice", "quantity": 3 },
    { "type": "writeBlanksFill", "quantity": 5 }
  ],
  "difficulty": ["B1"]
}
```

**Response (200):** `{ "exercises": [...] }`

#### `GET /api/exam/speaking/{task}`

Generate a speaking task prompt.

**Path params:** `task` = `1` | `2` | `3`

**Query params:** `topic`, `first_topic`, `second_topic`, `difficulty`

**Response (200):** Speaking part content (prompts array for Part 1/3, text block for Part 2)

#### `POST /api/exam/speaking/media`

Generate AI avatar video using ELAI.

**Request:**
```json
{
  "text": "Question text to speak",
  "avatar": "avatar_code"
}
```

**Response (200):** `{ "status": "STARTED", "result": null }` (async -- poll for completion)

#### `GET /api/exam/speaking/media/{vid_id}`

Poll for video generation status.

**Response (200):**
```json
{
  "status": "COMPLETED",
  "result": "https://elai-video-url..."
}
```

Possible statuses: `STARTED`, `IN_PROGRESS`, `COMPLETED`, `ERROR`

#### `GET /api/exam/speaking/avatars`

List available ELAI avatars.

**Response (200):** `[ { "name": "...", "avatar_code": "...", "avatar_url": "...", "gender": "..." } ]`

#### `GET /api/exam/writing/{task}`

Generate a writing task prompt.

**Path params:** `task` = `1` | `2`

**Query params:** `topic`, `difficulty`

**Response (200):** Writing task prompt text

#### `POST /api/exam/writing/{task}/attachment`

Generate an academic writing Task 1 with image attachment.

**Request:** Multipart form with `file` (chart/image) and optional `difficulty`

**Response (200):** Writing task prompt with image analysis

#### `POST /api/exam/level/`

Generate level test exercises.

**Request:**
```json
{
  "exercises": [
    { "type": "multipleChoice", "quantity": 10, "difficulty": "B1" },
    { "type": "fillBlanks", "quantity": 5, "text_size": 200, "topic": "education" }
  ],
  "difficulty": ["A2", "B1", "B2"]
}
```

**Response (200):** Generated exercises

#### `GET /api/exam/level/`

Get a pre-built level exam.

#### `GET /api/exam/level/utas`

Get a UTAS-format level exam.

#### `POST /api/exam/level/import/`

Import a level exam from file.

**Request:** Multipart form with `exercises` and optional `solutions` files

**Response (200):** Parsed exam object

#### `POST /api/exam/level/custom/`

Generate a custom level exam from JSON specification.

#### `POST /api/exam/reading/import`

Import a reading exam from Word/Excel file.

**Request:** Multipart form with `exercises` and optional `solutions` files

**Response (200):** Parsed exam object

#### `POST /api/exam/listening/import`

Import a listening exam from file.

### 4.2 Grading Endpoints

#### `POST /api/evaluate/writing`

Submit a writing answer for AI grading.

**Request:**
```json
{
  "userId": 1,
  "sessionId": 20,
  "exerciseId": "uuid",
  "question": "Write about...",
  "answer": "Student's essay text...",
  "task": 1,
  "attachment": "optional-image-url"
}
```

**Response (200):** `{ "ok": true }` (grading happens asynchronously)

**Implementation:** Creates `encoach.evaluation` and `encoach.ai.job`, then runs grading in a background thread (see Section 6.2).

#### `POST /api/evaluate/speaking`

Submit speaking audio for AI grading.

**Request:** Multipart form with `userId`, `sessionId`, `exerciseId`, `task`, and audio files (`audio_1`, `audio_2`, etc.) with corresponding `question_N` fields.

**Response (200):** `{ "ok": true }` (grading happens asynchronously)

**Implementation:** Whisper transcribes each audio, then GPT-4o grades the transcript (see Section 6.2).

#### `POST /api/evaluate/interactiveSpeaking`

Submit interactive speaking (multiple Q&A pairs) for grading.

**Request:** Same as speaking but with `question_N` and `audio_N` pairs.

#### `GET /api/evaluate/{sessionId}/{exerciseId}`

Poll for evaluation result.

**Response (200):**
```json
{
  "status": "completed",
  "result": { "...grading result..." }
}
```

#### `POST /api/grading/multiple`

Grade multiple short-answer exercises.

**Request:**
```json
{
  "text": "passage text",
  "questions": ["Q1", "Q2"],
  "answers": ["A1", "A2"]
}
```

**Response (200):** `{ "exercises": [{ "id": "...", "correct": true, "correct_answer": "..." }] }`

**Implementation:** GPT-4o evaluates each answer against the passage.

#### `POST /api/exam/grade/summary`

Generate a grading summary for a full exam session.

**Request:**
```json
{
  "sections": [
    { "code": "reading", "name": "Reading", "grade": 6.5 },
    { "code": "writing", "name": "Writing", "grade": 7.0 }
  ]
}
```

**Response (200):**
```json
{
  "sections": [
    {
      "code": "reading",
      "name": "Reading",
      "grade": 6.5,
      "evaluation": "Detailed evaluation text...",
      "suggestions": "Improvement suggestions...",
      "bullet_points": ["Focus on skimming", "Practice time management"]
    }
  ]
}
```

### 4.3 Training Endpoints

#### `POST /api/training`

Generate personalized training content.

**Request:**
```json
{
  "userID": 1,
  "stats": [{ "exam_id": "...", "date": 1710000000, "performance_comment": "...", "detailed_summary": "..." }]
}
```

**Response (200):** `{ "id": "training-record-id" }`

**Implementation:** Uses FAISS to find relevant tips, GPT to generate personalized recommendations (see Section 6.6).

#### `POST /api/training/tips`

Fetch contextual tips.

**Request:**
```json
{
  "context": "reading passage about climate change",
  "question": "What does the author suggest?",
  "answer": "student's wrong answer",
  "correct_answer": "the correct answer"
}
```

**Response (200):** `{ "tips": "Personalized tip text..." }`

#### `POST /api/transcribe`

Transcribe audio file.

**Request:** Multipart form with audio file

**Response (200):** Transcript text or dialog object

#### `POST /api/batch_users`

Bulk import users.

**Request:** `{ "makerID": "admin-id", "users": [{ ...userDTO }] }`

**Response (200):** `{ "ok": true }`

**Implementation:** In v1 this was handled by ielts-be (which imported into Firebase Auth + MongoDB). In v2, Odoo creates users directly in `res.users`. No Firebase import needed.

---

## 5. Authentication & Authorization

Identical to v1 document Section 5. Summary:

- **Recommended:** Odoo native auth with JWT tokens for API access
- **7 user roles** mapped to Odoo security groups with `ir.rule` record rules: student, teacher, corporate, admin, developer, agent, mastercorporate

---

## 6. AI/ML Services Integration

This is the new Section 6, replacing v1's "ielts-be Integration Specification." Odoo calls all external AI services directly.

### 6.1 External Service Overview

| Service | Purpose | API Type | Auth | Odoo Module |
|---------|---------|----------|------|-------------|
| **OpenAI GPT-4o** | Grading, content generation, text correction | REST API | API Key (`Authorization: Bearer`) | `encoach_ai` |
| **OpenAI GPT-3.5-turbo** | Secondary tasks (fixed text, perfect answers, summaries) | REST API | Same key | `encoach_ai` |
| **OpenAI Whisper** | Speech-to-text transcription | **Local model** (Python library) | N/A | `encoach_ai_media` |
| **AWS Polly** | Text-to-speech for listening exams | AWS SDK (boto3) | AWS Access Key + Secret | `encoach_ai_media` |
| **ELAI** | AI avatar video generation for speaking | REST API | Bearer token | `encoach_ai_media` |
| **GPTZero** | AI-generated text detection for writing | REST API | API Key (`x-api-key`) | `encoach_ai_grading` |
| **FAISS** | Semantic search over training tips | **Local library** (Python) | N/A | `encoach_training` |
| **sentence-transformers** | Embedding generation for FAISS | **Local model** (Python) | N/A | `encoach_training` |

### 6.2 Background Task Architecture

AI grading operations (writing and speaking) take 10-60 seconds. They must run asynchronously.

**Recommended approach:** Use Python threading within Odoo or the OCA `queue_job` module.

**Flow:**

1. Frontend calls `POST /api/evaluate/writing` (or speaking)
2. Odoo controller creates `encoach.evaluation` record with `status = 'pending'`
3. Odoo controller creates `encoach.ai.job` record and launches a background thread
4. Controller returns `200 OK` immediately to the frontend
5. Background thread:
   a. Calls OpenAI GPT-4o for grading (and GPTZero for AI detection in parallel)
   b. Updates `encoach.evaluation` with `status = 'completed'` and `result` JSON
   c. Updates `encoach.ai.job` with `status = 'completed'`
6. Frontend polls `GET /api/evaluate/{sessionId}/{exerciseId}` until status is `completed`

**Threading pattern (Odoo-compatible):**

```python
import threading
from odoo import api, SUPERUSER_ID

def _run_grading_in_background(dbname, evaluation_id, input_data):
    with api.Environment.manage():
        registry = odoo.registry(dbname)
        with registry.cursor() as cr:
            env = api.Environment(cr, SUPERUSER_ID, {})
            service = env['encoach.ai.grading']
            service.execute_grading(evaluation_id, input_data)

# In the controller:
thread = threading.Thread(
    target=_run_grading_in_background,
    args=(request.env.cr.dbname, evaluation.id, input_data)
)
thread.start()
```

**Alternative:** Use OCA `queue_job` for production-grade async processing with retry, monitoring, and worker management.

### 6.3 OpenAI Integration

#### 6.3.1 Client Configuration

Create a service class `EncoachOpenAIService` in `encoach_ai`:

| Parameter | Value |
|-----------|-------|
| **Library** | `openai` (Python package) |
| **Client** | `AsyncOpenAI(api_key=api_key)` or synchronous `OpenAI` |
| **API Key** | Odoo system parameter `encoach.openai_api_key` |
| **Response format** | `response_format={"type": "json_object"}` |

#### 6.3.2 Models Used

| Task | Model | Temperature |
|------|-------|-------------|
| Writing grading | `gpt-4o` | 0.1 |
| Speaking grading | `gpt-4o` | 0.1 |
| Short answer grading | `gpt-4o` | 0.1 |
| Fixed text / perfect answer | `gpt-3.5-turbo` | 0.1 |
| Grading summary | `gpt-3.5-turbo` | 0.1 |
| Reading passage generation | `gpt-4o` | 0.7 |
| Listening dialog generation | `gpt-4o` | 0.7 |
| Writing task generation | `gpt-4o` | 0.7 |
| Speaking task generation | `gpt-4o` | 0.7 |
| Level exercise generation | `gpt-4o` | 0.7 |
| Training tips | `gpt-4o` | 0.2 |
| Whisper overlap cleanup | `gpt-4o` | 0.1 |

#### 6.3.3 Writing Grading Prompts

**System message:**

`You are a helpful assistant designed to output JSON on this format: {template}`

Where `{template}` is:

```json
{
  "comment": "comment about student's response quality",
  "overall": 0.0,
  "task_response": {
    "Task Achievement": { "grade": 0.0, "comment": "..." },
    "Coherence and Cohesion": { "grade": 0.0, "comment": "..." },
    "Lexical Resource": { "grade": 0.0, "comment": "..." },
    "Grammatical Range and Accuracy": { "grade": 0.0, "comment": "..." }
  }
}
```

**User message (Task 2):**

`Evaluate the given Writing Task {task} response based on the IELTS grading system, ensuring a strict assessment that penalizes errors. Deduct points for deviations from the task, and assign a score of 0 if the response fails to address the question. Additionally, provide a detailed commentary highlighting both strengths and weaknesses in the response.\nQuestion: "{question}"\nAnswer: "{answer}"`

**User message (Task 1 General):**

Same as Task 2 plus: `Refer to the parts of the letter as: "Greeting Opener", "bullet 1", "bullet 2", "bullet 3", "closer (restate the purpose of the letter)", "closing greeting"`

**User message (Task 1 Academic with image):**

The image is sent as base64 in the message content using the vision API.

**Additional parallel tasks for writing grading:**

| Task | Model | Prompt |
|------|-------|--------|
| Perfect answer | `gpt-3.5-turbo` | `Write a perfect answer for this IELTS writing task: "{question}"` |
| Fixed text | `gpt-3.5-turbo` | `Fix the grammatical and spelling errors in this text, keeping the original meaning: "{answer}"` |
| AI detection | GPTZero API | See Section 6.7 |

#### 6.3.4 Speaking Grading Prompts

**System message:**

`You are a helpful assistant designed to output JSON on this format: {template}`

Where `{template}` is:

```json
{
  "comment": "extensive comment about answer quality",
  "overall": 0.0,
  "task_response": {
    "Fluency and Coherence": { "grade": 0.0, "comment": "extensive comment..." },
    "Lexical Resource": { "grade": 0.0, "comment": "..." },
    "Grammatical Range and Accuracy": { "grade": 0.0, "comment": "..." },
    "Pronunciation": { "grade": 0.0, "comment": "..." }
  }
}
```

**User message:**

`Evaluate the given Speaking Part {task} response based on the IELTS grading system, ensuring a strict assessment that penalizes errors. Deduct points for deviations from the task, and assign a score of 0 if the response fails to address the question. Additionally, provide detailed commentary highlighting both strengths and weaknesses in the response.`

Followed by question/answer pairs from transcription.

**Task-specific instructions:**

- Task 1: `Address the student as "you". If the answers are not 2 or 3 sentences long, warn the student that they should be.`
- Task 2: `Address the student as "you"`
- Task 3: `Address the student as "you" and pay special attention to coherence between the answers.`

**Speaking grading flow:**

1. Whisper transcribes each audio file
2. GPT-4o grades the full transcript against the rubric
3. GPT-3.5-turbo generates fixed text and perfect answer (for Task 2)
4. Results combined into the evaluation record

#### 6.3.5 Reading Passage Generation Prompts

**System message:**

`You are a helpful assistant designed to output JSON on this format: {"title": "title of the text", "text": "generated text"}`

**User message (Passage 1):**

`Generate an extensive text for IELTS Reading Passage 1, of at least {word_count} words, on the topic of "{topic}". The passage should offer a substantial amount of information relevant to the chosen subject matter. It should be fairly easy and consist of multiple paragraphs. Make sure that the generated text does not contain forbidden subjects in muslim countries.`

**User message (Passage 2):**

Same structure but: `fairly hard and consist of multiple paragraphs`

**User message (Passage 3):**

Same structure but: `very hard, present different points or theories, cite different sources, and consist of multiple paragraphs`

#### 6.3.6 Listening Dialog Generation Prompts

**Section 1 (conversation, 2 people):**

System: `{"conversation": [{"name": "name", "gender": "gender", "text": "text"}]}`

User: `Compose an authentic conversation between two individuals on the topic of "{topic}". Please include random names and genders. Include misleading discourse (dates, colors, etc.) and spelling of names. Make sure that the generated conversation does not contain forbidden subjects in muslim countries.`

**Section 2 (monologue, social):**

System: `{"monologue": "monologue"}`

User: `Generate a comprehensive monologue set in the social context of "{topic}". Make sure that the generated monologue does not contain forbidden subjects in muslim countries.`

**Section 3 (conversation, up to 4 people):**

User: `Compose an authentic and elaborate conversation between up to four individuals on the topic of "{topic}". Please include random names and genders. Make sure that the generated conversation does not contain forbidden subjects in muslim countries.`

**Section 4 (monologue, academic):**

User: `Generate a comprehensive and complex monologue on the academic subject of "{topic}". Make sure that the generated monologue does not contain forbidden subjects in muslim countries.`

#### 6.3.7 Writing Task Generation Prompts

**General Task 1:**

`Craft a prompt for an IELTS Writing Task 1 General Training exercise that instructs the student to compose a letter based on the topic of "{topic}" of {difficulty} CEFR level difficulty. The prompt should end with "In the letter you should" followed by 3 bullet points. Make sure it does not contain forbidden subjects in muslim countries.`

**General Task 2:**

`Craft a comprehensive question of {difficulty} CEFR level difficulty like the ones for IELTS Writing Task 2 General Training that directs the candidate to delve into an in-depth analysis of contrasting perspectives on the topic of "{topic}". The question should lead to an answer with either "theories", "complicated information" or be "very descriptive" on the topic.`

**Academic Task 1 (with image):**

`Analyze the uploaded image and create a detailed IELTS Writing Task 1 Academic prompt. Describe the visual type, context, and create a prompt at {difficulty} CEFR level.`

#### 6.3.8 Speaking Task Generation Prompts

**Part 1:**

`Craft 5 simple and single questions of easy difficulty for IELTS Speaking Part 1 that encourages candidates to delve deeply into personal experiences, preferences, or insights on the topic of "{first_topic}" and the topic of "{second_topic}". The questions should lead to the usage of 4 verb tenses (present perfect, present, past and future). Make sure that the generated question does not contain forbidden subjects in muslim countries.`

**Part 2:**

`Create a question of medium difficulty for IELTS Speaking Part 2 that encourages candidates to narrate a personal experience or story related to the topic of "{topic}". Include 3 prompts that guide the candidate. The prompts must not be questions. Also include a suffix like the ones in the IELTS exams that start with "And explain why". Make sure that the generated question does not contain forbidden subjects in muslim countries.`

**Part 3:**

`Formulate a set of 5 single questions of hard difficulty for IELTS Speaking Part 3 that encourage candidates to engage in a meaningful discussion on the topic of "{topic}". They must be 1 single question each and not be double-barreled questions. Make sure that the generated question does not contain forbidden subjects in muslim countries.`

#### 6.3.9 Level Exercise Generation Prompts

**Multiple Choice:**

`Generate {quantity} multiple choice questions of 4 options for an english level exam of {difficulty} CEFR level. Ensure that the questions cover a range of topics such as verb tense, subject-verb agreement, pronoun usage, sentence structure, and punctuation. Make sure every question only has 1 correct answer.`

**Fill Blanks:**

`Generate a text of at least {size} words about the topic {topic}. From the generated text choose exactly {quantity} words (cannot be sequential words), replace each with {{id}}, and generate a JSON object containing: the modified text, solutions array, words array with four options per blank.`

#### 6.3.10 Retry and Validation Logic

- Retry up to 2 times if response contains blacklisted words or missing required fields
- Blacklisted words include topics related to religion, politics, explicit content, and culturally sensitive subjects (stored in constants)
- Token limit: `4097 - input_token_count - 300` for max_tokens
- All responses expected in JSON format via `response_format={"type": "json_object"}`

### 6.4 AWS Polly Integration (TTS)

Implemented in `encoach_ai_media` module.

| Parameter | Value |
|-----------|-------|
| **Library** | `boto3` / `aioboto3` |
| **Service** | `polly` |
| **Engine** | `neural` |
| **Output format** | `mp3` |
| **Auth** | `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` (Odoo system parameters) |

**Available voices:**

| Voice | Gender | Accent |
|-------|--------|--------|
| Danielle | Female | US |
| Gregory | Male | US |
| Kevin | Male | US |
| Ruth | Female | US |
| Stephen | Male | US |
| Arthur | Male | GB |
| Olivia | Female | GB |
| Ayanda | Female | ZA |
| Aria | Female | NZ |
| Kajal | Female | IN |
| Niamh | Female | IE |

**Conversation audio generation:**

1. For each dialog line, call Polly `synthesize_speech()` with the assigned voice
2. Chunk text at sentence boundaries if > 3000 characters
3. Concatenate all MP3 segments
4. Append final message: `"This audio recording, for the listening exercise, has finished."` (voice: Stephen)

**Monologue audio generation:**

1. Select a random voice
2. Chunk text at sentence boundaries if > 3000 characters
3. Synthesize each chunk and concatenate

### 6.5 Whisper Integration (STT)

Implemented in `encoach_ai_media` module.

| Parameter | Value |
|-----------|-------|
| **Library** | `openai-whisper` (local Python package) |
| **Model size** | `base` (~1 GB, or configurable) |
| **Model instances** | 4 (for parallel transcription) |
| **Thread pool** | `ThreadPoolExecutor(max_workers=4)` |
| **Audio resampling** | 16 kHz, normalized |
| **Chunk size** | 30 seconds (480,000 samples) with 1/4 overlap |
| **Options** | `fp16=False`, `language='English'`, `verbose=False` |
| **Retries** | 3 attempts via tenacity |

**Long audio handling:**

1. Split audio into 30-second chunks with 25% overlap
2. Transcribe each chunk independently
3. Use GPT-4o to remove duplicated words at chunk boundaries
4. Join cleaned segments into final transcript

**Audio-to-dialog conversion:**

After transcription, use GPT-4o to determine whether the transcript is a conversation or monologue and output structured JSON:

```
You are a helpful assistant designed to output JSON on either one of these formats:
1 - {"dialog": [{"name": "name", "gender": "gender", "text": "text"}]}
2 - {"dialog": "text"}

A transcription of an audio file will be provided to you. Based on that transcription you will need to determine whether the transcription is a conversation or a monologue. If it is a conversation, output format 1. If it is a monologue, output format 2.
```

### 6.6 ELAI Integration (AI Video)

Implemented in `encoach_ai_media` module.

| Parameter | Value |
|-----------|-------|
| **Base URL** | `https://apis.elai.io/api/v1/videos` |
| **Auth** | Bearer token (Odoo system parameter `encoach.elai_token`) |

**Video generation flow:**

1. `POST /api/v1/videos` -- Create video with avatar, text, voice
2. `POST /api/v1/videos/render/{video_id}` -- Start rendering
3. Poll `GET /api/v1/videos/{video_id}` -- Check status (`ready`, `failed`, or in progress)
4. Return video URL when `ready`

**Avatar configuration:** Stored in `encoach.elai.avatar` model (avatar codes, voice IDs, voice providers).

### 6.7 GPTZero Integration (AI Detection)

Implemented in `encoach_ai_grading` module.

| Parameter | Value |
|-----------|-------|
| **Endpoint** | `https://api.gptzero.me/v2/predict/text` |
| **Auth** | `x-api-key` header (Odoo system parameter `encoach.gptzero_api_key`) |

**Request:**
```json
{
  "document": "student's writing text",
  "version": "",
  "multilingual": false
}
```

**Response fields used:**
- `class_probabilities` -- probability scores for human/AI/mixed
- `predicted_class` -- `human`, `ai`, or `mixed`
- `sentences[].highlight_sentence_for_ai` -- boolean per sentence

**Result stored in evaluation:** `"ai_detection": { "probability": 0.12, "predicted_class": "human" }`

### 6.8 FAISS + Sentence-Transformers (Training)

Implemented in `encoach_training` module.

| Parameter | Value |
|-----------|-------|
| **Embeddings model** | `sentence-transformers/all-MiniLM-L6-v2` |
| **Index type** | `faiss.IndexFlatL2` (one per category) |
| **Top-K results** | 5 |

**Categories:**

| Category | Description |
|----------|-------------|
| `ct_focus` | Critical thinking focus tips |
| `language_for_writing` | Language for writing tips |
| `reading_skill` | Reading skills tips |
| `strategy` | Test strategy tips |
| `writing_skill` | Writing skills tips |
| `word_link` | Word linking tips |
| `word_partners` | Word partner/collocation tips |

**Index files:** Store as Odoo attachments or on disk:
- `{category}_tips_index.faiss` -- FAISS index file per category
- `tips_metadata.pkl` -- Metadata mapping index positions to tip content

**Query flow:**

1. Encode user query using sentence-transformers: `embedding = model.encode([query])`
2. Search FAISS index: `distances, indices = index.search(embedding, top_k=5)`
3. Retrieve tip content from metadata
4. Pass tips + user performance data to GPT for personalized recommendations

**Training content generation flow:**

1. Receive user's exam stats (performance comments, detailed summaries)
2. Use GPT to identify weak areas and generate training queries
3. For each query, search FAISS index to find relevant tips
4. Use GPT to generate personalized recommendations based on tips + performance
5. Store result in `encoach.training`

### 6.9 Topic and Content Constants

Maintain in `encoach_ai` module as constants or configuration records:

**Topics for content generation:**

- `TOPICS` -- General IELTS topics (education, technology, environment, health, etc.)
- `TWO_PEOPLE_SCENARIOS` -- Listening Section 1 scenarios
- `SOCIAL_MONOLOGUE_CONTEXTS` -- Listening Section 2 contexts
- `FOUR_PEOPLE_SCENARIOS` -- Listening Section 3 scenarios
- `ACADEMIC_SUBJECTS` -- Listening Section 4 subjects

**Difficulty levels:** `["A1", "A2", "B1", "B2", "C1", "C2"]`

**Blacklisted words:** List of culturally sensitive terms that trigger retry if found in AI-generated content (religion, politics, explicit content, etc.)

### 6.10 Python Dependencies

Add to the Odoo server's Python environment:

| Package | Version | Purpose |
|---------|---------|---------|
| `openai` | >= 1.50 | OpenAI API client (GPT-4o, GPT-3.5) |
| `openai-whisper` | latest | Local Whisper model for STT |
| `boto3` | >= 1.34 | AWS SDK for Polly TTS |
| `faiss-cpu` | >= 1.7 | FAISS vector search |
| `sentence-transformers` | >= 3.0 | Embeddings for FAISS |
| `httpx` | >= 0.27 | HTTP client for ELAI, GPTZero |
| `tiktoken` | >= 0.7 | Token counting for OpenAI |
| `librosa` | >= 0.10 | Audio processing for Whisper |
| `soundfile` | >= 0.12 | Audio file I/O |
| `numpy` | >= 1.26 | Numerical operations |
| `tenacity` | >= 8.2 | Retry logic for API calls |
| `torch` | >= 2.0 | Required by Whisper and sentence-transformers |

**Server requirements for Whisper:** The Odoo server needs at least 4 GB RAM for the Whisper base model and sentence-transformers model. Consider running Whisper on a GPU for better performance, or use the OpenAI Whisper API instead of local inference.

### 6.11 Configuration (Odoo System Parameters)

| Parameter Key | Description | Example |
|--------------|-------------|---------|
| `encoach.openai_api_key` | OpenAI API key | `sk-...` |
| `encoach.aws_access_key_id` | AWS access key | `AKIA...` |
| `encoach.aws_secret_access_key` | AWS secret key | `wJal...` |
| `encoach.elai_token` | ELAI API token | `Bearer ...` |
| `encoach.gptzero_api_key` | GPTZero API key | `...` |
| `encoach.whisper_model_size` | Whisper model size | `base` (or `small`, `medium`, `large`) |
| `encoach.whisper_workers` | Number of Whisper worker threads | `4` |
| `encoach.grading_temperature` | Temperature for grading prompts | `0.1` |
| `encoach.generation_temperature` | Temperature for content generation | `0.7` |
| `encoach.tips_temperature` | Temperature for training tips | `0.2` |

---

## 7. Payment Integration

Identical to v1 document Section 7. Summary:

- **Stripe:** Checkout session creation, webhook handling, subscription extension
- **PayPal:** Order creation, capture, subscription update
- **Paymob:** Intention creation, transaction webhook verification
- Subscription logic: extend `subscriptionExpirationDate` on successful payment; propagate to entity members for corporate

---

## 8. Business Rules & Workflows

Identical to v1 document Section 8, with one addition:

### 8.1-8.7 (Unchanged from v1)

See v1 for: Registration flow, Subscription management, Entity licensing, Assignment lifecycle, Grading polling, Corporate payment activation, User change propagation.

### 8.8 AI Content Moderation (New)

All AI-generated content must be validated before being returned to the user:

1. **Blacklist check:** Scan generated text against the blacklisted words list
2. **Retry logic:** If blacklisted words are found, regenerate with the same prompt (up to 2 retries)
3. **JSON validation:** Verify the AI response is valid JSON with all required fields
4. **Score validation:** For grading, verify all scores are between 0.0 and 9.0
5. **Content length:** For passages, verify minimum word count is met

### 8.9 Batch User Import (Updated)

In v1, batch import was proxied to ielts-be (which imported into Firebase Auth). In v2:

1. Admin uploads CSV/Excel with user data
2. Odoo parses the file and creates `res.users` records directly
3. Generates temporary passwords and sends welcome emails
4. Creates registration codes and group assignments
5. No Firebase Auth involvement

---

## 9. Data Migration Plan

### 9.1 Migration Strategy

Perform a one-time data migration from both data sources:

1. **MongoDB** (used by ielts-ui) -> PostgreSQL
2. **MongoDB** (used by ielts-be) -> PostgreSQL (evaluation records, training data)
3. **Firebase Auth** users -> Odoo `res.users`

### 9.2 Collection-to-Model Mapping

Identical to v1 Section 9.2, with additional ielts-be collections:

| ielts-be MongoDB Collection | Odoo Model | Notes |
|----------------------------|------------|-------|
| `evaluation` | `encoach.evaluation` | Grading records with result JSON |
| `training` | `encoach.training` | Training content |

### 9.3 Additional Migration: AI Assets

| Asset | Source | Target |
|-------|--------|--------|
| FAISS index files | ielts-be `./faiss/` directory | Odoo server filesystem or `ir.attachment` |
| Tips metadata | `tips_metadata.pkl` | `encoach.training.tip` model records |
| Tips source data | `pathways_2_rw_with_ids.json` | `encoach.training.tip` model records |
| Avatar config | `conf.json`, `avatars.json` | `encoach.elai.avatar` model records |
| Whisper model | Downloaded at runtime | Odoo server filesystem |

### 9.4 Migration Script Requirements & Import Order

Identical to v1 Section 9.3 and 9.4, with the addition of:

22. `encoach.training.tip` (from tips JSON)
23. `encoach.elai.avatar` (from avatar config JSON)
24. FAISS index rebuild (re-embed tips into FAISS after import)

---

## 10. Non-Functional Requirements

### 10.1-10.6 (Unchanged from v1)

See v1 for: API response format, CORS, file storage, performance targets, scalability, security.

### 10.7 AI/ML Performance Requirements (New)

| Operation | Target Response Time | Notes |
|-----------|---------------------|-------|
| Reading passage generation | < 15s | Single GPT-4o call |
| Listening dialog generation | < 15s | Single GPT-4o call |
| Listening MP3 generation | < 30s | Multiple Polly calls + concatenation |
| Writing task generation | < 10s | Single GPT-4o call |
| Speaking task generation | < 10s | Single GPT-4o call |
| Level exercise generation | < 20s | Multiple GPT-4o calls |
| Writing grading (async) | < 30s total | Parallel: GPT-4o grading + GPTZero + perfect answer + fixed text |
| Speaking grading (async) | < 60s total | Sequential: Whisper transcription + GPT-4o grading |
| Short answer grading | < 10s | Single GPT-4o call |
| Transcription (1 min audio) | < 15s | Local Whisper |
| FAISS query | < 1s | Local computation |
| Video generation (async) | 1-5 min | ELAI rendering; poll for completion |

### 10.8 AI/ML Scalability (New)

- Whisper model requires ~1 GB RAM per instance (4 instances = 4 GB)
- sentence-transformers model requires ~500 MB RAM
- FAISS indices are small (< 100 MB total)
- OpenAI API has rate limits -- implement request queuing if needed
- AWS Polly has a 3000-character limit per request -- chunking is already handled
- Consider GPU acceleration for Whisper if transcription volume is high

### 10.9 AI/ML Monitoring (New)

- Log all OpenAI API calls with model, token usage, response time, and cost
- Log all Polly synthesis calls with character count
- Log all ELAI video generation requests with status and duration
- Log all Whisper transcription jobs with audio duration and processing time
- Alert on repeated grading failures (> 3 consecutive errors)
- Track OpenAI API spend against budget thresholds

### 10.10 Logging, Testing (Unchanged from v1)

See v1 Sections 10.7 and 10.8.

---

## Appendix A: Exercise Type Reference

Identical to v1 Appendix A.

## Appendix B: Grading Rubric Details

Identical to v1 Appendix B.

## Appendix C: Environment Variables

Updated to include all AI/ML service credentials:

| Variable | Description | Example |
|----------|-------------|---------|
| `ENCOACH_JWT_SECRET` | JWT signing secret | Random 256-bit string |
| `OPENAI_API_KEY` | OpenAI API key | `sk-...` |
| `AWS_ACCESS_KEY_ID` | AWS access key for Polly | `AKIA...` |
| `AWS_SECRET_ACCESS_KEY` | AWS secret key for Polly | `wJal...` |
| `ELAI_TOKEN` | ELAI API bearer token | `...` |
| `GPT_ZERO_API_KEY` | GPTZero API key | `...` |
| `WHISPER_MODEL_SIZE` | Whisper model size | `base` |
| `STRIPE_SECRET_KEY` | Stripe secret key | `sk_live_...` |
| `STRIPE_WEBHOOK_SECRET` | Stripe webhook signing secret | `whsec_...` |
| `PAYPAL_CLIENT_ID` | PayPal client ID | `AX...` |
| `PAYPAL_CLIENT_SECRET` | PayPal client secret | `EL...` |
| `PAYPAL_ACCESS_TOKEN_URL` | PayPal OAuth URL | `https://api.paypal.com/v1/oauth2/token` |
| `PAYMOB_API_KEY` | Paymob API key | `ZXlK...` |
| `PAYMOB_SECRET` | Paymob webhook secret | `...` |

## Appendix D: Glossary

Identical to v1 Appendix D, with additions:

| Term | Definition |
|------|-----------|
| **Whisper** | OpenAI's speech-to-text model, run locally on the Odoo server |
| **Polly** | AWS text-to-speech service for generating listening exam audio |
| **ELAI** | AI video generation service for creating speaking exam avatar videos |
| **GPTZero** | AI text detection service for identifying AI-generated writing submissions |
| **FAISS** | Facebook AI Similarity Search -- vector search library for finding relevant training tips |
| **sentence-transformers** | Python library for generating text embeddings used with FAISS |

## Appendix E: Custom Odoo Module Summary

| Module | Complexity | Key Models | Key Services | External APIs |
|--------|------------|------------|-------------- |---------------|
| `encoach_core` | Medium | User extensions, Entity, Role, Permission, Code, Invite | Registration, entity management | -- |
| `encoach_exam` | High | Exam (5 modules), Exercise structures | Exam CRUD, import/export | -- |
| `encoach_classroom` | Low | Group | Group management | -- |
| `encoach_assignment` | Medium | Assignment | Assignment lifecycle | -- |
| `encoach_stats` | Medium | Session, Stat | Stats tracking, PDF export | -- |
| `encoach_evaluation` | Medium | Evaluation, AI Job | Async grading coordination | -- |
| `encoach_training` | Medium | Training, Training Tip | FAISS search, tip generation | -- |
| `encoach_subscription` | Medium | Package, Payment, Subscription Payment, Discount | Subscription management | Stripe, PayPal, Paymob |
| `encoach_registration` | Medium | Code, Invite | Multi-path registration | -- |
| `encoach_ticket` | Low | Ticket | Ticket CRUD | -- |
| `encoach_ai` | **Very High** | -- | OpenAI client, prompt management, retry logic | OpenAI API |
| `encoach_ai_grading` | High | -- | Writing grading, speaking grading, short answer grading | OpenAI, GPTZero |
| `encoach_ai_generation` | High | -- | Reading/listening/writing/speaking/level generation | OpenAI |
| `encoach_ai_media` | High | ELAI Avatar | Polly TTS, Whisper STT, ELAI video | AWS Polly, ELAI, Whisper (local) |
| `encoach_api` | High | -- | REST JSON controllers (~60 endpoints) | -- |