feat(backend): Phase 2/3 hardening release

Roadmap P0 — platform safety & ops
- Merge duplicate encoach.student.attempt/answer models into encoach_scoring
  and drop the stale encoach_exam_template copies.
- Remove duplicate /api/exam/* routes; canonicalize on one controller tree.
- Gate raw-SQL seeds in seed_demo_data.py behind an explicit env flag.
- Add /api/health and /api/health/ready (DB + LLM reachability) endpoints.
- Fix docker-compose + ship odoo-docker.conf for container-local runs.
- Enforce OpenAI request_timeout=30s and @jwt_required on all AI/coach routes.
- Promote canonical cefr_mapper to encoach_ai.services.cefr_mapper.
- JWT cache TTL=30s + invalidation hook on user mutation.

Roadmap P1 — exam correctness & data provenance
- Wire QualityChecker + IeltsValidator into exam submit with a
  pending_review gate (encoach_ai.services.question_validator).
- Populate RAG metadata (course_id, subject_id, entity_id, taxonomy) on
  encoach_vector embeddings and add a chunking pipeline (>2000 chars).
- Add provenance fields on encoach.question (model, prompt_hash, log_id)
  and validate LLM output with schema before DB insert.
- Unify response envelope to {items,total,page,size}.
- Approval reject rollback with savepoint atomicity.
- Ticket notifications on status/assignee change.

Roadmap P2 — performance & observability
- Reports: replace Python loops with SQL read_group aggregations.
- X-Request-ID middleware + structured JSON logs.
- In-process/Prometheus counters and openapi.py controller exporting a
  spec by scanning @http.route decorators.
- Paymob real checkout + HMAC-SHA512 webhook verification, backed by a
  new encoach.paymob.order model and ir.config_parameter credentials.
- JWT refresh tokens + revocation table.
- Composite DB indexes on hot report/ticket/attempt paths.

Roadmap P3 — human-in-the-loop & compliance
- Human-in-the-loop exam review workflow (pending_review → publish) with
  new review controller and status transitions.
- encoach.ai.prompt model + versioning + admin editor endpoints (one
  active version per key, render-preview dry run).
- Student feedback loop → encoach.ai.feedback (upsert per user/subject,
  admin triage + resolve endpoints).
- GDPR export (/api/gdpr/export) and right-to-erasure (/api/gdpr/delete)
  with anonymization, tombstone record, and admin-self-erasure guard.
- HttpCase smoke tests for /api/health and /api/health/ready.

Made-with: Cursor
This commit is contained in:
Yamen Ahmad
2026-04-19 14:16:09 +04:00
parent 1a0349c381
commit 3972023a30
64 changed files with 4121 additions and 701 deletions

View File

@@ -28,9 +28,17 @@ class EncoachEmbedding(models.Model):
content_text = fields.Text()
metadata_json = fields.Text(default='{}')
course_id = fields.Integer(string='Course ID', index=True)
subject_id = fields.Integer(string='Subject ID', index=True)
entity_id = fields.Integer(string='Entity ID', index=True)
taxonomy = fields.Char(string='Taxonomy', index=True)
content_hash = fields.Char(string='Content SHA256', index=True)
chunk_index = fields.Integer(string='Chunk #', default=0)
chunk_total = fields.Integer(string='Total Chunks', default=1)
_content_unique = models.Constraint(
'UNIQUE(content_type, content_id)',
'Each content item can only have one embedding.',
'UNIQUE(content_type, content_id, chunk_index)',
'Each content item chunk can only have one embedding.',
)
@api.model
@@ -82,27 +90,46 @@ class EncoachEmbedding(models.Model):
return index_all(self.env)
@api.model
def similarity_search(self, query_vector, *, content_type=None, limit=10):
"""Find similar embeddings using cosine distance."""
def similarity_search(self, query_vector, *, content_type=None, limit=10,
course_id=None, subject_id=None, entity_id=None,
taxonomy=None):
"""Find similar embeddings using cosine distance.
Optional metadata filters (course/subject/entity/taxonomy) are applied
as SQL equality predicates to narrow the candidate set before ranking.
"""
vec_str = '[' + ','.join(str(v) for v in query_vector) + ']'
where = "WHERE embedding IS NOT NULL"
params = [vec_str, limit]
conditions = ["embedding IS NOT NULL"]
extra_params = []
if content_type:
where += " AND content_type = %s"
params = [vec_str, content_type, limit]
conditions.append("content_type = %s")
extra_params.append(content_type)
if course_id:
conditions.append("course_id = %s")
extra_params.append(int(course_id))
if subject_id:
conditions.append("subject_id = %s")
extra_params.append(int(subject_id))
if entity_id:
conditions.append("entity_id = %s")
extra_params.append(int(entity_id))
if taxonomy:
conditions.append("taxonomy = %s")
extra_params.append(taxonomy)
where = "WHERE " + " AND ".join(conditions)
query = f"""
SELECT id, content_type, content_id, content_text, metadata_json,
course_id, subject_id, entity_id, taxonomy,
chunk_index, chunk_total,
1 - (embedding <=> %s::vector) AS similarity
FROM encoach_embedding
{where}
ORDER BY embedding <=> %s::vector
LIMIT %s
"""
if content_type:
self.env.cr.execute(query, (vec_str, content_type, vec_str, limit))
else:
self.env.cr.execute(query, (vec_str, vec_str, limit))
params = [vec_str] + extra_params + [vec_str, limit]
self.env.cr.execute(query, params)
results = []
for row in self.env.cr.dictfetchall():
@@ -117,6 +144,12 @@ class EncoachEmbedding(models.Model):
'content_id': row['content_id'],
'text': row['content_text'],
'metadata': metadata,
'course_id': row['course_id'] or None,
'subject_id': row['subject_id'] or None,
'entity_id': row['entity_id'] or None,
'taxonomy': row['taxonomy'] or None,
'chunk_index': row['chunk_index'],
'chunk_total': row['chunk_total'],
'similarity': round(row['similarity'], 4),
})
return results