devops/encoach_backend
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Updated this to the latest version of develop, got rid of most of the duplication, might be missing some packages in toml, needs testing

Browse Source
This commit is contained in:
Carlos Mesquita
2024-08-30 02:35:11 +01:00
parent 3cf9fa5cba
commit f92a803d96
73 changed files with 3642 additions and 2703 deletions
Download Patch File Download Diff File Expand all files Collapse all files

92
README.md
View File

@@ -1,27 +1,5 @@
# Disclaimer
Latest refactor from develop's branch commit 5d5cd21 2024-08-28
I didn't fully test all the endpoints, the main purpose of this release was for ielts-be to be async but I've also
separated logic through different layers, removed some duplication and implemented dependency injection, so there
could be errors and extensive testing is needed before even considering deploying (if you're even considering it).
The version this was refactored from was master's branch commit a4caecd 2024-06-13
# Changes
Since one of my use cases is load testing with 5000 concurrent users and ielts-be is sync, I've refactored ielts-be
into this fastapi app.
The ielts-be Dockerfile runs the container with:
```CMD exec gunicorn --bind 0.0.0.0:5000 --workers 1 --threads 8 --timeout 0 app:app```
And since gunicorn uses WSGI and ielts-be has mostly sync I/O blocking operations, everytime a request encounters
an I/O blocking operation a thread is blocked. Since this config is 1 worker with 8 threads, the container
will only be able to handle 8 concurrent requests at a time before gcloud run cold starts another instance.
Flask was built with WSGI in mind, having Quart as it's async alternative, even though you can serve Flask
with uvicorn using the [asgiref](https://pypi.org/project/asgiref/) adapter, FastAPI has better performance
than both alternatives and the sync calls would need to be modified either way.
# Endpoints
@@ -29,34 +7,38 @@ In ielts-ui I've added a wrapper to every backend request in '/src/utils/transla
new endpoints if the "BACKEND_TYPE" environment variable is set to "async", if the env variable is not present or
with another value, the wrapper will return the old endpoint.
| Method | ielts-be | This one |
|--------|--------------------------------------|------------------------------------------|
| GET | /healthcheck | /api/healthcheck |
| GET | /listening_section_1 | /api/listening/section/1 |
| GET | /listening_section_2 | /api/listening/section/2 |
| GET | /listening_section_3 | /api/listening/section/3 |
| GET | /listening_section_4 | /api/listening/section/4 |
| POST | /listening | /api/listening |
| POST | /writing_task1 | /api/grade/writing/1 |
| POST | /writing_task2 | /api/grade/writing/2 |
| GET | /writing_task1_general | /api/writing/1 |
| GET | /writing_task2_general | /api/writing/2 |
| POST | /speaking_task_1 | /api/grade/speaking/1 |
| POST | /speaking_task_2 | /api/grade/speaking/2 |
| POST | /speaking_task_3 | /api/grade/speaking/3 |
| GET | /speaking_task_1 | /api/speaking/1 |
| GET | /speaking_task_2 | /api/speaking/2 |
| GET | /speaking_task_3 | /api/speaking/3 |
| POST | /speaking | /api/speaking |
| POST | /speaking/generate_speaking_video | /api/speaking/generate_speaking_video |
| POST | /speaking/generate_interactive_video | /api/speaking/generate_interactive_video |
| GET | /reading_passage_1 | /api/reading/passage/1 |
| GET | /reading_passage_2 | /api/reading/passage/2 |
| GET | /reading_passage_3 | /api/reading/passage/3 |
| GET | /level | /api/level |
| GET | /level_utas | /api/level/utas |
| POST | /fetch_tips | /api/training/tips |
| POST | /grading_summary | /api/grade/summary |
| Method | ielts-be | This one |
|--------|--------------------------------------|---------------------------------------------|
| GET | /healthcheck | /api/healthcheck |
| GET | /listening_section_1 | /api/listening/section/1 |
| GET | /listening_section_2 | /api/listening/section/2 |
| GET | /listening_section_3 | /api/listening/section/3 |
| GET | /listening_section_4 | /api/listening/section/4 |
| POST | /listening | /api/listening |
| POST | /writing_task1 | /api/grade/writing/1 |
| POST | /writing_task2 | /api/grade/writing/2 |
| GET | /writing_task1_general | /api/writing/1 |
| GET | /writing_task2_general | /api/writing/2 |
| POST | /speaking_task_1 | /api/grade/speaking/1 |
| POST | /speaking_task_2 | /api/grade/speaking/2 |
| POST | /speaking_task_3 | /api/grade/speaking/3 |
| GET | /speaking_task_1 | /api/speaking/1 |
| GET | /speaking_task_2 | /api/speaking/2 |
| GET | /speaking_task_3 | /api/speaking/3 |
| POST | /speaking | /api/speaking |
| POST | /speaking/generate_speaking_video | /api/speaking/generate_speaking_video |
| POST | /speaking/generate_interactive_video | /api/speaking/generate_interactive_video |
| GET | /reading_passage_1 | /api/reading/passage/1 |
| GET | /reading_passage_2 | /api/reading/passage/2 |
| GET | /reading_passage_3 | /api/reading/passage/3 |
| GET | /level | /api/level |
| GET | /level_utas | /api/level/utas |
| POST | /fetch_tips | /api/training/tips |
| POST | /grading_summary | /api/grade/summary |
| POST | /grade_short_answers | /api/grade/short_answers |
| POST | /upload_level | /api/level/upload |
| POST | /training_content | /api/training/ |
| POST | /custom_level | /api/level/custom |
# Run the app
@@ -64,9 +46,7 @@ This is for Windows, creating venv and activating it may differ based on your OS
1. python -m venv env
2. env\Scripts\activate
3. pip install openai-whisper
4. pip install --upgrade numpy<2
5. pip install poetry
6. poetry install
7. python main.py
3. pip install poetry
4. poetry install
5. python app.py