Skip to content

Commit fd1de8e

Browse files
YaleChen299chowncesangelsl
authored
Multitenant backend (#775)
* update test.exs to include optional test setting * initial schema change to account * added schema for course_registration * updated accounts query * Removed settings context and shifted sublanguages to courses context * Added initial tests for courses context and shifted settings tests over * Added initial tests for courses context controllers * Added course table changeset tests * updated course_registration context function/getters * Updated course config functions and tests * Fix filename typo * Updated endpoint url for course config update to prevent clash with assessment config and types updates * Added assesment config routing, functions and tests * Temporarily fix references to Courses context in miscellaneous files * Added context functions and tests for assessment types * adding course_registration test set * Updated courses context tests * Added update assessment types routing and routing tests * updated course_registration test * Updated get course config tests to include assessment types for the specified course * added pipeline for course_registration to assign course_reg in conn * Temporary edits to make tests compilable * Updated admin courses controller routes and test urls * Updated courses controller routes and test urls * Update admin courses controller tests * Updated courses context tests * Updated course_registrations function * Follow up on router assign_course * addded course_registration test for changeset * fix group factory, partial fix for the ecto.setup flow * updated course_registration context function test getters * Updated sourcecast context functions and tests * updated test for course_registration insert and update * updated course_registration test * updated course_registration test * Updated sourcecast tests * Updated assessment types tests * fix course_registration_test.exs * Updated sourcecast changeset * Updated stories schema and tests * upadted assessment schema and relevant test * update assessment factory * updated submissions schema with test, and added db migration * added latest_viewed_course in user and course fk to group * fix snake case * updated accounts context function and adminUserController with test * update accounts_test with get_users_by and fix queries with no groups * update userController with test(except for stories) * fix accounts test * added latest_viewed context functions with test * fix snake_case issue * update seed.exs for initial test with frontend * updated seed with latest_viewed * fix user view snake_case issue * refactor assessment_config to belong to assessment type with test and seeds * update_assessment_config: move order from url param to json boday * upadte courseName and courseShortName * update assessment_type test in user controller * update seed * update admin controller test to test updated result and fix course field names * refactor admin_course_controller test * fix change of field name in course in user view * remove grade/max_grade/adjustment in assessments context * fix assessments show & update submission_votes voter * fix assessment submit and query avenger_of? with test * updated bonus_xp logit with assessment config * update formatting and seed * Updated google claim extractor * Added github auth provider * Namespacing for different auth providers * Updated dev.secrets.exs.example * Updated seed * Update github auth config * Update user seeds * added get assessment config * updated config route, combine assessment config table * debuging for working with frontend * remove decay rate and update assessment config reorder logit with test * Updated sign_in flow and user table to allow for NULL name * refactor reorder logit with test * Added update_role and delete_user endpoints (#781) * Update views to match frontend requirements * Added update user role route * Added delete user from course * tested reorder and mass_upsert_reorder * Refactor update_role and delete_course_registration * update test cases and fix some credo issues * updated bonus_xp logic * update notifications with test * Added add_users endpoint and tests * Format * Added create course endpoint and tests * updated notification controller with test * updated assessments context functions with test(skipping contest) * updated answer controller with test * added delete assessment_config route with test * Updated provider tests * Format * Temp updates to migration file * Complete migration file * remove required in notification * fixing test seed and update with migration * update assessment controller with test + alter assessment config table * finalise assessment controller with test * Updated assessment config json to proper camelCase * Remove capitalisation of assessment types inside assessment_config changeset * update /user call * update xml parser for assessment with test * updated assessment config booleans * rename /user view field * updated /user endpoint with skippable sent * updated admin assessment controller with test * update /user call and remove settings test * finalised /user call * fix auth_controller_test * updated contest test * remove mentor from group * setting up admin grading controller test * Updated admin PUT /users endpoint * Namespace existing usernames in migration file * Updated migration file assessment type configs * Updated add users logic * updated admin grading controller with test(less grading summary) * merging * Updated devices * refactor assessment config booleans to question * code formatting * Fix migration file * Fix errors when testing with frontend * added isManuallyGraded field to admin grading endpoint * fix testcases * delete assessment config deletes relations + tested * fix assessment grading view * fix format and credo issues * Update migration file * added unique constaints for assessment number and course_id * rename user view crId to courseRegId * updated jobs and autograder with test * update assessments controller test * updated grading summary with test * fix spec and rename field * reorder grading summary * fix grader_id in migration * grading index filter by course * read all notifications in the previous course * prepare for multitenant deploy * prepare for multitenant deploy * Debug workflow * Specify --overwrite * update init.sh url * add role in /user call course array * add role in /user call course array * clean up unused code * set latest_viewed_id to nil in users for non-viewable course * update question testcases format * add migration for update testcase format * update migration * Increase upsert users and groups transaction timeout to handle large requests of up to 1000 entries * update admin assets controller test * update achievement for multitenant upgrade * fix formatting * renaming and cleaning up code * rename latest viewed course * add course_reg_id to admin user routes * update migration * update migration * update migration * update achievement migration * update contest leaderboard * update contest test * fix assessment config update * fix source chapter and variant changeset condition * fix course changeset and validation * simplify function * update delete route * fix issues * fix migration Co-authored-by: Chow En Rong <[email protected]> Co-authored-by: angelsl <[email protected]>
1 parent 1514071 commit fd1de8e

File tree

161 files changed

+8336
-3559
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

161 files changed

+8336
-3559
lines changed

.github/workflows/cd.yml

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ on:
44
branches:
55
- stable
66
- master
7+
- multitenant-deploy
78
paths:
89
- 'config/**'
910
- 'lib/**'
@@ -53,9 +54,7 @@ jobs:
5354
run: |
5455
mix deps.get
5556
- name: mix release
56-
run: |
57-
rm -f _build/prod/cadet-0.0.1.tar.gz
58-
mix release
57+
run: mix release --overwrite
5958
- name: Create release
6059
uses: marvinpinto/action-automatic-releases@latest
6160
with:

config/dev.secrets.exs.example

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,17 @@ config :cadet,
2626
# # You may need to write your own claim extractor for other providers
2727
# claim_extractor: Cadet.Auth.Providers.CognitoClaimExtractor
2828
# }},
29+
# # To use authentication with GitHub
30+
# "github" =>
31+
# {Cadet.Auth.Providers.GitHub,
32+
# %{
33+
# # A map of GitHub client_id => client_secret
34+
# clients: %{
35+
# "client_id" => "client_secret"
36+
# },
37+
# token_url: "https://github.com/login/oauth/access_token",
38+
# user_api: "https://api.github.com/user"
39+
# }},
2940
"test" =>
3041
{Cadet.Auth.Providers.Config,
3142
[

config/test.exs

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -52,22 +52,22 @@ config :cadet,
5252
token: "admin_token",
5353
code: "admin_code",
5454
name: "Test Admin",
55-
username: "admin",
56-
role: :admin
55+
username: "admin"
56+
# role: :admin
5757
},
5858
%{
5959
token: "staff_token",
6060
code: "staff_code",
6161
name: "Test Staff",
62-
username: "staff",
63-
role: :staff
62+
username: "staff"
63+
# role: :staff
6464
},
6565
%{
6666
token: "student_token",
6767
code: "student_code",
68-
name: "Test Student",
69-
username: "student",
70-
role: :student
68+
name: "student 1",
69+
username: "E1234564"
70+
# role: :student
7171
}
7272
]}
7373
},

deployment/init.sh

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,9 @@
88
set -euxo pipefail
99

1010
BASEDIR=/opt/cadet
11-
PKGURL='https://github.com/source-academy/cadet/releases/download/latest-stable/cadet-0.0.1.tar.gz'
11+
PKGURL='https://github.com/source-academy/cadet/releases/download/latest-multitenant-deploy/cadet-0.0.1.tar.gz'
1212
PKGPATH='/run/cadet-init/cadet-0.0.1.tar.gz'
13-
SVCURL=${SVCURL:-'https://raw.githubusercontent.com/source-academy/cadet/stable/deployment/cadet.service'}
13+
SVCURL=${SVCURL:-'https://raw.githubusercontent.com/source-academy/cadet/multitenant-deploy/deployment/cadet.service'}
1414
SVCPATH='/etc/systemd/system/cadet.service'
1515

1616
if [ "$EUID" -ne 0 ]; then

lib/cadet/accounts/accounts.ex

Lines changed: 59 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -6,25 +6,16 @@ defmodule Cadet.Accounts do
66

77
import Ecto.Query
88

9-
alias Cadet.Accounts.{Query, User}
9+
alias Cadet.Accounts.{Query, User, CourseRegistration}
1010
alias Cadet.Auth.Provider
1111

1212
@doc """
1313
Register new User entity using Cadet.Accounts.Form.Registration
1414
1515
Returns {:ok, user} on success, otherwise {:error, changeset}
1616
"""
17-
def register(attrs = %{username: username}, role) when is_binary(username) do
18-
attrs |> Map.put(:role, role) |> insert_or_update_user()
19-
end
20-
21-
@doc """
22-
Creates User entity with specified attributes.
23-
"""
24-
def create_user(attrs \\ %{}) do
25-
%User{}
26-
|> User.changeset(attrs)
27-
|> Repo.insert()
17+
def register(attrs = %{username: username}) when is_binary(username) do
18+
attrs |> insert_or_update_user()
2819
end
2920

3021
@doc """
@@ -53,58 +44,79 @@ defmodule Cadet.Accounts do
5344
Repo.get(User, id)
5445
end
5546

47+
@get_all_role ~w(admin staff)a
5648
@doc """
5749
Returns users matching a given set of criteria.
5850
"""
59-
def get_users(filter \\ []) do
60-
User
61-
|> join(:left, [u], g in assoc(u, :group))
62-
|> preload([u, g], group: g)
63-
|> get_users(filter)
51+
def get_users_by(filter \\ [], %CourseRegistration{course_id: course_id, role: role})
52+
when role in @get_all_role do
53+
CourseRegistration
54+
|> where([cr], cr.course_id == ^course_id)
55+
|> join(:inner, [cr], u in assoc(cr, :user))
56+
|> preload([cr, u], user: u)
57+
|> join(:left, [cr, u], g in assoc(cr, :group))
58+
|> preload([cr, u, g], group: g)
59+
|> get_users_helper(filter)
6460
end
6561

66-
defp get_users(query, []), do: Repo.all(query)
62+
defp get_users_helper(query, []), do: Repo.all(query)
6763

68-
defp get_users(query, [{:group, group} | filters]),
69-
do: query |> where([u, g], g.name == ^group) |> get_users(filters)
64+
defp get_users_helper(query, [{:group, group} | filters]),
65+
do: query |> where([cr, u, g], g.name == ^group) |> get_users_helper(filters)
7066

71-
defp get_users(query, [filter | filters]), do: query |> where(^[filter]) |> get_users(filters)
67+
defp get_users_helper(query, [filter | filters]),
68+
do: query |> where(^[filter]) |> get_users_helper(filters)
7269

7370
@spec sign_in(String.t(), Provider.token(), Provider.provider_instance()) ::
7471
{:error, :bad_request | :forbidden | :internal_server_error, String.t()} | {:ok, any}
7572
@doc """
7673
Sign in using given user ID
7774
"""
7875
def sign_in(username, token, provider) do
79-
case Repo.one(Query.username(username)) do
80-
nil ->
81-
# user is not registered in our database
82-
with {:ok, role} <- Provider.get_role(provider, token),
83-
{:ok, name} <- Provider.get_name(provider, token),
84-
{:ok, _} <- register(%{name: name, username: username}, role) do
85-
sign_in(username, name, token)
86-
else
87-
{:error, :invalid_credentials, err} ->
88-
{:error, :forbidden, err}
89-
90-
{:error, :upstream, err} ->
91-
{:error, :bad_request, err}
92-
93-
{:error, _err} ->
94-
{:error, :internal_server_error}
95-
end
96-
97-
user ->
98-
{:ok, user}
76+
user = username |> Query.username() |> Repo.one()
77+
78+
if is_nil(user) or is_nil(user.name) do
79+
# user is not registered in our database or does not have a name
80+
# (accounts pre-created by instructors do not have a name, and has to be fetched
81+
# from the auth provider during sign_in)
82+
with {:ok, name} <- Provider.get_name(provider, token),
83+
{:ok, _} <- register(%{name: name, username: username}) do
84+
sign_in(username, name, token)
85+
else
86+
{:error, :invalid_credentials, err} ->
87+
{:error, :forbidden, err}
88+
89+
{:error, :upstream, err} ->
90+
{:error, :bad_request, err}
91+
92+
{:error, _err} ->
93+
{:error, :internal_server_error}
94+
end
95+
else
96+
{:ok, user}
9997
end
10098
end
10199

102-
def update_game_states(user = %User{}, new_game_state = %{}) do
103-
case user
104-
|> User.changeset(%{game_states: new_game_state})
105-
|> Repo.update() do
106-
result = {:ok, _} -> result
107-
{:error, changeset} -> {:error, {:internal_server_error, full_error_messages(changeset)}}
100+
def update_latest_viewed(user = %User{id: user_id}, latest_viewed_course_id)
101+
when is_ecto_id(latest_viewed_course_id) do
102+
CourseRegistration
103+
|> where(user_id: ^user_id)
104+
|> where(course_id: ^latest_viewed_course_id)
105+
|> Repo.one()
106+
|> case do
107+
nil ->
108+
{:error, {:bad_request, "user is not in the course"}}
109+
110+
_ ->
111+
case user
112+
|> User.changeset(%{latest_viewed_course_id: latest_viewed_course_id})
113+
|> Repo.update() do
114+
result = {:ok, _} ->
115+
result
116+
117+
{:error, changeset} ->
118+
{:error, {:internal_server_error, full_error_messages(changeset)}}
119+
end
108120
end
109121
end
110122
end
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
defmodule Cadet.Accounts.CourseRegistration do
2+
@moduledoc """
3+
The mapping table representing the registration of a user to a course.
4+
"""
5+
use Cadet, :model
6+
7+
alias Cadet.Accounts.{Role, User}
8+
alias Cadet.Courses.{Course, Group}
9+
10+
schema "course_registrations" do
11+
field(:role, Role)
12+
field(:game_states, :map)
13+
14+
belongs_to(:group, Group)
15+
belongs_to(:user, User)
16+
belongs_to(:course, Course)
17+
18+
timestamps()
19+
end
20+
21+
@required_fields ~w(user_id course_id role)a
22+
@optional_fields ~w(game_states group_id)a
23+
24+
def changeset(course_registration, params \\ %{}) do
25+
course_registration
26+
|> cast(params, @optional_fields ++ @required_fields)
27+
|> add_belongs_to_id_from_model([:user, :group, :course], params)
28+
|> validate_required(@required_fields)
29+
|> unique_constraint(:user_id, name: :course_registrations_user_id_course_id_index)
30+
end
31+
end

0 commit comments

Comments
 (0)