From ae1f6c4f00950b1e5e7927369d5add891421e7a8 Mon Sep 17 00:00:00 2001 From: MisRob Date: Tue, 21 Nov 2023 07:11:44 +0100 Subject: [PATCH 1/5] Move docs from readme to docs dir to make orientation easier and free up a visible spot in the readme for the contributing guidelines copy. - Organizes content into chapters - Adds index file - Slightly tweaks some of the chapters titles and sections heading levels Doesn't remove nor rewrites content. --- README.md | 256 --------------------------------------- docs/_index.md | 26 ++++ docs/dependencies.md | 11 ++ docs/dev_tools.md | 86 +++++++++++++ docs/local_dev_docker.md | 121 ++++++++++++++++++ docs/local_dev_host.md | 149 +++++++++++++++++++++++ docs/storybook.md | 35 ++++++ 7 files changed, 428 insertions(+), 256 deletions(-) create mode 100644 docs/_index.md create mode 100644 docs/dependencies.md create mode 100644 docs/dev_tools.md create mode 100644 docs/local_dev_docker.md create mode 100644 docs/local_dev_host.md create mode 100644 docs/storybook.md diff --git a/README.md b/README.md index aa0a9ad8d5..fceab4231d 100644 --- a/README.md +++ b/README.md @@ -12,259 +12,3 @@ Kolibri Studio uses the [Django framework](https://www.djangoproject.com/) for the backend and [Vue.js](https://vuejs.org/) for the frontend. If you are looking for help setting up custom content channels, uploading and organizing resources using Kolibri Studio, please refer to the [User Guide](https://kolibri-studio.readthedocs.io/en/latest/). - -## Local development instructions -The following guide utilizes docker and docker-compose to run select services required for Studio to function. If you would rather install these services on your host, please follow the [host-setup guide](docs/host_services_setup.md). - -### Prerequisites -Please install these prerequisites, or alternatives for setting up your local development environment: -- [volta](https://docs.volta.sh/guide/getting-started) or a different node.js manager -- [pyenv](https://kolibri-dev.readthedocs.io/en/develop/howtos/installing_pyenv.html) and [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv#installation) -- [docker](https://docs.docker.com/install/) and [docker-compose](https://docs.docker.com/compose/install/) - - -### Build your python virtual environment -To determine the preferred version of Python, you can check the `runtime.txt` file: -```bash -$ cat runtime.txt -# This is the required version of Python to run Studio currently. -# This is determined by the default Python 3 version that is installed -# inside Ubuntu Bionic, which is used to build images for Studio. -# We encode it here so that it can be picked up by Github's dependabot -# to manage automated package upgrades. -python-3.9.13 -``` -Use `pyenv` to install the version of Python listed in that file, and to also set up a virtual environment: -```bash -pyenv install 3.9.13 -pyenv virtualenv 3.9.13 studio-py3.9 -pyenv activate studio-py3.9 -``` -Now you may install Studio's Python dependencies: -```bash -pip install -r requirements.txt -r requirements-dev.txt -``` -To deactivate the virtual environment, when you're finished developing on Studio for the time being: -```bash -pyenv deactivate -``` - -#### A note about dependencies on Apple Silicon M1+ -If you run into an error with `pip install` related to the `grcpio` package, it is because it currently [does not support M1 with the version for `grcpio` Studio uses](https://github.com/grpc/grpc/issues/25082). In order to fix it, you will need to add the following environmental variables before running `pip install`: -```bash -export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 -export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 -export CFLAGS="-I/opt/homebrew/opt/openssl/include" -export LDFLAGS="-L/opt/homebrew/opt/openssl/lib" -``` - -### Install frontend dependencies -Install the version of node.js supported by Studio, and install `yarn` version 1.x: -```bash -volta install node@16 -volta install yarn@1 -``` -After installing `yarn`, you may now install frontend dependencies: -```bash -yarn install -``` - -### Install and run services - -Studio requires some background services to be running: - -* Minio - a local S3 storage emulation -* PostgreSQL (postgres) - a relational database -* Redis - a fast key/value store useful for caching -* Celery - the task manager and executor, which relies on the Studio codebase - -Generally speaking, you'll want to open a separate terminal/terminal-tab to run the services. With docker and docker-compose installed, running the above services is as easy as: -```bash -make run-services -``` - -The above command may take longer the first time it's run. It includes starting the `celery` workers, and the other dependent services through docker, which can be done separately with the following two commands: - -```bash -make dcservicesup -make devceleryworkers -``` - -To confirm that docker-based services are running, you should see three containers when executing `docker ps`. For example: - -```bash -> docker ps -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -e09c5c203b93 redis:6.0.9 "docker-entrypoint.sā€¦" 51 seconds ago Up 49 seconds 0.0.0.0:6379->6379/tcp studio_vue-refactor_redis_1 -6164371efb6b minio/minio "minio server /data" 51 seconds ago Up 49 seconds 0.0.0.0:9000->9000/tcp studio_vue-refactor_minio_1 -c86bbfa3a59e postgres:12.10 "docker-entrypoint.sā€¦" 51 seconds ago Up 49 seconds 0.0.0.0:5432->5432/tcp studio_vue-refactor_postgres_1 -``` - -To stop the services, press Ctrl + C in the terminal where you ran `make run-services` (or `dcservicesup`). Once you've done that, you may run the following command to remove the docker containers (they will be recreated when you run `run-services` or `dcservicesup` again): -```bash -make dcservicesdown -``` - -### Initializing Studio -With the services running, in a separate terminal/terminal-tab, we can now initialize the database for Studio development purposes. The command below will initialize the database tables, import constants, and a user account for development: -```bash -yarn run devsetup -``` - -### Running the development server -With the services running, in a separate terminal/terminal-tab, and the database initialized, we can start the dev server: -```bash -yarn run devserver:hot # with Vue hot module reloading -# or -yarn run devserver # without hot module reloading -``` - -Either of the above commands will take a few moments to build the frontend. When it finishes, you can sign in with the account created by the `yarn run devsetup` command: -- url: `http://localhost:8080/accounts/login/` -- username: `a@a.com` -- password: `a` - -### Running the celery service -Studio uses `celery` for executing asynchronous tasks, which are integral to Studio's channel editing architecture. The celery service does not reload when there are Python changes like the Django devserver does, so it's often preferred to run it separately. If you are developing changes against a task or the celery configuration, you'll need to use `make dcservicesup` to run only the docker-based services. - -In a separate terminal/terminal-tab, run the following to start the service and press Ctrl + C to stop it: -```bash -make devceleryworkers -``` - -Stop and restart the above to reload your changes. - -## Adding or updating dependencies - -We use `pip-tools` to ensure all our dependencies use the same versions on all deployments. - -To add a dependency, add it to either `requirements.in` or `requirements-dev.in`, then -run `pip-compile requirements[-dev|-docs].in` to generate the .txt file. Please make sure that -both the `.in` and `.txt` file changes are part of the commit when updating dependencies. - -To update a dependency, use `pip-compile --upgrade-package [package-name] requirements[-dev|-docs].in` - -For more details, please see the [pip-tools docs on Github](https://github.com/jazzband/pip-tools). - -## Additional tools - -### Running tests - -With Studio's services running, you may run tests with the following commands: - -```bash -# backend -make test -# frontend -yarn run test -``` - -View [more testing tips](docs/running_tests.md) - -### Linting - -Front-end linting is run using: - -```bash -yarn run lint-frontend -``` - -Some linting errors can be fixed automatically by running: - -```bash -yarn run lint-frontend:format -``` - -Make sure you've set up pre-commit hooks as described above. This will ensure that linting is automatically run on staged changes before every commit. - -### Profiling and local production testing - -If you want to test the performance of your changes, you can start up a local server with settings closer to a production environment like so: - -```bash -# build frontend dependencies -yarn run build -# run the server (no webpack) -yarn run runserver -# or for profiling production more closely -yarn run runserver:prod-profiling -``` - -Once the local production server is running, you can also use Locust to test your changes under scenarios of high demand like so: - -```bash -cd deploy/chaos/loadtest -make timed_run -make stop_slaves # mac: killall python -``` - -#### Profiling - -In case you need to profile the application to know which part of the code are more time consuming, there are two different profilers available to work in two different modes. Both will store the profiling output in a directory that's determined by the `PROFILE_DIR` env variable. If this variable is not set, the output files will be store in a folder called profiler inside the OS temp folder (`/tmp/profile` usually) -Note that both profiling modes are incompatible: you can either use one or the other, but not both at the same time. In case the env variables are set for both modes, _All request profiling mode_ will be used. - -##### All requests profiling mode - -This mode will create interactive html files with all the profiling information for every request the Studio server receives. The name of the files will contain the total execution time, the endpoint name and a timestamp. - -To activate it an env variable called `PROFILE_STUDIO_FULL` must be set. - -Example of use: - -`PROFILE_STUDIO_FULL=y yarn runserver` - -Afterwards no further treatment of the generated files is needed. You can open directly the html files in your browser. - -##### Endpoint profiling mode - -When using the _all requests mode_ it's usual that the profile folder is soon full of information for requests that are not interesting for the developer, obscuring the files for specific endpoints. - -If an env variable called `PROFILE_STUDIO_FILTER` is used, the profiler will be executed only on the http requests containing the text stated by the variable. - -Example of use: - -`PROFILE_STUDIO_FILTER=edit yarn localprodserver` - -For this case, only html requests having the text _edit_ in their request path will be profiled. The profile folder will not have html files, but binary dump files (with the timestamp as filename) of the profiler information that can be later seen by different profiling tools (`snakeviz` that can be installed using pip is recommended). Also while the server is running, the ten most time consuming lines of code of the filtered request will be shown in the console where Studio has been launched. - -Example of snakeviz use: - -`snakeviz /tmp/profile/studio\:20200909161405011678.prof` - -will open the browser with an interactive diagram with all the profiling information - -### Storybook - -Storybook is a development environment for UI components. If this is your first encounter with this tool, you can check [this presentation](https://docs.google.com/presentation/d/10JL4C9buygWsTbT62Ym149Yh9zSR9nY_ZqFumBKUY0o/edit?usp=sharing) or [its website](https://storybook.js.org/). You are encouraged to use it any time you need to develop a new UI component. It is especially suitable for smaller to middle size components that represent basic UI building blocks. - -An example is worth a thousand words so please have a look at these simple [stories of an example component](./contentcuration/contentcuration/frontend/shared/views/details/DetailsRow.stories.js) to see how to write yours. For detailed information on writing stories you can [go through this tutorial](https://www.learnstorybook.com/intro-to-storybook/). - -You can also check [official addons](https://storybook.js.org/addons/). - -**Run development server** - -```bash -yarn run storybook -``` - -With detailed webpack information (useful when debugging loaders, addons and similar): - -```bash -yarn run storybook:debug -``` - -**Bundle** - -```bash -yarn run storybook:build -``` - -The output is saved to *storybook-static/*. - -### Current usage notes - -We've decided not to push our stories to the codebase and keep them locally in the near future. Although this limits the number of advantages Storybook provides, it allows us to start using it as soon as possible without the need to agree on all conventions and it also gives the whole team enough time to test the development workflow so we can decide later if we want to adopt this tool in a larger scale. - -Taking into account the above-mentioned, all stories except of example *DetailsRow.stories.js* will be ignored by git as long as you use a naming convention for Storybook source files: *\*.stories.js*. - -Although we don't share stories at this point, Storybook is installed and configured in the codebase to prevent the need for everyone to configure everything locally. If you update Storybook Webpack settings, install a new plugin and similar, you are welcome to share such updates with other members of the team. diff --git a/docs/_index.md b/docs/_index.md new file mode 100644 index 0000000000..c7214d4ba3 --- /dev/null +++ b/docs/_index.md @@ -0,0 +1,26 @@ +# Studio Developer Documentation + +## Local development guides + +- [Local development instructions: With Docker](./local_dev_docker.md) +- [Local development instructions: Run everything on your host machine](./local_dev_host.md) +- [Local development tools](./dev_tools.md) +- [Running tests](./running_tests.md) +- [Adding or updating dependencies](./dependencies.md) + +## Additional development tools + +- [Storybook](./storybook.md) + +## Deployment + +- [Docker + Kubernetes Studio Instance Setup](./docker_kubernetes_setup.md) + +## API + +- [API Endpoints](./api_endpoints.md) + +## Architecture and components documentation + +- [Studio Architecture](./architecture.md) +- [Markdown Editor/Viewer](./markdown_editor_viewer.md) diff --git a/docs/dependencies.md b/docs/dependencies.md new file mode 100644 index 0000000000..9b27056ce4 --- /dev/null +++ b/docs/dependencies.md @@ -0,0 +1,11 @@ +# Adding or updating dependencies + +We use `pip-tools` to ensure all our dependencies use the same versions on all deployments. + +To add a dependency, add it to either `requirements.in` or `requirements-dev.in`, then +run `pip-compile requirements[-dev|-docs].in` to generate the .txt file. Please make sure that +both the `.in` and `.txt` file changes are part of the commit when updating dependencies. + +To update a dependency, use `pip-compile --upgrade-package [package-name] requirements[-dev|-docs].in` + +For more details, please see the [pip-tools docs on Github](https://github.com/jazzband/pip-tools). diff --git a/docs/dev_tools.md b/docs/dev_tools.md new file mode 100644 index 0000000000..7d23039f25 --- /dev/null +++ b/docs/dev_tools.md @@ -0,0 +1,86 @@ +# Local development tools + +## Running tests + +With Studio's services running, you may run tests with the following commands: + +```bash +# backend +make test +# frontend +yarn run test +``` + +View [more testing tips](./running_tests.md) + +## Linting + +Front-end linting is run using: + +```bash +yarn run lint-frontend +``` + +Some linting errors can be fixed automatically by running: + +```bash +yarn run lint-frontend:format +``` + +Make sure you've set up pre-commit hooks as described above. This will ensure that linting is automatically run on staged changes before every commit. + +## Profiling and local production testing + +If you want to test the performance of your changes, you can start up a local server with settings closer to a production environment like so: + +```bash +# build frontend dependencies +yarn run build +# run the server (no webpack) +yarn run runserver +# or for profiling production more closely +yarn run runserver:prod-profiling +``` + +Once the local production server is running, you can also use Locust to test your changes under scenarios of high demand like so: + +```bash +cd deploy/chaos/loadtest +make timed_run +make stop_slaves # mac: killall python +``` + +### Profiling + +In case you need to profile the application to know which part of the code are more time consuming, there are two different profilers available to work in two different modes. Both will store the profiling output in a directory that's determined by the `PROFILE_DIR` env variable. If this variable is not set, the output files will be store in a folder called profiler inside the OS temp folder (`/tmp/profile` usually) +Note that both profiling modes are incompatible: you can either use one or the other, but not both at the same time. In case the env variables are set for both modes, _All request profiling mode_ will be used. + +#### All requests profiling mode + +This mode will create interactive html files with all the profiling information for every request the Studio server receives. The name of the files will contain the total execution time, the endpoint name and a timestamp. + +To activate it an env variable called `PROFILE_STUDIO_FULL` must be set. + +Example of use: + +`PROFILE_STUDIO_FULL=y yarn runserver` + +Afterwards no further treatment of the generated files is needed. You can open directly the html files in your browser. + +#### Endpoint profiling mode + +When using the _all requests mode_ it's usual that the profile folder is soon full of information for requests that are not interesting for the developer, obscuring the files for specific endpoints. + +If an env variable called `PROFILE_STUDIO_FILTER` is used, the profiler will be executed only on the http requests containing the text stated by the variable. + +Example of use: + +`PROFILE_STUDIO_FILTER=edit yarn localprodserver` + +For this case, only html requests having the text _edit_ in their request path will be profiled. The profile folder will not have html files, but binary dump files (with the timestamp as filename) of the profiler information that can be later seen by different profiling tools (`snakeviz` that can be installed using pip is recommended). Also while the server is running, the ten most time consuming lines of code of the filtered request will be shown in the console where Studio has been launched. + +Example of snakeviz use: + +`snakeviz /tmp/profile/studio\:20200909161405011678.prof` + +will open the browser with an interactive diagram with all the profiling information diff --git a/docs/local_dev_docker.md b/docs/local_dev_docker.md new file mode 100644 index 0000000000..7cba837368 --- /dev/null +++ b/docs/local_dev_docker.md @@ -0,0 +1,121 @@ +# Local development instructions: With Docker + +The following guide utilizes docker and docker-compose to run select services required for Studio to function. If you would rather install these services on your host, please follow the [host-setup guide](./local_dev_host.md). + +## Prerequisites +Please install these prerequisites, or alternatives for setting up your local development environment: +- [volta](https://docs.volta.sh/guide/getting-started) or a different node.js manager +- [pyenv](https://kolibri-dev.readthedocs.io/en/develop/howtos/installing_pyenv.html) and [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv#installation) +- [docker](https://docs.docker.com/install/) and [docker-compose](https://docs.docker.com/compose/install/) + + +## Build your python virtual environment +To determine the preferred version of Python, you can check the `runtime.txt` file: +```bash +$ cat runtime.txt +# This is the required version of Python to run Studio currently. +# This is determined by the default Python 3 version that is installed +# inside Ubuntu Bionic, which is used to build images for Studio. +# We encode it here so that it can be picked up by Github's dependabot +# to manage automated package upgrades. +python-3.9.13 +``` +Use `pyenv` to install the version of Python listed in that file, and to also set up a virtual environment: +```bash +pyenv install 3.9.13 +pyenv virtualenv 3.9.13 studio-py3.9 +pyenv activate studio-py3.9 +``` +Now you may install Studio's Python dependencies: +```bash +pip install -r requirements.txt -r requirements-dev.txt +``` +To deactivate the virtual environment, when you're finished developing on Studio for the time being: +```bash +pyenv deactivate +``` + +### A note about dependencies on Apple Silicon M1+ +If you run into an error with `pip install` related to the `grcpio` package, it is because it currently [does not support M1 with the version for `grcpio` Studio uses](https://github.com/grpc/grpc/issues/25082). In order to fix it, you will need to add the following environmental variables before running `pip install`: +```bash +export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 +export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 +export CFLAGS="-I/opt/homebrew/opt/openssl/include" +export LDFLAGS="-L/opt/homebrew/opt/openssl/lib" +``` + +## Install frontend dependencies +Install the version of node.js supported by Studio, and install `yarn` version 1.x: +```bash +volta install node@16 +volta install yarn@1 +``` +After installing `yarn`, you may now install frontend dependencies: +```bash +yarn install +``` + +## Install and run services + +Studio requires some background services to be running: + +* Minio - a local S3 storage emulation +* PostgreSQL (postgres) - a relational database +* Redis - a fast key/value store useful for caching +* Celery - the task manager and executor, which relies on the Studio codebase + +Generally speaking, you'll want to open a separate terminal/terminal-tab to run the services. With docker and docker-compose installed, running the above services is as easy as: +```bash +make run-services +``` + +The above command may take longer the first time it's run. It includes starting the `celery` workers, and the other dependent services through docker, which can be done separately with the following two commands: + +```bash +make dcservicesup +make devceleryworkers +``` + +To confirm that docker-based services are running, you should see three containers when executing `docker ps`. For example: + +```bash +> docker ps +CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES +e09c5c203b93 redis:6.0.9 "docker-entrypoint.sā€¦" 51 seconds ago Up 49 seconds 0.0.0.0:6379->6379/tcp studio_vue-refactor_redis_1 +6164371efb6b minio/minio "minio server /data" 51 seconds ago Up 49 seconds 0.0.0.0:9000->9000/tcp studio_vue-refactor_minio_1 +c86bbfa3a59e postgres:12.10 "docker-entrypoint.sā€¦" 51 seconds ago Up 49 seconds 0.0.0.0:5432->5432/tcp studio_vue-refactor_postgres_1 +``` + +To stop the services, press Ctrl + C in the terminal where you ran `make run-services` (or `dcservicesup`). Once you've done that, you may run the following command to remove the docker containers (they will be recreated when you run `run-services` or `dcservicesup` again): +```bash +make dcservicesdown +``` + +## Initializing Studio +With the services running, in a separate terminal/terminal-tab, we can now initialize the database for Studio development purposes. The command below will initialize the database tables, import constants, and a user account for development: +```bash +yarn run devsetup +``` + +## Running the development server +With the services running, in a separate terminal/terminal-tab, and the database initialized, we can start the dev server: +```bash +yarn run devserver:hot # with Vue hot module reloading +# or +yarn run devserver # without hot module reloading +``` + +Either of the above commands will take a few moments to build the frontend. When it finishes, you can sign in with the account created by the `yarn run devsetup` command: +- url: `http://localhost:8080/accounts/login/` +- username: `a@a.com` +- password: `a` + +## Running the celery service +Studio uses `celery` for executing asynchronous tasks, which are integral to Studio's channel editing architecture. The celery service does not reload when there are Python changes like the Django devserver does, so it's often preferred to run it separately. If you are developing changes against a task or the celery configuration, you'll need to use `make dcservicesup` to run only the docker-based services. + +In a separate terminal/terminal-tab, run the following to start the service and press Ctrl + C to stop it: +```bash +make devceleryworkers +``` + +Stop and restart the above to reload your changes. diff --git a/docs/local_dev_host.md b/docs/local_dev_host.md new file mode 100644 index 0000000000..0931453322 --- /dev/null +++ b/docs/local_dev_host.md @@ -0,0 +1,149 @@ +# Local development instructions: Run everything on your host machine + +This guide will walk through setting up Kolibri Studio for local development, where you'll run Studio's Python apps and all of Studio's services on your host machine, without the need for docker. + +## Prerequisites +- [volta](https://docs.volta.sh/guide/getting-started) +- [pyenv](https://kolibri-dev.readthedocs.io/en/develop/howtos/installing_pyenv.html) and [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv#installation) + +## Install system dependencies and services +Studio requires some background services to be running: + +* Minio - a local S3 storage emulation +* PostgreSQL - a relational database +* Redis - a fast key/value store useful for caching + +### Ubuntu or Debian +```bash +# Install packages +sudo apt-get install -y python-tk \ + postgresql-server-dev-all postgresql-contrib postgresql-client postgresql-12 \ + ffmpeg libmagickwand-dev redis-server wkhtmltopdf + +# Install minio +wget https://dl.minio.io/server/minio/release/linux-amd64/archive/minio.RELEASE.2020-06-01T17-28-03Z -O bin/minio +sudo chmod +x bin/minio +``` + +### Mac OS +```bash +brew install postgresql@12 redis ffmpeg imagemagick@6 gs +# note, this version of minio may not be compatible with Studio +brew install minio/stable/minio +brew link --force postgresql@12 +brew link --force imagemagick@6 +``` + +### Windows + +Windows is no longer supported due to incompatibilities with some required packages. + +## Set up the database + +Make sure postgres is running: + +```bash +service postgresql start +# alternatively: pg_ctl -D /usr/local/var/postgresql@12 start +``` + +Start the client with: + +```bash +sudo su postgres # switch to the postgres account +psql # mac: psql postgres +``` + +Create a database user with username `learningequality` and password `kolibri`: + +```sql +CREATE USER learningequality with NOSUPERUSER INHERIT NOCREATEROLE CREATEDB LOGIN NOREPLICATION NOBYPASSRLS PASSWORD 'kolibri'; + ``` + +Create a database called `kolibri-studio`: + +```sql +CREATE DATABASE "kolibri-studio" WITH TEMPLATE = template0 ENCODING = "UTF8" OWNER = "learningequality"; +``` + +Press Ctrl+D to exit the `psql` client. Finally + +```bash +exit # leave the postgres account +``` + + +## Build your python virtual environment +To determine what version of Python studio needs, you can check the `runtime.txt` file: +```bash +$ cat runtime.txt +# This is the required version of Python to run Studio currently. +# This is determined by the default Python 3 version that is installed +# inside Ubuntu Bionic, which is used to build images for Studio. +# We encode it here so that it can be picked up by Github's dependabot +# to manage automated package upgrades. +python-3.9.13 +``` +So to install python 3.9.13 through `pyenv` and set up a virtual environment: +```bash +pyenv install 3.9.13 +pyenv virtualenv 3.9.13 studio-py3.9 +pyenv activate studio-py3.9 +``` +Now you may install Studio's Python dependencies: +```bash +pip install -r requirements.txt -r requirements-dev.txt +``` +To deactivate the virtual environment, when you're finished developing on Studio for the time being: +```bash +pyenv deactivate +``` + +### A note about `psycopg2` +The packages `postgresql-12`, `postgresql-contrib`, and `postgresql-server-dev-all` are required to build `psycopg2` python driver. + +### A note about dependencies on Apple Silicon M1+ +If you run into an error with `pip install` related to the `grcpio` package, it is because it currently [does not support M1 with the version for `grcpio` Studio uses](https://github.com/grpc/grpc/issues/25082). In order to fix it, you will need to add the following environmental variables before running `pip install`: +```bash +export GRPC_PYTHON_BUILD_SYSTEM_OPENSSL=1 +export GRPC_PYTHON_BUILD_SYSTEM_ZLIB=1 +export CFLAGS="-I/opt/homebrew/opt/openssl/include" +export LDFLAGS="-L/opt/homebrew/opt/openssl/lib" +``` + +## Install frontend dependencies +Ready the version of node.js supported by Studio, and install yarn. +```bash +volta install node@16 +volta install yarn +``` +Then you can install frontend dependencies +```bash +yarn install +``` + +## Run the services + +Having installed all the necessary services, initialized your python virtual environment, and installed `yarn`, you're now ready to start the services. Generally speaking, you'll want to open a separate terminal/terminal-tab to run the services. The following will ensure all services are started, in addition to starting the celery workers service: +```bash +yarn run services +``` + +## Initializing Studio +With the services running, in a separate terminal/terminal-tab, we can now initialize the database for Studio development purposes. The command below will initialize the database, in addition to adding a user account for development: +```bash +yarn run devsetup +``` + +## Running the development server +With the services running, in a separate terminal/terminal-tab, and the database initialized, we can start the dev server: +```bash +yarn run devserver:hot # with Vue hot module reloading +# or +yarn run devserver # without hot module reloading +``` + +Either of the above commands will take a few minutes to build the frontend. When it's done, you can sign in with the account created by the `yarn run devsetup` command: +- url: `http://localhost:8080/accounts/login/` +- username: `a@a.com` +- password: `a` diff --git a/docs/storybook.md b/docs/storybook.md new file mode 100644 index 0000000000..78eb9d5658 --- /dev/null +++ b/docs/storybook.md @@ -0,0 +1,35 @@ +# Storybook + +Storybook is a development environment for UI components. If this is your first encounter with this tool, you can check [this presentation](https://docs.google.com/presentation/d/10JL4C9buygWsTbT62Ym149Yh9zSR9nY_ZqFumBKUY0o/edit?usp=sharing) or [its website](https://storybook.js.org/). You are encouraged to use it any time you need to develop a new UI component. It is especially suitable for smaller to middle size components that represent basic UI building blocks. + +An example is worth a thousand words so please have a look at these simple [stories of an example component](./contentcuration/contentcuration/frontend/shared/views/details/DetailsRow.stories.js) to see how to write yours. For detailed information on writing stories you can [go through this tutorial](https://www.learnstorybook.com/intro-to-storybook/). + +You can also check [official addons](https://storybook.js.org/addons/). + +**Run development server** + +```bash +yarn run storybook +``` + +With detailed webpack information (useful when debugging loaders, addons and similar): + +```bash +yarn run storybook:debug +``` + +**Bundle** + +```bash +yarn run storybook:build +``` + +The output is saved to *storybook-static/*. + +## Current usage notes + +We've decided not to push our stories to the codebase and keep them locally in the near future. Although this limits the number of advantages Storybook provides, it allows us to start using it as soon as possible without the need to agree on all conventions and it also gives the whole team enough time to test the development workflow so we can decide later if we want to adopt this tool in a larger scale. + +Taking into account the above-mentioned, all stories except of example *DetailsRow.stories.js* will be ignored by git as long as you use a naming convention for Storybook source files: *\*.stories.js*. + +Although we don't share stories at this point, Storybook is installed and configured in the codebase to prevent the need for everyone to configure everything locally. If you update Storybook Webpack settings, install a new plugin and similar, you are welcome to share such updates with other members of the team. From d05b383d23ae3dcc1d0f247050cb2acb076bb960 Mon Sep 17 00:00:00 2001 From: MisRob Date: Sat, 25 Nov 2023 13:42:47 +0100 Subject: [PATCH 2/5] Add contributing guidelines --- CONTRIBUTING.md | 28 ++++++++++++++++++++++++++++ README.md | 29 +++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+) create mode 100644 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000000..35d0e2c4c1 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,28 @@ + +## How can I contribute? + +1. šŸ“™ **Skim through the [Developer documentation](./docs/_index.md)** to understand where to refer later on. +2. šŸ’» **Follow the [Local development instructions](./docs/local_dev_docker.md) to set up your development server.** +3. šŸ” **Search for issues tagged as [help wanted](https://github.com/learningequality/studio/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22+no%3Aassignee) or [good first issue](https://github.com/learningequality/studio/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22+no%3Aassignee).** +4. šŸ—£ļø **Ask us for an assignment in the comments of an issue you've chosen.** Please request assignment of a reasonable amount of issues at a time. Once you finish your current issue or two, you are welcome to ask for more. + +**ā“ Where to ask questions** + +- For anything development related, refer to the [Developer documentation](./docs/_index.md) at first. Some answers may already be there. +- For questions related to a specific issue or assignment requests, use the corresponding issue's comments section. +- Visit [GitHub Discussions](https://github.com/learningequality/studio/discussions) to ask about anything related to contributing or to troubleshoot development server issues. + +**šŸ‘„ How to connect** + +- We encourage you to visit [GitHub Discussions](https://github.com/learningequality/studio/discussions) to connect with the Learning Equality team as well as with other contributors. +- If you'd like to contribute on a regular basis, we are happy to invite you to our open-source community Slack channel. Get in touch with us at info@learningequality.org to receive an invitation. + +--- + +šŸ•– Please allow us a few days to reply to your comments. If you don't hear from us within a week, reach out via [GitHub Discussions](https://github.com/learningequality/studio/discussions). + +As soon as you open a pull request, it may take us a week or two to review it as we're a small team. We appreciate your contribution and will provide feedback. + +--- + +*Thank you for your interest in contributing! Learning Equality was founded by volunteers dedicated to helping make educational materials more accessible to those in need, and every contribution makes a difference.* diff --git a/README.md b/README.md index fceab4231d..c7bd366d4e 100644 --- a/README.md +++ b/README.md @@ -12,3 +12,32 @@ Kolibri Studio uses the [Django framework](https://www.djangoproject.com/) for the backend and [Vue.js](https://vuejs.org/) for the frontend. If you are looking for help setting up custom content channels, uploading and organizing resources using Kolibri Studio, please refer to the [User Guide](https://kolibri-studio.readthedocs.io/en/latest/). + + +## How can I contribute? + +1. šŸ“™ **Skim through the [Developer documentation](./docs/_index.md)** to understand where to refer later on. +2. šŸ’» **Follow the [Local development instructions](./docs/local_dev_docker.md) to set up your development server.** +3. šŸ” **Search for issues tagged as [help wanted](https://github.com/learningequality/studio/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22+no%3Aassignee) or [good first issue](https://github.com/learningequality/studio/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22+no%3Aassignee).** +4. šŸ—£ļø **Ask us for an assignment in the comments of an issue you've chosen.** Please request assignment of a reasonable amount of issues at a time. Once you finish your current issue or two, you are welcome to ask for more. + +**ā“ Where to ask questions** + +- For anything development related, refer to the [Developer documentation](./docs/_index.md) at first. Some answers may already be there. +- For questions related to a specific issue or assignment requests, use the corresponding issue's comments section. +- Visit [GitHub Discussions](https://github.com/learningequality/studio/discussions) to ask about anything related to contributing or to troubleshoot development server issues. + +**šŸ‘„ How to connect** + +- We encourage you to visit [GitHub Discussions](https://github.com/learningequality/studio/discussions) to connect with the Learning Equality team as well as with other contributors. +- If you'd like to contribute on a regular basis, we are happy to invite you to our open-source community Slack channel. Get in touch with us at info@learningequality.org to receive an invitation. + +--- + +šŸ•– Please allow us a few days to reply to your comments. If you don't hear from us within a week, reach out via [GitHub Discussions](https://github.com/learningequality/studio/discussions). + +As soon as you open a pull request, it may take us a week or two to review it as we're a small team. We appreciate your contribution and will provide feedback. + +--- + +*Thank you for your interest in contributing! Learning Equality was founded by volunteers dedicated to helping make educational materials more accessible to those in need, and every contribution makes a difference.* From c809bd10db890a1fbac0efd1929cc15dd693240d Mon Sep 17 00:00:00 2001 From: MisRob Date: Mon, 27 Nov 2023 09:17:44 +0100 Subject: [PATCH 3/5] Add contact links to the new issue template --- .github/ISSUE_TEMPLATE/config.yml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml index 3ba13e0cec..0b2ccf668e 100644 --- a/.github/ISSUE_TEMPLATE/config.yml +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -1 +1,8 @@ blank_issues_enabled: false +contact_links: + - name: Studio GitHub Discussions + url: https://github.com/learningequality/studio/discussions + about: Please ask general questions about contributing to Studio or report development server issues here. + - name: Learning Equality Community Forum + url: https://community.learningequality.org/ + about: Ask and answer questions about Learning Equality's products and tools, share your experiences using Kolibri, and connect with users around the world. From 7230cc9f075efb1831547f62e3d6f22fc0a5362f Mon Sep 17 00:00:00 2001 From: MisRob Date: Tue, 28 Nov 2023 16:03:52 +0100 Subject: [PATCH 4/5] Add a note about recommended setup --- docs/local_dev_docker.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/local_dev_docker.md b/docs/local_dev_docker.md index 7cba837368..8be73878ef 100644 --- a/docs/local_dev_docker.md +++ b/docs/local_dev_docker.md @@ -1,6 +1,6 @@ -# Local development instructions: With Docker +# Local development instructions: With Docker (recommended) -The following guide utilizes docker and docker-compose to run select services required for Studio to function. If you would rather install these services on your host, please follow the [host-setup guide](./local_dev_host.md). +The following guide utilizes docker and docker-compose to run select services required for Studio to function. It's our recommended setup. However, if you would rather install these services on your host, please follow the [host-setup guide](./local_dev_host.md). ## Prerequisites Please install these prerequisites, or alternatives for setting up your local development environment: From da38913cba08faa28e9a2ba046ee05f4b38acf1e Mon Sep 17 00:00:00 2001 From: MisRob Date: Tue, 28 Nov 2023 16:06:08 +0100 Subject: [PATCH 5/5] Add a note about recommended setup to index --- docs/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_index.md b/docs/_index.md index c7214d4ba3..a7ceea42b9 100644 --- a/docs/_index.md +++ b/docs/_index.md @@ -2,7 +2,7 @@ ## Local development guides -- [Local development instructions: With Docker](./local_dev_docker.md) +- [Local development instructions: With Docker (recommended)](./local_dev_docker.md) - [Local development instructions: Run everything on your host machine](./local_dev_host.md) - [Local development tools](./dev_tools.md) - [Running tests](./running_tests.md)