From 5065415ff7a319b430902c1e3ca9f149270c1aa0 Mon Sep 17 00:00:00 2001 From: jo Date: Mon, 20 Mar 2023 13:52:27 +0100 Subject: [PATCH] docs: extract dev workflows from contributing docs --- docs/contribute.md | 114 +++--------------- docs/contributor-manual/README.md | 1 + docs/contributor-manual/design/database.md | 12 ++ .../development-environment.md | 16 +++ .../development-workflows.md | 77 ++++++++++++ 5 files changed, 121 insertions(+), 99 deletions(-) create mode 100644 docs/contributor-manual/development-workflows.md diff --git a/docs/contribute.md b/docs/contribute.md index 205456afd..89f0700ff 100644 --- a/docs/contribute.md +++ b/docs/contribute.md @@ -10,120 +10,36 @@ In addition, LibreTime follow the standardized [C4 development process](https://rfc.zeromq.org/spec:42/c4/), in which you can find explanation about most of the development workflows for LibreTime. -**How to contribute** +## Contribute financially -- [Reporting bugs](#reporting-bugs) -- [Suggesting enhancements](#suggesting-enhancements) -- [Financially](https://libretime.org/contribute#financial) -- [Contributing to documentation](https://libretime.org/contribute#write-documentation) -- [Contributing to code](#code) +LibreTime is run by volunteers who manage the project in their spare time. Financial contributions help us pay for our domain and infrastructure. It can also be used to cover the costs of development for important features and conference attendance. If you wish to contribute financially, you can do so through our [OpenCollective](https://opencollective.com/libretime). -## Reporting bugs +## Write documentation -This section guides you through submitting a bug report for LibreTime. -Following these guidelines helps maintainers and the community understand your -report, reproduce the behavior, and find related reports. +One of the simplest ways to get started contributing to a project is through improving documentation. LibreTime is constantly evolving, this means that sometimes our documentation has gaps. You can help by adding missing sections, editing the existing content so it is more accessible or creating new content (tutorials, FAQs, etc). -Before creating bug reports, please check the following list, to be sure that -you need to create one: +Issues pertaining to the documentation are usually marked with the [documentation](https://github.com/libretime/libretime/issues?q=is%3Aopen+is%3Aissue+label%3A%22is%3A+documentation%22) label. -- **Check the [LibreTime forum](https://discourse.libretime.org/)** for existing - questions and discussion. -- **Check that your issue does not already exist in the - [issue tracker](https://github.com/libretime/libretime/issues?q=is%3aissue+label%3abug)**. +## Translate LibreTime -> **Note:** If you find a **Closed** issue that seems like it is the same thing -> that you're experiencing, open a new issue and include a link to the original -> issue in the body of your new one. +LibreTime can run in over 15 different languages due to the gracious help of our volunteers. -When you are creating a bug report, please include as many details as possible. -Fill out the [required template](https://github.com/libretime/libretime/issues/new?labels=bug&template=bug_report.md), -the information it asks helps the maintainers resolve the issue faster. +LibreTime is localized using [Weblate](https://weblate.org/). If you would like to contribute a language translation, create an account and start working on [our Weblate page](https://hosted.weblate.org/projects/libretime/). -Bugs are tracked on the [official issue tracker](https://github.com/libretime/libretime/issues). +## Report bugs or request features -## Suggesting enhancements +Help us improve LibreTime by [submitting a bug report or suggesting new features](https://github.com/libretime/libretime/issues). When you are creating a report, please include as many details as possible. -This section guides you through submitting an enhancement suggestion for -LibreTime, including completely new features and minor improvements to existing -functionality. Following these guidelines helps maintainers and the community -understand your suggestion and find related suggestions. +You will find more details on how to submit a report on the [issue creation page](https://github.com/libretime/libretime/issues/new/choose). -Before creating enhancement suggestions, please check the following list, as you -might find out that you don't need to create one: +The [official issue tracker](https://github.com/libretime/libretime/issues) is hosted on Github. -- **Check the [LibreTime forum](https://discourse.libretime.org/)** for existing - questions and discussion. -- **Check that your issue does not already exist in the - [issue tracker](https://github.com/libretime/libretime/issues?q=is%3aissue+label%3afeature-request)**. +## Contribute with code -When you are creating an enhancement suggestion, please include as many details -as possible. Fill in [the template](https://github.com/libretime/libretime/issues/new?labels=feature-request&template=feature_request.md), -including the steps that you imagine you would take if the feature you're -requesting existed. +Are you familiar with coding in PHP or Python? Have you made projects in [Liquidsoap](https://www.liquidsoap.info/) and some of the other services we use? Help us improve LibreTime by picking an issue in the [list of bugs and feature requests](https://github.com/libretime/libretime/issues). -## Code - -Are you familiar with coding in PHP or Python? Have you made projects in -Liquidsoap and some of the other services we use? Take a look at the -[list of bugs and feature requests](https://github.com/libretime/libretime/issues), -and then fork our repo and have a go! Just use the **Fork** button at the top of -our [GitHub page](https://github.com/libretime/libretime), clone the forked repo -to your desktop, open up a favorite editor and make some changes, and then -commit, push, and open a pull request. +Then [fork our repo](https://docs.github.com/en/get-started/quickstart/contributing-to-projects) and [setup a development environment](https://libretime.org/docs/contributor-manual/development-environment.md) to get you started! Make sure to setup [pre-commit](https://libretime.org/docs/contributor-manual/development-workflows.md#pre-commit) to enforce the project best practices before committing new code. Knowledge on how to use [Github](https://guides.github.com/activities/hello-world/) and [Git](https://git-scm.com/docs/gittutorial) will suit you well, use the links for a quick 101. - -LibreTime uses the [black](https://github.com/psf/black) coding style for Python -and you must ensure that your code follows it. If not, the CI will fail and your -Pull Request will not be merged. Similarly, the Python import statements are -sorted with [isort](https://github.com/pycqa/isort). There is configuration -provided for [pre-commit](https://pre-commit.com/), which will ensure that code -matches the expected style and conventions when you commit changes. It is set up -by running: - -```bash -sudo apt install pre-commit -pre-commit install -``` - -You can also run it anytime using: - -```bash -pre-commit run --all-files -``` - -## Testing and CI/CD - -Before submitting code to the project, it's a good idea to test it first. To do -this, it's easiest to install LibreTime in a virtual machine on your local -system or in a cloud VM. We have instructions for setting up a virtual instance -of LibreTime with [Vagrant](/docs/vagrant) and [Multipass](/docs/multipass). - -If you would like to try LibreTime in a Docker image, Odclive has instructions -[here](https://github.com/kessibi/libretime-docker) for setting up a test image -and a more persistent install. - -## Modifying the Database - -LibreTime is designed to work with a [PostgreSQL](https://www.postgresql.org/) -database server running locally. LibreTime uses [PropelORM](https://github.com/propelorm/Propel) -to interact with the ZendPHP components and create the database. The version 2 -API uses Django to interact with the same database. - -If you are a developer seeking to add new columns to the database here are the steps. - -1. Modify `legacy/build/schema.xml` with any changes. -2. Run `dev_tools/propel_generate.sh` -3. Update the upgrade.sql under `legacy/application/controllers/upgrade_sql/VERSION` for example - `ALTER TABLE imported_podcast ADD COLUMN album_override boolean default 'f' NOT NULL;` -4. Update the models under `api/libretime_api/models/` to reflect the new - changes. - -## Documentation and financial contributions - -More information about how to contribute documentation or financially -through our [OpenCollective](https://opencollective.com/libretime) can be found -on our [website](https://libretime.org/contribute). diff --git a/docs/contributor-manual/README.md b/docs/contributor-manual/README.md index 5940f0d40..36a804287 100644 --- a/docs/contributor-manual/README.md +++ b/docs/contributor-manual/README.md @@ -7,4 +7,5 @@ Welcome to the **LibreTime contributor manual**, you should find guides to impro - Learn about the [architecture of LibreTime](./design/architecture.md) - Learn about the [database design](./design/database.md) +- Learn about the [development workflows of LibreTime](./development-workflows.md) - Start coding by setting up a [development environment](./development-environment.md) diff --git a/docs/contributor-manual/design/database.md b/docs/contributor-manual/design/database.md index 898bc471a..cc350edd0 100644 --- a/docs/contributor-manual/design/database.md +++ b/docs/contributor-manual/design/database.md @@ -2,6 +2,8 @@ title: Database --- +LibreTime is designed to work with a [PostgreSQL](https://www.postgresql.org/) database server and uses both the [PropelORM](https://github.com/propelorm/Propel) in the legacy app and [Django ORM](https://docs.djangoproject.com/en/4.1/topics/db/models/) in the API to interact with the database. + ## Database schema creation and migrations The method to maintain the database schema, is to write both a migration file for already installed databases and to update a `schema.sql` file for fresh databases. On fresh installation, the database is filled with the `schema.sql` and `data.sql` files and LibreTime won't run any sql migration on top of it. Previously, when LibreTime was upgraded, the missing migrations were run using a custom php based migration tool, those migrations are now handled by Django. The missing migrations are tracked using both a `schema_version` field in the `cc_pref` table and a Django migration id. @@ -53,3 +55,13 @@ stateDiagram-v2 apply_django_migration --> [*] ``` + +## Modifying the database schema + +To modify the database schema use the following steps: + +1. Modify `legacy/build/schema.xml`. +2. Run `make -C legacy propel-gen` to regenerate the legacy files. +3. Run `pre-commit run --all` and `make -C legacy format` multiple times to clean and format the previously generated files. +4. Update the `api` models to reflect the changes made to the schema. +5. Create a new schema migration file in the `api/legacy/migrations` directory. diff --git a/docs/contributor-manual/development-environment.md b/docs/contributor-manual/development-environment.md index 56f89609e..a2ba60a71 100644 --- a/docs/contributor-manual/development-environment.md +++ b/docs/contributor-manual/development-environment.md @@ -2,6 +2,10 @@ title: Development environment --- +This page describes the different way to run LibreTime in a development environment. + +The recommended development environment is the [docker-compose setup](#docker-compose). + ## Docker-compose To setup a docker-compose development environment, run the following commands: @@ -21,6 +25,18 @@ docker-compose up -d docker-compose logs -f ``` +:::info + +You may also use the following `make clean dev` shortcut: + +```bash +make clean dev + +docker-compose logs -f +``` + +::: + ## Vagrant To use Vagrant, you need to install a virtualization engine: [VirtualBox](https://www.virtualbox.org) or Libvirt. The [vagrant-vbguest] package on Github can help maintain guest extensions on host systems using VirtualBox. diff --git a/docs/contributor-manual/development-workflows.md b/docs/contributor-manual/development-workflows.md new file mode 100644 index 000000000..867828d0a --- /dev/null +++ b/docs/contributor-manual/development-workflows.md @@ -0,0 +1,77 @@ +--- +title: Development workflows +--- + +## Git workflow + +LibreTime uses [Github pull requests to manage changes](https://docs.github.com/en/get-started/quickstart/contributing-to-projects). The workflow looks like this: + +- [Create a fork of the project](https://docs.github.com/en/get-started/quickstart/fork-a-repo). +- Check out the `main` branch. If you're making a minor or small documentation change you can check out the `stable` branch. +- Create a new branch based on the checked out branch. +- Work on your changes locally. Try to keep each commit small to make reviews easier. +- Lint and test the codebase, for example using the `make lint` or `make test` commands inside the app folder you want to check. +- Push your branch. +- Create a pull request in Github. +- We'll review your request and provide feed back. + +## Project layout + +The LibreTime repository is split into multiple tools/services/apps at the root of the project. You will find `Makefile` in each of the component of the project. Those `Makefile` describe the different commands available to develop the project. + +Here is a small description of the different components in the repository: + +```bash +. +├── analyzer # The LibreTime Analyzer service +├── api # The LibreTime API service +├── api-client # The API clients used internally by other services +├── docker # The docker related files +├── docs # The documentation +├── install # The install script +├── installer # The install script extra files +├── legacy # The LibreTime Legacy service +├── playout # The LibreTime Playout service +├── shared # A shared library using by our python based services +├── tools # Set of tools used to maintain the project +├── website # Website repository that is cloned when developing the documentation +└── worker # The LibreTime Worker service +``` + +For example, to lint and test the `analyzer` service, you can run the commands: + +```bash +make -C analyzer lint test + +# Or by changing into the analyzer directory +cd analyzer +make lint test +``` + +## Pre-commit + +LibreTime uses [pre-commit](https://pre-commit.com/) to ensure that the files you commit are properly formatted, follow best practice, and don’t contain syntax or spelling errors. + +You can install and setup pre-commit using the [quick-start guide on the pre-commit documentation](https://pre-commit.com/#quick-start). Make sure to install pre-commit and setup the git pre-commit hook so pre-commit runs before you commit any changes to the repository. + +The workflow looks like this: + +- Install pre-commit. +- After cloning the repository, setup the pre-commit git hooks: + + ```bash + pre-commit install + ``` + +- Make your changes and commit them. +- If pre-commit fails to validate your changes, the commit process stops. Fix any reported errors and try again. + +:::info + +You can also run pre-commit anytime on all the files using: + +```bash +pre-commit run --all-files +``` + +:::