From 4ae344799312f4e59658a77b927b69289cbed241 Mon Sep 17 00:00:00 2001 From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com> Date: Thu, 16 Mar 2023 12:15:41 +0000 Subject: [PATCH 01/10] chore(deps): update amannn/action-semantic-pull-request action to v5.2.0 --- .github/workflows/pr.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/pr.yml b/.github/workflows/pr.yml index a3ff44f55..ef4d21dcc 100644 --- a/.github/workflows/pr.yml +++ b/.github/workflows/pr.yml @@ -12,7 +12,7 @@ jobs: name: Validate PR title runs-on: ubuntu-latest steps: - - uses: amannn/action-semantic-pull-request@v5.1.0 + - uses: amannn/action-semantic-pull-request@v5.2.0 env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} with: From f722cec2eb23c7f3d50d60132b67b059f357870f Mon Sep 17 00:00:00 2001 From: jo Date: Thu, 16 Mar 2023 21:48:13 +0100 Subject: [PATCH 02/10] docs: upgrade by migrating to a new server --- docs/admin-manual/setup/upgrade.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/docs/admin-manual/setup/upgrade.md b/docs/admin-manual/setup/upgrade.md index 9d4250928..065728a7a 100644 --- a/docs/admin-manual/setup/upgrade.md +++ b/docs/admin-manual/setup/upgrade.md @@ -11,6 +11,12 @@ You should always have proper backups and a rollback scenario in place before up ::: +:::info + +You may upgrade LibreTime by migrating to a new server. This is probably the cleanest way if you are coming from [Airtime](./migrate-from-airtime.md), if you wish to upgrade the underlying system, or if you want to skip multiple upgrade procedures. + +::: + ## Stop the services Run the following commands to stop the services: @@ -33,6 +39,12 @@ Follow [the backup guide](../backup.md) to make an extra backup of your installa Be sure to carefully read **all** the [releases notes](../../releases/README.md), from your current version to the targeted version, to apply upgrade or breaking changes instructions to your installation. +:::danger + +If you are migrating LibreTime to a new server, you must **not** apply the upgrade procedure. + +::: + :::caution You might need to run steps before and after the install procedure. Be sure to follow these steps thoroughly. @@ -43,6 +55,12 @@ You might need to run steps before and after the install procedure. Be sure to f Follow [the install guide](./install.md#download) to download and install the new version, and re-run the `./install` script with the same arguments you used during the initial install. +:::caution + +If you are migrating LibreTime to a new server, you must **stop before the [setup tasks](./install.md#setup)** and [restore the backups](../backup.md#restore-a-backup) on the new server. + +::: + ## Apply migrations Run the following command to apply the database migrations: From bc745617fb81ac931351a70fd42ac27ba539a00b Mon Sep 17 00:00:00 2001 From: jo Date: Mon, 20 Mar 2023 11:54:22 +0100 Subject: [PATCH 03/10] docs: fix database backup and restore commands The commands should now work out of the box when copy pasted on most systems. The previous one required the users to read the docs and the man pages. --- docs/admin-manual/backup.md | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/docs/admin-manual/backup.md b/docs/admin-manual/backup.md index 3da3b9503..7aaf9c1f3 100644 --- a/docs/admin-manual/backup.md +++ b/docs/admin-manual/backup.md @@ -27,17 +27,17 @@ On common setups, you need to backup the entire `/etc/libretime` folder. You need to backup the PostgreSQL database, which holds the entire data of your installation. -Here is an example to dump your PostgreSQL database: +Here is an example to dump your PostgreSQL database to a plain text SQL file: ```bash -sudo -u postgres pg_dump --file=libretime.sql libretime +sudo -u postgres pg_dump --no-owner --no-privileges libretime > libretime.sql ``` :::note -Consider using the `--no-owner` and `--no-privileges` flags to ignore roles +We use the `--no-owner` and `--no-privileges` flags to ignore roles and permissions details about the database. This can be useful when restoring -to database or role that have different names. +to database or role that have different names (e.g. `airtime` to `libretime`). ::: @@ -65,15 +65,25 @@ If you are upgrading LibreTime, edit the configuration file to match the new con ### Restore the database -Restore the database by using the following command: +Restore the database by using the one of the following command depending on the format of you backup file: ```bash -sudo -u postgres pg_restore --dbname=libretime libretime.sql +# With a plain text SQL file +sudo -u libretime libretime-api dbshell < libretime.sql + +# With a custom pg_dump format +sudo -u postgres pg_restore --no-owner --no-privileges --dbname=libretime libretime.dump ``` +:::info + +The `libretime-api dbshell` command is a shortcut to the `psql` command, and automatically passes the database access details (e.g. database name, user, password). + +::: + :::note -Consider using the `--no-owner` and `--no-privileges` flags to ignore roles +We use the `--no-owner` and `--no-privileges` flags to ignore roles and permissions details about the database. This can be useful when restoring to database or role that have different names. From ba2a1220a0e811507e1cd74d7db8ffba0a11b260 Mon Sep 17 00:00:00 2001 From: Jonas L Date: Thu, 23 Mar 2023 17:42:54 +0100 Subject: [PATCH 04/10] chore: backport #2461 (#2480) fix(api): cast string value to int enum (#2461) c7381a4f809114708838f86aead0d9c769bc3b79 --- api/libretime_api/core/models/preference.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/api/libretime_api/core/models/preference.py b/api/libretime_api/core/models/preference.py index 4057ef2a2..8a99f807f 100644 --- a/api/libretime_api/core/models/preference.py +++ b/api/libretime_api/core/models/preference.py @@ -78,7 +78,9 @@ class Preference(models.Model): entries = dict(cls.site.values_list("key", "value")) return StreamPreferences( input_fade_transition=float(entries.get("default_transition_fade") or 0.0), - message_format=MessageFormatKind(entries.get("stream_label_format") or 0), + message_format=MessageFormatKind( + int(entries.get("stream_label_format") or 0) + ), message_offline=entries.get("off_air_meta") or "Offline", ) From 64746ad2f4a07c32922bd80a838494a4f941ec5e Mon Sep 17 00:00:00 2001 From: "renovate[bot]" <29139614+renovate[bot]@users.noreply.github.com> Date: Wed, 29 Mar 2023 06:58:38 +0100 Subject: [PATCH 05/10] chore(deps): update pre-commit hook psf/black to v23.3.0 (#2487) Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> --- .pre-commit-config.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 7546c4d79..693b4a4df 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -42,7 +42,7 @@ repos: args: [--py36-plus] - repo: https://github.com/psf/black - rev: 23.1.0 + rev: 23.3.0 hooks: - id: black From 2c35544c08f8c52efd6d0d015a32c433dd4c1c29 Mon Sep 17 00:00:00 2001 From: jo Date: Thu, 16 Mar 2023 23:03:25 +0100 Subject: [PATCH 06/10] docs: move contributing to docs/contribute --- CONTRIBUTING.md => docs/contribute.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename CONTRIBUTING.md => docs/contribute.md (100%) diff --git a/CONTRIBUTING.md b/docs/contribute.md similarity index 100% rename from CONTRIBUTING.md rename to docs/contribute.md From bb755f7a62f64e726ea8682c2eecc24bcdcdb2ea Mon Sep 17 00:00:00 2001 From: jo Date: Thu, 16 Mar 2023 23:04:15 +0100 Subject: [PATCH 07/10] chore: create symlink to the contributing docs --- CONTRIBUTING.md | 1 + 1 file changed, 1 insertion(+) create mode 120000 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 120000 index 000000000..c5621ea35 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1 @@ +docs/contribute.md \ No newline at end of file From a9b7513bc0206b92ec9023374ac69613f494f8a0 Mon Sep 17 00:00:00 2001 From: jo Date: Thu, 16 Mar 2023 23:20:20 +0100 Subject: [PATCH 08/10] docs: split developer and contributor manual --- docs/admin-manual/setup/install.md | 2 +- docs/contributor-manual/README.md | 10 ++++++++++ docs/contributor-manual/_category_.yml | 2 ++ .../design/_category_.yml | 0 .../design/architecture.md | 0 .../design/database.md} | 6 +++++- .../development-environment.md} | 2 +- .../development => contributor-manual}/releases.md | 0 docs/developer-manual/README.md | 7 +------ docs/developer-manual/development/_category_.yml | 1 - docs/releases/3.0.0-beta.0.md | 4 ++-- docs/releases/3.0.0.md | 2 +- 12 files changed, 23 insertions(+), 13 deletions(-) create mode 100644 docs/contributor-manual/README.md create mode 100644 docs/contributor-manual/_category_.yml rename docs/{developer-manual => contributor-manual}/design/_category_.yml (100%) rename docs/{developer-manual => contributor-manual}/design/architecture.md (100%) rename docs/{developer-manual/design/database-migrations.md => contributor-manual/design/database.md} (97%) rename docs/{developer-manual/development/environment.md => contributor-manual/development-environment.md} (98%) rename docs/{developer-manual/development => contributor-manual}/releases.md (100%) delete mode 100644 docs/developer-manual/development/_category_.yml diff --git a/docs/admin-manual/setup/install.md b/docs/admin-manual/setup/install.md index 12c0edc58..335247109 100644 --- a/docs/admin-manual/setup/install.md +++ b/docs/admin-manual/setup/install.md @@ -46,7 +46,7 @@ LibreTime requires the following default ports to be open: The installer is shipped in the released tarballs or directly in the project repository. -We recommend installing on one of the following [distribution releases](../../developer-manual/development/releases.md#distributions-releases-support): +We recommend installing on one of the following [distribution releases](../../contributor-manual/releases.md#distributions-releases-support): - [Debian 11](https://www.debian.org/releases/) - [Ubuntu 20.04 LTS](https://wiki.ubuntu.com/Releases) diff --git a/docs/contributor-manual/README.md b/docs/contributor-manual/README.md new file mode 100644 index 000000000..5940f0d40 --- /dev/null +++ b/docs/contributor-manual/README.md @@ -0,0 +1,10 @@ +--- +title: Contributor manual +--- + +Welcome to the **LibreTime contributor manual**, you should find guides to improve and contribute to LibreTime. + +- Learn about the [architecture of LibreTime](./design/architecture.md) +- Learn about the [database design](./design/database.md) + +- Start coding by setting up a [development environment](./development-environment.md) diff --git a/docs/contributor-manual/_category_.yml b/docs/contributor-manual/_category_.yml new file mode 100644 index 000000000..90c952ceb --- /dev/null +++ b/docs/contributor-manual/_category_.yml @@ -0,0 +1,2 @@ +label: Contributor manual +position: 50 diff --git a/docs/developer-manual/design/_category_.yml b/docs/contributor-manual/design/_category_.yml similarity index 100% rename from docs/developer-manual/design/_category_.yml rename to docs/contributor-manual/design/_category_.yml diff --git a/docs/developer-manual/design/architecture.md b/docs/contributor-manual/design/architecture.md similarity index 100% rename from docs/developer-manual/design/architecture.md rename to docs/contributor-manual/design/architecture.md diff --git a/docs/developer-manual/design/database-migrations.md b/docs/contributor-manual/design/database.md similarity index 97% rename from docs/developer-manual/design/database-migrations.md rename to docs/contributor-manual/design/database.md index 0253168d9..898bc471a 100644 --- a/docs/developer-manual/design/database-migrations.md +++ b/docs/contributor-manual/design/database.md @@ -1,4 +1,8 @@ -# Database schema creation and migrations +--- +title: Database +--- + +## Database schema creation and migrations The method to maintain the database schema, is to write both a migration file for already installed databases and to update a `schema.sql` file for fresh databases. On fresh installation, the database is filled with the `schema.sql` and `data.sql` files and LibreTime won't run any sql migration on top of it. Previously, when LibreTime was upgraded, the missing migrations were run using a custom php based migration tool, those migrations are now handled by Django. The missing migrations are tracked using both a `schema_version` field in the `cc_pref` table and a Django migration id. diff --git a/docs/developer-manual/development/environment.md b/docs/contributor-manual/development-environment.md similarity index 98% rename from docs/developer-manual/development/environment.md rename to docs/contributor-manual/development-environment.md index 90fd416e7..56f89609e 100644 --- a/docs/developer-manual/development/environment.md +++ b/docs/contributor-manual/development-environment.md @@ -130,7 +130,7 @@ multipass shell ltTEST ``` Multipass isn't currently able to do an automated install from the cloud-init script. -After you enter the shell for the first time, you will still need to [run the LibreTime installer](../../admin-manual/setup/install.md). +After you enter the shell for the first time, you will still need to [run the LibreTime installer](../admin-manual/setup/install.md). The IP address of your new VM can be found by running `multipass list`. Copy and paste it into your web browser to access the LibreTime interface and complete the setup wizard. diff --git a/docs/developer-manual/development/releases.md b/docs/contributor-manual/releases.md similarity index 100% rename from docs/developer-manual/development/releases.md rename to docs/contributor-manual/releases.md diff --git a/docs/developer-manual/README.md b/docs/developer-manual/README.md index d065a7af4..52147626a 100644 --- a/docs/developer-manual/README.md +++ b/docs/developer-manual/README.md @@ -2,13 +2,8 @@ title: Developer manual --- -Welcome to the **LibreTime developer manual**, you should find guides to integrate LibreTime and tools to improve and contribute to LibreTime. +Welcome to the **LibreTime developer manual**, you should find guides to integrate LibreTime. ## Integrate LibreTime - :construction: Work in progress - -## Improve and contribute to LibreTime - -- Learn about the [architecture of LibreTime](./design/architecture.md) -- Learn about the [database migrations](./design/database-migrations.md) diff --git a/docs/developer-manual/development/_category_.yml b/docs/developer-manual/development/_category_.yml deleted file mode 100644 index 2f6f485a6..000000000 --- a/docs/developer-manual/development/_category_.yml +++ /dev/null @@ -1 +0,0 @@ -label: Development diff --git a/docs/releases/3.0.0-beta.0.md b/docs/releases/3.0.0-beta.0.md index 9dd1f8fe4..b3144397a 100644 --- a/docs/releases/3.0.0-beta.0.md +++ b/docs/releases/3.0.0-beta.0.md @@ -109,7 +109,7 @@ The LibreTime project wants to thank the following contributors for authoring PR ### Ubuntu Bionic support deprecation -Support for Ubuntu Bionic is being deprecated, and will be removed in LibreTime v3.1.0. Maintenance only versions (3.0.x) for Ubuntu Bionic will be provided until the distribution release reaches its end of life. Please see the [supported distributions release policy](../developer-manual/development/releases.md#distributions-releases-support) for details. +Support for Ubuntu Bionic is being deprecated, and will be removed in LibreTime v3.1.0. Maintenance only versions (3.0.x) for Ubuntu Bionic will be provided until the distribution release reaches its end of life. Please see the [supported distributions release policy](../contributor-manual/releases.md#distributions-releases-support) for details. Along with the Ubuntu Bionic deprecation, the following dependencies versions are also being deprecated: @@ -119,7 +119,7 @@ Along with the Ubuntu Bionic deprecation, the following dependencies versions ar ### Debian Buster support deprecation -Support for Debian Buster is being deprecated, and will be removed in LibreTime v3.1.0. Maintenance only versions (3.0.x) for Debian Buster will be provided until the distribution release reaches its end of life. Please see the [supported distributions release policy](../developer-manual/development/releases.md#distributions-releases-support) for details. +Support for Debian Buster is being deprecated, and will be removed in LibreTime v3.1.0. Maintenance only versions (3.0.x) for Debian Buster will be provided until the distribution release reaches its end of life. Please see the [supported distributions release policy](../contributor-manual/releases.md#distributions-releases-support) for details. Along with the Debian Buster deprecation, the following dependencies versions are also being deprecated: diff --git a/docs/releases/3.0.0.md b/docs/releases/3.0.0.md index 23ab562af..97f39f8b0 100644 --- a/docs/releases/3.0.0.md +++ b/docs/releases/3.0.0.md @@ -31,4 +31,4 @@ AAC streams aren't working out of the box because the [current distributions pac ### On Ubuntu Bionic, analyzing some FLAC files fails -[On Ubuntu Bionic, analyzing some FLAC files fails with `UnplayableFileError`](https://github.com/libretime/libretime/issues/2218) because of an upstream bug in Liquidsoap. If you encounter this, you should [upgrade to a more recent distribution version](../developer-manual/development/releases.md#distributions-releases-support). +[On Ubuntu Bionic, analyzing some FLAC files fails with `UnplayableFileError`](https://github.com/libretime/libretime/issues/2218) because of an upstream bug in Liquidsoap. If you encounter this, you should [upgrade to a more recent distribution version](../contributor-manual/releases.md#distributions-releases-support). From 5065415ff7a319b430902c1e3ca9f149270c1aa0 Mon Sep 17 00:00:00 2001 From: jo Date: Mon, 20 Mar 2023 13:52:27 +0100 Subject: [PATCH 09/10] docs: extract dev workflows from contributing docs --- docs/contribute.md | 114 +++--------------- docs/contributor-manual/README.md | 1 + docs/contributor-manual/design/database.md | 12 ++ .../development-environment.md | 16 +++ .../development-workflows.md | 77 ++++++++++++ 5 files changed, 121 insertions(+), 99 deletions(-) create mode 100644 docs/contributor-manual/development-workflows.md diff --git a/docs/contribute.md b/docs/contribute.md index 205456afd..89f0700ff 100644 --- a/docs/contribute.md +++ b/docs/contribute.md @@ -10,120 +10,36 @@ In addition, LibreTime follow the standardized [C4 development process](https://rfc.zeromq.org/spec:42/c4/), in which you can find explanation about most of the development workflows for LibreTime. -**How to contribute** +## Contribute financially -- [Reporting bugs](#reporting-bugs) -- [Suggesting enhancements](#suggesting-enhancements) -- [Financially](https://libretime.org/contribute#financial) -- [Contributing to documentation](https://libretime.org/contribute#write-documentation) -- [Contributing to code](#code) +LibreTime is run by volunteers who manage the project in their spare time. Financial contributions help us pay for our domain and infrastructure. It can also be used to cover the costs of development for important features and conference attendance. If you wish to contribute financially, you can do so through our [OpenCollective](https://opencollective.com/libretime). -## Reporting bugs +## Write documentation -This section guides you through submitting a bug report for LibreTime. -Following these guidelines helps maintainers and the community understand your -report, reproduce the behavior, and find related reports. +One of the simplest ways to get started contributing to a project is through improving documentation. LibreTime is constantly evolving, this means that sometimes our documentation has gaps. You can help by adding missing sections, editing the existing content so it is more accessible or creating new content (tutorials, FAQs, etc). -Before creating bug reports, please check the following list, to be sure that -you need to create one: +Issues pertaining to the documentation are usually marked with the [documentation](https://github.com/libretime/libretime/issues?q=is%3Aopen+is%3Aissue+label%3A%22is%3A+documentation%22) label. -- **Check the [LibreTime forum](https://discourse.libretime.org/)** for existing - questions and discussion. -- **Check that your issue does not already exist in the - [issue tracker](https://github.com/libretime/libretime/issues?q=is%3aissue+label%3abug)**. +## Translate LibreTime -> **Note:** If you find a **Closed** issue that seems like it is the same thing -> that you're experiencing, open a new issue and include a link to the original -> issue in the body of your new one. +LibreTime can run in over 15 different languages due to the gracious help of our volunteers. -When you are creating a bug report, please include as many details as possible. -Fill out the [required template](https://github.com/libretime/libretime/issues/new?labels=bug&template=bug_report.md), -the information it asks helps the maintainers resolve the issue faster. +LibreTime is localized using [Weblate](https://weblate.org/). If you would like to contribute a language translation, create an account and start working on [our Weblate page](https://hosted.weblate.org/projects/libretime/). -Bugs are tracked on the [official issue tracker](https://github.com/libretime/libretime/issues). +## Report bugs or request features -## Suggesting enhancements +Help us improve LibreTime by [submitting a bug report or suggesting new features](https://github.com/libretime/libretime/issues). When you are creating a report, please include as many details as possible. -This section guides you through submitting an enhancement suggestion for -LibreTime, including completely new features and minor improvements to existing -functionality. Following these guidelines helps maintainers and the community -understand your suggestion and find related suggestions. +You will find more details on how to submit a report on the [issue creation page](https://github.com/libretime/libretime/issues/new/choose). -Before creating enhancement suggestions, please check the following list, as you -might find out that you don't need to create one: +The [official issue tracker](https://github.com/libretime/libretime/issues) is hosted on Github. -- **Check the [LibreTime forum](https://discourse.libretime.org/)** for existing - questions and discussion. -- **Check that your issue does not already exist in the - [issue tracker](https://github.com/libretime/libretime/issues?q=is%3aissue+label%3afeature-request)**. +## Contribute with code -When you are creating an enhancement suggestion, please include as many details -as possible. Fill in [the template](https://github.com/libretime/libretime/issues/new?labels=feature-request&template=feature_request.md), -including the steps that you imagine you would take if the feature you're -requesting existed. +Are you familiar with coding in PHP or Python? Have you made projects in [Liquidsoap](https://www.liquidsoap.info/) and some of the other services we use? Help us improve LibreTime by picking an issue in the [list of bugs and feature requests](https://github.com/libretime/libretime/issues). -## Code - -Are you familiar with coding in PHP or Python? Have you made projects in -Liquidsoap and some of the other services we use? Take a look at the -[list of bugs and feature requests](https://github.com/libretime/libretime/issues), -and then fork our repo and have a go! Just use the **Fork** button at the top of -our [GitHub page](https://github.com/libretime/libretime), clone the forked repo -to your desktop, open up a favorite editor and make some changes, and then -commit, push, and open a pull request. +Then [fork our repo](https://docs.github.com/en/get-started/quickstart/contributing-to-projects) and [setup a development environment](https://libretime.org/docs/contributor-manual/development-environment.md) to get you started! Make sure to setup [pre-commit](https://libretime.org/docs/contributor-manual/development-workflows.md#pre-commit) to enforce the project best practices before committing new code. Knowledge on how to use [Github](https://guides.github.com/activities/hello-world/) and [Git](https://git-scm.com/docs/gittutorial) will suit you well, use the links for a quick 101. - -LibreTime uses the [black](https://github.com/psf/black) coding style for Python -and you must ensure that your code follows it. If not, the CI will fail and your -Pull Request will not be merged. Similarly, the Python import statements are -sorted with [isort](https://github.com/pycqa/isort). There is configuration -provided for [pre-commit](https://pre-commit.com/), which will ensure that code -matches the expected style and conventions when you commit changes. It is set up -by running: - -```bash -sudo apt install pre-commit -pre-commit install -``` - -You can also run it anytime using: - -```bash -pre-commit run --all-files -``` - -## Testing and CI/CD - -Before submitting code to the project, it's a good idea to test it first. To do -this, it's easiest to install LibreTime in a virtual machine on your local -system or in a cloud VM. We have instructions for setting up a virtual instance -of LibreTime with [Vagrant](/docs/vagrant) and [Multipass](/docs/multipass). - -If you would like to try LibreTime in a Docker image, Odclive has instructions -[here](https://github.com/kessibi/libretime-docker) for setting up a test image -and a more persistent install. - -## Modifying the Database - -LibreTime is designed to work with a [PostgreSQL](https://www.postgresql.org/) -database server running locally. LibreTime uses [PropelORM](https://github.com/propelorm/Propel) -to interact with the ZendPHP components and create the database. The version 2 -API uses Django to interact with the same database. - -If you are a developer seeking to add new columns to the database here are the steps. - -1. Modify `legacy/build/schema.xml` with any changes. -2. Run `dev_tools/propel_generate.sh` -3. Update the upgrade.sql under `legacy/application/controllers/upgrade_sql/VERSION` for example - `ALTER TABLE imported_podcast ADD COLUMN album_override boolean default 'f' NOT NULL;` -4. Update the models under `api/libretime_api/models/` to reflect the new - changes. - -## Documentation and financial contributions - -More information about how to contribute documentation or financially -through our [OpenCollective](https://opencollective.com/libretime) can be found -on our [website](https://libretime.org/contribute). diff --git a/docs/contributor-manual/README.md b/docs/contributor-manual/README.md index 5940f0d40..36a804287 100644 --- a/docs/contributor-manual/README.md +++ b/docs/contributor-manual/README.md @@ -7,4 +7,5 @@ Welcome to the **LibreTime contributor manual**, you should find guides to impro - Learn about the [architecture of LibreTime](./design/architecture.md) - Learn about the [database design](./design/database.md) +- Learn about the [development workflows of LibreTime](./development-workflows.md) - Start coding by setting up a [development environment](./development-environment.md) diff --git a/docs/contributor-manual/design/database.md b/docs/contributor-manual/design/database.md index 898bc471a..cc350edd0 100644 --- a/docs/contributor-manual/design/database.md +++ b/docs/contributor-manual/design/database.md @@ -2,6 +2,8 @@ title: Database --- +LibreTime is designed to work with a [PostgreSQL](https://www.postgresql.org/) database server and uses both the [PropelORM](https://github.com/propelorm/Propel) in the legacy app and [Django ORM](https://docs.djangoproject.com/en/4.1/topics/db/models/) in the API to interact with the database. + ## Database schema creation and migrations The method to maintain the database schema, is to write both a migration file for already installed databases and to update a `schema.sql` file for fresh databases. On fresh installation, the database is filled with the `schema.sql` and `data.sql` files and LibreTime won't run any sql migration on top of it. Previously, when LibreTime was upgraded, the missing migrations were run using a custom php based migration tool, those migrations are now handled by Django. The missing migrations are tracked using both a `schema_version` field in the `cc_pref` table and a Django migration id. @@ -53,3 +55,13 @@ stateDiagram-v2 apply_django_migration --> [*] ``` + +## Modifying the database schema + +To modify the database schema use the following steps: + +1. Modify `legacy/build/schema.xml`. +2. Run `make -C legacy propel-gen` to regenerate the legacy files. +3. Run `pre-commit run --all` and `make -C legacy format` multiple times to clean and format the previously generated files. +4. Update the `api` models to reflect the changes made to the schema. +5. Create a new schema migration file in the `api/legacy/migrations` directory. diff --git a/docs/contributor-manual/development-environment.md b/docs/contributor-manual/development-environment.md index 56f89609e..a2ba60a71 100644 --- a/docs/contributor-manual/development-environment.md +++ b/docs/contributor-manual/development-environment.md @@ -2,6 +2,10 @@ title: Development environment --- +This page describes the different way to run LibreTime in a development environment. + +The recommended development environment is the [docker-compose setup](#docker-compose). + ## Docker-compose To setup a docker-compose development environment, run the following commands: @@ -21,6 +25,18 @@ docker-compose up -d docker-compose logs -f ``` +:::info + +You may also use the following `make clean dev` shortcut: + +```bash +make clean dev + +docker-compose logs -f +``` + +::: + ## Vagrant To use Vagrant, you need to install a virtualization engine: [VirtualBox](https://www.virtualbox.org) or Libvirt. The [vagrant-vbguest] package on Github can help maintain guest extensions on host systems using VirtualBox. diff --git a/docs/contributor-manual/development-workflows.md b/docs/contributor-manual/development-workflows.md new file mode 100644 index 000000000..867828d0a --- /dev/null +++ b/docs/contributor-manual/development-workflows.md @@ -0,0 +1,77 @@ +--- +title: Development workflows +--- + +## Git workflow + +LibreTime uses [Github pull requests to manage changes](https://docs.github.com/en/get-started/quickstart/contributing-to-projects). The workflow looks like this: + +- [Create a fork of the project](https://docs.github.com/en/get-started/quickstart/fork-a-repo). +- Check out the `main` branch. If you're making a minor or small documentation change you can check out the `stable` branch. +- Create a new branch based on the checked out branch. +- Work on your changes locally. Try to keep each commit small to make reviews easier. +- Lint and test the codebase, for example using the `make lint` or `make test` commands inside the app folder you want to check. +- Push your branch. +- Create a pull request in Github. +- We'll review your request and provide feed back. + +## Project layout + +The LibreTime repository is split into multiple tools/services/apps at the root of the project. You will find `Makefile` in each of the component of the project. Those `Makefile` describe the different commands available to develop the project. + +Here is a small description of the different components in the repository: + +```bash +. +├── analyzer # The LibreTime Analyzer service +├── api # The LibreTime API service +├── api-client # The API clients used internally by other services +├── docker # The docker related files +├── docs # The documentation +├── install # The install script +├── installer # The install script extra files +├── legacy # The LibreTime Legacy service +├── playout # The LibreTime Playout service +├── shared # A shared library using by our python based services +├── tools # Set of tools used to maintain the project +├── website # Website repository that is cloned when developing the documentation +└── worker # The LibreTime Worker service +``` + +For example, to lint and test the `analyzer` service, you can run the commands: + +```bash +make -C analyzer lint test + +# Or by changing into the analyzer directory +cd analyzer +make lint test +``` + +## Pre-commit + +LibreTime uses [pre-commit](https://pre-commit.com/) to ensure that the files you commit are properly formatted, follow best practice, and don’t contain syntax or spelling errors. + +You can install and setup pre-commit using the [quick-start guide on the pre-commit documentation](https://pre-commit.com/#quick-start). Make sure to install pre-commit and setup the git pre-commit hook so pre-commit runs before you commit any changes to the repository. + +The workflow looks like this: + +- Install pre-commit. +- After cloning the repository, setup the pre-commit git hooks: + + ```bash + pre-commit install + ``` + +- Make your changes and commit them. +- If pre-commit fails to validate your changes, the commit process stops. Fix any reported errors and try again. + +:::info + +You can also run pre-commit anytime on all the files using: + +```bash +pre-commit run --all-files +``` + +::: From ade4d2736fe5bace7511740cecc20ff5b348b1c8 Mon Sep 17 00:00:00 2001 From: jo Date: Thu, 23 Mar 2023 13:24:23 +0100 Subject: [PATCH 10/10] docs: add some history notes --- docs/README.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/README.md b/docs/README.md index 22d4723b1..70e430a2d 100644 --- a/docs/README.md +++ b/docs/README.md @@ -21,3 +21,9 @@ Having trouble? We’d like to help! - [:question: Try the Forum – it’s got answers to many common questions](https://discourse.libretime.org/). - [:bug: Report bugs with LibreTime in our ticket tracker.](https://github.com/libretime/libretime/issues) + +## History + +LibreTime is a fork of AirTime due to stalled development of the open source version. For background on this, see this [open letter to the Airtime community](https://gist.github.com/hairmare/8c03b69c9accc90cfe31fd7e77c3b07d). + +Airtime itself was based on the software called Campcaster. You can find more information about the [early days of Campcaster and Airtime in the repository](https://github.com/libretime/libretime/blob/main/LEGACY.md).