Merge branch '3.0.x' into main

2023-02-26 20:16:38 +01:00 · 2023-02-26 20:16:38 +01:00 · 9384df7be2
parent e88e843b65 aca205dd4c
commit 9384df7be2
16 changed files with 200 additions and 61 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -1,3 +1,36 @@
+<a name="3.0.2"></a>
+
+## [3.0.2](https://github.com/libretime/libretime/compare/3.0.1...3.0.2) (2023-02-21)
+
+- [Release note](https://libretime.org/docs/releases/3.0.2/)
+
+### Bug Fixes
+
+- **legacy:** advanced search by track type id
+- **legacy:** refresh lock files
+- **legacy:** move forked deps to the libretime namespace
+- **legacy:** improve error messages and logs
+- **installer:** allow different actions on template_file
+- **installer:** print diff on file deployment
+- **installer:** only setup nginx on first install
+- **installer:** print unsupported distribution error ([#2368](https://github.com/libretime/libretime/issues/2368))
+- **installer:** create systemd dirs if missing ([#2379](https://github.com/libretime/libretime/issues/2379))
+
+### Documentation
+
+- add DOCKER_BUILDKIT env variable for docker-compose v1 ([#2270](https://github.com/libretime/libretime/issues/2270))
+- check logs before checking services status
+- add small faq for troubleshooting
+
+### Tests
+
+- **playout:** refresh snapshots after major upgrade ([#2381](https://github.com/libretime/libretime/issues/2381))
+
+### CI
+
+- don't squash commits during docs sync
+- test project weekly
+
 <a name="3.0.1"></a>

 ## [3.0.1](https://github.com/libretime/libretime/compare/3.0.0...3.0.1) (2022-12-20)
--- a/analyzer/setup.py
+++ b/analyzer/setup.py
@ -2,7 +2,7 @@ from setuptools import find_packages, setup

 setup(
    name="libretime-analyzer",
-    version="3.0.1",
+    version="3.0.2",
    description="Libretime Analyzer",
    author="LibreTime Contributors",
    url="https://github.com/libretime/libretime",
--- a/api-client/setup.py
+++ b/api-client/setup.py
@ -2,7 +2,7 @@ from setuptools import find_packages, setup

 setup(
    name="libretime-api-client",
-    version="3.0.1",
+    version="3.0.2",
    description="LibreTime API Client",
    author="LibreTime Contributors",
    url="https://github.com/libretime/libretime",
--- a/api/setup.py
+++ b/api/setup.py
@ -2,7 +2,7 @@ from setuptools import find_packages, setup

 setup(
    name="libretime-api",
-    version="3.0.1",
+    version="3.0.2",
    description="LibreTime API",
    author="LibreTime Contributors",
    url="https://github.com/libretime/libretime",
--- a/docs/admin-manual/setup/install.md
+++ b/docs/admin-manual/setup/install.md
@ -150,6 +150,12 @@ When upgrading be sure to run the installer using the same arguments you used du

 :::

+:::warning
+
+To update the LibreTime nginx configuration file, for example to change the `--listen-port`, make sure to add the `--update-nginx` flag to allow overwriting the existing configuration file.
+
+:::
+
 If you need to change some configuration, the install script can be configured using flags or environment variables. Changing the listening port of LibreTime or whether you want to install some dependency by yourself, you could run the following:

 ```bash
--- a/docs/admin-manual/troubleshooting.md
+++ b/docs/admin-manual/troubleshooting.md
@ -5,36 +5,21 @@ sidebar_position: 90

 This guide walk you though the steps required to troubleshoot LibreTime.

-## Services status
+## FAQ

-When facing a problem with LibreTime the first reflex is to verify whether the services are running.
+### I cannot login the interface "Oops! Something went wrong!"

-In the web interface, go to **Settings** > **Status** to see the state of the services.
+You might be accessing the interface from an invalid URL. Make sure that the URL in your browser and the one set in your configuration file in `[general.public_url]` are the same.

-![](./troubleshooting-status-page.png)
+Make sure to check the legacy logs at `/var/log/libretime/legacy.log`.

-Or directly from a terminal:
+References:

-```bash
-sudo systemctl --all --plain | egrep 'libretime|nginx|php.*-fpm'
-```
-
-If a service isn't running, you should search for details using the tool running those services.
-On a common setup, you should use the systemd service status:
-
-```bash
-sudo systemctl status libretime-worker
-```
-
-:::note
-
-Be sure to replace the service name with the problematic one.
-
-:::
+- https://discourse.libretime.org/t/version-3-0-1-debian-oops-something-went-wrong/1400

 ## Logs

-The next place to search for details on potential errors are the log files.
+The first place to search for details on potential errors are the log files.

 The `/var/log/syslog` file contains most of the system logs combined. This log file may contain information that the application logger wasn't able to log, such as early startup errors. You can follow the logs using:

@ -61,6 +46,33 @@ sudo -u libretime libretime-analyzer --config /etc/libretime/config.yml --log-le

 The `/var/log/nginx/libretime.error.log` file contains logs from the web server.

+## Services status
+
+The next reflex is to verify whether the services are running.
+
+In the web interface, go to **Settings** > **Status** to see the state of the services.
+
+![](./troubleshooting-status-page.png)
+
+Or directly from a terminal:
+
+```bash
+sudo systemctl --all --plain | egrep 'libretime|nginx|php.*-fpm'
+```
+
+If a service isn't running, you should search for details using the tool running those services.
+On a common setup, you should use the systemd service status:
+
+```bash
+sudo systemctl status libretime-worker
+```
+
+:::note
+
+Be sure to replace the service name with the problematic one.
+
+:::
+
 ## Test the stream inputs

 To test or debug your input streams, you can use the [`tools/test-stream-input.py`](https://github.com/libretime/libretime/blob/main/tools/test-stream-input.py) script to send a test sound to your stream inputs.
--- a/docs/releases/3.0.2.md
+++ b/docs/releases/3.0.2.md
@ -0,0 +1,32 @@
+---
+title: LibreTime 3.0.2
+---
+
+import ReleaseHead from './\_release-head.md';
+
+<ReleaseHead date='2023-02-21' version='3.0.2'/>
+
+## :sparkling_heart: Contributors
+
+The LibreTime project wants to thank the following contributors for authoring PRs to this release:
+
+- @jooola
+- @paddatrapper
+
+## :bug: Bug fixes
+
+- **legacy:** advanced search by track type id
+- **legacy:** refresh lock files
+- **legacy:** move forked deps to the libretime namespace
+- **legacy:** improve error messages and logs
+- **installer:** allow different actions on template_file
+- **installer:** print diff on file deployment
+- **installer:** only setup nginx on first install
+- **installer:** print unsupported distribution error ([#2368](https://github.com/libretime/libretime/issues/2368))
+- **installer:** create systemd dirs if missing ([#2379](https://github.com/libretime/libretime/issues/2379))
+
+## :arrow_up: Upgrading
+
+### Nginx configuration deployment
+
+The installer now only deploy the Nginx configuration during the first installation. This is to prevent overwriting changes made by Certbot for example. If you need to update the Nginx configuration, you need to pass the `--update-nginx` flag to the installer.
--- a/106
+++ b/106
@ -73,6 +73,8 @@ Options:
  --no-setup-postgresql         Do not setup Postgresql.
  --no-setup-rabbitmq           Do not setup RabbitMQ.

+  --update-nginx                Update nginx files during upgrades.
+
 Environment variables:

  Advanced options can be changed using environment variables (flags will take
@ -113,6 +115,9 @@ LIBRETIME_SETUP_POSTGRESQL=${LIBRETIME_SETUP_POSTGRESQL:-true}
 # > Create a default rabbitmq user with a random password.
 LIBRETIME_SETUP_RABBITMQ=${LIBRETIME_SETUP_RABBITMQ:-true}

+# > Update nginx files during upgrades.
+LIBRETIME_UPDATE_NGINX=${LIBRETIME_UPDATE_NGINX:-false}
+
 # > Comma separated list of sections to exclude from packages list.
 LIBRETIME_PACKAGES_EXCLUDES=${LIBRETIME_PACKAGES_EXCLUDES:-}

@ -130,6 +135,10 @@ while [[ $# -gt 0 ]]; do
      LIBRETIME_INSTALL_IN_PLACE=true
      shift 1
      ;;
+    --update-nginx)
+      LIBRETIME_UPDATE_NGINX=true
+      shift 1
+      ;;
    --no-setup-icecast)
      LIBRETIME_SETUP_ICECAST=false
      shift 1
@ -208,24 +217,41 @@ backup_if_exists() {
  fi
 }

+# diff_if_exists <src> <dest>
+diff_if_exists() {
+  src="$1"
+  dest="$2"
+  shift 2
+
+  if [[ -f "$dest" ]]; then
+    src_explicit_filename="$(dirname "$src")/$(basename "$dest").new"
+    cp "$src" "$src_explicit_filename"
+    # print what the existing file $dest would become if $src is copied
+    diff -u --color=always "$dest" "$src_explicit_filename" || :
+    rm -f "$src_explicit_filename"
+  fi
+}
+
 # cp_if_different <src> <dest>
 cp_if_different() {
  if [[ -f "$2" ]] && diff -q "$1" "$2" > /dev/null; then
    return
  fi
+  diff_if_exists "$1" "$2"
  backup_if_exists "$2"
  cp "$1" "$2"
 }

-# template_file <src> <dest> <render...>
+# template_file <action> <src> <dest> <render...>
 template_file() {
-  src="$1"
-  dest="$2"
-  shift 2
+  action="$1"
+  src="$2"
+  dest="$3"
+  shift 3

  tmp_file=$(mktemp)
  "$@" "$src" > "$tmp_file"
-  cp_if_different "$tmp_file" "$dest"
+  $action "$tmp_file" "$dest"
  rm "$tmp_file"
 }

@ -268,7 +294,11 @@ install_service() {

  [[ -f "$service_src" ]] || error "service '$service_name' src path '$service_src' does not exists!"

-  template_file "$service_src" "$service_dest" \
+  if [[ ! -d "$SERVICE_DIR" ]]; then
+    mkdir -p "$SERVICE_DIR"
+  fi
+
+  template_file cp_if_different "$service_src" "$service_dest" \
    sed \
      -e "s|^User=.*|User=${LIBRETIME_USER}|" \
      -e "s|^Group=.*|Group=${LIBRETIME_USER}|" \
@ -308,17 +338,15 @@ check_distribution()  {
    ubuntu-20.04) is_ubuntu=true && distro="focal" ;;
    debian-11)    is_debian=true && distro="bullseye" ;;
    *)
-      error << "EOF"
-  could not determine supported distribution "$ID-$VERSION_ID"!
+      error "could not determine supported distribution '$ID-$VERSION_ID'

-  Support for installing LibreTime on Buster has dropped since 3.1.0.
-  Support for installing LibreTime on Bionic has dropped since 3.1.0.
-  Support for installing LibreTime on Xenial has dropped since 3.0.0-alpha.10.
-  Support for installing LibreTime on Stretch has dropped since 3.0.0-alpha.10.
-  Support for installing LibreTime on Jessie has dropped since 3.0.0-alpha.8.
+Please check the documentation to find the supported distributions.

-EOF
-      exit 1
+Support for installing LibreTime on Buster has dropped since 3.1.0.
+Support for installing LibreTime on Bionic has dropped since 3.1.0.
+Support for installing LibreTime on Xenial has dropped since 3.0.0-alpha.10.
+Support for installing LibreTime on Stretch has dropped since 3.0.0-alpha.10.
+Support for installing LibreTime on Jessie has dropped since 3.0.0-alpha.8."
      ;;
  esac
 }
@ -565,7 +593,9 @@ install_service "libretime-liquidsoap.service" "$SCRIPT_DIR/playout/install/syst
 install_service "libretime-playout.service" "$SCRIPT_DIR/playout/install/systemd/libretime-playout.service"

 info "deploying libretime-liquidsoap logrotate config"
-template_file "$SCRIPT_DIR/playout/install/logrotate/libretime-liquidsoap.conf" "/etc/logrotate.d/libretime-liquidsoap" \
+template_file cp_if_different \
+  "$SCRIPT_DIR/playout/install/logrotate/libretime-liquidsoap.conf" \
+  "/etc/logrotate.d/libretime-liquidsoap" \
  sed \
    -e "s|@@LOG_DIR@@|${LOG_DIR}|g" \
    -e "s|@@USER@@|${LIBRETIME_USER}|g"
@ -652,13 +682,17 @@ fi
 PHP_VERSION="$(php-config --version | awk -F . '{ print $1 "." $2 }')"

 info "deploying libretime-legacy php-fpm config"
-template_file "$SCRIPT_DIR/legacy/install/php-fpm/libretime-legacy.conf" "/etc/php/$PHP_VERSION/fpm/pool.d/libretime-legacy.conf" \
+template_file cp_if_different \
+  "$SCRIPT_DIR/legacy/install/php-fpm/libretime-legacy.conf" \
+  "/etc/php/$PHP_VERSION/fpm/pool.d/libretime-legacy.conf" \
  sed \
    -e "s|@@DEFAULT_WEB_USER@@|${DEFAULT_WEB_USER}|g" \
    -e "s|@@USER@@|${LIBRETIME_USER}|g"

 info "deploying libretime-legacy logrotate config"
-template_file "$SCRIPT_DIR/legacy/install/logrotate/libretime-legacy.conf" "/etc/logrotate.d/libretime-legacy" \
+template_file cp_if_different \
+  "$SCRIPT_DIR/legacy/install/logrotate/libretime-legacy.conf" \
+  "/etc/logrotate.d/libretime-legacy" \
  sed \
    -e "s|@@LOG_DIR@@|${LOG_DIR}|g" \
    -e "s|@@USER@@|${LIBRETIME_USER}|g"
@ -668,21 +702,35 @@ template_file "$SCRIPT_DIR/legacy/install/logrotate/libretime-legacy.conf" "/etc

 section "Nginx"

-install_packages nginx
+if $is_first_install || $LIBRETIME_UPDATE_NGINX; then
+  install_packages nginx

-info "disabling nginx default site"
-rm -f "/etc/nginx/sites-enabled/default"
+  info "disabling nginx default site"
+  rm -f "/etc/nginx/sites-enabled/default"
+fi

-info "deploying libretime nginx config"
-template_file "${SCRIPT_DIR}/installer/nginx/libretime.conf" "/etc/nginx/sites-available/libretime.conf" \
-  sed \
-    -e "s|@@LISTEN_PORT@@|${LIBRETIME_LISTEN_PORT}|g" \
+nginx_config_template_args=(
+  "${SCRIPT_DIR}/installer/nginx/libretime.conf"
+  "/etc/nginx/sites-available/libretime.conf"
+    sed
+    -e "s|@@LISTEN_PORT@@|${LIBRETIME_LISTEN_PORT}|g"
    -e "s|@@LEGACY_WEB_ROOT@@|${LEGACY_WEB_ROOT}|g"
+)

-info "enabling libretime nginx config"
-ln -s --force \
-  "/etc/nginx/sites-available/libretime.conf" \
-  "/etc/nginx/sites-enabled/libretime.conf"
+if $is_first_install || $LIBRETIME_UPDATE_NGINX; then
+  info "deploying libretime nginx config"
+  template_file cp_if_different "${nginx_config_template_args[@]}"
+else
+  info "printing libretime nginx config differences"
+  template_file diff_if_exists "${nginx_config_template_args[@]}"
+fi
+
+if $is_first_install || $LIBRETIME_UPDATE_NGINX; then
+  info "enabling libretime nginx config"
+  ln -s --force \
+    "/etc/nginx/sites-available/libretime.conf" \
+    "/etc/nginx/sites-enabled/libretime.conf"
+fi

 # Finalize
 ########################################################################################
--- a/legacy/application/common/CORSHelper.php
+++ b/legacy/application/common/CORSHelper.php
@ -10,7 +10,9 @@ class CORSHelper

        if (!($origin == '' || preg_match('/https?:\/\/localhost/', $origin) === 1 || in_array($origin, $allowedOrigins))) {
            // Don't allow CORS from other domains to prevent XSS.
-            Logging::error("request origin '{$origin}' is not in allowed '" . implode(', ', $allowedOrigins) . "'!");
+            Logging::error(
+                "request origin '{$origin}' is not in the configured 'allowed_cors_origins' '" . implode(', ', $allowedOrigins) . "'"
+            );

            throw new Zend_Controller_Action_Exception('Forbidden', 403);
        }
--- a/legacy/application/configs/constants.php
+++ b/legacy/application/configs/constants.php
@ -41,6 +41,7 @@ define('WHOS_USING_URL', 'https://github.com/orgs/libretime/people');
 define('TERMS_AND_CONDITIONS_URL', 'https://github.com/libretime/libretime/blob/main/README.md');
 define('PRIVACY_POLICY_URL', 'https://github.com/libretime/organization/blob/main/CODE_OF_CONDUCT.md');
 define('USER_MANUAL_URL', 'https://libretime.org/docs');
+define('TROUBLESHOOTING_URL', 'https://libretime.org/docs/admin-manual/troubleshooting/');
 define('ABOUT_AIRTIME_URL', 'https://libretime.org');
 define('LIBRETIME_CONTRIBUTE_URL', 'https://libretime.org/contribute');
 define('LIBRETIME_DISCOURSE_URL', 'https://discourse.libretime.org');
--- a/legacy/application/views/scripts/error/error-500.phtml
+++ b/legacy/application/views/scripts/error/error-500.phtml
@ -12,7 +12,12 @@
        <h2><?php echo _("Oops!") ?></h2>
        <p><?php echo _("Something went wrong!") ?></p>
        <div class="button-bar">
-            <a class="toggle-button" href="<?php echo $this->helpUrl; ?>"><?php echo _("Help") ?></a>
+            <a class="toggle-button" href="<?php echo TROUBLESHOOTING_URL; ?>" target="_blank">
+                <?php echo _("Troubleshooting help") ?>
+            </a>
+            <a class="toggle-button" href="<?php echo $this->helpUrl; ?>" target="_blank">
+                <?php echo _("More help") ?>
+            </a>
        </div>
    </div>
 </body>
--- a/legacy/composer.json
+++ b/legacy/composer.json
@ -28,9 +28,9 @@
    "adbario/php-dot-notation": "^3.0.0",
    "composer/semver": "^3.2",
    "james-heinrich/getid3": "^1.9",
+    "league/uri": "^6.7",
    "libretime/celery-php": "dev-main",
    "libretime/propel1": "dev-main",
-    "league/uri": "^6.7",
    "php-amqplib/php-amqplib": "^3.0",
    "simplepie/simplepie": "^1.5",
    "symfony/config": "^5.4",
--- a/legacy/composer.lock
+++ b/legacy/composer.lock
@ -4,7 +4,7 @@
        "Read more about it at https://getcomposer.org/doc/01-basic-usage.md#installing-dependencies",
        "This file is @generated automatically"
    ],
-    "content-hash": "bfb63fdad05ca51f1ca445d3652b1787",
+    "content-hash": "99a8bfaf51b5e36bb702f789ce200fea",
    "packages": [
        {
            "name": "adbario/php-dot-notation",
--- a/playout/setup.py
+++ b/playout/setup.py
@ -2,7 +2,7 @@ from setuptools import find_packages, setup

 setup(
    name="libretime-playout",
-    version="3.0.1",
+    version="3.0.2",
    description="LibreTime Playout",
    author="LibreTime Contributors",
    url="https://github.com/libretime/libretime",
--- a/shared/setup.py
+++ b/shared/setup.py
@ -2,7 +2,7 @@ from setuptools import find_packages, setup

 setup(
    name="libretime-shared",
-    version="3.0.1",
+    version="3.0.2",
    description="LibreTime Shared",
    url="https://github.com/libretime/libretime",
    author="LibreTime Contributors",
--- a/worker/setup.py
+++ b/worker/setup.py
@ -2,7 +2,7 @@ from setuptools import find_packages, setup

 setup(
    name="libretime-worker",
-    version="3.0.1",
+    version="3.0.2",
    description="LibreTime Worker",
    author="LibreTime Contributors",
    url="https://github.com/libretime/libretime",