### Description
Improves the way hashlib is called in libretime_playout/player so that
is isn't broken on systems with python < 3.9
The way it is currently called in
site-packages/libretime_playout/player/file.py, in the section where
scheduled files are copied to the cache dir for playout, specifies the
usedforsecurity=False flag as follows:
`hasher = hashlib.md5(usedforsecurity=False)`
hashlib.md5 did not support this flag until Python 3.9. Attempting to
specify the flag directly as an argument to hashlib.md5(), in an older
python environment (such as that in Ubuntu Focal 20.04), is unsafe, and
can cause hashlib.md5() to balk/throw an exception, which results in
file copy operations failing. This then precipitates into playout
problems, as scheduled media is missing from the cache folder.
This PR instead calls using hashlib.new(), and passes the argument to
that, as follows:
`hasher = hashlib.new('md5', usedforsecurity=False)`
This method of calling with the flag argument is safer, because the
constructor will take it or leave it gracefully, regardless of whether
the system has a version of hashlib that supports the `usedforsecurity`
flag. AFAICT, it improves (fixes) function on older systems without
negatively impacting others.
### Testing Notes
**What I did:**
Before applying this patch, we were experiencing occasional but fairly
regular periods of silence when the system was supposed to be playing a
song or track. This behavior was consistent with errors such as the
following being present in the playout log:
```
2025-01-15 14:01:28,331 | INFO | libretime_playout.player.file:copy_file:47 - copying file 19834 to cache /var/lib/libretime/playout/scheduler/19834.mp3
2025-01-15 14:01:28,466 | ERROR | libretime_playout.player.file:copy_file:77 - could not copy file 19834 to /var/lib/libretime/playout/scheduler/19834.mp3: 'usedforsecurity' is an invalid keyword argument for openssl_md5()
Traceback (most recent call last):
File "/opt/libretime/lib/python3.8/site-packages/libretime_playout/player/file.py", line 70, in copy_file
file_event.filesize = self.report_file_size_and_md5_to_api(
File "/opt/libretime/lib/python3.8/site-packages/libretime_playout/player/file.py", line 89, in report_file_size_and_md5_to_api
hasher = hashlib.md5(usedforsecurity=False)
TypeError: 'usedforsecurity' is an invalid keyword argument for openssl_md5()
```
_For more information about the characterization and results of this
problem, see issue #3134_
**Testing on running systems**
After the patch was applied, these errors were no longer seen. I first
tested this on a dev server, and then on a live server, with
approximately 100 distinct tracks of playout occurring over about 24
hours. There were no file copy failures, and no related playout
problems, which was a major and immediate improvement.
**Testing installer, fresh installs**
***Ubuntu 20.04***
I deployed a patch to the installer and installed it on a blank system
running Ubuntu 20.04 and tested it to make sure the fix was applied and
worked.
***Debian 11***
I deployed patch to the installer and installed it on a blank system
running Debian Bullseye (which runs python = 3.9) , and tested it there
to make sure it did not break anything or introduce a regression.
### **Links**
Closes: #3134
### Description
PRs are intended to be squashed to a single commit. Only checking the
last commit gives us the intended state of the repo and ensures that if
the author commits the schema fixes later, the CI passes as expected.
Currently, the CI will fail because the earlier commits still have an
out of date schema.
### Description
Added support for previewing FLAC files in the web interface, provided
in #509 by marmotte32 on Github in the comments for this issue.
I have tested this against a script-installed copy of 4.2.0, and FLAC
preview is working, although auto-play isn't. I haven't tested to
confirm if this behavior matches MP3 and OGG uploads.
**This is a new feature**:
_Do the changes in this PR implement a new feature?_
**I have updated the documentation to reflect these changes**:
No changes needed; this bug is a missing file format in preview, and it
requires no updates to the documentation.
### Testing Notes
**What I did:**
I installed a LibreTime 4.2.0 system using the default installation
script against a fully-updated, brand new Debian 11 system. I logged
into the web interface, uploaded some FLAC files, and attempted to
preview them. They failed to preview.
I then replaced the preview_jplayer.js file with the contents in this PR
and then refreshed the page. I was able to preview the FLAC files and
hear the results in my local browser audio output.
**How you can replicate my testing:**
Perform the same steps above, or replace the same file in the
libretime_legacy_1 docker image to see the same results.
### **Links**
Closes: #509
---------
Co-authored-by: Kyle Robbertze <kyle@paddatrapper.com>
### Description
Upstream has renamed docker-compose to docker compose for all commands
**This is a new feature**:
_Do the changes in this PR implement a new feature?_
**I have updated the documentation to reflect these changes**:
Yes
### Description
The existing implementation for exporting playout logs to a CSV file
incorporates a very simplified CSV format. Some aspects of the complete
[RFC](https://www.rfc-editor.org/rfc/rfc4180) are missing, such as
escaping of quotes, and quoting of fields that contain certain
characters. This is problematic for common office spreadsheet tools, and
practically, anything else. Many radio stations rely on this
functionality to work well for exporting playout data, for example, in
order to compile data for reporting requirements.
**This is a new feature**:
The changes in this PR add quoting of fields containing a comma, as well
as those containing a CR/LF. It also escapes quotes by doubling them.
I'm not sure it makes CSVexport.js completely RFC 4180 compliant, but it
is much closer than it was.
**I have updated the documentation to reflect these changes**:
I don't think there are any documentation changes necessary; this is
probably expected behavior for anyone trying to use the CSV exporter.
### Testing Notes
**What I did:**
To validate this, I did a clean install of Debian, cloned from the
official libretime repo, and applied the code as a patch to the
installer. I then proceeded with the install and then loaded a database
from a running system (so that I had some playout data to test with). I
then performed the playout history export and examined the resulting CSV
file and after seeing previously problematic fields properly quoted, was
convinced it looked the way I expected. I loaded the csv file into
Libreoffice Calc and did not see any errors.
**How you can replicate my testing:**
See "What I did" above, basically run the patch after cloning the
installer from the repo. You could also apply the changes to a running
system by applying the patch to the file:
/usr/share/libretime/legacy/public/js/libs/CSVexport.js
Be sure to clear your browser cache and do a hard reload of the web
interface, before re-testing.
### **Links**
Closes: #2477
### Description
This is a way of ensuring the schema is up to date with every change.
This should be extended by fixing the PR api schema pipeline to squash
down all PR commits into a single change and checking the API schema for
that. Otherwise people will fix the schema after a failed pipeline and
the pipeline will continue to fail
### Description
Added filters for genre and md5 to the files API, e.g.
`/api/v2/files?genre=soul`
**This is a new feature**: Yes
**I have updated the documentation to reflect these changes**: No
There should be a schema and docs that are generated automatically. I
don't know where that is.
### Testing Notes
**What I did:**
- Used docker to deploy locally
- Confirmed filters work at
http://localhost:8080/api/v2/files?genre=Soul
**How you can replicate my testing:**
- `make clean dev`
- Upload some files!
- Visit http://localhost:8080/api/v2/files
- You can use the filters
<img width="658" alt="Screenshot 2024-12-23 at 01 36 01"
src="https://github.com/user-attachments/assets/ba19f7f3-fb3e-495d-8937-d451c70d326c"
/>
<img width="652" alt="Screenshot 2024-12-23 at 01 35 56"
src="https://github.com/user-attachments/assets/c7191131-a963-463a-b52f-9d0952192555"
/>
_How can the reviewer validate this PR?_
- See above
- wrote tests to confirm filters work
### Description
This fixes various problems in legacy migrations that were preventing a
successful database migration from Airtime 2.5.1. Previously, following
[the
procedure](https://libretime.org/docs/admin-manual/install/migrate-from-airtime/)
using the migrations provided in the Libretime 4.2.0 installer, without
the fixes in this PR, would either fail completely, or cause all of the
imported data to be completely deleted.
_migrations.py
If schema_version is not found in the table cc_prefs, it then checks for
system_version to have a value of '2.5.1' (in case this is an airtime
2.5.1 migration which will not have any schema_version in cc_pref). If
found, it prevents loading of the schema file, which is critical to
preserving the imported Airtime data.
0006_2_5_5
Removed a redundant addition of the image_path and description columns
to cc_show (done in earlier migration 003_2_5_2)
0015_2_5_17
Fixed a syntax error with adding the artwork column to cc_files
0023_3_0_0_alpha_9_1
Removed a redundant addition of the artwork column to cc_files (done in
earlier migration 0015_2_5_7)
### Documentation Changes
The [airtime migration
documentation](https://libretime.org/docs/admin-manual/install/migrate-from-airtime/)
already suggests a procedure to be followed, it just didn't work because
of problems within these migrations. The procedure as documented should
now work for those coming from Airtime 2.5.1.
### Testing Notes
**What I did:**
I attempted to migrate an actual airtime 2.5.1 database from a
production system containing a large amount of shows, tracks, and users.
I observed that the migration completed without errors, and that the
expected system state was achieved within Libretime. Specifically, the
calendar, library, authentication, and other aspects are populated with
the data that was present in the migrated Airtime database, and
Libretime is able to function using this data.
**How you can replicate my testing:**
Install Libretime 4.2.0. Restore a sample postrgresql database backup
from an Airtime 2.5.1 server. Apply the database migration. Restart the
services. Login and view the library, calender, etc.
### **Links**
Closes: #3121
May also be related to, or even close (as a duplicate): #2563
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[adamchainz/django-upgrade](https://redirect.github.com/adamchainz/django-upgrade)
| repository | patch | `1.22.1` -> `1.22.2` |
Note: The `pre-commit` manager in Renovate is not supported by the
`pre-commit` maintainers or community. Please do not report any problems
there, instead [create a Discussion in the Renovate
repository](https://redirect.github.com/renovatebot/renovate/discussions/new)
if you have any questions.
---
### Release Notes
<details>
<summary>adamchainz/django-upgrade (adamchainz/django-upgrade)</summary>
###
[`v1.22.2`](https://redirect.github.com/adamchainz/django-upgrade/blob/HEAD/CHANGELOG.rst#1222-2024-12-02)
[Compare
Source](https://redirect.github.com/adamchainz/django-upgrade/compare/1.22.1...1.22.2)
- Make these fixers work when `django.contrib.gis.db.models` is used to
import objects from `django.db.models`:
- `check_constraint_condition`
- `index_together`
`Issue #​513
<https://github.com/adamchainz/django-upgrade/issues/513>`\__.
</details>
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Enabled.
♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/libretime/libretime).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS40Mi40IiwidXBkYXRlZEluVmVyIjoiMzkuNDIuNCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiZGVwZW5kZW5jaWVzIl19-->
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR contains the following updates:
| Update | Change |
|---|---|
| lockFileMaintenance | All locks refreshed |
🔧 This Pull Request updates lock files to use the latest dependency
versions.
---
### Configuration
📅 **Schedule**: Branch creation - "after 4am and before 5am on monday"
(UTC), Automerge - At any time (no schedule defined).
🚦 **Automerge**: Enabled.
♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.
👻 **Immortal**: This PR will be recreated if closed unmerged. Get
[config
help](https://redirect.github.com/renovatebot/renovate/discussions) if
that's undesired.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/libretime/libretime).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS40Mi40IiwidXBkYXRlZEluVmVyIjoiMzkuNDIuNCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiZGVwZW5kZW5jaWVzIiwicGhwIl19-->
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR contains the following updates:
| Update | Change |
|---|---|
| lockFileMaintenance | All locks refreshed |
🔧 This Pull Request updates lock files to use the latest dependency
versions.
---
### Configuration
📅 **Schedule**: Branch creation - "after 4am and before 5am on monday"
(UTC), Automerge - At any time (no schedule defined).
🚦 **Automerge**: Enabled.
♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.
👻 **Immortal**: This PR will be recreated if closed unmerged. Get
[config
help](https://redirect.github.com/renovatebot/renovate/discussions) if
that's undesired.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/libretime/libretime).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xOS4wIiwidXBkYXRlZEluVmVyIjoiMzkuMTkuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiZGVwZW5kZW5jaWVzIiwicGhwIl19-->
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
### Description
Upload comments from mp3 files into libretime `comments` and
`description` fields.
**This is a new feature**:
Yes
**I have updated the documentation to reflect these changes**:
No none required
### Testing Notes
**What I did:**
I uploaded tracks that contained comments into LibreTime and checked the
database to ensure that the `comments` and `description` fields were
correctly populated. I then went to the UI and confirmed that the
description field had the MP3 comment in it inside of the metadata
editor. I then uploaded some files that did not have comments to make
sure I did not break any existing functionality.
**How you can replicate my testing:**
Follow the steps in what I did
### **Links**
Fixes#526
---------
Co-authored-by: Kyle Robbertze <paddatrapper@users.noreply.github.com>
This PR contains the following updates:
| Package | Type | Update | Change |
|---|---|---|---|
|
[codecov/codecov-action](https://redirect.github.com/codecov/codecov-action)
| action | major | `v4` -> `v5` |
---
### Release Notes
<details>
<summary>codecov/codecov-action (codecov/codecov-action)</summary>
###
[`v5`](https://redirect.github.com/codecov/codecov-action/compare/v4...v5)
[Compare
Source](https://redirect.github.com/codecov/codecov-action/compare/v4...v5)
</details>
---
### Configuration
📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).
🚦 **Automerge**: Enabled.
♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/libretime/libretime).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xMS41IiwidXBkYXRlZEluVmVyIjoiMzkuMTEuNSIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiY2kiLCJkZXBlbmRlbmNpZXMiXX0=-->
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
### Description
Having a global intro and outro playlist in settings is not very
flexible for special programming. This adds an override intro/outro
playlist per show. If it is not set, the global one is used. also it's
ignored if there's no autloading at all.
**I have updated the documentation to reflect these changes**:
Yes
### Testing Notes
**What I did:**
Schedule 2 shows, one without defining custom lists, one with defining
custom lists. one hour before the show starts it should be populated
correctly. If you define a global list it shojuld be replaced with the
per-show list.
---------
Co-authored-by: Thomas Göttgens <tgoettgens@mail.com>
### Description
When running a radio station it is generally a good idea to reduce dead
air time. The current algorithm for adding tracks to a block/show can
leave a lot of dead air time at the end as it doesn't use a very good
algorithm. Adding tracks to a show until it is full while making it as
full as possible is a well known problem in computer science. It is the
[Subset Sum Problem](https://en.wikipedia.org/wiki/Subset_sum_problem).
This PR implements a Randomized Greedy with Local Improvement (RGLI)
approximation solution for the Subset Sum Problem. The new algorithm is
only used when sort type is random and overflow is not enabled and there
is no limit on the number of tracks that can be used.
**This is a new feature**:
Improvement on an existing feature.
**I have not updated the documentation to reflect these changes**:
I did not update the documentation because the current scheduling
algorithm is not currently documented and its existing features have not
changed.
### Testing Notes
**What I did:**
I first attempted a fully polynomial time approximation scheme solution,
however it is really bad at finding good solutions for high density
values and can kinda slow the more tracks/time you have. So I instead
implemented an RGLI which is O(nlogn) and has been giving much better
results.
I implemented the solution in a separate project and tested it and timed
the values with a normal distribution of 500 songs with a mean of 3
minutes and a standard deviation of 1 minute. With a show size of 1 hour
the algorithm took around 10-15 ms to run. When adjusting the block size
and track size the algorithm still was pretty quick to run. Am going to
be testing on an instance with lots of tracks later, will update PR when
I have done that.
**How you can replicate my testing:**
_How can the reviewer validate this PR?_
### **Links**
Closes#3018
### Description
Add File Name and Size to the metadata editor screen, and added a File
Name column to the tracks data table.
**This is a new feature**:
Yes
**I have updated the documentation to reflect these changes**:
No, just some simple UI additions so no documentation needed.
### Testing Notes
**What I did:**
I uploaded some tracks, clicked on edit, and saw that the filename and
size showed up at the top. I also went out to the tracks view and added
the File Name column and saw that the filename displayed properly.
**How you can replicate my testing:**
Do what I did
### **Links**
Fixes#3053
### Description
FFMPEG filters for silence detection are too aggressive.
**This is a new feature**:
No
**I have updated the documentation to reflect these changes**:
No
### Testing Notes
**What I did:**
Ran tests and made sure they passed
**How you can replicate my testing:**
Upload files that start out quiet or end quiet and see that the
cue-in/cue-out points don't cut off the track. Or just run tests as
there are files that do that
### **Links**
Closes: #2629
renovate[bot]
Scott McGrath
Kyle Robbertze
nosbig
Keoni Mahelona
Weblate (bot)
libretime-bot
dakriy
Kyle Robbertze
Thomas Göttgens
