'text' argument was added in Python 3.7 as an alias of 'universal_newlines'. Using 'universal_newlines' works on Python 3.5/3.6 and should be forward-compatible.
The real error was PypoMessageHandler wasn't consuming messages from RabbitMQ, thus schedules were only updated on PypoFetch time out.
Proably queue is in a stale state because it didn't recover from a broken connection.
The issue is a SimpleQueue was being used, which's 'python-like' *get* method apparently doesn't handle recovery.
Consumers are the way to go instead.
this is apparently the reason why schedules don't get pushed after a silence
producing desynchronization between the mvc and liquidsoap and cannot catch up until pypofetch times out.
(ON AIR turns off and there's a mess until the schedule is fetched. sometimes it even continues playing the wrong tune)
((logging inside the while True loop produces no output, probably it is locked on the queue?))
When Libretime streams a webstream, the ID3 data is put into a single title field, leaving the artist field blank. When processing the ID3 data, Libretime concatenates the blank 'artist' field with the title, leaving '- title'. ShoutCAST rejects title updates that begin with punctuation for stylistic reasons (thanks guys) and so effectively ignores all ID3 data coming with the stream, falling back to the 'Station off-air' message. This PR uses the title field alone if the artist field is blank, but both when the artist data is available e.g. when streaming an MP3.
Config file getting is allowed to fail. I didn't realize that it
would not just use the default instead of excepting and getting
caught. Sorry for that.
1st- Changes the file permissions of cache files from 444 to 644
2nd- Put the host variables into a single array, this way we can pass the full variable to the report_file_size_and_md5_to_airtime function.
This removes most of the legacy upstream config madness by not using
weird config files spread all over the place.
This isn't the solution to other config reading fragility issues, but
it does move the whole config back to the central airtime.conf file.
This adds TLS support to pypo when downloading files from the REST
API. I previously fixed some similar issues in the api_client and
wasn't aware that pypo isn't using it for every URL..
The theme of this commit is to make the install process more
resiliant to differences in distros by testing for specific
features or versions of executables rather than making as many
assumptions just on the distro and release.
* Distro and Release detection (install, Vagrantfile)
- Added detection logic for distro and release based on
/etc/os-release which is available on all potential supported
distros. The --distribution and --release options can be used
to override detection.
- Added distro and release checks to ensure values specified are
for a supported release.
- Changed some distro checks to use booleans, e.g.
if $is_centos_7; then ...
- Added a simple check so if --distribution is specified, it will
ensure it's at least sane, e.g. centos on debian or debian on
centos will be caught.
- In Vagrantfile, removed --distribution and --release options for
all distro and release combinations.
* Portable Init System Detection and Management (install, centos.sh)
- Added detection logic for init system type - systemd, Upstart or
System V in the function systemInitDetect().
- Added portable init system install in the function
systemInitInstall() which depends on systemInitDetect(). After
installing files, enables and starts service.
- Added portable init system commands in the function
systemInitCommand() for start, stop, reload, restart and status.
- Python services don't support systemd but unconditionally install
scripts for both Upstart and System V. Disabled by passing
--no-init-script to setup.py for each service.
- When upgrading, remove all old system init files in /etc/init,
/etc/init.d, /etc/default and /etc/systemd/system.
- In centos.sh, removed install of airtime systemd service files
as it's now handled by install.
- Created an Upstart .conf for airtime-celery
- In systemd file for airtime-celery, changed absolute path to
/usr/local/bin/celery and for centos, the install symlinks
celery to /usr/local/bin.
* External Dependencies Install (install)
- For external dependencies, only attempt to install for Debian-
based distros with apt-get. For centos, a warning is displayed.
- For systems with apt-get, detect version and if 1.1 or greater,
use new force options, otherwise use --force-yes option.
* Configuring Apache (install)
- For Apache, check for centos and use httpd for binary and
service, otherwise use apache2ctl and apache2.
- Detect Apache version 2 or better in a more reliable manner.
- Detect Apache root folder and conf file name by running
apache2ctl -V or httpd -V as appropriate.
- Various checks for centos as it doesn't support the Debian
Apache utilities a2ensite/a2dissite, a2enmod/a2dismod, etc.
* Installing Airtime Services (install)
- Detect Python version. If less than 2.7.9, install OpenSSL
support.
- Prevent installing init files by passing --no-init-script to
setup.py for each service.
- Use systemInitInstall to install, enable and start each service
after setup.py has run.
- Removed filtering for WEB_USER for files in
/etc/init/airtime*.template as that is handled in the
systemInitInstall() function.
* Configuring PHP in Apache (install)
- Detect PHP conf folder by checking a list of locations rather
than making assumptions based on the distro and release.
* Configuring PostgreSQL (install)
- Detects if the airtime user has already been created. If not,
then creates the user.
* Installing Locales (install)
- Minor changes to check for centos and prevent from running.
This gets the mime type using file-magic in a most minimal way. Since the python bindings have been available as a distro package for quite a while it is written in a way so it should also run on pre pypi installs of file-magic. This means not being able to use nice things like magic.detect_from_filename due to the fact that they where added rather recently (with recently being 2 years ago).
As the mime type is only used to check for wav files that mutagen can't handle it only reads the mime type and ignores the charset and other attributes that magic can find.
Due to the fact that file-magic is not properly unicode safe I'm checking the file based on it's first 2048 bytes as per <http://stackoverflow.com/questions/34836792/python-magic-cant-identify-unicode-filename#comment57418632_34838355>. This is not an issue since wav files need to start with a wav header by definition anyway.
I tested this sucessfully on both CentOS and Debian with files containing Unicode in their names.
I think the code was bundled into legacy upstream before it was
readily available in liquidsoap proper. These snippets are now
maintained at https://github.com/savonet/liquidsoap/tree/master/scripts.
On all distros I checked these scripts are installed to /usr/bin/liquidsoap
by the corresponding package. Liquidsoap loads them from there by
default.
By not bundling this we will be ready to profit from new features and
bugfixes in liquidsoap that is expected to land on opam soon. We'll
need to figure out how to get debian and others updated when that
happens.
This is part 2 of fixing ecasound recordings from line-in. Part 1 restored the User-Interface, part 2 takes care of getting to the point where ecasound gets started, records something and uploads it through rest when done. Part 3 will take care of making sure that the recorded file is mapped to the show and not just stored as a new track.
I refactored api_clients to not use urllib2 for posting multipart data since I was loosing my sanity over it and requests seems to have a modern approach to doing this compared to what api_clients was previously doing.
It took me way too long to figure this one out it wasn't logging nicely. With this ecasound actually gets called again. It's still failing on my install but I'm not yet sure why exactly.
The quite ugly hack with std_err_override seems to have been made for python < 2.7. Since all major distros
have al least python 2.7 installed we can stop using the std_err_override hack.
This removes it from pypo, media-monitor still uses the module and we can completely delete it when we have
remove media-monitor after having maybe backported the watched folders feature to analyzer.
Lines 399 and 400. Changed the values of s3_name and s3_description places.
(Because of this error, the Icecast confused the name and description of the flow)
_______________________
Строки 399 и 400. Поменял значения s3_name и s3_description местами. (Из-за этой ошибки Icecast путал название и описание потока)
_______________________
Sorry for my clumsy translation, I'm from Russia
This is the workaround for <https://github.com/savonet/liquidsoap/issues/390>.
I still need to do proper testing on it and maybe we should figure out the proper "formula" for getting to the 0.04 value.
This changes the Vagrant setup to support multiple installations as multiple
boxes. In addition to Ubuntu Vagrant can now be used to install on Debian
as well as on CentOS.
I took the chance to clean up the .deb install a bit and backported analyzer
and celery to SysV proper so it runs there. Some of the distro specfics were
moved to the install script from the python setup scripts to acheive this.
For the CentOS support I added a rather involved OS prepare script. In the
long term this will be added to the preparing-the-server docs we already have.
I had to switch the default port to http-alt (8080). On CentOS 9080 is registered
for ocsp and getting it to work for apache without hacking SELinux is hard. I
think 8080 is the RFC way to go anyhow. If anyone want to override this it
should be rather easy using the --web-port arg and by hacking Vagrantfile.
The PyOpenSSL code has been refactored for all the distros that the Vagrantfile
now supports.
As far as my checks go, I tried this code with all the distros, uploaded a track
and downloaded a unicode and a ssl podcast and was able to listen to them
in each case.
In the experimental CentOS case, the UI is not up to spec since services
need to get scheduled through systemctl and the status overview (ie. on the /?config page)
do not work properly. They need to be as follows:
```
sudo systemctl start airtime-playout
sudo systemctl start airtime-liquidsoap
sudo systemctl start airtime_analyzer.service
sudo systemctl start airtime-celery.service
```
The podcast downloader fails pretty badly when the podcast name contains non ascii chars. The main fail happens during logging; I have learnt way to much about pythons stupid unicode implementation.
This adds addtional debug logging and also outputs the real reason a download fails properly. The content of the tags should be written as UTF-8 or whater is input into it, this commit mainly touches (and fixes) logging.
* [x] regonfigured the build matrix with more php jobs and a separate python job (we can add more python jobs later)
* [x] run tests on travis' trusty beta container (it's closer to what we need anyway)
* [x] install packages needed for analyzer tests in build env
* [x] added docs on how to run nosetests locally
* [x] don't run initctl in analyzer setup so setup can also be used on travis (and add it to the install script directly)
* [x] ignore replaygain checks on travis (it has proven quite impossible to get the needed python-gi module to work in the provided virtualenv)
I tried a lot of solutions to get the replaygain checks to run. I needed to decide that this has gone far enough, maybe someone who is more of a pythonista than me can take a crack at it and get it solved. Even without running those tests on CI/CD there are still plenty others.
This PR only has parts of what are needed for getting python tests running on travis as per #15. I only took a quick shot at anything not analyzer and figured I would not be able to "fix" them without digging a bit deeper (ie. also getting rid of std_err_override).
This is the results of sed -i -e 's|/etc/airtime-saas/|/etc/airtime/|' `grep -irl 'airtime-saas' airtime_mvc/ python_apps/` :P
It might need more testing, the airtime-saas part never really made sense, zf1 has environments for that, ie you would create a saas env based on production for instance.
I beleive legacy upstream was using this to share configuration between customers (ie. analyser runs only once and writes to a shared S3 bucket). I assume they mount the airtime-saas folder onto individual customers instances with a global config. Like I said, I don't feel that this makes sense since all it does is make hacking at the configs in airtime-saas a bit easier. A serious SaaS operation should be using something like puppet or ansible to achieve this.
* Fix various small bugs in auto ingestion and tab implementation
* Update TaskManager run conditions to piggyback on API calls - guarantees a certain frequency of requests and greatly reduces chances of lock contention
rjhelms
Zachary Klosko
Kyle Robbertze
Jakub Skory
Robb
Keoni Mahelona
Keoni Mahelona
Kyle Robbertze
Jérôme Lebleu
Roberto Soto
Lucas Bickel
Lucas Bickel
frecuencialibre
vagrant
expert-geeks
Kyle Robertze
Robb Ebright
Xabi Spacebiker
Bill Burton
Lapotor
Hans-Joachim
Kirill Kadoshnikov
Albert Santoni
Duncan Sommerville