Commit Graph

2460 Commits

Author SHA1 Message Date
Robb 479bdb1c80 Merge pull request #76 from radiorabe/feature/no-more-std_err_override-for-pypo
Problem: std_err_override is a hack for python < 2.7
2017-03-13 14:44:01 -04:00
Lucas Bickel dfe3f077fd Replace std_err_override with logging.captureWarnings(True) in pypo
The quite ugly hack with std_err_override seems to have been made for python < 2.7. Since all major distros
have al least python 2.7 installed we can stop using the std_err_override hack.

This removes it from pypo, media-monitor still uses the module and we can completely delete it when we have
remove media-monitor after having maybe backported the watched folders feature to analyzer.
2017-03-13 16:28:07 +01:00
Lucas Bickel 99a0ce37cc Fix fix_cue_in to work with liquidsoap 1.2
The previous syntax only worked with 1.1, this one should work on both.
2017-03-13 14:30:51 +01:00
Robb a583d8fa1e Merge pull request #64 from radiorabe/feature/debian-and-experimental-centos
Vagrant Debian support (and experimental CentOS)
2017-03-12 21:42:37 -04:00
Kirill Kadoshnikov 0775bc4f59 Correction lines 399 and 400. ls_script.liq
Lines 399 and 400. Changed the values of s3_name and s3_description places. 
(Because of this error, the Icecast confused the name and description of the flow)
_______________________
Строки 399 и 400. Поменял значения s3_name и s3_description местами. (Из-за этой ошибки Icecast путал название и описание потока)
_______________________
Sorry for my clumsy translation, I'm from Russia
2017-03-12 20:03:30 +04:00
Lucas Bickel ac5f608bd8 Refactor into def check_version
Make the code more readable :)
2017-03-12 14:18:06 +01:00
Lucas Bickel 80373ca7bc cue_cut workaround
This is the workaround for <https://github.com/savonet/liquidsoap/issues/390>.

I still need to do proper testing on it and maybe we should figure out the proper "formula" for getting to the 0.04 value.
2017-03-12 14:18:06 +01:00
Lucas Bickel c29285ae48 Vagrant Debian support (and experimental CentOS)
This changes the Vagrant setup to support multiple installations as multiple
boxes. In addition to Ubuntu Vagrant can now be used to install on Debian
as well as on CentOS.

I took the chance to clean up the .deb install a bit and backported analyzer
and celery to SysV proper so it runs there. Some of the distro specfics were
moved to the install script from the python setup scripts to acheive this.

For the CentOS support I added a rather involved OS prepare script. In the
long term this will be added to the preparing-the-server docs we already have.

I had to switch the default port to http-alt (8080). On CentOS 9080 is registered
for ocsp and getting it to work for apache without hacking SELinux is hard. I
think 8080 is the RFC way to go anyhow. If anyone want to override this it
should be rather easy using the --web-port arg and by hacking Vagrantfile.

The PyOpenSSL code has been refactored for all the distros that the Vagrantfile
now supports.

As far as my checks go, I tried this code with all the distros, uploaded a track
and downloaded a unicode and a ssl podcast and was able to listen to them
in each case.

In the experimental CentOS case, the UI is not up to spec since services
need to get scheduled through systemctl and the status overview (ie. on the /?config page)
do not work properly. They need to be as follows:

```
sudo systemctl start airtime-playout
sudo systemctl start airtime-liquidsoap
sudo systemctl start airtime_analyzer.service
sudo systemctl start airtime-celery.service
```
2017-03-12 13:42:22 +01:00
Lucas Bickel 0cb5e2665e Fix unicode issues in podcast downloader
The podcast downloader fails pretty badly when the podcast name contains non ascii chars. The main fail happens during logging; I have learnt way to much about pythons stupid unicode implementation.

This adds addtional debug logging and also outputs the real reason a download fails properly. The content of the tags should be written as UTF-8 or whater is input into it, this commit mainly touches (and fixes) logging.
2017-03-11 21:01:52 +01:00
Lucas Bickel 0e8d0868e8 Make pypo media downloader port aware
This is needed if the base port is not 80 like in the vagrant setup.
2017-03-10 19:10:15 +01:00
Robb d86e223c45 Merge pull request #28 from radiorabe/feature/robbt/podcast-naming
Podcast naming feature as per #26
2017-03-06 18:12:57 -05:00
Lucas Bickel 188c127057 Merge branch 'master' into feature/analyzer-on-travis
# Conflicts:
#	.travis.yml
2017-03-04 16:35:11 +01:00
Lucas Bickel 2e9fc432b1 Only override album it is empty 2017-03-04 13:38:37 +01:00
Robb Ebright 2cebb7dfa2 first part of the adding album tags to podcast downloads 2017-03-04 13:38:36 +01:00
Lucas Bickel 9b3d0c90da mostly run analyzer tests on travis
* [x] regonfigured the build matrix with more php jobs and a separate python job (we can add more python jobs later)
* [x] run tests on travis' trusty beta container (it's closer to what we need anyway)
* [x] install packages needed for analyzer tests in build env
* [x] added docs on how to run nosetests locally
* [x] don't run initctl in analyzer setup so setup can also be used on travis (and add it to the install script directly)
* [x] ignore replaygain checks on travis (it has proven quite impossible to get the needed python-gi module to work in the provided virtualenv)

I tried a lot of solutions to get the replaygain checks to run. I needed to decide that this has gone far enough, maybe someone who is more of a pythonista than me can take a crack at it and get it solved. Even without running those tests on CI/CD there are still plenty others.

This PR only has parts of what are needed for getting python tests running on travis as per #15. I only took a quick shot at anything not analyzer and figured I would not be able to "fix" them without digging a bit deeper (ie. also getting rid of std_err_override).
2017-03-03 20:38:27 +01:00
Lucas Bickel e28ad471f9 Rewrite config from /etc/airtime-saas to plain /etc/airtime
This is the results of sed -i -e 's|/etc/airtime-saas/|/etc/airtime/|' `grep -irl 'airtime-saas' airtime_mvc/ python_apps/` :P

It might need more testing, the airtime-saas part never really made sense, zf1 has environments for that, ie you would create a saas env based on production for instance.

I beleive legacy upstream was using this to share configuration between customers (ie. analyser runs only once and writes to a shared S3 bucket). I assume they mount the airtime-saas folder onto individual customers instances with a global config. Like I said, I don't feel that this makes sense since all it does is make hacking at the configs in airtime-saas a bit easier. A serious SaaS operation should be using something like puppet or ansible to achieve this.
2017-03-03 15:57:41 +01:00
Albert Santoni a26f1a70e3 Respawn liquidsoap and pypo indefinitely if they fail to launch. Throttled it though. 2015-12-04 15:56:06 -05:00
Albert Santoni 91e1252b86 SAAS-1239: Listener stats with Shoutcast not working 2015-12-04 15:05:17 -05:00
Albert Santoni 2eea558dfd SAAS-1267: AAC streams stuck at 64 kbps, even at higher bitrates 2015-12-04 10:51:07 -05:00
Albert Santoni 965a78c54a Merge branch 'saas-dev' into saas-dev-publishing
Conflicts:
	airtime_mvc/application/controllers/ApiController.php
2015-11-18 12:38:18 -05:00
Duncan Sommerville 071de55329 Add some additional error handling and prevent disabled buttons from being triggered 2015-11-11 19:26:51 -05:00
Albert Santoni bb05fa996a SAAS-1132: Improve safety of XML parsing for listener stats 2015-11-09 19:32:56 -05:00
Albert Santoni c85944785b Comments 2015-10-30 18:09:56 -04:00
Albert Santoni b49bb2e262 Loosen up Mutagen vs. Silan cue out length threshold a bit 2015-10-30 17:54:24 -04:00
Albert Santoni 7be548f30e Fixed the airtime_analyzer unit tests 2015-10-30 17:15:33 -04:00
Albert Santoni fc51e63214 Fix typo in airtime_analyzer setup.py 2015-10-30 17:13:12 -04:00
Albert Santoni 76f202106b Defensive coding against Silan bugs and bump to Mutagen 1.31 2015-10-30 17:12:13 -04:00
Duncan Sommerville 4f281a30ed Add SoundCloud update and download tasks to Celery backend; requires airtime-celery reinstall 2015-10-30 16:10:16 -04:00
Duncan Sommerville e3feb17f0c * Change the Celery timeout to 15 minutes (from 10) to better accommodate automatic ingest
* Fix various small bugs in auto ingestion and tab implementation
* Update TaskManager run conditions to piggyback on API calls - guarantees a certain frequency of requests and greatly reduces chances of lock contention
2015-10-29 17:53:45 -04:00
Duncan Sommerville 0b1df6baf3 * SAAS-1084 - initial work on publishing API backend
* More work on automatic ingest
* Add automatic_ingest_timestamp column to ImportedPodcast
2015-10-20 19:03:34 -04:00
Duncan Sommerville bcbd8a8050 Fix bug in auto ingest, immediately retry on failure 2015-10-19 15:36:34 -04:00
Duncan Sommerville 43e9fb59ce SAAS-1071 - more work on celery backend for podcasts; add upgrade to make file_id field in third_party_track_references nullable 2015-09-24 15:57:38 -04:00
Duncan Sommerville ca51dcf3ae SAAS-1071 - more work on backend podcast implementation 2015-09-24 12:58:02 -04:00
Duncan Sommerville cf1de5fb66 SAAS-1071 - initial work on podcast celery backend; tweak SoundCloud service 2015-09-22 15:26:08 -04:00
Duncan Sommerville a78a0d0cc8 * Fix track number bug in analyzer
* Move library search bar out of header
2015-09-16 17:50:21 -04:00
Duncan Sommerville c8baa6e4b3 Merge branch 'saas-dev-publishing' into soundcloud
Conflicts:
	airtime_mvc/application/views/scripts/playlist/playlist.phtml
	airtime_mvc/application/views/scripts/playlist/smart-block.phtml
	airtime_mvc/public/js/airtime/library/spl.js
2015-09-15 14:03:11 -04:00
Duncan Sommerville dd09676f88 Hide celery user 2015-09-15 13:59:48 -04:00
Duncan Sommerville 63784d4e6b Fix some behaviour with playlist/smartblock fade button 2015-09-08 16:39:33 -04:00
Duncan Sommerville f243014d9f Merge branch 'saas-dev' into soundcloud 2015-09-08 16:01:06 -04:00
Albert Santoni 3592eceb22 Merge branch 'saas' into saas-dev 2015-08-24 17:11:52 -04:00
Albert Santoni f21fcb3dcc SAAS-1011 - Fixed one mimetype problem and fixed installer 2015-08-24 16:25:09 -04:00
Albert Santoni fdb2d4fae2 SAAS-1011: Files on NFS with incorrect file extensions won't play out
* Simplified the mime type detection code in pypo and made it actually work
2015-08-24 11:29:45 -04:00
Albert Santoni dab0f00736 Fixed m4a file extension mapping in pypo 2015-08-11 08:59:44 -04:00
Duncan Sommerville 1816be156f SAAS-922 - change file extensions in pypo based on mime-type, update lists of accepted mime types 2015-07-16 16:26:52 -04:00
Duncan Sommerville 04333aaa29 SoundCloud download initial commit 2015-07-16 11:27:27 -04:00
Albert Santoni 238bb15793 SAAS-941: Liquidsoap fails to start for stations with the substrings true/false in their names 2015-07-09 22:32:34 -04:00
Albert Santoni 67c5b73db4 airtime_analyzer unit test backwards compatibility with older versions of requests 2015-07-08 10:32:35 -04:00
Duncan Sommerville b4599317e9 Run build from installer; add upstart files and templating back in 2015-06-30 12:16:31 -04:00
Duncan Sommerville dd2ffd148d Fix bug where pypo fails to restart liquidsoap; uncomment adding pypo template files 2015-06-29 12:20:39 -04:00
Duncan Sommerville 5cc8774e56 Merge branch 'saas-dev' into soundcloud
Conflicts:
	airtime_mvc/application/configs/constants.php
2015-06-29 10:02:58 -04:00
Duncan Sommerville d9f17f3919 SAAS-871 - Fix consumer cancellation bug 2015-06-26 17:11:04 -04:00
Duncan Sommerville c44bae8140 Merge branch 'saas-dev' into soundcloud
Conflicts:
	airtime_mvc/application/views/scripts/form/preferences.phtml
	airtime_mvc/public/js/airtime/preferences/preferences.js
2015-06-26 10:37:30 -04:00
Albert Santoni d54eb37317 Merge branch 'saas' into saas-dev 2015-06-24 15:19:38 -04:00
Duncan Sommerville eeb1f7eff0 Fix airtime-celery setup 2015-06-24 11:11:46 -04:00
Duncan Sommerville 4c797cf100 Update installer to work with saas branch 2015-06-23 19:02:55 -04:00
Duncan Sommerville 9bce3e6850 Remove user creation from airtime-celery README, automated in setup 2015-06-23 15:11:53 -04:00
Duncan Sommerville 599354c22f Merge branch 'soundcloud' of https://github.com/sourcefabric/Airtime into soundcloud 2015-06-23 15:10:18 -04:00
Duncan Sommerville 70f6cbbc71 Fixes to airtime-celery setup 2015-06-23 15:10:02 -04:00
Albert Santoni fa7e16fc9d Added celery user creation to README 2015-06-23 11:54:08 -04:00
Albert Santoni 195935cccf Slightly better exception handling and logging to help with SAAS-884 2015-06-22 15:49:23 -04:00
Duncan Sommerville 76a7aa9a24 Fix SoundCloud and TaskManager bugs, switch tasks to use acks_late, and provide feedback from SoundCloud context menu items 2015-06-18 18:18:48 -04:00
Duncan Sommerville 17983167ed Add configobj import to setup 2015-06-18 15:14:42 -04:00
Duncan Sommerville 4a92ec4e12 Fix airtime-celery setup 2015-06-18 15:05:32 -04:00
Duncan Sommerville 03d2ad5474 Run initctl and update rc.d when using --no-init-script 2015-06-18 12:08:24 -04:00
Duncan Sommerville 89e3eaa986 Merge branch 'saas-dev' into soundcloud 2015-06-17 17:00:15 -04:00
Duncan Sommerville 92ad5bde6d Add fields to airtime.example.conf for compatibility, remove /etc/init scripts to prevent duplicate processes on startup 2015-06-17 14:49:12 -04:00
Duncan Sommerville 3902c8c746 Add SoundCloud delete functionality and fixes; implement TaskManager to run background jobs 2015-06-15 15:12:37 -04:00
Duncan Sommerville 272911fa7b Remove saas-specific configurations 2015-06-12 12:50:35 -04:00
Duncan Sommerville 15c7ef5885 Celery backend and support for dev-env worker parallelization 2015-06-12 12:31:55 -04:00
Duncan Sommerville 459f9494c3 Merge saas-dev into soundcloud 2015-06-10 16:08:52 -04:00
Duncan Sommerville 626489bb3b SAAS-853 - Celery backend for SoundCloud uploads 2015-06-10 15:04:49 -04:00
Albert Santoni 3996e757b7 Merge branch 'saas-dev' into saas-installer-albert 2015-06-10 12:02:08 -04:00
Duncan Sommerville f031d13867 Merge branch 'saas-dev' into soundcloud 2015-06-09 14:03:03 -04:00
Duncan Sommerville 49165e91d9 SAAS-854 - celery-based SoundCloud backend uploader 2015-06-09 14:02:29 -04:00
Albert Santoni 146146e35f CC-6052: Fix problem with stream passwords 2015-06-08 14:18:19 -04:00
Albert Santoni 01ea6f27ae Merge branch 'saas-dev' into saas-installer-albert
Conflicts:
	python_apps/api_clients/api_clients/api_client.py
2015-06-02 15:40:57 -04:00
Albert Santoni 700fddbada SAAS-818: If the network connection is interrupted, pypo sometimes can't recover 2015-06-02 14:51:20 -04:00
drigato 532bd1ea85 SAAS-772: Send metadata to Tunein
Made Improvement so Airtime always makes a request to TuneIn every 4
minutes so TuneIn does not turn metadata off
2015-05-25 15:37:45 -04:00
Albert Santoni 6fad0a3ee6 Merge branch 'saas-dev' into saas-installer-albert 2015-05-22 19:09:30 -04:00
Albert Santoni fc02de4920 Revert "Add on_metadata wrapper to liquidsoap master/show sources"
This reverts commit 415a0d21f9.
2015-05-22 19:09:04 -04:00
Albert Santoni d3f89c9507 Revert "Add on_metadata wrapper to liquidsoap master/show sources"
This reverts commit 82c801cb79.
2015-05-22 19:08:48 -04:00
Albert Santoni 9ffbdfbfa3 Merge branch '2.5.x-installer' into saas-installer-albert 2015-05-22 18:43:28 -04:00
Albert Santoni b81069ace3 Added missing requests dependency to pypo's requirements 2015-05-22 18:43:05 -04:00
Albert Santoni 6ca0e340a4 Merge branch '2.5.x-installer' into saas-installer-albert
Conflicts:
	python_apps/pypo/pypo/pypofile.py
2015-05-22 18:26:16 -04:00
Albert Santoni 027adc7749 Three bugfixes
* Removed Support Settings nav link
* SAAS-801: pypo log configuration files are overkill and must be removed
* CC-6043: CTRL-C doesn't work in pypo / airtime-playout
2015-05-22 18:20:45 -04:00
Albert Santoni 11c6818e61 Merge branch '2.5.x-installer' into saas-installer-albert
Conflicts:
	.gitignore
	airtime_mvc/application/Bootstrap.php
	airtime_mvc/application/configs/conf.php
	airtime_mvc/application/controllers/SystemstatusController.php
	airtime_mvc/application/controllers/UpgradeController.php
	airtime_mvc/application/upgrade/Upgrades.php
	airtime_mvc/application/views/scripts/systemstatus/index.phtml
	airtime_mvc/build/airtime.conf
	airtime_mvc/build/sql/defaultdata.sql
	airtime_mvc/public/index.php
	airtime_mvc/tests/application/helpers/AirtimeInstall.php
	install_minimal/airtime-install
	install_minimal/include/airtime-constants.php
	install_minimal/include/airtime-copy-files.sh
	install_minimal/include/airtime-db-install.php
	install_minimal/include/airtime-initialize.sh
	install_minimal/include/airtime-install.php
	install_minimal/include/airtime-installed-check.php
	install_minimal/include/airtime-remove-files.sh
	install_minimal/include/airtime-upgrade.php
	python_apps/media-monitor/install/media-monitor-copy-files.py
	python_apps/monit/monit-airtime-generic.cfg
	python_apps/pypo/airtime-playout
	python_apps/pypo/install/pypo-copy-files.py
	python_apps/pypo/liquidsoap/generate_liquidsoap_cfg.py
	python_apps/pypo/liquidsoap/ls_script.liq
	python_apps/pypo/pypo/__main__.py
	python_apps/pypo/pypo/media/update/replaygain.py
	python_apps/pypo/pypo/media/update/replaygainupdater.py
	python_apps/pypo/pypo/media/update/silananalyzer.py
	python_apps/python-virtualenv/airtime_virtual_env.pybundle
	python_apps/python-virtualenv/requirements
	utils/airtime-check-system.php
2015-05-22 16:05:29 -04:00
Albert Santoni 16fe0aa177 Merge branch 'saas' into saas-dev 2015-05-21 15:05:20 -04:00
Duncan Sommerville 196e08fc90 Fix bug where users can't connect to Master source due to hardcoded path in liquidsoap 2015-05-21 12:49:26 -04:00
Albert Santoni c7aa6b3e5e airtime_analyzer: use requests >= 2.7.0 2015-05-20 10:36:55 -04:00
Albert Santoni 3d8cd8e927 Added daemon dependency back in for airtime_analyzer 2015-05-19 13:45:08 -04:00
Albert Santoni e1c50ecc7b Squash urllib3 warnings to see if it helps with SAAS-714 2015-05-19 13:09:43 -04:00
Albert Santoni 1c88d6dccd Merge branch 'saas' into saas-dev 2015-05-19 12:18:54 -04:00
Albert Santoni add211d13c Cleanup to help with SAAS-714 2015-05-19 12:02:19 -04:00
Duncan Sommerville 415a0d21f9 Add on_metadata wrapper to liquidsoap master/show sources 2015-05-15 15:46:45 -04:00
Duncan Sommerville 82c801cb79 Add on_metadata wrapper to liquidsoap master/show sources 2015-05-15 15:15:49 -04:00
Albert Santoni b3a19ae49e Bugfix for embedded player sending too many requests on load, quieter logging for boto 2015-04-29 11:46:44 -04:00
drigato b48d10a45e Merge branch 'saas' into saas-embed-player 2015-04-13 09:03:59 -04:00
Albert Santoni 28fc1cf2fc Fixed some airtime_analyzer unit tests 2015-04-10 15:08:44 -04:00
Albert Santoni e3390b99ef Disable analyzer log file rotation (already handed by Upstart), clean up 2015-04-10 15:03:58 -04:00
drigato fbcd5bc5bd Merge branch 'saas' into saas-embed-player 2015-04-09 15:09:03 -04:00