Commit Graph

190 Commits

Author SHA1 Message Date
drigato 7b80915c16 Merge branch 'cc-5709-airtime-analyzer-cloud-storage' into cc-5709-airtime-analyzer-cloud-storage-saas 2014-12-12 09:44:54 -05:00
drigato e7a742dbf4 Fixed merge error 2014-12-12 09:38:24 -05:00
drigato 29d3d877ab Merge branch 'cc-5709-airtime-analyzer' into cc-5709-airtime-analyzer-cloud-storage
Conflicts:
	python_apps/airtime_analyzer/airtime_analyzer/analyzer_pipeline.py
	python_apps/airtime_analyzer/setup.py
2014-12-12 09:36:27 -05:00
Albert Santoni dbfb2fc69b CC-5739: Added Liquidsoap playability test to airtime_analyzer 2014-12-11 15:45:45 -05:00
Albert Santoni 3b1c776879 Removed all ReplayGain and Silan stuff from pypo, plus a bugfix
* Ripped out all the ReplayGain and Silan analysis from pypo, since
  it's now implemented in airtime_analyzer. This fixes a bunch of
  race conditions.
* Also renamed the replaygain field to replay_gain in airtime_analyzer
  to match Airtime.
2014-12-11 14:26:05 -05:00
Albert Santoni 4dd2768755 ReplayGain analysis in airtime_analyzer using python-rgain
* Implemented Replaygain analysis for Ogg Vorbis, MP3, MP4, and FLAC
  using python-rgain
* Added unit tests for ReplayGainAnalyzer
* Squashed stderr output in ReplayGainAnalyzer and CuePointAnalyzer
* Clean up
2014-12-11 14:12:41 -05:00
Albert Santoni 38bd45b8dc Silan cue-in/out analysis now moved to airtime_analyzer
* Added a new CuePointAnalyzer using silan (will remove silan stuff from pypo later)
* Makes silan analysis slightly more reliable for certain short files.
* Fixes CC-5961: Audio duration cutoff for WAVE files
* Added unit tests for the new analyzer and improved code coverage slightly
* Added unit tests for WAVE metadata extraction and an invalid WMA file
2014-12-10 18:44:35 -05:00
drigato 57868d2351 Merge branch 'cc-5709-airtime-analyzer-cloud-storage' into cc-5709-airtime-analyzer-cloud-storage-saas 2014-12-10 12:47:22 -05:00
drigato d73b331376 SAAS-515: Prefix object names with station name 2014-12-10 12:40:08 -05:00
drigato cf9eabbc74 Merge branch 'cc-5709-airtime-analyzer-cloud-storage' into cc-5709-airtime-analyzer-cloud-storage-saas
Conflicts:
	airtime_mvc/application/Bootstrap.php
	airtime_mvc/application/models/Schedule.php
	airtime_mvc/application/modules/rest/controllers/MediaController.php
	airtime_mvc/build/sql/schema.sql
	python_apps/airtime_analyzer/airtime_analyzer/analyzer_pipeline.py
2014-12-03 13:35:32 -05:00
drigato 432245b18e SAAS-502: Analyzer -> Set the station id and domain in the cloud object's metadata
Set the domain name in the cloud object's metadata
2014-11-27 16:54:22 -05:00
drigato 92feacd46f SAAS-501: Re-jig cloud_storage.conf 2014-11-27 15:50:40 -05:00
drigato 477ac337e5 Merge branch 'saas' into cc-5709-airtime-analyzer-cloud-storage-saas
Conflicts:
	airtime_mvc/application/models/StoredFile.php
	python_apps/airtime_analyzer/airtime_analyzer/analyzer_pipeline.py
2014-11-26 11:54:07 -05:00
drigato 884a4c493d Merge branch 'cc-5709-airtime-analyzer' into cc-5709-airtime-analyzer-cloud-storage
Conflicts:
	airtime_mvc/application/models/StoredFile.php
	airtime_mvc/application/modules/rest/controllers/MediaController.php
	python_apps/airtime_analyzer/airtime_analyzer/analyzer_pipeline.py
2014-11-26 11:42:40 -05:00
Albert Santoni cbb8ac7dc6 Merge branch 'saas' of github.com:sourcefabric/Airtime into saas 2014-11-18 17:45:00 -05:00
Albert Santoni 162995bb49 Merge branch 'cc-5709-airtime-analyzer' into saas 2014-11-18 17:14:01 -05:00
drigato 2cfe08b33a Updated FTP upload hook script with new vhost.map file location 2014-11-18 14:58:27 -05:00
drigato 4d8b32556b Merge branch 'saas' into cc-5709-airtime-analyzer-cloud-storage-saas
Conflicts:
	airtime_mvc/application/modules/rest/controllers/MediaController.php
2014-11-18 09:24:38 -05:00
Albert Santoni 32bdbe1ad6 Improved some comments 2014-11-14 11:12:10 -05:00
Albert Santoni 71f2cc11d2 Fix for incorrect watchdog disabling 2014-11-12 17:04:21 -05:00
Albert Santoni 864352c6a7 Merge branch 'cc-5709-airtime-analyzer' into saas
Conflicts:
	python_apps/airtime_analyzer/airtime_analyzer/status_reporter.py
2014-11-12 17:04:00 -05:00
Albert Santoni 8eaeaf71f9 airtime_analyzer status_reporter shutdown fix 2014-11-12 17:01:59 -05:00
Albert Santoni aff6104839 Disable airtime_analyzer watchdog 2014-11-12 16:12:33 -05:00
drigato 97b74a0997 Merge branch 'cc-5709-airtime-analyzer-cloud-storage' into cc-5709-airtime-analyzer-cloud-storage-saas
Conflicts:
	airtime_mvc/application/models/Schedule.php
2014-11-12 15:08:35 -05:00
drigato f1fa4929ae Merge branch 'cc-5709-airtime-analyzer' into cc-5709-airtime-analyzer-cloud-storage
Conflicts:
	python_apps/airtime_analyzer/setup.py
2014-11-12 14:51:18 -05:00
Albert Santoni 24cae3b3d6 Merge branch 'cc-5709-airtime-analyzer' into saas 2014-11-11 18:41:26 -05:00
Albert Santoni b324031ee6 Disable those new requirements because pip requires gcc to install them 2014-11-11 18:41:09 -05:00
drigato 49f3881da0 rmq_config_path fix 2014-11-11 13:56:02 -05:00
drigato eb0f12c709 Change back rabbitmq config file param name 2014-11-11 13:55:46 -05:00
Albert Santoni 024923ba6a Disable SSL verification for SaaS as a temporary workaround 2014-11-11 12:18:20 -05:00
Albert Santoni 5f27e3099a Merge branch 'cc-5709-airtime-analyzer' into saas 2014-11-11 11:19:31 -05:00
Albert Santoni 63b357f0ee Updated requirements for airtime_analyzer to make python-requests
support SNI properly
2014-11-11 11:18:55 -05:00
drigato 41acfd3840 rmq_config_path fix 2014-11-10 12:35:41 -05:00
drigato 413c7fd604 Change back rabbitmq config file param name 2014-11-10 12:25:40 -05:00
drigato da42e39178 Merge branch 'cc-5709-airtime-analyzer-cloud-storage' into cc-5709-airtime-analyzer-cloud-storage-saas 2014-11-07 12:49:27 -05:00
drigato ecc0225ff5 Added apache-libcloud dependency to airtime_analyzer 2014-11-07 12:49:10 -05:00
drigato e24b4f43fd Made AirtimeAnalyzerServer.read_config_file static so CloudStorageUploader can use it.
Reverted MessageListener.msg_received_callback and MessageListener.spawn_analyzer_process back to static methods.
Moved cloud storage settings out of MessageListener and into CloudStorageUploader
2014-11-05 16:25:11 -05:00
drigato 6adb33e89b Made AirtimeAnalyzerServer.read_config_file static so CloudStorageUploader can use it.
Reverted MessageListener.msg_received_callback and MessageListener.spawn_analyzer_process back to static methods.
Moved cloud storage settings out of MessageListener and into CloudStorageUploader
2014-11-05 16:24:03 -05:00
drigato 0a45de7fba Merge branch 'saas' into cc-5709-airtime-analyzer-cloud-storage-saas
Conflicts:
	.gitignore
	airtime_mvc/application/configs/airtime-conf.php
	airtime_mvc/application/configs/classmap-airtime-conf.php
	airtime_mvc/application/models/RabbitMq.php
	airtime_mvc/application/models/Schedule.php
	airtime_mvc/application/models/StoredFile.php
	airtime_mvc/application/models/airtime/map/CcShowInstancesTableMap.php
	airtime_mvc/application/models/airtime/map/CcShowTableMap.php
	airtime_mvc/application/models/airtime/om/BaseCcShow.php
	airtime_mvc/application/models/airtime/om/BaseCcShowInstances.php
	airtime_mvc/application/models/airtime/om/BaseCcShowInstancesPeer.php
	airtime_mvc/application/models/airtime/om/BaseCcShowInstancesQuery.php
	airtime_mvc/application/models/airtime/om/BaseCcShowPeer.php
	airtime_mvc/application/models/airtime/om/BaseCcShowQuery.php
	airtime_mvc/application/modules/rest/Bootstrap.php
	airtime_mvc/application/modules/rest/controllers/MediaController.php
	airtime_mvc/build/sql/schema.sql
	airtime_mvc/public/index.php
2014-11-03 12:29:58 -05:00
drigato 01ad2ab832 Commented cloud storage classes 2014-10-29 17:42:42 -04:00
drigato 978fc43a82 Started commenting CloudStorageUploader class 2014-10-29 10:47:32 -04:00
drigato 70ff67374b Merge branch 'cc-5709-airtime-analyzer' into cc-5709-airtime-analyzer-cloud-storage
Conflicts:
	python_apps/airtime_analyzer/airtime_analyzer/filemover_analyzer.py
2014-10-22 17:16:46 -04:00
Albert Santoni 363d022b0c Merge branch 'cc-5709-airtime-analyzer' into saas 2014-10-22 11:39:52 -04:00
Albert Santoni 53dc92b204 StatusReporter exception handler to prevent thread from dying under any circumstances 2014-10-22 11:39:22 -04:00
Albert Santoni 54523e264c Fix rare exception during shutdown in airtime_analyzer 2014-10-22 11:38:22 -04:00
Albert Santoni c4af1b6b19 Merge branch 'cc-5709-airtime-analyzer' into saas 2014-10-21 19:24:03 -04:00
Albert Santoni 82f251f061 Fix invalid python in StatusReporter 2014-10-21 19:23:48 -04:00
Albert Santoni a1ef9d60c0 Merge branch 'cc-5709-airtime-analyzer' into saas 2014-10-21 15:36:51 -04:00
Albert Santoni 28be5c6bd3 CC-5929: Certain long filenames can result in the extension getting cut off 2014-10-21 15:34:10 -04:00
drigato 544d20ebc2 Replaced spaces with dashes in resource_id generation
Fixed track preview and download signed urls
Removed cloud_storage_deleter.py file
Changed cloud_storage settings in airtime.conf
2014-10-17 17:16:53 -04:00
Robert Elder 9fb35c448e Resolved merge conflicts for merging cc-5709-airtime-analyzer into saas. 2014-10-10 19:57:22 +00:00
Albert Santoni 081ca0a6a4 Reworked upstart config for airtime_analyzer 2014-10-10 11:28:44 -04:00
drigato ca62086e16 Removed cloud file deletion from Airtime Analyzer. Deletion is done from PHP now. 2014-08-12 12:59:50 -04:00
drigato dd37ffbdd7 CC-5888: Handle file deletion if the file is stored in the cloud
Execute a python script that deletes a file from the cloud
2014-07-31 23:11:49 -04:00
drigato ecb072b84c CC-5896: Store cloud files in separate table, inherited from cc_files
Fixed broken metadata display on Library page
Fixed broken download/preview option from Library page
2014-07-28 15:17:46 -04:00
drigato f1ea100411 CC-5896: Store cloud files in separate table, inherited from cc_files 2014-07-24 16:56:15 -04:00
Albert Santoni 46542d2b2b Merge branch 'cc-5709-airtime-analyzer-cloud-storage' of github.com:sourcefabric/Airtime into cc-5709-airtime-analyzer-cloud-storage 2014-07-16 18:09:38 -04:00
Albert Santoni c09457ce7c Merge branch 'cc-5709-airtime-analyzer' into cc-5709-airtime-analyzer-cloud-storage 2014-07-16 15:03:39 -04:00
drigato 3f676be66c Use a different libcloud function for uploading files 2014-07-16 14:43:55 -04:00
drigato 7ca6b91cdf CC-5888: Handle file deletion if the file is stored in the cloud 2014-07-16 12:03:22 -04:00
drigato aaee522ec6 CC-5888: Handle file deletion if the file is stored in the cloud 2014-07-15 17:20:04 -04:00
drigato 8e714bcb64 CC-5888: Handle file deletion if the file is stored in the cloud 2014-07-15 07:26:39 -04:00
drigato 039a51121b CC-5885: Factor out cloud storage code into separate class 2014-07-11 16:16:30 -04:00
Albert Santoni bbade8401c FTP upload hook bugfix for files with commas in their names (CURL
restriction)
2014-07-10 16:43:13 -04:00
Albert Santoni 08f4610aec Fix CURL error handling in FTP upload hook 2014-07-10 12:12:16 -04:00
Albert Santoni e84f765696 Better FileMover permissions unit test (passes as root) 2014-07-07 15:53:25 -04:00
drigato 1393bad014 Modified config file variables to describe not only rabbitmq settings
Removed filemover_analyzer functionality and replaced with an uploader function into Amazon s3
2014-07-04 16:38:58 -04:00
Albert Santoni 5e7fdb9606 Don't retry Analyzer HTTP requests if the web app is broken 2014-06-04 13:46:25 -04:00
Albert Santoni a969f8fc44 CC-5856: Analyzer metadata exception
* Fixed parsing of metadata when there's empty lists returned (mutagen's
  API is awful)
* Return HTTP 422 if there's any exception in validating the metadata
2014-06-03 12:43:01 -04:00
Albert Santoni 4548123ad9 CC-5861: Long file paths will cause analyzer error
* Fixed it
2014-06-03 11:41:45 -04:00
Albert Santoni 1373d4984f CC-5862: Invalid UTF-8 chars cause DB error
* Strip and validate UTF-8 strings in the Media API
* Also properly parse track numbers containing "-"
2014-05-30 13:02:19 -04:00
Albert Santoni d21cb596cd CC-5860: Analyzer HTTP requests can hang indefinitely
* Log a backtrace when sent the USR2 signal (kill -SIGUSR2 <pid>)
* Rigged up up something to strace and restart when we see a request hanging
2014-05-26 18:59:28 -04:00
drigato 0040965222 CC-5840: Add Media -> Endless retries if there is a validation error
Fixed by not adding message to retry queue if request returns a
validation specific error code (422)
2014-05-08 13:01:49 -04:00
drigato c291439483 Adding back analyzer deadlock workaround.
Removing command to delete empty sub folders inside organize.
2014-05-07 15:01:31 -04:00
drigato 81dd1deba3 Removing something for testing 2014-05-07 13:12:18 -04:00
Albert Santoni 56c3d8070b Alternate bugfix for logging deadlock in airtime_analyzer 2014-05-06 15:01:25 -04:00
Albert Santoni ed494ac587 Some defensive code against multiprocess related deadlocks
* Reinitialize logging in child processes so we don't inherit locks.
  Might be causing a deadlock we're seeing on Pro right now.
2014-05-05 18:25:47 -04:00
Albert Santoni d063700254 CC-5834: Airtime Analyzer: Bitrate is missing
* Fixed
2014-05-05 14:56:32 -04:00
Albert Santoni 22343ed99c Fixed bitrate import in airtime_analyzer 2014-05-05 14:39:36 -04:00
Albert Santoni c11e39f933 Fixed python-magic dependency 2014-04-30 18:46:37 -04:00
Albert Santoni abcdf5ea6a CC-5824: Airtime Analyzer: Flac file was uploaded but AA says it's
failed

* Fixed
2014-04-30 18:42:13 -04:00
Albert Santoni ace03edd2e Merge branch 'cc-5709-airtime-analyzer' of github.com:sourcefabric/Airtime into cc-5709-airtime-analyzer 2014-04-30 18:32:42 -04:00
Albert Santoni 26a39ed2db CC-5823: Airtime Analyzer: Wav file uploading fails with zero length and
empty mime

* Fixed
2014-04-30 18:32:17 -04:00
Albert Santoni c2f6032c8b Fixed the upstart config 2014-04-30 17:56:02 -04:00
drigato f9f9e2458e Merge branch 'cc-5709-airtime-analyzer' of github.com:sourcefabric/Airtime into cc-5709-airtime-analyzer 2014-04-28 14:45:05 -04:00
drigato fae5b15a9f CC-5821: Airtime Analyzer: Ftp-uploaded folder doesn't get removed from organize
$_FILES does not store the original file path so we were losing the folder name.
Fixed by explicitly passing the full filepath in via the ftp-upload-hook.sh script
2014-04-28 14:44:52 -04:00
Albert Santoni 375ec54778 CC-5820: Airtime Analyzer: Length and MIME parsing skipped if file has
no audio metadata tags.
2014-04-28 11:54:59 -04:00
Albert Santoni e503c952d2 CC-5816: Publisher column
* Kanye West support. Surely
2014-04-24 17:05:38 -04:00
Albert Santoni bc72a6626e CC-5709: Airtime Analyzer
* Fixed some unit tests that broken when I dropped python-magic in
  favour of mutagen for MIME type detection
2014-04-23 18:20:50 -04:00
Albert Santoni 9fdfc0a957 CC-5709: Airtime Analyzer
* Catch exceptions related to unpickling the HTTP request queue
2014-04-23 17:42:09 -04:00
Albert Santoni f2ce77fecb Remove python-magic 2014-04-23 14:20:48 -04:00
Albert Santoni d77f107df5 CC-5808: Fixed breakage at shutdown 2014-04-22 15:03:46 -04:00
Albert Santoni 6a86a75cee CC-5709: Airtime Analyzer
* Fixed CC-5799: Incorrect Mime on import
2014-04-21 17:50:10 -04:00
Albert Santoni cdce4ca104 CC-5709: Airtime Analyzer
* Fixed the unit tests
* Improved length testing because it seems to fluctuate a bit with
  your mutagen version (great, eh?)
2014-04-16 16:42:37 -04:00
Albert Santoni 9a3c88f009 CC-5709: Airtime Analyzer
* Explicitly set the locale in upstart conf to avoid UTF-8/unicode
  errors from python. Fixes CC-5794.
2014-04-16 15:33:02 -04:00
Albert Santoni b0670f5f47 CC-5709: Airtime Analyzer
* Log tracebacks from child process
2014-04-16 13:05:02 -04:00
Albert Santoni 4d1843ca1f Merge branch 'cc-5709-airtime-analyzer' of github.com:sourcefabric/Airtime into cc-5709-airtime-analyzer 2014-04-16 11:11:24 -04:00
Albert Santoni f491211167 CC-5709: Airtime Analyzer
* Retry HTTP requests (kinda crappy, should find a better design pattern to
  solve this) - CC-5775
* Persist failed HTTP requests at shutdown
2014-04-16 11:10:06 -04:00
drigato 7aba416c3b Small fix to ftp hook script.
Removed space that was getting added to the api key
2014-04-16 10:54:55 -04:00
Albert Santoni e4af3a5a0e CC-5709: Airtime Analyzer
* Better SIGTERM handling for airtime_analyzer
* Nuke the .identifier files saved by Airtime
2014-04-08 17:28:50 -04:00