Commit Graph

45 Commits

Author SHA1 Message Date
Albert Santoni e3390b99ef Disable analyzer log file rotation (already handed by Upstart), clean up 2015-04-10 15:03:58 -04:00
Albert Santoni d5012c25cb Another small bugfix for error handling in the analyzer 2015-04-06 17:33:08 -04:00
Albert Santoni e182e73626 SAAS-602: airtime_analyzer deadlocks in S3 hostname lookup
* Stopped using multiprocess.subprocess because it's dangerous and
  unreliable due to Python bug 6721: http://bugs.python.org/issue6721
2015-03-11 18:33:08 -04:00
Albert Santoni edd07c18ae Fixed invalid detection of S3 backend in analyzer 2015-02-26 12:02:02 -05:00
Albert Santoni 51aa846a4c Merge branch 'cc-5709-airtime-analyzer-refactor' into saas-media-refactor
Conflicts:
	airtime_mvc/application/models/RabbitMq.php
	python_apps/airtime_analyzer/airtime_analyzer/analyzer_pipeline.py
	python_apps/airtime_analyzer/airtime_analyzer/message_listener.py
2015-02-20 18:25:38 -05:00
Albert Santoni c868136d26 Make airtime_analyzer respect the storage_backend setting passed to it
by Airtime
2015-02-20 18:21:49 -05:00
drigato e6171680ad SAAS-560: Deploy separate cloud storage config files for each development environment
Small fix
2015-02-04 14:34:04 -05:00
drigato c6fc184559 Merge branch 'saas' into saas-cloud-storage-config-per-dev-env
Conflicts:

python_apps/airtime_analyzer/airtime_analyzer/analyzer_pipeline.py

python_apps/airtime_analyzer/airtime_analyzer/cloud_storage_uploader.py

python_apps/airtime_analyzer/airtime_analyzer/message_listener.py
2015-02-04 13:36:35 -05:00
drigato 7b3f9af04c SAAS-560: Deploy separate cloud storage config files for each development environment
Changed analyzer upstart to take the cloud storage config file as a command line option
Dropped the dev env portion from the rabbitmq-analyzer.ini filename
2015-02-03 15:55:47 -05:00
drigato 7808fd4708 More analyzer log statements 2015-02-03 13:54:25 -05:00
drigato 3127507124 Analyzer log statements for debugging 2015-02-03 13:21:07 -05:00
Albert Santoni 2793509781 Merge branch 'cc-5709-airtime-analyzer-cloud-storage' into saas
Conflicts:
	python_apps/airtime_analyzer/airtime_analyzer/analyzer_pipeline.py
	python_apps/airtime_analyzer/airtime_analyzer/message_listener.py
2015-02-02 13:17:26 -05:00
Albert Santoni 9baed1b328 Make the unit tests pass again 2015-02-02 13:14:14 -05:00
drigato 3f97ed3d55 Merge branch 'cc-5709-airtime-analyzer' into cc-5709-airtime-analyzer-cloud-storage 2015-01-27 16:46:06 -05:00
drigato ea868607b4 Enabling cuepoint analyzer because the silan bug has been fixed 2015-01-27 16:44:21 -05:00
drigato 102c78474d Enabling cuepoint analyzer because the silan bug has been fixed 2015-01-27 13:21:44 -05:00
drigato 43c6475f1b Merge branch 'cc-5709-airtime-analyzer' into cc-5709-airtime-analyzer-cloud-storage 2015-01-26 09:18:32 -05:00
drigato 6adaf7a91a Disabling cue point analyzer 2015-01-26 09:17:53 -05:00
drigato a686eabb96 Disabling cue point analyzer 2015-01-23 17:33:57 -05:00
drigato c711adc48d Merge branch 'cc-5709-airtime-analyzer-cloud-storage' into cc-5709-airtime-analyzer-cloud-storage-saas
Conflicts:
	python_apps/airtime_analyzer/airtime_analyzer/analyzer_pipeline.py
	python_apps/airtime_analyzer/airtime_analyzer/cloud_storage_uploader.py
	python_apps/airtime_analyzer/airtime_analyzer/message_listener.py
2014-12-19 16:06:23 -05:00
Albert Santoni 3f5b4faf1c Cloud storage cleanup and fixed the unit tests 2014-12-19 12:58:55 -05:00
drigato 19e6f1a535 SAAS-526: Change prefix from station name to station id
Done.
2014-12-17 16:48:20 -05:00
drigato 1de326283e SAAS-527: Allow files to be uploaded to either the cloud or on local file storage
Not quite done.
2014-12-16 18:47:42 -05:00
drigato 29d3d877ab Merge branch 'cc-5709-airtime-analyzer' into cc-5709-airtime-analyzer-cloud-storage
Conflicts:
	python_apps/airtime_analyzer/airtime_analyzer/analyzer_pipeline.py
	python_apps/airtime_analyzer/setup.py
2014-12-12 09:36:27 -05:00
Albert Santoni dbfb2fc69b CC-5739: Added Liquidsoap playability test to airtime_analyzer 2014-12-11 15:45:45 -05:00
Albert Santoni 4dd2768755 ReplayGain analysis in airtime_analyzer using python-rgain
* Implemented Replaygain analysis for Ogg Vorbis, MP3, MP4, and FLAC
  using python-rgain
* Added unit tests for ReplayGainAnalyzer
* Squashed stderr output in ReplayGainAnalyzer and CuePointAnalyzer
* Clean up
2014-12-11 14:12:41 -05:00
Albert Santoni 38bd45b8dc Silan cue-in/out analysis now moved to airtime_analyzer
* Added a new CuePointAnalyzer using silan (will remove silan stuff from pypo later)
* Makes silan analysis slightly more reliable for certain short files.
* Fixes CC-5961: Audio duration cutoff for WAVE files
* Added unit tests for the new analyzer and improved code coverage slightly
* Added unit tests for WAVE metadata extraction and an invalid WMA file
2014-12-10 18:44:35 -05:00
drigato 432245b18e SAAS-502: Analyzer -> Set the station id and domain in the cloud object's metadata
Set the domain name in the cloud object's metadata
2014-11-27 16:54:22 -05:00
Albert Santoni 32bdbe1ad6 Improved some comments 2014-11-14 11:12:10 -05:00
drigato e24b4f43fd Made AirtimeAnalyzerServer.read_config_file static so CloudStorageUploader can use it.
Reverted MessageListener.msg_received_callback and MessageListener.spawn_analyzer_process back to static methods.
Moved cloud storage settings out of MessageListener and into CloudStorageUploader
2014-11-05 16:25:11 -05:00
drigato 039a51121b CC-5885: Factor out cloud storage code into separate class 2014-07-11 16:16:30 -04:00
drigato 1393bad014 Modified config file variables to describe not only rabbitmq settings
Removed filemover_analyzer functionality and replaced with an uploader function into Amazon s3
2014-07-04 16:38:58 -04:00
drigato c291439483 Adding back analyzer deadlock workaround.
Removing command to delete empty sub folders inside organize.
2014-05-07 15:01:31 -04:00
drigato 81dd1deba3 Removing something for testing 2014-05-07 13:12:18 -04:00
Albert Santoni 56c3d8070b Alternate bugfix for logging deadlock in airtime_analyzer 2014-05-06 15:01:25 -04:00
Albert Santoni ed494ac587 Some defensive code against multiprocess related deadlocks
* Reinitialize logging in child processes so we don't inherit locks.
  Might be causing a deadlock we're seeing on Pro right now.
2014-05-05 18:25:47 -04:00
Albert Santoni b0670f5f47 CC-5709: Airtime Analyzer
* Log tracebacks from child process
2014-04-16 13:05:02 -04:00
Albert Santoni 95b369c54d CC-5709: Airtime Analyzer
* Remove awful StoredFile::uploadFile() function
* Massive airtime_analyzer commenting and cleanup
* Cleaned up the upload code
* Temporarily disabled the liquidsoap playability test.
2014-04-03 16:13:26 -04:00
Albert Santoni 58a38ffd3a CC-5709: Airtime Analyzer
* Added unit test coverage report stuff
* Added more unit tests
* 98% test coverage :-)
* Fixed a few bugs and removed some dead code
2014-03-26 15:06:48 -04:00
Albert Santoni 61c2c90b7e CC-5709: Airtime Analyzer
* Remove the "hidden" field from the REST blacklist, the analyzer needs to set it.
* Added import_status column messages in the recent uploads table
* Auto-refresh the recent uploads table while imports are pending
* Moved the file moving stuff to its own analyzer in airtime_analyzer
* Basic error reporting to the REST API in airtime_analyzer, needs
  hardeneing though
* Fixed a bug with the number of recent uploads
* Prevent airtime_analyzer from running if media_monitor is running
2014-03-22 02:12:03 -04:00
Albert Santoni 878dd11ccc CC-5709: Airtime Analyzer
* Overhauled Add Media screen, now shows state of recent uploads
* Dropped old unused "state" column, added new file_import column to cc_files
* New PluploadController methods
* Save the filename as the track title for unprocessed uploads
* Hide pending files from the library until they've been processed.
* Don't overwrite files with duplicate names, we rename them instead.
2014-03-21 13:22:00 -04:00
Albert Santoni 50a42f12bb CC-5709: Airtime Analyzer
* File import via the Add Media page now works!
* Reworked StoredFile::copyFileToStor() because it's still needed.
* Reworked the way file paths are reported in airtime_analyzer
  and in the RESTful media API
2014-03-17 10:19:39 -04:00
Albert Santoni 6d7117f670 CC-5709: Airtime Analyzer
* Added MetadataAnalyzer unit tests and test data
* Improved debug logging and squashed pika logging
* Implemented file moving
* Extract the track number/total
* Fixed mapping of mutagen to Airtime fields in a few spots. The mapping
  matches the DB column names now.
* Fixed the bin/airtime_analyzer binary
* Started work on PluploadController to make it work with the new File
  API
2014-03-10 16:32:23 -04:00
Albert Santoni 59535850e2 CC-5709: Airtime Analyzer Queue
* Added StatusReporter class to make HTTP requests back to Airtime
* Handle even more errors now
* Added proper logging and log rotation
* Added --debug flag for increased logging.
2014-03-05 22:43:47 -05:00
Albert Santoni a6a64a2b9e CC-5709 / CC-5705 : Airtime Analyzer
* Finished the skeleton of the airtime_analyzer service.
* Basic round-robin, reliable AMQP messaging works.
* Using multiprocess arch so the daemon survives analyzer crashes and
  avoids failures propagating to other nodes.
* Basic metadata extractor using Mutagen is done.
* HTTP requests to the File API to are next to come...
2014-03-05 12:15:25 -05:00