Congegni
/
sintonia
7
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,260 Commits 3 Branches 0 Tags 405 MiB
Commit Graph

239 Commits

Author SHA1 Message Date
Albert Santoni 081ca0a6a4 Reworked upstart config for airtime_analyzer 2014-10-10 11:28:44 -04:00
drigato ca62086e16 Removed cloud file deletion from Airtime Analyzer. Deletion is done from PHP now. 2014-08-12 12:59:50 -04:00
drigato dd37ffbdd7 CC-5888: Handle file deletion if the file is stored in the cloud 
Execute a python script that deletes a file from the cloud
 2014-07-31 23:11:49 -04:00
drigato ecb072b84c CC-5896: Store cloud files in separate table, inherited from cc_files 
Fixed broken metadata display on Library page
Fixed broken download/preview option from Library page
 2014-07-28 15:17:46 -04:00
drigato f1ea100411 CC-5896: Store cloud files in separate table, inherited from cc_files 2014-07-24 16:56:15 -04:00
Albert Santoni 46542d2b2b Merge branch 'cc-5709-airtime-analyzer-cloud-storage' of github.com:sourcefabric/Airtime into cc-5709-airtime-analyzer-cloud-storage 2014-07-16 18:09:38 -04:00
Albert Santoni c09457ce7c Merge branch 'cc-5709-airtime-analyzer' into cc-5709-airtime-analyzer-cloud-storage 2014-07-16 15:03:39 -04:00
drigato 3f676be66c Use a different libcloud function for uploading files 2014-07-16 14:43:55 -04:00
drigato 7ca6b91cdf CC-5888: Handle file deletion if the file is stored in the cloud 2014-07-16 12:03:22 -04:00
drigato aaee522ec6 CC-5888: Handle file deletion if the file is stored in the cloud 2014-07-15 17:20:04 -04:00
drigato 8e714bcb64 CC-5888: Handle file deletion if the file is stored in the cloud 2014-07-15 07:26:39 -04:00
drigato 039a51121b CC-5885: Factor out cloud storage code into separate class 2014-07-11 16:16:30 -04:00
Albert Santoni bbade8401c FTP upload hook bugfix for files with commas in their names (CURL 
restriction)
 2014-07-10 16:43:13 -04:00
Albert Santoni 08f4610aec Fix CURL error handling in FTP upload hook 2014-07-10 12:12:16 -04:00
Albert Santoni e84f765696 Better FileMover permissions unit test (passes as root) 2014-07-07 15:53:25 -04:00
drigato 1393bad014 Modified config file variables to describe not only rabbitmq settings 
Removed filemover_analyzer functionality and replaced with an uploader function into Amazon s3
 2014-07-04 16:38:58 -04:00
Albert Santoni 5e7fdb9606 Don't retry Analyzer HTTP requests if the web app is broken 2014-06-04 13:46:25 -04:00
Albert Santoni a969f8fc44 CC-5856: Analyzer metadata exception 
* Fixed parsing of metadata when there's empty lists returned (mutagen's
  API is awful)
* Return HTTP 422 if there's any exception in validating the metadata
 2014-06-03 12:43:01 -04:00
Albert Santoni 4548123ad9 CC-5861: Long file paths will cause analyzer error 
* Fixed it
 2014-06-03 11:41:45 -04:00
Albert Santoni 1373d4984f CC-5862: Invalid UTF-8 chars cause DB error 
* Strip and validate UTF-8 strings in the Media API
* Also properly parse track numbers containing "-"
 2014-05-30 13:02:19 -04:00
Albert Santoni d21cb596cd CC-5860: Analyzer HTTP requests can hang indefinitely 
* Log a backtrace when sent the USR2 signal (kill -SIGUSR2 <pid>)
* Rigged up up something to strace and restart when we see a request hanging
 2014-05-26 18:59:28 -04:00
drigato 0040965222 CC-5840: Add Media -> Endless retries if there is a validation error 
Fixed by not adding message to retry queue if request returns a
validation specific error code (422)
 2014-05-08 13:01:49 -04:00
drigato c291439483 Adding back analyzer deadlock workaround. 
Removing command to delete empty sub folders inside organize.
 2014-05-07 15:01:31 -04:00
drigato 81dd1deba3 Removing something for testing 2014-05-07 13:12:18 -04:00
Albert Santoni 56c3d8070b Alternate bugfix for logging deadlock in airtime_analyzer 2014-05-06 15:01:25 -04:00
Albert Santoni ed494ac587 Some defensive code against multiprocess related deadlocks 
* Reinitialize logging in child processes so we don't inherit locks.
  Might be causing a deadlock we're seeing on Pro right now.
 2014-05-05 18:25:47 -04:00
Albert Santoni d063700254 CC-5834: Airtime Analyzer: Bitrate is missing 
* Fixed
 2014-05-05 14:56:32 -04:00
Albert Santoni 22343ed99c Fixed bitrate import in airtime_analyzer 2014-05-05 14:39:36 -04:00
Albert Santoni c11e39f933 Fixed python-magic dependency 2014-04-30 18:46:37 -04:00
Albert Santoni abcdf5ea6a CC-5824: Airtime Analyzer: Flac file was uploaded but AA says it's 
failed

* Fixed
 2014-04-30 18:42:13 -04:00
Albert Santoni ace03edd2e Merge branch 'cc-5709-airtime-analyzer' of github.com:sourcefabric/Airtime into cc-5709-airtime-analyzer 2014-04-30 18:32:42 -04:00
Albert Santoni 26a39ed2db CC-5823: Airtime Analyzer: Wav file uploading fails with zero length and 
empty mime

* Fixed
 2014-04-30 18:32:17 -04:00
Albert Santoni c2f6032c8b Fixed the upstart config 2014-04-30 17:56:02 -04:00
drigato f9f9e2458e Merge branch 'cc-5709-airtime-analyzer' of github.com:sourcefabric/Airtime into cc-5709-airtime-analyzer 2014-04-28 14:45:05 -04:00
drigato fae5b15a9f CC-5821: Airtime Analyzer: Ftp-uploaded folder doesn't get removed from organize 
$_FILES does not store the original file path so we were losing the folder name.
Fixed by explicitly passing the full filepath in via the ftp-upload-hook.sh script
 2014-04-28 14:44:52 -04:00
Albert Santoni 375ec54778 CC-5820: Airtime Analyzer: Length and MIME parsing skipped if file has 
no audio metadata tags.
 2014-04-28 11:54:59 -04:00
Albert Santoni e503c952d2 CC-5816: Publisher column 
* Kanye West support. Surely
 2014-04-24 17:05:38 -04:00
Albert Santoni bc72a6626e CC-5709: Airtime Analyzer 
* Fixed some unit tests that broken when I dropped python-magic in
  favour of mutagen for MIME type detection
 2014-04-23 18:20:50 -04:00
Albert Santoni 9fdfc0a957 CC-5709: Airtime Analyzer 
* Catch exceptions related to unpickling the HTTP request queue
 2014-04-23 17:42:09 -04:00
Albert Santoni f2ce77fecb Remove python-magic 2014-04-23 14:20:48 -04:00
Albert Santoni d77f107df5 CC-5808: Fixed breakage at shutdown 2014-04-22 15:03:46 -04:00
Albert Santoni 6a86a75cee CC-5709: Airtime Analyzer 
* Fixed CC-5799: Incorrect Mime on import
 2014-04-21 17:50:10 -04:00
Albert Santoni cdce4ca104 CC-5709: Airtime Analyzer 
* Fixed the unit tests
* Improved length testing because it seems to fluctuate a bit with
  your mutagen version (great, eh?)
 2014-04-16 16:42:37 -04:00
Albert Santoni 9a3c88f009 CC-5709: Airtime Analyzer 
* Explicitly set the locale in upstart conf to avoid UTF-8/unicode
  errors from python. Fixes CC-5794.
 2014-04-16 15:33:02 -04:00
Albert Santoni b0670f5f47 CC-5709: Airtime Analyzer 
* Log tracebacks from child process
 2014-04-16 13:05:02 -04:00
Albert Santoni 4d1843ca1f Merge branch 'cc-5709-airtime-analyzer' of github.com:sourcefabric/Airtime into cc-5709-airtime-analyzer 2014-04-16 11:11:24 -04:00
Albert Santoni f491211167 CC-5709: Airtime Analyzer 
* Retry HTTP requests (kinda crappy, should find a better design pattern to
  solve this) - CC-5775
* Persist failed HTTP requests at shutdown
 2014-04-16 11:10:06 -04:00
drigato 7aba416c3b Small fix to ftp hook script. 
Removed space that was getting added to the api key
 2014-04-16 10:54:55 -04:00
Albert Santoni e4af3a5a0e CC-5709: Airtime Analyzer 
* Better SIGTERM handling for airtime_analyzer
* Nuke the .identifier files saved by Airtime
 2014-04-08 17:28:50 -04:00
Albert Santoni 84ee3d26cb CC-5709: Airtime Analyzer 
* Fixed metadata import, whoops
 2014-04-08 14:16:07 -04:00
Albert Santoni b8b913ff68 CC-5709: Airtime Analyzer 
* Fixed daemonization stuff for upstart
* Adding missing unit test files
 2014-04-07 16:19:44 -04:00
Albert Santoni b02dc45a53 CC-5709: Airtime Analyzer 
* Fixed airtime_analyzer setup script
 2014-04-04 19:46:59 -04:00
Albert Santoni 20b056c9f0 CC-5709: Airtime Analyzer 
* Added upstart script for starting on bootup
 2014-04-04 19:09:46 -04:00
Albert Santoni cf492045cc CC-5709: Airtime Analyzer 
* A bunch of cleanup and fix for metadata extraction on files mutagen can't read
 2014-04-04 12:35:50 -04:00
Albert Santoni 95b369c54d CC-5709: Airtime Analyzer 
* Remove awful StoredFile::uploadFile() function
* Massive airtime_analyzer commenting and cleanup
* Cleaned up the upload code
* Temporarily disabled the liquidsoap playability test.
 2014-04-03 16:13:26 -04:00
Albert Santoni 86b94b970f CC-5709: Airtime Analyzer 
* Fix for incorrect length sometimes happening (VBR MP3?)
 2014-04-03 11:34:57 -04:00
Albert Santoni 2a172afe3e CC-5709: Even more slightly better exception handling 2014-04-02 16:51:38 -04:00
Albert Santoni f6369c02ee CC-5709: Airtime Analyzer 
* Catch bad callback_url
 2014-04-02 16:47:24 -04:00
Albert Santoni 12c16bc120 CC-5709: Airtime Analyzer 
* Make the airtime_analyzer config path a runtime param
 2014-04-01 12:28:40 -04:00
drigato 82958a10ae CC-5708: proftpd hook for new File API 
small fix for testing script
 2014-03-28 12:26:40 -04:00
drigato 49996a9986 CC-5708: proftpd hook for new File API 
encased var in {}
 2014-03-28 11:15:28 -04:00
drigato 2fdc4291e3 CC-5708: proftpd hook for new File API 
testing ftp hook script
 2014-03-27 16:48:57 -04:00
drigato cc4e43a8b5 CC-5708: proftpd hook for new File API 
variable escaping?
 2014-03-27 16:48:27 -04:00
drigato 2bc37b9fea CC-5708: proftpd hook for new File API 
added handling for if the web server is down
 2014-03-27 15:46:42 -04:00
drigato eb72f25151 CC-5708: proftpd hook for new File API 
uncomment curl command
 2014-03-27 11:11:10 -04:00
drigato 700b28ae38 CC-5708: proftpd hook for new File API 
removed testing lines
 2014-03-27 11:09:18 -04:00
drigato b37938dbd8 Merge branch 'cc-5709-airtime-analyzer' of github.com:sourcefabric/Airtime into cc-5709-airtime-analyzer 2014-03-27 11:05:36 -04:00
drigato ead1ec4795 CC-5708: proftpd hook for new File API 2014-03-27 11:05:23 -04:00
Albert Santoni 7e0c9bc684 Merge branch 'cc-5709-airtime-analyzer' of github.com:sourcefabric/Airtime into cc-5709-airtime-analyzer 2014-03-26 15:07:40 -04:00
Albert Santoni 58a38ffd3a CC-5709: Airtime Analyzer 
* Added unit test coverage report stuff
* Added more unit tests
* 98% test coverage :-)
* Fixed a few bugs and removed some dead code
 2014-03-26 15:06:48 -04:00
drigato 681fe4630b CC-5708: proftpd hook for new File API 
url and API key are hard coded right now.
Next task is to somehow decipher these with whatever ftp parameters we can pass in to the scipt.
 2014-03-25 18:09:47 -04:00
Albert Santoni 03267fe83b Merge branch 'cc-5709-airtime-analyzer' of github.com:sourcefabric/Airtime into cc-5709-airtime-analyzer 2014-03-25 14:35:07 -04:00
Albert Santoni f8c221a610 CC-5709: Airtime Analyzer 
* Fixed and updated the unit tests
 2014-03-25 14:34:50 -04:00
drigato 3bffb02bc0 CC-5708: proftpd hook for new File API 2014-03-24 18:22:34 -04:00
Albert Santoni 6952902b22 CC-5709: Airtime Analyzer 
* Unbreak stuff
 2014-03-24 16:05:53 -04:00
Albert Santoni eb37b39406 CC-5709: Airtime Analyzer 
* Added missing FileMoverAnalyzer file that I forgot to commit
 2014-03-24 13:03:20 -04:00
Albert Santoni 61c2c90b7e CC-5709: Airtime Analyzer 
* Remove the "hidden" field from the REST blacklist, the analyzer needs to set it.
* Added import_status column messages in the recent uploads table
* Auto-refresh the recent uploads table while imports are pending
* Moved the file moving stuff to its own analyzer in airtime_analyzer
* Basic error reporting to the REST API in airtime_analyzer, needs
  hardeneing though
* Fixed a bug with the number of recent uploads
* Prevent airtime_analyzer from running if media_monitor is running
 2014-03-22 02:12:03 -04:00
Albert Santoni 8f7ecafcf6 CC-5709: Airtime Analyzer 
* A bunch of changes that somehow didn't make it into my last commit...
* confused
 2014-03-21 14:03:17 -04:00
Albert Santoni 878dd11ccc CC-5709: Airtime Analyzer 
* Overhauled Add Media screen, now shows state of recent uploads
* Dropped old unused "state" column, added new file_import column to cc_files
* New PluploadController methods
* Save the filename as the track title for unprocessed uploads
* Hide pending files from the library until they've been processed.
* Don't overwrite files with duplicate names, we rename them instead.
 2014-03-21 13:22:00 -04:00
Albert Santoni 16c56e6aff CC-5709: Airtime Analyzer 
* Fixed error in media API authentication
* Improved documentation
 2014-03-17 14:43:50 -04:00
Albert Santoni 50a42f12bb CC-5709: Airtime Analyzer 
* File import via the Add Media page now works!
* Reworked StoredFile::copyFileToStor() because it's still needed.
* Reworked the way file paths are reported in airtime_analyzer
  and in the RESTful media API
 2014-03-17 10:19:39 -04:00
Albert Santoni 259edebadc CC-5709: Airtime Analyzer 
* Fixed import of length, MIME type, and bitrate into Airtime
 2014-03-13 16:52:11 -04:00
Albert Santoni c0aee37e37 CC-5709: Airtime Analyzer 
* Fixed the RabbitMQ setup instructions in the README
 2014-03-13 14:38:01 -04:00
Albert Santoni 451b19150b CC-5709: Airtime Analyzer 
* Notify airtime_analyzer of new uploads with RabbitMQ
* Use a durable exchange for airtime-uploads
 2014-03-13 13:35:48 -04:00
Albert Santoni f4ea417b83 CC-5709: Airtime Analyzer 
* Renamed a function in MediaController for clarity
* Updated airtime_analyzer README
 2014-03-13 12:12:12 -04:00
Albert Santoni 6d7117f670 CC-5709: Airtime Analyzer 
* Added MetadataAnalyzer unit tests and test data
* Improved debug logging and squashed pika logging
* Implemented file moving
* Extract the track number/total
* Fixed mapping of mutagen to Airtime fields in a few spots. The mapping
  matches the DB column names now.
* Fixed the bin/airtime_analyzer binary
* Started work on PluploadController to make it work with the new File
  API
 2014-03-10 16:32:23 -04:00
Albert Santoni 4e39fce701 CC-5709: Airtime Analyzer 
* Basic HTTP reporting back to the File API works (PUT)
* Use the database table names as JSON field names.
* Fixed result returning bug in message_listener.py
* Fixed API key verification to adhere with the HTTP Basic Auth spec
 2014-03-06 16:55:20 -05:00
Albert Santoni 59535850e2 CC-5709: Airtime Analyzer Queue 
* Added StatusReporter class to make HTTP requests back to Airtime
* Handle even more errors now
* Added proper logging and log rotation
* Added --debug flag for increased logging.
 2014-03-05 22:43:47 -05:00
Albert Santoni a6a64a2b9e CC-5709 / CC-5705 : Airtime Analyzer 
* Finished the skeleton of the airtime_analyzer service.
* Basic round-robin, reliable AMQP messaging works.
* Using multiprocess arch so the daemon survives analyzer crashes and
  avoids failures propagating to other nodes.
* Basic metadata extractor using Mutagen is done.
* HTTP requests to the File API to are next to come...
 2014-03-05 12:15:25 -05:00