Commit Graph

237 Commits

Author SHA1 Message Date
Lucas Bickel 188c127057 Merge branch 'master' into feature/analyzer-on-travis
# Conflicts:
#	.travis.yml
2017-03-04 16:35:11 +01:00
Lucas Bickel 9b3d0c90da mostly run analyzer tests on travis
* [x] regonfigured the build matrix with more php jobs and a separate python job (we can add more python jobs later)
* [x] run tests on travis' trusty beta container (it's closer to what we need anyway)
* [x] install packages needed for analyzer tests in build env
* [x] added docs on how to run nosetests locally
* [x] don't run initctl in analyzer setup so setup can also be used on travis (and add it to the install script directly)
* [x] ignore replaygain checks on travis (it has proven quite impossible to get the needed python-gi module to work in the provided virtualenv)

I tried a lot of solutions to get the replaygain checks to run. I needed to decide that this has gone far enough, maybe someone who is more of a pythonista than me can take a crack at it and get it solved. Even without running those tests on CI/CD there are still plenty others.

This PR only has parts of what are needed for getting python tests running on travis as per #15. I only took a quick shot at anything not analyzer and figured I would not be able to "fix" them without digging a bit deeper (ie. also getting rid of std_err_override).
2017-03-03 20:38:27 +01:00
Lucas Bickel e28ad471f9 Rewrite config from /etc/airtime-saas to plain /etc/airtime
This is the results of sed -i -e 's|/etc/airtime-saas/|/etc/airtime/|' `grep -irl 'airtime-saas' airtime_mvc/ python_apps/` :P

It might need more testing, the airtime-saas part never really made sense, zf1 has environments for that, ie you would create a saas env based on production for instance.

I beleive legacy upstream was using this to share configuration between customers (ie. analyser runs only once and writes to a shared S3 bucket). I assume they mount the airtime-saas folder onto individual customers instances with a global config. Like I said, I don't feel that this makes sense since all it does is make hacking at the configs in airtime-saas a bit easier. A serious SaaS operation should be using something like puppet or ansible to achieve this.
2017-03-03 15:57:41 +01:00
Albert Santoni 965a78c54a Merge branch 'saas-dev' into saas-dev-publishing
Conflicts:
	airtime_mvc/application/controllers/ApiController.php
2015-11-18 12:38:18 -05:00
Albert Santoni c85944785b Comments 2015-10-30 18:09:56 -04:00
Albert Santoni b49bb2e262 Loosen up Mutagen vs. Silan cue out length threshold a bit 2015-10-30 17:54:24 -04:00
Albert Santoni 7be548f30e Fixed the airtime_analyzer unit tests 2015-10-30 17:15:33 -04:00
Albert Santoni fc51e63214 Fix typo in airtime_analyzer setup.py 2015-10-30 17:13:12 -04:00
Albert Santoni 76f202106b Defensive coding against Silan bugs and bump to Mutagen 1.31 2015-10-30 17:12:13 -04:00
Duncan Sommerville a78a0d0cc8 * Fix track number bug in analyzer
* Move library search bar out of header
2015-09-16 17:50:21 -04:00
Albert Santoni 67c5b73db4 airtime_analyzer unit test backwards compatibility with older versions of requests 2015-07-08 10:32:35 -04:00
Duncan Sommerville 4c797cf100 Update installer to work with saas branch 2015-06-23 19:02:55 -04:00
Albert Santoni c7aa6b3e5e airtime_analyzer: use requests >= 2.7.0 2015-05-20 10:36:55 -04:00
Albert Santoni 3d8cd8e927 Added daemon dependency back in for airtime_analyzer 2015-05-19 13:45:08 -04:00
Albert Santoni e1c50ecc7b Squash urllib3 warnings to see if it helps with SAAS-714 2015-05-19 13:09:43 -04:00
Albert Santoni add211d13c Cleanup to help with SAAS-714 2015-05-19 12:02:19 -04:00
Albert Santoni b3a19ae49e Bugfix for embedded player sending too many requests on load, quieter logging for boto 2015-04-29 11:46:44 -04:00
Albert Santoni 28fc1cf2fc Fixed some airtime_analyzer unit tests 2015-04-10 15:08:44 -04:00
Albert Santoni e3390b99ef Disable analyzer log file rotation (already handed by Upstart), clean up 2015-04-10 15:03:58 -04:00
Albert Santoni d5012c25cb Another small bugfix for error handling in the analyzer 2015-04-06 17:33:08 -04:00
Albert Santoni 492a7f329a Minor airtime_analyzer error handling improvements and documentation 2015-04-06 17:22:13 -04:00
Albert Santoni 7fd58ac51b Merge branch 'saas-s3proxy' into saas 2015-04-01 13:49:47 -04:00
Albert Santoni ae02fe7816 Log traceback in analyzer status_reporter.py exception handler 2015-03-30 13:21:26 -04:00
Albert Santoni 271dc266fa S3 proxy cache support + 1 minor analyzer bugfix 2015-03-26 12:08:52 -04:00
Albert Santoni d1b28fd564 Fixed a bug in FTP upload hook, other minor improvements
* Ensure we have write permissions on any newly uploaded files by FTP
* Don't silence warnings in moveFileToStor()
* Reduced the FTP upload max retry count
* Log E_NOTICE and E_WARNING levels to Sentry
2015-03-24 14:56:03 -04:00
Albert Santoni e182e73626 SAAS-602: airtime_analyzer deadlocks in S3 hostname lookup
* Stopped using multiprocess.subprocess because it's dangerous and
  unreliable due to Python bug 6721: http://bugs.python.org/issue6721
2015-03-11 18:33:08 -04:00
Albert Santoni 6cecb7c239 Simplified the FTP upload hook 2015-03-06 11:06:17 -05:00
Albert Santoni ab6d83f49d Merge branch 'saas' into saas-media-refactor
Conflicts:
	airtime_mvc/application/Bootstrap.php
	airtime_mvc/application/modules/rest/controllers/MediaController.php
2015-02-26 13:10:13 -05:00
Albert Santoni d82f461f35 Fix missing filesize and md5 for WAVE imports 2015-02-26 13:06:29 -05:00
Albert Santoni edd07c18ae Fixed invalid detection of S3 backend in analyzer 2015-02-26 12:02:02 -05:00
Albert Santoni 3a1141d4c8 Merge branch 'saas' into saas-media-refactor
Conflicts:
	airtime_mvc/application/cloud_storage/ProxyStorageBackend.php
	airtime_mvc/application/controllers/ApiController.php
2015-02-25 13:02:11 -05:00
Albert Santoni 37f9f273c7 Fixed the airtime_analyzer unit tests 2015-02-24 17:03:38 -05:00
Albert Santoni 702de0f5f5 SAAS-602: Fix getaddrinfo deadlock (again)
Conflicts:
	python_apps/airtime_analyzer/airtime_analyzer/cloud_storage_uploader.py
2015-02-24 13:14:21 -05:00
Albert Santoni 8aaf300491 Attempt a workaround for unicode hostname resolution deadlock in the
analyzer
2015-02-23 18:22:16 -05:00
Albert Santoni 51aa846a4c Merge branch 'cc-5709-airtime-analyzer-refactor' into saas-media-refactor
Conflicts:
	airtime_mvc/application/models/RabbitMq.php
	python_apps/airtime_analyzer/airtime_analyzer/analyzer_pipeline.py
	python_apps/airtime_analyzer/airtime_analyzer/message_listener.py
2015-02-20 18:25:38 -05:00
Albert Santoni c868136d26 Make airtime_analyzer respect the storage_backend setting passed to it
by Airtime
2015-02-20 18:21:49 -05:00
drigato 58f9ffd0eb Merge branch 'cc-5709-airtime-analyzer-store-file-size-and-hash-in-db' into saas-store-file-size-and-hash-in-db 2015-02-19 15:40:27 -05:00
drigato b6acfb2dce SAAS-596: Store file size and hash in database
Renamed metadata md5 value to match cc_files column
2015-02-19 15:39:55 -05:00
drigato 6ef085f460 Merge branch 'saas' into saas-store-file-size-and-hash-in-db 2015-02-19 11:43:34 -05:00
drigato 0f921cb481 Merge branch 'cc-5709-airtime-analyzer-store-file-size-and-hash-in-db' into saas-store-file-size-and-hash-in-db 2015-02-19 10:51:53 -05:00
drigato cc9e6efbca SAAS-596: Store file size and hash in database
Removed try/except while getting file size and hash
2015-02-19 10:38:30 -05:00
Albert Santoni 70f5bd3748 Fixed the unit tests 2015-02-19 10:17:42 -05:00
Albert Santoni d0bd4bfe81 Merge branch 'saas' into saas-store-file-size-and-hash-in-db
Conflicts:
	airtime_mvc/application/modules/rest/controllers/MediaController.php
2015-02-18 19:29:45 -05:00
drigato 5bd3371ec2 SAAS-596: Store file size and hash in database
Removed getFileSize from CloudFile.php since it can call the parent
function.
Moved setting filesize metadata from cloud_storage_uploader.
2015-02-17 16:51:25 -05:00
drigato c1086842d3 Merge branch 'cc-5709-airtime-analyzer-store-file-size-and-hash-in-db' into saas-store-file-size-and-hash-in-db
Conflicts:
	airtime_mvc/application/Bootstrap.php

airtime_mvc/application/modules/rest/controllers/MediaController.php
2015-02-17 16:19:58 -05:00
drigato 66a6a8f985 SAAS-596: Store file size and hash in database
Added file size and hash to the metadata dict in the analyzer
2015-02-17 16:02:49 -05:00
Albert Santoni d8f5e41fca Set a connection timeout for Boto to prevent cloud storage deadlock 2015-02-17 14:58:19 -05:00
drigato cee0ff4881 SAAS-560: Deploy separate cloud storage config files for each development environment
Default to production config files if dev env specific files are not
found.
Fix analyzer unit tests.
2015-02-05 14:31:20 -05:00
drigato 6cfac054ca SAAS-560: Deploy separate cloud storage config files for each development environment
Renamed the analyzer config variables to be named specially for rabbitmq
config
2015-02-05 10:14:35 -05:00
drigato e6171680ad SAAS-560: Deploy separate cloud storage config files for each development environment
Small fix
2015-02-04 14:34:04 -05:00