This commit is contained in:
Marc Aymerich 2015-04-04 17:44:07 +00:00
parent 751dda7126
commit 7133bd31ea
49 changed files with 220 additions and 262 deletions

View File

@ -5,7 +5,7 @@ Django-orchestra ships with a set of management commands for automating some of
These commands are meant to be run within a **clean** Debian-like distribution, you should be specially careful while following this guide on a customized system. These commands are meant to be run within a **clean** Debian-like distribution, you should be specially careful while following this guide on a customized system.
Django-orchestra can be installed on any Linux system, however it is **strongly recommended** to chose the reference platform for your deployment (Debian 7.0 wheezy and Python 2.7). Django-orchestra can be installed on any Linux system, however it is **strongly recommended** to chose the reference platform for your deployment (Debian 8.0 jessie and Python 3.4).
1. Create a system user for running Orchestra 1. Create a system user for running Orchestra
@ -18,7 +18,7 @@ Django-orchestra can be installed on any Linux system, however it is **strongly
2. Install django-orchestra's source code 2. Install django-orchestra's source code
```bash ```bash
sudo apt-get install python-pip sudo apt-get install python3-pip
sudo pip install django-orchestra==dev sudo pip install django-orchestra==dev
``` ```
@ -38,21 +38,21 @@ Django-orchestra can be installed on any Linux system, however it is **strongly
5. Create and configure a Postgres database 5. Create and configure a Postgres database
```bash ```bash
sudo python manage.py setuppostgres --db_password <password> sudo python3 manage.py setuppostgres --db_password <password>
python manage.py syncdb python3 manage.py syncdb
python manage.py migrate python3 manage.py migrate
``` ```
7. Configure celeryd 7. Configure celeryd
```bash ```bash
sudo python manage.py setupcelery --username orchestra sudo python3 manage.py setupcelery --username orchestra
``` ```
8. Configure the web server: 8. Configure the web server:
```bash ```bash
python manage.py collectstatic --noinput python3 manage.py collectstatic --noinput
sudo apt-get install nginx-full uwsgi uwsgi-plugin-python sudo apt-get install nginx-full uwsgi uwsgi-plugin-python3
sudo python manage.py setupnginx sudo python3 manage.py setupnginx
``` ```
9. Start all services: 9. Start all services:
@ -65,17 +65,17 @@ Upgrade
======= =======
To upgrade your Orchestra installation to the last release you can use `upgradeorchestra` management command. Before rolling the upgrade it is strongly recommended to check the [release notes](http://django-orchestra.readthedocs.org/en/latest/). To upgrade your Orchestra installation to the last release you can use `upgradeorchestra` management command. Before rolling the upgrade it is strongly recommended to check the [release notes](http://django-orchestra.readthedocs.org/en/latest/).
```bash ```bash
sudo python manage.py upgradeorchestra sudo python3 manage.py upgradeorchestra
``` ```
Current in *development* version (master branch) can be installed by Current in *development* version (master branch) can be installed by
```bash ```bash
sudo python manage.py upgradeorchestra dev sudo python3 manage.py upgradeorchestra dev
``` ```
Additionally the following command can be used in order to determine the currently installed version: Additionally the following command can be used in order to determine the currently installed version:
```bash ```bash
python manage.py orchestraversion python3 manage.py orchestraversion
``` ```

29
TODO.md
View File

@ -286,7 +286,7 @@ ugettext("Description")
* saas validate_creation generic approach, for all backends. standard output * saas validate_creation generic approach, for all backends. standard output
* html code x: &times; * html code x: &times; for bill line verbose quantity
* periodic task to cleanup backendlogs, monitor data and metricstorage * periodic task to cleanup backendlogs, monitor data and metricstorage
@ -299,29 +299,36 @@ celery max-tasks-per-child
* postupgradeorchestra send signals in order to hook custom stuff * postupgradeorchestra send signals in order to hook custom stuff
* make base home for systemusers that ara homed into main account systemuser * make base home for systemusers that ara homed into main account systemuser, and prevent shell users to have nested homes (if nnot implemented already)
* user force_text instead of unicode for _()
* autoscale celery workers http://docs.celeryproject.org/en/latest/userguide/workers.html#autoscaling * autoscale celery workers http://docs.celeryproject.org/en/latest/userguide/workers.html#autoscaling
* Delete transaction middleware
* webapp has_website list filter * webapp has_website list filter
glic3rinu's django-fluent-dashboard glic3rinu's django-fluent-dashboard
* gevent is not ported to python3 :'( * gevent is not ported to python3 :'(
* uwsgi python3 * uwsgi python3
https://github.com/django-nose/django-nose/archive/master.zip
django_debug_toolbar-1.3.0-py2.py3-none-any.whl
# FIXME account deletion generates a integrity error # FIXME account deletion generates an integrity error
https://code.djangoproject.com/ticket/24576
# FIXME what to do when deleting accounts? set fk null and fill a username charfield? issues, invoices.. we whant all this to go away? # FIXME what to do when deleting accounts? set fk null and fill a username charfield? issues, invoices.. we whant all this to go away?
* implement delete All related services * implement delete All related services
* address name change does not remove old one :P * address name change does not remove old one :P
* read https://docs.djangoproject.com/en/dev/releases/1.8/ and fix deprecation warnings
* remove admin object links , like contents webapps
* SaaS and WebApp fieldsets, and helptexts !
* remove all six stuff "from django.utils.six.moves import input"
* replace make_option in management commands
* rename apps to contrib find . -type f -name "*py"|xargs grep apps | grep -v 'orchestra\.apps\.'
* replace staticcheck by run(flake8 {orchestra,project} | grep -v "W293\|E501")
* rename utils.system to utils.sys

View File

@ -1,4 +1,4 @@
from django.contrib import admin, messages from django.contrib import admin
from django.core.mail import send_mass_mail from django.core.mail import send_mass_mail
from django.shortcuts import render from django.shortcuts import render
from django.utils.translation import ungettext, ugettext_lazy as _ from django.utils.translation import ungettext, ugettext_lazy as _

View File

@ -1,11 +1,8 @@
from admin_tools.menu import items, Menu from admin_tools.menu import items, Menu
from django.core.urlresolvers import reverse from django.core.urlresolvers import reverse
from django.utils.encoding import force_text
from django.utils.text import capfirst from django.utils.text import capfirst
from django.utils.translation import ugettext_lazy as _ from django.utils.translation import ugettext_lazy as _
from django.utils.safestring import mark_safe
from orchestra import get_version, settings
from orchestra.core import services, accounts from orchestra.core import services, accounts
from orchestra.utils.apps import isinstalled from orchestra.utils.apps import isinstalled

View File

@ -5,19 +5,18 @@ from django.contrib.admin.options import IS_POPUP_VAR
from django.contrib.admin.utils import unquote from django.contrib.admin.utils import unquote
from django.contrib.auth import update_session_auth_hash from django.contrib.auth import update_session_auth_hash
from django.core.exceptions import PermissionDenied from django.core.exceptions import PermissionDenied
from django.http import HttpResponseRedirect, Http404 from django.http import HttpResponseRedirect
from django.forms.models import BaseInlineFormSet from django.forms.models import BaseInlineFormSet
from django.shortcuts import render, redirect, get_object_or_404 from django.shortcuts import get_object_or_404
from django.template.response import TemplateResponse from django.template.response import TemplateResponse
from django.utils.decorators import method_decorator from django.utils.decorators import method_decorator
from django.utils.html import escape from django.utils.html import escape
from django.utils.text import camel_case_to_spaces
from django.utils.translation import ugettext_lazy as _ from django.utils.translation import ugettext_lazy as _
from django.views.decorators.debug import sensitive_post_parameters from django.views.decorators.debug import sensitive_post_parameters
from .forms import AdminPasswordChangeForm from .forms import AdminPasswordChangeForm
#from django.contrib.auth.forms import AdminPasswordChangeForm #from django.contrib.auth.forms import AdminPasswordChangeForm
from .utils import set_url_query, action_to_view, wrap_admin_view from .utils import set_url_query, action_to_view
sensitive_post_parameters_m = method_decorator(sensitive_post_parameters()) sensitive_post_parameters_m = method_decorator(sensitive_post_parameters())

View File

@ -3,12 +3,6 @@ from rest_framework.reverse import reverse
from rest_framework.routers import replace_methodname from rest_framework.routers import replace_methodname
def replace_collectionmethodname(format_string, methodname):
ret = replace_methodname(format_string, methodname)
ret = ret.replace('{collectionmethodname}', methodname)
return ret
def link_wrap(view, view_names): def link_wrap(view, view_names):
def wrapper(self, request, view=view, *args, **kwargs): def wrapper(self, request, view=view, *args, **kwargs):
""" wrapper function that inserts HTTP links on view """ """ wrapper function that inserts HTTP links on view """

View File

@ -1,12 +1,12 @@
from django.conf import settings as django_settings from django.conf import settings as django_settings
from django.core.exceptions import ImproperlyConfigured from django.core.exceptions import ImproperlyConfigured
from django.utils.module_loading import autodiscover_modules from django.utils.module_loading import autodiscover_modules
from rest_framework.routers import DefaultRouter, Route, flatten, replace_methodname from rest_framework.routers import DefaultRouter, Route, replace_methodname
from orchestra import settings from orchestra import settings
from orchestra.utils.python import import_class from orchestra.utils.python import import_class
from .helpers import insert_links, replace_collectionmethodname from .helpers import insert_links
class LinkHeaderRouter(DefaultRouter): class LinkHeaderRouter(DefaultRouter):

View File

@ -1,5 +1,5 @@
from django.forms import widgets from django.forms import widgets
from django.utils.translation import ugettext, ugettext_lazy as _ from django.utils.translation import ugettext_lazy as _
from rest_framework import serializers from rest_framework import serializers
from ..core.validators import validate_password from ..core.validators import validate_password

View File

@ -1,8 +1,15 @@
from django.contrib import messages from django.contrib import messages
from django.core.urlresolvers import reverse from django.contrib.admin import helpers
from django.db import transaction from django.contrib.admin.utils import NestedObjects, quote, model_ngettext
from django.contrib.auth import get_permission_codename
from django.core.urlresolvers import reverse, NoReverseMatch
from django.db import router
from django.shortcuts import redirect, render from django.shortcuts import redirect, render
from django.template.response import TemplateResponse
from django.utils import timezone from django.utils import timezone
from django.utils.encoding import force_text
from django.utils.html import format_html
from django.utils.text import capfirst
from django.utils.translation import ungettext, ugettext_lazy as _ from django.utils.translation import ungettext, ugettext_lazy as _
from orchestra.admin.decorators import action_with_confirmation from orchestra.admin.decorators import action_with_confirmation
@ -11,7 +18,6 @@ from orchestra.core import services
from . import settings from . import settings
@transaction.atomic
@action_with_confirmation() @action_with_confirmation()
def disable(modeladmin, request, queryset): def disable(modeladmin, request, queryset):
num = 0 num = 0
@ -43,12 +49,13 @@ def service_report(modeladmin, request, queryset):
# TODO resources # TODO resources
accounts = [] accounts = []
fields = [] fields = []
registered_services = services.get()
# First we get related manager names to fire a prefetch related # First we get related manager names to fire a prefetch related
for name, field in queryset.model._meta._name_map.items(): for name, field in queryset.model._meta.fields_map.items():
model = field[0].model model = field.related_model
if model in services.get() and model != queryset.model: if model in registered_services and model != queryset.model:
fields.append((model, name)) fields.append((model, name))
sorted(fields, key=lambda i: i[0]._meta.verbose_name_plural.lower()) sorted(fields, key=lambda f: f[0]._meta.verbose_name_plural.lower())
fields = [field for model, field in fields] fields = [field for model, field in fields]
for account in queryset.prefetch_related(*fields): for account in queryset.prefetch_related(*fields):
@ -66,4 +73,112 @@ def service_report(modeladmin, request, queryset):
def delete_related_services(modeladmin, request, queryset): def delete_related_services(modeladmin, request, queryset):
pass opts = modeladmin.model._meta
app_label = opts.app_label
using = router.db_for_write(modeladmin.model)
collector = NestedObjects(using=using)
collector.collect(queryset)
registered_services = services.get()
related_services = []
to_delete = []
user = request.user
admin_site = modeladmin.admin_site
def format(obj):
has_admin = obj.__class__ in admin_site._registry
opts = obj._meta
no_edit_link = '%s: %s' % (capfirst(opts.verbose_name), force_text(obj))
if has_admin:
try:
admin_url = reverse('admin:%s_%s_change' % (opts.app_label, opts.model_name),
None, (quote(obj._get_pk_val()),)
)
except NoReverseMatch:
# Change url doesn't exist -- don't display link to edit
return no_edit_link
p = '%s.%s' % (opts.app_label, get_permission_codename('delete', opts))
if not user.has_perm(p):
perms_needed.add(opts.verbose_name)
# Display a link to the admin page.
return format_html('{}: <a href="{}">{}</a>', capfirst(opts.verbose_name), admin_url, obj)
else:
# Don't display link to edit, because it either has no
# admin or is edited inline.
return no_edit_link
def format_nested(objs, result):
if isinstance(objs, list):
current = []
for obj in objs:
format_nested(obj, current)
result.append(current)
else:
result.append(format(objs))
for account in collector.nested():
if isinstance(account, list):
current = []
is_service = False
for service in account:
if type(service) in registered_services:
if service == main_systemuser:
continue
current.append(format(service))
to_delete.append(service)
is_service = True
elif is_service and isinstance(service, list):
nested = []
format_nested(service, nested)
current.append(nested)
is_service = False
else:
is_service = False
related_services.append(current)
elif isinstance(account, modeladmin.model):
# Prevent the deletion of the main system user, which will delete the account
main_systemuser = account.main_systemuser
related_services.append(format(account))
# The user has already confirmed the deletion.
# Do the deletion and return a None to display the change list view again.
if request.POST.get('post'):
n = queryset.count()
if n:
for obj in to_delete:
obj_display = force_text(obj)
modeladmin.log_deletion(request, obj, obj_display)
# TODO This probably will fail in certain conditions, just capture exception
obj.delete()
modeladmin.message_user(request, _("Successfully deleted %(count)d %(items)s.") % {
"count": n, "items": model_ngettext(modeladmin.opts, n)
}, messages.SUCCESS)
# Return None to display the change list page again.
return None
if len(queryset) == 1:
objects_name = force_text(opts.verbose_name)
else:
objects_name = force_text(opts.verbose_name_plural)
context = dict(
modeladmin.admin_site.each_context(request),
title=_("Are you sure?"),
objects_name=objects_name,
deletable_objects=[related_services],
model_count=dict(collector.model_count).items(),
queryset=queryset,
opts=opts,
action_checkbox_name=helpers.ACTION_CHECKBOX_NAME,
)
request.current_app = modeladmin.admin_site.name
# Display the confirmation page
return TemplateResponse(request, modeladmin.delete_selected_confirmation_template or [
"admin/%s/%s/delete_selected_confirmation.html" % (app_label, opts.model_name),
"admin/%s/delete_selected_confirmation.html" % app_label,
"admin/delete_selected_confirmation.html"
], context)
delete_related_services.short_description = _("Delete related services")

View File

@ -19,7 +19,7 @@ from orchestra.core import services, accounts
from orchestra.forms import UserChangeForm from orchestra.forms import UserChangeForm
from . import settings from . import settings
from .actions import disable, list_contacts, service_report from .actions import disable, list_contacts, service_report, delete_related_services
from .filters import HasMainUserListFilter from .filters import HasMainUserListFilter
from .forms import AccountCreationForm from .forms import AccountCreationForm
from .models import Account from .models import Account
@ -62,7 +62,7 @@ class AccountAdmin(ChangePasswordAdminMixin, auth.UserAdmin, ExtendedModelAdmin)
filter_horizontal = () filter_horizontal = ()
change_readonly_fields = ('username', 'main_systemuser_link') change_readonly_fields = ('username', 'main_systemuser_link')
change_form_template = 'admin/accounts/account/change_form.html' change_form_template = 'admin/accounts/account/change_form.html'
actions = [disable, list_contacts, service_report, SendEmail()] actions = [disable, list_contacts, service_report, SendEmail(), delete_related_services]
change_view_actions = [disable, service_report] change_view_actions = [disable, service_report]
list_select_related = ('billcontact',) list_select_related = ('billcontact',)
ordering = () ordering = ()

View File

@ -1,6 +1,7 @@
from collections import OrderedDict from collections import OrderedDict
from django import forms from django import forms
from django.core.exceptions import ValidationError
from django.db.models.loading import get_model from django.db.models.loading import get_model
from django.utils.translation import ugettext_lazy as _ from django.utils.translation import ugettext_lazy as _
@ -47,7 +48,7 @@ def create_account_creation_form():
if model.objects.filter(**kwargs).exists(): if model.objects.filter(**kwargs).exists():
verbose_name = model._meta.verbose_name verbose_name = model._meta.verbose_name
field_name = 'create_%s' % model._meta.model_name field_name = 'create_%s' % model._meta.model_name
errors[field] = ValidationError( errors[field_name] = ValidationError(
_("A %(type)s with this name already exists."), _("A %(type)s with this name already exists."),
params={'type': verbose_name}) params={'type': verbose_name})
if errors: if errors:

View File

@ -1,5 +1,4 @@
from django.contrib.auth import models as auth from django.contrib.auth import models as auth
from django.conf import settings as djsettings
from django.core import validators from django.core import validators
from django.db import models from django.db import models
from django.db.models.loading import get_model from django.db.models.loading import get_model

View File

@ -3,6 +3,7 @@ from io import StringIO
from django.contrib import messages from django.contrib import messages
from django.contrib.admin import helpers from django.contrib.admin import helpers
from django.core.exceptions import ValidationError
from django.core.urlresolvers import reverse from django.core.urlresolvers import reverse
from django.db import transaction from django.db import transaction
from django.http import HttpResponse from django.http import HttpResponse

View File

@ -1,4 +1,5 @@
from django.core.exceptions import ValidationError from django.core.exceptions import ValidationError
from django.utils.translation import ugettext_lazy as _
from rest_framework import serializers from rest_framework import serializers
from orchestra.api.serializers import HyperlinkedModelSerializer from orchestra.api.serializers import HyperlinkedModelSerializer

View File

@ -29,6 +29,7 @@ class RouteAdmin(admin.ModelAdmin):
] ]
list_editable = ['host', 'match', 'is_active'] list_editable = ['host', 'match', 'is_active']
list_filter = ['host', 'is_active', 'backend'] list_filter = ['host', 'is_active', 'backend']
ordering = ('backend',)
BACKEND_HELP_TEXT = { BACKEND_HELP_TEXT = {
backend: "This backend operates over '%s'" % ServiceBackend.get_backend(backend).model backend: "This backend operates over '%s'" % ServiceBackend.get_backend(backend).model

View File

@ -161,6 +161,7 @@ class Order(models.Model):
elif orders: elif orders:
order = orders.get() order = orders.get()
order.cancel(commit=commit) order.cancel(commit=commit)
logger.info("CANCELLED order id: {id}".format(id=order.id))
updates.append((order, 'cancelled')) updates.append((order, 'cancelled'))
return updates return updates
@ -284,8 +285,6 @@ def cancel_orders(sender, **kwargs):
# Account delete will delete all related orders, no need to maintain order consistency # Account delete will delete all related orders, no need to maintain order consistency
# if isinstance(instance, Order.account.field.rel.to): # if isinstance(instance, Order.account.field.rel.to):
# return # return
if sender is Order.account.field.rel.to:
return
print('delete', sender, instance, instance.pk) print('delete', sender, instance, instance.pk)
if type(instance) in services: if type(instance) in services:
for order in Order.objects.by_object(instance).active(): for order in Order.objects.by_object(instance).active():
@ -299,10 +298,10 @@ def cancel_orders(sender, **kwargs):
# return # return
print('related', type(related), related, related.pk) print('related', type(related), related, related.pk)
# try: # try:
# type(related).objects.get(pk=related.pk) type(related).objects.get(pk=related.pk)
# except related.DoesNotExist: # except related.DoesNotExist:
# print('not exists', type(related), related, related.pk) # print('not exists', type(related), related, related.pk)
Order.update_orders(related) print([(str(a).encode('utf8'), b) for a, b in Order.update_orders(related)])
@receiver(post_save, dispatch_uid="orders.update_orders") @receiver(post_save, dispatch_uid="orders.update_orders")
def update_orders(sender, **kwargs): def update_orders(sender, **kwargs):

View File

@ -124,7 +124,7 @@ class Transaction(models.Model):
if amount >= self.bill.total: if amount >= self.bill.total:
raise ValidationError(_("New transactions can not be allocated for this bill.")) raise ValidationError(_("New transactions can not be allocated for this bill."))
def check_state(*args): def check_state(self, *args):
if self.state not in args: if self.state not in args:
raise TypeError("Transaction not in %s" % ' or '.join(args)) raise TypeError("Transaction not in %s" % ' or '.join(args))
@ -176,7 +176,7 @@ class TransactionProcess(models.Model):
def __str__(self): def __str__(self):
return '#%i' % self.id return '#%i' % self.id
def check_state(*args): def check_state(self, *args):
if self.state not in args: if self.state not in args:
raise TypeError("Transaction process not in %s" % ' or '.join(args)) raise TypeError("Transaction process not in %s" % ' or '.join(args))

View File

@ -25,7 +25,7 @@ class Last(DataMethod):
def filter(self, dataset): def filter(self, dataset):
try: try:
return dataset.order_by('object_id', '-id').distinct('object_id') return dataset.order_by('object_id', '-id').distinct('object_id')
except MonitorData.DoesNotExist: except dataset.model.DoesNotExist:
return dataset.none() return dataset.none()
def compute_usage(self, dataset): def compute_usage(self, dataset):

View File

@ -1,6 +1,5 @@
from django.contrib import admin from django.contrib import admin
from django.core.urlresolvers import reverse from django.utils.translation import ugettext_lazy as _
from django.utils.translation import ugettext, ugettext_lazy as _
from orchestra.admin import ExtendedModelAdmin, ChangePasswordAdminMixin from orchestra.admin import ExtendedModelAdmin, ChangePasswordAdminMixin
from orchestra.apps.accounts.admin import AccountAdminMixin from orchestra.apps.accounts.admin import AccountAdminMixin

View File

@ -1,5 +1,4 @@
import pkgutil import pkgutil
import textwrap
class SaaSServiceMixin(object): class SaaSServiceMixin(object):

View File

@ -1,4 +1,3 @@
import json
import re import re
import requests import requests
@ -6,8 +5,6 @@ from django.utils.translation import ugettext_lazy as _
from orchestra.apps.orchestration import ServiceController from orchestra.apps.orchestration import ServiceController
from .. import settings
class PhpListSaaSBackend(ServiceController): class PhpListSaaSBackend(ServiceController):
verbose_name = _("phpList SaaS") verbose_name = _("phpList SaaS")
@ -16,7 +13,6 @@ class PhpListSaaSBackend(ServiceController):
block = True block = True
def _save(self, saas, server): def _save(self, saas, server):
base_domain = settings.SAAS_PHPLIST_BASE_DOMAIN
admin_link = 'http://%s/admin/' % saas.get_site_domain() admin_link = 'http://%s/admin/' % saas.get_site_domain()
admin_content = requests.get(admin_link).content admin_content = requests.get(admin_link).content
if admin_content.startswith('Cannot connect to Database'): if admin_content.startswith('Cannot connect to Database'):

View File

@ -1,5 +1,4 @@
from django import forms from django import forms
from django.core.exceptions import ValidationError
from django.utils.translation import ugettext_lazy as _ from django.utils.translation import ugettext_lazy as _
from rest_framework import serializers from rest_framework import serializers

View File

@ -1,12 +1,11 @@
from django import forms from django import forms
from django.core.exceptions import ValidationError
from django.core.urlresolvers import reverse from django.core.urlresolvers import reverse
from django.utils.safestring import mark_safe from django.utils.safestring import mark_safe
from django.utils.translation import ugettext_lazy as _ from django.utils.translation import ugettext_lazy as _
from rest_framework import serializers
from orchestra.apps.databases.models import Database, DatabaseUser from orchestra.apps.databases.models import Database, DatabaseUser
from orchestra.forms import widgets from orchestra.forms import widgets
from orchestra.plugins.forms import PluginDataForm
from .. import settings from .. import settings
from .options import SoftwareService, SoftwareServiceForm from .options import SoftwareService, SoftwareServiceForm

View File

@ -8,7 +8,6 @@ from django.utils.translation import ugettext_lazy as _
from orchestra.admin import ChangeViewActionsMixin from orchestra.admin import ChangeViewActionsMixin
from orchestra.admin.filters import UsedContentTypeFilter from orchestra.admin.filters import UsedContentTypeFilter
from orchestra.apps.accounts.admin import AccountAdminMixin
from orchestra.core import services from orchestra.core import services
from .actions import update_orders, view_help, clone from .actions import update_orders, view_help, clone

View File

@ -116,6 +116,8 @@ class ServiceHandler(plugins.Plugin, metaclass=plugins.PluginMount):
instance._meta.model_name: instance, instance._meta.model_name: instance,
'instance': instance, 'instance': instance,
'math': math, 'math': math,
'logsteps': lambda n, size=1: \
round(n/(size*10**int(math.log10(max(n, 1)))))*size*10**int(math.log10(max(n, 1))),
'log10': math.log10, 'log10': math.log10,
'Decimal': decimal.Decimal, 'Decimal': decimal.Decimal,
} }

View File

@ -1,9 +1,7 @@
import decimal import decimal
from django.contrib.contenttypes.models import ContentType from django.contrib.contenttypes.models import ContentType
from django.core.validators import ValidationError
from django.db import models from django.db import models
from django.db.models import Q
from django.db.models.loading import get_model from django.db.models.loading import get_model
from django.utils.functional import cached_property from django.utils.functional import cached_property
from django.utils.module_loading import autodiscover_modules from django.utils.module_loading import autodiscover_modules
@ -11,8 +9,6 @@ from django.utils.translation import string_concat, ugettext_lazy as _
from orchestra.core import caches, validators from orchestra.core import caches, validators
from orchestra.core.translations import ModelTranslation from orchestra.core.translations import ModelTranslation
from orchestra.core.validators import validate_name
from orchestra.models import queryset
from orchestra.utils.python import import_class from orchestra.utils.python import import_class
from . import settings from . import settings

View File

@ -1,5 +1,3 @@
from datetime import timedelta
from django.conf import settings from django.conf import settings
from django.utils.translation import ugettext_lazy as _ from django.utils.translation import ugettext_lazy as _

View File

@ -1,9 +0,0 @@
from orchestra.apps.accounts.models import Account
from orchestra.utils.tests import BaseTestCase, random_ascii
# TODO remove this shit
class BaseBillingTest(BaseTestCase):
pass
# TODO web disk

View File

@ -16,6 +16,8 @@ class SystemUserBackend(ServiceController):
def save(self, user): def save(self, user):
context = self.get_context(user) context = self.get_context(user)
if not context['user']:
return
groups = ','.join(self.get_groups(user)) groups = ','.join(self.get_groups(user))
context['groups_arg'] = '--groups %s' % groups if groups else '' context['groups_arg'] = '--groups %s' % groups if groups else ''
# TODO userd add will fail if %(user)s group already exists # TODO userd add will fail if %(user)s group already exists
@ -37,6 +39,8 @@ class SystemUserBackend(ServiceController):
def delete(self, user): def delete(self, user):
context = self.get_context(user) context = self.get_context(user)
if not context['user']:
return
self.append(textwrap.dedent("""\ self.append(textwrap.dedent("""\
{ sleep 2 && killall -u %(user)s -s KILL; } & { sleep 2 && killall -u %(user)s -s KILL; } &
killall -u %(user)s || true killall -u %(user)s || true

View File

@ -1,4 +1,5 @@
from django.core.exceptions import ValidationError from django.core.exceptions import ValidationError
from django.utils.translation import ugettext_lazy as _
from orchestra import plugins from orchestra import plugins
from orchestra.plugins.forms import PluginDataForm from orchestra.plugins.forms import PluginDataForm
@ -31,7 +32,7 @@ class AppType(plugins.Plugin):
def validate(self): def validate(self):
""" Unique name validation """ """ Unique name validation """
if self.unique_name: if self.unique_name:
if not self.instance.pk and Webapp.objects.filter(name=self.instance.name, type=self.instance.type).exists(): if not self.instance.pk and type(self.instance).objects.filter(name=self.instance.name, type=self.instance.type).exists():
raise ValidationError({ raise ValidationError({
'name': _("A WordPress blog with this name already exists."), 'name': _("A WordPress blog with this name already exists."),
}) })

View File

@ -118,6 +118,7 @@ class PHPApp(AppType):
wrapper_path = os.path.normpath(self.FCGID_WRAPPER_PATH % context) wrapper_path = os.path.normpath(self.FCGID_WRAPPER_PATH % context)
return ('fcgid', self.instance.get_path(), wrapper_path) return ('fcgid', self.instance.get_path(), wrapper_path)
else: else:
php_version = self.get_php_version()
raise ValueError("Unknown directive for php version '%s'" % php_version) raise ValueError("Unknown directive for php version '%s'" % php_version)
def get_php_version(self): def get_php_version(self):

View File

@ -208,7 +208,7 @@ class Apache2Backend(ServiceController):
proxies = [] proxies = []
for proxy in directives.get('proxy', []): for proxy in directives.get('proxy', []):
location, target = proxy.split() location, target = proxy.split()
location = normurlpath(source) location = normurlpath(location)
proxy = textwrap.dedent("""\ proxy = textwrap.dedent("""\
ProxyPass {location}/ {target} ProxyPass {location}/ {target}
ProxyPassReverse {location}/ {target}""".format( ProxyPassReverse {location}/ {target}""".format(

View File

@ -1,6 +1,7 @@
from django import forms from django import forms
from django.core.exceptions import ValidationError from django.core.exceptions import ValidationError
from django.utils.encoding import force_text from django.utils.encoding import force_text
from django.utils.translation import ugettext_lazy as _
from .validators import validate_domain_protocol from .validators import validate_domain_protocol

View File

@ -1,5 +1,4 @@
from django.core.exceptions import ValidationError from django.core.exceptions import ValidationError
from django.db.models import Q
from django.shortcuts import get_object_or_404 from django.shortcuts import get_object_or_404
from rest_framework import serializers from rest_framework import serializers

View File

@ -173,6 +173,7 @@ function install_requirements () {
xvfbwrapper \ xvfbwrapper \
freezegun \ freezegun \
coverage \ coverage \
flake8 \
orchestra-orm==dev \ orchestra-orm==dev \
django-debug-toolbar==1.3.0 \ django-debug-toolbar==1.3.0 \
https://github.com/django-nose/django-nose/archive/master.zip \ https://github.com/django-nose/django-nose/archive/master.zip \

View File

@ -48,11 +48,10 @@ MIDDLEWARE_CLASSES = (
'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware', 'django.contrib.messages.middleware.MessageMiddleware',
'orchestra.core.caches.RequestCacheMiddleware', 'orchestra.core.caches.RequestCacheMiddleware',
# also handles transations, ATOMIC REQUESTS does not wrap middlewares # also handles transations, ATOMIC_REQUESTS does not wrap middlewares
'orchestra.apps.orchestration.middlewares.OperationsMiddleware', 'orchestra.apps.orchestration.middlewares.OperationsMiddleware',
# Uncomment the next line for simple clickjacking protection: # Uncomment the next line for simple clickjacking protection:
# 'django.middleware.clickjacking.XFrameOptionsMiddleware', # 'django.middleware.clickjacking.XFrameOptionsMiddleware',
) )
@ -69,7 +68,6 @@ TEMPLATE_CONTEXT_PROCESSORS =(
INSTALLED_APPS = ( INSTALLED_APPS = (
# django-orchestra apps # django-orchestra apps
'orchestra', 'orchestra',
'orchestra.apps.accounts', 'orchestra.apps.accounts',
'orchestra.apps.contacts', 'orchestra.apps.contacts',

View File

@ -23,7 +23,7 @@ class Register(object):
def get(self, *args): def get(self, *args):
if args: if args:
return self._registry[arg[0]] return self._registry[args[0]]
return self._registry return self._registry

View File

@ -1,47 +0,0 @@
from django.db import connection, transaction
class TransactionMiddleware(object):
"""
Transaction middleware. If this is enabled, each view function will be run
with commit_on_response activated - that way a save() doesn't do a direct
commit, the commit is done when a successful response is created. If an
exception happens, the database is rolled back.
"""
pass
# def process_request(self, request):
# """Enters transaction management"""
# transaction.enter_transaction_management()
#
# def process_exception(self, request, exception):
# """Rolls back the database and leaves transaction management"""
# if transaction.is_dirty():
# # This rollback might fail because of network failure for example.
# # If rollback isn't possible it is impossible to clean the
# # connection's state. So leave the connection in dirty state and
# # let request_finished signal deal with cleaning the connection.
# transaction.rollback()
# transaction.leave_transaction_management()
#
# def process_response(self, request, response):
# """Commits and leaves transaction management."""
# if not transaction.get_autocommit():
# if transaction.is_dirty():
# # Note: it is possible that the commit fails. If the reason is
# # closed connection or some similar reason, then there is
# # little hope to proceed nicely. However, in some cases (
# # deferred foreign key checks for exampl) it is still possible
# # to rollback().
# try:
# transaction.commit()
# except Exception:
# # If the rollback fails, the transaction state will be
# # messed up. It doesn't matter, the connection will be set
# # to clean state after the request finishes. And, we can't
# # clean the state here properly even if we wanted to, the
# # connection is in transaction but we can't rollback...
# transaction.rollback()
# transaction.leave_transaction_management()
# raise
# transaction.leave_transaction_management()
# return response

View File

@ -1,7 +1,6 @@
import re import re
import crack import crack
import localflavor
import phonenumbers import phonenumbers
from django.core import validators from django.core import validators
@ -47,7 +46,7 @@ def validate_ipv6_address(value):
def validate_ip_address(value): def validate_ip_address(value):
msg = _("%s is not a valid IP address") % value msg = _("%s is not a valid IP address") % value
try: try:
ip = IP(value) IP(value)
except: except:
raise ValidationError(msg) raise ValidationError(msg)

View File

@ -1,6 +1,6 @@
from django import forms from django import forms
from django.contrib.auth import forms as auth_forms from django.contrib.auth import forms as auth_forms
from django.utils.translation import ugettext, ugettext_lazy as _ from django.utils.translation import ugettext_lazy as _
from orchestra.utils.python import random_ascii from orchestra.utils.python import random_ascii

View File

@ -1,102 +1,12 @@
# Adapted from http://djangosnippets.org/snippets/1762/
import ast
import os
import sys
from django.core.management.base import BaseCommand from django.core.management.base import BaseCommand
from pyflakes import checker, messages
from orchestra.utils.paths import get_orchestra_dir from orchestra.utils.paths import get_orchestra_dir, get_site_dir
from orchestra.utils.system import run
# BlackHole, PySyntaxError and checking based on
# https://github.com/patrys/gedit-pyflakes-plugin.git
class BlackHole(object):
write = flush = lambda *args, **kwargs: None
def __enter__(self):
self.stderr, sys.stderr = sys.stderr, self
def __exit__(self, *args, **kwargs):
sys.stderr = self.stderr
class PySyntaxError(messages.Message):
message = 'syntax error in line %d: %s'
def __init__(self, filename, e):
super(PySyntaxError, self).__init__(filename, e)
self.message_args = (e.offset, e.text)
def check(codeString, filename):
"""
Check the Python source given by C{codeString} for flakes.
@param codeString: The Python source to check.
@type codeString: C{str}
@param filename: The name of the file the source came from, used to report errors.
@type filename: C{str}
@return: The number of warnings emitted.
@rtype: C{int}
"""
try:
with BlackHole():
tree = ast.parse(codeString, filename)
except SyntaxError as e:
return [PySyntaxError(filename, e)]
else:
# Okay, it's syntactically valid. Now parse it into an ast and check it
w = checker.Checker(tree, filename)
lines = codeString.split('\n')
# honour pyflakes: ignore comments
messages = [message for message in w.messages
if lines[message.lineno-1].find('pyflakes:ignore') < 0]
messages.sort(lambda a, b: cmp(a.lineno, b.lineno))
return messages
def checkPath(filename):
"""
Check the given path, printing out any warnings detected.
@return: the number of warnings printed
"""
try:
return check(file(filename, 'U').read() + '\n', filename)
except IOError as msg:
return ["%s: %s" % (filename, msg.args[1])]
except TypeError:
pass
def checkPaths(filenames):
warnings = []
for arg in filenames:
if os.path.isdir(arg):
for dirpath, dirnames, filenames in os.walk(arg):
for filename in filenames:
if filename.endswith('.py'):
warnings.extend(checkPath(os.path.join(dirpath, filename)))
else:
warnings.extend(checkPath(arg))
return warnings
#### pyflakes.scripts.pyflakes ends.
class Command(BaseCommand): class Command(BaseCommand):
help = "Run pyflakes syntax checks." help = "Run flake8 syntax checks."
args = '[filename [filename [...]]]'
def handle(self, *filenames, **options): def handle(self, *filenames, **options):
if not filenames: flake = run('flake8 {%s,%s} | grep -v "W293\|E501"' % (get_orchestra_dir(), get_site_dir()))
filenames = [get_orchestra_dir(), '.'] print(flake.stdout)
warnings = checkPaths(filenames)
for warning in warnings:
print(warning)
if warnings:
print('Total warnings: %d' % len(warnings))
raise SystemExit(1)

View File

@ -4,7 +4,6 @@ from django.shortcuts import render, redirect
from django.utils.translation import ugettext_lazy as _ from django.utils.translation import ugettext_lazy as _
from orchestra.admin.utils import wrap_admin_view from orchestra.admin.utils import wrap_admin_view
from orchestra.utils.functional import cached
class SelectPluginAdminMixin(object): class SelectPluginAdminMixin(object):

View File

@ -63,7 +63,7 @@ def runiterator(command, display=False, error_codes=[0], silent=False, stdin='',
# Async reading of stdout and sterr # Async reading of stdout and sterr
while True: while True:
stdout = '' stdout = ''
sdterr = '' stderr = ''
# Get complete unicode chunks # Get complete unicode chunks
select.select([p.stdout, p.stderr], [], []) select.select([p.stdout, p.stderr], [], [])
@ -71,7 +71,7 @@ def runiterator(command, display=False, error_codes=[0], silent=False, stdin='',
stderrPiece = read_async(p.stderr) stderrPiece = read_async(p.stderr)
stdout += (stdoutPiece or b'').decode('utf8') stdout += (stdoutPiece or b'').decode('utf8')
sdterr += (stderrPiece or b'').decode('utf8') stderr += (stderrPiece or b'').decode('utf8')
if display and stdout: if display and stdout:
sys.stdout.write(stdout) sys.stdout.write(stdout)
@ -79,7 +79,7 @@ def runiterator(command, display=False, error_codes=[0], silent=False, stdin='',
sys.stderr.write(stderr) sys.stderr.write(stderr)
state = _Attribute(stdout) state = _Attribute(stdout)
state.stderr = sdterr state.stderr = stderr
state.return_code = p.poll() state.return_code = p.poll()
yield state yield state

View File

@ -16,7 +16,6 @@ from orchestra.apps.accounts.models import Account
from .python import random_ascii from .python import random_ascii
class AppDependencyMixin(object): class AppDependencyMixin(object):
DEPENDENCIES = () DEPENDENCIES = ()

View File

@ -3,6 +3,7 @@
# This is a helper script for creating a basic LXC container with some convenient packages # This is a helper script for creating a basic LXC container with some convenient packages
# ./create.sh [container_name] # ./create.sh [container_name]
set -u set -u
NAME=${1:-orchestra} NAME=${1:-orchestra}
@ -35,7 +36,7 @@ chroot $CONTAINER locale-gen
chroot $CONTAINER apt-get install -y --force-yes \ chroot $CONTAINER apt-get install -y --force-yes \
nano git screen sudo iputils-ping python2.7 python-pip wget curl dnsutils rsyslog nano git screen sudo iputils-ping python3 python3-pip wget curl dnsutils rsyslog
chroot $CONTAINER apt-get clean chroot $CONTAINER apt-get clean

View File

@ -41,13 +41,13 @@ chown $USER.$USER $HOME
run adduser $USER sudo run adduser $USER sudo
CURRENT_VERSION=$(python -c "from orchestra import get_version; print get_version();" 2> /dev/null || false) CURRENT_VERSION=$(python3 -c "from orchestra import get_version; print get_version();" 2> /dev/null || false)
if [[ ! $CURRENT_VERSION ]]; then if [[ ! $CURRENT_VERSION ]]; then
# First Orchestra installation # First Orchestra installation
run "apt-get -y install git python-pip" run "apt-get -y install git python3-pip"
surun "git clone https://github.com/glic3rinu/django-orchestra.git ~/django-orchestra" surun "git clone https://github.com/glic3rinu/django-orchestra.git ~/django-orchestra"
echo $HOME/django-orchestra/ | sudo tee /usr/local/lib/python2.7/dist-packages/orchestra.pth echo $HOME/django-orchestra/ | sudo tee /usr/local/lib/python3*/dist-packages/orchestra.pth
run "cp $HOME/django-orchestra/orchestra/bin/orchestra-admin /usr/local/bin/" run "cp $HOME/django-orchestra/orchestra/bin/orchestra-admin /usr/local/bin/"
fi fi
@ -71,33 +71,33 @@ if [[ ! $(sudo su postgres -c "psql -lqt" | awk {'print $1'} | grep '^orchestra$
/etc/postgresql/${POSTGRES_VERSION}/main/postgresql.conf /etc/postgresql/${POSTGRES_VERSION}/main/postgresql.conf
run "service postgresql restart" run "service postgresql restart"
run "python $MANAGE setuppostgres --db_name orchestra --db_user orchestra --db_password orchestra" run "python3 $MANAGE setuppostgres --db_name orchestra --db_user orchestra --db_password orchestra"
# Create database permissions are needed for running tests # Create database permissions are needed for running tests
sudo su postgres -c 'psql -c "ALTER USER orchestra CREATEDB;"' sudo su postgres -c 'psql -c "ALTER USER orchestra CREATEDB;"'
fi fi
if [[ $CURRENT_VERSION ]]; then if [[ $CURRENT_VERSION ]]; then
# Per version upgrade specific operations # Per version upgrade specific operations
run "python $MANAGE postupgradeorchestra --no-restart --from $CURRENT_VERSION" run "python3 $MANAGE postupgradeorchestra --no-restart --from $CURRENT_VERSION"
else else
run "python $MANAGE syncdb --noinput" run "python3 $MANAGE syncdb --noinput"
run "python $MANAGE migrate --noinput" run "python3 $MANAGE migrate --noinput"
fi fi
sudo python $MANAGE setupcelery --username $USER --processes 2 sudo python $MANAGE setupcelery --username $USER --processes 2
# Install and configure Nginx web server # Install and configure Nginx web server
surun "mkdir $BASE_DIR/static" surun "mkdir $BASE_DIR/static"
surun "python $MANAGE collectstatic --noinput" surun "python3 $MANAGE collectstatic --noinput"
run "apt-get install -y nginx uwsgi uwsgi-plugin-python" run "apt-get install -y nginx uwsgi uwsgi-plugin-python3"
run "python $MANAGE setupnginx" run "python3 $MANAGE setupnginx"
run "service nginx start" run "service nginx start"
# Apply changes # Apply changes
run "python $MANAGE restartservices" run "python3 $MANAGE restartservices"
# Create a orchestra user # Create a orchestra user
cat <<- EOF | python $MANAGE shell cat <<- EOF | python3 $MANAGE shell
from orchestra.apps.accounts.models import Account from orchestra.apps.accounts.models import Account
if not Account.objects.filter(username="$USER").exists(): if not Account.objects.filter(username="$USER").exists():
print 'Creating orchestra superuser' print 'Creating orchestra superuser'