pangea
/
django-orchestra
7
0
Fork 0
Code Issues 1 Pull Requests 2 Packages Projects Releases Wiki Activity

Merge pull request #3 from ribaguifi/docker 

Docker and deployment
This commit is contained in:
cayop 2021-02-01 12:45:15 +01:00 committed by GitHub
parent e6e434f525 cedb8d690b
commit 350d93f820
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
24 changed files with 1018 additions and 180 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
.env.example Normal file
View File

@ -0,0 +1,5 @@
SECRET_KEY=k_=*vfue(^campsl63)7w5m&cu9u4o4-!vaw94qzyrymyv0hgg
DEBUG=True
ALLOWED_HOSTS=.localhost,127.0.0.1
DATABASE_URL=postgres://USER:PASSWORD@HOST:PORT/NAME
STATIC_ROOT=PATH_TO_STATIC_ROOT

7
INSTALLDEV.md
View File

@ -11,7 +11,7 @@ If you are planing to do some development you may want to consider doing it unde
2. Build a new image, create and start a container
```bash
curl -L http://git.io/orchestra-Dockerfile > /tmp/Dockerfile
curl -L https://raw.githubusercontent.com/ribaguifi/django-orchestra/master/scripts/containers/Dockerfile > /tmp/Dockerfile
docker build -t orchestra /tmp/
docker create --name orchestra -i -t -u orchestra -w /home/orchestra orchestra bash
docker start orchestra
@ -21,12 +21,13 @@ If you are planing to do some development you may want to consider doing it unde
3. Deploy django-orchestra development environment, inside the container
```bash
bash <( curl -L http://git.io/orchestra-deploy ) --dev
bash <( curl -L https://raw.githubusercontent.com/ribaguifi/django-orchestra/master/scripts/containers/deploy.sh ) --dev
```
3. Nginx should be serving on port 80, but Django's development server can be used as well:
```bash
cd panel
python3 manage.py migrate
python3 manage.py runserver 0.0.0.0:8888
```
@ -34,5 +35,5 @@ If you are planing to do some development you may want to consider doing it unde
5. To upgrade to current master just re-run the deploy script
```bash
git pull origin master
bash <( curl -L http://git.io/orchestra-deploy ) --dev
bash <( curl -L https://raw.githubusercontent.com/ribaguifi/django-orchestra/master/scripts/containers/deploy.sh ) --dev
```

70
INSTALL_RIBAGUIFI_STYLE.md Normal file
View File

@ -0,0 +1,70 @@
We need have python3.6
#Install Packages
```bash
apt=(
bind9utils
ca-certificates
gettext
libcrack2-dev
libxml2-dev
libxslt1-dev
ssh-client
wget
xvfb
zlib1g-dev
git
iceweasel
dnsutils
postgresql-contrib
)
sudo apt-get install --no-install-recommends -y ${apt[@]}
```
It is necessary install *wkhtmltopdf*
You can install it from https://wkhtmltopdf.org/downloads.html
Clone this repository
```bash
git clone https://github.com/ribaguifi/django-orchestra
```
Prepare env and install requirements
```bash
cd django-orchestra
python3.6 -m venv env
source env/bin/activate
pip3 install --upgrade pip
pip3 install -r total_requirements.txt
pip3 install -e .
```
Configure project using environment file (you can use provided example as quickstart):
```bash
cp .env.example .env
```
Prepare your Postgres database (create database, user and grant permissions):
```sql
CREATE DATABASE myproject;
CREATE USER myuser WITH PASSWORD 'password';
GRANT ALL PRIVILEGES ON DATABASE myproject TO myuser;
```
Prepare a new project:
```bash
django-admin.py startproject PROJECT_NAME --template="orchestra/conf/ribaguifi_template"
```
Run migrations:
```bash
python3 manage.py migrate
```
(Optional) You can start a Django development server to check that everything is ok.
```bash
python3 manage.py runserver
```
Open [http://127.0.0.1:8000/](http://127.0.0.1:8000/) in your browser.

132
install_manually.md Normal file
View File

@ -0,0 +1,132 @@
# System requirements:
The most important requirement is use python3.6
we need install this packages:
```
bind9utils
ca-certificates
gettext
libcrack2-dev
libxml2-dev
libxslt1-dev
python3
python3-pip
python3-dev
ssh-client
wget
xvfb
zlib1g-dev
git
iceweasel
dnsutils
```
We need install too a *wkhtmltopdf* package
You can use one of your OS or get it from original.
This it is in https://wkhtmltopdf.org/downloads.html
# pip installations
We need install this packages:
```
Django==1.10.5
django-fluent-dashboard==0.6.1
django-admin-tools==0.8.0
django-extensions==1.7.4
django-celery==3.1.17
celery==3.1.23
kombu==3.0.35
billiard==3.3.0.23
Markdown==2.4
djangorestframework==3.4.7
ecdsa==0.11
Pygments==1.6
django-filter==0.15.2
jsonfield==0.9.22
python-dateutil==2.2
https://github.com/glic3rinu/passlib/archive/master.zip
django-iban==0.3.0
requests
phonenumbers
django-countries
django-localflavor
amqp
anyjson
pytz
cracklib
lxml==3.3.5
selenium
xvfbwrapper
freezegun
coverage
flake8
django-debug-toolbar==1.3.0
django-nose==1.4.4
sqlparse
pyinotify
PyMySQL
```
If you want to use Orchestra you need to install from pip like this:
```
pip3 install http://git.io/django-orchestra-dev
```
But if you want develop orquestra you need to do this:
```
git clone https://github.com/ribaguifi/django-orchestra
pip install -e django-orchestra
```
# Database
For default use sqlite3 if you want to use postgresql you need install this packages:
```
psycopg2 postgresql
```
You can use it for debian or ubuntu:
```
sudo apt-get install python3-psycopg2 postgresql-contrib
```
Remember create a database for your project and give permitions for the correct user like this:
```
psql -U postgres
psql (12.4)
Digite «help».
postgres=# CREATE database orchesta;
postgres=# CREATE USER orchesta WITH PASSWORD 'orquesta';
postgres=# GRANT ALL PRIVILEGES ON DATABASE orchesta TO orchesta;
```
# Create new project
You can use orchestra-admin for create your new project
```
orchestra-admin startproject <project_name> # e.g. panel
cd <project_name>
```
Next we need change the settings.py for configure the correct database
In settings.py we need change the DATABASE section like this:
```
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'orchestra'
'USER': 'orchestra',
'PASSWORD': 'orchestra',
'HOST': 'localhost',
'PORT': '5432',
'CONN_MAX_AGE': 60*10
}
}
```
For end you need to do the migrations:
```
python3 manage.py migrate
```

2
orchestra/bin/orchestra-admin
View File

@ -174,7 +174,7 @@ function install_requirements () {
minor=$(echo -e "$wkhtmltox_version\n0.12.2.1" | sort -V | head -n 1)
if [[ ! $wkhtmltox_version ]] || [[ $wkhtmltox_version != 0.12.2.1 && $minor == ${wkhtmltox_version} ]]; then
wkhtmltox=$(mktemp)
wget http://download.gna.org/wkhtmltopdf/0.12/0.12.2.1/wkhtmltox-0.12.2.1_linux-jessie-amd64.deb -O ${wkhtmltox}
wget https://github.com/wkhtmltopdf/packaging/releases/download/0.12.6-1/wkhtmltox_0.12.6-1.buster_amd64.deb -O ${wkhtmltox}
dpkg -i ${wkhtmltox} || { echo "Installing missing dependencies for wkhtmltox..." && apt-get -f -y install; }
fi
}

0
orchestra/conf/ribaguifi_template/locale/.gitignore vendored Normal file
View File

13
orchestra/conf/ribaguifi_template/manage.py Executable file
View File

@ -0,0 +1,13 @@
#!/usr/bin/env python3
import os
import sys
if __name__ == "__main__":
if sys.version_info < (3, 3):
cmd = ' '.join(sys.argv)
sys.stderr.write("Sorry, Orchestra requires at least Python 3.3, try with:\n$ python3 %s\n" % cmd)
sys.exit(1)
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "{{ project_name }}.settings")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)

0
orchestra/conf/ribaguifi_template/media/.gitignore vendored Normal file
View File

0
orchestra/conf/ribaguifi_template/project_name/__init__.py Normal file
View File

257
orchestra/conf/ribaguifi_template/project_name/settings.py Normal file
View File

@ -0,0 +1,257 @@
"""
Django settings for {{ project_name }} project.
Generated by 'django-admin startproject' using Django {{ django_version }}.
For more information on this file, see
https://docs.djangoproject.com/en/{{ docs_version }}/topics/settings/
For the full list of settings and their values, see
https://docs.djangoproject.com/en/{{ docs_version }}/ref/settings/
"""
import os
from decouple import config, Csv
from dj_database_url import parse as db_url
# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/{{ docs_version }}/howto/deployment/checklist/
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = '{{ secret_key }}'
# SECRET_KEY = config('SECRET_KEY')
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = config('DEBUG', default=False, cast=bool)
ALLOWED_HOSTS = config('ALLOWED_HOSTS', default=[], cast=Csv())
# Application definition
INSTALLED_APPS = [
# django-orchestra apps
'orchestra',
'orchestra.contrib.accounts',
'orchestra.contrib.systemusers',
'orchestra.contrib.contacts',
'orchestra.contrib.orchestration',
'orchestra.contrib.bills',
'orchestra.contrib.payments',
'orchestra.contrib.tasks',
'orchestra.contrib.mailer',
'orchestra.contrib.history',
'orchestra.contrib.issues',
'orchestra.contrib.services',
'orchestra.contrib.plans',
'orchestra.contrib.orders',
'orchestra.contrib.domains',
'orchestra.contrib.mailboxes',
'orchestra.contrib.lists',
'orchestra.contrib.webapps',
'orchestra.contrib.websites',
'orchestra.contrib.letsencrypt',
'orchestra.contrib.databases',
'orchestra.contrib.vps',
'orchestra.contrib.saas',
'orchestra.contrib.miscellaneous',
# Third-party apps
'django_extensions',
'djcelery',
'fluent_dashboard',
'admin_tools',
'admin_tools.theming',
'admin_tools.menu',
'admin_tools.dashboard',
'rest_framework',
'rest_framework.authtoken',
'passlib.ext.django',
'django_countries',
# 'debug_toolbar',
# Django.contrib
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.admin.apps.SimpleAdminConfig',
# Last to load
'orchestra.contrib.resources',
'orchestra.contrib.settings',
# 'django_nose',
]
ROOT_URLCONF = '{{ project_name }}.urls'
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [],
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
'orchestra.core.context_processors.site',
],
'loaders': [
'admin_tools.template_loaders.Loader',
'django.template.loaders.filesystem.Loader',
'django.template.loaders.app_directories.Loader',
],
},
},
]
WSGI_APPLICATION = '{{ project_name }}.wsgi.application'
# Database
# https://docs.djangoproject.com/en/{{ docs_version }}/ref/settings/#databases
DATABASES = {
'default': config(
'DATABASE_URL',
default='sqlite:///' + os.path.join(BASE_DIR, 'db.sqlite3'),
cast=db_url
)
}
# Internationalization
# https://docs.djangoproject.com/en/{{ docs_version }}/topics/i18n/
LANGUAGE_CODE = 'en-us'
try:
TIME_ZONE = open('/etc/timezone', 'r').read().strip()
except IOError:
TIME_ZONE = 'UTC'
USE_I18N = True
USE_L10N = True
USE_TZ = True
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/{{ docs_version }}/howto/static-files/
STATIC_URL = '/static/'
# Absolute path to the directory static files should be collected to.
# Don't put anything in this directory yourself; store your static files
# in apps' "static/" subdirectories and in STATICFILES_DIRS.
# Example: "/home/media/media.lawrence.com/static/"
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
# Absolute filesystem path to the directory that will hold user-uploaded files.
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
# Path used for database translations files
LOCALE_PATHS = (
os.path.join(BASE_DIR, 'locale'),
)
ORCHESTRA_SITE_NAME = '{{ project_name }}'
MIDDLEWARE_CLASSES = (
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
# 'django.middleware.locale.LocaleMiddleware'
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
'django.middleware.security.SecurityMiddleware',
'orchestra.core.caches.RequestCacheMiddleware',
# also handles transations, ATOMIC_REQUESTS does not wrap middlewares
'orchestra.contrib.orchestration.middlewares.OperationsMiddleware',
)
AUTH_USER_MODEL = 'accounts.Account'
AUTHENTICATION_BACKENDS = [
'orchestra.permissions.auth.OrchestraPermissionBackend',
'django.contrib.auth.backends.ModelBackend',
]
EMAIL_BACKEND = 'orchestra.contrib.mailer.backends.EmailBackend'
# Needed for Bulk operations
DATA_UPLOAD_MAX_NUMBER_FIELDS = None
#################################
## 3RD PARTY APPS CONIGURATION ##
#################################
# Admin Tools
ADMIN_TOOLS_MENU = 'orchestra.admin.menu.OrchestraMenu'
# Fluent dashboard
ADMIN_TOOLS_INDEX_DASHBOARD = 'orchestra.admin.dashboard.OrchestraIndexDashboard'
FLUENT_DASHBOARD_ICON_THEME = '../orchestra/icons'
# Django-celery
import djcelery
djcelery.setup_loader()
CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'
# rest_framework
REST_FRAMEWORK = {
'DEFAULT_PERMISSION_CLASSES': (
'orchestra.permissions.api.OrchestraPermissionBackend',
),
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework.authentication.SessionAuthentication',
'rest_framework.authentication.TokenAuthentication',
),
'DEFAULT_FILTER_BACKENDS': (
('rest_framework.filters.DjangoFilterBackend',)
),
}
# Use a UNIX compatible hash
PASSLIB_CONFIG = (
"[passlib]\n"
"schemes = sha512_crypt, django_pbkdf2_sha256, django_pbkdf2_sha1, "
" django_bcrypt, django_bcrypt_sha256, django_salted_sha1, des_crypt, "
" django_salted_md5, django_des_crypt, hex_md5, bcrypt, phpass\n"
"default = sha512_crypt\n"
"deprecated = django_pbkdf2_sha1, django_salted_sha1, django_salted_md5, "
" django_des_crypt, des_crypt, hex_md5\n"
"all__vary_rounds = 0.05\n"
"django_pbkdf2_sha256__min_rounds = 10000\n"
"sha512_crypt__min_rounds = 80000\n"
"staff__django_pbkdf2_sha256__default_rounds = 12500\n"
"staff__sha512_crypt__default_rounds = 100000\n"
"superuser__django_pbkdf2_sha256__default_rounds = 15000\n"
"superuser__sha512_crypt__default_rounds = 120000\n"
)
SHELL_PLUS_PRE_IMPORTS = (
('orchestra.contrib.orchestration.managers', ('orchestrate',)),
)

6
orchestra/conf/ribaguifi_template/project_name/urls.py Normal file
View File

@ -0,0 +1,6 @@
from django.conf.urls import include, url
urlpatterns = [
url(r'', include('orchestra.urls')),
]

14
orchestra/conf/ribaguifi_template/project_name/wsgi.py Normal file
View File

@ -0,0 +1,14 @@
"""
WSGI config for {{ project_name }} project.
It exposes the WSGI callable as a module-level variable named ``application``.
For more information on this file, see
https://docs.djangoproject.com/en/{{ docs_version }}/howto/deployment/wsgi/
"""
import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "{{ project_name }}.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()

47
orchestra/management/commands/postupgradeorchestra.py
View File

@ -1,6 +1,5 @@
import re
import os
from optparse import make_option
from django.core.management.base import BaseCommand
@ -19,26 +18,44 @@ def deprecate_periodic_tasks(names):
class Command(BaseCommand):
def __init__(self, *args, **kwargs):
super(Command, self).__init__(*args, **kwargs)
self.option_list = BaseCommand.option_list + (
make_option('--no-restart', action='store_false', dest='restart', default=True,
help='Only install local requirements'),
make_option('--specifics', action='store_true', dest='specifics_only',
default=False, help='Only run version specific operations'),
make_option('--no-upgrade-notes', action='store_false', default=True,
dest='print_upgrade_notes', help='Do not print specific upgrade notes'),
make_option('--from', dest='version', default=False,
help="Orchestra's version from where you are upgrading, i.e 0.0.1"),
)
option_list = BaseCommand.option_list
help = 'Upgrades django-orchestra installation'
# This command must be able to run in an environment with unsatisfied dependencies
leave_locale_alone = True
can_import_settings = False
requires_model_validation = False
def __init__(self, *args, **kwargs):
super(Command, self).__init__(*args, **kwargs)
def add_arguments(self, parser):
parser.add_argument(
'--no-restart',
action='store_false',
dest='restart',
default=True,
help='Only install local requirements'
)
parser.add_argument(
'--specifics',
action='store_true',
dest='specifics_only',
default=False,
help='Only run version specific operations'
)
parser.add_argument(
'--no-upgrade-notes',
action='store_false',
default=True,
dest='print_upgrade_notes',
help='Do not print specific upgrade notes'
)
parser.add_argument(
'--from',
dest='version',
default=False,
help="Orchestra's version from where you are upgrading, i.e 0.0.1"
)
@check_root
def handle(self, *args, **options):
version = options.get('version')

1
orchestra/management/commands/restartservices.py
View File

@ -7,5 +7,4 @@ from orchestra.management.commands.startservices import ManageServiceCommand
class Command(ManageServiceCommand):
services = settings.ORCHESTRA_RESTART_SERVICES
action = 'restart'
option_list = BaseCommand.option_list
help = 'Restart all related services. Usefull for reload configuration and files.'

37
orchestra/management/commands/setupcelery.py
View File

@ -1,5 +1,4 @@
import textwrap
from optparse import make_option
from os import path
from django.core.management.base import BaseCommand
@ -10,21 +9,33 @@ from orchestra.utils.sys import run, check_root
class Command(BaseCommand):
help = 'Configure Celeryd to run with your orchestra instance.'
def __init__(self, *args, **kwargs):
super(Command, self).__init__(*args, **kwargs)
self.option_list = BaseCommand.option_list + (
make_option('--username', dest='username', default='orchestra',
help='Specifies the system user that would run celeryd.'),
make_option('--processes', dest='processes', default=2,
help='Number of celeryd processes.'),
make_option('--noinput', action='store_false', dest='interactive', default=True,
help='Tells Django to NOT prompt the user for input of any kind. '
'You must use --username with --noinput, and must contain the '
'cleleryd process owner, which is the user how will perform tincd updates'),
)
option_list = BaseCommand.option_list
help = 'Configure Celeryd to run with your orchestra instance.'
def add_arguments(self, parser):
parser.add_argument(
'--username',
dest='username',
default='orchestra',
help='Specifies the system user that would run celeryd.'
)
parser.add_argument(
'--processes',
dest='processes',
default=2,
help='Number of celeryd processes.'
)
parser.add_argument(
'--noinput',
action='store_false',
dest='interactive',
default=True,
help='''Tells Django to NOT prompt the user for input of any kind.
You must use --username with --noinput, and must contain the
cleleryd process owner, which is the user how will perform tincd updates'''
)
@check_root
def handle(self, *args, **options):

152
orchestra/management/commands/setupnginx.py
View File

@ -1,6 +1,5 @@
import os
import textwrap
from optparse import make_option
from os.path import expanduser
from django.conf import settings
@ -11,54 +10,117 @@ from orchestra.utils.sys import run, check_root, confirm
class Command(BaseCommand):
help = 'Configures nginx + uwsgi to run with your Orchestra instance.'
def __init__(self, *args, **kwargs):
super(Command, self).__init__(*args, **kwargs)
self.option_list = BaseCommand.option_list + (
make_option('--cert', dest='cert', default='',
help='Nginx SSL certificate, one will be created by default.'),
make_option('--cert-key', dest='cert_key', default='',
help='Nginx SSL certificate key.'),
make_option('--cert-path', dest='cert_path',
default=os.path.join(paths.get_site_dir(), 'ssl', 'orchestra.crt'),
help='Nginx SSL certificate, one will be created by default.'),
make_option('--cert-key-path', dest='cert_key_path',
default=os.path.join(paths.get_site_dir(), 'ssl', 'orchestra.key'),
help='Nginx SSL certificate key.'),
# Cert options
make_option('--cert-override', dest='cert_override', action='store_true',
default=False, help='Force override cert and keys if exists.'),
make_option('--cert-country', dest='cert_country', default='ES',
help='Certificate Distinguished Name Country.'),
make_option('--cert-state', dest='cert_state', default='Spain',
help='Certificate Distinguished Name STATE.'),
make_option('--cert-locality', dest='cert_locality', default='Barcelona',
help='Certificate Distinguished Name Country.'),
make_option('--cert-org_name', dest='cert_org_name', default='Orchestra',
help='Certificate Distinguished Name Organization Name.'),
make_option('--cert-org_unit', dest='cert_org_unit', default='DevOps',
help='Certificate Distinguished Name Organization Unity.'),
make_option('--cert-email', dest='cert_email', default='orchestra@orchestra.lan',
help='Certificate Distinguished Name Email Address.'),
make_option('--cert-common_name', dest='cert_common_name', default=None,
help='Certificate Distinguished Name Common Name.'),
make_option('--server-name', dest='server_name', default='',
help='Nginx SSL certificate key.'),
make_option('--user', dest='user', default='',
help='uWSGI daemon user.'),
make_option('--group', dest='group', default='',
help='uWSGI daemon group.'),
make_option('--processes', dest='processes', default=4,
help='uWSGI number of processes.'),
make_option('--noinput', action='store_false', dest='interactive', default=True,
help='Tells Django to NOT prompt the user for input of any kind. '
'You must use --username with --noinput, and must contain the '
'cleeryd process owner, which is the user how will perform tincd updates'),
def add_arguments(self, parser):
parser.add_argument(
'--cert',
dest='cert',
default='',
help='Nginx SSL certificate, one will be created by default.',
)
parser.add_argument(
'--cert-key',
dest='cert_key',
default='',
help='Nginx SSL certificate key.'
)
parser.add_argument(
'--cert-path',
dest='cert_path',
default=os.path.join(paths.get_site_dir(), 'ssl', 'orchestra.crt'),
help='Nginx SSL certificate, one will be created by default.'
)
parser.add_argument(
'--cert-key-path',
dest='cert_key_path',
default=os.path.join(paths.get_site_dir(), 'ssl', 'orchestra.key'),
help='Nginx SSL certificate key.'
)
parser.add_argument(
'--cert-override',
dest='cert_override',
action='store_true',
default=False, help='Force override cert and keys if exists.'
)
parser.add_argument(
'--cert-country',
dest='cert_country',
default='ES',
help='Certificate Distinguished Name Country.'
)
parser.add_argument(
'--cert-state',
dest='cert_state',
default='Spain',
help='Certificate Distinguished Name STATE.'
)
parser.add_argument(
'--cert-locality',
dest='cert_locality',
default='Barcelona',
help='Certificate Distinguished Name Country.'
)
parser.add_argument(
'--cert-org_name',
dest='cert_org_name',
default='Orchestra',
help='Certificate Distinguished Name Organization Name.'
)
parser.add_argument(
'--cert-org_unit',
dest='cert_org_unit',
default='DevOps',
help='Certificate Distinguished Name Organization Unity.'
)
parser.add_argument(
'--cert-email',
dest='cert_email',
default='orchestra@orchestra.lan',
help='Certificate Distinguished Name Email Address.'
)
parser.add_argument(
'--cert-common_name',
dest='cert_common_name',
default=None,
help='Certificate Distinguished Name Common Name.'
)
parser.add_argument(
'--server-name',
dest='server_name',
default='',
help='Nginx SSL certificate key.'
)
parser.add_argument(
'--user',
dest='user',
default='',
help='uWSGI daemon user.'
)
parser.add_argument(
'--group',
dest='group',
default='',
help='uWSGI daemon group.'
)
parser.add_argument(
'--processes',
dest='processes',
default=4,
help='uWSGI number of processes.'
)
parser.add_argument(
'--noinput',
action='store_false',
dest='interactive',
default=True,
help='''Tells Django to NOT prompt the user for input of any kind.
You must use --username with --noinput, and must contain the
cleeryd process owner, which is the user how will perform tincd updates'''
)
option_list = BaseCommand.option_list
help = 'Configures nginx + uwsgi to run with your Orchestra instance.'
def generate_certificate(self, **options):
override = options.get('cert_override')

121
orchestra/management/commands/setuppostfix.py
View File

@ -1,52 +1,97 @@
import os
from optparse import make_option
from django.core.management.base import BaseCommand
from orchestra.utils.sys import run, check_root
class Command(BaseCommand):
help = 'Setup Postfix.'
def __init__(self, *args, **kwargs):
super(Command, self).__init__(*args, **kwargs)
self.option_list = BaseCommand.option_list + (
make_option('--db_name', dest='db_name', default='orchestra',
help='Specifies the database to create.'),
make_option('--db_user', dest='db_user', default='orchestra',
help='Specifies the database to create.'),
make_option('--db_password', dest='db_password', default='orchestra',
help='Specifies the database to create.'),
make_option('--db_host', dest='db_host', default='localhost',
help='Specifies the database to create.'),
make_option('--vmail_username', dest='vmail_username', default='vmail',
help='Specifies username in the operating system (default=vmail).'),
make_option('--vmail_uid', dest='vmail_uid', default='5000',
help='UID of user <vmail_username> (default=5000).'),
make_option('--vmail_groupname', dest='vmail_groupname', default='vmail',
help='Specifies the groupname in the operating system (default=vmail).'),
make_option('--vmail_gid', dest='vmail_gid', default='5000',
help='GID of user <vmail_username> (default=5000).'),
make_option('--vmail_home', dest='vmail_home', default='/var/vmail',
help='$HOME of user <vmail_username> (default=/var/vmail).'),
make_option('--dovecot_dir', dest='dovecot_dir', default='/etc/dovecot',
help='Dovecot root directory (default=/etc/dovecot).'),
make_option('--postfix_dir', dest='postfix_dir', default='/etc/postfix',
help='Postfix root directory (default=/etc/postfix).'),
make_option('--amavis_dir', dest='amavis_dir', default='/etc/amavis',
help='Amavis root directory (default=/etc/amavis).'),
make_option('--noinput', action='store_false', dest='interactive', default=True,
help='Tells Django to NOT prompt the user for input of any kind. '
'You must use --username with --noinput, and must contain the '
'cleeryd process owner, which is the user how will perform tincd updates'),
def add_arguments(self, parser):
parser.add_argument(
'--db_name',
dest='db_name',
default='orchestra',
help='Specifies the database to create.'
)
parser.add_argument(
'--db_user',
dest='db_user',
default='orchestra',
help='Specifies the database to create.'
)
parser.add_argument(
'--db_password',
dest='db_password',
default='orchestra',
help='Specifies the database to create.'
)
parser.add_argument(
'--db_host',
dest='db_host',
default='localhost',
help='Specifies the database to create.'
)
parser.add_argument(
'--vmail_username',
dest='vmail_username',
default='vmail',
help='Specifies username in the operating system (default=vmail).'
)
parser.add_argument(
'--vmail_uid',
dest='vmail_uid',
default='5000',
help='UID of user <vmail_username> (default=5000).'
)
parser.add_argument(
'--vmail_groupname',
dest='vmail_groupname',
default='vmail',
help='Specifies the groupname in the operating system (default=vmail).'
)
parser.add_argument(
'--vmail_gid',
dest='vmail_gid',
default='5000',
help='GID of user <vmail_username> (default=5000).'
)
parser.add_argument(
'--vmail_home',
dest='vmail_home',
default='/var/vmail',
help='$HOME of user <vmail_username> (default=/var/vmail).'
)
parser.add_argument(
'--dovecot_dir',
dest='dovecot_dir',
default='/etc/dovecot',
help='Dovecot root directory (default=/etc/dovecot).'
)
parser.add_argument(
'--postfix_dir',
dest='postfix_dir',
default='/etc/postfix',
help='Postfix root directory (default=/etc/postfix).'
)
parser.add_argument(
'--amavis_dir',
dest='amavis_dir',
default='/etc/amavis',
help='Amavis root directory (default=/etc/amavis).'
)
parser.add_argument(
'--noinput',
action='store_false',
dest='interactive',
default=True,
help='''Tells Django to NOT prompt the user for input of any kind.
You must use --username with --noinput, and must contain the
cleeryd process owner, which is the user how will perform tincd updates'''
)
option_list = BaseCommand.option_list
help = 'Setup Postfix.'
@check_root
def handle(self, *args, **options):

78
orchestra/management/commands/setuppostgres.py
View File

@ -1,6 +1,5 @@
import os
import textwrap
from optparse import make_option
from django.conf import settings
from django.core.management.base import BaseCommand, CommandError
@ -11,41 +10,64 @@ from orchestra.utils.sys import run, check_root
class Command(BaseCommand):
help = 'Setup PostgreSQL database.'
def __init__(self, *args, **kwargs):
super(Command, self).__init__(*args, **kwargs)
# Get defaults from settings, if exists
try:
defaults = settings.DATABASES['default']
self.defaults = settings.DATABASES['default']
except (AttributeError, KeyError):
defaults = {}
else:
if defaults['ENGINE'] != 'django.db.backends.postgresql_psycopg2':
defaults = {}
if self.defaults['ENGINE'] != 'django.db.backends.postgresql_psycopg2':
self.defaults = {}
self.option_list = BaseCommand.option_list + (
make_option('--db_name', dest='db_name',
default=defaults.get('DB_NAME', 'orchestra'),
help='Specifies the database to create.'),
make_option('--db_user', dest='db_user',
default=defaults.get('DB_USER', 'orchestra'),
help='Specifies the database to create.'),
make_option('--db_password', dest='db_password',
default=defaults.get('PASSWORD', ''),
help='Specifies the database password, random if not specified.'),
make_option('--db_host', dest='db_host',
default=defaults.get('HOST', 'localhost'),
help='Specifies the database to create.'),
make_option('--db_port', dest='db_port',
default=defaults.get('PORT', '5432'),
help='Specifies the database to create.'),
make_option('--noinput', action='store_false', dest='interactive', default=True,
help='Tells Django to NOT prompt the user for input of any kind. '
'You must use --username with --noinput, and must contain the '
'cleeryd process owner, which is the user how will perform tincd updates'),
def add_arguments(self, parser):
parser.add_argument(
'--db_name',
dest='db_name',
default=self.defaults.get('DB_NAME', 'orchestra'),
help='Specifies the database to create.',
type=str
)
parser.add_argument(
'--db_user',
dest='db_user',
default=self.defaults.get('DB_USER', 'orchestra'),
help='Specifies the database to create.',
type=str
)
parser.add_argument(
'--db_password',
dest='db_password',
default=self.defaults.get('PASSWORD', ''),
help='Specifies the database password, random if not specified.',
type=str
)
parser.add_argument(
'--db_host',
dest='db_host',
default=self.defaults.get('HOST', 'localhost'),
help='Specifies the database to create.',
type=str
)
parser.add_argument(
'--db_port',
dest='db_port',
default=self.defaults.get('PORT', '5432'),
help='Specifies the database to create.',
type=str
)
parser.add_argument(
'--noinput',
action='store_false',
dest='interactive',
default=True,
help='''Tells Django to NOT prompt the user for input of any kind.
You must use --username with --noinput, and must contain the
cleeryd process owner, which is the user how will perform tincd updates'''
)
option_list = BaseCommand.option_list
help = 'Setup PostgreSQL database.'
def run_postgres(self, cmd, *args, **kwargs):
return run('su postgres -c "psql -c \\"%s\\""' % cmd, *args, **kwargs)
@ -82,7 +104,7 @@ class Command(BaseCommand):
self.stdout.write(msg % context)
else:
raise CommandError("Postgres user '%(db_user)s' already exists and "
"--db_pass has not been provided." % context)
"--db_password has not been provided." % context)
else:
context['db_password'] = context['default_db_password']
msg = "Created new Postgres user '%(db_user)s' with password '%(db_password)s'"

16
orchestra/management/commands/startservices.py
View File

@ -1,5 +1,3 @@
from optparse import make_option
from django.core.management.base import BaseCommand
from orchestra import settings
@ -30,9 +28,16 @@ def flatten(nested, depth=0):
class ManageServiceCommand(BaseCommand):
def __init__(self, *args, **kwargs):
super(ManageServiceCommand, self).__init__(*args, **kwargs)
self.option_list = BaseCommand.option_list + tuple(
make_option('--no-%s' % service, action='store_false', dest=service, default=True,
help='Do not %s %s' % (self.action, service)) for service in flatten(self.services)
def add_arguments(self, parser):
for service in flatten(self.services):
parser.add_argument(
'--no-%s' % service,
action='store_false',
dest=service,
default=True,
help='Do not %s %s' % (self.action, service)
)
@check_root
@ -54,5 +59,4 @@ class ManageServiceCommand(BaseCommand):
class Command(ManageServiceCommand):
services = settings.ORCHESTRA_START_SERVICES
action = 'start'
option_list = BaseCommand.option_list
help = 'Start all related services. Usefull for reload configuration and files.'

1
orchestra/management/commands/stopservices.py
View File

@ -7,5 +7,4 @@ from orchestra.management.commands.startservices import ManageServiceCommand
class Command(ManageServiceCommand):
services = settings.ORCHESTRA_STOP_SERVICES
action = 'stop'
option_list = BaseCommand.option_list
help = 'Stop all related services. Usefull for reload configuration and files.'

29
orchestra/management/commands/upgradeorchestra.py
View File

@ -3,7 +3,6 @@ import os
import random
import string
from distutils.sysconfig import get_python_lib
from optparse import make_option
from django.core.management import call_command
from django.core.management.base import BaseCommand, CommandError
@ -26,20 +25,28 @@ def get_existing_pip_installation():
class Command(BaseCommand):
def __init__(self, *args, **kwargs):
super(Command, self).__init__(*args, **kwargs)
self.option_list = BaseCommand.option_list + (
make_option('--pip_only', action='store_true', dest='pip_only', default=False,
help='Only run "pip install django-orchestra --upgrade"'),
make_option('--orchestra_version', dest='version', default=False,
help='Specifies what version of the Orchestra you want to install'),
)
option_list = BaseCommand.option_list
help = "Upgrading Orchestra's installation. Desired version is accepted as argument"
can_import_settings = False
leave_locale_alone = True
def __init__(self, *args, **kwargs):
super(Command, self).__init__(*args, **kwargs)
def add_arguments(self, parser):
parser.add_argument(
'--pip_only',
action='store_true',
dest='pip_only',
default=False,
help='Only run "pip install django-orchestra --upgrade"'
)
parser.add_argument(
'--orchestra_version',
dest='version',
default=False,
help='Specifies what version of the Orchestra you want to install'
)
@check_root
def handle(self, *args, **options):
current_version = get_version()

136
scripts/containers/Dockerfile
View File

@ -1,8 +1,138 @@
FROM debian:latest
#
# NOTE: THIS DOCKERFILE IS GENERATED VIA "update.sh"
#
# PLEASE DO NOT EDIT IT DIRECTLY.
#
FROM buildpack-deps:buster
# ensure local python is preferred over distribution python
ENV PATH /usr/local/bin:$PATH
# http://bugs.python.org/issue19846
# > At the moment, setting "LANG=C" on a Linux system *fundamentally breaks Python 3*, and that's not OK.
ENV LANG C.UTF-8
# extra dependencies (over what buildpack-deps already includes)
RUN apt-get update && apt-get install -y --no-install-recommends \
libbluetooth-dev \
tk-dev \
&& rm -rf /var/lib/apt/lists/*
ENV GPG_KEY 0D96DF4D4110E5C43FBFB17F2D347EA6AA65421D
ENV PYTHON_VERSION 3.6.12
RUN set -ex \
\
&& wget -O python.tar.xz "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz" \
&& wget -O python.tar.xz.asc "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz.asc" \
&& export GNUPGHOME="$(mktemp -d)" \
&& gpg --batch --keyserver ha.pool.sks-keyservers.net --recv-keys "$GPG_KEY" \
&& gpg --batch --verify python.tar.xz.asc python.tar.xz \
&& { command -v gpgconf > /dev/null && gpgconf --kill all || :; } \
&& rm -rf "$GNUPGHOME" python.tar.xz.asc \
&& mkdir -p /usr/src/python \
&& tar -xJC /usr/src/python --strip-components=1 -f python.tar.xz \
&& rm python.tar.xz \
\
&& cd /usr/src/python \
&& gnuArch="$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)" \
&& ./configure \
--build="$gnuArch" \
--enable-loadable-sqlite-extensions \
--enable-optimizations \
--enable-option-checking=fatal \
--enable-shared \
--with-system-expat \
--with-system-ffi \
--without-ensurepip \
&& make -j "$(nproc)" \
# setting PROFILE_TASK makes "--enable-optimizations" reasonable: https://bugs.python.org/issue36044 / https://github.com/docker-library/python/issues/160#issuecomment-509426916
PROFILE_TASK='-m test.regrtest --pgo \
test_array \
test_base64 \
test_binascii \
test_binhex \
test_binop \
test_bytes \
test_c_locale_coercion \
test_class \
test_cmath \
test_codecs \
test_compile \
test_complex \
test_csv \
test_decimal \
test_dict \
test_float \
test_fstring \
test_hashlib \
test_io \
test_iter \
test_json \
test_long \
test_math \
test_memoryview \
test_pickle \
test_re \
test_set \
test_slice \
test_struct \
test_threading \
test_time \
test_traceback \
test_unicode \
' \
&& make install \
&& rm -rf /usr/src/python \
\
&& find /usr/local -depth \
\( \
\( -type d -a \( -name test -o -name tests -o -name idle_test \) \) \
-o \( -type f -a \( -name '*.pyc' -o -name '*.pyo' -o -name '*.a' \) \) \
-o \( -type f -a -name 'wininst-*.exe' \) \
\) -exec rm -rf '{}' + \
\
&& ldconfig \
\
&& python3 --version
# make some useful symlinks that are expected to exist
RUN cd /usr/local/bin \
&& ln -s idle3 idle \
&& ln -s pydoc3 pydoc \
&& ln -s python3 python \
&& ln -s python3-config python-config
# if this is called "PIP_VERSION", pip explodes with "ValueError: invalid truth value '<VERSION>'"
ENV PYTHON_PIP_VERSION 21.0
# https://github.com/pypa/get-pip
ENV PYTHON_GET_PIP_URL https://github.com/pypa/get-pip/raw/8cc88aca7d9775fce279e8b84ef163cf1d3e8a2e/get-pip.py
ENV PYTHON_GET_PIP_SHA256 ffb67da2e976f48dd29714fc64812d1ac419eb7d48079737166dd95640d1debd
RUN set -ex; \
\
wget -O get-pip.py "$PYTHON_GET_PIP_URL"; \
echo "$PYTHON_GET_PIP_SHA256 *get-pip.py" | sha256sum --check --strict -; \
\
python get-pip.py \
--disable-pip-version-check \
--no-cache-dir \
"pip==$PYTHON_PIP_VERSION" \
; \
pip --version; \
\
find /usr/local -depth \
\( \
\( -type d -a \( -name test -o -name tests -o -name idle_test \) \) \
-o \
\( -type f -a \( -name '*.pyc' -o -name '*.pyo' \) \) \
\) -exec rm -rf '{}' +; \
rm -f get-pip.py
RUN apt-get -y update && apt-get install -y curl sudo
RUN export TERM=xterm; curl -L http://git.io/orchestra-admin | bash -s install_requirements
RUN export TERM=xterm; curl -L https://raw.githubusercontent.com/ribaguifi/django-orchestra/master/orchestra/bin/orchestra-admin | bash -s install_requirements
RUN apt-get clean
@ -10,3 +140,5 @@ RUN useradd orchestra --shell /bin/bash && \
{ echo "orchestra:orchestra" | chpasswd; } && \
mkhomedir_helper orchestra && \
adduser orchestra sudo
CMD ["python3"]

19
scripts/containers/deploy.sh
View File

@ -46,13 +46,16 @@ function install_orchestra () {
if [[ $dev ]]; then
# Install from source
python_path=$(python3 -c "import sys; print([path for path in sys.path if path.startswith('/usr/local/lib/python')][0]);")
python_path=$(python3 -c "import sys; print([path for path in sys.path if path.startswith('/usr/local/lib/python')][1]);")
if [[ -d $python_path/orchestra ]]; then
run sudo rm -fr $python_path/orchestra
fi
orch_version=$(python3 -c "from orchestra import get_version; print(get_version());" 2> /dev/null)
orch_version=$(python3 -c "from orchestra import get_version; print(get_version());" 2> /dev/null || echo '')
if [[ ! $orch_version ]]; then
# First Orchestra installation
run sudo mkdir -p /usr/share/man/man1
run sudo mkdir -p /usr/share/man/man7
run sudo apt-get update
run sudo apt-get -y install git python3-pip
surun "git clone $repo $home/django-orchestra" || {
# Finishing partial installation
@ -65,13 +68,12 @@ function install_orchestra () {
if [[ -L /usr/local/bin/orchestra-admin || -f /usr/local/bin/orchestra-admin ]]; then
run sudo rm -f /usr/local/bin/{orchestra-admin,orchestra-beat}
fi
run sudo ln -s $home/django-orchestra/orchestra/bin/orchestra-admin /usr/local/bin/
run sudo ln -s $home/django-orchestra/orchestra/bin/orchestra-beat /usr/local/bin/
run sudo pip3 install -e $home/django-orchestra
run sudo orchestra-admin install_requirements --testing
else
# Install from pip
run sudo pip3 install http://git.io/django-orchestra-dev
run sudo orchestra-admin install_requirements
run sudo pip3 install -e git+https://github.com/ribaguifi/django-orchestra.git#egg=django-orchestra
fi
}
@ -79,12 +81,13 @@ function install_orchestra () {
function setup_database () {
dev=$1
noinput=$2
run sudo apt-get install -y postgresql python3-psycopg2
run sudo apt-get install -y postgresql
run sudo pip install psycopg2
# Setup Database
if [[ $dev ]]; then
# Speeding up tests, don't do this in production!
. /usr/share/postgresql-common/init.d-functions
pg_version=$(psql --version | head -n1 | sed -r "s/^.*\s([0-9]+\.[0-9]+).*/\1/")
pg_version=$(psql --version | head -n1 | awk '{print $3}' | sed -e "s,\..*,,")
sudo sed -i \
-e "s/^#fsync =\s*.*/fsync = off/" \
-e "s/^#full_page_writes =\s*.*/full_page_writes = off/" \
@ -170,7 +173,7 @@ function main () {
dev=
noinput=
user=$(whoami)
repo='https://github.com/glic3rinu/django-orchestra.git'
repo='https://github.com/ribaguifi/django-orchestra.git'
brepo=
project_name="panel"
bproject_name=

39
total_requirements.txt Normal file
View File

@ -0,0 +1,39 @@
Django==1.10.5
django-fluent-dashboard==0.6.1
django-admin-tools==0.8.0
django-extensions==1.7.4
django-celery==3.1.17
celery==3.1.23
kombu==3.0.35
billiard==3.3.0.23
Markdown==2.4
djangorestframework==3.4.7
ecdsa==0.11
Pygments==1.6
django-filter==0.15.2
jsonfield==0.9.22
python-dateutil==2.2
django-iban==0.3.0
requests
phonenumbers
django-countries
django-localflavor
amqp
anyjson
pytz
cracklib
lxml==3.3.5
selenium
xvfbwrapper
freezegun
coverage
flake8
django-debug-toolbar==1.3.0
django-nose==1.4.4
sqlparse
pyinotify
PyMySQL
dj_database_url==0.5.0
psycopg2-binary
python-decouple
https://github.com/glic3rinu/passlib/archive/master.zip