General Linux Setup¶
While Amazon Web Services (AWS) provides the preferred hosting for SEED, running on a bare-bones Linux server follows a similar setup, replacing the AWS services with their Linux package counterparts, namely: PostgreSQL and Redis.
SEED is a Django project and Django’s documentation is an excellent place to general understanding of this project’s layout.
Prerequisites¶
Ubuntu server 14.04 or newer
Install the following base packages to run SEED:
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install libpq-dev python-dev python-pip libatlas-base-dev \
gfortran build-essential g++ npm libxml2-dev libxslt1-dev git mercurial \
libssl-dev libffi-dev curl uwsgi-core uwsgi-plugin-python
sudo apt-get install redis-server
sudo apt-get install postgresql postgresql-contrib
Note
postgresql >=9.3
is required to support JSON Type
Configure PostgreSQL¶
$ sudo su - postgres
$ createdb "seed-deploy"
$ createuser -P DBUsername
$ psql
postgres=# GRANT ALL PRIVILEGES ON DATABASE "seed-deploy" TO DBUsername;
postgres=# \q
$ exit
Note
Any database name and username can be used here in place of “seed-deploy” and DBUsername
Python Dependencies¶
clone the seed repository from github
$ git clone git@github.com:SEED-platform/seed.git
enter the repo and install the python dependencies from requirements
$ cd seed
$ sudo pip install -r requirements/local.txt
JavaScript Dependencies¶
npm
is required to install the JS dependencies. The bin/install_javascript_dependencies.sh
script will
download all JavaScript dependencies and build them. bower
and gulp
should be installed globally for
convenience.
$ curl -sL https://deb.nodesource.com/setup_5.x | sudo -E bash -
$ sudo apt-get install -y nodejs
$ sudo npm install -g bower gulp
$ bin/install_javascript_dependencies.sh
Django Database Configuration¶
Copy the local_untracked.py.dist
file in the config/settings
directory to
config/settings/local_untracked.py
, and add a DATABASES
configuration with your database username, password,
host, and port. Your database configuration can point to an AWS RDS instance or a PostgreSQL 9.4 database instance
you have manually installed within your infrastructure.
# Database
DATABASES = {
'default': {
'ENGINE':'django.db.backends.postgresql_psycopg2',
'NAME': 'seed-deploy',
'USER': 'DBUsername',
'PASSWORD': '',
'HOST': 'localhost',
'PORT': '5432',
}
}
Note
Other databases could be used such as MySQL, but are not supported due to the postgres-specific JSON Type
In in the above database configuration, seed
is the database name, this is arbitrary and any valid name can be
used as long as the database exists. Enter the database name, user, password you set above.
The database settings can be tested using the Django management command, ./manage.py dbshell
to connect to the
configured database.
create the database tables and migrations:
$ python manage.py migrate
Cache and Message Broker¶
The SEED project relies on redis for both cache and message brokering, and
is available as an AWS ElastiCache service or with the redis-server
Linux package. (sudo apt-get install redis-server
)
local_untracked.py
should be updated with the CACHES
and CELERY_BROKER_URL
settings.
CACHES = {
'default': {
'BACKEND': 'redis_cache.cache.RedisCache',
'LOCATION': "127.0.0.1:6379",
'OPTIONS': {'DB': 1},
'TIMEOUT': 300
}
}
CELERY_BROKER_URL = 'redis://127.0.0.1:6379/1'
Creating the initial user¶
create a superuser to access the system
$ python manage.py create_default_user --username=demo@example.com --organization=example --password=demo123
Note
Every user must be tied to an organization, visit /app/#/profile/admin
as the superuser to create parent organizations and add users to them.
Running celery the background task worker¶
Celery is used for background tasks (saving data, matching, creating
projects, etc) and must be connected to the message broker queue. From the
project directory, celery
can be started:
celery -A seed worker -l INFO -c 2 -B --events --maxtasksperchild 1000
Running the development web server¶
The Django dev server (not for production use) can be a quick and easy way to get an instance up and running. The dev server runs by default on port 8000 and can be run on any port. See Django’s runserver documentation for more options.
$ python manage.py runserver --settings=config.settings.dev
Running a production web server¶
Our recommended web server is uwsgi sitting behind nginx. The python package uwsgi
is needed for this, and
should install to /usr/local/bin/uwsgi
Since AWS S3, is not being used here, we recommend using dj-static
to load static files.
Note
The use of the dev
settings file is production ready, and should be
used for non-AWS installs with DEBUG
set to False
for production use.
$ sudo pip install uwsgi dj-static
Generate static files:
$ sudo ./manage.py collectstatic --settings=config.settings.dev
Update config/settings/local_untracked.py
:
DEBUG = False
# static files
STATIC_ROOT = 'collected_static'
STATIC_URL = '/static/'
Start the web server:
$ sudo /usr/local/bin/uwsgi --http :80 --module standalone_uwsgi --max-requests 5000 --pidfile /tmp/uwsgi.pid --single-interpreter --enable-threads --cheaper-initial 1 -p 4
Warning
Note that uwsgi has port set to 80
. In a production setting, a dedicated web server such as NGINX would be
receiving requests on port 80 and passing requests to uwsgi running on a different port, e.g 8000.
Environmental Variables¶
The following environment variables can be set within the ~/.bashrc
file to
override default Django settings.
export SENTRY_DSN=https://xyz@app.getsentry.com/123
export DEBUG=False
export ONLY_HTTPS=True
SMTP service¶
In the AWS setup, we can use SES to provide an email service for Django. The service is configured in the config/settings/main.py:
EMAIL_BACKEND = 'django_ses.SESBackend'
Many options for setting up your own SMTP service/server or using other SMTP third party services are available and compatible including gmail.
Django can likewise send emails via python’s smtplib with sendmail or postfix installed. See their docs for more info.
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
local_untracked.py¶
# PostgreSQL DB config
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'seed',
'USER': 'your-username',
'PASSWORD': 'your-password',
'HOST': 'your-host',
'PORT': 'your-port',
}
}
# config for local storage backend
DOMAIN_URLCONFS = {}
DOMAIN_URLCONFS['default'] = 'urls.main'
CACHES = {
'default': {
'BACKEND': 'redis_cache.cache.RedisCache',
'LOCATION': "127.0.0.1:6379",
'OPTIONS': {'DB': 1},
'TIMEOUT': 300
}
}
CELERY_BROKER_URL = 'redis://127.0.0.1:6379/1'
# SMTP config
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
# static files
STATIC_ROOT = 'collected_static'
STATIC_URL = '/static/'