Aaron

Merge branch 'develop' into 'master'

New things;



See merge request !11
This application operates under the assumption of the following directory structure:
Building a new project
======================
-- work_dir
|-- html (this directory, named whatever you want)
|-- Serve.py (run this to start the CherryPy web application
|-- src (source for the CherryPy framework additions)
|-- README.md (this file)
|-- site.config (config file for application, includes [apps] section)
|-- MyModule (code that does the work)
Remove app/ directory and replace with application module using 'git submodule'
As a convention, MyModule contains an 'apps' directory/submodule which defines an
application interface. For example, if MyModule is structured like this:
```
git submodule add git@gitlab.onnix.io:internal/prioritize.git prioritize
```
-- MyModule
|-- apps
|-- cherrypy
|-- root.py
|-- __init__.py
|-- cli
|-- cliapp.py
|-- __init__.py
Remove html/ directory and replace with HTTP framework using 'git submodule'
Then I can do this:
```
git submodule add git@gitlab.onnix.io:code/cherryex.git html
```
from MyModule.apps.cherrypy import Root
Remove public/ directory and replace with coded templates and assets using 'git submodule'
because MyModule/apps/cherrypy/__init__.py contains the following line:
```
git@gitlab.onnix.io:templates/onnix-gentelella.git public
```
from .root import Root
If you want to use any of the existing directory structure, you will run into a problem after deleting:
... Which makes imports look nice.
```
$ rm -rf html/
$ git submodule add git@gitlab.onnix.io:code/cherryex.git html
'html' already exists in the index
```
The takeaway of all this is Serve.py and site.config. In site.config we create a
section that looks like this:
This can be resolved by removing the directory from cache:
[apps]
Root = MyModule.apps.cherrypy
```
$ git rm -r --cached html
rm 'html/README.md'
```
And Serve.py iterates the 'apps' config dict and essentially does a dynamic import
that is equivalent to:
Once you have resolved all the GIT nuances to get your project in a state ready for development, make sure you change the project origin to point to your new Git repository location:
from MyModule.apps.cherrypy import Root
cherrypy.tree.mount(Root(), config='site.config')
```
git remote add origin git@gitlab.onnix.io:internal/test-project.git
```
Finally, the benefit to all of this is that you can write your CherryPy application
completely isolated from the CherryPy framework, which exists in its own repository,
while keeping your module decoupled from the CherryPy framework and in ITS own repo.
Finally, you can push the project, complete with submodule references
Super convenient.
```
$ git commit
[project_import 70bb83e] Imported project code;
8 files changed, 12 insertions(+), 12 deletions(-)
create mode 100644 .gitmodules
delete mode 100644 app/README.md
create mode 160000 html
delete mode 100644 html/README.md
create mode 160000 prioritize
create mode 160000 public
delete mode 100644 public/assets/README.md
delete mode 100644 public/templates/README.md
$ git push -u origin project_import;
Enter passphrase for key '/home/harry/.ssh/id_rsa':
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 606 bytes | 303.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To gitlab.onnix.io:internal/web-app-template.git
* [new branch] project_import -> project_import
Branch 'project_import' set up to track remote branch 'project_import' from 'origin'.
$ git checkout -b project_import
A .gitmodules
D app/README.md
A html
D html/README.md
A prioritize
A public
D public/assets/README.md
D public/templates/README.md
Switched to a new branch 'project_import'
```
Cloning a Project
=================
When you clone this project, none of the submodules will be cloned.
```
git submodule init
git submodule update
```
... ...
This install required the Postgres database be created with UTF8 as its encoding.
The default encoding for template1 is ASCII, so we must perform the following as
the postgres user (/usr/local/pgsql/bin/psql)
Create UTF8 template:
update pg_database set datallowconn=TRUE where datname='template0';
\c template0
update pg_database set datistemplate=FALSE where datname='template1';
drop database template1;
create database template1 with template=template0 encoding='UTF8';
update pg_database set datistemplate=TRUE where datname='template1';
\c template1
update pg_database set datallowconn=FALSE where datname='template0';
Now you can create the database. Here we'll use the createdb binary:
/usr/local/pgsql/bin/createdb --encoding=UTF8 --owner=tccdev tccdev
And now reset template1 back to its defaults:
update pg_database set datistemplate=FALSE where datname='template1';
drop database template1;
create database template1 with template=template0
update pg_database set datistemplate=TRUE where datname='template1';
... ...
... ... @@ -14,7 +14,7 @@ if __name__ == '__main__':
# Import web framework modules
from src.plugins import SQLAlchemyPlugin
from src.tools import Jinja2Tool, SQLAlchemyTool, DebugTool, ResponseHeaders
from src.tools import AuthValidate, SQLAlchemyTool, DebugTool, ResponseHeaders
# Load config before CherryPy
p = Parser()
... ... @@ -43,10 +43,10 @@ if __name__ == '__main__':
pw = conf['db']['pass']).subscribe()
# Load Tools
cherrypy.tools.render = Jinja2Tool(conf['path']['templates'])
cherrypy.tools.db = SQLAlchemyTool()
cherrypy.tools.debug = DebugTool()
cherrypy.tools.db = SQLAlchemyTool()
cherrypy.tools.response_headers = ResponseHeaders(conf['response-headers'])
cherrypy.tools.authreq = AuthValidate(conf['jwt'])
# Import application modules via config file
for classname, pair in conf['apps'].items():
... ...
# A generic, single database configuration.
[alembic]
# path to migration scripts
script_location = migrate
# template used to generate migration files
# file_template = %%(rev)s_%%(slug)s
# timezone to use when rendering the date
# within the migration file as well as the filename.
# string value is passed to dateutil.tz.gettz()
# leave blank for localtime
# timezone =
# max length of characters to apply to the
# "slug" field
# truncate_slug_length = 40
# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false
# set to 'true' to allow .pyc and .pyo files without
# a source .py file to be detected as revisions in the
# versions/ directory
# sourceless = false
# version location specification; this defaults
# to migrate/versions. When using multiple version
# directories, initial revisions must be specified with --version-path
# version_locations = %(here)s/bar %(here)s/bat migrate/versions
# the output encoding used when revision files
# are written from script.py.mako
# output_encoding = utf-8
sqlalchemy.url = driver://user:pass@localhost/dbname
[post_write_hooks]
# post_write_hooks defines scripts or Python functions that are run
# on newly generated revision scripts. See the documentation for further
# detail and examples
# format using "black" - use the console_scripts runner, against the "black" entrypoint
# hooks=black
# black.type=console_scripts
# black.entrypoint=black
# black.options=-l 79
# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
qualname =
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S
... ...
To generate new migration:
cd .. && alembic -c ../../alembic.ini revision -m "Comment..."
To execute migration:
alembic -c ../../alembic.ini upgrade head
... ...
from logging.config import fileConfig
from sqlalchemy import engine_from_config
from sqlalchemy import pool
from alembic import context
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config
# Interpret the config file for Python logging.
# This line sets up loggers basically.
fileConfig(config.config_file_name)
# add your model's MetaData object here
# for 'autogenerate' support
# from myapp import mymodel
# target_metadata = mymodel.Base.metadata
target_metadata = None
# other values from the config, defined by the needs of env.py,
# can be acquired:
# my_important_option = config.get_main_option("my_important_option")
# ... etc.
def run_migrations_offline():
"""Run migrations in 'offline' mode.
This configures the context with just a URL
and not an Engine, though an Engine is acceptable
here as well. By skipping the Engine creation
we don't even need a DBAPI to be available.
Calls to context.execute() here emit the given string to the
script output.
"""
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online():
"""Run migrations in 'online' mode.
In this scenario we need to create an Engine
and associate a connection with the context.
"""
connectable = engine_from_config(
config.get_section(config.config_ini_section),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
context.configure(
connection=connection, target_metadata=target_metadata
)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
... ...
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
# revision identifiers, used by Alembic.
revision = ${repr(up_revision)}
down_revision = ${repr(down_revision)}
branch_labels = ${repr(branch_labels)}
depends_on = ${repr(depends_on)}
def upgrade():
${upgrades if upgrades else "pass"}
def downgrade():
${downgrades if downgrades else "pass"}
... ...
[global]
server.socket_host: '127.0.0.6'
server.socket_port: 9009
server.socket_host: '0.0.0.0'
server.socket_port: 9001
server.wsgi: False
log.screen: False
[/]
tools.trailing_slash.on: False
tools.render.on: False
tools.db.on: True
tools.debug.on: False
tools.sessions.on: True
tools.sessions.storage_class: cherrypy.lib.sessions.FileSession
tools.sessions.storage_path: '/var/www/prioritize.onnix.io/html/sessions'
tools.sessions.timeout = 60
tools.sessions.name = 'prioritize_sid'
tools.response_headers.on: True
request.dispatch: cherrypy.dispatch.MethodDispatcher()
[logs]
access: '/var/www/prioritize.onnix.io/html/logs/access.log'
error: '/var/www/prioritize.onnix.io/html/logs/error.log'
access: '/var/www/html/logs/access.log'
error: '/var/www/html/logs/error.log'
level: 'DEBUG'
[apps]
Root: ('app', '')
List: ('app', '/list')
Item: ('app', '/item')
Root: ('app', '/api/v1')
Auth: ('app', '/api/v1/auth')
Priorities: ('app', '/api/v1/priorities')
#Items: ('app', '/items')
[path]
templates: '/var/www/prioritize.onnix.io/public/templates'
libraries: '/var/www/prioritize.onnix.io/lib'
libraries: '/var/www/html/guss-api/lib'
[db]
host: 'DB_HOST'
port: 1234
name: 'DB_NAME'
user: 'DB_USER'
pass: 'DB_PORT'
host: '12.34.45.67'
port: 5432
name: 'dbname'
user: 'dbuser'
pass: 'dbpass'
[response-headers]
Access-Control-Allow-Origin: 'https://example.com'
Access-Control-Allow-Origin: '*'
Access-Control-Allow-Headers: 'Content-Type, Access-Control-Allow-Headers, Authorization, X-Requested-With'
Access-Control-Allow-Methods: 'GET,POST,PUT,DELETE,OPTIONS'
Server: None
[jwt]
private_key: 'jwt.pem'
... ...
from .debugtool import DebugTool
from .jinja2tool import Jinja2Tool
from .authvalidate import AuthValidate
from .sqlalchemytool import SQLAlchemyTool
from .responseheaders import ResponseHeaders
... ...
# -*- coding: utf-8 -*-
"""
Validates JSON Web Tokens
"""
import json
import cherrypy
from jwcrypto import jwt, jwk
from jwcrypto.jwt import JWTExpired
from lib.authenticate.user.controllers import UserAuth
from lib.authenticate.exceptions import AuthorizationError, UserNotFound
class AuthValidate(cherrypy.Tool):
def __init__(self, jwt_config):
self.token = None
self.cert_path = jwt_config['private_key']
cherrypy.Tool.__init__(self, 'on_start_resource', self.token_validate, priority=5)
def token_validate(self):
cherrypy.log('AUTH_REQUIRED')
if 'Authorization' not in cherrypy.request.headers:
self.auth_data = None
return
header = cherrypy.request.headers['Authorization']
if header[0:7] != 'Bearer ':
raise AuthorizationError('Malformed bearer token. Header value must start with "Bearer ..."')
raw_token = header[7:]
with open(self.cert_path, 'rb') as f:
key = jwk.JWK.from_pem(f.read())
try:
self.token = jwt.JWT(jwt=raw_token, key=key)
except JWTExpired as e:
self.token = None
if self.token is None:
raise cherrypy.HTTPError(401)
else:
request = cherrypy.serving.request
request.jwt = json.loads(self.token.claims)
... ...
... ... @@ -18,4 +18,7 @@ class ResponseHeaders(cherrypy.Tool):
def append_headers(self):
for header, value in self.headers.items():
cherrypy.response.headers[header] = value
if not value:
del cherrypy.response.headers[header]
else:
cherrypy.response.headers[header] = value
... ...