Decorative image frame

Hexo & Theme Usage

Hexo, documentation, troubleshooting, GitHub Issues.

Hexo

Install & Setup

1
$ npm install -g hexo-cli

Initialize

1
2
3
$ hexo init <folder>
$ cd <folder>
$ npm install

File Structure:

1
2
3
4
5
6
7
8
9
10
|-- _config.yml # configuration file for hexo
|-- source
|-- _posts
|-- one_blog.md # blog file
|-- _drafts
|-- one_draft.md # draft file
|-- themes
|-- landscape # default theme
|-- ocean
|-- _config.yml # configuration file for ocean

Create a new post

1
$ hexo new "My New Post" / $ hexo n "My New Post"

More info: Writing

Run server

1
$ hexo server / $ hexo s

More info: Server

Generate static files

1
$ hexo generate / $ hexo g

More info: Generating

Deploy to remote sites

1
$ hexo deploy / $ hexo d

More info: Deployment

Deploy to github

1
$ yarn add hexo-deployer-git

Then, go to hexo’s _config.yml, add this to deploy session:

1
2
3
4
deploy:
type: git
repo: https://github.com/xyshell/xyshell.github.io.git
branch: master

Theme

Install & Setup & Update

1
$ git clone https://github.com/zhwangart/hexo-theme-ocean.git themes/ocean

Modify theme setting in hexo’s _config.yml to ocean:

1
theme: ocean

Further update:

1
2
cd themes/ocean
git pull

Theme Reference

Ocean

https://github.com/zhwangart/hexo-theme-ocean

Minos

https://github.com/ppoffice/hexo-theme-minos

cka-note

Installation and Configuration

Intall kubectl

  • configuration file location: $HOME/.kube/config.

  • handy when going from a local environment to a cluster in the cloud, or from one cluster to another, such as from development to production.

1
$ kubectl config use-context foobar

GKE Quickstart

  1. GKE Install

  2. GKE Quickstart

  3. quick start command

1
2
3
4
5
6
7
$ gcloud container clusters create linuxfoundation

$ gcloud container clusters list

$ kubectl get nodes # kubectl comes free from gcloud

$ gcloud container clusters delete linuxfoundation

Minikube

  1. github repo

  2. install

1
2
3
4
5
$ curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-darwin-amd64

$ chmod +x minikube

$ sudo mv minikube /usr/local/bin
  1. quick start command
1
2
3
$ minikube start

$ kubectl get nodes

Main Deployment configuration

  1. Single-node

With a single-node deployment, all the components run on the same server. This is great for testing, learning, and developing around Kubernetes.

  1. Single head node, multiple workers

Adding more workers, a single head node and multiple workers typically will consist of a single node etcd instance running on the head node with the API, the scheduler, and the controller-manager.

  1. Multiple head nodes with HA, multiple workers

Multiple head nodes in an HA configuration and multiple workers add more durability to the cluster. The API server will be fronted by a load balancer, the scheduler and the controller-manager will elect a leader (which is configured via flags). The etcd setup can still be single node.

  1. HA etcd, HA head nodes, multiple workers

The most advanced and resilient setup would be an HA etcd cluster, with HA head nodes and multiple workers. Also, etcd would run as a true cluster, which would provide HA and would run on nodes separate from the Kubernetes head nodes.

CSS Note

CSS selectors

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
h1,
h2 {
...;
} /* html tag selector, h1-h6 */

#id {
...;
} /* id selector */

.class {
...;
} /* class selector */

.class img {
...;
} /* descendant combinator */

.class:hover {
...;
} /* user action pseudo class */

.class:nth-of-type(odd) {
...;
} /* location pseudo class
nth-of-type: number or odd/even
first-of-type/last-of-type
nth-child
first-child/last-child
*/

#leftmenu > ul > li > a {
...;
} /* child combinator */

.sidebar + h2 {
...;
} /* adjacent sibling combinator */

.sidebar ~ h2 {
...;
} /* general sibling combinator */

CSS property reference

text style

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
color: white; /* default black */
font-size: 18px; /* default 16px */
font-weight: bold; /* default normal */
font-style: italic; /* default normal */

/* whitespace between lines of test */
line-height: 1.5; /* default 1, relative to font-size */
/* space between characters */
letter-spacing: 2px; /* default 0 */

/* visit cssfontstack.com to find more web-safe fonts. */
font-family: Verdana, Arial, sans-serif, cursive, fantasy; /* web-safe fonts, default Arial */
/* serif: Georgia, Times New Roman
sans-serif: Arial, Verdana, Comic Sans, Trebuchet
monospace: Courier New */

text-align: center; /* justify, default left */
text-transform: capitalize; /* uppercase, default none */

container style

The Box Model: element < padding < border < margin

Use chrome dev tools to inspect the box model.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
width: 100%;
/* relative to parent width, default auto
width: 1em; relative to parent font-size
width: 1rem; relative to html font-size */
backgroud-image: url(../images/image.png);
background-repeat: repeat; /* repeat, repeat-x, repeat-y, no-repeat */
background-color: #23cea6; /* RGB, default transparent */
border: 10px solid #a693c2;
border-color: #a693c2;
/* border inside, width, style, color(default text color)
style: solid, dashed, dotted, ridge, double, groove, inset, outset
border-left: xxx; left only */
outline: 10px solid red; /* border outside */

padding: 10px 20px 30px 40px;
/* space between container and content
top right bottom left
top/bottom left/right
top/right/bottom/left */
padding-left: 10px;

margin: 10px 20px 30px 40px;
/* space between container and next element
margin collapse, max(margin-bottom, margin-top) applied */
margin-bottom: 10px;

CSS templates

normalize.css

normalize.css v8.0.1 is a collection of CSS resets and normalizations.

Mobile first design

small.css

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
html {
-webkit-text-size-adjust: 100%;
-ms-text-size-adjust: 100%;
}

* {
-moz-box-sizing: border-box;
-webkit-box-sizing: border-box;
box-sizing: border-box;
}

body {
font-family: "Lato", sans-serif;
font-size: 16px;
}

medium.css

1
2
3
@media only screen and (min-width: 768px) {
...;
}

large.css

1
2
3
@media only screen and (min-width: 1024px) {
...;
}

IDE Note

IDE features

Multi-cursor editing

Using multiple cursors allows you to edit multiple parts of the document at once, greatly improving your productivity.

Vscode

  • Box Selection - ⇧↓, ⇧→, ⇧↑, ⇧← to select a block of text. ⇧⌥ while selecting text with the mouse or drag-select using the middle mouse button.
  • Add a cursor - ⌥⌘↑ to add a new cursor above, or ⌥⌘↓ to add a new cursor below. You can also use your mouse with ⌥+Click to add a cursor anywhere.
  • Create cursors on all occurrences of a string - select one instance of a string e.g. background-color and press ⇧⌘L. Now you can replace all instances by simply typing.
1
2
3
4
5
6
7
8
9
#p1 {
background-color: #ff0000;
} /* red in HEX format */
#p2 {
background-color: hsl(120, 100%, 50%);
} /* green in HSL format */
#p3 {
background-color: rgba(0, 4, 255, 0.733);
} /* blue with alpha channel in RGBA format */

IntelliSense

Vscode

  • invoke IntelliSense: ⌃Space or ⌥Esc
1
2
3
4
5
const canvas = document.querySelector('canvas');
const context = canvas.getContext('2d');

context.strokeStyle = 'blue';
context.

Line Actions

Vscode

  • Copy a line and insert it above or below: ⇧⌥↓ or ⇧⌥↑
  • Move an entire line or selection of lines up or down: ⌥↑ and ⌥↓
  • Delete the entire line: ⇧⌘K.
  • Comment out a line: ⌘/
1
2
3
4
5
{
"name": "John",
"age": 31,
"city": "New York"
}

Refactoring

Vscode

  • Rename a function/variable: F2 or right-click on the function/variable name.

Formatting

Vscode

  • Format a document: ⇧⌘F
  • Format a selection: ⌘K ⌘F

ps: enable editor.formatOnSave

Code Folding

Vscode

  • fold and unfold a block of code: ⌥⌘[ and ⌥⌘]
  • fold and unfold all blocks: ⌘K ⌘0 and ⌘K ⌘J
  • fold and unfold a number of levels: ⌘K ⌘1 to ⌘K ⌘5

Errors and Warnings

Vscode

  • navigate to the next error: F8

Code Snipets

Vscode

  • create a code snippet: Code > Preferences > User Snippets

CPA Note

CPA – C++ Certified Associate Programmer

Readings

More books: https://stackoverflow.com/questions/388242/the-definitive-c-book-guide-and-list

Reference

cout manipulator

1
2
3
4
int byte = 255;
cout << "Byte in hex: " << hex << byte;
cout << "Byte in decimal: " << dec << byte;
cout << "Byte in octal: " << oct << byte;
1
2
3
4
5
6
7
8
9
10
#include <iostream>
#include <iomanip>

using namespace std;
int main(void)
{
int byte = 255;
cout << setbase(16) << byte;
return 0;
}
1
2
3
float x = 2.5, y = 0.0000000025;
cout << fixed << x << " " << y << endl;
cout << scientific << x << " " << y << endl;

String methods

compare

str1.compare(str2)

  • str1.compare(str2) == 0 when str1 == str2

  • str1.compare(str2) > 0 when str1 > str2

  • str1.compare(str2) < 0 when str1 < str2

S.compare(substr_start, substr_length, other_string)

S.compare(substr_start, substr_length, other_string, other_substr_start, other_substr_length)

substr

1
string newstr = oldstr.substr(substring_start_position, length_of_substring)

length, size, capacity, max_size

1
2
3
4
5
6
7
int string_size = S.size();

int string_length = S.length();

int string_capacity = s.capacity();

int string_max_size = s.max_size();
1
TheString.reserve(100);

reserve, resize, clear, empty

1
2
3
4
5
6
7
bool is_empty = TheString.empty();

TheString.resize(50,'?');

TheString.resize(4);

TheString.clear();

find

1
2
3
int where_it_begins = S.find(another_string, start_here);

int where_it_is = S.find(any_character, start_here);
1
2
3
4
int comma = greeting.find(',');
if(comma != string::npos){
//found
};

append, push_back, insert

1
2
3
NewString.append(TheString);
NewString.append(TheString,0,3);
NewString.append(2,'!');
1
TheString.push_back(car);
1
2
string quote = "Whyserious?", anyword = "monsoon";
quote.insert(3,2,' ').insert(4,anyword,3,2); // Why so serious?

assign

1
2
string sky; 
sky.assign(80,'*');

replace

1
2
3
4
string ToDo = "I'll think about that in one hour"; 
string Schedule = "today yesterday tomorrow";

ToDo.replace(22, 12, Schedule, 16, 8); // I'll think about that tomorrow

erase

1
2
3
4
string WhereAreWe = "I've got a feeling we're not in Kansas anymore"; 

WhereAreWe.erase(38, 8).erase(25, 4); // I've got a feeling we're in Kansas
TheString.erase();

swap

1
2
3
4
string Drink = "A martini";
string Needs = "Shaken, not stirred";

Drink.swap(Needs);

Sql Note

This note follows Essential SQLAlchemy 2nd Edition

Github repo: https://github.com/oreillymedia/essential-sqlalchemy-2e/

SQLAlchemy Core

Create Schema

Relationship-visualization

Full in-memory SQLite code sample

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
from datetime import datetime
from sqlalchemy import (MetaData, Table, Column, Integer, Numeric, String, DateTime, ForeignKey, create_engine)

metadata = MetaData()
cookies = Table('cookies', metadata,
Column('cookie_id', Integer(), primary_key=True),
Column('cookie_name', String(50), index=True),
Column('cookie_recipe_url', String(255)),
Column('cookie_sku', String(55)),
Column('quantity', Integer()),
Column('unit_cost', Numeric(12, 2))
)

users = Table('users', metadata,
Column('user_id', Integer(), primary_key=True),
Column('customer_number', Integer(), autoincrement=True),
Column('username', String(15), nullable=False, unique=True),
Column('email_address', String(255), nullable=False),
Column('phone', String(20), nullable=False),
Column('password', String(25), nullable=False),
Column('created_on', DateTime(), default=datetime.now),
Column('updated_on', DateTime(), default=datetime.now, onupdate=datetime.now)
)

orders = Table('orders', metadata,
Column('order_id', Integer(), primary_key=True),
Column('user_id', ForeignKey('users.user_id'))
)

line_items = Table('line_items', metadata,
Column('line_items_id', Integer(), primary_key=True),
Column('order_id', ForeignKey('orders.order_id')),
Column('cookie_id', ForeignKey('cookies.cookie_id')),
Column('quantity', Integer()),
Column('extended_cost', Numeric(12, 2))
)

engine = create_engine('sqlite:///:memory:')
metadata.create_all(engine)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
from datetime import datetime

from sqlalchemy import (MetaData, Table, Column, Integer, Numeric, String, DateTime, ForeignKey, Boolean, create_engine, CheckConstraint)
metadata = MetaData()

cookies = Table('cookies', metadata,
Column('cookie_id', Integer(), primary_key=True),
Column('cookie_name', String(50), index=True),
Column('cookie_recipe_url', String(255)),
Column('cookie_sku', String(55)),
Column('quantity', Integer()),
Column('unit_cost', Numeric(12, 2)),
CheckConstraint('quantity >= 0', name='quantity_positive')
)

users = Table('users', metadata,
Column('user_id', Integer(), primary_key=True),
Column('username', String(15), nullable=False, unique=True),
Column('email_address', String(255), nullable=False),
Column('phone', String(20), nullable=False),
Column('password', String(25), nullable=False),
Column('created_on', DateTime(), default=datetime.now),
Column('updated_on', DateTime(), default=datetime.now, onupdate=datetime.now)
)

orders = Table('orders', metadata,
Column('order_id', Integer()),
Column('user_id', ForeignKey('users.user_id')),
Column('shipped', Boolean(), default=False)
)

line_items = Table('line_items', metadata,
Column('line_items_id', Integer(), primary_key=True),
Column('order_id', ForeignKey('orders.order_id')),
Column('cookie_id', ForeignKey('cookies.cookie_id')),
Column('quantity', Integer()),
Column('extended_cost', Numeric(12, 2))
)

engine = create_engine('sqlite:///:memory:')
metadata.create_all(engine)
connection = engine.connect()

Data Operation

Insert

Way1:

1
2
3
4
5
6
7
8
9
10
ins = cookies.insert().values(
cookie_name="chocolate chip",
cookie_recipe_url="http://some.aweso.me/cookie/recipe.html",
cookie_sku="CC01",
quantity="12",
unit_cost="0.50"
)
# str(ins)
result = connection.execute(ins)
# result.inserted_primary_key

Way2:

1
2
3
4
5
6
7
8
from sqlalchemy import insert
ins = insert(cookies).values(
cookie_name="chocolate chip",
cookie_recipe_url="http://some.aweso.me/cookie/recipe.html",
cookie_sku="CC01",
quantity="12",
unit_cost="0.50"
)

Way3:

1
2
3
4
5
6
7
8
ins = cookies.insert()
result = connection.execute(
ins,
cookie_name='dark chocolate chip',
cookie_recipe_url='http://some.aweso.me/cookie/recipe_dark.html',
cookie_sku='CC02',
quantity='1',
unit_cost='0.75')

Multiple:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
inventory_list = [
{
'cookie_name': 'peanut butter',
'cookie_recipe_url': 'http://some.aweso.me/cookie/peanut.html',
'cookie_sku': 'PB01',
'quantity': '24',
'unit_cost': '0.25'
},
{
'cookie_name': 'oatmeal raisin',
'cookie_recipe_url': 'http://some.okay.me/cookie/raisin.html',
'cookie_sku': 'EWW01',
'quantity': '100',
'unit_cost': '1.00'
}
]
result = connection.execute(ins, inventory_list)

Select

Way1:

1
2
3
4
5
from sqlalchemy.sql import select
s = select([cookies])
# str(s)
rp = connection.execute(s)
results = rp.fetchall()

Way2:

1
2
3
4
s = cookies.select()
rp = connection.execute(s)
for record in rp:
print(record.cookie_name)

Sepcify columns, order by, limit, cast:

1
2
3
4
5
6
7
8
9
10
11
from sqlalchemy import desc
from sqlalchemy import cast

s = select([cookies.c.cookie_name, cookies.c.quantity, 'SKU-' + cookies.c.cookie_sku,
cast((cookies.c.quantity * cookies.c.unit_cost), Numeric(12,2)).label('inv_cost')]])
s = s.order_by(desc(cookies.c.quantity), cookies.c.cookie_name)
s = s.limit(2) # Also by, s = cookies.select(limit=1)
rp = connection.execute(s)
# rp.keys()
for cookie in rp:
print('{} - {}'.format(cookie.quantity, cookie.cookie_name))

Aggregate:

1
2
3
4
5
6
7
from sqlalchemy.sql import func

s = select([func.count(cookies.c.cookie_name).label('inventory_count')])
rp = connection.execute(s)
record = rp.first()
# record.keys()
# record.inventory_count

Filter(Where):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
s = select([cookies]).where(cookies.c.cookie_name == 'chocolate chip')
rp = connection.execute(s)
record = rp.first()
# record.items()

s = select([cookies]).where(cookies.c.cookie_name.like('%chocolate%')).where(cookies.c.quantity == 12)
# str(s)
rp = connection.execute(s)
for record in rp.fetchall():
print(record.cookie_name)

from sqlalchemy import and_, or_, not_
s = select([cookies]).where(or_(
cookies.c.quantity.between(10, 50),
cookies.c.cookie_name.contains('chip')
))
for row in connection.execute(s):
print(row.cookie_name)

Update

1
2
3
4
5
from sqlalchemy import update
u = update(cookies).where(cookies.c.cookie_name == "chocolate chip")
u = u.values(quantity=(cookies.c.quantity + 120))
result = connection.execute(u)
print(result.rowcount)

Delete

1
2
3
4
from sqlalchemy import delete
u = delete(cookies).where(cookies.c.cookie_name == "dark chocolate chip")
result = connection.execute(u)
print(result.rowcount)

Join

1
2
3
4
5
6
columns = [orders.c.order_id, users.c.username, users.c.phone, cookies.c.cookie_name, line_items.c.quantity, line_items.c.extended_cost]
cookiemon_orders = select(columns)
cookiemon_orders = cookiemon_orders.select_from(users.join(orders).join(line_items).join(cookies)).where(users.c.username == 'cookiemon')
result = connection.execute(cookiemon_orders).fetchall()
for row in result:
print(row)

outerjoin:

1
2
3
4
5
6
columns = [users.c.username, orders.c.order_id]
all_orders = select(columns)
all_orders = all_orders.select_from(users.outerjoin(orders))
result = connection.execute(all_orders).fetchall()
for row in result:
print(row)

Alias

1
2
3
manager = employee_table.alias()
stmt = select([employee_table.c.name], and_(employee_table.c.manager_id==manager.c.id, manager.c.name=='Fred'))
print(stmt)

Groupby

1
2
3
4
5
6
7
columns = [users.c.username, func.count(orders.c.order_id)]
all_orders = select(columns)
all_orders = all_orders.select_from(users.outerjoin(orders)).group_by(users.c.username)
print(str(all_orders))
result = connection.execute(all_orders).fetchall()
for row in result:
print(row)

Raw Queries

1
result = connection.execute("select * from orders").fetchall()
1
2
3
from sqlalchemy import text
stmt = select([users]).where(text('username="cookiemon"'))
print(connection.execute(stmt).fetchall())

Exception and Transaction

Transaction Example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
# Use second schema

from sqlalchemy import select, insert, update
ins = insert(users).values(
username="cookiemon",
email_address="mon@cookie.com",
phone="111-111-1111",
password="password"
)
result = connection.execute(ins)

ins = cookies.insert()
inventory_list = [
{
'cookie_name': 'chocolate chip',
'cookie_recipe_url': 'http://some.aweso.me/cookie/recipe.html',
'cookie_sku': 'CC01',
'quantity': '12',
'unit_cost': '0.50'
},
{
'cookie_name': 'dark chocolate chip',
'cookie_recipe_url': 'http://some.aweso.me/cookie/recipe_dark.html',
'cookie_sku': 'CC02',
'quantity': '1',
'unit_cost': '0.75'
}
]
result = connection.execute(ins, inventory_list)

ins = insert(orders).values(user_id=1, order_id=1)
result = connection.execute(ins)
ins = insert(line_items)
order_items = [
{
'order_id': 1,
'cookie_id': 1,
'quantity': 9,
'extended_cost': 4.50
}
]
result = connection.execute(ins, order_items)


ins = insert(orders).values(user_id=1, order_id=2)
result = connection.execute(ins)
ins = insert(line_items)
order_items = [
{
'order_id': 2,
'cookie_id': 2,
'quantity': 1,
'extended_cost': 1.50
},
{
'order_id': 2,
'cookie_id': 1,
'quantity': 4,
'extended_cost': 4.50
}
]
result = connection.execute(ins, order_items)

def ship_it(order_id):

s = select([line_items.c.cookie_id, line_items.c.quantity])
s = s.where(line_items.c.order_id == order_id)
cookies_to_ship = connection.execute(s)
for cookie in cookies_to_ship:
u = update(cookies).where(cookies.c.cookie_id == cookie.cookie_id)
u = u.values(quantity = cookies.c.quantity - cookie.quantity)
result = connection.execute(u)
u = update(orders).where(orders.c.order_id == order_id)
u = u.values(shipped=True)
result = connection.execute(u)
print("Shipped order ID: {}".format(order_id))

ship_it(1)

s = select([cookies.c.cookie_name, cookies.c.quantity])
connection.execute(s).fetchall()

ship_it(2)

u = update(cookies).where(cookies.c.cookie_name == "dark chocolate chip")
u = u.values(quantity = 1)
result = connection.execute(u)

from sqlalchemy.exc import IntegrityError
def ship_it(order_id):
s = select([line_items.c.cookie_id, line_items.c.quantity])
s = s.where(line_items.c.order_id == order_id)
transaction = connection.begin()
cookies_to_ship = connection.execute(s).fetchall()
try:
for cookie in cookies_to_ship:
u = update(cookies).where(cookies.c.cookie_id == cookie.cookie_id)
u = u.values(quantity = cookies.c.quantity-cookie.quantity)
result = connection.execute(u)
u = update(orders).where(orders.c.order_id == order_id)
u = u.values(shipped=True)
result = connection.execute(u)
print("Shipped order ID: {}".format(order_id))
transaction.commit()
except IntegrityError as error:
transaction.rollback()
print(error)
ship_it(2)

s = select([cookies.c.cookie_name, cookies.c.quantity])
connection.execute(s).fetchall()

Reflection

Reflecting Individual Tables

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
from sqlalchemy import MetaData, Table, create_engine
metadata = MetaData()
engine = create_engine('sqlite:///Chinook_Sqlite.sqlite')


artist = Table('artist', metadata, autoload=True, autoload_with=engine)
album = Table('album', metadata, autoload=True, autoload_with=engine)

artist.columns.keys()
from sqlalchemy import select
s = select([artist]).limit(10)
engine.execute(s).fetchall()

album.foreign_keys
from sqlalchemy import ForeignKeyConstraint
album.append_constraint(
ForeignKeyConstraint(['ArtistId'], ['artist.ArtistId'])
)
metadata.tables['album']
str(artist.join(album))

Reflecting a Whole Database

1
2
3
4
5
6
7
8
metadata.reflect(bind=engine)

metadata.tables.keys()

playlist = metadata.tables['Playlist']
from sqlalchemy import select
s = select([playlist]).limit(10)
engine.execute(s).fetchall()

SQLAlchemy ORM

Defining Tables via ORM Classes

sql_orm_setup

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
from datetime import datetime

from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Table, Column, Integer, Numeric, String, Boolean, DateTime, ForeignKey
from sqlalchemy import create_engine
from sqlalchemy.orm import relationship, backref

Base = declarative_base()


class Cookie(Base):
__tablename__ = 'cookies'

cookie_id = Column(Integer(), primary_key=True, )
cookie_name = Column(String(50), index=True)
cookie_recipe_url = Column(String(255))
cookie_sku = Column(String(55))
quantity = Column(Integer())
unit_cost = Column(Numeric(12, 2))

def __repr__(self):
return "Cookie(cookie_name='{self.cookie_name}', " \
"cookie_recipe_url='{self.cookie_recipe_url}', " \
"cookie_sku='{self.cookie_sku}', " \
"quantity={self.quantity}, " \
"unit_cost={self.unit_cost})".format(self=self)


class User(Base):
__tablename__ = 'users'

user_id = Column(Integer(), primary_key=True)
username = Column(String(15), nullable=False, unique=True)
email_address = Column(String(255), nullable=False)
phone = Column(String(20), nullable=False)
password = Column(String(25), nullable=False)
created_on = Column(DateTime(), default=datetime.now)
updated_on = Column(DateTime(), default=datetime.now,
onupdate=datetime.now)

def __repr__(self):
return "User(username='{self.username}', " \
"email_address='{self.email_address}', " \
"phone='{self.phone}', " \
"password='{self.password}')".format(self=self)


class Order(Base):
__tablename__ = 'orders'
order_id = Column(Integer(), primary_key=True)
user_id = Column(Integer(), ForeignKey('users.user_id'))
shipped = Column(Boolean(), default=False)
user = relationship("User", backref=backref(
'orders', order_by=order_id)) # one-to-many relationship

def __repr__(self):
return "Order(user_id={self.user_id}, " \
"shipped={self.shipped})".format(self=self)


class LineItem(Base):
__tablename__ = 'line_items'
line_items_id = Column(Integer(), primary_key=True)
order_id = Column(Integer(), ForeignKey('orders.order_id'))
cookie_id = Column(Integer(), ForeignKey('cookies.cookie_id'))
quantity = Column(Integer())
extended_cost = Column(Numeric(12, 2))
order = relationship("Order", backref=backref(
'line_items', order_by=line_items_id))
cookie = relationship("Cookie", uselist=False) # one-to-one relationship

def __repr__(self):
return "LineItems(order_id={self.order_id}, " \
"cookie_id={self.cookie_id}, " \
"quantity={self.quantity}, " \
"extended_cost={self.extended_cost})".format(
self=self)


engine = create_engine('sqlite:///sql_orm_setup.sqlite', echo=True)
Base.metadata.create_all(engine)

sql_orm_setup_cons

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
import os
from sqlalchemy.orm import relationship, backref
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Table, Column, Integer, Numeric, String, DateTime, ForeignKey, Boolean, CheckConstraint
from datetime import datetime
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker

dbname = os.path.basename(__file__).replace('.py', '.sqlite')
engine = create_engine('sqlite:///{}'.format(dbname), echo=True)

Session = sessionmaker(bind=engine)

session = Session()


Base = declarative_base()


class Cookie(Base):
__tablename__ = 'cookies'
__table_args__ = (CheckConstraint(
'quantity >= 0', name='quantity_positive'),)

cookie_id = Column(Integer, primary_key=True)
cookie_name = Column(String(50), index=True)
cookie_recipe_url = Column(String(255))
cookie_sku = Column(String(55))
quantity = Column(Integer())
unit_cost = Column(Numeric(12, 2))

def __init__(self, name, recipe_url=None, sku=None, quantity=0, unit_cost=0.00):
self.cookie_name = name
self.cookie_recipe_url = recipe_url
self.cookie_sku = sku
self.quantity = quantity
self.unit_cost = unit_cost

def __repr__(self):
return "Cookie(cookie_name='{self.cookie_name}', " \
"cookie_recipe_url='{self.cookie_recipe_url}', " \
"cookie_sku='{self.cookie_sku}', " \
"quantity={self.quantity}, " \
"unit_cost={self.unit_cost})".format(self=self)


class User(Base):
__tablename__ = 'users'

user_id = Column(Integer(), primary_key=True)
username = Column(String(15), nullable=False, unique=True)
email_address = Column(String(255), nullable=False)
phone = Column(String(20), nullable=False)
password = Column(String(25), nullable=False)
created_on = Column(DateTime(), default=datetime.now)
updated_on = Column(DateTime(), default=datetime.now,
onupdate=datetime.now)

def __init__(self, username, email_address, phone, password):
self.username = username
self.email_address = email_address
self.phone = phone
self.password = password

def __repr__(self):
return "User(username='{self.username}', " \
"email_address='{self.email_address}', " \
"phone='{self.phone}', " \
"password='{self.password}')".format(self=self)


class Order(Base):
__tablename__ = 'orders'
order_id = Column(Integer(), primary_key=True)
user_id = Column(Integer(), ForeignKey('users.user_id'))
shipped = Column(Boolean(), default=False)

user = relationship("User", backref=backref('orders', order_by=order_id))

def __repr__(self):
return "Order(user_id={self.user_id}, " \
"shipped={self.shipped})".format(self=self)


class LineItem(Base):
__tablename__ = 'line_items'
line_item_id = Column(Integer(), primary_key=True)
order_id = Column(Integer(), ForeignKey('orders.order_id'))
cookie_id = Column(Integer(), ForeignKey('cookies.cookie_id'))
quantity = Column(Integer())
extended_cost = Column(Numeric(12, 2))

order = relationship("Order", backref=backref(
'line_items', order_by=line_item_id))
cookie = relationship("Cookie", uselist=False)

def __repr__(self):
return "LineItems(order_id={self.order_id}, " \
"cookie_id={self.cookie_id}, " \
"quantity={self.quantity}, " \
"extended_cost={self.extended_cost})".format(
self=self)


Base.metadata.create_all(engine)


cookiemon = User('cookiemon', 'mon@cookie.com', '111-111-1111', 'password')
cc = Cookie('chocolate chip', 'http://some.aweso.me/cookie/recipe.html', 'CC01', 12, 0.50)
dcc = Cookie('dark chocolate chip',
'http://some.aweso.me/cookie/recipe_dark.html',
'CC02',
1,
0.75)
session.add(cookiemon)
session.add(cc)
session.add(dcc)


o1 = Order()
o1.user = cookiemon
session.add(o1)

line1 = LineItem(order=o1, cookie=cc, quantity=9, extended_cost=4.50)


session.add(line1)
session.commit()
o2 = Order()
o2.user = cookiemon
session.add(o2)

line1 = LineItem(order=o2, cookie=cc, quantity=2, extended_cost=1.50)
line2 = LineItem(order=o2, cookie=dcc, quantity=9, extended_cost=6.75)


session.add(line1)
session.add(line2)
session.commit()

sql_orm_setup_test

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
import os
from datetime import datetime

from sqlalchemy import (Column, Integer, Numeric, String, DateTime, ForeignKey,
Boolean, create_engine)
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship, backref, sessionmaker


conn_string = os.path.basename(__file__).replace('.py', '.sqlite')
Base = declarative_base()


class Cookie(Base):
__tablename__ = 'cookies'

cookie_id = Column(Integer, primary_key=True)
cookie_name = Column(String(50), index=True)
cookie_recipe_url = Column(String(255))
cookie_sku = Column(String(55))
quantity = Column(Integer())
unit_cost = Column(Numeric(12, 2))

def __repr__(self):
return "Cookie(cookie_name='{self.cookie_name}', " \
"cookie_recipe_url='{self.cookie_recipe_url}', " \
"cookie_sku='{self.cookie_sku}', " \
"quantity={self.quantity}, " \
"unit_cost={self.unit_cost})".format(self=self)


class User(Base):
__tablename__ = 'users'

user_id = Column(Integer(), primary_key=True)
username = Column(String(15), nullable=False, unique=True)
email_address = Column(String(255), nullable=False)
phone = Column(String(20), nullable=False)
password = Column(String(25), nullable=False)
created_on = Column(DateTime(), default=datetime.now)
updated_on = Column(DateTime(), default=datetime.now, onupdate=datetime.now)

def __repr__(self):
return "User(username='{self.username}', " \
"email_address='{self.email_address}', " \
"phone='{self.phone}', " \
"password='{self.password}')".format(self=self)


class Order(Base):
__tablename__ = 'orders'
order_id = Column(Integer(), primary_key=True)
user_id = Column(Integer(), ForeignKey('users.user_id'))
shipped = Column(Boolean(), default=False)

user = relationship("User", backref=backref('orders', order_by=order_id))

def __repr__(self):
return "Order(user_id={self.user_id}, " \
"shipped={self.shipped})".format(self=self)


class LineItem(Base):
__tablename__ = 'line_items'
line_item_id = Column(Integer(), primary_key=True)
order_id = Column(Integer(), ForeignKey('orders.order_id'))
cookie_id = Column(Integer(), ForeignKey('cookies.cookie_id'))
quantity = Column(Integer())
extended_cost = Column(Numeric(12, 2))

order = relationship("Order", backref=backref('line_items',
order_by=line_item_id))
cookie = relationship("Cookie", uselist=False)

def __repr__(self):
return "LineItems(order_id={self.order_id}, " \
"cookie_id={self.cookie_id}, " \
"quantity={self.quantity}, " \
"extended_cost={self.extended_cost})".format(
self=self)


class DataAccessLayer:

def __init__(self):
self.engine = None
self.session = None
self.conn_string = conn_string

def connect(self):
self.engine = create_engine(self.conn_string)
Base.metadata.create_all(self.engine)
self.Session = sessionmaker(bind=self.engine)


dal = DataAccessLayer()


def prep_db(session):
c1 = Cookie(cookie_name='dark chocolate chip',
cookie_recipe_url='http://some.aweso.me/cookie/dark_cc.html',
cookie_sku='CC02',
quantity=1,
unit_cost=0.75)
c2 = Cookie(cookie_name='peanut butter',
cookie_recipe_url='http://some.aweso.me/cookie/peanut.html',
cookie_sku='PB01',
quantity=24,
unit_cost=0.25)
c3 = Cookie(cookie_name='oatmeal raisin',
cookie_recipe_url='http://some.okay.me/cookie/raisin.html',
cookie_sku='EWW01',
quantity=100,
unit_cost=1.00)
session.bulk_save_objects([c1, c2, c3])
session.commit()

cookiemon = User(username='cookiemon',
email_address='mon@cookie.com',
phone='111-111-1111',
password='password')
cakeeater = User(username='cakeeater',
email_address='cakeeater@cake.com',
phone='222-222-2222',
password='password')
pieperson = User(username='pieperson',
email_address='person@pie.com',
phone='333-333-3333',
password='password')
session.add(cookiemon)
session.add(cakeeater)
session.add(pieperson)
session.commit()

o1 = Order()
o1.user = cookiemon
session.add(o1)

line1 = LineItem(cookie=c1, quantity=2, extended_cost=1.00)

line2 = LineItem(cookie=c3, quantity=12, extended_cost=3.00)

o1.line_items.append(line1)
o1.line_items.append(line2)
session.commit()

o2 = Order()
o2.user = cakeeater

line1 = LineItem(cookie=c1, quantity=24, extended_cost=12.00)
line2 = LineItem(cookie=c3, quantity=6, extended_cost=6.00)

o2.line_items.append(line1)
o2.line_items.append(line2)

session.add(o2)
session.commit()

Data Operation

Session

1
2
3
4
5
from sqlalchemy.orm import sessionmaker

Session = sessionmaker(bind=engine)

session = Session()

Insert

Inserting a single object:

1
2
3
4
5
6
7
8
9
cc_cookie = Cookie(cookie_name='chocolate chip',
cookie_recipe_url='http://some.aweso.me/cookie/recipe.html',
cookie_sku='CC01',
quantity=12,
unit_cost=0.50)
session.add(cc_cookie)
session.commit()

cc_cookie.cookie_id

Multiple inserts:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
dcc = Cookie(cookie_name='dark chocolate chip',
cookie_recipe_url='http://some.aweso.me/cookie/recipe_dark.html',
cookie_sku='CC02',
quantity=1,
unit_cost=0.75)
mol = Cookie(cookie_name='molasses',
cookie_recipe_url='http://some.aweso.me/cookie/recipe_molasses.html',
cookie_sku='MOL01',
quantity=1,
unit_cost=0.80)
session.add(dcc)
session.add(mol)
session.flush()

print(dcc.cookie_id)
print(mol.cookie_id)

Bulk inserting multiple records:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
c1 = Cookie(cookie_name='peanut butter',
cookie_recipe_url='http://some.aweso.me/cookie/peanut.html',
cookie_sku='PB01',
quantity=24,
unit_cost=0.25)
c2 = Cookie(cookie_name='oatmeal raisin',
cookie_recipe_url='http://some.okay.me/cookie/raisin.html',
cookie_sku='EWW01',
quantity=100,
unit_cost=1.00)
session.bulk_save_objects([c1,c2])
session.commit()
# If you are inserting multiple records and don’t need access to relationships or the inserted primary key, use bulk_save_objects or its related methods.
c1.cookie_id

Query

1
2
3
4
5
cookies = session.query(Cookie).all()
print(cookies)

for cookie in session.query(Cookie):
print(cookie)

Controlling the Columns in the Query:

1
print(session.query(Cookie.cookie_name, Cookie.quantity).first())

Ordering:

1
2
3
4
5
6
7
for cookie in session.query(Cookie).order_by(Cookie.quantity):
print('{:3} - {}'.format(cookie.quantity, cookie.cookie_name))


from sqlalchemy import desc
for cookie in session.query(Cookie).order_by(desc(Cookie.quantity)):
print('{:3} - {}'.format(cookie.quantity, cookie.cookie_name))

Limit:

1
2
3
4
5
query = session.query(Cookie).order_by(Cookie.quantity)[:2] # inefficient
print([result.cookie_name for result in query])

query = session.query(Cookie).order_by(Cookie.quantity).limit(2)
print([result.cookie_name for result in query])

Built-In SQL Functions and Labels:

1
2
3
4
5
6
7
8
9
10
11
from sqlalchemy import func
inv_count = session.query(func.sum(Cookie.quantity)).scalar()
print(inv_count)

rec_count = session.query(func.count(Cookie.cookie_name)).first()
print(rec_count)

rec_count = session.query(func.count(Cookie.cookie_name) \
.label('inventory_count')).first()
print(rec_count.keys())
print(rec_count.inventory_count)

Filter:

1
2
3
4
5
6
7
8
9
record = session.query(Cookie).filter(Cookie.cookie_name == 'chocolate chip').first()
print(record)

record = session.query(Cookie).filter_by(cookie_name='chocolate chip').first()
print(record)

query = session.query(Cookie).filter(Cookie.cookie_name.like('%chocolate%'))
for record in query:
print(record.cookie_name)

Operators:

1
2
3
4
5
6
7
8
9
10
11
results = session.query(Cookie.cookie_name, 'SKU-' + Cookie.cookie_sku).all()
for row in results:
print(row)


from sqlalchemy import cast
query = session.query(Cookie.cookie_name,
cast((Cookie.quantity * Cookie.unit_cost),
Numeric(12,2)).label('inv_cost'))
for result in query:
print('{} - {}'.format(result.cookie_name, result.inv_cost))

Boolean Operators:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
from sqlalchemy import and_, or_, not_
query = session.query(Cookie).filter(
Cookie.quantity > 23,
Cookie.unit_cost < 0.40
)
for result in query:
print(result.cookie_name)


from sqlalchemy import and_, or_, not_
query = session.query(Cookie).filter(
or_(
Cookie.quantity.between(10, 50),
Cookie.cookie_name.contains('chip')
)
)
for result in query:
print(result.cookie_name)

Update

1
2
3
4
5
query = session.query(Cookie)
cc_cookie = query.filter(Cookie.cookie_name == "chocolate chip").first()
cc_cookie.quantity = cc_cookie.quantity + 120
session.commit()
print(cc_cookie.quantity)
1
2
3
4
5
6
query = session.query(Cookie)
query = query.filter(Cookie.cookie_name == "chocolate chip")
query.update({Cookie.quantity: Cookie.quantity - 20})

cc_cookie = query.first()
print(cc_cookie.quantity)

Delete

1
2
3
4
5
6
7
query = session.query(Cookie)
query = query.filter(Cookie.cookie_name == "dark chocolate chip")
dcc_cookie = query.one()
session.delete(dcc_cookie)
session.commit()
dcc_cookie = query.first()
print(dcc_cookie)
1
2
3
4
5
query = session.query(Cookie)
query = query.filter(Cookie.cookie_name == "molasses")
query.delete()
mol_cookie = query.first()
print(mol_cookie)

setup:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
cookiemon = User(username='cookiemon',
email_address='mon@cookie.com',
phone='111-111-1111',
password='password')
cakeeater = User(username='cakeeater',
email_address='cakeeater@cake.com',
phone='222-222-2222',
password='password')
pieperson = User(username='pieperson',
email_address='person@pie.com',
phone='333-333-3333',
password='password')
session.add(cookiemon)
session.add(cakeeater)
session.add(pieperson)
session.commit()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
o1 = Order()
o1.user = cookiemon
session.add(o1)

cc = session.query(Cookie).filter(Cookie.cookie_name =="chocolate chip").one()
line1 = LineItem(cookie=cc, quantity=2, extended_cost=1.00)

pb = session.query(Cookie).filter(Cookie.cookie_name =="peanut butter").one()
line2 = LineItem(quantity=12, extended_cost=3.00)
line2.cookie = pb
line2.order = o1

o1.line_items.append(line1)
o1.line_items.append(line2)
session.commit()
1
2
3
4
5
6
7
8
9
10
11
12
13
14
o2 = Order()
o2.user = cakeeater

cc = session.query(Cookie).filter(Cookie.cookie_name == "chocolate chip").one()
line1 = LineItem(cookie=cc, quantity=24, extended_cost=12.00)

oat = session.query(Cookie).filter(Cookie.cookie_name == "oatmeal raisin").one()
line2 = LineItem(cookie=oat, quantity=6, extended_cost=6.00)

o2.line_items.append(line1)
o2.line_items.append(line2)

session.add(o2)
session.commit()

Joins

Using join to select from multiple tables:

1
2
3
4
query = session.query(Order.order_id, User.username, User.phone, Cookie.cookie_name, LineItem.quantity, LineItem.extended_cost)
query = query.join(User).join(LineItem).join(Cookie)
results = query.filter(User.username == 'cookiemon').all()
print(results)

Using outerjoin to select from multiple tables:

1
2
3
4
query = session.query(User.username, func.count(Order.order_id))
query = query.outerjoin(Order).group_by(User.username)
for row in query:
print(row)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
class Employee(Base):
__tablename__ = 'employees'

id = Column(Integer(), primary_key=True)
manager_id = Column(Integer(), ForeignKey('employees.id'))
name = Column(String(255), nullable=False)

manager = relationship("Employee", backref=backref('reports'), remote_side=[id])

Base.metadata.create_all(engine)

marsha = Employee(name='Marsha')
fred = Employee(name='Fred')
marsha.reports.append(fred)
session.add(marsha)
session.commit()

for report in marsha.reports:
print(report.name)

Group

1
2
3
4
query = session.query(User.username, func.count(Order.order_id))
query = query.outerjoin(Order).group_by(User.username)
for row in query:
print(row)

Chain

Chaining:

1
2
3
4
5
6
7
def get_orders_by_customer(cust_name):
query = session.query(Order.order_id, User.username, User.phone, Cookie.cookie_name, LineItem.quantity,LineItem.extended_cost)
query = query.join(User).join(LineItem).join(Cookie)
results = query.filter(User.username == cust_name).all()
return results

get_orders_by_customer('cakeeater')

Conditional chaining:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
def get_orders_by_customer(cust_name, shipped=None, details=False):
query = session.query(Order.order_id, User.username, User.phone)
query = query.join(User)
if details:
query = query.add_columns(Cookie.cookie_name, LineItem.quantity,
LineItem.extended_cost)
query = query.join(LineItem).join(Cookie)
if shipped is not None:
query = query.filter(Order.shipped == shipped)
results = query.filter(User.username == cust_name).all()
return results

print(get_orders_by_customer('cakeeater'))

print(get_orders_by_customer('cakeeater', details=True))

print(get_orders_by_customer('cakeeater', shipped=True))

print(get_orders_by_customer('cakeeater', shipped=False))

print(get_orders_by_customer('cakeeater', shipped=False, details=True))

Raw Queries

1
2
3
from sqlalchemy import text
query = session.query(User).filter(text("username='cookiemon'"))
print(query.all())

Session and Exceptions

from sql_orm_setup import *

Session States

1
2
3

cc_cookie = Cookie(
cookie_name='chocolate chip', cookie_recipe_url='http://some.aweso.me/cookie/recipe.html', cookie_sku='CC01', quantity=12, unit_cost=0.50)

transient:

1
2
3
4
from sqlalchemy import inspect
insp = inspect(cc_cookie)
for state in ['transient', 'pending', 'persistent', 'detached']:
print('{:>10}: {}'.format(state, getattr(insp, state)))

pending:

1
2
3
session.add(cc_cookie)
for state in ['transient','pending','persistent','detached']:
print('{:>10}: {}'.format(state, getattr(insp, state)))

persistent:

1
2
3
session.commit()
for state in ['transient','pending','persistent','detached']:
print('{:>10}: {}'.format(state, getattr(insp, state)))

detached:

1
2
3
session.expunge(cc_cookie)
for state in ['transient','pending','persistent','detached']:
print('{:>10}: {}'.format(state, getattr(insp, state)))
1
2
3
4
5
6
7
8
session.add(cc_cookie)
cc_cookie.cookie_name = 'Change chocolate chip'

insp.modified
for attr, attr_state in insp.attrs.items():
if attr_state.history.has_changes():
print('{}: {}'.format(attr, attr_state.value))
print('History: {}\n'.format(attr_state.history))

MultipleResultsFound Exception

1
2
3
4
5
6
7
8
9
10
11
12
13
dcc = Cookie('dark chocolate chip',
'http://some.aweso.me/cookie/recipe_dark.html',
'CC02', 1, 0.75)
session.add(dcc)
session.commit()

from sqlalchemy.orm.exc import MultipleResultsFound
try:
results = session.query(Cookie).one()
except MultipleResultsFound as exc:
print('We found too many cookies... is that even possible?')

session.query(Cookie).all()

DetachedInstanceError

1
2
3
4
5
6
7
8
9
10
11
12
cookiemon = User('cookiemon', 'mon@cookie.com', '111-111-1111', 'password')
session.add(cookiemon)
o1 = Order()
o1.user = cookiemon
session.add(o1)

cc = session.query(Cookie).filter(Cookie.cookie_name ==
"Change chocolate chip").one()
line1 = LineItem(order=o1, cookie=cc, quantity=2, extended_cost=1.00)

session.add(line1)
session.commit()
1
2
3
order = session.query(Order).first()
session.expunge(order)
order.line_items.all()

Transactions

1
from sql_orm_setup_cons import *

Defining the ship_it function:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
def ship_it(order_id):
order = session.query(Order).get(order_id)
for li in order.line_items:
li.cookie.quantity = li.cookie.quantity - li.quantity
session.add(li.cookie)
order.shipped = True
session.add(order)
session.commit()
print("shipped order ID: {}".format(order_id))

ship_it(1)
print(session.query(Cookie.cookie_name, Cookie.quantity).all())

ship_it(2)
print(session.query(Cookie.cookie_name, Cookie.quantity).all())
session.rollback()
print(session.query(Cookie.cookie_name, Cookie.quantity).all())

Transactional ship_it:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
from sqlalchemy.exc import IntegrityError
def ship_it(order_id):
order = session.query(Order).get(order_id)
for li in order.line_items:
li.cookie.quantity = li.cookie.quantity - li.quantity
session.add(li.cookie)
order.shipped = True
session.add(order)
try:
session.commit()
print("shipped order ID: {}".format(order_id))
except IntegrityError as error:
print('ERROR: {!s}'.format(error.orig))
session.rollback()

ship_it(2)

Testing with SQLAlchemy ORM

1
from sql_orm_setup_test import *

Testing with a Test Database

1
2
3
4
5
6
7
8
9
10
11
12
13
14
from db import Cookie, LineItem, Order, User,  dal


def get_orders_by_customer(cust_name, shipped=None, details=False):
query = dal.session.query(Order.order_id, User.username, User.phone)
query = query.join(User)
if details:
query = query.add_columns(Cookie.cookie_name, LineItem.quantity,
LineItem.extended_cost)
query = query.join(LineItem).join(Cookie)
if shipped is not None:
query = query.filter(Order.shipped == shipped)
results = query.filter(User.username == cust_name).all()
return results
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
import unittest

from decimal import Decimal

from db import prep_db, dal

from app import get_orders_by_customer


class TestApp(unittest.TestCase):
cookie_orders = [(1, u'cookiemon', u'111-111-1111')]
cookie_details = [
(1, u'cookiemon', u'111-111-1111',
u'dark chocolate chip', 2, Decimal('1.00')),
(1, u'cookiemon', u'111-111-1111',
u'oatmeal raisin', 12, Decimal('3.00'))]

@classmethod
def setUpClass(cls):
dal.conn_string = 'sqlite:///:memory:'
dal.connect()
dal.session = dal.Session()
prep_db(dal.session)
dal.session.close()

def setUp(self):
dal.session = dal.Session()

def tearDown(self):
dal.session.rollback()
dal.session.close()

def test_orders_by_customer_blank(self):
results = get_orders_by_customer('')
self.assertEqual(results, [])

def test_orders_by_customer_blank_shipped(self):
results = get_orders_by_customer('', True)
self.assertEqual(results, [])

def test_orders_by_customer_blank_notshipped(self):
results = get_orders_by_customer('', False)
self.assertEqual(results, [])

def test_orders_by_customer_blank_details(self):
results = get_orders_by_customer('', details=True)
self.assertEqual(results, [])

def test_orders_by_customer_blank_shipped_details(self):
results = get_orders_by_customer('', True, True)
self.assertEqual(results, [])

def test_orders_by_customer_blank_notshipped_details(self):
results = get_orders_by_customer('', False, True)
self.assertEqual(results, [])

def test_orders_by_customer_bad_cust(self):
results = get_orders_by_customer('bad name')
self.assertEqual(results, [])

def test_orders_by_customer_bad_cust_shipped(self):
results = get_orders_by_customer('bad name', True)
self.assertEqual(results, [])

def test_orders_by_customer_bad_cust_notshipped(self):
results = get_orders_by_customer('bad name', False)
self.assertEqual(results, [])

def test_orders_by_customer_bad_cust_details(self):
results = get_orders_by_customer('bad name', details=True)
self.assertEqual(results, [])

def test_orders_by_customer_bad_cust_shipped_details(self):
results = get_orders_by_customer('bad name', True, True)
self.assertEqual(results, [])

def test_orders_by_customer_bad_cust_notshipped_details(self):
results = get_orders_by_customer('bad name', False, True)
self.assertEqual(results, [])

def test_orders_by_customer(self):
results = get_orders_by_customer('cookiemon')
self.assertEqual(results, self.cookie_orders)

def test_orders_by_customer_shipped_only(self):
results = get_orders_by_customer('cookiemon', True)
self.assertEqual(results, [])

def test_orders_by_customer_unshipped_only(self):
results = get_orders_by_customer('cookiemon', False)
self.assertEqual(results, self.cookie_orders)

def test_orders_by_customer_with_details(self):
results = get_orders_by_customer('cookiemon', details=True)
self.assertEqual(results, self.cookie_details)

def test_orders_by_customer_shipped_only_with_details(self):
results = get_orders_by_customer('cookiemon', True, True)
self.assertEqual(results, [])

def test_orders_by_customer_unshipped_only_details(self):
results = get_orders_by_customer('cookiemon', False, True)
self.assertEqual(results, self.cookie_details)

Using Mocks

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
import unittest
from decimal import Decimal

import mock

from app import get_orders_by_customer


class TestApp(unittest.TestCase):
cookie_orders = [(1, u'cookiemon', u'111-111-1111')]
cookie_details = [
(1, u'cookiemon', u'111-111-1111',
u'dark chocolate chip', 2, Decimal('1.00')),
(1, u'cookiemon', u'111-111-1111',
u'oatmeal raisin', 12, Decimal('3.00'))]

@mock.patch('app.dal.session')
def test_orders_by_customer_blank(self, mock_dal):
mock_dal.query.return_value.join.return_value.filter.return_value. \
all.return_value = []
results = get_orders_by_customer('')
self.assertEqual(results, [])

@mock.patch('app.dal.session')
def test_orders_by_customer_blank_shipped(self, mock_dal):
mock_dal.query.return_value.join.return_value.filter.return_value. \
filter.return_value.all.return_value = []
results = get_orders_by_customer('', True)
self.assertEqual(results, [])

@mock.patch('app.dal.session')
def test_orders_by_customer(self, mock_dal):
mock_dal.query.return_value.join.return_value.filter.return_value. \
all.return_value = self.cookie_orders
results = get_orders_by_customer('cookiemon')
self.assertEqual(results, self.cookie_orders)

Reflection with SQLAlchemy ORM and Automap

Reflecting a Database with Automap

setup:

1
2
3
4
5
6
7
8
9
10
11
12
from sqlalchemy.ext.automap import automap_base
from sqlalchemy.orm import Session
from sqlalchemy import create_engine

Base = automap_base()

from sqlalchemy import create_engine

engine = create_engine('sqlite:///Chinook_Sqlite.sqlite')

Base.prepare(engine, reflect=True)
Base.classes.keys()
1
2
3
4
5
6
7
8
9
10
11
12
Artist = Base.classes.Artist
Album = Base.classes.Album

from sqlalchemy.orm import Session

session = Session(engine)
for artist in session.query(Artist).limit(10):
print(artist.ArtistId, artist.Name)

artist = session.query(Artist).first()
for album in artist.album_collection:
print('{} - {}'.format(artist.Name, album.Title))

Trick Note

References of some interesting tricks.

Show password in web

1
document.querySelector("input[type=password]").value;

Show IP address

1
curl ifconfig.me

Download file and unzip

1
2
wget https://training.linuxfoundation.org/cm/LFS258/LFS258V2021-09-20SOLUTIONS.tar.xz --user=xxx --password=xxx
tar -xvf LFS258V2021-09-20SOLUTIONS.tar.xz

Send email from gmail

Reference:

Note:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import smtplib
from email.mime.text import MIMEText
import ssl

port = 465
password = input("your password")
context = ssl.create_default_context()

msg = MIMEText("The body of the email is here")
msg['Subject'] = "An Email Alert"
msg['From'] = "my@gmail.com"
msg['To'] = "other@xxx.xxx"

with smtplib.SMTP_SSL("smtp.gmail.com", port, context=context) as server:
server.login("my@gmail.com", password)
server.send_message(msg)

Check the quality of a website

Algo Note

Merge Sort

Explanation:

  1. Wikipedia: https://en.wikipedia.org/wiki/Merge_sort
  2. Youtube: https://www.youtube.com/watch?time_continue=17&v=KF2j-9iSf4Q&feature=emb_logo

Time complexity: O(nlogn)
Space complexity: O(n)

Implementation:

Python

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

def merge(arr1: list, arr2: list) -> tuple:
"""Merge two arrays

Args:
arr1 (list): first array
arr2 (list): second array

Returns:
tuple(int, list): number of swaps, sorted array
"""
result = []
count = 0
i, j = 0, 0
m, n = len(arr1), len(arr2)
while i < m and j < n:
if arr1[i] <= arr2[j]:
result.append(arr1[i])
i += 1
else:
result.append(arr2[j])
count += m - i
j += 1
result += arr1[i:]
result += arr2[j:]
return count, result


def msort(arr: list) -> tuple:
"""Merge Sort Algorithm

Args:
arr (list): unsorted array

Returns:
tuple(int, list): number of swaps, sorted array
"""
n = len(arr)
if n > 1:
mid = n // 2
left_swaps, left_result = msort(arr[:mid])
right_swaps, right_result = msort(arr[mid:])
merge_swaps, result = merge(left_result, right_result)
return left_swaps+right_swaps+merge_swaps, result
return 0, arr

assert msort([1, 2, 5, 6, 3, 7, 4, 8]) == (5, [1, 2, 3, 4, 5, 6, 7, 8])

LCS(longest common subsequence) problem

Explanation:

  1. Wikipedia: https://en.wikipedia.org/wiki/Longest_common_subsequence_problem

complexity: O(n × m)

Implementation

Python

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
def commonChild(s1: str, s2: str) -> int:
"""Find length of common child string

Args:
s1 (str): first string
s2 (str): second string

Returns:
int: length of common child string
"""
m = len(s1)
n = len(s2)
mat = [[0 for _ in range(n+1)] for _ in range(m+1)]
for r in range(1, m+1):
for c in range(1, n+1):
i, j = r - 1, c - 1
if s1[i] == s2[j]:
mat[r][c] = mat[r-1][c-1] + 1
else:
mat[r][c] = max(mat[r-1][c], mat[r][c-1])
return mat[m][n]

assert commonChild("SHINCHAN", "NOHARAAA") == 3