CS50 Notes
Index
Intro
- Computational thinking follows this general logic to solve a problem:
- input -> process (algorithms) -> output
- Computers always use a finite amount of information to represent something in the case of number you use a certain amount of finite number of bits
- Computer doesn't know how to make assumptions it will only do precisely what you tell it to do
- Everything on the SW world ends up taking up much longer than anticipated
Basic Elements of Programming language
- Statements
- Loops
- Conditionals
- Boolean expressions
- Fork / branches
- Data types
- primitives
- modifiers / qualifiers
void is a type but not a data type
- scope refers to the context in which a variable exist or is usable
precedence
- precedence answers the question what operation should we do first. Rules:
- innermost parentheses to outwards
- if operators have equal priority you solve from left to right
- dereference operator has priority over math operators but not over suffix increment or decrement operators
Typecasting
- It is an easy way to alter computer interpretation by implicitly (you let compiler do the casting) or explicitly (you specify the type with a parentheses) changing its data type
- The data type determines how the computer and user interprets a variable
- In typecasting you can make a variable lose precision but never gain precision (truncation)
Compiler / Compilation
- source code: code that is on a programming languages and it can be understood by humans who know that programming languages
- object code: Fancy way of saying binary output or "code that can be understand by a machine"
- Compile in the most general sense means transforming code from one programming language into another language or into another format
- Compiler: Piece of SW that takes source code as input and produces object code as output
Compilation main steps (Considering C/C++) CS50 Compilers Short
- Compile is the general term to describe the entire process needed to generate an executable, which precisely involves other processes
- Preprocessing
- Replaces all pre-processor directives (
#define, #include...) which basically means copy and paste all the included .h files or libraries and replace all the occurrences of a #define with the correspondent value
- This step is done by the C pre-processor
- Compilation
- Transform C language code to assembly code
- Assembler
- Translates Assembly code into object code or machine code which is the binary code or ones and zeros a CPU can actually understand
- This step is considered a translation and NOT a transformation since is a very direct/easy to go from assembly to object code
- Linking
- Combines object files into an executable file or application
- To link multiple files you specify all of the when linking with the only restriction that only one can have a main function
- Adding libraries flags is the same that specifying the .o with the compiled code you use on your program
put all -l options at the end
make
- make is an utility mostly used for compiling source code
- It is a program the sequences other operations to produce a successful compilation output (usually an application or a library)
- It is a program that organizes invocations of the compiler on source code files so that things get compiled in the proper order
- Makefiles are the configuration files that tell make exactly what to do.
Library
- A Library is a collection of related prewritten code, and its a way a programmer can share common code
- Technically it is just binary or compiled file that has been produced by linking together a collection of object files
- A Library is usually composed of two parts
- Header files (*.h) include only declaration not implementation of functionality, this is an example of information hiding or encapsulation
- implementation files (*.c) include the actual code implementation
when sharing a library we usually don't share the implementation files (*.c) we just share a compiled binary file which is represent all the .c files
Computational and Algorithm complexity
- Computational complexity asks the questions how an algorithm handles larger and larger data set or how does this algorithm scale
as we throw more and more data at it
- Its a more uniform way of measuring algorithm efficiency
- asymptotic complexity: analyze how fast a programs run time grows asymptotically, so the input tend to infinity
Big O
- Big O notation is a notation that computer scientist uses to describe an upper bound on an algorithm's running time
- Describes the maximum number of steps or maximum amount of time an algorithm will take to complete
- It gives an idea of how much time an algorithm takes to complete in the the worst case scenario
- for example we may say an algorithm takes at worst n squared (n^2) steps
Omega
- Omega notation is a notation that computer scientist use to describe a lower bound on an algorithm's running time
- It gives an idea of how much time an algorithm takes to complete in the best case scenario
Sorting and Searching Algorithms
Selection Sort - O(n^2) and Om(n^2)
- Find the smallest unsorted element and swap it with the first unsorted element of the list (put it a the beginning of the list)
- Pick the criteria, find the first element that meets that criteria and order that element.
- Build a sorted list one element at a time
Bubble Sort - O(n^2) and Om(n)
- Look at adjacent pair elements on the list and if they are out of order swap them, then move to the next pair and repeat this process until no swaps are needed
- The general idea is to swap elements to move the bigger elements to end smaller elements to the beginning
Insertion Sort - O(n^2) and Om(n)
- For the first element state a one element array is sorted, find the next smallest element and insert it in the proper position making sure all elements behind are sorted
- It basically find element and insert it where belongs, shifting elements as necessary
Merge Sort - O(n*log(n)) and Om(n*log(n))
- The idea is to sort smaller arrays and combine them together in sorted array using recursion
- sort left side of the array, sort right side of the array and merge the two halves together
Linear search - O(n) and Om(1)
- Iterate across the array from left to right looking for the specified element
Binary search - O(log(n)) and Om(1)
- It makes the assumption the data is sorted, it requires the condition that the data is sorted in order to work
- Divide the amount of data by halve each time we are trying to find an element until the sub-list is zero
- It makes a binary decision
Data structures
- A data structure is just a container for data
- Something that provides a way to store data in a specific format
Binary search tree
It's a data structure with the following properties:
- If you look at any node its left child is smaller and its right child is bigger
- The left subtree of a node only contains values that are less than or equal to the node's value
- The right subtree of a node only contains values that are greater than or equal to the node's value
- Both left and right subtrees are also binary search tree
Hash table
- Combine the random access ability of an array with the dynamism of a linked list
- Insert, delete and lookup start to tend toward average of O(1)
- The data itself will produce an identifier that can be used to add/remove/update/search it
- They don't solve the problem of sorting/ordering data
- Elements:
- Hash Function math function returns a non-neg integer called hash code
- It should use all the data and only the data, not incorporated or add anything extra without reason
- Be deterministic. same input -> same output
- Uniformly distribute data
- Generate very different hash codes for similar data
- Array capable of storing the data we want to add
- It runs the data through the hash function, this return a hash code which is used as the index of the array in which we are going to store the data
- collision occurs when two pieces of data that are put through the same hash function produce the same hash code
- chaining technique of using linked list instead of simple arrays to store data
Tries
- They are a specific type of tree
- Use the data as a roadmap, no two different pieces of data cant have exactly the same roadmap
- It uses the idea that keys should be unique to avoid collisions
- to insert we need to build the correct path from the root to the leaf
Data Structures Summary
| Data structure |
Pros |
Cons |
| Arrays |
Lookup is great due to random access instant time |
Random insertion and deletion is bad because it requires shifting |
|
Small memory usage and Relatively easy to sort |
Fixed size, no flexibility or scalability |
| Linked List |
Small memory usage and Insertion/deletion is easy |
Difficult to sort and lookup can be slow |
| Hash tables |
Deletion is easy and lookup is relatively fast |
Hard to sort and insertion can become difficult (bad hash function) |
| Tries |
Deletion is easy, lookup is fast, it sorts as you build |
Insertion is complex and uses a lot of memory in the general case |
Web Fundamentals
- We use/get a public IP from the ISP and then use a router and network address translation to assign unique/private addresses (ranges: 10.#.#.#, 172.16.#.#, 192.168.#.#)
- Router allows data requests from all of the devices on a local network to be processed through a single IP address
- Web server is a software which purpose is to serve web pages
- DHCP protocol used to dynamically assign IP addresses
- DNS protocol that translates domain names to IP addresses and vice versa, the DNS servers are a decentralized network in which each node holds local names and large
- The internet is just interconnected networks, connection of multiple smaller networks which create a bigger network, to communicate with one an other
- Not all the networks are directly connected to each other, we rely on routers to distribute the communications
TCP/IP
HTTP
- It is a application layer protocol, set of conventions/rules that dictates how client talks to a server
- Dictates the format by which clients request webpages from servers and the format servers use to return information to clients
- Specifies how to make requests to webpages and how the response will be formatted
- get sends the request parameters directly in the URL publicly, post sends the parameters privately
HTML
- Markup language used to create webpages. It is not a programming language (no loops, no functions, no conditions, no logic, no variables, etc.)
- Consists of tags that tell a browser what to do or how to create/render? a web page and attributes which modify the behavior of tags
- Every html page contains a head and a body which is the actual content of a web page
CSS
- Language used to create style sheets which are used to customize the looks and style of a website
- Style sheets are used to cascade properties to all the child tags of an html parent tag
- Allows to define style (a set of properties) for tags
- A CSS file of style sheet is constructed with declarations that consist of:
- selectors: Identifier of the tag that will be modified
- key-value pairs of properties and value of that property
- Selectors don't have to apply only to HTML generic tags, you can define:
- An id selector (#) which will apply only to an HTML tag with an unique ID
- A class (.) selector which allows you to define a name for a set/group of tags and it will apply only to HTML tags that are given the same class attribute
- Other CSS selectors
- Media query: A specific way of assigning CSS to a particular HTML on a particular type of media (specific window size on a PC, resolution size, for printing)
- Uses the
@media on .css files
- SASS: Extension to CSS that let us programmatically generate CSS. Uses
.scss extension instead of usual .css
- Allows to define variables using
$, nesting and inheritance using templates (something that other things are going to inherit), uses % to define templates
- Web browsers usually don't understand SASS natively so we need a pre-processor to covert it to CSS
Dynamic programming
- It is a way of looking to a certain kinds of problems
- Category of algorithms that store the results of past computations and use those to calculate new values
- Use of look-up tables which are the ones that store results of the previous calculations
Python
- Scope is more flexible in python because is tide to sequence but not necessarily to indentation
from <lib> import * imports all functions and variables, using only import <lib> requires to prefix function calls with library name- Named parameters is supported so to pass parameters to function without relying on the order
- In Python classes are capitalized and file names (modules) are not by convention
- A decorator (
@) is used to modify the behavior of a function without actually changing the code of the function.
- It modifies the behavior of the original function by defining a function that takes as input the function we want to modify and returning a new function
Web-server
Flask:
- Web server software. Its a Web Micro framework
- You define routes for the web server. This means you tell what to do when a request for a certain source is received
cookie
- digital stamp or sign given by a webpage to a browser
- the browser stores digital sign and is constantly reminding the server of this through the http header in the http requests its sending to the server
Session
- Whatever it is the server is remembering on that users behalf per his cookie
- Box filled with values related to the user
setup
- install git:
sudo apt-get install git
- install python:
sudo apt-get install python && sudo apt-get install python3
- install sqlite3:
sudo apt-get install sqlite3
- install virtualenv:
sudo pip3 install virtualenvsudo apt-get install python-virtualenv
- change to project directory, create a virtualenv and activate environment:
cd ~/my_projectvirtualenv -v python3 myvenvsource myvenv/bin/activate
you can go back to the "real world" by deactivating your virtual environment with the deactivate command
- install your project requirements
pip install -r requirements.txt
verify pip --version returns that pip3 is being used and
make sure to add Flask and Flask-Session to the requirements.txt file
- export
Flask env variables
export FLASK_APP=application.py
export FLASK_DEBUG=1
MVC (Model view Controller)
- Paradigm (practices) to structure web server software or web projects. Its primary motivation is security to prevent the user to have a direct connection with the model (data)
- Controller is the programing logic that responds to inputs and updates/changes content (Code in java, python, etc.)
- View aesthetics, view or user interface the user actually see (html, CSS, jinja, templates, layouts, etc.)
- Model Refers to the data or where all the important data for your site lives (Database). Also refers to the logic to read/write/store data
Database and Database Design
- Database Piece of software or database program stores a hierarchical organized set of tables and listens for connections, gets requests and serves responses
- In a database it's usually better to separate data into smaller pieces like First name, Last name, Street address, City, Country, etc.
- Separating data creates the problem of unnecessary redundancy and waste of storage space
- Relational databases solve the problem of unnecessary redundancy, the idea is to factor out common information and create relation between tables using IDs/keys
- For this we use foreign key, which is the method used to connect multiple tables together
- Relational databases is not user friendly for humans is good for machines (computers)
- All queries to a database refers to the rows of a certain table
- Transactions: is the method used to solve race condition problems when accessing a database, the idea is to lock the database while running a transaction (sequence of commands)
SQL (Structure Query Language) and SQLite
- Language to interact with a database (query a database)
- phpLiteAdmin/phpMyAdmin: GUI tool to visualize/edit a binary file that contains a database
- Fundamental SQL Data types:
BLOB (Binary Large OObject), INTEGER, NULL, REAL, TEXT, DATETIME, NUMERIC
- SQL uses
BEGIN and COMMIT to lock the database and complete transaction
- Modifiers:
PRIMARY KEY: There can only be one primary key per Table and signals that the data in this column will be used as IdentifierUNIQUE: make a column unique and make sure we don't have data duplicatesINDEX: indicate the database to index the data in this column to make it easily/efficiently searchableNOT NULL: Don't let NULL be inserted in this column, it cant be blankFOREIGN KEY: This is the primary key in another table, use it to link data in different tables
- Built-in Functions:
date, datetime, time
- Basic Operations:
| Operation |
Canonical |
Example |
CREATE |
CREATE TABLE <table_name> ('<column_name>' <SQL Type> <SQL MOdifiers>, ... ) |
CREATE TABLE 'registrants' ('id' INTEGER PRIMRAY KEY, 'name' TEXT, 'dorm' TEXT) |
INSERT |
INSERT INTO "<table_name>" ('<columns>') VALUES (<values>) |
INSERT INTO "registrants" ("id", "name", "dorm") VALUES (1, 'Pepe', 'GDL') |
SELECT |
SELECT <columns> FROM "<table>" WHERE <condition> ORDER BY <column> |
SELECT * FROM "registrants" |
| - |
SELECT <columns> FROM "<table1>" JOIN "<table2>" ON <condition> |
SELECT * FROM "Album" JOIN "Artist" ON Album.ArtistId = Artist.ArtistId |
UPDATE |
UPDATE "<table>" SET <column> = <value> WHERE <condition> |
UPDATE "registrants" SET "name" = 'Pepe Ch' WHERE id = 1 |
DELETE |
DELETE FROM "<table>" WHERE <condition> |
DELETE FROM "registrants" WHERE id = 1 |
sqlite commands
sqlite database_file_name.db starts and creates a database with the given name- Commands that start with
. is SQLite command line program specific not part of SQL
~/workspace/ $ sqlite3 database_file_name.db
SQLite version 3.8.2 2013-12-06 14:53:30
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .help
...
sqlite> CREATE TABLE 'registrants' ('id' INTEGER PRIMRAY KEY, 'name' TEXT, 'dorm' TEXT);
...
sqlite> .schema
...
sqlite> .quit
SQLAlchemy
- It is a Python library used to connect Python and SQL, it allows python code to run SQL queries
ORM (Object-Relational Mapping)
- Allows us to tie OOP with databases
- Uses method and classes to interact with databases
- For any table in our database we will have a class
JavaScript
- Event based programming: paradigm in which programs listen and react to events
- The language, built into the browser (JS), essentially, just itself has this infinite loop that it's constantly listening for events and reacting accordingly as told by a programmer
- Language that is commonly used in the confines of a browser (on the client side)
- Recently it has been used on the server side in something called node.js
- Everything is an object and all variables are JSON (JavaScript Object Notation) objects which are list of key-value pairs
- Supports anonymous functions also known as lambda functions
- The
document object is a special global variable/object that refers to the web document, it's the object that organizes the entire content of a web page so we can manipulate the page's elements programmatically
- DOM common properties:
innerHTML, nodeName, id, parentNode, childNodes, attributes, style
- DOM common methods:
getElementById(id), getElementsByTagName(tag), appendChild(node), removeChild(node)
navigator is another special global variable from the browser that has geo functionality- variables can be declared with
const (a constant with local scope), let (variable exists only in the scope of innermost curly braces) or var (variable exists in the whole scope of function where it was declared)
- Arrow Functions (
=>) are defined with the list of parameters taken as input and without the use of the word function
this refers to the specific object that calls a function but only if it was defined in the traditional way using the keyword function()
- This means that using arrow notation
=> will bound this to whatever it was found in the code that was enclosing that arrow function
jQuery
- It is an open-source library that is designed to "simplify" common usages of JavaScript like DOM manipulation
- Bootstrap requires jQuery to work
- jQuery defines a global variable called
jQuery which alias is $
- jQuery Documentation
AJAX
- AJAX is a feature in browsers that lets them execute HTTP request after the first/primary request is done (the primary request will be visiting the web page)
- Extra HTTP requests usually return application/json in response instead of html/text like when visiting a page
- AJAX allows to interact with the server side without having to actually reload the whole page, so it allows us to update the webpage dynamically
- This means we can asynchronously refresh sections of the page
- To get asynchronous update functionality we use a special JavaScript object called
XMLHttpRequest, this is the general process:
- Create
XMLHttpRequest object
- Define its
onreadystatechange or onload behavior, which is just the callback that will be called when the HTTP request has completed
- (optional) In this callback use the
readyState property to check status of the request
- Make async requests
- Use the
open() method to define the request
- Use the
send() to send the request
- AJAX jQuery Documentation
Web Sockets
- Web sockets allows full duplex real-time communication between client and server
- For uses cases where the request-response paradigm of a client-server is not enough and real-time communication between client and server is needed like for messaging apps, streaming...
- socket.io is the JS library to work with sockets
- The idea behind sockets is that the web client and server are constantly sending/emitting events which are broadcasted and also listening/receiving events
- The general process of using sockets is the following
- Server
- Inits websocket
socketio = SocketIO(app)
- Define routes for events emitted by the client
@socketio.on("event name")
- In case is necessary broadcast an event to notify other clients of the received event
emit("event name", data, broadcast = True)
- Client
- Connects to the websocket
var socket = io.connect(location.protocol + '//' + document.domain + ':' + location.port);
- Defines what happens when the socket connects to the server
socket.on('connect', () => {...};);
- Defines how to emit events using the configured socket, give a name to the event and the data to be emitted
socket.emit('event name', data);
- Defines what to do when receiving an event
socket.on('event name', data => {...};);
DevOps
- Devs or Developers are who create SW, they code functionality, implement features, etc.
- Ops or Operators are the ones that deploy the SW, they are in charge of knowing how to setup a system to deploy/install and make the SW work
- DevOps goal is to integrate developers and operators, so both share responsibility of building and deploying a product
- DevOps:
- Cultural philosophy
- Practices: Refers to continuous integration, delivery and deployment, Infrastructure config as code (YAML), monitor and logging
- Tools to implement devops practices
- Faster deployments
- Accelerate hoe SW development works
DevOps AWS Services
- CodeBuild: Helps to compile SW, run tests and produce software packages. Manage by a text config files. Works with CodePipeline and Jenkins tools
- CodeDeploy: Automates code deployments to instances, update your application, supports rollback and avoids downtime during deployment
- CodePipeline: Handles continuous delivery, manages the build (uses CodeBuild), test and deploy (uses CodeDeploy) of code every time there is change
- CodeStar: Allows to develop, build and deploy SW
- AWSEducate:
React
- JavaScript library for building user interfaces
- JS library that helps develop responsive web pages
- React JS was developed by Facebook and it rival is Angular JS which was developed by Google
- JS standalone is mutational because it first fetches an HTML element and then change it, the paradigm of React is abstract the mutational elements
- into data or variables (React states) and then use a
render() function to paint/draw/render the screen like in game development
Install and Setup
- Install
Node and npm
Node Allows to run JS in the systems is installednpm Node Package Manager, it is the package manager for Node
- Install
create-react-app which is a tool that help create React projects faster
npm install -g create-react-app
- Create a React project and go to
localhost:3000
~/ $ create-react-app react_app_name
~/ $ cd react_app_name
~/react_app_name/ $ npm start
JSX
- JavaScript Extension (JSX) allows us to easily mix JavaScript and HTML
- It allows to easily return HTML tags so it can be displayed properly
Component
- Fundamental building block for React, which have internal state and external prop
- State: internal information about the component, use
this.setState() to change it
- Props: external quantities handed to the component
- Contained view of the project which can be changed by altering its state and prop
- Can be nested inside each other and used across multiple files and projects
- Every component must have the
render() method
- Called when the state changes, although is called every time the state changes not everything is reloaded on the page
- The method where we decide what the user should see based on the state and props
- This edits a Virtual DOM which React uses as input to compute the most efficient way to actually update the actual web page DOM
- It returns the HTML we want to be rendered
G Suite
- A platform of all the Google applications
- Google APIs that let you interact with Google applications
- Google Apps Script
- JavaScript platform by which you can interact with the Google APIs
- Python Google API examples: [Core Python Programming Blog] ()
Tkinter
- Light-weight GUI, useful for simple GUI building
- Install:
sudo pip3 install tkinter
- General steps when creating a GUI in Tkinter
- Import Tkinter
- Create
Tk object
- Set title and geometry
- Create elements and add them to the grid
- connect elements through functions
- Start window main loop
import tkinter as tk
def on_button_clicked():
greeting = "Hello " + str(entry1.get())
display = tk.Text(master = window, height =10, width = 30)
display.grid(column = 0, row = 3)
display.insert(tk.END, greeting)
window = tk.Tk()
window.title("My App")
window.geometry("400x400)
# label
label_title = tk.Label(text = "Hello World")
label_title.grid(column = 0, row = 0)
# entry
entry1 = tk.Entry()
entry1.grid(column = 0, row = 1)
# button
button1 = tk.Button(text = "clickme", command = on_button_clicked)
button1.grid(column = 0, row = 2)
# start window
window.mainloop()
Django:
- Web server software. Its a ** Full Web Server framework**
- Django 2 is the latest version
- Django divides all its web application into projects (a project is the whole web application)
- Django is based on the idea of integrating multiple Python packages to create a website
- A Django project refers to a entire web application and it consists of one or more Django apps, each app serves a particular purpose in the project
- Django has some built-in apps
a Python package is collection of python files that are grouped together for some purpose
a __init__.py tells you that the folder storing this file is a python module
Setup a Django project
Creating a project: django-admin startproject project_name
- Creates the main python package
Creating a Django app inside a Django project:
- Make sure you have the python Django package:
python -m pip install django or pip3 install django
- Make sure you are inside the Django project folder where the
manage.py file is located
- Run:
python manage.py startapp app_name
- Edit
views.py file that was created under the app_name folder, this file contains the views the user will see when accessing a certain route.
So a common practice is to have a function for each route which must receive a request object as argument/parameter. For example it should look something like this:
from django.http import HttpResponse
from django.shortcuts import render
# Create your views here.
def index(request):
return HttpResponse("Hello World")
- Create
urls.py inside the app folder (should be something like this:.../project_name/app_name)
from django.urls import path
from . import views
urlpatterns = [
path("", views.index)
]
Link project's urls.py with app's urls.py, this means edit urls.py inside project folder, which is the global routes file.
It should look something like this
from django.conf.urls import url
from django.contrib import admin
from django.urls import include, path
urlpatterns = [
path('', include('app_name.urls')),
url(r'^admin/', admin.site.urls),
]
Run Django app: python manage.py runserver
- You may need to give a specific IP and port like
python manage.py runserver 172.17.0.15:8080 and add hostname to ALLOWED_HOSTS list in settings.py in case you receive some permission errors when accessing the site
Django migrations and models
- Migrations allows that every time we make a change to the
models.py file Django generates SQL needed to update the database tables
- If you are using the models (
models.py) in your Django app you will need to add them in the INSTALLED_APP list in the Django project settings.py and run the python manage.py makemigrations and python manage.py migrate commands
- For example, assuming we have a flights app the
models.py of this app may look something like this:
# Create your models here.
class Flight(models.Model):
origin = models.CharField(max_length = 64)
destination = models.CharField(max_length = 64)
duration = models.IntegerField()
def __str__ (self):
return f"{self.id} - {self.origin} to {self.destination}"
And for this same example the settings.py may have a line like this: INSTALLED_APPS = ['flights.apps.FlightsConfig', 'django.contrib.admin', 'django.contrib.auth', ... ]
The python manage.py makemigrations command will take the changes in the models file and generate a migration (a 000X_initial.py file) which represents the changes that need to be made to the database so the changes in the model work
~/ $ python manage.py makemigrations
Migrations for 'flights':
flights/migrations/0001_initial.py
- Create model Flight
python manage.py sqlmigrate flights 0001 will let you examine the SQL code (usually create table commands) that is needed to store the model in the database
The python manage.py migrate command will apply a migration to the database, actually creates the tables inside the database that is defined in the DATABASES variable in the settings.py file
Python Django shell example run with python manage.py shell:
Python 3.6.0 (default, Apr 9 2018, 15:01:03)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from flights.models import Flight
>>> f = Flight(origin="NYC", destination="LDN", duration=415)
>>> f.save()
>>> Flight.objects.all()
<QuerySet [<Flight: 1 - NYC to LDN>]>
>>> f = Flight.objects.first()
>>> f
<Flight: 1 - NYC to LDN>
>>> f.delete()
(1, {'flights.Flight': 1})
Django Render Template
Create a file to render, usually the html files are organized like this in a Django App: .../project_name/app_name/templates/app_name/index.html
Modify views.py to return/render an html file and use id needed.
views.py examplefrom django.http import HttpResponse
from django.shortcuts import render
# Create your views here.
def index(request):
context = {
"stuff": ["Hello", "World"] # Flight.objects.all()
}
return render(request, "app_name/index.html", context)
index.html example<!DOCTYPE HTML>
<html lang="en-US">
<head>
<title>Index</title>
</head>
<body>
<h1>My Page title</h1>
<ul>
{% for s in stuff %}
<li>
{{ s }}
</li>
{% endfor %}
</ul>
</body>
</html>
Django admin app
- It's a Django built-in app that allows us to create, add, modify and remove existing data
Django admin app Setup
- Register your models in your app
admin.py file so the Admin site can manipulate these types of data. For example it may look something like this:
from django.contrib import admin
from .models import Airport, Flight
# Register your models here.
admin.site.register(Airport)
admin.site.register(Flight)
- Create a user account with command:
python manage.py createsuperuser, you will need to provide: username, email and password (at least 8 characters)
- This creates an entry in a user table stored in the database that is managed automatically by Django admin app
- You can access the
admin app by going to the app_name/admin route to manipulate your database
Testing
pull-request let you start a conversation about a change that you intend to be included in a certain branch (usually master)
Functional Testing tests a specific functionality, test that certain function produces the expected outputs (meets the requirements it's supposed to)
- Unit tests test single functionality of a component
- Components test test component as an entity
- Integration tests test the components interacting and integrated together are working correctly
- end-to-end tests test entire flow or how user uses the application
Non-Functional Testing test reliability, security, operational functionality
Selenium
- Selenium is a browser testing tool that allows to test things that are happening inside the web browser, it uses a webdriver that allows us to use Python Code to control
what's happening in a browser, in other words pretend to be the user manipulating the page. Install:
pip install selenium
CI/CD (Continuous Integration / Continuous Delivery)
Continuous Integration: Refers to consistently and continuously integrating code and automatically testing it
- Request merging of code between different development efforts (different people and/or branches)
- This aspect can be solved using a version control tool like
git
- Automated unit testing - This aspect can be solved using a unit test framework for example
unittest in Python and an automation tool like Travis CI or Jenkins
Continuous Delivery: Refers to continuously updating code of an application, making continuous deliveries to the ultimate application a user or client use
- Continuous deployment of code - This aspect can be solved using
CI-CD flow:
- Push code to repo. E.g. push to repo in GitHub
- Host of repo notifies CI tool. E.g. GitHub notifies Travis
- CI tool pull code from repo. E.g Travis pull new code from GitHub
- CI tool runs tests. E.g Travis runs defined unit test
- CD tool notifies or triggers deployment of application in other tool. E.g. Travis sends emails or triggers deployment using AWS, Azure or other
Travis CI
Continuous integration tool that uses files in YAML format for configuration
It is GitHub-based so GitHub repos can talk to Travis directly
The basic config is usually a .travis.yaml file inside the repo with contents:
language: python
python:
- 3.6
install:
- pup install -r requirements.txt
script:
- python manage.py test
Containers
A container is an isolated environment that contains all the needed dependencies to run an application
They are not complete operating systems, they are isolated environments (containers) that have just the things we want installed,
so we define the image which is what should go into a particular container and have that same image in other systems
Docker
- Docker is a tool to create an manage containers
- A Docker image contains the instructions to create a Docker container where the application will live
- Docker creates a link between the local folder where you have your application and the Docker container where you actually run you application. In other words it copies files from the local folder to the container
- Deployment tools can use Docker images and configuration files to setup the right environment to run an application
- Docker commands:
docker-compose command to start the Docker container and run the application there. For this you will need Dockerfile and docker-compose.yml filesdocker ps check the containers running usually there is a container for each servicedocker -it <docker_container_id> -l start a shell on the container
Dockerfile is the file that defines a Docker image. For example:
FROM python:3
WORKDIR /usr/src/app
ADD requirements.txt /usr/src/app
RUN pip install -r requirements.txt
ADD . /usr/src/app
docker-compose.yaml is a file that defines all the different services that make up the application that will we run in the Docker container. For example a database
version: '3'
services:
db:
image: postgres
migration:
build: .
command: python3 manage.py migrate
volumes:
- .:/usr/src/app
depends_on:
- db
web:
build: .
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- .:/usr/src/app
ports:
- "8000:8000"
depends_on:
- db
- migration
Scalability
Vertical scaling: Refers to the process of scaling or increasing the resources of a server, so adding processing power, memory and storage are examples of vertical scaling
Horizontal scaling: Refers to the process of adding more servers that work together handling requests
Load Balancer: Refers to the system that decides how the work loads and requests are distributed between servers
Session-Aware Load Balancing: Refers to to the idea of making sure the load balancer is session-aware, knows about users session to send same user requests to the same servers, this can be implemented with sticky sessions, sessions in database, client-side session, etc.
Auto-scaling: Refers to the idea that the number of servers can scale depending upon traffic coming in
Database Partitioning: Refers to the idea of partitioning data tables to operate on more manageable units, have queries run on tables more efficiently and quickly
- Vertical Database Partitioning : refers to separating data into multiple tables to reduce number of columns in each table
- Horizontal Database Partitioning: refers to separating data into multiple tables to reduce the number of rows in a table
Database Sharding Refers to the idea of splitting a database among different servers with the possibility of gaining concurrent access
Database Replication Refer to having multiple copies of the database in different servers to distribute load between the databases
- Single-Primary Replication has a unique primary database to write/update data to it and other secondary databases to only read data from it and data on the secondary databases gets updated every time the primary database changes
- Multi-primary Replication has multiple primary databases
Caching refers to the idea of storing data in some temporary place where is easy and fast to access it at a later time
- Client-Side Caching: http response can send a cache-control header to indicate to a browser that some client-side files wont cache for a certain amount of time so it can be cached (stored) in local memory for that time
- Server-Side Caching: Connect all servers to the cache that can cache information from database
Security
References
