chore: move all entries to flat file
This commit is contained in:
parent
2e69cb0d7c
commit
5e00c654b3
407 changed files with 18 additions and 705 deletions
|
@ -1,35 +0,0 @@
|
|||
---
|
||||
categories:
|
||||
- Databases
|
||||
tags: [relational-databases]
|
||||
---
|
||||
|
||||
# ACID principle
|
||||
|
||||
> A database is a collection of organised data that can be efficiently stored,
|
||||
> sorted, and searched.
|
||||
|
||||
How the data is organised will often determine the _type_ of database used.
|
||||
There are many different types of database; some examples of the different types
|
||||
are relational, object-orientated, graphical, NoSQL, and distributed. All should
|
||||
meet the principles of ACID.
|
||||
|
||||
To ensure the integrity of a database, each change or transaction must conform
|
||||
to a set of rules known as ACID:
|
||||
|
||||
- **atomicity**
|
||||
- when changing data within a database, if any part of the change fails, the
|
||||
whole change will fail and the data will remain as it was before the change
|
||||
was made; this is a safeguard that prevents partial records being created.
|
||||
- **consistency**
|
||||
- before data can be changed in a database, it must be validated against a set
|
||||
of rules
|
||||
- **isolation**
|
||||
- databases allow multiple changes at the same time, but each change is
|
||||
isolated from others
|
||||
- **durability**
|
||||
- once a change has been made, the data is safe, even in the event of system
|
||||
failure
|
||||
|
||||
> Databases will have mechanisms for **backup**, **distribution**, and
|
||||
> **redundancy**, to ensure data is not lost.
|
|
@ -1,134 +0,0 @@
|
|||
---
|
||||
categories:
|
||||
- DevOps
|
||||
- Backend
|
||||
tags: [AWS, docker]
|
||||
---
|
||||
|
||||
# Local AWS development with SAM
|
||||
|
||||
We can run a local instance of our SAM stack for a given application without
|
||||
sending requests to the cloud. This is implemented through Docker.
|
||||
|
||||
The SAM CLI handles all the Docker-related tasks, such as pulling the required
|
||||
Lambda runtime images, creating containers, mounting your code and dependencies,
|
||||
and running your Lambda functions inside those containers.
|
||||
|
||||
## Basic set up
|
||||
|
||||
Enter your project directory.
|
||||
|
||||
First build your SAM application:
|
||||
|
||||
```sh
|
||||
sam build
|
||||
```
|
||||
|
||||
We then run:
|
||||
|
||||
```sh
|
||||
sam local start-api
|
||||
```
|
||||
|
||||
If you have an API Gateway endpoint that you want to call over the local server.
|
||||
You will be able to call it after executing the above.
|
||||
|
||||
This will be indicated by:
|
||||
|
||||

|
||||
|
||||
If we want to invoke the function directly we use:
|
||||
|
||||
```sh
|
||||
sam local invoke [FunctionName]
|
||||
```
|
||||
|
||||
## Using environment variables
|
||||
|
||||
If you have an API key or database credentials, you are going to typically want
|
||||
to use different values dependent on environment.
|
||||
|
||||
Even if the values are the same accross environments, it's a good idea to not
|
||||
call a secret when working locally since this request is billable.
|
||||
|
||||
In the example below I show how to set up environment variables for an API key
|
||||
locally and in production.
|
||||
|
||||
## Create secret
|
||||
|
||||
Go to AWS SecretsManager and add the API key as a secret. This will be sourced
|
||||
in production.
|
||||
|
||||
## Create local ENV file
|
||||
|
||||
> These must be in JSON to work with SAM:
|
||||
|
||||
### Local env
|
||||
|
||||
```json
|
||||
// local-env.json
|
||||
{
|
||||
"FunctionName": {
|
||||
"API_KEY": "xxx-yyy-xxx",
|
||||
"NODE_ENV": "development"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
We save these to the root of the given function's directory not at the global
|
||||
repo level.
|
||||
|
||||
> Be sure to add this to `.gitignore` so that it does not become public
|
||||
|
||||
### Update `template.yaml`
|
||||
|
||||
Every environment variable you intend to use, must exist in the `template.yaml`,
|
||||
otherwise it will not be sourced at runtime:
|
||||
|
||||
```yaml
|
||||
...
|
||||
Resources:
|
||||
Properties:
|
||||
Environment:
|
||||
Variables:
|
||||
SECRET_ARN: "arn:aws:secretsmanager:eu-west-2:885135949562:secret:wakatime-api-key-X9oF3v",
|
||||
NODE_ENV: production
|
||||
API_KEY:
|
||||
...
|
||||
```
|
||||
|
||||
> We go ahead and populate the values for production. But we leave the variable
|
||||
> we use in development blank, since we don't want it committed and we will
|
||||
> source it at the SAM invocation. It still needs to exist though.
|
||||
|
||||
### Pass in the environment variable at invocation:
|
||||
|
||||
```sh
|
||||
sam local start-api --env-vars /home/thomas/repos/lambdas/wakatime-api/get-coding-stats/local-env.json
|
||||
```
|
||||
|
||||
In production, the variables required will be automatically sourced from the
|
||||
`template.yaml`
|
||||
|
||||
### Create handler within the Lambda itself
|
||||
|
||||
You are obviously going to need to distinguish between the different deployments
|
||||
when the Lambda executes. Here is an example helper function:
|
||||
|
||||
```ts
|
||||
import * as AWS from "aws-sdk";
|
||||
|
||||
const secretsManager = new AWS.SecretsManager();
|
||||
|
||||
async function getApiKey(): Promise<string> {
|
||||
if (process.env.NODE_ENV === "production") {
|
||||
const response = await secretsManager
|
||||
.getSecretValue({ SecretId: process.env.SECRET_ARN as string })
|
||||
.promise();
|
||||
const secretValues = JSON.parse(response.SecretString as string);
|
||||
return secretValues.API_KEY;
|
||||
} else {
|
||||
return process.env.API_KEY as string;
|
||||
}
|
||||
}
|
||||
```
|
|
@ -1,299 +0,0 @@
|
|||
---
|
||||
categories:
|
||||
- DevOps
|
||||
- Backend
|
||||
tags: [AWS]
|
||||
---
|
||||
|
||||
# AWS SAM
|
||||
|
||||
SAM stands for **serverless application model**. It is a framework developed by
|
||||
AWS to simplify the process of building, deploying and managing serverless
|
||||
applications. It provides a concise syntax for defining the components of a
|
||||
serverless application, such as
|
||||
[Lambda functions](/DevOps/AWS/AWS_Lambda/Lambda_programming_model.md),
|
||||
[API gateway](/DevOps/AWS/AWS_API_Gateway.md) and database tables.
|
||||
|
||||
The SAM infrastructure is defined in a YAML file which is then deployed to AWS.
|
||||
SAM syntax gets transformed into CloudFormation during the deployment process.
|
||||
(CloudFormation is a broader and more robust AWS tool for large, highly
|
||||
scaleable infrastructures).
|
||||
|
||||
## Key features of SAM
|
||||
|
||||
- Single deployment configuration
|
||||
- Integration with development tools
|
||||
- Local testing and debugging
|
||||
- Built on AWS CloudFormation
|
||||
|
||||
## Main technologies required
|
||||
|
||||
### Docker
|
||||
|
||||
Whilst SAM can be used to create a deployable file for AWS it can also be run as
|
||||
a container for local development with Docker.
|
||||
|
||||
### AWS CLI
|
||||
|
||||
This is installed using Python and allows you to interact directly with AWS via
|
||||
the command-line.
|
||||
|
||||
### AWS SAM CLI
|
||||
|
||||
See
|
||||
[https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html)
|
||||
|
||||
## Setting up credentials for the AWS CLI
|
||||
|
||||
You require an access key for the given
|
||||
[IAM user](/DevOps/AWS/AWS_User_management_and_roles.md#iam). You should create
|
||||
an IAM account specific to the project with bounded permissions.
|
||||
|
||||
```
|
||||
aws configure
|
||||
AWS Access Key ID [None]: AK*******
|
||||
AWS Secret Access Key [None]: ukp******
|
||||
Default region name [None]:
|
||||
Default output format [None]:
|
||||
```
|
||||
|
||||
This information can be found in the Security Credentials section of the given
|
||||
[IAM](/DevOps/AWS/AWS_User_management_and_roles.md#iam) user:
|
||||
|
||||

|
||||
|
||||
### Switching between credentials
|
||||
|
||||
You should set up a different IAM user for each project.
|
||||
|
||||
You can do this with:
|
||||
|
||||
```sh
|
||||
aws configure --profile <profile-name>
|
||||
```
|
||||
|
||||
This will then ask you to add the credentials for the user.
|
||||
|
||||
You can switch between different credentials for the user as follows:
|
||||
|
||||
```sh
|
||||
AWS_PROFILE=<profile-name> sam build
|
||||
|
||||
```
|
||||
|
||||
## Starting a SAM project
|
||||
|
||||
First create a directory for your project which will serve as the repository:
|
||||
|
||||
```sh
|
||||
mkdir aws-sam-learning
|
||||
cd aws-sam-learning
|
||||
```
|
||||
|
||||
Then we can use the `sam` cli to bootstrap the project:
|
||||
|
||||
```sh
|
||||
sam init --runtime nodejs16.x
|
||||
```
|
||||
|
||||
We can just click through and accept the basic HelloWorld Lambda.
|
||||
|
||||
This will create the Lambda as well as an API Gateway trigger URL.
|
||||
|
||||
### `template.yaml`
|
||||
|
||||
This is autogenerated and details the main constituents of the project. There
|
||||
are lots of fields but the most important are the following:
|
||||
|
||||
```yaml
|
||||
HelloWorldFunction:
|
||||
Type: AWS::Serverless::Function
|
||||
Properties:
|
||||
CodeUri: hello-world/
|
||||
Handler: app.lambdaHandler
|
||||
Runtime: nodejs16.x
|
||||
Architectures:
|
||||
- x86_64
|
||||
Events:
|
||||
HelloWorld:
|
||||
Type: Api
|
||||
Properties:
|
||||
Path: /hello
|
||||
Method: get
|
||||
```
|
||||
|
||||
This details the location of the
|
||||
[handler function](/DevOps/AWS/AWS_Lambda/Lambda_handler_function.md) which is
|
||||
contained at the path `hello-world/app.js`:
|
||||
|
||||
```js
|
||||
exports.lambdaHandler = async (event, context) => {
|
||||
try {
|
||||
// const ret = await axios(url);
|
||||
response = {
|
||||
statusCode: 200,
|
||||
body: JSON.stringify({
|
||||
message: "hello world",
|
||||
// location: ret.data.trim()
|
||||
}),
|
||||
};
|
||||
} catch (err) {
|
||||
console.log(err);
|
||||
return err;
|
||||
}
|
||||
|
||||
return response;
|
||||
};
|
||||
```
|
||||
|
||||
It also lists the `get` event that we can use to call API Gateway and trigger
|
||||
the Lambda.
|
||||
|
||||
The full template is below:
|
||||
|
||||

|
||||
|
||||
## Adding our own code
|
||||
|
||||
We will create our own function and API Gateway trigger.
|
||||
|
||||
We will place our function after the existing `HelloWorldFunction`
|
||||
|
||||
```yaml
|
||||
ClockFunction:
|
||||
Type: AWS::Serverless::Function
|
||||
Properties:
|
||||
CodeUri: clock/
|
||||
Handler: handler.clock
|
||||
Runtime: nodejs16.x
|
||||
Events:
|
||||
ClockApi:
|
||||
Type: Api
|
||||
Properties:
|
||||
Path: /clock
|
||||
Method: get
|
||||
```
|
||||
|
||||
We can test the syntax with:
|
||||
|
||||
```sh
|
||||
sam validate
|
||||
```
|
||||
|
||||
Just like with `HelloWorld`, we will create a directory for this function:
|
||||
`clock` and we will initialise it as an `npm` project.
|
||||
|
||||
```sh
|
||||
mkdir clock
|
||||
cd clock
|
||||
npm init
|
||||
```
|
||||
|
||||
We will use `handler.js` as our root, handler function.
|
||||
|
||||
We have said in the template file that our `Handler: handler.clock`, therefore
|
||||
the main function in the `handler` module should be `clock`:
|
||||
|
||||
```js
|
||||
const moment = require("moment");
|
||||
|
||||
exports.clock = async (event) => {
|
||||
console.log("Clock function run");
|
||||
const message = moment().format();
|
||||
const response = {
|
||||
statusCode: 200,
|
||||
body: JSON.stringify(message),
|
||||
};
|
||||
return response;
|
||||
};
|
||||
```
|
||||
|
||||
The directory structure is as follows:
|
||||
|
||||

|
||||
|
||||
When we call the API Gateway path `/clock` with `GET`, our function will be
|
||||
triggered.
|
||||
|
||||
## Deploying the project
|
||||
|
||||
We will now deploy our project to AWS from the local environment.
|
||||
|
||||
The process is as follows:
|
||||
|
||||
1. Build
|
||||
2. Package
|
||||
3. Deploy
|
||||
|
||||
### Build
|
||||
|
||||
We need to install the runtime dependencies for the function. We do this by
|
||||
running `sam build`. This ignores test files and development dependencies and
|
||||
installs the project dependencies and source files to a temporary subdirectory.
|
||||
|
||||

|
||||
|
||||
The build directory is `.aws-sam/build/`. There will be a subdirectory for each
|
||||
of our files.
|
||||
|
||||
### Package
|
||||
|
||||
As noted, CloudFront handles the deployment of the application. It can only
|
||||
receive one file as an input. The packaging process consists in creating that
|
||||
single file.
|
||||
|
||||
The packaging proces will first archive all of the project artefacts into a zip
|
||||
file and then upload that to [S3](/DevOps/AWS/AWS_S3.md). A reference to this S3
|
||||
entity is then provided to CloudFormation.
|
||||
|
||||

|
||||
|
||||
The command is as follows:
|
||||
|
||||
```sh
|
||||
sam package
|
||||
--template-file template.yaml
|
||||
--output-template-file pkg.yml
|
||||
--region eu-west-1
|
||||
```
|
||||
|
||||
This will automatically create a hashed bucket name for you in S3 (I have tried
|
||||
to add my own naming but it doesn't comply.)
|
||||
|
||||
### Local development with Docker
|
||||
|
||||
In order to work with your application locally without actually sending requests
|
||||
to AWS and using credit, you can run a local instance.
|
||||
|
||||
See
|
||||
[Local AWS Development with SAM](/DevOps/AWS/SAM/Local_AWS_development_with_SAM.md).
|
||||
|
||||
### Deploy
|
||||
|
||||
Once you have packaged the app you can deploy with `sam deploy --guided`. This
|
||||
will talk you through the defaults and will deploy the package to
|
||||
CloudFormation. In CloudFormation each individual project is called a **stack**.
|
||||
|
||||
If we then go to Cloud Formation we will see the deployed application.
|
||||
|
||||

|
||||
|
||||
## Call the endpoint
|
||||
|
||||
If we now go to the Lambda console, we will see our function listed, and the API
|
||||
Gateway endpoint under `triggers`:
|
||||
|
||||

|
||||
|
||||
We can then call this from Postman to check everything is working as it should:
|
||||
|
||||

|
||||
|
||||
## Clean up and erase the stack
|
||||
|
||||
We can delete the stack and remove all the resources we have created with a
|
||||
single CLI method:
|
||||
|
||||
```sh
|
||||
aws cloudformation delete-stack --stack-name <name> --region <region>
|
||||
```
|
|
@ -1,156 +0,0 @@
|
|||
---
|
||||
categories:
|
||||
- Programming Languages
|
||||
tags:
|
||||
- javascript
|
||||
- react
|
||||
- react-classes
|
||||
---
|
||||
|
||||
# Managing forms with class components
|
||||
|
||||
We are going to create a simple form that takes two integers from input fields
|
||||
and returns their sum. Here is the raw component:
|
||||
|
||||
```jsx
|
||||
class Form extends React.Component {
|
||||
render() {
|
||||
return (
|
||||
<form>
|
||||
<input type="number" placeholder="Please enter an integer" required />
|
||||
<input type="number" placeholder="Please enter an integer" required />
|
||||
<button>Calculate sum</button>
|
||||
<output></output>
|
||||
</form>
|
||||
);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Adding handlers
|
||||
|
||||
```jsx
|
||||
class Form extends React.Component {
|
||||
handleSubmit = (event) => {
|
||||
event.preventDefault();
|
||||
// Specific state change on submit
|
||||
};
|
||||
render() {
|
||||
return (
|
||||
<form onSubmit={this.handleSubmit}>
|
||||
<input type="number" placeholder="Please enter an integer" required />
|
||||
<input type="number" placeholder="Please enter an integer" required />
|
||||
<button>Calculate sum</button>
|
||||
<output></output>
|
||||
</form>
|
||||
);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- The overall handling of the form is going to execute on the submission of the
|
||||
form. So we create an `onSubmit` event within the form body and tie this to a
|
||||
handling function (`handleSubmit`) on the class.
|
||||
|
||||
- We follow the custom of generic naming of event methods with `handle[event]` ,
|
||||
just like we use `handleClick` with `onClick` events
|
||||
- The `event` object is a React wrapper for the standard DOM event. We operate
|
||||
on this object when managing the state of forms using React.
|
||||
- `preventDefault` is just the React method of the traditional prevent default
|
||||
method that we can apply to forms. It stops the page updating when the submit
|
||||
button is clicked.
|
||||
|
||||
## Capturing input values through state change
|
||||
|
||||
In order to retrieve the values that the user enters we need to introduce state
|
||||
to the form component. This is managed using the `onChange` event. Even though
|
||||
the overall form action should be managed through `onSubmit` we need to
|
||||
initialize state on the inputs we intend to use when the form is submitted. This
|
||||
enshrines the React principle of **lifting state up**. As the `input` elements
|
||||
are constituents of the overall `form` component, their state should be handled
|
||||
by this common parent.
|
||||
|
||||
### Initializing `input` state with `onChange` events
|
||||
|
||||
To begin with let's just look at how we would do this in the abstract with a
|
||||
single `input` before applying it to the two inputs in our example:
|
||||
|
||||
```html
|
||||
<input
|
||||
type="number"
|
||||
placeholder="Please enter an integer"
|
||||
value="this.state.integerFirst"
|
||||
onChange="this.handleChange"
|
||||
required
|
||||
></input>
|
||||
```
|
||||
|
||||
We tie the state parameters in the `input` element to a `handleChange` function
|
||||
on the parent component but we first need to add the starting state (using
|
||||
`this.state` on the `Form` components' constructor:
|
||||
|
||||
```jsx
|
||||
class Form extends React.Component {
|
||||
constuctor(props) {
|
||||
super(props);
|
||||
this.state = {
|
||||
integerFirst: " "
|
||||
};
|
||||
}
|
||||
handleChange = (event) => {
|
||||
this.setState(
|
||||
{
|
||||
integerFirst: event.target.value
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Now the `Form` component is kept in the loop. Whenever the user alters the
|
||||
`input` field, this change will be logged as a state change within React. This
|
||||
will allow us to retrieve the values that the inputs have when the submit event
|
||||
fires.
|
||||
|
||||
## Handling multiple inputs
|
||||
|
||||
It would be inefficient to have a change handler for every input in a form with
|
||||
many inputs. In this scenario it is better to use destructuring to capture all
|
||||
the values:
|
||||
|
||||
```jsx
|
||||
class Form extends React.Component {
|
||||
constuctor(props) {
|
||||
super(props);
|
||||
this.state = {
|
||||
integerFirst: "",
|
||||
integerSecond: ""
|
||||
};
|
||||
}
|
||||
|
||||
returnSum = (x,y) => x + y;
|
||||
|
||||
handleSubmit = (event) => {
|
||||
event.preventDefault()
|
||||
this.setState({
|
||||
outputSum: this.returnSum(
|
||||
Number(this.state.integerFirst), Number(this.state.integerSecond)
|
||||
)
|
||||
});
|
||||
|
||||
handleChange = (event) => {
|
||||
this.setState(
|
||||
{
|
||||
[event.targe.name]: value
|
||||
}
|
||||
)}
|
||||
render() {
|
||||
return(
|
||||
<form onSubmit={this.handleSubmit}>
|
||||
<input type="number" name="integerFirst" placeholder="Please enter an integer" onChange="this.handleChange" required />
|
||||
<input type="number" name="integerSecond" placeholder="Please enter an integer" onChange="this.handleChange" required />
|
||||
<button>Calculate sum</button>
|
||||
<output></output>
|
||||
</form>
|
||||
)
|
||||
}};
|
||||
|
||||
```
|
|
@ -1,77 +0,0 @@
|
|||
---
|
||||
categories:
|
||||
- Programming Languages
|
||||
tags:
|
||||
- typescript
|
||||
---
|
||||
|
||||
# Functions
|
||||
|
||||
## Basic typing within a function: arguments and return values
|
||||
|
||||
With functions we can apply types to the return value, the parameters and any
|
||||
values that are included within the function body.
|
||||
|
||||
```ts
|
||||
function search(query: string, tags: string[]): string {}
|
||||
```
|
||||
|
||||
We can also specify optional parameters with use of the `?` symbol:
|
||||
|
||||
```ts
|
||||
function search(query: string, tags?: string[]): string {}
|
||||
```
|
||||
|
||||
### Utilising custom types
|
||||
|
||||
Whilst we can use standard JS types with the parameters and return value, the
|
||||
real benefit comes when you use custom types. For instance we can specify that
|
||||
an object passed to a function must match the shape of a custom type or
|
||||
interface. Similarly we can ensure that for functions that return objects, the
|
||||
object that is returned must satisfy the shape of the custom object.
|
||||
|
||||
```ts
|
||||
async function getContributorData(
|
||||
contributorName: string
|
||||
): Promise<IContributor> {}
|
||||
```
|
||||
|
||||
For example, this function has a return signature which indicates that it will
|
||||
return a promise matching a type of shape `IContributor`
|
||||
|
||||
## Functions as types
|
||||
|
||||
As well as typing the values that a function receives and returns, you can type
|
||||
the function itself. **This is most useful when you are using higher-order
|
||||
functions and passing functions as parameters to another function.** In these
|
||||
scenarios you will want to type the function that is being passed as a
|
||||
parameter. There are several ways to do this. We'll use the following basic
|
||||
function as our demonstration:
|
||||
|
||||
```ts
|
||||
function higherOrderFunction(integer: number, addFunction: any): number {
|
||||
return addFunction(integer);
|
||||
}
|
||||
```
|
||||
|
||||
### Use `typeof`
|
||||
|
||||
```ts
|
||||
// Declare an adding function
|
||||
const addTwo = (int: number) => int + 2;
|
||||
|
||||
// Apply it:
|
||||
higherOrderFunction(3, addTwo);
|
||||
|
||||
// We can now define the higher-order function with a specific type:
|
||||
|
||||
function higherOrderFunction(
|
||||
integer: number,
|
||||
addFunction: typeof addTwo
|
||||
): number {
|
||||
return addFunction(integer);
|
||||
}
|
||||
```
|
||||
|
||||
This way we just use the native `typeof` keyword to assert that any call of
|
||||
`hoFunc` should pass a function of the type `addTwo`
|
File diff suppressed because one or more lines are too long
Before Width: | Height: | Size: 13 KiB |
15
_scripts/flatten_directories.sh
Executable file
15
_scripts/flatten_directories.sh
Executable file
|
@ -0,0 +1,15 @@
|
|||
#!/bin/bash
|
||||
directories_to_parse="../Computer_Architecture ../Databases ../Electronics_and_Hardware ../Operating_Systems ../Programming_Languages ../DevOps"
|
||||
|
||||
directory="/home/thomas/repos/eolas-bak"
|
||||
|
||||
mapfile -t directories < <(find "$directory" -type d)
|
||||
file_matches=()
|
||||
|
||||
# Return array of all files belonging to source dirs...
|
||||
for ele in ${directories[@]}; do
|
||||
file_matches+=( $(find $ele -name "*.md" -type f) )
|
||||
done
|
||||
|
||||
echo ${file_matches[1]}
|
||||
|
|
@ -6,3 +6,6 @@ find /home/thomas/repos/eolas/ -type f -name "*.md" | while
|
|||
read file; do
|
||||
sed -i 's/\/img\//\/_img\//g' $file
|
||||
done
|
||||
|
||||
|
||||
|
||||
|
|
Loading…
Add table
Reference in a new issue