thomasabishop/eolas
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

aws: add missing entries

Browse source
This commit is contained in:
thomasabishop 2024-02-25 16:26:09 +00:00
parent 86a79ef0a4
commit 7ebe6dcd5d
3 changed files with 449 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

24
zk/AWS_CLI.md Normal file
View file

@ -0,0 +1,24 @@
---
tags: [AWS]
---
# SAM frequent commands
### Retrieve current user
```
aws-sts get-caller-identity
```
### List users
```
aws configure list
aws configure list-profiles
```
### View profile data
```
vim ./aws/credentials
```

131
zk/Local_AWS_development_with_SAM.md Normal file
View file

@ -0,0 +1,131 @@
---
tags: [AWS, docker]
---
# Local AWS development with SAM
We can run a local instance of our SAM stack for a given application without
sending requests to the cloud. This is implemented through Docker.
The SAM CLI handles all the Docker-related tasks, such as pulling the required
Lambda runtime images, creating containers, mounting your code and dependencies,
and running your Lambda functions inside those containers.
## Basic set up
Enter your project directory.
First build your SAM application:
```sh
sam build
```
We then run:
```sh
sam local start-api
```
If you have an API Gateway endpoint that you want to call over the local server.
You will be able to call it after executing the above.
This will be indicated by:
![](/img/local-sam-docker.png)
If we want to invoke the function directly we use:
```sh
sam local invoke [FunctionName]
```
## Using environment variables
If you have an API key or database credentials, you are going to typically want
to use different values dependent on environment.
Even if the values are the same accross environments, it's a good idea to not
call a secret when working locally since this request is billable.
In the example below I show how to set up environment variables for an API key
locally and in production.
## Create secret
Go to AWS SecretsManager and add the API key as a secret. This will be sourced
in production.
## Create local ENV file
> These must be in JSON to work with SAM:
### Local env
```json
// local-env.json
{
"FunctionName": {
"API_KEY": "xxx-yyy-xxx",
"NODE_ENV": "development"
}
}
```
We save these to the root of the given function's directory not at the global
repo level.
> Be sure to add this to `.gitignore` so that it does not become public
### Update `template.yaml`
Every environment variable you intend to use, must exist in the `template.yaml`,
otherwise it will not be sourced at runtime:
```yaml
...
Resources:
Properties:
Environment:
Variables:
SECRET_ARN: "arn:aws:secretsmanager:eu-west-2:885135949562:secret:wakatime-api-key-X9oF3v",
NODE_ENV: production
API_KEY:
...
```
> We go ahead and populate the values for production. But we leave the variable
> we use in development blank, since we don't want it committed and we will
> source it at the SAM invocation. It still needs to exist though.
### Pass in the environment variable at invocation:
```sh
sam local start-api --env-vars /home/thomas/repos/lambdas/wakatime-api/get-coding-stats/local-env.json
```
In production, the variables required will be automatically sourced from the
`template.yaml`
### Create handler within the Lambda itself
You are obviously going to need to distinguish between the different deployments
when the Lambda executes. Here is an example helper function:
```ts
import * as AWS from "aws-sdk";
const secretsManager = new AWS.SecretsManager();
async function getApiKey(): Promise<string> {
if (process.env.NODE_ENV === "production") {
const response = await secretsManager
.getSecretValue({ SecretId: process.env.SECRET_ARN as string })
.promise();
const secretValues = JSON.parse(response.SecretString as string);
return secretValues.API_KEY;
} else {
return process.env.API_KEY as string;
}
}
```

294
zk/SAM.md Normal file
View file

@ -0,0 +1,294 @@
---
tags: [AWS]
---
# AWS SAM
SAM stands for **serverless application model**. It is a framework developed by
AWS to simplify the process of building, deploying and managing serverless
applications. It provides a concise syntax for defining the components of a
serverless application, such as
[Lambda functions](zk/Lambda_programming_model.md),
[API gateway](/zk/AWS_API_Gateway.md) and database tables.
The SAM infrastructure is defined in a YAML file which is then deployed to AWS.
SAM syntax gets transformed into CloudFormation during the deployment process.
(CloudFormation is a broader and more robust AWS tool for large, highly
scaleable infrastructures).
## Key features of SAM
- Single deployment configuration
- Integration with development tools
- Local testing and debugging
- Built on AWS CloudFormation
## Main technologies required
### Docker
Whilst SAM can be used to create a deployable file for AWS it can also be run as
a container for local development with Docker.
### AWS CLI
This is installed using Python and allows you to interact directly with AWS via
the command-line.
### AWS SAM CLI
See
[https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html)
## Setting up credentials for the AWS CLI
You require an access key for the given
[IAM user](zk/AWS_User_management_and_roles.md#iam). You should create an IAM
account specific to the project with bounded permissions.
```
aws configure
AWS Access Key ID [None]: AK*******
AWS Secret Access Key [None]: ukp******
Default region name [None]:
Default output format [None]:
```
This information can be found in the Security Credentials section of the given
[IAM](zk/AWS_User_management_and_roles.md#iam) user:
![](/img/access-key-aws.png)
### Switching between credentials
You should set up a different IAM user for each project.
You can do this with:
```sh
aws configure --profile <profile-name>
```
This will then ask you to add the credentials for the user.
You can switch between different credentials for the user as follows:
```sh
AWS_PROFILE=<profile-name> sam build
```
## Starting a SAM project
First create a directory for your project which will serve as the repository:
```sh
mkdir aws-sam-learning
cd aws-sam-learning
```
Then we can use the `sam` cli to bootstrap the project:
```sh
sam init --runtime nodejs16.x
```
We can just click through and accept the basic HelloWorld Lambda.
This will create the Lambda as well as an API Gateway trigger URL.
### `template.yaml`
This is autogenerated and details the main constituents of the project. There
are lots of fields but the most important are the following:
```yaml
HelloWorldFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: hello-world/
Handler: app.lambdaHandler
Runtime: nodejs16.x
Architectures:
- x86_64
Events:
HelloWorld:
Type: Api
Properties:
Path: /hello
Method: get
```
This details the location of the [handler function](/Lambda_handler_function.md)
which is contained at the path `hello-world/app.js`:
```js
exports.lambdaHandler = async (event, context) => {
try {
// const ret = await axios(url);
response = {
statusCode: 200,
body: JSON.stringify({
message: "hello world",
// location: ret.data.trim()
}),
};
} catch (err) {
console.log(err);
return err;
}
return response;
};
```
It also lists the `get` event that we can use to call API Gateway and trigger
the Lambda.
The full template is below:
![](/img/sam-template-yaml.png)
## Adding our own code
We will create our own function and API Gateway trigger.
We will place our function after the existing `HelloWorldFunction`
```yaml
ClockFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: clock/
Handler: handler.clock
Runtime: nodejs16.x
Events:
ClockApi:
Type: Api
Properties:
Path: /clock
Method: get
```
We can test the syntax with:
```sh
sam validate
```
Just like with `HelloWorld`, we will create a directory for this function:
`clock` and we will initialise it as an `npm` project.
```sh
mkdir clock
cd clock
npm init
```
We will use `handler.js` as our root, handler function.
We have said in the template file that our `Handler: handler.clock`, therefore
the main function in the `handler` module should be `clock`:
```js
const moment = require("moment");
exports.clock = async (event) => {
console.log("Clock function run");
const message = moment().format();
const response = {
statusCode: 200,
body: JSON.stringify(message),
};
return response;
};
```
The directory structure is as follows:
![](/_img/sam-directory.png)
When we call the API Gateway path `/clock` with `GET`, our function will be
triggered.
## Deploying the project
We will now deploy our project to AWS from the local environment.
The process is as follows:
1. Build
2. Package
3. Deploy
### Build
We need to install the runtime dependencies for the function. We do this by
running `sam build`. This ignores test files and development dependencies and
installs the project dependencies and source files to a temporary subdirectory.
![](/img/sam-build.png)
The build directory is `.aws-sam/build/`. There will be a subdirectory for each
of our files.
### Package
As noted, CloudFront handles the deployment of the application. It can only
receive one file as an input. The packaging process consists in creating that
single file.
The packaging proces will first archive all of the project artefacts into a zip
file and then upload that to [S3](zk/AWS_S3.md). A reference to this S3 entity
is then provided to CloudFormation.
![](/img/s3-package-again.svg)
The command is as follows:
```sh
sam package
--template-file template.yaml
--output-template-file pkg.yml
--region eu-west-1
```
This will automatically create a hashed bucket name for you in S3 (I have tried
to add my own naming but it doesn't comply.)
### Local development with Docker
In order to work with your application locally without actually sending requests
to AWS and using credit, you can run a local instance.
See [Local AWS Development with SAM](zk/Local_AWS_development_with_SAM.md).
### Deploy
Once you have packaged the app you can deploy with `sam deploy --guided`. This
will talk you through the defaults and will deploy the package to
CloudFormation. In CloudFormation each individual project is called a **stack**.
If we then go to Cloud Formation we will see the deployed application.
![](/img/cloud-formation-stack.png)
## Call the endpoint
If we now go to the Lambda console, we will see our function listed, and the API
Gateway endpoint under `triggers`:
![](/img/gateway-trigger.png)
We can then call this from Postman to check everything is working as it should:
![](/img/postman-aws-output.png)
## Clean up and erase the stack
We can delete the stack and remove all the resources we have created with a
single CLI method:
```sh
aws cloudformation delete-stack --stack-name <name> --region <region>
```