diff --git a/zk/Messaging_services.md b/zk/Messaging_services.md index 0d0d416..1f5a70c 100644 --- a/zk/Messaging_services.md +++ b/zk/Messaging_services.md @@ -4,21 +4,6 @@ tags: [AWS] # AWS Messaging services -## SQS - -> SQS: Simple Queue Service - -SQS is a service that allows you to send, store and receive messages between -apps and software components built in AWS, with automatic encryption. It helps -with decoupling and scaling. - -As the name indicates, its operating mode is that of a -[queue](Queue.md) data structure offering first-in, first-out -and other queue implementations. - -An example application of this would be to set up an SQS queue that receives -messages and triggers a lambda whenever a new message is added. - ## SNS > SNS: Simple Notification Service @@ -30,3 +15,5 @@ through SMS, text, push notifications and email. **SNS pushes messages out to subscribers while SQS stores the messages until someone reads them.** + +Relation between SNS and SQS - typically paired aren't they. diff --git a/zk/SQS.md b/zk/SQS.md new file mode 100644 index 0000000..657f0b7 --- /dev/null +++ b/zk/SQS.md @@ -0,0 +1,52 @@ +--- +tags: [AWS, SQS] +--- + +# AWS SQS + +SQS ("Simple Queue Service") is a service that allows you to send, store and +receive messages between apps and software components built in AWS. It helps +with decoupling and scaling. + +> Amazon SQS is a distributed queue system that enables web service applications +> to quickly and reliably queue messages that one component in the application +> generates to be consumed by another component where a queue is a temporary +> repository for messages that are awaiting processing. + +As the name indicates, its operating mode is that of a [queue](Queue.md) data +structure offering first-in, first-out and other queue implementations. + +## Example use case + +![SQS example diagram](../img/SQS-example.png) + +A request is made to an [API Gateway](API_Gateway.md) endpoint with a body. The +body is then parsed and inserted into a database. + +The benefit of adding SQS as a buffer between the request and the updating of +the database: + +- It can better handle spikes in activity, buffering requests to the database + until it is ready to handle them. This prevents the messages getting lost if + the server is overloaded + +- There is a retry mechanism built into SQS. If the database write fails, the + message stays in the queue allowing for retries + +- It facilitates decoupling. Without SQS as the middleman the responsibilites of + the lambda would be compounded - it would receive requests and update the DB, + plus any additional processes such sending a message to SNS. In the solution + we have two lambdas co-ordinating actions in a decouped manner via SQS. + +## Dead letters + +As SQS allows for multiple retries we could end up in a situation where a +malformed message is continually processed in a loop. To avoid this you can set +a maxiumum retry limit and, when this is exceeded, shift the problematic message +to the dead letter queue (DLQ) and remove it from the main queue. + +![SQS deadletter example](../img/SQS-deadletter.png) + +If the DLQ reaches a certain threshold this can trigger additional handling such +as raising an Alert in [CloudWatch](./AWS_CloudWatch.md) or other monitoring +tool.