Amazon Web Services

Warning: This blogpost has been posted over two years ago. That is a long time in development-world! The story here may not be relevant, complete or secure. Code might not be complete or obsoleted, and even my current vision might have (completely) changed on the subject. So please do read further, but use it with caution.
Posted on 20 May 2020
Tagged with: [ aws ]  [ amazon

More often than not, I’m using Amazon Web Services (AWS) as my “cloud”. Not only for my own projects, but almost all customers I’m working for use Amazon for hosting their applications. So over time you build up a lot of experience on AWS service: you know how to (correctly) setup VPC’s, know when to you ECS, EC2 or lambda to host code and even services like S3, SNS and SQS pose no challenges anymore.

But there are a lot of AWS services available. And I do mean: a LOT. Currently, there are 163 (!) different services that are available from the Amazon Dashboard, each with their own way of working, difficulties, catches and best practises.

Discovering AWS

You might realise that it’s probably near impossible to dive into each service and completely understand how they work and most likely, you don’t really need to know the exact ins and outs. But, having a basic understanding on each service can be a major benefit as a developer, architect or administrator. It makes it easier to see if there is an already existing solution for your problem at hand.

So, I dove into each and every service to figure out what it exactly was for and how it works in the basics. I tried to experiment with as many components as possible (time and money permitting, I didn’t want to spend 15.000$ on AWS Data Exchange). I tried to capture what the service does in a single one-liner to give you a global overview.

I think most of them are correct enough but if you have any suggestions or corrections, please tell me!

Compute  
EC2 Virtual Private Servers
Lightsail Amazon’s hosting provider (vps, dns, storage)
Lambda Functions you can run, written in Python, NodeJS, Go etc. Can run many in parallel.
Batch Run software jobs in Docker containers on EC2 machines
Elastic Beanstalk Run software on managed virtual machines
Serverless Application Repository Repository of serverless applications that you can deploy (on lambda)
AWS Outposts Run Amazon services in your own data center
EC2 Image Builder Create EC2 (ami?) images automatically
Storage  
S3 File / object storage. Not primarily used for mounting as filesystem, but you can directly download files through HTTP.
EFS NFS. Mount network disks to your machines.
FSx Windows / Lustre filesystems you can connect to your ec2 machines
S3 Glacier Low cost storage system for backups and archives and such
Storage Gateway iSCSI so you can connect s3 to your own (remote) machine.
AWS Backup Automatically create backups of different AWS service (ec2, rds etc)
Database  
RDS Managed mysql, postgres databases etc.
DynamoDB Large & scalable non-relational database
ElastiCache Managed memcache and redis machines
Neptune Graph database
Amazon Redshift Warehousing. Store lots of data that can be processed through streams.
Amazon QLDB Database for immutable and cryptographically verifiable data (money transactions etc)
Amazon DocumentDB MongoDB clone (but not really compatible anymore)
Amazon Keyspaces Managed Apache Cassandra clone
Migration & Transfer  
AS Migration Hub Migrate things from your DC to AWS
Application Discovery Service Discover services in your datacenter
Database Migration Service Migrate databases to RDS while staying online (can convert structures as well)
Server Migration Service Migrate virtual machines to amazon.
AWS Transfer Family (s)FTP service with S3 backend. Upload to FTP, directly store on S3 bucket.
Snowball Get a machine from AWS, plug in your DC, transfer data fast to AWS, return machine
DataSync Sync data between your datacenter and AWS
Networking & Content Delivery  
VPC Create your own virtual private network within AWS.
CloudFront Content Delivery Network.
Route 53 Manage domain names and records.
API Gateway Create HTTP APIs and let them connect to different backends.
Direct Connect Create a (physical) connection between you (or DC) to AWS.
AWS App Mesh Automatically run Envoy as a sidecar for your containers (ECS or EKS).
AWS Cloud Map Service discovery for your containers.
Global Accelerator Run your app on edge locations so they are closer to your customers (CDN for apps).
Developer Tools  
CodeStar Quickly develop applications by using template code and codecommit, codebuild etc
CodeCommit Amazon source repositories (git repo’s etc)
CodeBuild CI service
CodeDeploy Deployment service
CodePipeline Code delivery with workflows
Cloud9 Online IDE
X-Ray Allows tracing in your applications, supports Python, NodeJs, Go etc.
Robotics  
AWS RoboMaker Cloud solution for robotic developers to simulate, test and securely deploy robotic applications
Customer Enablement  
AWS IQ Job board: Hire AWS experts for whatever you need.
Support AWS support center
Managed Services Let AWS handle your AWS services for you.
Blockchain  
Amazon Managed Blockchain Block chains
Satellite  
Ground Station Timeshare radios and large antennas pointed at space
Quantum Technologies  
Amazon Braket Some quantum thing. It’s in preview so I have no idea what it is.
Management & Governance  
AWS Organizations Configure (sub)organisations and accounts
CloudWatch Logging from various AWS components
AWS Auto Scaling Scale resources based on your custom inputs and rules
CloudFormation Templates to create and configure AWS components (think terraform/sls)
CloudTrail Figure out who did what in your AWS services
Config Audit the configurations of your AWS resources
OpsWorks Use Ansible to automate stuff
Service Catalog Manage list of items/codes etc you have in the cloud
Systems Manager View data from your resources grouped in ways you like (like application specific etc)
AWS AppConfig Store and publish application configuration data
Trusted Advisor Checks your account for issues (costs, performance, security etc)
Control Tower Manage multi-accounts
AWS License Manager Manage licenses
AWS Well-Architected Tool Generate questionnaires about your architecture to see if you follow best practices
Personal Health Dashboard StatusPage for AWS
AWS Chatbot Connect AWS to slack
Launch Wizard Deploy MSSQL or SAP
AWS Compute Optimizer Finds your resources and advices on how to save costs
Media Services  
Elastic Transcoder Encode files from S3 into different other formats and store back at S3
Kinesis Video Streams Capture media streams
MediaConnect ?
MediaConvert Convert media into different formats
MediaLive Share live video with many others
MediaPackage ?
MediaStore ?
MediaTailor Insert advertisements into your broadcasts
Elemental Appliances & Software create videos on-premise. Basically a mix of all of the above services. Seems expensive. Probably is.
Machine Learning  
Amazon SageMaker Machine learning tools
Amazon CodeGuru Profile java code with machine learning
Amazon Comprehend Understand and classify data like emails, tweets etc
Amazon Forecast Create forecasts from data
Amazon Fraud Detector in preview so no idea.
Amazon Kendra Search service where you can ask questions
Amazon Lex Create voice and chatbots
Amazon Machine Learning Deprecated. Use SageMaker instead.
Amazon Personalize Create personalized recommendations based on data (mahout??)
Amazon Polly Convert text to speech in different languages
Amazon Rekognition Recognize objects and people in images
Amazon Textract Convert text found in images to text (OCR)
Amazon Transcribe Convert audio to text
Amazon Translate Translates text from one language to another
AWS DeepLens A video camera that does machine learning
AWS DeepRacer Some kind of game where you program a racecar to race against others.
Amazon Augmented AI Let humans in the loop to make AI learn things better
AWS DeepComposer Computer generated music. It’s as horrible as it sounds.
Analytics  
Athena Query data stored in s3 buckets.
EMR Elastic Map/Reduce
CloudSearch AWS version of managed document search system (like elasticsearch)
Elasticsearch Service Elasticsearch as a service
Kinesis Collect massive amount of data so you can do analytics (like ELK?)
QuickSight Business Intelligence service
Data Pipeline Move and transform data to dynamodb, rds, s3 etc.
AWS Data Exchange Find APIs which data you can consume, which can be very expensive
AWS Glue ETL service. Enrich, validate data.
AWS Lake Formation Create data lakes
MSK Kafka as a service
Security, Identity, & Compliance  
IAM AWS’s permission system that can control users and AWS services.
Resource Access Manager Share certain AWS resources like Route53, licenses, ec2 with other accounts.
Cognito User and password management system. Useful for managing users for your applications.
Secrets Manager Secrets key/value store. Can automatically rotate secrets.
GuardDuty Automatically scan your cloudtrail/vpc logs for threats.
Inspector Automatically find (security) issues in your network and machines.
Amazon Macie Analyzes data in your S3 buckets and check for PII data.
AWS Single Sign-On Allow single-sign on to your applications.
Certificate Manager Manage and even create (free) SSL certificates.
Key Management Service Manage secret keys
CloudHSM Hardware security modules. Allows you to generate and operate on cryptographic keys.
Directory Service Active directory as a service
WAF & Shield Web Application Firewall (for loadbalancers, cloudfront, api gateway). Can setup your own rules or use predefined ones
AWS Firewall Manager Firewall manager for different accounts in your organisation
Artifact Documents for cloud compliance (things like 27001 certification etc)
Security Hub Overall security checker that uses guardduty, inspector, macie etc
Detective Log security issues found (from security hub etc)
Mobile  
AWS Amplify Let AWS automatically generate frontend & backend apps and deploy them automatically.
Mobile Hub Part of AWS Amplify now.
AWS AppSync Create API backends that you can connect to. Can be created through AWS Amplify as well.
Device Farm AWS BrowserStack. Automatically test apps on many different mobile devices and browsers.
AR & VR  
Amazon Sumerian 3D Editor and Engine for rapidly prototyping AR/VR and 3D experiences
Application Integration  
Step Functions State machines written in amazon’s own language
Amazon AppFlow Automatically connects apps together (zapier?). For instance: slack to s3 buckets.
Amazon EventBridge Some kind of eventbus system
Amazon MQ ActiveMQ
Simple Notification Service Notification system that can notify through email, api endpoints, sms etc.
Simple Queue Service Message queue system
SWF Create workflows.
AWS Cost Management  
AWS Cost Explorer Gives an overview and projection of your budgets
AWS Budgets Create budgets for your AWS components
AWS Marketplace Subscriptions Find (and buy) AMI’s with software installed
Customer Engagement  
Amazon Connect AWS call center platform
Pinpoint Create transactional emails, SMS or voice calls based on templates.
Simple Email Service Send out emails. Email provider.
Business Applications  
Alexa for Business Connect Alexa to your business needs.
Amazon Chime AWS version of Zoom.
WorkMail AWS version of Gmail / Calendar.
End User Computing  
WorkSpaces Virtual desktops from Windows or Linux.
AppStream 2.0 Stream applications running native onto your browser
WorkDocs Store your documents and manage them online.
WorkLink Connect mobile users to your intranet.
Internet Of Things  
IoT Core Manage fleets of IOT devices through MQTT broker
FreeRTOS RTOS operating system for microcontrollers to automatically connect to IOT-Core or greengrass.
IoT 1-Click Manage 1-click buttons that can be connected to other systems like Lambda
IoT Analytics Clean up and save messages from topics into a data-store for analytics
IoT Device Defender Detect unwanted issues on your devices and take actions
IoT Device Management Organize IoT devices into groups, schedule jobs on the devices and configure remote access
IoT Events Monitor telemetry from devices and then trigger other AWS services or jobs on the devices themselves
IoT Greengrass A message broker can buffer messages for groups of up to 200 devices which can communicate and process data locally if connectivity to IoT Core is intermittent.
IoT SiteWise Collect, organize, analyze and visualize data from industrial equipment at scale
IoT Things Graph Cloudformation-like designer for graphing how devices should communicate with other AWS services
Game Development  
Amazon GameLift Deploy game servers with low latency on AWS
Containers  
Elastic Container Registry Store docker images like on DockerHub
Elastic Container Service Run containers, either on your own EC2 machines, or on managed machines called Fargate.
Elastic Kubernetes Service Kubernetes as a service

Thanks to Brian Thomas Smith for filling in the blanks on IoT. Thanks to all the others from #HN who suggested changes and updates on the different services.

Conclusion

With over 150 services running on Amazon AWS, it’s near impossible to be an expert on all of them. And that’s ok: when you deal with large EKS or ECS clusters, changes are you never will touch the IOT services for instances. I found that most services are pretty well explained and easy to start with.

One of the biggest issues are the IOT services: since I have no experience with smart devices, MQTT or IOT in general, i finally managed to get a simple GO application to connect to IOT-core, and connect different rules, pipelines, analytics etc to it. But even still it’s very unclear what the different services actually do. Hopefully there will be AWS IOT experts out there that can enlighten me.