- Published on
Amazon Cloud - AWS
- Authors
- Name
- Jackson Chen
Reference
AWS Training
AWS and Azure Cloud Technical Training - Always Up to day https://learn.cantrill.io/
ACloud Guru https://learn.acloud.guru/dashboard
AWS Workshops and Video https://www.workshops.aws/
AWS Training https://www.aws.training/
AWS Services
AWS services are categorised, for example
Analytics
Athena
CloudSearch
Data Pipeline
EMR # Managed Hadoop Framework
AWS Glue # serverless data integration service
Amazon Redshift # Data warehousing
Application Integration
Amazon AppFlow
Amazon EventBridge
Amazon MQ # Managed messaeg broker service for ActiveMQ and RabbitMQ
Simple Notification Service (SNS) # message topics for Pub/Sub
Simple Queue Service # Managed message queues
Step Functions # Coordinated distributed applications
SWF # Workflow for coordinated application components
AWS Cost Management
Business Applications
Compute
AWS App Runner # Build and run production web application at scale
Batch # Managed batch processing at any scale
EC2 # virtual servers in the cloud
EC2 Image Builder # Managed service to automate build, customize and deploy OS images
Elastic Beanstalk # Run and manage web apps
Lambda # run code without thinking about servers
Lightsail # Lauch and manage virtual private servers
AWS Outpost # Run AWS services on premises
Serverless Application Repository # Assemble, deploy, and share serverless applicationis within teams or publicly
AWS SimSpace Weaver # Build and run large scale spatical simulations
Containers
Elastic Container Registry # Managed Docker container registry
Elastic Container Service # Highly secure, reliable, and scalable way to run containers
Elastic Kubernetes Service # Start, run and scale Kubernetes
Red Hat OpenShift Service on AWS # Managed Red Hat OpenShift service on AWS
Database
Amazon DocumentDB # Managed MongoDB-compatible database service
DynamoDB # Mnaged NoSQL database
ElasticCache # In-memory cache (database cache)
Amazon Keyspaces # serverless cassandra-compatible database
Amazon MemoryDB for Redis # Managed Redis-compatible, im-memory database service
Neptune # Graph database build for the cloud
Amazon QLDB # managed ledger database
RDS # Managed relational database service
MySQL
PostgreSQL
Microsoft SQL
Oracle
Amazon Timestream # Serverless time series database for IoT and operational applications
Developer Tools
AWS AppConfig
Application Composer # visually design, build serverless application
Cloud9 # Cloud IDE for writing, running and debugging code
CloudShell # AWS CLI access
CodeArtifact # management of software development
CodeBuild # Build and test code
Amazon CodeCatalyst # Integrated DevOps service
Code Commit # store code in private Git repositories, version control
Code Deploy # automate code deployments
Code Pipeline # Release software using continuous delivery
Code Star # Develop, build and deply applications
Amzon CodeWhisperer # Build applications with ML-powered coding companion
AWS FIS # Improve resilience and performance with controlled experiments
X-Ray # Analyze and debug your applications
End User Computing
AppStream # Steam desktop applications securely to any web browser
WorkSpaces # desktop in the cloud
WorkSpace Web # cloud-native secure web access
Machine Learning
Management & Governance
AWS Auto Scaling # quickly scale entire application in AWS
CloudFormation # Create and manage resources with templates
CloudTrail # Track user activity and API usage
CloudWatch # Monitor resources and applications
AWS Compute Optimizer # Recommend optimal AWS Compute resources for your workloads
Config # Track resource inventory and changes
Control Tower # Setup and govern a secure, compliant multi-account environment
Amazon Grafana # Managed Grafana service for interactive data visualization and dashboarding
Incident Manager
Launch Wizard # guided deployment for enterprise applications and complex workloads
AWS License Manager
OpsWorks # Configuration management with Chef and Puppet
AWS Organizations # Central governance and management across AWS accounts
Amazon Prometheus # Manage Prometheus-compatiable monitoring service
AWS Proton # Manage your infrastructure, so developers can focus on coding
AWS Resilience Hub # A central place to define, validate and track the resiliency of applications on AWS
AWS Resource Explorer # Search and discover relevant resources across AWS
Resource Groups & Tag Editor # Search and group AWS resources
Service Catalog # Create, share, organize and govern your infrastructure as code (IaC) templates
Service Quotas
System Manager # View and manage AWS resources
Trusted Advisor # Optimize performance and security
AWS User Notifications
AWS Well-Architected Tool # Learn best practices, measure and improve your workloads
AWS Health Dashboard
Migration & Transfer
Application Disovery Service # Discover on-premises application inventory and dependencies
AWS Application Migration Service
Database Migration Service
DataSync # automate and accelerate moving data
AWS Migration Hub
AWS Snow Family # large scale data transport
AWS Transfer Family # Managed support for SFTP, FTPS and FTP
Networking & Content Delivery
API Gateway # Build, deploy and manage APIs
AWS App Mesh # monitor and control microservices
CloudFront # Clobal content delivery network (CDN)
Direct Connect # Dedicated network connection to AWS from your on-premises
Global Accelerator # Improve application availability and performance using AWS global network
Route 53 # Scalable DNS and domain name registration
Route 53 Application Recovery Controller # Monitor application recovery readiness and manage failovers
VPC # Isolated cloud resources (virtual private network)
Security, Identity & Compliance
AWS Artifact # security compliance reports and agreements
AWS Audit Manager # continuously assess controls for risk and compliance
Certificate Manager
CloudHSM # managed hardware security modules in the cloud
Cognito # Consumer identity management and AWS credentials for federated identities
Detective # Investigate and analyze potential security issues
Director Service # Host and managed Active Directory
AWS Firewall Manager # central management of firewall rules
GuardDuty # Intelligent threat detection to protect your AWS accounts and workloads
IAM Identity Center (successor to AWS Single Sign-On)
Amazon Inspector # continual vulnerability management at scale
Key Management Service # Generate and manage AWS encryption keys
AWS Private Certificate Authority
Resource Access Manager # Share AWS resources with other accounts or AWS organizations
Secrets Manager # Rotate, manage and retrieve secrets
Security Hub # AWS's security and compliance center
Security Lake # automatically centralize all your security data with a few clicks
AWS Signer # Ensuring trust and integrity of your code
Amazon Verified permissions # Manage, analyze and enforce permissions across your applications
WAF & Shield # Protet agains DDoS attacks and malicious web traffic
Storage
AWS Backup
EFS # Managed file storage for EC2 - shared storage / network share
AWS Elastic Disaster Recovery # Scalable application recovery to AWS
FSx # Managed third party file systems optimized for variety of workloads
S3 # Scalable storage in the cloud - Amazon file system
S3 Galcier # archive storage in the cloud
Storage Gateway # Hybrid storage integration
Using search to search for service, rather than browsing for the service.
Regions and Availability Zones
Inside region, there are availability zones to provide redundancies and site redundancies.
Security
Data protection:
- Amazon Macie - Discover and protect yoru sensitive data
- AWS key Management Service - Store and manage encryption keys
- AWS CloudHSM - hardware based key
Infrastructure Protection
- AWS Shield - Denied of service protection
- AWS web appliction firewall - Filter malicious website traffic
- AWS Firewall Manager - centrally manage firewall rules
Threat Detection
- Amazon GuardDuty - detect threats
- Amazon Inspector - application security
- AWS Config - Record and evaluate configuration of your AWS resources
- AWS CloudTrail - Track user activity and API usage
Identity Management
- AWS IAM
- AWS Single Sign-On
- AWS Cognito - manage identity inside applications
- AWS Directory Service
. Managed Microsoft Active Directory
. Managed Simple Active Directory
. AD Connector
- AWS Organizations - centrally govevrn and manage multiple AWS acccounts
AWS terminologies
Compute - Instance (aka virtual machine)
AWS Management
AWS configuration can be done by using JSON file or JSON format
# Example - IAM Policy
{
"Version": "2023-08-31"
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::bucket-name
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject"
],
"Resource": "arn:aws:s3:::bucket-name/*"
}
]
}
Where
* Means everything
To use SecretManager secret rather than using the password
# Using secret string
import mysql.connector
connection = mysql.connector.connect (
host='localhost',
database='test',
user='root',
password=get_secert_value_response[''SecretString]
)
Secret Manager can store different secrets, such as token, password, keys
AWS Directory Service
- Managed Microsoft Active Directory
- Managed Simple Active Directory
- AD Connector
- Distributed service with automatic failover
- Compatible with other AWS services
Amazon Chime
Amazon Connect
Amazon EC2 Intances
Amazon FSx for Windows File Server
Amazon QuickSight
Amazon RDS for MySQL
Amazon RDS for Oracle
Amazon RDS for PostgreSQL
Amazon RDS for SQL Server
Amazon Single Sign On
Amazon WorkDocs
Amazon WorkMail
Amazon WorkSpaces
Amazon Client VPN
Amazon Management Console
IAM
Manage users and groups from IAM
# Policies
1. Custoomer Inline # Normally used for one off situation
2. Managed policies # Amazon has many managed policies
Important: All permissions are implicit DENY, unless ALLOW.
EC2 Instances
# Instances
. Amazon EC2 # Secure and resizable virtual machines in the cloud
. Amazon EC2 Spot # Run fault tolerant workload at 90% of the normal price
. Amazon EC2 Auto Scaling # Automatically add or remove capacity based on demand
. Amazon LightSail # An easy-to-use cloud platform to build applications or websites
Containers
# Containters
ECS - Elastic container service
ECR - Elastic container registry
EKS - Elastic Kubernetes service
Lambda - serverless
All to run code without virtual machine # A compute service to run code without server
# Lambda is super powerful and pay by milliseconds, it can be super cheap as it is triggered by events, then does a function or action.
Note:
Lambda can not run more than 15 minutes. If longer runtime, then use Batch
Edge
# There are following can run on Edge
1. AWS Outpost # Run AWS services on-premises
2. AWS Snow Family # Bring your data to AWS
3. AWS Wavelength # Access AWS service via 5G networks
4. VMware Cloud on AWS # Migrate VMware workloads to AWS
5. AWS Local Zones # Run latency sensitive applications closer to end users
EC2 - Elastic Compute Cloud
. Rent virtual computers
. Choose from various type of OS with different CPU, RAM and storage
. Different optimizations are available
. Pay by the hour or second
Storage
There are different storage methods or type, File, Block or Object.
In AWS, they are stored as object with unique identifier number (UUID)
it is globally unique. It can be retrieved by its UUID
1. Amazon EFS - Elastic File System
# A scalable, elastic and cloud native netowrk file system.
Highly available and durable (11's 9 - 99.999999999%)
Build-in protection from AZ outages and other failures
2. Amazon FSx - For Windows File Server
# A fully managed file storage for Windows servers
3. Amazon EBS - Block Storage
# Easy to use, high performance block storage
# It is just hard disk, can only be mounted to one single computer/instance
4. Amazon S3 - Object storage (Amazon Simple Storage Service - S3)
# Store and retrieve any amount of data from anywhere in the world
5. AWS Backup
Storage class
There are different storage classes
Availibility Zone Min Storge Charge
1. S3 Standard 3 N/A
2. S3 Standard - Infrequent 3 30 days
3. S3 One Zone - Infrequent 1 30 days
4. S3 Glacier >=3 90 days
5. S3 Glacier Deep Archive >=3 180 days
6. S3 Intelligent Tiering >=3 30 days
Using access policy, if file not frequently access, then move them to infrequent access tier.
Note: file are encrypted
There are storage gateway to provide access to files stored in AWS, and Tape Gateway to provide access to virtual tapes stored in AWS.
Virtual tapes support by:
CommVault
Veeam
Veritas
There is volume gateway, as it presents an iSCSI block storage volume as storag to your on-premises applications.
Data Transfer
# Data transfer service
1. AWS Storage Gateway # Provide on-premises access to unlimited cloud storage
# Give you access to SMB and NFS interface to S3
2. ASWS DataSync # Transfer data to and from AWS up to 10 times faster than normal
3. AWS Transfer Family # Transfer files to AWS S3 using SFTP, FTP and FTPS
4. AWS Snow
Database
There are different database types
# Database types
Relational
Amazon Aurora # mySQL and PostgreSQL compatible
Amazon RDS
Amazon RedShift # data warehouse
RDS - (MySQL, PostgreSQL, MariaDB, Oracle, Microsoft SQL)
# using structural sql query
# Easy to setup, automated backup
Key-value
Amazon DynamoDB # NoSQL for any scale
1. where value can be different types
2. Can have different attributes, i.e. 1, 2 or more attributes
# suitable for e-commerce, gaming
# single digit millisecond performance
# Can handle more than 20 millions requests per second
Note: Build for big internet companies
In-memory
Amazon ElasticCache # Managed, in-memory data store service for Redis, and Memcached
# suitable for gaming and geo-spatial applications
Users -> Website -> ElasticCache -> Database
Document
Amazon DocumentDB # MongoDB compatible database
Graph
Time series
Wide column
Ledger
Networking
# Networking includes
1. Amzon VPC # Define and provision an isolated network for your AWS resources
2. Amazon Route 53 # Host your own managed DNS
3. AWS Transit Gateway # Connect VPCs and on-premisess networks
4. AWS Privatelink # Provide private connectivity between VPCs and on-premises applications
Network scaling
1. Elastic Load Balancing # automatically distribute network traffice across pool of resources
2. AWS Global Accelerator # Direct traffic through the AWS global network, to
improve global application performance
Content Dilivery
Amazon CloudFront
# secure deliver data, videos and applications to users globally with low latency and high transfer speeds
VPC
There are from gateways
1. NAT gateway
2. Internet gateway
Network access control list (NACL)
CloudFront
AWS has many CloudFront edge locations around the world for faster website access.
Users -> CloudFront -> Load Balancers -> Websites/applications
# Provides benefits
Increase security # Protect backend systems
Traffic spike protection
Lambda at Edge
Realtime metrics
It shields your website or applications from users, and protect your applications
Route 53
It is Amazon DNS services
# Provide services
weighted policy
simple routing
geolocation policy
latency policy
failover policy
multivalue answer policy (similar to load balancing)
Account management
1. AWS Control Tower # setup and govern a secure multi-account AWS environment
2. AWS Organizations # Centrally govern and manage across multiple AWS accounts
3. AWS Budgets # Plan and cost control
Provisioning Services
1. AWS CloudFormation # Model and provision resources via code
2. AWS Service Catalog # Create, organize and govern your curated catalog of AWS service/instances, etc
3. AWS OpsWorks # Automate operations with Chef and Puppet
4. AWS MarketPlace # find, test and buy deploy software that run on AWS
Operation Services
1. Amazon CloudWatch # Observe your services via metrics and logging
2. AWS Config # Record and evaluate configurations of AWS resources
3. AWS CloudTrail # Track user activity and API usage
4. AWS Systems Manager # Optimize performance and security while manage a large amount of systems
5. Amazon X-Ray # Aanlyze and debug applications
CloudFormation
Create and manage cloud infrastructure services, such as VPC, instances, etc CloudFormation templates are written using either JSON or YAML
# Benefits
Using Git, GitHub or AWS CodeCommit as version control
CloudFormation allow DevOps for automation
Deploy across multiple regions
First, create CloudFormation stack, then create create resources.
When longer required, delete the CloudFormation stack, it will delete all the resources associated with the stack.
CloudWatch
Monitoring and observatory
. Collect metrics from services
. Integrates with 70+ AWS services
. Lots of pre-defined metrics
Note:
Configure on-premises to send logs to CloudWatch
Auto Scaling
Scale instances automatically, create an autoscaling group, then create desired capacity.
--------
Min size
--------------------
Desired capacity
-----------------------------------------------
Scale out as needed
-------------------------------------------------------------------
Max size / capcity
Note:
When implement with load balancer, provides
High availability
Fault tolerance
Support - EC2, DynamoDB, Aurora
Machine Learning
1. Amazon Kendra # intelligent search
2. Amazon Personalize # personalized recommendations
3. Amazon Rekognition # analyze images and videos, and extract meaning
4. Learning languages services
Amazon Polly # Turn text to speech
Amazon Transcribe # Add speed to text capabilities to your applications
Amazon Lex # Build conversational agents or chat bots