AWS Notes

​​🆕 Step Functions Distributed Map: https://aws.amazon.com/blogs/aws/step-functions-distributed-map-a-serverless-solution-for-large-scale-parallel-data-processing/ Step Function’s map state executes the same processing steps for multiple entries in a dataset. The existing map state is limited to 40 parallel iterations at a time. Glue vs EMR vs S3 Batch Operations ◆ Data scientists and data engineers use Glue and EMR to process large amounts of data. ◆ On the other hand, application developers will use Step Functions to add serverless data processing into their applications. Step Functions is able to scale from zero quickly, which makes it a good fit for interactive workloads where customers may be waiting for the results. ◆ System administrators and IT operation teams are likely to use S3 Batch Operations for single-step IT automation operations such as copying, tagging, or changing permissions on billions of S3 objects. #Step_Functions #serverless

🆕 Amazon EventBridge Pipes: https://aws.amazon.com/blogs/aws/new-create-point-to-point-integrations-between-event-producers-and-consumers-with-amazon-eventbridge-pipes/ EventBridge Pipes supports the following services as event sources: ✺ DynamoDB ✺ Kinesis ✺ Kafka/MSK ✺ SQS ✺ MQ #EventBridge_Pipes

🆕 AWS Application Composer 🔥

AWS re:Invent 2022 обзор анонсов часть 2 https://www.youtube.com/watch?v=cbxNxHIkd8M Присоединяйтесь к нам прямо сейчас! #reinvent

🆕 New features of Amazon SageMaker: ✺ AutoML step in SageMaker Model Building Pipelines ✺ Collaboration with shared spaces ✺ Data Wrangler data preparation widget ✺ Inference shadow tests ✺ Notebook-based Workflows ✺ SageMaker Model Cards ✺ SageMaker Model Dashboard ✺ SageMaker Role Manager ✺ SageMaker geospatial capabilities ✺ Studio Git extension #SageMaker #reinvent2022

🆕 New features of Amazon SageMaker: ✺ AutoML step in SageMaker Model Building Pipelines ✺ Collaboration with shared spaces ✺ Data Wrangler data preparation widget ✺ Inference shadow tests ✺ Notebook-based Workflows ✺ SageMaker Model Cards ✺ SageMaker Model Dashboard ✺ SageMaker Role Manager ✺ SageMaker geospatial capabilities ✺ Studio Git extension #SageMaker #reinvent2022

​​AWS re:Invent 2022 обзор анонсов часть 1: https://www.youtube.com/watch?v=dZyDPAZZ_CY 4:25 Новые Local Zones 8:20 OpenSearch Serverless 11:30 SimSpace Weave 21:57 DataZone 26:52 Security Lake 30:01 Clean Rooms 36:25 Nitro v5 44:17 EC2 C7gn 50:12 EC2 HPC6id 54:43 Lambda SnapStart 58:00 CloudWatch Internet Monitor 1:02:20 Verified Access 1:04:05 ECS Service Connect 1:06:05 Amazon Omics #reinvent

Присоейдиняйтесь прямо сейчас! https://www.youtube.com/watch?v=dZyDPAZZ_CY #reinvent

🆕 Amazon Security Lake: https://aws.amazon.com/blogs/aws/preview-amazon-security-lake-a-purpose-built-customer-owned-data-lake-service/ Security Lake automatically collects logs for: √ AWS Health Dashboard √ CloudTrail √ Firewall Manager √ GuardDuty √ IAM Access Analyzer √ Inspector √ Lambda √ Macie √ Route 53 √ S3 √ Security Hub √ Systems Manager Patch Manager √ VPC Security Lake automatically partitions and converts incoming log data to a storage and query-efficient Apache Parquet and OCSF format. Security Lake supports third-party sources providing OCSF security data, including Barracuda Networks, Cisco, Cribl, CrowdStrike, CyberArk, Lacework, Laminar, Netscout, Netskope, Okta, Orca, Palo Alto Networks, Ping Identity, SecurityScorecard, Tanium, The Falco Project, Trend Micro, Vectra AI, VMware, Wiz, and Zscaler. #Security_Lake

🆕 OpenSearch Serverless 🔥

💥 EC2 R7iz on Intel Xeon Scalable Gen 4: https://aws.amazon.com/ec2/instance-types/r7iz/ R7iz ▹ Architecture: x86 ▹ Frequency: 3.9 GHz ▹ CPU ⇨ up to 128 vCPUs ▹ Memory ⇨ up to 1 TiB ▹ Network ⇨ up to 50 Gbps ▹ EBS ⇨ up to 40 Gbps Если вы думаете, что "Intel всё", то нет. Новые виртуалки на свежайшем 4-м поколении Intel на текущий момент становятся самыми мощными для аналитики и других нагрузок, требовательных к памяти. #EC2

🆕 Lambda SnapStart: https://aws.amazon.com/blogs/aws/new-accelerate-your-lambda-functions-with-lambda-snapstart/ ✻ Lambda SnapStart can improve startup performance for latency-sensitive applications by up to 10x at no extra cost, typically with no changes to your function code. The largest contributor to startup latency (often referred to as cold start time) is the time that Lambda spends initializing the function, which includes loading the function's code, starting the runtime, and initializing the function code. ✻ With SnapStart, Lambda initializes your function when you publish a function version. Lambda takes a Firecracker microVM snapshot of the memory and disk state of the initialized execution environment, encrypts the snapshot, and caches it for low-latency access. When you invoke the function version for the first time, and as the invocations scale up, Lambda resumes new execution environments from the cached snapshot instead of initializing them from scratch, improving startup latency. ⚠️ Important ➣ If your applications depend on uniqueness of state, you must evaluate your function code and verify that it is resilient to snapshot operations. For more information, see Handling uniqueness with Lambda SnapStart. SnapStart supports: ✅ Java 11 runtime SnapStart does not support: ❌ provisioned concurrency ❌ arm64 ❌ Lambda Extensions ❌ EFS ❌ > 512 MB ephemeral storage ℹ️ You can't use SnapStart on a function's unpublished version ($LATEST). SnapStart vs Provisioned Concurrency 👉 Use Provisioned Concurrency if your application has strict cold start latency requirements. 👉 SnapStart helps you improve startup performance by up to 10x at no extra cost. ❗ You can't use both SnapStart and Provisioned Concurrency on the same function version. #Lambda

​​💥 Graviton3E + Nitro v5 = C7gn and HPC7g https://aws.amazon.com/blogs/aws/new-amazon-ec2-instance-types-in-the-works-c7gn-r7iz-and-hpc7g/ C7gn and HPC7g ▹ CPU ⇨ up to 64 vCPUs ▹ Memory ⇨ up to 128 GiB ▹ Network ⇨ up to 200 Gbps #Graviton3E #Nitro

Смотрите, какая прелесть! http://awguess.com

Control Tower Account Factory Customization: 🎉 https://docs.aws.amazon.com/controltower/latest/userguide/af-customization-page.html Кастомизация AWS аккаунтов, которую можно применять как к свежесоздаваемым аккаунтам, так и уже имеющимся. Год назад сделали такое для Terraform (Control Tower Account Factory for Terraform) и вот теперь аналогичное для CloudFormation. 👉 Ещё раз: 1️⃣ Сначала с поддержкой Terraform. 2️⃣ Через год (❗) — с поддержкой CloudFormation. Кто там рассказывает, что AWS форсит CloudFormation?!? #Control_Tower

​​Control Tower Comprehensive Controls Management: https://aws.amazon.com/blogs/aws/new-for-aws-control-tower-comprehensive-controls-management-preview/ Соответствие требованиям PCI DSS и другим compliance раньше нужно было самому настраивать для AWS аккаунтов с помощью SCP. Теперь же это можно сделать сразу с помощью Control Tower. Да ещё сразу плюс интеграция с Security Hub (Service-Managed Standard: AWS Control Tower). В общем, вердикт — можно брать. Как минимум, на превью обещают бесплатно. There is no additional charge to use these new capabilities during the preview. #Control_Tower

​​AWS Wickr for enterprises with auditing and regulatory requirements: https://aws.amazon.com/blogs/aws/aws-wickr-a-secure-end-to-end-encrypted-communication-service-for-enterprises-with-auditing-and-regulatory-requirements/ End-to-End Encryption ☐ Wickr provides secure communication between two or more correspondents. It means that the system provides authenticity and confidentiality: no unauthorized party can inject a message into the system, and no unintended party can access or understand the communications without being given them by one of the correspondents. ☐ Each message gets a unique AES encryption key and a unique ECDH public key to negotiate the key exchange with other recipients. The message content (text, files, audio, or video) is encrypted on the sending device (your iPhone, for example) using the message-specific AES key. The message-specific AES key is exchanged with recipients via a Diffie-Hellman EDCH521 mechanism. This ensures that only intended recipients have the message-specific AES key to decrypt the message. #Wickr

Delegated administrator for AWS Organizations: https://docs.aws.amazon.com/organizations/latest/userguide/orgs_delegate_policies.html Это могло стать одной из самых крутых новостей ещё не начавшегося (или вот-вот) re:Invent 2022. Но нет. Кто, как и я, бросился включать AWS Organizations Delegated administrator в консоли, тоже гарантированно сначала получат: ❌Failed to edit delegation policy. ▼API response This resource-based policy contains an unsupported action. С энной попытки таки решат, наконец, почитать примеры полиси в документации и получат уже синтаксические ошибки (пропущены запятые в JSON), чем часто грешат свежевыпущенные тексты. Но даже исправив все ошибки, так нахрапом создать delegated policy не получится. Если коротко, то звёздочка в Actions не прокатит. Вот максимальный набор правил, что можно передать в AWS Organizations Delegated Administrator account: {  "Version": "2012-10-17",  "Statement": [   {    "Sid": "MaximumPolicy",    "Effect": "Allow",    "Principal": {     "AWS": "arn:aws:iam::022015898756:root"    },    "Action": [     "organizations:List*",     "organizations:Describe*",     "organizations:CreatePolicy",     "organizations:UpdatePolicy",     "organizations:DeletePolicy",     "organizations:AttachPolicy",     "organizations:DetachPolicy",     "organizations:EnablePolicyType",     "organizations:DisablePolicyType",     "organizations:TagResource",     "organizations:UntagResource"    ],    "Resource": "*"   }  ] } Ответ в самом начале документа. Это "ненастоящий админ", это policy admin. Правда права у него побольше, нежели были раньше доступны для Delegated админов: https://docs.aws.amazon.com/organizations/latest/APIReference/action-reference.html#actions-management-or-delegated-admin Добавились тэги и, собственно, управление Policy. Но это всё. Управлять (CRUD и т.п.) непосредственно AWS аккаунтами, OU — нельзя, только из Management account. Что ж, пока не всё, что хотелось, но уже хоть что-то. И важно — заложен Resource-Based принцип для работы с AWS Organizations в целом и её частями в частности. А это значит, что «Лёд тронулся, господа присяжные заседатели!» © 😀 #Organizations