The best way to optimize IOPS on RDS MySQL

Amazon RDS fully supports the InnoDB storage engine for MySQL DB instances and there are features like snapshot restore that are supported for the InnoDB storage engine only.

show storage engines;

But InnoDB is NOT the best performing storage engine on RDS.

No, I am not talking about MyISAM, I am talking about BLACKHOLE.

Best way to optimize IOPS and have fewer problems with data is not to have the data in the first place.

Don’t store data you do not need to store. You might not even need to change your application to achieve that. Go ahead with that ALTER TABLE.


Keep Your Presentation Simple: How a Pigeon Hijacked My Talk

I love discussing technical challenges on AWS using unconventional examples.

My session at re:Invent was “Drawing the New York City skyline with Amazon Aurora Serverless v2”. I used marathon runners to test Amazon Rekognition. I will challenge participants on cycling the Tour de France in a workshop next month.

Load tests using JMeter and shopping patterns are more meaningful but often not powerful enough to explain a new concept. Absurd examples might do better and keep the audience focused, the risk is to take the game too far.

The idea

A couple of months ago I agreed on a talk with the tentative title “A Blue Tit, a Pigeon and Amazon Rekognition Streaming Video Events”.

AWS added Streaming Video Events, a feature of Amazon Rekognition to provide real-time alerts on live video streams. My wife had recently added a birdhouse to feed a small passerine bird but a pigeon decided to claim the space.

A Blue Tit, a Pigeon and Amazon Rekognition Streaming Video Events

What better option to test the new AWS service and see if it would do better than a camera with built-in motion detectors? Can Streaming Video Events differentiate between the two birds and rescue the nest for the little blue tit? Could a notification trigger an audio alarm?

The failure

I did a quick test to demo real-time alerts on live video streams and I waited for the conference this autumn. The talk was approved and I was ready to collect some more real data. But while I was committed to the talk, the birds were not. Pretty soon the small passerine found a new home and the pigeon lost interest in the birdhouse too. I was so busy overthinking the presentation that I missed the obvious. I had no data.

A pigeon as a dependency is a silly mistake, others might be more subtle but equally challenging. Doing a demo at re:Invent on a service in preview had its own risks, you cannot assume it is going to be GA by the time of the conference. The capabilities might change significantly too. For example, the preview of Aurora Serverless v2 was for the MySQL 5.7-compatible edition, a version that never made it to GA.

The lesson

Keep your idea and presentation simple. And don’t feed the pigeons.

InfoQ – August 2022

From free trial instances on Google Cloud Spanner to AWS Event Ruler now open source, from s3 support on Azure Data Explorer to Google Cloud Certificate Manager: a recap of my articles for InfoQ in September.

Amazon Switched Compression from Gzip to Zstd for Own Service Data

A tweet from Adrian Cockcroft, former VP at Amazon, recently highlighted the benefits of switching from gzip to Zstandard compression at Amazon and triggered discussions in the community about the compression algorithm. Other large corporations, including Twitter and Honeycomb, shared interesting gains using zstd.

Google Cloud Spanner Introduces Free Trial Instances and Fine-Grained Access Control

Google Cloud recently announced different improvements to their managed databases. The cloud provider introduced free trial instances and fine-grained access control for Spanner to let developers try the managed service and configure access to data at the table and column level.

Amazon SNS Introduces Message Data Protection to Discover Sensitive Data in Motion

Amazon SNS recently announced the public preview of message data protection. Identifying PII data and other sensitive information in flight, the new SNS feature leverages pattern matching, machine learning models, and data protection policies to simplify data protection and compliance in applications that exchange high volumes of data.

AWS IAM Identity Center Introduces APIs to Manage Users and Groups at Scale

AWS recently introduced IAM Identity Center APIs to create users and groups at scale. Administrators can use these new APIs to manage identities programmatically and gain visibility into users in the Identity Center directory.

AWS Open Sources Event Ruler

AWS recently announced that Event Ruler, the component managing the routing rules of Amazon EventBridge, is now open source. The project is a new option for developers in need to match lots of patterns, policies, or expressions against any amount of events in near real-time.

Google Cloud Certificate Manager Generally Available

Google Cloud recently announced the general availability of Certificate Manager, a service to acquire, manage, and deploy TLS certificates for use with Google Cloud workloads.

AWS Adds VMware Cloud on AWS Integration with Amazon FSx for NetApp ONTAP

AWS and VMware recently announced VMware Cloud on AWS integration with Amazon FSx for NetApp ONTAP. Designed for data-intensive VMware workloads, the new option is a supplemental datastore for VMware Cloud on AWS, reducing storage costs and simplifying migration to the cloud for enterprises.

Azure Data Explorer Supports Native Ingestion from Amazon S3

Microsoft recently announced the ability to natively ingest data from Amazon S3 into Azure Data Explorer (ADX). The new feature simplifies multi-cloud data analytics deployments, bringing data from Amazon S3 to Azure, without relying on custom ETL pipelines.

Cloud Providers Target Middle East: AWS Adds Region in the United Arab Emirates, Microsoft in Qatar

Amazon recently announced a new region in the United Arab Emirates, its second one in the Middle East. Microsoft just opened its first global datacenter region in Qatar where Google plans to open a new region too.

Amazon Introduces Encrypted Communication Service AWS Wickr

A year after the acquisition of the company Wickr, Amazon recently announced the preview of the collaboration suite AWS Wickr. Built on a proprietary encryption protocol, the new managed service provides enterprises and government agencies with security and administrative controls to meet security and compliance requirements.

More news? A recap of my articles for InfoQ in August.

ADDO 2022: Drawing the NYC Skyline with a serverless database

I am pleased to be virtually on stage at All Day DevOps for the third year in a row on November 10th. In the last two years I talked at the the world’s largest DevOps conference about the good, the bad and the ugly of Serverless Databases and cost optimization on AWS. My session this year will be about drawing the NYC Skyline with a serverless database. See you live soon!

Drawing the NYC Skyline with a serverless database

Serverless databases are a challenging area for cloud deployments, with providers trying to extend the elasticity of managed solutions to RDBMS databases. If the new options are so elastic, with CPU and database capacity strictly correlated, can we perform some creative benchmarking using a serverless database?

In this session, we will discuss database elasticity and serverless databases, seeing how they can adapt the capacity vertically and incrementally according to the load and the resources consumed. Ignoring standard load testing tools and simulation models, we will perform different tests with simple SQL statements. If Aurora Serverless v2 is elastic and can scale quickly, is it possible to create a load where the database capacity plots the NYC skyline in Amazon CloudWatch?

InfoQ – August 2022

From private 5G networks on AWS to the largest DDoS attack on Google Cloud, from Microsoft Dev Box to the risk of recursive serverless functions: a recap of my articles for InfoQ in August.

Amazon CloudFront Supports HTTP/3

AWS recently announced that CloudFront distributions support HTTP version 3 (HTTP/3) requests over QUIC. HTTP/3 is optional and provides faster response times and enhanced security over previous HTTP versions.

Microsoft Dev Box Now in Public Preview

Azure recently announced the public preview of Microsoft Dev Box, a workstation in the cloud targeted to developers. Integrated with Windows 365, the new service supports any developer IDE, SDK, or tool that runs on Windows and simplifies onboarding of new developers.

Google Cloud Blocks Largest Layer 7 DDoS Attack

Google claims to have recently fended off the largest ever HTTPS-based distributed denial of service attack, which peaked at 46 million requests per second. According to the cloud provider, the DDoS attack was quickly detected and stopped at the edge of Google’s network, and the customer was not impacted.

AWS Private 5G Mobile Network Generally Available But without 5G Support

Amazon recently announced the general availability of AWS Private 5G, a managed service to set up and deploy private mobile networks. The current lack of 5G support, the minimum price of a radio unit and the availability only in the United States raised some concerns in the community.

AWS Adds Coding Assistant CodeWhisperer to Lambda Console

AWS recently announced the preview of Amazon CodeWhisperer in the AWS Lambda console. Available as a native code suggestion feature in the code editor, the new functionality of the coding assistant can make code recommendations during Lambda function definition.

Amazon GuardDuty Introduces Malware Detection

At the recent re:Inforce security conference, AWS announced the availability of malware detection for Amazon GuardDuty. The new functionality of the managed threat detection service initiates a scan of the EBS volumes when it detects suspicious behavior indicative of malware on EC2 or containers.

Google Cloud Introduces Batch, a Service for Scheduling Batch Jobs

Google Cloud recently announced the preview of Batch, a managed service to run batch jobs at scale. The new service supports the latest T2A Arm-based instances and Spot VMs for large batch jobs utilizing task parallelization.

Are Recursive Serverless Functions the Biggest Billing Risk on the Cloud?

Forrest Brazeal, head of content at Google Cloud, recently argued that serverless functions are the cloud’s biggest billing risk for developers as there’s no simple way to protect against recursive calls and they can scale out almost indefinitely on all the cloud providers.

More news? A recap of my articles for InfoQ in July.

InfoQ – July 2022

From AMD R6a insances to Rocky Linux on Google Cloud, from Amazon Redshift Serverless to API backend options for Azure Static Web Apps: a recap of my articles for InfoQ in July.

AWS Announces AMD Based R6a Instances for Memory-Intensive Workloads

AWS recently announced the general availability of the R6a instances, EC2 designed for memory-intensive workloads like SQL and NoSQL databases.. The new instances are built on the AWS Nitro System and are powered by AMD Milan processors.

Google Cloud Introduces Optimized Rocky Linux Images for Customers Moving off CentOS

Google recently announced the general availability of Rocky Linux optimized for Google Cloud. The new images are customized variants of Rocky Linux, the open-source enterprise distribution compatible with Red Hat Enterprise.

Amazon Redshift Serverless Generally Available to Automatically Scale Data Warehouse

Amazon recently announced the general availability of Redshift Serverless, an elastic option to scale data warehouse capacity. The new service allows data analysts, developers and data scientists to run and scale analytics without provisioning and managing data warehouse clusters.

Amazon Announces General Availability of EC2 M1 Mac Instances to Build and Test on macOS

AWS recently announced the general availability of the EC2 M1 Mac instances based on the Apple ARM-based processor and designed for CI/CD of Apple-based applications. The M1 Mac option is faster and cheaper than the existing x86-based Mac version but still requires a minimum 24 hours commitment.

Azure Static Web Apps Introduces API Backend Options

Azure recently announced the preview of new API backend options in Azure Static Web Apps. Developers can now create an end-to-end authenticated application calling APIs hosted on Azure App Service, Azure Container Apps, or Azure API Management.

Amazon Aurora Supports PostgreSQL 14

Amazon recently announced that Aurora PostgreSQL supports PostgreSQL major version 14. The new release adds performance improvements and new capabilities, including support for SCRAM password encryption.

PostgreSQL Interface for Cloud Spanner Now Generally Available

Google Cloud recently announced the general availability of the PostgreSQL interface for Cloud Spanner. The new interface increases the portability of workloads to and from Spanner and provides a globally distributed option to developers already familiar with PostgreSQL.

TLS 1.2 Becoming the Minimum TLS Protocol Level on AWS

AWS recently announced that TLS 1.2 is going to become the minimum protocol level for API endpoints. The cloud provider will remove backward compatibility and support for versions 1.0 and 1.1 on all APIs and regions by June 2023.

More news? A recap of my articles for InfoQ in June.