Renato joins Adam to discuss the differences between Aurora Serverless v1 and v2, how he’s used AWS certifications to learn topics he might not dive into otherwise, and the benefits of speaking at conferences when you’re introverted.
Amazon RDS fully supports the InnoDB storage engine for MySQL DB instances and there are features like snapshot restore that are supported for the InnoDB storage engine only.
But InnoDB is NOT the best performing storage engine on RDS.
No, I am not talking about MyISAM, I am talking about BLACKHOLE.
Best way to optimize IOPS and have fewer problems with data is not to have the data in the first place.
Don’t store data you do not need to store. You might not even need to change your application to achieve that. Go ahead with that ALTER TABLE.
ALTER TABLE myUselessTable ENGINE=BLACKHOLE;
I love discussing technical challenges on AWS using unconventional examples.
My session at re:Invent was “Drawing the New York City skyline with Amazon Aurora Serverless v2”. I used marathon runners to test Amazon Rekognition. I will challenge participants on cycling the Tour de France in a workshop next month.
Load tests using JMeter and shopping patterns are more meaningful but often not powerful enough to explain a new concept. Absurd examples might do better and keep the audience focused, the risk is to take the game too far.
The idea
A couple of months ago I agreed on a talk with the tentative title “A Blue Tit, a Pigeon and Amazon Rekognition Streaming Video Events”.
AWS added Streaming Video Events, a feature of Amazon Rekognition to provide real-time alerts on live video streams. My wife had recently added a birdhouse to feed a small passerine bird but a pigeon decided to claim the space.
What better option to test the new AWS service and see if it would do better than a camera with built-in motion detectors? Can Streaming Video Events differentiate between the two birds and rescue the nest for the little blue tit? Could a notification trigger an audio alarm?
The failure
I did a quick test to demo real-time alerts on live video streams and I waited for the conference this autumn. The talk was approved and I was ready to collect some more real data. But while I was committed to the talk, the birds were not. Pretty soon the small passerine found a new home and the pigeon lost interest in the birdhouse too. I was so busy overthinking the presentation that I missed the obvious. I had no data.
A pigeon as a dependency is a silly mistake, others might be more subtle but equally challenging. Doing a demo at re:Invent on a service in preview had its own risks, you cannot assume it is going to be GA by the time of the conference. The capabilities might change significantly too. For example, the preview of Aurora Serverless v2 was for the MySQL 5.7-compatible edition, a version that never made it to GA.
The lesson
Keep your idea and presentation simple. And don’t feed the pigeons.
From free trial instances on Google Cloud Spanner to AWS Event Ruler now open source, from s3 support on Azure Data Explorer to Google Cloud Certificate Manager: a recap of my articles for InfoQ in September.
Amazon Switched Compression from Gzip to Zstd for Own Service Data
A tweet from Adrian Cockcroft, former VP at Amazon, recently highlighted the benefits of switching from gzip to Zstandard compression at Amazon and triggered discussions in the community about the compression algorithm. Other large corporations, including Twitter and Honeycomb, shared interesting gains using zstd.
Google Cloud Spanner Introduces Free Trial Instances and Fine-Grained Access Control
Google Cloud recently announced different improvements to their managed databases. The cloud provider introduced free trial instances and fine-grained access control for Spanner to let developers try the managed service and configure access to data at the table and column level.
Amazon SNS Introduces Message Data Protection to Discover Sensitive Data in Motion
Amazon SNS recently announced the public preview of message data protection. Identifying PII data and other sensitive information in flight, the new SNS feature leverages pattern matching, machine learning models, and data protection policies to simplify data protection and compliance in applications that exchange high volumes of data.
AWS IAM Identity Center Introduces APIs to Manage Users and Groups at Scale
AWS recently introduced IAM Identity Center APIs to create users and groups at scale. Administrators can use these new APIs to manage identities programmatically and gain visibility into users in the Identity Center directory.
AWS Open Sources Event Ruler
AWS recently announced that Event Ruler, the component managing the routing rules of Amazon EventBridge, is now open source. The project is a new option for developers in need to match lots of patterns, policies, or expressions against any amount of events in near real-time.
Google Cloud Certificate Manager Generally Available
Google Cloud recently announced the general availability of Certificate Manager, a service to acquire, manage, and deploy TLS certificates for use with Google Cloud workloads.
AWS Adds VMware Cloud on AWS Integration with Amazon FSx for NetApp ONTAP
AWS and VMware recently announced VMware Cloud on AWS integration with Amazon FSx for NetApp ONTAP. Designed for data-intensive VMware workloads, the new option is a supplemental datastore for VMware Cloud on AWS, reducing storage costs and simplifying migration to the cloud for enterprises.
Azure Data Explorer Supports Native Ingestion from Amazon S3
Microsoft recently announced the ability to natively ingest data from Amazon S3 into Azure Data Explorer (ADX). The new feature simplifies multi-cloud data analytics deployments, bringing data from Amazon S3 to Azure, without relying on custom ETL pipelines.
Cloud Providers Target Middle East: AWS Adds Region in the United Arab Emirates, Microsoft in Qatar
Amazon recently announced a new region in the United Arab Emirates, its second one in the Middle East. Microsoft just opened its first global datacenter region in Qatar where Google plans to open a new region too.
Amazon Introduces Encrypted Communication Service AWS Wickr
A year after the acquisition of the company Wickr, Amazon recently announced the preview of the collaboration suite AWS Wickr. Built on a proprietary encryption protocol, the new managed service provides enterprises and government agencies with security and administrative controls to meet security and compliance requirements.
More news? A recap of my articles for InfoQ in August.
I am pleased to be virtually on stage at All Day DevOps for the third year in a row on November 10th. In the last two years I talked at the the world’s largest DevOps conference about the good, the bad and the ugly of Serverless Databases and cost optimization on AWS. My session this year will be about drawing the NYC Skyline with a serverless database. See you live soon!
Drawing the NYC Skyline with a serverless database
Serverless databases are a challenging area for cloud deployments, with providers trying to extend the elasticity of managed solutions to RDBMS databases. If the new options are so elastic, with CPU and database capacity strictly correlated, can we perform some creative benchmarking using a serverless database?
In this session, we will discuss database elasticity and serverless databases, seeing how they can adapt the capacity vertically and incrementally according to the load and the resources consumed. Ignoring standard load testing tools and simulation models, we will perform different tests with simple SQL statements. If Aurora Serverless v2 is elastic and can scale quickly, is it possible to create a load where the database capacity plots the NYC skyline in Amazon CloudWatch?
No system provides 100% availability, so the pragmatic question is whether or not CloudWildBoar delivers availability that is so high that most users don’t worry about its outages. For example, given there are many sources of outages for an application, if CloudWildBoar is an insignificant contributor to its downtime, then users are correct to not worry about it
CloudWildBoar does not exist. It is not a service on AWS (yet?) or any other cloud provider. I made it up.
The sentence is not mine, it originally referred to Google Spanner and it is from an old article “Inside Cloud Spanner and the CAP Theorem”.
But I always keep the sentence handy for any meetup, conference, tech discussion when someone asks or brags about availability, number of nines of a specific provider, database, service or feature.
The cloud provider and specific services might have occasional failures and you should be aware of those. And there are limited scenarios where it really matters. But usually the weakest links are your own application and deployment.
Fix those, sit down and relax. And keep that sentence ready for your next chat.
Photo by Austin Neill on Unsplash
There are different ways to learn and test Amazon DynamoDB on-demand and test how the database process requests without capacity planning. There are conventional ones, where you follow the AWS documentation and “start 1,000 Lambda functions in parallel to generate load on the API endpoint, using random HTTP methods and random data for the item”.
Or you can try a more absurd way to demonstrate how the write capacity and your billing are predictable in an on-demand mode. You can cycle the Tour de France from Briançon to Alpe d’Huez, invoking a Lambda function to generate the required load and match the profile of the stage. Climbing the legendary and gruelling Col du Galibier and Col de la Croix de Fer on two wheels is significantly harder than on DynamoDB.
ConsumedWriteCapacityUnits
Lambda Invocations
The correlation between the ConsumedWriteCapacityUnits that we pay on DynamoDB – as there is no provisioned one – and the Lambda Invocations metrics is pretty striking. And they both match the route we had to cycle.
Conclusions
DynamoDB is indeed elastic but It is just a game, do not take me too seriously. Now go out there and start cycling!
AWS ha recentemente annunciato che le distribuzioni CloudFront supportano richieste HTTP versione 3 (HTTP/3) su QUIC. L’utilizzo di HTTP/3 è opzionale ma fornisce tempi di risposta più rapidi e maggiore sicurezza rispetto alle versioni HTTP precedenti.
Il supporto HTTP/3 si basa su s2n-quic, una implementazione open source in Rust di QUIC migliorando le prestazioni e la user experience dell’utente. Channy Yun, principal principal developer advocate presso AWS, spiega come funziona HTTP/3:
HTTP/3 utilizza QUIC e supera molte delle limitazioni del protocollo TCP. Quando si utilizza HTTP/2 esistente su TCP e TLS, TCP necessita di un handshake per stabilire una sessione tra un client e un server e anche TLS necessita del proprio handshake per garantire che la sessione sia protetta. Ogni handshake deve compiere l’intero viaggio di andata e ritorno tra client e server, che può richiedere molto tempo quando client e server sono molto distanti. Tuttavia, QUIC ha bisogno solo di un singolo handshake per stabilire una sessione sicura.
Yan Cui, consulente cloud e AWS Serverless Hero, auspica:
Le distribuzioni CloudFront gestite come quelle gestite da Amplify, AppSync e API Gateway saranno automaticamente abilitate a HTTP/3?
Essendo una impostazione opzionale non di default, HTTP/3 non è al momento disponibile per le API dei servizi gestiti AWS. Per abilitare HTTP/3 su una distribuzione CloudFront, bisogna modificare la configurazione utilizzando la console, l’API UpdateDistribution o CloudFormation. I client che non supportano HTTP/3 possono comunque utilizzare versioni HTTP precedenti.
Yun aggiunge:
HTTP/3 offre vantaggi a tutti gli utenti CloudFront sotto forma di tempi di connessione più rapidi, stream multiplexing e meno round trip nel processo di handshake.
Amazon Cloudfront non è l’unico CDN che supporta il nuovo standard, con il supporto QUIC e HTTP/3 disponibile su Cloudflare dal 2019. Google Cloud CDN e HTTPS Load Balancing supportano HTTP/3 dallo scorso anno. Google afferma che sul proprio motore di ricerca la feature ha ridotto la latenza del 2% e i tempi di rebuffer video su YouTube del 9%.
Il supporto HTTP/3 è disponibile in tutte le edge location CloudFront senza costi aggiuntivi. Affinché i client e le distribuzioni utilizzino HTTP/3, i client devono supportare TLSv1.3 e Server Name Indication (SNI). CloudFront supporta la migrazione della connessione a HTTP/3 per cambiare rete senza perdere la connessione.
Vuoi leggere altre news su AWS?
Amazon Redshift Serverless è GA
Amazon annuncia le istanze EC2 Mac M1 per sviluppare e testare applicazioni su macOS