AWS Kinesis Data Streams  sandbox 

Table of Contents

kinesis-data-streams

  • Real-time data ingestion, aggregation and then loading the data into a data-warehouse like [[#Redshift]] or [[#EMR]] cluster.
  • Replicated across 3-AZ, so high availability, and data durability.
  • Auto-scaling
  • Delay between data ingestion, and retrieval is typically less than a second.
  • Multiple applications can consume data from the same stream and do different things in parallel, like for ex - processing and archiving.

Use cases

  • Logs intake, processing
  • Real-time metrics and reporting
  • Real-time streaming
  • Complex stream processing like DAG (Directed Acyclic Graphs) by ingesting from multiple [[#Kinesis#Data Streams]].