AWS has introduced a real-time event processing service called Kinesis. Amazon’s description of Kinesis: “A fully managed service for real-time processing of streaming data at massive scale.” Put another way, Kinesis is designed to support applications that generate enormous numbers of events that an application or organization want to store, analyze, or operate upon.
What is an event? Most people think of an event as, well, an event that occurs at a moment of time — some noticeable or important bit of information that is generated by one entity that another entity wants to keep track of. Examples that Amazon offers up in its Kinesis introduction are application log and clickstream records — both obvious generators of large amounts of real-time data that organizations might very well like to capture, track, and analyze. Both area aspects of IT applications that today are quite difficult to manage in high-throughput situations, and Kinesis is an excellent solution to address these application requirements.
However, I think that Kinesis, much like EC2 before it, will become a platform that enables entirely new application categories, reaching far beyond the initial uses envisioned by early users and even Amazon itself. Specifically, I think Kinesis will be an incredible enabler of the Internet of Things and will become the de facto backbone of communication from a society suffused with devices. Given my involvement in the area, I was very interested in Kinesis when it was announced and dived into the documentation to understand it better.
While the potential of Kinesis is clear, the details of the service — its architecture, how it operates, how users can integrate it — are not well-known or, frankly, that easy to understand. Rather than force every person interested in Kinesis to replicate my journey to understanding, I decided to share what I learned via a video tutorial, which you can click on at the top of this post. I hope you enjoy it and find it useful. I welcome any comments or questions.