Spark, Mesos, Akka, Cassandra and Kafka (SMACK) stack real time Big Data
Enterprise Big Data Cloud architect/developer need to use Big data solutions deployed in Cloud for reliability, scalability, agility, availability, and security. There are no comprehensive end-to-end hands on experience for Developers and Enterprise architects have to be able to design Big data solutions which can maximize for linear scalability and high availability. Why? Big data Analytics use-cases require sub-second response time with at least 99.99 % availability using massively parallel processing.
We will start with Apache Cassandra. Apache Cassandra is one of the best solutions for storing data for high availability and scalability. Modeling strategies help in designing solutions based on the application flow. Later, we will explore Cassandra time series data modeling.
Big Data applications require a faster speed of data processing and analysis. We will study topology with examples for building Spark applications. We will deep dive into Apache Spark applications, which connect into Cassandra. Spark streaming is used for processing real-time data.
We will explore Spark, Mesos, Akka, Cassandra and Kafka (SMACK) stack. We will look at how SMACK stack is helping build massively scalable applications in the cloud. Mesos is distributed systems kernel that can run on every machine and provides applications like Hadoop, Spark, Kafka, Elastic Search with APIs for resource management and scheduling across entire datacenter and cloud environments. Mesos help with scalability to 10,000 nodes and is fault-tolerant replicated master slave using ZooKeeper. Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM. Kafka is a distributed publish-subscribe messaging system that is designed to be fast, scalable, and durable.
The objective for this talk is to give clear direction on how to create scalable cloud ready Big Data applications with practical recopies to solve common problems. Building Elastic, Resilient, Scalable, Multi-workload, Efficient, and Isolated architecture using SMACK stack.
Enterprise Big Data Cloud architect/developer need to use Big data solutions deployed in Cloud for reliability, scalability, agility, availability, and security. There are no comprehensive end-to-end hands on experience for Developers and Enterprise architects have to be able to design Big data solutions which can maximize for linear scalability and high availability. Why? Big data Analytics use-cases require sub-second response time with at least 99.99 % availability using massively parallel processing.
We will start with Apache Cassandra. Apache Cassandra is one of the best solutions for storing data for high availability and scalability. Modeling strategies help in designing solutions based on the application flow. Later, we will explore Cassandra time series data modeling.
Big Data applications require a faster speed of data processing and analysis. We will study topology with examples for building Spark applications. We will deep dive into Apache Spark applications, which connect into Cassandra. Spark streaming is used for processing real-time data.
We will explore Spark, Mesos, Akka, Cassandra and Kafka (SMACK) stack. We will look at how SMACK stack is helping build massively scalable applications in the cloud. Mesos is distributed systems kernel that can run on every machine and provides applications like Hadoop, Spark, Kafka, Elastic Search with APIs for resource management and scheduling across entire datacenter and cloud environments. Mesos help with scalability to 10,000 nodes and is fault-tolerant replicated master slave using ZooKeeper. Akka is a toolkit and runtime for building highly concurrent, distributed, and resilient message-driven applications on the JVM. Kafka is a distributed publish-subscribe messaging system that is designed to be fast, scalable, and durable.
The objective for this talk is to give clear direction on how to create scalable cloud ready Big Data applications with practical recopies to solve common problems. Building Elastic, Resilient, Scalable, Multi-workload, Efficient, and Isolated architecture using SMACK stack.
About Rohit Bhardwaj
Rohit Bhardwaj is a Director of AI & Data Architecture at Salesforce, where he focuses on enterprise AI, agentic systems, cloud-native architecture, distributed systems, data platforms, security, and large-scale transformation.
Over his career, Rohit has designed and led complex enterprise platforms across AWS, Google Cloud, microservices, real-time data, API ecosystems, resilient distributed systems, and AI-enabled architectures. His work increasingly focuses on the challenges enterprises face as software evolves from deterministic services to AI-native and agentic systems—particularly around reliability, governance, evidence, security, observability, cost, and safe autonomy.
Rohit is the author of System Design with AI Interview Guide: Designing Scalable, Agentic, and Defensible Systems, published by Apress. The book presents a modern approach to system design covering scalability, distributed systems, AI architecture primitives, security, reliability, economics, agentic systems, and real-world architectures including e-commerce, ride sharing, payments, fraud detection, messaging, video streaming, file storage, and search. (Springer Link)
Book:
Amazon: https://a.co/d/09Zs1twa
Publisher / Springer Nature: https://link.springer.com/book/10.1007/979-8-8688-2782-2
O'Reilly: https://learning.oreilly.com/library/view/system-design-with/9798868827822/
Rohit is also an O’Reilly instructor and a frequent speaker at technology conferences including No Fluff Just Stuff, UberConf, GIDS, and other international events. His talks focus on practical architecture lessons from building and operating complex systems, including AI control planes, trusted agents, inference at scale, evidence-first RAG, AI security, distributed-system failure, and AI-era software architecture.
As a trusted advisor and architecture leader, Rohit works at the intersection of business strategy and deep technical architecture—helping teams translate complex business problems into scalable, resilient, secure, and economically sustainable systems.
Rohit holds an MBA in Corporate Entrepreneurship from Babson College and graduate-level education in Computer Science from Boston University and Harvard University.
Connect with Rohit:
LinkedIn: http://linkedin.com/in/rohit-bhardwaj-cloud
X / Twitter: @rbhardwaj1