Machine Learning Data Pipelines Deep Dive

An intense workshop designed to operationalize machine learning. The focus of this full-day workshop is to divide specializations; data engineer and data scientist. The data engineer ensures that data is delivered, manipulated, and harnessed. The data engineer does this so that it is useful for the data scientist. The data engineer is versant in Java and Scala.

The data scientist uses that data, does their own cleaning, and investigates possible patterns designing a machine learning model that we can use to either find regressions or classifications for our data. The data scientists use Python, Jupyter Notebooks, Tensorflow, and Matplotlib as their tools of choice for constructing a machine learning model to make decisions about the data.

How do we move information in real-time and connect machine learning models to make decisions on our business data? This presentation goes through machine learning and Kafka tools that would help achieve that goal.

In this workshop, we start with machine learning for beginners; what is training, and what is testing? How do you clean and prepare the data? We then use a neural network to create a model that classifies newspaper articles. From there, we operationalize it by creating a docker image that we can deploy onto Kubernetes. Then we apply Redis for caching and Kafka and Kafka Streams from real-time machine learning data processing. We then sum up with a discussion of other processing solutions and how KSQLDB can fit into your data processing picture.

About Daniel Hinojosa

Daniel is a programmer, consultant, instructor, speaker, and recent author. With over 20 years of experience, he does work for private, educational, and government institutions. He is also currently a speaker for No Fluff Just Stuff tour. Daniel loves JVM languages like Java, Groovy, and Scala; but also dabbles with non JVM languages like Haskell, Ruby, Python, LISP, C, C++. He is an avid Pomodoro Technique Practitioner and makes every attempt to learn a new programming language every year. For downtime, he enjoys reading, swimming, Legos, football, and barbecuing.

More About Daniel »