Stories Tag: event-driven-architecture

Towards Event-Driven Architecture for Machine Learning — Part I | by Venkat Ganesh | Oct, 2021 | Medium

admin 4 days ago in

In this article, I will talk about the importance of event-driven architectures for Machine Learning and will also discuss a design pattern for building a large-scale ML training Infrastructure for a Virtual Agent use case using Knative Eventing Let’s take a real-life example to explain this, imagine you’re mailing a package from Boston to NYC. You buy a shipping label online, stick it on to…

Simplifying Kafka Connectors Configuration with an Integration Language | by Sebastien Goasguen | Jul, 2021 | Medium

admin 3 months ago in

In a previous post about configuring Kafka sources and sinks using Knative and Kubernetes API objects you might have told yourself “face full of YAML”. Indeed, one of the most painful points with Kubernetes since its inception has been the face full of YAML problem. Once you start declaring the desired state of your application with Kubernetes objects you end up with a lot and…

Sending Messages to Kafka in Kubernetes | by Sebastien Goasguen | Jun, 2021 | Medium

admin 4 months ago in

In this previous blog I showed how to consume messages from Kafka declaratively. Today I am going to show you how to do the opposite, how to produce messages into Kafka aka how to send Messages to a Kafka topic declaratively. Head over to the previous blog for context. Basically we are trying to write Kubernetes manifests to declare how we send and receive or…

Deferred tasks in a microservice architecture | by Vlad Pavlenko | Feb, 2021 | Medium

admin 8 months ago in

I am not a native speaker, so I apologize for the translation in advance. Often in projects, there is a need to perform deferred tasks, such as sending email, push, and other specific tasks specific to the domain area of your application. The difficulties begin when the usual crontab is no longer enough, when batch processing is not suitable, and when each task unit has…

Auto-Scaling Your Long Running Workloads on Kubernetes With KEDA v2 | by Oren Lederman, PhD | Jan, 2021 | Medium

admin 9 months ago in

In one of my projects we use an event-based architecture to process new data as it arrives to make it available to our developers quickly. The data arrives in batches, and take anything from minutes to hours to process. Processing the data includes, for example, composing a video from individual images and saving it as a mp4 file, or performing a battery of tests created…

Data Streaming Platform Using Kafka, Kubernetes, MongoDB, InfluxDB and Grafana. Part 1 | by Harsh Dhillon | Oct, 2020 |...

admin 1 year ago in

I am building a data streaming platform for a visual effects production company. This article will follow my current architecture for the platform. Goal : Visualize the render jobs being spooled on our render farm to analyze which shows are taking up the most resources and allocate them appropriately. This architecture can be divided into 4 key parts, All our application containers will be deployed…

Teknasyon’un Sunucu Mimarisindeki Dönüşümü | by Doğan Aydın | Sep, 2020 | Teknasyon Engineering

admin 1 year ago in

Bugün DevOps, yazılım yaklaşımımızı, kodlama yapımız değiştiriyor. Teknasyon’un zaman içinde yaşadığı bu evrimi konu alan konuşmamızda bu süreç içinde neler yaşadık bunlardan Webrazzi Developer 2020 etkinliğinde bahsettik. Bu süreçi 4 döneme ayırdık: Her bir dönemin çıktısı olan süreci artıları ve eksileri ile birlikte değerlendirdik. Mimari yaklaşımımızda “Event Driven Architecture” modele geçişimizi ve bu mimarisel yaklaşım ile süreçlerin event tabanlı olması kazanımını Big Data Platformu üretiminde…

Serverless: dissecting the jargon (part 1) – Pedro Freitas – Medium

admin 1 year ago in

The tech industry likes to talk, share and create a lot of buzz around new terminology and new technology. In regards to advancements in cloud computing, the term “serverless” has been somewhat vague, heavily depending on the technology your stack is being built around, but in the end sums up to the following concepts, quoting the official definition from the Cloud Native Computing Foundation: Serverless…