The old-fashioned batch processing is not applicable in the current applications. Pipelines need to respond to events in real time when businesses rely on real-time data to track user behavior, to process financial transactions, or to monitor Internet of Things (IoT) devices, instead of hours after the event.
Why Event-Driven Architecture Matters
Event-driven processing versus batch processing is a paradigm shift in the flow of data through the systems. With batch pipelines, data is idle until it is run. In event pipelines, each change is followed by an immediate response. This difference is crucial in the development of fraud detection systems that demand sub-second response times or in systems that offer recommendations that are updated in real-time according to who is currently using it.