Release v0.3.1 (What’s new?).

Documentation Status!--None.svg?style=social!--None.svg?style=social

Welcome to abstract_producer Documentation#

📔 See Full Documentation HERE.

This library provides the abstraction of data producer, which is a common client sending data to stream processing system, such as Apache Kafka, Apache Pulsar, AWS Kinesis, AWS SQS, AWS CloudWatch logs, etc.

It has the following business critical features out-of-box:

  1. group records into micro batch to maximize the utilization of network bandwidth.

  2. use local write-ahead-log to prevent unexpected error and data loss.

  3. auto-retry using exponential backoff strategy to handle transient error.

With this library, it is easy to create data producer client library for any stream processing system.

Usage Examples#


abstract_producer is released on PyPI, so all you need is to:

$ pip install abstract-producer

To upgrade to latest version:

$ pip install --upgrade abstract-producer

Table of Content#

About the Author#

(\ (\
( -.-)o

Sanhe Hu is a seasoned software engineer with a deep passion for Python development since 2010. As an author and maintainer of 20+ open-source projects, I bring a wealth of experience to the table. As a Senior Solution Architect and Subject Matter Expert in Amazon Web Services, Cloud Engineering, DevOps, Big Data, and Machine Learning, I thrive on helping clients with platform design, enterprise architecture, and strategic roadmaps.

Talk is cheap, show me the code:

API Document#