zz Metamorphosis - Distributed Message Middleware - Open Source China

Mar 16, 201213

AI Translation

This post is translated from Chinese into English through AI.View Original

AI-generated summary

Metamorphosis is an open-source distributed messaging middleware similar to LinkedIn's Kafka. It is designed for high performance, high availability, and scalability, with features such as sequential message storage, high throughput, and support for local and XA transactions. It is widely used in scenarios with high throughput, sequential messaging, broadcasting, and log data transmission, particularly in Taobao and Alipay. The overall structure and main features of Metamorphosis are described, including its support for distributed producers, servers, and consumers, sequential message storage, high performance, support for message ordering, local and XA transactions, client pull with random reading and zero-copy, support for consumer-side transactions, message broadcasting, asynchronous message sending, support for HTTP protocol, message retry and recovery, transparent data migration and scaling, client-side storage of consumer state, support for synchronous and asynchronous replication, and group commit.

Metamorphosis is a high-performance, highly available, and scalable distributed message middleware, similar to LinkedIn's Kafka. It has features such as sequential message storage, high throughput, and support for local and XA transactions. It is suitable for scenarios with high throughput, sequential messages, broadcasting, and log data transmission. It is widely used in Taobao and Alipay and is now open source.

Overall structure:

Internal structure:

Main features:

Producers, servers, and consumers can all be distributed

Sequential message storage

Extremely high performance, large throughput

Supports message ordering

Supports local and XA transactions

Client pull, random read, utilizing the sendfile system call, zero-copy, batch data pulling

Supports consumer-side transactions

Supports message broadcasting mode

Supports asynchronous message sending

Supports HTTP protocol

Supports message retry and recovery

Data migration and capacity expansion are transparent to users

Consumer state is saved on the client

Supports synchronous and asynchronous replication

Supports group commit

More...

Through Metamorphosis - Distributed Message Middleware - Open Source China.