Use EzyKafka For Push Message System

Updated at 1685687630000
In a push message system, maybe there're a lot of messages in a time need to send to clients. So we can use kafka like a message broker to receive messages from producers and wait for consumers poll the messages to consume.

1. Simple System Design

In above design we have:

  1. Message sender client: it will call API to push messages.
  2. HTTP API server: to receive the messages from the client and push the messages to Kafka. We will EzyHTTP and Ezykafka for this one.
  3. Kafka: Kafka cluster.
  4. Socket server</code>: to poll the messages from kafka and push them to clients. We will [EzyFox Server](https://github.com/youngmonkeys/ezyfox-server) and EzyKafka for this one.
  5. Message receiver clients: to receive the messages from socket server.

2. Create HTTP Api Server

Firstly, let's add dependencies:

<dependency>
    <groupId>com.tvd12</groupId>
    <artifactId>ezymq-kafka</artifactId>
    <version>1.2.5</version>
</dependency>
<dependency>
    <groupId>com.tvd12</groupId>
    <artifactId>ezyhttp-server-boot</artifactId>
    <version>1.2.8.1</version>
</dependency>

The latest version can be found in the Maven Central repository.

You can configure EzyKafka like this:

@EzyConfigurationBefore
public class KafkaConfig implements EzySingletonFactoryAware, EzyBeanConfig {
    @Setter
    private EzySingletonFactory singletonFactory;
    @Override
    public void config() {
        EzyKafkaProxy kafkaProxy = EzyKafkaProxy.builder()
            .scan("org.youngmonkeys.example.push_message.gateway.api.kafka")
            .build();
        singletonFactory.addSingleton("messageProducer", kafkaProxy.getProducer("message"));
    }
}

You can add to your configuration file like this:

# for application.yaml
kafka:
  producers:
    message:
      topic: message
# for application.properties
kafka.producers.message.topic=message

You can create a controller like this:

@Controller("/api/v1/message")
public class MessageController {
    @EzyAutoBind
    private EzyKafkaProducer messageProducer;
    @DoPost("/push")
    public boolean pushMessage(@RequestBody Message message) {
        messageProducer.send(
            "push",
            new KafkaMessage(message.getUsername(), message.getData())
        );
        return Boolean.TRUE;
    }
}

3. Create Socket Server

Fistly, we need add dependency:

<dependencies>
    <dependency>
        <groupId>com.tvd12</groupId>
        <artifactId>ezyfox-server-embedded</artifactId>
        <version>1.2.8.1</version>
    </dependency>
    <dependency>
        <groupId>com.tvd12</groupId>
        <artifactId>ezymq-kafka</artifactId>
        <version>1.2.5</version>
    </dependency>
</dependencies>

The latest version can be found in the Maven Central repository.

If you don't need to use ezyfox-server-embedded you can look at Create Ezyfox Server Server Project to see how to create an EzyFox Server project.

You can config EzyKafka like this:

@Setter
@EzyConfigurationAfter
public class KafkaConfig implements EzyBeanConfig {
    @EzyAutoBind
    private EzyResponseFactory responseFactory;
    @Override
    public void config() {
        EzyKafkaProxy.builder()
            .scan("org.youngmonkeys.example.push_message.kafka")
            .addSingleton(responseFactory)
            .build();
    }
}

You can add to your configuration file like this:

# for application.yaml
kafka:
  consumers:
    message:
      topic: message
# for application.properties
kafka.consumers.message.topic=message

You can create a message data class like this:

@EzyMessage
@Data
@AllArgsConstructor
@NoArgsConstructor
public class KafkaMessage {
    private String username;
    private EzyObject data;
}

And then you can create a message handler class like this:

@AllArgsConstructor
@EzyKafkaHandler(topic = "message", command = "push")
public class KafkaMessageHandler implements EzyKafkaMessageHandler<KafkaMessage> {
    private final EzyResponseFactory responseFactory;
    @Override
    public void process(KafkaMessage message) {
        responseFactory.newObjectResponse()
            .command("message")
            .usernames(message.getUsername())
            .data(message.getData())
            .execute();
    }
}

4. Create a Message Receiver Client

You can create any client for web and mobile to receive the messages from socket server. You can find out some clients for android, iOS and web in push message example repo.

5. Run the system

  1. Run socket server first
  2. Next one, run HTTP Api Server
  3. Run a client, let's say you choose Vue.js client, you can run by command: npm start
  4. Let's login on the client with username youngmonkeys with any password (length >= 6)
  5. Let's run command:
		
curl --location --request POST 'http://localhost:8080/api/v1/message/push' 
--header 'Content-Type: application/json' 
--data-raw '{
    "username": "youngmonkeys",
    "data" : {
        "title": "Hello",
        "content": "World"
    }
}'

And you can see the result like this:

6. Summary

With EzyKafka you can create a message system quickly and easily. For entire source code you can clone push-message example from Github.