Overview

Pagination is one of the common features used in almost any software project. There are three types of pagination:

Fetch all then paginate: Convenient but slow, capable of displaying the order of records.
Offset pagination: Quite convenient but slow, and gets slower with larger data. It can display the order of records.
Cursor pagination: Complex but ensures performance in all cases, but cannot display the order of records.

EzyPlatform will provide code bases for options 2 and 3.

Cursor Pagination

We need to create the following classes:

XxxFilter: Contains information about filtering conditions.
XxxPaginationParameter: Contains information about comparison conditions for pagination.
XxxPaginationParameterConverter: Used to convert between text and PaginationParameter objects.
PaginationXxxRepository: Communicates with the database.
PaginationXxxService: Calls the Repository class and converts Entity to Model.

1. XxxFilter

You should create a base interface inheriting from CommonStorageFilter to characterize the pagination business you want to implement. For example, here I declare the interface DefaultBookFilter to characterize the book pagination business.

public interface BookFilter extends CommonStorageFilter {}

Next, you can declare a specific class implementing the base interface. This class will contain data fields for filtering conditions. Fields that are null mean they are ignored.

You also need to implement one or all of the following functions:

selectionFields: List of selected fields, default is all fields in the table.
countField: Field used for counting in the count query.
decorateQueryStringBeforeWhere: Add to the query before the where keyword, you can use join here.
matchingCondition: Filtering condition for the query after the where keyword.
groupBy: List of fields needed for the summary query with group by, separated by commas.

For example, the DefaultBookFilter class provides filtering conditions for the book pagination business.

@Getter
@Builder
public class DefaultBookFilter implements BookFilter {
    public final String status;
    public final String likeKeyword;
    public final Collection<String> keywords;

    @Override
    public void decorateQueryStringBeforeWhere(
        StringBuilder queryString
    ) {
        queryString.append(" INNER JOIN Product a ON e.productId = a.id");
        if (keywords != null) {
            queryString.append(" INNER JOIN DataIndex k ON e.productId = k.dataId");
        }
    }

    @Override
    public String matchingCondition() {
        EzyQueryConditionBuilder answer = new EzyQueryConditionBuilder();
        if (status != null) {
            answer.append("e.status = :status");
        }
        if (keywords != null) {
            answer
                .and("k.dataType = '" + TABLE_NAME_PRODUCT + "'")
                .and("k.keyword IN :keywords");
        }
        if (likeKeyword != null) {
            String query = new EzyQueryConditionBuilder()
                .append("(")
                .append("a.productName LIKE CONCAT('%', :likeKeyword, '%')")
                .or("a.productCode LIKE CONCAT('%', :likeKeyword, '%')")
                .or("e.author LIKE CONCAT('%', :likeKeyword, '%')")
                .or("e.affiliate LIKE CONCAT('%', :likeKeyword, '%')")
                .or("e.distributionCompany LIKE CONCAT('%', :distributionCompany, '%')")
                .or("e.publisher LIKE CONCAT('%', :publisher, '%')")
                .append(")")
                .build();
            answer.or(query);
        }
        return answer.build();
    }
}

2. XxxPaginationParameter

Similar to the filter, you should create a base interface inheriting from CommonPaginationParameter to characterize the pagination business you want to implement. For example, here I declare the interface BookPaginationParameter to characterize the book pagination business.

public interface BookPaginationParameter extends CommonPaginationParameter {}

Next, you can declare a specific class implementing the base interface. This class will contain data fields for pagination conditions. Fields that are null mean they are ignored.

You also need to implement one or all of the following functions:

selectionFields: List of selected fields, default is all fields in the table.
decorateQueryStringBeforeWhere: Add to the query before the where keyword, you can use join here.
paginationCondition: Pagination condition for the query after the where keyword.
groupBy: List of fields needed for the summary query with group by, separated by commas.
orderBy: Fields and sorting direction after the order by keyword.
sortOrder: Sorting direction used to convert from text to the corresponding PaginationParameter class.

Below is the IdDescBookPaginationParameter class providing pagination conditions for the book pagination business.

@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public class IdDescBookPaginationParameter implements BookPaginationParameter {
    public Long id;

    @Override
    public String paginationCondition(boolean nextPage) {
        if (id == null) {
            return null;
        }
        return nextPage
            ? "e.id < :id"
            : "e.id > :id";
    }

    @Override
    public String orderBy(boolean nextPage) {
        return nextPage
            ? "e.id DESC"
            : "e.id ASC";
    }

    @Override
    public String sortOrder() {
        return BookPaginationSortOrder.ID_DESC.toString();
    }
}

3. XxxPaginationParameterConverter

When calling APIs from the client, data can only be in text form. For example, the query for searching by keyword looks like this:

https://admin.ezyplatform.com/api/v1/books-store/books
?limit=12
&nextPageToken=eyJzb3J0T3JkZXIiOiJJRF9ERVNDIiwidmFsdWUiOnsiaWQiOjQ4ODN9fQ==

The value of nextPageToken will be the pagination condition, which is in base64 format. When decoded, we get data in text (JSON) format:

{"sortOrder":"ID_DESC","value":{"id":4883}}

From this text data, we need to convert it to the corresponding PaginationParameter object.

When the API is executed, the client also receives data like this:

{
    "items": [
        ... list of items
    ],
    "pageToken": {
        "next": "eyJzb3J0T3JkZXIiOiJJRF9ERVNDIiwidmFsdWUiOnsiaWQiOjQ4NzF9fQ==",
        "prev": "eyJzb3J0T3JkZXIiOiJJRF9ERVNDIiwidmFsdWUiOnsiaWQiOjQ4ODJ9fQ=="
    },
    "continuation": {
        "hasNext": true,
        "hasPrevious": true
    },
    "count": 12,
    "total": 4737,
    "timestamp": 1690813511661
}

Here, next and prev are also pagination conditions in base64 format for the

next and previous pages. To get next and prev, we need to convert the PaginationParameter object to text (JSON) format and then convert it to base64 format.

The conversion between JSON data and PaginationParameter object is the role of the PaginationParameterConverter class.

For example, suppose we have an enum class:

public enum BookPaginationSortOrder {
    ID_DESC
}

Containing the value ID_DESC representing sorting records by the id field in descending order. We can create the BookPaginationParameterConverter class as follows to convert between text data (JSON) and the corresponding PaginationParameter object, specifically the pair of enum BookPaginationSortOrder.ID_DESC and the IdDescBookPaginationParameter class.

public class BookPaginationParameterConverter
    extends ComplexPaginationParameterConverter<
        String,
        ProductBookModel
    > {

    private final ClockProxy clock;

    public BookPaginationParameterConverter(
        ClockProxy clock,
        PaginationParameterConverter converter
    ) {
        super(converter);
        this.clock = clock;
    }

    @Override
    protected void mapPaginationParametersToTypes(
        Map<String, Class<?>> map
    ) {
        map.put(
            BookPaginationSortOrder.ID_DESC.toString(),
            IdDescBookPaginationParameter.class
        );
        map.put(
            BookPaginationSortOrder.RELEASED_AT_DESC_ID_DESC.toString(),
            ReleasedAtDescIdDescBookPaginationParameter.class
        );
    }

    @Override
    protected void addPaginationParameterExtractors(
        Map<String, Function<ProductBookModel, Object>> map
    ) {
        map.put(
            BookPaginationSortOrder.ID_DESC.toString(),
            model -> new IdDescBookPaginationParameter(
                model.getProductId()
            )
        );
        map.put(
            BookPaginationSortOrder.RELEASED_AT_DESC_ID_DESC.toString(),
            model -> new ReleasedAtDescIdDescBookPaginationParameter(
                clock.toLocalDateTime(model.getReleasedAt()),
                model.getProductId()
            )
        );
    }
}

4. PaginationXxxRepository

This class uses Filter and PaginationParameter to generate JPQL queries and access the database. You need to implement the getEntityType function to return the Entity class. The example below is the PaginationBookRepository class for the book pagination business.

public class PaginationBookRepository extends CommonPaginationRepository<
    BookFilter,
    BookPaginationParameter,
    Long,
    ProductBook> {

    @Override
    protected Class<ProductBook> getEntityType() {
        return ProductBook.class;
    }
}

5. PaginationXxxService

This class is responsible for:

Using the PaginationParameterConverter class to convert between text data and PaginationParameter objects.
Calling the Repository to query data.
Converting data from Entity to Model.

You need to implement:

serializeToPageToken: Function to convert PaginationParameter object to text.
deserializePageToken: Function to convert text data to PaginationParameter object.
defaultPaginationParameter: Function to provide default PaginationParameter when no information about PaginationParameter is provided.
convertEntity: Function to convert data from Entity to Model.

In the PaginationBookService class below for book pagination business, we have:

Id type as Long.
Entity class as ProductBook.
Filter class as BookFilter.
PaginationParameter class as BookPaginationParameter.
Model class as ProductBookModel.

public class PaginationBookService extends CommonPaginationService<
    ProductBookModel,
    BookFilter,
    BookPaginationParameter,
    Long,
    ProductBook> {

    private final EcommerceEntityToModelConverter entityToModelConverter;

    public PaginationBookService(
        PaginationBookRepository repository,
        EcommerceEntityToModelConverter entityToModelConverter,
        BookPaginationParameterConverter paginationParameterConverter
    ) {
        super(repository, paginationParameterConverter);
        this.entityToModelConverter = entityToModelConverter;
    }


    @Override
    protected ProductBookModel convertEntity(ProductBook entity) {
        return entityToModelConverter.toModel(entity);
    }

    @Override
    protected BookPaginationParameter defaultPaginationParameter() {
        return new IdDescBookPaginationParameter();
    }
}

Usage

When using, you only need to focus on the PaginationService and Filter classes:

Step one: Create a filter object.
Step two: Use the PaginationService to retrieve a data page.

Below is an example:

@Service
@AllArgsConstructor
public class AdminBookControllerService {

    private final AdminPaginationBookService paginationBookService;
    private final AdminBookModelDecorator bookModelDecorator;

    public PaginationModel<AdminBookResponse> getBooks(
        BookFilter filter,
        String nextPageToken,
        String prevPageToken,
        boolean lastPage,
        int limit,
        long currencyId,
        String currencyFormat
    ) {
        PaginationModel<ProductBookModel> pagination = getPaginationModel(
            paginationBookService,
            filter,
            nextPageToken,
            prevPageToken,
            lastPage,
            limit
        );
        return

 bookModelDecorator.decorate(
            pagination,
            currencyId,
            currencyFormat
        );
    }
}

Another way of usage when we don't want to use default pagination parameters:

public class ApiContractController {

    private final WebPaginationAddressService paginationAddressService;
    private final WebChainValidator chainValidator;
    private final WebCommonValidator webCommonValidator;
    private final WebAddressPaginationParameterConverter addressPaginationParameterConverter;

    @DoGet("/chains/{chainId}/contracts")
    public PaginationModel<ContractModel> chainsChainIdContractsGet(
        @PathVariable(value = "chainId") String chainId,
        @RequestParam(value = "nextPageToken") String nextPageToken,
        @RequestParam(value = "prevPageToken") String prevPageToken,
        @RequestParam(value = "lastPage") boolean lastPage,
        @RequestParam(value = "limit", defaultValue = "15") int limit
    ) {
        chainValidator.validateChainId(chainId);
        webCommonValidator.validatePageSize(limit);
        AddressFilter filter = new AddressTypeAddressFilter(
            chainId,
            AddressType.CONTRACT
        );
        String actualNextPageToken = nextPageToken;
        String actualPrevPageToken = prevPageToken;
        String sortOrder = AddressPaginationSortOrder
            .FIRST_BLOCK_HEIGHT_DESC_ADDRESS_DESC
            .toString();
        if (nextPageToken == null && prevPageToken == null) {
            if (lastPage) {
                actualPrevPageToken = addressPaginationParameterConverter
                    .getDefaultPageToken(sortOrder);
            } else {
                actualNextPageToken = addressPaginationParameterConverter
                    .getDefaultPageToken(sortOrder);
            }
        }
        return PaginationModelFetchers.<AddressModel>getPaginationModel(
            paginationAddressService,
            filter,
            actualNextPageToken,
            actualPrevPageToken,
            lastPage,
            limit
        ).map(AddressModel::toContractModel);
    }
}

Offset pagination

Pagination by offset will be much simpler than using cursors because the offset itself acts as a pagination condition, so the number of classes we need to create will be fewer, and the complexity will also decrease significantly.

1. Create the XxxOffsetPaginationParameter class

This class provides offset, sorting field, and sorting direction, for example:

public class YmProjectOffsetPaginationParameter
    extends OffsetPaginationParameter {

    public YmProjectOffsetPaginationParameter(
        long offset
    ) {
        super(offset, "e.priority DESC");
    }
}

2. Create the XxxPaginationRepository class

Similar to cursor pagination, for example:

@EzyRepository
public class WebYmPaginationProjectRepository extends PaginationRepository<
    PostFilter,
    YmProjectOffsetPaginationParameter,
    Long,
    Post> {

    @Override
    protected Class<Post> getEntityType() {
        return Post.class;
    }
}

3. Create the XxxPaginationService class

Similar to cursor pagination, for example:

@Service
public class WebYmPaginationProjectService extends OffsetPaginationService<
    PostModel,
    PostFilter,
    YmProjectOffsetPaginationParameter,
    Long,
    Post> {

    private final WebEzyArticleEntityToModelConverter entityToModelConverter;

    public WebYmPaginationProjectService(
        WebYmPaginationProjectRepository repository,
        WebEzyArticleEntityToModelConverter entityToModelConverter
    ) {
        super(
            repository,
            YmProjectOffsetPaginationParameter::new
        );
        this.entityToModelConverter = entityToModelConverter;
    }

    @Override
    protected PostModel convertEntity(Post entity) {
        return entityToModelConverter.toModel(entity);
    }
}

Usage

Similar to cursor pagination, for example:

private final WebYmPaginationProjectService paginationProjectService;
  private final WebYmProjectModelDecorator projectModelDecorator;

  public PaginationModel<WebYmProjectResponse> getProjectPagination(
      String nextPageToken,
      int limit
  ) {
      PaginationModel<PostModel> pagination = getPaginationModel(
          paginationProjectService,
          DefaultPostFilter
              .builder()
              .postType(YmPostType.PROJECT.toString())
              .postStatus(PostStatus.PUBLISHED.toString())
              .build(),
          nextPageToken,
          null,
          false,
          limit
      );
      return projectModelDecorator.decorate(pagination);
  }

Some utility functions

EzyPlatform provides some utility functions to help you easily create the PaginationParameter class as follows:

Values.isAllNull: This function helps check if all values are null.
PaginationParameters.makePaginationConditionDesc: This function helps set the paginationCondition function more easily.
PaginationParameters.makeOrderByDesc: This function helps set the orderBy function more easily.

For example:

import lombok.AllArgsConstructor;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;

import java.time.LocalDateTime;

import static org.youngmonkeys.ezyplatform.pagination.PaginationParameters.makeOrderByDesc;
import static org.youngmonkeys.ezyplatform.pagination.PaginationParameters.makePaginationConditionDesc;
import static org.youngmonkeys.ezyplatform.util.Values.isAllNull;

@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public class ReleasedAtDescIdDescBookPaginationParameter
    implements BookPaginationParameter {

    public LocalDateTime releasedAt;
    public Long id;

    @Override
    public String paginationCondition(boolean nextPage) {
        return isEmpty()
            ? null
            : makePaginationConditionDesc(
                nextPage,
                "releasedAt",
                "id"
            );
    }

    @Override
    public String orderBy(boolean nextPage) {
        return makeOrderByDesc(nextPage, "releasedAt", "id");
    }

    @Override
    public String sortOrder() {
        return BookPaginationSortOrder.RELEASED_AT_DESC_ID_DESC.toString();
    }

    @Override
    public boolean isEmpty() {
        return isAllNull(releasedAt, id);
    }
}

Example source code

You can take a look here.

Pagination Implementation

Overview

Cursor Pagination

1. XxxFilter

2. XxxPaginationParameter

3. XxxPaginationParameterConverter

4. PaginationXxxRepository

5. PaginationXxxService

Usage

Offset pagination

1. Create the XxxOffsetPaginationParameter class

2. Create the XxxPaginationRepository class

3. Create the XxxPaginationService class

Usage

Some utility functions

Example source code