Home
Projects
Download
Support
Sponsors
Sponsor Benefits
Buy Gifts
Donate Now
Join Us
Open Cooperation
Contribute
Contributors (20)
Events (1)
Outsourcing
Contact Us
Outsourcing
Open Cooperation
Sponsor
Others
Ask Us
About Us
Back to project ezyplatform

Pagination Implementation

Updated at 1693492047000

Overview

Pagination is one of the common features used in almost any software project. There are 3 types of pagination:

  1. Retrieve all and then paginate: Convenient but slow, capable of displaying the order of records.
  2. Offset-based pagination: Quite convenient but slow, and as the data grows, it becomes even slower, capable of displaying the order of records.
  3. Cursor-based pagination: Complex but guarantees performance in all cases, but cannot display the order of records.

EzyPlatform will provide codebase for the 2nd and 3rd options.

Cursor-based Pagination

We need to create the following classes:

Pagination.png
  1. XxxFilter: Contains information about filtering conditions.
  2. XxxPaginationParameter: Contains information about comparison conditions for pagination.
  3. XxxPaginationParameterConverter: Used to convert between text and PaginationParameter objects.
  4. PaginationXxxRepository: Communicates with the database.
  5. PaginationXxxService: Calls the Repository class and converts Entity to Model.

1. XxxFilter

You should create a base interface that inherits from CommonStorageFilter to characterize the pagination business logic you want to implement. For example, here I declare the AdminActivityFilter interface to represent the pagination of admin activity history. 

public interface AdminActivityFilter extends CommonStorageFilter {}

Next, you can declare a specific class that implements the base interface. This class will contain the data fields for filtering conditions, and any field with a null value will be ignored.

You also need to implement one or all of the following methods:

  1. selectionFields: List of selected fields, default is all fields in the table.
  2. countField: The field used for counting in the count query.
  3. decorateQueryStringBeforeWhere: Adds to the query before the where keyword, you can use join here.
  4. matchingCondition: The filtering condition for the query after the where keyword.
  5. groupBy: List of fields required for the group by aggregation query, separated by commas.

For example, the AdminAdminActivityFilter class provides the filtering condition for the pagination of admin activity history. 

@Getter
public class AdminAdminActivityFilter implements AdminActivityFilter {
    public final Long adminId;

    protected AdminAdminActivityFilter(Builder builder) {
        this.adminId = builder.adminId;
    }

    @Override
    public String matchingCondition() {
        EzyQueryConditionBuilder answer = new EzyQueryConditionBuilder();
        if (adminId != null) {
            answer.and("e.adminId = :adminId");
        }
        return answer.build();
    }

    public static Builder builder() {
        return new Builder();
    }

    public static class Builder
        implements EzyBuilder<AdminAdminActivityFilter> {

        private Long adminId;

        public Builder adminId(Long adminId) {
            this.adminId = adminId;
            return this;
        }

        @Override
        public AdminAdminActivityFilter build() {
            return new AdminAdminActivityFilter(this);
        }
    }
}

2. XxxPaginationParameter

Similar to the filter, you should also create a base interface that inherits from CommonPaginationParameter to characterize the pagination business logic you want to implement. For example, here I declare the interface AdminActivityPaginationParameter to represent the pagination of admin activity history. 

public interface AdminActivityPaginationParameter extends CommonPaginationParameter {}

Next, you can declare a specific class that implements the base interface. This class will contain the data fields for pagination conditions, and any field with a null value will be ignored.

You also need to implement one or all of the following methods:

  1. selectionFields: List of selected fields, default is all fields in the table.
  2. decorateQueryStringBeforeWhere: Adds to the query before the where keyword, you can use join here.
  3. paginationCondition: Pagination condition for the query after the where keyword.
  4. groupBy: List of fields required for the group by aggregation query, separated by commas.
  5. orderBy: Fields and sorting direction after the order by keyword.
  6. sortOrder: Sorting direction used to convert from text to the corresponding PaginationParameter class.

Below is the AdminIdDescAdminActivityPaginationParameter class, which provides the pagination condition for the pagination of admin activity history. 

@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public class AdminIdDescAdminActivityPaginationParameter implements AdminActivityPaginationParameter {
    public Long id;

    @Override
    public String paginationCondition(boolean nextPage) {
        if (id == null) {
            return null;
        }
        return nextPage ? "e.id < :id" : "e.id > :id";
    }

    @Override
    public String orderBy(boolean nextPage) {
        return nextPage ? "e.id DESC" : "e.id ASC";
    }

    @Override
    public String sortOrder() {
        return AdminActivityPaginationSortOrder.ID_DESC.toString();
    }
}

3. XxxPaginationParameterConverter

When we call an API from the client, the data can only be in text form. For example, the search query with this keyword: 

https://admin.ezyplatform.com/api/v1/admins/me/activities?limit=12&nextPageToken=eyJzb3J0T3JkZXIiOiJJRF9ERVNDIiwidmFsdWUiOnsiaWQiOjQ4ODN9fQ==

The value of nextPageToken will be the pagination condition, in base64 form. When decoded, we will receive data in text (json) form: 

{"sortOrder":"ID_DESC","value":{"id":4883}}

From this text data, we need to convert it back to the corresponding PaginationParameter object.

When the API is executed, the client will also receive data in this format: 

{
    "items": [
        ... list of items
    ],
    "pageToken": {
        "next": "eyJzb3J0T3JkZXIiOiJJRF9ERVNDIiwidmFsdWUiOnsiaWQiOjQ4NzF9fQ==",
        "prev": "eyJzb3J0T3JkZXIiOiJJRF9ERVNDIiwidmFsdWUiOnsiaWQiOjQ4ODJ9fQ=="
    },
    "continuation": {
        "hasNext": true,
        "hasPrevious": true
    },
    "count": 12,
    "total": 4737,
    "timestamp": 1690813511661
}

The next and prev values are also pagination conditions in base64 format for the next page and the previous page. To obtain next and prev, we also need to convert the PaginationParameter object to text (json) form and then convert it to base64 form.

The conversion of data between json and the corresponding PaginationParameter object is the role of the PaginationParameterConverter class.

For example, we have an enum class: 

public enum AdminActivityPaginationSortOrder {
    ID_DESC
}

Which contains the value ID_DESC, representing the sorting of records by the id field in descending order. We can create the AdminActivityPaginationParameterConverter class as shown below to convert between data in text (json) form and the corresponding PaginationParameter object, specifically the pair of enum AdminActivityPaginationSortOrder.ID_DESC and the AdminIdDescAdminActivityPaginationParameter class. 

@EzySingleton
public class AdminActivityPaginationParameterConverter extends ComplexPaginationParameterConverter<String, AdminActivityHistoryModel> {
    public AdminActivityPaginationParameterConverter(AdminPaginationParameterConverter converter) {
        super(converter);
    }

    @Override
    protected void mapPaginationParametersToTypes(Map<String, Class<?>> map) {
        map.put(AdminActivityPaginationSortOrder.ID_DESC.toString(), AdminIdDescAdminActivityPaginationParameter.class);
    }

    @Override
    protected void addPaginationParameterExtractors(Map<String, Function<AdminActivityHistoryModel, Object>> map) {
        map.put(AdminActivityPaginationSortOrder.ID_DESC.toString(), model -> new AdminIdDescAdminActivityPaginationParameter(model.getId()));
    }
}

4. PaginationXxxRepository

This class will use Filter and PaginationParameter to create JPQL queries and access the database. You will need to implement the getEntityType method to return the Entity class. Below is an example of the AdminPaginationAdminActivityRepository class for the pagination of admin activity history:

  1. The Id type is Long.
  2. The Entity class is AdminActivityHistory.
  3. The filter class is AdminActivityFilter.
  4. The PaginationParameter class is AdminActivityPaginationParameter.
@EzyRepository
public class AdminPaginationAdminActivityRepository extends CommonPaginationRepository<
    AdminActivityFilter,
    AdminActivityPaginationParameter,
    Long,
    AdminActivityHistory
> {

    @Override
    protected Class<AdminActivityHistory> getEntityType() {
        return AdminActivityHistory.class;
    }
}

5. PaginationXxxService

This class is responsible for:

  1. Using the PaginationParameterConverter to convert data between text and PaginationParameter objects.
  2. Calling the Repository to query the data.
  3. Converting the data from Entity to Model format.

You will need to implement the following:

  1. serializeToPageToken: Function to convert PaginationParameter object to text.
  2. deserializePageToken: Function to convert text data to PaginationParameter object.
  3. defaultPaginationParameter: Function to provide the default PaginationParameter when no information about PaginationParameter is provided.
  4. convertEntity: Function to convert data from Entity to Model format.

In the AdminPaginationAdminActivityService class below for the pagination of admin activity history, we have:

  1. The Id type is Long.
  2. The Entity class is AdminActivityHistory.
  3. The filter class is AdminActivityFilter.
  4. The PaginationParameter class is AdminActivityPaginationParameter.
  5. The Model class is AdminActivityHistoryModel.
@Service
public class AdminPaginationAdminActivityService extends DefaultPaginationService<
    AdminActivityHistoryModel,
    AdminActivityFilter,
    AdminActivityPaginationParameter,
    Long,
    AdminActivityHistory
> {

    private final AdminEntityToModelConverter entityToModelConverter;
    private final AdminActivityPaginationParameterConverter adminActivityPaginationParameterConverter;

    public AdminPaginationAdminActivityService(
        AdminPaginationAdminActivityRepository repository,
        AdminEntityToModelConverter entityToModelConverter,
        AdminActivityPaginationParameterConverter adminActivityPaginationParameterConverter
    ) {
        super(repository);
        this.entityToModelConverter = entityToModelConverter;
        this.adminActivityPaginationParameterConverter = adminActivityPaginationParameterConverter;
    }

    @Override
    protected AdminActivityHistoryModel convertEntity(AdminActivityHistory adminActivity) {
        return entityToModelConverter.toModel(adminActivity);
    }

    @Override
    protected String serializeToPageToken(
        AdminActivityPaginationParameter paginationParameter,
        AdminActivityHistoryModel model
    ) {
        return adminActivityPaginationParameterConverter.serialize(
            paginationParameter.sortOrder(),
            model
        );
    }

    @Override
    protected AdminActivityPaginationParameter deserializePageToken(String value) {
        return adminActivityPaginationParameterConverter.deserialize(value);
    }

    @Override
    protected AdminActivityPaginationParameter defaultPaginationParameter() {
        return new AdminIdDescAdminActivityPaginationParameter();
    }
}

Usage

When using this, we only need to pay attention to the PaginationService and the Filter:

  1. Step one: Create a filter object.
  2. Step two: Use the PaginationService to retrieve a data page.

Below is an example: 

public class AdminApiAdminActivitiesController {

    private final AdminPaginationAdminActivityService paginationAdminActivityService;
    private final AdminCommonValidator commonValidator;

    @DoGet("/admins/me/activities")
    public PaginationModel<AdminActivityHistoryModel> adminsMeActivities(
        @AdminId long adminId,
        @RequestParam(value = "nextPageToken") String nextPageToken,
        @RequestParam(value = "prevPageToken") String prevPageToken,
        @RequestParam(value = "lastPage") boolean lastPage,
        @RequestParam(value = "limit", defaultValue = "5") int limit
    ) {
        commonValidator.validatePageSize(limit);
        AdminAdminActivityFilter filter = AdminAdminActivityFilter.builder()
            .adminId(adminId)
            .build();
        return PaginationModelFetchers.getPaginationModel(
            paginationAdminActivityService,
            filter,
            nextPageToken,
            prevPageToken,
            lastPage,
            limit
        );
    }

    @EzyFeature("admin_management")
    @DoGet("/admins/{adminId}/activities")
    public PaginationModel<AdminActivityHistoryModel> adminsAdminIdActivities(
        @PathVariable(value = "adminId") long adminId,
        @RequestParam(value = "nextPageToken") String nextPageToken,
        @RequestParam(value = "prevPageToken") String prevPageToken,
        @RequestParam(value = "lastPage") boolean lastPage,
        @RequestParam(value = "limit", defaultValue = "5") int limit
    ) {
        commonValidator.validatePageSize(limit);
        AdminAdminActivityFilter filter = AdminAdminActivityFilter.builder()
            .adminId(adminId)
            .build();
        return PaginationModelFetchers.getPaginationModel(
            paginationAdminActivityService,
            filter,
            nextPageToken,
            prevPageToken,
            lastPage,
            limit
        );
    }
}

Another usage when we do not want to use the default pagination object: 

public class ApiContractController {

    private final WebPaginationAddressService paginationAddressService;
    private final WebChainValidator chainValidator;
    private final WebCommonValidator webCommonValidator;
    private final WebAddressPaginationParameterConverter addressPaginationParameterConverter;

    @DoGet("/chains/{chainId}/contracts")
    public PaginationModel<ContractModel> chainsChainIdContractsGet(
        @PathVariable(value = "chainId") String chainId,
        @RequestParam(value = "nextPageToken") String nextPageToken,
        @RequestParam(value = "prevPageToken") String prevPageToken,
        @RequestParam(value = "lastPage") boolean lastPage,
        @RequestParam(value = "limit", defaultValue = "15") int limit
    ) {
        chainValidator.validateChainId(chainId);
        webCommonValidator.validatePageSize(limit);
        AddressFilter filter = new AddressTypeAddressFilter(
            chainId,
            AddressType.CONTRACT
        );
        String actualNextPageToken = nextPageToken;
        String actualPrevPageToken = prevPageToken;
        String sortOrder = AddressPaginationSortOrder.FIRST_BLOCK_HEIGHT_DESC_ADDRESS_DESC.toString();
        if (nextPageToken == null && prevPageToken == null) {
            if (lastPage) {
                actualPrevPageToken = addressPaginationParameterConverter.getDefaultPageToken(sortOrder);
            } else {
                actualNextPageToken = addressPaginationParameterConverter.getDefaultPageToken(sortOrder);
           

 }
        }
        return PaginationModelFetchers.<AddressModel>getPaginationModel(
            paginationAddressService,
            filter,
            actualNextPageToken,
            actualPrevPageToken,
            lastPage,
            limit
        ).map(AddressModel::toContractModel);
    }
}

Offset pagination

Pagination using offset will be much simpler compared to cursor pagination because the offset itself acts as a condition for pagination. Therefore, the number of classes we need to create will be fewer, and the complexity will also decrease significantly.

Pagination Offset.png

The only difference is the XxxOffsetPaginationParameter class, while all other classes will be the same as cursor pagination.

1. Create XxxOffsetPaginationParameter class

This class provides offset, the field used for sorting, and the sorting direction, for example: 

public class YmProjectOffsetPaginationParameter extends OffsetPaginationParameter {

    public YmProjectOffsetPaginationParameter(long offset) {
        super(offset, "e.priority DESC");
    }
}

2. Create XxxPaginationRepository class

Similar to cursor pagination, for example: 

@EzyRepository
public class WebYmPaginationProjectRepository extends PaginationRepository<
    PostFilter,
    YmProjectOffsetPaginationParameter,
    Long,
    Post
> {

    @Override
    protected Class<Post> getEntityType() {
        return Post.class;
    }
}

3. Create XxxPaginationService class

Similar to cursor pagination, for example: 

@Service
public class WebYmPaginationProjectService extends OffsetPaginationService<
    PostModel,
    PostFilter,
    YmProjectOffsetPaginationParameter,
    Long,
    Post
> {

    private final WebEzyArticleEntityToModelConverter entityToModelConverter;

    public WebYmPaginationProjectService(
        WebYmPaginationProjectRepository repository,
        WebEzyArticleEntityToModelConverter entityToModelConverter
    ) {
        super(repository, YmProjectOffsetPaginationParameter::new);
        this.entityToModelConverter = entityToModelConverter;
    }

    @Override
    protected PostModel convertEntity(Post entity) {
        return entityToModelConverter.toModel(entity);
    }
}

Usage

It will be similar to cursor pagination, for example: 

private final WebYmPaginationProjectService paginationProjectService;
private final WebYmProjectModelDecorator projectModelDecorator;

public PaginationModel<WebYmProjectResponse> getProjectPagination(
    String nextPageToken,
    int limit
) {
    PaginationModel<PostModel> pagination = getPaginationModel(
        paginationProjectService,
        DefaultPostFilter.builder()
            .postType(YmPostType.PROJECT.toString())
            .postStatus(PostStatus.PUBLISHED.toString())
            .build(),
        nextPageToken,
        null,
        false,
        limit
    );
    return projectModelDecorator.decorate(pagination);
}

Some Utility Functions

EzyPlatform provides several utility functions to help you easily create a PaginationParameter class as follows:

  1. Values.isAllNull: This function helps check if all values are null.
  2. PaginationParameters.makePaginationConditionDesc: This function helps set up the paginationCondition function more easily.
  3. PaginationParameters.makeOrderByDesc: This function helps set up the orderBy function more easily.

For example: 

import static org.youngmonkeys.ezyplatform.pagination.PaginationParameters.makeOrderByDesc;
import static org.youngmonkeys.ezyplatform.pagination.PaginationParameters.makePaginationConditionDesc;
import static org.youngmonkeys.ezyplatform.util.Values.isAllNull;

@Getter
@Setter
@NoArgsConstructor
@AllArgsConstructor
public class PriceDescIdDescProductPaginationParameter
    implements ProductPricePaginationParameter {

    public BigDecimal price;
    public Long id;

    @Override
    public String paginationCondition(boolean nextPage) {
        return isEmpty()
            ? null
            : makePaginationConditionDesc(
                nextPage,
                "price",
                "id"
            );
    }

    @Override
    public String orderBy(boolean nextPage) {
        return makeOrderByDesc(nextPage, "price", "id");
    }

    @Override
    public String sortOrder() {
        return ProductPricePaginationSortOrder.PRICE_DESC_ID_DESC.toString();
    }

    @Override
    public boolean isEmpty() {
        return isAllNull(price, id);
    }
}