Implementing React table pagination handling one million records

Have you ever faces a situation where you react application stuck with a loading state or slow when the data starts to grow?. Well, Most of us would have faced this scenario in our work. Here, we will see how to solve it using react table pagination on the server-side to handle one million records.

Since we will use server-side pagination, we need to set up a nodejs server to fetch the data from MongoDB. checkout this article to set up a nodejs server with pagination, or you can create a mock API with data (1 million records)

React Table

React table is a lightweight and extensible data table for react. It's a headless UI for data tables in react applications. If you're new to react table, I will suggest getting started with basics and try pagination.

Here's a simple react table component without pagination

1export const BasicTable = ({ columns, data }) => {
2 const {
3 getTableProps,
4 getTableBodyProps,
5 headerGroups,
6 footerGroups,
7 rows,
8 prepareRow,
9 } = useTable({
10 columns,
11 data,
12 })
13
14 return (
15 <>
16 <table {...getTableProps()}>
17 <thead>
18 {headerGroups.map(headerGroup => (
19 <tr {...headerGroup.getHeaderGroupProps()}>
20 {headerGroup.headers.map(column => (
21 <th {...column.getHeaderProps()}>{column.render("Header")}</th>
22 ))}
23 </tr>
24 ))}
25 </thead>
26 <tbody {...getTableBodyProps()}>
27 {rows.map(row => {
28 prepareRow(row)
29 return (
30 <tr {...row.getRowProps()}>
31 {row.cells.map(cell => {
32 return <td {...cell.getCellProps()}>{cell.render("Cell")}</td>
33 })}
34 </tr>
35 )
36 })}
37 </tbody>
38 </table>
39 </>
40 )
41}

useTable hooks provide all the properties and methods for the table. So, for example, we need to map getTableProps props to the table jsx element and take care of all the functionalities.

As you can see, that table is scrollable because of rendering the whole data in the table. To make it a better experience for the users, we can implement pagination.

Pagination

To use pagination in the table, we need usePagination hooks from react table. import the hook and add it to the useTable

1import { useTable, usePagination } from "react-table"
2const {
3 getTableProps,
4 getTableBodyProps,
5 headerGroups,
6 footerGroups,
7 rows,
8 prepareRow,
9} = useTable(
10 {
11 columns,
12 data,
13 },
14 usePagination
15)

Once we have pagination hooks, replace the rows prop with page prop inside the useTable hooks because every data inside the table will be based on the page hereafter. So, we need to render pages instead of rows.

1export const BasicTable = ({ columns, data }) => {
2 const {
3 getTableProps,
4 getTableBodyProps,
5 headerGroups,
6 footerGroups,
7 page,
8 prepareRow,
9 } = useTable(
10 {
11 columns,
12 data,
13 },
14 usePagination
15 )
16
17 return (
18 <>
19 <table {...getTableProps()}>
20 <thead>
21 {headerGroups.map(headerGroup => (
22 <tr {...headerGroup.getHeaderGroupProps()}>
23 {headerGroup.headers.map(column => (
24 <th {...column.getHeaderProps()}>{column.render("Header")}</th>
25 ))}
26 </tr>
27 ))}
28 </thead>
29 <tbody {...getTableBodyProps()}>
30 {page.map(row => {
31 prepareRow(row)
32 return (
33 <tr {...row.getRowProps()}>
34 {row.cells.map(cell => {
35 return <td {...cell.getCellProps()}>{cell.render("Cell")}</td>
36 })}
37 </tr>
38 )
39 })}
40 </tbody>
41 </table>
42 </>
43 )
44}

Once we make the change, our table will render only the first 10 elements from the data. So, it's time to implement the pagination buttons in the footer to navigate or fetch the next set of data.

To implement navigation for the paginated table, react table provides four properties. They are,

  • nextPage
  • previousPage
  • canPreviousPage
  • canNextPage

nextPage is an action-based prop that we can use inside the onClick method. canNextPage returns a boolean to add a condition whether to render the next button or not. For example, if we reach the last page. We can disable the next button based on the canNextPage prop.

Also usePagination provides state that returns pageSize and pageIndex . we can use it to show the number of items on the page and also for server-side pagination. We will come back to that in the latter part of the article.

1const {
2 getTableProps,
3 getTableBodyProps,
4 headerGroups,
5 prepareRow,
6 page,
7 state: { pageIndex, pageSize },
8} = useTable(
9 {
10 columns,
11 data,
12 },
13 usePagination
14)

Here's the Paginated Table with all the basic functionalities,

React Table Pagination

Everything is great on the in-built pagination of react table. But, there are use-cases where we need to have more control over the component. One of the use-cases is Server-side pagination.

In the above example, we send the whole data to react table component. What if we get the data from the paginated API. We will get the first 20 rows from API, and we need to call API to fetch the next row of data.

How does react table handle this situation? we are discussing it here because this is the common scenario we will face in real product development(not just simple pagination). First, let's see how to handle this situation in an efficient way.

Here's the flow for server side pagination in react table,

  • Change to Manual Pagination in react table
  • Call API whenever pageIndex changes

Now, add this property inside useTable

1const {
2 getTableProps,
3 getTableBodyProps,
4 headerGroups,
5 prepareRow,
6 page,
7 canPreviousPage,
8 canNextPage,
9 pageOptions,
10 pageCount,
11 gotoPage,
12 nextPage,
13 previousPage,
14 setPageSize,
15 setHiddenColumns,
16 state: { pageIndex, pageSize },
17} = useTable(
18 {
19 columns,
20 data,
21 manualPagination: true,
22 pageCount: controlledPageCount,
23 },
24 usePagination
25)

After that add an useEffect hook, so whenever the pageIndex changes we can fetch the data

1React.useEffect(() => {
2 fetchData && fetchData({ pageIndex, pageSize })
3}, [fetchData, pageIndex, pageSize])

fetchData function can be a prop from the parent component or the same component. In most cases, we will be abstracting this Table component. So, it will be a prop from the parent component.

Here's the fetchData function takes the current element and check if it's the same as the previous. If not, it will fetch the new data.

1const fetchData = useCallback(
2 ({ pageSize, pageIndex }) => {
3 // console.log("fetchData is being called")
4 // This will get called when the table needs new data
5 // You could fetch your data from literally anywhere,
6 // even a server. But for this example, we'll just fake it.
7 // Give this fetch an ID
8 const fetchId = ++fetchIdRef.current
9 setLoading(true)
10 if (fetchId === fetchIdRef.current) {
11 fetchAPIData({
12 limit: pageSize,
13 skip: pageSize * pageIndex,
14 search: searchTerm,
15 })
16 }
17 },
18 [searchTerm]
19)

Since we have the fetchData function in the parent component, we use the useCallback hook that helps us to avoid unnecessary re-render. We also pass the searchTerm for filtering functionality.

fetchAPIData function fetches the data and set it in state value

1const fetchAPIData = async ({ limit, skip, search }) => {
2 try {
3 setLoading(true)
4 const response = await fetch(
5 `/companies?limit=${limit}&skip=${skip}&search=${search}`
6 )
7 const data = await response.json()
8
9 setData(data.data)
10
11 setPageCount(data.paging.pages)
12 setLoading(false)
13 } catch (e) {
14 console.log("Error while fetching", e)
15 // setLoading(false)
16 }
17}

complete source is available here

To Read More

Modern React Redux Toolkit - Login ...

User Authentication is one of the common workflow in web applications. In this t...

Building Nodejs Microservice - A Cl...

This Article explains everything about how to build Nodejs Microservices in clou...

I Accidentally wiped the entire dat...

One of the tragic accident in my job turned out to be good learning for me in re...