Background Writer

This section describes background writer parameters. The background writer process is used to write dirty data (new or modified data) in shared buffers to disks. This mechanism ensures that database processes seldom or never need to wait for a write action to occur when handling user queries.

It also mitigates performance deterioration caused by checkpoints because only a few of dirty pages need to be flushed to the disk when the checkpoints arrive. This mechanism, however, increases the overall net I/O load because while a repeatedly-dirtied page may otherwise be written only once per checkpoint interval, the background writer may write it several times as it is dirtied in the same interval. In most cases, continuous light loads are preferred, instead of periodical load peaks. The parameters discussed in this section can be set based on actual requirements.

bgwriter_delay

Parameter description: Specifies the interval at which the background writer writes dirty shared buffers. The background writer initiates write operations for some dirty shared buffers (the volume of data to be written is specified by the bgwriter_lru_maxpages parameter), sleep for the milliseconds specified by bgwriter_delay, and then restarts.

In many systems, the effective resolution of sleep delays is 10 milliseconds. Therefore, setting this parameter to a value that is not a multiple of 10 has the same effect as setting it to the next higher multiple of 10.

This parameter is a SIGHUP parameter. Set it based on instructions provided in Table 1 GUC parameters.

Value range: an integer ranging from 10 to 10000. The unit is millisecond.

Default value: 2s

Setting suggestion: Reduce this value in slow data writing scenarios to reduce the checkpoint load.

candidate_buf_percent_target

Parameter description: Specifies the expected percentage of available buffers in the shared_buffer memory buffer in the candidate buffer chain when the incremental checkpoint is enabled. If the number of available buffers in the current candidate chain is less than the target value, the bgwriter thread starts flushing dirty pages that meet the requirements.

This parameter is a SIGHUP parameter. Set it based on instructions provided in Table 1 GUC parameters.

Value range: a double-precision floating point number ranging from 0.1 to 0.85

Default value: 0.3

bgwriter_lru_maxpages

Parameter description: Specifies the number of dirty buffers the background writer can write in each round.

This parameter is a SIGHUP parameter. Set it based on instructions provided in Table 1 GUC parameters.

Value range: an integer ranging from 0 to 1000

NOTE: When this parameter is set to 0, the background writer is disabled. This setting does not affect checkpoints.

Default value: 100

bgwriter_lru_multiplier

Parameter description: Specifies the coefficient used to estimate the number of dirty buffers the background writer can write in the next round.

The number of dirty buffers written in each round depends on the number of buffers used by server processes during recent rounds. The estimated number of buffers required in the next round is calculated using the following formula: Average number of recently used buffers x bgwriter_lru_multiplier. The background writer writes dirty buffers until sufficient clean and reusable buffers are available. The number of buffers the background writer writes in each round is always equal to or less than bgwriter_lru_maxpages.

Therefore, the value 1.0 represents a just-in-time policy of writing exactly the number of dirty buffers predicted to be required. Larger values provide some cushion against spikes in demand, whereas smaller values intentionally leave more writes to be done by server processes.

Smaller values of bgwriter_lru_maxpages and bgwriter_lru_multiplier reduce the extra I/O load caused by the background writer, but make it more likely that server processes will have to issue writes for themselves, delaying interactive queries.

This parameter is a SIGHUP parameter. Set it based on instructions provided in Table 1 GUC parameters.

Value range: a floating point number ranging from 0 to 10

Default value: 2

pagewriter_thread_num

Parameter description: Specifies the number of threads for background page flushing after the incremental checkpoint is enabled. Dirty pages are flushed in sequence to disks, promoting recovery points.

This parameter is a POSTMASTER parameter. Set it based on instructions provided in Table 1 GUC parameters.

Value range: an integer ranging from 1 to 16

Default value: 4

dirty_page_percent_max

Parameter description: Specifies the percentage of dirty pages to shared_buffersafter the incremental checkpoint is enabled. When the value of this parameter is reached, the background page flush thread flushes dirty pages based on the maximum value of max_io_capacity.

This parameter is a SIGHUP parameter. Set it based on instructions provided in Table 1 GUC parameters.

Value range: a floating point number ranging from 0.1 to 1

Default value: 0.9

pagewriter_sleep

Parameter description: Specifies the interval for the pagewriter thread to flush dirty pages to disks after the incremental checkpoint is enabled. When the ratio of dirty pages to shared_buffers reaches dirty_page_percent_max, the number of pages in each batch is calculated based on the value of max_io_capacity. In other cases, the number of pages in each batch decreases proportionally.

This parameter is a SIGHUP parameter. Set it based on instructions provided in Table 1 GUC parameters.

Value range: an integer ranging from 0 to 3600000. The unit is ms.

Default value: 2000ms(2s)

max_io_capacity

Parameter description: Specifies the maximum I/O per second for the backend write process to flush pages in batches. Set this parameter based on the service scenario and disk I/O capability of the host. If the RTO is short or the data volume is much larger than the shared memory, and the service access data volume is random, the value of this parameter cannot be too small. A small parameter value reduces the number of pages flushed by the backend write process. If a large number of pages are eliminated due to service triggering, the services are affected.

This parameter is a SIGHUP parameter. Set it based on instructions provided in Table 1 GUC parameters.

Value range: an integer ranging from 30720 to 10485760. The unit is kB.

Default value: 512000 kB (500 MB)

enable_consider_usecount

Parameter description: Specifies whether the backend thread considers the page popularity during page replacement. You are advised to enable this parameter in large-capacity scenarios.

This parameter is a SIGHUP parameter. Set it based on instructions provided in Table 1 GUC parameters.

Value range: Boolean

• on/true: The page popularity is considered.
• off/false: The page popularity is not considered.

Default value: off

dw_file_num

Parameter description: Specifies the number of doublewrite files to be written in batches. The value is related to pagewriter_thread_num and cannot be greater than pagwriter_thread_num. If the value is too large, it will be corrected to the value of pagewriter_thread_num.

This parameter is a POSTMASTER parameter. Set it based on instructions provided in Table 1 GUC parameters.

Value range: an integer ranging from 1 to 16

Default value: 1

dw_file_size

Parameter description: Specifies the size of each doulewrite file.

This parameter is a POSTMASTER parameter. Set it based on instructions provided in Table 1 GUC parameters.

Value range: an integer, in the range [32,256]

Default value: 256

extreme_flush_dirty_page

Parameter description: Whether or not to enable Extreme Brush Dirty Mode. Enabling it will make brushing faster but increase the write amplification. (Introduced in MogDB 5.0.8)

This parameter is a POSTMASTER parameter. Set it based on instructions provided in Table 1 GUC parameters.

Value range: Boolean

Default value: off

Note: Please confirm that the bottleneck of the current system brush dirty slow is not in the system IO capacity before enabling this parameter. You can confirm that there is no bottleneck in disk IO through monitoring tools such as iostat and Node-exporter. For shared storage services, you should also confirm the IO capacity limit of the shared storage service.

checkpoint_target_time

Parameter description: Maximum time to execute checkpoint. The smaller the value, the faster the brushing dirty, the smaller the actual time consumed to execute checkpoint, but the write amplification increases, when IO becomes a bottleneck, the value is very low and may affect the business; the corresponding upstream operations are: stop (stop), switchover (primary and standby nodes), and manually executing the checkpoint statement. (Introduced in MogDB 5.0.8)

This parameter is a POSTMASTER parameter. Set it based on instructions provided in Table 1 GUC parameters.

Value range: 5～60s

Default value: 30s