This section applies only to MRS 3.3.0 or later.
Alarm Description
The system checks the disk capacity of the ClickHouseServer node every 1 minute. This alarm is generated when the system detects that the disk capacity exceeds 90% and the ClickHouseServer node enters the read-only mode.
This alarm is automatically cleared when the system detects that the disk capacity is lower than 90% and the ClickHouseServer node exits the read-only mode.
If the ClickHouseServer node is in read-only mode and you need to log in to the client to clear data, you can manually exit the read-only mode using the following method:
Log in to FusionInsight Manager, choose Cluster > Services > ClickHouse > Configurations > All Configurations, search for profiles.default.readonly, and change its value to 0.
Alarm Attributes
Alarm ID |
Alarm Severity |
Auto Cleared |
|---|---|---|
45439 |
Major |
Yes |
Alarm Parameters
Parameter |
Description |
|---|---|
Source |
Specifies the cluster or system for which the alarm is generated. |
ServiceName |
Specifies the service for which the alarm is generated. |
RoleName |
Specifies the role for which the alarm is generated. |
HostName |
Specifies the host for which the alarm is generated. |
DiskPath |
Specifies the path of the disk for which the alarm is generated. |
Impact on the System
After the ClickHouseServer node enters the read-only mode, all write, modification, and deletion operations fail.
Possible Causes
The disk usage of the ClickHouse node exceeds 90%.
Handling Procedure
- Log in to FusionInsight Manager, choose O&M > Alarm > Alarms, and view the role name and the IP address for the hostname in Location.
- Expand the disk capacity of the node for which the alarm is generated.
- Go to 4 if the expansion fails or the alarm persists after the expansion.
After the capacity expansion, this alarm can be automatically cleared only when profiles.default.readonly is auto. If its value has been manually changed, change it back to auto. If profiles.default.readonly needs to be set to 0 or 1 based on service requirements, manually clear this alarm.
Collect fault information.
- On FusionInsight Manager, choose O&M. In the navigation pane on the left, choose Log > Download.
- Expand the Service drop-down list, and select ClickHouse for the target cluster.
- Expand the Hosts drop-down list. In the Select Host dialog box that is displayed, select the abnormal host, and click OK.
- Click the edit icon in the upper right corner, and set Start Date and End Date for log collection to 1 hour ahead of and after the alarm generation time, respectively. Then, click Download.
- Contact O&M personnel and provide the collected logs.
Alarm Clearance
This alarm is automatically cleared after the fault is rectified.
Related Information
None.