2018-04-03
diff --git a/docs/dms/umn/kafka-ug-00002.html b/docs/dms/umn/kafka-ug-00002.html
new file mode 100644
index 00000000..03d8f17c
--- /dev/null
+++ b/docs/dms/umn/kafka-ug-00002.html
@@ -0,0 +1,122 @@
+
+
+Acronyms and Abbreviations
+
+ Abbreviation
+ |
+Full Name
+ |
+
|---|
+
+ACL
+ |
+Access Control List
+ |
+
+AK/SK
+ |
+Access Key ID/Secret Access Key
+ |
+
+AZ
+ |
+Availability Zone
+ |
+
+CRT
+ |
+Certificate
+ |
+
+CLI
+ |
+Command Line Interface
+ |
+
+CTS
+ |
+Cloud Trace Service
+ |
+
+DMS
+ |
+Distributed Message Service
+ |
+
+ECS
+ |
+Elastic Cloud Server
+ |
+
+EIP
+ |
+Elastic IP
+ |
+
+EOS
+ |
+End of Service & support
+ |
+
+EVS
+ |
+Elastic Volume Service
+ |
+
+GC
+ |
+Garbage Collection
+ |
+
+JKS
+ |
+Java Key Storage
+ |
+
+JVM
+ |
+Java Virtual Machine
+ |
+
+KMS
+ |
+Key Management Service
+ |
+
+PEM
+ |
+Privacy Enhanced Mail
+ |
+
+SASL
+ |
+Simple Authentication and Security Layer
+ |
+
+SSL
+ |
+Secure Sockets Layer
+ |
+
+TMS
+ |
+Tag Management Service
+ |
+
+TPS
+ |
+Transactions Per Second
+ |
+
+VPC
+ |
+Virtual Private Cloud
+ |
+
+
+
+
+
+ Getting Started
+
+
+
+
+
diff --git a/docs/dms/umn/kafka-ug-0001.html b/docs/dms/umn/kafka-ug-0001.html
index d31c02cb..34a928ca 100644
--- a/docs/dms/umn/kafka-ug-0001.html
+++ b/docs/dms/umn/kafka-ug-0001.html
@@ -1,8 +1,8 @@
-Cross-VPC Access to a Kafka Instance
+Accessing Kafka Using a VPC Endpoint Across VPCs
ContextVPCs are logically isolated from each other. If a Kafka instance and a Kafka client are in different VPCs within a region, they cannot communicate with each other. In this case, you can use one of the following methods to access a Kafka instance across VPCs:
- - Establish a VPC peering connection to allow two VPCs to communicate with each other. For details, see section "VPC Peering Connection" in Virtual Private Cloud User Guide.
- Use VPC Endpoint (VPCEP) to establish a cross-VPC connection.
+ - Establish a VPC peering connection to allow two VPCs to communicate with each other. For details, see VPC Peering Connection.
- Use VPC Endpoint (VPCEP) to establish a cross-VPC connection.
ScenarioThe following describes how to use VPCEP to implement cross-VPC access.
VPCEP provides two types of resources: VPC endpoint services and VPC endpoints.
@@ -11,33 +11,33 @@
ProcedureFigure 2 Process for accessing a Kafka instance across VPCs
- Creating a VPC Endpoint Service- Log in to the management console.
- Click
 in the upper left corner to select a region. Select the region where your Kafka instance is located.
+ Creating a VPC Endpoint Service- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
- - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- In the Advanced Settings section on the Basic Information tab page, obtain the listeners IP addresses and port IDs of the instance for Cross-VPC Access.
Figure 3 Cross-VPC access–related listeners IP addresses and corresponding port IDs of the Kafka instance
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view its details.
- In the Advanced Settings section on the Basic Information tab page, obtain the listeners IP addresses and port IDs of the instance for Cross-VPC Access.
Figure 3 Cross-VPC access–related listeners IP addresses and corresponding port IDs of the Kafka instance
- In the Network section on the Basic Information tab page, view the VPC to which the Kafka instance belongs.
Figure 4 Viewing the VPC to which the Kafka instance belongs
- Click the VPC to obtain the VPC ID on the VPC console.
Figure 5 Obtaining the VPC ID
- - Call the VPC Endpoint API to create a VPC endpoint service. For details, see "Creating a VPC Endpoint Service" in VPC Endpoint API Reference.
curl -i -k -H 'Accept:application/json' -H 'Content-Type:application/json;charset=utf8' -X POST -H "X-Auth-Token:$token" -d '{"port_id":"38axxxeac","vpc_id":"706xxx888","ports":[{"protocol":"TCP","client_port":9011,"server_port":9011 }],"approval_enabled":false,"service_type":"interface","server_type":"VM"}' https://{endpoint}/v1/{project_id}/vpc-endpoint-services
-Parameter description:
-- token: an access credential issued to an IAM user to bear its identity and permissions. For details on how to obtain a token, see Obtaining a User Token.
- port_id: one of the port IDs obtained in 5.
- vpc_id: VPC ID obtained in 7.
- endpoint: VPCEP endpoint obtained from Regions and Endpoints. The region must be the same as that of the Kafka instance.
- project_id: project ID obtained from "Obtaining a Project ID". The region must be the same as that of the Kafka instance. For details about how to obtain the value, see section "Common Parameters" > "Obtaining a Project ID" in the VPC Endpoint API Reference.
+ - Call the VPC Endpoint API to create a VPC endpoint service. For details, see Creating a VPC Endpoint Service.
POST https://{endpoint}/v1/{project_id}/vpc-endpoint-services
+Set the following request parameter to the specified values, and other parameters as required.
+- port_id: one of the port IDs obtained in 5.
- vpc_id: VPC ID obtained in 7.
- server_type: VM
- client_port: 9011
- server_port: 9011
- protocol: TCP
- approval_enabled: false
- service_type: interface
- endpoint: VPCEP endpoint obtained from Regions and Endpoints. The region must be the same as that of the Kafka instance.
- project_id: project ID obtained from Obtaining a Project ID. The region must be the same as that of the Kafka instance.
Record the value of service_name in the response. This parameter indicates the name of the VPC endpoint service.
- Repeat 8 to create VPC endpoint services for other port IDs obtained in 5 and record the VPC endpoint service names.
- (Optional) Adding a Whitelist RecordIf the Kafka client and Kafka instance belong to different accounts, add the ID of the account to which the Kafka client belongs to the whitelist of the endpoint service. For details, see Add a Whitelist Record.
+ (Optional) Adding a Whitelist RecordIf the Kafka client and Kafka instance belong to different accounts, add the ID of the account to which the Kafka client belongs to the whitelist of the endpoint service. For details, see Add a Whitelist Record.
- Creating a VPC Endpoint- Click Service List. Then choose Networking > VPC Endpoint.
- Click Create VPC Endpoint.
- Set the following parameters:
- Region: Select the region that the Kafka instance is in.
- Service Category: Select Find a service by name.
- VPC Endpoint Service Name: Enter the VPC endpoint service name recorded in 8 and click Verify. If Service name found is displayed, proceed with subsequent operations.
- VPC: Select the VPC that the Kafka instance is in.
- Subnet: Select the subnet that the Kafka instance is in.
- Private IP Address: Select Automatic.
+Creating a VPC Endpoint- Click Service List. Then choose Network > VPC Endpoint.
- Click Create VPC Endpoint.
- Set the following parameters:
- Region: Select the region that the Kafka instance is in.
- Service Category: Select Find a service by name.
- VPC Endpoint Service Name: Enter the VPC endpoint service name recorded in 8 and click Verify. If Service name found is displayed, proceed with subsequent operations.
- VPC: Select the VPC that the Kafka client is in.
- Subnet: Select the subnet that the Kafka client is in.
- Private IP Address: Select Automatic.
Retain the default values for other parameters. For details, see Creating a VPC Endpoint.
- - Click Create Now.
- Confirm the configurations and submit the request.
- Go back to the VPC endpoint list and check whether the status of the created VPC endpoint has changed to Accepted. The Accepted state means that the VPC endpoint has been connected to the VPC endpoint service.
Figure 6 Checking the VPC endpoint status
+ - Click Create Now.
- Confirm the configurations and submit the request.
- Go back to the VPC endpoint list and check whether the status of the created VPC endpoint has changed to Accepted. The Accepted state means that the VPC endpoint has been connected to the VPC endpoint service.
Figure 6 Checking the VPC endpoint status
- Click the VPC endpoint ID. On the Summary tab page, obtain the private IP address.
You can use the private IP address to access the VPC endpoint service.
Figure 7 Viewing the private IP address
- Repeat 1 to 7 to create a VPC endpoint for each VPC endpoint service created in 9, and view and record the private IP addresses of the VPC endpoint services.
Changing the advertised.listeners IP Address- Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- On the Advanced Settings section of the Basic Information tab page, click Modify for Cross-VPC Access to change the value of advertised.listeners IP address to the private IP addresses recorded in 7 and 8. Click Save.
 Each IP address must match the corresponding port ID. Otherwise, the network will be disconnected.
+ Changing the advertised.listeners IP Addresses- Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view its details.
- On the Advanced Settings section of the Basic Information tab page, click Modify for Cross-VPC Access to change the value of advertised.listeners IP address to the private IP addresses recorded in 7 and 8. Click Save.
Each IP address must match the corresponding port ID. Otherwise, the network will be disconnected.
Figure 8 Changing the advertised.listeners IP addresses
- Verifying ConnectivityCheck whether messages can be created and retrieved by referring to Accessing a Kafka Instance Without SASL or Accessing a Kafka Instance with SASL.
+ Verifying ConnectivityCheck whether messages can be created and retrieved by referring to Connecting to Kafka Using the Client (Plaintext Access) or Connecting to Kafka Using the Client (Ciphertext Access).
Notes:
- The address for connecting to a Kafka instance is in the format of "advertised.listeners IP:9011". For example, the addresses for connecting to the Kafka instance shown in Figure 8 are 10.158.0.151:9011,10.158.0.162:9011,10.158.0.164:9011.
- Configure inbound rules for the security group of the Kafka instance to allow access from 198.19.128.0/17 over port 9011.
- If a network access control list (ACL) has been configured for the subnet of this instance, configure inbound rules for the network ACL to allow access from 198.19.128.0/17 and from the subnet used by the VPC endpoint.
198.19.128.0/17 is the network segment allocated to the VPCEP service. To use VPCEP, allow access from this network segment.
@@ -46,7 +46,7 @@
diff --git a/docs/dms/umn/kafka-ug-0002.html b/docs/dms/umn/kafka-ug-0002.html
index 3dddf617..4cde3d02 100644
--- a/docs/dms/umn/kafka-ug-0002.html
+++ b/docs/dms/umn/kafka-ug-0002.html
@@ -1,29 +1,34 @@
- Granting Permissions to a SASL_SSL User
- DMS supports ACL permission management for topics. You can differentiate the operations that different users are allowed to perform on a topic by granting the users different permissions.
- This section describes how to grant topic permissions to a SASL_SSL user. For details about how to create a SASL_SSL user, see Creating a SASL_SSL User.
- If no SASL_SSL user is granted any permission for a topic, all users can subscribe to or publish messages to the topic.
- If one or more SASL_SSL users are granted permissions for a topic, only the authorized users can subscribe to or publish messages to the topic.
- Prerequisites- SASL_SSL has been enabled when you create the Kafka instance.
- (Optional) A SASL_SSL user has been created. For details, see Creating a SASL_SSL User.
-
- Granting Permissions to a SASL_SSL User- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+ Configuring Kafka Topic Permissions
+ DMS supports access control list (ACL) for topics. You can differentiate user permissions by granting users different permissions in a topic.
+ This section describes how to grant topic permissions to users after ciphertext access is enabled for Kafka instances. For details about how to create a user, see Configuring Kafka Users.
+ This function is unavailable for single-node instances.
- - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- In the navigation pane, choose the Topics tab.
- In the row that contains the topic for which you want to configure user permissions, click Grant User Permission.
In the upper part of the Grant User Permission dialog box, the topic information is displayed, including the topic name, number of partitions, aging time, number of replicas, and whether synchronous flushing is enabled. In the middle part, you can use the search box to search for a user if there are many SASL_SSL users. In the Users area, the list of created SASL_SSL users is displayed. In the Selected area, you can grant permissions to the SASL_SSL users.
- - In the Users area of the Grant User Permission dialog box, select target users. In the Selected area, configure permissions (Subscribe, Publish, and Publish/Subscribe) for the users.
Figure 1 Granting user permissions
-As shown in Figure 1, only the test, send, and receive users can subscribe to or publish messages to topic-01. The send_receive user cannot subscribe to or publish messages to topic-01.
+Constraints- If no user is granted any permission for a topic and allow.everyone.if.no.acl.found is set to true, all users can subscribe to or publish messages to the topic.
- If allow.everyone.if.no.acl.found is set to false, only the authorized users can subscribe to or publish messages to the topic. The value of allow.everyone.if.no.acl.found can be modified.
- If one or more users are granted permissions for a topic, only the authorized users can subscribe to or publish messages to the topic.
- If both the default and individual user permissions are configured for a topic, the union of the permissions is used.
+ Configuring Topic Permissions- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- In the navigation pane, choose Topics.
- In the row containing the desired topic, click Grant User Permission.
In the upper part of the Grant User Permission dialog box, the topic information is displayed, including the topic name, number of partitions, aging time, number of replicas, and whether synchronous flushing and replication are enabled. You can enable Default permissions to grant the same permissions for all users. You can use the search box to search for a user if there are many users. In the Users area, the list of created users is displayed. In the Selected area, grant permissions to the users.
+ - Grant topic permissions to users.
- To grant the same permissions to all users, select Default permissions and then select permissions. As shown in the following figure, all users have the permission to publish messages to this topic.
Figure 1 Granting the same permissions to all users
+ - To grant different permissions to different users, do not select Default permissions. In the Users area of the Grant User Permission dialog box, select target users. In the Selected area, configure permissions (Subscribe, Publish, or Publish/Subscribe) for the users. As shown in the following figure, only the test, send, and receive users can subscribe to or publish messages to this topic. The send_receive user cannot subscribe to or publish messages to this topic.
Figure 2 Granting permissions to individual users
+
+If both the default and individual user permissions are configured for a topic, the union of the permissions is used. As shown in the following figure, the test and receive users can subscribe to and publish messages to this topic.
+Figure 3 Granting topic permissions to users
- Click OK.
On the Topics tab page, click  next to the topic name to view the authorized users and their permissions.
-Figure 2 Viewing authorized users and their permissions
+Figure 4 Viewing authorized users and their permissions
(Optional) Removing Permissions from a SASL_SSL User- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+ (Optional) Deleting Topic Permissions- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
- - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- In the navigation pane, choose the Topics tab.
- In the row that contains the topic for which you want to remove user permissions, click Grant User Permission.
- In the Selected area of the displayed Grant User Permission dialog box, locate the row that contains the SASL_SSL user whose permissions are to be removed, click Delete, and click OK.
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- In the navigation pane, choose Topics.
- In the row containing the desired topic, click Grant User Permission.
- In the Selected area of the displayed Grant User Permission dialog box, locate the row that contains the user whose permissions are to be removed, click Delete, and click OK.
diff --git a/docs/dms/umn/kafka-ug-0003.html b/docs/dms/umn/kafka-ug-0003.html
index affec0b4..2485753a 100644
--- a/docs/dms/umn/kafka-ug-0003.html
+++ b/docs/dms/umn/kafka-ug-0003.html
@@ -1,19 +1,46 @@
- Creating a SASL_SSL User
- DMS supports ACL permission management for topics. You can differentiate the operations that different users are allowed to perform on a topic by granting the users different permissions.
- This section describes how to create a SASL_SSL user after SASL_SSL is enabled for a Kafka instance. For details about how to grant user permissions, see Granting Permissions to a SASL_SSL User.
- PrerequisitesSASL_SSL has been enabled when you create the Kafka instance.
+ Configuring Kafka Users
+ DMS supports access control list (ACL) for topics. You can differentiate user permissions by granting users different permissions in a topic.
+ This section describes how to create users, reset the password, and delete users with ciphertext access enabled. For details about how to grant topic permissions for users, see Configuring Kafka Topic Permissions.
+ The maximum number of users that can be created for a Kafka instance is 20 or 500. Check the console for the actual limit.
+ There are two ways to create a user on the console. Accordingly, there are two ways to reset the user's password:
+
+ Prerequisites- Ciphertext access has been enabled for the Kafka instance.
- Kafka users can be configured only for Kafka instances in the Running state.
- Procedure- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+ Constraints- This function is unavailable for single-node instances.
- Resetting a user password will interrupt services. Change the user password in the client configuration file or code as soon as possible.
+
+ Creating a User- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
- - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- On the Users tab page, click Create User.
- In the displayed Create User dialog box, set the username and password, and click OK.
After the SASL_SSL user is created, grant permissions to the user by referring to Granting Permissions to a SASL_SSL User.
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view its details.
- On the Users page, click Create User.
- In the displayed Create User dialog box, set the username and password, and click OK.
After the user is created, grant permissions to the user by referring to Configuring Kafka Topic Permissions.
+ Resetting the Password (for the Initial User)- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Reset the password for the initial user in either of the following ways.
- Choose More > Reset Kafka Password in the row containing the desired Kafka instance.
- Click the desired Kafka instance to view its details. Choose More > Reset Kafka Password in the upper left corner.
- Click the desired Kafka instance to view its details. On the Basic Information page, click Reset Password next to Username in the Connection section.
- Click the desired Kafka instance to view its details. On the Users page, click Reset Password in the row containing the desired user.
+ - Enter and confirm a new password, and click OK.
- If the password is successfully reset, a success message is displayed.
- If the password fails to be reset, a failure message is displayed. In this case, reset the password again. If you still fail to reset the password after multiple attempts, contact customer service.
+ The system will display a success message only after the password is successfully reset on all brokers.
+
+
+
+ Resetting the User Password (for Non-initial Users)- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view its details.
- On the Users page, click Reset Password in the row containing the desired user.
- Enter and confirm a new password, and click OK.
- If the password is successfully reset, a success message is displayed.
- If the password fails to be reset, a failure message is displayed. In this case, reset the password again. If you still fail to reset the password after multiple attempts, contact customer service.
+ The system will display a success message only after the password is successfully reset on all brokers.
+
+
+
+ Deleting a User- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view its details.
- Delete a user in either of the following ways:
- On the Users page, click Delete in the row containing the desired user.
- On the Users page, select one or more users and click Delete above the list.
+ The initial user set when ciphertext access is enabled for the first time cannot be deleted.
+
+ - In the displayed Delete User dialog box, click OK to delete the user.
+
diff --git a/docs/dms/umn/kafka-ug-0004.html b/docs/dms/umn/kafka-ug-0004.html
index 17a6c961..b6a6de2b 100644
--- a/docs/dms/umn/kafka-ug-0004.html
+++ b/docs/dms/umn/kafka-ug-0004.html
@@ -1,10 +1,12 @@
- Viewing Disk Usage
- On the Kafka console, you can view the disk usage of each broker.
- Procedure- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+ Viewing Kafka Disk Usage
+ This section describes how to view the disk usage of each broker on the Kafka console.
+ This function is unavailable for single-node instances.
- - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click a Kafka instance to go to the Basic Information tab page.
- Click the Disk Usage Statistics tab.
Figure 1 Viewing disk usage
+Viewing Kafka Disk Usage- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click a Kafka instance to go to the Basic Information page.
- Go to the Disk Usage Statistics page.
Figure 1 Viewing disk usage
You can query topics that use the most disk space or topics that have used a specified amount or percentage of disk space.
In the upper right corner of the page, click View Metric. On the displayed Cloud Eye page, you can view metrics of Kafka instances.
diff --git a/docs/dms/umn/kafka-ug-0006.html b/docs/dms/umn/kafka-ug-0006.html
index e7d13e0a..37b68942 100644
--- a/docs/dms/umn/kafka-ug-0006.html
+++ b/docs/dms/umn/kafka-ug-0006.html
@@ -1,29 +1,31 @@
- Changing Partition Quantity
- After creating a topic, you can increase the number of partitions based on service requirements.
- Changing the number of partitions does not affect services.
+ Changing Kafka Partition Quantity
+ After creating a topic, you can increase the number of partitions as required.
+ Changing the number of partitions does not restart the instance or affect services.
Methods for changing the partition quantity:
-
- Method 1: By Using the Console- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ Method 1: By using Kafka console- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
- - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- Click the Topics tab.
- Modify the number of partitions using either of the following methods:
- Select one or more topics and click Edit Topic in the upper left corner.
- In the row containing the desired topic, click Edit.
- - In the Edit Topic dialog box, enter the number of partitions and click OK.
- The number of partitions can only be increased.
- To ensure performance, the Kafka console allows a maximum of 100 partitions for each topic.
- The total number of partitions of all topics cannot exceed the maximum number of partitions allowed by the instance.
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- In the navigation pane, choose Topics.
- Modify the number of partitions using either of the following methods:
- Select one or more topics and click Edit Topic in the upper left corner.
- In the row containing the desired topic, click Edit.
+ - In the Edit Topic dialog box, enter the number of partitions and click OK.
- The number of partitions can only be increased.
- To ensure performance, the Kafka console allows a maximum of 200 partitions for each topic.
- The total number of partitions of all topics cannot exceed the maximum number of partitions allowed by the instance.
Method 2: By Using Kafka CLIIf your Kafka client version is later than 2.2, you can use kafka-topics.sh to change the partition quantity.
-
diff --git a/docs/dms/umn/kafka-ug-0007.html b/docs/dms/umn/kafka-ug-0007.html
index 0d9d30b8..215d0b40 100644
--- a/docs/dms/umn/kafka-ug-0007.html
+++ b/docs/dms/umn/kafka-ug-0007.html
@@ -1,235 +1,312 @@
-Modifying Kafka Parameters
+Modifying Kafka Instance Configuration Parameters
ScenarioYour Kafka instances, topics, and consumers come with default configuration parameter settings. You can modify common parameters on the DMS console. For details about parameters that are not listed on the console, see the Kafka official website.
- Parameters of v1.1.0 instances are all static parameters. v2.3.0/2.7 instances have both dynamic and static parameters.
+ Kafka instances have dynamic and static parameters:
- Dynamic parameters: Modifying dynamic parameters will not restart the instance.
- Static parameters: After static parameters are modified, you must manually restart the instance.
- Configuration parameters of some old instances cannot be modified. Check whether your instance parameters can be modified on the console. If they cannot be modified, contact customer service.
+ - Configuration parameters of some old instances cannot be modified. Check whether your instance parameters can be modified on the console. If they cannot be modified, contact customer service.
- This function is not available for single-node instances.
PrerequisitesYou can modify configuration parameters of a Kafka instance when the instance is in the Running state.
- Procedure- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+ Modifying Kafka Instance Configuration Parameters- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
- - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- On the Parameters tab page, click Edit in the row containing the parameter to modify. Table 1 describes the parameters of v1.1.0 instances. Table 2 and Table 3 describe the parameters of v2.3.0/2.7 instances.
- Table 1 Static parameters (v1.1.0 instances)Parameter
+- Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- On the Parameters page, click Edit in the row containing the parameter to modify.
Parameters of v1.1.0 instances are described in Table 2 and Table 1. Parameters of v2.3.0/v2.7/3.x instances are described in Table 3 and Table 4.
+
+Table 1 Dynamic parameters (v1.1.0 instances)Parameter
|
-Description
+ | Description
|
-Value Range
+ | Value Range
|
-Default Value
+ | Default Value
|
|---|
-min.insync.replicas
+ | auto.create.groups.enable
|
-If a producer sets the acks parameter to all (or -1), the min.insync.replicas parameter specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful.
+ | Whether to automatically create consumer groups.
+This parameter is not displayed on the console for some earlier instances. The function of automatically creating consumer groups is enabled by default and cannot be disabled on the console.
|
-1–3
+ | true/false
|
-1
+ | true
|
-message.max.bytes
+ | offsets.retention.minutes
|
-Maximum length of a single message, in bytes.
+ | The longest period a consumption position can be retained starts from the time of submission. Positions retained beyond this duration will be deleted. Each time a consumption position is submitted to a topic partition, its retention period resets to 0. The unit is minute.
+This parameter is displayed as a static one for certain earlier instances.
|
-0–10,485,760
+ | 1440–30240
|
-10,485,760
+ | 20160
|
-unclean.leader.election.enable
+ |
+
+ Table 2 Static parameters (v1.1.0 instances)Parameter
+ |
+Description
+ |
+Value Range
+ |
+Default Value
+ |
+
|---|
+
+min.insync.replicas
|
-Indicates whether to allow replicas not in the ISR set to be elected as the leader as a last resort, even though doing so may result in data loss.
+ | If a producer sets the acks parameter to all (or -1), the min.insync.replicas parameter specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful.
|
-true or false
+ | 1–3
|
-true
+ | 1
|
-connections.max.idle.ms
+ | message.max.bytes
|
-Idle connection timeout (in ms). Connections that are idle for the duration specified by this parameter will be closed.
+ | Maximum length of a single message, in bytes.
|
-5000–600,000
+ | 0–10,485,760
|
-600,000
+ | 10,485,760
|
-log.retention.hours
+ | unclean.leader.election.enable
|
-Duration (in hours) for retaining a log file.
+ | Indicates whether to allow replicas not in the ISR set to be elected as the leader as a last resort, even though doing so may result in data loss.
+ |
+true or false
+ |
+false
+ |
+
+connections.max.idle.ms
+ |
+Idle connection timeout (in ms). Connections that are idle for the duration specified by this parameter will be closed.
+ |
+5000–600,000
+ |
+600,000
+ |
+
+log.retention.hours
+ |
+Duration (in hours) for retaining a log file.
This parameter takes effect only for topics that have no aging time configured. If there is aging time configured for topics, it overrides this parameter.
|
-1–168
+ | 1–168
|
-72
+ | 72
|
-max.connections.per.ip
+ | max.connections.per.ip
|
-The maximum number of connections allowed from each IP address. Request for new connections will be rejected once the limit is reached. The limit set using this parameter will be replaced if there are overrides configured using the max.connections.per.ip.overrides parameter.
+ | The maximum number of connections allowed from each IP address. Request for new connections will be rejected once the limit is reached.
|
-100–20,000
+ | 100–20,000
|
-1000
+ | 1000
|
-group.max.session.timeout.ms
+ | group.max.session.timeout.ms
|
-The maximum session timeout (in ms) for consumers. A longer timeout gives consumers more time to process messages between heartbeats but results in a longer time to detect failures.
+ | The maximum session timeout (in ms) for consumers. A longer timeout gives consumers more time to process messages between heartbeats but results in a longer time to detect failures.
|
-6000–1,800,000
+ | 6000–1,800,000
|
-1,800,000
+ | 1,800,000
|
-default.replication.factor
+ | default.replication.factor
|
-The default number of replicas configured for an automatically created topic.
+ | The default number of replicas configured for an automatically created topic.
|
-1–3
+ | 1–3
|
-3
+ | 3
|
-num.partitions
+ | allow.everyone.if.no.acl.found
|
-The default number of partitions configured for each automatically created topic.
+ | When this parameter is set to true, all users can access resources without ACL rules.
+This parameter is displayed only when is enabled for the instance or ciphertext access is used.
+This parameter cannot be modified for certain earlier instances.
|
-1–100
+ | true/false
|
-3
+ | true
|
-group.min.session.timeout.ms
+ | num.partitions
|
-The minimum session timeout (in ms) for consumers. A shorter timeout enables quicker failure detection but results in more frequent consumer heartbeating, which can overwhelm broker resources.
+ | The default number of partitions configured for each automatically created topic.
|
-6000–300,000
+ | 1 ~ 200
|
-6000
+ | 3
+ |
+
+group.min.session.timeout.ms
+ |
+The minimum session timeout (in ms) for consumers. A shorter timeout enables quicker failure detection but results in more frequent consumer heartbeating, which can overwhelm broker resources.
+ |
+6000–300,000
+ |
+6000
|
Table 2 Dynamic parameters (v2.3.0/2.7 instances)Parameter
+Table 3 Dynamic parameters (2.3.0/2.7/3.x instances)Parameter
|
-Description
+ | Description
|
-Value Range
+ | Value Range
|
-Default Value
+ | Default Value
|
|---|
-min.insync.replicas
+ | min.insync.replicas
|
-If a producer sets the acks parameter to all (or -1), the min.insync.replicas parameter specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful.
+ | If a producer sets the acks parameter to all (or -1), the min.insync.replicas parameter specifies the minimum number of replicas that must acknowledge a write for the write to be considered successful.
|
-1–3
+ | 1–3
|
-1
+ | 1
|
-message.max.bytes
+ | message.max.bytes
|
-Maximum length of a single message, in bytes.
+ | Maximum length of a single message, in bytes.
|
-0–10,485,760
+ | 0–10,485,760
|
-10,485,760
+ | 10,485,760
|
-unclean.leader.election.enable
+ | auto.create.groups.enable
|
-Indicates whether to allow replicas not in the ISR set to be elected as the leader as a last resort, even though doing so may result in data loss.
+ | Whether to automatically create consumer groups.
+This parameter is not displayed on the console for some earlier instances. The function of automatically creating consumer groups is enabled by default and cannot be disabled on the console.
|
-true or false
+ | true/false
|
-true
+ | true
+ |
+
+max.connections.per.ip
+ |
+The maximum number of connections allowed from each IP address. Request for new connections will be rejected once the limit is reached.
+ |
+100–20,000
+ |
+1000
+ |
+
+unclean.leader.election.enable
+ |
+Indicates whether to allow replicas not in the ISR set to be elected as the leader as a last resort, even though doing so may result in data loss.
+ |
+true or false
+ |
+false
+ |
+
+offsets.retention.minutes
+ |
+The longest period a consumption position can be retained starts from the time of submission. Positions retained beyond this duration will be deleted. Each time a consumption position is submitted to a topic partition, its retention period resets to 0. The unit is minute.
+This parameter is displayed as a static one for certain earlier instances.
+ |
+1440–30240
+ |
+20160
|
Table 3 Static parameters (v2.3.0/2.7 instances)Parameter
+Table 4 Static parameters (2.3.0/2.7/3.x instances)Parameter
|
-Description
+ | Description
|
-Value Range
+ | Value Range
|
-Default Value
+ | Default Value
|
|---|
-connections.max.idle.ms
+ | connections.max.idle.ms
|
-Idle connection timeout (in ms). Connections that are idle for the duration specified by this parameter will be closed.
+ | Idle connection timeout (in ms). Connections that are idle for the duration specified by this parameter will be closed.
|
-5000–600,000
+ | 5000–600,000
|
-600,000
+ | 600,000
|
-log.retention.hours
+ | log.retention.hours
|
-Duration (in hours) for retaining a log file.
+ | Duration (in hours) for retaining a log file.
This parameter takes effect only for topics that have no aging time configured. If there is aging time configured for topics, it overrides this parameter.
|
-1–168
+ | 1–168
|
-72
+ | 72
|
-max.connections.per.ip
+ | group.max.session.timeout.ms
|
-The maximum number of connections allowed from each IP address. Request for new connections will be rejected once the limit is reached. The limit set using this parameter will be replaced if there are overrides configured using the max.connections.per.ip.overrides parameter.
+ | The maximum session timeout (in ms) for consumers. A longer timeout gives consumers more time to process messages between heartbeats but results in a longer time to detect failures.
|
-100–20,000
+ | 6000–1,800,000
|
-1000
+ | 1,800,000
|
-group.max.session.timeout.ms
+ | default.replication.factor
|
-The maximum session timeout (in ms) for consumers. A longer timeout gives consumers more time to process messages between heartbeats but results in a longer time to detect failures.
+ | The default number of replicas configured for an automatically created topic.
|
-6000–1,800,000
+ | 1–3
|
-1,800,000
+ | 3
|
-default.replication.factor
+ | allow.everyone.if.no.acl.found
|
-The default number of replicas configured for an automatically created topic.
+ | When this parameter is set to true, all users can access resources without ACL rules.
+This parameter is displayed only when is enabled for the instance or ciphertext access is used.
+This parameter of some earlier instances cannot be modified.
|
-1–3
+ | true/false
|
-3
+ | true
|
-num.partitions
+ | num.partitions
|
-The default number of partitions configured for each automatically created topic.
+ | The default number of partitions configured for each automatically created topic.
|
-1–100
+ | 1 ~ 200
|
-3
+ | 3
|
-group.min.session.timeout.ms
+ | group.min.session.timeout.ms
|
-The minimum session timeout (in ms) for consumers. A shorter timeout enables quicker failure detection but results in more frequent consumer heartbeating, which can overwhelm broker resources.
+ | The minimum session timeout (in ms) for consumers. A shorter timeout enables quicker failure detection but results in more frequent consumer heartbeating, which can overwhelm broker resources.
|
-6000–300,000
+ | 6000–300,000
|
-6000
+ | 6000
|
@@ -242,4 +319,9 @@
+
diff --git a/docs/dms/umn/kafka-ug-0009.html b/docs/dms/umn/kafka-ug-0009.html
deleted file mode 100644
index f10e2323..00000000
--- a/docs/dms/umn/kafka-ug-0009.html
+++ /dev/null
@@ -1,21 +0,0 @@
-
-
-
- Managing Users
-
-
-
-
-
diff --git a/docs/dms/umn/kafka-ug-0011.html b/docs/dms/umn/kafka-ug-0011.html
index 057742bb..f13db34a 100644
--- a/docs/dms/umn/kafka-ug-0011.html
+++ b/docs/dms/umn/kafka-ug-0011.html
@@ -1,20 +1,21 @@
-
- Managing Consumer Groups
-
-
+ Managing Consumer Groups
+
-
diff --git a/docs/dms/umn/kafka-ug-0012.html b/docs/dms/umn/kafka-ug-0012.html
index 2914fa65..a43395e4 100644
--- a/docs/dms/umn/kafka-ug-0012.html
+++ b/docs/dms/umn/kafka-ug-0012.html
@@ -1,30 +1,32 @@
- Deleting a Consumer Group
- You can delete a consumer group using either of the following methods:
+ Deleting a Kafka Consumer Group
+ You can delete a consumer group in either of the following ways:
- Method 1: Delete a consumer group on the console.
- Method 2: Use Kafka CLI to delete a consumer group. (Ensure that the Kafka instance version is the same as the CLI version.)
PrerequisitesThe status of the consumer group to be deleted is EMPTY.
- Method 1: Deleting a Consumer Group on the Console- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+ Constraints- If auto.create.groups.enable is set to true, the consumer group status is EMPTY, and no offset has been submitted, the system automatically deletes the consumer group 10 minutes later.
- If auto.create.groups.enable is set to false, the system does not automatically delete consumer groups. You can manually delete them.
- If a consumer group has never committed an offset, the group will be deleted after the Kafka instance restarts.
+
+ Method 1: Deleting a Consumer Group on the Console- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
- Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- In the navigation pane, choose the Consumer Groups tab.
- Delete consumer groups using either of the following methods:
- Select one or more consumer groups and click Delete Consumer Group above the consumer group list.
- In the row containing the consumer group you want to delete, click Delete.
A consumer group can be deleted only when its status is EMPTY.
Consumer group statuses include:
- DEAD: The consumer group has no member or metadata.
- EMPTY: The consumer group has metadata but has no member.
- PREPARING_REBALANCE: The consumer group is to be rebalanced.
- COMPLETING_REBALANCE: All members have joined the consumer group.
- STABLE: Members in the consumer group can consume messages normally.
- - In the displayed Delete Consumer Group dialog box, click Yes.
+ - In the displayed Delete Consumer Group dialog box, click OK.
Method 2: Using the CLI to Delete a Consumer GroupThe following uses Linux as an example.
- - Download Kafka CLI v1.1.0, v2.7.2, or v2.3.0. Ensure that the Kafka instance and the CLI are of the same version.
- Use the CLI to connect to the Kafka instance. For details, see Accessing a Kafka Instance Without SASL or Accessing a Kafka Instance with SASL.
- In the /{directory where the CLI is located}/kafka_{version}/bin/ directory, run the following command to delete a consumer group:
kafka-consumer-groups.sh --bootstrap-server {Kafka instance connection address} --delete --group {consumer group name}
+- Download Kafka CLI v1.1.0, v2.3.0, or v2.7.2, or v3.4.0. Ensure that the Kafka instance and the CLI are of the same version.
- Use the CLI to connect to the Kafka instance. For details, see Connecting to Kafka Using the Client (Plaintext Access) or Connecting to Kafka Using the Client (Ciphertext Access).
- In the /bin directory of the Kafka client, run the following command:
./kafka-consumer-groups.sh --bootstrap-server ${connection-address} --delete --group ${consumer-group-name}
+Example:
[root@zk-server-1 bin]# ./kafka-consumer-groups.sh --bootstrap-server 192.168.1.245:9091,192.168.1.86:9091,192.168.1.128:9091 --delete --group bbbb
Note: This will not show information about old Zookeeper-based consumers.
Deletion of requested consumer groups ('bbbb') was successful.
- If SASL authentication is enabled for the Kafka instance, the --command-config {consumer.properties file with SASL authentication} parameter must be added to the preceding commands. For details about the consumer.properties file, see Accessing a Kafka Instance with SASL.
+ If ciphertext access is enabled for the Kafka instance, the --command-config {consumer.properties file with SASL authentication} parameter must be added to the preceding commands. For details about the consumer.properties file, see Connecting to Kafka Using the Client (Ciphertext Access).
-
diff --git a/docs/dms/umn/kafka-ug-0014.html b/docs/dms/umn/kafka-ug-0014.html
index 916b6b9c..659d939e 100644
--- a/docs/dms/umn/kafka-ug-0014.html
+++ b/docs/dms/umn/kafka-ug-0014.html
@@ -1,37 +1,50 @@
- Resetting the Consumer Offset
- Resetting the consumer offset is to change the retrieval position of a consumer.
+ Viewing and Resetting Kafka Consumption Offsets
+ This section describes how to view and reset consumption offsets. Resetting consumption offsets is to change the consumption position for consumers.
Messages may be retrieved more than once after the offset is reset. Exercise caution when performing this operation.
PrerequisitesThe consumer offset cannot be reset on the fly. You must first stop retrieval of the desired consumer group.
-
- Procedure- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+ After a client is stopped, the server considers the client offline only after the time period specified in ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG (1000 ms by default).
- - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- In the navigation pane, choose the Consumer Groups tab.
- Click the name of the desired consumer group.
- On the Consumer Offset tab page, you can perform the following operations:
- To reset the consumer offset of all partitions of a topic, click Reset Consumer Offset in the row containing the desired topic.
- To reset the consumer offset of a single partition of a topic, click Reset Consumer Offset in the row containing the desired partition.
+
+ Viewing Consumer Offsets (Console)- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- In the navigation pane, choose Consumer Groups.
- Click the name of the desired consumer group.
- On the Consumer Offset tab page, view the list of topics that the consumer group has subscribed to, total number of messages accumulated in the topic, message consumption progress in each partition of the topic (accumulated messages, offset, latest offset, consumer ID, consumer address, and client ID).
- (Optional) To query the consumer offsets of a specific topic, enter the topic name in the search box and press Enter.
+
+ Viewing Consumer Offsets (Kafka CLI)
+
+ Resetting Consumer Offsets- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- In the navigation pane, choose the Consumer Groups tab.
- Click the name of the desired consumer group.
- On the Consumer Offset tab page, you can perform the following operations:
- To reset the consumer offset of all partitions of a single topic, click Reset Consumer Offset in the row containing the desired topic.
- To reset the consumer offset of a single partition of a single topic, click Reset Consumer Offset in the row containing the desired partition.
- To reset the consumer offset of all partitions in all topics, click One-touch Reset Consumer Offset above the list.
- In the displayed Reset Consumer Offset dialog box, set the parameters by referring to Table 1.
- Table 1 Parameters for resetting the consumer offsetParameter
+Table 1 Parameters for resetting the consumer offsetParameter
|
-Description
+ | Description
|
|---|
-Reset By
+ | Reset By
|
-You can reset an offset by:
+ | You can reset an offset by:
- Time: Reset the offset to the specified time.
- Offset: Reset the offset to the specified position.
+If you reset offsets in batches, they can only be reset to the specified time.
|
-Time
+ | Time
|
-Set this parameter if Reset By is set to Time.
+ | Set this parameter if Reset By is set to Time.
Select a time point. After the reset is complete, retrieval starts from this time point.
-- Earliest: earliest offset
- Custom Time Range: a custom time point
- Latest: latest offset
+- Earliest: earliest offset
- Custom: a custom time point
- Latest: latest offset
|
-Offset
+ | Offset
|
-Set this parameter if Reset By is set to Offset.
+ | Set this parameter if Reset By is set to Offset.
Enter an offset, which is greater than or equal to 0. After the reset is complete, retrieval starts from this offset.
|
diff --git a/docs/dms/umn/kafka-ug-0015.html b/docs/dms/umn/kafka-ug-0015.html
index dd56601a..9d4bcef7 100644
--- a/docs/dms/umn/kafka-ug-0015.html
+++ b/docs/dms/umn/kafka-ug-0015.html
@@ -1,15 +1,24 @@
-Viewing Consumer Connection Addresses
-View consumer connection addresses on the DMS console.
- The connection address of a consumer can be viewed only when the consumer is connected to a Kafka instance.
+ Viewing Kafka Consumer Details
+ This section describes how to view the consumer list and consumer connection addresses.
+ PrerequisitesThe consumer list and connection address can be viewed only when consumers in a consumer group are connected to the Kafka instance (that is, the consumer group is in the STABLE state).
+
+ Viewing the Consumer List (Console)- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
-Viewing Consumer Addresses on Console- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- In the navigation pane, choose Consumer Groups.
- Click the name of the desired consumer group.
- On the Consumers tab page, view the consumer list.
In the consumer list, you can view the consumer ID, consumer address, and client ID.
+ - (Optional) To query a specific consumer, enter the consumer ID in the search box and press Enter.
+
+ Viewing the Consumer List (Kafka CLI)
+
+ Viewing Consumer Connection Addresses (Console)- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
- Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- In the navigation pane, choose Consumer Groups.
- Click the desired consumer group.
- On the Consumers tab page, view the consumer addresses.
-
-
diff --git a/docs/dms/umn/kafka-ug-0038.html b/docs/dms/umn/kafka-ug-0038.html
new file mode 100644
index 00000000..eecbf6cf
--- /dev/null
+++ b/docs/dms/umn/kafka-ug-0038.html
@@ -0,0 +1,63 @@
+
+
+ Modifying Kafka Topic Configurations
+ This section describes how to modify following configurations of a Kafka topic on the console.
+
+ Table 1 Kafka topic configuration parametersParameter
+ |
+Description
+ |
+
|---|
+
+Partitions
+ |
+Number of partitions in a topic. For details about how to change, see Changing Kafka Partition Quantity.
+ |
+
+Aging Time (h)
+ |
+Maximum message retention. For details about how to change, see Changing Kafka Message Retention Period.
+ |
+
+Synchronous Replication
+ |
+A message is returned to the client only after the message creation request has been received and the message has been acknowledged by all replicas.
+ |
+
+Synchronous Flushing
+ |
+- Enabled: A message is immediately flushed to disk once it is created, bringing higher reliability.
- Disabled: A message is stored in the memory instead of being immediately flushed to disk once created.
+ |
+
+Message Timestamp
+ |
+Timestamp type of a message. Options: - CreateTime: time when the producer created the message.
- LogAppendTime: time when the broker appended the message to the log.
+ |
+
+Max. Message Size
+ |
+Maximum size of messages to be processed in batches. If message compression is enabled, this parameter indicates the size after compression.
+If this value is increased and the consumer version is earlier than 0.10.2, the consumers' fetch size must also be increased so that they can obtain the latest value.
+ |
+
+
+
+
+ Modifying Synchronous Replication, Synchronous Flushing, Message Timestamp, or Max. Message Size does not require instance restart.
+
+ Modifying Topic Configurations- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- In the navigation pane, choose Topics.
- Modify topic configurations in either of the following ways:
- Select one or more topics and click Edit Topic above the topic list.
- In the row containing the desired topic, click Edit.
+ - In the Edit Topic dialog box, change configurations and click OK.
- If there is only one replica, Synchronous Replication cannot be enabled.
- After enabling synchronous replication, set acks to all or –1 on the client. Otherwise, this function will not take effect.
+
+
+
+
+
+
+
diff --git a/docs/dms/umn/kafka-ug-0041.html b/docs/dms/umn/kafka-ug-0041.html
new file mode 100644
index 00000000..2b7312a6
--- /dev/null
+++ b/docs/dms/umn/kafka-ug-0041.html
@@ -0,0 +1,36 @@
+
+
+ Creating a Kafka Consumer Group
+ Create a consumer group on the console.
+ auto.create.groups.enable: a consumer group is automatically created when a consumer attempts to enter a group that does not exist.
+ - This operation is optional when auto.create.groups.enable is true in Configuring Parameters.
- A consumer group is required before consuming messages when auto.create.groups.enable is false in Configuring Parameters. Otherwise, consumption will fail.
+ - If auto.create.groups.enable is set to true, the consumer group status is EMPTY, and no offset has been submitted, the system automatically deletes the consumer group 10 minutes later.
- If auto.create.groups.enable is set to false, the system does not automatically delete consumer groups. You can manually delete them.
- If a consumer group has never committed an offset, the group will be deleted after the Kafka instance restarts.
- Creating a consumer group on the console does not require instance restart.
+
+ Creating a Kafka Consumer Group- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- In the navigation pane, choose Consumer Groups.
- Click Create Consumer Group.
- Set consumer group parameters by referring to Table 1 and click OK.
+ Table 1 Consumer group parametersParameter
+ |
+Description
+ |
+
|---|
+
+Consumer Group Name
+ |
+Enter 3 to 64 characters, starting with a letter or underscore (_). Use only letters, digits, periods (.), hyphens (-), and underscores (_).
+If a consumer group name starts with a special character, for example, an underscore (_) or a number sign (#), monitoring data cannot be displayed.
+ |
+
+
+
+ View the new consumer group in the consumer group list.
+
+
+
+
+
diff --git a/docs/dms/umn/kafka-ug-0053.html b/docs/dms/umn/kafka-ug-0053.html
new file mode 100644
index 00000000..4a5b1b8f
--- /dev/null
+++ b/docs/dms/umn/kafka-ug-0053.html
@@ -0,0 +1,17 @@
+
+
+ Exporting the Kafka Instance List
+ ScenarioThis section describes how to export the Kafka instance list from the console.
+
+ Exporting the Kafka Instance List- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Export the instance list using either of the following methods:
- Select the desired instances and choose Export > Export selected data to an XLSX file to export specified instances.
- Choose Export > Export all data to an XLSX file to export all instances.
+
+
+
+
+
diff --git a/docs/dms/umn/kafka-ug-0056.html b/docs/dms/umn/kafka-ug-0056.html
new file mode 100644
index 00000000..f9ba335c
--- /dev/null
+++ b/docs/dms/umn/kafka-ug-0056.html
@@ -0,0 +1,17 @@
+
+
+ Exporting Kafka Consumer Groups
+ ScenarioExport the consumer group list from the console.
+
+ Exporting Kafka Consumer Groups- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view its details.
- In the navigation pane, choose Consumer Groups.
- Export consumer groups in either of the following ways:
- Select the desired consumer groups and choose Export > Export selected data to an XLSX file to export specified consumer groups.
- Choose Export > Export all data to an XLSX file to export all consumer groups.
+
+
+
+
+
diff --git a/docs/dms/umn/kafka-ug-0058.html b/docs/dms/umn/kafka-ug-0058.html
new file mode 100644
index 00000000..34616522
--- /dev/null
+++ b/docs/dms/umn/kafka-ug-0058.html
@@ -0,0 +1,31 @@
+
+
+
+ Managing Topics
+
+
+
+
+
diff --git a/docs/dms/umn/kafka-ug-0059.html b/docs/dms/umn/kafka-ug-0059.html
new file mode 100644
index 00000000..8b79b9fd
--- /dev/null
+++ b/docs/dms/umn/kafka-ug-0059.html
@@ -0,0 +1,25 @@
+
+
+
+ Configuring Kafka Network Connections
+
+
+
+
+
diff --git a/docs/dms/umn/kafka-ug-0061.html b/docs/dms/umn/kafka-ug-0061.html
new file mode 100644
index 00000000..2624f327
--- /dev/null
+++ b/docs/dms/umn/kafka-ug-0061.html
@@ -0,0 +1,21 @@
+
+
+
+ Configuring Kafka Access Control
+
+
+
+
+
diff --git a/docs/dms/umn/kafka-ug-0062.html b/docs/dms/umn/kafka-ug-0062.html
new file mode 100644
index 00000000..ae1c00c8
--- /dev/null
+++ b/docs/dms/umn/kafka-ug-0062.html
@@ -0,0 +1,21 @@
+
+
+
+ Configuring the Kafka Client
+
+
+
+
+
diff --git a/docs/dms/umn/kafka-ug-0069.html b/docs/dms/umn/kafka-ug-0069.html
new file mode 100644
index 00000000..560f0e56
--- /dev/null
+++ b/docs/dms/umn/kafka-ug-0069.html
@@ -0,0 +1,14 @@
+
+
+ Process of Using Kafka
+ The following figure shows the process of using a Kafka instance to produce and consume messages.
+ Figure 1 Process of using Kafka
+ - Creating a User and Granting DMS for Kafka Permissions
Create IAM users and grant them only the DMS for Kafka permissions required to perform a given task based on their job responsibilities.
+ - Creating a Kafka Instance
Kafka instances are tenant-exclusive, and physically isolated in deployment.
+ - Creating a Kafka Topic
Create a topic for storing messages so that producers can produce messages and consumers can subscribe to messages.
+ - Connecting to an Instance
The client uses commands to connect to Kafka instances in a private or public network, and produces and consumes messages.
+ - Managing Messages
View messages on the console.
+
+
+
+
diff --git a/docs/dms/umn/kafka-ug-0319001.html b/docs/dms/umn/kafka-ug-0319001.html
index e67676ff..616f1639 100644
--- a/docs/dms/umn/kafka-ug-0319001.html
+++ b/docs/dms/umn/kafka-ug-0319001.html
@@ -1,64 +1,104 @@
- Configuring Public Access
+ Configuring Kafka Public Access
To access a Kafka instance over a public network, enable public access and configure EIPs for the instance.
If you no longer need public access to the instance, you can disable it as required.
- Procedure- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+ Prerequisites- You can change the public access setting only when the Kafka instance is in the Running state.
- Kafka instances only support IPv4 EIPs. IPv6 EIPs are not supported.
+
+ Enabling Public Access- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
- - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click a Kafka instance to go to the Basic Information tab page.
- Configure public access.
- You can change the public access setting only when the Kafka instance is in the Running state.
- Only IPv4 EIPs can be bound to Kafka instances.
-
-Enabling public access
-Click  next to Public Access to enable public access. For Elastic IP Address, select an EIP for each broker and then click  .
-You can view the operation progress on the Background Tasks page. If the task status is Successful, the modification has succeeded.
-Figure 1 Configuring public access
-After public access is enabled, configure security group rules listed in Table 1 before attempting to access Kafka. For details about accessing Kafka, see Accessing a Kafka Instance.
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click a Kafka instance to go to the Basic Information page.
- Click
 next to Public Access to enable public access. For Elastic IP Address, select an EIP for each broker. If the number of EIPs is insufficient, click Create Elastic IP to go to the Create EIP page and create EIPs. For details, see Assigning an EIP. After the creation is complete, return to the page for enabling the public access. Click  next to Elastic IP Address, select EIPs from the drop-down list. The number of EIPs must be the same as the number of brokers. Then click  and the Background Tasks page is displayed.If the status of the task turns to Successful, public access is successfully enabled. Figure 1 Enabling public access
+ After public access is enabled, configure the access mode (plaintext or ciphertext) and security group rules listed in Table 1 before attempting to access Kafka. For details about accessing Kafka, see Connecting to an Instance.
-Table 1 Security group rulesDirection
+Table 1 Security group rulesDirection
|
-Protocol
+ | Protocol
|
-Port
+ | Port
|
-Source
+ | Source
|
-Description
+ | Description
|
|---|
-Inbound
+ | Inbound
|
-TCP
+ | TCP
|
-9094
+ | 9094
|
-0.0.0.0/0
+ | 0.0.0.0/0
|
-Access Kafka through the public network (without SSL encryption).
+ | Accessing Kafka over a public network (in plaintext)
|
-Inbound
+ | Inbound
|
-TCP
+ | TCP
|
-9095
+ | 9095
|
-0.0.0.0/0
+ | 0.0.0.0/0
|
-Access Kafka through the public network (with SSL encryption).
+ | Accessing Kafka over a public network (in ciphertext)
|
Disabling public access
-Click  next to Public Access.
-You can view the operation progress on the Background Tasks page. If the task status is Successful, the modification has succeeded.
+
+
+Disabling Public Access- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click a Kafka instance to go to the Basic Information page.
- Before disabling public access, disable Plaintext Access and Ciphertext Access next to Public Network Access. Then click
 next to Public Access. - Click OK. The Background Tasks page is displayed. If the status of the task turns to Successful, public access is successfully disabled.
After public access is disabled, configure security group rules listed in Table 2 before attempting to access Kafka in a VPC. For details about accessing Kafka, see Connecting to an Instance.
+ Table 2 Security group rules (private network access)Direction
+ |
+Protocol
+ |
+Port
+ |
+Source
+ |
+Description
+ |
+
|---|
+
+Inbound
+ |
+TCP
+ |
+9092
+ |
+0.0.0.0/0
+ |
+Accessing a Kafka instance over a private network within a VPC (in plaintext)
+ |
+
+Inbound
+ |
+TCP
+ |
+9093
+ |
+0.0.0.0/0
+ |
+Accessing a Kafka instance over a private network within a VPC (in ciphertext)
+ |
+
+
+
+
+ After a security group is created, its default inbound rule allows communication among ECSs within the security group and its default outbound rule allows all outbound traffic. In this case, you can access a Kafka instance within a VPC, and do not need to add rules according to Table 2.
+
+
Managing Topics
-
-
-
+Configuring Topics
+
diff --git a/docs/dms/umn/kafka-ug-0720002.html b/docs/dms/umn/kafka-ug-0720002.html
index 7198c52e..5d9085d5 100644
--- a/docs/dms/umn/kafka-ug-0720002.html
+++ b/docs/dms/umn/kafka-ug-0720002.html
@@ -1,14 +1,15 @@
-
- Managing Messages
-
-
+ Managing Messages
+
-
diff --git a/docs/dms/umn/kafka-ug-0723001.html b/docs/dms/umn/kafka-ug-0723001.html
index 67d5e797..5976de14 100644
--- a/docs/dms/umn/kafka-ug-0723001.html
+++ b/docs/dms/umn/kafka-ug-0723001.html
@@ -13,6 +13,8 @@
- Specifications
+ - Comparing Single-node and Cluster Kafka Instances
+
- Comparing DMS for Kafka and Open-Source Kafka
- Notes and Constraints
@@ -23,7 +25,7 @@
- Best Practices
- - Permissions
+ - Permission
- Instances
+- Specification Modification
+
- Connections
- Topics and Partitions
diff --git a/docs/dms/umn/kafka-ug-0723006.html b/docs/dms/umn/kafka-ug-0723006.html
new file mode 100644
index 00000000..ddfaeafc
--- /dev/null
+++ b/docs/dms/umn/kafka-ug-0723006.html
@@ -0,0 +1,23 @@
+
+
+
+ Troubleshooting
+
+
+
+
+
diff --git a/docs/dms/umn/kafka-ug-180413001.html b/docs/dms/umn/kafka-ug-180413001.html
index 62b30b1e..bc4eee52 100644
--- a/docs/dms/umn/kafka-ug-180413001.html
+++ b/docs/dms/umn/kafka-ug-180413001.html
@@ -1,12 +1,14 @@
-Monitoring
+Monitoring and Alarms
diff --git a/docs/dms/umn/kafka-ug-180413002.html b/docs/dms/umn/kafka-ug-180413002.html
new file mode 100644
index 00000000..0e2248b4
--- /dev/null
+++ b/docs/dms/umn/kafka-ug-180413002.html
@@ -0,0 +1,684 @@
+
+
+Kafka Metrics
+IntroductionThis section describes metrics reported by DMS to Cloud Eye as well as their namespaces and dimensions. You can use the Cloud Eye console or APIs to query the Kafka metrics and alarms, or view Kafka instance metrics on the Monitoring page of the DMS console.
+ For example, you can call the API to query the monitoring data of the Disk Capacity Usage metric.
+
+
+ Instance Metrics
+ Table 1 Instance metricsMetric ID
+ |
+Metric Name
+ |
+Description
+ |
+Value Range
+ |
+Monitored Object
+ |
+Monitoring Period (Raw Data)
+ |
+
|---|
+
+current_partitions
+ |
+Partitions
+ |
+Number of used partitions in the instance
+Unit: count
+ |
+0~100000
+ |
+Kafka instance
+ |
+1 minute
+ |
+
+current_topics
+ |
+Topics
+ |
+Number of created topics in the instance
+Unit: count
+ |
+0~100000
+ |
+Kafka instance
+ |
+1 minute
+ |
+
+group_msgs
+ |
+Accumulated Messages
+ |
+Total number of accumulated messages in all consumer groups of the instance
+Unit: count
+ |
+0–1,000,000,000
+ |
+Kafka instance
+ |
+1 minute
+ |
+
+
+
+
+
+ Broker Metrics
+ Table 2 Broker metricsMetric ID
+ |
+Metric Name
+ |
+Description
+ |
+Value Range
+ |
+Monitored Object
+ |
+Monitoring Period (Raw Data)
+ |
+
|---|
+
+broker_data_size
+ |
+Message Size
+ |
+Total size of messages in the broker
+Unit: byte, KB, MB, GB, TB or PB
+ |
+0–5,000,000,000,000
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_messages_in_rate
+ |
+Message Creation Rate
+ |
+Number of messages created per second
+Unit: count/s
+ |
+0–500,000
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_bytes_out_rate
+ |
+Message Retrieval
+ |
+Number of bytes retrieved per second
+Unit: byte/s, KB/s, MB/s, or GB/s
+ |
+0–500,000,000
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_bytes_in_rate
+ |
+Message Creation
+ |
+Number of bytes created per second
+Unit: byte/s, KB/s, MB/s, or GB/s
+ |
+0–500,000,000
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_public_bytes_in_rate
+ |
+Public Inbound Traffic
+ |
+Inbound traffic over public networks per second
+Unit: byte/s, KB/s, MB/s, or GB/s
+ NOTE: You can view this metric on the EIP console if public access has been enabled and EIPs have been assigned to the instance.
+
+ |
+0–500,000,000
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_public_bytes_out_rate
+ |
+Public Outbound Traffic
+ |
+Outbound traffic over public networks per second
+Unit: byte/s, KB/s, MB/s, or GB/s
+ NOTE: You can view this metric on the EIP console if public access has been enabled and EIPs have been assigned to the instance.
+
+ |
+0–500,000,000
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_fetch_mean
+ |
+Average Message Retrieval Processing Duration
+ |
+Average time that the broker spends processing message retrieval requests
+Unit: ms
+ |
+0–10,000
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_produce_mean
+ |
+Average Message Creation Processing Duration
+ |
+Average time that the broker spends processing message creation requests
+Unit: ms
+ |
+0–10,000
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_cpu_core_load
+ |
+Average Load per CPU Core
+ |
+Average load of each CPU core of the Kafka VM
+Unit: %
+ |
+0–20
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_disk_usage
+ |
+Disk Capacity Usage
+ |
+Disk usage of the Kafka VM
+Unit: %
+ |
+0–100
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_memory_usage
+ |
+Memory Usage
+ |
+Memory usage of the Kafka VM
+Unit: %
+ |
+0–100
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_heap_usage
+ |
+JVM Heap Memory Usage of Kafka
+ |
+Heap memory usage of the Kafka JVM
+Unit: %
+ |
+0–100
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_alive
+ |
+Broker Alive
+ |
+Whether the Kafka broker is alive
+ |
+
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_connections
+ |
+Connections
+ |
+Total number of TCP connections on the Kafka broker
+Unit: count
+ |
+0–65,535
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_cpu_usage
+ |
+CPU Usage
+ |
+CPU usage of the Kafka VM
+Unit: %
+ |
+0–100
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_disk_read_await
+ |
+Average Disk Read Time
+ |
+Average time for each disk I/O read in the monitoring period
+Unit: ms
+ |
+> 0
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_disk_write_await
+ |
+Average Disk Write Time
+ |
+Average time for each disk I/O write in the monitoring period
+Unit: ms
+ |
+> 0
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_total_bytes_in_rate
+ |
+Inbound Traffic
+ |
+Inbound traffic per second
+Unit: byte/s
+ |
+0–1,000,000,000
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_total_bytes_out_rate
+ |
+Outbound Traffic
+ |
+Outbound traffic per second
+Unit: byte/s
+ |
+0–1,000,000,000
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_disk_read_rate
+ |
+Disk Read Speed
+ |
+Read traffic on the disk
+Unit: byte/s, KB/s, MB/s, or GB/s
+ |
+≥ 0
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+broker_disk_write_rate
+ |
+Disk Write Speed
+ |
+Write traffic on the disk
+Unit: byte/s, KB/s, MB/s, or GB/s
+ |
+≥ 0
+ |
+Kafka instance broker
+ |
+1 minute
+ |
+
+
+
+
+
+ Topic Metrics
+ Table 3 Topic metricsMetric ID
+ |
+Metric Name
+ |
+Description
+ |
+Value Range
+ |
+Monitored Object
+ |
+Monitoring Period (Raw Data)
+ |
+
|---|
+
+topic_bytes_in_rate
+ |
+Message Creation
+ |
+Number of bytes created per second
+Unit: byte/s, KB/s, MB/s, or GB/s
+ NOTE: - On the Cloud Eye console, this metric is available only when Scope is set to Basic monitoring on the Queues tab page.
- On the Monitoring page of the DMS console, this metric is available only when Monitoring Type is set to Basic monitoring on the By Topic tab page.
+
+ |
+0–500,000,000
+ |
+Topic in a Kafka instance
+ |
+1 minute
+ |
+
+topic_bytes_out_rate
+ |
+Message Retrieval
+ |
+Number of bytes retrieved per second
+Unit: byte/s, KB/s, MB/s, or GB/s
+ NOTE: - On the Cloud Eye console, this metric is available only when Scope is set to Basic monitoring on the Queues tab page.
- On the Monitoring page of the DMS console, this metric is available only when Monitoring Type is set to Basic monitoring on the By Topic tab page.
+
+ |
+0–500,000,000
+ |
+Topic in a Kafka instance
+ |
+1 minute
+ |
+
+topic_data_size
+ |
+Message Size
+ |
+Total size of messages in the queue
+Unit: byte, KB, MB, GB, TB or PB
+ NOTE: - On the Cloud Eye console, this metric is available only when Scope is set to Basic monitoring on the Queues tab page.
- On the Monitoring page of the DMS console, this metric is available only when Monitoring Type is set to Basic monitoring on the By Topic tab page.
+
+ |
+0–5,000,000,000,000
+ |
+Topic in a Kafka instance
+ |
+1 minute
+ |
+
+topic_messages
+ |
+Total Messages
+ |
+Total number of messages in the queue
+Unit: count
+ NOTE: - On the Cloud Eye console, this metric is available only when Scope is set to Basic monitoring on the Queues tab page.
- On the Monitoring page of the DMS console, this metric is available only when Monitoring Type is set to Basic monitoring on the By Topic tab page.
+
+ |
+≥ 0
+ |
+Topic in a Kafka instance
+ |
+1 minute
+ |
+
+topic_messages_in_rate
+ |
+Message Creation Rate
+ |
+Number of messages created per second
+Unit: count/s
+ NOTE: - On the Cloud Eye console, this metric is available only when Scope is set to Basic monitoring on the Queues tab page.
- On the Monitoring page of the DMS console, this metric is available only when Monitoring Type is set to Basic monitoring on the By Topic tab page.
+
+ |
+0–500,000
+ |
+Topic in a Kafka instance
+ |
+1 minute
+ |
+
+partition_messages
+ |
+Partition Messages
+ |
+Total number of messages in the partition
+Unit: count
+ NOTE: - On the Cloud Eye console, this metric is available only when Scope is set to Partition monitoring on the Queues tab page.
- On the Monitoring page of the DMS console, this metric is available only when Monitoring Type is set to Partition monitoring on the By Topic tab page.
+
+ |
+≥ 0
+ |
+Topic in a Kafka instance
+ |
+1 minute
+ |
+
+produced_messages
+ |
+Created Messages
+ |
+Number of messages that have been created
+Unit: count
+ NOTE: - On the Cloud Eye console, this metric is available only when Scope is set to Partition monitoring on the Queues tab page.
- On the Monitoring page of the DMS console, this metric is available only when Monitoring Type is set to Partition monitoring on the By Topic tab page.
+
+ |
+≥ 0
+ |
+Topic in a Kafka instance
+ |
+1 minute
+ |
+
+
+
+
+
+ Consumer Group Metrics
+ Table 4 Consumer group metricsMetric ID
+ |
+Metric Name
+ |
+Description
+ |
+Value Range
+ |
+Monitored Object
+ |
+Monitoring Period (Raw Data)
+ |
+
|---|
+
+messages_consumed
+ |
+Retrieved Messages
+ |
+Number of messages that have been retrieved in the consumer group
+Unit: count
+ NOTE: - On the Cloud Eye console, this metric is available only when Queue is set to a specific topic name and Scope is set to Partition monitoring on the Consumer Groups tab page.
- On the Monitoring page of the DMS console, this metric is available only when Topic is set to a specific topic name and Monitoring Type is set to Partition monitoring on the By Consumer Group tab page.
+
+ |
+≥ 0
+ |
+Consumer group of a Kafka instance
+ |
+1 minute
+ |
+
+messages_remained
+ |
+Available Messages
+ |
+Number of messages that can be retrieved in the consumer group
+Unit: count
+ NOTE: - On the Cloud Eye console, this metric is available only when Queue is set to a specific topic name and Scope is set to Partition monitoring on the Consumer Groups tab page.
- On the Monitoring page of the DMS console, this metric is available only when Topic is set to a specific topic name and Monitoring Type is set to Partition monitoring on the By Consumer Group tab page.
+
+ |
+≥ 0
+ |
+Consumer group of a Kafka instance
+ |
+1 minute
+ |
+
+topic_messages_remained
+ |
+Topic Available Messages
+ |
+Number of remaining messages that can be retrieved from the specified topic in the consumer group
+Unit: Count
+ NOTE: - On the Cloud Eye console, this metric is available only when Queue is set to a specific topic name and Scope is set to Basic monitoring on the Consumer Groups tab page.
- On the Monitoring page of the DMS console, this metric is available only when Topic is set to a specific topic name and Monitoring Type is set to Basic monitoring on the By Consumer Group tab page.
+
+ |
+0 to 263–1
+ |
+Consumer group of a Kafka instance
+ |
+1 minute
+ |
+
+topic_messages_consumed
+ |
+Topic Retrieved Messages
+ |
+Number of messages that have been retrieved from the specified topic in the consumer group
+Unit: Count
+ NOTE: - On the Cloud Eye console, this metric is available only when Queue is set to a specific topic name and Scope is set to Basic monitoring on the Consumer Groups tab page.
- On the Monitoring page of the DMS console, this metric is available only when Topic is set to a specific topic name and Monitoring Type is set to Basic monitoring on the By Consumer Group tab page.
+
+ |
+0 to 263–1
+ |
+Consumer group of a Kafka instance
+ |
+1 minute
+ |
+
+consumer_messages_remained
+ |
+Consumer Available Messages
+ |
+Number of remaining messages that can be retrieved in the consumer group
+Unit: Count
+ NOTE: - On the Cloud Eye console, this metric is available only when Queues is set to All queues on the Consumer Groups tab page.
- On the Monitoring page of the DMS console, this metric is available only when Topic is set to All topics on the By Consumer Group tab page.
+
+ |
+0 to 263–1
+ |
+Consumer group of a Kafka instance
+ |
+1 minute
+ |
+
+consumer_messages_consumed
+ |
+Consumer Retrieved Messages
+ |
+Number of messages that have been retrieved in the consumer group
+Unit: Count
+ NOTE: - On the Cloud Eye console, this metric is available only when Queues is set to All queues on the Consumer Groups tab page.
- On the Monitoring page of the DMS console, this metric is available only when Topic is set to All topics on the By Consumer Group tab page.
+
+ |
+0 to 263–1
+ |
+Consumer group of a Kafka instance
+ |
+1 minute
+ |
+
+
+
+
+
+ Dimension
+ Key
+ |
+Value
+ |
+
|---|
+
+kafka_instance_id
+ |
+Kafka instance
+ |
+
+kafka_broker
+ |
+Kafka instance broker
+ |
+
+kafka_topics
+ |
+Kafka instance topic
+ |
+
+kafka_partitions
+ |
+Partition in a Kafka instance
+ |
+
+kafka_groups-partitions
+ |
+Partition consumer group in a Kafka instance
+ |
+
+kafka_groups_topics
+ |
+Topic consumer group in a Kafka instance
+ |
+
+kafka_groups
+ |
+Consumer group of a Kafka instance
+ |
+
+
+
+
+
+ Auditing
-
-
-
diff --git a/docs/dms/umn/kafka-ug-180418002.html b/docs/dms/umn/kafka-ug-180418002.html
index 606839da..bd88e26b 100644
--- a/docs/dms/umn/kafka-ug-180418002.html
+++ b/docs/dms/umn/kafka-ug-180418002.html
@@ -1,282 +1,395 @@
-Operations Logged by CTS
-With Cloud Trace Service (CTS), you can record operations associated with DMS for later query, audit, and backtrack operations.
-
- Table 1 DMS operations that can be recorded by CTSOperation
+Viewing Kafka Audit Logs
+ScenarioWith Cloud Trace Service (CTS), you can record operations associated with DMS for later query, audit, and backtrack operations.
+
+ PrerequisiteCTS has been enabled.
+
+ DMS Operations Supported by CTS
+ Table 1 DMS operations that can be recorded by CTSOperation
|
-Resource Type
+ | Resource Type
|
-Trace Name
+ | Trace Name
|
|---|
-Successfully deleting a background task
+ | Successfully creating an instance
|
-kafka
+ | kafka
|
-deleteDMSBackendJobSuccess
+ | createDMSInstanceTaskSuccess
|
-Failing to delete a background task
+ | Failing to create an instance
|
-kafka
+ | kafka
|
-deleteDMSBackendJobFailure
+ | createDMSInstanceTaskFailure
|
-Successfully creating an order for creating an instance
+ | Successfully deleting an instance that failed to be created
|
-kafka
+ | kafka
|
-createDMSInstanceOrderSuccess
+ | deleteDMSCreateFailureInstancesSuccess
|
-Failing to create an order for creating an instance
+ | Failing to delete an instance that failed to be created
|
-kafka
+ | kafka
|
-createDMSInstanceOrderFailure
+ | deleteDMSCreateFailureInstancesFailure
|
-Successfully submitting a request to modify an instance order
+ | Successfully deleting an instance
|
-kafka
+ | kafka
|
-modifyDMSInstanceOrderSuccess
+ | deleteDMSInstanceTaskSuccess
|
-Failing to submit a request to modify an instance order
+ | Failing to delete an instance
|
-kafka
+ | kafka
|
-modifyDMSInstanceOrderFailure
+ | deleteDMSInstanceTaskFailure
|
-Successfully submitting a request to scale up an instance
+ | Deleting multiple instance tasks at a time
|
-kafka
+ | kafka
|
-extendDMSInstanceSuccess
+ | batchDeleteDMSInstanceTask
|
-Failing to submit a request to scale up an instance
+ | Successfully submitting a request to delete multiple instances at a time
|
-kafka
+ | kafka
|
-extendDMSInstanceFailure
+ | batchDeleteDMSInstanceSuccess
|
-Successfully submitting a request to reset instance password
+ | Successfully deleting multiple instances at a time
|
-kafka
+ | kafka
|
-resetDMSInstancePasswordSuccess
+ | batchDeleteDMSInstanceTaskSuccess
|
-Failing to submit a request to reset instance password
+ | Failing to submit a request to delete multiple instances at a time
|
-kafka
+ | kafka
|
-resetDMSInstancePasswordFailure
+ | batchDeleteDMSInstanceFailure
|
-Successfully creating a topic for a Kafka instance
+ | Failing to delete multiple instances at a time
|
-kafka
+ | kafka
|
-Kafka_platinum_create_topicSuccess
+ | batchDeleteDMSInstanceTaskFailure
|
-Failing to create a topic for a Kafka instance
+ | Successfully submitting a request to scale up an instance
|
-kafka
+ | kafka
|
-Kafka_platinum_create_topicFailure
+ | extendDMSInstanceSuccess
|
-Successfully deleting a topic from a Kafka instance
+ | Successfully scaling up an instance
|
-kafka
+ | kafka
|
-Kafka_platinum_delete_topicsSuccess
+ | extendDMSInstanceTaskSuccess
|
-Failing to delete a topic for a Kafka instance
+ | Failing to submit a request to scale up an instance
|
-kafka
+ | kafka
|
-Kafka_platinum_delete_topicsFailure
+ | extendDMSInstanceFailure
|
-Successfully deleting an instance that failed to be created
+ | Failing to scale up an instance
|
-kafka
+ | kafka
|
-deleteDMSCreateFailureInstancesSuccess
+ | extendDMSInstanceTaskFailure
|
-Failing to delete an instance that failed to be created
+ | Successfully submitting a request to reset instance password
|
-kafka
+ | kafka
|
-deleteDMSCreateFailureInstancesFailure
+ | resetDMSInstancePasswordSuccess
|
-Successfully submitting a request to restart an instance
+ | Failing to submit a request to reset instance password
|
-kafka
+ | kafka
|
-restartDMSInstanceSuccess
+ | resetDMSInstancePasswordFailure
|
-Failing to submit a request to restart an instance
+ | Successfully submitting a request to restart an instance
|
-kafka
+ | kafka
|
-restartDMSInstanceFailure
+ | restartDMSInstanceSuccess
|
-Successfully submitting a request to delete multiple instances at a time
+ | Successfully restarting an instance
|
-kafka
+ | kafka
|
-batchDeleteDMSInstanceSuccess
+ | restartDMSInstanceTaskSuccess
|
-Failing to submit a request to delete multiple instances at a time
+ | Failing to submit a request to restart an instance
|
-kafka
+ | kafka
|
-batchDeleteDMSInstanceFailure
+ | restartDMSInstanceFailure
|
-Successfully submitting a request to restart multiple instances at a time
+ | Failing to restart an instance
|
-kafka
+ | kafka
|
-batchRestartDMSInstanceSuccess
+ | restartDMSInstanceTaskFailure
|
-Failing to submit a request to restart multiple instances at a time
+ | Successfully submitting a request to restart multiple instances at a time
|
-kafka
+ | instance
|
-batchRestartDMSInstanceFailure
+ | batchRestartDMSInstanceSuccess
|
-Successfully submitting a request to modify instance information
+ | Successfully restarting multiple instances at a time
|
-kafka
+ | kafka
|
-modifyDMSInstanceInfoSuccess
+ | batchRestartDMSInstanceTaskSuccess
|
-Failing to submit a request to modify instance information
+ | Failing to submit a request to restart multiple instances at a time
|
-kafka
+ | instance
|
-modifyDMSInstanceInfoFailure
+ | batchRestartDMSInstanceFailure
|
-Deleting multiple instance tasks at a time
+ | Failing to restart multiple instances at a time
|
-kafka
+ | kafka
|
-batchDeleteDMSInstanceTask
+ | batchRestartDMSInstanceTaskFailure
|
-Successfully deleting an instance
+ | Successfully submitting a request to modify instance information
|
-kafka
+ | kafka
|
-deleteDMSInstanceTaskSuccess
+ | modifyDMSInstanceInfoSuccess
|
-Failing to delete an instance
+ | Successfully modifying instance information
|
-kafka
+ | kafka
|
-deleteDMSInstanceTaskFailure
+ | modifyDMSInstanceInfoTaskSuccess
|
-Successfully creating an instance
+ | Failing to submit a request to modify instance information
|
-kafka
+ | kafka
|
-createDMSInstanceTaskSuccess
+ | modifyDMSInstanceInfoFailure
|
-Failing to create an instance
+ | Failing to modify instance information
|
-kafka
+ | kafka
|
-createDMSInstanceTaskFailure
+ | modifyDMSInstanceInfoTaskFailure
|
-Successfully scaling up an instance
+ | Successfully deleting a background task
|
-kafka
+ | kafka
|
-extendDMSInstanceTaskSuccess
+ | deleteDMSBackendJobSuccess
|
-Failing to scale up an instance
+ | Failing to delete a background task
|
-kafka
+ | kafka
|
-extendDMSInstanceTaskFailure
+ | deleteDMSBackendJobFailure
|
-Successfully restarting an instance
+ | Successfully creating a topic for a Kafka instance
|
-kafka
+ | kafka
|
-restartDMSInstanceTaskSuccess
+ | Kafka_create_topicSuccess
|
-Failing to restart an instance
+ | Failing to create a topic for a Kafka instance
|
-kafka
+ | kafka
|
-restartDMSInstanceTaskFailure
+ | Kafka_create_topicFailure
|
-Successfully restarting multiple instances at a time
+ | Successfully deleting a topic from a Kafka instance
|
-kafka
+ | kafka
|
-batchRestartDMSInstanceTaskSuccess
+ | Kafka_delete_topicsSuccess
|
-Failing to restart multiple instances at a time
+ | Failing to delete a topic for a Kafka instance
|
-kafka
+ | kafka
|
-batchRestartDMSInstanceTaskFailure
+ | Kafka_delete_topicsFailure
|
-Successfully modifying instance information
+ | Successfully enabling automatic topic creation
|
-kafka
+ | kafka
|
-modifyDMSInstanceInfoTaskSuccess
+ | enable_auto_topicSuccess
|
-Failing to modify instance information
+ | Failing to enable automatic topic creation
|
-kafka
+ | kafka
|
-modifyDMSInstanceInfoTaskFailure
+ | enable_auto_topicFailure
+ |
+
+Successfully modifying a topic
+ |
+kafka
+ |
+Kafka_alter_topicsSuccess
+ |
+
+Failing to modify a topic
+ |
+kafka
+ |
+Kafka_alter_topicsFailure
+ |
+
+Successfully reassigning partitions
+ |
+kafka
+ |
+kafka_reassignmentTaskSuccess
+ |
+
+Failing to reassign partitions
+ |
+kafka
+ |
+kafka_reassignmentTaskFailure
+ |
+
+Successfully submitting a partition reassignment request
+ |
+kafka
+ |
+kafka_reassignmentSuccess
+ |
+
+Failing to submit a partition reassignment request
+ |
+kafka
+ |
+kafka_reassignmentFailure
+ |
+
+Successfully resetting the consumer offset
+ |
+kafka
+ |
+Kafka_reset_consumer_offsetSuccess
+ |
+
+Failing to reset the consumer offset
+ |
+kafka
+ |
+Kafka_reset_consumer_offsetFailure
+ |
+
+Successfully deleting consumer groups in batches
+ |
+kafka
+ |
+Kafka_batch_delete_groupSuccess
+ |
+
+Failing to delete consumer groups in batches
+ |
+kafka
+ |
+Kafka_batch_delete_groupFailure
+ |
+
+Successfully creating a user
+ |
+kafka
+ |
+createUserSuccess
+ |
+
+Failing to create a user
+ |
+kafka
+ |
+createUserFailure
+ |
+
+Successfully deleting a user
+ |
+kafka
+ |
+deleteUserSuccess
+ |
+
+Failing to delete a user
+ |
+kafka
+ |
+deleteUserFailure
+ |
+
+Successfully updating user policies
+ |
+kafka
+ |
+updateUserPoliciesTaskSuccess
+ |
+
+Failing to update user policies
+ |
+kafka
+ |
+updateUserPoliciesTaskFailure
|
-
-
diff --git a/docs/dms/umn/kafka-ug-180418003.html b/docs/dms/umn/kafka-ug-180418003.html
deleted file mode 100644
index 46d2a6d2..00000000
--- a/docs/dms/umn/kafka-ug-180418003.html
+++ /dev/null
@@ -1,20 +0,0 @@
-
-
- Viewing Audit Logs
- ScenarioThis section describes how to view operation records of the last 7 days on the CTS console.
-
- Procedure- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
-
- - Click Service List and choose Management & Deployment > Cloud Trace Service.
- In the navigation pane, choose Trace List.
- Set filters to search for desired traces. The following filters are available:
- Trace Source: Select DMS.
- Resource Type: Select kafka or Instance.
- Search By: Select an option from the drop-down list.
- When you select Trace name, you also need to select a specific trace name.
- If you select Resource ID for Search By, you need to enter a specific resource ID. The corresponding operation trace can be queried only when the resource ID is completely matched.
- When you select Resource name, you also need to select a specific resource name.
- - Operator: Select a specific operator (a user other than tenant).
- Trace Status: Available options include All trace statuses, normal, warning, and incident. You can only select one of them.
- Time Range: In the upper right corner, choose Last 1 hour, Last 1 day, or Last 1 week, or specify a custom time range. If you select Customize, you also need to select the start time and end time, and then click OK.
- - Click on the left of a trace to expand its details.
Figure 1 Expanding trace details
- - Click View Trace in the Operation column. In the dialog box, the trace details are displayed, as shown in Figure 2.
Figure 2 Viewing a trace
-
-
-
-
-
diff --git a/docs/dms/umn/kafka-ug-180524001.html b/docs/dms/umn/kafka-ug-180524001.html
new file mode 100644
index 00000000..4c0edf43
--- /dev/null
+++ b/docs/dms/umn/kafka-ug-180524001.html
@@ -0,0 +1,143 @@
+
+
+ Configuring a Kafka Alarm Rule
+ This section describes the alarm rules of some metrics and how to configure them. In actual services, you are advised to configure alarm rules for metrics based on the following alarm policies:
+
+ Table 1 Alarm policies and handling of Kafka instancesMetric ID
+ |
+Metric
+ |
+Monitored Object
+ |
+Alarm Policy
+ |
+Description
+ |
+Handling Suggestion
+ |
+
|---|
+
+broker_disk_usage
+ |
+Disk Capacity Usage
+ |
+Broker
+ |
+Alarm threshold: original value > 80%
+Number of consecutive periods: 1
+Alarm severity: critical
+ |
+Disk usage of the Kafka VM
+ |
+Modify the instance storage space. For details, see Modifying Instance Specifications.
+ |
+
+broker_cpu_core_load
+ |
+Average Load per CPU Core
+ |
+Broker
+ |
+Alarm threshold: original value > 2
+Number of consecutive periods: 3
+Alarm severity: major
+ |
+Average load of each CPU core of the Kafka VM.
+ |
+Check whether the metric has been approaching or exceeding the alarm threshold for a long time. If yes, modify the number of brokers. For details, see Modifying Instance Specifications.
+ |
+
+broker_memory_usage
+ |
+Memory Usage
+ |
+Broker
+ |
+Alarm threshold: original value > 90%
+Number of consecutive periods: 3
+Alarm severity: critical
+ |
+Memory usage of the Kafka VM.
+ |
+Modify the number of brokers. For details, see Modifying Instance Specifications.
+ |
+
+current_partitions
+ |
+Partitions
+ |
+Instance
+ |
+Alarm threshold: original value > 90% of the maximum allowed number of partitions. The partition limit varies depending on instance specifications. For details, see Specifications.
+Number of consecutive periods: 1
+Alarm severity: major
+ |
+Number of used partitions in the instance.
+ |
+If new topics are required, modify the number of brokers, or split the service to multiple instances. For details about how to modify the number of brokers, see Modifying Instance Specifications.
+ |
+
+broker_cpu_usage
+ |
+CPU Usage
+ |
+Broker
+ |
+Alarm threshold: original value > 90%
+Number of consecutive periods: 3
+Alarm severity: major
+ |
+CPU usage of the Kafka VM.
+ |
+Check whether the metric has been approaching or exceeding the alarm threshold for a long time. If yes, modify the number of brokers. For details, see Modifying Instance Specifications.
+ |
+
+group_msgs
+ |
+Accumulated Messages
+ |
+Instance
+ |
+Alarm threshold: original value > 90% of the upper limit. The upper limit is customized.
+Number of consecutive periods: 1
+Alarm severity: major
+ |
+Total number of accumulated messages in all consumer groups of the instance
+ |
+Delete idle consumer groups, if any. You can also accelerate message retrieval, for example, by increasing the number of consumers.
+ |
+
+topic_messages_remained
+ |
+Topic Available Messages
+ |
+Consumer group
+ |
+Alarm threshold: original value > 90% of the upper limit. The upper limit is customized.
+Number of consecutive periods: 1
+Alarm severity: major
+ |
+Number of remaining messages that can be retrieved from the specified topic in the consumer group.
+ |
+Check whether the consumer code logic is correct, for example, by checking whether the consumer stops consuming messages due to an exception. You can also accelerate message retrieval, for example, by adding topic consumers. Ensure that the number of partitions is greater than or equal to the number of consumers.
+ |
+
+
+
+
+ Configuring a Kafka Alarm Rule- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- In the row containing the desired instance, click View Metric.
You are redirected to the Cloud Eye console page displaying metrics of the selected instance.
+ - Hover the mouse pointer over a metric and click
 to create an alarm rule for the metric. - Specify the alarm details.
For more information about creating alarm rules, see Creating an Alarm Rule.
+- Set the alarm name and description.
- Specify the alarm policy and alarm severity.
As shown in the following figure, if the original disk capacity usage is equal to or higher than 85% for three consecutive periods, an alarm is generated. If the alarm is not handled on time, an alarm notification is sent.
+Figure 1 Setting the alarm policy and alarm severity
+ - Set the alarm notification configurations. If you enable Alarm Notification, specify Notification Window, Notification Object, and Trigger Condition.
- Click Create.
+
+
+
+
+
diff --git a/docs/dms/umn/kafka-ug-180604011.html b/docs/dms/umn/kafka-ug-180604011.html
index 6b0ed21d..e04ae183 100644
--- a/docs/dms/umn/kafka-ug-180604011.html
+++ b/docs/dms/umn/kafka-ug-180604011.html
@@ -4,23 +4,23 @@
diff --git a/docs/dms/umn/kafka-ug-180604012.html b/docs/dms/umn/kafka-ug-180604012.html
index 1e98d929..a88004fd 100644
--- a/docs/dms/umn/kafka-ug-180604012.html
+++ b/docs/dms/umn/kafka-ug-180604012.html
@@ -1,139 +1,139 @@
- Preparing Required Resources
- OverviewBefore creating a Kafka instance, ensure the availability of resources, including a virtual private cloud (VPC), subnet, security group, and security group rules. Each Kafka instance is deployed in a VPC and bound to a specific subnet and security group. In this way, Kafka provides an isolated virtual network environment and security protection policies that you can easily configure and manage.
- To access a Kafka instance over a public network, prepare an elastic IP address (EIP) in advance.
- To encrypt the disk, prepare a KMS key in advance.
-
- Required ResourcesTable 1 lists the resources required by a Kafka instance.
+ Kafka Network Connection Conditions
+ A client can connect to a Kafka instance in public or private networks. Notes before using a private network:
+ - By default, a client and a Kafka instance are interconnected when they are deployed in a VPC.
- If they are not, you need to interconnect them because of isolation among VPCs.
+ Table 1 lists how a client can connect to a Kafka instance.
- Table 1 Kafka resourcesResource
+Table 1 Connection modesMode
|
-Requirement
+ | How To Do
|
-Operations
+ | Reference
|
|---|
-VPC and subnet
+ | Public access
|
-Different Kafka instances can use the same or different VPCs and subnets based on site requirements. Note the following when creating a VPC and a subnet:
-- The VPC must be created in the same region as the Kafka instance.
- Use the default settings when creating a VPC and subnet.
+ | Enable public access on the Kafka console and configure elastic IPs (EIPs). The client can connect to the Kafka instance through EIPs.
|
-For details about how to create a VPC and subnet, see the Virtual Private Cloud User Guide.
+ | Configuring Kafka Public Access
|
-Security group
+ | Configure port mapping using DNAT. The client can connect to the Kafka instance in a public network.
|
-Different Kafka instances can use the same or different security groups. Note the following when creating a security group:
-- To use Kafka instances, add the security group rules described in Table 2. Other rules can be added based on site requirements.
NOTE: After a security group is created, its default inbound rule allows communication among ECSs within the security group and its default outbound rule allows all outbound traffic. In this case, you can access a Kafka instance within a VPC, and do not need to add rules according to Table 2.
+ Accessing Kafka in a Public Network Using DNAT
+ |
+
|
+Private access
+ |
+A client and a Kafka instance are interconnected when they are deployed in a VPC.
+ |
+-
+ |
+
+When a client and a Kafka instance are deployed in different VPCs of the same region, connect the client and the Kafka instance across VPCs using a VPC endpoint.
+ |
+Accessing Kafka Using a VPC Endpoint Across VPCs
+ |
+
+When a client and a Kafka instance are deployed in different VPCs of the same region, interconnect two VPCs using a VPC peering connection.
+ |
+VPC Peering Connection
+ |
+
+
+
+ Before connecting a client to a Kafka instance, allow accesses for the following security groups.
+ After a security group is created, its default inbound rule allows communication among ECSs within the security group and its default outbound rule allows all outbound traffic. In this case, you can access a Kafka instance within a VPC, and do not need to add rules according to Table 2.
-
-
- | For details about how to create a security group and configure security group rules, see the Virtual Private Cloud User Guide.
- |
-
|---|
-EIP
- |
-Note the following when creating EIPs:
-- The EIPs must be created in the same region as the Kafka instance.
- The number of EIPs must be the same as the number of Kafka instance brokers.
- |
-For details about how to create an EIP, see "Assigning an EIP" in Elastic IP User Guide.
- |
-
-KMS key
- |
-To encrypt the disk for a Kafka instance, prepare a KMS key in advance.
-The KMS key must be created in the same region as the Kafka instance.
- |
-For details about how to create a KMS key, see "Creating a Key" in the Key Management Service User Guide.
- |
-
-
-
-
- Table 2 Security group rulesDirection
+Table 2 Security group rulesDirection
|
-Protocol
+ | Protocol
|
-Port
+ | Port
|
-Source
+ | Source
|
-Description
+ | Description
|
|---|
-Inbound
+ | Inbound
|
-TCP
+ | TCP
|
-9094
+ | 9094
|
-0.0.0.0/0
+ | 0.0.0.0/0
|
-Access a Kafka instance through the public network (without SSL encryption).
+ | Accessing a Kafka instance over a public network (in plaintext)
|
-Inbound
+ | Inbound
|
-TCP
+ | TCP
|
-9092
+ | 9092
|
-0.0.0.0/0
+ | 0.0.0.0/0
|
-Access a Kafka instance within a VPC (without SSL encryption).
+ | - Accessing a Kafka instance over a private network within a VPC (in plaintext)
- Accessing a Kafka instance using a peering connection across VPCs (in plaintext)
|
-Inbound
+ | Inbound
|
-TCP
+ | TCP
|
-9095
+ | 9095
|
-0.0.0.0/0
+ | 0.0.0.0/0
|
-Access a Kafka instance through the public network (with SSL encryption).
+ | Accessing a Kafka instance over a public network (in ciphertext)
|
-Inbound
+ | Inbound
|
-TCP
+ | TCP
|
-9093
+ | 9093
|
-0.0.0.0/0
+ | 0.0.0.0/0
|
-Access a Kafka instance within a VPC (with SSL encryption).
+ | - Accessing a Kafka instance over a private network within a VPC (in ciphertext)
- Accessing a Kafka instance using a peering connection across VPCs (in ciphertext)
|
-Inbound
+ | Inbound
|
-TCP
+ | TCP
|
-9011
+ | 9011
|
-198.19.128.0/17
+ | 198.19.128.0/17
|
-Access a Kafka instance using VPC Endpoint (VPCEP).
+ | Accessing a Kafka instance using a VPC endpoint across VPCs (in cipher- or plaintext)
|
-Inbound
+ | Inbound
|
-TCP
+ | TCP
|
-9011
+ | 9011
|
-0.0.0.0/0
+ | 0.0.0.0/0
|
-Access a Kafka instance using DNAT.
+ | Accessing a Kafka instance using DNAT (in cipher- or plaintext)
|
Creating a Kafka Instance
+ScenarioKafka instances are tenant-exclusive, and physically isolated in deployment. You can customize the computing capabilities and storage space of a Kafka instance as required.
+
+ PrerequisitesBefore creating a Kafka instance, prepare the resources listed in Table 1.
+
+ Table 1 Kafka resourcesResource
+ |
+Requirement
+ |
+Operations
+ |
+
|---|
+
+VPC and subnet
+ |
+You need to configure a VPC and subnet for the Kafka instance as required. You can use the current account's existing VPC and subnet, or create new ones.
+Note: VPCs must be created in the same region as the Kafka instance.
+ |
+For details on how to create a VPC and a subnet, see Creating a VPC. If you need to create and use a new subnet in an existing VPC, see Creating a Subnet for the VPC.
+ |
+
+Security group
+ |
+Different Kafka instances can use the same or different security groups.
+Before accessing a Kafka instance, configure security groups based on the access mode. For details, see Table 2.
+ |
+For details on how to create a security group, see Creating a Security Group. For details on how to add rules to a security group, see Adding a Security Group Rule.
+ |
+
+EIP
+ |
+To create a Kafka instance and access it over a public network, create EIPs in advance.
+Note the following when creating EIPs:
+- The EIPs must be created in the same region as the Kafka instance.
- The number of EIPs must be the same as the number of Kafka instance brokers.
- The Kafka console cannot identify IPv6 EIPs.
+ |
+For details about how to create an EIP, see Assigning an EIP.
+ |
+
+KMS key
+ |
+To encrypt the disk for a Kafka instance, prepare a KMS key.
+The KMS key must be created in the same region as the Kafka instance.
+ |
+For details about how to create a KMS key, see Creating a Key.
+ |
+
+
+
+
+
+ Creating a Kafka Instance- Log in to the console.
- Click in the upper left corner to select a region.
Select a region near you to ensure the lowest latency possible.
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click Create Instance in the upper right corner of the page.
- Specify Region, Project, and AZ.
Select one AZ or at least three AZs.
+ - Enter an Instance Name.
You can customize a name that complies with the rules: 4–64 characters; starts with a letter; can contain only letters, digits, hyphens (-), and underscores (_).
+ - Select an Enterprise Project.
This parameter is for enterprise users. An enterprise project manages cloud resources. The enterprise project management service unifies cloud resources in projects, and resources and members in a project. The default project is default.
+ - Configure the following instance parameters:
Specifications: Select Cluster or Single-node.
+- If you select Cluster, specify the version, broker flavor and quantity, disk type, and storage space to be supported by the cluster Kafka instance as required. Cluster instances support Kafka versions 2.3.0, 2.7, and 3.x.
- Single-node: Create a Kafka 2.7 instance with one broker. For details about single-node instances, see Comparing Single-node and Cluster Kafka Instances.
+If you select Cluster, specify the version, broker flavor and quantity, disk type, and storage space to be supported by the Kafka instance as required.
+- Version: Cluster instances support Kafka 2.3.0, 2.7, and 3.x. The version cannot be changed once the instance is created.
- CPU Architecture: The x86 architecture is supported.
- Broker Flavor: Select a broker flavor that best fit your needs.
Maximum number of partitions per broker x Number of brokers = Maximum number of partitions of an instance. If the total number of partitions of all topics exceeds the upper limit of partitions, topic creation fails.
+ - For Brokers, specify the broker quantity. A 3.x instance can only have three brokers (not changeable).
- Storage space per broker: Disk type and size for storing the instance data.
The disk type cannot be changed once the Kafka instance is created.
+The storage space is consumed by message replicas, logs, and metadata. Specify the storage space based on the expected service message size, the number of replicas, and the reserved disk space. Each Kafka broker reserves 33 GB disk space for storing logs and metadata.
+Disks are formatted when an instance is created. As a result, the actual available disk space is 93% to 95% of the total disk space.
+The disk supports high I/O and ultra-high I/O types. For more information, see Disk Types and Performance.
+Each broker of a 3.x instance can only have 100 GB storage space (not changeable).
+ - Disk Encryption: Specify whether to enable disk encryption.
Enabling disk encryption improves data security, but slows down disk read/write. Disk encryption depends on Key Management Service (KMS). If you enable disk encryption, select a KMS key. This parameter cannot be modified once the Kafka instance is created.
+ - Capacity Threshold Policy: Policy used when the disk usage reaches the threshold. The capacity threshold is 95%.
- Automatically delete: Messages can be created and retrieved, but 10% of the earliest messages will be deleted to ensure sufficient disk space. This policy is suitable for scenarios where no service interruption can be tolerated. Data may be lost.
- Stop production: New messages cannot be created, but existing messages can still be retrieved. This policy is suitable for scenarios where no data loss can be tolerated.
+
+If you select Single-node, a v2.7 instance with one broker will be created.
+- Version: Kafka version, which can only be 2.7.
- CPU Architecture: The x86 architecture is supported.
- Broker Flavor: Select a broker flavor that best fit your needs.
- Brokers: The instance can have only one broker.
- Storage space per broker: Disk type and size for storing the instance data.
The disk type cannot be changed once the Kafka instance is created.
+The storage space is consumed by message replicas, logs, and metadata. Specify the storage space based on the expected service message size, the number of replicas, and the reserved disk space. Each Kafka broker reserves 33 GB disk space for storing logs and metadata.
+Disks are formatted when an instance is created. As a result, the actual available disk space is 93% to 95% of the total disk space.
+The disk supports high I/O and ultra-high I/O types. For more information, see Disk Types and Performance.
+ - Disk Encryption: Specify whether to enable disk encryption.
Enabling disk encryption improves data security, but slows down disk read/write. Disk encryption depends on Key Management Service (KMS). If you enable disk encryption, select a KMS key. This parameter cannot be modified once the Kafka instance is created.
+ - Capacity Threshold Policy: Policy used when the disk usage reaches the threshold. The capacity threshold is 95%.
- Automatically delete: Messages can be created and retrieved, but 10% of the earliest messages will be deleted to ensure sufficient disk space. This policy is suitable for scenarios where no service interruption can be tolerated. Data may be lost.
- Stop production: New messages cannot be created, but existing messages can still be retrieved. This policy is suitable for scenarios where no data loss can be tolerated.
+
+Figure 1 Single-node instances
+ - Configure the instance network parameters.
- Select the created VPC and subnet from the VPC drop-down list.
A VPC provides an isolated virtual network for your Kafka instances. You can configure and manage the network as required.
+ After the Kafka instance is created, its VPC and subnet cannot be changed.
+
+ - Select a security group.
A security group is a set of rules for accessing a Kafka instance. You can click Manage Security Group to view or create security groups on the network console.
+Before accessing a Kafka instance on the client, configure security group rules based on the access mode. For details about security group rules, see Table 2.
+
+ - Configure the instance access mode.
+ Table 2 Instance access modesPublic or Private Network
+ |
+Plaintext or Ciphertext
+ |
+Description
+ |
+
|---|
+
+Private Network Access
+
+
+ |
+Plaintext Access
+ |
+Clients connect to the Kafka instance without SASL authentication.
+Once enabled, private network access cannot be disabled. Enable plaintext or ciphertext access, or both.
+ |
+
+Ciphertext Access
+ |
+Clients connect to the Kafka instance with SASL authentication.
+Once enabled, private network access cannot be disabled. Enable plaintext or ciphertext access, or both. To disable ciphertext access, contact customer service.
+If you enable Ciphertext Access, specify a security protocol, SASL/PLAIN, username, and password.
+After an instance is created, disabling and re-enabling Ciphertext Access do not affect users.
+ |
+
+Public Network Access
+
+
+ |
+Plaintext Access
+ |
+Clients connect to the Kafka instance without SASL authentication.
+Enable or disable plaintext access, and configure addresses for public network access.
+ |
+
+Ciphertext Access
+ |
+Clients connect to the Kafka instance with SASL authentication.
+Enable or disable ciphertext access, and configure addresses for public network access.
+If you enable Ciphertext Access, specify a security protocol, SASL/PLAIN, username, and password.
+After an instance is created, disabling and re-enabling Ciphertext Access do not affect users.
+ |
+
+Public IP Addresses
+ |
+Select the number of public IP addresses as required.
+If EIPs are insufficient, click Create Elastic IP to create EIPs. Then, return to the Kafka console and click  next to Public IP Address to refresh the public IP address list.
+Kafka instances only support IPv4 EIPs.
+ |
+
+
+
+ Ciphertext access is unavailable for single-node instances.
+
+The security protocol, SASL/PLAIN mechanism, username, and password are described as follows.
+
+Table 3 Ciphertext access parametersParameter
+ |
+Value
+ |
+Description
+ |
+
|---|
+
+Security Protocol
+
+ |
+SASL_SSL
+ |
+SASL is used for authentication. Data is encrypted with SSL certificates for high-security transmission.
+SCRAM-SHA-512 is enabled by default. To use PLAIN, enable SASL/PLAIN.
+What are SCRAM-SHA-512 and PLAIN mechanisms?
+- SCRAM-SHA-512: uses the hash algorithm to generate credentials for usernames and passwords to verify identities. SCRAM-SHA-512 is more secure than PLAIN.
- PLAIN: a simple username and password verification mechanism.
+ |
+
+SASL_PLAINTEXT
+ |
+SASL is used for authentication. Data is transmitted in plaintext for high performance.
+SCRAM-SHA-512 is enabled by default. To use PLAIN, enable SASL/PLAIN. SCRAM-SHA-512 authentication is recommended for plaintext transmission.
+ |
+
+SASL/PLAIN
+ |
+-
+ |
+- If SASL/PLAIN is disabled, the SCRAM-SHA-512 mechanism is used for username and password authentication.
- If SASL/PLAIN is enabled, both the SCRAM-SHA-512 and PLAIN mechanisms are supported. You can select either of them as required.
+The SASL/PLAIN setting cannot be changed once ciphertext access is enabled.
+ |
+
+Username and Password
+ |
+-
+ |
+Username and password used by the client to connect to the Kafka instance.
+A username should contain 4 to 64 characters, start with a letter, and contain only letters, digits, hyphens (-), and underscores (_).
+A password must meet the following requirements:
+- Contains 8 to 32 characters.
- Contains at least three types of the following characters: uppercase letters, lowercase letters, digits, and special characters `~! @#$ %^&*()-_=+\|[{}];:'",<.>? and spaces, and cannot start with a hyphen (-).
- Cannot be the username spelled forwards or backwards.
+The username cannot be changed once ciphertext access is enabled.
+ |
+
+
+
+ - Click Advanced Settings to configure more parameters.
- Configure Automatic Topic Creation.
This setting is disabled by default. You can enable or disable it as required.
+If this option is enabled, a topic will be automatically created when a message is produced in or consumed from a topic that does not exist. By default, the topic has parameters listed in Table 4.
+After you change the value of the log.retention.hours, default.replication.factor, or num.partitions parameter, the value will be used in later topics that are automatically created. For example, assume that num.partitions is changed to 5, an automatically created topic has parameters listed in Table 4.
+
+Table 4 Topic parametersParameter
+ |
+Default Value
+ |
+Modified Value
+ |
+
|---|
+
+Partitions
+ |
+3
+ |
+5
+ |
+
+Replicas
+ |
+3
+ |
+3
+ |
+
+Aging Time (h)
+ |
+72
+ |
+72
+ |
+
+Synchronous Replication
+ |
+Disabled
+ |
+Disabled
+ |
+
+Synchronous Flushing
+ |
+Disabled
+ |
+Disabled
+ |
+
+Message Timestamp
+ |
+CreateTime
+ |
+CreateTime
+ |
+
+Max. Message Size (bytes)
+ |
+10,485,760
+ |
+10,485,760
+ |
+
+
+
+ - Specify Tags.
Tags are used to identify cloud resources. When you have multiple cloud resources of the same type, you can use tags to classify them based on usage, owner, or environment.
+- If you have predefined tags, select a predefined pair of tag key and value. You can click View predefined tags to go to the Tag Management Service (TMS) console and view or create tags.
- You can also create new tags by specifying Tag key and Tag value.
+Up to 20 tags can be added to each Kafka instance. For details about the requirements on tags, see Configuring Kafka Instance Tags.
+ - Enter a Description of the instance for 0–1024 characters.
+ - Click Create.
- Confirm the instance information, and click Submit.
- Return to the instance list and check whether the Kafka instance has been created.
It takes 3 to 15 minutes to create an instance. During this period, the instance status is Creating.
+- If the instance is created successfully, its status changes to Running.
- If the instance is in the Creation failed state, delete it by referring to Deleting Kafka Instances. Then create a new one. If the instance creation fails again, contact customer service.
Instances that fail to be created do not occupy other resources.
+
+
+
+
+ Viewing an Instance
-ScenarioView detailed information about a Kafka instance on the DMS console, for example, the IP addresses and port numbers for accessing the instance.
-
- Procedure- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+ Viewing and Modifying Basic Information of a Kafka Instance
+ ScenarioThis section describes how to view the details, and modify the basic information of a Kafka instance on the console.
+ After creating a Kafka instance, you can modify some parameters of it as required, including the instance name, description, security group, and capacity threshold policy.
+ Single-node instances do not support reconfiguration of private network access.
- - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Search for a Kafka instance by tag, status, name, ID, or connection address. Table 1 describes the various possible statuses of a Kafka instance.
- Table 1 Kafka instance status descriptionStatus
+
+PrerequisiteYou can modify basic information of a Kafka instance when the instance is in the Running state.
+ Viewing Kafka Instance Details- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Search for a Kafka instance by specifying filters. You can filter instances by tag, status, name, version, flavor, used/available storage space, maximum partitions, billing mode, and enterprise project.Only enterprise users can filter instances by enterprise projects. For Kafka instance statuses, see Table 1.
+ Table 1 Kafka instance status descriptionStatus
|
-Description
+ | Description
|
|---|
-Creating
+ | Creating
|
-The instance is being created.
+ | The instance is being created.
|
-Running
+ | Creation failed
|
-The instance is running properly.
+ | The instance failed to be created.
+ |
+
+Running
+ |
+The instance is running properly.
Only instances in the Running state can provide services.
|
-Faulty
+ | Faulty
|
-The instance is not running properly.
+ | The instance is not running properly.
|
-Starting
+ | Restarting
|
-The status between Frozen and Running.
+ | The instance is being restarted.
|
-Restarting
+ | Changing
|
-The instance is being restarted.
+ | The instance specifications or public access configurations are being modified.
|
-Changing
+ | Change failed
|
-The instance specifications or public access configurations are being modified.
- |
-
-Change failed
- |
-The instance specifications or public access configurations failed to be modified.
- |
-
-Frozen
- |
-The instance is frozen.
- |
-
-Freezing
- |
-The status between Running and Frozen.
- |
-
-Upgrading
- |
-The instance is being upgraded.
- |
-
-Rolling back
- |
-The instance is being rolled back.
+ | The instance specifications or public access configurations failed to be modified.
+You cannot restart, delete, or modify an instance in the Change failed state. Contact customer service.
|
- Click the name of the desired Kafka instance and view detailed information about the instance on the Basic Information tab page.
+ - Click the name of the desired Kafka instance and view detailed information about the instance on the Basic Information tab page.
Table 2 describes the parameters for connecting to a Kafka instance. For details about other parameters, see the Basic Information tab page of the Kafka instance on the console.
+
+Table 2 Connection parametersSection
+ |
+Parameter
+ |
+Sub-Parameter
+ |
+Description
+ |
+
|---|
+
+Connection
+
+
+
+
+
+ |
+Username
+ |
+-
+ |
+Username for accessing the instance with ciphertext access enabled.
+ |
+
+Private Network Access
+
+
+
+
+ |
+Plaintext Access
+ |
+Indicates whether plaintext access is enabled.
+ |
+
+Address (Private Network, Plaintext)
+ |
+This parameter is displayed only after you enable Plaintext Access.
+ |
+
+Ciphertext Access
+ |
+Indicates whether ciphertext access is enabled.
+This function is unavailable for single-node instances.
+ |
+
+Address (Private Network, Ciphertext)
+ |
+This parameter is displayed only after you enable Ciphertext Access.
+ |
+
+Security Protocol
+ |
+This parameter is displayed only after you enable Ciphertext Access.
+ |
+
+Public Network Access
+
+
+
+
+
+ |
+Toggle switch
+ |
+Indicates whether public access has been enabled.
+ |
+
+Plaintext Access
+ |
+This parameter is displayed only when Public Access is enabled.
+Indicates whether plaintext access is enabled.
+ |
+
+Address (Public Network, Plaintext)
+ |
+This parameter is displayed only after you enable Plaintext Access.
+ |
+
+Ciphertext Access
+ |
+This parameter is displayed only when Public Access is enabled.
+Indicates whether ciphertext access is enabled.
+This function is unavailable for single-node instances.
+ |
+
+Address (Public Network, Ciphertext)
+ |
+This parameter is displayed only after you enable Ciphertext Access.
+ |
+
+Security Protocol
+ |
+This parameter is displayed only after you enable Ciphertext Access.
+ |
+
+SASL Mechanism
+ |
+-
+ |
+This parameter is displayed only after you enable Ciphertext Access.
+ |
+
+SSL Certificate
+ |
+-
+ |
+This parameter is displayed only when SASL_SSL is enabled.
+Click Download to download the SSL certificate for accessing the instance.
+ |
+
+
+
+
+ Modifying Basic Information of a Kafka Instance- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view its details.
- Modify the following parameters if needed:
+ Table 3 Modifiable Kafka parametersParameter
+ |
+How to Modify
+ |
+Result
+ |
+
|---|
+
+Instance Name
+ |
+Click  , enter a new name, and click  .
+Naming rules: 4–64 characters; starts with a letter; can contain only letters, digits, hyphens (-), and underscores (_).
+ |
+The modification result is displayed in the upper right corner of the page.
+ |
+
+Enterprise Project
+ |
+Click , select a new enterprise project from the drop-down list, and click .
+Only for enterprise users. Modifying this parameter does not restart the instance.
+ |
+The modification result is displayed in the upper right corner of the page.
+ |
+
+Description
+ |
+Click , enter a new description, and click .
+0 to 1024 characters.
+ |
+The modification result is displayed in the upper right corner of the page.
+ |
+
+Security Group
+ |
+Click , select a new security group from the drop-down list, and click .
+Modifying this parameter does not restart the instance.
+ |
+The modification result is displayed in the upper right corner of the page.
+ |
+
+Private Network Access
+ |
+See Configuring Plaintext or Ciphertext Access to Kafka Instances.
+ |
+You will be redirected to the Background Tasks page, which displays the modification progress and result.
+ |
+
+Public Access
+ |
+See Configuring Kafka Public Access.
+ |
+You will be redirected to the Background Tasks page, which displays the modification progress and result.
+ |
+
+Capacity Threshold Policy
+ |
+Click the desired policy. In the displayed Confirm dialog box, click OK.
+Modifying this parameter does not restart the instance.
+ |
+You will be redirected to the Background Tasks page, which displays the modification progress and result.
+ |
+
+Automatic Topic Creation
+ |
+Enable/Disable this Automatic Topic Creation. In the displayed Confirm dialog box, click OK.
+Changing this option may restart the instance.
+ |
+You will be redirected to the Background Tasks page, which displays the modification progress and result.
+ |
+
+Cross-VPC Access
+ |
+See Accessing Kafka Using a VPC Endpoint Across VPCs and Accessing Kafka in a Public Network Using DNAT.
+ |
+The modification result is displayed in the upper right corner of the page.
+ |
+
+
+
+
diff --git a/docs/dms/umn/kafka-ug-180604015.html b/docs/dms/umn/kafka-ug-180604015.html
index 20640e21..ee836265 100644
--- a/docs/dms/umn/kafka-ug-180604015.html
+++ b/docs/dms/umn/kafka-ug-180604015.html
@@ -1,16 +1,16 @@
- Restarting an Instance
+ Restarting a Kafka Instance
ScenarioRestart one or more Kafka instances at a time on the DMS console.
When a Kafka instance is being restarted, message retrieval and creation requests of clients will be rejected.
PrerequisitesThe status of the Kafka instance you want to restart is either Running or Faulty.
- Procedure- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+ Restarting a Kafka Instance- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
- - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Restart Kafka instances using one of the following methods:
- Select one or more Kafka instances and click Restart in the upper left corner.
- In the row containing the desired instance, click Restart.
- - In the Restart Instance dialog box, click Yes to restart the Kafka instance.
It takes 3 to 15 minutes to restart a Kafka instance. After the instance is successfully restarted, its status should be Running.
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Restart Kafka instances using one of the following methods:
- Select one or more Kafka instances and click Restart in the upper left corner.
- In the row containing the desired instance, click Restart.
- Click the desired Kafka instance to view its details. In the upper right corner, click Restart.
+ - In the Restart Instance dialog box, click Yes to restart the Kafka instance.
It takes 3 to 15 minutes to restart a Kafka instance. After the instance is successfully restarted, its status should be Running.
Restarting a Kafka instance only restarts the instance process and does not restart the VM where the instance is located.
diff --git a/docs/dms/umn/kafka-ug-180604016.html b/docs/dms/umn/kafka-ug-180604016.html
index 71495983..77f93ffe 100644
--- a/docs/dms/umn/kafka-ug-180604016.html
+++ b/docs/dms/umn/kafka-ug-180604016.html
@@ -1,25 +1,18 @@
- Deleting an Instance
- ScenarioOn the DMS console, you can delete one or more Kafka instances that have been created or failed to be created.
+ Deleting Kafka Instances
+ ScenarioDelete one or more Kafka instances at a time on the DMS console.
Deleting a Kafka instance will delete the data in the instance without any backup. Exercise caution when performing this operation.
PrerequisitesThe status of the Kafka instance you want to delete is Running or Faulty.
- Deleting Kafka Instances- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+ Deleting Kafka Instances- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
- - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Delete Kafka instances using one of the following methods:
- Select one or more Kafka instances and click Delete in the upper left corner.
- In the row containing the Kafka instance to be deleted, choose More > Delete.
- Kafka instances in the Creating, Starting, Changing, Change failed, or Restarting state cannot be deleted.
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Delete pay-per-use Kafka instances in either of the following ways:
- Select one or more Kafka instances and click Delete in the upper left corner.
- In the row containing the Kafka instance to be deleted, choose More > Delete.
- Click the desired Kafka instance to view its details. In the upper right corner, choose More > Delete.
+ Kafka instances in the Creating, Changing, Change failed, or Restarting state cannot be deleted.
- - In the Delete Instance dialog box, click Yes to delete the Kafka instance.
It takes 1 to 60 seconds to delete a Kafka instance.
-
-
- Deleting Kafka Instances That Failed to Be Created- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
-
- - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- If there are Kafka instances that failed to be created, Instance Creation Failures and quantity information will be displayed.
Instances that fail to be created do not occupy other resources.
-
- - Click Instance Creation Failures or the icon or quantity next to it.
- Delete Kafka instances that failed to be created in either of the following ways:
- To delete all Kafka instances that failed to be created at once, click Clear Failed Instance.
- To delete a single Kafka instance that failed to be created, click Delete in the row containing the chosen Kafka instance.
+ - In the Delete Instance dialog box, enter DELETE and click OK to delete the Kafka instance.
It takes 1 to 60 seconds to delete a Kafka instance.
diff --git a/docs/dms/umn/kafka-ug-180604017.html b/docs/dms/umn/kafka-ug-180604017.html
deleted file mode 100644
index fe0359a0..00000000
--- a/docs/dms/umn/kafka-ug-180604017.html
+++ /dev/null
@@ -1,18 +0,0 @@
-
-
-Modifying the Information About an Instance
-After creating a Kafka instance, you can modify some parameters of the instance based on service requirements, including the instance name, description, security group, and capacity threshold policy.
- Procedure- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
-
- - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- Modify the following parameters if needed:
- Instance Name
- Description
- Security Group
- Public Access (For details about how to change the public access configuration, see Configuring Public Access.)
- Capacity Threshold Policy (Modifying this setting will not restart the instance.)
- Automatic Topic Creation (Modifying this setting will restart the instance.)
-After the parameters are modified, view the modification result in one of the following ways:
-- If Capacity Threshold Policy, Public Access, or Automatic Topic Creation has been modified, you will be redirected to the Background Tasks page, which displays the modification progress and result.
- If Instance Name, Description, or Security Group has been modified, the modification result will be displayed on the upper right corner of the page.
-
-
- Creating a Kafka Topic
+Topics store messages created by producers and subscribed by consumers. If automatic topic creation is not enabled during Kafka instance creation, you need to manually create topics. If automatic topic creation has been enabled for the instance, this operation is optional.
+ Automatic topic creation: A topic will be automatically created when a message is produced in or consumed from a topic that does not exist. By default, the topic has parameters listed in Table 1.
+ After you change the value of the log.retention.hours, default.replication.factor, or num.partitions parameter, the value will be used in later topics that are automatically created. For example, assume that num.partitions is changed to 5, an automatically created topic has parameters listed in Table 1.
+
+ Table 1 Topic parametersParameter
+ |
+Default Value
+ |
+Modified Value
+ |
+
|---|
+
+Partitions
+ |
+3
+ |
+5
+ |
+
+Replicas
+ |
+3
+ |
+3
+ |
+
+Aging Time (h)
+ |
+72
+ |
+72
+ |
+
+Synchronous Replication
+ |
+Disabled
+ |
+Disabled
+ |
+
+Synchronous Flushing
+ |
+Disabled
+ |
+Disabled
+ |
+
+Message Timestamp
+ |
+CreateTime
+ |
+CreateTime
+ |
+
+Max. Message Size (bytes)
+ |
+10,485,760
+ |
+10,485,760
+ |
+
+
+
+
+ Methods that can be used to manually create a topic:
+
+ Constraints- The total number of partitions in topics is limited. When the partition quantity limit is reached, you can no longer create topics. The total number of partitions varies by instance specifications. For details, see Specifications.
- If an instance node is faulty, an internal service error may be reported when you query messages in a topic with only one replica. Therefore, you are not advised using a topic with only one replica.
+
+ Method 1: Creating a Topic on the Console- Log in to the console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
+
+ - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- In the navigation pane, choose Topics. Then click Create Topic.
Figure 1 Creating a topic
+ - Specify the topic parameters listed in the following table.
+ Table 2 Topic parametersParameter
+ |
+Description
+ |
+
|---|
+
+Topic Name
+ |
+Customize a name that contains 3 to 200 characters, starts with a letter or underscore (_), and contains only letters, digits, periods (.), hyphens (-), and underscores (_).
+The name must be different from preset topics:
+- _consumer_offsets
- _transaction_state
- _trace
- _connect-status
- _connect-configs
- _connect-offsets
+Once the topic is created, you cannot modify its name.
+ |
+
+Partitions
+ |
+Number of partitions in the topic.
+If the number of partitions is the same as that of consumers, the larger the partitions, the higher the consumption concurrency.
+If this parameter is set to 1, messages will be retrieved in the FIFO order.
+Value range: 1 to 200
+Default value: 3
+ |
+
+Replicas
+ |
+A higher number of replicas delivers higher reliability. Data is automatically backed up on each replica. When one Kafka broker becomes faulty, data is still available on other brokers.
+If this parameter is set to 1, only one set of data is available.
+Value range: 1 to number of brokers
+ NOTE: If an instance node is faulty, an internal service error may be reported when you query messages in a topic with only one replica. Therefore, you are not advised using a topic with only one replica.
+
+ |
+
+Aging Time (h)
+ |
+The period that messages are retained for. Consumers must retrieve messages before this period ends. Otherwise, the messages will be deleted and can no longer be consumed.
+Value range: 1–720
+Default value: 72
+ |
+
+Synchronous Replication
+ |
+A message is returned to the client only after the message creation request has been received and the message has been acknowledged by all replicas.
+After enabling this, set the parameter acks to all or -1 in the configuration file or production code on the producer client.
+If there is only one replica, synchronous replication cannot be enabled.
+ |
+
+Synchronous Flushing
+ |
+- Enabled: A message is immediately flushed to disk once it is created, bringing higher reliability.
- Disabled: A message is stored in the memory instead of being immediately flushed to disk once created.
+ |
+
+Message Timestamp
+ |
+Timestamp type of a message. Options:
+- CreateTime: time when the producer created the message.
- LogAppendTime: time when the broker appended the message to the log.
+ |
+
+Max. Message Size
+ |
+Maximum batch processing size allowed by Kafka. If message compression is enabled in the client configuration file or code of producers, this parameter indicates the size after compression.
+If this is increased and there are consumers older than 0.10.2, the consumers' fetch size must also be increased so that they can fetch record batches this large.
+Value range: 0 to 10,485,760
+ |
+
+
+
+ - Click OK.
+
+ Method 2: Creating a Topic by Using Kafka CLIIf your client is v2.2 or later, you can use kafka-topics.sh to create topics and manage topic parameters.
+ - If a topic name starts with a special character, for example, an underscore (_) or a number sign (#), monitoring data cannot be displayed.
- For an instance with ciphertext access enabled, if allow.everyone.if.no.acl.found is set to false, topics cannot be created through the client.
+
+
+
+ Deleting a Topic
+Deleting a Kafka Topic
Delete a topic using either of the following methods:
-
+
Prerequisites- A Kafka instance has been created, and a topic has been created in this instance.
- The Kafka instance is in the Running state.
- Deleting a Topic on the Console- Log in to the management console.
- Click in the upper left corner to select a region.
Select the region where your Kafka instance is located.
-
- - Click Service List and choose Application > Distributed Message Service. The Kafka instance list is displayed.
- Click the desired Kafka instance to view the instance details.
- Click the Topics tab.
- Delete topics using either of the following methods:
- Select one or more topics and click Delete Topic in the upper left corner.
- In the row containing the topic you want to delete, click Delete.
- - In the Delete Topic dialog box that is displayed, click Yes to delete the topic.
+ ConstraintIf your Kafka instances are connected using Logstash, stop Logstash before deleting topics. Otherwise, services may crash.
- Deleting a Topic with the Kafka CLIIf your Kafka client version is later than 2.2, you can use kafka-topics.sh to delete topics.
-
diff --git a/docs/dms/umn/kafka-ug-180604020.html b/docs/dms/umn/kafka-ug-180604020.html
index f94200c3..f2216070 100644
--- a/docs/dms/umn/kafka-ug-180604020.html
+++ b/docs/dms/umn/kafka-ug-180604020.html
@@ -1,15 +1,15 @@
- Accessing a Kafka Instance Without SASL
- This section describes how to use an open-source Kafka client to access a Kafka instance if SASL access is not enabled for the instance. There are two scenarios. For cross-VPC access, see Cross-VPC Access to a Kafka Instance. For DNAT-based access, see Using DNAT to Access a Kafka Instance.
- For details on how to use Kafka clients in different languages, visit https://cwiki.apache.org/confluence/display/KAFKA/Clients.
-
|
|---|
|
|---|
|
|---|
|
|---|
|
|---|
|
|---|
|
|---|
|
|---|
|
