ApsaraMQ for Kafka:Manage a connector

Precautions

• You can modify connector configurations only for Function Compute sink connectors and MySQL source connectors.
• For information about how to manage a Data Lake Analytics (DLA) sink connector, see Create a DLA sink connector.
• You can suspend a connector only if the connector is in the Running state. You can enable a connector only if the connector is in the Suspended state.

View the configurations of a connector

After you create a connector, you can view the information about the connector, such as the basic information, source information, and destination information.

1. Log on to the ApsaraMQ for Kafka console.
2. In the Resource Distribution section of the Overview page, select the region where your instance is deployed.
3. In the left-side navigation pane, click Connectors.
4. On the Connectors page, select an instance to which your connector belongs from the Select Instance drop-down list. In the connector list, find the connector and click Details in the Actions column.
   On the page that appears, view the details of the connector in the Basic Information, Source Information, and Destination Information sections.

View log information about a connector

If an error occurs in a connector, you can view the log information about the connector in the ApsaraMQ for Kafka console to troubleshoot the error.

1. Log on to the ApsaraMQ for Kafka console.
2. In the Resource Distribution section of the Overview page, select the region where your instance is deployed.
3. In the left-side navigation pane, click Connectors.
4. On the Connectors page, select the instance to which your connector belongs from the Select Instance drop-down list. In the connector list, find the connector and choose More > View Logs in the Actions column.
   You can also click the name of the connector or click Details in the Actions column. On the details page that appears, click View Logs in the upper-right corner.

   Note The position of View Logs in the Actions column varies based on the status of the connector.
   In the dialog box that appears, view the log information about the connector.

Modify the configurations of a connector

After a Function Compute sink connector or a MySQL source connector is created, you can modify the configurations of the connector in the ApsaraMQ for Kafka console.

1. Log on to the ApsaraMQ for Kafka console.
2. In the Resource Distribution section of the Overview page, select the region where your instance is deployed.
3. In the left-side navigation pane, click Connectors.
4. On the Connectors page, select the instance to which your connector belongs from the Select Instance drop-down list. In the connector list, find the connector and choose More > Modify Configuration in the Actions column.
   You can also click the name of the connector or click Details in the Actions column. On the details page that appears, click Modify Configuration in the upper-right corner.
5. In the Modify Configuration panel, change the values of the parameters that are described in the following table based on your business requirements. Then click OK.
   • Modify the configurations of a Function Compute sink connector.

     Parameter	Description
     Consumer Thread Concurrency	The number of concurrent consumer threads that are used to synchronize data from the source topic. Default value: 6. Valid values: 1 2 3 6 12
     Failure Handling Policy	Specifies whether to retain the subscription to the partition in which an error occurs after the relevant message fails to be sent. Valid values: Continue Subscription: retains the subscription to the partition in which an error occurs and generates an error log entry. Stop Subscription: stops the subscription to the partition where an error occurs and generates an error log entry. Note For information about how to view log information, see Manage a connector. For information about how to troubleshoot errors based on error codes, see Error codes.
     Transmission Mode	The mode in which messages are sent. Valid values: Synchronous: The producer and the consumer must be running to get ready for data exchanges. Asynchronous: After the producer sends a message, the producer does not need to wait for a response from the broker and can continue processing other tasks.
     Data Size	The maximum number of messages that can be sent at a time. Valid values: 1 to 5000. Default value: 20.
     Retries	The maximum number of retries allowed after a message fails to be sent. Default value: 2. Valid values: 1 to 3. In specific cases, when a message fails to be sent, retries are not supported. For more information about error codes, see Error codes. The following content describes whether retries are supported when specific error codes are returned: For 4xx, retries are not supported except in the case when 429 is returned. For 5xx, retries are supported. Note The connector calls the InvokeFunction operation to send messages to Function Compute.

   • Modify the configurations of a MySQL source connector.

     Parameter

     Description

     Database Table

     The name of one or more ApsaraDB RDS for MySQL tables from which you want to synchronize data. Separate multiple table names with commas (,). Source tables and destination topics are related based on one-to-one mappings. Note If you add or modify a table, you must create the corresponding topic in the ApsaraMQ for Kafka console. The topic name must be in the <Topic prefix>_<Table name> format. Otherwise, an error occurs when you run the connector. For information about how to create a topic, see Step 1: Create a topic.

   After the configurations are modified, go to the Connectors page and find the connector. Click Details in the Actions column. On the Connector Details page, you can view the updated configurations.

Suspend or enable a connector

If you need to temporarily suspend a running connector or resume a suspended connector, you can suspend or enable the connector in the ApsaraMQ for Kafka console.

1. Log on to the ApsaraMQ for Kafka console.
2. In the Resource Distribution section of the Overview page, select the region where your instance is deployed.
3. In the left-side navigation pane, click Connectors.
4. On the Connectors page, select the instance to which your connector belongs from the Select Instance drop-down list. In the connector list, find the connector and suspend or enable the connector.
   • Suspend the connector: In the Actions column, choose More > Suspend. In the Note message, click OK.
   • Enable the connector: In the Actions column, click Enable. In the Note message, click OK.

   You can also click the name of the connector or click Details in the Actions column. On the details page that appears, click Suspend or Enable in the upper-right corner.

   On the Connectors page, find the connector you suspended or resumed. The value of the Status parameter for the connector becomes Suspended or Running.

   Note If you attempt to resume a MySQL source connector that remains suspended for a long period of time, the operation may fail. In this case, you can click Task Configurations in the Actions column. On the DataWorks page that appears, find the corresponding connector, and click Start. In the Start dialog box, select Reset Offset, configure the Start Time Point parameter, and then click OK.

Test a connector

After you create a connector, you can send a message to the connector from the ApsaraMQ for Kafka console to test the connector.

1. Log on to the ApsaraMQ for Kafka console.
2. In the Resource Distribution section of the Overview page, select the region where your instance is deployed.
3. In the left-side navigation pane, click Connectors.
4. On the Connectors page, select the instance to which your connector belongs from the Select Instance drop-down list. In the connector list, find the connector and click Test in the Actions column.
   You can also click the name of the connector or click Details in the Actions column. On the details page that appears, click Test in the upper-right corner.

   Note The position of Test in the Actions column varies based on the status of the connector.
5. In the Send Message panel, send a test message.
   1. In the Message Key field, enter the key of the test message, such as demo.
   2. In the Message Content field, enter the content of the test message, such as {"key": "test"}.
   3. Configure the Send to Specified Partition parameter to specify whether to send the test message to a specific partition.
      • If you want to send the test message to a specific partition, click Yes and enter the partition ID, such as 0, in the Partition ID field. For information about how to query partition IDs, see View partition status.
      • If you do not want to send the test message to a specific partition, click No.

View the consumption progress for a connector

After a connector is created and enters the Running state, you can view the consumption details of the consumer group that corresponds to the connector.Group The name of the consumer group is connect-{Connector name}.Group

1. Log on to the ApsaraMQ for Kafka console.
2. In the Resource Distribution section of the Overview page, select the region where your instance is deployed.
3. In the left-side navigation pane, click Connectors.
4. On the Connectors page, select the instance to which your connector belongs from the Select Instance drop-down list. In the connector list, find the connector and click Consumption Progress in the Actions column.
   You can also click the name of the connector or click Details in the Actions column. On the details page that appears, click Consumption Progress in the upper-right corner.
   In the panel that appears, you can view the consumption details of the consumer group that corresponds to the connector in the source topic.

Delete a connector

ApsaraMQ for Kafka limits the number of connectors for each instance. If you no longer need a connector, you can delete it in the ApsaraMQ for Kafka console.

1. Log on to the ApsaraMQ for Kafka console.
2. In the Resource Distribution section of the Overview page, select the region where your instance is deployed.
3. In the left-side navigation pane, click Connectors.
4. On the Connectors page, select the instance to which your connector belongs from the Select Instance drop-down list. In the connector list, find the connector and choose More > Delete in the Actions column.
   You can also click the name of the connector or click Details in the Actions column. On the details page that appears, click Delete in the upper-right corner.
5. In the Note message, click OK to delete the connector.