DataWorks:SAP HANA data source

Limits

• Data of views can be read during batch synchronization.

• SAP HANA data sources support only exclusive resource groups for Data Integration.

Data type mappings

Develop a data synchronization task

For information about the entry point for and the procedure of configuring a data synchronization task, see the following sections. For information about the parameter settings, view the infotip of each parameter on the configuration tab of the task.

Add a data source

Before you configure a data synchronization task to synchronize data from or to a specific data source, you must add the data source to DataWorks. For more information, see Add and manage data sources.

Configure a batch synchronization task to synchronize data of a single table

• For more information about the configuration procedure, see Configure a batch synchronization task by using the codeless UI and Configure a batch synchronization task by using the code editor.

• For information about all parameters that are configured and the code that is run when you use the code editor to configure a batch synchronization task, see Appendix: Code and parameters.

Appendix: Code and parameters

Appendix: Configure a batch synchronization task by using the code editor

If you use the code editor to configure a batch synchronization task, you must configure parameters for the Reader and Writer of the related data source based on the format requirements in the code editor. For more information about the format requirements, see Configure a batch synchronization task by using the code editor. The following information describes the configuration details of parameters for the Reader and Writer in the code editor.

Code for SAP HANA Reader

• Configure a synchronization task to read data from a table that is not sharded

  {
      "type":"job",
      "version":"2.0",// The version number. 
      "steps":[
          {
              "stepType":"saphana",// The plug-in name. 
              "parameter":{
                  "column":[// The names of the columns. 
                      "id"
                  ],
                  "connection":[
                      {   "querySql":["select a,b from join1 c join join2 d on c.id = d.id;"], // The SQL statement that is used to read data from the source table. 
                          "datasource":"",// The name of the data source. 
                          "table":[// The name of the source table. The table name must be enclosed in brackets []. 
                              "xxx"
                          ]
                      }
                  ],
                  "where":"",// The WHERE clause. 
                  "splitPk":"", // The shard key. 
                  "encoding":"UTF-8"// The encoding format. 
              },
              "name":"Reader",
              "category":"reader"
          },
          {
              "stepType":"stream",
              "parameter":{},
              "name":"Writer",
              "category":"writer"
          }
      ],
      "setting":{
          "errorLimit":{
              "record":"0"// The maximum number of dirty data records allowed. 
          },
          "speed":{
              "throttle":true,// Specifies whether to enable throttling. The value false indicates that throttling is disabled, and the value true indicates that throttling is enabled. The mbps parameter takes effect only when the throttle parameter is set to true. 
              "concurrent":1, // The maximum number of parallel threads. 
              "mbps":"12"// The maximum transmission rate. Unit: MB/s. 
          }
      },
      "order":{
          "hops":[
              {
                  "from":"Reader",
                  "to":"Writer"
              }
          ]
      }
  }

• Configure a synchronization task to read data from SAP HANA tables in sharded databases

  Note

  When you configure a synchronization task to read data from SAP HANA tables in sharded databases, you can select multiple SAP HANA tables that have the same schema.

  {
      "type": "job",
      "version": "1.0",
      "configuration": {
          "reader": {
              "plugin": "saphana",
              "parameter": {
                  "connection": [
                      {
                          "table": [
                              "tbl1",
                              "tbl2",
                              "tbl3"
                          ],
                          "datasource": "datasourceName1"
                      },
                      {
                          "table": [
                              "tbl4",
                              "tbl5",
                              "tbl6"
                          ],
                          "datasource": "datasourceName2"
                      }
                  ],
                  "singleOrMulti": "multi",
                  "splitPk": "db_id",
                  "column": [
                      "id", "name", "age"
                  ],
                  "where": "1 < id and id < 100"
              }
          },
          "writer": {            
          }
      }
  }

Parameters in code for SAP HANA Reader

Code for RestAPI Writer

{
    "type":"job",
    "version":"2.0",// The version number. 
    "steps":[
        {
            "stepType":"stream",
            "parameter":{},
            "name":"Reader",
            "category":"reader"
        },
        {
            "stepType":"saphana",// The plug-in name. 
            "parameter":{
                "postSql":[],// The SQL statement that you want to execute after the synchronization task is run. 
                "datasource":"",// The name of the data source. 
                "column":[// The names of the columns. 
                    "id",
                    "value"
                ],
                "batchSize":1024,// The number of data records to write at a time. 
                "table":"",// The name of the table to which you want to write data. 
                "preSql":[
                     "delete from XXX;" // The SQL statement that you want to execute before the synchronization task is run. 
                   ]
            },
            "name":"Writer",
            "category":"writer"
        }
    ],
    "setting":{
        "errorLimit":{// The maximum number of dirty data records allowed. 
            "record":"0"
        },
        "speed":{
            "throttle":true,// Specifies whether to enable throttling. The value false indicates that throttling is disabled, and the value true indicates that throttling is enabled. The mbps parameter takes effect only when the throttle parameter is set to true. 
            "concurrent":1, // The maximum number of parallel threads. 
            "mbps":"12"// The maximum transmission rate. Unit: MB/s. 
        }
    },
    "order":{
        "hops":[
            {
                "from":"Reader",
                "to":"Writer"
            }
        ]
    }
}

Parameters in code for SAP HANA Writer

truncate table tablename