In specific IoT scenarios, devices that have limited resources and low specifications or devices that require high network throughput cannot use Thing Specification Language (TSL) models in the JSON format to communicate with IoT Platform. To resolve this issue, you can pass through raw data from devices to IoT Platform. IoT Platform provides the data parsing feature that converts data between device-specific formats and the JSON format based on the scripts that you submitted. You can use the feature before you process data.
Background information
The following figure shows how data flows between devices and servers.
In this example, a device that collects environmental data is used.
Connect a device to IoT Platform
Log on to the IoT Platform console IoT Platform console.
On the Overview page, click All environment. On the All environment tab, find the instance that you want to manage and click the instance ID or instance name.
In the left-side navigation pane, choose . On the Products page, click Create Product to create a product. In this example, a product named EnvSensor is created.
Set the Data Type parameter to Custom. Use the default values for other parameters. For more information, see Create a product.
After the product is created, click Define TSL to add TSL features. Then, publish the TSL model.
In this example, a sample TSL model is provided. You can use the TSL model to add multiple TSL features at a time. For more information, see Batch add TSL features.
In the left-side navigation pane, choose Devices > Devices. On the Devices page, click Device List. On the Device List tab, click Add Device to create a device for the EnvSensor product. In this example, a device named Esensor is created.
After the device is created, obtain information about the device certificate. The information includes the ProductKey, DeviceName, and DeviceSecret.
Configure a device and perform a test.
Use a Link SDK to simulate a device and submit messages to IoT Platform. In this example, Link SDK for Node.js is used. For more information about how to configure a device, see Connect the device to IoT Platform and submit data.
For more information about how to use Link SDKs, see Device connection by using a Link SDK
Sample code:
const mqtt = require('aliyun-iot-mqtt'); // Specify information about the device certificate. var options = { productKey: "g18lk****", deviceName: "Esensor", deviceSecret: "c39fe9dc32f97bfd753******b", host: "iot******.mqtt.iothub.aliyuncs.com" }; // Establish a connection. const client = mqtt.getAliyunIotMqttClient(options); // Listen to commands from IoT Platform. client.subscribe(`/${options.productKey}/${options.deviceName}/user/get`) client.on('message', function(topic, message) { console.log("topic " + topic) console.log("message " + message) }) setInterval(function() { // Submit environmental data. const payloadArray = new Buffer.from([0xaa,0x1f,0xc8,0x00,0x00,0x37,0x10,0xff,0x00,0x05,0xd7,0x6b,0x15,0x00,0x1c,0x01,0x34,0x00,0xad,0x04,0xff,0xff,0x04,0x00,0xff,0xff,0x18,0x00,0x30,0x00,0xff,0x2e]) client.publish(`/sys/${options.productKey}/${options.deviceName}/thing/model/up_raw`, payloadArray, { qos: 0 }); }, 5 * 1000);After you connect the device to IoT Platform, the device is in the Online state. You can view the status on the Devices page of the instance to which the device is connected.
Click View in the Actions column of Esensor. Then, click TSL Data. In this example, the data type of the product is set to Custom. In this case, the submitted TSL data is not displayed on the Status tab.
Choose . On the Cloud run log tab, select Device-to-cloud Messages to view the submitted messages that are displayed in the hexadecimal format.
In this example, the following data is displayed:
0xaa1fc800003710ff0005d76b15001c013400ad04ffff0400ffff18003000ff2e.
Write a data parsing script
You can edit, submit, and debug data parsing scripts in the IoT Platform console.
Log on to the IoT Platform console. Click the name of the instance that you want to manage. In the left-side navigation pane of the Instance Details page, choose .
On the Products page, find the product and click View in the Actions column.
On the product details page, click Message Analysis.
On the Message Analysis tab, write a script in the Edit Script field.
Modify the script based on the communication protocol that is used by the device. The following table describes the message body structure of the device.
Byte
Description
Remarks
12
Low byte value of the PM2.5 property
Returns the value of the PM2.5 property. Valid values: 0 to 999. Unit: ug/m3.
13
High byte value of the PM2.5 property
14
Low byte value of the temperature property × 10
Returns the value of the temperature property. Valid values: -10 to 50. Unit: °C.
15
High byte value of the temperature property × 10
16
Low byte value of the humidity property
Returns the value of the humidity property. Valid values: 0 to 99. Unit: %.
17
High byte value of the humidity property
18
Low byte value of the carbon dioxide property
Returns the value of the carbon dioxide property. Valid values: 0 to 9999. Unit: mg/m3.
19
High byte value of the carbon dioxide property
22
Low byte value of the formaldehyde property × 100
Returns the value of the formaldehyde property. Valid values: 0 to 9.99.
23
High byte value of the formaldehyde property × 100
28
Low byte value of the illuminance property
Returns the value of the illuminance property. Unit: lux.
29
High byte value of the illuminance property
In this example, the device submits only environmental data. In this case, you need to configure only the rawDataToProtocol() function to parse upstream data. You do not need to configure the protocolToRawData() function. You can convert custom data into property data, service data, or event data. In this example, custom data is converted into property data.
var PROPERTY_REPORT_METHOD = 'thing.event.property.post'; // Submit upstream data. The following function converts data in a custom format into a TSL model in the JSON format. function rawDataToProtocol(bytes) { var uint8Array = new Uint8Array(bytes.length); for (var i = 0; i < bytes.length; i++) { uint8Array[i] = bytes[i] & 0xff; } var dataView = new DataView(uint8Array.buffer, 0); var jsonMap = new Object(); // Specify the request method. jsonMap['method'] = PROPERTY_REPORT_METHOD; // Specify the protocol version. Set the value to 1.0. jsonMap['version'] = '1.0'; // Specify the ID of the request. jsonMap['id'] = new Date().getTime(); var params = {}; // The values 12 and 13 are the values of the PM2.5 property. params['PM25'] = (dataView.getUint8(13)*256+dataView.getUint8(12)); // The values 14 and 15 are the values of the temperature property. params['temperature'] = (dataView.getUint8(15)*256+dataView.getUint8(14))/10; // The values 16 and 17 are the values of the humidity property. params['humidity'] = (dataView.getUint8(17)*256+dataView.getUint8(16)); // The values 18 and 19 are the values of the co2 property. params['co2'] = (dataView.getUint8(19)*256+dataView.getUint8(18)); // The values 22 and 23 are the values of the hcho property. params['hcho'] = (dataView.getUint8(23)*256+dataView.getUint8(22))/100; // The values 28 and 29 are the values of the lightLux property. params['lightLux'] = (dataView.getUint8(29)*256+dataView.getUint8(28)); jsonMap['params'] = params; return jsonMap; } // Send downstream commands. The following function converts a TSL model in the JSON format into data of a custom format. function protocolToRawData(json) { var payloadArray = [1];// The device can only submit data and cannot receive commands from IoT Platform. return payloadArray; } // Parse topic data in a custom format to Alink JSON data. function transformPayload(topic, rawData) { var jsonObj = {} return jsonObj; }Test data parsing.
Select Upstreamed Device Data from the Simulation Type drop-down list.
On the Input Simulation tab, enter test data in the field.
You can use the hexadecimal data that is submitted by the device as test data. To view the hexadecimal data, go to the Device Log page. Example: 0xaa1fc800003710ff0005d76b15001c013400ad04ffff0400ffff18003000ff2e.
Click Run.
The following figure shows the result on the Parsing Results tab.
After you confirm that test data can be parsed by the script, click Submit to submit the script to IoT Platform.
Debug the device and submit data
After you submit the script, you can use the device SDK to debug the script.
When the device submits data to IoT Platform, IoT Platform uses the script to parse the data. Choose . On the Cloud run log tab, view the logs for data parsing.
To view the parsed data, go to the Device Details page and choose .
Example of upstream TSL data
{
"schema": "https://iotx-tsl.oss-ap-southeast-1.aliyuncs.com/schema.json",
"profile": {
"version": "1.0",
"productKey": "g4****"
},
"properties": [
{
"identifier": "lightLux",
"name": "illuminance",
"accessMode": "rw",
"required": false,
"dataType": {
"type": "float",
"specs": {
"min": "0",
"max": "10000",
"unit": "Lux",
"unitName": "luminous flux per unit area",
"step": "0.1"
}
}
},
{
"identifier": "PM25",
"name": "PM25",
"accessMode": "rw",
"required": false,
"dataType": {
"type": "int",
"specs": {
"min": "0",
"max": "1000",
"unit": "μg/m³",
"unitName": "micrograms per cubic meter",
"step": "1"
}
}
},
{
"identifier": "hcho",
"name": "formaldehyde",
"accessMode": "rw",
"desc": "HCHOValue",
"required": false,
"dataType": {
"type": "float",
"specs": {
"min": "0",
"max": "10",
"unit": "ppm",
"unitName": "parts per million",
"step": "0.1"
}
}
},
{
"identifier": "co2",
"name": "carbon dioxide",
"accessMode": "rw",
"required": false,
"dataType": {
"type": "int",
"specs": {
"min": "0",
"max": "10000",
"unit": "mg/m³",
"unitName": "milligrams per cubic meter",
"step": "1"
}
}
},
{
"identifier": "humidity",
"name": "humidity",
"accessMode": "rw",
"required": false,
"dataType": {
"type": "int",
"specs": {
"min": "0",
"max": "100",
"unit": "%",
"unitName": "percentage",
"step": "1"
}
}
},
{
"identifier": "temperature",
"name": "temperature",
"accessMode": "rw",
"desc": "motor working temperature",
"required": false,
"dataType": {
"type": "float",
"specs": {
"min": "-10",
"max": "50",
"unit": "℃",
"step": "0.1"
}
}
}
],
"events": [
{
"identifier": "post",
"name": "post",
"type": "info",
"required": true,
"desc": "submit properties",
"method": "thing.event.property.post",
"outputData": [
{
"identifier": "lightLux",
"name": "illuminance",
"dataType": {
"type": "float",
"specs": {
"min": "0",
"max": "10000",
"unit": "Lux",
"unitName": "luminous flux per unit area",
"step": "0.1"
}
}
},
{
"identifier": "PM25",
"name": "PM25",
"dataType": {
"type": "int",
"specs": {
"min": "0",
"max": "1000",
"unit": "μg/m³",
"unitName": "micrograms per cubic meter",
"step": "1"
}
}
},
{
"identifier": "hcho",
"name": "formaldehyde",
"dataType": {
"type": "float",
"specs": {
"min": "0",
"max": "10",
"unit": "ppm",
"unitName": "parts per million",
"step": "0.1"
}
}
},
{
"identifier": "co2",
"name": "carbon dioxide",
"dataType": {
"type": "int",
"specs": {
"min": "0",
"max": "10000",
"unit": "mg/m³",
"unitName": "milligrams per cubic meter",
"step": "1"
}
}
},
{
"identifier": "humidity",
"name": "humidity",
"dataType": {
"type": "int",
"specs": {
"min": "0",
"max": "100",
"unit": "%",
"unitName": "percentage",
"step": "1"
}
}
},
{
"identifier": "temperature",
"name": "temperature",
"dataType": {
"type": "float",
"specs": {
"min": "-10",
"max": "50",
"unit": "℃",
"step": "0.1"
}
}
}
]
}
],
"services": [
{
"identifier": "set",
"name": "set",
"required": true,
"callType": "async",
"desc": "set properties",
"method": "thing.service.property.set",
"inputData": [
{
"identifier": "lightLux",
"name": "illuminance",
"dataType": {
"type": "float",
"specs": {
"min": "0",
"max": "10000",
"unit": "Lux",
"unitName": "luminous flux per unit area",
"step": "0.1"
}
}
},
{
"identifier": "PM25",
"name": "PM25",
"dataType": {
"type": "int",
"specs": {
"min": "0",
"max": "1000",
"unit": "μg/m³",
"unitName": "micrograms per cubic meter",
"step": "1"
}
}
},
{
"identifier": "hcho",
"name": "formaldehyde",
"dataType": {
"type": "float",
"specs": {
"min": "0",
"max": "10",
"unit": "ppm",
"unitName": "parts per million",
"step": "0.1"
}
}
},
{
"identifier": "co2",
"name": "carbon dioxide",
"dataType": {
"type": "int",
"specs": {
"min": "0",
"max": "10000",
"unit": "mg/m³",
"unitName": "milligrams per cubic meter",
"step": "1"
}
}
},
{
"identifier": "humidity",
"name": "humidity",
"dataType": {
"type": "int",
"specs": {
"min": "0",
"max": "100",
"unit": "%",
"unitName": "percentage",
"step": "1"
}
}
},
{
"identifier": "temperature",
"name": "temperature",
"dataType": {
"type": "float",
"specs": {
"min": "-10",
"max": "50",
"unit": "℃",
"step": "0.1"
}
}
}
],
"outputData": []
},
{
"identifier": "get",
"name": "get",
"required": true,
"callType": "async",
"desc": "obtain properties",
"method": "thing.service.property.get",
"inputData": [
"lightLux",
"PM25",
"hcho",
"co2",
"humidity",
"temperature"
],
"outputData": [
{
"identifier": "lightLux",
"name": "illuminance",
"dataType": {
"type": "float",
"specs": {
"min": "0",
"max": "10000",
"unit": "Lux",
"unitName": "luminous flux per unit area",
"step": "0.1"
}
}
},
{
"identifier": "PM25",
"name": "PM25",
"dataType": {
"type": "int",
"specs": {
"min": "0",
"max": "1000",
"unit": "μg/m³",
"unitName": "micrograms per cubic meter",
"step": "1"
}
}
},
{
"identifier": "hcho",
"name": "formaldehyde",
"dataType": {
"type": "float",
"specs": {
"min": "0",
"max": "10",
"unit": "ppm",
"unitName": "parts per million",
"step": "0.1"
}
}
},
{
"identifier": "co2",
"name": "carbon dioxide",
"dataType": {
"type": "int",
"specs": {
"min": "0",
"max": "10000",
"unit": "mg/m³",
"unitName": "milligrams per cubic meter",
"step": "1"
}
}
},
{
"identifier": "humidity",
"name": "humidity",
"dataType": {
"type": "int",
"specs": {
"min": "0",
"max": "100",
"unit": "%",
"unitName": "percentage",
"step": "1"
}
}
},
{
"identifier": "temperature",
"name": "temperature",
"dataType": {
"type": "float",
"specs": {
"min": "-10",
"max": "50",
"unit": "℃",
"step": "0.1"
}
}
}
]
}
]
}