Tair (Redis® OSS-Compatible):Lua script syntax and solutions to common errors

Precautions

Commands related to Lua scripts cannot be used in the Data Management (DMS) console. For more information about DMS, see Overview. You can use a client or redis-cli to connect to instances and use Lua scripts.

Syntax

For more information about the commands, visit the Redis official website.

Reduce memory and network overheads

Problem description:

EVAL "return redis.call('set', 'k1', 'v1')" 0
EVAL "return redis.call('set', 'k2', 'v2')" 0

Flush the Lua script cache

Problem description:

-OOM command not allowed when used memory > 'maxmemory'.

Solution:

Do not write Lua scripts that may take up excessive amounts of memory. Moreover, do not write Lua scripts that involve large amounts of data. Otherwise, memory usage significantly increases and an OOM error may occur. To reduce memory usage, we recommend that you enable data eviction by using the volatile-lru policy. By default, data eviction is enabled for the instance. However, the instance does not evict the Lua script cache regardless of whether data eviction is enabled.

Handle the NOSCRIPT error

Problem description:

(error) NOSCRIPT No matching script. Please use EVAL.

Solution:

Run the EVAL or SCRIPT LOAD command to cache the script in the instance and try again. In specific scenarios, such as instance migrations and configuration changes, the instance flushes the Lua script cache because the instance cannot ensure the persistence and replicability of Lua scripts. In this case, your client must be able to handle the error. For more information, see Caching, persistence, and replication of scripts.

import redis
import hashlib

# strin specifies a string in Lua scripts. The following function returns the SHA1 value of strin in the string format. 
def calcSha1(strin):
    sha1_obj = hashlib.sha1()
    sha1_obj.update(strin.encode('utf-8'))
    sha1_val = sha1_obj.hexdigest()
    return sha1_val

class MyRedis(redis.Redis):

    def __init__(self, host="localhost", port=6379, password=None, decode_responses=False):
        redis.Redis.__init__(self, host=host, port=port, password=password, decode_responses=decode_responses)

    def prepend_inLua(self, key, value):
        script_content = """\
        local suffix = redis.call("get", KEYS[1])
        local prefix = ARGV[1]
        local new_value = prefix..suffix
        return redis.call("set", KEYS[1], new_value)
        """
        script_sha1 = calcSha1(script_content)
        if self.script_exists(script_sha1)[0] == True:      # Check whether the script is already cached in Tair (Redis OSS-compatible). 
            return self.evalsha(script_sha1, 1, key, value) # If the script is already cached, the EVALSHA command is used to run the script.
        else:
            return self.eval(script_content, 1, key, value) # Otherwise, use the EVAL command to run the script. Note that the EVAL command can cache scripts. Another method is to use the SCRIPT LOAD and EVALSHA commands. 

r = MyRedis(host="r-******.redis.rds.aliyuncs.com", password="***:***", port=6379, decode_responses=True)

print(r.prepend_inLua("k", "v"))
print(r.get("k"))
            

Handle timeouts of Lua scripts

• Problem description:

  Slow Lua requests may block the instance because a Lua script is atomically executed in the instance. One Lua script can block the instance for up to 5 seconds. After 5 seconds, the instance returns a BUSY error for other commands until the script execution is complete.

  BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.

  Solution:

  Run the SCRIPT KILL command to terminate the Lua script or wait until the Lua script finishes running.

  Note

  • During the first 5 seconds when a slow Lua script is being executed, the SCRIPT KILL command does not take effect because the instance is being blocked.

  • To prevent the instance from being blocked for an extended period of time, we recommend that you estimate the amount of time required to execute a Lua script when you write the Lua script, check for infinite loop, and split the Lua script if necessary.

• Problem description:

  If a Lua script has already run write commands against the dataset, the SCRIPT KILL command does not take effect. Error example:

  (error) UNKILLABLE Sorry the script already executed write commands against the dataset. You can either wait the script termination or kill the server in a hard way using the SHUTDOWN NOSAVE command.

  Solution:

  On the Instances page of the console, find the instance that you want to manage and click restart in the Actions column.

Caching, persistence, and replication of scripts

Problem description:

Tair (Redis OSS-compatible) keeps caching the Lua scripts that have been executed in an instance if the instance is not restarted or the SCRIPT FLUSH command is not run for the instance. However, Tair (Redis OSS-compatible) cannot ensure the persistence of Lua scripts or the synchronization of Lua scripts from the current node to other nodes in scenarios such as instance migrations, configuration changes, version upgrades, and instance switchovers.

Solution:

Store all Lua scripts in your on-premise device. Recache the Lua scripts in Tair (Redis OSS-compatible) by using the EVAL or SCRIPT LOAD command if necessary. This prevents a NOSCRIPT error from occurring when Lua scripts are cleared during an instance restart or a high availability (HA) switchover.

Limits on Lua scripts in cluster instances

• To ensure the execution atomicity, a Lua script cannot be split and can be executed only on one shard in a cluster instance. In most cases, a key is used to determine which shard Lua scripts are routed to. Therefore, you must specify at least one key when you run a Lua script in a cluster instance. If you want to read or write multiple keys, the keys in one Lua script must belong to the same slot. Otherwise, an abnormal execution result is returned. Lua scripts that do not have keys (such as KEYS, SCAN, and FLUSHDB) can be executed normally. However, only the data of a single shard is returned. This limit is caused by the architecture of cluster instances.

• A Lua script may not be stored in other nodes when you run the SCRIPT LOAD command on one node.

Error codes and causes for custom Lua script errors in proxy mode

Proxy nodes perform syntax checks to identify the keys that belong to multiple slots and throw exceptions in advance to assist troubleshooting. Proxy nodes use a different approach to check for errors than Lua virtual machines. This places additional limits on the execution of Lua scripts in proxy mode. For example, the UNPACK command is not supported, and the EVAL, EVALSHA, and SCRIPT commands are not supported in MULTI and EXEC transactions. You can also set the script_check_enable parameter to disable additional checks on Lua syntax in proxy mode.

If the readonly_lua_route_ronode_enable parameter is set to 1 for a read/write splitting instance, proxy nodes check whether Lua scripts contain only read-only commands and determine whether to forward them to read-only nodes. This check logic imposes limits on Lua syntax.

The following table describes the error codes and causes.

# Example of valid command usage:
EVAL "return redis.call('get', KEYS[1])" 1 fooeval

# Example of invalid command usage:
EVAL "return redis.call('get', 'foo')" 0

# Example of valid command usage:
EVAL "return redis.call('mget', KEYS[1], KEYS[2])" 2 foo {foo}bar

# Example of invalid command usage:
EVAL "return redis.call('mget', KEYS[1], KEYS[2])" 2 foo foobar

# Example of valid command usage:
EVAL "local value = redis.call('GET', KEYS[1]); redis.call('SET', KEYS[2], value)" 2 foo bar

# Example of invalid command usage:
EVAL "redis.call('SET', KEYS[1], redis.call('GET', KEYS[2]))" 2 foo bar

# Example of valid command usage:
eval "redis.call('GET', KEYS[1])" 1 foo

# Example of invalid command usage:
eval "local cmd = 'GET'; redis.call(cmd, KEYS[1])" 1 foo

# Example of valid command usage:
EVAL "return redis.call('mget', KEYS[1], KEYS[2])" 2 foo {foo}bar

# Example of invalid command usage:
EVAL "return redis.call('mget', KEYS[1], '{foo}bar')" 1 foo                      # This command is invalid because the '{foo}bar' key must be specified by using the KEYS array. 
EVAL "local i = 2 return redis.call('mget', KEYS[1], KEYS[i])" 2 foo {foo}bar    # This command is invalid because the index of keys consists of variables, which is not allowed for an instance in proxy mode. Instances in direct connection mode are not subject to this limit. 
EVAL "return redis.call('mget', KEYS[1], ARGV[1])" 1 foo {foo}bar                # This command is invalid because ARGV[1] cannot be specified as a key.