All Products
Search
Document Center

Tair (Redis® OSS-Compatible):Lua script syntax and solutions to common errors

Last Updated:Oct 31, 2024

Tair (Redis OSS-compatible) instances support Lua commands. Lua scripts can be used to efficiently process compare-and-set (CAS) commands. This improves the performance of Redis and simplifies the implementation of features. This topic describes the syntax and usage of Lua scripts.

Precautions

Commands related to Lua scripts cannot be used in the Data Management (DMS) console. For more information about DMS, see Overview. You can use a client or redis-cli to connect to Redis instances and use Lua scripts.

Syntax

Command

Syntax

Description

EVAL

EVAL script numkeys [key [key ...]] [arg [arg ...]]

Executes a specific script that includes parameters and returns the output.

Parameters:

  • script: the Lua script.

  • numkeys: the number of arguments in the KEYS array. The number is a non-negative integer.

  • KEYS[]: the Redis keys that you want to pass to the script as arguments.

  • ARGV[]: the additional arguments that you want to pass to the script. The indexes of the KEYS[] and ARGV[] parameters start from 1.

Note
  • The EVAL command loads a script into the script cache of Tair in the same manner that the SCRIPT LOAD command loads a script.

  • Mixed use or misuse of the KEYS[] and ARGV[] parameters may cause Tair instances, especially cluster instances, to run abnormally. For more information, see Limits on Lua scripts in cluster instances.

  • We recommend that you pass values in the KEYS[] and ARGV[] parameters to call Lua scripts, instead of encoding parameters into Lua scripts. Otherwise, the memory usage of the Lua virtual machine increases and cannot be reduced at the earliest opportunity. In a worst-case scenario, an out of memory (OOM) error occurs on the instance and results in data loss.

EVALSHA

EVALSHA sha1 numkeys key [key ...] arg [arg ...]

Evaluates a cached script by its SHA1 digest and runs the script.

If the script is not cached in Redis when you use the EVALSHA command, Redis returns the NOSCRIPT error. Cache the script in Redis by using the EVAL or SCRIPT LOAD command and try again. For more information, see Handle the NOSCRIPT error.

SCRIPT LOAD

SCRIPT LOAD script

Caches a specified script in Redis and returns the SHA1 digest of the script.

SCRIPT EXISTS

SCRIPT EXISTS script [script ...]

Returns information about the existence of one or more scripts in the script cache by using their corresponding SHA1 digests. If a specific script exists, a value of 1 is returned. Otherwise, a value of 0 is returned.

SCRIPT KILL

SCRIPT KILL

Terminates a running Lua script.

SCRIPT FLUSH

SCRIPT FLUSH

Removes all Lua scripts from the script cache in the Redis server.

For more information about Redis commands, visit the Redis official website.

The following sample code provides examples of specific Redis commands. Before the following commands are run, the SET foo value_test command is run.

  • Sample EVAL command:

    EVAL "return redis.call('GET', KEYS[1])" 1 foo

    Sample output:

    "value_test"
  • Sample SCRIPT LOAD command:

    SCRIPT LOAD "return redis.call('GET', KEYS[1])"

    Sample output:

    "620cd258c2c9c88c9d10db67812ccf663d96bdc6"
  • Sample EVALSHA command:

    EVALSHA 620cd258c2c9c88c9d10db67812ccf663d96bdc6 1 foo

    Sample output:

    "value_test"
  • Sample SCRIPT EXISTS command:

    SCRIPT EXISTS 620cd258c2c9c88c9d10db67812ccf663d96bdc6 ffffffffffffffffffffffffffffffffffffffff

    Sample output:

    1) (integer) 1
    2) (integer) 0
  • Sample SCRIPT FLUSH command:

    Warning

    This command deletes all cached Lua scripts from the instance. Make sure that you back up the Lua scripts before you run this command.

    SCRIPT FLUSH

    Sample output:

    OK

Reduce memory and network overheads

Issue:

A large number of scripts that serve the same purposes are cached in Redis. These scripts take up large amounts of memory and may cause an out of memory (OOM) error. Example of invalid usage:

EVAL "return redis.call('set', 'k1', 'v1')" 0
EVAL "return redis.call('set', 'k2', 'v2')" 0

Solution:

  • Do not pass parameters to Lua scripts as constants to reduce memory usage.

    # The following commands serve the same purposes as the preceding sample commands but cache scripts only once. 
    EVAL "return redis.call('set', KEYS[1], ARGV[1])" 1 k1 v1
    EVAL "return redis.call('set', KEYS[1], ARGV[1])" 1 k2 v2
  • Use the following command syntax to reduce memory and network overheads:

    SCRIPT LOAD "return redis.call('set', KEYS[1], ARGV[1])"    # After this command is run, the following output is returned: "55b22c0d0cedf3866879ce7c854970626dcef0c3".
    EVALSHA 55b22c0d0cedf3866879ce7c854970626dcef0c3 1 k1 v1
    EVALSHA 55b22c0d0cedf3866879ce7c854970626dcef0c3 1 k2 v2

Flush the Lua script cache

Issue:

Used memory of an instance may be higher than expected because the Lua script cache takes up memory of the instance. When the used memory of the instance approaches or exceeds the upper limit specified by maxmemory, an OOM error is triggered. Error example:

-OOM command not allowed when used memory > 'maxmemory'.

Solution:

Flush the Lua script cache by running the SCRIPT FLUSH command on the client. Different from the FLUSHALL command, the SCRIPT FLUSH command is synchronous. If Redis caches a large number of Lua scripts, the SCRIPT FLUSH command can block Redis for an extended period of time and the involved instance may become unavailable. Proceed with caution. We recommend that you perform this operation during off-peak hours.

Note

If you click Clear Data in the console, data can be cleared but the Lua script cache cannot be flushed.

Do not write Lua scripts that may consume excessive amounts of memory resources or Lua scripts that involve large amounts of data. Otherwise, memory usage significantly increases and an OOM error may occur. To reduce memory usage, we recommend that you enable data eviction by using the volatile-lru policy. By default, data eviction is enabled in Tair. However, Tair does not evict the Lua script cache regardless of whether data eviction is enabled.

Handle the NOSCRIPT error

Issue:

If the script is not cached in Redis when you use the EVALSHA command, Redis returns the NOSCRIPT error. Error example:

(error) NOSCRIPT No matching script. Please use EVAL.

Solution:

Run the EVAL or SCRIPT LOAD command to cache the script in Redis and try again. In specific scenarios, such as instance migrations and configuration changes, Redis flushes the Lua script cache because Redis cannot ensure the persistence and replicability of Lua scripts. In this case, your client must be able to handle the error. For more information, see Caching, persistence, and replication of scripts.

The following sample Python code shows a method for handling the NOSCRIPT error. The sample code prepends strings by using Lua scripts.

Note

You can also use redis-py to handle the error. redis-py provides the Script class that encapsulates the judgment logic for Lua scripts of Redis, such as a catch statement for the NOSCRIPT error.

import redis
import hashlib

# strin specifies a string in Lua scripts. The following function returns the SHA1 value of strin in the string format. 
def calcSha1(strin):
    sha1_obj = hashlib.sha1()
    sha1_obj.update(strin.encode('utf-8'))
    sha1_val = sha1_obj.hexdigest()
    return sha1_val

class MyRedis(redis.Redis):

    def __init__(self, host="localhost", port=6379, password=None, decode_responses=False):
        redis.Redis.__init__(self, host=host, port=port, password=password, decode_responses=decode_responses)

    def prepend_inLua(self, key, value):
        script_content = """\
        local suffix = redis.call("get", KEYS[1])
        local prefix = ARGV[1]
        local new_value = prefix..suffix
        return redis.call("set", KEYS[1], new_value)
        """
        script_sha1 = calcSha1(script_content)
        if self.script_exists(script_sha1)[0] == True:      # Check whether the script is already cached in Redis. 
            return self.evalsha(script_sha1, 1, key, value) # If the script is already cached, the EVALSHA command is used to run the script.
        else:
            return self.eval(script_content, 1, key, value) # Otherwise, use the EVAL command to run the script. Note that the EVAL command can cache scripts in Redis. You can also use the SCRIPT LOAD and EVALSHA commands. 

r = MyRedis(host="r-******.redis.rds.aliyuncs.com", password="***:***", port=6379, decode_responses=True)

print(r.prepend_inLua("k", "v"))
print(r.get("k"))
            

Handle timeouts of Lua scripts

  • Issue:

    Slow Lua requests may block Redis because Lua scripts are atomically executed in Redis. One Lua script can block Redis for up to 5 seconds. After 5 seconds, Redis returns a BUSY error for other commands until the script finishes running.

    BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.

    Solution:

    Run the SCRIPT KILL command to terminate the Lua script or wait until the Lua script finishes running.

    Note
    • During the first 5 seconds when a slow Lua script is running, the SCRIPT KILL command does not take effect because Tair is being blocked.

    • To prevent Redis from being blocked for an extended period of time, we recommend that you estimate the amount of time required to execute a Lua script when you write the Lua script, check for infinite loop, and split the Lua script if necessary.

  • Issue:

    If a Lua script has already run write commands against the dataset, the SCRIPT KILL command does not take effect. Error example:

    (error) UNKILLABLE Sorry the script already executed write commands against the dataset. You can either wait the script termination or kill the server in a hard way using the SHUTDOWN NOSAVE command.

    Solution:

    On the Instances page of the console, find the instance that you want to manage and click Restart in the Actions column.

Caching, persistence, and replication of scripts

Issue:

Redis keeps caching the Lua scripts that have been executed in an instance if the instance is not restarted or the SCRIPT FLUSH command is not run for the instance. However, Redis cannot ensure the persistence of Lua scripts or the synchronization of Lua scripts from the current node to other nodes in scenarios such as instance migrations, configuration changes, version upgrades, and instance switchovers.

Solution:

Store all Lua scripts on your on-premise device. Recache the Lua scripts in Redis by using the EVAL or SCRIPT LOAD command if necessary. This prevents the NOSCRIPT error from occurring when Lua scripts are cleared during an instance restart or a high availability (HA) switchover.

Limits on Lua scripts in cluster instances

Open source Redis clusters have limits on the use of Lua scripts. Redis cluster instances have the following additional limits on the use of Lua scripts:

  • Instances that use specific minor versions do not support EVAL-related commands. If the ERR command eval not support for normal user error message is returned, update the minor version of your instance and try again. For more information, see Update the minor version of an instance.

  • All keys must be in the same slot. Otherwise, the system returns the -ERR eval/evalsha command keys must be in same slot\r\n error.

    You can run the CLUSTER KEYSLOT command to obtain the hash slot of a key.

  • When you run the SCRIPT LOAD command on one node, a Lua script may not be stored in other nodes.

  • The following Pub/Sub commands are not supported: PSUBSCRIBE, PUBSUB, PUBLISH, PUNSUBSCRIBE, SUBSCRIBE, and UNSUBSCRIBE.

  • The UNPACK function is not supported.

Note

If all the operations can be performed in the same hash slot and you want to break through the limits that are imposed on your Lua scripts by the Redis cluster architecture, you can set the script_check_enable parameter to 0 in the console. This way, the system does not check your Lua scripts at the backend. In this case, you must specify at least one key in the KEYS array to allow the proxy nodes to route the requests involved in your Lua scripts. If you cannot make sure that all the operations are performed in the same hash slot, an error is returned. For more information, see Configure instance parameters.

Additional checks on Lua scripts in proxy mode

You can specify the script_check_enable parameter to disable the following checks on Lua scripts. We recommend that you do not disable the checks.

  • Lua scripts use the redis.call or redis.pcall function to run Redis commands. All keys must be specified by using the KEYS array, and cannot be replaced by Lua variables. If you do not use the KEYS array to specify the keys, the following error message is returned: -ERR bad lua script for redis cluster, all the keys that the script uses should be passed using the KEYS array\r\n.

    Note

    This limit applies only to instances that run Redis 5.0 and whose minor version is earlier than 5.0.8 and instances that run Redis 4.0 or earlier.

    Examples of valid and invalid command usage:

    # To prepare for this example, run the following commands: 
    SET foo foo_value
    SET {foo}bar bar_value
    
    # Example of valid command usage:
    EVAL "return redis.call('mget', KEYS[1], KEYS[2])" 2 foo {foo}bar
    
    # Example of invalid command usage:
    EVAL "return redis.call('mget', KEYS[1], '{foo}bar')" 1 foo                      # This command is invalid because the '{foo}bar' key must be specified by using the KEYS array. 
    EVAL "local i = 2 return redis.call('mget', KEYS[1], KEYS[i])" 2 foo {foo}bar    # This command is invalid because the index of keys consists of variables, which is not allowed for an instance in proxy mode. Instances in direct connection mode are not subject to this limit. 
    EVAL "return redis.call('mget', KEYS[1], ARGV[1])" 1 foo {foo}bar                # This command is invalid because ARGV[1] cannot be specified as a key. 
  • When you use Lua scripts with the redis.call or redis.pcall function, the first parameter you provide must be a string literal representing the Redis command that you want to run. If you attempt to use a variable or a non-literal string as the first parameter, the following error message is returned: -ERR bad lua script for redis cluster, first parameter of redis.call/redis.pcall must be a single literal string.

    Examples of valid and invalid command usage:

    # Example of valid command usage
    eval "redis.call('GET', KEYS[1])" 1 foo
    
    # Example of invalid command usage
    eval "local cmd = 'GET'; redis.call(cmd, KEYS[1])" 1 foo
  • You must include keys in all commands that you want to run. Otherwise, the following error message is returned: -ERR for redis cluster, eval/evalsha number of keys can't be negative or zero\r\n.

    Note

    This limit applies only to instances that run Redis 5.0 and whose minor version is earlier than 5.0.8 and instances that run Redis 4.0 or earlier.

    Examples of valid and invalid command usage:

    # Example of valid command usage
    EVAL "return redis.call('get', KEYS[1])" 1 fooeval
    
    # Example of invalid command usage
    EVAL "return redis.call('get', 'foo')" 0
  • Nested calls are not supported within Lua scripts for Redis clusters. When you attempt to run a script that contains nested calls, the following error message is returned: -ERR bad lua script for redis cluster, nested redis.call/redis.pcall.

    You can use local variables to work around this limitation. Examples of valid and invalid command usage:

    # Example of valid command usage
    EVAL "local value = redis.call('GET', KEYS[1]); redis.call('SET', KEYS[2], value)" 2 foo bar
    
    # Example of invalid command usage
    EVAL "redis.call('SET', KEYS[1], redis.call('GET', KEYS[2]))" 2 foo bar
  • You cannot run the EVAL, EVALSHA, or SCRIPT command in the MULTI or EXEC transactions.

  • Commands that involve multiple Redis nodes, such as KEYS and SCAN, are not supported in Lua scripts.

    To ensure that a Lua script is atomically executed, a proxy node uses the KEYS parameter to route the Lua script to a Redis node for execution, and then retrieves the result. This results in different command outputs for the Redis node and the other nodes.

Note

If you want to use features that are unavailable in proxy mode, you can enable the direct connection mode for your ApsaraDB for Redis cluster instance. However, you cannot perform migrations or configuration changes for ApsaraDB for Redis cluster instances if Lua scripts that do not meet the requirements of the proxy mode are executed in direct connection mode. This is because cluster instances rely on proxy nodes during migrations and configuration changes.

To resolve this issue, make sure that Lua scripts meet the requirements of the proxy mode before you execute the scripts in direct connection mode.