https://www.compose.com/articles/a-quick-guide-to-redis-lua-scripting/
What's Lua?
Lua is a language which has been around since 1993. Its origins in engineering made for a compact language which could be embedded in other applications. It's been embedded in applications as diverse as World of Warcraft and the Nginx web server. And Redis, which is why we are here.
What does Redis let you do with Lua?
It lets you create your own scripted extensions to the Redis database. That means that with Redis you can execute Lua scripts like this:
> EVAL 'local val="Hello Compose" return val' 0
"Hello Compose"
The string after the EVAL is the Lua script.
local val="Hello Compose"
return val
So at its simplest you can run Lua scripts. But more importantly, you can run Lua scripts that act like an intelligent transaction. You can handle errors smartly, so instead of just rolling back, you can carry on processing. Of course, the intelligence of the transaction will be up to you.
But to start tapping into that power you'll probably want to pass the script some keys and arguments.
Keys and arguments?
The 0 at the end of the EVAL, thats the number of keys being passed to the Lua code, in that example there were none. But if instead of 0 it was 2 foo bar fizz buzzthen the first two items in the list, foo and bar, would be passed as keys and fizz and buzz would be arguments.
If you do pass keys, they are available to the Lua script in the KEYS table (A table is Lua's associative array which also is used as an 1-based array). If you have arguments, they appear in the ARGV table. For example:
return ARGV[1]..' '..KEYS[1]
The .. is Lua's string concatenation operator so this returns whatever the argument is concatenated with a space and then the key name given in the arguments.
> EVAL "return ARGV[1]..' '..KEYS[1]" 1 name:first "Hello"
"Hello name:first"
No magic is applied to the KEYS, they are just strings so we still have to look up their value.
And how do you call Redis from inside Lua?
We can get at Redis's functions through the redis.call() command. If we use this script:
return ARGV[1].." "..redis.call("get",KEYS[1])
and EVAL it, assuming we've SET the key name:first to something we'll see this:
> EVAL 'return ARGV[1].." "..redis.call("get",KEYS[1])' 1 name:first "Hello"
"Hello Brian"
So now, in a script, we've taken a parameter, looked up a key's value and created a string and returned that as a result.
This EVAL command is going to get pretty messy isn't it?
Yes. That's why there's other ways to get Lua scripts up to the server. The one we like is by using the command line arguments of the redis-cli command. Just write your "more complex" Lua script...
lua local name=redis.call("get", KEYS[1]) local greet=ARGV[1] local result=greet.." "..name return result
We'll save that as longhello.lua and the run this at the command line:
$ redis-cli -h aws-us-east-1-portal.15.dblayer.com -p 11260 -a secret --eval longhello.lua name:first , Hello
"Hello Brian"
From the top, we're running the redis-cli, complete with -h, -p and -a parameters to connect to the database. Then comes the new bit, --eval. This lets you name a file to be sent up to the server and eval'd. So we follow that with longhello.lua. Now our script needs keys and arguments. The keys come first and redis-cli counts them for us; each command line argument is a key, right up to the comma. What comes after the comma are arguments.
So now we can write Lua code for Redis locally and quickly test it on the server, even a remote one.
I'm pretty good for greetings functions, got anything meatier?
Why yes, here's a little problem for you that Lua solves nicely. Consider a situation where various divisions of a company increment different counters in the big scheme of things. So say "region:one" bumps the counter keys "count:emea", "count:usa", "count:atlantic" while "region:two" just bumps "count:usa". These counter lists may be added to in the future, but you really want to make sure it happens all in one fell swoop. Remember what we said about this being an "intelligent transaction", well, here we can do all that in one script.
Let's set up our regions as lists:
> rpush region:one count:emea count:usa count:atlantic
(integer) 3
> rpush region:two "count:usa"
(integer) 1
Now we'll create a Lua script locally:
local count=0
We start with a count variable - we will count all the increments we do and return that value.
local broadcast=redis.call("lrange", KEYS[1], 0,-1)
Here we ask Redis to give us all the values in the list that should be referred to in the first key.
for _,key in ipairs(broadcast) do
This is the opener of a Lua for loop. The ipairs function iterates through the Lua table we just got in order and we take the key from each.
redis.call("INCR",key)
count=count+1
And for each key we ask Redis to increment it. Oh and then we bump up our counter. And thats nearly it. All thats left is...
end
return count
... to end the for loop and return the count. Save that to a file and then run it with an argument:
$ redis-cli -h aws-us-east-1-portal.15.dblayer.com -p 11260 -a secret --eval broadcast.lua region:one
(integer) 3
and if we go look we find:
> mget count:usa count:atlantic count:emea
1) "1"
2) "1"
3) "1"
And if we use the script on region 2...
$ redis-cli -h aws-us-east-1-portal.15.dblayer.com -p 11260 -a secret --eval broadcast.lua region:two
(integer) 1
...
> mget count:usa count:atlantic count:emea
1) "2"
2) "1"
3) "1"
So now we've got ourselves a useful little function. What if there was an error? Ah, well as it stands this script would error out too. That's because we're using redis.call()which explicitly does that. If we used redis.pcall(), if there was an error, the error details would be returned instead and we could decide what to do. But this is a speed guide and...
Wait a minute, I have to upload the script every time?
No, Redis has a script cache and a command SCRIPT LOAD to just load scripts into the cache. We'll use it from the command line here, but you can incorporate it into your applications like any other Redis command.
$ redis-cli -h aws-us-east-1-portal.15.dblayer.com -p 11260 -a secret SCRIPT LOAD "$(cat broadcast.lua)"
"84ffc8b6e4b45af697cfc5cd83894417b7946cc1"
That "$(cat broadcast.lua)" just turns our script into a quoted argument. The important bit is the number that comes back (its in hex). It's the SHA1 signature of the script. We can use this to invoke the script using the EVALSHA command like this:
> EVALSHA 84ffc8b6e4b45af697cfc5cd83894417b7946cc1 1 region:one
(integer) 3
There's commands to check for scripts (SCRIPT EXISTS) and to flush them out (SCRIPT FLUSH) so you can manage your script loading too.
So what's the catch?
After a particular time limit (5 seconds by default), Lua scripts will start getting errors in response to queries and an error will be written to the log - the only commands available will either kill the script (KILL SCRIPT) or shutdown the server (SHUTDOWN NOSAVE) in that situation. The five second time limit is insanely generous as you should be writing your Lua scripts to run very quickly, in milliseconds. Why? Because while your script is running, everything else is on hold.
Hey, Lua has a lot of libraries, can I use them?
No, afraid not. The documentation lists the ones that are loaded; base, table, string, math, struct, cjson, cmsgpack, bitop, redis.sha1hex, ref
Where next?
The EVAL commands documentation is where you start as it goes into detail on how Redis types are converted to Lua types. The Lua Manual covers the entire language - and remember there's versions to download that you can run outside Redis for practice. There's also an alternative introduction from 2013 which touches on using the libraries and common gotchas too.
