Recently I’ve been discussing datastores with some friends, and it’s motivated me to make a post about how I work with datastores. My method ensures that data loss and duplication glitches are impossible, and enables multiple servers to write to the same key, all without the need for session locking. Under my method, session locking can still be used when needed, but I think after reading this post you’ll agree that it’s not needed very often.
What is session locking?
Session locking is a technique where a server locks a key for some period of time before unlocking it. For the duration that the server holds the lock only that server can write to the key. This is often used for player data - when a player joins, the server locks the key, and when the player leaves, the server unlocks the key.
Session locking is a good way to prevent data loss and some duplication glitches, but it has some downsides.
- When players move between servers, they must either wait some time before their data is available on the new server, or some amount of data could be lost if the new server steals the lock from the old server. This one is especially bad for games that use teleports or encourage players to rejoin frequently.
- When a server crashes, the lock is not released, and the key is locked until it expires. Depending on the expiration time, this could be a considerable period of time, at least for the player who cannot join the game.
- By it’s very nature, you cannot edit a key from multiple servers at the same time, or from outside roblox with the open cloud api.
Changing how we think about datastores
For a very long time I thought about datastores like it was a file. When a player joins, the server reads the file, and when the player leaves, the server saves the file. This model works well with session locking - it’s like opening a file so that no other application can open it. Eventually, I stopped thinking about datastores like a file and started thinking about them like a database. In database software, one does not load data, modify it, and save it back. Instead one writes queries that modify the data in place.
Roblox provides an api for precisely this purpose - DataStore:UpdateAsync. UpdateAsync will read the data, pass it to a function you provide, and then save the data back. It uses an optimistic lock to ensure that your update is always applied with the latest data, and that no data is lost. To top it off, it returns the stored data so that you don’t have to do another GetAsync.
Purity and atomicity, or how to write safe update functions
Update functions must be pure and atomic to prevent data loss and duplication glitches.
Update purity
For my purposes, a function is pure if for the same inputs it gives the same outputs. Let’s look at a few examples.
local function addone(n: number): number
return n + 1
end
This function is pure. We can call addone(1) infinitely many times and always get 2.
local function addrandom(n: number): number
return n + math.random()
end
This function is not pure. addrandom(1) will give a different output every time it’s called. Let’s look at this in actual update functions.
local function removemoney(data)
if data.money < 10 then
return data
end
data.money -= 10
return data
end
This update function is pure. If we call removemoney({money = 100}) we will always get {money = 90}. It’s also important to note that the function has a failure case. If the player doesn’t have enough money the function will error, and the datastore will not be updated. I’ll refactor this so that any amount of money can be removed, not just 10.
local function removemoney(amount: number)
return function(data)
if data.money < amount then
return data
end
data.money -= amount
return data
end
end
Here we can see that we have a function that returns an update function. This pattern is very useful when working with datastores; it can be thought of as inserting arguments into an sql string query.
Update atomicity
Atomicity is the idea that in a series of updates, either all updates are applied, or none are.
Here is a series of update functions that are run when a player buys an item.
store:UpdateAsync(key, removemoney(10))
store:UpdateAsync(key, additem("sword"))
The problem with this code is that it does not care if the player has enough money to buy the item, it will always give the player the sword. Updates that rely on each other should be run in a single update function.
local function buyitem(cost: number, item: string)
return function(data)
if data.money < cost then
return data
end
data.money -= cost
data.items[item] = true
return data
end
end
A few convenient tweaks
Roblox only allows UpdateAsync to be called once every 6 seconds per key. This means that a queue for updates is almost always desirable. Here is my simple implementation.
local function onplayeradded(player: Player)
local queue = {}
local cache = nil
local key = getplayerkey(player)
task.spawn(function()
while true do
local swap = queue
queue = {}
cache = pcall(store.UpdateAsync, key, function(data)
for _, update in swap do
data = update(data)
end
return data
end)
updatestate(player, cache)
task.wait(10)
end
end)
local function applyupdate(update)
table.insert(queue, update)
update(cache)
updatestate(player, cache)
end
end
This code will run updates every 10 seconds, and also retrieve the latest data at the same time. It also applies updates immediately to the cached data, so that the player can see the changes before the update is saved. This code is not perfect, but it has the core functionality that I use.
Frequently asked questions
What about ratelimits?
At the time of writing, roblox has a ratelimit of 10 UpdateAsync calls per minute per player. This means at a maximum level, we can make one call every 6 seconds. I do 10 seconds in my code for a bit a buffer room. Remember: roblox will not pay you more if you don’t use your request budget, nobody will be impressed either.