lua table - Behavior of nested arrays in Lua

Question

I have this piece of code :

json = require('dkjson')
dataStr = "{}"

function addEntry(position, hour, id)

        local data = json.decode(dataStr) or {} 
        
        if not data.detections then
            print("create data.detections")
            data.detections = {}
        end
        if not data.detections[position] then
            print("data.detections[position]")
            data.detections[position] = {}
        end              
        if not data.detections[position][hour] then
            print("data.detections[position][hour]")
            data.detections[position][hour] = {}
        end
        table.insert(data.detections[position][hour], id)
                
        dataStr = json.encode(data)
        print(dataStr)
end

addEntry("toto", 28000, 11111)
addEntry("toto", 28000, 22222)
addEntry("toto", 28000, 33333)
addEntry("toto", 28000, 44444)
addEntry("toto", 28000, 55555)

And I have this output on Zerobrane :

create data.detections
create data.detections[position]
create data.detections[position][hour]
{"detections":{"toto":{"28000":[11111,11111]}}}
create data.detections[position][hour]
{"detections":{"toto":{"28000":[22222,22222],"28000":[11111,11111]}}}
create data.detections[position][hour]
{"detections":{"toto":{"28000":[33333,33333],"28000":[11111,11111]}}}
create data.detections[position][hour]
{"detections":{"toto":{"28000":[44444,44444],"28000":[11111,11111]}}}
create data.detections[position][hour]
{"detections":{"toto":{"28000":[55555,55555],"28000":[11111,11111]}}}

I would have expected to have this in the final string : 

```{"detections":{"toto":{"28000":[11111,22222,33333,44444,55555]}}}

Can somebody explain to me this Lua behavior ? The override instead of adding a new value, and this remaining separate first value ? 

Answer 1

From the dkjson documentation:

It can also be used to save Lua data structures, but you should be aware that not every Lua table can be represented by the JSON standard. For example tables that contain both string keys and an array part cannot be exactly represented by JSON.

addEntry("toto", 28000, 11111)

prints

create data.detections
create data.detections[position]
create data.detections[position][hour]
{"detections":{"toto":{"28000":[11111,11111]}}}

You created and encoded the table

{detections = {toto = {[28000] = {11111, 11111}}}}

Which gives you the json string

'{"detections":{"toto":{"28000":[11111,11111]}}}'

When you decode this json string you'll have a lua table like

{detections = {toto = {["28000"] = {11111, 11111}}}}

As you see after decoding 28000 is now a string key and not the integer you encoded.

So when you call

addEntry("toto", 28000, 22222)

where hour is a number again, you'll end up with the table

{detections = {toto = {[28000] = {22222, 22222}, ["28000"] = {11111, 11111}}}}

that contains values for both the numberic and the string key.

If you encode this again you get the json string

'{"detections":{"toto":{"28000":[22222,22222],"28000":[11111,11111]}}}'

Now we decode this again leaving us with the table

{detections = {toto = {["28000"] = {11111, 11111}}}}

as the first "28000" entry {22222, 22222} is of course overwritten by the second one {11111, 11111} as we cannot have two elements for the same key.

Now the same thing happens for each of your following calls. You add a numeric key and lose it in the en/decoding process.

Numeric keys will only be used if your table is a sequence with consecutive integer keys from 1 to n. All other table keys are convertet to a string.

The double values are caused by the fact that you do this

 data.detections[position][hour] = {id}

When data.detections[position][hour] is nil which due to the problem described above is true in every call.

As you add id to the table after that you end up with a table containing the id twice. Create an empty table instead.