json - What is the correct way of "refreshing" a config file which is constantly being updated?

Question

I'm a bit stooped on an issue I'm experiencing.

I have a python program which sends signals over a GPIO interface (Pi 4). The signals are dependant on a config.json

The layout of the json looks like the following:

{
   "key1" : val1
   "key2" : val2
   "key3" : val3
}

The config data is passed to the callers as lists and saved as a dict/json config file on the device for reuse when no new config arrives. The program uses the following bit of code to read, edit and save existing or new configs:

def check_json(self, source: str, write=False, val1=940, val2=5, val3=10):
    """check_json checks whether there's an existing config on the machine and in the same folder/location
    if it does exist, it gets returned to the caller. If it doesn't exist, a new file will be created
    with preset data"""
    if os.path.isfile(source):
        if write:
            try:
                with open(source, "r+") as json_data:  # update the existing values and save them
                    try:
                        config_data = js.load(json_data)
                        config_data["key1"] = val1
                        config_data["key2"] = val2
                        config_data["key3"] = val3
                        print(config_data)
                        json_data.seek(0)
                        json_data.truncate()
                        js.dump(config_data, json_data, indent=2)
                        json_data.flush()
                    except TypeError:
                        pass
            except TypeError:
                pass
        else:
            json_data = open(source, "r")
            dict_data = js.load(json_data)
            config_data = [dict_data["light_lvl"],
                           dict_data["on_time"], dict_data["off_time"]]
            json_data.close()
            return config_data
    # create new with presets if json config does not exist
    else:
        json_data = open(source, "w")
        dict_data = {"key1": val1,
                     "key2": on_time, "key3": val}
        js.dump(dict_data, json_data, indent=2)
        json_data.flush()
        json_data.close()
        return self.check_json(source)

Once a new config arrives, the program crashes with the following error:

"key1" = config_data[0]
TypeError: 'NoneType' object is not subscriptable

The error occurs even though the json does arrive with it's contents intact. I tried using multiple try and excepts as visible with the hope that it would just continue read the new config data on a new iteration. The try and except blocks haven't helped a bit and I don't know how else I could fix this,

Any input, tips/tricks are greatly appreciated

question from:https://stackoverflow.com/questions/65909784/what-is-the-correct-way-of-refreshing-a-config-file-which-is-constantly-being

Answer 1

Seems like my approach was not the correct way. The way I programmed my solution was with the idea that I can simply access my values without any worries. But that isn't the case as the file is somewhat volatile, in the sense that it could be still be in the processes of being "received".

Trying to get the values of file that still isn't fully received can only lead to problems. With that out of the way, I went back to the drawing board and revised my concept a little bit.

class FileOps:

def __init__(self, file_name: str):
    self.file_name = file_name
    self.config_name = "config.json"
    self.preset_config = {"key1" : val1, "key2" : val2, "key3" : val3}


def save_config(self):
    "saves the newer config file and deletes it in order to avoid clutter."
    if isfile(self.file_name):
        with open(self.config_name, "w") as file:
            js.dump(self.pass_new(), file, indent=2)
            remove(self.file_name) # delete the newly arrived config to prevent clutter


def read_config(self):
    """pass the existing config over to the caller if present. 
    Creates a preset config using the field "preset_config" if config.json doesn't exist """
    if not isfile(self.config_name):
        with open(self.config_name,"w") as file:
            js.dump(self.preset_config, file, indent=2)
            return self.preset_config # return the preset config for now
    with open(self.config_name, "r") as file:
        json_string = ""
        for line in file:
            json_string += line
        return js.loads(json_string)


def pass_new(self): 
    """checks if a new config is present and passes the data through to the caller"""
    if isfile(self.file_name):
        with open(self.file_name, "r") as file:
            json_string = ""
            for line in file:
                json_string += line
            return js.loads(json_string)

I seperated the file reading operations to a different file and class in order to keep everything specific and concise and I just programmed it so that a file is read line by line instead of instantly trying to parse it. This fixed my issue with TypeError: 'NoneType' object is not subscriptable as well as correctly passing the information.

The class FIleOps is given to the main class as a composition relation and called when needed through a simple object.