Handle process externally in ModelicaSystem.simulate and ModelicaSystem.linearized

[kiroorooki]

Hello!
I've noticed that we could run simulations in parallel if we handled the process outside the simulate() method.

It is useful to me because I need to run a log of simulations, and running them in parallel saves me a lot of time.
In order to keep the same functionality, I added a boolean handleProcessOutside, and if set to true, I don't wait() and terminate() the process, I simply return it, so the user can handle the process themselves. That way people can use the library the same way, or get extended functionality if needed with just a boolean.

In my case, I run all my simulations, and I make sure all of them are done later when I save my results (opening .mat file, converting things to csv...).

I just realized the library changed quite a bit, but I believe the same functionality can still be implemented in _run_cmd and propagated to the methods using it (simulate and linearized).

Thanks for your hard work, I hope my suggestion makes sense!

This is how I currently implemented it in my version for reference (quite outdated):
`

def simulate(self, resultfile=None, simflags=None, verbose=True, handleProcessOutside=False):  # 11
    """
    This method simulates model according to the simulation options.
    usage
    >>> simulate()
    >>> simulate(resultfile="a.mat")
    >>> simulate(simflags="-noEventEmit -noRestart -override=e=0.3,g=10) set runtime simulation flags
    """
    if(resultfile is None):
        r=""
        self.resultfile = os.path.join(self.tempdir, self.modelName + "_res.mat").replace("\\", "/")
    else:
        r=" -r=" + resultfile
        self.resultfile = resultfile

    # allow runtime simulation flags from user input
    if(simflags is None):
        simflags=""
    else:
        simflags=" " + simflags

    overrideFile = os.path.join(self.tempdir, '{}.{}'.format(self.modelName + "_override", "txt")).replace("\\", "/")
    if (self.overridevariables or self.simoptionsoverride):
        tmpdict=self.overridevariables.copy()
        tmpdict.update(self.simoptionsoverride)
        # write to override file
        file = open(overrideFile, "w")
        for (key, value) in tmpdict.items():
            name = key + "=" + value + "\n"
            file.write(name)
        file.close()
        override =" -overrideFile=" + overrideFile
    else:
        override =""

    if (self.inputFlag):  # if model has input quantities
        for i in self.inputlist:
            val=self.inputlist[i]
            if(val==None):
                val=[(float(self.simulateOptions["startTime"]), 0.0), (float(self.simulateOptions["stopTime"]), 0.0)]
                self.inputlist[i]=[(float(self.simulateOptions["startTime"]), 0.0), (float(self.simulateOptions["stopTime"]), 0.0)]
            if float(self.simulateOptions["startTime"]) != val[0][0]:
                print("!!! startTime not matched for Input ",i)
                return
            if float(self.simulateOptions["stopTime"]) != val[-1][0]:
                print("!!! stopTime not matched for Input ",i)
                return
            if val[0][0] < float(self.simulateOptions["startTime"]):
                print('Input time value is less than simulation startTime for inputs', i)
                return
        self.createCSVData()  # create csv file
        csvinput=" -csvInput=" + self.csvFile
    else:
        csvinput=""

    if (platform.system() == "Windows"):
        getExeFile = os.path.join(self.tempdir, '{}.{}'.format(self.modelName, "exe")).replace("\\", "/")
    else:
        getExeFile = os.path.join(self.tempdir, self.modelName).replace("\\", "/")
    currentDir = os.getcwd()
    if (os.path.exists(getExeFile)):
        cmd = getExeFile + override + csvinput + r + simflags
        #print(cmd)
        os.chdir(self.tempdir)
        if (platform.system() == "Windows"):
            omhome = os.path.join(os.environ.get("OPENMODELICAHOME"))
            dllPath = os.path.join(omhome, "bin").replace("\\", "/") + os.pathsep + os.path.join(omhome, "lib/omc").replace("\\", "/") + os.pathsep + os.path.join(omhome, "lib/omc/cpp").replace("\\", "/") +  os.pathsep + os.path.join(omhome, "lib/omc/omsicpp").replace("\\", "/")
            my_env = os.environ.copy()
            my_env["PATH"] = dllPath + os.pathsep + my_env["PATH"]
            if not verbose:
                p = subprocess.Popen(cmd, env=my_env, stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT)
            else:
                p = subprocess.Popen(cmd, env=my_env)
            if not handleProcessOutside:
                p.wait()
                p.terminate()
        else:
            if not verbose:
                p = subprocess.Popen(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT)
            else:
                p = subprocess.Popen(cmd)
        os.chdir(currentDir)
        self.simulationFlag = True
        if handleProcessOutside:
            return p
    else:
        raise Exception("Error: Application file path not found: " +  getExeFile)

`

[casella]

@arun3688, @adrpo what do you think?

[arun3688]

@kiroorooki The suggestion looks fine, can you update your OMPython to the latest master and make a PR with your changes

[adrpo]

I guess we could also add some facility to OMPython similar to what we have for Library Testing in parallel. Basically we first build the model, then run it in parallel with the Python Parallel jobs feature.

[adrpo]

The only problem with the suggestion might be conflicting generated files and result file names. If those are different for each parallel simulate command then it should work fine.

[arun3688]

ModelicaSystem() creates a separate working directory for each session, so there should not be a problem in conflicting files with same names. We can use python's Multiprocessing library to achieve parallel simulation

[casella]

This would be very interesting, e.g., to run massively parallel black-box optimization on the cloud.

[FarzanehMousaviM]

Hi everyone! :) This is what we tried to achieve with this change:

We first needed to build multiple versions of a model (tweaked) in different directories (In order to have multiple instances of a model using different parameter values).
This is why we requested the feature to build in a specific folder, as we re-use this path later to simulate a specific instance of the model. (#204)

Later we needed to simulate each model instance multiple times with different parameter values (using mod.setParameters()).
However, this step needed to be parallelized. This is why we made the current request.

We are now simulating each model instance (sequentially) multiple times with different parameter values (parallelized).
As the next step, it would be great if we could also parallelize at the instance level to speed up the simulation even more.

--

This is our current approach (with the current feature request), hopefully it brings something to the table.
Feel free to ask more questions if anything is unclear.

mod.simulate() returns a process instead of waiting and terminating the process.
p = mod.simulate(resultfile = RESULT_FILEPATH + resultFileName + ".mat", simflags = simflag)

One issue we had was that a process could fail (for no clear reason), so we added this loop to retry up to 4 times:

while not os.path.exists(myFileName) and retryIndex < 4:
    poll = p.poll()
    if poll is None: #if process is still running
        if os.path.exists(myFileName):
           break
    else:
        if os.path.exists(myFileName):
           break
        p = mod.simulate(resultfile = RESULT_FILEPATH + resultFileName + ".mat", simflags = simflag)
        retryIndex += 1

We then check that the processes are done before post-processing the results:

if omcProcess != None:
        poll = omcProcess.poll()
        if poll is None:
            omcProcess.wait()
            omcProcess.terminate()

    while(True):
        try:
            results = DyMat.DyMatFile(RESULT_FILEPATH + resultFileName + ".mat")
        except:
            continue
        break
postProcess(results)

[syntron]

@FarzanehMousaviM , @kiroorooki could you please test if the changes in PR #312 help you? These should enable you to run a DoE based on OMPython.ModelicaSystemDoE ...