Ability to control cplex memory consumption

Nowadays optimization runs are more and more conducted in the cloud, in kubernetes environments. In kubernetes, the processes run inside containers in what are called pods. To prevent a pod to be killed when the node (machine) capacity is exceeded, one needs to set a cpu and memory limit that the process inside the pod does not exceed; Typically CPU and memory limits. If a pod exceeds the given limits, the kubernetes orchestrator kills it.

The problem with cplex is that there is no way to control its cpu and memory consumption so that they do not exceed the given limit. Even setting various cplex parameters like IloCplex.Param.Threads, IloCplex.Param.WorkMem, Emphasis.Memory and MIP.Limits.TreeMemory does not enforce a global limit. Indeed, only the branch&bound procedure seems affected by these parameters, not presolve, probing, lexicographic, etc. The fact that lexicographic optimization explodes in terms of memory consumption is particularly impacting. I even opened a bug ticket because I have a case where settings these parameters to get a 1GB memory limit results into a 8GB memory consumption during the branch&bound.

To workaround this issue, we set the cplex parameters to try to control the memory consumption and then set a pod limit far above the cplex limit to keep some slack. But still some optimization runs are randomly killed when the datasets are in such a way that cplex consumes more memory than usual. The impact is very significant as cplex cannot be used in a reliable manner in production in a cloud environment.