Memory usage when repeatedly solving a problem

Would you be able to share the complete Mosel code?

Also, which version of Xpress / Mosel are you working with?

Are you defining (sub)problems by using entities of type 'mpproblem' in your model?

Hi Susanne,

thanks for your prompt response, the code below worked for me in the past (as mentionend, based on our discussions about 15 years ago :) ). I am using Win 11 with Mosel 6.0.1:

model "ND Sandbox"
version 1.0.0
uses  "mmxprs", "mmsheet", "mmsystem", "math"
!uses "mminsight" ! Mandatory for Xpress insight
 
!this file contains definition of data elements, loading of data from spreadsheet, and internal processing of arrays for later use in models
!nomenclature used: CTR: Controls, EXT: read from spreadsheet, INT: processed internally, RES: results shown to user
 
!*********** Section DEFINITION ***********
 
parameters
	CSTR= 'Sandbox Template.xlsx'													! Name of Excel file (data template)
	!CSTRout = 'E2E Risk Guru Results.xlsx' 
	CTR_MAX_ROW=2400
	CTR_ROAD_FACT=1.18
end-parameters
 
[email protected] input
public declarations 																! Insert declarations of entities that are used by the UI here
	DC: set of string
	!Sheet DC
	EXT_DC_CITY: array(DC) of string	
	EXT_DC_COUNTRY: array(DC) of string	
	EXT_DC_LAT:  array(DC) of real													! DC latitude
	EXT_DC_LON:  array(DC) of real 													! DC longitude
	EXT_DC_STATUS: array(DC) of integer	
	
	PLANT: set of string
	!Sheet Sites
	EXT_PLANT_CITY: array(PLANT) of string	
	EXT_PLANT_COUNTRY: array(PLANT) of string	
	EXT_PLANT_LAT:  array(PLANT) of real											! DC latitude
	EXT_PLANT_LON:  array(PLANT) of real 											! DC longitude
	
	CUSTOMER: set of string															! Set of customers
	! Sheet Customer
	
	EXT_CUSTOMER_CITY: array(CUSTOMER) of string	
	EXT_CUSTOMER_COUNTRY: array(CUSTOMER) of string	
	EXT_CUSTOMER_LAT:  array(CUSTOMER) of real										! Customer latitude
	EXT_CUSTOMER_LON:  array(CUSTOMER) of real 										! Customer longitude
	EXT_CUSTOMER_CHANNEL:  array(CUSTOMER) of string 								! Customer longitude
	EXT_CUSTOMER_LT:  array(CUSTOMER) of real 										! Customer longitude
 
	PRODUCT: set of string
	! Sheet demand
	EXT_DEMAND_QUANTITY: array(CUSTOMER, PRODUCT) of real
	
end-declarations
 
declarations
	ORIG_SITES: set of string
	DEST_SITES: set of string
	INT_DISTANCE: dynamic array(ORIG_SITES, DEST_SITES) of real
	INT_TRANS_OUT_COST: real
	INT_TRANS_INB_COST: real
	INT_WHS_FIX_COST: real
	INT_PLANT_SHARE: array(PRODUCT, PLANT) of real
	INT_BIGM: array(PRODUCT) of real
	INT_MAX_DISTANCE_CUST: array(CUSTOMER) of real
	!emissions
	INT_EM_OUT: real
	INT_EM_INB: real
	INT_CERTIFICATE_COST: real
	INT_EM_WHS: real
	!Controls
	INT_WHS_STRUCTURE: string
	INT_SCEN: integer
	INT_ITER: integer
	Obj: linctr
	INT_DEMAND_MULTIPLIER: real
end-declarations
 
!*********** Section LOAD DATA AND INTERNAL PROCESSING ***********
 
procedure loaddata
	![...]
	!this intialized data
end-initializations 
 
end-procedure
 
loaddata
 
![...] data preprocessing
 
procedure ND
	
declarations
	z: array(PLANT, DC, PRODUCT) of mpvar
	x: array(DC, CUSTOMER, PRODUCT) of mpvar
	y: array(DC) of mpvar
	penalty_dist: array(CUSTOMER) of mpvar
	CO2: mpvar
end-declarations
 
forall(i in DC) y(i) is_binary
 
obj:=sum(i in DC, j in CUSTOMER, k in PRODUCT) x(i,j,k)*INT_DISTANCE(i,j)*INT_TRANS_OUT_COST + sum(i in DC) y(i)*INT_WHS_FIX_COST + sum(i in PLANT, j in DC, k in PRODUCT) z(i,j,k)*INT_DISTANCE(i,j)*INT_TRANS_INB_COST + sum(j in CUSTOMER)penalty_dist(j)*9999999 + CO2/1000*INT_CERTIFICATE_COST
 
C0:=CO2=sum(i in DC, j in CUSTOMER, k in PRODUCT) x(i,j,k)*INT_DISTANCE(i,j)*INT_EM_OUT + sum(i in PLANT, j in DC, k in PRODUCT) z(i,j,k)*INT_DISTANCE(i,j)*INT_EM_INB
 
forall(i in DC, k in PRODUCT)
	C1(i,k):=sum(m in PLANT)z(m,i,k) = sum(j in CUSTOMER)x(i,j,k)
 
forall(i in DC, k in PRODUCT)
	C2(i,k):=sum(j in CUSTOMER)x(i,j,k) <= y(i)*INT_BIGM(k)
	
forall(j in CUSTOMER, k in PRODUCT)
	C3(j,k):=sum(i in DC)x(i,j,k) >= EXT_DEMAND_QUANTITY(j,k)*INT_DEMAND_MULTIPLIER
	
forall(i in DC) do
if EXT_DC_STATUS(i)=1 then
	y(i)=1
end-if
if EXT_DC_STATUS(i)=0 then
	y(i)=0
end-if
end-do
 
forall(i in DC, j in CUSTOMER, k in PRODUCT)
	C4(i,j,k):=x(i,j,k)*INT_DISTANCE(i,j) <= x(i,j,k)*INT_MAX_DISTANCE_CUST(j) + penalty_dist(j)
	
minimize(obj)
 
![...] write output to console
 
!Release Constraints
C0:=0
Obj:=0
forall(i in DC, k in PRODUCT)
	C1(i,k):=0
forall(i in DC, k in PRODUCT)
	C2(i,k):=0	
forall(j in CUSTOMER, k in PRODUCT)
	C3(j,k):=0
forall(i in DC, j in CUSTOMER, k in PRODUCT)
	C4(i,j,k):=0
 
!unloadprob
end-procedure
 
!****************************************************************************************
 
!ENUMERATION
 
!HERE I Call ND in a loop that enumerates all combination of y(i): 12 DCs --> 2^12 combinations plus further loops for sensitivity to data such as fixed cost
 
 
end-model

I shortened [...] this a little bit. the procedure ND should be called repeatedly for enumerating results and sensitivity analysis.

Thanks

David

Your subroutine contains some unary constraints ('is_binary' and bounds on variables) that are not reset. The easiest solution would probably be to rewrite your subroutine as follows:

procedure ND
	
declarations
	z: array(PLANT, DC, PRODUCT) of mpvar
	x: array(DC, CUSTOMER, PRODUCT) of mpvar
	y: array(DC) of mpvar
	penalty_dist: array(CUSTOMER) of mpvar
	CO2: mpvar
end-declarations
 
with mpproblem do
 
forall(i in DC) y(i) is_binary
 
obj:=sum(i in DC, j in CUSTOMER, k in PRODUCT) x(i,j,k)*INT_DISTANCE(i,j)*INT_TRANS_OUT_COST + sum(i in DC) y(i)*INT_WHS_FIX_COST + sum(i in PLANT, j in DC, k in PRODUCT) z(i,j,k)*INT_DISTANCE(i,j)*INT_TRANS_INB_COST + sum(j in CUSTOMER)penalty_dist(j)*9999999 + CO2/1000*INT_CERTIFICATE_COST
 
C0:=CO2=sum(i in DC, j in CUSTOMER, k in PRODUCT) x(i,j,k)*INT_DISTANCE(i,j)*INT_EM_OUT + sum(i in PLANT, j in DC, k in PRODUCT) z(i,j,k)*INT_DISTANCE(i,j)*INT_EM_INB
 
forall(i in DC, k in PRODUCT)
	C1(i,k):=sum(m in PLANT)z(m,i,k) = sum(j in CUSTOMER)x(i,j,k)
 
forall(i in DC, k in PRODUCT)
	C2(i,k):=sum(j in CUSTOMER)x(i,j,k) <= y(i)*INT_BIGM(k)
	
forall(j in CUSTOMER, k in PRODUCT)
	C3(j,k):=sum(i in DC)x(i,j,k) >= EXT_DEMAND_QUANTITY(j,k)*INT_DEMAND_MULTIPLIER
	
forall(i in DC) do
if EXT_DC_STATUS(i)=1 then
	y(i)=1
end-if
if EXT_DC_STATUS(i)=0 then
	y(i)=0
end-if
end-do
 
forall(i in DC, j in CUSTOMER, k in PRODUCT)
	C4(i,j,k):=x(i,j,k)*INT_DISTANCE(i,j) <= x(i,j,k)*INT_MAX_DISTANCE_CUST(j) + penalty_dist(j)
	
minimize(obj)
 
![...] write output to console
 
end-do    ! Leave the subproblem (resets solution info and deletes all constraints)
end-procedure

While working on your model, in order to track the memory consumption, you could add a call to ''memoryuse" after terminating an iteration of the subroutine (remove such calls for production runs as they add to the execution time).

Hello Susannne,

actually, I had tried mpproblem before and I have done it now again.

Variables are defined locally within procedure, mpproblem loop embraces the remaining code.

But the memory usage still grows. I did a little test with the numbers generated by memoryuse per iteration of "ND":

y (memory usage) = 287,63x2 + 1375x + 4E+06, R² = 1 (x: iteration counter).

So unfortunately, this doesn't work.

CanI send you my code?

Thanks

David

I probably need to see the complete example to understand what exactly is happening.

Can you send it directly to me at FirstnameSurname_at_fico_dot_com

Summary of offline discussion: the model contains a string entity that collects output from iterations and therefore keeps growing. If such a functionality is required, the entity should be declared as 'text' - this avoids storing intermediate strings in Mosel's dictionary which caused most of the overhead.

The remark to work with 'mpproblem' for each iteration also applies.