If you are working with the FICO® Xpress Python API, have you ever wished your optimization variables could live inside your DataFrame, right alongside your data? Or that you could formulate constraints using the same Pandas features you already know — groupby(), vectorized arithmetic, aggregations — without writing a single loop?
With Xpress 9.8, this workflow becomes significantly closer to reality. While the Xpress Python API has supported Pandas for some time, version 9.8 introduces enhanced compatibility features that fundamentally change how you build optimization models in Python, bringing the world of mathematical optimization closer to the data-centric workflows that analysts and data scientists already use every day. In this post, we'll explore why this matters, what's possible, and how it can transform your modeling practice.
Pandas thinks in DataFrames and vectorized operations; traditional optimization APIs think in indexed variables and explicit loops. The enhanced integration in Xpress 9.8 builds upon existing Pandas support with a key improvement: you can now store Xpress variables and expressions directly in DataFrame columns using the xpressobj data type. This enhancement unlocks a fundamentally different way of modeling Xpress problems with an extended set of features.
Decision variables are created by adding a new column (Series) to the Dataframe as follows:
With this one step, decision variables become first-class citizens in the data structure. Now every Pandas operation that works on numeric columns — array-based arithmetic, aggregations, filtering, grouping — works equally well on optimization variables. The DataFrame is no longer just a data container; it's the foundation of your optimization model.
The element-wise multiplication (*) creates a new series of Xpress expressions — one for each stock — and sum() aggregates them. No loops, no index management, and the code reads exactly like the mathematical formulation.
Boolean indexing filters your DataFrame, and the constraint applies only to the selected variables. The modeling intent is clear, and there's no manual index tracking.
The groupby('sector') operation partitions your stocks by sector, ‘alloc_var’ selects the decision variables that belong to the specific sector, and sum() aggregates them within each group. The result is a series of constraints (one per sector) automatically generated and added to the model. No explicit iteration, no manual constraint creation for each category, and the code directly expresses the business rule: "no sector should exceed 25% of the portfolio."
One of the biggest advantages of this approach is that when your categories are not known in advance or change based on input data, this approach works seamlessly and becomes powerful. The same code works whether you have 5 sectors or 50, and adding a new category requires no model changes — it's handled automatically by the groupby() operation. The same pattern extends to multi-level grouping: groupby(['sector', 'region']) automatically generates constraints for each sector-region combination, handling multi-dimensional aggregations without nested loops.
Note: currently the groupby() method for constraint creation with Xpress is only compatible for Pandas versions < 3.0.
A portfolio optimization notebook (portfolio_pandas.ipynb) using Pandas operations with Xpress is publicly available on our GitHub page. It provides a complete example with data visualization, parameter experiments, and trade-off analysis that can be run online without any need for a local Xpress installation.
Speed through vectorization: Pandas operations are implemented in optimized C code and execute orders of magnitude faster than Python loops. Building constraints through vectorized operations means models assemble faster, especially for large-scale problems. This performance advantage compounds when experimenting with different formulations during model development.
Error reduction: Manually synchronizing indices between data and variables is a common source of errors. When variables live in the DataFrame, there's nothing to synchronize. The index alignment is automatic and guaranteed by Pandas.
Maintainability: When a business rule changes - say, you need to add a new constraint on a different category - you don't need to refactor loops or index structures. The model structure is self-documenting.
Workflow integration: Data preparation, exploratory analysis and post-processing happen in Pandas. Now optimization modeling can happen in Pandas too. The entire pipeline uses consistent data schema and API. This enables running optimization as part of ETL pipelines, making it easier to embed decision-making into data workflows.
Prototyping speed: Trying different constraint formulations, exploring scenario variations, or experimenting with model structures becomes much faster when you can leverage Pandas' expressive API. What might have taken dozens of lines with explicit loops now takes one or two lines with vectorized operations.
For even more efficiency with large-scale models, the xp.Dot() operator supports matrix operations including sparse matrices from SciPy:
This approach is orders of magnitude faster than loop-based formulations for large-scale problems, and it's particularly valuable when working with network models, assignment problems, or any application with sparse constraint matrices or multi-dimensional arrays.
1. Install or upgrade Xpress: pip install --upgrade xpress
2. Explore the examples: Clone the FICO Xpress Python notebooks repository
3. Check the documentation: Xpress Python API Reference
Alternatively, run the notebooks directly on GitHub codespaces (no local installation needed) by following these steps.
The result is code that's more concise, more readable, and more maintainable. Operations like groupby() that would have required manual loops and index management now generate constraint families automatically. Weighted-sum objectives and constraints become simple one-liners using Pandas operations. And your entire data pipeline, from preparation through optimization to post-processing, can use a single, unified data structure.
If you're building data-driven optimization models in Python, these features are worth exploring. They won't just save you time — they'll change how you think about modeling. Try the portfolio optimization example, experiment with your own problems, and share your experiences. We'd love to hear how you're using Pandas and Xpress together in your work.
With Xpress 9.8, this workflow becomes significantly closer to reality. While the Xpress Python API has supported Pandas for some time, version 9.8 introduces enhanced compatibility features that fundamentally change how you build optimization models in Python, bringing the world of mathematical optimization closer to the data-centric workflows that analysts and data scientists already use every day. In this post, we'll explore why this matters, what's possible, and how it can transform your modeling practice.
The Disconnect Between Data and Modeling
A common pattern for Python users developing data-driven optimization projects is: you have business data in a Pandas DataFrame and separate structures for storing the optimization objects. You create decision variables in separate collections, write explicit loops to match indices between your data and variables, and carefully synchronize everything when building constraints. Want to sum allocations by region? Write a loop. Need constraints by category? Another loop. The result is more code that's harder to maintain, and a workflow that feels disconnected from your data pipeline.Pandas thinks in DataFrames and vectorized operations; traditional optimization APIs think in indexed variables and explicit loops. The enhanced integration in Xpress 9.8 builds upon existing Pandas support with a key improvement: you can now store Xpress variables and expressions directly in DataFrame columns using the xpressobj data type. This enhancement unlocks a fundamentally different way of modeling Xpress problems with an extended set of features.
What Xpress 9.8 Changes: Variables as Data
Let's walk through an example: consider a portfolio optimization problem where we're selecting stocks to maximize return subject to risk limits, sector diversification, and ESG (Environmental, Social, Governance) constraints. Our data includes 100 stocks with attributes like expected return, sector classification, and ESG scores. The head of the dataframe (stocks_df) could look like:Decision variables are created by adding a new column (Series) to the Dataframe as follows:
With this one step, decision variables become first-class citizens in the data structure. Now every Pandas operation that works on numeric columns — array-based arithmetic, aggregations, filtering, grouping — works equally well on optimization variables. The DataFrame is no longer just a data container; it's the foundation of your optimization model.
The Power of Vectorized Modeling
Once variables live in your DataFrame, the full power of Pandas' vectorized operations becomes available for model building. Consider three common modeling patterns:1. Weighted sum in Objectives and Constraints
Many optimization problems involve weighted sums: maximize revenue weighted by quantities, minimize risk weighted by exposures, or enforce weighted average constraints. With variables in a DataFrame, these become one-liners:The element-wise multiplication (*) creates a new series of Xpress expressions — one for each stock — and sum() aggregates them. No loops, no index management, and the code reads exactly like the mathematical formulation.
2. Conditional Constraints with Boolean Indexing
Pandas dataframes excel at filtering data based on conditions. This translates directly to formulating constraints that apply only to subsets of variables:Boolean indexing filters your DataFrame, and the constraint applies only to the selected variables. The modeling intent is clear, and there's no manual index tracking.
3. Aggregation by Category with groupby()
Many real-world constraints involve aggregating across categories: limit spending per region, balance workload across departments, or diversify investments across sectors. Traditionally, this requires nested loops or dictionary comprehensions. With Pandas integration, it's a single line:The groupby('sector') operation partitions your stocks by sector, ‘alloc_var’ selects the decision variables that belong to the specific sector, and sum() aggregates them within each group. The result is a series of constraints (one per sector) automatically generated and added to the model. No explicit iteration, no manual constraint creation for each category, and the code directly expresses the business rule: "no sector should exceed 25% of the portfolio."
One of the biggest advantages of this approach is that when your categories are not known in advance or change based on input data, this approach works seamlessly and becomes powerful. The same code works whether you have 5 sectors or 50, and adding a new category requires no model changes — it's handled automatically by the groupby() operation. The same pattern extends to multi-level grouping: groupby(['sector', 'region']) automatically generates constraints for each sector-region combination, handling multi-dimensional aggregations without nested loops.
Note: currently the groupby() method for constraint creation with Xpress is only compatible for Pandas versions < 3.0.
A portfolio optimization notebook (portfolio_pandas.ipynb) using Pandas operations with Xpress is publicly available on our GitHub page. It provides a complete example with data visualization, parameter experiments, and trade-off analysis that can be run online without any need for a local Xpress installation.
Why This Matters for Modelers
The Pandas integration isn't just about writing less code, it fundamentally changes the modeling experience in several important ways:Speed through vectorization: Pandas operations are implemented in optimized C code and execute orders of magnitude faster than Python loops. Building constraints through vectorized operations means models assemble faster, especially for large-scale problems. This performance advantage compounds when experimenting with different formulations during model development.
Error reduction: Manually synchronizing indices between data and variables is a common source of errors. When variables live in the DataFrame, there's nothing to synchronize. The index alignment is automatic and guaranteed by Pandas.
Maintainability: When a business rule changes - say, you need to add a new constraint on a different category - you don't need to refactor loops or index structures. The model structure is self-documenting.
Workflow integration: Data preparation, exploratory analysis and post-processing happen in Pandas. Now optimization modeling can happen in Pandas too. The entire pipeline uses consistent data schema and API. This enables running optimization as part of ETL pipelines, making it easier to embed decision-making into data workflows.
Prototyping speed: Trying different constraint formulations, exploring scenario variations, or experimenting with model structures becomes much faster when you can leverage Pandas' expressive API. What might have taken dozens of lines with explicit loops now takes one or two lines with vectorized operations.
When to Use This Approach
The Pandas integration is most valuable when:- Your problem data is naturally tabular (that is, information about a specific model object type can be represented as table with columns corresponding to properties or attributes)
- You need constraints that aggregate by categories or groups
- The categories might vary with input data
- You're building multiple related models or variants
- You want your optimization code to integrate seamlessly with data preparation and analysis
Beyond Pandas: NumPy Arrays and Matrix Operations
The integration extends to NumPy arrays as well, which is essential for models with matrix-based formulations, such as transportation problems, multi-commodity flows, or any problem naturally expressed with 2D decision arrays. Xpress uses xpress.ndarray, a subclass of numpy.ndarray that customizes operators for constraint creation:For even more efficiency with large-scale models, the xp.Dot() operator supports matrix operations including sparse matrices from SciPy:
This approach is orders of magnitude faster than loop-based formulations for large-scale problems, and it's particularly valuable when working with network models, assignment problems, or any application with sparse constraint matrices or multi-dimensional arrays.
Getting Started
The enhanced Pandas compatibility is available in Xpress 9.8 and later. To try it:1. Install or upgrade Xpress: pip install --upgrade xpress
2. Explore the examples: Clone the FICO Xpress Python notebooks repository
3. Check the documentation: Xpress Python API Reference
Alternatively, run the notebooks directly on GitHub codespaces (no local installation needed) by following these steps.
Conclusion
The enhanced Pandas compatibility in Xpress 9.8 represents more than an incremental API improvement — it's a shift in how we think about writing optimization code in Python. By allowing variables and expressions to live directly in DataFrames, Xpress enables a more natural, data-centric approach to model building that aligns with how data scientists and analysts already work.The result is code that's more concise, more readable, and more maintainable. Operations like groupby() that would have required manual loops and index management now generate constraint families automatically. Weighted-sum objectives and constraints become simple one-liners using Pandas operations. And your entire data pipeline, from preparation through optimization to post-processing, can use a single, unified data structure.
If you're building data-driven optimization models in Python, these features are worth exploring. They won't just save you time — they'll change how you think about modeling. Try the portfolio optimization example, experiment with your own problems, and share your experiences. We'd love to hear how you're using Pandas and Xpress together in your work.
Updated 8 months ago
Version 3.0