The FICO® Platform – DecisionOps Capability gives decision science and data science teams the same type of structured workflow and tooling that DevOps gives software teams and MLOps gives machine learning teams. Without DecisionOps, model developers are often required to write custom scripts and rely heavily on engineering teams to operationalize their models, which can become a bottleneck as models and use cases scale and evolve.
In this introduction, we’ll walk through how to leverage DecisionOps alongside a simple assignment model through the lens of a decision model developer. All of the concepts we’ll cover apply to connecting prototype or production-grade decision models for use cases in supply chain and logistics, healthcare, energy, retail, and beyond. We’ll demonstrate how DecisionOps accelerates time to value by taking a developer-authored decision model from a local prototype to a live decision service that teams can confidently test, collaborate on, and manage at scale.
Problem statement for our assignment example
Let's imagine we're school administrators planning for the upcoming school year. We have a data science team that’s responsible for handling all assignment planning using an in-house built decision model. The objective of our decision model is to maximize the number of students assigned to classrooms while accounting for student preferences, per-student costs, classroom capacity, and extended-day eligibility and requests. In addition to planning class assignments, we want to be prepared for situations that may arise throughout the course of the school year and have posed the following questions to the data science team.
Scenario 1: A new honor student enrolls mid-term and they cannot be enrolled in class with their sibling. How does accommodating this request impact the overall class assignments?
Scenario 2: The school recently launched a program to allow certain classrooms to have extended days. However, the school is short staffed and we’re considering ending the extended day program early to compensate. What are the impacts of ending an hour or two earlier than normal?
We’ll follow along as the data science team uses the FICO® Platform – DecisionOps Capability to answer these questions with custom metrics in shareable visualizations.
To make the model connection process smooth, FICO® Platform - DecisionOps Capability includes an onboarding experience that guides users through each step. As we answer four simple questions about the model, the tool first scaffolds the app.
Voilà! This is all the information required to build the structure of the app. If we open the newly created app in our working directory, we’ll see the following:
The app.yaml is the app manifest which holds the instructions for deploying and running the application, and the main.py is the entrypoint. (If you’re working with your own model rather than an example, the main.py is where you’d place your model code.)
This project structure follows best practices for operationalizing decision models as microservices. Separating the model input (e.g., input.json) from the decision code (e.g., main.py) allows for faster troubleshooting, simpler iteration, and streamlined code maintenance.
Here’s the result:
In the run result, alongside the other metrics, we can see that 25 students were successfully assigned to classrooms. The result also includes empty sections that will populate for configurations used and experiment metadata (which we’ll see later in this post).
For now, since we’ve verified the app is running correctly, we can push the app up as a hosted microservice.
Behind the scenes, it will build the application, bundle the dependencies, copy the files, compress them, package them, and push up the app.
Ta-da! We now have a live decision service accessible via a unique API endpoint.
Each time we push the app, a new version (a snapshot of the app’s executable code) is created. This functionality is important for tracking updates and creating a system of record for your decision app. Here’s a preview of what that looks like in the FICO® Platform – DecisionOps Capability UI.
Screenshot 1 Decision App Versions
Optionally, we can link the version to an instance, which is a named endpoint or environment (e.g., staging and production). We can then promote a new version to an instance the same way software developers promote a build in a software release pipeline.
Screenshot 2 Decision App Instances
The next CLI prompt will ask if we’d like to make a remote run. We’ll use the same input as we used for the local run, but now we’ll use the API to run the decision app remotely.
Fantastic! We successfully made a remote run of our decision app.
Screenshot 3 Decision App Overview
As we make more runs and perform experiments, we’ll see results and metadata populate this overview page.
Screenshot 4 Decision App Runs
Here we can access the input, output, user-defined metrics, metadata, logs, and visuals for that run. The summary tab outlines the run metrics and run configuration. Again, we see the 25 assigned students, 11 extended day requests granted, and 12 extended day requests. In the run configuration, we see the values for options such as `extended_day_time`.
Screenshot 5 Decision App Run Details View
Screenshot 6 Decision App Run Details View with Configuration
The output tab surfaces the solution which can be viewed in a table or raw format. In this case, the table displays which students were successfully assigned to which classrooms.
Screenshot 7 Decision App Run Details View with Output
In addition to metadata and logs, the run details view includes custom visual assets tied to each run. Here we see available vs enrolled seats for each classroom.
Screenshot 8 Decision App Run Details View with Visual Asset
The run details view provides access to the details a model developer would need to see understand results, troubleshoot issues, or replay a run.
Cloning a run
Now let’s make a run directly from the UI. We can select a run and replay it as is or quickly tweak a configurable option (e.g., extended-day bonus) or edit the input.
Screenshot 9 Creating a New Run
Cloning runs is helpful for recreating results in addition to doing lightweight run comparisons (as we’ll see in the next step).
We can answer this by comparing two runs (without the sibling vs with the sibling).
Screenshot 10 Cloning a Run
Then we’ll navigate to the ‘Input’ tab. We can edit the input directly in the UI to add a new student and a “separation” constraint preventing two specific students (siblings) from being placed in the same class. Below we’re updating a raw JSON file, but CSVs (with tabular data) can also be edited directly in the UI.
Screenshot 11 Editing Run Input to Add Student
We’ll click “Start run” to kick off the new run using the editing input file. Next, once the run has succeeded, we will select the two runs and click “Compare runs” to see the results.
Screenshot 12 Selecting Runs to Compare
This will allow us to compare the metrics, assets, and configuration for each run side by side.
Screenshot 13 Comparing Two Run Results
Comparing runs directly allows us to easily answer, “What did this change impact?” for any decision model.
We’ll use a scenario test to answer this question. A scenario test will allow us to make multiple configuration changes (ending the extended day at different times), execute runs in parallel, and then analyze the results to understand the impacts.
Next, we’ll build the scenario test by first selecting our input set (Input Set 1) and instance (Staging). Then we’ll define a scenario by varying extended-day end-time configuration across multiple values (e.g., 3pm/4pm/5pm/6pm). This one scenario test will fire off eight parallel runs (all configs × all inputs).
Screenshot 14 Creating a Scenario Test
The test runs every configuration + input combination in parallel and manages the orchestration for you, so getting a comprehensive answer is faster and simpler to analyze.
Screenshot 15 Results of Scenario Test
Here we see that the average number of extended days granted decreases as the extended day time decreases. This indicates that (as expected) the earlier the school day ends the fewer extended day requests we can grant. In this case and others, a scenario test is an efficient method for answering, “What are the impacts across a whole range of a variables?” for our assignment model.
In this introduction, we’ll walk through how to leverage DecisionOps alongside a simple assignment model through the lens of a decision model developer. All of the concepts we’ll cover apply to connecting prototype or production-grade decision models for use cases in supply chain and logistics, healthcare, energy, retail, and beyond. We’ll demonstrate how DecisionOps accelerates time to value by taking a developer-authored decision model from a local prototype to a live decision service that teams can confidently test, collaborate on, and manage at scale.
Key Takeaways
- A microservice approach accelerates dev-authored decision model iteration: The CLI scaffolds a repeatable app structure that prepares decision models to ship as microservices.
- A system of record streamlines troubleshooting and unlocks agentic workflows: A versioned snapshot of the app is created with every model push, and every run is logged with metadata, user-defined metrics, and more for full transparency.
- Repeatable offline and online testing enables confident rollbacks and rollouts: Features such as scenario testing and shadow testing allow teams to answer stakeholder questions faster and identify potential impacts before promoting new versions.
- Configurable environments scale model value across the organization: Environments can be individually created and configured to serve decision models as they evolve for richer testing and smarter management.
Problem statement for our assignment example
Let's imagine we're school administrators planning for the upcoming school year. We have a data science team that’s responsible for handling all assignment planning using an in-house built decision model. The objective of our decision model is to maximize the number of students assigned to classrooms while accounting for student preferences, per-student costs, classroom capacity, and extended-day eligibility and requests. In addition to planning class assignments, we want to be prepared for situations that may arise throughout the course of the school year and have posed the following questions to the data science team.
Scenario 1: A new honor student enrolls mid-term and they cannot be enrolled in class with their sibling. How does accommodating this request impact the overall class assignments?
Scenario 2: The school recently launched a program to allow certain classrooms to have extended days. However, the school is short staffed and we’re considering ending the extended day program early to compensate. What are the impacts of ending an hour or two earlier than normal?
We’ll follow along as the data science team uses the FICO® Platform – DecisionOps Capability to answer these questions with custom metrics in shareable visualizations.
Connecting your model
We’ll make our start in two steps: 1) Give our assignment model a standard app structure and 2) Validate everything worked with a local run. As we do this, we’ll be wiring up connection points between our local (or Git-managed) decision model and the FICO® Platform - DecisionOps Capability. This gives us the foundation to really fly with repeatable model runs, tests, and deployment patterns down the line.Structuring the decision app
First, we'll use the command line interface (CLI) for structuring our decision model. The CLI can also be used for running models (locally and remotely), performing experiments, pushing up new model versions, and more.To make the model connection process smooth, FICO® Platform - DecisionOps Capability includes an onboarding experience that guides users through each step. As we answer four simple questions about the model, the tool first scaffolds the app.
- Language
- Question: Which type (language) do you want to work with?
- Answer: python
- Example model vs. existing model
- Question: How do you want to start working with Nextmv?
- Answer: Example - class-room assignment
- Content format
- Question: Which type of I/O (input/output) content format do you prefer for your app?
- Answer: json
- Initialization path
- Question: Where would you like to initialize your Nextmv application template?
- Answer:[working directory or other path]
Voilà! This is all the information required to build the structure of the app. If we open the newly created app in our working directory, we’ll see the following:
The app.yaml is the app manifest which holds the instructions for deploying and running the application, and the main.py is the entrypoint. (If you’re working with your own model rather than an example, the main.py is where you’d place your model code.)
This project structure follows best practices for operationalizing decision models as microservices. Separating the model input (e.g., input.json) from the decision code (e.g., main.py) allows for faster troubleshooting, simpler iteration, and streamlined code maintenance.
Making a local run
Now that our assignment app is built, let’s make a run locally to ensure we’re getting expected results. We’ll continue to follow the CLI prompts to start a local run, select the input (provided in the example app), and get the results of the local run.Here’s the result:
In the run result, alongside the other metrics, we can see that 25 students were successfully assigned to classrooms. The result also includes empty sections that will populate for configurations used and experiment metadata (which we’ll see later in this post).
For now, since we’ve verified the app is running correctly, we can push the app up as a hosted microservice.
Pushing the decision app
In order to push the app and run it remotely, we’ll configure the CLI with our API key. Then we’ll continue to follow the CLI prompts.Behind the scenes, it will build the application, bundle the dependencies, copy the files, compress them, package them, and push up the app.
Ta-da! We now have a live decision service accessible via a unique API endpoint.
Each time we push the app, a new version (a snapshot of the app’s executable code) is created. This functionality is important for tracking updates and creating a system of record for your decision app. Here’s a preview of what that looks like in the FICO® Platform – DecisionOps Capability UI.
Screenshot 1 Decision App Versions
Optionally, we can link the version to an instance, which is a named endpoint or environment (e.g., staging and production). We can then promote a new version to an instance the same way software developers promote a build in a software release pipeline.
Screenshot 2 Decision App Instances
The next CLI prompt will ask if we’d like to make a remote run. We’ll use the same input as we used for the local run, but now we’ll use the API to run the decision app remotely.
Fantastic! We successfully made a remote run of our decision app.
Exploring the decision app with FICO
We can now see the app live in the FICO® Platform – DecisionOps Capability UI. In the app overview, we see the app name, the latest updates that were, last pushed executable (and who pushed it), custom run metrics, and more.Screenshot 3 Decision App Overview
As we make more runs and perform experiments, we’ll see results and metadata populate this overview page.
Run history and details
Scrolling down, we see a section that shows recent runs. Let’s click into the latest run.Screenshot 4 Decision App Runs
Here we can access the input, output, user-defined metrics, metadata, logs, and visuals for that run. The summary tab outlines the run metrics and run configuration. Again, we see the 25 assigned students, 11 extended day requests granted, and 12 extended day requests. In the run configuration, we see the values for options such as `extended_day_time`.
Screenshot 5 Decision App Run Details View
Screenshot 6 Decision App Run Details View with Configuration
The output tab surfaces the solution which can be viewed in a table or raw format. In this case, the table displays which students were successfully assigned to which classrooms.
Screenshot 7 Decision App Run Details View with Output
In addition to metadata and logs, the run details view includes custom visual assets tied to each run. Here we see available vs enrolled seats for each classroom.
Screenshot 8 Decision App Run Details View with Visual Asset
The run details view provides access to the details a model developer would need to see understand results, troubleshoot issues, or replay a run.
Cloning a run
Now let’s make a run directly from the UI. We can select a run and replay it as is or quickly tweak a configurable option (e.g., extended-day bonus) or edit the input.
Screenshot 9 Creating a New Run
Cloning runs is helpful for recreating results in addition to doing lightweight run comparisons (as we’ll see in the next step).
Comparing runs
With our assignment app running remotely, we can now answer questions for the school administrators. Let’s start with scenario 1: A new honor student enrolls mid term and they cannot be enrolled in class with their sibling. How does accommodating this request impact the overall class assignments?We can answer this by comparing two runs (without the sibling vs with the sibling).
Editing the input
We’ll clone an existing run by selecting the “+” button at the top right. Because we’re editing directly from an existing run rather than building a new input from scratch, we can isolate the effects of one change (adding a sibling to the roster).Screenshot 10 Cloning a Run
Then we’ll navigate to the ‘Input’ tab. We can edit the input directly in the UI to add a new student and a “separation” constraint preventing two specific students (siblings) from being placed in the same class. Below we’re updating a raw JSON file, but CSVs (with tabular data) can also be edited directly in the UI.
Screenshot 11 Editing Run Input to Add Student
We’ll click “Start run” to kick off the new run using the editing input file. Next, once the run has succeeded, we will select the two runs and click “Compare runs” to see the results.
Screenshot 12 Selecting Runs to Compare
This will allow us to compare the metrics, assets, and configuration for each run side by side.
Analyzing results
Below we see the metrics produced by each run. The run without the sibling resulted in 25 students being successfully assigned, and the run with the sibling resulted in 26 students being successfully assigned. This means the sibling was able to be assigned to a separate honors class and that adding the sibling did not negatively impact assigned students.Screenshot 13 Comparing Two Run Results
Comparing runs directly allows us to easily answer, “What did this change impact?” for any decision model.
Performing a scenario test
Let’s move to scenario 2: The school is short staffed and they’re considering ending the extended day program early to compensate. What are the impacts of ending an hour or two earlier than normal?We’ll use a scenario test to answer this question. A scenario test will allow us to make multiple configuration changes (ending the extended day at different times), execute runs in parallel, and then analyze the results to understand the impacts.
Building the test
Before we run this test, we need input data to use. We’ll create an input set (a saved collection of inputs) that you can reuse across experiments in the future instead of re-uploading test data each time. For this input set, we’ll use historical inputs from the past 3 months. We could also select input from specific runs or upload inputs directly.Next, we’ll build the scenario test by first selecting our input set (Input Set 1) and instance (Staging). Then we’ll define a scenario by varying extended-day end-time configuration across multiple values (e.g., 3pm/4pm/5pm/6pm). This one scenario test will fire off eight parallel runs (all configs × all inputs).
Screenshot 14 Creating a Scenario Test
The test runs every configuration + input combination in parallel and manages the orchestration for you, so getting a comprehensive answer is faster and simpler to analyze.
Exploring scenario test results
We get the results in an interactive pivot table: extended-day end time vs. percentage of extended-day requests granted. The pivot table aggregates results across every run in the scenario test automatically, so you can see the pattern across all configs and inputs at a glance.Screenshot 15 Results of Scenario Test
Here we see that the average number of extended days granted decreases as the extended day time decreases. This indicates that (as expected) the earlier the school day ends the fewer extended day requests we can grant. In this case and others, a scenario test is an efficient method for answering, “What are the impacts across a whole range of a variables?” for our assignment model.
Why this DecisionOps workflow matters
Without DecisionOps, navigating the path from dev-authored decision model prototype to operationalized decision service can be unpredictable, with many points of friction encountered along the way. DecisionOps empowers data science and decision science teams to move swiftly with a repeatable workflow that accelerates development, streamlines testing, and reduces engineering effort. Rooted in collaboration and transparency, DecisionOps removes friction so organizations can operationalize models safely and see the value of their dev-authored decision models faster.Updated 25 days ago
Version 2.0