Champion/Challenger Testing

Document created by Makenna.Brei Advocate on Aug 16, 2017Last modified by Makenna.Brei Advocate on Oct 9, 2017
Version 6Show Document
  • View in full screen mode

A long standing feature of TRIAD is the ability of users to introduce a new strategy, the Challenger, to a user-specified portion of the portfolio, while the remaining majority of the portfolio continues to be treated by an existing strategy, the Champion. Through this process TRIAD enables comparison of multiple competing strategies in a statistically valid way so users can determine which strategy produces the best results.

 

  • Prior to introducing a Challenger strategy to production, users should first test the strategy using the Estimator to validate that the treatment outcomes are as expected.
  • Champion/Challenger testing drives measurable improvement in portfolio management decisions for any licensed decision area within a Strategic Portfolio ID (SPID).
  • Multiple Challengers can be assigned to a SPID at the same time to investigate multiple strategy concepts within a single decision area, or to test a Challenger in multiple decision areas at the same time.
  • For a statistically significant test, the Sample Group defined within the SPID should have at least 35,000 customers/accounts; if introducing an aggressive Challenger to production, it may be better to test on a smaller Sample Group of a few hundred customers/accounts as a litmus test.
  • Some, if not many assigned Challengers will not perform as well as the Champion strategy, but unsuccessful Challengers should provide some directional guidance for a better iteration in the next round of Champion/Challenger testing.
  • Work with your FICO Strategy Consultant regarding the optimal size of your Sample Groups, and for input on constructing or revising Challengers that align with the strategies and policies of your organization.

Attachments

    Outcomes