Drools Planner supports several optimization algorithms (as solver phases), but you're probably wondering which is the best one? Although some optimization algorithms generally perform better then others, it really depends on your problem domain. Most solver phases have settings which can be tweaked. Those settings can influence the results a lot, although most solver phases work pretty well out-of-the-box.
Luckily, Drools Planner includes a benchmarker, which allows you to play out different solver phases with different settings against each other, so you can pick the best configuration for your planning problem.
The benchmarker is current in the drools-planner-core modules, but it requires an extra dependency on the JFreeChart library.
If you use maven, add a dependency in your pom.xml file:
<dependency>
<groupId>jfree</groupId>
<artifactId>jfreechart</artifactId>
<version>1.0.13</version>
</dependency>This is similar for gradle, ivy and buildr.
If you use ANT, you've probably already copied the required jars from the download zip's
binaries directory.
You can build a PlannerBenchmark instance with the XmlPlannerBenchmarkFactory.
Configure it with a benchmark configuration xml file:
XmlPlannerBenchmarkFactory plannerBenchmarkFactory = new XmlPlannerBenchmarkFactory();
plannerBenchmarkFactory.configure("/org/drools/planner/examples/nqueens/benchmark/nqueensBenchmarkConfig.xml");
PlannerBenchmark plannerBenchmark = benchmarkFactory.buildPlannerBenchmark();
plannerBenchmark.benchmark();
A basic benchmark configuration file looks something like this:
<?xml version="1.0" encoding="UTF-8"?>
<plannerBenchmark>
<benchmarkDirectory>local/data/nqueens</benchmarkDirectory>
<warmUpSecondsSpend>30</warmUpSecondsSpend>
<inheritedSolverBenchmark>
<problemBenchmarks>
<xstreamAnnotatedClass>org.drools.planner.examples.nqueens.domain.NQueens</xstreamAnnotatedClass>
<inputSolutionFile>data/nqueens/unsolved/unsolvedNQueens32.xml</inputSolutionFile>
<inputSolutionFile>data/nqueens/unsolved/unsolvedNQueens64.xml</inputSolutionFile>
<problemStatisticType>BEST_SOLUTION_CHANGED</problemStatisticType>
</problemBenchmarks>
<solver>
<solutionClass>org.drools.planner.examples.nqueens.domain.NQueens</solutionClass>
<planningEntityClass>org.drools.planner.examples.nqueens.domain.Queen</planningEntityClass>
<scoreDrl>/org/drools/planner/examples/nqueens/solver/nQueensScoreRules.drl</scoreDrl>
<scoreDefinition>
<scoreDefinitionType>SIMPLE</scoreDefinitionType>
</scoreDefinition>
<termination>
<maximumSecondsSpend>20</maximumSecondsSpend>
</termination>
<constructionHeuristic>
<constructionHeuristicType>FIRST_FIT_DECREASING</constructionHeuristicType>
<constructionHeuristicPickEarlyType>FIRST_LAST_STEP_SCORE_EQUAL_OR_IMPROVING</constructionHeuristicPickEarlyType>
</constructionHeuristic>
</solver>
</inheritedSolverBenchmark>
<solverBenchmark>
<name>Solution tabu</name>
<solver>
<localSearch>
<selector>
<moveFactoryClass>org.drools.planner.examples.nqueens.solver.move.factory.RowChangeMoveFactory</moveFactoryClass>
</selector>
<acceptor>
<solutionTabuSize>1000</solutionTabuSize>
</acceptor>
<forager>
<pickEarlyType>NEVER</pickEarlyType>
</forager>
</localSearch>
</solver>
</solverBenchmark>
<solverBenchmark>
<name>Move tabu</name>
<solver>
<localSearch>
<selector>
<moveFactoryClass>org.drools.planner.examples.nqueens.solver.move.factory.RowChangeMoveFactory</moveFactoryClass>
</selector>
<acceptor>
<moveTabuSize>5</moveTabuSize>
</acceptor>
<forager>
<pickEarlyType>NEVER</pickEarlyType>
</forager>
</localSearch>
</solver>
</solverBenchmark>
<solverBenchmark>
<name>Property tabu</name>
<solver>
<localSearch>
<selector>
<moveFactoryClass>org.drools.planner.examples.nqueens.solver.move.factory.RowChangeMoveFactory</moveFactoryClass>
</selector>
<acceptor>
<propertyTabuSize>5</propertyTabuSize>
</acceptor>
<forager>
<pickEarlyType>NEVER</pickEarlyType>
</forager>
</localSearch>
</solver>
</solverBenchmark>
</plannerBenchmark>
This PlannerBenchmark will try 3 configurations (1 solution tabu, 1 move tabu and 1 property tabu) on 2 data sets
(32 and 64 queens), so it will run 6 solvers.
Every solverBenchmark entity contains a solver configuration (for example with a local
search solver phase) and one or more inputSolutionFile elements. It will run the solver
configuration on each of those unsolved solution files. A name is optional and generated if
absent.
The common part of multiple solverBenchmark entities can be extracted to the
inheritedSolverBenchmark entity, but that can still be overwritten per
solverBenchmark entity. Note that inherited solver phases such as
<constructionHeuristic> or <localSearch> are not overwritten but
instead are added to the head of the solver phases list.
You need to specify a benchmarkDirectory (relative to the working directory). The best
solution of each Solver run and a handy overview HTML file will be written in that
directory.
The benchmarker needs to be able to read the input files to contain a Solution write
the best Solution of each benchmark to an output file. For that it uses a class that
implements the ProblemIO interface:
public interface ProblemIO {
String getFileExtension();
Solution read(File inputSolutionFile);
void write(Solution solution, File outputSolutionFile);
}By default, a XStreamProblemIO instance is used and you just need to configure your
Solution class as being annotated with XStream:
<problemBenchmarks>
<xstreamAnnotatedClass>org.drools.planner.examples.nqueens.domain.NQueens</xstreamAnnotatedClass>
<inputSolutionFile>data/nqueens/unsolved/unsolvedNQueens32.xml</inputSolutionFile>
...
</problemBenchmarks>However, your input files need to have been written with a XStreamProblemIO instance.
Alternatively, you can implement your own ProblemIO implementation and configure it
with the problemIOClass element:
<problemBenchmarks>
<problemIOClass>org.drools.planner.examples.machinereassignment.persistence.MachineReassignmentProblemIO</problemIOClass>
<inputSolutionFile>data/machinereassignment/input/model_a1_1.txt</inputSolutionFile>
...
</problemBenchmarks>Without a warm up, the results of the first (or first few) benchmarks are not reliable, because they will have lost CPU time on hotspot JIT compilation (and possibly DRL compilation too).
The avoid that distortion, the benchmarker can run some of the benchmarks for a specified amount of time, before running the real benchmarks. Generally, a warm up of 30 seconds suffices:
<plannerBenchmark> ... <warmUpSecondsSpend>30</warmUpSecondsSpend> ... </plannerBenchmark>
The benchmarker supports outputting statistics as graphs and CSV (comma separated values) files to the
benchmarkDirectory.
To configure graph and CSV output of a statistic, just add a problemStatisticType
line:
<plannerBenchmark>
<benchmarkDirectory>local/data/nqueens/solved</benchmarkDirectory>
<inheritedSolverBenchmark>
<problemBenchmarks>
...
<problemStatisticType>BEST_SOLUTION_CHANGED</problemStatisticType>
<problemStatisticType>CALCULATE_COUNT_PER_SECOND</problemStatisticType>
</problemBenchmarks>
...
</inheritedSolverBenchmark>
...
</plannerBenchmark>
Multiple problemStatisticType elements are allowed. Some statistic types might influence
performance noticeably. The following types are supported:
To see how the best score evolves over time, add BEST_SOLUTION_CHANGED as a
problemStatisticType.
The best score over time statistic is very useful to detect abnormalities, such as score traps.
Note
Don't be fooled by the simulated annealing line in this graph. If you give simulated annealing only 5 minutes, it might still be better than 5 minutes of tabu search. That's because this simulated annealing implementation automatically determines it's velocity based on the amount of time that can be spend. On the other hand, for the tabu search, what you see is what you'd get.
To see how fast the scores are calculated, add CALCULATE_COUNT_PER_SECOND as a
problemStatisticType.
Note
The initial high calculate count is typical during solution initialization. In this example, it's far easier to calculate the score of a solution if only a handful exams have been added, in contrast to all of them. After those few seconds of initialization, the calculate count is relatively stable, apart from an occasional stop-the-world garbage collector disruption.
