1) Benchmarking CPU's for Snowie rollouts,
2) Measuring Snowie rollout runtime for the various parameter settings available,
3) Comparison of rollout results for different Snowie settings as well asdifferent robots.

This article is a report on the current results of part 1.

A standard position and set of rollout parameters were chosen. The position isthe opening setup with a diceroll of 62. The rollouts consisted of 36 trials ofeach of three candidate plays: 24/18, 13/11; 24/16; and 13/5. The rolloutparameters were: cubeless money play, 3-ply truncated at 10 rolls, truncationif bearoff database reached, medium search space, 20% speed, seed = 12345.Several GoL members took on the task with their respective CPU's and they arecredited below for their contributions.

Although cputime for a given rollout should roughly correlate with CPU clockspeed, different processors handle programs differently, particularly when bothinteger and floating point arithmetic are used. The table below is a summary ofthe various results reported both publically on the forum and to me privately. Also included is a plot of Snowie's runspeed (1000/runtime where 1000 is chosen to make the resulting numbers manageable and 'runtime' is given in seconds).The Snowie runspeed is plotted as a function of CPU clockspeed.

Contributor      SW ver.    Processor       Proc. Speed     mem.  OS       tot time    time(h:m:s) tot time normalized   projected ration (sec)            speed           speed       norm./proj.
JP White         3.0        P-166           166             64MB  Win98    2h  5m 55s   2:5:55     7555       0.13         0.15      0.87
Chuck Bower      3.0        PII-233         233             32MB  Win98    1h  7m 22s   1:7:22     4042       0.25         0.25      0.99
Jason Lee        3.0        PII-233         233             64MB  Win98    0h 54m 59s   0:54:59    3299       0.30         0.25      1.21
JP White         3.0        AMD K6-300      300             128MB  Win95    1h  5m 17s  1:5:17     3917       0.26         0.35      0.73
Gregg Cattanach  3.2        PII-350         350             64MB   Win98    0h 36m 14s  0:36:14    2174       0.46         0.42      1.09
David Montgomery 3.2        PII-400         400                    Win98    0h 33m 50s  0:33:50    2030       0.49         0.49      1.00
David Montgomery 2.3        PII-400         400                    Win98    0h 32m 28s  0:32:28    1948       0.51         0.49      1.04
Robi             3.2        PII-400         400             128MB  Win98    0h 35m  4s  0:35:4     2104       0.48         0.49      0.96
Chuck Bower      3.2        PIII-450        450             128MB  WinNT4.0 0h 27m 47s  0:27:47    1667       0.60         0.57      1.06
Gregg Cattanach  3.2        PIII-450        450             128MB  Win98    0h 29m 12s  0:29:12    1752       0.57         0.57      1.01
JP White         3.2        AMD K6-2 450    450             128MB  Win98    0h 36m  1s  0:36:1     2161       0.46         0.57      0.82
JP White         3.2        AMD K6-2 450    450             128MB  Win98    0h 37m 33s  0:37:33    2253       0.44         0.57      0.78
Rob Maier        3.2        Celeron-466     466                             0h 30m 10s  0:30:10    1810       0.55         0.59      0.94
Chris Yep        3.2        PIII-500        500             96MB   Win98    0h 26m 11s  0:26:11    1571       0.64         0.64      1.00
TarHeelFan       2.3        PIII-600        600             256MB  Win2000  0h 20m 36s  0:20:36    1236       0.81         0.79      1.03
Bob Stringer     3.2        PIII-1000       1000            256MB  Win98    0h 13m  2s  0:13:2      782       1.28         1.37      0.93
Petter Bengsston 3.2        AMD-1000        1000            384MB  Win2000  0h 10m 36s  0:10:36     636       1.57         1.37      1.15

You will notice that the data conform roughly to a linear relationship, whichshould not be surprising. I performed an 'eyeball' linear fit to the data. Thesecond to last column of the table is a predicted speed based upon this fit. The last column is the ratio of the normalized speed (1000/actual runtime) divided by the predicted speed. Although one must be careful about drawing too much from thiscomparison, a few generalities seem to be evident.

The early AMD processors (K6 models) underperform their Intel cousins by about20-25%. However the latest generations of processors reverses this trend. In particular the AMD Thunderbird (Athlon?) running at 1 GHz beats the prediction by 15%, while the Intel processor running the same Clock speed might even be a few percent below the projection. Both of these occurences were predicted by forum posts prior to the results becoming available and I refer the interested reader to the thread which I initiated on January 6th for more details. Finally, admittedly based upon limited data, the Intel Celeron processor may be a bit slower than it's PII and PIII siblings, though if the effect is real it is only a few percent. I believe all of this is consistent with known performance differences of these processors, particluarly when performing floating point operations.

Again cautioning that the data is limited, there may be a few percent affectdue to the version of Snowie being run (2.x or 3.y) with the older version runninga bit faster. There is not an obvious connection relating to either operatingsystem nor amount of available memory.

Concerning part II (runtime dependence on Snowie parameter settings), I havecompliled quite a bit of data and will report the results next month. There is still considerable help needed for part III (rollout results for different robots and for different Snowie settings). Again I'm asking for help from GoL members. Here the position and candidate plays are the same as in the other parts, but the settings (including number of trials) can and should be different. In particular, since there is the potential of combining results, the use of a random seed would be safest, but in particular DON'T use 12345. If you have a favorite seed, that is probably OK.

Part III (when combined with part II) may help answer a longstanding question:what are the most efficient Snowie settings for getting fast, reliable rolloutresults? In addition, by comparing rollout results from different robots, we maybe able to speculate about the relative strengths of different creations. I planon setting up the old Expert Backgammon (version I). Gary Wong has submitted datafor gnu-backgammon (but more would be welcome). Jellyfish is a must (any volunteers?)and I'm going to ask at least a couple authors of homegrown robots for their input.Again, any contributions you can make would be valuable and appreciated.