I want to stress that this is one area where my expertise is limited. I donot have a working knowledge of neural networks, nor do I understand exactlyhow the bots are programmed in many cases. A lot of what I will be sayingin this article will be conclusions based on what I have observed, and someof these conclusions may be quite wrong. I encourage knowledgable readerswho are more familiar with these areas to send in articles themselvescorrecting my mistakes or adding to the concepts I am discussing. In thisway I am hoping this article will lead to a collection of articles by thebest computer and backgammon minds in the world, which will be of the mostvalue to everybody.

There are several bots which play a competent game. The two commercial onesare Jellyfish and Snowie. I am going to limit this discussion to use ofSnowie for one main reason: Snowie has the ability to import a match (withdifferent formats) and then automatically run through the match, giving itsevaluation of all the plays. This is an extremely valuable tool forimproving one's game. The question is, how can we make the most of it.

I am using Snowie version 3.2, on a fairly fast computer. Earlier versionsof Snowie may have features different from what I am describing. Slowercomputers will take more time, making some of the rollout suggestionstoo time-consuming.

First, a little background. The idea of a neural network learning toplay backgammon was originally conceived of by Gerald Tesauro around 1991.His program, TD-Gammon, was the pioneer for the neural nets. The way theneural nets work is roughly as follows: Starting with little more than therules of the game, they play thousands of games against themselves. Fromthe results of these games, they "learn" through trial and error the valuesof important paramaters such as points, blots, etc. and construct their ownequations for weighing these paramaters for any given position. At theconclusion of this training period, the bot is able to look at any givenposition and give an estimate of the number of wins, gammons, and backgammonswhich each side will win. This estimate is what the bot uses for playing.

How good is this estimate? It depends a lot upon the type of position.Oddly enough, for simple positions such as races the bots make strangemisevaluations and thus do weird things. In order to get around this, I believethat Snowie uses a data base (which is 100% accurate) to make its racingplays when both sides have all their men in. Other relatively simple positionssuch as coming in against an anchor cause the bot to do some strange-lookingthings, although a surprising number of these plays turn out to be correctupon analysis. Other types of positions can give the bots problems. Primingbattles where timing is critical can cause the bots to be way off. Backgames, which require special technique, create problems for the bots.However in the average position the bots are generally correct in theirassessments.

This snapshot evaluation is what is called the 1-ply evaluation. There isno looking ahead by the bot, just a weighing of paramaters to evaluate theposition. If the bot can look ahead a bit, this evaluation improvesmarkedly. Just examining all the possible dice rolls for the opponent,which is the 2-ply evaluation, will have the evaluations much moreaccurate. If the bot looks further down the road at not only the opponent'srolls but its own rolls, the 3-ply evaluation, its accuracy is better still.The problem with these 2 and 3-ply evaluations is time. There are 21different possible dice rolls (grouping 4-3 and 3-4 as the same). Thus todo a full 3-ply evaluation the bot has to examine 21 X 21 or 441 sequences.It then has to find the best plays for each of these sequences (using itssnapshot or 1-ply evaluation). Even with our ever-faster computers, thiscan take some time. However with a fast computer the bot can play at3-ply in what would be considered a normal playing pace, and it is then aformidable opponent.

So how well do the bots really play at their 3-ply analysis? The answer isvery well -- probably as good or better than the best players in the world.They may slip up on some of the technical plays, but they more than compensatefor this with their judgment in the murky but critical positional decisions and balanceof priorities.

A true test of this came a few years ago. Jellyfish (backed by Malcolm Davis)played 300 games against Mike Senkiewicz and 300 games against Nack Ballard.The stakes were significant, so the players were taking this very seriously.The conditions were simulated to actual play -- dice were rolled by hand,the players were playing on a regular backgammon board, a human opponentwas playing Jellyfish's moves. The moves and dice rolls were keyed intothe computer by a third person, and that person called out the Jellyfishplays which were then made by the human opponent. Thus, from the player'spoint of view it was a regular backgammon game. It worked quite smoothly --I was there and helped set it up and do much of the inputting. The resultswere Jellyfish came out dead even -- +58 points vs. Senkiewicz and -58 pointsvs. Ballard. Since if anybody claimed that Senkiewicz and Ballard were thetwo best players in the world it would be difficult to dispute that, I believethis match was pretty solid proof that Jellyfish can play at world classlevel. Snowie plays at least as well as Jellyfish in my opinion, so weare talking about top level play.

The really nice feature on Snowie is the ability to have it run through anentire match, evaluating your play. Using the File/Import File button, youcan import an match into Snowie. There are several different formatswhich can be accepted. For example, if you play a match on GamesGrid, youcan then save it in .sgg format, and can import this match directly intoSnowie.

Now to have Snowie run through it. If you click on Batch/Specify Analysis,a dialog box comes up which allows you to prepare the runthrough. In orderto get the most out of Snowie's analytical capabilities, I recommend usingthe 3-ply analysis for the match, search space huge, 3-ply speed 100%. I'mnot exactly sure what is going on here, but I believe these paramaterscontrol how the 3-ply analysis is done. If you use a smaller search space,the analysis is shortcutted in some way, so it won't be as accurate. Thetradeoff of course is that it goes faster. On my computer the analysis ofa 7-point match generally takes about 10 or 20 minutes, which I find quiteacceptable. I can play a match, run the analysis, and play through it tosee where I screwed up. You can also set the program to automaticallyroll out what it considers an error or a blunder -- this takes some extratime of course. Fiddle around with the paramaters and find what best suitsyou.

Now the hard part -- interpreting what Snowie is saying. First of all, howdoes Snowie handle things? Obviously it would be silly for it to run a full3-ply analysis on every legal move, since some of the moves are ridiculous.What it does, I believe, is as follows: It checks out every legal move onthe 1-ply (this it can do very quickly). Those moves which are within someamount of the "best" play, are then run on a 2-ply. After that secondscreening, candidates which are still close are run on the 3-ply. That iswhy you will see some candidates on 3-ply, some on 2-ply, and some on 1-ply.And if the move is 100% obvious the 3-ply or even the 2-ply may be skippedaltogether. I don't know exactly what paramaters are used to determingwhen to include a candidate in the next pass, but for the most part thescreening seems to be accurate. It is rare that a serious candidate doesn'tmake it to the 3-ply stage. And of course if when going through the matchyou are suspicious about a move which didn't make it to the 3-ply stage youcan just do a 3-ply check on it very easily.

Okay, so now we have a bunch of information on each of the candidate movesfor a position. What does all this info tell us? The move at the top isthe move Snowie thought was the best. Other moves are inferior according toSnowie, and the numbers will hopefully tell us how much inferior they are.The move actually played will have an asterisk next to it.

For each candidate move, there are six numbers at the bottom. These, inorder, tell us Snowie's estimates (in percentages) of backgammons won,gammons won, single games won, single games lost, gammons lost, and backgammonslost. It should be noted that the single games won column includes gammonsand backgammons, and the gammons column includes backgammons. Thus, thesingle games won and the single games lost will always add up to 100%.These are very important results to keep an eye on, as they will be tellingthe true story of what Snowie is thinking.

To the right of the move is a number which is some kind of measure of theequity of the move. What does this number mean? I'll admit I don't understandit, and I would be very appreciative of a good explanation. Of course thisis the number by which the candidate moves are ranked, and the number whichSnowie uses to rank our overall play in the match. While it might not beclear what it means, the difference between that number for the best playand the number for our actual play is supposed to be some kind of measureof the magnitude of our error.

A third piece of information can be gotten by simply pointing the cursorinside the box where the move is. A yellow popup box comes up giving uswhat it calls score based cubeless equities -- current, doubled, andredoubled. Once again, it is not clear to me exactly what these figuresstand for at match play -- for money, it is just the normal cubeless equity. However the difference between the current figure for the bestplay and the current figure for the actual play signifies something.Interestingly enough, the best play doesn't always come out on top.Go figure.

It would seem as though the magnitude of the difference between the bestplay and the actual play (which is expressed by a number in parenthesis)should be a good measure of how bad the play is. Unfortunately the makersof Snowie are trying to figure in the cube, and by doing so the resultscan be quite distorted. For example:

142 157

White money game Blue

Blue has just rolled a pretty bad number from the bar. Let's suppose thatwe thought the priority was to play as safely as possible, and we hadplayed B/23, 8/2. As we will see, this is not a good play. But we wantto know more than that -- we want to get a feel for how bad a play we havemade so that when we see a similar position (but for which the argumentsfor the Safe play are stronger, say one of the checkers on the midpointmoved down to the eight point). In other words, when we see an error wehave made we need to know how bad the error was so that we have mentallybuilt up a reference position for the future.

The Snowie results at full 3 play, 100% are as follows:

B/23 13/7               -.6130.2%  7.0%  37.3%  62.7%  25.3%  0.8%
Cubeless equities  current:  -0.254 - Jacoby  doubled:  -0.443
B/23 8/2                -.764 (-0.151)0.3%  6.8%  36.1%  63.9%  29.5%  1.2%
Cubeless equities  current:  -0.278 (-0.024) - Jacoby  doubled:  -0.514 (-0.071)

But wait. As an experiment, let's look at exactly the same position butgive Blue ownership of the cube. Suddenly we have a different story!Snowie says:

B/23 13/7               -.3070.2%  7.0%  37.3%  62.7%  25.3%  0.8%
Cubeless equities  current:  -0.443
B/23 8/2                -.382 (-0.075)0.3%  6.8%  36.1%  63.9%  29.5%  1.2%
Cubeless equities  current:  -0.514 (-0.071)

What it looks like is going on is as follows: Clearly White has a strongposition after either play, and the cubeless equities of -.443 and -.514indicate that after either play the proper cube action (if the cube is inthe center) is double and take. So Snowie, in its infinite wisdom. doubleseverything because it anticipates a cube turn and is trying to churn outnumbers based on the cube. If you weren't aware of this, you would havea very distorted picture of how large the error actually was.

When we get to match situations, these distortions can be even worse.As an illustation, I'll give you the "size of error" for the above playat each match score in a 5-point match. You can try to figure out foryourself what is going on. However if you were just trying to analyzeyour checker play, you can come up with some very wrong impressions.

In each case the cube will be assumed to be in the center, and if oneplayer has 4 points it will be assumed to be the Crawford game. Blue'sscore will be given first.

Score           Error                
0-0             .165
1-0             .196
2-0             .184
3-0             .090
4-0             .030
0-1             .194
1-1             .171
2-1             .000
3-1             .000
4-1             .069
0-2             .073
1-2             .095
2-2             .175
3-2             .000
4-2             .033
0-3             .085
1-3             .086
2-3             .095
3-3             .064
4-3             .041
0-4             .023
1-4             .027
2-4             .023
3-4             .029
4-4             .024

The problem is that when you are zipping through a match examining yourblunders and errors, you aren't thinking about the cube magnification effect.You just want a quick look at the number to see what is going on. Untila future version of Snowie changes this and puts things in the properperspective, the user has to be very aware of this situation.

A slight modification in the position illustrates how serious this problemcan be:

142 160

White money game Blue

Once again, B/23, 13/7 is the correct play. This time, however, thedifference between that and B/23, 8/2 is only .083. It isn't that thetwo men on the 11 point structurally change much. The difference is thatdue to the improvement in Blue's position Snowie doesn't think White hasa double after either play, so the difference isn't magnified.

One more minor modifaction:

142 167

White money game Blue

Now the difference jumps all the way to .196! What happened was that weweakened Blue's position just enough so that Snowie thinks White has adouble after the poorer B/23, 8/2, but not after B/23, 13/7. This assessmentmay well be correct, but the results magnify the size of the error way outof proportion.

I believe the above distortions only occur when a cube turn is imminent, sothey won't happen too often. However they can be very distracting when theydo occur.

Snowie attempts to take the match score into account when evaluating variouscandidate plays. If some plays are much more gammonish than others, thiscan lead to weird-looking results which may be difficult to interpretproperly. The following position is a typical example:

160 163

White money game Blue

Blue's obvious choices are to attack with 8/4(2), 6/2(2)* or to playpositionally with 24/20(2), 13/9(2). Both plays leave him with a strongposition. The attacking play will generate far more gammons, and willalso lose a few more gammons. The positional play figures to win the gamemore often. For money (assuming no Jacoby rule), Snowie's 3-ply opinion is:

8/4(2) 6/2(2)*          +.5881.2%  25.6%  61.5%  38.5%  9.7%  0.4%
Cubeless equity: +.39724/20(2) 
13/9(2)        +.536 (-0.052)0.6%  15.9%  63.8%  36.2%  5.7%  0.2%
Cubeless equity:  +.381 (-0.016)

Now let's look at what Snowie thinks of this play at a couple ofdifferent match scores in a 5-point match:

0-0: The attacking play is superior by .078
1-0: The attacking play is superior by .004
0-1: The attacking play is superior by .118

Can these figures be right? True one should go more for gammonish playswhen behind in the match, but can the difference between 1-0 and 0-1 ina five point match with the cube still in the center be this great?Snowie thinks the plays are a photo when ahead 1-0, but considers thepure play a fairly major blunder when behind 1-0.

What is the reason for these big differences? I can't figure it out.In Snowie's opinion if Blue attacks and White flunks, it is double andpass at each of these three match scores. Snowie thinks the pass is closest atthe 0-0 score compared to the other scores, but not by that much. Itdoesn't make a whole lot of sense to me.

Let's try following the same theme, but strengthening White's position a bit:

150 163

White money game Blue

For money Snowie rates the two plays as follows:

8/4(2) 6/2(2)*          +.4341.0%  22.7%  59.0%  41.0%  10.6%  0.4%
Cubeless equity:  .30624/20(2) 
13/9(2)        +.431 (-.003)0.5%  13.8%  61.7%  38.3%  6.0%  0.2%
Cubeless equity:  .313 (+0.007)

In a 5-point match, Snowie thinks a little differently:

0-0: Attacking play better by .044
1-0: Attacking play better by .059
0-1: Attacking play better by .044

This seems odd. Why should the attacking play be so much better at 0-0 ina five-point match than it is for money? Part of the reason might be thatin the match Snowie thinks the double is clearer (rather than borderline)if White flunks after the attacking play, although the take is still veryclear.

I took a look at the figures assuming that White held the cube. Thewins, gammons, backgammons, and cubeless equity remain the same, of course.However, for money Snowie now rates the pure play as superior by .018.It looks like Snowie is taking into account the cube potential after theattacking play. Whether this is the proper approach is questionable.The problem, I think, is that Snowie is only looking at the cube potentialon its next roll.

For the match scores we have been considering (and White owning the cube),Snowie says:

0-0: Attacking play better by .002
1-0: Pure play better by .005
0-1: Attacking play better by .010

This makes some sense, since the player behind in the match can use gammonsmore than the leader. However the differences are pretty small, so thematch score doesn't appear to be too relevant a factor. The largerdifferences in the play evaluation appear to come when a cube turn ispossible in some variations.

As a final check, let's strenghten White's position a bit more so thatthere won't be a cube involved next turn.

152 163

White money game Blue

Snowie's money estimates are:

24/20(2) 13/9(2)                0.4050.5%  14/6%  60.4%  39.6%  6.1%  0.2%
Cubeless equity:  .2968/4(2) 
6/2(2)*                  0.273 (-0.126)0.6%  15/0%  56.8%  43.2%  6.9%  0.3%
Cubeless equity:  .220 (-0.077)

For the match scores, we have the following results:

0-0: Pure play better by .103
1-0: Pure play better by .104
0-1: Pure play better by .123

In none of these cases does Snowie think that after attack and flunk doesBlue have a double (although it is close). Thus, the differences inthe Snowie estimates of the plays doesn't change much. I don't understandwhy Snowie prefers the pure play more when behind 1-0 than at the otherscores -- that may have something to do with the quirks of the matchequity table. Anyway, the difference isn't much. The key appears to bethat when no cube turn is coming on the next roll, the results will bepretty much consistent regardless of the match score. However if there isa potential cube turn on the next roll, then the results of play vs. playevaluations may be quite distorted from reality. This is important toremember when running through a match.

When Snowie gives its cube analysis, there is plenty of information availableto the user. The full wins, gammons, and backgammons to each side of course,as well as the cubeless equity. In addition, Snowie presents three numbers.These are the equities if it goes double-take, if it goes double-pass, andif it goes no double. The equity for double-pass is always 1.000, of course.The other equities take the match score into account -- how this is doneis not totally obvious to me. However Snowie seems to do a pretty goodjob on this. If the equity after double-take is less then 1.000, thedouble should be taken -- if it is greater, the double should be passed.If the equity after no double is greater than after double take, thendoubling is wrong -- if the equity after double-take is greater thendoubling is correct (except if the equity after no double is greaterthan 1.000 -- then the position is too good to double). For example, lookat the following position -- Blue is ahead 2-1 in a 5-point match:

132 117

White 1 5 point match Blue 2

The Snowie estimates are:

Money equity: .4830.1%  2.0%  73.9%  26.1%  1.5%  0.1%
Double, take:  .986
No Double:     .888  (-0.097)
Double, pass: 1.000  (+0.014)

Of course, Snowie's cube actions have to be dependent upon Snowie'sevaluation of the position. If Snowie is way off in the evaluations, thenthe cube actions will be correspondingly wrong. In positions such as theabove it is likely that the estimates are on target. However for certaintypes of games, Snowie's estimates can be far off. In general, Snowie(like the other bots) has trouble evaluating timing positions and backgames. The results are more serious for the user than with play decisions.Even if Snowie is misestimating the overall equity for a given type ofposition, the difference between two plays is likely to be accurate as longas the position type resulting from the two plays isn't too different.When it comes to cube action, it is necessary to get the absolute equitycorrect in order to make a proper decision.

Here is an example of a sequence of plays from a match I played recentlywhich illustrates the difficulties Snowie has:

109 130

White 0 7 point match Blue 1

White is on roll. At the time I thought my opponent should have redoubled.He was a favorite to make his four point, and then the timing would probablygo his way. If he made his four point and I busted my prime, he would losehis market by quite a lot. Of course it looked like I still had a take, sincehe might roll a horror number or I might survive the priming battle even ifhe does make his four point. Snowie said:

1-ply   Money equity
    -0.0070.5%  10.9%  48.1%  51.9%  8.1%  0.2%
No Redouble     0.189
Redouble, take -0.197 (-0.387)
Redouble, pass  1.000 (+0.811)

2-ply   Money equity
    0.2280.4%  10.6%  57.8%  42.2%  3.8%  0.1%
No redouble     0.458
Redouble, take  0.292 (-0.166)
Redouble, pass  1.000 (+0.542)

3-ply   Money Equity
    0.3520.7%  16.1%  61.5%  38.5%  4.4%  0.1%
No redouble     0.616Redouble, take  0.587 (-0.028)
Redouble, pass  1.000 (+0.384)

How about rolling the position out? I will discuss the bot rollouts laterin this article in detail, but for now accept that a 2-ply rollout of 72trials with no truncation, which is what I did, is likely to give a prettyaccurate result. And the rollout said:

Rollout         Money equity
    0.4051.1%  21.5%  62.9%  37.1%  7.9%  0.2%
Redouble, take  0.757
No redoube      0.682 (-0.075)
Redouble, pass  1.000 (+0.243)

The game continued as follows: My opponent did not redouble, rolled 6-5,and played 10/4, 9/4. I rolled 5-2 and played 8/3*, 5/3. This was the position:

101 123

White 0 7 point match Blue 1

Now what is goingon? I was sure my opponent was supposed to redouble. All he had to do wasflunk and my position would probably crack, and even if he entered thepriming battle would probably be in his favor. Frankly I wasn't sure whetheror not I was supposed to take. This time Snowie was willing to producea 3-ply result, which said:

3-ply           Money equity
    0.3751.0%  19.6%  60.7%  39.3%  4.4%  0.2%
No redouble     0.656
Redouble, take  0.624 (-0.032)
Redouble, pass  1.000 (+0.344)

Rollout         Money equity
    0.5881.4%  27.9%  67.7%  32.3%  5.8%  0.1%
Redouble, pass  1.000
No redouble     0.838 (-0.162)
Redouble, take  1.138 (+0.138)

My opponent chose not to redouble and rolled 5-2, playing B/23, 6/1*. Iresponded with 1-1, playing B/24*, 24/23, 9/8(2). Here we were again.This time it was quite clear that he didn't have a redouble. If he enteredhe would probably be the one to crack, and if he flunked I would still havea reasonable chance to win the priming battle with my spare on the eightpoint able to absorb some pips. Snowie's opinion was that I was the slightfavorite. This may or may not be accurate, but it is definitely not aredouble for him. No need to roll this one out.

He now danced. I rolled 6-2, and played 8/6, 8/2*. New cube problem.This is what the position now looked like.

120 112

White 0 7 point match Blue 1

My reaction was that this is a monster double. I wouldn't take it in amillion years. The timing was now very likely to go against me. Snowiedidn't see it that way, however.

3-ply           Money equity
    0.4360.5%  12.6%  68.5%  31.5%  6.0%  0.4%
Redouble, take  0.846
No redouble     0.787 (-0.058)
Redouble, pass  1.000 (+0.154)

Rollout         Money equity
    0.6891.0%  22.3%  76.1%  23.9%  6.2%  0.4%
No redouble     1.006
Redouble, pass  1.000 (-0.006)
Redouble, take  1.431 (+0.425)

Predictibaly enough, my opponent didn't redouble. This may have been thetheoretically correct action, but from his previous cube action it is likelyhe thought he wasn't good enough rather than thinking he was too good.The game continued with him rolling 6-2, playing the forced B/23*. I rolled3-1, playing B/24, 8/5. The position now was:

118 131

White 0 7 point match Blue 1

Now he finally redoubled (indicating that hisreason for not redoubling the roll before was the wrong reason), and Ipassed of course. On the 3-ply Snowie finally recognized that hisposition was pretty strong.

3-ply           Money equity
    0.5300.9%  20.8%  70.0%  30.0%  8.2%  0.4%
Redouble, pass  1.000
No redouble     0.783 (-0.217)
Redouble, take  1.086 (+0.086)

Rollout         Money equity
    0.6511.5%  23.0%  73.7%  26.3%  6.6%  0.3%
Redouble, pass  1.000
No redouble     0.874 (-0.126)
Redouble, take  1.326 (+0.326)

So we have seen how Snowie even on its powerful 3-ply can make very largemisevaluations on cube decisions in some types of positions. In order toget closer to the real truth, we need to go to rollouts.

Having the bot roll out the position is an ideal way to see what is goingon. Unfortunately there are two drawbacks to rollouts. One is that theyare time-consuming. The other is that the bot may not handle the positionwell enough for the rollout results to be meaningful.

How many times do we need to roll out a position before we can trust theresults from a statistical point of view. Assuming we are rolling outthe position all the way to the end, it can take quite a few trials. It mightseem like 1000 or so trials would be sufficient, but the truth is that therecan be quite a bit of variance (or luck) involved, and it is not uncommonfor a rollout result to be several percent away from what it should beon 1000 trials. You need 4000 or 5000 trials to be fairly safe onstatistical grounds, and this takes a fair amount of time even with today'sfast computers.

One way around the time problem is to use truncated rollouts (or what arecalled mini-rollouts by Snowie). The idea is that instead of having the botplay the position to the end, you have it play it out a few moves, and thensimply use its equity estimate after those few moves as the result of thetrial. This is a time-saver for two reasons. First of all, it takes muchless time to play out a few moves than to complete the whole game. Secondly,a smaller number of trials is generally sufficient. The reason is thatthe luck factor is cut down, because you don't have the wild one-roll swingswhich occur at the end of the game. The program has already averaged themout when forming its estimate. Thus, 1296 trials is usually quite sufficient.The default setting on Snowie is 5 rolls deep, but my personal preferenceis 7 rolls. I believe it is worth a little extra computer time in orderto get a bit farther into the position.

Truncated rollouts are dependent, of course, on Snowie's ability to estimatethe equity of the resulting positions accurately. For most simple positionsSnowie does a pretty good job on this. However for some complex backgametypes of positions which won't change much in nature over the next fewrolls, Snowie's estimate may be far off. For these types of positions,truncated rollouts may not be the answer.

The other problem with the rollouts is that Snowie might not be playing toowell. Due to time considerations, we can't have Snowie play at itspowerful 3-ply level for the rollouts. They will just take too long.We have to live with the 1-ply level of play if we are going to get asufficient number of trials in. As we have seen, Snowie can come up withsome pretty bad plays on its 1-ply level. If these bad plays occur at thewrong time, they may mess up the rollout results.

The answer to this problem is to have Snowie play at a higher level forthe rollouts. It may seem as though this requires too small a number oftrials for the results to be meaningful, but it turns out that this is notthe case. By using a method called variance reduction, Snowie can make theresults of about 50 trials equivalent to about 1000 trials. The generalidea is that Snowie evaluates the luck factor from each roll in the trialand filters it in. If Snowie's evaluation of this luck factor is accuratethe number of trials represented will be far more than the actual numberrolled out.

How much can we trust this variance reduction method? David Montgomery wrotea truly excellent article on variance reduction in the February 2000 issueof GammOnLine. I fully recommend this article to anybody who is interestedin this sort of material. I'll admit I don't understand the article orvariance reduction fully, but it does seem to make sense. Mathematiciansand other knowledgable people I have talked to about it all say that theapproach looks valid. And, most importantly, from what I have seen theresults appear to be very reasonable. When I roll a position out 1-ply(no variance reduction) and 2-ply (with variance reduction), if the resultsdiffer the 2-ply results are almost always more in line with my intuitionthan the 1-ply results. Thus, I have concluded that 2-ply rollouts usingvariance reduction are pretty valid.

What setting should you use for the 2-ply, and how many trials? This islargly a matter of time and computer power. I generally run 72 trials,played to completion. I would rather not get involved with truncatedrollouts with the variance reduction, since then I have to worry aboutthe Snowie evaluations. It appears as though 72 trials is usually roughlyequivalent to 1296 trials without variance reduction. For most positionsit takes from 2 to 10 minutes to roll out those 72 trials on my computer.This time will vary depending on one's computer speed, of course.

When should you use 2-ply and when should you be satisfied with the 1-plyrollouts? You just have to get a feel for this. Simple positions suchas holding games, blitzes, and races are generally handled okay by the1-ply rollout. More complex games usually need the 2-ply rollout to getclose to the truth. After having tried several different types of positionsyou get to know pretty well what you should be using.

The advantage of the 2-ply is that the bot plays considerably better than itdoes on 1-ply, making the rollout results more trustworthy from that point ofview. So how about using 3-ply rollouts? This doesn't gain you anythingas far as the variance reduction goes, it turns out. If 72 trials on the2-ply are about equivalent to 1296 on 1-ply, then 72 trials on 3-ply willalso be about equivalent to 1296 on 1-ply. The gain is simply that the botplays better on 3-ply than on 2-ply. The loss is that it takes a lot longer.

How about cubeful rollouts? The rollouts can be set so that Snowie takes thecube into account during the rollout. My personal preference is to not getinvolved with these when I am examining a play vs. play problem which I appearto have bungled. There are enough things which might go wrong with therollout without getting Snowie's cube opinions involved in its evalutions.If I get a look at the cubeless equities, these will generally be sufficientfor me to form my own conclusions about the cube considerations if need be.

So, what procedures should one follow to use the bots for improvement? Thisis largely a matter of one's personal taste. This is what I do. First Iwill play a match, say on GamesGrid. After completing the match, save thegame on my computer (in .sgg format), and then import the file to Snowie.I then have Snowie run through the match. My settings are 3-ply with hugesearch space and 100% 3-ply speed. With these settings, it generally takesSnowie about 10 to 20 minutes to run through a 7 point match. I have itlook at both my opponent's plays and mine -- you never know when somethingof interest might have come up on the other side of the board.

Now for the analysis. I don't just look at my errors and blunders, althoughobviously these are the most important. I go through every move step bystep. It is just as important to see the difficult positions which youhave handled correctly as the ones you got wrong. It is also important tonote by how much Snowie prefers move A to move B. If you thought thechoice was very close but Snowie says they are miles apart, then you havesome rethinking to do about the position.

What about my errors or blunders. Generally I will roll them out, unlessI can agree immediately that I was wrong. It is the results of theserollouts which give me the most insight into the position, and tell mewhere I need to improve my thinking. Also I will roll out some verypivotal plays which I didn't really know what to do even though Snowie mayhave thought I got it right. In addition I may roll out a play of myopponent if I am surprised by the Snowie result. Always keep in mindthat it isn't just which play Snowie thinks is best that is important -- itis the magnitude of the difference between the two plays which matters.If Snowie thinks your play is .020 worse than the best play you can probablyignore its opinion if you still think your play is better -- quite likelyit is. However if Snowie believes your play is .150 worse than the bestplay you can be pretty sure that your play is wrong, and it is up to youto figure out why.

In addition to the above, it is vital to keep an eye open for the typesof quirks described earlier in this article which may magnify an errorout of proportion due to the match score or potential cube action. Alwaysbe willing to look at the cubeless equities and at the percentage of winsand gammons for each side. These figures don't lie, and will often giveyou a truer picture of what is going on than the number Snowie uses torank the plays and evaluate the size of your errors. And of course ifyou are still suspicious, roll it out -- 2 ply if necessary. If therollout confirms what Snowie is saying, you can be pretty sure that it is right.

One further thing. While it in our nature to think that we are right andSnowie is wrong, most of the time when Snowie says we make a blunder we havemade one. It is true that occasionally Snowie will give us false information,as has been seen from several of the examples. But when in doubt, believeSnowie and try to understand what it is trying to tell you. Snowie usuallyknows what it is doing.