I'm working on this game engine which plays some board game, I implemented minimax with alpha beta pruning, and move ordering. Now when I play with other existing engines, everything looks fine. However, I found two subtle problems which I don't know what causes them, maybe I have some gap in my knowledge.
The first issue:
When I increase the depth of the search, intuitively I should get better results even if consumes more time, but in my case moving from depth 8 (winning) to depth 9 causes me to lose all the time, now if I increase it to 10 I suddenly win again. Keep in mind that I keep everything constant except the depth. At first I thought that maybe its because at depth 9 we find a promising move at first which the opponent can refute it easily, but then this could happen at every depth. So my question is what causes this weird behaviour? Is there an optimal depth, which if we go beyond could backfires?
The second issue:
Its similar to move ordering but done to get the values of the next play to pick what to play. For example:
For every possible move starting from weakest move do:
minimax(move)
Now since we have to go through all possible moves why having the strong moves first wins the game but if I started with the weakest moves loses the game. It doesn't make since we have to evaluate all possible moves anyway. What causes this behaviour?
When I increase the depth of the search, intuitively I should get
better results even if consumes more time, but in my case moving from
depth 8 (winning) to depth 9 causes me to lose all the time, now if I
increase it to 10 I suddenly win again.
Certainly sounds like a bug to me.
For every possible move starting from weakest move do:
minimax(move)
You should always start from the strongest move in order to benefit from alpha-beta pruning
Now since we have to go through all possible moves
What about alpha-beta pruning? You do not have to go through all possible moves.
Related
After reading the chessprogramming wiki and other sources, I've been confused about what the exact purpose of iterative deepening. My original understanding was the following:
It consisted of minimax search performed at depth=1, depth=2, etc. until reaching the desired depth. After a minimax search of each depth, sort the root-node moves according to the results from that search, to make for optimal move ordering in the next search with depth+1, so in the next deeper search,the PV-move is searched, then the next best move, then the next best move after that, and so on.
Is this correct? Doubts emerged when I read about MVV-LVA ordering, specifically about ordering captures, and additionally, using hash tables and such. For example, this page recommends a move ordering of:
PV-move of the principal variation from the previous iteration of an iterative deepening framework for the leftmost path, often implicitly done by 2.
Hash move from hash tables
Winning captures/promotions
Equal captures/promotions
Killer moves (non capture), often with mate killers first
Non-captures sorted by history heuristic and that like
Losing captures
If so, then what's the point of sorting the minimax from each depth, if only the PV-move is needed? On the other hand, if the whole point of ID is the PV-move, won't it be a waste to search from every single minimax depth up till desired depth just to calculate the PV-move of each depth?
What is the concrete purpose of ID, and how much computation does it save?
Correct me if I am wrong, but I think you are mixing 2 different concepts here.
Iterative deepening is mainly used to set a maximum search time for each move. The AI will go deeper and deeper, and then when the decided time is up it returns the move from the latest depth it finished searching. Since each increase in depth leads to exponentially longer search times, searching each depth from e.g. 1 to 12 take almost the same time as only searching with depth 12.
Sorting the moves is done to maximize the effect of alpha-beta pruning. If you want an optimal alpha-beta pruning you look at the best move first. Which is of course impossible to know beforehand, but the points you stated above is a good guess. Just make sure that the sorting algorithm doens't slow down your recursive function, and by that removing the effect from the alhpa-beta.
Hope this helps and that I understood your question correctly.
For some time now, I've been thinking about how games calculate physics. Take as an example the game TrackMania. There are special routes where you only have to accelerate from the beginning to get to the finish. As an example, I take the following YouTube video (https://www.youtube.com/watch?v=uK7Y7zyP_SY). Unfortunately, I'm not a specialist in game development, but I know roughly how an engine works.
Most engines use a game loop, which means they use the delta value between the last call and the current call. This delta value is used to move objects, detect collisions and so on. The higher the delta value, the farther the object must have moved. The principle works fine with many games, but not with TrackMania.
A PC that can only display 25 FPS would calculate the physics differently than a PC with 120 FPS, because the collision detection is more accurate (impact is detected earlier, speed adjusted accordingly, ...). Now you can assume that the delta value is always the same (as with Super Mario Maker, at least that's my assumption), then this would work. But that would cause problems similar to old games (https://superuser.com/questions/630769/why-do-some-old-games-run-much-to-quickly-on-modern-hardware/).
Now my question, why do such maps work on every PC and why is the physics always exactly the same? Did I miss any aspect of game development / engine development?
The answer is simple, first the physic of the game is predictable, based on the input the result will always be the same.
Then the physic loop is not the same as the render, the game ensure the physic loop will be call with exactly the same period every time during the whole execution. So, yes a delta is needed for the render part, but the physic as a constant time in ms between each iteration.
One last think : you wont find "Press Forward" maps on the multiplayer, these kind of maps will not work correctly, this is directly linked to specificities in the physic to avoid TAS (Tool Assisted Speedrun).
TLDR: I have an asymmetric evaluation function for an implementation of negamax - is that acceptable? Or do I need to make it symmetric?
Longer:
I'm writing a game AI (for the chess-like board game "Hive") that was using minimax with alpha-beta pruning and an asymmetric evaluation function.
But I was having trouble adding transposition tables correctly, and was losing confidence in my minimax implementation, so I decided to switch to negamax using the pseudo-code here: https://en.wikipedia.org/wiki/Negamax#Negamax_with_alpha_beta_pruning_and_transposition_tables
I've got everything "working" and AFAIK accurately following the pseudo-code, but my AI is now making some wildly different moves than before and games that usually ended after 10-15 turns now take 30+, and I'm not convinced the AI is actually playing better than it was before. I'm worried that having an asymmetric evaluation function means I'm scoring nodes differently than before (because of the negamax flip-flopping).
I don't want to change to a symmetric function unless I really have to - I've been trying to produce an optimal function experimentally (AI vs AI battles) and have put in hundreds if not thousands of compute hours into producing a strong evaluation function.
Negamax support asymmetric evaluation functions but it does not lead to optimal play (assuming you have no knowlege about your opponent).
I don't know enough about Hive, but in computer chess it is, in general, a bug to have an asymmetric evaluation function. The reasons behind it should be the same for chess and Hive.
For instance, take the starting position (in chess). White is next to move and let us assume your evaluation function gives the position a score of +0.08.
Now change the position, so black is first to move. Everything is the same, only that the roles of white and black has been changed. Under the assumption, that +0.08 was the optimal score for the white position, why should the position for black not also be evaluated as +0.08?
The same argument goes for any position. If you reverse everything, there is no good reason for playing the position differently.
There is only one exception to this rule. If one opponent is clearly stronger than the other, there are arguments for an asymmetric evaluation. For instance, take a completely drawn position like this:
FEN: 4k3/8/8/p1p1p1p1/PpPpPpPp/1P1P1P1P/8/4K3 b - - 0 1
This position could safely be evaluated as 0. Now imaging the starting position but white starts without one knight. This should be a strong advantage for black.
Let us assume you are Magnus Carlsen and you are playing against on opponent who does not even know the chess rules. Which position would you prefer? Here, I would argue that an asymmetric evaluation could make sense (e.g., evaluate a likely draw similar to a loss). Carlsen should avoid the drawn position, while the beginner should prefer it.
The chances that the beginner can hold its own against the world champion, even at one knight odds, are practically zero. On the other hand, in the drawn position, the skill advantage does not matter, as no order of moves can result in a win or loss.
In computer chess, Rebel had a function to prefer tactical positions when playing against humans (see ANTI GRANDMASTER PLAY). There is also the common concept of "contempt", which is the score that engines give for a remis.
But note that in both my examples, this is not optimal play. Magnus Carlsen would not choose the position without the knight when playing a strong (or unknown) opponent. Also Rebel would not use the anti-human strategy against other machines, which also excel in tactical battles. (Even though, depending on the position, Rebel 10 did use ANTI GRANDMASTER PLAY against computers.)
In a game I've created Negamax works well for low depth searches but larger depth increases causes it to freeze. I thought about changing depth to type 'long' instead of 'integer' but not sure what else I can do. I know computation will take longer so it is possible it is calculating behind the scenes and I'm interpreting as freeze up. Any advice would be appreciated. In the game a player can only make 1 of 3 possible moves in a position and it is not like chess where there are large numbers of moves possible in anyone position and terminal position is difficult to reach.
Thanks
Daz
What counts as larger depth?
Remember that these trees grow exponentially, so if you have 3 options on the first choice, you have 9 when you're 2 deep, 59049 options to check when you're 10 deep, and so on. Another possible reason for things to slow down drastically is if you start using the page file; that is if you're storing your whole tree and suddenly run out of Ram once you get to a "larger" depth. You can probably hear that, or see the blinking hard drive light, if that's contributing.
Your best bet is to get some feedback; get it to print out a new number every x thousand options it checks, so that you can find out instead of guessing at whether it's still trying and how far it has to go. Once you know what it's doing and assuming it is just munching through, look into something like alpha-beta pruning to prevent the tree from growing as quickly.
I have been playing Bejeweled Blitz for a while now. Yes, it is an addiction. In thinking about the game, I have observed that on some boards, the bottom runs dry (no moves) leaving only the top part of the board playable. Frequently that part of the board drys up, and one is left with moves in area cleared by the last move.
The board never runs completely dry, so clearly the program is doing some sorts of calculation that allows it to choose what to drop to prevent it from running dry.
I have noticed in this 'mode' that it is very common for the algorithm to drop jewels which causes more non-dry area to appear in the horizontal area. Perhaps less frequent is a drop which seems designed to open up the bottom part of the board again.
So my question is "How would one go about designing an algorithm guarantee that there is always a move available.?"
I wrote three-in-a-row game a while ago and the way I dealt with that problem is by selecting gems to drop at random and counting all valid moves. If selected gems did not provide at least 1 valid move I would select another set of gems and so on.
This is solvable by exploring the space of possible "jewel drops" and then you apply a rating based on your rules you have asked for. Higher rating could mean follows rule and low ratings mean not following then you choose one slot with the highest rating.
This is not easy to explain, but a rule based approach should do the trick.