#ml-agents

a compilation of the past few months on and off experimenting with ml-agents.

the simple idea is that you give the algorithm the position of the agent (the cube) and the position of the reward (the ball) and you give the agent some actions it can take at each point in time, and it experiments on which actions to take until it learns to take the correct actions at the right time to get to the reward eventually.

it was pretty awesome seeing the agent learn and fail repeatedly and then eventually succeed. i imagine maybe similar to seeing your kid take their first steps.

results were pretty instant and predictable for getting to and chasing the ball, but the setup is a bit frustrating to iterate so it was a struggle to up the complexity. i tried to get the agent to both run away from an enemy (negative -1 reward) and get to the ball at the same time, but the creature never learned to run away. i think maybe i need to train the two behaviours separately but haven't gotten to trying that yet.

i also tried to see if putting 2 agents in the scene and giving them a negative reward upon touching each other would result in some emergent behaviour, such as running away from each other and going a different route to get to the ball instead, but the most that happened was the agents got stuck in this lock step state where they just inched towards the ball and not towards each other.

i want to go back to experimenting with adding more actions to the agents to let them fight each other a bit, to see if that would create more emergent behaviour.

but before i got to that step i got kind of impatient and i just decided to through them into a graybox scene with a character controller and run around and make them chase the ball. that was way more fun than having to do the whole ML setup again.

and then i added legs to them without retraining the agents and they were definitely struggling to keep up on them LOL.