Autonomous

Autonomous Vehicle Project

I) Intro:

OK, Autonomous Vehicle is a bit of a stretch, it's a rectangle going around a track. But it has sensors, a bit of inertia, and it does not know it's location at any point. It's only inputs are how close the walls are around it.

The idea here was to create an agent that could move it's way around a track. It has no positional input, that would lead to memory of the track and that isn't what I was looking for.
The inputs into the agent are raycasted 'sensors' that calculate the distance from the agent to the nearest obstacle. There are 8 sensors, 1 every 45 degrees around the agent.

Here's what each agent's inputs are like:
(open in a new tab)

II) Model Training:

   a)   The first goal is to solve the environment, to completely circuit the track.
    b) The second goal is to save the weights of the neural network and load them into an agent on another track and see how it performs.

             Demonstration:

Opens in new tab - just a running window. To view the code and run the program use the links at the top of the page.

Run Demonstration

Run Alternate Track

So First if you hit the 1 key it will stop drawing all sensors so it can run faster.
Now you can see the agents going around the track.

If you are patient enough they will complete the track usually around 50 generations. They can solve the environment sooner if by random mutation they get lucky.
Or they can even diverge leading to a never ending cycle.

Or you can hit the l key which will load a pre-trained model and it will instantly be able to solve the track.
As soon as I have time I'll create a new track so the pre-trained model can solve it as well with no additional training.

video

video 1 of agent completing track one

video 2 of agent loading into new track and completing it immediately

Analysis:

I initially programmed this in Java using the Processing library. I then made a Javascript version to be able to be run in a browser.
Saving/loading the neural network weights in java was simple, but accomplishing it in javascript is proving difficult, as soon as I can get that accomplished I'll post it here.
As for now I have linked the Java code in the following gitHub link.

As you can see it takes the agent quite a long time to figure out how to traverse the course. But once it is accomplished the agents neural network is in fact ready to be run an an alternate course.
The link below shows 2 java projects, the one labeled 'alternate track' is solved immediately after loading the agent with the pre-trained model.
note: the reverse is not true. The alternate track is much simpler than the original track, a trained model on the simpler track will not immediately solve the original track, but it does then solve it quicker.

In a later project I learned that neural networks like this do not like negative numbers in their inputs. I had to add 2 extra inputs, one to say the target is on the left, one to say on the right, along with the distance inputs.
I will try to add a similar approach here to see if the agent can perform better.
As you can see, the agent can complete the first right turns relatively quickly, within 5 generations, but then can take upward of 30-40 generations to make the next left turn. Once it makes the first right turns, the network is heavily biased to go right.
Perhaps I can try giving the agent inputs as to the location of the gates. The gates are what gives the agent points when traversed. Currently the agent does not see the gates, it's only inputs are the distances between the walls and the agent.

The big takeaway here:

Pre-trained models are powerful. Once the agent was trained on the hard course, it easily solved the second, simpler, course with no added training needed. As you can see, some of the agents even ran the secondary course backwards with no difficulty at all.