Continuing the theme of “why this version is better than the last”, I’m going to briefly talk about the new neurite system. In biology neurites include both axons and dendrites and grow out of neurons. The same applies for Distributed Neuron.

In the last version (I need to come up with a numbering scheme), neurite growth was very simplistic. Attractants and repellents were scattered around the growth space and influenced how far and in what direction the neurites grew. The static neurons then sprouted neurites which grew in their preferred direction. Once the neurite reached its destination, it randomly selected a neuron in the near vicinity of the neurite head.

There are two important problem with this scheme. The first arises from the fact that neurites only grow in one direction and do so in a completely straight line. This prevents the development of complex structures. Secondly, neurites only make one synapse at the termination point. If, for instance, a dendrite runs into an axon on the way to the destination, no synapse is made.

The new system is a bit more complicated but yields far superior results. After neurons have finished their migration phase, they enter a neurite growth phase. Neurites grow in a more three-dimensional manner now. Axons and dendrites have a direction they tend to grow in but can swerve from their path. In addition, they can also randomly branch along the main growth path, sprouting a new child neurite which continues growing. This branch can also branch, leading to an elaborate tree structure. This three-dimensional growth system allows multiple synapses. Anywhere an axon and dendrite collide a synapse is formed, resulting in a more realistic model.

The details of this (direction, length, branching probability, etc) are fully configurable and specific to the each cell type. Axons have separate parameters from dendrites, leading to different morphologies for each type of neurite. For instance, axons could have a high length value but zero branching leading to a long process. Conversely, dendrites could have medium length and high branching, leading to a large “tree”.

Of course, the neurites are not limited to the stereotypes we see in our brain. Evolution doesn’t really care as long as it works. It should be interesting to see the various cell types and morphologies associated as the system begins to evolve.

And of course, here is a pretty picture. You can see a single cell (magenta) with its axon and dendrite showing. The axon is grey, the dendrite cyan. I don’t know the parameters off hand, but they are relatively generic. The dendrite appears to have a higher branching factor while both appear to have relatively small lengths.

Click for big.

A lot of the work I’ve been doing in lab lately is related to brain development. Your brain is an absolutely amazing mess of pathways, fibers and tracts. The interconnected nature of the brain is far more sophisticated than simple dendrite trees in a big cluster. Furthermore, your brain and my brain are nearly identical, as far as gross anatomy is concerned. You have a hypothalamus, an amygdala, a corpus callosum and a suprachiasmatic nucleus, etc etc ad nauseum. The connections between the various brain regions is likewise similar. This simple fact is actually quite astonishing: every human brain is hardwired to organize into the same gross pattern.

Indeed, hordes of researchers are actively pursuing the mysteries of brain development. Many diseases are caused by improper maturation of the brain. It is becoming increasingly clear that the development of the brain, while complicated, is by no means magic. Various signaling chemicals and peptides direct axons to their destination, either attracting or repelling. Chemical gradients create complex migration behavior depending on your location in the gradient. Importantly, this intricate chemical dance is entirely reproducible - it is the reason your brain looks like mine.

The previous iteration of Distributed Neuron consisted of essentially random placement and a semi-random connection system (through the use of simple “chemical” attractants). This system left a lot to be desired. Results were unpredictable each iteration due to the inherent random nature. Gross anatomy was difficult to develop because neuron placement was random and fixed. Once a neuron was placed, it was permanent. The connection system allowed minor segregation by influencing the direction and distance neurites could travel. It was, in my opinion, too little of an affect to be worthwhile.

Enter the newest revision. Again, neurons are placed into a three-dimensional environment. Each neuron is a 1×1x1 cube. Initial placement is fixed and predictable. This iteration, instead of immediately growing neurites for connections, enters a “migration” phase. Each neuron has a base migration value that defines how far and in what direction it travels. Chemo-emitters are strewn about the three-dimensional space. As neurons migrate, they may change their differentiation pattern according to what chemo-emitters are nearby. This differentiation change affects the base migration properties, altering migration direction. Importantly, no randomness is involved so the results are identical each time.

In this way, neurons not only migrate to different positions but also change their cell fate depending on their location. The non-random nature makes testing/analysis easy because the results are identical from one run to the next. Small variations in the placement and quantity of chemo-emitters can drastically alter the gross anatomy of the network, providing an infinite supply of variations.

There are still pieces I would like to iron out, but for now, the system works very well and can produce some interesting results. Here is one such sample network, from two different views. It consists of 10,000 neurons and has already performed its “migration” phase.

Colors indicate different cell “types”. This particular network has 4 cell types (red, magenta, blue, and one lonely green). As you can see, the network formed a thick magenta sheet. Pressed against this are two separate sheets, one blue and one red. It should be noted that the initial placement of neurons was actually in the bottom-right corner of the screen, meaning the neurons migrated top-left. It should also be known that initial placement was in a small cube, which proceeded to flatten and elongate into a sheet.

Click for big.

“Your results are only as good as the fitness function”

This is generally the difficulty in genetic algorithms. Evolution is a funny thing and capable of coming up with all kinds of wacky solutions to whatever problem is at hand. In a simulated environment where there is a distinct goal, you only want the wacky solutions that pertain to your problem at hand. The fitness function evalutating your gene pool must be very specific. Any loopholes in the fitness function can, and will, be exploited by the simulation. This is one of the greatest difficulties in working with genetic algorithms - creating a tight fitness function that drives evolution towards the solution that you want.

This makes for an interesting situation when your goal is to evolve intelligence. How do you create a fitness function for that? Thoughts on the subject after the jump.
Read the rest of this entry

It turns out digital sex is a lot more complicated than I had originally thought. A key aspect to genetic algorithms is crossover between pairs of individuals. This crossover is made to reflect the genetic pairing seen in most eukaryotes. It is an artificial way to distribute genes from the parents to the children. Crossover is important because without it, a genetic algorithm is little more than a random search. Crossover (theoretically) lets the organisms traverse the fitness landscape quicker because common building blocks are passed from parents to children.

Techniques, thoughts and a cool adaptive study after the jump.
Read the rest of this entry

This is the first of many posts in the new category “Design Decisions”. These posts will chronicle my thought process as I work through various problems, both technical and philosophical.

The first problem that should pop up in any brain simulation project is also one of the most difficult to answer. How does a brain grow? Is the particular structure of the human brain required for intelligence?

Discussion after the jump.
Read the rest of this entry