Following is an endeavour to bridge the gap between human and computational thinking. We are all bounded by defects when it comes to memory, clarity in thought or general computation with our brain. These defects stop us from reaching our maximum potential and to work on things real time. With the invention of computers, which are just an extension of our brain , we can divide these tasks and focus on the things we are better at. However, the agony of punching digits into a calculator or constant googling to search for what we exactly need , is inefficient and extremely slow.
What if there was a device connected to our brains that could fill this knowledge gap real-time? If such a device was to be made, it would require the combined knowledge of biology, machine learning and electronics. This project aims to provide an enhanced version of the human brain neural net architecture or "host", that dynamically changes its functionality and performance due to an external "carrier". The carrier will cut the host and glue itself in between the layers of the host, resulting in a combined symbiotic neural net to modify the final output. In this study, the "host" will have an auto-encoder architecture whereas the "carrier" can be in the form of a function, layer or new neural net altogether.
Architechture
Out[]=
This process can be understood by relating it with the Lysogenic Cycle of how a virus attacks the DNA and is broken down in three simple stage: encoding, interference and decoding.
1
.In the encoding stage, the host will start forward propagation on the testing data. This depicts the normal functioning of a cell in the body.
2
.This will be interrupted by the carrier, which will do arbitrary symbolic computing not easily implementable by the conventional host. The I/O type signature will be kept intact to ensure the carrier can digest the output of stage 1 and, stage 3 can decode the output of the carrier effectively.
Similarly, the virus will attach itself to the host cell and inject its genetic material . Viral DNA integrates into a specific place in the cell's DNA by identifying genetic markers.
Similarly, the virus will attach itself to the host cell and inject its genetic material . Viral DNA integrates into a specific place in the cell's DNA by identifying genetic markers.
3
.Finally, forward propagation will be completed in the decoding stage. The output of both the host net and host cell has changed due to the foreign substance.
Implementation
Implementation
The “LeNet Trained on MNIST Data” has been used for the host cell whereas a newly designed neural net has been used for the carrier. The host cell is cut in between layers 6 and 7. This has explicitly been chosen by me but a follow-up project would be to identify where to make the cut for a given carrier.
Next the decoder of the carrier is trained in a GAN like architecture where it competes withe decoder of the host to reduce the cost function. After training, the encoder is extracted from this network and is ready to go!
Out[]=
The encoder of the carrier is trained in a similar way. The carrier is trained without any active help from the host, giving one the capability of inserting arbitrary symbolic programs in neural nets inside the brain.
Out[]=
The carrier consists of the encoder, decoder and a Wolfram language function that is responsible to tweak the results. The function chosen here is: Count[IntegerDigits[#,2],0]. Next the carrier attaches itself to the host, forming a symbiotic net that looks as follows:
In[]:=
carrier=Module[{decoder=carrierDecoder1,encoder=pipe[Framed[Style[#,Black],BackgroundWhite,FrameStyleWhite,FrameMargins5]&,Rasterize[#,RasterSize{100,100}]&,ImagePad[#,-30]&,Echo[#,"Image inside carrier :"]&,carrierEncoder1],core=(Count[IntegerDigits[#,2],0]&)},pipe[decoder,core,encoder]]
This symbiotic net is tested with the image of the number 8. Because of the carrier, it spits out the number 3. We have hence proved that our symbiotic net is indeed working.
In[]:=
//pipe[host1,(Echo[NumericArray@#,"hostPart1 "];#)&,carrier,(Echo[NumericArray@#,"carrier "];#)&,host2]
»
hostPart1 NumericArray
»
Image inside carrier :
»
carrier NumericArray
Out[]=
3
The carrier can take in any sort of function that has an input type of a 1 digit number and output as an arbitrary image. Below is an example of another type of function along with its changed results when is passed through the symbiotic net via the function Symbiosis[func_][image_].
ClearAll[func]func[n_]:=Module{x,volume},volume=Integrate[1,x∈Simplex[IdentityMatrix[n]]];//Rationalizen,&//Denominator;Symbiosis[func]
1/n
volume
1
10
»
hostPart1 NumericArray
»
Image inside carrier :
»
carrier NumericArray
Out[]=
1
Another interesting application is to feed the decoder of the carrier with a completely unique image like that of a barcode.The point of these functions are to trick the host and make it produce things that it doesn’t really understand , but still passes through. This is similar to how a virus attacks cells in the body.
ClearAll[func]func[n_]:=Switchn,0,π,1,
2
,2,,3,n,4,EulerGamma,_,RandomInteger[100,n]//Total//N[n,50]&//ToString//BarcodeImage[n,"QR"]&Symbiosis[func]
Out[]=
»
hostPart1 NumericArray
»
Image inside carrier :
»
carrier NumericArray
Out[]=
5
Future Plan
Future Plan
Some of the future plans of the project include:
1
.Create carrier for all categories of host nets including image, audio or numerical
2
.Create a classification system which depicts the compatibility of various host-carrier combinations
3
.Train the carrier encoder without passing backpropagation or freezing any parts of the host. This might be done by replacing the pre-trained decoder of the carrier by the host decoder during the training cycle using the function NetGANOperator[]
Have we solved the problem of brain machine interface? Absolutely not! The work done in this project is a step in the right direction to create biologically-inspired machine learning architectures that can be tweaked using symbolic functional programming.
Keywords
Keywords
◼
Host Net
◼
Carrier Net
◼
Symbiotic Net
Acknowledgment
Acknowledgment
Mentor: Silvia Hao
I would like to thank my mentor Silvia Hao for all her tireless assistance and general camaraderie. Without her guidance, this project would not have been possible.
Thanks also to the invaluable support of Wolfram Summer School team which makes the 2020 Summer School possible in these unprecedented circumstances.
Thanks also to the invaluable support of Wolfram Summer School team which makes the 2020 Summer School possible in these unprecedented circumstances.