In my last blog post I talked about the underlying principle for holography, neural networks and quantum computers. You can check it out here: The Holographic Model – A key in understanding neural networks and quantum computers?

Let’s take a step forward and incorporate some principles of holography into neural networks.

Properties of HNNs

Complex numbers instead of weights

Holographic neural networks need to work with complex numbers to represent amplitude and phase information instead of regular weights. After performing the FFT decomposition of an image (or text?), the frequency domain and phase information are obtained. An architecture called CVNN (Complex-Valued Neural Networks), which works with complex numbers, already exists. I found a paper online titled ‚Theory and Implementation of Complex-Valued Neural Networks.‘ The network must be capable of performing complex backpropagation, having complex layers, and using complex activation functions.

Interference layers

Interference layers could play a role analogous to attention mechanisms in standard neural networks but with a fundamentally different approach. By simulating light wave superposition (constructive and destructive interference), these layers could naturally highlight relationships between input data points, potentially improving tasks like image segmentation or clustering in high-dimensional spaces.

Other properties

Also the design should include: coherence, non-locality, scale-invariance, relational grouping, and fast similarity search.

Coherence

Definition: Coherent systems maintain the phase relationships between components, ensuring that the whole system works in harmony.

Non-locality

Definition: The information is not stored in a single location but is distributed across the system.

Scale-Invariance

Definition: The representation should be independent of the physical size of the input.

Relational Grouping

Definition: Data points that are semantically related should be grouped together.

Fast Similarity Search

Amplitude-Phase Matching: Represent data using both amplitude and phase.
Hashing in Frequency Space: Use locality-sensitive hashing on the frequency domain representation for fast approximate nearest neighbor search.
Holographic Memory Systems: Store data holographically, where interference patterns allow for instantaneous similarity retrieval.

To be continued

I don’t know how to implement all these properties, and I’m not sure if such a neural network is even possible. However, it’s fun to combine ideas from different fields and speculate about how it could work.