Introduction
Hopfieldneuralnetworkisarecurrentneuralnetwork,inventedbyJohnHopfieldin1982.Hopfieldnetworkisaneuralnetworkthatcombinesstoragesystemandbinarysystem.Itguaranteesconvergencetothelocalminimum,butconvergestothewronglocalminimum(localminimum)insteadoftheglobalminimum(globalminimum)mayalsooccur.TheHopfieldnetworkalsoprovidesamodelthatsimulateshumanmemory.
DiscreteHopfieldnetworkisasingle-layernetworkwithnneuronnodes,andtheoutputofeachneuronisconnectedtotheinputofotherneurons.Eachnodehasnoself-feedback.Eachnodecanbeinapossiblestate(1or-1),thatis,whenthestimulationoftheneuronexceedsitsthreshold,theneuronisinastate(suchas1),otherwisetheneuronwillalwaysbeInanotherstate(suchas-1).Theentirenetworkhastwoworkingmodes:asynchronousmodeandsynchronousmode.
Construction
TheunitsoftheHopfieldnetworkarebinary,thatis,theseunitscanonlyaccepttwodifferentvalues,andthevaluedependsonwhetherthesizeoftheinputreachesthethreshold.TheHopfieldnetworkusuallyacceptsavalueof-1or1,butitcanalsobe0or1.Theinputisprocessedbythesigmoidfunction.Thesigmoidfunctionisdefinedas:
Itisusedtoreducetheinputtotwoextremevalues.
EverypairofHopfilednetworkunitsiandjhaveapairofconnectionswithacertainweight{\displaystylew_{ij}}.Therefore,theHopfilednetworkcanbedescribedasacompleteundirectedgraphG=,whereVisasetofartificialneurons.
TheconnectionoftheHopfilednetworkhasthefollowingcharacteristics:
(noneuronsconnectedtoitself)
(Theconnectionweightissymmetric)
Therequirementofweightsymmetryisanimportantfeature,becauseitensuresthattheenergyequation(theprocessofconvergingtoacertainpointofthefunctionisconvertedintoenergyfrompotentialenergy)whentheneuronisactivatedMonotonouslydecreasing,asymmetricalweightsmaycauseperiodicincreasesornoise.However,theHopfilednetworkalsoprovesthatthenoiseprocesswillbelimitedtoasmallrangeanddoesnotaffectthefinalperformanceofthenetwork.
Application
AssociativememoryfunctionisanimportantapplicationrangeofdiscreteHopfieldnetwork.Torealizeassociativememory,thefeedbacknetworkmusthavetwobasicconditions:
①Thenetworkcanconvergetoastableequilibriumstateanduseitasthesamplememoryinformation;
②haveTheabilitytorecallistheabilitytorecallcompletememoryinformationfromacertainincompleteinformation.TheprocessofdiscreteHopfieldnetworktorealizeassociativememoryisdividedintotwostages:learningandmemorystageandassociativememorystage.Inthelearningandmemorystage,thedesignerdeterminesasetofappropriateweightsthroughacertaindesignmethodtomakethenetworkmemorythedesiredstablebalancepoint.TheLenovorecallstageistheworkingprocessofthenetwork.
ThediscreteHopfieldnetworkusedforassociativememoryhastwooutstandingcharacteristics:thatis,memoryisdistributed,whileassociationisdynamic.ThelimitationsofthediscreteHopfieldnetworkaremainlymanifestedinthefollowingpoints:
①Thefinitenessofmemorycapacity;
②Theassociationandmemoryofpseudo-stablepoints;
③Whenthememorysamplesareclose,thenetworkcannotalwaysrecallthecorrectmemory,etc.Inaddition,thebalanceandstabilitypointofthenetworkcannotbearbitrarilyset,andthereisnouniversalwaytoknowthebalanceandstabilitypointinadvance.
See
Boltzmannmachine-likeaHopfieldnetwork,annealingGibbssamplingcanbeusedinsteadofgradientdescent
IssingModel
HebbTheory