Learning Centre

Neural systems in light of distribution codes and information theory

 |  Login

Show simple item record

dc.contributor Aalto-yliopisto fi
dc.contributor Aalto University en
dc.contributor.author Pohja, Seppo
dc.date.accessioned 2012-02-13T12:11:44Z
dc.date.available 2012-02-13T12:11:44Z
dc.date.issued 2000-11-03
dc.identifier.isbn 951-22-5300-3
dc.identifier.issn 1455-0474
dc.identifier.uri https://aaltodoc.aalto.fi/handle/123456789/2303
dc.description.abstract Distribution code is defined as a code that uses the joint probability distribution of a vector of binary-valued random variables to represent numeric values. When applied to neural systems, the values of the source variables (i.e., input variables) determine the relative frequencies of input vectors either deterministically or stochastically. Similarly, the relative frequencies of the output vectors determine the values of the target variables (i.e., output variables). A mathematical model is proposed allowing unified treatment of biological neural systems and binary-valued artificial neural systems. The distribution function of a neural system is defined as the mapping from the distribution of the input vectors to the distribution of output vectors. It is conceptually similar to the input-output function. Information-theoretic properties of probabilistic and deterministic one-dimensional distribution codes are studied. One result is that the channel capacity of such codes has an upper limit with logarithmic asymptotic behaviour in terms of the size of the sample set and that there are codes whose channel capacity reaches the upper limit. The distribution function of a deterministic memoryless neural system using a one-dimensional distribution code is shown to be monotonic. Several other kinds of neural systems are studied and shown not to have this limitation. It is shown that, when the input weights are fixed, the distribution function of a binary neuron can be controlled to much finer degree – it is an adjustable linear combination of several parts – than the input-output function of a real-valued neuron, which can merely be translated. Multidimensional distribution codes can be used to encode source variable values whose number greatly exceeds the number of the physical inputs of the neural system. There are multidimensional distribution codes that make it possible to change the output distribution of the system without changing any of the marginal distributions of the physical inputs. It can be argued that real-valued neurons do not have an analogue for this property. The last part of this work focuses on experimental work and biological neural systems. A literature survey of information theoretic study on biological neural systems shows a need for new methods for generating data in a controlled and easy manner. A simulation-based methodology is proposed, implemented and evaluated. Two sets of simulations are carried out as a proof of concept. They focus on studying the information-theoretic implications of the topology of a pyramidal neuron. en
dc.format.extent 133
dc.format.mimetype application/pdf
dc.language.iso en en
dc.publisher Helsinki University of Technology en
dc.publisher Teknillinen korkeakoulu fi
dc.relation.ispartofseries Helsinki University of Technology Laboratory of Computational Engineering publications. Report B en
dc.relation.ispartofseries 22 en
dc.subject.other Electrical engineering en
dc.subject.other Biotechnology en
dc.title Neural systems in light of distribution codes and information theory en
dc.type G4 Monografiaväitöskirja fi
dc.description.version reviewed en
dc.contributor.department Department of Electrical and Communications Engineering en
dc.contributor.department Sähkö- ja tietoliikennetekniikan osasto fi
dc.subject.keyword neural networks en
dc.subject.keyword distribution codes en
dc.subject.keyword pulse coding en
dc.subject.keyword information theory en
dc.identifier.urn urn:nbn:fi:tkk-002482
dc.type.dcmitype text en
dc.type.ontasot Väitöskirja (monografia) fi
dc.type.ontasot Doctoral dissertation (monograph) en
dc.contributor.lab Laboratory of Computational Engineering en
dc.contributor.lab Laskennallisen tekniikan laboratorio fi
local.aalto.digifolder Aalto_67669
local.aalto.digiauth ask

Files in this item

This item appears in the following Collection(s)

Show simple item record

Search archive

Advanced Search

article-iconSubmit a publication