Dissertation/ Thesis
Mixed order hyper-networks for function approximation and optimisation
| Τίτλος: | Mixed order hyper-networks for function approximation and optimisation |
|---|---|
| Συγγραφείς: | Swingler, Kevin |
| Συνεισφορές: | Smith, Leslie, Hussain, Amir |
| Στοιχεία εκδότη: | University of Stirling, 2016. |
| Έτος έκδοσης: | 2016 |
| Θεματικοί όροι: | 006.3, Neural networks, Optimisation, Machine Learning, Estimation of Distribution Algorithms, Computers--Data processing, Computer algorithms, Combinatorial analysis--Data processing |
| Περιγραφή: | Many systems take inputs, which can be measured and sometimes controlled, and outputs, which can also be measured and which depend on the inputs. Taking numerous measurements from such systems produces data, which may be used to either model the system with the goal of predicting the output associated with a given input (function approximation, or regression) or of finding the input settings required to produce a desired output (optimisation, or search). Approximating or optimising a function is central to the field of computational intelligence. There are many existing methods for performing regression and optimisation based on samples of data but they all have limitations. Multi layer perceptrons (MLPs) are universal approximators, but they suffer from the black box problem, which means their structure and the function they implement is opaque to the user. They also suffer from a propensity to become trapped in local minima or large plateaux in the error function during learning. A regression method with a structure that allows models to be compared, human knowledge to be extracted, optimisation searches to be guided and model complexity to be controlled is desirable. This thesis presents such as method. This thesis presents a single framework for both regression and optimisation: the mixed order hyper network (MOHN). A MOHN implements a function f:{-1,1}^n →R to arbitrary precision. The structure of a MOHN makes the ways in which input variables interact to determine the function output explicit, which allows human insights and complexity control that are very difficult in neural networks with hidden units. The explicit structure representation also allows efficient algorithms for searching for an input pattern that leads to a desired output. A number of learning rules for estimating the weights based on a sample of data are presented along with a heuristic method for choosing which connections to include in a model. Several methods for searching a MOHN for inputs that lead to a desired output are compared. Experiments compare a MOHN to an MLP on regression tasks. The MOHN is found to achieve a comparable level of accuracy to an MLP but suffers less from local minima in the error function and shows less variance across multiple training trials. It is also easier to interpret and combine from an ensemble. The trade-off between the fit of a model to its training data and that to an independent set of test data is shown to be easier to control in a MOHN than an MLP. A MOHN is also compared to a number of existing optimisation methods including those using estimation of distribution algorithms, genetic algorithms and simulated annealing. The MOHN is able to find optimal solutions in far fewer function evaluations than these methods on tasks selected from the literature. |
| Τύπος εγγράφου: | Thesis Or Dissertation |
| Γλώσσα: | English |
| Σύνδεσμος πρόσβασης: | https://ethos.blethoshyku.com/concern/thesis_or_dissertations/714665 |
| Αριθμός Καταχώρησης: | edsble.714665 |
| Βάση Δεδομένων: | British Library EThOS |
| FullText | Text: Availability: 0 CustomLinks: – Url: https://ethos.blethoshyku.com/concern/thesis_or_dissertations/714665 Name: EDS - British Library EThOS (ns324271) Category: fullText Text: View record in EThOS |
|---|---|
| Header | DbId: edsble DbLabel: British Library EThOS An: edsble.714665 RelevancyScore: 937 AccessLevel: 3 PubType: Dissertation/ Thesis PubTypeId: dissertation PreciseRelevancyScore: 937.178283691406 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Mixed order hyper-networks for function approximation and optimisation – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Swingler%2C+Kevin%22">Swingler, Kevin</searchLink> – Name: Author Label: Contributors Group: Au Data: Smith, Leslie<br />Hussain, Amir – Name: Publisher Label: Publisher Information Group: PubInfo Data: University of Stirling, 2016. – Name: DatePubCY Label: Publication Year Group: Date Data: 2016 – Name: Subject Label: Subject Terms Group: Su Data: <searchLink fieldCode="DE" term="%22006%2E3%22">006.3</searchLink><br /><searchLink fieldCode="DE" term="%22Neural+networks%22">Neural networks</searchLink><br /><searchLink fieldCode="DE" term="%22Optimisation%22">Optimisation</searchLink><br /><searchLink fieldCode="DE" term="%22Machine+Learning%22">Machine Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Estimation+of+Distribution+Algorithms%22">Estimation of Distribution Algorithms</searchLink><br /><searchLink fieldCode="DE" term="%22Computers--Data+processing%22">Computers--Data processing</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+algorithms%22">Computer algorithms</searchLink><br /><searchLink fieldCode="DE" term="%22Combinatorial+analysis--Data+processing%22">Combinatorial analysis--Data processing</searchLink> – Name: Abstract Label: Description Group: Ab Data: Many systems take inputs, which can be measured and sometimes controlled, and outputs, which can also be measured and which depend on the inputs. Taking numerous measurements from such systems produces data, which may be used to either model the system with the goal of predicting the output associated with a given input (function approximation, or regression) or of finding the input settings required to produce a desired output (optimisation, or search). Approximating or optimising a function is central to the field of computational intelligence. There are many existing methods for performing regression and optimisation based on samples of data but they all have limitations. Multi layer perceptrons (MLPs) are universal approximators, but they suffer from the black box problem, which means their structure and the function they implement is opaque to the user. They also suffer from a propensity to become trapped in local minima or large plateaux in the error function during learning. A regression method with a structure that allows models to be compared, human knowledge to be extracted, optimisation searches to be guided and model complexity to be controlled is desirable. This thesis presents such as method. This thesis presents a single framework for both regression and optimisation: the mixed order hyper network (MOHN). A MOHN implements a function f:{-1,1}^n →R to arbitrary precision. The structure of a MOHN makes the ways in which input variables interact to determine the function output explicit, which allows human insights and complexity control that are very difficult in neural networks with hidden units. The explicit structure representation also allows efficient algorithms for searching for an input pattern that leads to a desired output. A number of learning rules for estimating the weights based on a sample of data are presented along with a heuristic method for choosing which connections to include in a model. Several methods for searching a MOHN for inputs that lead to a desired output are compared. Experiments compare a MOHN to an MLP on regression tasks. The MOHN is found to achieve a comparable level of accuracy to an MLP but suffers less from local minima in the error function and shows less variance across multiple training trials. It is also easier to interpret and combine from an ensemble. The trade-off between the fit of a model to its training data and that to an independent set of test data is shown to be easier to control in a MOHN than an MLP. A MOHN is also compared to a number of existing optimisation methods including those using estimation of distribution algorithms, genetic algorithms and simulated annealing. The MOHN is able to find optimal solutions in far fewer function evaluations than these methods on tasks selected from the literature. – Name: TypeDocument Label: Document Type Group: TypDoc Data: Thesis Or Dissertation – Name: Language Label: Language Group: Lang Data: English – Name: URL Label: Access URL Group: URL Data: <link linkTarget="URL" linkTerm="https://ethos.blethoshyku.com/concern/thesis_or_dissertations/714665" linkWindow="_blank">https://ethos.blethoshyku.com/concern/thesis_or_dissertations/714665</link> – Name: AN Label: Accession Number Group: ID Data: edsble.714665 |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edsble&AN=edsble.714665 |
| RecordInfo | BibRecord: BibEntity: Languages: – Text: English Subjects: – SubjectFull: 006.3 Type: general – SubjectFull: Neural networks Type: general – SubjectFull: Optimisation Type: general – SubjectFull: Machine Learning Type: general – SubjectFull: Estimation of Distribution Algorithms Type: general – SubjectFull: Computers--Data processing Type: general – SubjectFull: Computer algorithms Type: general – SubjectFull: Combinatorial analysis--Data processing Type: general Titles: – TitleFull: Mixed order hyper-networks for function approximation and optimisation Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Swingler, Kevin – PersonEntity: Name: NameFull: Smith, Leslie – PersonEntity: Name: NameFull: Hussain, Amir IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 01 Type: published Y: 2016 Identifiers: – Type: issn-locals Value: edsble |
| ResultId | 1 |