Performance Improvement of a Parsimonious Learning Machine Using Metaheuristic Approaches
Abstract : Autonomous learning algorithms operate in an online fashion in dealing with data stream mining, where minimum computational complexity is a desirable feature. For such applications, parsimonious learning machines (PALMs) are suitable candidates due to their structural simplicity. However, these parsimonious algorithms depend upon predefined thresholds to adjust their structures in terms of adding or deleting rules. Besides, another adjustable parameter of PALM is the fuzziness in membership grades. The best set of such hyper parameters is determined by experts' knowledge or by optimization techniques such as greedy algorithms. To mitigate such experts' dependency or usage of computationally expensive greedy algorithms, in this work, a meta heuristic-based optimization technique, called the multimethod-based optimization technique (MOT), is utilized to develop an advanced PALM. The performance has been compared with some popular optimization techniques, namely, the greedy search, local search, genetic algorithm (GA), and particle swarm optimization (PSO). The proposed parsimonious learning algorithm with MOT outperforms the others in most cases. It validates the multioperator-based optimization technique's advantages over the single operator-based variants in selecting the best feasible hyperparameters for the autonomous learning algorithm by maintaining a compact architecture.
? The idea is to iteratively breed a new child or two, assess their fitness, and then reintroduce them directly into the population itself, killing off some preexisting individuals to make room for them.
? The new locations of children are entirely based on the existing parents and which combinations we can make of adding and subtracting them.
? The difficulty here is, of course, that the swapped-in subgraph will be disjoint with the existing nodes in that individual’s graph.
? Then after a while a few new rules are bred from the population and reinserted into it, displacing some existing low-fitness rules.
? If we create a new rule, we check first to see if it’s identical to an existing rule.
? Therefore, the selection of proper thresholds has a dependency on experts’ knowledge or requires several iterations to achieve the desired accuracy in regression-based problems.
? However, the performance of DE for solving any optimization problem depends on its control parameter. Such fixed parameters may not work well for a wide range of optimization problems.
? To overcome such limitations, in this research, we formulate the problem as an EA-based optimization problem, where the objective is to maximize the predictive accuracy of the PALM in modeling various dynamical systems from data streams.
? In this work, the problem of selecting the most feasible hyper-parameters of PALM has been formulated as a challenging optimization problem.
• In contrast, cooperative coevolution, proposed by Mitchell Potter and Ken De Jong,105 strives to find individuals who work well together.
• The Sequential approach is the original method proposed by Potter and De Jong, and it still remains popular.
• If you’re a graduate student and would like some tough feedback on your proposed thesis work, a great pick is the GECCO Graduate Student Workshop, where you present your work in front of a panel of luminaries who then critique it (and they’re not nice!).
• The most common form of fitness sharing, proposed by David Goldberg and Jon Richardson, requires you to define a neighborhood radius s.
? The structural simplicity of PALM provides us the courage to dig further to improve its performance through the usage of EAs, where PALM’s structural compactness should be maintained.
? In that high-dimensional space, the performance of a FLS may form a hyper-surface to achieve predefined modeling or predictive accuracy.
? In DE, the greedy selection procedure is employed by selecting the better among new solutions and their parents, which gives the DE advantages over the GA in converging the performance.
? To observe the performance improvement of the MOT over some benchmark EAs, we have used a PSO algorithm, and recent self-adaptive real-coded GA.
? To observe the performance improvement of PALM, it has been evaluated using three different synthetic and real-world data sets.
|