A Novel Statistical and Neural Network Combined Approach for the Cloud Spot Market
Abstract : The instance price in the Amazon EC2 spot model is often much lower than in the on-demand counterpart. However, this price reduction comes with a decrease in the availability guarantees. To our knowledge, there is no work that accurately captures the short-term trade-off between spot price and availability, and does long-term analysis for spot price tendencies in favor of user decision making. In this work, we propose a utility-based strategy, that balances cost and availability of spot instances and is targeted to short-term analysis; and a LSTM neural network framework for long term spot price tendency analysis. Our experiments show that, for r4.2xlarge, 90% of spot bid suggestions ensured at least 5.73 hours of availability, with a bid price of approximately 38% of the on-demand price. The LSTM experiments predicted spot price tendencies for several instance types with low error. Our LSTM framework predicted an average value of 0.19 USD/hour for the r5.2xlarge instance type, which is about 37% of the on-demand price. Finally, we used our combined mechanism on an application that compares thousands of SARS-CoV-2 sequences and show that our approach is able to provide good choices of instances, with low bids and very good availability.
? Even though our focus is particularly on DL implementations of financial time series prediction studies, it will be beneficial to briefly mention about the existing surveys covering ML-based financial time series forecasting studies in order to gain historical perspective.
? NLP based ensemble models that integrate data semantics with time-series data might increase the accuracy of the existing models.
? In our survey, we wanted to review the existing studies to provide a snapshot of the current research status of DL implementations for financial time series forecasting.
? We grouped the studies according to their intended asset class along with the preferred DL model associated with the problem.
? By treating Cloud resources as assets, the study formulates the spot service pricing function as an option pricing problem.
? PADB is supposed to achieve a nearoptimal bidding solution to the profit maximization problem from the service broker’s perspective.
? On the one hand, service broker and secondary provider are essentially consumers of spot instance service, and therefore they have to deal with the same issues as the normal service consumers, such as bidding strategies and fault tolerance.
? However, given the observations that spot prices frequently surpassed on-demand prices in the price history across different instance types and datacenters, the consumers may have issued irrational biddings unless they tried to bid as high as possible to decrease the chance of service interruptions.
• The spot instance was frstly proposed by Amazon, and then, other cloud service providers also proposed spot instances, such as Alibaba and Tencent.
• We propose a price prediction method for spot instance, and taking Amazon as the representative to elaborate the method.
• A price prediction model based on k-Nearest Neighbors (kNN) regression is proposed to predict the future price of cloud spot instances.
• The method proposed in this paper is applicable to the spot instance price prediction of other cloud providers.
• Spot instances allow users to propose a bid, which is the maximum price user can aford.
? A similar function is described in, while a simple auto-regression (AR) method is further used to model demand quantity independently of the price.
? After mapping these parameters to the BSM model, the BSM equation can be used to price the Cloud resources .
? In particular, the up-time fraction of the total time interval is used to measure service availability in.
? Although it is impossible to determine an optimal spot instance for migration, three different heuristics (i.e. Lowest price, Lowest failure rate, and Highest failure rate) can be used to facilitate selecting the next instance type.
|