Water Flow-Like Algorithm with Simulated Annealing for Travelling Salesman Problems

— Water Flow-like Algorithm (WFA) has been proved its ability obtaining a fast and quality solution for solving Travelling Salesman Problem (TSP). The WFA uses the insertion move with 2-neighbourhood search to get better flow splitting and moving decision. However, the algorithms can be improved by making a good balance between its solution search exploitation and exploration. Such improvement can be achieved by hybridizing good search algorithm with WFA. This paper presents a hybrid of WFA with various three neighbourhood search in Simulated Annealing (SA) for TSP problem. The performance of the proposed method is evaluated using 18 large TSP benchmark datasets. The experimental result shows that the hybrid method has improved the solution quality compare with the basic WFA and state of art algorithm for TSP.


I. INTRODUCTION
The Travelling Salesman Problem (TSP) is a classic routing problem that attracts many researchers from different fields, including operational research, mathematics and several scientific and engineering fields. The TSP solution has contributed many real world applications such as decision making, navigation, stock market, transportation and problems. The TSP searches for the shortest path among a set of cities with known distances between a pairs of cities, and it can be formulated as a complete graph with a set of vertices, which is a set of edges weighted by the distance between two vertices (cities). The problem is finding the most inexpensive Hamiltonian cycle associated with visiting each city exactly once and returning to the original city. The TSP is well known as a Non-deterministic Polynomial time (NP-hard) problem, and the determination of the exact solution is difficult [1]. Previous studies have considered many approaches for solving the TSP problem, which are varied in their complexity and efficiency. There have been important advances in the development of exact and approximate algorithms. The exact algorithms are the oldest approaches used to solve the problem. Dantzig, et al. [2] [3][4][5][6] [7] [8][9][10]. The exact methods can only be used for small problem instances. Therefore, for large problem instances, the recent researches have been focused on applying approximate methods, i.e., greedy constructive heuristics, iterative improvement or local search heuristics and metaheuristics, in solving the problem.
Earliest greedy constructive heuristics for the TSP build solutions from scratch by adding an unvisited city in each step based on cost-saving path. These methods are relatively fast to generate feasible solution and the solution can be iteratively improving by using local search heuristics such as 2-opt or 3-opt. Nevertheless, the main drawback of applying the local search heuristics only is the fact that the algorithm is easy to trap in local optima. Past researches has shown meta-heuristics can improved the problem, where the two algorithms such as tabu search (TS) [11], simulated annealing (SA) [12][13][14][15][16], etc, are popular used as local search in meta-heuristic. They have achieved some success with solving TSP in an acceptable amount of computation. Recently, Water Flow-like Algorithm (WFA) [32 -34] has successfully applied to solve TSP [17]. The major key success of WFA for TSP (WFA-TSP) is the fact that WFA-TSP search problem solution space with dynamic population (solution search agents) size. Unlike traditional metaheuristic such as genetic algorithm or ant colony system which search problem solution space with fixed population size. The main advantages WFA is the fact that it presented a dynamic behaviour in which the WFA has the ability to adapt its population number during the optimization process according to problem instance size and complexity. Now a day, the current research trend is move toward developing an efficient and effective algorithm for solving TSP by improving the existing meta-heuristics. Hybridization meta-heuristics with other meta-heuristics is a  famous method used to improve a metaheuristic by  employing the strength of other metaheuristic feature and by  exploiting the complementary character of different  optimization strategies. thus, having an adequate combination of complementary algorithmic concepts can be the key for achieving top performance in solving many hard optimization problems [18,19].
In general, population based meta-heuristics are characterized by a good exploration of the problem search space, whereas single solution-based meta-heuristics are good for exploitation [20]. Therefore, the combination between both populations based meta-heuristics and single solution-based meta-heuristics have been widely recorded in the literature [18,19,[21][22][23][24][25]. One of the major key usage of such combination is the fact that it can provides a good balance between exploration and exploitation, which then can improve the overall performance of the algorithm [26]. The balancing between exploration and exploitation in population based meta-heuristics is performed by increasing the algorithm exploitation capability. The algorithm might combine with strong local search or single solution metaheuristic algorithm in order to achieve a such balance, as the single solution-based meta-heuristics are strong in solution search exploitation Zhao, et al. [27].
Simulated annealing algorithm have been widely used as local search algorithm to improve the population based meta-heuristics [28][29][30][31]. The main feature of using SA is it can improve the solution search exploitation without being trap in local optima. In this study, a combination between WFA algorithm and SA algorithm is proposed in order to improve the overall performance of WFA algorithm by balancing between exploration and exploitation. Making such balance between exploration and exploitation can be achieved by improving WFA algorithm exploitation using SA algorithm. This paper is organised as follows. Section 2 discusses the related works including a brief background of WFA and SA algorithms performance. Section 3 details the propose hybrid WFA with SA for solving TSP problem. The experimental results and conclusion is presented in sections 4 and 5 respectively.

A. Water flow-like algorithm
Water flow-like algorithm (WFA) [32] considered as a new meta-heuristic algorithm, inspired by the natural behaviour of water flowing from higher to lower levels. The water flows can split or merge according to the surface scenery. The advantages of the WFA are that it is selfadaptive and dynamic in its population sizes and parameter settings. The solution agent size is not fixed, unlike in the traditional population based meta-heuristics. The flow number is subject to increase or decrease during the optimisation process. The population size changes are based on the problem diminution and solution quality found by the agents. However, Yang and Wang [32] describe and map the dynamic size of the solution agents based on the natural behaviour of water flows as they split, move and merge.
The first version of the WFA was developed by Yang and Wang as the bin-packing [32]. WFA also has presented is good performance is various domain such [32], manufacturing cell fraction [33] and nurse scheduling problems [34] and recent work [17] has been applied in Travel sales problem. The most advantages of WFA is the fact that its ability to reach the quality solution very fast, especially in large data set. WFA has performed more 90% faster than ACS. This is because the nature of WFA itself behave dynamic population size. The number changes over the time and the needs of agent solution optimized during the finding the solutions.
The basic operations of the WFA for solving the Travelling salesman Problem (TSP) include initialization, flow splitting and moving, flow merging, water evaporation, and water precipitation [17]. Initialization includes parameter setting and initial solution generation. The original WFA is adopted for the parameter setting [2]. The initial solution is generated by the Nearest Neighbour (NN). Normally, the process of enhancement of the WFA only starts after the initialization process. The flow splitting operation is conducted and depends on the flow momentum value. After that, the moving operation is conducted where the design is based on the type of target problem being solved. The algorithm for the flow moving process combines two types of neighbourhood structures, namely the insertion move and k-opt.
The flow merging operation is the operation that combines more than two flow moves to the same location. This operation merges more than one flow into a single flow. Masses and momentums accumulate to compose an integrated flow to reinforce the solution search. This accumulation helps the stagnant flow to escape from the trapped location. This operation checks a flow with others whether they share the same location, and if it does, the latter flow will merge into the former one. The merging operation is executed to eliminate redundant flows.
The water evaporation operation is performed after the flow merging operation. This operation aims to simulate the natural evaporation of water into the air. Water evaporation is executed when evaporation conditions are met. The WFA uses the concept of water evaporation for preparing regeneration flows to increase the wideness of a solution search. When the evaporated water accumulates to a certain amount, it will return to the ground. This process is known as the precipitation operation, and it is the natural rainfall behaviour. This operation achieves redistribution of flows that escape from the local optima and spread the solution search range. Two types of precipitation are used in the WFA known as the enforced precipitation and regular precipitation.
There are many methods that can improve the metaheuristic solution quality such as a hybrid of different metaheuristics, adding heuristics and using some memory to avoid redundant searches. However, this research aims to improve the local search using simulated annealing.

B. Simulated Annealing
Simulated Annealing (SA) is one of the oldest among the metaheuristic methods, which is developed for solving COP's by locating a good approximation to the global optimum of the objective function in the search space. SA was first proposed by [35,36]. The basic idea of SA is stimulating the physical process, where the annealing process requires heating, then slowly cooling of the material in order to increase the size of its crystals and reduce their defects and then obtaining a strong crystalline structure [36,37].
SA has obtained some degrees of successes [12][13][14][15][16]. Although it differs from other meta-heuristics, [12][13][14][15][16] SA has shown a comparative performance in improving local optimization in terms of computation time. This is due to its requirement of less memory space, in comparison to the population-based meta-heuristics. Therefore, SA has a potential to improve several meta-heuristics. As recorded in literature, SA have been widely used as local search algorithm to improve the population based meta-heuristics [28][29][30][31]. The main feature of using SA is it can improve the solution search exploitation without being trap in local optima.

C. Proposed Method
The proposed solution approach has adopted WFA-TSP basic operations of initialization, flow splitting and moving, flow merging, water evaporation and water precipitation [3]. The SA is embedded into WFA-TSP as local search procedure in order to improve solutions by making an intensive search for the best solution that produced by WFA-TSP after flow moving and splitting operation. Ayman in [17] has used 2-opt local search. The search process of the embedded SA depends on the neighbourhood structure of the TSP. There are 4 additional steps added to the original algorithm between steps 6 and 7 in [17], which earlier uses different random move of neighbourhood. However this research has used various structure such as random swap move, 2-opt move, 3-opt move and 4-opt move in SA and the step 7 in [17] move to steps 11 onwards.
Step 7: Start and initialized the SA procedure.
Step 8: Search for new neighbourhood solution of SolutionBest using random neighbourhood move. If the new solution found is better than SolutionBest, Then update the best solution and skip to step 9, else accept the new solution with probability.
Step 9: Update the temperature.
Step 10: Repeat steps 8 and 9 until the termination condition is met. Figure 1 shows the algorithm WFA with SA for TSP adopt WFA for TSP in [17], where line 6 is call the three variants Simulated Annealing using three neighbourhood structures show in Figure 2 16 Update best solution record.

Fig. 1 WFA with Simulated Annealing
The SA implementation starts from a single solution, which is obtained by the best subflows after flow splitting and moving operation. It search systematically a random move generation of solution neighbourhood and allow worst move to be accepted or to be rejected according to the SA criterion. In the pseudo-code, the algorithm starts with initializing the SA parameters such as initial solution Sinitial, initial temperature T0, current temperature Tc, max iteration Maxitr and cooling rate C. Next, each iteration in the SA procedure, a stochastic approach is used to guide the search. It guides the search in the following way. If the generated neighbourhood solution S' using one move of solution S decreases the objective value or leaves it unchanged, then the move is always accepted. Formally, the solution S' is always accepted as current solution in case of ∆≤0, where ∆ =f(s)-f(s') and the f(s) is the value of the objective function. On the other hand, the solution s' is accepted with probability e^(∆/T_c ) in order to allow the search to explore more region in the search space and to prevent it to be trapped in local optimum.
In our implementation and after experimentation, the initial parameter for cooling rate is set to at T0=100, Tc=0.001, and max iteration is set Maxitr=50. The temperature is decreased according to the cooling rate C which was set to 0.5.
In the implemented SA, the random move generation method of solution neighbourhood can be performed using any applicable neighbourhood structure of TSP. However, in this paper we test and implement four common types of neighbourhood structure, namely, swap move (swap two randomly selected cities), 2-opt move, 3-opt move and 4 opt move. Each single neighbourhood structure are implemented and tested separately with SA. The pseudo-code for the four neighbourhood structures used in this paper are illustrated in Figures 2 (a), (b) and (c) , respectively.  In this section, the experimental results of WFA-SA-TSP are presented. However, to measure the performance of WFA-SA-TSP, a number of experiments were carried out using TSP benchmark datasets. Eighteen TSP benchmark datasets, available at TSPLIB [38], were used in the conducted experiments as similar in [17]. The number of cities of the datasets are ranged from 51 to 3705 cities. The WFA-TSP and WFA-SA-TSP were implemented using Java platform JDK 1.6, a Windows environment and a personal computer with an Intel core i5 (3.00 GHz CPU speed and 4 GB RAM). The WFA-TSP and WFA-SA-TSP were implemented to compare them and measure their performance. The experiments used the parameter setting as in [17]. Table 1 and Table 2 present a comparison results in terms of the best solution quality, average iterations number and computation time of the algorithms tested. The tables also show the comparison between WFA-TSP and WFA-SA-TSP in terms of the solution accuracy (in percentage) and the solution deviation of the mean values regarding the bestknown solution. The p-values are provided for solution quality to see if there is any significant difference between the WFA-TSP and WFA-SA-TSP algorithms.
The experiments measured the solution quality of the WFA-SA-TSP obtained from 10 runs for each neighbourhood structure, such as swap, 2-opt, 3-opt and 4opt. That means for each data sets, the WFA-SA-TSP was executed 40 times. The termination condition of the WFA-SA-TSP is activated once it reach 10,000 iterations for each independent run. The number of iterations required to reach the best solution was also considered. The results of top 10 out of 40 independent run that includes the best solution, the time elapsed to find the best solution, the average of the solution cost and the best neighbourhood structure were also recorded.
The results of the experiments are presented in Table 2 in which the performance of the WFA-SA-TSP is compared with the basic WFA-TSP, by considering the best solution found and the average solutions from 10 independent runs. In the table, BKS represents the best-known solution, Mean represents the average, best represents the best solution found and Best NS represents the best neighbourhood structure of 4o independent ruins. It can be seen that the results of the WFA-SA-TSP when the embedded SA use the four-neighbourhood structure are generally better than WFA-TSP in large size datasets. Where the WFA-SA-TSP outperformed the WFA-TSP 9 datasets in terms of the best solution found. These datasets are lin318, rat576, rat787, u1060, fl1400, d1655, u1817, d2103 and fl3795. In small size datasets, both the WFA-TSP and the WFA-SA-TSP can obtain the optimal solution in 9 out of 18 datasets in terms of the best solution found. The WFA-SA-TSP also outperformed the WFA-TSP in nearly all datasets in terms of the average solutions, except for datasets eil76, eil101 and ch150. Table 1 shows the average computational time of both algorithms changes when the number of cities is increased. It can be seen from the table that the average computation time of both algorithms is clearly increasing with the larger dataset size in both algorithms. From the table, it can be deduced that with small, medium and large dataset size, the computation time of WFA-SA-TSP is only slightly higher that WFA-TSP.  Table 2 presents a comparison of the experimental results by using the percentage deviations of the average and best solutions from the best-known solution of the WFA-TSP and WFA-SA-TSP. The percentage deviation of the average solution from the best-known solution as PDavg while the calculated deviation of the best solution from the best-known solution is denoted as PDbest. The table shows that the deviation of the average solution from the PDavg of the WFA-SA-TSP for the major datasets was significantly better than that of the other algorithms. The difference in the performance of WFA-SA-TSP in each dataset compared with that of the WFA-TSP in terms of the PDavg value. Table 2 also shows that the deviation of the best solution from the best-known solution (PDbest) of the WFA-SA-TSP is generally better than that of WFA-TSP for datasets lin318, rat576, rat787, u1060, fl1400, d1655, u1817, d2103 and fl3795. The table also demonstrates the significant of the statistical results of all datasets. It can be seen that WFA-SA-TSP with 4-opt has a significant improvement in terms of the solution quality where the p-values for the large size datasets are less than 0.05.  Figure 3 shows a general WFA-SA-TSP searching behaviour and the performance of the algorithm in terms of exploration and exploitation when using SA with swap, 2opt, 3-opt and 4-opt move to solve ch130 dataset problem. The use of SA as local search algorithm has improved the algorithm performance in finding good solution in less amount of time, which prevents the algorithm from early convergence and avoids it to trap in local optima.
The dashed lime arrow points to the best objective values that obtained by WFA-SA-TSP in respect to WFA-TSP. The pointed area indicates that the exploitation process which using SA is performed better in this region comparing to the WFA-TSP. On the other hand, the solid line arrow point to a sample of solution exploration regions with worst solution area more that WFA-TSP which are clearly shows that the WFA-SA-TSP can explore move more solution area in the problem search space. As shown in the figure sketch a, b, c and d, a good balancing between exploration and exploitation process is clearly seen in data sets ch130, lin318, rat783 and d1655, respectively, where the shaded area of the local best values are fluctuating somehow in more systematic way.  [17] in terms of solution quality using in 18 TSP datasets (see Table 3).   [39] algorithms. The WFA-SA-TSP also better compare with discrete invasive weed optimization algorithm, in all three data sets shown in [40].

IV. CONCLUSION
WFA for TSP has shown its abilities in solving TSP, as it characterized with dynamic population size which make it more suitable of solving variant instance size of TSP without reconfiguring the parameter setting. In this paper, we propose an improved version of WFA for solving TSP problem (WFA-SA-TSP). The performance of WFA-SA-TSP was tested using 18 TSP benchmark datasets considering the best solution, average solution, and standard deviation, the percentage deviation of the average solution to the best-known solution and the percentage deviation of the best solution to the best-known solution. The experimental results show that the WFA-SA-TSP is generally outperforms the WFA-TSP. This study demonstrates that the WFA-TSP. can be subject for further improvement in terms of solution search exploration and exploitation using various neighborhood structure.
Since WFA-TSP consists of several components that influencing in the algorithm therefore many potential improvements can be made using the WFA algorithm for the TSP especially improving the water flow splitting and movement using more better neighbor search strategies.