In the ultra-dense network environment

each access point is deployed in the hotspot area

which forms a complex heterogeneous network. Users need to choose the appropriate network to access

so as to achieve the best performance. Network selection problem is to choose the optimal network for the user

so that the user or network performance reaches the best. In order to solve the access selection problem of users in ultra-dense networks

we proposed an ultra-dense network access selection algorithm based on the improved deep Q network (DQN)

considering network states

user preferences

and service types

and combining with load balancing strategies. First

by analyzing the influence of network attributes and user preferences on network selection

the appropriate network parameters were selected as the parameters of the access selection algorithm. Then

the problem of network access selection was modeled by Markov decision-making process

and the states

actions

and reward functions of the model were designed. Finally

the optimal network strategy was obtained by using DQN to solve the network selection model. In addition

the target function of traditional DQN was optimized to avoid overestimation of Q value by DQN

and a priority experience replay mechanism was introduced to improve learning efficiency. Simulation results show that the method could well solve the problem of overestimation of traditional DQN

accelerate the convergence of neural network

effectively reduce user congestion

and improve network throughput performance.