The recurrent neural networks (RNN) with richly distributed internal states and flexible non-linear transition functions, have gradually overtaken the dynamic Bayesian networks in modeling highly structured sequential data. These data, which may come from speech and handwriting, often contain complex relationships between the underlying variational factors such as speaker characteristic and the observed data. The standard RNN model has very limited randomness or variability in its structure, which comes from the output conditional probability model. To improve the variability and performance, we study the new latent variable models with novel regularization methods. This paper will present different ways of using high level latent random variables in RNN to model the variability in the sequential data. We will explore possible ways of using adversarial methods to train a variational RNN model. Through theoretical analysis we show that, contrary to competing approaches our schemes are theoretical optimum in the model training and the symmetric objective function in the adversarial training provides better model training stability. Our approach also improves the posterior approximation in the variational inference network by a separated adversarial training step. Numerical results simulated from TIMIT speech data show that reconstruction loss and evidence lower bound converge to the same level and adversarial training loss converges in a stable course. The results also show our approach of regularization provides stability and smoothness on probability distribution distance minimization between prior and posterior of the latent variables.

Time Series Prediction (TSP) has been used in mobile network traffic data analysis to produce predictive results for network planning and resource allocation. In the first part of this thesis, we propose a novel method of predicting mobile network traffic using neural networks based on conditional probability modeling between adjacent data windows in the time series sequence. Firstly, we develop a pre-processing method to aggregate the raw traffic log data and sample the aggregated time series to adjacent data windows, as training samples. Secondly, we use neural networks to parameterize the conditional probability between adjacent data windows and estimate the probability by training the neural networks with sampled data. The estimated conditional probability is then used to ensemble the prediction. Thirdly, we show theoretically that the prediction based on all historical data is equivalent to the prediction based on just previous data window, given the estimation of conditional probability between adjacent data windows. We also analyze computation complexity and show that seasonality will reduce the computational complexity. In the experiment, we compare the prediction performance among the models with different seasonality, sample size and number of hidden layers, and show that the proposed schemes achieve better prediction accuracy than state-of-the-art.

The Recurrent Neural Networks (RNN) with richly distributed internal states and flexible non-linear transition functions, havegradually overtaken the dynamic Bayesian networks in modeling highly structured sequential data. These data, which may come fromspeech and handwriting, often contain complex relationships between the underlying variational factors such as speakercharacteristic and the observed data. The standard RNN model has very limited randomness or variability in its structure, which comes from the output conditional probability model. To improve the variability and performance, we study the new latent variable models with novel regularization methods. The second part of this thesis will present different ways of using high level latent random variables in RNN to model the variability in the sequential data. We will explore possible ways of using adversarial methods to train a variational RNN model. Through theoretical analysis we show that, contrary to competing approaches our schemes have theoretical optimum in the model training and the symmetric objective function in the adversarial training provides better model training stability. Our approach also improves the posterior approximation in the variational inference network by a separated adversarial training step. Numerical results simulated from TIMIT speech data show that reconstruction loss and evidence lower bound converge to the same level and adversarial training loss converges in a stable course. The results also show our approach of regularization provides stability and smoothness on probability distribution distance minimization between prior and posterior of the latent variables. 

In the last part of this thesis, we  studies potential challenges and opportunities in intelligent road traffic sensing from the data mining and learning point of view with mobile network generated data. This part of the thesis only include qualitative analysis. Firstly, we classify the data resources available in the commercial mobile network according to different taxonomy criteria. Then, we outline the broken-down problems that fit in the framework of road traffic sensing based on mobile user network log data. We study the existing data processing and learning algorithms on extracting road traffic condition information from a large amount of mobile network log data. Finally we make suggestion on potential future work for road traffic sensing on data from mobile networks.

Time Series Prediction (TSP) har använts i mobilnätverkstrafikdataanalys för att producera prediktiva resultat för nätverksplanering och resursallokering. I den första delen av denna avhandling föreslår vi en ny metod för att förutsäga mobilnätverkstrafik med hjälp av neurala nätverk baserat på villkorad sannolikhetsmodellering mellan intilliggande datafönster i tidsseriesekvensen. För det första utvecklar vi en förbearbetningsmetod för att aggregera rå trafikloggdata och ta prov på de aggregerade tidsserierna till angränsande datafönster, som träningsexempel. För det andra använder vi neurala nätverk för att parametrisera den villkorliga sannolikheten mellan intilliggande datafönster och uppskatta sannolikheten genom att träna de neurala nätverken med samplade data. Den uppskattade villkorade sannolikheten används sedan för att sammanställa förutsägelsen. För det tredje visar vi teoretiskt att förutsägelsen baserad på alla historiska data är ekvivalent med förutsägelsen baserad på just tidigare datafönster, givet uppskattningen av villkorlig sannolikhet mellan angränsande datafönster. Vi analyserar också beräkningskomplexiteten och visar att säsongsvariationer kommer att minska beräkningskomplexiteten. I experimentet jämför vi prediktionsprestandan bland modellerna med olika säsongsvariationer, urvalsstorlek och antal dolda lager, och visar att de föreslagna scheman uppnår bättre prediktionsnoggrannhet än toppmoderna.

De Recurrent Neural Networks (RNN) med rikt fördelade interna tillstånd och flexibla icke-linjära övergångsfunktioner, hargradvis gått om de dynamiska Bayesianska nätverken vid modellering av högstrukturerade sekventiella data. Dessa data, som kan komma fråntal och handstil, innehåller ofta komplexa samband mellan de underliggande variationsfaktorerna såsom talareegenskaper och observerade data. Standard-RNN-modellen har mycket begränsad slumpmässighet eller variabilitet i sin struktur, vilket kommer från den betingade sannolikhetsmodellen för output. För att förbättra variabiliteten och prestandan studerar vi de nya latenta variabelmodellerna med nya regulariseringsmetoder. Den andra delen av denna avhandling kommer att presentera olika sätt att använda latenta slumpvariabler på hög nivå i RNN för att modellera variabiliteten i sekventiell data. Vi kommer att utforska möjliga sätt att använda kontradiktoriska metoder för att träna en variationsrik RNN-modell. Genom teoretisk analys visar vi att, i motsats till konkurrerande tillvägagångssätt, har våra system teoretiskt optimum i modellträningen och den symmetriska objektivfunktionen i den kontradiktoriska träningen ger bättre modellträningsstabilitet. Vårt tillvägagångssätt förbättrar också den bakre approximationen i det variationsmässiga slutledningsnätverket genom ett separerat kontradiktoriskt träningssteg. Numeriska resultat simulerade från TIMIT-taldata visar att rekonstruktionsförlust och nedre gräns för bevis konvergerar till samma nivå och kontradiktorisk träningsförlust konvergerar i en stabil kurs. Numeriska resultat visar också att vårt tillvägagångssätt för regularisering ger stabilitet och jämnhet på sannolikhetsfördelningen minimering av avståndsförlust mellan föregående och bakre av de latenta variablerna. Detta är en klar förbättring än den ursprungliga RNN-modellen.

I den sista delen av denna avhandling studerar vi potentiella utmaningar och möjligheter inom intelligent vägtrafikavkänning ur datautvinnings- och inlärningssynpunkt med mobilnätsgenererad data. Denna del av avhandlingen innehåller endast kvalitativ analys. För det första klassificerar vi de tillgängliga dataresurserna i det kommersiella mobilnätet enligt olika taxonomikriterier. Sedan skisserar vi de nedbrutna problemen som passar inom ramen för vägtrafikavkänning baserat på mobilanvändarnätverksloggdata. Vi studerar befintliga databearbetnings- och inlärningsalgoritmer för att extrahera information om vägtrafiktillstånd från en stor mängd mobilnätverksloggdata. Slutligen ger vi förslag på potentiellt framtida arbete för vägtrafikavkänning på data från mobilnät.

Mobile operators collect and store the network generated traffic data for analysis. Time Series Prediction (TSP) has been used in mobile network traffic data analysis to produce predictive results for network planning and resource allocation. We propose a novel method of predicting mobile network traffic using neural networks based on conditional probability modeling between adjacent data windows. Firstly, we develop a pre-processing method to aggregate the raw traffic log data and sample the aggregated time series to adjacent data windows, as training samples. Secondly, we use neural networks to parameterize the conditional probability between adjacent data windows and estimate the probability by training the neural networks with sampled data. The estimated conditional probability is then used to ensemble the prediction. Thirdly, we show theoretically that the prediction based on all historical data is equivalent to the prediction based on just previous data window, given the estimation of conditional probability between adjacent data windows. We also analyze computation complexity and show that seasonality will reduce the computational complexity. In the experiment, we compare the prediction performance among the models with different seasonality, sample size and number of hidden layers, and show that the proposed schemes achieve better prediction accuracy than state-of-the-art. IEEE

The highly structured sequential data, such as from speech and handwriting, often contain complex relationships between the underlaying variational factors and the observed data. This paper will present different ways of using high level latent random variables in RNN to model the variability in the sequential data. We have developed the two-steps training algorithms of such RNN model under the VAE (Variational Autoencoder) principle. We proposed novel approach of using adversarial training to regularize the latent variable distributions in the variational RNN model. Contrary to competing approaches, our approach has theoretical optimum in the model training and provides better model training stability. Our approach also improves the posterior approximation in the variational inference network by a separated adversarial training step. Numerical results simulated from TIMIT speech data show that reconstruction loss and evidence lower bound converge to the same level and adversarial training loss converges to 0.

With the improvement of the storage and big data processing technology, mobile operators are able to extract and store a large amount of mobile network generated user behavior data, in order to develop various intelligent applications. One interesting application based on these data is traffic sensing, which uses techniques of learning human mobility patterns from updated location information in network interaction log data. Mobile networks, under which a huge amount of frequently updated location information of mobile users are tracked, can provide complete coverage to estimate traffic condition on roads and highways. This paper studies potential challenges and opportunities in intelligent traffic sensing from the data science point of view with mobile network generated data. Firstly, we classify the data resources available in the commercial radio network according to different taxonomy criteria. Then we outline the broken-down problems that fit in the framework of traffic sensing based on mobile user network log data. We study the existing data processing and learning algorithms on extracting traffic condition information from a large amount of mobile network log data. Finally we make suggestion on potential future work for traffic sensing on data from mobile networks. We believe the techniques and insights provided here will inspire the research community in data science to develop the machine learning models of traffic sensing on the widely collected mobile user behavior data.

In the telecom business, there has been a heavy competition from Internet, media and handset vendors companies. These over-the-top (OTT) players offer compiling telecom services, cause a transformation in the telecom business ecosystem, and the most challenging services posed here are media services. China, India and Indonesia, as world’s emerging markets in Asia, are predicted to take the largest share in the global mobile traffic explosion by 2015. It is critical for mobile network operators (MNOs) in this region to explore strategy for mobile media services, as mobile broadband is likely preferred compared to fixed broadband. In this paper, we analyze and compare mobile music business models used in these markets and structure the relation models between the key actors, using Actors, Relations and Business Activities (ARA) model. We present the economic models that are emerging, and an insight of why and how these multitudes actors are betting on currently. We found that the MNOs generally have a much stronger position compared to their counterparts in the developed markets, and the personalization services, like ring-back tone, are still a huge success. The actors tend to deliver the services by their own, rather than to collaborate in a horizontal business setting.