Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Improving real-time bidding using a constrained markov decision process
Show others and affiliations
2017 (English)In: 13th International Conference on Advanced Data Mining and Applications, ADMA 2017, Springer, 2017, Vol. 10604, 711-726 p.Conference paper (Refereed)
Abstract [en]

Online advertising is increasingly switching to real-time bidding on advertisement inventory, in which the ad slots are sold through real-time auctions upon users visiting websites or using mobile apps. To compete with unknown bidders in such a highly stochastic environment, each bidder is required to estimate the value of each impression and to set a competitive bid price. Previous bidding algorithms have done so without considering the constraint of budget limits, which we address in this paper. We model the bidding process as a Constrained Markov Decision Process based reinforcement learning framework. Our model uses the predicted click-through-rate as the state, bid price as the action, and ad clicks as the reward. We propose a bidding function, which outperforms the state-of-the-art bidding functions in terms of the number of clicks when the budget limit is low. We further simulate different bidding functions competing in the same environment and report the performances of the bidding strategies when required to adapt to a dynamic environment.

Place, publisher, year, edition, pages
Springer, 2017. Vol. 10604, 711-726 p.
Series
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), ISSN 0302-9743 ; 10604
Keyword [en]
Display Advertising, Markov Decision Process, Real-time bidding, Reinforcement Learning
National Category
Computer Systems
Identifiers
URN: urn:nbn:se:kth:diva-218314DOI: 10.1007/978-3-319-69179-4_50Scopus ID: 2-s2.0-85033689734ISBN: 9783319691787 (print)OAI: oai:DiVA.org:kth-218314DiVA: diva2:1160460
Conference
13th International Conference on Advanced Data Mining and Applications, ADMA 2017, Singapore, 5 November 2017 through 6 November 2017
Note

QC 20171127

Available from: 2017-11-27 Created: 2017-11-27 Last updated: 2017-11-27Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full textScopus

Authority records BETA

Brorsson, Mats

Search in DiVA

By author/editor
Brorsson, Mats
By organisation
Software and Computer systems, SCS
Computer Systems

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 2 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf