CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Generative Adversarial Networks in Text Generation
KTH, School of Electrical Engineering and Computer Science (EECS).
2019 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesis
Abstract [en]

The Generative Adversarial Network (GAN) was firstly proposed in 2014, and it has been highly studied and developed in recent years. It has obtained great success in the problems that cannot be explicitly defined by a math equation such as generating real images. However, since the GAN was initially designed to solve the problem in a continuous domain (image generation, for example), the performance of GAN in text generation is developing because the sentences are naturally discrete (no interpolation exists between “hello" and “bye").

In the thesis, it firstly introduces fundamental concepts in natural language processing, generative models, and reinforcement learning. For each part, some state-of-art methods and commonly used metrics are introduced. The thesis also proposes two models for the random sentence generation and the summary generation based on context, respectively. Both models involve the technique of the GAN and are trained on the large-scale dataset. Due to the limitation of resources, the model is designed and trained as a prototype. Therefore, it cannot achieve the state-of-art performance. However, the results still show the promising performance of the application of GAN in text generation. It also proposes a novel model-based metric to evaluate the quality of summary referring both the source text and the summary.

The source code of the thesis will be available soon in the GitHub repository: https://github.com/WangZesen/Text-Generation-GAN.

Abstract [sv]

Det generativa motståndsnätverket (GAN) introducerades först 2014 och det har studerats samt utvecklats starkt under senare år. GAN har uppnått stor framgång för problem som inte kan definieras uttryckligen av en matematisk ekvation, som att generera riktiga bilder. Men eftersom GAN ursprungligen var utformat för att lösa problemet i en kontinuerlig domän (till exempel bildgenerering), utvecklas GAN:s prestanda i textgenerering eftersom meningarna är naturligt diskreta (ingen interpolering finns mellan “hej" och “hejdå").

I examensarbetet introduceras grundläggande begrepp i naturlig språkbearbetning, generativa modeller och förstärkningslärande. För varje del introduceras några bästa tillgängliga metoder och vanligt förekommande mätvärden. Examensarbetet föreslår också två modeller för slumpmässig meningsgenerering respektive sammanfattningsgenerering baserat på sammanhang. Båda modellerna involverar tekniken för GAN och är tränade på storskaliga datamängder.

På grund av begränsningen av resurser är modellen designad och tränad som en prototyp. Därför kan den inte heller uppnå bästa möjliga prestanda. Resultaten visar ändå lovande prestanda för tillämpningen av GAN i textgenerering. Den föreslår också en ny modellbaserad metrik för att utvärdera kvaliteten på sammanfattningen som hänvisar både till källtexten och sammanfattningen.

Examensarbetets källkod kommer snart att finnas tillgänglig i GitHubförvaret:

https://github.com/WangZesen/Text-Generation-GAN.

Place, publisher, year, edition, pages
2019. , p. 51
Series
TRITA-EECS-EX ; 2019:612
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:kth:diva-264575OAI: oai:DiVA.org:kth-264575DiVA, id: diva2:1374343
External cooperation
Seavus
Supervisors
Examiners
Available from: 2019-11-29 Created: 2019-11-29

Open Access in DiVA

fulltext(3343 kB)6 downloads
File information
File name FULLTEXT01.pdfFile size 3343 kBChecksum SHA-512
c6ddbfc9f6c96303aff29fd42c6468279f50c2cda784540d64b937e623a3019de84081715ae110153a08664d62ef113a6dcc5e92dec2bdb9443d0e8cf321252f
Type fulltextMimetype application/pdf

By organisation
School of Electrical Engineering and Computer Science (EECS)
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 6 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 31 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf