Accurate prediction of highway traffic is of vital importance to proactive traffic monitoring, operation and controls. In the data mining of highway traffic, abstracting temporal knowledge is often prioritized than exploring topological relationship. In this study, we propose a deep learning model, called Knowledge-Sequence-to-Sequence (K-Seq2Seq), to solve the short-term highway traffic prediction problem in two stages: representing temporal knowledge and predicting future traffic. Through computational experiment in a road section of a Swedish motorway, we show that our model outperforms the conventional Seq2Seq model significantly, more than 20% when predicting information of longer time step.