Ship behavior is the semantic expression of corresponding trajectory in spatial-temporal space. The intelligent identification of ship behavior is critical for safety supervision in the waterborne transport. In particular, the complicated behavior reflects the long-term intentions of a ship, but it is challenging to recognize it automatically for computers without a proper understanding. For this purpose, this study provides a method to model the behavior for computers from the perspective of knowledge modeling that is explainable. Based on our previous work, a semantic model for ship behavior representation is given considering the multi-scale features of ship behavior in cognitive space. Firstly, the multi-scale features of ship behavior are analyzed in spatial-temporal dimension and semantic dimension individually. Then, a method for multi-scale behaviors modeling from the perspective of semantics is determined, which divides the behavior scale into four sub-scales in cognitive space, considering spatial and temporal dimensions: action, activity, process, and event. Furthermore, an ontology model is introduced to construct the multi-scale semantic model for ship behavior, where behaviors with different semantic scales are expressed using the functions of ontology from a microscopic perspective to a macroscopic perspective consecutively. To validate the model, a case study is conducted in which ship behavior with different scales occurred in port water areas. Typical behaviors, which include leveraging the axioms expression and semantic web rule language (SWRL) of the ontology, are then deduced using a reasoner, such as Pellet. The results show that the model is reasonable and feasible to represent multi-scale ship behavior in various scenarios and provides the potential to construct a smart supervision network for maritime authorities.