In this paper, we study the consensus problem of multi-agent systems in which each agent adopts the same linear model that can be of any order. We consider the case where only the relative output error between the neighboring agents can be measured. In order to solve the consensus problem, two kinds of decentralized control laws are designed. We first show that a static output error feedback control can solve the consensus problem if some further constraints on the system model is imposed. Then we use an observer based approach to design a dynamic output error feedback consensus control. We note that with the observer based approach, some information exchange between the agents is needed unless the associated adjacent graph is completely connected.