In smart home services, non-intrusive load monitoring (NILM) can reveal individual appliances’ power consumption from the aggregate power and requires only one measurement point at the entrance by a smart meter. Most of the existing load disaggregation methods are based on deep and complex neural networks, and excessively long input sequences could increase the model disaggregation time. Meanwhile, constructing representative features and designing effective disaggregation model is becoming increasingly important. Therefore, we utilize a gramian summation difference angular field (GASDF) image, taking any two power sample points’ temporal correlations as input to our baseline model, to better recognize different appliances from the aggregate power sequence. Then, since GASDF could not provide statistical characteristics, we further build the expert feature encoder (EFE) to realize the multi-dimensional representation of power by encoding both current aggregate power and statistical characteristics from historical data as prior knowledge. Afterwards, a batch-normalization (BN)-based normalization fusion (NF) method is proposed to lower the disaggregation error incurred by the distribution difference between GASDF and prior knowledge. Finally, to verify the proposed method's effectiveness, named ApplianceFilter, experiments are conducted on the UK-DALE and REDD data, showing that load disaggregation is improved using prior knowledge fusion, superior to the existing end-to-end neural network model.