Matrix polysaccharides in primary and secondary plant cell walls are biochemically diverse and include xylans, glucomannans, β-glucans (i.e., mixed linkage β-glucan and xyloglucan), pectins, and β-galactan. Their composition and molecular structure in specific cell walls depend on the type of plant, type of tissue, and the temporal development of the plant. The supramolecular organization of matrix polysaccharides around the cellulose microfibrils affects the flexibility and strength of the cell wall. However, the molecular level details of the interface between the cellulose microfibrils and the matrix polysaccharides are not fully understood. Here, the interaction of unsubstituted model oligosaccharides with cellulose microfibrils was investigated through molecular dynamics simulations of the adsorption of model oligosaccharides representing their respective backbone motifs. The simulations show that induced conformational changes of the polysaccharide backbone upon adsorption and its alignment with the cellulose microfibril lead to stronger interactions with cellulose. This differentiates typical primary and secondary cell wall hemicelluloses (xylans, glucomannans, and β-glucans) from pectins and β-galactan and explains why mixed-linkage β-glucan can be classified as a hemicellulose. Our study contributes to the development of accurate molecular models for plant cell walls, which will improve our understanding of lignocellulosic biomass and its conversion into functional biobased materials.