Polygonatum odoratum is a perennial rhizomatous medicinal plant and different plant parts have been used in the treatment of various ailments. Herein, we have investigated the structural compositions of rhizome, leaf, and stem cell walls. We found 30–44% of polysaccharides in these wall preparations were cyclohexanediaminetetraacetic acid (CDTA) extractable, the proportion of heteromannans (HMs) in the rhizome is nearly three-fold compared to that of the leave and stem. The pectic polysaccharides of the rhizome are also structurally more diverse, with arabinans and type I and type II arabinogalactans being richest as shown by linkage study of the sodium carbonate (Na2CO3) extract. In addition, the 2-linked Araf was rhizome-specific, suggesting the cell walls in the rhizome had adapted to a more complex structure compared to that of the leaf and stem. Water-soluble polysaccharide fractions were also investigated, high proportion of Man as in 4-linked Manp indicated high proportion of HMs. The 21.4 kDa pectic polysaccharides and HMs derived from rhizome cell walls induced specific immune response in mice macrophage cells producing IL-1α and hematopoietic growth factors GM-CSF and G-CSF in vitro.