Cellulose and lignin are the two main components of secondary plant cell walls with substantial impact on stalk in the field and on straw during industrial processing. The amount of fermentable sugar that can be accessed is another important parameter affecting various industrial applications. In the present study, genetic variability of rice (Oryza sativa L.) genotypes for cellulose, lignin, and fermentable sugars contents was analyzed in rice straw. A genome-wide association study of 33,484 single nucleotide polymorphisms (SNPs) with a minor allele frequency (MAF) >0.05 was performed. The genome-wide association study identified seven, three, and three genomic regions to be significantly associated with cellulose, lignin, and fermentable sugar contents, respectively. Candidate genes in the associated genomic regions were enzymes mainly involved in cell wall metabolism. Novel SNP markers associated with cellulose were tagged to GH16, peroxidase, GT6, GT8, and CSLD2. For lignin content, Villin protein, OsWAK1/50/52/53, and GH16 were identified. For fermentable sugar content, UTP-glucose-1-phosphate uridylyltransferase, BRASSINOSTEROID INSENSITIVE 1, and receptor-like protein kinase 5 were found. The results of this study should improve our understanding of the genetic basis of the factors that might be involved in biosynthesis, turnover, and modification of major cell wall components and saccharides in rice straw.