Diabetes detrimentally affects the musculoskeletal system by stiffening the collagen matrix due to increased advanced glycation end products (AGEs). In this study, tibiae and tendon from Zucker diabetic Sprague-Dawley (ZDSD) rats were compared to Sprague-Dawley derived controls (CD) using Atomic Force Microscopy. ZDSD and CD tibiae were compared using Raman Spectroscopy and Reference Point Indentation (RPI). ZDSD bone had a significantly different distribution of collagen D-spacing than CD (p = 0.015; ZDSD n = 294 fibrils; CD n = 274 fibrils) which was more variable and shifted to higher values. This shift between ZDSD and CD D-spacing distribution was more pronounced in tendon (p < 0.001; ZDSD n = 350; CD n = 371). Raman revealed significant increases in measures of bone matrix mineralization in ZDSD (PO43− ν1/Amide I p = 0.008; PO43− ν1/CH2 wag p = 0.047; n = 5 per group) despite lower bone mineral density (aBMD) and ash fraction indicating diabetes may preferentially reduce the Raman signature of collagen. Decreased indentation distance increase (p = 0.010) and creep indentation distance (p = 0.040) measured by RPI (n = 9 per group) in ZDSD rats suggest a matrix more resistant to indentation under the high stresses associated with RPI at this length scale. There were significant correlations between Raman and RPI measurements in the ZDSD population (n = 18 locations) but not the CD population (n = 16 locations) indicating that while RPI is relatively unaffected by biological noise, it is sensitive to disease-induced compositional changes. In conclusion, diabetes in the ZDSD rat causes changes to the nanoscale morphology of collagen that result in compositional and mechanical effects in bone at the microscale.