Ultrasound imaging has been proposed for diagnostics of osteoarthritis and cartilage injuries in vivo. However, the specific contribution of chondrocytes and collagen to ultrasound scattering in articular cartilage has not been systematically studied. We investigated the role of these tissue structures by measuring ultrasound scattering in agarose scaffolds with varying collagen and chondrocyte concentrations. Ultrasound catheters with center frequencies of 9 MHz (7.1–11.0 MHz, −6 dB) and 40 MHz (30.1–45.3 MHz, −6 dB) were applied using an intravascular ultrasound device. Ultrasound backscattering quantified in a region of interest starting right below sample surface differed significantly (p < 0.05) with the concentrations of collagen and chondrocytes. An ultrasound frequency of 40 MHz, as compared with 9 MHz, was more sensitive to variations in collagen and chondrocyte concentrations. The present findings may improve diagnostic interpretation of arthroscopic ultrasound imaging and provide information necessary for development of models describing ultrasound propagation within cartilage.