In the process of analyzing the yarn-dyed fabric, two kinds of color information about color yarns should be detected: (1) the number of yarn colors; (2) the layout of the color yarns. The traditional detection methods are time-consuming and labor-intensive. An automatic method based on image analysis is proposed in this study. The image of yarn-dyed fabric captured with a flat scanner is analyzed by a fuzzy C-means clustering (FCM) algorithm. By the analysis of the image of the yarn-dyed fabric based with the FCM algorithm, we can conclude that the number of yarn colors can be obtained with cluster validity analysis, and the layout of color yarns can be inspected automatically with the help of Hough transform. Experiments on two actual fabrics show that the approach proposed in this study is effective for detecting the number of yarn colors and the layout of color yarns in the yarn-dyed fabric.