In this paper, an image processing and recognition system of the coal and gangue has been studied, which is made up by the chip DM6437 and some peripheral auxiliary equipment. The image processing arithmetic is realized by the CCS programming environment, and loaded into the target board through the simulator XDS510. The system takes advantage of the high performance of the chip to process the real-time image collected by the CCD camera, and makes count of coal and gangue. In this paper, we mainly use the coal and gangue’s trend of contours to make count，and we achieved to make count of the continuous and irregular coal and gangue in the rapid movement of the belt.