A convenient and reliable SHM (Structural Health Monitoring) system based on non-contact displacement measurements was proposed for applying to the real-time crack growth monitoring of concrete structures in this work. The displacement vectors of the nodes of discretized elements on digital image are measured with a CCD camera and an image processor. The maximum principal strain and the principal direction, which are calculated from the displacement vectors, are the prime parameters for crack identification. The validity of the proposed SHM system was demonstrated through the experimental results using a four-point bending specimen.