A vision-based real-time monitoring system adopting pattern matching techniques has been developed for long-distance remote measurement of static and dynamic displacements of high-rise structures. In application to the Guangzhou New TV Tower (GNTVT) of 610 m high, the vision monitoring system has been used together with a GPS system and accelerometers to measure the structural dynamic characteristics. The results indicate that the dynamic displacement response time histories obtained by the vision monitoring system matches well with those obtained from GPS, and are less noise-corrupted. It is illustrated that the devised vision monitoring system provides a viable means for structural health monitoring of super-tall structures.