Damage detection in civil structure is a challenging task, mainly because of the strong environmental variations and the variable and unknown excitation. There is still a lack of a robust damage detection process. Taking advantage of the development of the nonlinear dynamical systems theory which represents time series in a reconstructed state-space, a novel damage sensitive feature vector is proposed. Statistical modelling using extreme value theory is conducted to classify measurement as damaged or undamaged. The whole approach is tested on two case studies. The first one is a simple 4dof mass/spring numerical model, damaged by stiffness reduction. The second one is a concrete beam subjected to temperature variations to simulate realistic conditions. Damage is introduced by loading cycles. From 30% of stiffness reduction, damage is correctly detected with a monotonic trend. In the more realistic case, only few true detections are observed before macro-cracking whereas all points are well classified after. Furthermore, the method is robust against strong temperature variations.