The new advanced very high resolution (VHR) synthetic aperture radar (SAR) sensors are capable of achieving sub-metric spatial resolution, which offers the opportunity for a fine level of analysis of man-made structures in such data. In this paper, we first proposed a decomposed model to analyze the signature of rectangular buildings in VHR SAR images. In the model, the scattering effects were subdivided according to different contributors. Accordingly, detailed geometrical structures and spatial distributions of image features of building objects with respect to varied SAR illumination conditions were exploited. Then, based on the generalized effects, a new detection and reconstruction algorithm was proposed for 3-D reconstruction from single VHR SAR images. Specifically, a matching procedure was applied to extract image features, and prior knowledge was employed in the reconstruction steps to jointly perform dimensions retrieval and building type identification. Finally, the proposed approach was applied to VHR SAR imagery and the reconstruction results were discussed.