The ground motion on dipping faults was observed that is larger in the hanging wall than in the footwall. In order to show that the ground motion is affected mainly by dynamic source effects, the rupture process of dipping faults with different asperity size but with the same seismic moment is modeled. The results suggest that the effect of the asperity size on ground motion is important in the generation of high frequency components. The smaller is the asperity, the more is the high frequency motion generation. Based on these results, we simulate the rupture process of the southern and northern part of the 1999 Chi-Chi (Taiwan) earthquake in order to explain the damage distribution on buildings caused by the earthquake in which, although the strongest ground motion occurred at the northern part of the causative fault, structural damage was heavier in the southern part than in the former.