This thesis addresses the surprising features of zero-temperature statics and dynamics of several spin glass models, including correlations between soft spins that arise spontaneously during avalanches, and the discovery of localized states that involve the presence of two-level systems.