I Can Only Judge Your Future by His Past