Model-free optimization methods typically use only current cost samples (e.g., one per iteration) by discarding all the past cost samples. We introduce a simple yet memory mechanism to maintain and use them until new data become available.