We identify critical flaws in existing datasets and benchmarking protocols for crystal structure prediction in generative modeling of inorganic crystals; we revise datasets and make new benchmarks that account for crystal polymorphism.