After 5 years of thought, we removed and prevent duplicates. This readies us for mass input from Lyft perhaps…. Previously, I for one would upload redundant data “just to make sure it got in”. Then it would take hours to audit them out. No more. Duplicates all gone. It took a month of making sure the proper original owner stayed in and newer-non-deleted copies went away.
This is the first time we have actually physically removed data, typically we mark it, not remove it. So if you are making a local copy of wikispeedia, you need to remove duplicates yourself, live with them, or start over…
If you have performance issues, put a comment below immediately.
Below are the before and after stats which may be interesting, maybe not.