

wasn’t common crawl the one that pulled a similar trick to goog’s “if you label a thing as $x we won’t include you”[0]? I could swear I heard their name in association with some derpshit intake management stuff above and beyond the typical fundamental “free/open scraper set” problems
[0] - a tactic google first pulled with Streetview cars pulling in a pile of wifi beacons and tying it to location - “if you don’t want it just rename your AP to ‘{prefix} - {apname}’”. a reply that was just dumb and aggravating but also it fucking sucks that basically no standards have taken this problem to heart in the ~15y hence







also