

You can’t impersonate search engine crawlers, because the server can verify that it’s really a search engine crawler
See this from google: https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests
Here, duckduckgo: https://duckduckgo.com/duckduckgo-help-pages/results/duckduckbot


I just realized this place is full of KDE devs :3