Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Thanks for your comment, JSDevOps! I appreciate your input. I understand that robots.txt can be bypassed by some. The idea behind "Copy from URL" is to address a specific need when interacting with AI chatbots through their websites, like ChatGPT or Claude, where we don't have control over robots.txt settings (that's up to OpenAI or Anthropic). If you use the API, you can just ignore this, but that's not the case I'm targeting. This tool is designed for those times when you want to quickly copy content from a site to use in your AI conversations. It's just a simple way to make that process smoother. Hope that clarifies the intention behind the tool!


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: