extra credit Search 85,062 technologies ⌘K Download for Mac

webrobots

This library helps write robots.txt compliant web robots in Ruby.

Category
Ruby gems
gem install webrobots
↑↓ move ↵ open