Stars
grobotstxt is a native Go port of Google's robots.txt parser and matcher library.
The repository contains Google's robots.txt parser and matcher as a C++ library (compliant to C++11).
{{ message }}
grobotstxt is a native Go port of Google's robots.txt parser and matcher library.
The repository contains Google's robots.txt parser and matcher as a C++ library (compliant to C++11).