cpet/openbsd-ports
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

import p5-WWW-Robot 0.024

Browse Source 
This module implements a configurable web traversal engine, for a robot
or other web agent. Given an initial web page (URL), the Robot will get
the contents of that page, and extract all links on the page, adding
them to a list of URLs to visit.

Features of the Robot module include:

    * Follows the Robot Exclusion Protocol.
    * Supports the META element proposed extensions to the Protocol.
    * Implements many of the Guidelines for Robot Writers.
    * Configurable.
    * Builds on standard Perl 5 modules for WWW, HTTP, HTML, etc.
This commit is contained in:
jasper 2006-12-17 18:09:41 +00:00
parent 2ca7ee38e8
commit 1eeb4dd082
4 changed files with 41 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
www/p5-WWW-Robot/Makefile Executable file
View File

@ -0,0 +1,22 @@
# $OpenBSD: Makefile,v 1.1.1.1 2006/12/17 18:09:41 jasper Exp $
COMMENT= "configurable web traversal engine"
DISTNAME= WWW-Robot-0.024
CATEGORIES= www
# Artistic + GPL
PERMIT_PACKAGE_CDROM= Yes
PERMIT_PACKAGE_FTP= Yes
PERMIT_DISTFILES_CDROM= Yes
PERMIT_DISTFILES_FTP= Yes
MODULES= cpan
BUILD_DEPENDS= ${RUN_DEPENDS}
RUN_DEPENDS= ::www/p5-HTML-Parser \
::www/p5-HTML-Tree \
::www/p5-URI \
::www/p5-libwww
.include <bsd.port.mk>

4
www/p5-WWW-Robot/distinfo Normal file
View File

@ -0,0 +1,4 @@
MD5 (WWW-Robot-0.024.tar.gz) = 893df367ff4eabc477a62a2a5cbbfda3
RMD160 (WWW-Robot-0.024.tar.gz) = 2c4360a488a47ce10e9a0c380aa8302c9bddd216
SHA1 (WWW-Robot-0.024.tar.gz) = 2837684c6905cce3570dfa7bdad33608422ea86c
SIZE (WWW-Robot-0.024.tar.gz) = 17465

12
www/p5-WWW-Robot/pkg/DESCR Executable file
View File

@ -0,0 +1,12 @@
This module implements a configurable web traversal engine, for a robot
or other web agent. Given an initial web page (URL), the Robot will get
the contents of that page, and extract all links on the page, adding
them to a list of URLs to visit.
Features of the Robot module include:
* Follows the Robot Exclusion Protocol.
* Supports the META element proposed extensions to the Protocol.
* Implements many of the Guidelines for Robot Writers.
* Configurable.
* Builds on standard Perl 5 modules for WWW, HTTP, HTML, etc.

3
www/p5-WWW-Robot/pkg/PLIST Executable file
View File

@ -0,0 +1,3 @@
@comment $OpenBSD: PLIST,v 1.1.1.1 2006/12/17 18:09:41 jasper Exp $
${P5SITE}/WWW/Robot.pm
@man man/man3p/WWW::Robot.3p