cpet/openbsd-ports
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
05628c87ee
openbsd-ports/net/crawl/Makefile

30 lines
602 B
Makefile
Raw Normal View History

use REVISION, checked with before/after make show=PKGNAMES (plus some extra-careful checking where there are complicated PSEUDO_FLAVORS).
2010-07-12 18:07:37 -04:00
# $OpenBSD: Makefile,v 1.23 2010/07/12 22:07:38 sthen Exp $
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 17:57:12 -04:00
Remove surrounding quotes in COMMENT*/PERMIT_*/BROKEN/ERRORS Add $OpenBSD$ to p5-SNMP-Info/Makefile (ok kili@, simon@)
2007-09-15 18:36:51 -04:00
COMMENT= small and efficient HTTP crawler
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 17:57:12 -04:00
upgrade to crawl 0.4 -- From: Robert Nagy <thuglife at bsd dot hu>
2004-04-07 18:51:03 -04:00
DISTNAME= crawl-0.4
use REVISION, checked with before/after make show=PKGNAMES (plus some extra-careful checking where there are complicated PSEUDO_FLAVORS).
2010-07-12 18:07:37 -04:00
REVISION= 2
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 17:57:12 -04:00
CATEGORIES= net
HOMEPAGE= http://www.monkey.org/~provos/crawl/
drop maintainership on some stuff i don't use anymore, lop off WWW: ${HOMEPAGE} while touching 'em
2003-09-25 02:25:13 -04:00
# BSD
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 17:57:12 -04:00
PERMIT_PACKAGE_CDROM= Yes
PERMIT_PACKAGE_FTP= Yes
PERMIT_DISTFILES_CDROM= Yes
PERMIT_DISTFILES_FTP= Yes
use REVISION, checked with before/after make show=PKGNAMES (plus some extra-careful checking where there are complicated PSEUDO_FLAVORS).
2010-07-12 18:07:37 -04:00
missing libevent
2005-11-01 11:24:15 -05:00
WANTLIB= c event
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 17:57:12 -04:00
MASTER_SITES= http://www.monkey.org/~provos/
fix pkgspec
2009-03-16 07:05:54 -04:00
LIB_DEPENDS= lib/db/db.=3:db->=3,<4:databases/db/v3
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 17:57:12 -04:00
- remove unnecessary patches - use DESTDIRNAME
2002-06-23 14:23:03 -04:00
CONFIGURE_STYLE= gnu
CONFIGURE_ARGS= --with-db="${LOCALBASE}"
use new databases/db layout db update and these modifications by Aleksander Piotrowski <aleksander dot piotrowski at nic dot com dot pl>
2003-12-08 12:42:34 -05:00
CONFIGURE_ENV= CFLAGS="${CFLAGS} -I/usr/local/include/db"
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 17:57:12 -04:00
No regression tests available.
2002-10-27 20:38:44 -05:00
NO_REGRESS= Yes
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 17:57:12 -04:00
.include <bsd.port.mk>
Reference in New Issue Copy Permalink