cpet/openbsd-ports
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
openbsd-ports/net/crawl/Makefile

31 lines
607 B
Makefile
Raw Normal View History

zap trailing whitespace in files where it's present on PERMIT_* lines
2012-10-15 15:18:31 +00:00
# $OpenBSD: Makefile,v 1.32 2012/10/15 15:18:32 sthen Exp $
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
Remove surrounding quotes in COMMENT*/PERMIT_*/BROKEN/ERRORS Add $OpenBSD$ to p5-SNMP-Info/Makefile (ok kili@, simon@)
2007-09-15 22:36:51 +00:00
COMMENT= small and efficient HTTP crawler
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
upgrade to crawl 0.4 -- From: Robert Nagy <thuglife at bsd dot hu>
2004-04-07 22:51:03 +00:00
DISTNAME= crawl-0.4
after PKGSPEC change, bump all packages that depend on db/v3
2011-09-25 14:42:51 +00:00
REVISION= 5
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
CATEGORIES= net
HOMEPAGE= http://www.monkey.org/~provos/crawl/
drop maintainership on some stuff i don't use anymore, lop off WWW: ${HOMEPAGE} while touching 'em
2003-09-25 06:25:13 +00:00
# BSD
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
PERMIT_PACKAGE_CDROM= Yes
zap trailing whitespace in files where it's present on PERMIT_* lines
2012-10-15 15:18:31 +00:00
PERMIT_PACKAGE_FTP= Yes
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
PERMIT_DISTFILES_CDROM= Yes
PERMIT_DISTFILES_FTP= Yes
use REVISION, checked with before/after make show=PKGNAMES (plus some extra-careful checking where there are complicated PSEUDO_FLAVORS).
2010-07-12 22:07:37 +00:00
last remaining old-style lib version numbers
2010-11-22 10:27:12 +00:00
WANTLIB= c event lib/db/db=3
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
MASTER_SITES= http://www.monkey.org/~provos/
db/v3 meets PKGSPEC
2010-11-26 14:23:31 +00:00
LIB_DEPENDS= databases/db/v3
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
- remove unnecessary patches - use DESTDIRNAME
2002-06-23 18:23:03 +00:00
CONFIGURE_STYLE= gnu
CONFIGURE_ARGS= --with-db="${LOCALBASE}"
use new databases/db layout db update and these modifications by Aleksander Piotrowski <aleksander dot piotrowski at nic dot com dot pl>
2003-12-08 17:42:34 +00:00
CONFIGURE_ENV= CFLAGS="${CFLAGS} -I/usr/local/include/db"
WANTLIB conversion
2010-11-11 11:54:08 +00:00
USE_GROFF = Yes
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
No regression tests available.
2002-10-28 01:38:44 +00:00
NO_REGRESS= Yes
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
.include <bsd.port.mk>
Reference in New Issue Copy Permalink