cpet/openbsd-ports
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
openbsd-ports/net/crawl/Makefile

29 lines
612 B
Makefile
Raw Normal View History

fix pkgspec
2009-03-16 11:05:54 +00:00
# $OpenBSD: Makefile,v 1.22 2009/03/16 11:05:54 sthen Exp $
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
Remove surrounding quotes in COMMENT*/PERMIT_*/BROKEN/ERRORS Add $OpenBSD$ to p5-SNMP-Info/Makefile (ok kili@, simon@)
2007-09-15 22:36:51 +00:00
COMMENT= small and efficient HTTP crawler
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
upgrade to crawl 0.4 -- From: Robert Nagy <thuglife at bsd dot hu>
2004-04-07 22:51:03 +00:00
DISTNAME= crawl-0.4
fix pkgspec
2009-03-16 11:05:54 +00:00
PKGNAME= ${DISTNAME}p2
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
CATEGORIES= net
HOMEPAGE= http://www.monkey.org/~provos/crawl/
drop maintainership on some stuff i don't use anymore, lop off WWW: ${HOMEPAGE} while touching 'em
2003-09-25 06:25:13 +00:00
# BSD
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
PERMIT_PACKAGE_CDROM= Yes
PERMIT_PACKAGE_FTP= Yes
PERMIT_DISTFILES_CDROM= Yes
PERMIT_DISTFILES_FTP= Yes
missing libevent
2005-11-01 16:24:15 +00:00
WANTLIB= c event
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
MASTER_SITES= http://www.monkey.org/~provos/
fix pkgspec
2009-03-16 11:05:54 +00:00
LIB_DEPENDS= lib/db/db.=3:db->=3,<4:databases/db/v3
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
- remove unnecessary patches - use DESTDIRNAME
2002-06-23 18:23:03 +00:00
CONFIGURE_STYLE= gnu
CONFIGURE_ARGS= --with-db="${LOCALBASE}"
use new databases/db layout db update and these modifications by Aleksander Piotrowski <aleksander dot piotrowski at nic dot com dot pl>
2003-12-08 17:42:34 +00:00
CONFIGURE_ENV= CFLAGS="${CFLAGS} -I/usr/local/include/db"
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
No regression tests available.
2002-10-28 01:38:44 +00:00
NO_REGRESS= Yes
crawl-0.1b import - provos@ ok The crawl utility starts a depth-first traversal of the web at the specified URLs. It stores all JPEG images that match the configured constraints. Crawl is fairly fast and allows for graceful termination. After terminating crawl, it is possible to restart it at exactly the same spot where it was terminated. Crawl keeps a persistent database that allows multiple crawls without revisiting sites. The main reason for writing crawl was the lack of simple open source web crawlers. Crawl is only a few thousand lines of code and fairly easy to debug and customize. Features + Saves encountered JPEG images + Image selection based on regular expressions and size contraints + Resume previous crawl after graceful termination + Persistent database of visited URLs + Very small and efficient code + Supports robots.txt
2001-09-09 21:57:12 +00:00
.include <bsd.port.mk>
Reference in New Issue Copy Permalink