espie
8f14ffd213
missing libevent
2005-11-01 16:24:15 +00:00
sturm
b1d1e479a4
better and consistent LIB_DEPENDS on db; this fixes a few possible
...
build time failures, where the wrong version of db could be found
2005-01-12 21:31:05 +00:00
naddy
fedfc3867f
SIZE
2005-01-05 17:14:37 +00:00
alek
2344f6d18c
Add WANTLIB markers
2004-12-16 00:31:20 +00:00
espie
52b24f12ea
new plists, kill a few INSTALL scripts.
2004-09-15 18:17:38 +00:00
naddy
25989f293a
fix build; from Robert Nagy <thuglife@bsd.ru>
2004-04-16 15:06:48 +00:00
brad
292a790802
upgrade to crawl 0.4
...
--
From: Robert Nagy <thuglife at bsd dot hu>
2004-04-07 22:51:03 +00:00
sturm
04a09edf10
fix db dependencies to ensure db/v3 is installed
...
ensure db/v3 is used
also fixes build on NO_SHARED_ARCHS
with conceptual help from brad@
2004-01-10 08:33:11 +00:00
sturm
0d2f5634f8
use new databases/db layout
...
db update and these modifications by
Aleksander Piotrowski <aleksander dot piotrowski at nic dot com dot pl>
2003-12-08 17:42:34 +00:00
jolan
912e6da097
drop maintainership on some stuff i don't use anymore, lop off
...
WWW: ${HOMEPAGE} while touching 'em
2003-09-25 06:25:13 +00:00
jolan
67e58ed296
update e-mail address, ok brad@
2003-05-12 08:23:45 +00:00
pvalchev
6bfe425725
new maintainer Jolan Luff <jolan@cryptonomicon.org>
2003-03-27 17:41:49 +00:00
david
e74e4a4773
fix more spelling errors/typos
...
ok pvalchev@
2003-03-02 05:35:10 +00:00
pvalchev
de5a1ce02c
cast NULL sentinel to void * so it is 64bit on alpha & sparc64
2002-12-28 23:29:13 +00:00
brad
fb4ec1cc36
change e-mail address.
...
--
From: MAINTAINER
2002-12-09 15:37:23 +00:00
naddy
c53eb436cb
No regression tests available.
2002-10-28 01:38:44 +00:00
naddy
559f9fe989
* libevent timeout_ -> evtimer_
...
* printf() size_t by casting to unsigned long and using %lu format
ok mark@
2002-07-16 16:08:23 +00:00
brad
db1541d1f7
- remove unnecessary patches
...
- use DESTDIRNAME
2002-06-23 18:23:03 +00:00
naddy
c66a73f77a
kill libevent dependency
2002-06-16 15:26:58 +00:00
mark
065b7237e3
getopt() returns int, not char
2002-02-24 23:37:51 +00:00
mark
e0ce1d4f7f
crawl-0.3 update
2002-02-08 08:16:10 +00:00
mark
ce7c609bd9
update maintainer address
2001-12-24 07:51:05 +00:00
obecian
bf24f2f3e3
crawl-0.2 update
2001-12-22 10:49:48 +00:00
espie
0e6f6f93bb
Kill first component of BUILD_DEPENDS.
...
Fix sawfish comment while I'm at it.
`window manager for the X windows system'.
Like DUH ! It's going to be a window manager for my microwave oven ?
2001-11-10 15:00:47 +00:00
espie
2ba875bfa5
fix dependencies
2001-10-15 12:27:46 +00:00
obecian
4b01747aa5
crawl-0.1b import - provos@ ok
...
The crawl utility starts a depth-first traversal of the web at the
specified URLs. It stores all JPEG images that match the configured
constraints. Crawl is fairly fast and allows for graceful termination.
After terminating crawl, it is possible to restart it at exactly the
same spot where it was terminated. Crawl keeps a persistent database
that allows multiple crawls without revisiting sites.
The main reason for writing crawl was the lack of simple open source
web crawlers. Crawl is only a few thousand lines of code and fairly
easy to debug and customize.
Features
+ Saves encountered JPEG images
+ Image selection based on regular expressions and size contraints
+ Resume previous crawl after graceful termination
+ Persistent database of visited URLs
+ Very small and efficient code
+ Supports robots.txt
2001-09-09 21:57:12 +00:00
