mwells
|
04d2d46f0d
|
fix errors from restarting collection and
spiders not going.
fixed error from resetting/restarting crawl
and spiders going again even though were paused.
|
2014-04-10 23:59:30 -07:00 |
|
Matt Wells
|
27e8e810d2
|
use collnum instead of coll string.
more stable since resetting collections
keeps string the same but changes the collnum.
|
2014-03-06 15:48:11 -08:00 |
|
mwells
|
82494baa89
|
move CollectionRec stuff into Collectiondb files
for simplicity.
|
2013-12-10 15:28:04 -08:00 |
|
Matt Wells
|
54d3375a00
|
fixes when crawling on distributed 2x2
|
2013-10-25 14:54:24 -07:00 |
|
Matt Wells
|
fc17521697
|
Merge branch 'master' into diffbot
Conflicts:
Hostdb.cpp
Makefile
PageResults.cpp
PageRoot.cpp
Pages.cpp
Rdb.cpp
SearchInput.cpp
SearchInput.h
Spider.cpp
Spider.h
XmlDoc.cpp
|
2013-10-16 14:28:42 -07:00 |
|
Matt Wells
|
fe97e08281
|
move from groups to shards. got rid of annoying
groupid bit mask thing.
|
2013-10-04 16:18:56 -07:00 |
|
mwells
|
a0c79932bb
|
catdb is now generated successfully.
|
2013-10-02 23:36:49 -06:00 |
|
mwells
|
6c2c9f7774
|
trying to bring back dmoz integration.
|
2013-10-02 22:34:21 -06:00 |
|
Matt Wells
|
f6e560c1f4
|
Initial file population.
|
2013-08-02 13:12:24 -07:00 |
|