Matt Wells
7ec1513d41
updates
2014-03-12 08:09:45 -07:00
Matt Wells
953b7c558d
parm updates
2014-02-10 21:45:03 -07:00
Matt Wells
c041d47a0c
html formatting updates
2014-02-10 00:15:04 -07:00
Matt Wells
6c9a44367f
code checkpoint
2014-02-09 12:38:40 -07:00
Matt Wells
fa59c62264
more bug fixes associated with collections
...
and site page counts in url filters.
2014-01-18 11:54:58 -08:00
Matt Wells
5b7170e8c6
Merge branch 'diffbot' of github.com:gigablast/open-source-search-engine into diffbot
...
Conflicts:
Json.cpp
PageAddUrl.cpp
PageStats.cpp
Spider.cpp
2014-01-17 21:07:08 -08:00
Matt Wells
4e803210ee
tons of changes from live github on neo.
...
lots of core fixes.
took out ppthtml powerpoint convert, it hangs.
dynamic rdbmap to save memory per coll.
fixed disk page cache logic and brought it
back.
2014-01-17 21:01:43 -08:00
Matt Wells
4f64677b4f
get new global preemptive cache
...
logic compiling, with section voting
stats.
2014-01-05 11:51:09 -08:00
mwells
82494baa89
move CollectionRec stuff into Collectiondb files
...
for simplicity.
2013-12-10 15:28:04 -08:00
Matt Wells
fc17521697
Merge branch 'master' into diffbot
...
Conflicts:
Hostdb.cpp
Makefile
PageResults.cpp
PageRoot.cpp
Pages.cpp
Rdb.cpp
SearchInput.cpp
SearchInput.h
Spider.cpp
Spider.h
XmlDoc.cpp
2013-10-16 14:28:42 -07:00
mwells
612f2872f7
use addurl to add the gbdmoz url
...
files to gigablast. it should index
just those dmoz urls, and not spider their links.
it should ignore external errors like
ETCPTIMEDOUT when indexing so it will be
identical to dmoz.
2013-10-05 23:22:51 -06:00
mwells
923d1becce
support &spiderlinks=1 in addition to &spiderLinks=1
...
for add url in PageAddUrl.cpp.
2013-09-30 14:59:48 -06:00
Matt Wells
f6e560c1f4
Initial file population.
2013-08-02 13:12:24 -07:00