gigablast/open-source-search-engine
1
0
Fork 0
mirror of https://github.com/gigablast/open-source-search-engine.git synced 2024-10-05 04:37:39 +03:00
Code Issues Projects Releases Wiki Activity
642 Commits 3 Branches 0 Tags 222 MiB
020d7741b9
Commit Graph

18 Commits

Author SHA1 Message Date
Matt Wells
43e40208b8 Merge branch 'master' into diffbot 
Conflicts:
	SafeBuf.cpp
	SafeBuf.h
	SearchInput.cpp
	XmlDoc.cpp
 2013-11-20 15:51:58 -08:00
mwells
46a683a904 label the bigger safebuf chunks of mem 
so we can see a better breakdown of mem
on the stats page, not just a big "SafeBuf"
allocation.
 2013-11-19 23:53:40 -07:00
Matt Wells
7248641bc4 fix mem leaks. turn off electric fence. 2013-11-11 09:58:14 -08:00
Matt Wells
3e4db4f1bc show all crawl details in url webhook 
notification in the post body.
 2013-11-07 13:59:43 -08:00
Matt Wells
c39b45ff88 fix crawl round end detection etc. 
inc round counter even if not repeating crawl
 2013-10-23 15:53:59 -07:00
Matt Wells
64a1c7c2f2 more bug fixes. if spiders disabled for row 
in url filters, don't spider the url.
 2013-10-21 14:45:12 -07:00
Matt Wells
84a3aded94 spider round updates correction 2013-10-17 17:18:05 -07:00
Matt Wells
df7fd21253 spider rounds update. 2013-10-17 17:17:19 -07:00
Matt Wells
fc17521697 Merge branch 'master' into diffbot 
Conflicts:
	Hostdb.cpp
	Makefile
	PageResults.cpp
	PageRoot.cpp
	Pages.cpp
	Rdb.cpp
	SearchInput.cpp
	SearchInput.h
	Spider.cpp
	Spider.h
	XmlDoc.cpp
 2013-10-16 14:28:42 -07:00
mwells
d4b5c37f45 Merge branch 'master' into testing 2013-10-13 00:20:37 -07:00
mwells
c283e85e40 add support for noindex meta tag. 
use it in the gbdmoz.urls.txt.* files
that contain the dmoz urls we want to spider.
 2013-10-12 22:50:23 -07:00
Matt Wells
0b4bbf926e fix potential compiler error. 2013-10-09 11:52:58 -07:00
Matt Wells
283ec2f6b4 email and webhook alerts when spider runs out of urls 
to spider.
 2013-10-09 11:42:56 -07:00
Matt Wells
a412c798bf Merge branch 'master' into diffbot 
Conflicts:
	PageResults.cpp
 2013-09-13 09:24:28 -07:00
Matt Wells
5dc7bd2ab4 integrate diffbot from svn back into git. 2013-09-13 09:23:18 -07:00
Matt Wells
76b390aea2 fix typo 2013-09-08 19:51:57 -07:00
mwells
d930a833cc try to fix compiler error related to bad 
delete function override. added "throw()"
before the first "{" in the function
body.
 2013-09-08 20:15:39 -06:00
Matt Wells
f6e560c1f4 Initial file population. 2013-08-02 13:12:24 -07:00