gigablast/open-source-search-engine
1
0
Fork 0
mirror of https://github.com/gigablast/open-source-search-engine.git synced 2024-10-04 20:27:43 +03:00
Code Issues Projects Releases Wiki Activity
2,312 Commits 3 Branches 0 Tags 222 MiB
4e7152b487
Commit Graph

2312 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
mwells
4e7152b487 fix more bugs in squid proxy implementation. 
force squid proxy stack to use floaters.
 2014-10-02 11:54:50 -07:00
mwells
42b891219d several fixes for floater proxy through squid proxy. 
gb needs to act like squid for the rendering machines so
it can do crawl delay backoff and load balancing over the
floaters.
 2014-10-02 02:08:38 -07:00
mwells
00a09104ca Merge branch 'diffbot-testing' into testing 2014-10-01 21:27:18 -07:00
mwells
8f96ba0187 tell diffbot to use gigablast host #0 as 
its proxy and diffbot will in turn send
the request to one of the spider proxies in its list.
this way it can do its load balancing etc. algo.
 2014-10-01 21:25:39 -07:00
Matt Wells
3a0d60d6b9 remove graphix 2014-10-01 20:00:35 -07:00
Matt Wells
7d2ee6df37 Merge branch 'diffbot-testing' of github.com:gigablast/open-source-search-engine into diffbot-testing 2014-10-01 19:55:45 -07:00
Matt Wells
abd75e5eca remove size limit on coll.conf 2014-10-01 19:55:24 -07:00
mwells
c7a5073139 added sorting by site # inlinks/pop to menu for testing 2014-10-01 12:09:43 -07:00
mwells
854767e074 add example for gbsortby:sitenuminlinks into syntax page 2014-10-01 12:07:01 -07:00
mwells
0075dfee84 fix nytimes.com cookie/redir bug again. 2014-10-01 11:53:47 -07:00
mwells
145341dcb3 import patch from diffbot-testing 2014-10-01 11:36:02 -07:00
Matt Wells
1d22e9525a fixes to pass smoke tests. 2014-10-01 11:33:39 -07:00
mwells
65840d969e update to spider proxy choose set logic 2014-10-01 10:00:24 -07:00
mwells
7d4c4e8db1 update spider proxy logic. 2014-10-01 09:26:41 -07:00
mwells
603b350e09 misnomer 2014-09-30 17:40:02 -07:00
mwells
b4ca812ef8 added parm to reset proxy stats in table. erases 
all our knowledge/stats for each proxy.
 2014-09-30 17:38:59 -07:00
Matt Wells
3ae773a1f3 Merge branch 'testing' into diffbot-testing 2014-09-30 16:22:37 -07:00
Matt Wells
83111cb5c1 fixes before smoke testing 2014-09-30 16:22:18 -07:00
Matt Wells
23d26e26ba Merge branch 'testing' into diffbot-testing 2014-09-30 16:02:07 -07:00
mwells
ce56fb93ab fix qa test so we can roll out proxy code. 2014-09-30 15:40:02 -07:00
mwells
2af806993b update proxy algo so not all proxies get cutoff 
at once.
 2014-09-30 13:08:35 -07:00
Matt Wells
98ce40967f more collection swapping fixes 2014-09-29 21:52:58 -07:00
Matt Wells
8c6d216a14 lots of fixes for collection swapping. 2014-09-29 20:16:39 -07:00
mwells
7275765fbb get collection/root login system working 2014-09-29 19:56:31 -07:00
mwells
e3dbeafa5f more updates to cloud code 2014-09-29 18:28:36 -07:00
Matt Wells
cfb2ab7e82 fix core when deleting collection 
that is not swapped out.
 2014-09-29 14:00:10 -07:00
mwells
bca24fb0e6 fix collection swap logic a bunch. seems to work now. 2014-09-29 13:05:20 -07:00
mwells
257a7e3c10 first stab at swapping out collection recs 
to save memory when # of collections is high
 2014-09-29 11:37:05 -07:00
mwells
46290fa52f new password systems. individual collection passwords/accessIps. 2014-09-28 18:59:49 -07:00
mwells
66dcf61fd7 fix empty summary related core. 
added \n to scoring info on serps to make diffs
in qa.cpp simpler.
 2014-09-28 14:31:56 -07:00
mwells
235d69571b hid indexbody parm. 
show pure xml in cached page if it's xml.
do not show summaries for xml/json docs in the serps, pointless.
fix hashSections().
 2014-09-28 13:47:54 -07:00
mwells
a8c5d6a46e fix gbfacetstr: operator for xml docs 2014-09-28 12:09:04 -07:00
mwells
2366776da3 fix parsing inconsistency bug from fixing the 
hashing gblang:de etc.
 2014-09-28 11:43:02 -07:00
mwells
7d3bcd7672 1 spider out at a time for qa test consistency 2014-09-28 11:00:31 -07:00
mwells
7a0f9fe370 fix support for indexing xml docs. 
no longer use hacks gbxmltitle and gbxmllinks.
no longer convert html entities for xml docs using hacks
since we have XmlDoc::hashXmlFields() function.
added qaxml() qa test to test xml doc indexing and searching.
ignore <?xml> tag when generating xml tag compound name.
 2014-09-28 10:43:41 -07:00
mwells
52c88aee94 index xml docs properly like we do json 2014-09-28 09:20:16 -07:00
Matt Wells
a2beb23d87 added Xml::getCompoundName() 2014-09-28 08:39:46 -07:00
Matt Wells
47eecf4165 msie img border fix 2014-09-27 21:49:48 -07:00
Matt Wells
8308915654 render tabs cleaner for MSIE 2014-09-27 21:41:15 -07:00
Matt Wells
903e53a239 fix login cookie for msie 2014-09-27 21:13:14 -07:00
mwells
8e6365f476 minor fixes in docs 2014-09-27 20:26:21 -07:00
mwells
675d21df0e fix so gblang:en gblang:"zh_cn" terms work 2014-09-27 19:35:30 -07:00
mwells
4b611edf8d link to add gigablast to browser search bar 2014-09-27 19:21:00 -07:00
mwells
8ffa4fe24e doc updates 2014-09-27 17:49:49 -07:00
mwells
0267e865b8 minor fixes 2014-09-27 17:01:16 -07:00
mwells
afd41676d2 bring back meta tag display in results again. 
added qa tests for advanced search and api parms.
various api parm fixes and hides.
do not do test url on proxies if test url empty.
 2014-09-27 15:54:55 -07:00
mwells
9d738cdb8b more advanced search fixes 2014-09-27 11:51:37 -07:00
mwells
6de7a3f6b3 get advanced search working again 2014-09-27 11:12:47 -07:00
mwells
6c94cfceef add <omitCount> stuff. fix getDocIds() recalls 
when too many results invisible (dedupped etc.).
show in xml, json, html. provide link in html to
show omitted results.
 2014-09-27 09:56:23 -07:00
mwells
8c53ce2a79 undo hashtab change. too much overhead. 2014-09-27 08:39:22 -07:00