Matt
c2567ad244
a hopeful fix for host #0 always crashing from
...
streaming socket timeouts.
2015-04-02 15:17:49 -06:00
Matt Wells
f76f0c77d8
fix core while streaming and getting EPIPE.
...
but it seems like firefox and firefox only
has a bug it in when streaming json
with high start values like &s=20000&q=type:json
2015-01-28 19:53:38 -08:00
Matt
4e8a42e024
text replacements for bad int32_t substitutions
2014-11-17 18:24:38 -08:00
Matt
931a1c4bc6
good checkpoint. quite a few fixes.
2014-11-17 18:13:36 -08:00
Matt
96b8197ad3
now it compiles with -m32
2014-11-10 14:45:11 -08:00
Matt Wells
e7dd8f7956
replace long long with int64_t
2014-10-30 13:36:39 -06:00
Matt Wells
5c7fc3b083
fix OOM for large &n=1000000000 values when searching.
...
just alloc for the docids found, not the docids asked for.
2014-10-09 11:35:35 -07:00
Matt Wells
ce61090b52
more fixes for federated search going OOM
2014-10-07 11:03:10 -07:00
mwells
6c94cfceef
add <omitCount> stuff. fix getDocIds() recalls
...
when too many results invisible (dedupped etc.).
show in xml, json, html. provide link in html to
show omitted results.
2014-09-27 09:56:23 -07:00
mwells
efc7f3c064
support facet ranges now like
...
gbfacetint:gblang,0-1,1-2,2-3,3-4,5-6,6-18
2014-09-04 20:41:37 -07:00
mwells
a9e61b5aca
facet text lookup fixes.
2014-07-29 19:32:27 -07:00
mwells
312b39c059
lookup facet values to get their text representations.
2014-07-29 16:17:18 -07:00
Matt Wells
d93d44250a
fix debug print statements
2014-07-01 11:46:01 -07:00
Matt Wells
48a98df71d
make &s=20000 search much faster by skipping
...
generation of first 20000 summaries if
deduping is off, site clustering is off and
gigabit generation are off (&dr=0&sc=0&dsrt=0).
turn gigabits off on load for all customcrawls(diffbot)
2014-06-23 14:44:21 -07:00
mwells
3cf3cddc5c
beginning of total parm overhaul.
...
new injection parms, just need to engage them.
2014-06-12 21:27:06 -07:00
Matt Wells
6664faa792
fix printing back-to-back commas when showing
...
results in json with &icc=1.
2014-05-20 13:23:29 -07:00
Matt Wells
2f331d55e5
widget updates
2014-05-06 10:47:57 -07:00
Matt Wells
5c0d646133
fix invalid json when doing &s=1
2014-04-25 13:46:20 -07:00
Matt Wells
75032da5b9
fix pagination for &stream=1
2014-04-22 11:18:21 -07:00
Matt Wells
e351d2a6f1
get searching on token working
2014-03-06 17:01:41 -08:00
Matt Wells
25cf0efdbf
first compiled stab at multi collection searching.
2014-03-06 10:45:13 -08:00
Matt Wells
2ab9aaeeaa
streaming csv fixes
2014-03-04 11:04:26 -08:00
Matt Wells
b1381cc610
make csv streamable, faster and take almost no memory.
2014-03-04 10:45:57 -08:00
Matt Wells
b429f12346
add logic to save memory when streaming
...
over 200 results back. should fix oom when
streaming back hundreds of thousands of results.
2014-02-26 20:33:35 -08:00
Matt Wells
cd6069e5a6
send single space to socket if not streaming
...
and search results still not ready after 10 seconds.
send it every 10 seconds to prevent client from closing socket.
sped up all downloads, json and csv, but not doing "fuzzy"
deduping of search results, but just deduping on page
content hash. added TcpSocket::m_numDestroys to ensure we
do not send heartbeat on a socket that was closed and
re-opened for another client.
2014-02-13 08:45:13 -08:00
Matt Wells
8d534b8ed8
many more fixes for streaming mode
2014-02-06 18:21:22 -08:00
Matt Wells
f9dbd64056
get streaming time sliced results working
2014-02-06 14:25:44 -08:00
Matt Wells
189999509b
code checkpoint. time slicing, faster spider code
...
compiling. now needs debug.
2014-02-04 17:34:43 -08:00
Matt Wells
7874c8d832
added ifdef NEEDSLICENSE
2013-12-01 14:47:08 -07:00
mwells
6c2c9f7774
trying to bring back dmoz integration.
2013-10-02 22:34:21 -06:00
Matt Wells
f6e560c1f4
Initial file population.
2013-08-02 13:12:24 -07:00
