Matt
39214a9dc6
Merge branch 'diffbot-testing' into testing
2015-10-02 19:26:15 -06:00
Matt Wells
e4adc99c0c
fix empty winner tree bug.
...
try to improve rdbcache promotion logic for all
caches. -O2 on spider.cpp.
2015-10-02 12:16:48 -07:00
Matt Wells
9178d67b2f
fix churn bug in winnerlistcache in spider.cpp
...
so do not add the dolebuf list of spiderrequests
back into the cache, but just modify the "jump"
in the first 4 bytes of the cached record. because
when we re-added it back to the cache it created too
much churn and we'd lose cached records unnecessarily.
2015-10-01 19:35:34 -07:00
Matt
100888d691
fix file/dir creation permissions bugs
2015-09-21 12:44:41 -06:00
Matt
74cde33a3a
just use the user's umask val for all file/dir creation
2015-09-21 11:33:38 -06:00
Matt
ce7b06fc4d
all files made are now group writable.
...
if you don't like that then you can make
a special group and set the directory just
group writable for that group using chmod g+s <dir>.
2015-09-21 11:19:34 -06:00
Matt Wells
3c8c415d04
fix file cache.
...
fix core when file cache max mem is 0.
2015-09-10 22:21:55 -07:00
Matt
8611a06dc8
fix log spam
2015-09-10 23:07:06 -06:00
Matt
09de59f026
do not store cblock, etc. tags into tagdb to save
...
disk space. added tagdb file cache for better performance,
less disk accesses. will help reduce disk load.
put file cache sizes in master controls and if they change
then update the cache size dynamically.
2015-09-10 12:46:00 -06:00
Matt Wells
c6a59d0810
fix rdbcache corruption bugs for winnerlistcache.
2015-03-07 11:09:06 -08:00
Matt
f6723ddaa3
new much faster spider. cache the winner tree
...
basically. TODO: need to update cache if
new spiderrequests are added that should be
in the cached winner tree.
2015-02-10 21:27:21 -08:00
mwells
87285ba3cd
use gbmemcpy not memcpy so we can get profiler working again
...
since memcpy can't be interrupted and backtrace() called.
2015-01-13 12:25:42 -07:00
Matt Wells
375069bd15
fix url dup cache bug
2014-12-01 07:45:59 -08:00
Matt Wells
8e315504a2
fix empty rdbcache bug of not enough buf mem.
2014-11-27 13:17:00 -08:00
Matt
958e7db0c2
fix rdbcache bug from 64bit conversino i guess
2014-11-19 10:31:31 -08:00
Matt
2977845375
simplify Inlinks class in LinkInfo.cpp.
...
fix some more 64-bit related cores.
2014-11-18 16:50:31 -08:00
Matt
4e8a42e024
text replacements for bad int32_t substitutions
2014-11-17 18:24:38 -08:00
Matt
931a1c4bc6
good checkpoint. quite a few fixes.
2014-11-17 18:13:36 -08:00
Matt
4a0554c76f
more 64bit fixes
2014-11-14 17:30:32 -08:00
Matt
4c19453ea9
working with -m32 for basic testing.
...
compiles for 64-bit.
2014-11-12 11:38:37 -08:00
Matt
96b8197ad3
now it compiles with -m32
2014-11-10 14:45:11 -08:00
Matt Wells
e7dd8f7956
replace long long with int64_t
2014-10-30 13:36:39 -06:00
Matt Wells
b13f3d24d7
replaced unsigned long long with uint64_t
2014-10-30 13:30:39 -06:00
mwells
060e887f08
misc/various bug fixes.
...
fix canonical redir url bug with iframes.
2014-08-28 18:07:22 -07:00
Matt Wells
b634d06287
fix some cores. use olddoc contenthash
...
for msg13 call for EDOCUNCHANGED errors.
2014-02-07 18:28:09 -08:00
Matt Wells
4029b0b937
more faster spider fixes. tried to fix
...
corrupt rdbcache.
2014-02-06 09:25:27 -08:00
Matt Wells
4606e88721
code cleanups.
...
xmldoc::injectDoc(), and it'll
add a SpiderRequest as well.
better collectiondb init code.
2014-01-18 21:19:26 -08:00
Matt Wells
43e40208b8
Merge branch 'master' into diffbot
...
Conflicts:
SafeBuf.cpp
SafeBuf.h
SearchInput.cpp
XmlDoc.cpp
2013-11-20 15:51:58 -08:00
mwells
5baf6a95d4
handle a bunch of oom conditions that
...
caused core. found using oom tester.
2013-11-20 10:14:02 -07:00
Matt Wells
889583ec4b
now we can reset collection mid stream
2013-10-18 17:49:36 -07:00
Matt Wells
94e6492916
removed MAX_COLL_RECS so we can have unlimited
...
collections, really limited by the sizeof(collnum_t) only now,
which is 16bits, 15bits unsigned, which is the limitation.
can always expand this so we can have more than 32k collections.
2013-08-30 16:20:38 -07:00
Matt Wells
f6e560c1f4
Initial file population.
2013-08-02 13:12:24 -07:00
