Matt
4e8a42e024
text replacements for bad int32_t substitutions
2014-11-17 18:24:38 -08:00
Matt
96b8197ad3
now it compiles with -m32
2014-11-10 14:45:11 -08:00
Matt Wells
5ca411e3e2
tuning the rebalance loop
2014-03-15 14:56:11 -07:00
Matt Wells
86147fe22c
tight merge during rebalance to save
...
disk space, so neg recs annihilate pos recs.
2014-03-14 23:37:30 -07:00
Matt Wells
e37eebd76f
when rebalancing wait for merge to complete before scanning
...
more
2014-03-14 22:16:25 -07:00
Matt Wells
553aefdb55
keep files tightly merged when doing rebalanced
...
to avoid running out of disk space
2014-03-14 19:19:41 -07:00
Matt Wells
27e8e810d2
use collnum instead of coll string.
...
more stable since resetting collections
keeps string the same but changes the collnum.
2014-03-06 15:48:11 -08:00
Matt Wells
c4a6ad1145
update "this round" counts to at least
...
the total counts if round # is 0 so we do not
double spider everyone's jobs!
put a check in rebalance loop to see if gb
is exiting so we don't get into an infinite loop.
this should be in redmine now...
2014-01-23 18:22:13 -08:00
Matt Wells
a4be05d8d0
more shard rebalancer fixes
2014-01-22 00:44:33 -08:00
Matt Wells
066d910934
try to fix rebalancing some more.
2014-01-21 22:39:01 -08:00
Matt Wells
01a3282020
fix problem scanning spiderdb.
...
move dedup spiderdb code to
RdbMerge.cpp where it really should be.
2014-01-16 17:04:08 -08:00
Matt Wells
980d63632a
more msg5 re-read fixes.
...
stop re-reading if increasing minrecsizes did nothing.
fix tight merges so they work over all colls.
fix merge counting to be fast and not loop over
all rdbbases which could be thousands.
add num mirrors to rebalance.txt.
fix updateCrawlInfo to wait for all replies. critical error!
2014-01-16 13:38:22 -08:00
Matt Wells
ae3aa445e8
rebalancer working pretty well now
2014-01-15 19:08:47 -08:00
Matt Wells
4b27b22949
git rebalancing working right
2014-01-15 17:40:17 -08:00
Matt Wells
4a04542829
rebalancer is running now
2014-01-15 16:09:38 -08:00
Matt Wells
f8c2329bd2
rebalancer fixes
2014-01-15 15:42:59 -08:00
Matt Wells
b8058cc16c
rebalance fix
2014-01-15 14:49:11 -08:00
Matt Wells
cb5b4af271
show reason spiders are not going above
...
the spider queue page.
2014-01-11 21:40:45 -08:00
Matt Wells
eed606601e
added emergency msg box on all admin pages
2014-01-11 20:14:44 -08:00
Matt Wells
8a49e87a61
got code with shard rebalancing compiling.
...
now we store a "sharded by termid" bit in posdb
key for checksums, etc keys that are not sharded
by docid. save having to do disk seeks on every
host in the cluster to do a dup check, etc.
2014-01-11 16:08:42 -08:00
Matt Wells
f64b53bfb3
almost done with rebalancing code
2014-01-10 14:12:58 -08:00
