Commit Graph

1856 Commits

Author SHA1 Message Date
mwells
e4ce9bc9ac squidproxycache/floaters/sectiondbtagging all compiles.
need to do run-time debugging now.
2014-06-11 17:57:28 -07:00
mwells
6f70282ba2 almost got sectiondb integration compiling 2014-06-11 17:24:58 -07:00
mwells
1e10c676d5 parm updates for injecting 2014-06-11 17:24:33 -07:00
Matt Wells
66f8f3926d raise MAX_EXPRESSIONS 2014-06-10 19:32:46 -07:00
Matt Wells
27ffd23345 handle boolean query overflow errors better. 2014-06-10 17:21:55 -07:00
Matt Wells
365f29b293 made &spiderRoundStart=1 (or 0) force the next
spider round to begin.
also added pageUrl to XmlDoc::getContentHashJSON32()
so it's not included in the hash to fix some spider-time
deduping issues.
2014-06-10 14:20:41 -07:00
mwells
108c281c33 fix annoying bug when adding new parms. 2014-06-10 12:29:50 -07:00
mwells
77241ecee0 fix make cygwin in Makefile 2014-06-09 18:48:58 -07:00
mwells
29e90d1d55 squid proxy fixes 2014-06-09 16:10:24 -07:00
mwells
5bf3042633 fix squid proxy cache key generation 2014-06-09 14:37:13 -07:00
mwells
b71ea7f7c6 fixes for squid proxy simulator 2014-06-09 14:31:48 -07:00
mwells
4a2717a88f Merge branch 'diffbot-testing' into diffbot-matt 2014-06-09 12:42:54 -07:00
mwells
7d452a766c completed squid proxy simulation code 2014-06-09 12:42:05 -07:00
Matt Wells
8968f094c0 ignore gbsortby:offerprice gbrevsortby:whatever query
operators when evaluating boolean expressions.
fix for '(title:fourth OR text:water)  gbsortby:offerPrice'
query
2014-06-09 11:00:27 -07:00
Matt Wells
bc6c6b3ab7 Merge branch 'testing' into diffbot-testing
Conflicts:
	Makefile
2014-06-09 10:18:25 -07:00
Matt Wells
56af753c3e fixed nasty bug of resetting RdbBases for
random collnums, causing data loss and corruption.
2014-06-09 10:16:29 -07:00
mwells
778e67130f File::set() fix for //'s 2014-06-08 15:24:30 -07:00
mwells
81fed12705 minor makefile updates 2014-06-08 11:49:26 -07:00
mwells
6fddeb416a fixes for 'make debian-testing' package building code
for ubuntu/debian
2014-06-08 11:35:39 -07:00
mwells
01bfebaaaf admin.html updates 2014-06-07 19:45:56 -07:00
mwells
c713454318 admin.html updates 2014-06-07 18:50:24 -07:00
mwells
9067013425 cygwin fixes 2014-06-07 16:30:56 -07:00
mwells
4e5cf747dc cygwin fixes 2014-06-07 16:29:39 -07:00
mwells
e5cb5ab907 cygwin cleanups 2014-06-07 15:59:32 -07:00
mwells
c07996d700 cygwin updates 2014-06-07 14:58:57 -07:00
mwells
27cc896a6c DEFS2 to Makefile 2014-06-07 14:48:19 -07:00
mwells
f55d7cfd68 CYGWIN updates 2014-06-07 14:39:48 -07:00
mwells
778430a543 cygwin updates 2014-06-07 14:37:21 -07:00
mwells
d57ce8a2df simplify compilation more. remove clones() 2014-06-07 14:26:11 -07:00
mwells
1553663d82 compiler cleanups for cygwin compile 2014-06-07 14:20:04 -07:00
mwells
628fe2336f make code compile cleaner. 2014-06-07 14:11:12 -07:00
mwells
04c5d78efe updated email. 2014-06-07 11:20:01 -07:00
mwells
de3f51d30f add ubuntu package link in admin.html. 2014-06-07 10:47:23 -07:00
mwells
4a4fccfd93 added 'make testing-deb' support to build debian packages. 2014-06-07 10:21:51 -07:00
Matt Wells
a809c99abb email update 2014-06-06 19:31:24 -07:00
Matt Wells
d16a1f3422 Merge branch 'diffbot-testing' into testing 2014-06-06 19:22:52 -07:00
Matt Wells
3b2ed3bdb4 fix compile issues on some machines by including
the bits/ include subdir directly in the repo.
also added -Ibits to the Makefile.
2014-06-06 19:05:01 -07:00
Matt Wells
b9777d3f55 fix domain only bug in serps 2014-06-06 18:00:18 -07:00
mwells
0fd85b788b halfway done coding up proxy (squid) support into gb 2014-06-06 17:27:18 -07:00
mwells
72df0d25d2 added safebuf base64decode func 2014-06-06 16:20:15 -07:00
Matt Wells
08e42a64cb any /admin/ cmd as well should not trunc posts 2014-06-06 16:14:47 -07:00
Matt Wells
c7c39005f1 do not truncate crawl jobs POSTs 2014-06-06 16:08:37 -07:00
mwells
965d992f98 Merge branch 'diffbot-testing' into diffbot-matt
Conflicts:
	Msg13.cpp
2014-06-06 15:14:41 -07:00
mwells
3f2dcda4e1 got new floater/proxy logic compiling. 2014-06-06 15:11:51 -07:00
Matt Wells
6b5b83ac85 fixes for gbmin/gbmax being first query term. 2014-06-06 10:20:12 -07:00
Matt Wells
5b6100c77d log format change for errcnt 2014-06-06 09:29:57 -07:00
Matt Wells
d850f5f006 try to prevent job status flip flop from error retries. 2014-06-05 23:38:54 -07:00
Matt Wells
74f0a41290 bulk jobs give up after downloading a url
3 times. crawls don't give up on tmperrors,
but retry every 30 days.
2014-06-05 23:11:14 -07:00
Matt Wells
172d7071a7 fix to rename tagdb0000.002.dat 2014-06-05 22:21:41 -07:00
Matt Wells
8ac691f324 fix merging getting clogged by so many
collections tring to merge tagdb at once
2014-06-05 21:27:33 -07:00