gigablast/open-source-search-engine
1
0
Fork 0
mirror of https://github.com/gigablast/open-source-search-engine.git synced 2024-10-04 20:27:43 +03:00
Code Issues Projects Releases Wiki Activity
Nov 20 2017 -- A distributed open source search engine and spider/crawler written in C/C++ for Linux on Intel/AMD. From gigablast dot com, which has binaries for download. See the README.md file at the very bottom of this page for instructions.
3,070 Commits 3 Branches 0 Tags 222 MiB
C++ 96.7%
HTML 1.4%
C 1.2%
Python 0.5%
Makefile 0.1%
bc54282339
Go to file
Matt bc54282339 complete overhaul of injection pipeline now compiles. 
should distribute injection requests evenly over the cluster.
uses new InjectionRequest class which sets from httprequest
using parms in Parms.cpp. and easily serializes into a udp request.
very nice. we should use this model going forward.
2015-05-03 19:07:44 -07:00
antiword-dir Initial file population. 2013-08-02 13:12:24 -07:00
diffbot-widget widget updates 2014-04-21 09:21:28 -07:00
html calls NEWS BLOG 2015-04-12 12:33:09 -06:00
openssl we already include our own 32-bit 2013-09-15 18:25:49 -06:00
ucdata Initial file population. 2013-08-02 13:12:24 -07:00
.gitignore added Codeblocks project file 2014-10-31 11:00:18 -07:00
Abbreviations.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Abbreviations.h replace long long with int64_t 2014-10-30 13:36:39 -06:00
Accessdb.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Accessdb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Address.cpp fixed langid based query stop words. 2015-03-08 15:44:23 -07:00
Address.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
addtest.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Ads.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Ads.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
AdultBit.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
AdultBit.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
animate.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
antiword fix ulimit and antiword bugs 2014-06-18 04:06:20 -07:00
AutoBan.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
AutoBan.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
badcattable.dat Initial file population. 2013-08-02 13:12:24 -07:00
BigFile.cpp update log msg 2015-04-13 07:49:57 -07:00
BigFile.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Bits.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Bits.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
blaster2.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Blaster.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Blaster.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
bmptopnm Initial file population. 2013-08-02 13:12:24 -07:00
Cachedb.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Cachedb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
camsort.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
catcountry.dat Initial file population. 2013-08-02 13:12:24 -07:00
Catdb.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Catdb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Categories.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Categories.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
CatRec.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
CatRec.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
character-sets Initial file population. 2013-08-02 13:12:24 -07:00
check_unicode.cpp Initial file population. 2013-08-02 13:12:24 -07:00
Clusterdb.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Clusterdb.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Collectiondb.cpp added logdebugmsg13 to find out why urls are getting stuck 2015-05-01 13:19:45 -07:00
Collectiondb.h Merge branch 'diffbot-testing' into ia 2015-04-30 17:51:14 -07:00
Conf.cpp put in some fixes to deal with doledb tree 2015-03-08 20:36:13 -07:00
Conf.h added hack to log controls to avoid sending 2015-05-02 10:32:13 -07:00
control.deb package bldg updates 2014-06-16 21:50:32 -06:00
convert.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
copyright.head package bldg updates 2014-06-16 21:50:32 -06:00
copyright.tail package bldg updates 2014-06-16 21:50:32 -06:00
CountryCode.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
CountryCode.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
create_ucd_tables.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
DailyMerge.cpp added new indicators so we can make gb more stable. 2014-12-16 16:22:50 -08:00
DailyMerge.h move CollectionRec stuff into Collectiondb files 2013-12-10 15:28:04 -08:00
DataFeed.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
DataFeed.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Datedb.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Datedb.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Dates.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Dates.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Diff.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Diff.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Dir.cpp support arc injections 2015-02-07 12:07:42 -08:00
Dir.h replace long long with int64_t 2014-10-30 13:36:39 -06:00
DiskPageCache.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
DiskPageCache.h fix up diskpagecache. how did it work before 2014-11-20 15:05:34 -08:00
dlstubs.c Initial file population. 2013-08-02 13:12:24 -07:00
dmozparse.cpp fix dmozparse compiler error 2015-03-06 20:37:30 -08:00
Dns.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Dns.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
DnsProtocol.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
dnstest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Domains.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Domains.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
dumpcore.cpp Initial file population. 2013-08-02 13:12:24 -07:00
Entities.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Entities.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Errno.cpp added logic to retry diffbot reply on connection reset, 2015-03-09 20:54:34 -07:00
Errno.h added logic to retry diffbot reply on connection reset, 2015-03-09 20:54:34 -07:00
errnotest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Events.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Facebook.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Facebook.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
fastIndexTable.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
fctypes.cpp complete overhaul of injection pipeline now compiles. 2015-05-03 19:07:44 -07:00
fctypes.h complete overhaul of injection pipeline now compiles. 2015-05-03 19:07:44 -07:00
File.cpp the new urls.csv format is ready. 2015-04-15 12:13:27 -06:00
File.h try to speed up host #4 on seraph 2015-04-10 09:20:18 -06:00
filterquerylogs.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Flags.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Flags.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
gb-1.0.spec make it so we don't need --nodeps with 2014-05-25 22:08:46 -04:00
gb-include.h replace memcpy_ass with bcopy 2015-01-14 14:12:55 -08:00
gb.deb.rules if netpbm pkg already installed use it. 2014-07-06 09:54:28 -07:00
gb.pem add old gb.pem file, not used by gigablast 2013-10-09 17:37:01 -06:00
gbfilter.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
gbtitletest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
geneaology.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
generateSuperMergeCode.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
geo_ip_table.cpp Initial file population. 2013-08-02 13:12:24 -07:00
geo_ip_table.h Initial file population. 2013-08-02 13:12:24 -07:00
GeoIP_internal.h Initial file population. 2013-08-02 13:12:24 -07:00
GeoIP.c Initial file population. 2013-08-02 13:12:24 -07:00
GeoIP.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
GeoIPCity.c Initial file population. 2013-08-02 13:12:24 -07:00
GeoIPCity.h Initial file population. 2013-08-02 13:12:24 -07:00
getsample.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
giftopnm Initial file population. 2013-08-02 13:12:24 -07:00
gigablast.cbp added Codeblocks project file 2014-10-31 11:00:18 -07:00
gigablast.layout added Codeblocks project file 2014-10-31 11:00:18 -07:00
hash.cpp some optimizations for speed 2015-01-22 13:04:42 -07:00
hash.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
HashTable.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
HashTable.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
HashTableT.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
HashTableT.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
HashTableX.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
HashTableX.h fixes to pass internal qa tests in light 2015-04-01 11:20:36 -06:00
hashtest2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
hashtest3.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
hashtest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Highlight.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Highlight.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Hostdb.cpp fix bug of never calling callback 2015-04-13 09:56:21 -06:00
Hostdb.h just use INCOMING udp slots/sockets for jam detection. 2015-04-08 15:52:43 -06:00
hosts.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
HttpMime.cpp injecting warc files nicely now 2015-04-29 19:55:06 -07:00
HttpMime.h more fixes 2015-04-28 23:20:16 -07:00
HttpRequest.cpp fix bug of parms being set at seemingly random. 2015-02-03 17:52:44 -08:00
HttpRequest.h added support for supplying basic proxy authorization 2015-02-02 13:23:38 -08:00
HttpServer.cpp wget cookie support 2015-05-02 21:52:58 -07:00
HttpServer.h added support for supplying basic proxy authorization 2015-02-02 13:23:38 -08:00
iana_charset.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
iana_charset.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
iconv.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Images.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Images.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Indexdb.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Indexdb.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
IndexList.cpp cleanup all warning when not using -m32 2014-11-12 14:11:27 -08:00
IndexList.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
IndexReadInfo.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
IndexReadInfo.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
IndexTable2.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
IndexTable2.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
IndexTable.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
IndexTable.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
init.gb.conf minor make install changes 2014-05-22 18:46:38 -07:00
injectme3 added injectme3 file and documentation into compare.html 2013-08-17 11:02:26 -06:00
injectmedemo fix sections.cpp to not set root title section 2014-12-11 19:54:33 -08:00
injector.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
iostream.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
ip.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
ip.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
ipconfig.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Iso8859.cpp Initial file population. 2013-08-02 13:12:24 -07:00
Iso8859.h Initial file population. 2013-08-02 13:12:24 -07:00
jointest.cpp Initial file population. 2013-08-02 13:12:24 -07:00
jpegtopnm Initial file population. 2013-08-02 13:12:24 -07:00
Json.cpp fix gbssdocid bug better 2015-04-13 14:33:57 -06:00
Json.h fix gbssDocId:123456789, et al, query. will only work for docs indexed 2015-04-13 14:13:16 -06:00
keepalive.cpp Initial file population. 2013-08-02 13:12:24 -07:00
Lang.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Lang.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
LangList.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
LangList.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Language.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Language.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
LanguageIdentifier.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
LanguageIdentifier.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
LanguagePages.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
LanguagePages.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
libc.a Initial file population. 2013-08-02 13:12:24 -07:00
libcrypto.a turn off hearbeats when compiling openssl libs 2014-04-22 16:39:40 -07:00
libgcc.a Initial file population. 2013-08-02 13:12:24 -07:00
libiconv64.a added 64 bit libiconv64.a 2014-11-14 17:34:11 -08:00
libiconv.a Initial file population. 2013-08-02 13:12:24 -07:00
libiconv.la Initial file population. 2013-08-02 13:12:24 -07:00
libjpeg.so.62 thumbnail generation support back in. 2014-04-24 10:13:45 -07:00
libm.a Initial file population. 2013-08-02 13:12:24 -07:00
libnetpbm.so.10 thumbnail generation support back in. 2014-04-24 10:13:45 -07:00
libpng12.so.0 thumbnail generation support back in. 2014-04-24 10:13:45 -07:00
libpthread.a Initial file population. 2013-08-02 13:12:24 -07:00
libssl.a turn off hearbeats when compiling openssl libs 2014-04-22 16:39:40 -07:00
libstdc++.a Initial file population. 2013-08-02 13:12:24 -07:00
libtiff.so.4 thumbnail generation support back in. 2014-04-24 10:13:45 -07:00
libz64.a add libz64.a 2014-11-17 14:53:59 -08:00
libz.a Initial file population. 2013-08-02 13:12:24 -07:00
libz.so.1 thumbnail generation support back in. 2014-04-24 10:13:45 -07:00
LICENSE license fix 2014-06-16 13:52:51 -07:00
Linkdb.cpp 4th time is a charm for links with spaces 2015-04-29 10:50:18 -07:00
Linkdb.h fix links parser so it harvests outlinks from rss feeds' 2015-03-12 17:35:47 -07:00
LinkedList.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
linkspam.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
linkspam.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Log.cpp use STDERR_FILENO not 2. 2015-02-08 20:02:05 -08:00
Log.h make new logfile when current logfile hits 1GB. 2015-01-05 11:29:49 -08:00
Loop.cpp better fixes for core from sig alarms 2015-04-13 10:28:43 -06:00
Loop.h added proper write callback registration into 2015-02-16 14:48:39 -07:00
looptest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
main.cpp gb -h fix 2015-04-22 12:51:32 -06:00
Make.depend force gb to recompile version every time 2014-09-19 12:23:40 -07:00
Makefile warc injects from file in spider pipeline working now. 2015-05-03 12:28:02 -07:00
malloc.c Initial file population. 2013-08-02 13:12:24 -07:00
matches2.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
matches2.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Matches.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Matches.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Mem.cpp proxy fixes 2015-03-05 11:10:40 -08:00
Mem.h replace memcpy_ass with bcopy 2015-01-14 14:12:55 -08:00
membustest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
MemPool.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
MemPool.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
MemPoolTree.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
MemPoolTree.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
memtest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
mergetest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
MetaContainer.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
MetaContainer.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Mime.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Mime.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
mixfile.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
mmseg.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
monitor.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Monitordb.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Monitordb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg0.cpp prevent core when injecting when not in sync with host #0 2015-04-28 15:29:26 -07:00
Msg0.h try to handle those quick tagdb lookups first. 2015-01-29 20:55:02 -07:00
Msg1.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg1.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Msg1f.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg1f.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg2.cpp allocate enough nodes in top tree for 2014-11-27 08:44:24 -08:00
Msg2.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg2a.cpp working with -m32 for basic testing. 2014-11-12 11:38:37 -08:00
Msg2a.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg2b.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg2b.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg3.cpp fix some more 2015-04-13 10:08:01 -06:00
Msg3.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg3a.cpp now return the total # of docs that have 2015-04-24 10:59:34 -06:00
Msg3a.h add skippedshards and totalshards to search results in xml/json 2015-02-27 08:17:32 -07:00
Msg3e.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg3e.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg4.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg4.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg5.cpp try to reduce log spam 2015-01-05 11:03:49 -08:00
Msg5.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg6b.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg6b.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg8b.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg8b.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg9b.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg9b.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg13.cpp Merge branch 'diffbot-testing' into ia 2015-05-03 12:08:25 -07:00
Msg13.h retry download if was in the twitchy table 2015-04-30 16:06:13 -07:00
Msg17.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg17.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg20.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg20.h fixed query reindex for diffbot json docs. 2014-12-15 16:49:20 -08:00
Msg22.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg22.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg24.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg28.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg28.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg30.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg30.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg35.cpp working with -m32 for basic testing. 2014-11-12 11:38:37 -08:00
Msg35.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg36.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Msg36.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg37.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg37.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg39.cpp now return the total # of docs that have 2015-04-24 10:59:34 -06:00
Msg39.h now show "totalDocsWithField" for each facet, so we know 2015-04-01 09:16:42 -06:00
Msg40.cpp fix csv when &stream=1 2015-04-25 09:07:03 -07:00
Msg40.h a hopeful fix for host #0 always crashing from 2015-04-02 15:17:49 -06:00
Msg40Cache.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg40Cache.h Initial file population. 2013-08-02 13:12:24 -07:00
Msg42.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Msg42.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg51.cpp fix core 2014-11-27 14:33:04 -07:00
Msg51.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msgaa.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msgaa.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
MsgC.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
MsgC.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msge0.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Msge0.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msge1.cpp loop.cpp cleanups. 2015-02-13 12:07:10 -08:00
Msge1.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Multicast.cpp fix a couple bugs. 2014-12-17 14:28:32 -08:00
Multicast.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
mysynonyms.txt mysyn fixes 2015-04-22 08:34:29 -06:00
numwords.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
PageAddColl.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
PageAddUrl.cpp do not consider .gz a 'media' url extension any more 2015-05-02 14:52:17 -07:00
PageBasic.cpp fix core from adding a lot of sites 2015-03-07 20:57:17 -07:00
PageCatdb.cpp return ENOPERM on certain pages if not 2015-01-29 09:46:48 -07:00
PageCrawlBot.cpp fix support for _html.json 2015-04-25 14:37:16 -07:00
PageCrawlBot.h more api updates 2014-07-13 09:35:44 -07:00
PageDirectory.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageEvents.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageGet.cpp now we add the spider status docs as json documents. 2015-03-19 16:17:36 -06:00
PageHosts.cpp clarify jam ups 2015-04-08 18:30:27 -07:00
PageIndexdb.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageInject.cpp complete overhaul of injection pipeline now compiles. 2015-05-03 19:07:44 -07:00
PageInject.h complete overhaul of injection pipeline now compiles. 2015-05-03 19:07:44 -07:00
PageLogView.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageNetTest.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageNetTest.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
PageOverview.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
PageParser.cpp quite a few bug fixes from adding the new query 2014-12-11 18:24:28 -08:00
PageParser.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
PagePerf.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageReindex.cpp now we add the spider status docs as json documents. 2015-03-19 16:17:36 -06:00
PageReindex.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
PageResults.cpp show diffbot uri in csv output 2015-04-25 08:48:03 -07:00
PageResults.h tap in some fixes for running the newly updated smokes 2015-04-21 15:20:57 -07:00
PageRoot.cpp complete overhaul of injection pipeline now compiles. 2015-05-03 19:07:44 -07:00
Pages.cpp clarify jam ups 2015-04-08 18:30:27 -07:00
Pages.h return ENOPERM on certain pages if not 2015-01-29 09:46:48 -07:00
PageSockets.cpp fix excessive looping when calling makeCallbacks() 2015-04-03 12:12:58 -06:00
PageSpam.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
PageStats.cpp fix spider request overflow/dropping algo. 2015-03-10 13:07:00 -07:00
PageStatsdb.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageSubmit.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageThesaurus.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
PageThreads.cpp working with -m32 for basic testing. 2014-11-12 11:38:37 -08:00
PageTitledb.cpp Merge branch 'diffbot-testing' into diffbot-matt 2014-11-20 16:53:07 -08:00
PageTurk.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageTurk.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Parms.cpp checkpoint #2 2015-05-03 17:51:47 -07:00
Parms.h checkpoint #2 2015-05-03 17:51:47 -07:00
parse_iana_charsets.pl move CollectionRec stuff into Collectiondb files 2013-12-10 15:28:04 -08:00
pdftohtml fix rdbcache init core 2014-12-01 12:37:51 -08:00
Phrases.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Phrases.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
PingServer.cpp tested auto proxy use and auto spider (non-proxy) backoff to 2015-04-30 15:31:09 -07:00
PingServer.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Placedb.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Placedb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
pngtopnm Initial file population. 2013-08-02 13:12:24 -07:00
pnmscale Initial file population. 2013-08-02 13:12:24 -07:00
Pops.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Pops.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
porter.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Pos.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Pos.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Posdb.cpp now return the total # of docs that have 2015-04-24 10:59:34 -06:00
Posdb.h now return the total # of docs that have 2015-04-24 10:59:34 -06:00
postalCodes.txt Initial file population. 2013-08-02 13:12:24 -07:00
PostQueryRerank.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PostQueryRerank.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
ppmtojpeg Initial file population. 2013-08-02 13:12:24 -07:00
Process.cpp Merge branch 'diffbot-testing' into ia 2015-05-03 12:08:25 -07:00
Process.h more fixes for new spider updates 2015-02-11 21:54:36 -08:00
Profiler.cpp comment update 2015-02-05 10:35:52 -08:00
Profiler.h more fixes for profiler. 2015-01-20 17:02:10 -07:00
Proxy.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Proxy.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
pstotext Initial file population. 2013-08-02 13:12:24 -07:00
qa.cpp fix qa test to make it easier to see the real diffs 2015-04-30 19:38:27 -07:00
QAClient.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
QAClient.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
quarantine.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Query.cpp update search api doc 2015-04-22 18:55:25 -06:00
Query.h fix counting of # docs that have facet field. 2015-04-06 14:41:44 -06:00
Rdb.cpp fix cores associated with new spider status docs. 2015-04-07 10:33:54 -07:00
Rdb.h added tree mem reclaimer for doledb since it 2015-02-12 12:12:25 -08:00
RdbBase.cpp removed buggy close least used linked list logic. 2015-04-14 15:26:46 -07:00
RdbBase.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
RdbBuckets.cpp added RdbBuckets::cleanBuckets() corresponding to 2015-03-21 22:28:34 -06:00
RdbBuckets.h added RdbBuckets::cleanBuckets() corresponding to 2015-03-21 22:28:34 -06:00
RdbCache.cpp fix rdbcache corruption bugs for winnerlistcache. 2015-03-07 11:09:06 -08:00
RdbCache.h fix rdbcache bug from 64bit conversino i guess 2014-11-19 10:31:31 -08:00
RdbDump.cpp try to fix a core from restarting a collection 2015-04-22 16:07:16 -07:00
RdbDump.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
RdbList.cpp miscellaneous spider/injection speedups. 2015-02-03 14:04:53 -08:00
RdbList.h added undeletable posdb key test to qainject1(). 2014-12-16 13:29:04 -08:00
RdbMap.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
RdbMap.h Merge branch 'diffbot-testing' into diffbot-matt 2014-11-20 16:53:07 -08:00
RdbMem.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
RdbMem.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
RdbMerge.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
RdbMerge.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
RdbScan.cpp Merge branch 'diffbot-testing' into diffbot-matt 2014-11-20 16:53:07 -08:00
RdbScan.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
rdbtest2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
rdbtest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
RdbTree.cpp fixed bug of gb not saving 2015-02-22 13:11:20 -07:00
RdbTree.h fixed bad deletenode call causing dups in 2015-02-12 16:12:23 -08:00
README.md update README.md 2015-03-19 23:31:09 -06:00
readRec.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Rebalance.cpp emergency fixes 2014-12-11 08:39:26 -08:00
Rebalance.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
reindex2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Repair.cpp clean out rebuild trees/buckets too 2015-03-21 22:42:49 -06:00
Repair.h Merge branch 'diffbot-testing' into diffbot-matt 2014-11-20 16:53:07 -08:00
RequestTable.cpp cleanup all warning when not using -m32 2014-11-12 14:11:27 -08:00
RequestTable.h cleanup all warning when not using -m32 2014-11-12 14:11:27 -08:00
rescue.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Revdb.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Revdb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
rmbots.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
S99gb added S99gb for loading at boot. 2014-06-23 07:32:38 -06:00
SafeBuf.cpp proxy fixes 2015-03-05 11:10:40 -08:00
SafeBuf.h proxy fixes 2015-03-05 11:10:40 -08:00
SafeList.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Sanity.h Merge branch 'diffbot-testing' into diffbot-matt 2014-11-20 16:53:07 -08:00
Scores.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Scores.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Scraper.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Scraper.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
SearchInput.cpp tap in some fixes for running the newly updated smokes 2015-04-21 15:20:57 -07:00
SearchInput.h added support for &nf=50 to limit to top 50 facets. 2015-01-29 10:34:22 -07:00
Sections.cpp fix sections.cpp to not set root title section 2014-12-11 19:54:33 -08:00
Sections.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
seektest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
seo.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
SiteGetter.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
SiteGetter.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
sitelinks.txt fixed missing sites in sitelinks.txt 2015-03-05 20:32:01 -08:00
sleepandlog.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
sort.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
sort.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Speller.cpp query stop words now based on selected langid. 2015-03-08 15:16:24 -07:00
Speller.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Spider.cpp do not consider .gz a 'media' url extension any more 2015-05-02 14:52:17 -07:00
Spider.h fix corrupt spider replies from causing a url with 2015-05-01 14:50:05 -07:00
SpiderProxy.cpp fix slow spider proxy loop 2015-04-23 21:17:55 -06:00
SpiderProxy.h proxy fixes 2015-03-05 11:10:40 -08:00
Stats.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Stats.h show total link overflows in spiderdb 2015-02-18 19:18:38 -07:00
Statsdb.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Statsdb.h fix signed/unsigned bug 2014-12-10 11:04:37 -08:00
StopWords.cpp fixed langid based query stop words. 2015-03-08 15:44:23 -07:00
StopWords.h fixed langid based query stop words. 2015-03-08 15:44:23 -07:00
streambuf.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Strings.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Strings.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Summary.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Summary.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
superMergeTest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
supported_charsets.cpp Initial file population. 2013-08-02 13:12:24 -07:00
supported_charsets.txt Initial file population. 2013-08-02 13:12:24 -07:00
Syncdb.cpp fixed bad deletenode call causing dups in 2015-02-12 16:12:23 -08:00
Syncdb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Synonyms.cpp query stop words now based on selected langid. 2015-03-08 15:16:24 -07:00
Synonyms.h nomenclature change 2014-12-04 11:02:54 -07:00
Tagdb.cpp emergency fixes for corrupt tagdb tag id 2015-03-20 17:21:52 -07:00
Tagdb.h added support for sitelinks.txt file 2015-01-31 15:18:06 -07:00
TcpServer.cpp Merge branch 'diffbot' into diffbot-testing 2015-05-03 11:21:22 -07:00
TcpServer.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
TcpSocket.h added proper write callback registration into 2015-02-16 14:48:39 -07:00
test2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
test_convert.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
test_hash.cpp replace long long with int64_t 2014-10-30 13:36:39 -06:00
test_norm.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
test_parser2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
test_parser.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
test_unicode.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Test.cpp now we add the spider status docs as json documents. 2015-03-19 16:17:36 -06:00
Test.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
testfloats.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Tfndb.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Tfndb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Thesaurus.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Thesaurus.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Threads.cpp count threads whose callback has not been called 2015-05-03 10:58:23 -07:00
Threads.h 64 bit somewhat working at runtime. need to test all functionality 2014-11-12 19:18:25 -08:00
threadtest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
thunder.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
tifftopnm Initial file population. 2013-08-02 13:12:24 -07:00
Timedb.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Timedb.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Timer.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Title.cpp query stop words now based on selected langid. 2015-03-08 15:16:24 -07:00
Title.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Titledb.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Titledb.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
TopTree.cpp fixed bad deletenode call causing dups in 2015-02-12 16:12:23 -08:00
TopTree.h fix cores in top tree with last commit. this one 2014-12-08 09:29:21 -08:00
treetest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
TuringTest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
TuringTest.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Turkdb.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
types.h allocate enough nodes in top tree for 2014-11-27 08:44:24 -08:00
UCNormalizer.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
UCNormalizer.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
UCPropTable.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
UCPropTable.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
UCWordIterator.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
UCWordIterator.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
UdpProtocol.h no limit to tagdb lookups even if niceness 1 2015-01-29 21:38:10 -07:00
UdpServer.cpp fix a core on seraph host #6 2015-04-22 15:46:35 -07:00
UdpServer.h just use INCOMING udp slots/sockets for jam detection. 2015-04-08 15:52:43 -06:00
UdpSlot.cpp fix bug of not able to get ip address because 2015-04-07 12:29:19 -07:00
UdpSlot.h fix udp linked list thing again 2015-04-13 10:13:59 -06:00
udptest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Unicode.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Unicode.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
UnicodeProperties.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
UnicodeProperties.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
unifiedDict.txt Initial file population. 2013-08-02 13:12:24 -07:00
uniq2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Url.cpp now all container doc logic is in xmldoc 2015-05-01 20:32:54 -07:00
Url.h checkpoint. moved warc and arc looping into xmldoc. 2015-05-01 19:11:13 -07:00
urlinfo.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Users.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Users.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
ValidPointer.cpp Initial file population. 2013-08-02 13:12:24 -07:00
ValidPointer.h Initial file population. 2013-08-02 13:12:24 -07:00
Vector.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Vector.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Version.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Version.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Weights.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Weights.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Wiki.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Wiki.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
wikititles.txt.part1 Initial file population. 2013-08-02 13:12:24 -07:00
wikititles.txt.part2 Initial file population. 2013-08-02 13:12:24 -07:00
wiktionary-buf.txt when user searches for a word without the 2014-06-01 09:37:00 -07:00
wiktionary-lang.txt when user searches for a word without the 2014-06-01 09:37:00 -07:00
wiktionary-syns.dat when user searches for a word without the 2014-06-01 09:37:00 -07:00
Wiktionary.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Wiktionary.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Words.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Words.h query stop words now based on selected langid. 2015-03-08 15:16:24 -07:00
Xml.cpp fix 'delete' checkbox in url filters. 2015-03-17 21:20:27 -06:00
Xml.h fix links parser so it harvests outlinks from rss feeds' 2015-03-12 17:35:47 -07:00
XmlDoc.cpp complete overhaul of injection pipeline now compiles. 2015-05-03 19:07:44 -07:00
XmlDoc.h checkpoint 2015-05-04 00:17:17 +00:00
XmlNode.cpp sitemap.xml support for harvesting loc urls. 2015-03-17 14:26:16 -06:00
XmlNode.h sitemap.xml support for harvesting loc urls. 2015-03-17 14:26:16 -06:00
zconf.h updated to a new libz64.a. updated zconf.h and 2014-11-17 14:53:15 -08:00
zlib.h updated to a new libz64.a. updated zconf.h and 2014-11-17 14:53:15 -08:00

README.md

open-source-search-engine

An open source web and enterprise search engine and spider/crawler. As can be seen on http://www.gigablast.com/ .

RUNNING GIGABLAST

See html/faq.html for all administrative documentation including the quick start instructions.

Alternatively, visit http://www.gigablast.com/faq.html

CODE ARCHITECTURE

See html/developer.html for all code documentation.

Alternatively, visit http://www.gigablast.com/developer.html

CONTACT

Contact me for feature requests or help in general. I will work for free for good use cases. mattdwells@hotmail.com.