Nov 20 2017 -- A distributed open source search engine and spider/crawler written in C/C++ for Linux on Intel/AMD. From gigablast dot com, which has binaries for download. See the README.md file at the very bottom of this page for instructions.
Go to file
Matt 94a8210586 added CSV to output dropdown. show all json fields
for spider status doc csv files. support spider status
docs in csv output.
2015-04-01 13:53:03 -06:00
antiword-dir Initial file population. 2013-08-02 13:12:24 -07:00
diffbot-widget widget updates 2014-04-21 09:21:28 -07:00
html Merge branch 'testing' 2015-02-18 07:26:01 -07:00
openssl we already include our own 32-bit 2013-09-15 18:25:49 -06:00
ucdata Initial file population. 2013-08-02 13:12:24 -07:00
.gitignore added Codeblocks project file 2014-10-31 11:00:18 -07:00
Abbreviations.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Abbreviations.h replace long long with int64_t 2014-10-30 13:36:39 -06:00
Accessdb.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Accessdb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Address.cpp fixed langid based query stop words. 2015-03-08 15:44:23 -07:00
Address.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
addtest.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Ads.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Ads.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
AdultBit.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
AdultBit.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
animate.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
antiword fix ulimit and antiword bugs 2014-06-18 04:06:20 -07:00
AutoBan.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
AutoBan.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
badcattable.dat Initial file population. 2013-08-02 13:12:24 -07:00
BigFile.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
BigFile.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Bits.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Bits.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
blaster2.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Blaster.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Blaster.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
bmptopnm Initial file population. 2013-08-02 13:12:24 -07:00
Cachedb.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Cachedb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
camsort.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
catcountry.dat Initial file population. 2013-08-02 13:12:24 -07:00
Catdb.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Catdb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Categories.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Categories.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
CatRec.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
CatRec.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
character-sets Initial file population. 2013-08-02 13:12:24 -07:00
check_unicode.cpp Initial file population. 2013-08-02 13:12:24 -07:00
Clusterdb.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Clusterdb.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Collectiondb.cpp added RdbBuckets::cleanBuckets() corresponding to 2015-03-21 22:28:34 -06:00
Collectiondb.h do away with filtered/banned spider priorities. 2015-03-17 20:27:23 -06:00
Conf.cpp put in some fixes to deal with doledb tree 2015-03-08 20:36:13 -07:00
Conf.h put in some fixes to deal with doledb tree 2015-03-08 20:36:13 -07:00
control.deb package bldg updates 2014-06-16 21:50:32 -06:00
convert.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
copyright.head package bldg updates 2014-06-16 21:50:32 -06:00
copyright.tail package bldg updates 2014-06-16 21:50:32 -06:00
CountryCode.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
CountryCode.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
create_ucd_tables.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
DailyMerge.cpp added new indicators so we can make gb more stable. 2014-12-16 16:22:50 -08:00
DailyMerge.h move CollectionRec stuff into Collectiondb files 2013-12-10 15:28:04 -08:00
DataFeed.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
DataFeed.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Datedb.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Datedb.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Dates.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Dates.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Diff.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Diff.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Dir.cpp support arc injections 2015-02-07 12:07:42 -08:00
Dir.h replace long long with int64_t 2014-10-30 13:36:39 -06:00
DiskPageCache.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
DiskPageCache.h fix up diskpagecache. how did it work before 2014-11-20 15:05:34 -08:00
dlstubs.c Initial file population. 2013-08-02 13:12:24 -07:00
dmozparse.cpp fix dmozparse compiler error 2015-03-06 20:37:30 -08:00
Dns.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Dns.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
DnsProtocol.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
dnstest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Domains.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Domains.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
dumpcore.cpp Initial file population. 2013-08-02 13:12:24 -07:00
Entities.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Entities.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Errno.cpp added logic to retry diffbot reply on connection reset, 2015-03-09 20:54:34 -07:00
Errno.h added logic to retry diffbot reply on connection reset, 2015-03-09 20:54:34 -07:00
errnotest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Events.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Facebook.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Facebook.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
fastIndexTable.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
fctypes.cpp speed up spiderDoledUrls() loop calling of 2015-02-10 11:47:53 -08:00
fctypes.h fix gb thrutest disk tests 2015-02-09 10:29:08 -08:00
File.cpp fix getFileSize(). fix warc injector. 2015-01-20 19:12:58 -07:00
File.h fix getFileSize(). fix warc injector. 2015-01-20 19:12:58 -07:00
filterquerylogs.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Flags.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Flags.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
gb-1.0.spec make it so we don't need --nodeps with 2014-05-25 22:08:46 -04:00
gb-include.h replace memcpy_ass with bcopy 2015-01-14 14:12:55 -08:00
gb.deb.rules if netpbm pkg already installed use it. 2014-07-06 09:54:28 -07:00
gb.pem so we have spider https sites add 2013-10-13 00:15:39 -07:00
gbfilter.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
gbtitletest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
geneaology.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
generateSuperMergeCode.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
geo_ip_table.cpp Initial file population. 2013-08-02 13:12:24 -07:00
geo_ip_table.h Initial file population. 2013-08-02 13:12:24 -07:00
GeoIP_internal.h Initial file population. 2013-08-02 13:12:24 -07:00
GeoIP.c Initial file population. 2013-08-02 13:12:24 -07:00
GeoIP.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
GeoIPCity.c Initial file population. 2013-08-02 13:12:24 -07:00
GeoIPCity.h Initial file population. 2013-08-02 13:12:24 -07:00
getsample.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
giftopnm Initial file population. 2013-08-02 13:12:24 -07:00
gigablast.cbp added Codeblocks project file 2014-10-31 11:00:18 -07:00
gigablast.layout added Codeblocks project file 2014-10-31 11:00:18 -07:00
hash.cpp some optimizations for speed 2015-01-22 13:04:42 -07:00
hash.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
HashTable.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
HashTable.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
HashTableT.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
HashTableT.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
HashTableX.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
HashTableX.h fixes to pass internal qa tests in light 2015-04-01 11:20:36 -06:00
hashtest2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
hashtest3.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
hashtest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Highlight.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Highlight.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Hostdb.cpp ignore dead shards when doing queries so they remain fast. 2015-02-27 08:02:19 -07:00
Hostdb.h if a shard is dead skip it when searching. 2015-02-27 07:28:41 -07:00
hosts.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
HttpMime.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
HttpMime.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
HttpRequest.cpp fix bug of parms being set at seemingly random. 2015-02-03 17:52:44 -08:00
HttpRequest.h added support for supplying basic proxy authorization 2015-02-02 13:23:38 -08:00
HttpServer.cpp proxy fixes 2015-03-05 11:10:40 -08:00
HttpServer.h added support for supplying basic proxy authorization 2015-02-02 13:23:38 -08:00
iana_charset.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
iana_charset.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
iconv.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Images.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Images.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Indexdb.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Indexdb.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
IndexList.cpp cleanup all warning when not using -m32 2014-11-12 14:11:27 -08:00
IndexList.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
IndexReadInfo.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
IndexReadInfo.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
IndexTable2.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
IndexTable2.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
IndexTable.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
IndexTable.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
init.gb.conf minor make install changes 2014-05-22 18:46:38 -07:00
injectme3 added injectme3 file and documentation into compare.html 2013-08-17 11:02:26 -06:00
injectmedemo fix sections.cpp to not set root title section 2014-12-11 19:54:33 -08:00
injector.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
iostream.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
ip.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
ip.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
ipconfig.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Iso8859.cpp Initial file population. 2013-08-02 13:12:24 -07:00
Iso8859.h Initial file population. 2013-08-02 13:12:24 -07:00
jointest.cpp Initial file population. 2013-08-02 13:12:24 -07:00
jpegtopnm Initial file population. 2013-08-02 13:12:24 -07:00
Json.cpp added logic to retry diffbot reply on connection reset, 2015-03-09 20:54:34 -07:00
Json.h added logic to retry diffbot reply on connection reset, 2015-03-09 20:54:34 -07:00
keepalive.cpp Initial file population. 2013-08-02 13:12:24 -07:00
Lang.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Lang.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
LangList.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
LangList.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Language.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Language.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
LanguageIdentifier.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
LanguageIdentifier.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
LanguagePages.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
LanguagePages.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
libc.a Initial file population. 2013-08-02 13:12:24 -07:00
libcrypto.a turn off hearbeats when compiling openssl libs 2014-04-22 16:39:40 -07:00
libgcc.a Initial file population. 2013-08-02 13:12:24 -07:00
libiconv64.a added 64 bit libiconv64.a 2014-11-14 17:34:11 -08:00
libiconv.a Initial file population. 2013-08-02 13:12:24 -07:00
libiconv.la Initial file population. 2013-08-02 13:12:24 -07:00
libjpeg.so.62 thumbnail generation support back in. 2014-04-24 10:13:45 -07:00
libm.a Initial file population. 2013-08-02 13:12:24 -07:00
libnetpbm.so.10 thumbnail generation support back in. 2014-04-24 10:13:45 -07:00
libpng12.so.0 thumbnail generation support back in. 2014-04-24 10:13:45 -07:00
libpthread.a Initial file population. 2013-08-02 13:12:24 -07:00
libssl.a turn off hearbeats when compiling openssl libs 2014-04-22 16:39:40 -07:00
libstdc++.a Initial file population. 2013-08-02 13:12:24 -07:00
libtiff.so.4 thumbnail generation support back in. 2014-04-24 10:13:45 -07:00
libz64.a add libz64.a 2014-11-17 14:53:59 -08:00
libz.a Initial file population. 2013-08-02 13:12:24 -07:00
libz.so.1 thumbnail generation support back in. 2014-04-24 10:13:45 -07:00
LICENSE license fix 2014-06-16 13:52:51 -07:00
Linkdb.cpp sitemap.xml support for harvesting loc urls. 2015-03-17 14:26:16 -06:00
Linkdb.h fix links parser so it harvests outlinks from rss feeds' 2015-03-12 17:35:47 -07:00
LinkedList.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
linkspam.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
linkspam.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Log.cpp use STDERR_FILENO not 2. 2015-02-08 20:02:05 -08:00
Log.h make new logfile when current logfile hits 1GB. 2015-01-05 11:29:49 -08:00
Loop.cpp fix for https urls through proxies 2015-02-21 09:25:54 -08:00
Loop.h added proper write callback registration into 2015-02-16 14:48:39 -07:00
looptest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
main.cpp warc injection fixes 2015-03-07 15:01:47 -08:00
Make.depend force gb to recompile version every time 2014-09-19 12:23:40 -07:00
Makefile fixes to pass internal qa tests in light 2015-04-01 11:20:36 -06:00
malloc.c Initial file population. 2013-08-02 13:12:24 -07:00
matches2.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
matches2.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Matches.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Matches.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Mem.cpp proxy fixes 2015-03-05 11:10:40 -08:00
Mem.h replace memcpy_ass with bcopy 2015-01-14 14:12:55 -08:00
membustest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
MemPool.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
MemPool.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
MemPoolTree.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
MemPoolTree.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
memtest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
mergetest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
MetaContainer.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
MetaContainer.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Mime.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Mime.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
mixfile.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
mmseg.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
monitor.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Monitordb.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Monitordb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg0.cpp try to handle those quick tagdb lookups first. 2015-01-29 20:55:02 -07:00
Msg0.h try to handle those quick tagdb lookups first. 2015-01-29 20:55:02 -07:00
Msg1.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg1.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Msg1f.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg1f.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg2.cpp allocate enough nodes in top tree for 2014-11-27 08:44:24 -08:00
Msg2.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg2a.cpp working with -m32 for basic testing. 2014-11-12 11:38:37 -08:00
Msg2a.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg2b.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg2b.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg3.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg3.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg3a.cpp now show "totalDocsWithField" for each facet, so we know 2015-04-01 09:16:42 -06:00
Msg3a.h add skippedshards and totalshards to search results in xml/json 2015-02-27 08:17:32 -07:00
Msg3e.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg3e.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg4.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg4.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg5.cpp try to reduce log spam 2015-01-05 11:03:49 -08:00
Msg5.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg6b.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg6b.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg8b.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg8b.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg9b.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg9b.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg13.cpp proxy fixes 2015-03-05 11:10:40 -08:00
Msg13.h more fixes 2015-02-02 14:06:38 -08:00
Msg17.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg17.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg20.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg20.h fixed query reindex for diffbot json docs. 2014-12-15 16:49:20 -08:00
Msg22.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg22.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg24.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg28.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg28.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg30.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg30.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg35.cpp working with -m32 for basic testing. 2014-11-12 11:38:37 -08:00
Msg35.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg36.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Msg36.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg37.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg37.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg39.cpp now show "totalDocsWithField" for each facet, so we know 2015-04-01 09:16:42 -06:00
Msg39.h now show "totalDocsWithField" for each facet, so we know 2015-04-01 09:16:42 -06:00
Msg40.cpp now show "totalDocsWithField" for each facet, so we know 2015-04-01 09:16:42 -06:00
Msg40.h fix core while streaming and getting EPIPE. 2015-01-28 19:53:38 -08:00
Msg40Cache.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg40Cache.h Initial file population. 2013-08-02 13:12:24 -07:00
Msg42.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Msg42.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg51.cpp fix core 2014-11-27 14:33:04 -07:00
Msg51.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msgaa.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msgaa.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
MsgC.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
MsgC.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msge0.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Msge0.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msge1.cpp loop.cpp cleanups. 2015-02-13 12:07:10 -08:00
Msge1.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Multicast.cpp fix a couple bugs. 2014-12-17 14:28:32 -08:00
Multicast.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
mysynonyms.txt Initial file population. 2013-08-02 13:12:24 -07:00
numwords.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
PageAddColl.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
PageAddUrl.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
PageBasic.cpp fix core from adding a lot of sites 2015-03-07 20:57:17 -07:00
PageCatdb.cpp return ENOPERM on certain pages if not 2015-01-29 09:46:48 -07:00
PageCrawlBot.cpp now we add the spider status docs as json documents. 2015-03-19 16:17:36 -06:00
PageCrawlBot.h more api updates 2014-07-13 09:35:44 -07:00
PageDirectory.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageEvents.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageGet.cpp now we add the spider status docs as json documents. 2015-03-19 16:17:36 -06:00
PageHosts.cpp more fixes for new spider updates 2015-02-11 21:54:36 -08:00
PageIndexdb.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageInject.cpp add switches to disable injections or querying 2015-03-04 10:49:37 -08:00
PageInject.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
PageLogView.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageNetTest.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageNetTest.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
PageOverview.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
PageParser.cpp quite a few bug fixes from adding the new query 2014-12-11 18:24:28 -08:00
PageParser.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
PagePerf.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageReindex.cpp now we add the spider status docs as json documents. 2015-03-19 16:17:36 -06:00
PageReindex.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
PageResults.cpp added CSV to output dropdown. show all json fields 2015-04-01 13:53:03 -06:00
PageResults.h Revert "fix json search results formatting." 2015-03-12 15:42:20 -07:00
PageRoot.cpp dont show add url and widgets tabs in serps 2015-03-07 15:32:35 -08:00
Pages.cpp fix a couple critical xml parsing bugs. fixes 2015-03-10 19:13:21 -07:00
Pages.h return ENOPERM on certain pages if not 2015-01-29 09:46:48 -07:00
PageSockets.cpp try to fix fdisset for writes on udpserver when 2015-03-06 09:30:36 -08:00
PageSpam.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
PageStats.cpp fix spider request overflow/dropping algo. 2015-03-10 13:07:00 -07:00
PageStatsdb.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageSubmit.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageThesaurus.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
PageThreads.cpp working with -m32 for basic testing. 2014-11-12 11:38:37 -08:00
PageTitledb.cpp Merge branch 'diffbot-testing' into diffbot-matt 2014-11-20 16:53:07 -08:00
PageTurk.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageTurk.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Parms.cpp inline doc update 2015-03-17 21:50:10 -06:00
Parms.h do away with filtered/banned spider priorities. 2015-03-17 20:27:23 -06:00
parse_iana_charsets.pl move CollectionRec stuff into Collectiondb files 2013-12-10 15:28:04 -08:00
pdftohtml fix rdbcache init core 2014-12-01 12:37:51 -08:00
Phrases.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Phrases.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
PingServer.cpp add # of tcp connections to hosts table 2015-02-03 14:14:17 -08:00
PingServer.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Placedb.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Placedb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
pngtopnm Initial file population. 2013-08-02 13:12:24 -07:00
pnmscale Initial file population. 2013-08-02 13:12:24 -07:00
Pops.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Pops.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
porter.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Pos.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Pos.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Posdb.cpp now show "totalDocsWithField" for each facet, so we know 2015-04-01 09:16:42 -06:00
Posdb.h fixed query reindex for diffbot json docs. 2014-12-15 16:49:20 -08:00
postalCodes.txt Initial file population. 2013-08-02 13:12:24 -07:00
PostQueryRerank.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PostQueryRerank.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
ppmtojpeg Initial file population. 2013-08-02 13:12:24 -07:00
Process.cpp put in some fixes to deal with doledb tree 2015-03-08 20:36:13 -07:00
Process.h more fixes for new spider updates 2015-02-11 21:54:36 -08:00
Profiler.cpp comment update 2015-02-05 10:35:52 -08:00
Profiler.h more fixes for profiler. 2015-01-20 17:02:10 -07:00
Proxy.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Proxy.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
pstotext Initial file population. 2013-08-02 13:12:24 -07:00
qa.cpp one more qa test fix for spider status docs 2015-04-01 12:47:32 -06:00
QAClient.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
QAClient.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
quarantine.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Query.cpp now show "totalDocsWithField" for each facet, so we know 2015-04-01 09:16:42 -06:00
Query.h now show "totalDocsWithField" for each facet, so we know 2015-04-01 09:16:42 -06:00
Rdb.cpp now we add the spider status docs as json documents. 2015-03-19 16:17:36 -06:00
Rdb.h added tree mem reclaimer for doledb since it 2015-02-12 12:12:25 -08:00
RdbBase.cpp dont attempt merge in quickpoll. just return do not 2015-03-02 07:26:38 -08:00
RdbBase.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
RdbBuckets.cpp added RdbBuckets::cleanBuckets() corresponding to 2015-03-21 22:28:34 -06:00
RdbBuckets.h added RdbBuckets::cleanBuckets() corresponding to 2015-03-21 22:28:34 -06:00
RdbCache.cpp fix rdbcache corruption bugs for winnerlistcache. 2015-03-07 11:09:06 -08:00
RdbCache.h fix rdbcache bug from 64bit conversino i guess 2014-11-19 10:31:31 -08:00
RdbDump.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
RdbDump.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
RdbList.cpp miscellaneous spider/injection speedups. 2015-02-03 14:04:53 -08:00
RdbList.h added undeletable posdb key test to qainject1(). 2014-12-16 13:29:04 -08:00
RdbMap.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
RdbMap.h Merge branch 'diffbot-testing' into diffbot-matt 2014-11-20 16:53:07 -08:00
RdbMem.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
RdbMem.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
RdbMerge.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
RdbMerge.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
RdbScan.cpp Merge branch 'diffbot-testing' into diffbot-matt 2014-11-20 16:53:07 -08:00
RdbScan.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
rdbtest2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
rdbtest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
RdbTree.cpp fixed bug of gb not saving 2015-02-22 13:11:20 -07:00
RdbTree.h fixed bad deletenode call causing dups in 2015-02-12 16:12:23 -08:00
README.md update README.md 2015-03-19 23:31:09 -06:00
readRec.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Rebalance.cpp emergency fixes 2014-12-11 08:39:26 -08:00
Rebalance.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
reindex2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Repair.cpp clean out rebuild trees/buckets too 2015-03-21 22:42:49 -06:00
Repair.h Merge branch 'diffbot-testing' into diffbot-matt 2014-11-20 16:53:07 -08:00
RequestTable.cpp cleanup all warning when not using -m32 2014-11-12 14:11:27 -08:00
RequestTable.h cleanup all warning when not using -m32 2014-11-12 14:11:27 -08:00
rescue.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Revdb.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Revdb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
rmbots.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
S99gb added S99gb for loading at boot. 2014-06-23 07:32:38 -06:00
SafeBuf.cpp proxy fixes 2015-03-05 11:10:40 -08:00
SafeBuf.h proxy fixes 2015-03-05 11:10:40 -08:00
SafeList.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Sanity.h Merge branch 'diffbot-testing' into diffbot-matt 2014-11-20 16:53:07 -08:00
Scores.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Scores.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Scraper.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Scraper.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
SearchInput.cpp fix searchinput::clear() bug. final fix 2015-03-15 07:48:29 -07:00
SearchInput.h added support for &nf=50 to limit to top 50 facets. 2015-01-29 10:34:22 -07:00
Sections.cpp fix sections.cpp to not set root title section 2014-12-11 19:54:33 -08:00
Sections.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
seektest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
seo.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
SiteGetter.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
SiteGetter.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
sitelinks.txt fixed missing sites in sitelinks.txt 2015-03-05 20:32:01 -08:00
sleepandlog.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
sort.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
sort.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Speller.cpp query stop words now based on selected langid. 2015-03-08 15:16:24 -07:00
Speller.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Spider.cpp now we add the spider status docs as json documents. 2015-03-19 16:17:36 -06:00
Spider.h now we add the spider status docs as json documents. 2015-03-19 16:17:36 -06:00
SpiderProxy.cpp proxy fixes 2015-03-05 11:10:40 -08:00
SpiderProxy.h proxy fixes 2015-03-05 11:10:40 -08:00
Stats.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Stats.h show total link overflows in spiderdb 2015-02-18 19:18:38 -07:00
Statsdb.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Statsdb.h fix signed/unsigned bug 2014-12-10 11:04:37 -08:00
StopWords.cpp fixed langid based query stop words. 2015-03-08 15:44:23 -07:00
StopWords.h fixed langid based query stop words. 2015-03-08 15:44:23 -07:00
streambuf.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Strings.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Strings.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Summary.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Summary.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
superMergeTest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
supported_charsets.cpp Initial file population. 2013-08-02 13:12:24 -07:00
supported_charsets.txt Initial file population. 2013-08-02 13:12:24 -07:00
Syncdb.cpp fixed bad deletenode call causing dups in 2015-02-12 16:12:23 -08:00
Syncdb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Synonyms.cpp query stop words now based on selected langid. 2015-03-08 15:16:24 -07:00
Synonyms.h nomenclature change 2014-12-04 11:02:54 -07:00
Tagdb.cpp emergency fixes for corrupt tagdb tag id 2015-03-20 17:21:52 -07:00
Tagdb.h added support for sitelinks.txt file 2015-01-31 15:18:06 -07:00
TcpServer.cpp speed up overflow check for firstip with a little 2015-03-10 13:24:03 -07:00
TcpServer.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
TcpSocket.h added proper write callback registration into 2015-02-16 14:48:39 -07:00
test2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
test_convert.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
test_hash.cpp replace long long with int64_t 2014-10-30 13:36:39 -06:00
test_norm.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
test_parser2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
test_parser.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
test_unicode.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Test.cpp now we add the spider status docs as json documents. 2015-03-19 16:17:36 -06:00
Test.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
testfloats.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Tfndb.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Tfndb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Thesaurus.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Thesaurus.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Threads.cpp try to speed up threads more 2015-02-05 15:00:18 -08:00
Threads.h 64 bit somewhat working at runtime. need to test all functionality 2014-11-12 19:18:25 -08:00
threadtest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
thunder.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
tifftopnm Initial file population. 2013-08-02 13:12:24 -07:00
Timedb.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Timedb.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Timer.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Title.cpp query stop words now based on selected langid. 2015-03-08 15:16:24 -07:00
Title.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Titledb.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Titledb.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
TopTree.cpp fixed bad deletenode call causing dups in 2015-02-12 16:12:23 -08:00
TopTree.h fix cores in top tree with last commit. this one 2014-12-08 09:29:21 -08:00
treetest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
TuringTest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
TuringTest.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Turkdb.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
types.h allocate enough nodes in top tree for 2014-11-27 08:44:24 -08:00
UCNormalizer.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
UCNormalizer.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
UCPropTable.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
UCPropTable.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
UCWordIterator.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
UCWordIterator.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
UdpProtocol.h no limit to tagdb lookups even if niceness 1 2015-01-29 21:38:10 -07:00
UdpServer.cpp try to fix fdisset for writes on udpserver when 2015-03-06 09:30:36 -08:00
UdpServer.h try to fix fdisset for writes on udpserver when 2015-03-06 09:30:36 -08:00
UdpSlot.cpp fix for infinite loop hang. 2015-03-07 16:28:53 -08:00
UdpSlot.h limit downloaded doc size to something 2015-02-16 09:43:39 -07:00
udptest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Unicode.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Unicode.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
UnicodeProperties.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
UnicodeProperties.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
unifiedDict.txt Initial file population. 2013-08-02 13:12:24 -07:00
uniq2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Url.cpp try ddomain only urls with www. when looking up 2015-01-31 15:33:37 -07:00
Url.h try ddomain only urls with www. when looking up 2015-01-31 15:33:37 -07:00
urlinfo.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Users.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Users.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
ValidPointer.cpp Initial file population. 2013-08-02 13:12:24 -07:00
ValidPointer.h Initial file population. 2013-08-02 13:12:24 -07:00
Vector.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Vector.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Version.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Version.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Weights.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Weights.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Wiki.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Wiki.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
wikititles.txt.part1 Initial file population. 2013-08-02 13:12:24 -07:00
wikititles.txt.part2 Initial file population. 2013-08-02 13:12:24 -07:00
wiktionary-buf.txt when user searches for a word without the 2014-06-01 09:37:00 -07:00
wiktionary-lang.txt when user searches for a word without the 2014-06-01 09:37:00 -07:00
wiktionary-syns.dat when user searches for a word without the 2014-06-01 09:37:00 -07:00
Wiktionary.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Wiktionary.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Words.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Words.h query stop words now based on selected langid. 2015-03-08 15:16:24 -07:00
Xml.cpp fix 'delete' checkbox in url filters. 2015-03-17 21:20:27 -06:00
Xml.h fix links parser so it harvests outlinks from rss feeds' 2015-03-12 17:35:47 -07:00
XmlDoc.cpp fixes to pass internal qa tests in light 2015-04-01 11:20:36 -06:00
XmlDoc.h now we add the spider status docs as json documents. 2015-03-19 16:17:36 -06:00
XmlNode.cpp sitemap.xml support for harvesting loc urls. 2015-03-17 14:26:16 -06:00
XmlNode.h sitemap.xml support for harvesting loc urls. 2015-03-17 14:26:16 -06:00
zconf.h updated to a new libz64.a. updated zconf.h and 2014-11-17 14:53:15 -08:00
zlib.h updated to a new libz64.a. updated zconf.h and 2014-11-17 14:53:15 -08:00

open-source-search-engine

An open source web and enterprise search engine and spider/crawler. As can be seen on http://www.gigablast.com/ .

RUNNING GIGABLAST

See html/faq.html for all administrative documentation including the quick start instructions.

Alternatively, visit http://www.gigablast.com/faq.html

CODE ARCHITECTURE

See html/developer.html for all code documentation.

Alternatively, visit http://www.gigablast.com/developer.html

CONTACT

Contact me for feature requests or help in general. I will work for free for good use cases. mattdwells@hotmail.com.