gigablast/open-source-search-engine
1
0
Fork 0
mirror of https://github.com/gigablast/open-source-search-engine.git synced 2024-10-04 04:07:13 +03:00
Code Issues Projects Releases Wiki Activity
Nov 20 2017 -- A distributed open source search engine and spider/crawler written in C/C++ for Linux on Intel/AMD. From gigablast dot com, which has binaries for download. See the README.md file at the very bottom of this page for instructions.
3,694 Commits 3 Branches 0 Tags 222 MiB
C++ 96.7%
HTML 1.4%
C 1.2%
Python 0.5%
Makefile 0.1%
a07c840a6a
Go to file
Matt Wells a07c840a6a join with threads when exiting -- to no avail 
exit status is still foobar.
2015-12-17 10:15:39 -08:00
antiword-dir Initial file population. 2013-08-02 13:12:24 -07:00
diffbot-widget widget updates 2014-04-21 09:21:28 -07:00
doxygen put in place doxygen stuffs 2015-05-15 14:47:47 -07:00
html added micro.html 2015-06-23 17:45:25 -06:00
openssl we already include our own 32-bit 2013-09-15 18:25:49 -06:00
script Increase time to mark item as stale in warc injector. 2015-11-01 19:45:29 -07:00
ucdata Initial file population. 2013-08-02 13:12:24 -07:00
.gitignore added Codeblocks project file 2014-10-31 11:00:18 -07:00
Abbreviations.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Abbreviations.h replace long long with int64_t 2014-10-30 13:36:39 -06:00
Accessdb.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Accessdb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Address.cpp fixed langid based query stop words. 2015-03-08 15:44:23 -07:00
Address.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
addtest.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Ads.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Ads.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
AdultBit.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
AdultBit.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
animate.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
antiword fix ulimit and antiword bugs 2014-06-18 04:06:20 -07:00
AutoBan.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
AutoBan.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
badcattable.dat Initial file population. 2013-08-02 13:12:24 -07:00
BigFile.cpp added FIXBUG code to fix seg fault from 2015-12-08 10:30:16 -08:00
BigFile.h all files made are now group writable. 2015-09-21 11:19:34 -06:00
Bits.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Bits.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
blaster2.cpp fix right 2015-10-08 13:42:42 -07:00
Blaster.cpp bring back max mem control into master controls. 2015-08-14 12:58:54 -06:00
Blaster.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
bmptopnm Initial file population. 2013-08-02 13:12:24 -07:00
Cachedb.cpp fix compiler warnings 2015-09-10 13:24:59 -06:00
Cachedb.h do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
camsort.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
catcountry.dat Initial file population. 2013-08-02 13:12:24 -07:00
Catdb.cpp do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
Catdb.h do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
Categories.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Categories.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
CatRec.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
CatRec.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
character-sets Initial file population. 2013-08-02 13:12:24 -07:00
check_unicode.cpp Initial file population. 2013-08-02 13:12:24 -07:00
Clusterdb.cpp fix compiler warnings 2015-09-10 13:24:59 -06:00
Clusterdb.h do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
Collectiondb.cpp zero out crazy local spider stats. 2015-12-17 09:43:41 -08:00
Collectiondb.h complete merge of ia code into testing. 2015-11-09 12:46:06 -07:00
Conf.cpp fix permissions bug when creating directories, 2015-10-07 08:26:27 -06:00
Conf.h fix file/dir creation permissions bugs 2015-09-21 12:44:41 -06:00
control.deb package bldg updates 2014-06-16 21:50:32 -06:00
convert.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
copyright.head package bldg updates 2014-06-16 21:50:32 -06:00
copyright.tail package bldg updates 2014-06-16 21:50:32 -06:00
CountryCode.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
CountryCode.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
create_ucd_tables.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
DailyMerge.cpp fix so we can generate posdb map for 2015-11-01 14:56:39 -08:00
DailyMerge.h move CollectionRec stuff into Collectiondb files 2013-12-10 15:28:04 -08:00
DataFeed.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
DataFeed.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Datedb.cpp do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
Datedb.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Dates.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Dates.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Diff.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Diff.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Dir.cpp try to fix core dumps. not sure how 2015-08-22 08:52:28 -07:00
Dir.h replace long long with int64_t 2014-10-30 13:36:39 -06:00
DiskPageCache.cpp re-disbale page cache. wtf? 2015-09-09 22:06:00 -07:00
DiskPageCache.h the new disk page cache. temporarily disabled. 2015-08-14 15:52:24 -06:00
dlstubs.c Initial file population. 2013-08-02 13:12:24 -07:00
dmozparse.cpp fix make dmozparse 2015-09-13 13:21:36 -07:00
Dns.cpp More fixes to prevent spider traffic from hitting hosts with nospider 2015-11-13 15:03:02 -07:00
Dns.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
DnsProtocol.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
dnstest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Domains.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Domains.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
dumpcore.cpp Initial file population. 2013-08-02 13:12:24 -07:00
Entities.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Entities.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Errno.cpp complete merge of ia code into testing. 2015-11-09 12:46:06 -07:00
Errno.h complete merge of ia code into testing. 2015-11-09 12:46:06 -07:00
errnotest.cpp errnotest.cpp fix 2015-08-24 16:22:11 -06:00
Events.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Facebook.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Facebook.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
fastIndexTable.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
fctypes.cpp Merge branch 'ia' into ia-zak 2015-10-10 14:07:15 -06:00
fctypes.h Merge branch 'ia' into testing 2015-10-12 10:40:16 -06:00
File.cpp fix file/dir creation permissions bugs 2015-09-21 12:44:41 -06:00
File.h just use the user's umask val for all file/dir creation 2015-09-21 11:33:38 -06:00
filterquerylogs.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Flags.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Flags.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
gb-1.0.spec make it so we don't need --nodeps with 2014-05-25 22:08:46 -04:00
gb-include.h replace memcpy_ass with bcopy 2015-01-14 14:12:55 -08:00
gb.deb.rules if netpbm pkg already installed use it. 2014-07-06 09:54:28 -07:00
gb.pem so we have spider https sites add 2013-10-13 00:15:39 -07:00
gbfilter.cpp fix file/dir creation permissions bugs 2015-09-21 12:44:41 -06:00
gbtitletest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
geneaology.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
generateSuperMergeCode.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
geo_ip_table.cpp Initial file population. 2013-08-02 13:12:24 -07:00
geo_ip_table.h Initial file population. 2013-08-02 13:12:24 -07:00
GeoIP_internal.h Initial file population. 2013-08-02 13:12:24 -07:00
GeoIP.c Initial file population. 2013-08-02 13:12:24 -07:00
GeoIP.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
GeoIPCity.c Initial file population. 2013-08-02 13:12:24 -07:00
GeoIPCity.h Initial file population. 2013-08-02 13:12:24 -07:00
getsample.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
giftopnm Initial file population. 2013-08-02 13:12:24 -07:00
gigablast.cbp added Codeblocks project file 2014-10-31 11:00:18 -07:00
gigablast.layout added Codeblocks project file 2014-10-31 11:00:18 -07:00
hash.cpp fix more possible unicode errors 2015-07-19 12:05:09 -06:00
hash.h fix more possible unicode errors 2015-07-19 12:05:09 -06:00
HashTable.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
HashTable.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
HashTableT.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
HashTableT.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
HashTableX.cpp fix file/dir creation permissions bugs 2015-09-21 12:44:41 -06:00
HashTableX.h quite a few bug fixes. 2015-07-02 17:42:05 -06:00
hashtest2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
hashtest3.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
hashtest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Highlight.cpp fix pesky memory leak finally 2015-07-13 17:47:34 -06:00
Highlight.h allow up to 3000 query terms. really we can allow 2015-07-10 19:02:30 -06:00
Hostdb.cpp fix getLeastLoadedInShard() to only return 2015-11-16 09:53:40 -07:00
Hostdb.h Fix host selection for downloading when nospider directives are present. 2015-11-29 21:36:19 -07:00
hosts.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
HttpMime.cpp Fix atotime5 to parse ISO8601. 2015-07-06 00:51:18 -06:00
HttpMime.h more fixes 2015-04-28 23:20:16 -07:00
HttpRequest.cpp can inject a single doc now 2015-05-03 21:14:28 -07:00
HttpRequest.h added support for supplying basic proxy authorization 2015-02-02 13:23:38 -08:00
HttpServer.cpp fix a couple of cores happening on crawlbot. 2015-11-30 13:26:43 -08:00
HttpServer.h added support for supplying basic proxy authorization 2015-02-02 13:23:38 -08:00
iana_charset.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
iana_charset.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
iconv.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Images.cpp fix file/dir creation permissions bugs 2015-09-21 12:44:41 -06:00
Images.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Indexdb.cpp fix compiler warnings 2015-09-10 13:24:59 -06:00
Indexdb.h do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
IndexList.cpp cleanup all warning when not using -m32 2014-11-12 14:11:27 -08:00
IndexList.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
IndexReadInfo.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
IndexReadInfo.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
IndexTable2.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
IndexTable2.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
IndexTable.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
IndexTable.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
init.gb.conf minor make install changes 2014-05-22 18:46:38 -07:00
injectme3 added injectme3 file and documentation into compare.html 2013-08-17 11:02:26 -06:00
injectmedemo fix sections.cpp to not set root title section 2014-12-11 19:54:33 -08:00
injector.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
iostream.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
ip.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
ip.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
ipconfig.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Iso8859.cpp Initial file population. 2013-08-02 13:12:24 -07:00
Iso8859.h Initial file population. 2013-08-02 13:12:24 -07:00
jointest.cpp Initial file population. 2013-08-02 13:12:24 -07:00
jpegtopnm Initial file population. 2013-08-02 13:12:24 -07:00
Json.cpp Add gbcapturedate to individual doc's metadata when injecting warcs. 2015-10-04 01:53:54 -06:00
Json.h Add gbcapturedate to individual doc's metadata when injecting warcs. 2015-10-04 01:53:54 -06:00
keepalive.cpp Initial file population. 2013-08-02 13:12:24 -07:00
Lang.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Lang.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
LangList.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
LangList.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Language.cpp fix file/dir creation permissions bugs 2015-09-21 12:44:41 -06:00
Language.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
LanguageIdentifier.cpp Add gbcapturedate to individual doc's metadata when injecting warcs. 2015-10-04 01:53:54 -06:00
LanguageIdentifier.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
LanguagePages.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
LanguagePages.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
libc.a Initial file population. 2013-08-02 13:12:24 -07:00
libcrypto.a turn off hearbeats when compiling openssl libs 2014-04-22 16:39:40 -07:00
libgcc.a Initial file population. 2013-08-02 13:12:24 -07:00
libiconv64.a added 64 bit libiconv64.a 2014-11-14 17:34:11 -08:00
libiconv.a Initial file population. 2013-08-02 13:12:24 -07:00
libiconv.la Initial file population. 2013-08-02 13:12:24 -07:00
libjpeg.so.62 thumbnail generation support back in. 2014-04-24 10:13:45 -07:00
libm.a Initial file population. 2013-08-02 13:12:24 -07:00
libnetpbm.so.10 thumbnail generation support back in. 2014-04-24 10:13:45 -07:00
libpng12.so.0 thumbnail generation support back in. 2014-04-24 10:13:45 -07:00
libpthread.a Initial file population. 2013-08-02 13:12:24 -07:00
libssl.a turn off hearbeats when compiling openssl libs 2014-04-22 16:39:40 -07:00
libstdc++.a Initial file population. 2013-08-02 13:12:24 -07:00
libtiff.so.4 thumbnail generation support back in. 2014-04-24 10:13:45 -07:00
libz64.a add libz64.a 2014-11-17 14:53:59 -08:00
libz.a Initial file population. 2013-08-02 13:12:24 -07:00
libz.so.1 thumbnail generation support back in. 2014-04-24 10:13:45 -07:00
LICENSE license fix 2014-06-16 13:52:51 -07:00
Linkdb.cpp Revert "hash the normalized outlinks in the diffbot reply" 2015-12-02 13:04:56 -07:00
Linkdb.h Revert "hash the normalized outlinks in the diffbot reply" 2015-12-02 13:04:56 -07:00
LinkedList.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
linkspam.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
linkspam.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Log.cpp fix file/dir creation permissions bugs 2015-09-21 12:44:41 -06:00
Log.h make new logfile when current logfile hits 1GB. 2015-01-05 11:29:49 -08:00
Loop.cpp added some more quickpolls. 2015-12-04 09:02:03 -08:00
Loop.h Fix load balance of msg22s to use the udp slots in pinginfo. 2015-11-03 11:51:19 -07:00
looptest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
main.cpp added gb rwtest and exposed seektest and thrutest in gb -h. 2015-11-30 21:29:17 -07:00
Make.depend force gb to recompile version every time 2014-09-19 12:23:40 -07:00
Makefile undo a change that didn't work. 2015-12-01 09:03:59 -08:00
malloc.c Initial file population. 2013-08-02 13:12:24 -07:00
matches2.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
matches2.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Matches.cpp Fix anomalous link text detector to take into consideration the total 2015-11-20 10:42:46 -07:00
Matches.h Fix anomalous link text detector to take into consideration the total 2015-11-20 10:42:46 -07:00
Mem.cpp remove fixme log comment 2015-11-18 08:11:45 -07:00
Mem.h fixes for umsg00 electric fence. 2015-08-24 11:35:33 -06:00
membustest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
MemPool.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
MemPool.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
MemPoolTree.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
MemPoolTree.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
memtest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
mergetest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
MetaContainer.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
MetaContainer.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Mime.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Mime.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
mixfile.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
mmseg.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
monitor.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Monitordb.cpp do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
Monitordb.h do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
Msg0.cpp prevent core when injecting when not in sync with host #0 2015-04-28 15:29:26 -07:00
Msg0.h try to handle those quick tagdb lookups first. 2015-01-29 20:55:02 -07:00
Msg1.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg1.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Msg1f.cpp fix file/dir creation permissions bugs 2015-09-21 12:44:41 -06:00
Msg1f.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg2.cpp fix pesky memory leak finally 2015-07-13 17:47:34 -06:00
Msg2.h allow up to 3000 query terms. really we can allow 2015-07-10 19:02:30 -06:00
Msg2a.cpp working with -m32 for basic testing. 2014-11-12 11:38:37 -08:00
Msg2a.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg2b.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg2b.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg3.cpp added some more quickpolls. 2015-12-04 09:02:03 -08:00
Msg3.h added cache validation logic 2015-09-10 13:56:38 -06:00
Msg3a.cpp Filter link text anomalies at query time. 2015-11-19 12:25:25 -07:00
Msg3a.h allow up to 3000 query terms. really we can allow 2015-07-10 19:02:30 -06:00
Msg3e.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg3e.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg4.cpp fix file/dir creation permissions bugs 2015-09-21 12:44:41 -06:00
Msg4.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg5.cpp do not hit file cache when merging files on disk. 2015-09-11 11:09:15 -07:00
Msg5.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg6b.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg6b.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg8b.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg8b.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg9b.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg9b.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg13.cpp Fix host selection for downloading when nospider directives are present. 2015-11-29 21:36:19 -07:00
Msg13.h in the sockets table page, 2015-08-25 09:34:45 -07:00
Msg17.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg17.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg20.cpp Merge branch 'testing' into diffbot-testing 2015-12-09 23:11:37 -07:00
Msg20.h do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
Msg22.cpp fix getLeastLoadedInShard() to only return 2015-11-16 09:53:40 -07:00
Msg22.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg24.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg28.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg28.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg30.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msg30.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg35.cpp working with -m32 for basic testing. 2014-11-12 11:38:37 -08:00
Msg35.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg36.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Msg36.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg37.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg37.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg39.cpp fix cores on gi #0 2015-09-25 08:09:05 -07:00
Msg39.h fix some mem leaks from allowing really big queries. 2015-07-13 23:17:53 -06:00
Msg40.cpp undo a change that didn't work. 2015-12-01 09:03:59 -08:00
Msg40.h Fix double call of gotSummary when computing facets in msg40. Fixes 2015-10-20 17:21:37 -06:00
Msg40Cache.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg40Cache.h Initial file population. 2013-08-02 13:12:24 -07:00
Msg42.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Msg42.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msg51.cpp fix core 2014-11-27 14:33:04 -07:00
Msg51.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msgaa.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Msgaa.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
MsgC.cpp Allow nospider and noquery on the same host. 2015-09-13 17:15:31 -06:00
MsgC.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msge0.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Msge0.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Msge1.cpp loop.cpp cleanups. 2015-02-13 12:07:10 -08:00
Msge1.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Multicast.cpp Fix load balance of msg22s to use the udp slots in pinginfo. 2015-11-03 11:51:19 -07:00
Multicast.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
mysynonyms.txt mysyn fixes 2015-04-22 08:34:29 -06:00
numwords.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
OldDiskPageCache.cpp bring back max mem control into master controls. 2015-08-14 12:58:54 -06:00
OldDiskPageCache.h undo #define thing 2015-08-14 13:08:11 -06:00
PageAddColl.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
PageAddUrl.cpp do not consider .gz a 'media' url extension any more 2015-05-02 14:52:17 -07:00
PageBasic.cpp fix core from adding a lot of sites 2015-03-07 20:57:17 -07:00
PageCatdb.cpp return ENOPERM on certain pages if not 2015-01-29 09:46:48 -07:00
PageCrawlBot.cpp add new link to page crawlbot to see spider attempt 2015-12-15 16:22:53 -08:00
PageCrawlBot.h more api updates 2014-07-13 09:35:44 -07:00
PageDirectory.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageEvents.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageGet.cpp get mike's super long query working 2015-07-13 14:59:44 -06:00
PageHosts.cpp Allow nospider and noquery on the same host. 2015-09-13 17:15:31 -06:00
PageIndexdb.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageInject.cpp Merge branch 'ia' into testing 2015-11-09 11:14:00 -07:00
PageInject.h show inject requests in the spider queue table now 2015-09-11 14:16:26 -06:00
PageLogView.cpp More testing on nospider, noquery. 2015-08-31 10:47:19 -06:00
PageNetTest.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageNetTest.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
PageOverview.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
PageParser.cpp quite a few bug fixes from adding the new query 2014-12-11 18:24:28 -08:00
PageParser.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
PagePerf.cpp Fixes to injector script. 2015-08-13 23:29:20 -06:00
PageReindex.cpp make query reindex (not query delete) distribute 2015-05-07 09:08:59 -07:00
PageReindex.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
PageResults.cpp Merge branch 'diffbot-testing' into testing 2015-10-12 10:39:35 -06:00
PageResults.h some debug statement to track down the socket snafu on host 0 2015-09-10 19:18:48 -07:00
PageRoot.cpp fix empty url condition in add url. 2015-11-19 16:14:12 -07:00
Pages.cpp turn off profiler automatically after 60 seconds. 2015-09-10 13:37:14 -06:00
Pages.h return ENOPERM on certain pages if not 2015-01-29 09:46:48 -07:00
PageSockets.cpp fix bug of losing the line waiter header 2015-11-19 19:40:30 -07:00
PageSpam.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
PageStats.cpp fix empty winner tree bug. 2015-10-02 12:16:48 -07:00
PageStatsdb.cpp Warc pipe fixes. Fix arcs not processing https. Fix nulls being left 2015-10-12 00:30:28 -06:00
PageSubmit.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageThesaurus.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
PageThreads.cpp undo some possible averse changes 2015-09-04 11:31:43 -07:00
PageTitledb.cpp Merge branch 'diffbot-testing' into diffbot-matt 2014-11-20 16:53:07 -08:00
PageTurk.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PageTurk.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Parms.cpp fix for all urls getting malformed url (EBADURL) 2015-12-15 10:06:06 -08:00
Parms.h move 2nd occurence of same collnum_t collection id 2015-08-18 18:59:01 -07:00
parse_iana_charsets.pl move CollectionRec stuff into Collectiondb files 2013-12-10 15:28:04 -08:00
pdftohtml fix rdbcache init core 2014-12-01 12:37:51 -08:00
Phrases.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Phrases.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
PingServer.cpp fix core from sending a url alert, then customer deleting 2015-09-08 15:57:46 -07:00
PingServer.h fix core from sending a url alert, then customer deleting 2015-09-08 15:57:46 -07:00
Placedb.cpp do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
Placedb.h do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
pngtopnm Initial file population. 2013-08-02 13:12:24 -07:00
pnmscale Initial file population. 2013-08-02 13:12:24 -07:00
Pops.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Pops.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
porter.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Pos.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Pos.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Posdb.cpp Merge branch 'diffbot-testing' into testing 2015-10-12 10:39:35 -06:00
Posdb.h do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
postalCodes.txt Initial file population. 2013-08-02 13:12:24 -07:00
PostQueryRerank.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
PostQueryRerank.h get mike's super long query working 2015-07-13 14:59:44 -06:00
ppmtojpeg Initial file population. 2013-08-02 13:12:24 -07:00
Process.cpp try to fix exiting w/ pthreads some more (part 2) 2015-12-17 08:38:12 -07:00
Process.h more fixes for new spider updates 2015-02-11 21:54:36 -08:00
Profiler.cpp Merge branch 'diffbot-testing' into testing 2015-11-09 11:13:42 -07:00
Profiler.h turn off profiler automatically after 60 seconds. 2015-09-10 13:37:14 -06:00
Proxy.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Proxy.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
pstotext Initial file population. 2013-08-02 13:12:24 -07:00
Punycode.cpp Start to detect non-asci urls and encode them to ascii. 2015-09-12 15:47:33 -06:00
Punycode.h Start to detect non-asci urls and encode them to ascii. 2015-09-12 15:47:33 -06:00
qa.cpp complete merge of ia code into testing. 2015-11-09 12:46:06 -07:00
QAClient.cpp good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
QAClient.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
quarantine.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Query.cpp Filter link text anomalies at query time. 2015-11-19 12:25:25 -07:00
Query.h fix more cores from the dynamic query size changes. 2015-07-18 14:15:47 -06:00
Rdb.cpp added some more quickpolls. 2015-12-04 09:02:03 -08:00
Rdb.h do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
RdbBase.cpp fix urgent merge mode bug some more? 2015-11-24 08:51:18 -08:00
RdbBase.h fix bug of dumping too many files to disk and not 2015-11-17 09:52:41 -08:00
RdbBuckets.cpp fix file/dir creation permissions bugs 2015-09-21 12:44:41 -06:00
RdbBuckets.h added RdbBuckets::cleanBuckets() corresponding to 2015-03-21 22:28:34 -06:00
RdbCache.cpp Merge branch 'diffbot-testing' into ia 2015-10-10 14:05:27 -06:00
RdbCache.h do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
RdbDump.cpp we were allocating too many nodes in top tree. tone that down. 2015-10-09 14:30:57 -06:00
RdbDump.h do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
RdbList.cpp fix for RdbList::constrain() core 2015-12-01 17:09:34 -07:00
RdbList.h fix churn bug in winnerlistcache in spider.cpp 2015-10-01 19:35:34 -07:00
RdbMap.cpp fix so we can generate posdb map for 2015-11-01 14:56:39 -08:00
RdbMap.h fix so we can generate posdb map for 2015-11-01 14:56:39 -08:00
RdbMem.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
RdbMem.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
RdbMerge.cpp fix core when exiting while merging 2015-10-24 12:50:57 -07:00
RdbMerge.h do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
RdbScan.cpp remove unnecessary line 2015-09-13 17:54:38 -07:00
RdbScan.h do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
rdbtest2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
rdbtest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
RdbTree.cpp Merge branch 'diffbot-testing' into ia-zak 2015-09-25 08:23:42 -06:00
RdbTree.h fixed bad deletenode call causing dups in 2015-02-12 16:12:23 -08:00
README.md update README.md 2015-03-19 23:31:09 -06:00
readRec.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Rebalance.cpp fix save prevention when coring in malloc/free. 2015-08-23 11:51:46 -07:00
Rebalance.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
reindex2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Repair.cpp clean out rebuild trees/buckets too 2015-03-21 22:42:49 -06:00
Repair.h Merge branch 'diffbot-testing' into diffbot-matt 2014-11-20 16:53:07 -08:00
RequestTable.cpp cleanup all warning when not using -m32 2014-11-12 14:11:27 -08:00
RequestTable.h cleanup all warning when not using -m32 2014-11-12 14:11:27 -08:00
rescue.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Revdb.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Revdb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
rmbots.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
S99gb added S99gb for loading at boot. 2014-06-23 07:32:38 -06:00
SafeBuf.cpp Merge branch 'ia' into testing 2015-11-09 11:14:00 -07:00
SafeBuf.h Merge branch 'ia' into testing 2015-11-09 11:14:00 -07:00
SafeList.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Sanity.h Merge branch 'diffbot-testing' into diffbot-matt 2014-11-20 16:53:07 -08:00
Scores.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Scores.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Scraper.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Scraper.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
SearchInput.cpp fix more cores from the dynamic query size changes. 2015-07-18 14:15:47 -06:00
SearchInput.h added support for &nf=50 to limit to top 50 facets. 2015-01-29 10:34:22 -07:00
Sections.cpp do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
Sections.h do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
seektest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
seo.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
SiteGetter.cpp fix infinite loop bug from EBADRBDID 2015-07-31 08:56:26 -07:00
SiteGetter.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
sitelinks.txt fixed missing sites in sitelinks.txt 2015-03-05 20:32:01 -08:00
sleepandlog.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
sort.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
sort.h use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Speller.cpp fix file/dir creation permissions bugs 2015-09-21 12:44:41 -06:00
Speller.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Spider.cpp join with threads when exiting -- to no avail 2015-12-17 10:15:39 -08:00
Spider.h improve spider performance when we have lots of collections. 2015-11-01 20:23:18 -08:00
SpiderProxy.cpp fix spider proxy table bug that seemed to be the 2015-11-21 10:43:23 -08:00
SpiderProxy.h spider proxy fixes for negative ports 2015-10-21 15:32:58 -07:00
Stats.cpp allow up to 3000 query terms. really we can allow 2015-07-10 19:02:30 -06:00
Stats.h allow up to 3000 query terms. really we can allow 2015-07-10 19:02:30 -06:00
Statsdb.cpp Fix repeating label. 2015-09-24 01:33:51 -06:00
Statsdb.h fix signed/unsigned bug 2014-12-10 11:04:37 -08:00
StopWords.cpp fixed langid based query stop words. 2015-03-08 15:44:23 -07:00
StopWords.h fixed langid based query stop words. 2015-03-08 15:44:23 -07:00
streambuf.h good checkpoint. quite a few fixes. 2014-11-17 18:13:36 -08:00
Strings.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Strings.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Summary.cpp fix some mem leaks from allowing really big queries. 2015-07-13 23:17:53 -06:00
Summary.h nomenclature changes 2015-07-13 18:42:13 -06:00
superMergeTest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
supported_charsets.cpp Initial file population. 2013-08-02 13:12:24 -07:00
supported_charsets.txt Initial file population. 2013-08-02 13:12:24 -07:00
Syncdb.cpp try to fix core dumps. not sure how 2015-08-22 08:52:28 -07:00
Syncdb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Synonyms.cpp fix pesky memory leak finally 2015-07-13 17:47:34 -06:00
Synonyms.h nomenclature change 2014-12-04 11:02:54 -07:00
Tagdb.cpp fix getLeastLoadedInShard() to only return 2015-11-16 09:53:40 -07:00
Tagdb.h do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
TcpServer.cpp fix bug of having a meta redirect tag 2015-09-16 11:03:38 -06:00
TcpServer.h prevent double ./gb start calls from messing 2015-08-31 11:13:33 -06:00
TcpSocket.h added proper write callback registration into 2015-02-16 14:48:39 -07:00
test2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
test_convert.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
test_hash.cpp replace long long with int64_t 2014-10-30 13:36:39 -06:00
test_norm.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
test_parser2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
test_parser.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
test_unicode.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Test.cpp now we add the spider status docs as json documents. 2015-03-19 16:17:36 -06:00
Test.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
testfloats.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Tfndb.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Tfndb.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Thesaurus.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Thesaurus.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Threads.cpp join with threads when exiting -- to no avail 2015-12-17 10:15:39 -08:00
Threads.h try to fix exiting w/ pthreads some more (part 2) 2015-12-17 08:38:12 -07:00
threadtest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
thunder.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
tifftopnm Initial file population. 2013-08-02 13:12:24 -07:00
Timedb.cpp text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Timedb.h text replacements for bad int32_t substitutions 2014-11-17 18:24:38 -08:00
Timer.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Title.cpp query stop words now based on selected langid. 2015-03-08 15:16:24 -07:00
Title.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Titledb.cpp do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
Titledb.h do not store cblock, etc. tags into tagdb to save 2015-09-10 12:46:00 -06:00
TopTree.cpp fixed bad deletenode call causing dups in 2015-02-12 16:12:23 -08:00
TopTree.h fix cores in top tree with last commit. this one 2014-12-08 09:29:21 -08:00
treetest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
TuringTest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
TuringTest.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Turkdb.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
types.h allocate enough nodes in top tree for 2014-11-27 08:44:24 -08:00
UCNormalizer.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
UCNormalizer.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
UCPropTable.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
UCPropTable.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
UCWordIterator.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
UCWordIterator.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
UdpProtocol.h no limit to tagdb lookups even if niceness 1 2015-01-29 21:38:10 -07:00
UdpServer.cpp Merge branch 'ia-zak' of https://github.com/gigablast/open-source-search-engine into ia-zak 2015-09-10 21:32:36 -06:00
UdpServer.h Add logic to limit number of msg7s to 100 per hosts, then we drop the 2015-09-03 22:17:16 -06:00
UdpSlot.cpp fix bug of not able to get ip address because 2015-04-07 12:29:19 -07:00
UdpSlot.h Merge branch 'diffbot-testing' of github.com:gigablast/open-source-search-engine into diffbot-testing 2015-12-17 06:08:47 -07:00
udptest.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Unicode.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Unicode.h Optimize UTF-8 handling in getUtf8CharSize() by using logic instead of table lookup (memory fetch) for bytes<128 2015-09-07 13:32:36 +02:00
UnicodeProperties.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
UnicodeProperties.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
unifiedDict.txt Initial file population. 2013-08-02 13:12:24 -07:00
uniq2.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Url.cpp added gb rwtest and exposed seektest and thrutest in gb -h. 2015-11-30 21:29:17 -07:00
Url.h Show utf8 url in page results. 2015-09-21 16:44:40 -06:00
urlinfo.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Users.cpp fix save prevention when coring in malloc/free. 2015-08-23 11:51:46 -07:00
Users.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
ValidPointer.cpp Initial file population. 2013-08-02 13:12:24 -07:00
ValidPointer.h Initial file population. 2013-08-02 13:12:24 -07:00
Vector.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Vector.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Version.cpp now it compiles with -m32 2014-11-10 14:45:11 -08:00
Version.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Weights.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Weights.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Wiki.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Wiki.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
wikititles.txt.part1 Initial file population. 2013-08-02 13:12:24 -07:00
wikititles.txt.part2 Initial file population. 2013-08-02 13:12:24 -07:00
wiktionary-buf.txt when user searches for a word without the 2014-06-01 09:37:00 -07:00
wiktionary-lang.txt when user searches for a word without the 2014-06-01 09:37:00 -07:00
wiktionary-syns.dat when user searches for a word without the 2014-06-01 09:37:00 -07:00
Wiktionary.cpp use gbmemcpy not memcpy so we can get profiler working again 2015-01-13 12:25:42 -07:00
Wiktionary.h now it compiles with -m32 2014-11-10 14:45:11 -08:00
Words.cpp quite a few bug fixes. 2015-07-02 17:42:05 -06:00
Words.h query stop words now based on selected langid. 2015-03-08 15:16:24 -07:00
Xml.cpp fix </script> tag detection stuff again. 2015-08-31 14:06:44 -06:00
Xml.h fix links parser so it harvests outlinks from rss feeds' 2015-03-12 17:35:47 -07:00
XmlDoc.cpp fix a fix 2015-12-14 17:06:58 -08:00
XmlDoc.h Merge branch 'ia' into testing 2015-11-09 11:14:00 -07:00
XmlNode.cpp sitemap.xml support for harvesting loc urls. 2015-03-17 14:26:16 -06:00
XmlNode.h sitemap.xml support for harvesting loc urls. 2015-03-17 14:26:16 -06:00
zconf.h updated to a new libz64.a. updated zconf.h and 2014-11-17 14:53:15 -08:00
zlib.h updated to a new libz64.a. updated zconf.h and 2014-11-17 14:53:15 -08:00

README.md

open-source-search-engine

An open source web and enterprise search engine and spider/crawler. As can be seen on http://www.gigablast.com/ .

RUNNING GIGABLAST

See html/faq.html for all administrative documentation including the quick start instructions.

Alternatively, visit http://www.gigablast.com/faq.html

CODE ARCHITECTURE

See html/developer.html for all code documentation.

Alternatively, visit http://www.gigablast.com/developer.html

CONTACT

Contact me for feature requests or help in general. I will work for free for good use cases. mattdwells@hotmail.com.