4 Commits

Author	SHA1	Message	Date
mwells	7d3cc672c8	use ./gb blaster -u <fileofurls> to just inject urls, ... but use -i to also add the outlinks to spiderdb.	2013-08-19 16:33:27 -06:00
mwells	95a020574c	set spiderlinks=1 when doing ... ./gb blaster -i <fileofurls> to index/inject a file of urls so that we add the outlinks to spiderdb. this will slow things down a little since we will have to do a dns lookup of the subdomain of each outlink, unless it is cached.	2013-08-19 16:15:58 -06:00
mwells	2c83b96ba4	Added support for 'gb blaster -i <fileofurls> <maxThreads>' to ... inject/index a file of urls. Committing older work for compare.html that shows differences between gigablast and solr, but has a lot of blanks.	2013-08-19 13:26:46 -06:00
Matt Wells	f6e560c1f4	Initial file population.	2013-08-02 13:12:24 -07:00