8 Commits

Author	SHA1	Message	Date
mwells	87285ba3cd	use gbmemcpy not memcpy so we can get profiler working again ... since memcpy can't be interrupted and backtrace() called.	2015-01-13 12:25:42 -07:00
Matt	96b8197ad3	now it compiles with -m32	2014-11-10 14:45:11 -08:00
Matt Wells	e7dd8f7956	replace long long with int64_t	2014-10-30 13:36:39 -06:00
mwells	b24071caee	do not add crazy urls into spiderdb	2014-09-20 08:26:22 -06:00
Matt Wells	5ee2be8fcf	fixed data corruption bug. m_finalCrawlDelay ... was being stored in xmldoc titlerec header.	2013-11-27 14:18:15 -08:00
mwells	f562e6da9a	just ignore all urls with # (hashtag) in them ... from the dmoz dump. we were truncating http://twitter.com/#!/ronpaul to http://twitter.com/ and when looking up the catids of twitter.com got that ronpaul url. so that's bad. people should respect the hashtag.	2013-10-03 23:33:55 -06:00
David Sparks	0783c7395e	Copied these global vars from main.cpp to fix compilation error	2013-08-04 22:37:01 -07:00
Matt Wells	f6e560c1f4	Initial file population.	2013-08-02 13:12:24 -07:00