gigablast/open-source-search-engine
1
0
Fork 0
mirror of https://github.com/gigablast/open-source-search-engine.git synced 2024-10-04 12:17:35 +03:00
Code Issues Projects Releases Wiki Activity

Merge branch 'diffbot' into diffbot-testing

Browse Source
This commit is contained in:
Matt Wells 2014-03-20 15:39:03 -07:00
commit 67202f3731
1 changed files with 6 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
Spider.cpp
View File

@ -5134,7 +5134,12 @@ void SpiderLoop::startLoop ( ) {
"is permanently disabled. Restart to fix.");
// crawlinfo updating
if ( !g_loop.registerSleepCallback(1000,
// save bandwidth for now make this every 4 seconds not 1 second
// then try not to send crawlinfo the host should already have.
// each collrec can have a checksum for each host of the last
// info we sent it. but we should resend all every 100 secs anyway
// in case host when dead
if ( !g_loop.registerSleepCallback(4000,
this,
updateAllCrawlInfosSleepWrapper))
log("build: failed to register updatecrawlinfowrapper");