Commit Graph

7 Commits

Author SHA1 Message Date
Matt Wells
48b5330d9c only skip checking to spider a url of its
doleip table is empty
2014-03-03 13:22:27 -08:00
Matt Wells
189999509b code checkpoint. time slicing, faster spider code
compiling. now needs debug.
2014-02-04 17:34:43 -08:00
Matt Wells
4f64677b4f get new global preemptive cache
logic compiling, with section voting
stats.
2014-01-05 11:51:09 -08:00
mwells
82494baa89 move CollectionRec stuff into Collectiondb files
for simplicity.
2013-12-10 15:28:04 -08:00
mwells
40192249f9 spider speedups and fixes. 2013-09-25 11:58:03 -06:00
Matt Wells
991e2f30f7 speed up whitelist hashtable like 20x
using hashtable key magic.
2013-09-15 21:10:53 -07:00
Matt Wells
f6e560c1f4 Initial file population. 2013-08-02 13:12:24 -07:00