2097 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
mwells	55a1a4626f	more signal count stats	2014-09-03 09:18:30 -07:00
mwells	bb76ffa8b2	do not bother doing fcntl(F_SETSIG,fd) since ... we use select() now. however we still do get thread-cleanup signals from a thread when it is done.	2014-09-03 09:02:57 -07:00
mwells	cd97944d1e	fix cpu usage reporting	2014-09-03 08:38:43 -07:00
mwells	959187893e	default parm val updates for easier max spiders setting	2014-09-03 07:48:07 -07:00
mwells	5a508cad69	upped MAX_SPIDERS from 100 to 300. ... watch out for oom though.	2014-09-03 07:25:40 -07:00
mwells	7866ff1196	added some files	2014-09-03 06:40:04 -07:00
mwells	d9462e740b	handle threads on EINTR from select() in Loop.cpp.	2014-09-03 06:39:19 -07:00
mwells	82edb9814f	Merge branch 'master' into testing	2014-09-03 06:10:03 -07:00
mwells	19feb94f29	imported new Loop.cpp using select() from ... testing branch so ppl can use it on ubuntu 14.04	2014-09-02 22:22:39 -07:00
mwells	71e4574c71	added status file type to search filter bar	2014-09-02 22:17:27 -07:00
mwells	048aa60fd9	try polling fds now with select(). real-time signals ... were somewhat of an artifact from bygone days and the select() call when done right seems to not use any more cpu. plus it should work well with apple os x and cygwin etc.	2014-09-02 22:05:15 -07:00
mwells	2c81157031	fix search filters bar not coming up	2014-09-02 14:07:04 -07:00
mwells	c392e6eade	fix minserpscore parm. support TYPE_DOUBLE parms.	2014-09-01 20:37:17 -07:00
mwells	91ab558eb5	fix widget from initializing with serps ... because 'cd' was a text node, "No Results Found..." and did not have the getAttribute() function available thus causing a javascript error.	2014-09-01 19:24:04 -07:00
mwells	65e533bbb7	website updates	2014-09-01 17:23:15 -07:00
mwells	58d8861a34	widget page updates	2014-09-01 17:04:08 -07:00
mwells	48a3239276	spider status widget fixes	2014-09-01 14:22:03 -07:00
mwells	25b79684c5	website gui fixes	2014-09-01 13:31:13 -07:00
mwells	2ccc4626dc	more support for cloud initiative	2014-08-31 21:55:27 -07:00
mwells	7f622bd416	fixes for cloud support.	2014-08-31 16:23:11 -07:00
mwells	0aca84a5b2	website updates	2014-08-31 15:35:53 -07:00
mwells	1bc5fecb33	website updates	2014-08-31 11:11:12 -07:00
mwells	ef8cb47590	website updates.	2014-08-31 10:51:37 -07:00
mwells	0d4baf228e	fix display of dmoz pages	2014-08-31 10:11:16 -07:00
mwells	754d5b4755	rename admin.html to faq.html etc. file juggling.	2014-08-31 09:51:21 -07:00
mwells	6a5ad8a3e4	bio update	2014-08-30 22:41:44 -07:00
mwells	c68874240a	gigabot advice blurb fixes	2014-08-30 22:26:12 -07:00
mwells	253ed05074	gigabot advice blurb.	2014-08-30 22:14:53 -07:00
mwells	61f919a19c	website updates	2014-08-30 21:25:14 -07:00
mwells	aafacf5207	website updates	2014-08-30 20:49:11 -07:00
mwells	acd1672277	get crawlinfo every 3 seonds not 5.	2014-08-29 16:18:46 -07:00
mwells	2fdea42e78	lower SPIDER_DONE_TIMER so we can be more zippy ... to complete jobs and also so we can pass smoke tests for crawlCreationCompletionTime etc.	2014-08-29 13:50:25 -07:00
mwells	9d6437c2f8	remove html column from data csv output ... of json objects.	2014-08-29 11:58:30 -07:00
mwells	9de4e4bf3d	Merge branch 'testing' into diffbot-testing	2014-08-29 11:23:13 -07:00
mwells	aa966043ad	added quickpolls.	2014-08-28 19:45:25 -07:00
mwells	fb161e0102	more minor bug fixes.	2014-08-28 18:11:07 -07:00
mwells	4dbc6ed745	minor bug fix	2014-08-28 18:09:23 -07:00
mwells	060e887f08	misc/various bug fixes. ... fix canonical redir url bug with iframes.	2014-08-28 18:07:22 -07:00
mwells	3457245893	fix printf compiler warnings	2014-08-28 13:23:46 -07:00
mwells	caee238c46	fixes to make easier to compile on max os x.	2014-08-28 12:55:02 -07:00
mwells	94e8b43cc9	fix bug of not running df -ka to get disk usage	2014-08-28 09:49:47 -07:00
mwells	c641666b45	Merge branch 'master' of github.com:gigablast/open-source-search-engine	2014-08-28 08:52:48 -06:00
mwells	adcdf672bd	Merge branch 'testing'	2014-08-28 07:46:35 -07:00
mwells	38cef7d52e	fix # docs and recs bug.	2014-08-28 07:45:43 -07:00
mwells	e7aa933959	awesome updates to help.html page	2014-08-27 22:21:30 -07:00
mwells	51baca8917	Merge branch 'testing' of github.com:gigablast/open-source-search-engine into testing	2014-08-27 23:11:18 -06:00
mwells	ad8168f214	updates for query help table	2014-08-27 23:10:27 -06:00
mwells	8f6b82261f	makefile updates	2014-08-27 22:09:21 -06:00
mwells	f7af7ea2af	re-enable support for canonical url "redirects".	2014-08-27 19:28:48 -07:00
mwells	2a34e6b2b8	Merge branch 'testing'	2014-08-27 19:25:16 -07:00