Dmitry Smirnov
b1ace63607
codespell: spelling corrections
2021-05-06 01:52:55 +10:00
Matt
a5e6a12ff8
added support for TLS SNI (Server name identification)
2015-12-23 13:30:49 -07:00
Matt
13e0ba7bff
fix bug of having a meta redirect tag
...
in <script> tags. we have to use Xml class
to make sure it is a legit refresh tag.
2015-09-16 11:03:38 -06:00
Matt
cb03dec928
log debug tcp buf switch
2015-09-13 14:20:16 -07:00
Matt Wells
d0055cda6b
do not hit file cache when merging files on disk.
...
1. it prevents corruption. 2. it drains our cache of good stuff.
2015-09-11 11:09:15 -07:00
Matt Wells
1b9e509a55
more streaming socket fixes and debug stuff
2015-09-11 09:17:41 -07:00
Matt Wells
f3b2883268
fix core from new file caching logic
2015-09-10 20:42:45 -07:00
Matt Wells
ed9c5c580a
some debug statement to track down the socket snafu on host 0
2015-09-10 19:18:48 -07:00
Matt
efa93aad18
prevent double ./gb start calls from messing
...
things up.
2015-08-31 11:13:33 -06:00
Matt
289c6b90cf
ssl connect calls malloc
2015-08-24 18:12:28 -06:00
Matt Wells
c54e9cd96e
fix bug of printing results in csv when one
...
summary had an error.
2015-05-16 09:28:13 -07:00
Matt
a5c11c666c
try to fix double close bug while streaming
2015-05-12 12:37:19 -06:00
Matt Wells
79b81ede00
Merge branch 'diffbot' into diffbot-testing
2015-05-03 11:21:22 -07:00
Matt Wells
31e54df0c8
fix core from ::read returning 0.
...
do not do ban checks when autobackoff and autoproxies are disabled.
2015-05-03 10:17:36 -07:00
Matt
b15b7cdb94
log a note to hint how to debug ssl library errors
...
of some kind.
2015-05-01 18:15:53 -07:00
Matt
ef66e1e69f
speed up overflow check for firstip with a little
...
cache thingy. reduce log spam from ssl msgs.
2015-03-10 13:24:03 -07:00
Matt Wells
459c65100c
fix querying disabled bug.
2015-03-05 07:07:30 -08:00
mwells
4355665fa9
fix ssl handshake hanging forever. time it out now.
...
spider performance fix for when a firstip has massive # of
urls and can only have one spidered in the future.
2015-02-25 11:39:13 -07:00
Matt Wells
b80a70a6fd
fix for https urls through proxies
...
using newly updated tcp/loop code.
2015-02-21 09:25:54 -08:00
mwells
2bcbfd6852
fix double close/destroy bug.
2015-02-17 22:49:44 -07:00
Matt
ce8c97e8e2
fix log spam
2015-02-17 20:55:36 -07:00
Matt
dce8d9f930
fix qa bug of not resetting s_i.
...
fix tcpserver.cpp bug of destroying a streaming
socket after what is really not the final write.
2015-02-17 20:10:13 -07:00
Matt
2488c1a338
added proper write callback registration into
...
TcpServer.cpp so we only register write callbacks
when a non-blocking write does not write all the
bytes requested of it, or when a connection does not
complete. also fixed up the sslHandshake() function
which calls SSL_connect().
2015-02-16 14:48:39 -07:00
Matt Wells
f76f0c77d8
fix core while streaming and getting EPIPE.
...
but it seems like firefox and firefox only
has a bug it in when streaming json
with high start values like &s=20000&q=type:json
2015-01-28 19:53:38 -08:00
mwells
87285ba3cd
use gbmemcpy not memcpy so we can get profiler working again
...
since memcpy can't be interrupted and backtrace() called.
2015-01-13 12:25:42 -07:00
Matt Wells
b7cb2b56e1
try to debug the core
2014-12-23 18:53:11 -07:00
Matt
730b131bbf
added new indicators so we can make gb more stable.
...
now hosts table reports # ooms, disk read corruptions,
closed sockets from overloads, and we # of outstanding
spiders. made ping request a class so we can easily add
new indicators.
2014-12-16 16:22:50 -08:00
Matt
4e8a42e024
text replacements for bad int32_t substitutions
2014-11-17 18:24:38 -08:00
Matt
931a1c4bc6
good checkpoint. quite a few fixes.
2014-11-17 18:13:36 -08:00
Matt
4c19453ea9
working with -m32 for basic testing.
...
compiles for 64-bit.
2014-11-12 11:38:37 -08:00
Matt
96b8197ad3
now it compiles with -m32
2014-11-10 14:45:11 -08:00
Matt Wells
e7dd8f7956
replace long long with int64_t
2014-10-30 13:36:39 -06:00
Matt Wells
b13f3d24d7
replaced unsigned long long with uint64_t
2014-10-30 13:30:39 -06:00
Matt Wells
393cd90c63
remove log debug msg
2014-10-13 20:51:22 -07:00
Matt Wells
ef0e37d5a4
remove debug prints
2014-10-13 17:05:39 -07:00
Matt Wells
18d3b210ce
added some debug logic. (will remove shortly)
...
fix for mike's commit.
2014-10-13 16:36:43 -07:00
Matt Wells
8bb3545b71
emergency fixes for out of sockets core and
...
get proxy request timing out causing spider to hang bug.
2014-10-09 07:20:04 -07:00
Matt Wells
6a650a3856
fixed ssl CONNECT reply from being truncated.
...
after receiving the CONNECT reply of ok we had
to reset the total bytes to read to 0 to avoid
using the old value and truncating the follow up reply
of the actual webserver response.
2014-10-02 14:47:55 -07:00
mwells
4e7152b487
fix more bugs in squid proxy implementation.
...
force squid proxy stack to use floaters.
2014-10-02 11:54:50 -07:00
mwells
b50d3ee016
more select polling fixes
2014-09-11 07:16:39 -07:00
mwells
0b230995ee
more fixes for inner loop code
2014-09-11 05:56:47 -07:00
mwells
ee070d9378
try to fix inner loop logic some more
2014-09-11 05:26:14 -07:00
mwells
2870c1b0be
fixed system call hanging. i guess system calls fork
...
and it did not like or SIGALRMs because when i set the iterm
off for those it was ok.
2014-09-11 04:57:27 -07:00
mwells
e5aa7a0247
fix loop stuff some more. now seems fast again.
2014-09-10 14:56:56 -07:00
mwells
3623a49d6a
we no longer use write callbacks they seem pointless.
2014-09-04 10:56:27 -07:00
Matt Wells
ed66bf57b7
git ride of select on writefds. pretty pointless unless
...
we tried to write to a socket before and the buffer was full
so the write failed. then we'd want to know if it was ready for writing
again i guess.... anyway i'm not so sure that happens a lot so i took it
out and i guess we'll see what happens. also added more udp/loop
debugging statements.
2014-09-03 21:24:51 -07:00
mwells
9b94ce2e40
Merge branch 'diffbot-testing' into testing
...
Conflicts:
HttpRequest.cpp
TcpServer.cpp
2014-08-07 15:15:08 -07:00
mwells
5149f855af
more nyt.com bug fixes
2014-08-07 10:26:30 -07:00
mwells
6a28250e94
get qa test working after nyt bug fix
2014-08-06 16:00:25 -07:00
mwells
6434e5cc04
Merge branch 'testing' into diffbot-matt
...
Conflicts:
Errno.cpp
Errno.h
Parms.h
2014-07-07 09:49:59 -07:00
