git.iko.soy/typeable/oauth2-proxy
on master help sign in
Merge pull request #1047 from oauth2-proxy/http-server

Refactor HTTP Server and add ServerGroup to handle graceful shutdown of multiple servers

commit6894738d975a66e9c2273b020896e2aa290e1714 authorJoel Speed <Joel.speed@hotmail.co.uk> dateSun 7 Mar 2021, 21:12 (6 years ago) parent602dac736887a0
20 files changed+1259−249
CHANGELOG.md+1 docs/docs/configuration/alpha_config.md+26−1 go.mod+1 go.sum+3 http.go−136 http_test.go−39 main.go+3−53 main_test.go+4−3 oauthproxy.go+62−4 oauthproxy_test.go+1 pkg/apis/options/alpha_options.go+16 pkg/apis/options/legacy_options.go+77 pkg/apis/options/legacy_options_test.go+93 pkg/apis/options/options.go+3−13 pkg/apis/options/server.go+27 pkg/http/http_suite_test.go+88 pkg/http/server.go+245 pkg/http/server_group.go+35 pkg/http/server_group_test.go+102 pkg/http/server_test.go+472
CHANGELOG.md+1
@@ -8,6 +8,7 @@
88
99## Changes since v7.0.1
1010
11+- [#1047](https://github.com/oauth2-proxy/oauth2-proxy/pull/1047) Refactor HTTP Server and add ServerGroup to handle graceful shutdown of multiple servers (@JoelSpeed)
1112- [#1070](https://github.com/oauth2-proxy/oauth2-proxy/pull/1070) Refactor logging middleware to middleware package (@NickMeves)
1213- [#1064](https://github.com/oauth2-proxy/oauth2-proxy/pull/1064) Add support for setting groups on session when using basic auth (@stefansedich)
1314- [#1056](https://github.com/oauth2-proxy/oauth2-proxy/pull/1056) Add option for custom logos on the sign in page (@JoelSpeed)
docs/docs/configuration/alpha_config.md+26−1
@@ -117,6 +117,8 @@ They may change between releases without notice.
117117| `upstreams` | _[Upstreams](#upstreams)_ | Upstreams is used to configure upstream servers.<br/>Once a user is authenticated, requests to the server will be proxied to<br/>these upstream servers based on the path mappings defined in this list. |
118118| `injectRequestHeaders` | _[[]Header](#header)_ | InjectRequestHeaders is used to configure headers that should be added<br/>to requests to upstream servers.<br/>Headers may source values from either the authenticated user's session<br/>or from a static secret value. |
119119| `injectResponseHeaders` | _[[]Header](#header)_ | InjectResponseHeaders is used to configure headers that should be added<br/>to responses from the proxy.<br/>This is typically used when using the proxy as an external authentication<br/>provider in conjunction with another proxy such as NGINX and its<br/>auth_request module.<br/>Headers may source values from either the authenticated user's session<br/>or from a static secret value. |
120+| `server` | _[Server](#server)_ | Server is used to configure the HTTP(S) server for the proxy application.<br/>You may choose to run both HTTP and HTTPS servers simultaneously.<br/>This can be done by setting the BindAddress and the SecureBindAddress simultaneously.<br/>To use the secure server you must configure a TLS certificate and key. |
121+| `metricsServer` | _[Server](#server)_ | MetricsServer is used to configure the HTTP(S) server for metrics.<br/>You may choose to run both HTTP and HTTPS servers simultaneously.<br/>This can be done by setting the BindAddress and the SecureBindAddress simultaneously.<br/>To use the secure server you must configure a TLS certificate and key. |
120122
121123### ClaimSource
122124
@@ -172,7 +174,7 @@ make up the header value
172174
173175### SecretSource
174176
175−(**Appears on:** [ClaimSource](#claimsource), [HeaderValue](#headervalue))
177+(**Appears on:** [ClaimSource](#claimsource), [HeaderValue](#headervalue), [TLS](#tls))
176178
177179SecretSource references an individual secret value.
178180Only one source within the struct should be defined at any time.
@@ -183,6 +185,29 @@ Only one source within the struct should be defined at any time.
183185| `fromEnv` | _string_ | FromEnv expects the name of an environment variable. |
184186| `fromFile` | _string_ | FromFile expects a path to a file containing the secret value. |
185187
188+### Server
189+
190+(**Appears on:** [AlphaOptions](#alphaoptions))
191+
192+Server represents the configuration for an HTTP(S) server
193+
194+| Field | Type | Description |
195+| ----- | ---- | ----------- |
196+| `BindAddress` | _string_ | BindAddress is the the address on which to serve traffic.<br/>Leave blank or set to "-" to disable. |
197+| `SecureBindAddress` | _string_ | SecureBindAddress is the the address on which to serve secure traffic.<br/>Leave blank or set to "-" to disable. |
198+| `TLS` | _[TLS](#tls)_ | TLS contains the information for loading the certificate and key for the<br/>secure traffic. |
199+
200+### TLS
201+
202+(**Appears on:** [Server](#server))
203+
204+TLS contains the information for loading a TLS certifcate and key.
205+
206+| Field | Type | Description |
207+| ----- | ---- | ----------- |
208+| `Key` | _[SecretSource](#secretsource)_ | Key is the the TLS key data to use.<br/>Typically this will come from a file. |
209+| `Cert` | _[SecretSource](#secretsource)_ | Cert is the TLS certificate data to use.<br/>Typically this will come from a file. |
210+
186211### Upstream
187212
188213(**Appears on:** [Upstreams](#upstreams))
go.mod+1
@@ -30,6 +30,7 @@ require (
3030 golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9
3131 golang.org/x/net v0.0.0-20200707034311-ab3426394381
3232 golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d
33+ golang.org/x/sync v0.0.0-20201207232520-09787c993a3a
3334 google.golang.org/api v0.20.0
3435 gopkg.in/natefinch/lumberjack.v2 v2.0.0
3536 gopkg.in/square/go-jose.v2 v2.4.1
go.sum+3
@@ -506,7 +506,10 @@ golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJ
506506golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
507507golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
508508golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
509+golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e h1:vcxGaoTs7kV8m5Np9uUNQin4BrLOthgV7252N8V+FwY=
509510golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
511+golang.org/x/sync v0.0.0-20201207232520-09787c993a3a h1:DcqTD9SDLc+1P/r1EmRBwnVsrOwW+kk2vWf9n+1sGhs=
512+golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
510513golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
511514golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
512515golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
http.godeleted−136
@@ -1,136 +0,0 @@
1−package main
2−
3−import (
4− "context"
5− "crypto/tls"
6− "errors"
7− "net"
8− "net/http"
9− "strings"
10− "time"
11−
12− "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/apis/options"
13− "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/logger"
14−)
15−
16−// Server represents an HTTP server
17−type Server struct {
18− Handler http.Handler
19− Opts *options.Options
20− stop chan struct{} // channel for waiting shutdown
21−}
22−
23−// ListenAndServe will serve traffic on HTTP or HTTPS depending on TLS options
24−func (s *Server) ListenAndServe() {
25− if s.Opts.TLSKeyFile != "" || s.Opts.TLSCertFile != "" {
26− s.ServeHTTPS()
27− } else {
28− s.ServeHTTP()
29− }
30−}
31−
32−// ServeHTTP constructs a net.Listener and starts handling HTTP requests
33−func (s *Server) ServeHTTP() {
34− HTTPAddress := s.Opts.HTTPAddress
35− var scheme string
36−
37− i := strings.Index(HTTPAddress, "://")
38− if i > -1 {
39− scheme = HTTPAddress[0:i]
40− }
41−
42− var networkType string
43− switch scheme {
44− case "", "http":
45− networkType = "tcp"
46− default:
47− networkType = scheme
48− }
49−
50− slice := strings.SplitN(HTTPAddress, "//", 2)
51− listenAddr := slice[len(slice)-1]
52−
53− listener, err := net.Listen(networkType, listenAddr)
54− if err != nil {
55− logger.Fatalf("FATAL: listen (%s, %s) failed - %s", networkType, listenAddr, err)
56− }
57− logger.Printf("HTTP: listening on %s", listenAddr)
58− s.serve(listener)
59− logger.Printf("HTTP: closing %s", listener.Addr())
60−}
61−
62−// ServeHTTPS constructs a net.Listener and starts handling HTTPS requests
63−func (s *Server) ServeHTTPS() {
64− addr := s.Opts.HTTPSAddress
65− config := &tls.Config{
66− MinVersion: tls.VersionTLS12,
67− MaxVersion: tls.VersionTLS13,
68− }
69− if config.NextProtos == nil {
70− config.NextProtos = []string{"http/1.1"}
71− }
72−
73− var err error
74− config.Certificates = make([]tls.Certificate, 1)
75− config.Certificates[0], err = tls.LoadX509KeyPair(s.Opts.TLSCertFile, s.Opts.TLSKeyFile)
76− if err != nil {
77− logger.Fatalf("FATAL: loading tls config (%s, %s) failed - %s", s.Opts.TLSCertFile, s.Opts.TLSKeyFile, err)
78− }
79−
80− ln, err := net.Listen("tcp", addr)
81− if err != nil {
82− logger.Fatalf("FATAL: listen (%s) failed - %s", addr, err)
83− }
84− logger.Printf("HTTPS: listening on %s", ln.Addr())
85−
86− tlsListener := tls.NewListener(tcpKeepAliveListener{ln.(*net.TCPListener)}, config)
87− s.serve(tlsListener)
88− logger.Printf("HTTPS: closing %s", tlsListener.Addr())
89−}
90−
91−func (s *Server) serve(listener net.Listener) {
92− srv := &http.Server{Handler: s.Handler}
93−
94− // See https://golang.org/pkg/net/http/#Server.Shutdown
95− idleConnsClosed := make(chan struct{})
96− go func() {
97− <-s.stop // wait notification for stopping server
98−
99− // We received an interrupt signal, shut down.
100− if err := srv.Shutdown(context.Background()); err != nil {
101− // Error from closing listeners, or context timeout:
102− logger.Printf("HTTP server Shutdown: %v", err)
103− }
104− close(idleConnsClosed)
105− }()
106−
107− err := srv.Serve(listener)
108− if err != nil && !errors.Is(err, http.ErrServerClosed) {
109− logger.Errorf("ERROR: http.Serve() - %s", err)
110− }
111− <-idleConnsClosed
112−}
113−
114−// tcpKeepAliveListener sets TCP keep-alive timeouts on accepted
115−// connections. It's used by ListenAndServe and ListenAndServeTLS so
116−// dead TCP connections (e.g. closing laptop mid-download) eventually
117−// go away.
118−type tcpKeepAliveListener struct {
119− *net.TCPListener
120−}
121−
122−func (ln tcpKeepAliveListener) Accept() (net.Conn, error) {
123− tc, err := ln.AcceptTCP()
124− if err != nil {
125− return nil, err
126− }
127− err = tc.SetKeepAlive(true)
128− if err != nil {
129− logger.Printf("Error setting Keep-Alive: %v", err)
130− }
131− err = tc.SetKeepAlivePeriod(3 * time.Minute)
132− if err != nil {
133− logger.Printf("Error setting Keep-Alive period: %v", err)
134− }
135− return tc, nil
136−}
http_test.godeleted−39
@@ -1,39 +0,0 @@
1−package main
2−
3−import (
4− "net/http"
5− "sync"
6− "testing"
7− "time"
8−
9− "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/apis/options"
10− "github.com/stretchr/testify/assert"
11−)
12−
13−func TestGracefulShutdown(t *testing.T) {
14− opts := options.NewOptions()
15− stop := make(chan struct{}, 1)
16− srv := Server{Handler: http.DefaultServeMux, Opts: opts, stop: stop}
17− var wg sync.WaitGroup
18− wg.Add(1)
19− go func() {
20− defer wg.Done()
21− srv.ServeHTTP()
22− }()
23−
24− stop <- struct{}{} // emulate catching signals
25−
26− // An idiomatic for sync.WaitGroup with timeout
27− c := make(chan struct{})
28− go func() {
29− defer close(c)
30− wg.Wait()
31− }()
32− select {
33− case <-c:
34− case <-time.After(1 * time.Second):
35− t.Fatal("Server should return gracefully but timeout has occurred")
36− }
37−
38− assert.Len(t, stop, 0) // check if stop chan is empty
39−}
main.go+3−53
@@ -1,20 +1,15 @@
11package main
22
33import (
4− "context"
54 "fmt"
65 "math/rand"
7− "net/http"
86 "os"
9− "os/signal"
107 "runtime"
11− "syscall"
128 "time"
139
1410 "github.com/ghodss/yaml"
1511 "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/apis/options"
1612 "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/logger"
17− "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/middleware"
1813 "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/validation"
1914 "github.com/spf13/pflag"
2015)
@@ -67,54 +62,9 @@ func main() {
6762
6863 rand.Seed(time.Now().UnixNano())
6964
70− oauthProxyStop := make(chan struct{}, 1)
71− metricsStop := startMetricsServer(opts.MetricsAddress, oauthProxyStop)
72−
73− s := &Server{
74− Handler: oauthproxy,
75− Opts: opts,
76− stop: oauthProxyStop,
77− }
78− // Observe signals in background goroutine.
79− go func() {
80− sigint := make(chan os.Signal, 1)
81− signal.Notify(sigint, os.Interrupt, syscall.SIGTERM)
82− <-sigint
83− s.stop <- struct{}{} // notify having caught signal stop oauthproxy
84− close(metricsStop) // and the metrics endpoint
85− }()
86− s.ListenAndServe()
87−}
88−
89−// startMetricsServer will start the metrics server on the specified address.
90−// It always return a channel to signal stop even when it does not run.
91−func startMetricsServer(address string, oauthProxyStop chan struct{}) chan struct{} {
92− stop := make(chan struct{}, 1)
93−
94− // Attempt to setup the metrics endpoint if we have an address
95− if address != "" {
96− s := &http.Server{Addr: address, Handler: middleware.DefaultMetricsHandler}
97− go func() {
98− // ListenAndServe always returns a non-nil error. After Shutdown or
99− // Close, the returned error is ErrServerClosed
100− if err := s.ListenAndServe(); err != http.ErrServerClosed {
101− logger.Println(err)
102− // Stop the metrics shutdown go routine
103− close(stop)
104− // Stop the oauthproxy server, we have encounter an unexpected error
105− close(oauthProxyStop)
106− }
107− }()
108−
109− go func() {
110− <-stop
111− if err := s.Shutdown(context.Background()); err != nil {
112− logger.Print(err)
113− }
114− }()
115− }
116−
117− return stop
65+ if err := oauthproxy.Start(); err != nil {
66+ logger.Fatalf("ERROR: Failed to start OAuth2 Proxy: %v", err)
67+ }
11868}
11969
12070// loadConfiguration will load in the user's configuration.
main_test.go+4−3
@@ -15,6 +15,7 @@ import (
1515
1616var _ = Describe("Configuration Loading Suite", func() {
1717 const testLegacyConfig = `
18+http_address="127.0.0.1:4180"
1819upstreams="http://httpbin"
1920set_basic_auth="true"
2021basic_auth_password="super-secret-password"
@@ -54,10 +55,11 @@ injectResponseHeaders:
5455 prefix: "Basic "
5556 basicAuthPassword:
5657 value: c3VwZXItc2VjcmV0LXBhc3N3b3Jk
58+server:
59+ bindAddress: "127.0.0.1:4180"
5760`
5861
5962 const testCoreConfig = `
60−http_address="0.0.0.0:4180"
6163cookie_secret="OQINaROshtE9TcZkNAm-5Zs2Pv3xaWytBmc5W7sPX7w="
6264provider="oidc"
6365email_domains="example.com"
@@ -82,7 +84,6 @@ redirect_url="http://localhost:4180/oauth2/callback"
8284 opts, err := options.NewLegacyOptions().ToOptions()
8385 Expect(err).ToNot(HaveOccurred())
8486
85− opts.HTTPAddress = "0.0.0.0:4180"
8687 opts.Cookie.Secret = "OQINaROshtE9TcZkNAm-5Zs2Pv3xaWytBmc5W7sPX7w="
8788 opts.ProviderType = "oidc"
8889 opts.EmailDomains = []string{"example.com"}
@@ -203,7 +204,7 @@ redirect_url="http://localhost:4180/oauth2/callback"
203204 configContent: testCoreConfig,
204205 alphaConfigContent: testAlphaConfig + ":",
205206 expectedOptions: func() *options.Options { return nil },
206− expectedErr: errors.New("failed to load alpha options: error unmarshalling config: error converting YAML to JSON: yaml: line 34: did not find expected key"),
207+ expectedErr: errors.New("failed to load alpha options: error unmarshalling config: error converting YAML to JSON: yaml: line 36: did not find expected key"),
207208 }),
208209 Entry("with alpha configuration and bad core configuration", loadConfigurationTableInput{
209210 configContent: testCoreConfig + "unknown_field=\"something\"",
oauthproxy.go+62−4
@@ -8,8 +8,11 @@ import (
88 "net"
99 "net/http"
1010 "net/url"
11+ "os"
12+ "os/signal"
1113 "regexp"
1214 "strings"
15+ "syscall"
1316 "time"
1417
1518 "github.com/justinas/alice"
@@ -21,6 +24,7 @@ import (
2124 "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/authentication/basic"
2225 "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/cookies"
2326 "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/encryption"
27+ proxyhttp "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/http"
2428 "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/ip"
2529 "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/logger"
2630 "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/middleware"
@@ -102,6 +106,7 @@ type OAuthProxy struct {
102106 headersChain alice.Chain
103107 preAuthChain alice.Chain
104108 pageWriter pagewriter.Writer
109+ server proxyhttp.Server
105110}
106111
107112// NewOAuthProxy creates a new instance of OAuthProxy from the options provided
@@ -184,7 +189,7 @@ func NewOAuthProxy(opts *options.Options, validator func(string) bool) (*OAuthPr
184189 return nil, fmt.Errorf("could not build headers chain: %v", err)
185190 }
186191
187− return &OAuthProxy{
192+ p := &OAuthProxy{
188193 CookieName: opts.Cookie.Name,
189194 CSRFCookieName: fmt.Sprintf("%v_%v", opts.Cookie.Name, "csrf"),
190195 CookieSeed: opts.Cookie.Secret,
@@ -223,7 +228,60 @@ func NewOAuthProxy(opts *options.Options, validator func(string) bool) (*OAuthPr
223228 headersChain: headersChain,
224229 preAuthChain: preAuthChain,
225230 pageWriter: pageWriter,
226− }, nil
231+ }
232+
233+ if err := p.setupServer(opts); err != nil {
234+ return nil, fmt.Errorf("error setting up server: %v", err)
235+ }
236+
237+ return p, nil
238+}
239+
240+func (p *OAuthProxy) Start() error {
241+ if p.server == nil {
242+ // We have to call setupServer before Start is called.
243+ // If this doesn't happen it's a programming error.
244+ panic("server has not been initialised")
245+ }
246+
247+ ctx, cancel := context.WithCancel(context.Background())
248+
249+ // Observe signals in background goroutine.
250+ go func() {
251+ sigint := make(chan os.Signal, 1)
252+ signal.Notify(sigint, os.Interrupt, syscall.SIGTERM)
253+ <-sigint
254+ cancel() // cancel the context
255+ }()
256+
257+ return p.server.Start(ctx)
258+}
259+
260+func (p *OAuthProxy) setupServer(opts *options.Options) error {
261+ serverOpts := proxyhttp.Opts{
262+ Handler: p,
263+ BindAddress: opts.Server.BindAddress,
264+ SecureBindAddress: opts.Server.SecureBindAddress,
265+ TLS: opts.Server.TLS,
266+ }
267+
268+ appServer, err := proxyhttp.NewServer(serverOpts)
269+ if err != nil {
270+ return fmt.Errorf("could not build app server: %v", err)
271+ }
272+
273+ metricsServer, err := proxyhttp.NewServer(proxyhttp.Opts{
274+ Handler: middleware.DefaultMetricsHandler,
275+ BindAddress: opts.MetricsServer.BindAddress,
276+ SecureBindAddress: opts.MetricsServer.BindAddress,
277+ TLS: opts.MetricsServer.TLS,
278+ })
279+ if err != nil {
280+ return fmt.Errorf("could not build metrics server: %v", err)
281+ }
282+
283+ p.server = proxyhttp.NewServerGroup(appServer, metricsServer)
284+ return nil
227285}
228286
229287// buildPreAuthChain constructs a chain that should process every request before
@@ -233,9 +291,9 @@ func buildPreAuthChain(opts *options.Options) (alice.Chain, error) {
233291 chain := alice.New(middleware.NewScope(opts.ReverseProxy))
234292
235293 if opts.ForceHTTPS {
236− _, httpsPort, err := net.SplitHostPort(opts.HTTPSAddress)
294+ _, httpsPort, err := net.SplitHostPort(opts.Server.SecureBindAddress)
237295 if err != nil {
238− return alice.Chain{}, fmt.Errorf("invalid HTTPS address %q: %v", opts.HTTPAddress, err)
296+ return alice.Chain{}, fmt.Errorf("invalid HTTPS address %q: %v", opts.Server.SecureBindAddress, err)
239297 }
240298 chain = chain.Append(middleware.NewRedirectToHTTPS(httpsPort))
241299 }
oauthproxy_test.go+1
@@ -2341,6 +2341,7 @@ func baseTestOptions() *options.Options {
23412341 },
23422342 },
23432343 }
2344+
23442345 return opts
23452346}
23462347
pkg/apis/options/alpha_options.go+16
@@ -28,6 +28,18 @@ type AlphaOptions struct {
2828 // Headers may source values from either the authenticated user's session
2929 // or from a static secret value.
3030 InjectResponseHeaders []Header `json:"injectResponseHeaders,omitempty"`
31+
32+ // Server is used to configure the HTTP(S) server for the proxy application.
33+ // You may choose to run both HTTP and HTTPS servers simultaneously.
34+ // This can be done by setting the BindAddress and the SecureBindAddress simultaneously.
35+ // To use the secure server you must configure a TLS certificate and key.
36+ Server Server `json:"server,omitempty"`
37+
38+ // MetricsServer is used to configure the HTTP(S) server for metrics.
39+ // You may choose to run both HTTP and HTTPS servers simultaneously.
40+ // This can be done by setting the BindAddress and the SecureBindAddress simultaneously.
41+ // To use the secure server you must configure a TLS certificate and key.
42+ MetricsServer Server `json:"metricsServer,omitempty"`
3143}
3244
3345// MergeInto replaces alpha options in the Options struct with the values
@@ -36,6 +48,8 @@ func (a *AlphaOptions) MergeInto(opts *Options) {
3648 opts.UpstreamServers = a.Upstreams
3749 opts.InjectRequestHeaders = a.InjectRequestHeaders
3850 opts.InjectResponseHeaders = a.InjectResponseHeaders
51+ opts.Server = a.Server
52+ opts.MetricsServer = a.MetricsServer
3953}
4054
4155// ExtractFrom populates the fields in the AlphaOptions with the values from
@@ -44,4 +58,6 @@ func (a *AlphaOptions) ExtractFrom(opts *Options) {
4458 a.Upstreams = opts.UpstreamServers
4559 a.InjectRequestHeaders = opts.InjectRequestHeaders
4660 a.InjectResponseHeaders = opts.InjectResponseHeaders
61+ a.Server = opts.Server
62+ a.MetricsServer = opts.MetricsServer
4763}
pkg/apis/options/legacy_options.go+77
@@ -18,6 +18,9 @@ type LegacyOptions struct {
1818 // Legacy options for injecting request/response headers
1919 LegacyHeaders LegacyHeaders `cfg:",squash"`
2020
21+ // Legacy options for the server address and TLS
22+ LegacyServer LegacyServer `cfg:",squash"`
23+
2124 Options Options `cfg:",squash"`
2225}
2326
@@ -35,6 +38,11 @@ func NewLegacyOptions() *LegacyOptions {
3538 SkipAuthStripHeaders: true,
3639 },
3740
41+ LegacyServer: LegacyServer{
42+ HTTPAddress: "127.0.0.1:4180",
43+ HTTPSAddress: ":443",
44+ },
45+
3846 Options: *NewOptions(),
3947 }
4048}
@@ -44,6 +52,7 @@ func NewLegacyFlagSet() *pflag.FlagSet {
4452
4553 flagSet.AddFlagSet(legacyUpstreamsFlagSet())
4654 flagSet.AddFlagSet(legacyHeadersFlagSet())
55+ flagSet.AddFlagSet(legacyServerFlagset())
4756
4857 return flagSet
4958}
@@ -56,6 +65,8 @@ func (l *LegacyOptions) ToOptions() (*Options, error) {
5665 l.Options.UpstreamServers = upstreams
5766
5867 l.Options.InjectRequestHeaders, l.Options.InjectResponseHeaders = l.LegacyHeaders.convert()
68+ l.Options.Server, l.Options.MetricsServer = l.LegacyServer.convert()
69+
5970 return &l.Options, nil
6071}
6172
@@ -403,3 +414,69 @@ func getXAuthRequestAccessTokenHeader() Header {
403414 },
404415 }
405416}
417+
418+type LegacyServer struct {
419+ MetricsAddress string `flag:"metrics-address" cfg:"metrics_address"`
420+ MetricsSecureAddress string `flag:"metrics-secure-address" cfg:"metrics_address"`
421+ MetricsTLSCertFile string `flag:"metrics-tls-cert-file" cfg:"tls_cert_file"`
422+ MetricsTLSKeyFile string `flag:"metrics-tls-key-file" cfg:"tls_key_file"`
423+ HTTPAddress string `flag:"http-address" cfg:"http_address"`
424+ HTTPSAddress string `flag:"https-address" cfg:"https_address"`
425+ TLSCertFile string `flag:"tls-cert-file" cfg:"tls_cert_file"`
426+ TLSKeyFile string `flag:"tls-key-file" cfg:"tls_key_file"`
427+}
428+
429+func legacyServerFlagset() *pflag.FlagSet {
430+ flagSet := pflag.NewFlagSet("server", pflag.ExitOnError)
431+
432+ flagSet.String("metrics-address", "", "the address /metrics will be served on (e.g. \":9100\")")
433+ flagSet.String("metrics-secure-address", "", "the address /metrics will be served on for HTTPS clients (e.g. \":9100\")")
434+ flagSet.String("metrics-tls-cert-file", "", "path to certificate file for secure metrics server")
435+ flagSet.String("metrics-tls-key-file", "", "path to private key file for secure metrics server")
436+ flagSet.String("http-address", "127.0.0.1:4180", "[http://]<addr>:<port> or unix://<path> to listen on for HTTP clients")
437+ flagSet.String("https-address", ":443", "<addr>:<port> to listen on for HTTPS clients")
438+ flagSet.String("tls-cert-file", "", "path to certificate file")
439+ flagSet.String("tls-key-file", "", "path to private key file")
440+
441+ return flagSet
442+}
443+
444+func (l LegacyServer) convert() (Server, Server) {
445+ appServer := Server{
446+ BindAddress: l.HTTPAddress,
447+ SecureBindAddress: l.HTTPSAddress,
448+ }
449+ if l.TLSKeyFile != "" || l.TLSCertFile != "" {
450+ appServer.TLS = &TLS{
451+ Key: &SecretSource{
452+ FromFile: l.TLSKeyFile,
453+ },
454+ Cert: &SecretSource{
455+ FromFile: l.TLSCertFile,
456+ },
457+ }
458+ // Preserve backwards compatibility, only run one server
459+ appServer.BindAddress = ""
460+ } else {
461+ // Disable the HTTPS server if there's no certificates.
462+ // This preserves backwards compatibility.
463+ appServer.SecureBindAddress = ""
464+ }
465+
466+ metricsServer := Server{
467+ BindAddress: l.MetricsAddress,
468+ SecureBindAddress: l.MetricsSecureAddress,
469+ }
470+ if l.MetricsTLSKeyFile != "" || l.MetricsTLSCertFile != "" {
471+ metricsServer.TLS = &TLS{
472+ Key: &SecretSource{
473+ FromFile: l.MetricsTLSKeyFile,
474+ },
475+ Cert: &SecretSource{
476+ FromFile: l.MetricsTLSCertFile,
477+ },
478+ }
479+ }
480+
481+ return appServer, metricsServer
482+}
pkg/apis/options/legacy_options_test.go+93
@@ -106,6 +106,10 @@ var _ = Describe("Legacy Options", func() {
106106
107107 opts.InjectResponseHeaders = []Header{}
108108
109+ opts.Server = Server{
110+ BindAddress: "127.0.0.1:4180",
111+ }
112+
109113 converted, err := legacyOpts.ToOptions()
110114 Expect(err).ToNot(HaveOccurred())
111115 Expect(converted).To(Equal(opts))
@@ -759,4 +763,93 @@ var _ = Describe("Legacy Options", func() {
759763 }),
760764 )
761765 })
766+
767+ Context("Legacy Servers", func() {
768+ type legacyServersTableInput struct {
769+ legacyServer LegacyServer
770+ expectedAppServer Server
771+ expectedMetricsServer Server
772+ }
773+
774+ const (
775+ insecureAddr = "127.0.0.1:8080"
776+ insecureMetricsAddr = ":9090"
777+ secureAddr = ":443"
778+ secureMetricsAddr = ":9443"
779+ crtPath = "tls.crt"
780+ keyPath = "tls.key"
781+ )
782+
783+ var tlsConfig = &TLS{
784+ Cert: &SecretSource{
785+ FromFile: crtPath,
786+ },
787+ Key: &SecretSource{
788+ FromFile: keyPath,
789+ },
790+ }
791+
792+ DescribeTable("should convert to app and metrics servers",
793+ func(in legacyServersTableInput) {
794+ appServer, metricsServer := in.legacyServer.convert()
795+ Expect(appServer).To(Equal(in.expectedAppServer))
796+ Expect(metricsServer).To(Equal(in.expectedMetricsServer))
797+ },
798+ Entry("with default options only starts app HTTP server", legacyServersTableInput{
799+ legacyServer: LegacyServer{
800+ HTTPAddress: insecureAddr,
801+ HTTPSAddress: secureAddr,
802+ },
803+ expectedAppServer: Server{
804+ BindAddress: insecureAddr,
805+ },
806+ }),
807+ Entry("with TLS options specified only starts app HTTPS server", legacyServersTableInput{
808+ legacyServer: LegacyServer{
809+ HTTPAddress: insecureAddr,
810+ HTTPSAddress: secureAddr,
811+ TLSKeyFile: keyPath,
812+ TLSCertFile: crtPath,
813+ },
814+ expectedAppServer: Server{
815+ SecureBindAddress: secureAddr,
816+ TLS: tlsConfig,
817+ },
818+ }),
819+ Entry("with metrics HTTP and HTTPS addresses", legacyServersTableInput{
820+ legacyServer: LegacyServer{
821+ HTTPAddress: insecureAddr,
822+ HTTPSAddress: secureAddr,
823+ MetricsAddress: insecureMetricsAddr,
824+ MetricsSecureAddress: secureMetricsAddr,
825+ },
826+ expectedAppServer: Server{
827+ BindAddress: insecureAddr,
828+ },
829+ expectedMetricsServer: Server{
830+ BindAddress: insecureMetricsAddr,
831+ SecureBindAddress: secureMetricsAddr,
832+ },
833+ }),
834+ Entry("with metrics HTTPS and tls cert/key", legacyServersTableInput{
835+ legacyServer: LegacyServer{
836+ HTTPAddress: insecureAddr,
837+ HTTPSAddress: secureAddr,
838+ MetricsAddress: insecureMetricsAddr,
839+ MetricsSecureAddress: secureMetricsAddr,
840+ MetricsTLSKeyFile: keyPath,
841+ MetricsTLSCertFile: crtPath,
842+ },
843+ expectedAppServer: Server{
844+ BindAddress: insecureAddr,
845+ },
846+ expectedMetricsServer: Server{
847+ BindAddress: insecureMetricsAddr,
848+ SecureBindAddress: secureMetricsAddr,
849+ TLS: tlsConfig,
850+ },
851+ }),
852+ )
853+
854+ })
762855})
pkg/apis/options/options.go+3−13
@@ -22,9 +22,6 @@ type Options struct {
2222 ProxyPrefix string `flag:"proxy-prefix" cfg:"proxy_prefix"`
2323 PingPath string `flag:"ping-path" cfg:"ping_path"`
2424 PingUserAgent string `flag:"ping-user-agent" cfg:"ping_user_agent"`
25− MetricsAddress string `flag:"metrics-address" cfg:"metrics_address"`
26− HTTPAddress string `flag:"http-address" cfg:"http_address"`
27− HTTPSAddress string `flag:"https-address" cfg:"https_address"`
2825 ReverseProxy bool `flag:"reverse-proxy" cfg:"reverse_proxy"`
2926 RealClientIPHeader string `flag:"real-client-ip-header" cfg:"real_client_ip_header"`
3027 TrustedIPs []string `flag:"trusted-ip" cfg:"trusted_ips"`
@@ -33,8 +30,6 @@ type Options struct {
3330 ClientID string `flag:"client-id" cfg:"client_id"`
3431 ClientSecret string `flag:"client-secret" cfg:"client_secret"`
3532 ClientSecretFile string `flag:"client-secret-file" cfg:"client_secret_file"`
36− TLSCertFile string `flag:"tls-cert-file" cfg:"tls_cert_file"`
37− TLSKeyFile string `flag:"tls-key-file" cfg:"tls_key_file"`
3833
3934 AuthenticatedEmailsFile string `flag:"authenticated-emails-file" cfg:"authenticated_emails_file"`
4035 KeycloakGroups []string `flag:"keycloak-group" cfg:"keycloak_groups"`
@@ -68,6 +63,9 @@ type Options struct {
6863 InjectRequestHeaders []Header `cfg:",internal"`
6964 InjectResponseHeaders []Header `cfg:",internal"`
7065
66+ Server Server `cfg:",internal"`
67+ MetricsServer Server `cfg:",internal"`
68+
7169 SkipAuthRegex []string `flag:"skip-auth-regex" cfg:"skip_auth_regex"`
7270 SkipAuthRoutes []string `flag:"skip-auth-route" cfg:"skip_auth_routes"`
7371 SkipJwtBearerTokens bool `flag:"skip-jwt-bearer-tokens" cfg:"skip_jwt_bearer_tokens"`
@@ -136,10 +134,7 @@ func NewOptions() *Options {
136134 return &Options{
137135 ProxyPrefix: "/oauth2",
138136 ProviderType: "google",
139− MetricsAddress: "",
140137 PingPath: "/ping",
141− HTTPAddress: "127.0.0.1:4180",
142− HTTPSAddress: ":443",
143138 RealClientIPHeader: "X-Real-IP",
144139 ForceHTTPS: false,
145140 Cookie: cookieDefaults(),
@@ -162,14 +157,10 @@ func NewOptions() *Options {
162157func NewFlagSet() *pflag.FlagSet {
163158 flagSet := pflag.NewFlagSet("oauth2-proxy", pflag.ExitOnError)
164159
165− flagSet.String("http-address", "127.0.0.1:4180", "[http://]<addr>:<port> or unix://<path> to listen on for HTTP clients")
166− flagSet.String("https-address", ":443", "<addr>:<port> to listen on for HTTPS clients")
167160 flagSet.Bool("reverse-proxy", false, "are we running behind a reverse proxy, controls whether headers like X-Real-Ip are accepted")
168161 flagSet.String("real-client-ip-header", "X-Real-IP", "Header used to determine the real IP of the client (one of: X-Forwarded-For, X-Real-IP, or X-ProxyUser-IP)")
169162 flagSet.StringSlice("trusted-ip", []string{}, "list of IPs or CIDR ranges to allow to bypass authentication. WARNING: trusting by IP has inherent security flaws, read the configuration documentation for more information.")
170163 flagSet.Bool("force-https", false, "force HTTPS redirect for HTTP requests")
171− flagSet.String("tls-cert-file", "", "path to certificate file")
172− flagSet.String("tls-key-file", "", "path to private key file")
173164 flagSet.String("redirect-url", "", "the OAuth Redirect URL. ie: \"https://internalapp.yourcompany.com/oauth2/callback\"")
174165 flagSet.StringSlice("skip-auth-regex", []string{}, "(DEPRECATED for --skip-auth-route) bypass authentication for requests path's that match (may be given multiple times)")
175166 flagSet.StringSlice("skip-auth-route", []string{}, "bypass authentication for requests that match the method & path. Format: method=path_regex OR path_regex alone for all methods")
@@ -204,7 +195,6 @@ func NewFlagSet() *pflag.FlagSet {
204195 flagSet.String("proxy-prefix", "/oauth2", "the url root path that this proxy should be nested under (e.g. /<oauth2>/sign_in)")
205196 flagSet.String("ping-path", "/ping", "the ping endpoint that can be used for basic health checks")
206197 flagSet.String("ping-user-agent", "", "special User-Agent that will be used for basic health checks")
207− flagSet.String("metrics-address", "", "the address /metrics will be served on (e.g. \":9100\")")
208198 flagSet.String("session-store-type", "cookie", "the session storage provider to use")
209199 flagSet.Bool("session-cookie-minimal", false, "strip OAuth tokens from cookie session stores if they aren't needed (cookie session store only)")
210200 flagSet.String("redis-connection-url", "", "URL of redis server for redis session storage (eg: redis://HOST[:PORT])")
pkg/apis/options/server.gonew file+27
@@ -0,0 +1,27 @@
1+package options
2+
3+// Server represents the configuration for an HTTP(S) server
4+type Server struct {
5+ // BindAddress is the the address on which to serve traffic.
6+ // Leave blank or set to "-" to disable.
7+ BindAddress string
8+
9+ // SecureBindAddress is the the address on which to serve secure traffic.
10+ // Leave blank or set to "-" to disable.
11+ SecureBindAddress string
12+
13+ // TLS contains the information for loading the certificate and key for the
14+ // secure traffic.
15+ TLS *TLS
16+}
17+
18+// TLS contains the information for loading a TLS certifcate and key.
19+type TLS struct {
20+ // Key is the the TLS key data to use.
21+ // Typically this will come from a file.
22+ Key *SecretSource
23+
24+ // Cert is the TLS certificate data to use.
25+ // Typically this will come from a file.
26+ Cert *SecretSource
27+}
pkg/http/http_suite_test.gonew file+88
@@ -0,0 +1,88 @@
1+package http
2+
3+import (
4+ "bytes"
5+ "crypto/rand"
6+ "crypto/rsa"
7+ "crypto/tls"
8+ "crypto/x509"
9+ "crypto/x509/pkix"
10+ "encoding/pem"
11+ "math/big"
12+ "net"
13+ "net/http"
14+ "testing"
15+ "time"
16+
17+ "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/apis/options"
18+ "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/logger"
19+ . "github.com/onsi/ginkgo"
20+ . "github.com/onsi/gomega"
21+)
22+
23+var certData []byte
24+var certDataSource, keyDataSource options.SecretSource
25+var client *http.Client
26+
27+func TestHTTPSuite(t *testing.T) {
28+ logger.SetOutput(GinkgoWriter)
29+ logger.SetErrOutput(GinkgoWriter)
30+
31+ RegisterFailHandler(Fail)
32+ RunSpecs(t, "HTTP")
33+}
34+
35+var _ = BeforeSuite(func() {
36+ By("Generating a self-signed cert for TLS tests", func() {
37+ priv, err := rsa.GenerateKey(rand.Reader, 2048)
38+ Expect(err).ToNot(HaveOccurred())
39+
40+ keyOut := bytes.NewBuffer(nil)
41+ privBytes, err := x509.MarshalPKCS8PrivateKey(priv)
42+ Expect(err).ToNot(HaveOccurred())
43+ Expect(pem.Encode(keyOut, &pem.Block{Type: "PRIVATE KEY", Bytes: privBytes})).To(Succeed())
44+ keyDataSource.Value = keyOut.Bytes()
45+
46+ serialNumberLimit := new(big.Int).Lsh(big.NewInt(1), 128)
47+ serialNumber, err := rand.Int(rand.Reader, serialNumberLimit)
48+ Expect(err).ToNot(HaveOccurred())
49+
50+ template := x509.Certificate{
51+ SerialNumber: serialNumber,
52+ Subject: pkix.Name{
53+ Organization: []string{"OAuth2 Proxy Test Suite"},
54+ },
55+ NotBefore: time.Now(),
56+ NotAfter: time.Now().Add(time.Hour),
57+ IPAddresses: []net.IP{net.ParseIP("127.0.0.1")},
58+ KeyUsage: x509.KeyUsageDigitalSignature | x509.KeyUsageKeyEncipherment,
59+ ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
60+ }
61+
62+ certBytes, err := x509.CreateCertificate(rand.Reader, &template, &template, &priv.PublicKey, priv)
63+ Expect(err).ToNot(HaveOccurred())
64+ certData = certBytes
65+
66+ certOut := bytes.NewBuffer(nil)
67+ Expect(pem.Encode(certOut, &pem.Block{Type: "CERTIFICATE", Bytes: certBytes})).To(Succeed())
68+ certDataSource.Value = certOut.Bytes()
69+ })
70+
71+ By("Setting up a http client", func() {
72+ cert, err := tls.X509KeyPair(certDataSource.Value, keyDataSource.Value)
73+ Expect(err).ToNot(HaveOccurred())
74+
75+ certificate, err := x509.ParseCertificate(cert.Certificate[0])
76+ Expect(err).ToNot(HaveOccurred())
77+
78+ certpool := x509.NewCertPool()
79+ certpool.AddCert(certificate)
80+
81+ transport := http.DefaultTransport.(*http.Transport).Clone()
82+ transport.TLSClientConfig.RootCAs = certpool
83+
84+ client = &http.Client{
85+ Transport: transport,
86+ }
87+ })
88+})
pkg/http/server.gonew file+245
@@ -0,0 +1,245 @@
1+package http
2+
3+import (
4+ "context"
5+ "crypto/tls"
6+ "errors"
7+ "fmt"
8+ "net"
9+ "net/http"
10+ "strings"
11+ "time"
12+
13+ "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/apis/options"
14+ "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/apis/options/util"
15+ "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/logger"
16+ "golang.org/x/sync/errgroup"
17+)
18+
19+// Server represents an HTTP or HTTPS server.
20+type Server interface {
21+ // Start blocks and runs the server.
22+ Start(ctx context.Context) error
23+}
24+
25+// Opts contains the information required to set up the server.
26+type Opts struct {
27+ // Handler is the http.Handler to be used to serve http pages by the server.
28+ Handler http.Handler
29+
30+ // BindAddress is the address the HTTP server should listen on.
31+ BindAddress string
32+
33+ // SecureBindAddress is the address the HTTPS server should listen on.
34+ SecureBindAddress string
35+
36+ // TLS is the TLS configuration for the server.
37+ TLS *options.TLS
38+}
39+
40+// NewServer creates a new Server from the options given.
41+func NewServer(opts Opts) (Server, error) {
42+ s := &server{
43+ handler: opts.Handler,
44+ }
45+ if err := s.setupListener(opts); err != nil {
46+ return nil, fmt.Errorf("error setting up listener: %v", err)
47+ }
48+ if err := s.setupTLSListener(opts); err != nil {
49+ return nil, fmt.Errorf("error setting up TLS listener: %v", err)
50+ }
51+
52+ return s, nil
53+}
54+
55+// server is an implementation of the Server interface.
56+type server struct {
57+ handler http.Handler
58+
59+ listener net.Listener
60+ tlsListener net.Listener
61+}
62+
63+// setupListener sets the server listener if the HTTP server is enabled.
64+// The HTTP server can be disabled by setting the BindAddress to "-" or by
65+// leaving it empty.
66+func (s *server) setupListener(opts Opts) error {
67+ if opts.BindAddress == "" || opts.BindAddress == "-" {
68+ // No HTTP listener required
69+ return nil
70+ }
71+
72+ networkType := getNetworkScheme(opts.BindAddress)
73+ listenAddr := getListenAddress(opts.BindAddress)
74+
75+ listener, err := net.Listen(networkType, listenAddr)
76+ if err != nil {
77+ return fmt.Errorf("listen (%s, %s) failed: %v", networkType, listenAddr, err)
78+ }
79+ s.listener = listener
80+
81+ return nil
82+}
83+
84+// setupTLSListener sets the server TLS listener if the HTTPS server is enabled.
85+// The HTTPS server can be disabled by setting the SecureBindAddress to "-" or by
86+// leaving it empty.
87+func (s *server) setupTLSListener(opts Opts) error {
88+ if opts.SecureBindAddress == "" || opts.SecureBindAddress == "-" {
89+ // No HTTPS listener required
90+ return nil
91+ }
92+
93+ config := &tls.Config{
94+ MinVersion: tls.VersionTLS12,
95+ MaxVersion: tls.VersionTLS13,
96+ NextProtos: []string{"http/1.1"},
97+ }
98+ if opts.TLS == nil {
99+ return errors.New("no TLS config provided")
100+ }
101+ cert, err := getCertificate(opts.TLS)
102+ if err != nil {
103+ return fmt.Errorf("could not load certificate: %v", err)
104+ }
105+ config.Certificates = []tls.Certificate{cert}
106+
107+ listenAddr := getListenAddress(opts.SecureBindAddress)
108+
109+ listener, err := net.Listen("tcp", listenAddr)
110+ if err != nil {
111+ return fmt.Errorf("listen (%s) failed: %v", listenAddr, err)
112+ }
113+
114+ s.tlsListener = tls.NewListener(tcpKeepAliveListener{listener.(*net.TCPListener)}, config)
115+ return nil
116+}
117+
118+// Start starts the HTTP and HTTPS server if applicable.
119+// It will block until the context is cancelled.
120+// If any errors occur, only the first error will be returned.
121+func (s *server) Start(ctx context.Context) error {
122+ g, groupCtx := errgroup.WithContext(ctx)
123+
124+ if s.listener != nil {
125+ g.Go(func() error {
126+ if err := s.startServer(groupCtx, s.listener); err != nil {
127+ return fmt.Errorf("error starting insecure server: %v", err)
128+ }
129+ return nil
130+ })
131+ }
132+
133+ if s.tlsListener != nil {
134+ g.Go(func() error {
135+ if err := s.startServer(groupCtx, s.tlsListener); err != nil {
136+ return fmt.Errorf("error starting secure server: %v", err)
137+ }
138+ return nil
139+ })
140+ }
141+
142+ return g.Wait()
143+}
144+
145+// startServer creates and starts a new server with the given listener.
146+// When the given context is cancelled the server will be shutdown.
147+// If any errors occur, only the first error will be returned.
148+func (s *server) startServer(ctx context.Context, listener net.Listener) error {
149+ srv := &http.Server{Handler: s.handler}
150+ g, groupCtx := errgroup.WithContext(ctx)
151+
152+ g.Go(func() error {
153+ <-groupCtx.Done()
154+
155+ if err := srv.Shutdown(context.Background()); err != nil {
156+ return fmt.Errorf("error shutting down server: %v", err)
157+ }
158+ return nil
159+ })
160+
161+ g.Go(func() error {
162+ if err := srv.Serve(listener); err != nil && !errors.Is(err, http.ErrServerClosed) {
163+ return fmt.Errorf("could not start server: %v", err)
164+ }
165+ return nil
166+ })
167+
168+ return g.Wait()
169+}
170+
171+// getNetworkScheme gets the scheme for the HTTP server.
172+func getNetworkScheme(addr string) string {
173+ var scheme string
174+ i := strings.Index(addr, "://")
175+ if i > -1 {
176+ scheme = addr[0:i]
177+ }
178+
179+ switch scheme {
180+ case "", "http":
181+ return "tcp"
182+ default:
183+ return scheme
184+ }
185+}
186+
187+// getListenAddress gets the address for the HTTP server.
188+func getListenAddress(addr string) string {
189+ slice := strings.SplitN(addr, "//", 2)
190+ return slice[len(slice)-1]
191+}
192+
193+// getCertificate loads the certificate data from the TLS config.
194+func getCertificate(opts *options.TLS) (tls.Certificate, error) {
195+ keyData, err := getSecretValue(opts.Key)
196+ if err != nil {
197+ return tls.Certificate{}, fmt.Errorf("could not load key data: %v", err)
198+ }
199+
200+ certData, err := getSecretValue(opts.Cert)
201+ if err != nil {
202+ return tls.Certificate{}, fmt.Errorf("could not load cert data: %v", err)
203+ }
204+
205+ cert, err := tls.X509KeyPair(certData, keyData)
206+ if err != nil {
207+ return tls.Certificate{}, fmt.Errorf("could not parse certificate data: %v", err)
208+ }
209+
210+ return cert, nil
211+}
212+
213+// getSecretValue wraps util.GetSecretValue so that we can return an error if no
214+// source is provided.
215+func getSecretValue(src *options.SecretSource) ([]byte, error) {
216+ if src == nil {
217+ return nil, errors.New("no configuration provided")
218+ }
219+ return util.GetSecretValue(src)
220+}
221+
222+// tcpKeepAliveListener sets TCP keep-alive timeouts on accepted
223+// connections. It's used by so that dead TCP connections (e.g. closing laptop
224+// mid-download) eventually go away.
225+type tcpKeepAliveListener struct {
226+ *net.TCPListener
227+}
228+
229+// Accept implements the TCPListener interface.
230+// It sets the keep alive period to 3 minutes for each connection.
231+func (ln tcpKeepAliveListener) Accept() (net.Conn, error) {
232+ tc, err := ln.AcceptTCP()
233+ if err != nil {
234+ return nil, err
235+ }
236+ err = tc.SetKeepAlive(true)
237+ if err != nil {
238+ logger.Errorf("Error setting Keep-Alive: %v", err)
239+ }
240+ err = tc.SetKeepAlivePeriod(3 * time.Minute)
241+ if err != nil {
242+ logger.Printf("Error setting Keep-Alive period: %v", err)
243+ }
244+ return tc, nil
245+}
pkg/http/server_group.gonew file+35
@@ -0,0 +1,35 @@
1+package http
2+
3+import (
4+ "context"
5+
6+ "golang.org/x/sync/errgroup"
7+)
8+
9+// NewServerGroup creates a new Server to start and gracefully stop a collection
10+// of Servers.
11+func NewServerGroup(servers ...Server) Server {
12+ return &serverGroup{
13+ servers: servers,
14+ }
15+}
16+
17+// serverGroup manages the starting and graceful shutdown of a collection of
18+// servers.
19+type serverGroup struct {
20+ servers []Server
21+}
22+
23+// Start runs the servers in the server group.
24+func (s *serverGroup) Start(ctx context.Context) error {
25+ g, groupCtx := errgroup.WithContext(ctx)
26+
27+ for _, server := range s.servers {
28+ srv := server
29+ g.Go(func() error {
30+ return srv.Start(groupCtx)
31+ })
32+ }
33+
34+ return g.Wait()
35+}
pkg/http/server_group_test.gonew file+102
@@ -0,0 +1,102 @@
1+package http
2+
3+import (
4+ "context"
5+ "errors"
6+
7+ . "github.com/onsi/ginkgo"
8+ . "github.com/onsi/gomega"
9+)
10+
11+var _ = Describe("Server Group", func() {
12+ var m1, m2, m3 *mockServer
13+ var ctx context.Context
14+ var cancel context.CancelFunc
15+ var group Server
16+
17+ BeforeEach(func() {
18+ ctx, cancel = context.WithCancel(context.Background())
19+
20+ m1 = newMockServer()
21+ m2 = newMockServer()
22+ m3 = newMockServer()
23+ group = NewServerGroup(m1, m2, m3)
24+ })
25+
26+ AfterEach(func() {
27+ cancel()
28+ })
29+
30+ It("starts each server in the group", func() {
31+ go func() {
32+ defer GinkgoRecover()
33+ Expect(group.Start(ctx)).To(Succeed())
34+ }()
35+
36+ Eventually(m1.started).Should(BeClosed(), "mock server 1 not started")
37+ Eventually(m2.started).Should(BeClosed(), "mock server 2 not started")
38+ Eventually(m3.started).Should(BeClosed(), "mock server 3 not started")
39+ })
40+
41+ It("stop each server in the group when the context is cancelled", func() {
42+ go func() {
43+ defer GinkgoRecover()
44+ Expect(group.Start(ctx)).To(Succeed())
45+ }()
46+
47+ cancel()
48+ Eventually(m1.stopped).Should(BeClosed(), "mock server 1 not stopped")
49+ Eventually(m2.stopped).Should(BeClosed(), "mock server 2 not stopped")
50+ Eventually(m3.stopped).Should(BeClosed(), "mock server 3 not stopped")
51+ })
52+
53+ It("stop each server in the group when the an error occurs", func() {
54+ err := errors.New("server error")
55+ go func() {
56+ defer GinkgoRecover()
57+ Expect(group.Start(ctx)).To(MatchError(err))
58+ }()
59+
60+ m2.errors <- err
61+ Eventually(m1.stopped).Should(BeClosed(), "mock server 1 not stopped")
62+ Eventually(m2.stopped).Should(BeClosed(), "mock server 2 not stopped")
63+ Eventually(m3.stopped).Should(BeClosed(), "mock server 3 not stopped")
64+ })
65+})
66+
67+// mockServer is used to test the server group can start
68+// and stop multiple servers simultaneously.
69+type mockServer struct {
70+ started chan struct{}
71+ startClosed bool
72+ stopped chan struct{}
73+ stopClosed bool
74+ errors chan error
75+}
76+
77+func newMockServer() *mockServer {
78+ return &mockServer{
79+ started: make(chan struct{}),
80+ stopped: make(chan struct{}),
81+ errors: make(chan error),
82+ }
83+}
84+
85+func (m *mockServer) Start(ctx context.Context) error {
86+ if !m.startClosed {
87+ close(m.started)
88+ m.startClosed = true
89+ }
90+ defer func() {
91+ if !m.stopClosed {
92+ close(m.stopped)
93+ m.stopClosed = true
94+ }
95+ }()
96+ select {
97+ case <-ctx.Done():
98+ return nil
99+ case err := <-m.errors:
100+ return err
101+ }
102+}
pkg/http/server_test.gonew file+472
@@ -0,0 +1,472 @@
1+package http
2+
3+import (
4+ "context"
5+ "errors"
6+ "fmt"
7+ "io/ioutil"
8+ "net/http"
9+
10+ "github.com/oauth2-proxy/oauth2-proxy/v7/pkg/apis/options"
11+ . "github.com/onsi/ginkgo"
12+ . "github.com/onsi/ginkgo/extensions/table"
13+ . "github.com/onsi/gomega"
14+)
15+
16+const hello = "Hello World!"
17+
18+var _ = Describe("Server", func() {
19+ handler := http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
20+ rw.Write([]byte(hello))
21+ })
22+
23+ Context("NewServer", func() {
24+ type newServerTableInput struct {
25+ opts Opts
26+ expectedErr error
27+ expectHTTPListener bool
28+ expectTLSListener bool
29+ }
30+
31+ DescribeTable("When creating the new server from the options", func(in *newServerTableInput) {
32+ srv, err := NewServer(in.opts)
33+ if in.expectedErr != nil {
34+ Expect(err).To(MatchError(ContainSubstring(in.expectedErr.Error())))
35+ Expect(srv).To(BeNil())
36+ return
37+ }
38+
39+ Expect(err).ToNot(HaveOccurred())
40+
41+ s, ok := srv.(*server)
42+ Expect(ok).To(BeTrue())
43+
44+ Expect(s.listener != nil).To(Equal(in.expectHTTPListener))
45+ if in.expectHTTPListener {
46+ Expect(s.listener.Close()).To(Succeed())
47+ }
48+ Expect(s.tlsListener != nil).To(Equal(in.expectTLSListener))
49+ if in.expectTLSListener {
50+ Expect(s.tlsListener.Close()).To(Succeed())
51+ }
52+ },
53+ Entry("with a valid http bind address", &newServerTableInput{
54+ opts: Opts{
55+ Handler: handler,
56+ BindAddress: "127.0.0.1:0",
57+ },
58+ expectedErr: nil,
59+ expectHTTPListener: true,
60+ expectTLSListener: false,
61+ }),
62+ Entry("with a valid https bind address, with no TLS config", &newServerTableInput{
63+ opts: Opts{
64+ Handler: handler,
65+ SecureBindAddress: "127.0.0.1:0",
66+ },
67+ expectedErr: errors.New("error setting up TLS listener: no TLS config provided"),
68+ expectHTTPListener: false,
69+ expectTLSListener: false,
70+ }),
71+ Entry("with a valid https bind address, and valid TLS config", &newServerTableInput{
72+ opts: Opts{
73+ Handler: handler,
74+ SecureBindAddress: "127.0.0.1:0",
75+ TLS: &options.TLS{
76+ Key: &keyDataSource,
77+ Cert: &certDataSource,
78+ },
79+ },
80+ expectedErr: nil,
81+ expectHTTPListener: false,
82+ expectTLSListener: true,
83+ }),
84+ Entry("with a both a valid http and valid https bind address, and valid TLS config", &newServerTableInput{
85+ opts: Opts{
86+ Handler: handler,
87+ BindAddress: "127.0.0.1:0",
88+ SecureBindAddress: "127.0.0.1:0",
89+ TLS: &options.TLS{
90+ Key: &keyDataSource,
91+ Cert: &certDataSource,
92+ },
93+ },
94+ expectedErr: nil,
95+ expectHTTPListener: true,
96+ expectTLSListener: true,
97+ }),
98+ Entry("with a \"-\" for the bind address", &newServerTableInput{
99+ opts: Opts{
100+ Handler: handler,
101+ BindAddress: "-",
102+ },
103+ expectedErr: nil,
104+ expectHTTPListener: false,
105+ expectTLSListener: false,
106+ }),
107+ Entry("with a \"-\" for the secure bind address", &newServerTableInput{
108+ opts: Opts{
109+ Handler: handler,
110+ SecureBindAddress: "-",
111+ },
112+ expectedErr: nil,
113+ expectHTTPListener: false,
114+ expectTLSListener: false,
115+ }),
116+ Entry("with an invalid bind address scheme", &newServerTableInput{
117+ opts: Opts{
118+ Handler: handler,
119+ BindAddress: "invalid://127.0.0.1:0",
120+ },
121+ expectedErr: errors.New("error setting up listener: listen (invalid, 127.0.0.1:0) failed: listen invalid: unknown network invalid"),
122+ expectHTTPListener: false,
123+ expectTLSListener: false,
124+ }),
125+ Entry("with an invalid secure bind address scheme", &newServerTableInput{
126+ opts: Opts{
127+ Handler: handler,
128+ SecureBindAddress: "invalid://127.0.0.1:0",
129+ TLS: &options.TLS{
130+ Key: &keyDataSource,
131+ Cert: &certDataSource,
132+ },
133+ },
134+ expectedErr: nil,
135+ expectHTTPListener: false,
136+ expectTLSListener: true,
137+ }),
138+ Entry("with an invalid bind address port", &newServerTableInput{
139+ opts: Opts{
140+ Handler: handler,
141+ BindAddress: "127.0.0.1:a",
142+ },
143+ expectedErr: errors.New("error setting up listener: listen (tcp, 127.0.0.1:a) failed: listen tcp: lookup tcp/a: "),
144+ expectHTTPListener: false,
145+ expectTLSListener: false,
146+ }),
147+ Entry("with an invalid secure bind address port", &newServerTableInput{
148+ opts: Opts{
149+ Handler: handler,
150+ SecureBindAddress: "127.0.0.1:a",
151+ TLS: &options.TLS{
152+ Key: &keyDataSource,
153+ Cert: &certDataSource,
154+ },
155+ },
156+ expectedErr: errors.New("error setting up TLS listener: listen (127.0.0.1:a) failed: listen tcp: lookup tcp/a: "),
157+ expectHTTPListener: false,
158+ expectTLSListener: false,
159+ }),
160+ Entry("with an invalid TLS key", &newServerTableInput{
161+ opts: Opts{
162+ Handler: handler,
163+ SecureBindAddress: "127.0.0.1:0",
164+ TLS: &options.TLS{
165+ Key: &options.SecretSource{
166+ Value: []byte("invalid"),
167+ },
168+ Cert: &certDataSource,
169+ },
170+ },
171+ expectedErr: errors.New("error setting up TLS listener: could not load certificate: could not parse certificate data: tls: failed to find any PEM data in key input"),
172+ expectHTTPListener: false,
173+ expectTLSListener: false,
174+ }),
175+ Entry("with an invalid TLS cert", &newServerTableInput{
176+ opts: Opts{
177+ Handler: handler,
178+ SecureBindAddress: "127.0.0.1:0",
179+ TLS: &options.TLS{
180+ Key: &keyDataSource,
181+ Cert: &options.SecretSource{
182+ Value: []byte("invalid"),
183+ },
184+ },
185+ },
186+ expectedErr: errors.New("error setting up TLS listener: could not load certificate: could not parse certificate data: tls: failed to find any PEM data in certificate input"),
187+ expectHTTPListener: false,
188+ expectTLSListener: false,
189+ }),
190+ Entry("with no TLS key", &newServerTableInput{
191+ opts: Opts{
192+ Handler: handler,
193+ SecureBindAddress: "127.0.0.1:0",
194+ TLS: &options.TLS{
195+ Cert: &certDataSource,
196+ },
197+ },
198+ expectedErr: errors.New("error setting up TLS listener: could not load certificate: could not load key data: no configuration provided"),
199+ expectHTTPListener: false,
200+ expectTLSListener: false,
201+ }),
202+ Entry("with no TLS cert", &newServerTableInput{
203+ opts: Opts{
204+ Handler: handler,
205+ SecureBindAddress: "127.0.0.1:0",
206+ TLS: &options.TLS{
207+ Key: &keyDataSource,
208+ },
209+ },
210+ expectedErr: errors.New("error setting up TLS listener: could not load certificate: could not load cert data: no configuration provided"),
211+ expectHTTPListener: false,
212+ expectTLSListener: false,
213+ }),
214+ Entry("when the bind address is prefixed with the http scheme", &newServerTableInput{
215+ opts: Opts{
216+ Handler: handler,
217+ BindAddress: "http://127.0.0.1:0",
218+ },
219+ expectedErr: nil,
220+ expectHTTPListener: true,
221+ expectTLSListener: false,
222+ }),
223+ Entry("when the secure bind address is prefixed with the https scheme", &newServerTableInput{
224+ opts: Opts{
225+ Handler: handler,
226+ SecureBindAddress: "https://127.0.0.1:0",
227+ TLS: &options.TLS{
228+ Key: &keyDataSource,
229+ Cert: &certDataSource,
230+ },
231+ },
232+ expectedErr: nil,
233+ expectHTTPListener: false,
234+ expectTLSListener: true,
235+ }),
236+ )
237+ })
238+
239+ Context("Start", func() {
240+ var srv Server
241+ var ctx context.Context
242+ var cancel context.CancelFunc
243+
244+ BeforeEach(func() {
245+ ctx, cancel = context.WithCancel(context.Background())
246+ })
247+
248+ AfterEach(func() {
249+ cancel()
250+ })
251+
252+ Context("with an http server", func() {
253+ var listenAddr string
254+
255+ BeforeEach(func() {
256+ var err error
257+ srv, err = NewServer(Opts{
258+ Handler: handler,
259+ BindAddress: "127.0.0.1:0",
260+ })
261+ Expect(err).ToNot(HaveOccurred())
262+
263+ s, ok := srv.(*server)
264+ Expect(ok).To(BeTrue())
265+
266+ listenAddr = fmt.Sprintf("http://%s/", s.listener.Addr().String())
267+ })
268+
269+ It("Starts the server and serves the handler", func() {
270+ go func() {
271+ defer GinkgoRecover()
272+ Expect(srv.Start(ctx)).To(Succeed())
273+ }()
274+
275+ resp, err := client.Get(listenAddr)
276+ Expect(err).ToNot(HaveOccurred())
277+ Expect(resp.StatusCode).To(Equal(http.StatusOK))
278+
279+ body, err := ioutil.ReadAll(resp.Body)
280+ Expect(err).ToNot(HaveOccurred())
281+ Expect(string(body)).To(Equal(hello))
282+ })
283+
284+ It("Stops the server when the context is cancelled", func() {
285+ go func() {
286+ defer GinkgoRecover()
287+ Expect(srv.Start(ctx)).To(Succeed())
288+ }()
289+
290+ _, err := client.Get(listenAddr)
291+ Expect(err).ToNot(HaveOccurred())
292+
293+ cancel()
294+
295+ Eventually(func() error {
296+ _, err := client.Get(listenAddr)
297+ return err
298+ }).Should(HaveOccurred())
299+ })
300+ })
301+
302+ Context("with an https server", func() {
303+ var secureListenAddr string
304+
305+ BeforeEach(func() {
306+ var err error
307+ srv, err = NewServer(Opts{
308+ Handler: handler,
309+ SecureBindAddress: "127.0.0.1:0",
310+ TLS: &options.TLS{
311+ Key: &keyDataSource,
312+ Cert: &certDataSource,
313+ },
314+ })
315+ Expect(err).ToNot(HaveOccurred())
316+
317+ s, ok := srv.(*server)
318+ Expect(ok).To(BeTrue())
319+
320+ secureListenAddr = fmt.Sprintf("https://%s/", s.tlsListener.Addr().String())
321+ })
322+
323+ It("Starts the server and serves the handler", func() {
324+ go func() {
325+ defer GinkgoRecover()
326+ Expect(srv.Start(ctx)).To(Succeed())
327+ }()
328+
329+ resp, err := client.Get(secureListenAddr)
330+ Expect(err).ToNot(HaveOccurred())
331+ Expect(resp.StatusCode).To(Equal(http.StatusOK))
332+
333+ body, err := ioutil.ReadAll(resp.Body)
334+ Expect(err).ToNot(HaveOccurred())
335+ Expect(string(body)).To(Equal(hello))
336+ })
337+
338+ It("Stops the server when the context is cancelled", func() {
339+ go func() {
340+ defer GinkgoRecover()
341+ Expect(srv.Start(ctx)).To(Succeed())
342+ }()
343+
344+ _, err := client.Get(secureListenAddr)
345+ Expect(err).ToNot(HaveOccurred())
346+
347+ cancel()
348+
349+ Eventually(func() error {
350+ _, err := client.Get(secureListenAddr)
351+ return err
352+ }).Should(HaveOccurred())
353+ })
354+
355+ It("Serves the certificate provided", func() {
356+ go func() {
357+ defer GinkgoRecover()
358+ Expect(srv.Start(ctx)).To(Succeed())
359+ }()
360+
361+ resp, err := client.Get(secureListenAddr)
362+ Expect(err).ToNot(HaveOccurred())
363+ Expect(resp.StatusCode).To(Equal(http.StatusOK))
364+
365+ Expect(resp.TLS.VerifiedChains).Should(HaveLen(1))
366+ Expect(resp.TLS.VerifiedChains[0]).Should(HaveLen(1))
367+ Expect(resp.TLS.VerifiedChains[0][0].Raw).Should(Equal(certData))
368+ })
369+ })
370+
371+ Context("with both an http and an https server", func() {
372+ var listenAddr, secureListenAddr string
373+
374+ BeforeEach(func() {
375+ var err error
376+ srv, err = NewServer(Opts{
377+ Handler: handler,
378+ BindAddress: "127.0.0.1:0",
379+ SecureBindAddress: "127.0.0.1:0",
380+ TLS: &options.TLS{
381+ Key: &keyDataSource,
382+ Cert: &certDataSource,
383+ },
384+ })
385+ Expect(err).ToNot(HaveOccurred())
386+
387+ s, ok := srv.(*server)
388+ Expect(ok).To(BeTrue())
389+
390+ listenAddr = fmt.Sprintf("http://%s/", s.listener.Addr().String())
391+ secureListenAddr = fmt.Sprintf("https://%s/", s.tlsListener.Addr().String())
392+ })
393+
394+ It("Starts the server and serves the handler on http", func() {
395+ go func() {
396+ defer GinkgoRecover()
397+ Expect(srv.Start(ctx)).To(Succeed())
398+ }()
399+
400+ resp, err := client.Get(listenAddr)
401+ Expect(err).ToNot(HaveOccurred())
402+ Expect(resp.StatusCode).To(Equal(http.StatusOK))
403+
404+ body, err := ioutil.ReadAll(resp.Body)
405+ Expect(err).ToNot(HaveOccurred())
406+ Expect(string(body)).To(Equal(hello))
407+ })
408+
409+ It("Starts the server and serves the handler on https", func() {
410+ go func() {
411+ defer GinkgoRecover()
412+ Expect(srv.Start(ctx)).To(Succeed())
413+ }()
414+
415+ resp, err := client.Get(secureListenAddr)
416+ Expect(err).ToNot(HaveOccurred())
417+ Expect(resp.StatusCode).To(Equal(http.StatusOK))
418+
419+ body, err := ioutil.ReadAll(resp.Body)
420+ Expect(err).ToNot(HaveOccurred())
421+ Expect(string(body)).To(Equal(hello))
422+ })
423+
424+ It("Stops both servers when the context is cancelled", func() {
425+ go func() {
426+ defer GinkgoRecover()
427+ Expect(srv.Start(ctx)).To(Succeed())
428+ }()
429+
430+ _, err := client.Get(listenAddr)
431+ Expect(err).ToNot(HaveOccurred())
432+ _, err = client.Get(secureListenAddr)
433+ Expect(err).ToNot(HaveOccurred())
434+
435+ cancel()
436+
437+ Eventually(func() error {
438+ _, err := client.Get(listenAddr)
439+ return err
440+ }).Should(HaveOccurred())
441+ Eventually(func() error {
442+ _, err := client.Get(secureListenAddr)
443+ return err
444+ }).Should(HaveOccurred())
445+ })
446+ })
447+ })
448+
449+ Context("getNetworkScheme", func() {
450+ DescribeTable("should return the scheme", func(in, expected string) {
451+ Expect(getNetworkScheme(in)).To(Equal(expected))
452+ },
453+ Entry("with no scheme", "127.0.0.1:0", "tcp"),
454+ Entry("with a tcp scheme", "tcp://127.0.0.1:0", "tcp"),
455+ Entry("with a http scheme", "http://192.168.0.1:1", "tcp"),
456+ Entry("with a unix scheme", "unix://172.168.16.2:2", "unix"),
457+ Entry("with a random scheme", "random://10.10.10.10:10", "random"),
458+ )
459+ })
460+
461+ Context("getListenAddress", func() {
462+ DescribeTable("should remove the scheme", func(in, expected string) {
463+ Expect(getListenAddress(in)).To(Equal(expected))
464+ },
465+ Entry("with no scheme", "127.0.0.1:0", "127.0.0.1:0"),
466+ Entry("with a tcp scheme", "tcp://127.0.0.1:0", "127.0.0.1:0"),
467+ Entry("with a http scheme", "http://192.168.0.1:1", "192.168.0.1:1"),
468+ Entry("with a unix scheme", "unix://172.168.16.2:2", "172.168.16.2:2"),
469+ Entry("with a random scheme", "random://10.10.10.10:10", "10.10.10.10:10"),
470+ )
471+ })
472+})