Commit Graph

23886 Commits

Author SHA1 Message Date
J
347d51fde9 interface: cleaner API, robust GcpManager retry
GcpApi now acts like other APIs.

Since GcpManager can no longer get at the token exipry by inspecting the
raw update, it must depend on the global store instead. This also means
it can check whether the user has configured S3, and not try to refresh
the token in that case.

In the case where no storage is configured, this will spam the console
with request failures since the thread returns 500 if there is no token.
Perhaps this is a good argument for making  the thread return a unit.
2021-02-26 22:10:15 +00:00
J
723a5a050e interface: check accessKey !== undefined
Also make it type-level optional rather than using explicit null.

Perhaps one day we will want to use undefined to denote "the thread
hasn't returned yet" and null for "GCP Storage is not configured."
Perhaps.
2021-02-26 20:29:05 +00:00
J
6b3397bd9f interface: supports both S3 and GCP Storage
The S3 client has another layer of indirection we missed. To support it
expediently, we just make the promise() method on GcpUpload do all the
work in GcpClient.
2021-02-26 20:18:40 +00:00
J
6ec574d32b interface: GcpManager fixes 2021-02-26 20:18:07 +00:00
J
92dd46b176 GcpClient: remove unused BASE_URL 2021-02-26 00:49:57 +00:00
J
1c0d8e524e interface: thread gcp through props
Basically just a grep for 's3', and I added gcp wherever it looked like
I ought to.
2021-02-26 00:38:01 +00:00
J
11a58115b0 interface: implement useStorage and withStorage
useS3 and withS3 are now gone. Storage should work with both GCP and S3,
preferring GCP if available and falling back.
2021-02-26 00:07:15 +00:00
J
4814d61c48 interface: unnecessary const 2021-02-25 23:08:41 +00:00
J
a6d4c3a431 interface: proof-of-concept GCP upload
As of this commit, the Links app uploads files to GCP storage.

(Note that it no longer uploads to S3. Still need to support both.)
2021-02-25 22:52:18 +00:00
J
c718894eb3 interface: explicitly depend on querystring
It will be used by GcpClient.
2021-02-25 22:50:50 +00:00
J
efea38f842 interface: GcpReducer
Now we are correctly storing the GCP accessKey in state.
2021-02-25 00:41:47 +00:00
J
9fa086e0d3 interface: extract GcpManager
Now the API is just an API. (I did still have it produce an expiry time,
since it makes the refresh loop easier to do.)

Not yet storing the token now.
2021-02-25 00:31:39 +00:00
J
3c2ce636e7 interface: storing the gcp token 2021-02-25 00:31:16 +00:00
J
0a23c0f4a4 -get-gcp-token: docs, formatting
Correct expiry from @da -> @dr, and set max cols to 72.
2021-02-24 21:26:59 +00:00
J
a80c3504a8 interface: refresh loop
Still not actually storing the token anywhere...
2021-02-24 21:22:28 +00:00
J
d0bb1cc849 gcp: camel-case json key names
Now we can do ({accessKey, expiresIn}) => {...} in JavaScript.
2021-02-24 21:20:39 +00:00
J
d60c3183f7 gcp: lib spacing 2021-02-24 20:55:40 +00:00
J
94556ce936 interface: GCP skeleton 2021-02-24 20:42:57 +00:00
J
2a82da55ca -get-gcp-token: error messages
Don't dump extra console output on thread failure. Say 'gcp' in failures
to aid debugging.
2021-02-24 20:37:37 +00:00
J
03e99f52c9 gcp: make expiry relative
This indeed makes it easier on the Landscape side. It also makes it
easier to coerce from the JSON Google gives us, so a win all around.
2021-02-24 20:35:24 +00:00
J
713954a8bb gcp: mark for tokens, token-to-json 2021-02-24 19:30:49 +00:00
J
328159da67 -get-gcp-token: renamed from -get-gcp-jwt 2021-02-24 00:49:47 +00:00
J
d94f35aa5c gcp: add token type
Also use it in -get-gcp-jwt.
2021-02-24 00:49:47 +00:00
J
a8113182a8 -get-gcp-jwt: vertical 2021-02-23 23:48:50 +00:00
J
6a07130f0d -get-gcp-jwt: correct scope, clean up call
It turns out 'devstorage.read_write' also gives us an access token
instead of a JWT, and is probably more the thing that we want.

Took the opportunity to make scope a macro to clean up the make-jwt call
site.
2021-02-23 23:12:48 +00:00
J
aa93ac49c1 -get-gcp-jwt: sot -> jot, moar vertical
Since we only have one JWT (the thing we produce is an "access token",
not a JWT), we can just call it jot.
2021-02-23 23:03:07 +00:00
J
61d8030ec9 -get-gcp-jwt: documentation, clearer return faces 2021-02-23 22:53:42 +00:00
J
9f43fef85d -get-gcp-jwt: returns time of expiry
Couldn't bring myself to pass in a relative date without a base, so we
instead pass now into get-access-token, and return the time of expiry.
2021-02-23 22:40:05 +00:00
J
b0bb659f12 -get-gcp-jwt: @dr expires_in, cleanup
The expiry field we get from Google is not a timestamp, but a count of
seconds after which the token is invalid (hence, 'expires_in'). We
represent this as a @dr by multiplying the integer we get from Google by
~1s. (Perhaps it will wind up being easier on the Landscape side to
consume the raw integer, but we may as well start off more
type-correct.)

Also cleans up some names now that I'm less confused about how the token
API works, and makes some syntax more vertical.
2021-02-23 22:34:48 +00:00
J
d978e6a551 -get-gcp-jwt: produces an access token
The correct scope to ask for is not devstorage.editor, but
cloud-platform. This will, rather than returning a signed JWT from
Google that looks like it works but doesn't, return a JSON object
containing an access token and expiration time.
2021-02-23 22:18:59 +00:00
J
860343e3d4 -get-gcp-jwt: clean up references, use +ot
Moves =, closer to call sites so it's clearer what's coming from where.
Also uses +ot, allowing a less horrifying +sign-jwt. This also seems to
not jump back and forth between tapes and cords as much, for what that's
worth.
2021-02-23 21:14:35 +00:00
Jōshin
ac40b5cf24 -get-gcp-jwt: produce a JWT for use in GCP Storage 2021-02-23 21:13:30 +00:00
Jōshin
08791901b7 sh/poke-gcp-account-json: poke gcp storage values
To support GCP storage, we want to poke entries from a service account
JSON file into settings-store. This script does that.
2021-02-23 21:10:50 +00:00
Jōshin
ed3d5a233e
Merge pull request #4474 from urbit/jo/zuse-enjs-time
Supports @da-to-unix and unix-to@da conversion with both second and millisecond resolution in both chrono:userlib and en/dejs:format. Stops rounding up to the next millisecond in enjs.
2021-02-23 13:07:15 -08:00
Jōshin
1a0d26f03b pill: solid 2021-02-23 20:44:18 +00:00
Jōshin
92a0b5eadf zuse: simplify +di/du:dejs:format 2021-02-23 19:46:21 +00:00
Jōshin
9e69892631 zuse: time conversion fixes
State before: in chrono:userlib, there were second-resolution
@da-to-unix and unix-to-@da functions. In en/dejs:format, there were
millisecond-resolution @da-to-unix and unix-to-@da functions. The
@da-to-unix path in time:enjs confusingly rounded to the nearest
millisecond, meaning millisecond n was a label for [n-0.5, n+0.5) rather
than [n, n+1).

This adds a millisecond-resolution @da-to-unix and unix-to-@da to
chrono:userlib, and a second-resolution conversion to en/dejs:format.
It makes use of the chrono:userlib functions in en/dejs, and doesn't do
any rounding.

Backwards-incompatible changes:

- made unt:chrono:userlib take a @da rather than @.
2021-02-23 19:46:21 +00:00
Ted Blackman
42c120aaf1
Merge pull request #4454 from urbit/ted/gall-q
gall: fixes to %boon handling
2021-02-19 18:16:24 -05:00
Ted Blackman
674b5edb1f Merge branch 'release/next-sys' into ted/gall-q 2021-02-19 13:42:58 -05:00
fang
a3acc70384
Merge branch 'master' into release/next-sys 2021-02-19 19:31:24 +01:00
Ted Blackman
eb56fbd3f4 ames: shorten error printing; update pill 2021-02-19 11:54:34 -05:00
Ted Blackman
669169be0e pill: update solid 2021-02-18 22:02:54 -05:00
Ted Blackman
d317a0847b gall: crash properly on failed %boon from ames 2021-02-18 21:51:44 -05:00
Ted Blackman
b702505ac8 clay: print if +read-at-aeon crashes 2021-02-18 21:50:59 -05:00
Liam Fitzgerald
fab9a47a92
Merge remote-tracking branch 'origin/ixv/settings-existence-scry' 2021-02-19 12:47:47 +10:00
Ted Blackman
473a520b27 ames: more informative printing 2021-02-18 21:47:02 -05:00
Logan Allen
c4dead9661 contact-store: fix is-public being set to false on frontend 2021-02-18 13:19:52 -06:00
Logan Allen
6ee6a99acf contact-store: upon %edit of a nonexistent contact, make an empty contact and set that field 2021-02-18 11:51:15 -06:00
Isaac Visintainer
815c64b074 settings-store: add existence scry 2021-02-17 16:52:45 -08:00
Ted Blackman
22295f08fb gall: fix request queue desynchronization bug 2021-02-17 17:57:44 -05:00