3.4 KiB
- katip
- katip-elasticsearch
- Efficient UUID generation (don't use system rng)
- Async size-capped buffer for incoming items
- Be resilient to server disconnects and crashes
- Connection pooling (bloodhound)
- Stop using Strings everywhere (bloodhound)
- update index settings on setup (when the API becomes available in bloodhound)
- by-day or user scheduled index sharding
- katip-rotated-log
- katip-retry
katip
TODO Should we remove orphan IO instance and add a LoggingT?
DONE Add a "Maybe LocInfo" using a TH version of logF
CLOSED: [2015-02-28 Sat 21:23]
DONE Log current PID in Item
CLOSED: [2015-02-28 Sat 20:01]
DONE Rethink important fields
CLOSED: [2015-02-28 Sat 20:01]
Maybe add a logF variant that uses the class, but it's not
mandatory
Another variant that explicitly passes the fields of interest
DONE Add Verbosity to the mix
CLOSED: [2015-02-28 Sat 20:01]
DONE AppName -> [Text]
CLOSED: [2015-02-28 Sat 20:21]
- LogEnv defines the base name-space
- logF gets a [Text] argument for AppName namespace
- Whe two are concatted for each message
katip-elasticsearch
DONE Efficient UUID generation (don't use system rng)
CLOSED: [2015-03-17 Tue 10:27]
Make RNG once, store it behind an mvar and use it in all uuid generation. Use the rng-utils module for this.
Turns out uuid is already doing some clever stuff in their random
instance to make this plenty fast. Benchmarks show we can generate
> 1M ids/sec.
DONE Async size-capped buffer for incoming items
CLOSED: [2015-03-20 Fri 17:08] Don't block log requests for server connection, add to buffer and move on. If buffer fills up, drop further logs until there's space.
We can use a limited chan with a pre-check to make sure that if the size limit is reached, we drop the request instead of blocking until limit is reached.
TODO Be resilient to server disconnects and crashes
The async buffer method described above certainly helps with this. In addition, make sure that nothing out of the ordinary happens if the server is unreachable. Logging should not be an element in any stack that can potentially induce a crash - that would defeat the purpose!
This means:
- No slowdowns, no crashes even if server is unreachable
- No file descriptor hogging (tight loop retries, etc.)
- Gently try once in a while to see if the server has become reachable again
DONE Connection pooling (bloodhound)
CLOSED: [2015-03-20 Fri 17:09]
Exposing Manager would solve this, as it does internal pooling.
DONE Stop using Strings everywhere (bloodhound)
CLOSED: [2015-03-20 Fri 17:09]
Seems to be a limitation of http-client that we'll have to live with for now.
DONE update index settings on setup (when the API becomes available in bloodhound)
CLOSED: [2015-11-12 Thu 08:33]
TODO by-day or user scheduled index sharding
Consider using templates and write to indexes per-day. So rather than building the direct index the user asks for, set up a template that automatically gets applied to indices matching the prefix, then create an index per day. Apparently this allows more efficient searches. Its yet to be seen when one-big-index breaks down
katip-rotated-log
TODO Create a rotated-log scribe
This works just like the built-in Handle scribe, but pushes the log through the rotated-log packages and therefore rotates files when they get too large
katip-retry
TODO Implement combinator for creating exception Handler that log to katip
Along the lines of the built-in 'logRetries' in retry, but instead captures structured information through katip.
Explicitly capture fields to aid in filtering:
- Retry number
- The exception
- The exception type - maybe via Typeable's typeOf?
- Decision result: Crashing vs. Retrying