git.iko.soy/rowtype-yoga/purescript-yoga-om-workerbees
on main help sign in
rowtype-yoga/purescript-yoga-om-workerbees/README.md
204 lines 5.1 kB history raw
1# purescript-yoga-om-workerbees
3Multi-threaded computation for the Om effect monad using Node.js worker threads.
5This library provides a clean, type-safe interface for running CPU-bound computations across multiple worker threads, with support for shared memory primitives and atomic operations.
7## Features
9- **Worker Pool Management**: Create and manage pools of worker threads
10- **Shared Memory Primitives**:
11 - `SharedInt`: Atomic Int32 reference with compare-and-swap
12 - `SharedState`: JSON-serialized state with spinlock synchronization
13 - `SharedMap`: Concurrent hash map with striped locks
14- **Type-Safe**: Full PureScript type safety for worker communication
15- **Om Integration**: Seamless integration with the Om effect monad
16- **Automatic Work Distribution**: Distribute tasks across available workers
18## Installation
20```bash
21spago install yoga-om-workerbees
22```
24**Dependencies:**
25- `purescript-yoga-om` - The Om effect monad
26- `purescript-node-workerbees` - Low-level worker thread bindings
27- `purescript-yoga-json` - JSON serialization
29## Quick Start
31### 1. Create a Worker Module
33```purescript
34module MyWorker where
36import Prelude
37import Node.WorkerBees as WB
39-- Worker function: Int -> Int
40fibWorker :: Int -> Int
41fibWorker n
42 | n <= 1 = n
43 | otherwise = fibWorker (n - 1) + fibWorker (n - 2)
45-- Export as worker main
46main :: Effect Unit
47main = WB.makeAsMain fibWorker
48```
50### 2. Bundle the Worker
52```bash
53spago bundle \
54 --module MyWorker \
55 --outfile dist/workers/MyWorker.js \
56 --platform node
57```
59### 3. Use the Worker Pool
61```purescript
62module Main where
64import Prelude
65import Effect.Aff (launchAff_)
66import Yoga.Om.WorkerBees (makePool, distributeWork, terminatePool)
68main :: Effect Unit
69main = launchAff_ do
70 -- Create worker pool
71 pool <- makePool
72 { workerPath: "./dist/workers/MyWorker.js"
73 , poolSize: 4
74 }
76 -- Distribute work
77 let inputs = [35, 36, 37, 38]
78 results <- distributeWork pool inputs
80 -- Clean up
81 terminatePool pool
82```
84## Shared Memory
86### SharedInt (Atomic Counter)
88```purescript
89import Yoga.Om.WorkerBees.SharedInt as SharedInt
91-- Create atomic counter
92counter <- SharedInt.new 0 # liftEffect
94-- Atomic operations
95SharedInt.add counter 1 # liftEffect
96SharedInt.sub counter 1 # liftEffect
97value <- SharedInt.read counter # liftEffect
99-- Compare-and-swap
100success <- SharedInt.compareAndSwap counter 0 42 # liftEffect
102-- Atomic modify
103SharedInt.modify counter (_ + 1) # liftEffect
104```
106### SharedState (JSON State)
108```purescript
109import Yoga.Om.WorkerBees.SharedState as SharedState
111type AppState = { counter :: Int, message :: String }
113-- Create shared state (1KB buffer)
114state <- SharedState.new 1024 { counter: 0, message: "Hello" } # liftEffect
116-- Read/write
117current <- SharedState.read state # liftEffect
118SharedState.write state { counter: 1, message: "Updated" } # liftEffect
120-- Atomic modify
121SharedState.modify state (\s -> s { counter = s.counter + 1 }) # liftEffect
122```
124### SharedMap (Concurrent Hash Map)
126```purescript
127import Yoga.Om.WorkerBees.SharedMap as SharedMap
129-- Create map (16 stripes, 4KB per entry)
130map <- SharedMap.new 16 4096 # liftEffect
132-- Insert/lookup/delete
133SharedMap.insert map "key" { value: 42 } # liftEffect
134maybeValue <- SharedMap.lookup map "key" # liftEffect
135SharedMap.delete map "key" # liftEffect
137-- Atomic modify
138SharedMap.modify map "key" (\v -> v { value = v.value + 1 }) # liftEffect
139```
141## Examples
143Run the console examples:
145```bash
146# Install dependencies
147bun install
149# Fibonacci calculation with worker pool
150bun run example:fibonacci
152# Shared atomic counter
153bun run example:counter
155# Shared JSON state
156bun run example:state
158# Concurrent hash map
159bun run example:map
160```
162See the `examples/` directory for full source code.
164## When to Use WorkerBees
166**Use WorkerBees for:**
167- CPU-bound computations (cryptography, image processing, parsing)
168- Parallel processing of large datasets
169- Tasks that would block the event loop
171**Don't use WorkerBees for:**
172- I/O operations (use `Aff` instead - Node.js already handles I/O concurrency)
173- Simple computations (overhead of worker communication isn't worth it)
174- Shared state when Aff + `Ref` would suffice
176## Architecture
178- **WorkerPool**: Manages a pool of worker threads
179- **Sendable**: Type-safe serialization for worker communication
180- **SharedArrayBuffer**: Native shared memory between threads
181- **Atomics**: Low-level atomic operations (load, store, compareExchange)
182- **SharedInt/SharedState/SharedMap**: High-level shared memory abstractions
184## Performance Tips
1861. **Pool Size**: Default is 4 workers. Adjust based on CPU cores and workload:
187 ```purescript
188 makePool { workerPath, poolSize: 8 }
189 ```
1912. **Batch Work**: Distribute many small tasks rather than few large ones for better load balancing
1933. **Shared Memory**: Use `makePoolWithData` to share memory instead of copying data:
194 ```purescript
195 counter <- SharedInt.new 0 # liftEffect
196 pool <- makePoolWithData (SharedInt.toSendable counter) config
197 ```
1994. **Worker Bundling**: Always bundle workers with `--platform node` for optimal performance
201## License
203MIT