neilotoole/sq
1
1
Fork 0
mirror of https://github.com/neilotoole/sq.git synced 2024-12-18 21:52:28 +03:00
Code Issues Actions Packages Projects Releases Wiki Activity
470 Commits 7 Branches 98 Tags 55 MiB
e0462c1125
Commit Graph

97 Commits

Author SHA1 Message Date
Neil O'Toole
611ad30f60
Test ingest Excel file format variants (#303) 
* Tests for additional formats

* Tests for ingest Excel file format variants
 2023-08-20 07:22:24 -06:00
Neil O'Toole
db092d5453
CSV ingest improvements (#302) 
* csv: additional datetime detect formats
 2023-08-18 09:21:11 -06:00
Neil O'Toole
edc933d897
#200: Feature/200 excel driver (#301) 
* Switch to excelize driver
 2023-08-16 09:09:50 -06:00
Neil O'Toole
c7bba4dfe4
go1.21: changes to support slog as part of stdlib (#299) 
* go1.21: changes to support slog as part of stdlib

* Removed accidentially checked-in line of code

* Fixed minor linting issues; reenable typecheck

* go1.21: switched to stdlib slices pkg
 2023-08-12 12:54:14 -06:00
Neil O'Toole
21c1dea9c4
#200: Switch excel driver for output (#291) 
* Switched from xlsxw to excelw for Excel outpt
 2023-07-26 22:19:11 -06:00
Neil O'Toole
6ca26f4e4f
Column rename: template now has Alpha field. (#285) 
* wip: refactor col name mungeing

* Finished refactoring FieldMeta

* Renamed tpl .AlphaIndex to .Alpha

* wip: debugging source config override

* Source config override passing tests

* CHANGELOG update
 2023-07-08 19:34:53 -06:00
Neil O'Toole
9c5836ef1c
#191: XLSX driver auto-detects header row (#284) 
* xlsx driver now detects header row.
 2023-07-08 09:21:27 -06:00
Neil O'Toole
4ffaae925f
#99: Rename duplicate ingest headers (#283) 
* CSV now renames duplicate ingest headers

* Fix broken test

* xlsx ingester now handles duplicate col names

* Update CHANGELOG

* Additional tests for ingest.column.rename

* Removed dead comment in grammar
 2023-07-04 11:31:47 -06:00
Neil O'Toole
7396aadb9e
#12: multiple joins (#280) 
* The query language now supports multiple joins.
 2023-07-03 09:34:19 -06:00
Neil O'Toole
a5c4f9d6bd
#263: Inspect --overview; improved output (#273) 
* Refactor tablew md writer

* wip: cmd inspect -v

* cmd inspect output improved

* cmd inspect tests

* cmd inspect: help text
 2023-06-21 23:48:58 -06:00
Neil O'Toole
2abf7222ef
Simplify temp file handling (#260) 2023-06-18 00:44:01 -06:00
Neil O'Toole
2ba633fc2a
#258: Alias can be an arbitrary string. (#259) 
* Fixed space issues with expressions

* Alias can now be an arbitrary string

* Alias can now be an arbitrary string (fixed)

* Alias now automatically applied to expressions

* Ignore .run

* Fixed issue with TestRun not logging correctly to testing.T

* Fiddling with sqlite3 temp file closing

* Re-enable tests
 2023-06-18 00:05:09 -06:00
Neil O'Toole
44d27207f8
#256: column-only queries (#257) 
* Column-only queries
 2023-06-17 19:28:11 -06:00
Neil O'Toole
e0812e9a65
#242 - Handle *uint64 returned from db (#253) 2023-06-15 07:50:07 -06:00
Neil O'Toole
9cb42bf579
#244: shell completion for "sq add LOCATION" (#246) 
- Shell completion for `sq add LOCATION`.
 2023-06-13 10:06:18 -06:00
Neil O'Toole
3ecdde5595
record.Record elements are now values, not pointers (#243) 
* Updated record pkg

* sqlite migrated

* postgres migrated

* All DBs migrated

* CSV working

* Tests passing
 2023-05-27 07:57:07 -06:00
neilotoole
4eb917f276 Make testh.OptLongOpen even longer 2023-05-25 23:02:11 -06:00
Neil O'Toole
c810d17eec
#229: sq diff ready for beta release (#239) 
* sq config edit: fixed glaring bug that prevented editing a source

* Refine sq diff
 2023-05-25 21:58:43 -06:00
Neil O'Toole
c3a543aa84
#229: refactor mysql source metadata impl (#236) 
* refactor mysql source metadata impl

* Made loop condition stronger
 2023-05-23 18:42:36 -06:00
Neil O'Toole
2f2dfd6e47
#229: More diff (#233) 
- Implement `sq diff --data`.
 2023-05-22 09:08:14 -06:00
Neil O'Toole
1ea24dac4a
#229: sq diff core (#230) 
* "sq diff" initial implementation

* Refactor "cli" pkg.
 2023-05-19 08:24:18 -06:00
Neil O'Toole
c5cf6c0cc0
#217: Configurable timestamp format (#218) 
* Moved time functions from pkg stringz to pkg timez

* Refactor options.Opt

* wip: initial work on configurable time layout

* wip: most printers now respect format.datetime and friends

* Folded pkg timefmt into timez

* Refactor options.Opt; refine options

* Add 'sq config set OPTION --help' mechanism

* Finished completion of OptDateFormatAsNumber and OptTimeFormatAsNumber
 2023-05-06 20:36:34 -06:00
Neil O'Toole
3180334c0c
#199: Config overhaul (#214) 
* refactor: partially moved over driver.Tuning params to options

* All knobs moved to options

* sq config edit: now has comments for options

* Major work complete on config/options overhaul

* Major work complete on config/options overhaul

* Updated help text for 'sq version'
 2023-05-03 06:36:10 -06:00
Neil O'Toole
4521008b72
#199: sq config set (#212) 
* Initial impl of 'sq config set'

* Verbose output for 'sq config get'

* config/options: refining 'sq config get'

* Refine 'sq config get': now prints defaults when --verbose

* cli: 'sq config get option.key' implemented

* cli: initial impl of 'sq config get --src'

* cli: refine 'sq config get'

* cli: 'sq config set --src' is implemented
 2023-04-30 21:59:34 -06:00
Neil O'Toole
e5db7990bc
#199: Kill RunContext.Log (#211) 
* refactor: RunContext.Log is dead

* Fiddling with options

* Changed default max conns to zero (no limit)

* Linting
 2023-04-30 08:18:56 -06:00
Neil O'Toole
5e31da4644
Refactor config options (#209) 
* Refactor config and options.
 2023-04-26 09:16:42 -06:00
Neil O'Toole
57cddd7590
Edit config (#208) 
* cobra: Added 'sq config edit' cmd

* Linting

* Implemented 'sq config edit'

* whitespace

* Dialing in config cmd help
 2023-04-22 16:31:28 -06:00
Neil O'Toole
81f631e135
Refactoring (#206) 
* Renamed source.Type to source.DriverType for clarity

* More renaming wrt source.DriverType

* Renamed output.Formatting to output.Printing
 2023-04-21 21:36:32 -06:00
neilotoole
c1b71fa68a Renamed testh.OptLongDB to testh.OptLongOpen; added it to other tests. 2023-04-19 08:08:26 -06:00
Neil O'Toole
98b47a2666
#199 - Config, refactoring (#204) 
* refactor: moved cli flags to pkg cli/flag

* testh: add OptLongDB for long-running tests

* implement 'sq config dir'

* legacy dir migration: probably a bad idea

* cleanup

* Refactored SQ_CONFIG and --config

* added yaml writer

* Dialing in tests

* YAML output for 'sq driver ls'

* Significant refactoring of config

* Minor test for ioz

* Rename source.Set to source.Collection

* Cleaning up references to source.Set
 2023-04-18 23:28:09 -06:00
Neil O'Toole
958d509088
Source groups (#198) 
* Implemented source groups mechanism.
 2023-04-15 16:28:51 -06:00
Neil O'Toole
ebb88b3a44
CSV empty string now treated as NULL (#190) 2023-04-09 19:29:13 -06:00
Neil O'Toole
ad23d5c116
Auto-detect CSV headers (#188) 
* wip: initial header detection

* wip: refactored csv driver

* wip: Fixes to kind.Detector

* typo

* Refactor pkg csv files

* Update to changelog

* workflow: now running codacy and codeql on cron, not on push/pr
 2023-04-09 08:44:27 -06:00
Neil O'Toole
54a0155ed6
Deal with lots of connections, especially for sq inspect. (#186) 
* wip: deal with lots of connections

* Clean up TestDatabase_SourceMetadata_concurrent

* Fixed error message
 2023-04-08 12:09:27 -06:00
Neil O'Toole
256d9c1ab6
go.mod: upgrade to jackc/pgx v5 (#184) 
* jackc/pgx upgrade

* jackc/pgx upgrade
 2023-04-08 06:39:06 -06:00
Neil O'Toole
9a1c6a7d09
Feature/173 args (#183) 
- Implement --arg feature
- Refactor sqlbuilder package (now called "render").
- Bug fixes, especially around expressions.
 2023-04-07 02:00:49 -06:00
Neil O'Toole
791d41ad1d
Removed dependencies on neilotoole/errgroup in favor of sync/errgroup (#176) 
* Removed dependencies on neilotoole/errgroup in favor of sync/errgroup

* Removed deadcode
 2023-04-02 15:17:15 -06:00
Neil O'Toole
3f6157c4c4
Change logging library to slog (#175) 
- Switch to slog logger.
 2023-04-02 13:49:45 -06:00
Neil O'Toole
bfd5542f75
Refactor/introduce query context (#174) 
* Moved query_X_test.go from /drivers to /libsq

* Refactor: introduced libsq.QueryContext type
 2023-04-01 03:48:24 -06:00
Neil O'Toole
8d47ef806d
go1.20 upgrade; lots of linting (#172) 2023-04-01 02:38:32 -06:00
Neil O'Toole
29e33ed2b1
Implement unique / DISTINCT (#165) 
- Implemented "unique" function
- Implemented "count_unique" function
 2023-03-28 00:48:24 -06:00
Neil O'Toole
fac3a27d7b
group_by() function (#163) 
- `group_by()` now accepts function as argument.
- Refactored grammar.
- Broad improvements to function implementation.
 2023-03-26 20:03:40 -06:00
Neil O'Toole
d9b56eea8a
Implement groupby() (#161) 
* test: slq2sql more test cases; now executes the generated query

* groupby: first test case working against all DBs

* sakila: added sqlserver 2019 source

* groupby: tests for groupby synonyms

* groupby: final touches
 2023-03-26 02:01:41 -06:00
Neil O'Toole
9746f4c1a2
orderby() (#159) 
* wip: orderby impl

* Tests passing (note: ast.checkASTIntegrity is disabled)

* ExprNode now rendered via renderSelectorNode

* linting

* CHAGELOG for v0.27.0
 2023-03-25 19:20:53 -06:00
Neil O'Toole
a1a89ee9dd
Support table and column names with spaces. (#156) 
* sakila: initial test data

* sakila: more test data

* sakila: yet more test data setup

* whitespace cols: now working for sqlite

* grammar cleanup

* whitespace cols: now working inside count() func for sqlite

* whitespace cols: tests mostly passing; begining refactoring

* grammar: refactor handle

* grammar: more refactoring

* grammar: rename selElement to selector

* wip

* all tests passing

* all tests passing

* linting

* driver: implement CurrentSchema for all driver.SQLDriver impls

* driver: tests for AlterTableRename and AlterTableRenameColumn

* undo reformat of SQL

* undo reformat of SQL

* undo reformat of SQL

* undo reformat of SQL
 2023-03-22 00:17:34 -06:00
Neil O'Toole
d3e6f89829
SLQ support for column aliases (#150) 
* alias: more early work

* alias: test cases working for sqlite

* alias: SQL builder tests

* alias: func (col expr) aliases now working for SQLite

* linting

* CHANGELOG update

* Docs update

* Docs update

* Rename buildAst() -> buildAST()

* CHANGELOG typo
 2023-03-18 22:58:00 -06:00
Neil O'Toole
161d6f9ee7
Fix to docs for cmd_sql (#148) 2023-03-15 01:43:48 -06:00
Neil O'Toole
7148505a84
CSV: check for mismatch field count vs explicitly specified column names (#147) 
* csv: check for mismatch field count vs explicitly specified column names

* CHANGELOG update
 2023-03-14 23:04:49 -06:00
Neil O'Toole
f71b124a1f
Testing for postgres alternate schema 2023-03-13 19:25:12 -06:00
Neil O'Toole
5d799a8820
Bug/windows file slash munge location (#140) 
* Munge windows file location

* Windows filepath fix

* More Windows filepath issues

* Main workflow runs on matrix each time
 2022-12-31 22:35:33 -07:00