neilotoole/sq
1
1
Fork 0
mirror of https://github.com/neilotoole/sq.git synced 2024-12-25 17:25:36 +03:00
Code Issues Actions Packages Projects Releases Wiki Activity
604 Commits 7 Branches 98 Tags 55 MiB
2d3f68bb47
Commit Graph

164 Commits

Author SHA1 Message Date
Neil O'Toole
c7bba4dfe4
go1.21: changes to support slog as part of stdlib (#299) 
* go1.21: changes to support slog as part of stdlib

* Removed accidentially checked-in line of code

* Fixed minor linting issues; reenable typecheck

* go1.21: switched to stdlib slices pkg
 2023-08-12 12:54:14 -06:00
Neil O'Toole
3c4b61b806
#200: Excel date format config (#294) 
* Config options to control Excel date/time output format
 2023-08-03 23:41:33 -06:00
Neil O'Toole
21c1dea9c4
#200: Switch excel driver for output (#291) 
* Switched from xlsxw to excelw for Excel outpt
 2023-07-26 22:19:11 -06:00
Neil O'Toole
6ca26f4e4f
Column rename: template now has Alpha field. (#285) 
* wip: refactor col name mungeing

* Finished refactoring FieldMeta

* Renamed tpl .AlphaIndex to .Alpha

* wip: debugging source config override

* Source config override passing tests

* CHANGELOG update
 2023-07-08 19:34:53 -06:00
Neil O'Toole
9c5836ef1c
#191: XLSX driver auto-detects header row (#284) 
* xlsx driver now detects header row.
 2023-07-08 09:21:27 -06:00
Neil O'Toole
4ffaae925f
#99: Rename duplicate ingest headers (#283) 
* CSV now renames duplicate ingest headers

* Fix broken test

* xlsx ingester now handles duplicate col names

* Update CHANGELOG

* Additional tests for ingest.column.rename

* Removed dead comment in grammar
 2023-07-04 11:31:47 -06:00
Neil O'Toole
7396aadb9e
#12: multiple joins (#280) 
* The query language now supports multiple joins.
 2023-07-03 09:34:19 -06:00
Neil O'Toole
072cb4f515
Tidy go.mod; revert osquery lib to original package (#276) 2023-06-22 21:07:27 -06:00
Neil O'Toole
9ebd1c4b01
Fix cmd version bug on windows (#275) 2023-06-22 11:34:35 -06:00
Neil O'Toole
a5c4f9d6bd
#263: Inspect --overview; improved output (#273) 
* Refactor tablew md writer

* wip: cmd inspect -v

* cmd inspect output improved

* cmd inspect tests

* cmd inspect: help text
 2023-06-21 23:48:58 -06:00
Neil O'Toole
85d6f7a2f9
#263: cmd version: structure yaml output correctly (#272) 
* cmd version: structure yaml output correctly

* Fix bug_report.md to match 'sq version --yaml' output
 2023-06-21 09:15:40 -06:00
Neil O'Toole
97739da1e1
#263: sq version host info (#271) 
* sq version host info

* workflow: update bug_report.md with version instructions
 2023-06-21 06:28:15 -06:00
Neil O'Toole
eedc11ec46
#261: jsonw: deadlock occurred when record was large, due to mutex on Flush() call (#262) 
* jsonw: deadlock occurred when record was large, due to mutex on Flush() call
 2023-06-19 10:55:38 -06:00
Neil O'Toole
2ba633fc2a
#258: Alias can be an arbitrary string. (#259) 
* Fixed space issues with expressions

* Alias can now be an arbitrary string

* Alias can now be an arbitrary string (fixed)

* Alias now automatically applied to expressions

* Ignore .run

* Fixed issue with TestRun not logging correctly to testing.T

* Fiddling with sqlite3 temp file closing

* Re-enable tests
 2023-06-18 00:05:09 -06:00
Neil O'Toole
44d27207f8
#256: column-only queries (#257) 
* Column-only queries
 2023-06-17 19:28:11 -06:00
Neil O'Toole
e93f462fff
#254: explicit where() clause (#255) 
Implement explicit `where()` clause.
 2023-06-16 22:54:25 -06:00
Neil O'Toole
9cb42bf579
#244: shell completion for "sq add LOCATION" (#246) 
- Shell completion for `sq add LOCATION`.
 2023-06-13 10:06:18 -06:00
Neil O'Toole
3ecdde5595
record.Record elements are now values, not pointers (#243) 
* Updated record pkg

* sqlite migrated

* postgres migrated

* All DBs migrated

* CSV working

* Tests passing
 2023-05-27 07:57:07 -06:00
Neil O'Toole
debb282386
diff: renamed --count flag to --counts as intended (#240) 
* diff: renamed --count flag to --counts as intended

* CHANGELOG for v0.36.1
 2023-05-26 14:45:08 -06:00
Neil O'Toole
c810d17eec
#229: sq diff ready for beta release (#239) 
* sq config edit: fixed glaring bug that prevented editing a source

* Refine sq diff
 2023-05-25 21:58:43 -06:00
Neil O'Toole
76d48154e7
#229: Yet more sq diff implementation. (#237) 
* sq config edit: fixed glaring bug that prevented editing a source

* Refine sq diff
 2023-05-24 17:36:10 -06:00
Neil O'Toole
2f2dfd6e47
#229: More diff (#233) 
- Implement `sq diff --data`.
 2023-05-22 09:08:14 -06:00
Neil O'Toole
1ea24dac4a
#229: sq diff core (#230) 
* "sq diff" initial implementation

* Refactor "cli" pkg.
 2023-05-19 08:24:18 -06:00
Neil O'Toole
12b33d5b49
Implemented yaml record writer (#226) 2023-05-10 20:03:45 -06:00
Neil O'Toole
f5e7e7b8cb
Restore --markdown flag (#220) 
* Restore --markdown flag (also suppport --md)

* Update README and splash image
 2023-05-08 07:39:47 -06:00
neilotoole
ceab8b700e jsonw & yamlw: sq config -v: print options even if empty 2023-05-07 02:15:02 -06:00
neilotoole
7b3f9f9dab Doc cleanup 2023-05-07 00:29:42 -06:00
Neil O'Toole
11525eae10
#199: Last spit n' polish for config overhaul (#219) 
* Split 'sq config ls' into its own command

* Tidy up 'sq config' help

* Fixed help text
 2023-05-07 00:01:29 -06:00
Neil O'Toole
c5cf6c0cc0
#217: Configurable timestamp format (#218) 
* Moved time functions from pkg stringz to pkg timez

* Refactor options.Opt

* wip: initial work on configurable time layout

* wip: most printers now respect format.datetime and friends

* Folded pkg timefmt into timez

* Refactor options.Opt; refine options

* Add 'sq config set OPTION --help' mechanism

* Finished completion of OptDateFormatAsNumber and OptTimeFormatAsNumber
 2023-05-06 20:36:34 -06:00
Neil O'Toole
964417dba7
Use flag --compact instead of --pretty (#216) 2023-05-05 11:41:22 -06:00
Neil O'Toole
f0aa65791b
#199: More config/options work (#215) 
* CHANGELOG text clarification

* Dialing in config/options

* Yet more dialing in of config/options

* Refactor output writers

* YAML output for more commands
 2023-05-05 08:32:50 -06:00
Neil O'Toole
3180334c0c
#199: Config overhaul (#214) 
* refactor: partially moved over driver.Tuning params to options

* All knobs moved to options

* sq config edit: now has comments for options

* Major work complete on config/options overhaul

* Major work complete on config/options overhaul

* Updated help text for 'sq version'
 2023-05-03 06:36:10 -06:00
Neil O'Toole
4521008b72
#199: sq config set (#212) 
* Initial impl of 'sq config set'

* Verbose output for 'sq config get'

* config/options: refining 'sq config get'

* Refine 'sq config get': now prints defaults when --verbose

* cli: 'sq config get option.key' implemented

* cli: initial impl of 'sq config get --src'

* cli: refine 'sq config get'

* cli: 'sq config set --src' is implemented
 2023-04-30 21:59:34 -06:00
Neil O'Toole
e5db7990bc
#199: Kill RunContext.Log (#211) 
* refactor: RunContext.Log is dead

* Fiddling with options

* Changed default max conns to zero (no limit)

* Linting
 2023-04-30 08:18:56 -06:00
Neil O'Toole
5e31da4644
Refactor config options (#209) 
* Refactor config and options.
 2023-04-26 09:16:42 -06:00
Neil O'Toole
57cddd7590
Edit config (#208) 
* cobra: Added 'sq config edit' cmd

* Linting

* Implemented 'sq config edit'

* whitespace

* Dialing in config cmd help
 2023-04-22 16:31:28 -06:00
Neil O'Toole
a766340382
Pass context to config.Store (#207) 2023-04-22 07:37:07 -06:00
Neil O'Toole
81f631e135
Refactoring (#206) 
* Renamed source.Type to source.DriverType for clarity

* More renaming wrt source.DriverType

* Renamed output.Formatting to output.Printing
 2023-04-21 21:36:32 -06:00
neilotoole
c1b71fa68a Renamed testh.OptLongDB to testh.OptLongOpen; added it to other tests. 2023-04-19 08:08:26 -06:00
Neil O'Toole
98b47a2666
#199 - Config, refactoring (#204) 
* refactor: moved cli flags to pkg cli/flag

* testh: add OptLongDB for long-running tests

* implement 'sq config dir'

* legacy dir migration: probably a bad idea

* cleanup

* Refactored SQ_CONFIG and --config

* added yaml writer

* Dialing in tests

* YAML output for 'sq driver ls'

* Significant refactoring of config

* Minor test for ioz

* Rename source.Set to source.Collection

* Cleaning up references to source.Set
 2023-04-18 23:28:09 -06:00
Neil O'Toole
958d509088
Source groups (#198) 
* Implemented source groups mechanism.
 2023-04-15 16:28:51 -06:00
Neil O'Toole
ebb88b3a44
CSV empty string now treated as NULL (#190) 2023-04-09 19:29:13 -06:00
neilotoole
150dfc01a9 Fixed some tests that had neglected to drop tables they had created 2023-04-08 14:58:10 -06:00
neilotoole
8f0d8d0874 Use --md instead of --markdown 2023-04-08 13:19:06 -06:00
Neil O'Toole
54a0155ed6
Deal with lots of connections, especially for sq inspect. (#186) 
* wip: deal with lots of connections

* Clean up TestDatabase_SourceMetadata_concurrent

* Fixed error message
 2023-04-08 12:09:27 -06:00
Neil O'Toole
9a1c6a7d09
Feature/173 args (#183) 
- Implement --arg feature
- Refactor sqlbuilder package (now called "render").
- Bug fixes, especially around expressions.
 2023-04-07 02:00:49 -06:00
Neil O'Toole
3f6157c4c4
Change logging library to slog (#175) 
- Switch to slog logger.
 2023-04-02 13:49:45 -06:00
Neil O'Toole
bfd5542f75
Refactor/introduce query context (#174) 
* Moved query_X_test.go from /drivers to /libsq

* Refactor: introduced libsq.QueryContext type
 2023-04-01 03:48:24 -06:00
Neil O'Toole
8d47ef806d
go1.20 upgrade; lots of linting (#172) 2023-04-01 02:38:32 -06:00
Neil O'Toole
9746f4c1a2
orderby() (#159) 
* wip: orderby impl

* Tests passing (note: ast.checkASTIntegrity is disabled)

* ExprNode now rendered via renderSelectorNode

* linting

* CHAGELOG for v0.27.0
 2023-03-25 19:20:53 -06:00
Neil O'Toole
a1a89ee9dd
Support table and column names with spaces. (#156) 
* sakila: initial test data

* sakila: more test data

* sakila: yet more test data setup

* whitespace cols: now working for sqlite

* grammar cleanup

* whitespace cols: now working inside count() func for sqlite

* whitespace cols: tests mostly passing; begining refactoring

* grammar: refactor handle

* grammar: more refactoring

* grammar: rename selElement to selector

* wip

* all tests passing

* all tests passing

* linting

* driver: implement CurrentSchema for all driver.SQLDriver impls

* driver: tests for AlterTableRename and AlterTableRenameColumn

* undo reformat of SQL

* undo reformat of SQL

* undo reformat of SQL

* undo reformat of SQL
 2023-03-22 00:17:34 -06:00
Neil O'Toole
db189e55a7
Bug/153 text table long lines (#154) 
* text table: fixed long lines padding

* text table: fixed long lines padding
 2023-03-19 01:51:05 -06:00
Neil O'Toole
89ba5a04f6
Feature/151 sq add active (#152) 
* sq add: Now with --active flag

* sq add: Add test for shorthand -a flag

* sq add: docs update
 2023-03-19 00:18:54 -06:00
Neil O'Toole
d3e6f89829
SLQ support for column aliases (#150) 
* alias: more early work

* alias: test cases working for sqlite

* alias: SQL builder tests

* alias: func (col expr) aliases now working for SQLite

* linting

* CHANGELOG update

* Docs update

* Docs update

* Rename buildAst() -> buildAST()

* CHANGELOG typo
 2023-03-18 22:58:00 -06:00
Neil O'Toole
161d6f9ee7
Fix to docs for cmd_sql (#148) 2023-03-15 01:43:48 -06:00
Neil O'Toole
143d2a1101
Bug/no active source (#145) 
* Clean up misc text

* config: will repair itself if active source is corrupted

* Clean up error handling code

* CHANGELOG update

* Update config tests to allow invalid active source
 2023-03-11 21:21:05 -07:00
neilotoole
b227dd7112 docs updates 2023-03-11 20:25:19 -07:00
Neil O'Toole
82dfd1c703
PIng command json output (#141) 
* cmd_ping now respects --json

* ctrl-c doesn't print the cancel message for sq ping

* changelog update
 2022-12-31 23:00:07 -07:00
Neil O'Toole
5d799a8820
Bug/windows file slash munge location (#140) 
* Munge windows file location

* Windows filepath fix

* More Windows filepath issues

* Main workflow runs on matrix each time
 2022-12-31 22:35:33 -07:00
Neil O'Toole
ed9aa38a67
Improvements to source commands (#139) 
* Expose source.Set.Data() method

* jsonw.writeJSON cleaned up

* sq add now respects --json

* Location strings are subject to more scrutiny

* Ignore .db files in project dir

* sq add is more restrictive about location string

* source.RedactedLocation now uses 'xxxxx' per stdlib url.URL.Redacted()

* Update changelog for v0.23.0

* typos
 2022-12-31 20:17:44 -07:00
neilotoole
23000ef0ee Adjustment to sq version output 2022-12-30 10:59:27 -07:00
neilotoole
3672f28f72 Linting 2022-12-30 10:38:49 -07:00
neilotoole
433d83d5b2 linting 2022-12-30 10:28:01 -07:00
neilotoole
19aa8c76ff More error checking in cmd_version 2022-12-30 10:25:45 -07:00
Neil O'Toole
66a8c39844
Improvements to 'sq version' command (#137) 2022-12-30 10:10:56 -07:00
Neil O'Toole
d9693bde17
Feature/manpages (#136) 
* manpage support

* Polish up man work

* Set DisableAutoGenTag=true for all commands

* Set DisableAutoGenTag=true for root cmd

* CHANGELOG
 2022-12-29 19:02:10 -07:00
neilotoole
70e650ef87 linting 2022-12-24 21:09:20 -07:00
Neil O'Toole
a1ba6578da
"add" command supports hiding password input (#132) 
* renamed cmdFlagChanged to flagChanged

* initial stdin stuff working

* wip: mostly working as expected

* Docs and lots of cleanup

* Mostly docs

* fixed behavior of source.LocationWithPassword, and tests
 2022-12-24 21:04:18 -07:00
Neil O'Toole
fff01a32db
Chore/yet more mod upgrades (#128) 
* gopkg.in/yaml.v3 upgrade

* emirpasic/gods@v1.18.1

* yet more module stuff
 2022-12-23 14:05:24 -07:00
Neil O'Toole
97da9a53a3
'__complete sql @offline_handle' no longer errors, but instead returns the handle (#124) 
* '__complete sql @offline_handle' no longer errors, but instead returns the handle

* docs update
 2022-12-23 09:32:07 -07:00
Neil O'Toole
fab365f43c
gofumpt on files (#122) 
* gofumpt on files

* more gofumpt
 2022-12-18 01:35:59 -07:00
Neil O'Toole
c778c8ced5
Linting of _test.go files (#121) 
* test linting

* test linting

* test linting

* test linting
 2022-12-18 00:18:35 -07:00
Neil O'Toole
f9c19785e3
Almost at the end of the linting road (#120) 
* yet more linting

* yet more linting

* yet more linting

* yet more linting

* yet more linting

* yet more linting

* yet more linting

* yet more linting
 2022-12-17 23:42:11 -07:00
Neil O'Toole
425702e7ba
The linting goes on forever (#119) 
* linting

* yet more linting

* yet more linting

* yet more linting

* yet more linting

* yet more linting
 2022-12-17 23:07:38 -07:00
Neil O'Toole
ac7535609d
Yet more linting (#118) 
* lint fixes

* lint gosec
 2022-12-17 22:16:10 -07:00
Neil O'Toole
08dfa10325
Errors linting (#117) 
* errors linting

* errors linting

* formatting
 2022-12-17 21:31:06 -07:00
Neil O'Toole
85b8879324
got rid of dead notify package (#116) 2022-12-17 20:05:46 -07:00
Neil O'Toole
540adfac58
Lint long lines (#115) 
* lint config now sorts results

* linted long lines

* linted long lines
 2022-12-17 19:43:53 -07:00
Neil O'Toole
2831211ae9
Yet more linting (#114) 
* wip: bunch o' linting

* bunch more linting
 2022-12-17 17:51:33 -07:00
Neil O'Toole
5187e8000a
bunch of linting issues fixed (#113) 2022-12-17 16:11:33 -07:00
Neil O'Toole
27b70731e9
cobra upgrade; help output now the same for 'sq help' and 'sq --help' (#112) 2022-12-16 21:59:42 -07:00
Neil O'Toole
703f6d2427
Refactored verbose/header flags; inspect cmd now less verbose by default (#111) 
* Refactored verbose/header flags; inspect cmd now less verbose by default

* Updated changelog

* README update
 2022-12-16 20:46:37 -07:00
Neil O'Toole
6a0878bc6b
fixed broken mysql tests (parseTime param); moved some test funcs to pkg tutil (#109) 2022-12-16 19:09:49 -07:00
Neil O'Toole
e674cdc724
change 'interface{}' to 'any' (#108) 2022-12-16 16:34:33 -07:00
Neil O'Toole
a419e0b693
Misc cleanup, goimports, updates (#107) 
* debugging

* generic retry functionality; some additions to stringz.go

* fiddling with magefile_sakila

* added test lock function testh.Lock

* bunch of linting, goimports, what not
 2022-12-16 15:54:09 -07:00
Neil O'Toole
23022f2b7f
Go1.19 compatibility (#104) 
* upgraded mattn/go-isatty version because it was importing a incompatible version of golang.org/x/sys/unix

* changed parsing of SQL Server DSN due to changes to url.ParseQuery no longer accepting semicolon

* updated xo/dburl pkg to v0.11.0

* removed dead underscore assignment

* updated GH workflow go action to 1.19
 2022-08-08 13:14:42 -06:00
Neil O'Toole
b06b631e76
Bug #87: generated SQL should always quote table and column names in join statement (#89) 
* BaseFragmentBuilder now quotes table and col names for joins

* Refactored libsq.engine so that the SQL generated from SLQ input can be tested

* Deleted dead code; additional comments
 2021-03-07 23:27:35 -07:00
Neil O'Toole
f1335c4a30
Cmd ping to use --all flag instead of arg (#83) 
* cmd ping now uses --all flag instead of arg
 2021-02-27 07:44:08 -07:00
Neil O'Toole
6870327508
Cobra upgrade: includes shell completion work (#81) 
Addressed #80
 2021-02-22 00:37:00 -07:00
Neil O'Toole
a1cfd5d525
Xlsx import handle empty sheets (#79) 
* can now import XLSX with empty sheets

* renamed XLSX test article
 2021-02-19 23:22:35 -07:00
Neil O'Toole
8fd0b292c7 flag usage update 2021-01-03 19:56:59 -07:00
Neil O'Toole
71a4f78e52
--insert now also creates the table (#78) 2021-01-03 19:56:22 -07:00
Neil O'Toole
e2f51f97c7 fixed panic with insert to non-existent table 2021-01-03 18:20:05 -07:00
Neil O'Toole
45b64d4848 docs update 2021-01-03 17:40:32 -07:00
Neil O'Toole
8345e6743b
removed notify stuff (#74) 2020-12-30 12:18:22 -07:00
Neil O'Toole
5aebc04356
Fixed issues with files and databases not being closed correctly (#73) 
* fiddling with scratch database close order

* files debugging

* files debugging2

* files debugging3

* files debugging 4

* files debugging 5

* didn't close the ReadCloser in csv import

* more closing cleanup
 2020-12-30 11:57:58 -07:00
Neil O'Toole
06900f8c84
fixed some panics with oddly-shaped xlsx imports (#71) 2020-11-02 10:40:29 -07:00
Neil O'Toole
a92b9abf34
Initial work on a JSON driver (#70) 
* implementation work for json importers

* json driver checkpoint

* working on json.ParseObjectsInArray

* json.ParseObjectsInArray seems to be working

* checkpoint while tidying up ParseObjectsInArray

* more tidy checkpoint

* more tidy checkpoint 2

* tidying up ParseObjectsInArray

* tidy up

* code/docs cleanup

* more cleanup of json driver

* more cleanup of json driver

* flat json import seemingly working

* improvements to json driver

* json writer now prints empty [] for postgres empty tables
 2020-10-20 09:05:43 -06:00
Neil O'Toole
0f3bc26f52
JSON driver: initial jsona driver (#69) 
* jsona driver appears to be mostly working

* jsona verified as working

* minor tidy
 2020-08-23 20:32:59 -06:00
Neil O'Toole
1818d50546
moved pkg libsq/sqlmodel to libsq/core/sqlmodel (#68) 2020-08-23 13:00:13 -06:00