This adds support for additional url query params which make stuff like
```
pgcli "postgres://user:pass@host.invalid/db?\
sslmode=verify-full&sslcert=my.pem&sslkey=my-key.pem&sslrootcert=ca.pem"
```
work. More generally it allows one to pass through various extra connection
options.
Having typed an alias name in an `ORDER BY` or (`SELECT`) `DISTINCT`
clause, the alias was not taken account of, and the completion simply
listed all columns. This change fixes that, and makes the autocompletion
behave the same as in `SELECT` and `WHERE` clauses.
A bunch of cosmetic changes to be more compliant with PEP80 and pylint.
And I added a pylintrc so that we can ignore some rules that just add
too much clutter.
This introduces functionality for running the same test case for
different completer settings. This should improve code-path coverage.
I've set up all the old tests to run with all configs that should have
identical output.
Consider this script
```
CREATE FUNCTION foo() returns text LANGUAGE SQL AS $func$
SELECT 1 FROM Bar;
SELECT <cursor> FROM Baz;
$func$;
```
The change here is that `SELECT <cursor> FROM Baz;` will be seen as the
current statement, instead of the whole function definition.
This means we'll no longer get column suggestions from `Bar`.
And schema-qualifying them, of course, so that for `SELECT * FROM bar`
we might suggest `buildings.barns` and for `select dopi` we might
suggest `maintenance.delete_old_personal_info()`, for example.
Controlled by a config setting, `search_path_filter`, in case anyone
prefers the old behaviour.
E.g. for table `Foo` with columns `FooBar` and `Fabulous`, inputting
`SELECT * FROM Foo F WHERE F.fb` will suggest `FooBar` as the #1 choice.
Likewise, given the functions `baz_baz()` and `baab()`, inputting
`SELECT bb` will have `baz_baz()` as the first option.
The idea is that if you have e.g. a table called `FooBar`, where we'd
use the initialism `FB` as an alias (suggesting `FooBar FB`),
inputting e.g. `SELECT * FROM FB` should suggest `FooBar FB` (or
`FooBar` if aliasing is disabled). Ditto for functions, views and joins.
The solution is that we send `FB` as a `synonym` to `find_matches`,
which results in the `FooBar` suggestion rising to the top (above e.g.
`FabulousTable`).
... i.e. suggesting foo.fooid instead of just fooid
Controlled using a config-file setting:
**qualify_columns**: always/never/**if_more_than_one_table**.
To enable the proper sorting of qualified column suggestions, we
introduce the concept of synonyms for suggestions
(in `pgcompleter.find_matches`). They way synonyms work is that a
number of synonyms may be provided for a suggestion sent to
`find_matches`. If synonyms are provided, sorting is based on how
well the best synonym matches the input, instead of only comparing
the input to the suggestion text.
In this case, the unqualified name acts as a synonym.
I have a couple of other ideas of use cases where we can use synonyms
to get better completions with less typing for the user, which I intend
to explore later.
Python 3 forbids comparisons between different types: use a tuple
containing a single 0 (zero) as the priority for path matches so that it
can be compared with those generated by the workhorse method find_matches().
This commit makes it so that given `INSERT INTO foo(<cursor1>) SELECT <cursor2> FROM bar;`, we suggest `bar` columns for `<cursor2>` and `foo` columns for `<cursor1>`. Previous behaviour is sugggesting columns from both tables in both cases.
When the database URL contains a port, uri.port is (at least in Python 2.7.6) an integer, not a string, so urlparse.unquote chokes on it.
Fixes issue #536, but is probably worth verifying on Python 3.
If config.main.generate_aliases is True, for `SELECT * FROM `, we suggest `FooBar FB` and `foo_bar fb` instead of `FooBar` and `foo_bar`, respectively.
To be able to add a test, I had to add support for testing with different settings, which meant I could also add tests for casing.
There are two non-obvious changes that I can think of:
1. The lexical sorting of matches is modified so as to sort spaces and underscores before letters and to sort case-insensitively. This is so that e.g `Foob F` comes before 'FooBar FB' when `foob` is input.
2. We now suggest `some_func()` instead of `some_func` (because suggesting `some_func sf` didn't make any sense).
When self-joining a table with an FK to or from some other table, we got a false FK-join suggestion for that column.
There was also a problem with quoted tables not being quoted in the join condition.
And there were a couple of problems when trying to join a non-existent table or using a non-existent qualifier (`SELECT * FROM Foo JOIN Bar ON Meow.`).
I also rewrote get_join_condition_matches a bit in the process, hopefully making it a bit simpler.
If we have the input `SELECT * FROM Foo JOIN `, we now suggest `Foo Foo2 ON Foo2.ParentID = Foo.ID` (given the appropriate casing file and FK).
There were also some problems with quoted tables and with the casing of table aliases, which are now fixed.
I also made a few cosmetic changes to get_join_matches (pgcompleter.py) just to make it a bit easier to work with.
This is based on my previous work on suggesting join conditions, but here instead we suggest the whole join. What we do is simply check all the tables in the statement for FK relationships and then suggest joins based on those. I think this will not only save key presses, but also be rather useful when exploring an unfamiliar (part of a) database.
There's one non-obvious change in this commit (that I can think of): When calling **sqlcompletion.text_before_cursor**, the **text_before_cursor** argument now no longer includes **word_before_cursor**. This is because for 'SELECT * FROM foo JOIN bar<cursor>', we would otherwise consider the table **bar** already included in the statement and thus suggest e.g. 'baz on baz.barid = bar.barid'.
This adds support for defining the casing to be used in completions (for unquoted names). The casing is determined by the casing file, which consists of a \n-separated list of names using the preferred casing. This file can either be populated manually or created automatically (based on usage in SQL/PLPGSQL functions), with a config setting.
Getting the parameters from proargnames, proallargtypes and proargmodes instead of from parsing the arg_list string simplifies FunctionMetadata quite a bit.
I also made the ColumnMetadata for table/view columns use the same format for the type (i.e. regtype instead of typname). This means we now get join-condition suggestions for joining tables/views to functions, which didn't work before.
And add a setting to use alphabetic order instead.
We simply get the columns from the database in the canonical order, and then sort them if the user has opted for alphabetic order.
The user types 'SELECT * FROM parenttable p JOIN childtable c on '. We then suggest ['c.parenttableid = p.parenttableid', 'c.createdby = p.createdby', ...] as completions.
The suggestions are FK matches first, then name matches for int columns, then name matches for other columns, sorted by proximity from the cursor to the other table.
Some changes:
For "JOIN Foo USING(<cursor>)", now only columns present in Foo are suggested.
For all suggestions after 'ON', now only the tables before the cursor are considered.
meta[reltype]][relname] goes from being a list of columns to being an OrderedDict {column_name:ColumnMetadata}, where the ColumnMetaData contains the column's name, data type and a list of FKs involving the column. This entails modification of a number of functions dealing with columns.
Move *-expansion logic from find_matches to get_column_matches
Make the expansion always qualify column names if there's more than one table. For this I changed populate_scoped_columns to return a dict{TableReference:[column names]}; I took this code from my other branch joinconditions.
Add a bunch of tests
Change from filtering out all non-set-returning functions to only filteirng out windowing functions and aggregate functions.
Non-set-returning functions are legal in the FROM clause and can be quite useful.
As you know [RFC-3986][1] allows passing "disallowed" characters
as long as these are percent encoded.
Since we deal with the url parsing, we can expect this to happen
and need to use the values after they have been decoded.
[1]: https://tools.ietf.org/html/rfc3986#section-2.1
