Both `Database` and `Heap` were allowed to be opened twice. Prevent
this, and change SQLServer to only open databases that are not already
opened.
This fixes a Ladybird crash where opening the application twice would
erroneously duplicate free heap block indices.
Otherwise the `move(result)` statement inside the lambda does not
actually move anything, because `result` is constant without the mutable
attribute. Caught by clangd.
These are currently hitting the `decltype(nullptr)` constructor, which
marks the response as invalid, resulting in no response being sent to
the waiting client.
Currently, we create a new SQL::Database object for each database we are
requested to open. When multiple clients connect to the same database,
the same underlying database file is opened and cached each time. This
results in updates from one client not being propagated to others.
To prevent this, when a database is requested to be open, check if it is
already open. We can then re-use that SQL::Database object for the new
connection.
This also allows for overriding the path. Ladybird will want to store
the database files in a subdirectory of the standard data directory that
contains the Ladybird application name.
Fixes#16000.
The sql REPL had the created/updated rows swapped by mistake. Also make
sure SQLServer fills in the correct value depending on the executed
command, and that the DELETE command indicates the rows it deleted.
We've been sending the values converted to a string, but now that the
Value type is transferrable over IPC, send the values themselves. Any
client that wants the value as a string may do so easily, whereas this
will allow less trivial clients to avoid string parsing.
If a statement is executed multiple times in quick succession, we may
overwrite the results of a previous execution. Instead of storing the
result, pass it around as it is sent to the client.
Currently, when clients connect to SQL server, we inform them of any
errors opening the database via an asynchronous IPC. But we already know
about these errors before returning from the connect() IPC, so this
roundabout propagation is a bit unnecessary. Now if we fail to open the
database, we will simply not send back a valid connection ID.
Disconnect has a similar story. Rather than disconnecting and invoking
an asynchronous IPC to inform the client of the disconnect, make the
disconnect() IPC synchronous (because all it does is remove the database
from the map of open databases). Further, the only user of this command
is the SQL REPL when it wants to connect to a different database, so it
makes sense to block it. This did require moving a bit of logic around
in the REPL to accommodate this change.
In order to execute a prepared statement multiple times, and track each
execution's results, clients will need to be provided an execution ID.
This will create a monotonically increasing ID each time a prepared
statement is executed for this purpose.
When storing IDs and sending values over IPC, this changes SQLServer to:
1. Stop using -1 as a nominal "bad" ID. Store the IDs as unsigned, and
use Optional in the one place that the IPC needs to indicate an ID
was not allocated.
2. Let LibIPC encode/decode enumerations (SQLErrorCode) on our behalf.
3. Use size_t for array sizes.
One of the benefits of prepared statements is that the SQL string is
parsed just once and re-used. This updates SQLStatement to do just that
and store the parsed result.
This will make it easier to support both string types at the same time
while we convert code, and tracking down remaining uses.
One big exception is Value::to_string() in LibJS, where the name is
dictated by the ToString AO.
We have a new, improved string type coming up in AK (OOM aware, no null
state), and while it's going to use UTF-8, the name UTF8String is a
mouthful - so let's free up the String name by renaming the existing
class.
Making the old one have an annoying name will hopefully also help with
quick adoption :^)
Database::get_schema currently either returns a RefPtr to an existing
schema, a nullptr if the schema doesn't exist, or an Error if some
internal error occured. Change this to return a NonnullRefPtr to an
exisiting schema, or a SQL::Result with any error, including if the
schema was not found. Callers can then handle that specific error code
if they want.
Returning a NonnullRefPtr will enable some further cleanup. This had
some fallout of needing to change some other methods' return types from
AK::ErrorOr to SQL::Result so that TRY may continue to be used.
Rename sql_statement to prepare_statement and statement_execute to
execute_statement. The former aligns more with other database libraries
(e.g. Java's JDBC prepareStatement). The latter reads less awkwardly.
These lambdas were marked mutable as they captured a Ptr wrapper
class by value, which then only returned const-qualified references
to the value they point from the previous const pointer operators.
Nothing is actually mutating in the lambdas state here, and now
that the Ptr operators don't add extra const qualifiers these
can be removed.
Otherwise, we end up propagating those dependencies into targets that
link against that library, which creates unnecessary link-time
dependencies.
Also included are changes to readd now missing dependencies to tools
that actually need them.
We previously put the generated headers in SOURCES, which did not mark
them as GENERATED (and did not produce a proper dependency).
This commit moves all generated headers into GENERATED_SOURCES, and
removes useless header SOURCES.
The result of a SQL statement execution is either:
1. An error.
2. The list of rows inserted, deleted, selected, etc.
(2) is currently represented by a combination of the Result class and
the ResultSet list it holds. This worked okay, but issues start to
arise when trying to use Result in non-statement contexts (for example,
when introducing Result to SQL expression execution).
What we really need is for Result to be a thin wrapper that represents
both (1) and (2), and to not have any explicit members like a ResultSet.
So this commit removes ResultSet from Result, and introduces ResultOr,
which is just an alias for AK::ErrorOrr. Statement execution now returns
ResultOr<ResultSet> instead of Result. This further opens the door for
expression execution to return ResultOr<Value> in the future.
Lastly, this moves some other context held by Result over to ResultSet.
This includes the row count (which is really just the size of ResultSet)
and the command for which the result is for.
Ordering is done by replacing the straight Vector holding the query
result in the SQLResult object with a dedicated Vector subclass that
inserts result rows according to their sort key using a binary search.
This is done in the ResultSet class.
There are limitations:
- "SELECT ... ORDER BY 1" (or 2 or 3 etc) is supposed to sort by the
n-th result column. This doesn't work yet
- "SELECT ... column-expression alias ... ORDER BY alias" is supposed to
sort by the column with the given alias. This doesn't work yet
What does work however is something like
```SELECT foo FROM bar SORT BY quux```
i.e. sorted by a column not in the result set. Once functions are
supported it should be possible to sort by random functions.
This change unfortunately cannot be atomically made without a single
commit changing everything.
Most of the important changes are in LibIPC/Connection.cpp,
LibIPC/ServerConnection.cpp and LibCore/LocalServer.cpp.
The notable changes are:
- IPCCompiler now generates the decode and decode_message functions such
that they take a Core::Stream::LocalSocket instead of the socket fd.
- IPC::Decoder now uses the receive_fd method of LocalSocket instead of
doing system calls directly on the fd.
- IPC::ConnectionBase and related classes now use the Stream API
functions.
- IPC::ServerConnection no longer constructs the socket itself; instead,
a convenience macro, IPC_CLIENT_CONNECTION, is used in place of
C_OBJECT and will generate a static try_create factory function for
the ServerConnection subclass. The subclass is now responsible for
passing the socket constructed in this function to its
ServerConnection base; the socket is passed as the first argument to
the constructor (as a NonnullOwnPtr<Core::Stream::LocalServer>) before
any other arguments.
- The functionality regarding taking over sockets from SystemServer has
been moved to LibIPC/SystemServerTakeover.cpp. The Core::LocalSocket
implementation of this functionality hasn't been deleted due to my
intention of removing this class in the near future and to reduce
noise on this (already quite noisy) PR.
This encapsulates what our multi-client IPC servers typically do on
startup:
1. Create a Core::LocalServer
2. Take over a listening socket file descriptor from SystemServer
3. Set up an accept handler for incoming connections
IPC::MultiServer does all this for you! All you have to do is provide
the relevant client connection type as a template argument.
These ones all manage their storage internally, whereas the WebContent
and ImageDecoder ones require the caller to manage their lifetime. This
distinction is not obvious to the user without looking through the code,
so an API that makes this clearer would be nice.
The handling of filesystem level errors was basically non-existing or
consisting of `VERIFY_NOT_REACHED` assertions. Addressed this by
* Adding `open` methods to `Heap` and `Database` which return errors.
* Changing the interface of methods of these classes and clients
downstream to propagate these errors.
The constructors of `Heap` and `Database` don't open the underlying
filesystem file anymore.
The SQL statement handlers return an `SQLErrorCode::InternalError`
error code if an error comes back from the lower levels. Note that some
of these errors are things like duplicate index entry errors that should
be caught before the SQL layer attempts to actually update the database.
Added tests to catch attempts to open weird or non-existent files as
databases.
Finally, in between me writing this patch and submitting the PR the
AK::Result<Foo, Bar> template got deprecated in favour of ErrorOr<Foo>.
This resulted in more busywork.
Everyone used this hook in the same way: immediately accept() on the
socket and then do something with the newly accepted fd.
This patch simplifies the hook by having LocalServer do the accepting
automatically.
Because SQL is the craptastic language that it is, sometimes expressions
need to know details about the calling statement. For example the tables
in the 'FROM' clause may be needed to determine which columns are
referenced in 'WHERE' expressions. So the current statement is added
to the ExecutionContext and a new 'execute' overload on Statement is
created which takes the Database and the Statement and builds an
ExecutionContaxt from those.
Derivatives of Core::Object should be constructed through
ClassName::construct(), to avoid handling ref-counted objects with
refcount zero. Fixing the visibility means that misuses like this are
more difficult.
The database the sql client connected to was 'hardcoded' to the login
name of the calling user.
- Extended the IPC API to be more expressive when connecting, by
returning the name of the database the client connected to in the
'connected' callback.
- Gave the sql client a command line argument (-d/--database) allowing
an alternative database name to be specified
A subsequent commit will have a dot command allowing the user to
connect to different databases from the same sql session.
If you capture a stack variable by reference in a lamdba definition,
and this lambda outlives the scope of the stack variable, this reference
may point to garbage when the lambda is executed. Therefore capture as
little as possible (typically only ``this``), and what is captured is
captured by value
Only one place used this argument and it was to hold on to a strong ref
for the object. Since we already do that now, there's no need to keep
this argument around since this can be easily captured.
This commit contains no changes.
In the DatabaseConnection constructor, there's a deferred_invoke
callback that references the client_id. But depending on when the
callback occurs, the reference of the client_id can change. This created
a problem when connecting to SQLServer using the SQL utility because
depending on when the callback was invoked, the client_id could change.
m_client_id is set in the constructor and that reference will not change
depending on when the callback is invoked.
This patch provides very basic, bare bones implementations of the
INSERT and SELECT statements. They are *very* limited:
- The only variant of the INSERT statement that currently works is
SELECT INTO schema.table (column1, column2, ....) VALUES
(value11, value21, ...), (value12, value22, ...), ...
where the values are literals.
- The SELECT statement is even more limited, and is only provided to
allow verification of the INSERT statement. The only form implemented
is: SELECT * FROM schema.table
These statements required a bit of change in the Statement::execute
API. Originally execute only received a Database object as parameter.
This is not enough; we now pass an ExecutionContext object which
contains the Database, the current result set, and the last Tuple read
from the database. This object will undoubtedly evolve over time.
This API change dragged SQLServer::SQLStatement into the patch.
Another API addition is Expression::evaluate. This method is,
unsurprisingly, used to evaluate expressions, like the values in the
INSERT statement.
Finally, a new test file is added: TestSqlStatementExecution, which
tests the currently implemented statements. As the number and flavour of
implemented statements grows, this test file will probably have to be
restructured.
This patch introduces the SQLServer system server. This service is
supposed to be the only process/application talking to database storage.
This makes things like locking and caching more reliable, easier to
implement, and more efficient.
In LibSQL we added a client component that does the ugly IPC nitty-
gritty for you. All that's needed is setting a number of event handler
lambdas and you can connect to databases and execute statements on them.
Applications that wish to use this SQLClient class obviously need to
link LibSQL and LibIPC.
