Adding new Date/Time operations (-, date_add, date_diff, date_part) (#7221)

- Adds `Column.date_diff` for computing date/time difference as integer multiply of some unit.
- Adds `Column.date_add` for shifting date/time by a unit.
- Adds `Column.date_part` for extracting various parts of the date/time value as integer.
- Adds widgets for the 3 methods above whose content depends on the column value type.
- Adds shorthands: `Column.hour`, `Column.minute` and `Column.second` to extract these date parts.
- Extends `Time_Period` with support for milli-, micro- and nano- seconds; and adapts functions taking `Time_Period` to support these wherever possible.
This commit is contained in:
Radosław Waśko 2023-07-13 14:56:54 +02:00 committed by GitHub
parent 4be2c7b65b
commit ca68dd94da
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
39 changed files with 1552 additions and 175 deletions

View File

@ -517,6 +517,9 @@
`Column`, and in-memory `Table` to take a `Regex` in addition to a `Text`.]
[7223]
- [Added `cross_join` support to database tables.][7234]
- [Improving date/time support in Table - added `date_diff`, `date_add`,
`date_part` and some shorthands. Extended `Time_Period` with milli-, micro-
and nanosecond periods.][7221]
[debug-shortcuts]:
https://github.com/enso-org/enso/blob/develop/app/gui/docs/product/shortcuts.md#debug
@ -745,6 +748,7 @@
[7174]: https://github.com/enso-org/enso/pull/7174
[7223]: https://github.com/enso-org/enso/pull/7223
[7234]: https://github.com/enso-org/enso/pull/7234
[7221]: https://github.com/enso-org/enso/pull/7221
#### Enso Compiler

View File

@ -2,27 +2,53 @@ import project.Data.Time.Date.Date
import project.Data.Time.Date_Time.Date_Time
import project.Data.Time.Day_Of_Week.Day_Of_Week
import project.Data.Time.Period.Period
import project.Error.Error
import project.Errors.Illegal_State.Illegal_State
from project.Data.Boolean import Boolean, False, True
polyglot java import java.time.temporal.ChronoUnit
polyglot java import java.time.temporal.TemporalAdjuster
polyglot java import java.time.temporal.TemporalAdjusters
polyglot java import org.enso.base.time.Date_Period_Utils
polyglot java import java.time.temporal.TemporalUnit
polyglot java import org.enso.base.Time_Utils
polyglot java import org.enso.base.time.Date_Period_Utils
polyglot java import org.enso.base.time.CustomTemporalUnits
## Represents a unit of time longer on the scale of days (longer than a day).
type Date_Period
## Represents a date period of a calendar year.
Its length in days will depend on context (accounting for leap years).
Year
## Represents a date period of a quarter - 3 calendar months.
Quarter
## Represents a date period of a month.
Its length in days will depend on context of what month it is used.
Month
## Represents a 7-day week starting at a given day.
By default, the first day of the week is Monday, but this can be adjusted
to any other day.
The starting day will be ignored for methods that just compute the time
differences. It only matters for methods that need to find a beginning or
end of a specific period (like `start_of` or `end_of`).
The `date_part` method will return the ISO 8601 week of year number,
regardless of the starting day.
Week (first_day:Day_Of_Week = Day_Of_Week.Monday)
## Represents a time period of a single calendar day.
? Daylight Saving Time
Note that due to DST changes, some days may be slightly longer or
shorter. This date period will reflect that and still count such days
as one day. For a measure of exactly 24 hours, use `Time_Period.Day`.
Day
## PRIVATE
@ -55,3 +81,12 @@ type Date_Period
Date_Period.Month -> Period.new months=1
Date_Period.Week _ -> Period.new days=7
Date_Period.Day -> Period.new days=1
## PRIVATE
to_java_unit : TemporalUnit
to_java_unit self = case self of
Date_Period.Year -> ChronoUnit.YEARS
Date_Period.Quarter -> CustomTemporalUnits.QUARTERS
Date_Period.Month -> ChronoUnit.MONTHS
Date_Period.Week _ -> ChronoUnit.WEEKS
Date_Period.Day -> ChronoUnit.DAYS

View File

@ -6,39 +6,76 @@ from project.Data.Boolean import Boolean, False, True
polyglot java import java.time.temporal.ChronoUnit
polyglot java import java.time.temporal.TemporalUnit
polyglot java import org.enso.base.Time_Utils
polyglot java import org.enso.base.time.CustomTemporalUnits
## Represents a unit of time of a day or shorter.
type Time_Period
## Represents a time period of a single day, measured as 24 hours.
? Daylight Saving Time
Note that due to DST changes, some days may be slightly longer or
shorter. This is not reflected in the duration of this time period. For
a calendar-oriented day period, use `Date_Period.Day` instead.
Day
## Represents a time period of an hour.
Hour
## Represents a time period of a minute.
Minute
## Represents a time period of a second.
Second
## Represents a time period of a millisecond.
Millisecond
## Represents a time period of a microsecond.
Microsecond
## Represents a time period of a nanosecond.
Nanosecond
## PRIVATE
We treat the `Time_Period.Day` as a period of 24 hours, not a calendar day.
to_java_unit : TemporalUnit
to_java_unit self = case self of
Time_Period.Day -> ChronoUnit.DAYS
Time_Period.Day -> CustomTemporalUnits.DAY_AS_24_HOURS
Time_Period.Hour -> ChronoUnit.HOURS
Time_Period.Minute -> ChronoUnit.MINUTES
Time_Period.Second -> ChronoUnit.SECONDS
Time_Period.Millisecond -> ChronoUnit.MILLIS
Time_Period.Microsecond -> ChronoUnit.MICROS
Time_Period.Nanosecond -> ChronoUnit.NANOS
## PRIVATE
A special case for `adjust_start` and `adjust_end` methods.
In this particular case, it seems better to treat `Time_Period.Day` as a
calendar day. Otherwise, the behaviour of `start_of` and `end_of` methods
near DST would become unintuitive.
to_java_unit_for_adjust : TemporalUnit
to_java_unit_for_adjust self = case self of
Time_Period.Day -> ChronoUnit.DAYS
_ -> self.to_java_unit
## PRIVATE
adjust_start : (Time_Of_Day | Date_Time) -> (Time_Of_Day | Date_Time)
adjust_start self date =
(Time_Utils.utils_for date).start_of_time_period date self.to_java_unit
(Time_Utils.utils_for date).start_of_time_period date self.to_java_unit_for_adjust
## PRIVATE
adjust_end : (Time_Of_Day | Date_Time) -> (Time_Of_Day | Date_Time)
adjust_end self date =
(Time_Utils.utils_for date).end_of_time_period date self.to_java_unit
(Time_Utils.utils_for date).end_of_time_period date self.to_java_unit_for_adjust
## PRIVATE
to_duration : Duration
to_duration self = case self of
Time_Period.Day -> Duration.new 24
Time_Period.Hour -> Duration.new 1
Time_Period.Minute -> Duration.new 0 1
Time_Period.Second -> Duration.new 0 0 1
Time_Period.Day -> Duration.new hours=24
Time_Period.Hour -> Duration.new hours=1
Time_Period.Minute -> Duration.new minutes=1
Time_Period.Second -> Duration.new seconds=1
Time_Period.Millisecond -> Duration.new milliseconds=1
Time_Period.Microsecond -> Duration.new nanoseconds=1000
Time_Period.Nanosecond -> Duration.new nanoseconds=1

View File

@ -7,6 +7,7 @@ import Standard.Base.Internal.Rounding_Helpers
import Standard.Table.Data.Column.Column as Materialized_Column
import Standard.Table.Data.Type.Enso_Types
import Standard.Table.Data.Type.Value_Type_Helpers
import Standard.Table.Internal.Date_Time_Helpers
import Standard.Table.Internal.Java_Problems
import Standard.Table.Internal.Problem_Builder.Problem_Builder
import Standard.Table.Internal.Widget_Helpers
@ -99,7 +100,7 @@ type Column
## Returns a vector containing all the elements in this column.
to_vector : Vector Any
to_vector self =
self.to_table.read . at self.name . to_vector
self.to_table.read . at 0 . to_vector
## Returns the `Value_Type` associated with that column.
@ -133,8 +134,9 @@ type Column
- operands: A vector of additional operation arguments (the column itself
is always passed as the first argument).
- new_name: The name of the resulting column.
make_op : Text -> Vector Text -> (Text | Nothing) -> Column
make_op self op_kind operands new_name =
- metadata: Optional metadata for the `SQL_Expression.Operation`.
make_op : Text -> Vector Text -> (Text | Nothing) -> (Any | Nothing) -> Column
make_op self op_kind operands new_name metadata=Nothing =
checked_support = if self.connection.dialect.is_supported op_kind then True else
Error.throw (Unsupported_Database_Operation.Error "The operation "+op_kind+" is not supported by this backend.")
checked_support.if_not_error <|
@ -149,7 +151,7 @@ type Column
SQL_Expression.Constant constant
expressions = operands.map prepare_operand
new_expr = SQL_Expression.Operation op_kind ([self.expression] + expressions)
new_expr = SQL_Expression.Operation op_kind ([self.expression] + expressions) metadata
infer_from_database_callback expression =
SQL_Type_Reference.new self.connection self.context expression
@ -356,7 +358,7 @@ type Column
self.make_op "BETWEEN" [lower, upper] new_name
## ALIAS Add, Plus, Concatenate
Element-wise addition.
Element-wise addition. Works on numeric types or text.
Arguments:
- other: The other column to add to this column.
@ -366,13 +368,16 @@ type Column
between corresponding elements of `self` and `other`.
+ : Column | Any -> Column
+ self other =
op = Value_Type_Helpers.resolve_addition_kind self other
op = case Value_Type_Helpers.resolve_addition_kind self other of
Value_Type_Helpers.Addition_Kind.Numeric_Add -> "ADD_NUMBER"
Value_Type_Helpers.Addition_Kind.Text_Concat -> "ADD_TEXT"
op.if_not_error <|
new_name = self.naming_helpers.binary_operation_name "+" self other
self.make_binary_op op other new_name
## ALIAS Subtract, Minus
Element-wise subtraction.
## ALIAS Subtract, Minus, Time Difference
Element-wise subtraction. Allows to subtract numeric types or compute a
difference between two date/time values.
Arguments:
- other: The other column to subtract from this column.
@ -382,8 +387,11 @@ type Column
pairwise between corresponding elements of `self` and `other`.
- : Column | Any -> Column
- self other =
Value_Type_Helpers.check_binary_numeric_op self other <|
case Value_Type_Helpers.resolve_subtraction_kind self other of
Value_Type_Helpers.Subtraction_Kind.Numeric_Subtract ->
self.make_binary_op "-" other
Value_Type_Helpers.Subtraction_Kind.Date_Time_Difference ->
Error.throw (Unsupported_Database_Operation.Error "Subtracting date/time values is not supported in this database.")
## ALIAS Multiply, Times, Product
Element-wise multiplication.
@ -1144,7 +1152,7 @@ type Column
Returns a column of `Integer` type.
year : Column ! Invalid_Value_Type
year self = Value_Type.expect_has_date self <|
self.make_unary_op "year"
simple_unary_op self "year"
## Gets the month as a number (1-12) from the date stored in the column.
@ -1152,7 +1160,7 @@ type Column
Returns a column of `Integer` type.
month : Column ! Invalid_Value_Type
month self = Value_Type.expect_has_date self <|
self.make_unary_op "month"
simple_unary_op self "month"
## Gets the day of the month as a number (1-31) from the date stored in the
column.
@ -1161,7 +1169,99 @@ type Column
Returns a column of `Integer` type.
day : Column ! Invalid_Value_Type
day self = Value_Type.expect_has_date self <|
self.make_unary_op "day"
simple_unary_op self "day"
## Gets the hour as a number (0-23) from the time stored in the column.
Applies only to columns that hold the `Time_Of_Day` or `Date_Time` types.
Returns a column of `Integer` type.
hour : Column ! Invalid_Value_Type
hour self = Value_Type.expect_has_time self <|
simple_unary_op self "hour"
## Gets the minute as a number (0-59) from the time stored in the column.
Applies only to columns that hold the `Time_Of_Day` or `Date_Time` types.
Returns a column of `Integer` type.
minute : Column ! Invalid_Value_Type
minute self = Value_Type.expect_has_time self <|
simple_unary_op self "minute"
## Gets the second as an integer (0-60) from the time stored in the column.
Applies only to columns that hold the `Time_Of_Day` or `Date_Time` types.
Returns a column of `Integer` type.
second : Column ! Invalid_Value_Type
second self = Value_Type.expect_has_time self <|
simple_unary_op self "second"
## Gets the date part of the date/time value.
Returns a column of `Integer` type.
@period Date_Time_Helpers.make_period_selector_for_column
date_part : Date_Period | Time_Period -> Column ! Invalid_Value_Type
date_part self period =
Date_Time_Helpers.make_date_part_function self period simple_unary_op self.naming_helpers
## Computes a time difference between the two dates.
It returns a column of integers expressing how many periods fit between
the two dates/times.
The difference will be positive if `end` is greater than `self`.
Arguments:
- end: A date/time column or a date/time value to compute the difference
from. It should have the same type as the current column, i.e. a
`Date_Time` column cannot be compared to a `Date` - to do so you first
need to `cast`.
- period: The period to compute the difference in. For `Date` columns it
should be a `Date_Period` and for `Time` columns it should be a
`Time_Period`. For `Date_Time` columns it can be either.
? Time Zone handling
Some backends may not preserve the timezone data in a `Date_Time`
(preserving the represented time instant). This may lead to slight
differences in time calculations between backends, especially around
unusual events like DST.
@period Date_Time_Helpers.make_period_selector_for_column
date_diff : (Column | Date | Date_Time | Time_Of_Day) -> Date_Period | Time_Period -> Column
date_diff self end (period : Date_Period | Time_Period) =
Value_Type.expect_type self .is_date_or_time "date/time" <|
my_type = self.inferred_precise_value_type
Value_Type.expect_type end (== my_type) my_type.to_display_text <|
Date_Time_Helpers.check_period_aligned_with_value_type my_type period <|
new_name = self.naming_helpers.function_name "date_diff" [self, end, period.to_display_text]
metadata = self.connection.dialect.prepare_metadata_for_period period my_type
self.make_op "date_diff" [end] new_name metadata
## Shifts the date/time by a specified period, returning a new date/time
column of the same type.
Arguments:
- amount: An integer or integer column specifying by how many periods to
shift each date.
- period: The period by which to shift. For `Date` columns it should be a
`Date_Period` and for `Time` columns it should be a `Time_Period`. For
`Date_Time` columns it can be either.
? Time Zone handling
Some backends may not preserve the timezone data in a `Date_Time`
(preserving the represented time instant). This may lead to slight
differences in time calculations between backends, especially around
unusual events like DST.
@period Date_Time_Helpers.make_period_selector_for_column
date_add : (Column | Integer) -> Date_Period | Time_Period -> Column
date_add self amount (period : Date_Period | Time_Period) =
Value_Type.expect_type self .is_date_or_time "date/time" <|
my_type = self.inferred_precise_value_type
Value_Type.expect_integer amount <|
Date_Time_Helpers.check_period_aligned_with_value_type my_type period <|
new_name = self.naming_helpers.function_name "date_add" [self, amount, period.to_display_text]
metadata = self.connection.dialect.prepare_metadata_for_period period my_type
self.make_op "date_add" [amount] new_name metadata
## Checks for each element of the column if it is contained within the
provided vector or column.
@ -1471,3 +1571,8 @@ adapt_unified_column column expected_type =
SQL_Type_Reference.new column.connection column.context expression
adapted = dialect.adapt_unified_column column.as_internal expected_type infer_return_type
Column.Value name=column.name connection=column.connection sql_type_reference=adapted.sql_type_reference expression=adapted.expression context=column.context
## PRIVATE
A shorthand to be able to share the implementations between in-memory and
database.
simple_unary_op column op_kind = column.make_unary_op op_kind

View File

@ -223,6 +223,14 @@ type Dialect
_ = [connection, table_name]
Unimplemented.throw "This is an interface only."
## PRIVATE
Prepares metadata for an operation taking a date/time period and checks
if the given period is supported.
prepare_metadata_for_period : Date_Period | Time_Period -> Value_Type -> Any
prepare_metadata_for_period self period operation_input_type =
_ = [period, operation_input_type]
Unimplemented.throw "This is an interface only."
## PRIVATE
The dialect of SQLite databases.

View File

@ -45,6 +45,7 @@ import project.Internal.Helpers
import project.Internal.IR.Context.Context
import project.Internal.IR.From_Spec.From_Spec
import project.Internal.IR.Internal_Column.Internal_Column
import project.Internal.IR.Operation_Metadata
import project.Internal.IR.Order_Descriptor.Order_Descriptor
import project.Internal.IR.Query.Query
import project.Internal.IR.SQL_Expression.SQL_Expression
@ -580,11 +581,11 @@ type Table
descriptors -> descriptors
grouping_expressions = grouping_columns.map .expression
separator = SQL_Expression.Literal Base_Generator.row_number_parameter_separator
# The SQL row_number() counts from 1, so we adjust the offset.
offset = from - step
params = [SQL_Expression.Constant offset, SQL_Expression.Constant step] + order_descriptors + [separator] + grouping_expressions
new_expr = SQL_Expression.Operation "ROW_NUMBER" params
params = [SQL_Expression.Constant offset, SQL_Expression.Constant step] + order_descriptors + grouping_expressions
metadata = Operation_Metadata.Row_Number_Metadata.Value grouping_expressions.length
new_expr = SQL_Expression.Operation "ROW_NUMBER" params metadata
type_mapping = self.connection.dialect.get_type_mapping
infer_from_database_callback expression =

View File

@ -55,7 +55,7 @@ make_aggregate_column table aggregate new_name dialect infer_return_type problem
Count_Empty c _ -> simple_aggregate "COUNT_EMPTY" [c]
Percentile p c _ ->
op_kind = "PERCENTILE"
expression = SQL_Expression.Operation op_kind [SQL_Expression.Constant p, c.expression]
expression = SQL_Expression.Operation op_kind [SQL_Expression.Literal p.to_text, c.expression]
sql_type_ref = infer_return_type op_kind [c] expression
Internal_Column.Value new_name sql_type_ref expression
Mode c _ ->

View File

@ -11,6 +11,7 @@ import project.Internal.IR.Query.Query
import project.Internal.IR.SQL_Expression.SQL_Expression
import project.Internal.IR.SQL_Join_Kind.SQL_Join_Kind
from project.Errors import Unsupported_Database_Operation
from project.Internal.IR.Operation_Metadata import Row_Number_Metadata
type Internal_Dialect
@ -21,8 +22,9 @@ type Internal_Dialect
Arguments:
- operation_map: The mapping which maps operation names to their
implementations; each implementation is a function which takes SQL
builders for the arguments and should return a builder yielding the
whole operation.
builders for the arguments, and optionally an additional metadata
argument, and should return a builder yielding code for the whole
operation.
- wrap_identifier: A function that converts an arbitrary supported
identifier name in such a way that it can be used in the query; that
usually consists of wrapping the name in quotes and escaping any quotes
@ -267,37 +269,18 @@ make_is_in_column arguments = case arguments.length of
## PRIVATE
make_row_number : Vector Builder -> Builder
make_row_number arguments = if arguments.length < 4 then Error.throw (Illegal_State.Error "Wrong amount of parameters in ROW_NUMBER IR. This is a bug in the Database library.") else
make_row_number arguments (metadata : Row_Number_Metadata) = if arguments.length < 3 then Error.throw (Illegal_State.Error "Wrong amount of parameters in ROW_NUMBER IR. This is a bug in the Database library.") else
offset = arguments.at 0
step = arguments.at 1
separator_ix = arguments.index_of code->
code.build.prepare.first == row_number_parameter_separator
ordering = arguments.take (Range.new 2 separator_ix)
grouping = arguments.drop (separator_ix+1)
ordering_and_grouping = arguments.drop 2
ordering = ordering_and_grouping.drop (Last metadata.groupings_count)
grouping = ordering_and_grouping.take (Last metadata.groupings_count)
group_part = if grouping.length == 0 then "" else
Builder.code "PARTITION BY " ++ Builder.join ", " grouping
Builder.code "(row_number() OVER (" ++ group_part ++ " ORDER BY " ++ Builder.join ", " ordering ++ ") * " ++ step.paren ++ " + " ++ offset.paren ++ ")"
## PRIVATE
This is a terrible hack, but I could not figure a decent way to have an
operation take a variable number of arguments of multiple kinds (here both
groups and orders are varying).
Currently, the IR just allows to put a list of parameters for the operation
and they are all converted into SQL code before being passed to the
particular operation builder. So at this stage there is no way to distinguish
the arguments.
So to distinguish different groups of arguments, we use this 'fake' parameter
to act as a separator. This parameter is not supposed to end up in the
generated SQL code.
This is yet another argument for the IR redesign.
row_number_parameter_separator =
"--<!PARAMETER_SEPARATOR!>--"
## PRIVATE
Builds code for an expression.
@ -311,10 +294,14 @@ generate_expression dialect expr = case expr of
dialect.wrap_identifier origin ++ '.' ++ dialect.wrap_identifier name
SQL_Expression.Constant value -> Builder.interpolation value
SQL_Expression.Literal value -> Builder.code value
SQL_Expression.Operation kind arguments ->
SQL_Expression.Operation kind arguments metadata ->
op = dialect.operation_map.get kind (Error.throw <| Unsupported_Database_Operation.Error kind)
parsed_args = arguments.map (generate_expression dialect)
op parsed_args
result = op parsed_args
# If the function expects more arguments, we pass the metadata as the last argument.
case result of
_ : Function -> result metadata
_ -> result
query : Query -> generate_query dialect query
descriptor : Order_Descriptor -> generate_order dialect descriptor

View File

@ -0,0 +1,13 @@
from Standard.Base import all
import Standard.Table.Data.Type.Value_Type.Value_Type
## PRIVATE
type Row_Number_Metadata
## PRIVATE
Value groupings_count:Integer
## PRIVATE
type Date_Period_Metadata
## PRIVATE
Value (period : Date_Period | Time_Period) (input_value_type : Value_Type)

View File

@ -47,4 +47,8 @@ type SQL_Expression
dialect.
- expressions: a list of expressions which are arguments to the operation
different operations support different amounts of arguments.
Operation (kind : Text) (expressions : Vector SQL_Expression)
- metadata: additional metadata tied to the operation. This will be
`Nothing` for most operations, but some operations that need to be
parametrized by additional settings can use this field to pass that
information to the code generator.
Operation (kind : Text) (expressions : Vector SQL_Expression) (metadata : Any | Nothing = Nothing)

View File

@ -37,6 +37,7 @@ import project.Internal.SQL_Type_Mapping.SQL_Type_Mapping
import project.Internal.SQL_Type_Reference.SQL_Type_Reference
import project.Internal.Statement_Setter.Statement_Setter
from project.Errors import SQL_Error, Unsupported_Database_Operation
from project.Internal.IR.Operation_Metadata import Date_Period_Metadata
## PRIVATE
@ -227,6 +228,17 @@ type Postgres_Dialect
fetch_primary_key self connection table_name =
Dialect.default_fetch_primary_key connection table_name
## PRIVATE
Prepares metadata for an operation taking a date/time period and checks
if the given period is supported.
prepare_metadata_for_period : Date_Period | Time_Period -> Value_Type -> Any
prepare_metadata_for_period self period operation_input_type =
case period of
Time_Period.Nanosecond ->
Error.throw (Unsupported_Database_Operation.Error "Postgres backend does not support nanosecond precision in date/time operations.")
_ ->
Date_Period_Metadata.Value period operation_input_type
## PRIVATE
make_internal_generator_dialect =
cases = [["LOWER", Base_Generator.make_function "LOWER"], ["UPPER", Base_Generator.make_function "UPPER"]]
@ -238,7 +250,7 @@ make_internal_generator_dialect =
stddev_pop = ["STDDEV_POP", Base_Generator.make_function "stddev_pop"]
stddev_samp = ["STDDEV_SAMP", Base_Generator.make_function "stddev_samp"]
stats = [agg_median, agg_mode, agg_percentile, stddev_pop, stddev_samp]
date_ops = [make_extract_as_int "year" "YEAR", make_extract_as_int "month" "MONTH", make_extract_as_int "day" "DAY"]
date_ops = [make_extract_as_int "year", make_extract_as_int "quarter", make_extract_as_int "month", make_extract_as_int "week", make_extract_as_int "day", make_extract_as_int "hour", make_extract_as_int "minute", make_extract_fractional_as_int "second", make_extract_fractional_as_int "millisecond" modulus=1000, make_extract_fractional_as_int "microsecond" modulus=1000, ["date_add", make_date_add], ["date_diff", make_date_diff]]
special_overrides = [is_null, is_empty]
my_mappings = text + counts + stats + first_last_aggregators + arith_extensions + bool + date_ops + special_overrides
Base_Generator.base_dialect . extend_with my_mappings
@ -481,10 +493,130 @@ decimal_mod = Base_Generator.lift_binary_op "DECIMAL_MOD" x-> y->
x ++ " - FLOOR(CAST(" ++ x ++ " AS decimal) / CAST(" ++ y ++ " AS decimal)) * " ++ y
## PRIVATE
make_extract_as_int enso_name sql_name =
make_extract_as_int enso_name sql_name=enso_name =
Base_Generator.lift_unary_op enso_name arg->
extract = Builder.code "EXTRACT(" ++ sql_name ++ " FROM " ++ arg ++ ")"
Builder.code "CAST(" ++ extract ++ " AS integer)"
as_int32 <| Builder.code "EXTRACT(" ++ sql_name ++ " FROM " ++ arg ++ ")"
## PRIVATE
make_extract_fractional_as_int enso_name sql_name=enso_name modulus=Nothing =
Base_Generator.lift_unary_op enso_name arg->
result = as_int32 <| Builder.code "TRUNC(EXTRACT(" ++ sql_name ++ " FROM " ++ arg ++ "))"
case modulus of
Nothing -> result
_ : Integer ->
(result ++ (" % "+modulus.to_text)).paren
## PRIVATE
make_date_add arguments (metadata : Date_Period_Metadata) =
if arguments.length != 2 then Error.throw (Illegal_State.Error "date_add expects exactly 2 sub expressions. This is a bug in Database library.") else
expr = arguments.at 0
amount = arguments.at 1
interval_arg = case metadata.period of
Date_Period.Year ->
"years=>1"
Date_Period.Quarter ->
"months=>3"
Date_Period.Month ->
"months=>1"
Date_Period.Week _ ->
"weeks=>1"
Date_Period.Day ->
"days=>1"
Time_Period.Day ->
"hours=>24"
Time_Period.Hour ->
"hours=>1"
Time_Period.Minute ->
"mins=>1"
Time_Period.Second ->
"secs=>1"
Time_Period.Millisecond ->
"secs=>0.001"
Time_Period.Microsecond ->
"secs=>0.000001"
interval_expression = Builder.code "make_interval(" ++ interval_arg ++ ")"
shifted = Builder.code "(" ++ expr ++ " + (" ++ amount ++ " * " ++ interval_expression ++ "))"
case metadata.input_value_type of
Value_Type.Date ->
Builder.code "(" ++ shifted ++ "::date)"
_ -> shifted
## PRIVATE
make_date_diff arguments (metadata : Date_Period_Metadata) =
if arguments.length != 2 then Error.throw (Illegal_State.Error "date_diff expects exactly 2 sub expressions. This is a bug in Database library.") else
start = arguments.at 0
end = arguments.at 1
truncate expr =
Builder.code "TRUNC(" ++ expr ++ ")"
# `age` computes a 'symbolic' difference expressed in years, months and days.
extract_years =
as_int32 <| Builder.code "EXTRACT(YEARS FROM age(" ++ end ++ ", " ++ start ++ "))"
# To get total months, we need to sum up with whole years.
extract_months =
months = as_int32 <|
Builder.code "EXTRACT(MONTHS FROM age(" ++ end ++ ", " ++ start ++ "))"
Builder.code "(" ++ extract_years ++ " * 12 + " ++ months ++ ")"
## To get total days, we cannot use `age`, because we cannot convert an
amount of months to days (month lengths vary). Instead we rely on `-`
returning an interval based in 'raw' days.
extract_days =
as_int32 <| case metadata.input_value_type of
## For pure 'date' datatype, the difference is a simple integer
count of days.
Value_Type.Date -> (end ++ " - " ++ start).paren
# For others, it is an interval, so we need to extract.
_ -> Builder.code "EXTRACT(DAYS FROM (" ++ end ++ " - " ++ start ++ "))"
## We round the amount of seconds towards zero, as we only count full
elapsed seconds in the interval.
Note that it is important the interval is computed using `-`. The
symbolic `age` has no clear mapping to the count of days, skewing the
result.
extract_seconds =
seconds_numeric = Builder.code "EXTRACT(EPOCH FROM (" ++ end ++ " - " ++ start ++ "))"
as_int64 (truncate seconds_numeric)
case metadata.period of
Date_Period.Year -> extract_years
Date_Period.Month -> extract_months
Date_Period.Quarter -> (extract_months ++ " / 3").paren
Date_Period.Week _ -> (extract_days ++ " / 7").paren
Date_Period.Day -> extract_days
## EXTRACT HOURS/MINUTES would yield only a date part, but we need
the total which is easiest achieved by EPOCH
Time_Period.Hour -> (extract_seconds ++ " / 3600").paren
Time_Period.Minute -> (extract_seconds ++ " / 60").paren
Time_Period.Second -> extract_seconds
Time_Period.Day -> case metadata.input_value_type of
Value_Type.Date -> extract_days
# Time_Period.Day is treated as 24 hours, so for types that support time we use the same algorithm like for hours, but divide by 24.
_ -> (extract_seconds ++ " / (3600 * 24)").paren
## The EPOCH gives back just the integer amount of seconds, without
the fractional part. So we get the fractional part using
MILLISECONDS - but that does not give the _total_ just the
'seconds of minute' part, expressed in milliseconds. So we need
to merge both - but then seconds of minute appear twice, so we %
the milliseconds to get just the fractional part from it and sum
both.
Time_Period.Millisecond ->
millis = truncate <|
Builder.code "EXTRACT(MILLISECONDS FROM (" ++ end ++ " - " ++ start ++ "))"
as_int64 <|
((extract_seconds ++ " * 1000").paren ++ " + " ++ (millis ++ " % 1000").paren).paren
Time_Period.Microsecond ->
micros = Builder.code "EXTRACT(MICROSECONDS FROM (" ++ end ++ " - " ++ start ++ "))"
as_int64 <|
((extract_seconds ++ " * 1000000").paren ++ " + " ++ (micros ++ " % 1000000").paren).paren
## PRIVATE
Alters the expression casting the value to a 64-bit integer.
as_int64 expr =
Builder.code "(" ++ expr ++ "::int8)"
## PRIVATE
Alters the expression casting the value to a 32-bit integer (the default choice for integers in Postgres).
as_int32 expr =
Builder.code "(" ++ expr ++ "::int4)"
## PRIVATE
postgres_statement_setter = Statement_Setter.default

View File

@ -257,6 +257,14 @@ type SQLite_Dialect
v = info_table.filter "pk" (>0) . order_by "pk" . at "name" . to_vector
if v.is_empty then Nothing else v
## PRIVATE
Prepares metadata for an operation taking a date/time period and checks
if the given period is supported.
prepare_metadata_for_period : Date_Period | Time_Period -> Value_Type -> Any
prepare_metadata_for_period self period operation_input_type =
_ = [period, operation_input_type]
Error.throw (Unsupported_Database_Operation.Error "SQLite backend does not support date/time operations.")
## PRIVATE
make_internal_generator_dialect =
text = [starts_with, contains, ends_with, make_case_sensitive]+concat_ops+trim_ops

View File

@ -13,6 +13,7 @@ import project.Data.Type.Storage
import project.Data.Type.Value_Type_Helpers
import project.Internal.Cast_Helpers
import project.Internal.Column_Ops
import project.Internal.Date_Time_Helpers
import project.Internal.Java_Problems
import project.Internal.Naming_Helpers.Naming_Helpers
import project.Internal.Parse_Values_Helper
@ -23,6 +24,7 @@ from project.Errors import Conversion_Failure, Floating_Point_Equality, Inexact_
from project.Internal.Column_Format import all
from project.Internal.Java_Exports import make_date_builder_adapter, make_double_builder, make_long_builder, make_string_builder
polyglot java import org.enso.base.Time_Utils
polyglot java import org.enso.table.data.column.operation.map.MapOperationProblemBuilder
polyglot java import org.enso.table.data.column.storage.Storage as Java_Storage
polyglot java import org.enso.table.data.table.Column as Java_Column
@ -358,7 +360,7 @@ type Column
result.rename new_name
## ALIAS Add, Plus, Concatenate
Element-wise addition.
Element-wise addition. Works on numeric types or text.
Arguments:
- other: The value to add to `self`. If `other` is a column, the addition
@ -387,7 +389,8 @@ type Column
run_vectorized_binary_op self '+' fallback_fn=Nothing other
## ALIAS Subtract, Minus
Element-wise subtraction.
Element-wise subtraction. Allows to subtract numeric types or compute a
difference between two date/time values.
Arguments:
- other: The value to subtract from `self`. If `other` is a column, the
@ -412,7 +415,21 @@ type Column
example_minus = Examples.integer_column - 10
- : Column | Any -> Column
- self other =
Value_Type_Helpers.check_binary_numeric_op self other <|
case Value_Type_Helpers.resolve_subtraction_kind self other of
Value_Type_Helpers.Subtraction_Kind.Numeric_Subtract ->
run_vectorized_binary_op self '-' fallback_fn=Nothing other
Value_Type_Helpers.Subtraction_Kind.Date_Time_Difference ->
case self.inferred_precise_value_type of
Value_Type.Date ->
## Special handling for `Period` since it is hard to
vectorize as there is no polyglot handling of it - we
_wrap_ a Java object and the Java wrapper will always
be an Enso callback, at which point its better to do
the whole operation in pure Enso.
fn = x-> y-> Period.between y x
new_name = Naming_Helpers.binary_operation_name "-" self other
run_binary_op self other fn new_name
_ ->
run_vectorized_binary_op self '-' fallback_fn=Nothing other
## ALIAS Multiply, Times, Product
@ -1239,6 +1256,108 @@ type Column
day self = Value_Type.expect_has_date self <|
simple_unary_op self "day"
## Gets the hour as a number (0-23) from the time stored in the column.
Applies only to columns that hold the `Time_Of_Day` or `Date_Time` types.
Returns a column of `Integer` type.
hour : Column ! Invalid_Value_Type
hour self = Value_Type.expect_has_time self <|
simple_unary_op self "hour"
## Gets the minute as a number (0-59) from the time stored in the column.
Applies only to columns that hold the `Time_Of_Day` or `Date_Time` types.
Returns a column of `Integer` type.
minute : Column ! Invalid_Value_Type
minute self = Value_Type.expect_has_time self <|
simple_unary_op self "minute"
## Gets the second as an integer (0-60) from the time stored in the column.
Applies only to columns that hold the `Time_Of_Day` or `Date_Time` types.
Returns a column of `Integer` type.
second : Column ! Invalid_Value_Type
second self = Value_Type.expect_has_time self <|
simple_unary_op self "second"
## Gets the date part of the date/time value.
Returns a column of `Integer` type.
@period Date_Time_Helpers.make_period_selector_for_column
date_part : Date_Period | Time_Period -> Column ! Invalid_Value_Type
date_part self period =
Date_Time_Helpers.make_date_part_function self period simple_unary_op Naming_Helpers
## Computes a time difference between the two dates.
It returns a column of integers expressing how many periods fit between
the two dates/times.
The difference will be positive if `end` is greater than `self`.
Arguments:
- end: A date/time column or a date/time value to compute the difference
from. It should have the same type as the current column, i.e. a
`Date_Time` column cannot be compared to a `Date` - to do so you first
need to `cast`.
- period: The period to compute the difference in. For `Date` columns it
should be a `Date_Period` and for `Time` columns it should be a
`Time_Period`. For `Date_Time` columns it can be either.
? Time Zone handling
Some backends may not preserve the timezone data in a `Date_Time`
(preserving the represented time instant). This may lead to slight
differences in time calculations between backends, especially around
unusual events like DST.
@period Date_Time_Helpers.make_period_selector_for_column
date_diff : (Column | Date | Date_Time | Time_Of_Day) -> Date_Period | Time_Period -> Column
date_diff self end (period : Date_Period | Time_Period) =
Value_Type.expect_type self .is_date_or_time "date/time" <|
my_type = self.inferred_precise_value_type
Value_Type.expect_type end (== my_type) my_type.to_display_text <|
Date_Time_Helpers.check_period_aligned_with_value_type my_type period <|
new_name = Naming_Helpers.function_name "date_diff" [self, end, period.to_display_text]
java_unit = period.to_java_unit
fn = case my_type of
Value_Type.Date_Time _ ->
start-> end-> Time_Utils.unit_datetime_difference java_unit start end
Value_Type.Date ->
start-> end-> Time_Utils.unit_date_difference java_unit start end
Value_Type.Time ->
start-> end-> Time_Utils.unit_time_difference java_unit start end
run_binary_op self end fn new_name
## Shifts the date/time by a specified period, returning a new date/time
column of the same type.
Arguments:
- amount: An integer or integer column specifying by how many periods to
shift each date.
- period: The period by which to shift. For `Date` columns it should be a
`Date_Period` and for `Time` columns it should be a `Time_Period`. For
`Date_Time` columns it can be either.
? Time Zone handling
Some backends may not preserve the timezone data in a `Date_Time`
(preserving the represented time instant). This may lead to slight
differences in time calculations between backends, especially around
unusual events like DST.
@period Date_Time_Helpers.make_period_selector_for_column
date_add : (Column | Integer) -> Date_Period | Time_Period -> Column
date_add self amount (period : Date_Period | Time_Period) =
Value_Type.expect_type self .is_date_or_time "date/time" <|
my_type = self.inferred_precise_value_type
Value_Type.expect_integer amount <|
Date_Time_Helpers.check_period_aligned_with_value_type my_type period <|
new_name = Naming_Helpers.function_name "date_add" [self, amount, period.to_display_text]
java_unit = period.to_java_unit
fn date amount =
java_unit.addTo date amount
run_binary_op self amount fn new_name
## Checks for each element of the column if it is contained within the
provided vector or column.
@ -1971,6 +2090,15 @@ run_vectorized_binary_op column name fallback_fn operand expected_result_type=No
Problem_Behavior.Report_Warning.attach_problems_after result <|
Java_Problems.parse_aggregated_problems problem_builder.getProblems
## PRIVATE
Runs a binary operation over the provided column and operand which may be
another column or a scalar value.
run_binary_op column operand function new_name =
new_column = case operand of
_ : Column -> column.zip operand function
_ -> column.map (function _ operand)
new_column.rename new_name
## PRIVATE
Executes a vectorized binary operation over the provided column.

View File

@ -188,6 +188,23 @@ type Value_Type
Value_Type.Date_Time _ -> True
_ -> False
## Checks if the `Value_Type` represents a type that holds a time of day.
It will return true for both `Time_Of_Day` and `Date_Time` types.
has_time : Boolean
has_time self = case self of
Value_Type.Time -> True
Value_Type.Date_Time _ -> True
_ -> False
## Checks if the `Value_Type` represents a date/time type.
is_date_or_time : Boolean
is_date_or_time self = case self of
Value_Type.Date -> True
Value_Type.Date_Time _ -> True
Value_Type.Time -> True
_ -> False
## PRIVATE
Specifies if values of the given type can be compared for ordering.
has_ordering : Boolean
@ -283,7 +300,7 @@ type Value_Type
a text type and runs the following action or reports a type error.
expect_text : Any -> Any -> Any ! Invalid_Value_Type
expect_text argument ~action =
expect_type argument .is_text "Char" action
Value_Type.expect_type argument .is_text "Char" action
## PRIVATE
ADVANCED
@ -291,7 +308,7 @@ type Value_Type
a text type and runs the following action or reports a type error.
expect_boolean : Any -> Any -> Any ! Invalid_Value_Type
expect_boolean argument ~action =
expect_type argument .is_boolean Value_Type.Boolean action
Value_Type.expect_type argument .is_boolean Value_Type.Boolean action
## PRIVATE
ADVANCED
@ -299,7 +316,7 @@ type Value_Type
a numeric type and runs the following action or reports a type error.
expect_numeric : Any -> Any -> Any ! Invalid_Value_Type
expect_numeric argument ~action =
expect_type argument .is_numeric "a numeric" action
Value_Type.expect_type argument .is_numeric "a numeric" action
## PRIVATE
ADVANCED
@ -308,15 +325,42 @@ type Value_Type
error.
expect_floating_point : Any -> Any -> Any ! Invalid_Value_Type
expect_floating_point argument ~action =
expect_type argument .is_floating_point "Float" action
Value_Type.expect_type argument .is_floating_point "Float" action
## PRIVATE
ADVANCED
Checks if the provided argument (which may be a value or a Column) is has
Checks if the provided argument (which may be a value or a Column) is of
an integer type and runs the following action or reports a type error.
expect_integer : Any -> Any -> Any ! Invalid_Value_Type
expect_integer argument ~action =
Value_Type.expect_type argument .is_integer "Integer" action
## PRIVATE
ADVANCED
Checks if the provided argument (which may be a value or a Column) has
type `Date` or `Date_Time`.
expect_has_date : Any -> Any -> Any ! Invalid_Value_Type
expect_has_date argument ~action =
expect_type argument .has_date "Date or Date_Time" action
Value_Type.expect_type argument .has_date "Date or Date_Time" action
## PRIVATE
ADVANCED
Checks if the provided argument (which may be a value or a Column) has
type `Time_Of_Day` or `Date_Time`.
expect_has_time : Any -> Any -> Any ! Invalid_Value_Type
expect_has_time argument ~action =
Value_Type.expect_type argument .has_time "Time_Of_Day or Date_Time" action
## PRIVATE
A helper for generating the `Value_Type.expect_` checks.
expect_type : Any -> (Value_Type -> Boolean) -> Text|Value_Type -> Any -> Any ! Invalid_Value_Type
expect_type value predicate type_kind ~action = case value of
# Special handling for `Nothing`. Likely, can be removed with #6281.
Nothing -> action
_ ->
typ = Value_Type_Helpers.find_argument_type value
if predicate typ then action else
Value_Type_Helpers.raise_unexpected_type type_kind value
## PRIVATE
Provides a text representation of the `Value_Type` meant for
@ -373,14 +417,3 @@ type Value_Type
- otherwise, `Text` is chosen as a fallback and the column is kept as-is
without parsing.
type Auto
## PRIVATE
A helper for generating the `Value_Type.expect_` checks.
expect_type : Any -> (Value_Type -> Boolean) -> Text|Value_Type -> Any -> Any ! Invalid_Value_Type
expect_type value predicate type_kind ~action = case value of
# Special handling for `Nothing`. Likely, can be removed with #6281.
Nothing -> action
_ ->
typ = Value_Type_Helpers.find_argument_type value
if predicate typ then action else
Value_Type_Helpers.raise_unexpected_type type_kind value

View File

@ -94,6 +94,14 @@ find_argument_type value = if Nothing == value then Nothing else
col_type = value.value_type
if col_type == Value_Type.Mixed then value.inferred_precise_value_type else col_type
## PRIVATE
type Addition_Kind
## PRIVATE
Numeric_Add
## PRIVATE
Text_Concat
## PRIVATE
A helper which resolves if numeric addition or string concatenation should be
used when the a `+` operator is used with the two provided types.
@ -101,12 +109,36 @@ find_argument_type value = if Nothing == value then Nothing else
resolve_addition_kind arg1 arg2 =
type_1 = find_argument_type arg1
type_2 = find_argument_type arg2
if type_1.is_numeric && (type_2.is_nothing || type_2.is_numeric) then 'ADD_NUMBER' else
if type_1.is_text && (type_2.is_nothing || type_2.is_text) then 'ADD_TEXT' else
if type_1.is_numeric && (type_2.is_nothing || type_2.is_numeric) then Addition_Kind.Numeric_Add else
if type_1.is_text && (type_2.is_nothing || type_2.is_text) then Addition_Kind.Text_Concat else
Error.throw <| Illegal_Argument.Error <|
if type_2.is_nothing then "Cannot perform addition on a value of type " + type_1.to_display_text + ". Addition can only be performed if the column is of some numeric type or is text." else
"Cannot perform addition on a pair of values of types " + type_1.to_display_text + " and " + type_2.to_display_text + ". Addition can only be performed if both columns are of some numeric type or are both are text."
## PRIVATE
type Subtraction_Kind
## PRIVATE
Numeric_Subtract
## PRIVATE
Date_Time_Difference
## PRIVATE
A helper which resolves if numeric subtraction or date-time difference should
be used when the a `-` operator is used with the two provided types.
It will return an error if the provided types are incompatible.
resolve_subtraction_kind arg1 arg2 =
type_1 = find_argument_type arg1
type_2 = find_argument_type arg2
if type_1.is_numeric && (type_2.is_nothing || type_2.is_numeric) then Subtraction_Kind.Numeric_Subtract else
case type_1.is_date_or_time of
True ->
if type_2.is_nothing || (type_2 == type_1) then Subtraction_Kind.Date_Time_Difference else
raise_unexpected_type type_1 arg2
False ->
raise_unexpected_type "numeric or date/time" arg1
## PRIVATE
Checks that both provided arguments have numeric type and runs the action
if they do.

View File

@ -0,0 +1,66 @@
from Standard.Base import all
import Standard.Base.Errors.Illegal_Argument.Illegal_Argument
import Standard.Base.Metadata.Display
import Standard.Base.Metadata.Widget
from Standard.Base.Metadata.Choice import Option
from Standard.Base.Metadata.Widget import Single_Choice
import project.Data.Column.Column
import project.Data.Type.Value_Type.Value_Type
## PRIVATE
check_period_aligned_with_value_type value_type period ~action = case value_type of
Value_Type.Date ->
## We don't 'officially' allow `Time_Period` for Date, but since
`Time_Period.Day` and `Date_Period.Day` in this context can be
interchangeable, we allow it as an exception.
if (period.is_a Date_Period) || (period == Time_Period.Day) then action else
Error.throw (Illegal_Argument.Error "`Time_Period` is not allowed for Date columns. Use `Date_Period`.")
Value_Type.Time ->
case period of
_ : Date_Period ->
Error.throw (Illegal_Argument.Error "`Date_Period` is not allowed for Time columns. Use `Time_Period`.")
Time_Period.Day ->
Error.throw (Illegal_Argument.Error "`Time_Period.Day` does not make sense for Time columns.")
_ -> action
Value_Type.Date_Time _ ->
## Both kinds are allowed for `Date_Time` columns.
action
## PRIVATE
Common logic for `Column.date_part`.
make_date_part_function column period make_unary_op naming_helpers =
Value_Type.expect_type column .is_date_or_time "date/time" <|
my_type = column.inferred_precise_value_type
check_period_aligned_with_value_type my_type period <|
new_name = naming_helpers.function_name "date_part" [column, period]
result = case period of
Date_Period.Year -> make_unary_op column "year"
Date_Period.Quarter -> make_unary_op column "quarter"
Date_Period.Month -> make_unary_op column "month"
Date_Period.Week _ -> make_unary_op column "week"
Date_Period.Day -> make_unary_op column "day"
Time_Period.Day -> make_unary_op column "day"
Time_Period.Hour -> make_unary_op column "hour"
Time_Period.Minute -> make_unary_op column "minute"
Time_Period.Second -> make_unary_op column "second"
Time_Period.Millisecond -> make_unary_op column "millisecond"
Time_Period.Microsecond -> make_unary_op column "microsecond"
Time_Period.Nanosecond -> make_unary_op column "nanosecond"
result.rename new_name
## PRIVATE
make_period_selector_for_column : Column -> Widget
make_period_selector_for_column column =
column_type = column.inferred_precise_value_type
date_periods = ["Year", "Quarter", "Month", "Week", "Day"].map name->
Option name "Date_Period."+name
time_periods = ["Hour", "Minute", "Second", "Millisecond", "Microsecond", "Nanosecond"].map name->
Option name "Time_Period."+name
values = case column_type of
Value_Type.Date -> date_periods
Value_Type.Date_Time _ -> date_periods + time_periods
Value_Type.Time -> time_periods
# Some fallback is needed for the type mismatch case. Throwing an error will not work as expected as just the widget code will fail. (TODO right?)
_ -> [Option ("Expected a date/time column but got "+column_type.to_display_text+".") "Date_Period.Day"]
Single_Choice display=Display.Always values=values

View File

@ -17,6 +17,7 @@ import java.time.format.DateTimeFormatter;
import java.time.format.DateTimeFormatterBuilder;
import java.time.temporal.ChronoUnit;
import java.time.temporal.TemporalField;
import java.time.temporal.TemporalUnit;
import java.time.temporal.WeekFields;
import java.util.Locale;
@ -266,4 +267,31 @@ public class Time_Utils {
public static ZonedDateTime with_zone_same_instant(ZonedDateTime dateTime, ZoneId zone) {
return dateTime.withZoneSameInstant(zone);
}
/**
* This wrapper function is needed to ensure that EnsoDate gets converted to LocalDate correctly.
* <p>
* The {@code ChronoUnit::between} takes a value of type Temporal which does not trigger a polyglot conversion.
*/
public static long unit_date_difference(TemporalUnit unit, LocalDate start, LocalDate end) {
return unit.between(start, end);
}
/**
* This wrapper function is needed to ensure that EnsoTimeOfDay gets converted to LocalTime correctly.
* <p>
* The {@code ChronoUnit::between} takes a value of type Temporal which does not trigger a polyglot conversion.
*/
public static long unit_time_difference(TemporalUnit unit, LocalTime start, LocalTime end) {
return unit.between(start, end);
}
/**
* This wrapper function is needed to ensure that EnsoDateTime gets converted to ZonedDateTime correctly.
* <p>
* The {@code ChronoUnit::between} takes a value of type Temporal which does not trigger a polyglot conversion.
*/
public static long unit_datetime_difference(TemporalUnit unit, ZonedDateTime start, ZonedDateTime end) {
return unit.between(start, end);
}
}

View File

@ -0,0 +1,97 @@
package org.enso.base.time;
import java.time.Duration;
import java.time.temporal.ChronoUnit;
import java.time.temporal.Temporal;
import java.time.temporal.TemporalUnit;
/**
* Some units that are not available in ChronoUnit but are used by Enso's Date_Period/Time_Period.
*/
public class CustomTemporalUnits {
/**
* A unit that represents a 24-hour period.
*
* <p>It will behave differently from DAYS if DST is involved in time-supporting Temporal values.
* However, if a pure-date-based Temporal value is provided (one that does not support time), it
* will act exactly as DAYS.
*/
public static final TemporalUnit DAY_AS_24_HOURS = new DayAs24Hours();
public static final TemporalUnit QUARTERS = new Quarters();
private static class DayAs24Hours implements TemporalUnit {
private final Duration duration = Duration.ofHours(24);
@Override
public Duration getDuration() {
return duration;
}
@Override
public boolean isDurationEstimated() {
return false;
}
@Override
public boolean isDateBased() {
return true;
}
@Override
public boolean isTimeBased() {
return true;
}
@SuppressWarnings("unchecked")
@Override
public <R extends Temporal> R addTo(R temporal, long amount) {
if (temporal.isSupported(ChronoUnit.HOURS)) {
return (R) temporal.plus(amount * 24, ChronoUnit.HOURS);
} else {
return (R) temporal.plus(amount, ChronoUnit.DAYS);
}
}
@Override
public long between(Temporal temporal1Inclusive, Temporal temporal2Exclusive) {
if (temporal1Inclusive.isSupported(ChronoUnit.HOURS)) {
return ChronoUnit.HOURS.between(temporal1Inclusive, temporal2Exclusive) / 24;
} else {
return ChronoUnit.DAYS.between(temporal1Inclusive, temporal2Exclusive);
}
}
}
private static class Quarters implements TemporalUnit {
@Override
public Duration getDuration() {
return ChronoUnit.MONTHS.getDuration().multipliedBy(3);
}
@Override
public boolean isDurationEstimated() {
return true;
}
@Override
public boolean isDateBased() {
return true;
}
@Override
public boolean isTimeBased() {
return false;
}
@Override
public <R extends Temporal> R addTo(R temporal, long amount) {
return ChronoUnit.MONTHS.addTo(temporal, amount * 3);
}
@Override
public long between(Temporal temporal1Inclusive, Temporal temporal2Exclusive) {
return ChronoUnit.MONTHS.between(temporal1Inclusive, temporal2Exclusive) / 3;
}
}
}

View File

@ -0,0 +1,113 @@
package org.enso.table.data.column.operation.map;
import org.enso.base.polyglot.Polyglot_Utils;
import org.enso.table.data.column.builder.Builder;
import org.enso.table.data.column.storage.Storage;
import org.enso.table.data.column.storage.type.AnyObjectType;
import org.enso.table.error.UnexpectedTypeException;
import org.graalvm.polyglot.Context;
public abstract class GenericBinaryObjectMapOperation<
InputType, InputStorageType extends Storage<InputType>, OutputType>
extends MapOperation<InputType, InputStorageType> {
protected GenericBinaryObjectMapOperation(
String name,
Class<InputType> inputTypeClass,
Class<? extends InputStorageType> inputStorageTypeClass) {
super(name);
this.inputTypeClass = inputTypeClass;
this.inputStorageTypeClass = inputStorageTypeClass;
}
private final Class<InputType> inputTypeClass;
private final Class<? extends InputStorageType> inputStorageTypeClass;
protected abstract Builder createOutputBuilder(int size);
protected abstract OutputType run(InputType value, InputType other);
@Override
public Storage<?> runMap(
InputStorageType storage, Object arg, MapOperationProblemBuilder problemBuilder) {
arg = Polyglot_Utils.convertPolyglotValue(arg);
if (arg == null) {
int n = storage.size();
Builder builder = createOutputBuilder(n);
builder.appendNulls(n);
return builder.seal();
} else if (inputTypeClass.isInstance(arg)) {
InputType casted = inputTypeClass.cast(arg);
int n = storage.size();
Builder builder = createOutputBuilder(n);
Context context = Context.getCurrent();
for (int i = 0; i < n; i++) {
if (storage.isNa(i)) {
builder.appendNulls(1);
} else {
OutputType result = run(storage.getItemBoxed(i), casted);
builder.appendNoGrow(result);
}
context.safepoint();
}
return builder.seal();
} else {
throw new UnexpectedTypeException(
"a " + inputTypeClass.getName() + " but got " + arg.getClass().getName());
}
}
@Override
public Storage<?> runZip(
InputStorageType storage, Storage<?> arg, MapOperationProblemBuilder problemBuilder) {
if (inputStorageTypeClass.isInstance(arg)) {
InputStorageType otherCasted = inputStorageTypeClass.cast(arg);
int n = storage.size();
Builder builder = createOutputBuilder(n);
Context context = Context.getCurrent();
for (int i = 0; i < n; ++i) {
if (storage.isNa(i) || otherCasted.isNa(i)) {
builder.appendNulls(1);
} else {
InputType left = storage.getItemBoxed(i);
InputType right = otherCasted.getItemBoxed(i);
OutputType result = run(left, right);
builder.append(result);
}
context.safepoint();
}
return builder.seal();
} else if (arg.getType() instanceof AnyObjectType) {
// TODO this case may not be needed once #7231 gets implemented
int n = storage.size();
Builder builder = createOutputBuilder(n);
Context context = Context.getCurrent();
for (int i = 0; i < n; ++i) {
if (storage.isNa(i) || arg.isNa(i)) {
builder.appendNulls(1);
} else {
InputType left = storage.getItemBoxed(i);
Object right = arg.getItemBoxed(i);
if (inputTypeClass.isInstance(right)) {
OutputType result = run(left, inputTypeClass.cast(right));
builder.append(result);
} else {
throw new UnexpectedTypeException(
"Got a mixed storage where values were assumed to be of type "
+ inputTypeClass.getName()
+ ", but got a value of type "
+ right.getClass().getName());
}
}
context.safepoint();
}
return builder.seal();
} else {
throw new UnexpectedTypeException(
"a " + inputStorageTypeClass.getName() + " or a mixed storage");
}
}
}

View File

@ -0,0 +1,93 @@
package org.enso.table.data.column.operation.map.datetime;
import java.time.temporal.ChronoField;
import java.time.temporal.IsoFields;
import java.time.temporal.Temporal;
import java.time.temporal.TemporalField;
import org.enso.table.data.column.operation.map.numeric.GenericUnaryIntegerOp;
import org.enso.table.data.column.storage.Storage;
public class DatePartExtractors {
public static <T extends Temporal, I extends Storage<T>>
GenericUnaryIntegerOp<Temporal, T, I> make_op(String name, TemporalField field) {
return new GenericUnaryIntegerOp<>(name) {
@Override
protected long doGenericOperation(Temporal value) {
return value.getLong(field);
}
};
}
public static <T extends Temporal, I extends Storage<T>>
GenericUnaryIntegerOp<Temporal, T, I> year() {
return make_op("year", ChronoField.YEAR);
}
public static <T extends Temporal, I extends Storage<T>>
GenericUnaryIntegerOp<Temporal, T, I> quarter() {
return new GenericUnaryIntegerOp<>("quarter") {
@Override
protected long doGenericOperation(Temporal value) {
long month = value.get(ChronoField.MONTH_OF_YEAR);
return (month - 1) / 3 + 1;
}
};
}
public static <T extends Temporal, I extends Storage<T>>
GenericUnaryIntegerOp<Temporal, T, I> month() {
return make_op("month", ChronoField.MONTH_OF_YEAR);
}
public static <T extends Temporal, I extends Storage<T>>
GenericUnaryIntegerOp<Temporal, T, I> week() {
return make_op("week", IsoFields.WEEK_OF_WEEK_BASED_YEAR);
}
public static <T extends Temporal, I extends Storage<T>>
GenericUnaryIntegerOp<Temporal, T, I> day() {
return make_op("day", ChronoField.DAY_OF_MONTH);
}
public static <T extends Temporal, I extends Storage<T>>
GenericUnaryIntegerOp<Temporal, T, I> hour() {
return make_op("hour", ChronoField.HOUR_OF_DAY);
}
public static <T extends Temporal, I extends Storage<T>>
GenericUnaryIntegerOp<Temporal, T, I> minute() {
return make_op("minute", ChronoField.MINUTE_OF_HOUR);
}
public static <T extends Temporal, I extends Storage<T>>
GenericUnaryIntegerOp<Temporal, T, I> second() {
return make_op("second", ChronoField.SECOND_OF_MINUTE);
}
public static <T extends Temporal, I extends Storage<T>>
GenericUnaryIntegerOp<Temporal, T, I> millisecond() {
return make_op("millisecond", ChronoField.MILLI_OF_SECOND);
}
public static <T extends Temporal, I extends Storage<T>>
GenericUnaryIntegerOp<Temporal, T, I> microsecond() {
return new GenericUnaryIntegerOp<>("microsecond") {
@Override
protected long doGenericOperation(Temporal value) {
long micros = value.get(ChronoField.MICRO_OF_SECOND);
return micros % 1000;
}
};
}
public static <T extends Temporal, I extends Storage<T>>
GenericUnaryIntegerOp<Temporal, T, I> nanosecond() {
return new GenericUnaryIntegerOp<>("nanosecond") {
@Override
protected long doGenericOperation(Temporal value) {
long micros = value.get(ChronoField.NANO_OF_SECOND);
return micros % 1000;
}
};
}
}

View File

@ -1,6 +1,8 @@
package org.enso.table.data.column.operation.map;
package org.enso.table.data.column.operation.map.numeric;
import java.util.BitSet;
import org.enso.table.data.column.operation.map.MapOperationProblemBuilder;
import org.enso.table.data.column.operation.map.UnaryMapOperationWithProblemBuilder;
import org.enso.table.data.column.storage.numeric.DoubleStorage;
import org.enso.table.data.column.storage.numeric.LongStorage;
import org.graalvm.polyglot.Context;

View File

@ -0,0 +1,17 @@
package org.enso.table.data.column.operation.map.numeric;
import org.enso.table.data.column.storage.Storage;
public abstract class GenericUnaryIntegerOp<U, T extends U, I extends Storage<T>>
extends UnaryIntegerOp<T, I> {
public GenericUnaryIntegerOp(String name) {
super(name);
}
protected abstract long doGenericOperation(U value);
@Override
protected long doOperation(T value) {
return doGenericOperation(value);
}
}

View File

@ -1,6 +1,7 @@
package org.enso.table.data.column.operation.map;
package org.enso.table.data.column.operation.map.numeric;
import java.util.BitSet;
import org.enso.table.data.column.operation.map.UnaryMapOperation;
import org.enso.table.data.column.storage.numeric.DoubleStorage;
import org.enso.table.data.column.storage.numeric.LongStorage;
import org.graalvm.polyglot.Context;

View File

@ -1,6 +1,7 @@
package org.enso.table.data.column.operation.map;
package org.enso.table.data.column.operation.map.numeric;
import java.util.BitSet;
import org.enso.table.data.column.operation.map.UnaryMapOperation;
import org.enso.table.data.column.storage.Storage;
import org.enso.table.data.column.storage.numeric.LongStorage;
import org.graalvm.polyglot.Context;

View File

@ -1,6 +1,7 @@
package org.enso.table.data.column.operation.map;
package org.enso.table.data.column.operation.map.numeric;
import java.util.BitSet;
import org.enso.table.data.column.operation.map.UnaryMapOperation;
import org.enso.table.data.column.storage.numeric.AbstractLongStorage;
import org.enso.table.data.column.storage.numeric.LongStorage;
import org.graalvm.polyglot.Context;

View File

@ -4,7 +4,7 @@ import java.time.LocalDate;
import org.enso.table.data.column.builder.Builder;
import org.enso.table.data.column.builder.DateBuilder;
import org.enso.table.data.column.operation.map.MapOpStorage;
import org.enso.table.data.column.operation.map.UnaryIntegerOp;
import org.enso.table.data.column.operation.map.datetime.DatePartExtractors;
import org.enso.table.data.column.operation.map.datetime.DateTimeIsInOp;
import org.enso.table.data.column.storage.ObjectStorage;
import org.enso.table.data.column.storage.SpecializedStorage;
@ -25,27 +25,11 @@ public final class DateStorage extends SpecializedStorage<LocalDate> {
private static MapOpStorage<LocalDate, SpecializedStorage<LocalDate>> buildOps() {
MapOpStorage<LocalDate, SpecializedStorage<LocalDate>> t = ObjectStorage.buildObjectOps();
t.add(new DateTimeIsInOp<>(LocalDate.class));
t.add(
new UnaryIntegerOp<>(Maps.YEAR) {
@Override
protected long doOperation(LocalDate date) {
return (long) date.getYear();
}
});
t.add(
new UnaryIntegerOp<>(Maps.MONTH) {
@Override
protected long doOperation(LocalDate date) {
return (long) date.getMonthValue();
}
});
t.add(
new UnaryIntegerOp<>(Maps.DAY) {
@Override
protected long doOperation(LocalDate date) {
return (long) date.getDayOfMonth();
}
});
t.add(DatePartExtractors.year());
t.add(DatePartExtractors.quarter());
t.add(DatePartExtractors.month());
t.add(DatePartExtractors.week());
t.add(DatePartExtractors.day());
return t;
}

View File

@ -1,10 +1,13 @@
package org.enso.table.data.column.storage.datetime;
import java.time.Duration;
import java.time.ZonedDateTime;
import org.enso.table.data.column.builder.Builder;
import org.enso.table.data.column.builder.DateTimeBuilder;
import org.enso.table.data.column.builder.ObjectBuilder;
import org.enso.table.data.column.operation.map.GenericBinaryObjectMapOperation;
import org.enso.table.data.column.operation.map.MapOpStorage;
import org.enso.table.data.column.operation.map.UnaryIntegerOp;
import org.enso.table.data.column.operation.map.datetime.DatePartExtractors;
import org.enso.table.data.column.operation.map.datetime.DateTimeIsInOp;
import org.enso.table.data.column.storage.ObjectStorage;
import org.enso.table.data.column.storage.SpecializedStorage;
@ -27,25 +30,29 @@ public final class DateTimeStorage extends SpecializedStorage<ZonedDateTime> {
MapOpStorage<ZonedDateTime, SpecializedStorage<ZonedDateTime>> t =
ObjectStorage.buildObjectOps();
t.add(new DateTimeIsInOp<>(ZonedDateTime.class));
t.add(DatePartExtractors.year());
t.add(DatePartExtractors.quarter());
t.add(DatePartExtractors.month());
t.add(DatePartExtractors.week());
t.add(DatePartExtractors.day());
t.add(DatePartExtractors.hour());
t.add(DatePartExtractors.minute());
t.add(DatePartExtractors.second());
t.add(DatePartExtractors.millisecond());
t.add(DatePartExtractors.microsecond());
t.add(DatePartExtractors.nanosecond());
t.add(
new UnaryIntegerOp<>(Maps.YEAR) {
new GenericBinaryObjectMapOperation<
ZonedDateTime, SpecializedStorage<ZonedDateTime>, Duration>(
Maps.SUB, ZonedDateTime.class, DateTimeStorage.class) {
@Override
protected long doOperation(ZonedDateTime date) {
return (long) date.getYear();
protected Builder createOutputBuilder(int size) {
return new ObjectBuilder(size);
}
});
t.add(
new UnaryIntegerOp<>(Maps.MONTH) {
@Override
protected long doOperation(ZonedDateTime date) {
return (long) date.getMonthValue();
}
});
t.add(
new UnaryIntegerOp<>(Maps.DAY) {
@Override
protected long doOperation(ZonedDateTime date) {
return (long) date.getDayOfMonth();
protected Duration run(ZonedDateTime value, ZonedDateTime other) {
return Duration.between(other, value);
}
});
return t;

View File

@ -1,9 +1,13 @@
package org.enso.table.data.column.storage.datetime;
import java.time.Duration;
import java.time.LocalTime;
import org.enso.table.data.column.builder.Builder;
import org.enso.table.data.column.builder.ObjectBuilder;
import org.enso.table.data.column.builder.TimeOfDayBuilder;
import org.enso.table.data.column.operation.map.GenericBinaryObjectMapOperation;
import org.enso.table.data.column.operation.map.MapOpStorage;
import org.enso.table.data.column.operation.map.datetime.DatePartExtractors;
import org.enso.table.data.column.operation.map.datetime.DateTimeIsInOp;
import org.enso.table.data.column.storage.ObjectStorage;
import org.enso.table.data.column.storage.SpecializedStorage;
@ -24,6 +28,25 @@ public final class TimeOfDayStorage extends SpecializedStorage<LocalTime> {
private static MapOpStorage<LocalTime, SpecializedStorage<LocalTime>> buildOps() {
MapOpStorage<LocalTime, SpecializedStorage<LocalTime>> t = ObjectStorage.buildObjectOps();
t.add(new DateTimeIsInOp<>(LocalTime.class));
t.add(DatePartExtractors.hour());
t.add(DatePartExtractors.minute());
t.add(DatePartExtractors.second());
t.add(DatePartExtractors.millisecond());
t.add(DatePartExtractors.microsecond());
t.add(DatePartExtractors.nanosecond());
t.add(
new GenericBinaryObjectMapOperation<LocalTime, SpecializedStorage<LocalTime>, Duration>(
Maps.SUB, LocalTime.class, TimeOfDayStorage.class) {
@Override
protected Builder createOutputBuilder(int size) {
return new ObjectBuilder(size);
}
@Override
protected Duration run(LocalTime value, LocalTime other) {
return Duration.between(other, value);
}
});
return t;
}

View File

@ -5,11 +5,11 @@ import org.enso.table.data.column.builder.Builder;
import org.enso.table.data.column.builder.NumericBuilder;
import org.enso.table.data.column.operation.map.MapOpStorage;
import org.enso.table.data.column.operation.map.MapOperationProblemBuilder;
import org.enso.table.data.column.operation.map.UnaryLongToLongOp;
import org.enso.table.data.column.operation.map.UnaryMapOperation;
import org.enso.table.data.column.operation.map.numeric.LongBooleanOp;
import org.enso.table.data.column.operation.map.numeric.LongIsInOp;
import org.enso.table.data.column.operation.map.numeric.LongNumericOp;
import org.enso.table.data.column.operation.map.numeric.UnaryLongToLongOp;
import org.enso.table.data.column.storage.BoolStorage;
import org.enso.table.data.column.storage.Storage;
import org.graalvm.polyglot.Context;

View File

@ -4,12 +4,12 @@ import java.util.BitSet;
import java.util.List;
import org.enso.table.data.column.builder.Builder;
import org.enso.table.data.column.builder.NumericBuilder;
import org.enso.table.data.column.operation.map.DoubleLongMapOpWithSpecialNumericHandling;
import org.enso.table.data.column.operation.map.MapOpStorage;
import org.enso.table.data.column.operation.map.MapOperationProblemBuilder;
import org.enso.table.data.column.operation.map.UnaryMapOperation;
import org.enso.table.data.column.operation.map.numeric.DoubleBooleanOp;
import org.enso.table.data.column.operation.map.numeric.DoubleIsInOp;
import org.enso.table.data.column.operation.map.numeric.DoubleLongMapOpWithSpecialNumericHandling;
import org.enso.table.data.column.operation.map.numeric.DoubleNumericOp;
import org.enso.table.data.column.storage.BoolStorage;
import org.enso.table.data.column.storage.Storage;

View File

@ -1,6 +1,11 @@
package org.enso.table.error;
/** An error thrown when a type error is encountered. */
/**
* An error thrown when a type error is encountered.
*
* <p>This is an internal error of the Table library and any time it is thrown indicates a bug.
* Normally, the types should be checked before being passed to a vectorized operation.
*/
public class UnexpectedTypeException extends RuntimeException {
private final String expected;

View File

@ -9,7 +9,7 @@ from Standard.Database.Errors import Unsupported_Database_Operation
from Standard.Test import Test, Problems
import Standard.Test.Extensions
from project.Common_Table_Operations.Util import run_default_backend
from project.Common_Table_Operations.Util import all
main = run_default_backend spec
@ -42,50 +42,45 @@ spec setup =
named_zone = Time_Zone.parse "US/Hawaii"
dt3 = Date_Time.new 2019 11 23 4 5 6 zone=named_zone
to_utc dt = dt.at_zone Time_Zone.utc
dates = [dt1, dt2, dt3]
xs = [1, 2, 3]
table = table_builder [["C", dates], ["X", xs]]
table.at "C" . value_type . should_equal Value_Type.Date_Time
## We compare the timestamps converted to UTC.
This ensures that the value we are getting back represents the
exact same instant in time as the one we put in.
We cannot guarantee that time time _zone_ itself will be the same
- for example Postgres stores all timestamps in UTC, regardless
of what timezone they were in at input.
table.at "C" . to_vector . map to_utc . should_equal (dates.map to_utc)
table.at "C" . to_vector . should_equal_tz_agnostic dates
table.at "X" . to_vector . should_equal xs
Test.group prefix+"Date-Time operations" pending=pending_datetime <|
dates = table_builder [["A", [Date.new 2020 12 31, Date.new 2024 2 29, Date.new 1990 1 1, Nothing]], ["X", [2020, 29, 1, 100]]]
times = table_builder [["A", [Time_Of_Day.new 23 59 59 millisecond=567 nanosecond=123, Time_Of_Day.new 2 30 44 nanosecond=1002000, Time_Of_Day.new 0 0 0, Nothing]], ["X", [2020, 29, 1, 100]]]
datetimes = table_builder [["A", [Date_Time.new 2020 12 31 23 59 59 millisecond=567 nanosecond=123, Date_Time.new 2024 2 29 2 30 44 nanosecond=1002000, Date_Time.new 1990 1 1 0 0 0, Nothing]], ["X", [2020, 29, 1, 100]]]
Test.specify "should allow to get the year/month/day of a Date" <|
t = table_builder [["A", [Date.new 2020 12 31, Date.new 2024 2 29, Date.new 1990 1 1, Nothing]], ["X", [2020, 29, 1, 100]]]
t.at "A" . year . to_vector . should_equal [2020, 2024, 1990, Nothing]
t.at "A" . month . to_vector . should_equal [12, 2, 1, Nothing]
t.at "A" . day . to_vector . should_equal [31, 29, 1, Nothing]
[t.at "A" . year, t.at "A" . month, t.at "A" . day].each c->
t = dates
a = t.at "A"
a.year . to_vector . should_equal [2020, 2024, 1990, Nothing]
a.month . to_vector . should_equal [12, 2, 1, Nothing]
a.day . to_vector . should_equal [31, 29, 1, Nothing]
[a.year, a.month, a.day].each c->
Test.with_clue "The column "+c.name+" value type ("+c.value_type.to_display_text+") should be an integer: " <|
c.value_type.is_integer.should_be_true
c.value_type.is_integer.should_be_true
((t.at "A" . year) == (t.at "X")).to_vector . should_equal [True, False, False, Nothing]
((t.at "A" . month) == (t.at "X")).to_vector . should_equal [False, False, True, Nothing]
((t.at "A" . day) == (t.at "X")).to_vector . should_equal [False, True, True, Nothing]
((a.year) == (t.at "X")).to_vector . should_equal [True, False, False, Nothing]
((a.month) == (t.at "X")).to_vector . should_equal [False, False, True, Nothing]
((a.day) == (t.at "X")).to_vector . should_equal [False, True, True, Nothing]
Test.specify "should allow to get the year/month/day of a Date_Time" <|
t = table_builder [["A", [Date_Time.new 2020 12 31 23 59 59, Date_Time.new 2024 2 29 2 30 44, Date_Time.new 1990 1 1 0 0 0, Nothing]], ["X", [2020, 29, 1, 100]]]
t.at "A" . year . to_vector . should_equal [2020, 2024, 1990, Nothing]
t.at "A" . month . to_vector . should_equal [12, 2, 1, Nothing]
t.at "A" . day . to_vector . should_equal [31, 29, 1, Nothing]
[t.at "A" . year, t.at "A" . month, t.at "A" . day].each c->
t = datetimes
a = t.at "A"
a.year . to_vector . should_equal [2020, 2024, 1990, Nothing]
a.month . to_vector . should_equal [12, 2, 1, Nothing]
a.day . to_vector . should_equal [31, 29, 1, Nothing]
[a.year, a.month, a.day].each c->
Test.with_clue "The column "+c.name+" value type ("+c.value_type.to_display_text+") should be an integer: " <|
c.value_type.is_integer.should_be_true
((t.at "A" . year) == (t.at "X")).to_vector . should_equal [True, False, False, Nothing]
((t.at "A" . month) == (t.at "X")).to_vector . should_equal [False, False, True, Nothing]
((t.at "A" . day) == (t.at "X")).to_vector . should_equal [False, True, True, Nothing]
((a.year) == (t.at "X")).to_vector . should_equal [True, False, False, Nothing]
((a.month) == (t.at "X")).to_vector . should_equal [False, False, True, Nothing]
((a.day) == (t.at "X")).to_vector . should_equal [False, True, True, Nothing]
Test.specify "should allow to evaluate expressions with year/month/day" <|
t = table_builder [["A", [Date.new 2020 12 31, Date.new 2024 2 29, Date.new 1990 1 1, Nothing]], ["X", [0, 2, 1, 100]], ["B", [Date_Time.new 2020 10 31 23 59 59, Date_Time.new 2024 4 29 2 30 44, Date_Time.new 1990 10 1 0 0 0, Nothing]]]
@ -94,13 +89,74 @@ spec setup =
c.value_type.is_integer.should_be_true
c.to_vector . should_equal [(2020 + 0 + 31 * 10), (2024 + 2 + 29 * 4), (1990 + 1 + 1 * 10), Nothing]
Test.specify "should report a type error if year/month/day is invoked on a non-date column" <|
t = table_builder [["A", [1, 2, 3]], ["B", ["a", "b", "c"]], ["C", [True, False, True]]]
r1 = t.at "A" . year
r1.should_fail_with Invalid_Value_Type
r1.catch . to_display_text . should_start_with "Expected type Date or Date_Time, but got a column [A] of type Integer"
t.at "B" . month . should_fail_with Invalid_Value_Type
t.at "C" . day . should_fail_with Invalid_Value_Type
Test.specify "should allow to get hour/minute/second of a Time_Of_Day" <|
a = times.at "A"
a.hour . to_vector . should_equal [23, 2, 0, Nothing]
a.minute . to_vector . should_equal [59, 30, 0, Nothing]
a.second . to_vector . should_equal [59, 44, 0, Nothing]
a.date_part Time_Period.Hour . to_vector . should_equal [23, 2, 0, Nothing]
a.date_part Time_Period.Minute . to_vector . should_equal [59, 30, 0, Nothing]
a.date_part Time_Period.Second . to_vector . should_equal [59, 44, 0, Nothing]
[a.hour, a.minute, a.second, a.date_part Time_Period.Hour, a.date_part Time_Period.Minute, a.date_part Time_Period.Second].each c->
Test.with_clue "The column "+c.name+" value type ("+c.value_type.to_display_text+") should be an integer: " <|
c.value_type.is_integer.should_be_true
Test.specify "should allow to get hour/minute/second of a Date_Time" <|
a = datetimes.at "A"
a.hour . to_vector . should_equal [23, 2, 0, Nothing]
a.minute . to_vector . should_equal [59, 30, 0, Nothing]
a.second . to_vector . should_equal [59, 44, 0, Nothing]
a.date_part Time_Period.Hour . to_vector . should_equal [23, 2, 0, Nothing]
a.date_part Time_Period.Minute . to_vector . should_equal [59, 30, 0, Nothing]
a.date_part Time_Period.Second . to_vector . should_equal [59, 44, 0, Nothing]
[a.hour, a.minute, a.second, a.date_part Time_Period.Hour, a.date_part Time_Period.Minute, a.date_part Time_Period.Second].each c->
Test.with_clue "The column "+c.name+" value type ("+c.value_type.to_display_text+") should be an integer: " <|
c.value_type.is_integer.should_be_true
Test.specify "should allow to get millisecond/nanosecond of Time_Of_Day through date_part" <|
a = times.at "A"
a.date_part Time_Period.Second . to_vector . should_equal [59, 44, 0, Nothing]
a.date_part Time_Period.Millisecond . to_vector . should_equal [567, 1, 0, Nothing]
a.date_part Time_Period.Microsecond . to_vector . should_equal [0, 2, 0, Nothing]
case setup.test_selection.supports_nanoseconds_in_time of
True ->
a.date_part Time_Period.Nanosecond . to_vector . should_equal [123, 0, 0, Nothing]
False ->
a.date_part Time_Period.Nanosecond . should_fail_with Unsupported_Database_Operation
[a.date_part Time_Period.Second, a.date_part Time_Period.Millisecond, a.date_part Time_Period.Microsecond, a.date_part Time_Period.Nanosecond].each c->
Test.with_clue "The column "+c.name+" value type ("+c.value_type.to_display_text+") should be an integer: " <|
c.value_type.is_integer.should_be_true
Test.specify "should allow to get week/quarter of Date through date_part" <|
a = dates.at "A"
a.date_part Date_Period.Quarter . to_vector . should_equal [4, 1, 1, Nothing]
a.date_part Date_Period.Week . to_vector . should_equal [53, 9, 1, Nothing]
[a.date_part Date_Period.Quarter, a.date_part Date_Period.Week].each c->
Test.with_clue "The column "+c.name+" value type ("+c.value_type.to_display_text+") should be an integer: " <|
c.value_type.is_integer.should_be_true
Test.specify "should allow to get various date_part of Date_Time" <|
a = datetimes.at "A"
a.date_part Date_Period.Quarter . to_vector . should_equal [4, 1, 1, Nothing]
a.date_part Date_Period.Week . to_vector . should_equal [53, 9, 1, Nothing]
a.date_part Time_Period.Millisecond . to_vector . should_equal [567, 1, 0, Nothing]
a.date_part Time_Period.Microsecond . to_vector . should_equal [0, 2, 0, Nothing]
case setup.test_selection.supports_nanoseconds_in_time of
True ->
a.date_part Time_Period.Nanosecond . to_vector . should_equal [123, 0, 0, Nothing]
False ->
a.date_part Time_Period.Nanosecond . should_fail_with Unsupported_Database_Operation
[a.date_part Date_Period.Quarter, a.date_part Date_Period.Week, a.date_part Time_Period.Second, a.date_part Time_Period.Millisecond, a.date_part Time_Period.Microsecond, a.date_part Time_Period.Nanosecond].each c->
Test.with_clue "The column "+c.name+" value type ("+c.value_type.to_display_text+") should be an integer: " <|
c.value_type.is_integer.should_be_true
Test.specify "should allow to compare dates" <|
t = table_builder [["X", [Date.new 2021 12 3]], ["Y", [Date.new 2021 12 5]]]
@ -126,16 +182,303 @@ spec setup =
op (t.at "X") (t.at "Y") . to_vector . should_succeed
op (t.at "X") (Time_Of_Day.new 12 30 0) . to_vector . should_succeed
Test.specify "should not allow to mix" <|
Test.specify "should not allow to mix types in ordering comparisons" <|
t = table_builder [["X", [Date.new 2021 12 3]], ["Y", [Date_Time.new 2021 12 5 12 30 0]], ["Z", [Time_Of_Day.new 12 30 0]]]
[(<), (<=), (>), (>=)].each op->
op (t.at "X") (t.at "Y") . should_fail_with Invalid_Value_Type
op (t.at "X") (t.at "Z") . should_fail_with Invalid_Value_Type
if setup.test_selection.supports_time_duration then
Test.specify "should allow to subtract two Dates" <|
t = table_builder [["X", [Date.new 2021 11 3]], ["Y", [Date.new 2021 12 5]]]
((t.at "Y") - (t.at "X")) . to_vector . should_equal [Period.new months=1 days=2]
((t.at "Y") - (Date.new 2020 12 5)) . to_vector . should_equal [Period.new years=1]
Test.specify "should allow to subtract two Date_Times" <|
dx = Date_Time.new 2021 11 30 10 15 0
t = table_builder [["X", [dx]], ["Y", [Date_Time.new 2021 12 5 12 30 20]]]
hours = 2 + 24 * 5
diff = Duration.new hours=hours minutes=15 seconds=20
((t.at "Y") - (t.at "X")) . to_vector . should_equal [diff]
((t.at "Y") - dx) . to_vector . should_equal [diff]
Test.specify "should allow to subtract two Time_Of_Days" <|
t = table_builder [["X", [Time_Of_Day.new 10 15 0, Time_Of_Day.new 1 0 0]], ["Y", [Time_Of_Day.new 12 30 20, Time_Of_Day.new 0 0 0]]]
((t.at "Y") - (t.at "X")) . to_vector . should_equal [Duration.new hours=2 minutes=15 seconds=20, Duration.new hours=(-1) minutes=0 seconds=0]
((t.at "Y") - (Time_Of_Day.new 0 0 0)) . to_vector . should_equal [Duration.new hours=12 minutes=30 seconds=20, Duration.zero]
if setup.test_selection.supports_time_duration.not then
Test.specify "should report unsupported operation for subtracting date/time" <|
t1 = table_builder [["X", [Date.new 2021 11 3]], ["Y", [Date.new 2021 12 5]]]
t2 = table_builder [["X", [Date_Time.new 2021 11 3 10 15 0]], ["Y", [Date_Time.new 2021 12 5 12 30 20]]]
t3 = table_builder [["X", [Time_Of_Day.new 10 15 0, Time_Of_Day.new 1 0 0]], ["Y", [Time_Of_Day.new 12 30 20, Time_Of_Day.new 0 0 0]]]
((t1.at "Y") - (t1.at "X")) . should_fail_with Unsupported_Database_Operation
((t1.at "Y") - (Date.new 2020 12 5)) . should_fail_with Unsupported_Database_Operation
((t2.at "Y") - (t2.at "X")) . should_fail_with Unsupported_Database_Operation
((t2.at "Y") - (Date_Time.new 2020 12 5 10 15 0)) . should_fail_with Unsupported_Database_Operation
((t3.at "Y") - (t3.at "X")) . should_fail_with Unsupported_Database_Operation
((t3.at "Y") - (Time_Of_Day.new 0 0 0)) . should_fail_with Unsupported_Database_Operation
Test.specify "should report an Invalid_Value_Type error when subtracting mixed date/time types" <|
t = table_builder [["X", [Date.new 2021 11 3]], ["Y", [Date_Time.new 2021 12 5 12 30 0]], ["Z", [Time_Of_Day.new 12 30 0]]]
((t.at "Y") - (t.at "X")) . should_fail_with Invalid_Value_Type
((t.at "Y") - (Time_Of_Day.new 12 30 0)) . should_fail_with Invalid_Value_Type
((t.at "X") - (t.at "Z")) . should_fail_with Invalid_Value_Type
((t.at "X") - (Date_Time.new 2021 12 5 12 30 0)) . should_fail_with Invalid_Value_Type
((t.at "Z") - (t.at "Y")) . should_fail_with Invalid_Value_Type
((t.at "Z") - (Date.new 2021 11 3)) . should_fail_with Invalid_Value_Type
Test.specify "should allow computing a SQL-like difference" <|
t1 = table_builder [["X", [Date.new 2021 11 3]], ["Y", [Date.new 2021 12 5]]]
(t1.at "X").date_diff (t1.at "Y") Date_Period.Day . to_vector . should_equal [32]
(t1.at "Y").date_diff (t1.at "X") Date_Period.Day . to_vector . should_equal [-32]
(t1.at "X").date_diff (Date.new 2021 11 3) Date_Period.Day . to_vector . should_equal [0]
(t1.at "X").date_diff (t1.at "Y") Date_Period.Month . to_vector . should_equal [1]
(t1.at "X").date_diff (Date.new 2021 12 1) Date_Period.Month . to_vector . should_equal [0]
(t1.at "X").date_diff (Date.new 2020 12 1) Date_Period.Month . to_vector . should_equal [-11]
(t1.at "X").date_diff (t1.at "Y") Date_Period.Quarter . to_vector . should_equal [0]
(t1.at "X").date_diff (Date.new 2021 5 1) Date_Period.Quarter . to_vector . should_equal [-2]
(t1.at "X").date_diff (Date.new 2023 7 1) Date_Period.Quarter . to_vector . should_equal [6]
(t1.at "X").date_diff (t1.at "Y") Date_Period.Year . to_vector . should_equal [0]
(t1.at "X").date_diff (Date.new 2021 12 1) Date_Period.Year . to_vector . should_equal [0]
(t1.at "X").date_diff (Date.new 2020 10 1) Date_Period.Year . to_vector . should_equal [-1]
# Ensure months of varying length (e.g. February) are still counted right.
t1_2 = table_builder [["X", [Date.new 2021 01 02]]]
(t1_2 . at "X").date_diff (Date.new 2021 03 02) Date_Period.Day . to_vector . should_equal [59]
(t1_2 . at "X").date_diff (Date.new 2021 03 02) Date_Period.Month . to_vector . should_equal [2]
(t1_2 . at "X").date_diff (Date.new 2021 03 01) Date_Period.Day . to_vector . should_equal [58]
(t1_2 . at "X").date_diff (Date.new 2021 03 01) Date_Period.Month . to_vector . should_equal [1]
# We do allow the `Time_Period.Day` as a kind of alias for `Date_Period.Day` here.
(t1.at "X").date_diff (t1.at "Y") Time_Period.Day . to_vector . should_equal [32]
(t1.at "X").date_diff (t1.at "Y") Time_Period.Hour . should_fail_with Illegal_Argument
t2 = table_builder [["X", [Date_Time.new 2021 11 3 10 15 0]], ["Y", [Date_Time.new 2021 12 5 12 30 20]]]
(t2.at "X").date_diff (t2.at "Y") Date_Period.Day . to_vector . should_equal [32]
(t2.at "Y").date_diff (t2.at "X") Date_Period.Day . to_vector . should_equal [-32]
(t2.at "X").date_diff (Date_Time.new 2021 11 3 10 15 0) Date_Period.Day . to_vector . should_equal [0]
(t2.at "X").date_diff (t2.at "Y") Date_Period.Month . to_vector . should_equal [1]
(t2.at "X").date_diff (Date_Time.new 2021 12 1 10 15 0) Date_Period.Month . to_vector . should_equal [0]
(t2.at "X").date_diff (t2.at "Y") Date_Period.Year . to_vector . should_equal [0]
(t2.at "X").date_diff (Date_Time.new 2031 12 1 10 15 0) Date_Period.Year . to_vector . should_equal [10]
(t2.at "X").date_diff (t2.at "Y") Time_Period.Day . to_vector . should_equal [32]
(t2.at "X").date_diff (t2.at "Y") Time_Period.Hour . to_vector . should_equal [770]
(t2.at "X").date_diff (Date_Time.new 2021 11 3 12 15 0) Time_Period.Hour . to_vector . should_equal [2]
(t2.at "X").date_diff (t2.at "Y") Time_Period.Minute . to_vector . should_equal [46215]
(t2.at "X").date_diff (Date_Time.new 2021 11 3 10 45 0) Time_Period.Minute . to_vector . should_equal [30]
(t2.at "X").date_diff (t2.at "Y") Time_Period.Second . to_vector . should_equal [2772920]
(t2.at "X").date_diff (Date_Time.new 2021 11 3 10 15 30) Time_Period.Second . to_vector . should_equal [30]
(t2.at "X").date_diff (t2.at "Y") Time_Period.Millisecond . to_vector . should_equal [2772920000]
(t2.at "X").date_diff (Date_Time.new 2021 11 3 10 15 30 123) Time_Period.Millisecond . to_vector . should_equal [30123]
(t2.at "X").date_diff (t2.at "Y") Time_Period.Microsecond . to_vector . should_equal [2772920000000]
(t2.at "X").date_diff (Date_Time.new 2021 11 3 10 15 30 123 456) Time_Period.Microsecond . to_vector . should_equal [30123456]
case setup.test_selection.supports_nanoseconds_in_time of
True ->
(t2.at "X").date_diff (t2.at "Y") Time_Period.Nanosecond . to_vector . should_equal [2772920000000000]
(t2.at "X").date_diff (Date_Time.new 2021 11 3 10 15 30 123 456 789) Time_Period.Nanosecond . to_vector . should_equal [30123456789]
False ->
(t2.at "X").date_diff (t2.at "Y") Time_Period.Nanosecond . should_fail_with Unsupported_Database_Operation
(t2.at "X").date_diff (Date_Time.new 2021 11 3 10 15 30 123 456 789) Time_Period.Nanosecond . should_fail_with Unsupported_Database_Operation
t3 = table_builder [["X", [Time_Of_Day.new 10 15 0]], ["Y", [Time_Of_Day.new 12 30 20]]]
# There is no default period:
(t3.at "X").date_diff (t3.at "Y") . should_be_a Function
(t3.at "X").date_diff (t3.at "Y") Date_Period.Month . should_fail_with Illegal_Argument
# This will always be 0, should it be allowed?
(t3.at "X").date_diff (t3.at "Y") Time_Period.Day . should_fail_with Illegal_Argument
(t3.at "X").date_diff (t3.at "Y") Time_Period.Hour . to_vector . should_equal [2]
(t3.at "X").date_diff (Time_Of_Day.new 9 15 0) Time_Period.Hour . to_vector . should_equal [-1]
(t3.at "X").date_diff (t3.at "Y") Time_Period.Minute . to_vector . should_equal [135]
(t3.at "X").date_diff (Time_Of_Day.new 10 04 0) Time_Period.Minute . to_vector . should_equal [-11]
(t3.at "X").date_diff (t3.at "Y") Time_Period.Second . to_vector . should_equal [8120]
(t3.at "X").date_diff (Time_Of_Day.new 10 15 12) Time_Period.Second . to_vector . should_equal [12]
(t3.at "X").date_diff (t3.at "Y") Time_Period.Millisecond . to_vector . should_equal [8120*1000]
(t3.at "X").date_diff (Time_Of_Day.new 10 15 12 34) Time_Period.Millisecond . to_vector . should_equal [12034]
(t3.at "X").date_diff (t3.at "Y") Time_Period.Microsecond . to_vector . should_equal [8120*1000*1000]
(t3.at "X").date_diff (Time_Of_Day.new 10 15 12 34 56) Time_Period.Microsecond . to_vector . should_equal [12034056]
case setup.test_selection.supports_nanoseconds_in_time of
True ->
(t3.at "X").date_diff (t3.at "Y") Time_Period.Nanosecond . to_vector . should_equal [8120*1000*1000*1000]
(t3.at "X").date_diff (Time_Of_Day.new 10 15 12 34 56 78) Time_Period.Nanosecond . to_vector . should_equal [12034056078]
False ->
(t3.at "X").date_diff (t3.at "Y") Time_Period.Nanosecond . should_fail_with Unsupported_Database_Operation
(t3.at "X").date_diff (Time_Of_Day.new 10 15 12 34 56 78) Time_Period.Nanosecond . should_fail_with Unsupported_Database_Operation
Test.specify "date_diff should return integers" <|
t = table_builder [["X", [Date.new 2021 01 31]], ["Y", [Time_Of_Day.new 12 30 20]], ["Z", [Date_Time.new 2021 12 5 12 30 20]]]
time_periods = [Time_Period.Hour, Time_Period.Minute, Time_Period.Second]
date_periods = [Date_Period.Day, Date_Period.Week, Date_Period.Month, Date_Period.Quarter, Date_Period.Year]
date_periods.each p->
(t.at "X").date_diff (Date.new 2021 12 05) p . value_type . is_integer . should_be_true
time_periods.each p->
(t.at "Y").date_diff (Time_Of_Day.new 01 02) p . value_type . is_integer . should_be_true
(date_periods+time_periods).each p->
(t.at "Z").date_diff (Date_Time.new 2021 12 05 01 02) p . value_type . is_integer . should_be_true
Test.specify "should not allow mixing types in date_diff" <|
t = table_builder [["X", [Date.new 2021 01 31]], ["Y", [Time_Of_Day.new 12 30 20]], ["Z", [Date_Time.new 2021 12 5 12 30 20]]]
(t.at "X").date_diff (t.at "Y") Date_Period.Day . should_fail_with Invalid_Value_Type
(t.at "Z").date_diff (t.at "X") Date_Period.Day . should_fail_with Invalid_Value_Type
(t.at "Y").date_diff (t.at "Z") Time_Period.Hour . should_fail_with Invalid_Value_Type
r1 = (t.at "X").date_diff (Date_Time.new 2021 12 5 12 30 20) Date_Period.Day
r1.should_fail_with Invalid_Value_Type
r1.catch.expected.to_text . should_equal "Date"
(t.at "Y").date_diff (Date.new 2021 12 5) Date_Period.Day . should_fail_with Invalid_Value_Type
(t.at "Z").date_diff (Time_Of_Day.new 12 30 20) Time_Period.Hour . should_fail_with Invalid_Value_Type
Test.specify "should allow an SQL-like shift" <|
t1 = table_builder [["X", [Date.new 2021 01 31, Date.new 2021 01 01, Date.new 2021 12 31]], ["Y", [5, -1, 0]]]
(t1.at "X").date_add (t1.at "Y") Date_Period.Day . to_vector . should_equal [Date.new 2021 02 05, Date.new 2020 12 31, Date.new 2021 12 31]
(t1.at "X").date_add -1 Date_Period.Day . to_vector . should_equal [Date.new 2021 01 30, Date.new 2020 12 31, Date.new 2021 12 30]
(t1.at "X").date_add (t1.at "Y") Date_Period.Month . to_vector . should_equal [Date.new 2021 06 30, Date.new 2020 12 01, Date.new 2021 12 31]
(t1.at "X").date_add 1 Date_Period.Month . to_vector . should_equal [Date.new 2021 02 28, Date.new 2021 02 01, Date.new 2022 01 31]
(t1.at "X").date_add (t1.at "Y") Date_Period.Year . to_vector . should_equal [Date.new 2026 01 31, Date.new 2020 01 01, Date.new 2021 12 31]
(t1.at "X").date_add 1 Date_Period.Year . to_vector . should_equal [Date.new 2022 01 31, Date.new 2022 01 01, Date.new 2022 12 31]
(t1.at "X").date_add (t1.at "Y") Date_Period.Week . to_vector . should_equal [Date.new 2021 03 07, Date.new 2020 12 25, Date.new 2021 12 31]
(t1.at "X").date_add 1 Date_Period.Week . to_vector . should_equal [Date.new 2021 02 07, Date.new 2021 01 08, Date.new 2022 01 07]
(t1.at "X").date_add (t1.at "Y") Date_Period.Quarter . to_vector . should_equal [Date.new 2022 04 30, Date.new 2020 10 01, Date.new 2021 12 31]
(t1.at "X").date_add 1 Date_Period.Quarter . to_vector . should_equal [Date.new 2021 04 30, Date.new 2021 04 01, Date.new 2022 03 31]
(t1.at "X").date_add 1 Time_Period.Hour . should_fail_with Illegal_Argument
# Will accept Time_Period.Day as alias of Date_Period.Day
(t1.at "X").date_add 1 Time_Period.Day . to_vector . should_equal [Date.new 2021 02 01, Date.new 2021 01 02, Date.new 2022 01 01]
t2 = table_builder [["X", [Date_Time.new 2021 01 31 12 30 0, Date_Time.new 2021 01 01 12 30 0, Date_Time.new 2021 12 31 12 30 0]], ["Y", [5, -1, 0]]]
(t2.at "X").date_add (t2.at "Y") Date_Period.Day . to_vector . should_equal_tz_agnostic [Date_Time.new 2021 02 05 12 30 0, Date_Time.new 2020 12 31 12 30 0, Date_Time.new 2021 12 31 12 30 0]
(t2.at "X").date_add -1 Time_Period.Day . to_vector . should_equal_tz_agnostic [Date_Time.new 2021 01 30 12 30 0, Date_Time.new 2020 12 31 12 30 0, Date_Time.new 2021 12 30 12 30 0]
(t2.at "X").date_add (t2.at "Y") Date_Period.Month . to_vector . should_equal_tz_agnostic [Date_Time.new 2021 06 30 12 30 0, Date_Time.new 2020 12 01 12 30 0, Date_Time.new 2021 12 31 12 30 0]
(t2.at "X").date_add 1 Date_Period.Month . to_vector . should_equal_tz_agnostic [Date_Time.new 2021 02 28 12 30 0, Date_Time.new 2021 02 01 12 30 0, Date_Time.new 2022 01 31 12 30 0]
(t2.at "X").date_add (t2.at "Y") Date_Period.Year . to_vector . should_equal_tz_agnostic [Date_Time.new 2026 01 31 12 30 0, Date_Time.new 2020 01 01 12 30 0, Date_Time.new 2021 12 31 12 30 0]
(t2.at "X").date_add 1 Date_Period.Year . to_vector . should_equal_tz_agnostic [Date_Time.new 2022 01 31 12 30 0, Date_Time.new 2022 01 01 12 30 0, Date_Time.new 2022 12 31 12 30 0]
(t2.at "X").date_add (t2.at "Y") Time_Period.Hour . to_vector . should_equal_tz_agnostic [Date_Time.new 2021 01 31 17 30 0, Date_Time.new 2021 01 01 11 30 0, Date_Time.new 2021 12 31 12 30 0]
(t2.at "X").date_add 1 Time_Period.Hour . to_vector . should_equal_tz_agnostic [Date_Time.new 2021 01 31 13 30 0, Date_Time.new 2021 01 01 13 30 0, Date_Time.new 2021 12 31 13 30 0]
(t2.at "X").date_add (t2.at "Y") Time_Period.Minute . to_vector . should_equal_tz_agnostic [Date_Time.new 2021 01 31 12 35 0, Date_Time.new 2021 01 01 12 29 0, Date_Time.new 2021 12 31 12 30 0]
(t2.at "X").date_add 1 Time_Period.Minute . to_vector . should_equal_tz_agnostic [Date_Time.new 2021 01 31 12 31 0, Date_Time.new 2021 01 01 12 31 0, Date_Time.new 2021 12 31 12 31 0]
(t2.at "X").date_add (t2.at "Y") Time_Period.Second . to_vector . should_equal_tz_agnostic [Date_Time.new 2021 01 31 12 30 5, Date_Time.new 2021 01 01 12 29 59, Date_Time.new 2021 12 31 12 30 0]
(t2.at "X").date_add 1 Time_Period.Second . to_vector . should_equal_tz_agnostic [Date_Time.new 2021 01 31 12 30 1, Date_Time.new 2021 01 01 12 30 1, Date_Time.new 2021 12 31 12 30 1]
(t2.at "X").date_add 1 Time_Period.Millisecond . to_vector . should_equal_tz_agnostic [Date_Time.new 2021 01 31 12 30 millisecond=1, Date_Time.new 2021 01 01 12 30 millisecond=1, Date_Time.new 2021 12 31 12 30 millisecond=1]
(t2.at "X").date_add 1 Time_Period.Microsecond . to_vector . should_equal_tz_agnostic [Date_Time.new 2021 01 31 12 30 microsecond=1, Date_Time.new 2021 01 01 12 30 microsecond=1, Date_Time.new 2021 12 31 12 30 microsecond=1]
case setup.test_selection.supports_nanoseconds_in_time of
True ->
(t2.at "X").date_add 1 Time_Period.Nanosecond . to_vector . should_equal [Date_Time.new 2021 01 31 12 30 nanosecond=1, Date_Time.new 2021 01 01 12 30 nanosecond=1, Date_Time.new 2021 12 31 12 30 nanosecond=1]
False ->
(t2.at "X").date_add 1 Time_Period.Nanosecond . should_fail_with Unsupported_Database_Operation
t3 = table_builder [["X", [Time_Of_Day.new 12 30 0, Time_Of_Day.new 23 45 0, Time_Of_Day.new 1 30 0]], ["Y", [5, -1, 0]]]
(t3.at "X").date_add (t3.at "Y") Time_Period.Hour . to_vector . should_equal [Time_Of_Day.new 17 30 0, Time_Of_Day.new 22 45 0, Time_Of_Day.new 1 30 0]
(t3.at "X").date_add 1 Time_Period.Hour . to_vector . should_equal [Time_Of_Day.new 13 30 0, Time_Of_Day.new 0 45 0, Time_Of_Day.new 2 30 0]
(t3.at "X").date_add (t3.at "Y") Time_Period.Minute . to_vector . should_equal [Time_Of_Day.new 12 35 0, Time_Of_Day.new 23 44 0, Time_Of_Day.new 1 30 0]
(t3.at "X").date_add 1 Time_Period.Minute . to_vector . should_equal [Time_Of_Day.new 12 31 0, Time_Of_Day.new 23 46 0, Time_Of_Day.new 1 31 0]
(t3.at "X").date_add (t3.at "Y") Time_Period.Second . to_vector . should_equal [Time_Of_Day.new 12 30 5, Time_Of_Day.new 23 44 59, Time_Of_Day.new 1 30 0]
(t3.at "X").date_add 1 Time_Period.Second . to_vector . should_equal [Time_Of_Day.new 12 30 1, Time_Of_Day.new 23 45 1, Time_Of_Day.new 1 30 1]
(t3.at "X").date_add 1 Time_Period.Millisecond . to_vector . should_equal [Time_Of_Day.new 12 30 millisecond=1, Time_Of_Day.new 23 45 millisecond=1, Time_Of_Day.new 1 30 millisecond=1]
(t3.at "X").date_add 1 Time_Period.Microsecond . to_vector . should_equal [Time_Of_Day.new 12 30 microsecond=1, Time_Of_Day.new 23 45 microsecond=1, Time_Of_Day.new 1 30 microsecond=1]
case setup.test_selection.supports_nanoseconds_in_time of
True ->
(t3.at "X").date_add 1 Time_Period.Nanosecond . to_vector . should_equal [Time_Of_Day.new 12 30 nanosecond=1, Time_Of_Day.new 23 45 nanosecond=1, Time_Of_Day.new 1 30 nanosecond=1]
False ->
(t3.at "X").date_add 1 Time_Period.Nanosecond . should_fail_with Unsupported_Database_Operation
# No sense to shift Time_Of_Day by days either or by a Date_Period
(t3.at "X").date_add (t3.at "Y") Time_Period.Day . to_vector . should_fail_with Illegal_Argument
(t3.at "X").date_add 1 Date_Period.Month . to_vector . should_fail_with Illegal_Argument
# There is no default period.
(t1.at "X").date_add (t1.at "Y") . should_be_a Function
Test.specify "should check shift_amount type in date_add" <|
t = table_builder [["X", [Date.new 2021 01 31]]]
t.at "X" . date_add "text" Date_Period.Day . should_fail_with Invalid_Value_Type
Test.specify "date_diff and date_add should work correctly with DST" <|
zone = Time_Zone.parse "Europe/Warsaw"
dt1 = Date_Time.new 2023 03 26 00 30 00 zone=zone
t = table_builder [["X", [dt1]]]
x = t.at "X"
# +24h will shift 1 day and 1 hour, because they 26th of March has only 23 hours within it
x.date_add 24 Time_Period.Hour . to_vector . should_equal_tz_agnostic [Date_Time.new 2023 03 27 01 30 00 zone=zone]
# But 1 day date shift will shift 1 day, keeping the time, even if that particular day is only 23 hours.
x.date_add 1 Date_Period.Day . to_vector . should_equal_tz_agnostic [Date_Time.new 2023 03 27 00 30 00 zone=zone]
# Time_Period.Day will shift by 24 hours.
x.date_add 1 Time_Period.Day . to_vector . should_equal_tz_agnostic [Date_Time.new 2023 03 27 01 30 00 zone=zone]
dt2 = Date_Time.new 2023 03 27 00 30 00 zone=zone
x.date_diff dt2 Time_Period.Hour . to_vector . should_equal [23]
# Date_Period.Day and Time_Period.Day are interchangeable.
## The results may vary between backends.
- In-memory we know this is a DST switch moment and so even if
there's 23 hours between the instants, it is a day difference.
- Postgres backend accepts times with timezone, but in its inner
storage, it converts them into UTC. In UTC there is no DST - so
these are 2 instants 23 hours from each other and the database
cannot 'guess' that it should be a day of a difference and
since it is 'just' 23 hours - there is 0 days.
[[0], [1]] . should_contain (x.date_diff dt2 Date_Period.Day . to_vector)
# Again consistent for both backends, when counting in hours - 23 hours is not a full 24-hour day.
x.date_diff dt2 Time_Period.Day . to_vector . should_equal [0]
dt3 = Date_Time.new 2023 03 28 01 30 00 zone=zone
dt4 = Date_Time.new 2023 03 29 00 30 00 zone=zone
t2 = table_builder [["X", [dt3]]]
# No DST switch here, so all backends agree that 0 days elapsed in the 23 hours.
(t2.at "X").date_diff dt4 Date_Period.Day . to_vector . should_equal [0]
(t2.at "X").date_diff dt4 Time_Period.Day . to_vector . should_equal [0]
(t2.at "X").date_diff dt4 Time_Period.Hour . to_vector . should_equal [23]
if setup.test_selection.date_time.not then
Test.group prefix+"partial Date-Time support" <|
Test.specify "will warn when uploading a Table containing Dates" <|
Test.specify "will fail when uploading a Table containing Dates" <|
d = Date.new 2020 10 24
table = table_builder [["A", [d]], ["X", [123]]]
table.should_fail_with Unsupported_Database_Operation
Test.specify "should report a type error when date operations are invoked on a non-date column" <|
t = table_builder [["A", [1, 2, 3]], ["B", ["a", "b", "c"]], ["C", [True, False, True]]]
r1 = t.at "A" . year
r1.should_fail_with Invalid_Value_Type
r1.catch . to_display_text . should_start_with "Expected type Date or Date_Time, but got a column [A] of type Integer"
t.at "B" . month . should_fail_with Invalid_Value_Type
t.at "C" . day . should_fail_with Invalid_Value_Type
t.at "A" . date_diff (t.at "B") Date_Period.Day . should_fail_with Invalid_Value_Type
t.at "A" . date_add 42 Date_Period.Day . should_fail_with Invalid_Value_Type

View File

@ -93,7 +93,11 @@ type Test_Selection
length text columns.
- supports_decimal_type: Specifies if the backend supports the `Decimal`
high-precision type.
Config supports_case_sensitive_columns=True order_by=True natural_ordering=False case_insensitive_ordering=True order_by_unicode_normalization_by_default=False case_insensitive_ascii_only=False take_drop=True allows_mixed_type_comparisons=True supports_unicode_normalization=False is_nan_and_nothing_distinct=True distinct_returns_first_row_from_group_if_ordered=True date_time=True fixed_length_text_columns=False supports_decimal_type=False
- supports_time_duration: Specifies if the backend supports a
`Duration`/`Period` type.
- supports_nanoseconds_in_time: Specifies if the backend supports
nanosecond precision in time values.
Config supports_case_sensitive_columns=True order_by=True natural_ordering=False case_insensitive_ordering=True order_by_unicode_normalization_by_default=False case_insensitive_ascii_only=False take_drop=True allows_mixed_type_comparisons=True supports_unicode_normalization=False is_nan_and_nothing_distinct=True distinct_returns_first_row_from_group_if_ordered=True date_time=True fixed_length_text_columns=False supports_decimal_type=False supports_time_duration=False supports_nanoseconds_in_time=False
spec setup =
Core_Spec.spec setup

View File

@ -24,3 +24,34 @@ run_default_backend spec =
within_table t <|
t.at "A" . to_vector . should_equal [1, 2, 3]
within_table table = Test.with_clue 'Resulting table:\n'+table.display+'\n\n'
## PRIVATE
Any.should_equal_tz_agnostic self other =
loc = Meta.get_source_location 1
_ = other
Test.fail "Expected a vector but got "+self.to_display_text+" (at "+loc+")."
## PRIVATE
A helper method that compares two vectors of Date_Time values.
It ensures that they represent the same instant in time, but ignore the
timezone that is attached to them. This is simply done by converting them to
UTC.
Vector.should_equal_tz_agnostic self other =
loc = Meta.get_source_location 1
case other of
_ : Vector ->
utc = Time_Zone.utc
normalize_date_time dt = case dt of
_ : Date_Time -> dt.at_zone utc
_ -> Test.fail "The vector should contain Date_Time objects but it contained "+dt.to_display_text+" (at "+loc+")"
self_normalized = self.map normalize_date_time
other_normalized = other.map normalize_date_time
self_normalized.should_equal other_normalized frames_to_skip=2
_ -> Test.fail "Expected a vector but got "+other.to_display_text+" (at "+loc+")"
## PRIVATE
Error.should_equal_tz_agnostic self other =
loc = Meta.get_source_location 1
_ = other
Test.fail "Expected a vector but got a dataflow error "+self.catch.to_display_text+" (at "+loc+")."

View File

@ -815,19 +815,17 @@ spec make_new_connection prefix persistent_connector=True =
tables_immediately_after = connection.base_connection.get_tables_advanced types=Nothing include_hidden=True . at "Name" . to_vector
# If no new tables are left out - we just finish.
passing_immediately = tables_immediately_after.sort == existing_tables.sort
if passing_immediately then Nothing else
## If there are some additional tables, we add some timeout to
allow the database to do the cleaning up.
## If there are some additional tables, we add some timeout to allow
the database to do the cleaning up.
additional_tables = (Set.from_vector tables_immediately_after).difference (Set.from_vector existing_tables)
if additional_tables.is_empty then
Test.fail "The Database contains less tables after the test than before! That is unexpected, please inspect manually."
if additional_tables.is_empty then Nothing else
additional_table = additional_tables.to_vector.first
wait_until_temporary_table_is_deleted_after_closing_connection connection additional_table
# After the wait we check again and now there should be no additional tables.
tables_after_wait = connection.base_connection.get_tables_advanced types=Nothing include_hidden=True . at "Name" . to_vector
tables_after_wait . should_contain_the_same_elements_as existing_tables
additional_tables_2 = (Set.from_vector tables_after_wait).difference (Set.from_vector existing_tables)
additional_tables_2.to_vector . should_equal []
database_table_builder name_prefix args primary_key=[] connection=connection =
in_memory_table = Table.new args

View File

@ -7,7 +7,7 @@ from Standard.Test import Test_Suite
import project.Common_Table_Operations
run_common_spec spec =
selection = Common_Table_Operations.Main.Test_Selection.Config supports_case_sensitive_columns=True order_by=True natural_ordering=True case_insensitive_ordering=True order_by_unicode_normalization_by_default=True supports_unicode_normalization=True
selection = Common_Table_Operations.Main.Test_Selection.Config supports_case_sensitive_columns=True order_by=True natural_ordering=True case_insensitive_ordering=True order_by_unicode_normalization_by_default=True supports_unicode_normalization=True supports_time_duration=True supports_nanoseconds_in_time=True
aggregate_selection = Common_Table_Operations.Aggregate_Spec.Test_Selection.Config
table = (enso_project.data / "data.csv") . read

View File

@ -3,7 +3,6 @@ import Standard.Base.Data.Text.Span.Span
import Standard.Base.Data.Text.Span.Utf_16_Span
import Standard.Base.Data.Text.Regex.Match.Match
import Standard.Base.Data.Text.Regex.No_Such_Group
import Standard.Base.Data.Text.Regex.Regex
import Standard.Base.Data.Text.Regex.Regex_Syntax_Error
import Standard.Base.Data.Text.Regex.Internal.Replacer.Replacer
import Standard.Base.Errors.Common.Type_Error

View File

@ -353,6 +353,10 @@ spec_with name create_new_datetime parse_datetime nanoseconds_loss_in_precision=
time+Time_Period.Hour . should_equal <| create_new_datetime 1970 1 1 1 (zone = Time_Zone.utc)
time+Time_Period.Minute . should_equal <| create_new_datetime 1970 1 1 0 1 (zone = Time_Zone.utc)
time+Time_Period.Second . should_equal <| create_new_datetime 1970 1 1 0 0 1 (zone = Time_Zone.utc)
time+Time_Period.Millisecond . should_equal <| create_new_datetime 1970 1 1 0 0 0 (10^6) (zone = Time_Zone.utc)
if nanoseconds_loss_in_precision.not then
time+Time_Period.Microsecond . should_equal <| create_new_datetime 1970 1 1 0 0 0 (10^3) (zone = Time_Zone.utc)
time+Time_Period.Nanosecond . should_equal <| create_new_datetime 1970 1 1 0 0 0 1 (zone = Time_Zone.utc)
Test.specify "should support subtraction of Time_Period" <|
time = create_new_datetime 1970 (zone = Time_Zone.utc)
@ -360,6 +364,11 @@ spec_with name create_new_datetime parse_datetime nanoseconds_loss_in_precision=
time-Time_Period.Hour . should_equal <| create_new_datetime 1969 12 31 23 (zone = Time_Zone.utc)
time-Time_Period.Minute . should_equal <| create_new_datetime 1969 12 31 23 59 (zone = Time_Zone.utc)
time-Time_Period.Second . should_equal <| create_new_datetime 1969 12 31 23 59 59 (zone = Time_Zone.utc)
second_in_nanos = 10^9
time-Time_Period.Millisecond . should_equal <| create_new_datetime 1969 12 31 23 59 59 (second_in_nanos - 10^6) (zone = Time_Zone.utc)
if nanoseconds_loss_in_precision.not then
time-Time_Period.Microsecond . should_equal <| create_new_datetime 1969 12 31 23 59 59 (second_in_nanos - 10^3) (zone = Time_Zone.utc)
time-Time_Period.Nanosecond . should_equal <| create_new_datetime 1969 12 31 23 59 59 (second_in_nanos - 1) (zone = Time_Zone.utc)
Test.specify "should support mixed addition and subtraction of Date_Period and Time_Period" <|
time = create_new_datetime 1970 (zone = Time_Zone.utc)
@ -502,6 +511,15 @@ spec_with name create_new_datetime parse_datetime nanoseconds_loss_in_precision=
d1.end_of Time_Period.Minute . should_equal (Date_Time.new 2022 9 12 15 37 59 nanosecond=max_nanos)
d1.start_of Time_Period.Second . should_equal (Date_Time.new 2022 9 12 15 37 58 0)
d1.end_of Time_Period.Second . should_equal (Date_Time.new 2022 9 12 15 37 58 nanosecond=max_nanos)
d1.start_of Time_Period.Millisecond . should_equal (Date_Time.new 2022 9 12 15 37 58 nanosecond=123000000)
d1.end_of Time_Period.Millisecond . should_equal (Date_Time.new 2022 9 12 15 37 58 nanosecond=123999999)
if nanoseconds_loss_in_precision.not then
d1.start_of Time_Period.Microsecond . should_equal (Date_Time.new 2022 9 12 15 37 58 nanosecond=123456000)
d1.end_of Time_Period.Microsecond . should_equal (Date_Time.new 2022 9 12 15 37 58 nanosecond=123456999)
# No change on nanosecond
d1.start_of Time_Period.Nanosecond . should_equal d1
d1.end_of Time_Period.Nanosecond . should_equal d1
d2 = create_new_datetime 1970 1 1 0 0 0
d2.start_of Time_Period.Day . should_equal (Date_Time.new 1970)
@ -535,6 +553,8 @@ spec_with name create_new_datetime parse_datetime nanoseconds_loss_in_precision=
d1_plus . should_equal d2
check_dates_spring date =
date.start_of Date_Period.Day . should_equal (Date_Time.new 2022 3 27 zone=tz)
date.end_of Date_Period.Day . should_equal (Date_Time.new 2022 3 27 23 59 59 nanosecond=max_nanos zone=tz)
date.start_of Time_Period.Day . should_equal (Date_Time.new 2022 3 27 zone=tz)
date.end_of Time_Period.Day . should_equal (Date_Time.new 2022 3 27 23 59 59 nanosecond=max_nanos zone=tz)

View File

@ -13,7 +13,7 @@ spec =
specWith "Time_Of_Day" enso_time Time_Of_Day.parse
specWith "JavaLocalTime" java_time java_parse
specWith name create_new_time parse_time =
specWith name create_new_time parse_time nanoseconds_loss_in_precision=False =
Test.group name <|
Test.specify "should create local time" <|
@ -123,6 +123,10 @@ specWith name create_new_time parse_time =
time+Time_Period.Hour . should_equal <| create_new_time 1
time+Time_Period.Minute . should_equal <| create_new_time 0 1
time+Time_Period.Second . should_equal <| create_new_time 0 0 1
time+Time_Period.Millisecond . should_equal <| create_new_time 0 0 0 10^6
if nanoseconds_loss_in_precision.not then
time+Time_Period.Microsecond . should_equal <| create_new_time 0 0 0 10^3
time+Time_Period.Nanosecond . should_equal <| create_new_time 0 0 0 1
Test.specify "should support subtraction of Time_Period" <|
time = create_new_time 12
@ -130,6 +134,11 @@ specWith name create_new_time parse_time =
time-Time_Period.Hour . should_equal <| create_new_time 11
time-Time_Period.Minute . should_equal <| create_new_time 11 59
time-Time_Period.Second . should_equal <| create_new_time 11 59 59
second_in_nanos = 10^9
time-Time_Period.Millisecond . should_equal <| create_new_time 11 59 59 (second_in_nanos - 10^6)
if nanoseconds_loss_in_precision.not then
time-Time_Period.Microsecond . should_equal <| create_new_time 11 59 59 (second_in_nanos - 10^3)
time-Time_Period.Nanosecond . should_equal <| create_new_time 11 59 59 (second_in_nanos - 1)
Test.specify "should support mixed addition and subtraction of Date_Period and Time_Period" <|
time = create_new_time 0
@ -164,6 +173,14 @@ specWith name create_new_time parse_time =
d1.end_of Time_Period.Minute . should_equal (Time_Of_Day.new 15 37 59 nanosecond=max_nanos)
d1.start_of Time_Period.Second . should_equal (Time_Of_Day.new 15 37 58 0)
d1.end_of Time_Period.Second . should_equal (Time_Of_Day.new 15 37 58 nanosecond=max_nanos)
d1.start_of Time_Period.Millisecond . should_equal (Time_Of_Day.new 15 37 58 nanosecond=123000000)
d1.end_of Time_Period.Millisecond . should_equal (Time_Of_Day.new 15 37 58 nanosecond=123999999)
if nanoseconds_loss_in_precision.not then
d1.start_of Time_Period.Microsecond . should_equal (Time_Of_Day.new 15 37 58 nanosecond=123456000)
d1.end_of Time_Period.Microsecond . should_equal (Time_Of_Day.new 15 37 58 nanosecond=123456999)
d1.start_of Time_Period.Nanosecond . should_equal d1
d1.end_of Time_Period.Nanosecond . should_equal d1
d2 = create_new_time 0 0 0
d2.start_of Time_Period.Day . should_equal (Time_Of_Day.new)