Home
last modified time | relevance | path

Searched +hist:3 +hist:ea321ce (Results 1 - 25 of 25) sorted by relevance

/sqlalchemy/lib/sqlalchemy/cextension/
H A Dimmutabledict.cdiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
H A Dresultproxy.cdiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff a6adc0a5 Fri Jul 26 18:50:18 MDT 2013 Mike Bayer <mike_mp@zzzcomputing.com> - The C extensions are ported to Python 3 and will build under
any supported CPython 2 or 3 environment. [ticket:2161]
diff a6adc0a5 Fri Jul 26 18:50:18 MDT 2013 Mike Bayer <mike_mp@zzzcomputing.com> - The C extensions are ported to Python 3 and will build under
any supported CPython 2 or 3 environment. [ticket:2161]
/sqlalchemy/examples/versioned_rows/
H A Dversioned_rows_w_versionid.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
/sqlalchemy/examples/elementtree/
H A Doptimized_al.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
/sqlalchemy/lib/sqlalchemy/sql/
H A Dlambdas.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 77c9534d Sat Dec 12 16:56:58 MST 2020 Mike Bayer <mike_mp@zzzcomputing.com> Major revisals to lambdas

1. Improve coercions._deep_is_literal to check sequences
for clause elements, thus allowing a phrase like
lambda: col.in_([literal("x"), literal("y")]) to be handled

2. revise closure variable caching completely. All variables
entering must be part of a closure cache key or rejected.
only objects that can be resolved to HasCacheKey or FunctionType
are accepted; all other types are rejected. This adds a high
degree of strictness to lambdas and will make them a little more
awkward to use in some cases, however prevents several classes
of critical issues:

a. previously, a lambda that had an expression derived from
some kind of state, like "self.x", or "execution_context.session.foo"
would produce a closure cache key from "self" or "execution_context",
objects that can very well be per-execution and would therefore
cause a AnalyzedFunction objects to overflow. (memory won't leak
as it looks like an LRUCache is already used for these)

b. a lambda, such as one used within DeferredLamdaElement, that
produces different SQL expressions based on the arguments
(which is in fact what it's supposed to do), however it would
through the use of conditionals produce different bound parameter
combinations, leading to literal parameters not tracked properly.
These are now rejected as uncacheable whereas previously they would
again be part of the closure cache key, causing an overflow of
AnalyizedFunction objects.

3. Ensure non-mapped mixins are handled correctly by
with_loader_criteria().

4. Fixed bug in lambda SQL system where we are not supposed to allow a Python
function to be embedded in the lambda, since we can't predict a bound value
from it. While there was an error condition added for this, it was not
tested and wasn't working; an informative error is now raised.

5. new docs for lambdas

6. consolidated changelog for all of these

Fixes: #5760
Fixes: #5765
Fixes: #5766
Fixes: #5768
Fixes: #5770
Change-Id: Iedaa636c3225fad496df23b612c516c8ab247ab7
3dc9a4a2 Mon Dec 16 15:06:43 MST 2019 Mike Bayer <mike_mp@zzzcomputing.com> introduce deferred lambdas

The coercions system allows us to add in lambdas as arguments
to Core and ORM elements without changing them at all. By allowing
the lambda to produce a deterministic cache key where we can also
cheat and yank out literal parameters means we can move towards
having 90% of "baked" functionality in a clearer way right in
Core / ORM.

As a second step, we can have whole statements inside the lambda,
and can then add generation with __add__(), so then we have
100% of "baked" functionality with full support of ad-hoc
literal values.

Adds some more short_selects tests for the moment for comparison.

Other tweaks inside cache key generation as we're trying to
approach a certain level of performance such that we can
remove the use of "baked" from the loader strategies.

As we have not yet closed #4639, however the caching feature
has been fully integrated as of
b0cfa7379cf8513a821a3dbe3028c4965d9f85bd, we will also
add complete caching documentation here and close that issue
as well.

Closes: #4639
Fixes: #5380
Change-Id: If91f61527236fd4d7ae3cad1f24c38be921c90ba
H A Dselectable.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 707e5d70 Tue Jul 06 09:26:53 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> labeling refactor

To service #6718 and #6710, the system by which columns are
given labels in a SELECT statement as well as the system that
gives them keys in a .c or .selected_columns collection have
been refactored to provide a single source of truth for
both, in constrast to the previous approach that included
similar logic repeated in slightly different ways.

Main ideas:

1. ColumnElement attributes ._label, ._anon_label, ._key_label
are renamed to include the letters "tq", meaning
"table-qualified" - these labels are only used when rendering
a SELECT that has LABEL_STYLE_TABLENAME_PLUS_COL for its
label style; as this label style is primarily legacy, the
"tq" names should be isolated so that in a 2.0 style application
these aren't being used at all

2. The means by which the "labels" and "proxy keys" for the elements
of a SELECT has been centralized to a single source of truth;
previously, the three of _generate_columns_plus_names,
_generate_fromclause_column_proxies, and _column_naming_convention
all had duplicated rules between them, as well as that there
were a little bit of labeling rules in compiler._label_select_column
as well; by this we mean that the various "anon_label" "anon_key"
methods on ColumnElement were called by all four of these methods,
where there were many cases where it was necessary that one method
comes up with the same answer as another of the methods. This
has all been centralized into _generate_columns_plus_names
for all the names except the "proxy key", which is generated
by _column_naming_convention.

3. compiler._label_select_column has been rewritten to both not make
any naming decisions nor any "proxy key" decisions, only whether
to label or not to label; the _generate_columns_plus_names method
gives it the information, where the proxy keys come from
_column_naming_convention; previously, these proxy keys were matched
based on restatement of similar (but not really the same) logic in
two places. The heuristics of "whether to label or not to label"
are also reorganized to be much easier to read and understand.

4. a new method compiler._label_returning_column is added for dialects
to use in their "generate returning columns" methods. A
github search reveals a small number of third party dialects also
doing this using the prior _label_select_column method so we
try to make sure _label_select_column continues to work the
exact same way for that specific use case; for the "SELECT" use
case it now needs

5. After some attempts to do it different ways, for the case where
_proxy_key is giving us some kind of anon label, we are hard
changing it to "_no_label" right now, as there's not currently
a way to fully match anonymized labels from stmt.c or
stmt.selected_columns to what will be in the result map. The
idea of "_no_label" is to encourage the user to use label('name')
for columns they want to be able to target by string name that
don't have a natural name.

Change-Id: I7a92a66f3a7e459ccf32587ac0a3c306650daf11
diff 20c0f774 Thu Apr 01 15:20:31 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> Default caching to opt-out for 3rd party dialects

Added a new flag to the :class:`_engine.Dialect` class called
:attr:`_engine.Dialect.supports_statement_cache`. This flag now needs to be present
directly on a dialect class in order for SQLAlchemy's
:ref:`query cache <sql_caching>` to take effect for that dialect. The
rationale is based on discovered issues such as :ticket:`6173` revealing
that dialects which hardcode literal values from the compiled statement,
often the numerical parameters used for LIMIT / OFFSET, will not be
compatible with caching until these dialects are revised to use the
parameters present in the statement only. For third party dialects where
this flag is not applied, the SQL logging will show the message "dialect
does not support caching", indicating the dialect should seek to apply this
flag once they have verified that no per-statement literal values are being
rendered within the compilation phase.

Fixes: #6184
Change-Id: I6fd5b5d94200458d4cb0e14f2f556dbc25e27e22
diff 3dc9a4a2 Mon Dec 16 15:06:43 MST 2019 Mike Bayer <mike_mp@zzzcomputing.com> introduce deferred lambdas

The coercions system allows us to add in lambdas as arguments
to Core and ORM elements without changing them at all. By allowing
the lambda to produce a deterministic cache key where we can also
cheat and yank out literal parameters means we can move towards
having 90% of "baked" functionality in a clearer way right in
Core / ORM.

As a second step, we can have whole statements inside the lambda,
and can then add generation with __add__(), so then we have
100% of "baked" functionality with full support of ad-hoc
literal values.

Adds some more short_selects tests for the moment for comparison.

Other tweaks inside cache key generation as we're trying to
approach a certain level of performance such that we can
remove the use of "baked" from the loader strategies.

As we have not yet closed #4639, however the caching feature
has been fully integrated as of
b0cfa7379cf8513a821a3dbe3028c4965d9f85bd, we will also
add complete caching documentation here and close that issue
as well.

Closes: #4639
Fixes: #5380
Change-Id: If91f61527236fd4d7ae3cad1f24c38be921c90ba
diff 3ab2364e Wed Jun 03 15:38:35 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Convert bulk update/delete to new execution model

This reorganizes the BulkUD model in sqlalchemy.orm.persistence
to be based on the CompileState concept and to allow plain
update() / delete() to be passed to session.execute() where
the ORM synchronize session logic will take place.
Also gets "synchronize_session='fetch'" working with horizontal
sharding.

Adding a few more result.scalar_one() types of methods
as scalar_one() seems like what is normally desired.

Fixes: #5160
Change-Id: I8001ebdad089da34119eb459709731ba6c0ba975
diff 3d55263c Tue May 19 16:55:16 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> fix table.schema version to 1.3.18

Fixes: #5338
Change-Id: I12ce8021f5627c306c768b74da991ef4af0a5b55
diff 3cb61400 Sat Feb 22 11:11:20 MST 2020 Mike Bayer <mike_mp@zzzcomputing.com> Ensure descendants of mixins don't become cacheable

HasPrefix / HasSuffixes / SupportsCloneAnnotations exported
a _traverse_internals attribute that does not represent a
complete traversal, meaning non-traversible subclasses would
seem traversible. rename these attributes so that this
does not occur. DML is currently not traversible (will be soon).

Change-Id: I2605e61c8c3d49965335e66e09f4aeedc5e73bd3
diff 3b85eeb5 Tue Sep 10 10:46:21 MDT 2019 Mike Bayer <mike_mp@zzzcomputing.com> Restore subquery.as_scalar() w/ deprecation

Apparently Alias had an .as_scalar() method, so restore an
equivalent to Subquery with an appropriate deprecation warning.

Fixes: #4854
Change-Id: I6255d61b7d82487ca90ba8ee79d4b3a74e7cbe38
diff 3b60ccae Mon Jul 01 11:09:44 MDT 2019 Mike Bayer <mike_mp@zzzcomputing.com> A few doc tweaks for alias / order_by / group_by

Change-Id: Ib3b46b45735529d68ebfb3784de4de5d2d0f4abc
H A Dcompiler.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 79a3dafb Wed Jul 14 08:29:50 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> Ensure alias traversal block works when adapt_from_selectables present

Fixed regression which appeared in version 1.4.3 due to :ticket:`6060`
where rules that limit ORM adaptation of derived selectables interfered
with other ORM-adaptation based cases, in this case when applying
adaptations for a :func:`_orm.with_polymorphic` against a mapping which
uses a :func:`_orm.column_property` which in turn makes use of a scalar
select that includes a :func:`_orm.aliased` object of the mapped table.

Fixes: #6762
Change-Id: Ice3dc34b97d12b59f044bdc0c5faaefcc4015227
diff 707e5d70 Tue Jul 06 09:26:53 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> labeling refactor

To service #6718 and #6710, the system by which columns are
given labels in a SELECT statement as well as the system that
gives them keys in a .c or .selected_columns collection have
been refactored to provide a single source of truth for
both, in constrast to the previous approach that included
similar logic repeated in slightly different ways.

Main ideas:

1. ColumnElement attributes ._label, ._anon_label, ._key_label
are renamed to include the letters "tq", meaning
"table-qualified" - these labels are only used when rendering
a SELECT that has LABEL_STYLE_TABLENAME_PLUS_COL for its
label style; as this label style is primarily legacy, the
"tq" names should be isolated so that in a 2.0 style application
these aren't being used at all

2. The means by which the "labels" and "proxy keys" for the elements
of a SELECT has been centralized to a single source of truth;
previously, the three of _generate_columns_plus_names,
_generate_fromclause_column_proxies, and _column_naming_convention
all had duplicated rules between them, as well as that there
were a little bit of labeling rules in compiler._label_select_column
as well; by this we mean that the various "anon_label" "anon_key"
methods on ColumnElement were called by all four of these methods,
where there were many cases where it was necessary that one method
comes up with the same answer as another of the methods. This
has all been centralized into _generate_columns_plus_names
for all the names except the "proxy key", which is generated
by _column_naming_convention.

3. compiler._label_select_column has been rewritten to both not make
any naming decisions nor any "proxy key" decisions, only whether
to label or not to label; the _generate_columns_plus_names method
gives it the information, where the proxy keys come from
_column_naming_convention; previously, these proxy keys were matched
based on restatement of similar (but not really the same) logic in
two places. The heuristics of "whether to label or not to label"
are also reorganized to be much easier to read and understand.

4. a new method compiler._label_returning_column is added for dialects
to use in their "generate returning columns" methods. A
github search reveals a small number of third party dialects also
doing this using the prior _label_select_column method so we
try to make sure _label_select_column continues to work the
exact same way for that specific use case; for the "SELECT" use
case it now needs

5. After some attempts to do it different ways, for the case where
_proxy_key is giving us some kind of anon label, we are hard
changing it to "_no_label" right now, as there's not currently
a way to fully match anonymized labels from stmt.c or
stmt.selected_columns to what will be in the result map. The
idea of "_no_label" is to encourage the user to use label('name')
for columns they want to be able to target by string name that
don't have a natural name.

Change-Id: I7a92a66f3a7e459ccf32587ac0a3c306650daf11
diff 3dcdc304 Wed Apr 28 10:14:33 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> accommodate for mutiple copies of bind in ckbm

Fixed critical regression where bound parameter tracking as used in the SQL
caching system could fail to track all parameters for the case where the
same SQL expression containing a parameter were used in an ORM-related
query using a feature such as class inheritance, which was then embedded in
an enclosing expression which would make use of that same expression
multiple times, such as a UNION. The ORM would individually copy the
individual SELECT statements as part of compilation with class inheritance,
which then embedded in the enclosing statement would fail to accommodate
for all parameters. The logic that tracks this condition has been adjusted
to work for multiple copies of a parameter.

Fixes: #6391
Change-Id: I6db5dee0d361a3bb58d753a2d27ef2eee2b369c5
diff 3a0a6e1d Tue Apr 20 13:09:51 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> Propagate compiler kw for visit_values to parameters

Fixed issue in SQL compiler where the bound parameters set up for a
:class:`.Values` construct wouldn't be positionally tracked correctly if
inside of a :class:`_sql.CTE`, affecting database drivers that support
VALUES + ctes and use positional parameters such as SQL Server in
particular as well as asyncpg. The fix also repairs support for
compiler flags such as ``literal_binds``.

Fixes: #6327
Change-Id: I2d549228691d0bfc10dadd0955b1549d7584db51
diff 3ebc3710 Thu Jan 07 13:22:52 MST 2021 Federico Caselli <cfederico87@gmail.com> ``Identity`` implies ``nullable=False``.

Altered the behavior of the :class:`_schema.Identity` construct such that
when applied to a :class:`_schema.Column`, it will automatically imply that
the value of :paramref:`_sql.Column.nullable` should default to ``False``,
in a similar manner as when the :paramref:`_sql.Column.primary_key`
parameter is set to ``True``. This matches the default behavior of all
supporting databases where ``IDENTITY`` implies ``NOT NULL``. The
PostgreSQL backend is the only one that supports adding ``NULL`` to an
``IDENTITY`` column, which is here supported by passing a ``True`` value
for the :paramref:`_sql.Column.nullable` parameter at the same time.

Fixes: #5775
Change-Id: I0516d506ff327cff35cda605e8897a27440e0373
diff 3e49b8d0 Mon Oct 19 08:19:29 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Ensure no compiler visit method tries to access .statement

Fixed structural compiler issue where some constructs such as MySQL /
PostgreSQL "on conflict / on duplicate key" would rely upon the state of
the :class:`_sql.Compiler` object being fixed against their statement as
the top level statement, which would fail in cases where those statements
are branched from a different context, such as a DDL construct linked to a
SQL statement.

Fixes: #5656
Change-Id: I568bf40adc7edcf72ea6c7fd6eb9d07790de189e
diff 3b4bbbb2 Sat Aug 15 13:08:09 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Create a real type for Tuple() and handle appropriately in compiler

Improved the :func:`_sql.tuple_` construct such that it behaves predictably
when used in a columns-clause context. The SQL tuple is not supported as a
"SELECT" columns clause element on most backends; on those that do
(PostgreSQL, not surprisingly), the Python DBAPI does not have a "nested
type" concept so there are still challenges in fetching rows for such an
object. Use of :func:`_sql.tuple_` in a :func:`_sql.select` or
:class:`_orm.Query` will now raise a :class:`_exc.CompileError` at the
point at which the :func:`_sql.tuple_` object is seen as presenting itself
for fetching rows (i.e., if the tuple is in the columns clause of a
subquery, no error is raised). For ORM use,the :class:`_orm.Bundle` object
is an explicit directive that a series of columns should be returned as a
sub-tuple per row and is suggested by the error message. Additionally ,the
tuple will now render with parenthesis in all contexts. Previously, the
parenthesization would not render in a columns context leading to
non-defined behavior.

As part of this change, Tuple receives a dedicated datatype
which appears to allow us the very desirable change of removing
the bindparam._expanding_in_types attribute as well as
ClauseList._tuple_values (which might already have not been
needed due to #4645).

Fixes: #5127
Change-Id: Iecafa0e0aac2f1f37ec8d0e1631d562611c90200
diff 3dc9a4a2 Mon Dec 16 15:06:43 MST 2019 Mike Bayer <mike_mp@zzzcomputing.com> introduce deferred lambdas

The coercions system allows us to add in lambdas as arguments
to Core and ORM elements without changing them at all. By allowing
the lambda to produce a deterministic cache key where we can also
cheat and yank out literal parameters means we can move towards
having 90% of "baked" functionality in a clearer way right in
Core / ORM.

As a second step, we can have whole statements inside the lambda,
and can then add generation with __add__(), so then we have
100% of "baked" functionality with full support of ad-hoc
literal values.

Adds some more short_selects tests for the moment for comparison.

Other tweaks inside cache key generation as we're trying to
approach a certain level of performance such that we can
remove the use of "baked" from the loader strategies.

As we have not yet closed #4639, however the caching feature
has been fully integrated as of
b0cfa7379cf8513a821a3dbe3028c4965d9f85bd, we will also
add complete caching documentation here and close that issue
as well.

Closes: #4639
Fixes: #5380
Change-Id: If91f61527236fd4d7ae3cad1f24c38be921c90ba
/sqlalchemy/lib/sqlalchemy/orm/
H A Ddecl_api.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
450f5c0d Mon Aug 31 09:46:55 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Build out new declarative systems; deprecate mapper()

The ORM Declarative system is now unified into the ORM itself, with new
import spaces under ``sqlalchemy.orm`` and new kinds of mappings. Support
for decorator-based mappings without using a base class, support for
classical style-mapper() calls that have access to the declarative class
registry for relationships, and full integration of Declarative with 3rd
party class attribute systems like ``dataclasses`` and ``attrs`` is now
supported.

Fixes: #5508
Change-Id: I130b2b6edff6450bfe8a3e6baa099ff04b5471ff
H A Dpath_registry.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
H A Drelationships.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 450f5c0d Mon Aug 31 09:46:55 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Build out new declarative systems; deprecate mapper()

The ORM Declarative system is now unified into the ORM itself, with new
import spaces under ``sqlalchemy.orm`` and new kinds of mappings. Support
for decorator-based mappings without using a base class, support for
classical style-mapper() calls that have access to the declarative class
registry for relationships, and full integration of Declarative with 3rd
party class attribute systems like ``dataclasses`` and ``attrs`` is now
supported.

Fixes: #5508
Change-Id: I130b2b6edff6450bfe8a3e6baa099ff04b5471ff
diff 4c2c2c40 Thu Dec 20 20:05:36 MST 2018 Mike Bayer <mike_mp@zzzcomputing.com> Add deprecation warnings to all deprecated APIs

A large change throughout the library has ensured that all objects, parameters,
and behaviors which have been noted as deprecated or legacy now emit
``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now
defaults to displaying deprecation warnings, as well as that modern test suites
based on tools like tox and pytest tend to display deprecation warnings,
this change should make it easier to note what API features are obsolete.

See the notes added to the changelog and migration notes for further
details.

Fixes: #4393
Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
diff 3ec9b9f6 Fri Dec 04 09:52:16 MST 2015 Mike Bayer <mike_mp@zzzcomputing.com> - The :meth:`.Session.merge` method now tracks pending objects by
primary key before emitting an INSERT, and merges distinct objects with
duplicate primary keys together as they are encountered, which is
essentially semi-deterministic at best. This behavior
matches what happens already with persistent objects.
fixes #3601
/sqlalchemy/test/sql/
H A Dtest_type_expressions.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3bb402ff Tue Aug 27 15:08:11 MDT 2019 Mike Bayer <mike_mp@zzzcomputing.com> Label simple column transformations as the column name

Additional logic has been added such that certain SQL expressions which
typically wrap a single database column will use the name of that column as
their "anonymous label" name within a SELECT statement, potentially making
key-based lookups in result tuples more intutive. The primary example of
this is that of a CAST expression, e.g. ``CAST(table.colname AS INTEGER)``,
which will export its default name as "colname", rather than the usual
"anon_1" label, that is, ``CAST(table.colname AS INTEGER) AS colname``.
If the inner expression doesn't have a name, then the previous "anonymous
label" logic is used. When using SELECT statements that make use of
:meth:`.Select.apply_labels`, such as those emitted by the ORM, the
labeling logic will produce ``<tablename>_<inner column name>`` in the same
was as if the column were named alone. The logic applies right now to the
:func:`.cast` and :func:`.type_coerce` constructs as well as some
single-element boolean expressions.

Fixes: #4449
Change-Id: Ie3b73470e3bea53f2386cd86514cdc556491564e
H A Dtest_compare.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff f1e96cb0 Sun Jan 10 11:44:14 MST 2021 Mike Bayer <mike_mp@zzzcomputing.com> reinvent xdist hooks in terms of pytest fixtures

To allow the "connection" pytest fixture and others work
correctly in conjunction with setup/teardown that expects
to be external to the transaction, remove and prevent any usage
of "xdist" style names that are hardcoded by pytest to run
inside of fixtures, even function level ones. Instead use
pytest autouse fixtures to implement our own
r"setup|teardown_test(?:_class)?" methods so that we can ensure
function-scoped fixtures are run within them. A new more
explicit flow is set up within plugin_base and pytestplugin
such that the order of setup/teardown steps, which there are now
many, is fully documented and controllable. New granularity
has been added to the test teardown phase to distinguish
between "end of the test" when lock-holding structures on
connections should be released to allow for table drops,
vs. "end of the test plus its teardown steps" when we can
perform final cleanup on connections and run assertions
that everything is closed out.

From there we can remove most of the defensive "tear down everything"
logic inside of engines which for many years would frequently dispose
of pools over and over again, creating for a broken and expensive
connection flow. A quick test shows that running test/sql/ against
a single Postgresql engine with the new approach uses 75% fewer new
connections, creating 42 new connections total, vs. 164 new
connections total with the previous system.

As part of this, the new fixtures metadata/connection/future_connection
have been integrated such that they can be combined together
effectively. The fixture_session(), provide_metadata() fixtures
have been improved, including that fixture_session() now strongly
references sessions which are explicitly torn down before
table drops occur afer a test.

Major changes have been made to the
ConnectionKiller such that it now features different "scopes" for
testing engines and will limit its cleanup to those testing
engines corresponding to end of test, end of test class, or
end of test session. The system by which it tracks DBAPI
connections has been reworked, is ultimately somewhat similar to
how it worked before but is organized more clearly along
with the proxy-tracking logic. A "testing_engine" fixture
is also added that works as a pytest fixture rather than a
standalone function. The connection cleanup logic should
now be very robust, as we now can use the same global
connection pools for the whole suite without ever disposing
them, while also running a query for PostgreSQL
locks remaining after every test and assert there are no open
transactions leaking between tests at all. Additional steps
are added that also accommodate for asyncio connections not
explicitly closed, as is the case for legacy sync-style
tests as well as the async tests themselves.

As always, hundreds of tests are further refined to use the
new fixtures where problems with loose connections were identified,
largely as a result of the new PostgreSQL assertions,
many more tests have moved from legacy patterns into the newest.

An unfortunate discovery during the creation of this system is that
autouse fixtures (as well as if they are set up by
@pytest.mark.usefixtures) are not usable at our current scale with pytest
4.6.11 running under Python 2. It's unclear if this is due
to the older version of pytest or how it implements itself for
Python 2, as well as if the issue is CPU slowness or just large
memory use, but collecting the full span of tests takes over
a minute for a single process when any autouse fixtures are in
place and on CI the jobs just time out after ten minutes.
So at the moment this patch also reinvents a small version of
"autouse" fixtures when py2k is running, which skips generating
the real fixture and instead uses two global pytest fixtures
(which don't seem to impact performance) to invoke the
"autouse" fixtures ourselves outside of pytest.
This will limit our ability to do more with fixtures
until we can remove py2k support.

py.test is still observed to be much slower in collection in the
4.6.11 version compared to modern 6.2 versions, so add support for new
TOX_POSTGRESQL_PY2K and TOX_MYSQL_PY2K environment variables that
will run the suite for fewer backends under Python 2. For Python 3
pin pytest to modern 6.2 versions where performance for collection
has been improved greatly.

Includes the following improvements:

Fixed bug in asyncio connection pool where ``asyncio.TimeoutError`` would
be raised rather than :class:`.exc.TimeoutError`. Also repaired the
:paramref:`_sa.create_engine.pool_timeout` parameter set to zero when using
the async engine, which previously would ignore the timeout and block
rather than timing out immediately as is the behavior with regular
:class:`.QueuePool`.

For asyncio the connection pool will now also not interact
at all with an asyncio connection whose ConnectionFairy is
being garbage collected; a warning that the connection was
not properly closed is emitted and the connection is discarded.
Within the test suite the ConnectionKiller is now maintaining
strong references to all DBAPI connections and ensuring they
are released when tests end, including those whose ConnectionFairy
proxies are GCed.

Identified cx_Oracle.stmtcachesize as a major factor in Oracle
test scalability issues, this can be reset on a per-test basis
rather than setting it to zero across the board. the addition
of this flag has resolved the long-standing oracle "two task"
error problem.

For SQL Server, changed the temp table style used by the
"suite" tests to be the double-pound-sign, i.e. global,
variety, which is much easier to test generically. There
are already reflection tests that are more finely tuned
to both styles of temp table within the mssql test
suite. Additionally, added an extra step to the
"dropfirst" mechanism for SQL Server that will remove
all foreign key constraints first as some issues were
observed when using this flag when multiple schemas
had not been torn down.

Identified and fixed two subtle failure modes in the
engine, when commit/rollback fails in a begin()
context manager, the connection is explicitly closed,
and when "initialize()" fails on the first new connection
of a dialect, the transactional state on that connection
is still rolled back.

Fixes: #5826
Fixes: #5827
Change-Id: Ib1d05cb8c7cf84f9a4bfd23df397dc23c9329bfe
diff 77c9534d Sat Dec 12 16:56:58 MST 2020 Mike Bayer <mike_mp@zzzcomputing.com> Major revisals to lambdas

1. Improve coercions._deep_is_literal to check sequences
for clause elements, thus allowing a phrase like
lambda: col.in_([literal("x"), literal("y")]) to be handled

2. revise closure variable caching completely. All variables
entering must be part of a closure cache key or rejected.
only objects that can be resolved to HasCacheKey or FunctionType
are accepted; all other types are rejected. This adds a high
degree of strictness to lambdas and will make them a little more
awkward to use in some cases, however prevents several classes
of critical issues:

a. previously, a lambda that had an expression derived from
some kind of state, like "self.x", or "execution_context.session.foo"
would produce a closure cache key from "self" or "execution_context",
objects that can very well be per-execution and would therefore
cause a AnalyzedFunction objects to overflow. (memory won't leak
as it looks like an LRUCache is already used for these)

b. a lambda, such as one used within DeferredLamdaElement, that
produces different SQL expressions based on the arguments
(which is in fact what it's supposed to do), however it would
through the use of conditionals produce different bound parameter
combinations, leading to literal parameters not tracked properly.
These are now rejected as uncacheable whereas previously they would
again be part of the closure cache key, causing an overflow of
AnalyizedFunction objects.

3. Ensure non-mapped mixins are handled correctly by
with_loader_criteria().

4. Fixed bug in lambda SQL system where we are not supposed to allow a Python
function to be embedded in the lambda, since we can't predict a bound value
from it. While there was an error condition added for this, it was not
tested and wasn't working; an informative error is now raised.

5. new docs for lambdas

6. consolidated changelog for all of these

Fixes: #5760
Fixes: #5765
Fixes: #5766
Fixes: #5768
Fixes: #5770
Change-Id: Iedaa636c3225fad496df23b612c516c8ab247ab7
diff 3dc9a4a2 Mon Dec 16 15:06:43 MST 2019 Mike Bayer <mike_mp@zzzcomputing.com> introduce deferred lambdas

The coercions system allows us to add in lambdas as arguments
to Core and ORM elements without changing them at all. By allowing
the lambda to produce a deterministic cache key where we can also
cheat and yank out literal parameters means we can move towards
having 90% of "baked" functionality in a clearer way right in
Core / ORM.

As a second step, we can have whole statements inside the lambda,
and can then add generation with __add__(), so then we have
100% of "baked" functionality with full support of ad-hoc
literal values.

Adds some more short_selects tests for the moment for comparison.

Other tweaks inside cache key generation as we're trying to
approach a certain level of performance such that we can
remove the use of "baked" from the loader strategies.

As we have not yet closed #4639, however the caching feature
has been fully integrated as of
b0cfa7379cf8513a821a3dbe3028c4965d9f85bd, we will also
add complete caching documentation here and close that issue
as well.

Closes: #4639
Fixes: #5380
Change-Id: If91f61527236fd4d7ae3cad1f24c38be921c90ba
diff 3ab2364e Wed Jun 03 15:38:35 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Convert bulk update/delete to new execution model

This reorganizes the BulkUD model in sqlalchemy.orm.persistence
to be based on the CompileState concept and to allow plain
update() / delete() to be passed to session.execute() where
the ORM synchronize session logic will take place.
Also gets "synchronize_session='fetch'" working with horizontal
sharding.

Adding a few more result.scalar_one() types of methods
as scalar_one() seems like what is normally desired.

Fixes: #5160
Change-Id: I8001ebdad089da34119eb459709731ba6c0ba975
diff 3cb61400 Sat Feb 22 11:11:20 MST 2020 Mike Bayer <mike_mp@zzzcomputing.com> Ensure descendants of mixins don't become cacheable

HasPrefix / HasSuffixes / SupportsCloneAnnotations exported
a _traverse_internals attribute that does not represent a
complete traversal, meaning non-traversible subclasses would
seem traversible. rename these attributes so that this
does not occur. DML is currently not traversible (will be soon).

Change-Id: I2605e61c8c3d49965335e66e09f4aeedc5e73bd3
H A Dtest_types.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3d284bb1 Mon Mar 29 15:57:20 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> TypeDecorator passes "outer" flag to itself for set_parent accounting

Fixed bug first introduced in as some combination of :ticket:`2892`,
:ticket:`2919` nnd :ticket:`3832` where the attachment events for a
:class:`_types.TypeDecorator` would be doubled up against the "impl" class,
if the "impl" were also a :class:`_types.SchemaType`. The real-world case
is any :class:`_types.TypeDecorator` against :class:`_types.Enum` or
:class:`_types.Boolean` would get a doubled
:class:`_schema.CheckConstraint` when the ``create_constraint=True`` flag
is set.

Fixes: #6152
Change-Id: I3218b7081297270c132421f6765b5c3673d10a5c
diff f1e96cb0 Sun Jan 10 11:44:14 MST 2021 Mike Bayer <mike_mp@zzzcomputing.com> reinvent xdist hooks in terms of pytest fixtures

To allow the "connection" pytest fixture and others work
correctly in conjunction with setup/teardown that expects
to be external to the transaction, remove and prevent any usage
of "xdist" style names that are hardcoded by pytest to run
inside of fixtures, even function level ones. Instead use
pytest autouse fixtures to implement our own
r"setup|teardown_test(?:_class)?" methods so that we can ensure
function-scoped fixtures are run within them. A new more
explicit flow is set up within plugin_base and pytestplugin
such that the order of setup/teardown steps, which there are now
many, is fully documented and controllable. New granularity
has been added to the test teardown phase to distinguish
between "end of the test" when lock-holding structures on
connections should be released to allow for table drops,
vs. "end of the test plus its teardown steps" when we can
perform final cleanup on connections and run assertions
that everything is closed out.

From there we can remove most of the defensive "tear down everything"
logic inside of engines which for many years would frequently dispose
of pools over and over again, creating for a broken and expensive
connection flow. A quick test shows that running test/sql/ against
a single Postgresql engine with the new approach uses 75% fewer new
connections, creating 42 new connections total, vs. 164 new
connections total with the previous system.

As part of this, the new fixtures metadata/connection/future_connection
have been integrated such that they can be combined together
effectively. The fixture_session(), provide_metadata() fixtures
have been improved, including that fixture_session() now strongly
references sessions which are explicitly torn down before
table drops occur afer a test.

Major changes have been made to the
ConnectionKiller such that it now features different "scopes" for
testing engines and will limit its cleanup to those testing
engines corresponding to end of test, end of test class, or
end of test session. The system by which it tracks DBAPI
connections has been reworked, is ultimately somewhat similar to
how it worked before but is organized more clearly along
with the proxy-tracking logic. A "testing_engine" fixture
is also added that works as a pytest fixture rather than a
standalone function. The connection cleanup logic should
now be very robust, as we now can use the same global
connection pools for the whole suite without ever disposing
them, while also running a query for PostgreSQL
locks remaining after every test and assert there are no open
transactions leaking between tests at all. Additional steps
are added that also accommodate for asyncio connections not
explicitly closed, as is the case for legacy sync-style
tests as well as the async tests themselves.

As always, hundreds of tests are further refined to use the
new fixtures where problems with loose connections were identified,
largely as a result of the new PostgreSQL assertions,
many more tests have moved from legacy patterns into the newest.

An unfortunate discovery during the creation of this system is that
autouse fixtures (as well as if they are set up by
@pytest.mark.usefixtures) are not usable at our current scale with pytest
4.6.11 running under Python 2. It's unclear if this is due
to the older version of pytest or how it implements itself for
Python 2, as well as if the issue is CPU slowness or just large
memory use, but collecting the full span of tests takes over
a minute for a single process when any autouse fixtures are in
place and on CI the jobs just time out after ten minutes.
So at the moment this patch also reinvents a small version of
"autouse" fixtures when py2k is running, which skips generating
the real fixture and instead uses two global pytest fixtures
(which don't seem to impact performance) to invoke the
"autouse" fixtures ourselves outside of pytest.
This will limit our ability to do more with fixtures
until we can remove py2k support.

py.test is still observed to be much slower in collection in the
4.6.11 version compared to modern 6.2 versions, so add support for new
TOX_POSTGRESQL_PY2K and TOX_MYSQL_PY2K environment variables that
will run the suite for fewer backends under Python 2. For Python 3
pin pytest to modern 6.2 versions where performance for collection
has been improved greatly.

Includes the following improvements:

Fixed bug in asyncio connection pool where ``asyncio.TimeoutError`` would
be raised rather than :class:`.exc.TimeoutError`. Also repaired the
:paramref:`_sa.create_engine.pool_timeout` parameter set to zero when using
the async engine, which previously would ignore the timeout and block
rather than timing out immediately as is the behavior with regular
:class:`.QueuePool`.

For asyncio the connection pool will now also not interact
at all with an asyncio connection whose ConnectionFairy is
being garbage collected; a warning that the connection was
not properly closed is emitted and the connection is discarded.
Within the test suite the ConnectionKiller is now maintaining
strong references to all DBAPI connections and ensuring they
are released when tests end, including those whose ConnectionFairy
proxies are GCed.

Identified cx_Oracle.stmtcachesize as a major factor in Oracle
test scalability issues, this can be reset on a per-test basis
rather than setting it to zero across the board. the addition
of this flag has resolved the long-standing oracle "two task"
error problem.

For SQL Server, changed the temp table style used by the
"suite" tests to be the double-pound-sign, i.e. global,
variety, which is much easier to test generically. There
are already reflection tests that are more finely tuned
to both styles of temp table within the mssql test
suite. Additionally, added an extra step to the
"dropfirst" mechanism for SQL Server that will remove
all foreign key constraints first as some issues were
observed when using this flag when multiple schemas
had not been torn down.

Identified and fixed two subtle failure modes in the
engine, when commit/rollback fails in a begin()
context manager, the connection is explicitly closed,
and when "initialize()" fails on the first new connection
of a dialect, the transactional state on that connection
is still rolled back.

Fixes: #5826
Fixes: #5827
Change-Id: Ib1d05cb8c7cf84f9a4bfd23df397dc23c9329bfe
diff dfb94052 Tue Oct 06 21:46:03 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> modernize requirements for boolean constraint test

this test was not actually working correctly against
mariadb and was "failing" (i.e. "passing") inappropriately
for pymysql, mysqlclient. it also started "passing" (i.e. "failing")
for mariadb as of 1.0.3.

modernize reqs here including for mssql bit behavior.

Change-Id: Ia88ee607ec8d59f6140a7e4a1174910382d53674
diff fc97854f Thu Aug 13 22:58:56 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Bump minimum MySQL version to 5.0.2; use all-numeric server version

MySQL dialect's server_version_info tuple is now all numeric. String
tokens like "MariaDB" are no longer present so that numeric comparison
works in all cases. The .is_mariadb flag on the dialect should be
consulted for whether or not mariadb was detected. Additionally removed
structures meant to support extremely old MySQL versions 3.x and 4.x;
the minimum MySQL version supported is now version 5.0.2.

In addition, as the "MariaDB" name goes away from server version,
expand upon the change in I330815ebe572b6a9818377da56621397335fa702
to support the name "mariadb" throughout the dialect and test suite
when mariadb-only mode is used. This changes the "name" field
on the MariaDB dialect to "mariadb", which then implies a change
throughout the testing requirements system as well as all the
dialect-specific DDL argument names such as "mysql_engine" is
now specified as "mariadb_engine", etc. Make use of the
recent additions to test suite URL provisioning so that we can
force MariaDB databases to have a "mariadb-only" dialect which
allows us to test this name change fully.

Update documentation to refer to MySQL / MariaDB explicitly
as well as indicating the "mariadb_" prefix used for options.

It seems likely that MySQL and MariaDB version numbers are going to
start colliding at some point so having the "mariadb" name
be available as a totally separate dialect name should give us
some options in this regard.

Currently also includes a date related fix to a test for
the postgresql dialect that was implicitly assuming a
non-UTC timezone

Fixes: #4189
Change-Id: I00e76d00f62971e1f067bd61915fa6cc1cf64e5e
diff af0e59ff Thu May 07 08:53:15 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Disable "check unicode returns" under Python 3

Disabled the "unicode returns" check that runs on dialect startup when
running under Python 3, which for many years has occurred in order to test
the current DBAPI's behavior for whether or not it returns Python Unicode
or Py2K strings for the VARCHAR and NVARCHAR datatypes. The check still
occurs by default under Python 2, however the mechanism to test the
behavior will be removed in SQLAlchemy 2.0 when Python 2 support is also
removed.

This logic was very effective when it was needed, however now that Python 3
is standard, all DBAPIs are expected to return Python 3 strings for
character datatypes. In the unlikely case that a third party DBAPI does
not support this, the conversion logic within :class:`.String` is still
available and the third party dialect may specify this in its upfront
dialect flags by setting the dialect level flag ``returns_unicode_strings``
to one of :attr:`.String.RETURNS_CONDITIONAL` or
:attr:`.String.RETURNS_BYTES`, both of which will enable Unicode conversion
even under Python 3.

As part of this change, disabling testing of the doctest tutorials under
Python 2.

Fixes: #5315
Change-Id: I1260e894611409d3b7fe1a92bd90c52043bbcf19
diff af0e59ff Thu May 07 08:53:15 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Disable "check unicode returns" under Python 3

Disabled the "unicode returns" check that runs on dialect startup when
running under Python 3, which for many years has occurred in order to test
the current DBAPI's behavior for whether or not it returns Python Unicode
or Py2K strings for the VARCHAR and NVARCHAR datatypes. The check still
occurs by default under Python 2, however the mechanism to test the
behavior will be removed in SQLAlchemy 2.0 when Python 2 support is also
removed.

This logic was very effective when it was needed, however now that Python 3
is standard, all DBAPIs are expected to return Python 3 strings for
character datatypes. In the unlikely case that a third party DBAPI does
not support this, the conversion logic within :class:`.String` is still
available and the third party dialect may specify this in its upfront
dialect flags by setting the dialect level flag ``returns_unicode_strings``
to one of :attr:`.String.RETURNS_CONDITIONAL` or
:attr:`.String.RETURNS_BYTES`, both of which will enable Unicode conversion
even under Python 3.

As part of this change, disabling testing of the doctest tutorials under
Python 2.

Fixes: #5315
Change-Id: I1260e894611409d3b7fe1a92bd90c52043bbcf19
diff af0e59ff Thu May 07 08:53:15 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Disable "check unicode returns" under Python 3

Disabled the "unicode returns" check that runs on dialect startup when
running under Python 3, which for many years has occurred in order to test
the current DBAPI's behavior for whether or not it returns Python Unicode
or Py2K strings for the VARCHAR and NVARCHAR datatypes. The check still
occurs by default under Python 2, however the mechanism to test the
behavior will be removed in SQLAlchemy 2.0 when Python 2 support is also
removed.

This logic was very effective when it was needed, however now that Python 3
is standard, all DBAPIs are expected to return Python 3 strings for
character datatypes. In the unlikely case that a third party DBAPI does
not support this, the conversion logic within :class:`.String` is still
available and the third party dialect may specify this in its upfront
dialect flags by setting the dialect level flag ``returns_unicode_strings``
to one of :attr:`.String.RETURNS_CONDITIONAL` or
:attr:`.String.RETURNS_BYTES`, both of which will enable Unicode conversion
even under Python 3.

As part of this change, disabling testing of the doctest tutorials under
Python 2.

Fixes: #5315
Change-Id: I1260e894611409d3b7fe1a92bd90c52043bbcf19
diff af0e59ff Thu May 07 08:53:15 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Disable "check unicode returns" under Python 3

Disabled the "unicode returns" check that runs on dialect startup when
running under Python 3, which for many years has occurred in order to test
the current DBAPI's behavior for whether or not it returns Python Unicode
or Py2K strings for the VARCHAR and NVARCHAR datatypes. The check still
occurs by default under Python 2, however the mechanism to test the
behavior will be removed in SQLAlchemy 2.0 when Python 2 support is also
removed.

This logic was very effective when it was needed, however now that Python 3
is standard, all DBAPIs are expected to return Python 3 strings for
character datatypes. In the unlikely case that a third party DBAPI does
not support this, the conversion logic within :class:`.String` is still
available and the third party dialect may specify this in its upfront
dialect flags by setting the dialect level flag ``returns_unicode_strings``
to one of :attr:`.String.RETURNS_CONDITIONAL` or
:attr:`.String.RETURNS_BYTES`, both of which will enable Unicode conversion
even under Python 3.

As part of this change, disabling testing of the doctest tutorials under
Python 2.

Fixes: #5315
Change-Id: I1260e894611409d3b7fe1a92bd90c52043bbcf19
diff af0e59ff Thu May 07 08:53:15 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Disable "check unicode returns" under Python 3

Disabled the "unicode returns" check that runs on dialect startup when
running under Python 3, which for many years has occurred in order to test
the current DBAPI's behavior for whether or not it returns Python Unicode
or Py2K strings for the VARCHAR and NVARCHAR datatypes. The check still
occurs by default under Python 2, however the mechanism to test the
behavior will be removed in SQLAlchemy 2.0 when Python 2 support is also
removed.

This logic was very effective when it was needed, however now that Python 3
is standard, all DBAPIs are expected to return Python 3 strings for
character datatypes. In the unlikely case that a third party DBAPI does
not support this, the conversion logic within :class:`.String` is still
available and the third party dialect may specify this in its upfront
dialect flags by setting the dialect level flag ``returns_unicode_strings``
to one of :attr:`.String.RETURNS_CONDITIONAL` or
:attr:`.String.RETURNS_BYTES`, both of which will enable Unicode conversion
even under Python 3.

As part of this change, disabling testing of the doctest tutorials under
Python 2.

Fixes: #5315
Change-Id: I1260e894611409d3b7fe1a92bd90c52043bbcf19
H A Dtest_compiler.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 707e5d70 Tue Jul 06 09:26:53 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> labeling refactor

To service #6718 and #6710, the system by which columns are
given labels in a SELECT statement as well as the system that
gives them keys in a .c or .selected_columns collection have
been refactored to provide a single source of truth for
both, in constrast to the previous approach that included
similar logic repeated in slightly different ways.

Main ideas:

1. ColumnElement attributes ._label, ._anon_label, ._key_label
are renamed to include the letters "tq", meaning
"table-qualified" - these labels are only used when rendering
a SELECT that has LABEL_STYLE_TABLENAME_PLUS_COL for its
label style; as this label style is primarily legacy, the
"tq" names should be isolated so that in a 2.0 style application
these aren't being used at all

2. The means by which the "labels" and "proxy keys" for the elements
of a SELECT has been centralized to a single source of truth;
previously, the three of _generate_columns_plus_names,
_generate_fromclause_column_proxies, and _column_naming_convention
all had duplicated rules between them, as well as that there
were a little bit of labeling rules in compiler._label_select_column
as well; by this we mean that the various "anon_label" "anon_key"
methods on ColumnElement were called by all four of these methods,
where there were many cases where it was necessary that one method
comes up with the same answer as another of the methods. This
has all been centralized into _generate_columns_plus_names
for all the names except the "proxy key", which is generated
by _column_naming_convention.

3. compiler._label_select_column has been rewritten to both not make
any naming decisions nor any "proxy key" decisions, only whether
to label or not to label; the _generate_columns_plus_names method
gives it the information, where the proxy keys come from
_column_naming_convention; previously, these proxy keys were matched
based on restatement of similar (but not really the same) logic in
two places. The heuristics of "whether to label or not to label"
are also reorganized to be much easier to read and understand.

4. a new method compiler._label_returning_column is added for dialects
to use in their "generate returning columns" methods. A
github search reveals a small number of third party dialects also
doing this using the prior _label_select_column method so we
try to make sure _label_select_column continues to work the
exact same way for that specific use case; for the "SELECT" use
case it now needs

5. After some attempts to do it different ways, for the case where
_proxy_key is giving us some kind of anon label, we are hard
changing it to "_no_label" right now, as there's not currently
a way to fully match anonymized labels from stmt.c or
stmt.selected_columns to what will be in the result map. The
idea of "_no_label" is to encourage the user to use label('name')
for columns they want to be able to target by string name that
don't have a natural name.

Change-Id: I7a92a66f3a7e459ccf32587ac0a3c306650daf11
diff 3dcdc304 Wed Apr 28 10:14:33 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> accommodate for mutiple copies of bind in ckbm

Fixed critical regression where bound parameter tracking as used in the SQL
caching system could fail to track all parameters for the case where the
same SQL expression containing a parameter were used in an ORM-related
query using a feature such as class inheritance, which was then embedded in
an enclosing expression which would make use of that same expression
multiple times, such as a UNION. The ORM would individually copy the
individual SELECT statements as part of compilation with class inheritance,
which then embedded in the enclosing statement would fail to accommodate
for all parameters. The logic that tracks this condition has been adjusted
to work for multiple copies of a parameter.

Fixes: #6391
Change-Id: I6db5dee0d361a3bb58d753a2d27ef2eee2b369c5
diff f1e96cb0 Sun Jan 10 11:44:14 MST 2021 Mike Bayer <mike_mp@zzzcomputing.com> reinvent xdist hooks in terms of pytest fixtures

To allow the "connection" pytest fixture and others work
correctly in conjunction with setup/teardown that expects
to be external to the transaction, remove and prevent any usage
of "xdist" style names that are hardcoded by pytest to run
inside of fixtures, even function level ones. Instead use
pytest autouse fixtures to implement our own
r"setup|teardown_test(?:_class)?" methods so that we can ensure
function-scoped fixtures are run within them. A new more
explicit flow is set up within plugin_base and pytestplugin
such that the order of setup/teardown steps, which there are now
many, is fully documented and controllable. New granularity
has been added to the test teardown phase to distinguish
between "end of the test" when lock-holding structures on
connections should be released to allow for table drops,
vs. "end of the test plus its teardown steps" when we can
perform final cleanup on connections and run assertions
that everything is closed out.

From there we can remove most of the defensive "tear down everything"
logic inside of engines which for many years would frequently dispose
of pools over and over again, creating for a broken and expensive
connection flow. A quick test shows that running test/sql/ against
a single Postgresql engine with the new approach uses 75% fewer new
connections, creating 42 new connections total, vs. 164 new
connections total with the previous system.

As part of this, the new fixtures metadata/connection/future_connection
have been integrated such that they can be combined together
effectively. The fixture_session(), provide_metadata() fixtures
have been improved, including that fixture_session() now strongly
references sessions which are explicitly torn down before
table drops occur afer a test.

Major changes have been made to the
ConnectionKiller such that it now features different "scopes" for
testing engines and will limit its cleanup to those testing
engines corresponding to end of test, end of test class, or
end of test session. The system by which it tracks DBAPI
connections has been reworked, is ultimately somewhat similar to
how it worked before but is organized more clearly along
with the proxy-tracking logic. A "testing_engine" fixture
is also added that works as a pytest fixture rather than a
standalone function. The connection cleanup logic should
now be very robust, as we now can use the same global
connection pools for the whole suite without ever disposing
them, while also running a query for PostgreSQL
locks remaining after every test and assert there are no open
transactions leaking between tests at all. Additional steps
are added that also accommodate for asyncio connections not
explicitly closed, as is the case for legacy sync-style
tests as well as the async tests themselves.

As always, hundreds of tests are further refined to use the
new fixtures where problems with loose connections were identified,
largely as a result of the new PostgreSQL assertions,
many more tests have moved from legacy patterns into the newest.

An unfortunate discovery during the creation of this system is that
autouse fixtures (as well as if they are set up by
@pytest.mark.usefixtures) are not usable at our current scale with pytest
4.6.11 running under Python 2. It's unclear if this is due
to the older version of pytest or how it implements itself for
Python 2, as well as if the issue is CPU slowness or just large
memory use, but collecting the full span of tests takes over
a minute for a single process when any autouse fixtures are in
place and on CI the jobs just time out after ten minutes.
So at the moment this patch also reinvents a small version of
"autouse" fixtures when py2k is running, which skips generating
the real fixture and instead uses two global pytest fixtures
(which don't seem to impact performance) to invoke the
"autouse" fixtures ourselves outside of pytest.
This will limit our ability to do more with fixtures
until we can remove py2k support.

py.test is still observed to be much slower in collection in the
4.6.11 version compared to modern 6.2 versions, so add support for new
TOX_POSTGRESQL_PY2K and TOX_MYSQL_PY2K environment variables that
will run the suite for fewer backends under Python 2. For Python 3
pin pytest to modern 6.2 versions where performance for collection
has been improved greatly.

Includes the following improvements:

Fixed bug in asyncio connection pool where ``asyncio.TimeoutError`` would
be raised rather than :class:`.exc.TimeoutError`. Also repaired the
:paramref:`_sa.create_engine.pool_timeout` parameter set to zero when using
the async engine, which previously would ignore the timeout and block
rather than timing out immediately as is the behavior with regular
:class:`.QueuePool`.

For asyncio the connection pool will now also not interact
at all with an asyncio connection whose ConnectionFairy is
being garbage collected; a warning that the connection was
not properly closed is emitted and the connection is discarded.
Within the test suite the ConnectionKiller is now maintaining
strong references to all DBAPI connections and ensuring they
are released when tests end, including those whose ConnectionFairy
proxies are GCed.

Identified cx_Oracle.stmtcachesize as a major factor in Oracle
test scalability issues, this can be reset on a per-test basis
rather than setting it to zero across the board. the addition
of this flag has resolved the long-standing oracle "two task"
error problem.

For SQL Server, changed the temp table style used by the
"suite" tests to be the double-pound-sign, i.e. global,
variety, which is much easier to test generically. There
are already reflection tests that are more finely tuned
to both styles of temp table within the mssql test
suite. Additionally, added an extra step to the
"dropfirst" mechanism for SQL Server that will remove
all foreign key constraints first as some issues were
observed when using this flag when multiple schemas
had not been torn down.

Identified and fixed two subtle failure modes in the
engine, when commit/rollback fails in a begin()
context manager, the connection is explicitly closed,
and when "initialize()" fails on the first new connection
of a dialect, the transactional state on that connection
is still rolled back.

Fixes: #5826
Fixes: #5827
Change-Id: Ib1d05cb8c7cf84f9a4bfd23df397dc23c9329bfe
diff 3e49b8d0 Mon Oct 19 08:19:29 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Ensure no compiler visit method tries to access .statement

Fixed structural compiler issue where some constructs such as MySQL /
PostgreSQL "on conflict / on duplicate key" would rely upon the state of
the :class:`_sql.Compiler` object being fixed against their statement as
the top level statement, which would fail in cases where those statements
are branched from a different context, such as a DDL construct linked to a
SQL statement.

Fixes: #5656
Change-Id: I568bf40adc7edcf72ea6c7fd6eb9d07790de189e
diff 3bb402ff Tue Aug 27 15:08:11 MDT 2019 Mike Bayer <mike_mp@zzzcomputing.com> Label simple column transformations as the column name

Additional logic has been added such that certain SQL expressions which
typically wrap a single database column will use the name of that column as
their "anonymous label" name within a SELECT statement, potentially making
key-based lookups in result tuples more intutive. The primary example of
this is that of a CAST expression, e.g. ``CAST(table.colname AS INTEGER)``,
which will export its default name as "colname", rather than the usual
"anon_1" label, that is, ``CAST(table.colname AS INTEGER) AS colname``.
If the inner expression doesn't have a name, then the previous "anonymous
label" logic is used. When using SELECT statements that make use of
:meth:`.Select.apply_labels`, such as those emitted by the ORM, the
labeling logic will produce ``<tablename>_<inner column name>`` in the same
was as if the column were named alone. The logic applies right now to the
:func:`.cast` and :func:`.type_coerce` constructs as well as some
single-element boolean expressions.

Fixes: #4449
Change-Id: Ie3b73470e3bea53f2386cd86514cdc556491564e
diff 4c2c2c40 Thu Dec 20 20:05:36 MST 2018 Mike Bayer <mike_mp@zzzcomputing.com> Add deprecation warnings to all deprecated APIs

A large change throughout the library has ensured that all objects, parameters,
and behaviors which have been noted as deprecated or legacy now emit
``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now
defaults to displaying deprecation warnings, as well as that modern test suites
based on tools like tox and pytest tend to display deprecation warnings,
this change should make it easier to note what API features are obsolete.

See the notes added to the changelog and migration notes for further
details.

Fixes: #4393
Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
diff 3e2f61c4 Sun Aug 19 20:19:59 MDT 2018 Mike Bayer <mike_mp@zzzcomputing.com> Add missing range_ / rows parameters to additional over() methods

Added missing window function parameters
:paramref:`.WithinGroup.over.range_` and :paramref:`.WithinGroup.over.rows`
parameters to the :meth:`.WithinGroup.over` and
:meth:`.FunctionFilter.over` methods, to correspond to the range/rows
feature added to the "over" method of SQL functions as part of
:ticket:`3049` in version 1.1.

Fixes: #4322
Change-Id: I77dcdac65c699a4b52a3fc3ee09a100ffb4fc20e
diff 3a5bbe4c Fri Aug 25 07:41:18 MDT 2017 Mike Bayer <mike_mp@zzzcomputing.com> Allow for multiple FOLLOWING/PRECEDING in a window range

Altered the range specification for window functions to allow
for two of the same PRECEDING or FOLLOWING keywords in a range
by allowing for the left side of the range to be positive
and for the right to be negative, e.g. (1, 3) is
"1 FOLLOWING AND 3 FOLLOWING".

Change-Id: I7d3a6c641151bb49219104968d18dac2266f3db8
Fixes: #4053
diff 3a5bbe4c Fri Aug 25 07:41:18 MDT 2017 Mike Bayer <mike_mp@zzzcomputing.com> Allow for multiple FOLLOWING/PRECEDING in a window range

Altered the range specification for window functions to allow
for two of the same PRECEDING or FOLLOWING keywords in a range
by allowing for the left side of the range to be positive
and for the right to be negative, e.g. (1, 3) is
"1 FOLLOWING AND 3 FOLLOWING".

Change-Id: I7d3a6c641151bb49219104968d18dac2266f3db8
Fixes: #4053
diff 3a5bbe4c Fri Aug 25 07:41:18 MDT 2017 Mike Bayer <mike_mp@zzzcomputing.com> Allow for multiple FOLLOWING/PRECEDING in a window range

Altered the range specification for window functions to allow
for two of the same PRECEDING or FOLLOWING keywords in a range
by allowing for the left side of the range to be positive
and for the right to be negative, e.g. (1, 3) is
"1 FOLLOWING AND 3 FOLLOWING".

Change-Id: I7d3a6c641151bb49219104968d18dac2266f3db8
Fixes: #4053
/sqlalchemy/test/dialect/mssql/
H A Dtest_types.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff a8b4ea88 Mon Feb 17 07:41:59 MST 2020 Mike Bayer <mike_mp@zzzcomputing.com> Further refine fractional seconds datetimeoffset fixture

in 55f6d61e85b7f16df0b77f1c55a4fb051cd696e3 we still
forgot to accommodate for Python 3 timezone constructor
rejecting fractional minutes so the test further needs lambdas
to prevent the constructor from invoking for Python versions
less than 3.7

Change-Id: I02a83888f5ffbbe53ed0ce16c46a70942add4a3c
diff f521577f Tue Nov 26 07:57:48 MST 2019 Gord Thompson <gord@gordthompson.com> Add DATETIMEOFFSET support for mssql+pyodbc

Repaired support for the :class:`.mssql.DATETIMEOFFSET` datatype on PyODBC,
by adding PyODBC-level result handlers as it does not include native
support for this datatype. This includes usage of the Python 3 "timezone"
tzinfo subclass in order to set up a timezone, which on Python 2 makes
use of a minimal backport of "timezone" in sqlalchemy.util.

Fixes: #4983
Closes: #4986
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4986
Pull-request-sha: e3a5a8dda01c7580dc93271092c4f9beaac4be11

Change-Id: Ia4c7c0d1971c6c0492515bd5fa0b1799f07fee1c
diff 3f55039e Mon Apr 11 13:29:03 MDT 2016 Sheila Allen <sallen@zeomega.com> Use new mxODBC 3.3.4 varbinary null symbol

Use new mxODBC 3.3.4 varbinary null symbol with VARBINARY
data types when value is None(based on similar change to pyodbc).
Fix a test to pass on mxODBC starting w 3.3.3 version

Change-Id: Id703ecb51ebc5db149c81fef124f673433606c7f
Pull-request: https://bitbucket.org/zzzeek/sqlalchemy/pull-requests/58
diff 3f55039e Mon Apr 11 13:29:03 MDT 2016 Sheila Allen <sallen@zeomega.com> Use new mxODBC 3.3.4 varbinary null symbol

Use new mxODBC 3.3.4 varbinary null symbol with VARBINARY
data types when value is None(based on similar change to pyodbc).
Fix a test to pass on mxODBC starting w 3.3.3 version

Change-Id: Id703ecb51ebc5db149c81fef124f673433606c7f
Pull-request: https://bitbucket.org/zzzeek/sqlalchemy/pull-requests/58
/sqlalchemy/test/dialect/postgresql/
H A Dtest_reflection.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3fe8d618 Thu Mar 25 14:08:21 MDT 2021 Federico Caselli <cfederico87@gmail.com> Fixed reflection of identity columns in tables
with mixed case names in PostgreSQL.

Fixes: #6129
Change-Id: I50480cec03fcb44a668c9b0f9c72950803b771d9
diff f1e96cb0 Sun Jan 10 11:44:14 MST 2021 Mike Bayer <mike_mp@zzzcomputing.com> reinvent xdist hooks in terms of pytest fixtures

To allow the "connection" pytest fixture and others work
correctly in conjunction with setup/teardown that expects
to be external to the transaction, remove and prevent any usage
of "xdist" style names that are hardcoded by pytest to run
inside of fixtures, even function level ones. Instead use
pytest autouse fixtures to implement our own
r"setup|teardown_test(?:_class)?" methods so that we can ensure
function-scoped fixtures are run within them. A new more
explicit flow is set up within plugin_base and pytestplugin
such that the order of setup/teardown steps, which there are now
many, is fully documented and controllable. New granularity
has been added to the test teardown phase to distinguish
between "end of the test" when lock-holding structures on
connections should be released to allow for table drops,
vs. "end of the test plus its teardown steps" when we can
perform final cleanup on connections and run assertions
that everything is closed out.

From there we can remove most of the defensive "tear down everything"
logic inside of engines which for many years would frequently dispose
of pools over and over again, creating for a broken and expensive
connection flow. A quick test shows that running test/sql/ against
a single Postgresql engine with the new approach uses 75% fewer new
connections, creating 42 new connections total, vs. 164 new
connections total with the previous system.

As part of this, the new fixtures metadata/connection/future_connection
have been integrated such that they can be combined together
effectively. The fixture_session(), provide_metadata() fixtures
have been improved, including that fixture_session() now strongly
references sessions which are explicitly torn down before
table drops occur afer a test.

Major changes have been made to the
ConnectionKiller such that it now features different "scopes" for
testing engines and will limit its cleanup to those testing
engines corresponding to end of test, end of test class, or
end of test session. The system by which it tracks DBAPI
connections has been reworked, is ultimately somewhat similar to
how it worked before but is organized more clearly along
with the proxy-tracking logic. A "testing_engine" fixture
is also added that works as a pytest fixture rather than a
standalone function. The connection cleanup logic should
now be very robust, as we now can use the same global
connection pools for the whole suite without ever disposing
them, while also running a query for PostgreSQL
locks remaining after every test and assert there are no open
transactions leaking between tests at all. Additional steps
are added that also accommodate for asyncio connections not
explicitly closed, as is the case for legacy sync-style
tests as well as the async tests themselves.

As always, hundreds of tests are further refined to use the
new fixtures where problems with loose connections were identified,
largely as a result of the new PostgreSQL assertions,
many more tests have moved from legacy patterns into the newest.

An unfortunate discovery during the creation of this system is that
autouse fixtures (as well as if they are set up by
@pytest.mark.usefixtures) are not usable at our current scale with pytest
4.6.11 running under Python 2. It's unclear if this is due
to the older version of pytest or how it implements itself for
Python 2, as well as if the issue is CPU slowness or just large
memory use, but collecting the full span of tests takes over
a minute for a single process when any autouse fixtures are in
place and on CI the jobs just time out after ten minutes.
So at the moment this patch also reinvents a small version of
"autouse" fixtures when py2k is running, which skips generating
the real fixture and instead uses two global pytest fixtures
(which don't seem to impact performance) to invoke the
"autouse" fixtures ourselves outside of pytest.
This will limit our ability to do more with fixtures
until we can remove py2k support.

py.test is still observed to be much slower in collection in the
4.6.11 version compared to modern 6.2 versions, so add support for new
TOX_POSTGRESQL_PY2K and TOX_MYSQL_PY2K environment variables that
will run the suite for fewer backends under Python 2. For Python 3
pin pytest to modern 6.2 versions where performance for collection
has been improved greatly.

Includes the following improvements:

Fixed bug in asyncio connection pool where ``asyncio.TimeoutError`` would
be raised rather than :class:`.exc.TimeoutError`. Also repaired the
:paramref:`_sa.create_engine.pool_timeout` parameter set to zero when using
the async engine, which previously would ignore the timeout and block
rather than timing out immediately as is the behavior with regular
:class:`.QueuePool`.

For asyncio the connection pool will now also not interact
at all with an asyncio connection whose ConnectionFairy is
being garbage collected; a warning that the connection was
not properly closed is emitted and the connection is discarded.
Within the test suite the ConnectionKiller is now maintaining
strong references to all DBAPI connections and ensuring they
are released when tests end, including those whose ConnectionFairy
proxies are GCed.

Identified cx_Oracle.stmtcachesize as a major factor in Oracle
test scalability issues, this can be reset on a per-test basis
rather than setting it to zero across the board. the addition
of this flag has resolved the long-standing oracle "two task"
error problem.

For SQL Server, changed the temp table style used by the
"suite" tests to be the double-pound-sign, i.e. global,
variety, which is much easier to test generically. There
are already reflection tests that are more finely tuned
to both styles of temp table within the mssql test
suite. Additionally, added an extra step to the
"dropfirst" mechanism for SQL Server that will remove
all foreign key constraints first as some issues were
observed when using this flag when multiple schemas
had not been torn down.

Identified and fixed two subtle failure modes in the
engine, when commit/rollback fails in a begin()
context manager, the connection is explicitly closed,
and when "initialize()" fails on the first new connection
of a dialect, the transactional state on that connection
is still rolled back.

Fixes: #5826
Fixes: #5827
Change-Id: Ib1d05cb8c7cf84f9a4bfd23df397dc23c9329bfe
diff b4be7ceb Thu Jun 13 08:37:16 MDT 2019 Eli Collins <elic@assurancetechnologies.com> PostgreSQL now reflects per-column sort order on indexes.

Added support for column sorting flags when reflecting indexes for
PostgreSQL, including ASC, DESC, NULLSFIRST, NULLSLAST. Also adds this
facility to the reflection system in general which can be applied to other
dialects in future releases. Pull request courtesy Eli Collins.

Fixes: #4717
Closes: #4725
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4725
Pull-request-sha: 3cbb067bd46776fdb125553ba0ac192cb45d060c

Change-Id: I8b0617d68580cfe4ff79d758a077263f33e852c2
diff 1a35188b Tue Mar 25 10:26:42 MDT 2014 Mike Bayer <mike_mp@zzzcomputing.com> - Fixed regression caused by release 0.8.5 / 0.9.3's compatibility
enhancements where index reflection on Postgresql versions specific
to only the 8.1, 8.2 series again
broke, surrounding the ever problematic int2vector type. While
int2vector supports array operations as of 8.1, apparently it only
supports CAST to a varchar as of 8.3.
fix #3000
diff 0326f3cf Sun Feb 02 14:33:54 MST 2014 Mike Bayer <mike_mp@zzzcomputing.com> - Added :paramref:`.MetaData.reflect.**dialect_kwargs`
to support dialect-level reflection options for all :class:`.Table`
objects reflected.

- Added a new dialect-level argument ``postgresql_ignore_search_path``;
this argument is accepted by both the :class:`.Table` constructor
as well as by the :meth:`.MetaData.reflect` method. When in use
against Postgresql, a foreign-key referenced table which specifies
a remote schema name will retain that schema name even if the name
is present in the ``search_path``; the default behavior since 0.7.3
has been that schemas present in ``search_path`` would not be copied
to reflected :class:`.ForeignKey` objects. The documentation has been
updated to describe in detail the behavior of the ``pg_get_constraintdef()``
function and how the ``postgresql_ignore_search_path`` feature essentially
determines if we will honor the schema qualification reported by
this function or not. [ticket:2922]
/sqlalchemy/test/orm/
H A Dtest_update_delete.pydiff 0277ec7b Sun Sep 12 12:46:48 MDT 2021 Daniel Black <daniel@mariadb.org> test_update_delete - fix the fixed comment (#7020)

Spelling Criteria right.

Fixes 3ea321ce22438c5b7450e027d087437474eb4498

Also makes it PEP8 compatible again to be under 80
characters.
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 0fd256d9 Wed Jun 24 07:51:56 MDT 2020 ramonvg <ramonvillalongaa@gmail.com> Fixes: #4556 - Test that prevents passing a filter to Session.query when running an update

### Description
There was a bug in 1.3.8 that allowed passing a `BinaryExpression` to `Session().query`. I assume it was fixed in https://github.com/sqlalchemy/sqlalchemy/commit/3ab2364e78641c4f0e4b6456afc2cbed39b0d0e6 since now it raises:
```
sqlalchemy.exc.ArgumentError: subject table for an INSERT, UPDATE or DELETE expected, got <sqlalchemy.sql.elements.BinaryExpression object at 0x7f3d90426370>.
```

I'm just adding a test that ensures the behavior. Fixes: [4556](https://github.com/sqlalchemy/sqlalchemy/issues/4556)

### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)

-->

This pull request is:

- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.

**Have a nice day!**

Closes: #5419
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5419
Pull-request-sha: af7601dff1a0dce94802627a9fe2398be60d8186

Change-Id: Ic1a9075e7e8520510964fdd3d7ac14fba550110c
diff 3ab2364e Wed Jun 03 15:38:35 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Convert bulk update/delete to new execution model

This reorganizes the BulkUD model in sqlalchemy.orm.persistence
to be based on the CompileState concept and to allow plain
update() / delete() to be passed to session.execute() where
the ORM synchronize session logic will take place.
Also gets "synchronize_session='fetch'" working with horizontal
sharding.

Adding a few more result.scalar_one() types of methods
as scalar_one() seems like what is normally desired.

Fixes: #5160
Change-Id: I8001ebdad089da34119eb459709731ba6c0ba975
H A Dtest_attributes.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff f1e96cb0 Sun Jan 10 11:44:14 MST 2021 Mike Bayer <mike_mp@zzzcomputing.com> reinvent xdist hooks in terms of pytest fixtures

To allow the "connection" pytest fixture and others work
correctly in conjunction with setup/teardown that expects
to be external to the transaction, remove and prevent any usage
of "xdist" style names that are hardcoded by pytest to run
inside of fixtures, even function level ones. Instead use
pytest autouse fixtures to implement our own
r"setup|teardown_test(?:_class)?" methods so that we can ensure
function-scoped fixtures are run within them. A new more
explicit flow is set up within plugin_base and pytestplugin
such that the order of setup/teardown steps, which there are now
many, is fully documented and controllable. New granularity
has been added to the test teardown phase to distinguish
between "end of the test" when lock-holding structures on
connections should be released to allow for table drops,
vs. "end of the test plus its teardown steps" when we can
perform final cleanup on connections and run assertions
that everything is closed out.

From there we can remove most of the defensive "tear down everything"
logic inside of engines which for many years would frequently dispose
of pools over and over again, creating for a broken and expensive
connection flow. A quick test shows that running test/sql/ against
a single Postgresql engine with the new approach uses 75% fewer new
connections, creating 42 new connections total, vs. 164 new
connections total with the previous system.

As part of this, the new fixtures metadata/connection/future_connection
have been integrated such that they can be combined together
effectively. The fixture_session(), provide_metadata() fixtures
have been improved, including that fixture_session() now strongly
references sessions which are explicitly torn down before
table drops occur afer a test.

Major changes have been made to the
ConnectionKiller such that it now features different "scopes" for
testing engines and will limit its cleanup to those testing
engines corresponding to end of test, end of test class, or
end of test session. The system by which it tracks DBAPI
connections has been reworked, is ultimately somewhat similar to
how it worked before but is organized more clearly along
with the proxy-tracking logic. A "testing_engine" fixture
is also added that works as a pytest fixture rather than a
standalone function. The connection cleanup logic should
now be very robust, as we now can use the same global
connection pools for the whole suite without ever disposing
them, while also running a query for PostgreSQL
locks remaining after every test and assert there are no open
transactions leaking between tests at all. Additional steps
are added that also accommodate for asyncio connections not
explicitly closed, as is the case for legacy sync-style
tests as well as the async tests themselves.

As always, hundreds of tests are further refined to use the
new fixtures where problems with loose connections were identified,
largely as a result of the new PostgreSQL assertions,
many more tests have moved from legacy patterns into the newest.

An unfortunate discovery during the creation of this system is that
autouse fixtures (as well as if they are set up by
@pytest.mark.usefixtures) are not usable at our current scale with pytest
4.6.11 running under Python 2. It's unclear if this is due
to the older version of pytest or how it implements itself for
Python 2, as well as if the issue is CPU slowness or just large
memory use, but collecting the full span of tests takes over
a minute for a single process when any autouse fixtures are in
place and on CI the jobs just time out after ten minutes.
So at the moment this patch also reinvents a small version of
"autouse" fixtures when py2k is running, which skips generating
the real fixture and instead uses two global pytest fixtures
(which don't seem to impact performance) to invoke the
"autouse" fixtures ourselves outside of pytest.
This will limit our ability to do more with fixtures
until we can remove py2k support.

py.test is still observed to be much slower in collection in the
4.6.11 version compared to modern 6.2 versions, so add support for new
TOX_POSTGRESQL_PY2K and TOX_MYSQL_PY2K environment variables that
will run the suite for fewer backends under Python 2. For Python 3
pin pytest to modern 6.2 versions where performance for collection
has been improved greatly.

Includes the following improvements:

Fixed bug in asyncio connection pool where ``asyncio.TimeoutError`` would
be raised rather than :class:`.exc.TimeoutError`. Also repaired the
:paramref:`_sa.create_engine.pool_timeout` parameter set to zero when using
the async engine, which previously would ignore the timeout and block
rather than timing out immediately as is the behavior with regular
:class:`.QueuePool`.

For asyncio the connection pool will now also not interact
at all with an asyncio connection whose ConnectionFairy is
being garbage collected; a warning that the connection was
not properly closed is emitted and the connection is discarded.
Within the test suite the ConnectionKiller is now maintaining
strong references to all DBAPI connections and ensuring they
are released when tests end, including those whose ConnectionFairy
proxies are GCed.

Identified cx_Oracle.stmtcachesize as a major factor in Oracle
test scalability issues, this can be reset on a per-test basis
rather than setting it to zero across the board. the addition
of this flag has resolved the long-standing oracle "two task"
error problem.

For SQL Server, changed the temp table style used by the
"suite" tests to be the double-pound-sign, i.e. global,
variety, which is much easier to test generically. There
are already reflection tests that are more finely tuned
to both styles of temp table within the mssql test
suite. Additionally, added an extra step to the
"dropfirst" mechanism for SQL Server that will remove
all foreign key constraints first as some issues were
observed when using this flag when multiple schemas
had not been torn down.

Identified and fixed two subtle failure modes in the
engine, when commit/rollback fails in a begin()
context manager, the connection is explicitly closed,
and when "initialize()" fails on the first new connection
of a dialect, the transactional state on that connection
is still rolled back.

Fixes: #5826
Fixes: #5827
Change-Id: Ib1d05cb8c7cf84f9a4bfd23df397dc23c9329bfe
diff 4c2c2c40 Thu Dec 20 20:05:36 MST 2018 Mike Bayer <mike_mp@zzzcomputing.com> Add deprecation warnings to all deprecated APIs

A large change throughout the library has ensured that all objects, parameters,
and behaviors which have been noted as deprecated or legacy now emit
``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now
defaults to displaying deprecation warnings, as well as that modern test suites
based on tools like tox and pytest tend to display deprecation warnings,
this change should make it easier to note what API features are obsolete.

See the notes added to the changelog and migration notes for further
details.

Fixes: #4393
Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
diff 3c9d2d7b Tue Nov 30 18:25:22 MST 2010 Mike Bayer <mike_mp@zzzcomputing.com> - replace GenericBackrefExtension with straight events
- add "backref" argument to register_attribute_impl
diff 3d389b19 Tue Sep 14 18:43:48 MDT 2010 Mike Bayer <mike_mp@zzzcomputing.com> - reorganization
- attrbutes.py splits into attribtes.py and instrumentation.py
- all the various Event subclasses go into events.py modules
- some ideas for orm events
- move *Extension out to deprecated_interfaces
diff 3e25e6e6 Tue Sep 02 11:57:35 MDT 2008 Mike Bayer <mike_mp@zzzcomputing.com> - AttributeListener has been refined such that the event
is fired before the mutation actually occurs. Addtionally,
the append() and set() methods must now return the given value,
which is used as the value to be used in the mutation operation.
This allows creation of validating AttributeListeners which
raise before the action actually occurs, and which can change
the given value into something else before its used.
A new example "validate_attributes.py" shows one such recipe
for doing this. AttributeListener helper functions are
also on the way.
diff 3cd10102 Sun Jun 08 19:24:08 MDT 2008 Mike Bayer <mike_mp@zzzcomputing.com> - Query.UpdateDeleteTest.test_delete_fallback fails on mysql due to subquery in DELETE; not sure how to do this exact operation in MySQL
- added query_cls keyword argument to sessionmaker(); allows user-defined Query subclasses to be generated by query().
- added @attributes.on_reconstitute decorator, MapperExtension.on_reconstitute, both receieve 'on_load' attribute event allowing
non-__init__ dependent instance initialization routines.
- push memusage to the top to avoid pointless heisenbugs
- renamed '_foostate'/'_fooclass_manager' to '_sa_instance_state'/'_sa_class_manager'
- removed legacy instance ORM state accessors
- query._get() will use _remove_newly_deleted instead of expunge() on ObjectDeleted, so that transaction rollback
restores the previous state
- removed MapperExtension.get(); replaced by a user-defined Query subclass
- removed needless **kwargs from query.get()
- removed Session.get(cls, id); this is redundant against Session.query(cls).get(id)
- removed Query.load() and Session.load(); the use case for this method has never been clear, and the same functionality is available in more explicit ways
diff 8693d4b2 Sat Dec 08 11:58:03 MST 2007 Mike Bayer <mike_mp@zzzcomputing.com> - flush() refactor merged from uow_nontree branch r3871-r3885
- topological.py cleaned up, presents three public facing functions which
return list/tuple based structures, without exposing any internals. only
the third function returns the "hierarchical" structure. when results
include "cycles" or "child" items, 2- or 3- tuples are used to represent
results.
- unitofwork uses InstanceState almost exclusively now. new and deleted lists
are now dicts which ref the actual object to provide a strong ref for the
duration that they're in those lists. IdentitySet is only used for the public
facing versions of "new" and "deleted".
- unitofwork topological sort no longer uses the "hierarchical" version of the sort
for the base sort, only for the "per-object" secondary sort where it still
helps to group non-dependent operations together and provides expected insert
order. the default sort deals with UOWTasks in a straight list and is greatly
simplified. Tests all pass but need to see if svilen's stuff still works,
one block of code in _sort_cyclical_dependencies() seems to not be needed anywhere
but i definitely put it there for a reason at some point; if not hopefully we
can derive more test coverage from that.
- the UOWEventHandler is only applied to object-storing attributes, not
scalar (i.e. column-based) ones. cuts out a ton of overhead when setting
non-object based attributes.
- InstanceState also used throughout the flush process, i.e. dependency.py,
mapper.save_obj()/delete_obj(), sync.execute() all expect InstanceState objects
in most cases now.
- mapper/property cascade_iterator() takes InstanceState as its argument,
but still returns lists of object instances so that they are not dereferenced.
- a few tricks needed when dealing with InstanceState, i.e. when loading a list
of items that are possibly fresh from the DB, you *have* to get the actual objects
into a strong-referencing datastructure else they fall out of scope immediately.
dependency.py caches lists of dependent objects which it loads now (i.e. history
collections).
- AttributeHistory is gone, replaced by a function that returns a 3-tuple of
added, unchanged, deleted. these collections still reference the object
instances directly for the strong-referencing reasons mentiontioned, but
it uses less IdentitySet logic to generate.
diff 8693d4b2 Sat Dec 08 11:58:03 MST 2007 Mike Bayer <mike_mp@zzzcomputing.com> - flush() refactor merged from uow_nontree branch r3871-r3885
- topological.py cleaned up, presents three public facing functions which
return list/tuple based structures, without exposing any internals. only
the third function returns the "hierarchical" structure. when results
include "cycles" or "child" items, 2- or 3- tuples are used to represent
results.
- unitofwork uses InstanceState almost exclusively now. new and deleted lists
are now dicts which ref the actual object to provide a strong ref for the
duration that they're in those lists. IdentitySet is only used for the public
facing versions of "new" and "deleted".
- unitofwork topological sort no longer uses the "hierarchical" version of the sort
for the base sort, only for the "per-object" secondary sort where it still
helps to group non-dependent operations together and provides expected insert
order. the default sort deals with UOWTasks in a straight list and is greatly
simplified. Tests all pass but need to see if svilen's stuff still works,
one block of code in _sort_cyclical_dependencies() seems to not be needed anywhere
but i definitely put it there for a reason at some point; if not hopefully we
can derive more test coverage from that.
- the UOWEventHandler is only applied to object-storing attributes, not
scalar (i.e. column-based) ones. cuts out a ton of overhead when setting
non-object based attributes.
- InstanceState also used throughout the flush process, i.e. dependency.py,
mapper.save_obj()/delete_obj(), sync.execute() all expect InstanceState objects
in most cases now.
- mapper/property cascade_iterator() takes InstanceState as its argument,
but still returns lists of object instances so that they are not dereferenced.
- a few tricks needed when dealing with InstanceState, i.e. when loading a list
of items that are possibly fresh from the DB, you *have* to get the actual objects
into a strong-referencing datastructure else they fall out of scope immediately.
dependency.py caches lists of dependent objects which it loads now (i.e. history
collections).
- AttributeHistory is gone, replaced by a function that returns a 3-tuple of
added, unchanged, deleted. these collections still reference the object
instances directly for the strong-referencing reasons mentiontioned, but
it uses less IdentitySet logic to generate.
H A Dtest_query.pydiff 3f220977 Fri Oct 01 14:16:14 MDT 2021 Federico Caselli <cfederico87@gmail.com> Warn when trying to execute a query object with a session.

Passing a :class:`.Query` object to :meth:`_orm.Session.execute` is not
the intended use of this object, and will now raise a deprecation warning.

Fixes: #6284
Change-Id: I30a406d5a72335f9405f10f0a2030a32ccda41b9
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 707e5d70 Tue Jul 06 09:26:53 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> labeling refactor

To service #6718 and #6710, the system by which columns are
given labels in a SELECT statement as well as the system that
gives them keys in a .c or .selected_columns collection have
been refactored to provide a single source of truth for
both, in constrast to the previous approach that included
similar logic repeated in slightly different ways.

Main ideas:

1. ColumnElement attributes ._label, ._anon_label, ._key_label
are renamed to include the letters "tq", meaning
"table-qualified" - these labels are only used when rendering
a SELECT that has LABEL_STYLE_TABLENAME_PLUS_COL for its
label style; as this label style is primarily legacy, the
"tq" names should be isolated so that in a 2.0 style application
these aren't being used at all

2. The means by which the "labels" and "proxy keys" for the elements
of a SELECT has been centralized to a single source of truth;
previously, the three of _generate_columns_plus_names,
_generate_fromclause_column_proxies, and _column_naming_convention
all had duplicated rules between them, as well as that there
were a little bit of labeling rules in compiler._label_select_column
as well; by this we mean that the various "anon_label" "anon_key"
methods on ColumnElement were called by all four of these methods,
where there were many cases where it was necessary that one method
comes up with the same answer as another of the methods. This
has all been centralized into _generate_columns_plus_names
for all the names except the "proxy key", which is generated
by _column_naming_convention.

3. compiler._label_select_column has been rewritten to both not make
any naming decisions nor any "proxy key" decisions, only whether
to label or not to label; the _generate_columns_plus_names method
gives it the information, where the proxy keys come from
_column_naming_convention; previously, these proxy keys were matched
based on restatement of similar (but not really the same) logic in
two places. The heuristics of "whether to label or not to label"
are also reorganized to be much easier to read and understand.

4. a new method compiler._label_returning_column is added for dialects
to use in their "generate returning columns" methods. A
github search reveals a small number of third party dialects also
doing this using the prior _label_select_column method so we
try to make sure _label_select_column continues to work the
exact same way for that specific use case; for the "SELECT" use
case it now needs

5. After some attempts to do it different ways, for the case where
_proxy_key is giving us some kind of anon label, we are hard
changing it to "_no_label" right now, as there's not currently
a way to fully match anonymized labels from stmt.c or
stmt.selected_columns to what will be in the result map. The
idea of "_no_label" is to encourage the user to use label('name')
for columns they want to be able to target by string name that
don't have a natural name.

Change-Id: I7a92a66f3a7e459ccf32587ac0a3c306650daf11
diff f1e96cb0 Sun Jan 10 11:44:14 MST 2021 Mike Bayer <mike_mp@zzzcomputing.com> reinvent xdist hooks in terms of pytest fixtures

To allow the "connection" pytest fixture and others work
correctly in conjunction with setup/teardown that expects
to be external to the transaction, remove and prevent any usage
of "xdist" style names that are hardcoded by pytest to run
inside of fixtures, even function level ones. Instead use
pytest autouse fixtures to implement our own
r"setup|teardown_test(?:_class)?" methods so that we can ensure
function-scoped fixtures are run within them. A new more
explicit flow is set up within plugin_base and pytestplugin
such that the order of setup/teardown steps, which there are now
many, is fully documented and controllable. New granularity
has been added to the test teardown phase to distinguish
between "end of the test" when lock-holding structures on
connections should be released to allow for table drops,
vs. "end of the test plus its teardown steps" when we can
perform final cleanup on connections and run assertions
that everything is closed out.

From there we can remove most of the defensive "tear down everything"
logic inside of engines which for many years would frequently dispose
of pools over and over again, creating for a broken and expensive
connection flow. A quick test shows that running test/sql/ against
a single Postgresql engine with the new approach uses 75% fewer new
connections, creating 42 new connections total, vs. 164 new
connections total with the previous system.

As part of this, the new fixtures metadata/connection/future_connection
have been integrated such that they can be combined together
effectively. The fixture_session(), provide_metadata() fixtures
have been improved, including that fixture_session() now strongly
references sessions which are explicitly torn down before
table drops occur afer a test.

Major changes have been made to the
ConnectionKiller such that it now features different "scopes" for
testing engines and will limit its cleanup to those testing
engines corresponding to end of test, end of test class, or
end of test session. The system by which it tracks DBAPI
connections has been reworked, is ultimately somewhat similar to
how it worked before but is organized more clearly along
with the proxy-tracking logic. A "testing_engine" fixture
is also added that works as a pytest fixture rather than a
standalone function. The connection cleanup logic should
now be very robust, as we now can use the same global
connection pools for the whole suite without ever disposing
them, while also running a query for PostgreSQL
locks remaining after every test and assert there are no open
transactions leaking between tests at all. Additional steps
are added that also accommodate for asyncio connections not
explicitly closed, as is the case for legacy sync-style
tests as well as the async tests themselves.

As always, hundreds of tests are further refined to use the
new fixtures where problems with loose connections were identified,
largely as a result of the new PostgreSQL assertions,
many more tests have moved from legacy patterns into the newest.

An unfortunate discovery during the creation of this system is that
autouse fixtures (as well as if they are set up by
@pytest.mark.usefixtures) are not usable at our current scale with pytest
4.6.11 running under Python 2. It's unclear if this is due
to the older version of pytest or how it implements itself for
Python 2, as well as if the issue is CPU slowness or just large
memory use, but collecting the full span of tests takes over
a minute for a single process when any autouse fixtures are in
place and on CI the jobs just time out after ten minutes.
So at the moment this patch also reinvents a small version of
"autouse" fixtures when py2k is running, which skips generating
the real fixture and instead uses two global pytest fixtures
(which don't seem to impact performance) to invoke the
"autouse" fixtures ourselves outside of pytest.
This will limit our ability to do more with fixtures
until we can remove py2k support.

py.test is still observed to be much slower in collection in the
4.6.11 version compared to modern 6.2 versions, so add support for new
TOX_POSTGRESQL_PY2K and TOX_MYSQL_PY2K environment variables that
will run the suite for fewer backends under Python 2. For Python 3
pin pytest to modern 6.2 versions where performance for collection
has been improved greatly.

Includes the following improvements:

Fixed bug in asyncio connection pool where ``asyncio.TimeoutError`` would
be raised rather than :class:`.exc.TimeoutError`. Also repaired the
:paramref:`_sa.create_engine.pool_timeout` parameter set to zero when using
the async engine, which previously would ignore the timeout and block
rather than timing out immediately as is the behavior with regular
:class:`.QueuePool`.

For asyncio the connection pool will now also not interact
at all with an asyncio connection whose ConnectionFairy is
being garbage collected; a warning that the connection was
not properly closed is emitted and the connection is discarded.
Within the test suite the ConnectionKiller is now maintaining
strong references to all DBAPI connections and ensuring they
are released when tests end, including those whose ConnectionFairy
proxies are GCed.

Identified cx_Oracle.stmtcachesize as a major factor in Oracle
test scalability issues, this can be reset on a per-test basis
rather than setting it to zero across the board. the addition
of this flag has resolved the long-standing oracle "two task"
error problem.

For SQL Server, changed the temp table style used by the
"suite" tests to be the double-pound-sign, i.e. global,
variety, which is much easier to test generically. There
are already reflection tests that are more finely tuned
to both styles of temp table within the mssql test
suite. Additionally, added an extra step to the
"dropfirst" mechanism for SQL Server that will remove
all foreign key constraints first as some issues were
observed when using this flag when multiple schemas
had not been torn down.

Identified and fixed two subtle failure modes in the
engine, when commit/rollback fails in a begin()
context manager, the connection is explicitly closed,
and when "initialize()" fails on the first new connection
of a dialect, the transactional state on that connection
is still rolled back.

Fixes: #5826
Fixes: #5827
Change-Id: Ib1d05cb8c7cf84f9a4bfd23df397dc23c9329bfe
diff fc97854f Thu Aug 13 22:58:56 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Bump minimum MySQL version to 5.0.2; use all-numeric server version

MySQL dialect's server_version_info tuple is now all numeric. String
tokens like "MariaDB" are no longer present so that numeric comparison
works in all cases. The .is_mariadb flag on the dialect should be
consulted for whether or not mariadb was detected. Additionally removed
structures meant to support extremely old MySQL versions 3.x and 4.x;
the minimum MySQL version supported is now version 5.0.2.

In addition, as the "MariaDB" name goes away from server version,
expand upon the change in I330815ebe572b6a9818377da56621397335fa702
to support the name "mariadb" throughout the dialect and test suite
when mariadb-only mode is used. This changes the "name" field
on the MariaDB dialect to "mariadb", which then implies a change
throughout the testing requirements system as well as all the
dialect-specific DDL argument names such as "mysql_engine" is
now specified as "mariadb_engine", etc. Make use of the
recent additions to test suite URL provisioning so that we can
force MariaDB databases to have a "mariadb-only" dialect which
allows us to test this name change fully.

Update documentation to refer to MySQL / MariaDB explicitly
as well as indicating the "mariadb_" prefix used for options.

It seems likely that MySQL and MariaDB version numbers are going to
start colliding at some point so having the "mariadb" name
be available as a totally separate dialect name should give us
some options in this regard.

Currently also includes a date related fix to a test for
the postgresql dialect that was implicitly assuming a
non-UTC timezone

Fixes: #4189
Change-Id: I00e76d00f62971e1f067bd61915fa6cc1cf64e5e
diff f9f9f0fe Sun Jul 12 08:48:03 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Correct mock call from 5de0f1cf50

mock in python 2.7 / 3.8 seems to have an "args" accessor
but in prior python 3 versions this is not the case,
so this is likely a recent addition to "mock" but wasn't
present in the python3 vendored versions.

Change-Id: Iff8afe75e891c9658c7c7b577831fefd5986940b
diff 4c2c2c40 Thu Dec 20 20:05:36 MST 2018 Mike Bayer <mike_mp@zzzcomputing.com> Add deprecation warnings to all deprecated APIs

A large change throughout the library has ensured that all objects, parameters,
and behaviors which have been noted as deprecated or legacy now emit
``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now
defaults to displaying deprecation warnings, as well as that modern test suites
based on tools like tox and pytest tend to display deprecation warnings,
this change should make it easier to note what API features are obsolete.

See the notes added to the changelog and migration notes for further
details.

Fixes: #4393
Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
diff 3fa38a1a Wed May 16 09:16:57 MDT 2018 Mike Bayer <mike_mp@zzzcomputing.com> Change query._identity_lookup into a normal instance method

Fixed regression in 1.2.7 caused by :ticket:`4228`, which itself was fixing
a 1.2-level regression, where the ``query_cls`` callable passed to a
:class:`.Session` was assumed to be a subclass of :class:`.Query` with
class method availability, as opposed to an arbitrary callable. In
particular, the dogpile caching example illustrates ``query_cls`` as a
function and not a :class:`.Query` subclass.

Change-Id: I3f86fcb12a6a9a89aa308b335e75c25969bcc30e
Fixes: #4256
diff 3c46eb17 Mon Feb 09 18:49:19 MST 2015 Mike Bayer <mike_mp@zzzcomputing.com> - additional mysql cast-related test fixes
/sqlalchemy/
H A Dtox.inidiff 23721113 Fri Oct 15 09:47:08 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> re-enable asnycmy >= 0.2.3; disable aiomysql totally

Change-Id: I94bc22db0462509003c8241d0ccef915fe6513c6
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 4476dca0 Wed Mar 24 09:33:04 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> Repair pysqlcipher and use sqlcipher3

The ``pysqlcipher`` dialect now imports the ``sqlcipher3`` module
for Python 3 by default. Regressions have been repaired such that
the connection routine was not working.

To better support the post-connection steps of the pysqlcipher
dialect, a new hook Dialect.on_connect_url() is added, which
supersedes Dialect.on_connect() and is passed the URL object.
The dialect now pulls the passphrase and other cipher args
from the URL directly without including them in the
"connect" args. This will allow any user-defined extensibility
to connecting to work as it would for other dialects.

The commit also builds upon the extended routines in
sqlite/provisioning.py to better support running tests against
multiple simultaneous SQLite database files. Additionally enables
backend for test_sqlite which was skipping everything
for aiosqlite too, fortunately everything there is passing.

Fixes: #5848
Change-Id: I43f53ebc62298a84a4abe149e1eb699a027b7915
diff f1e96cb0 Sun Jan 10 11:44:14 MST 2021 Mike Bayer <mike_mp@zzzcomputing.com> reinvent xdist hooks in terms of pytest fixtures

To allow the "connection" pytest fixture and others work
correctly in conjunction with setup/teardown that expects
to be external to the transaction, remove and prevent any usage
of "xdist" style names that are hardcoded by pytest to run
inside of fixtures, even function level ones. Instead use
pytest autouse fixtures to implement our own
r"setup|teardown_test(?:_class)?" methods so that we can ensure
function-scoped fixtures are run within them. A new more
explicit flow is set up within plugin_base and pytestplugin
such that the order of setup/teardown steps, which there are now
many, is fully documented and controllable. New granularity
has been added to the test teardown phase to distinguish
between "end of the test" when lock-holding structures on
connections should be released to allow for table drops,
vs. "end of the test plus its teardown steps" when we can
perform final cleanup on connections and run assertions
that everything is closed out.

From there we can remove most of the defensive "tear down everything"
logic inside of engines which for many years would frequently dispose
of pools over and over again, creating for a broken and expensive
connection flow. A quick test shows that running test/sql/ against
a single Postgresql engine with the new approach uses 75% fewer new
connections, creating 42 new connections total, vs. 164 new
connections total with the previous system.

As part of this, the new fixtures metadata/connection/future_connection
have been integrated such that they can be combined together
effectively. The fixture_session(), provide_metadata() fixtures
have been improved, including that fixture_session() now strongly
references sessions which are explicitly torn down before
table drops occur afer a test.

Major changes have been made to the
ConnectionKiller such that it now features different "scopes" for
testing engines and will limit its cleanup to those testing
engines corresponding to end of test, end of test class, or
end of test session. The system by which it tracks DBAPI
connections has been reworked, is ultimately somewhat similar to
how it worked before but is organized more clearly along
with the proxy-tracking logic. A "testing_engine" fixture
is also added that works as a pytest fixture rather than a
standalone function. The connection cleanup logic should
now be very robust, as we now can use the same global
connection pools for the whole suite without ever disposing
them, while also running a query for PostgreSQL
locks remaining after every test and assert there are no open
transactions leaking between tests at all. Additional steps
are added that also accommodate for asyncio connections not
explicitly closed, as is the case for legacy sync-style
tests as well as the async tests themselves.

As always, hundreds of tests are further refined to use the
new fixtures where problems with loose connections were identified,
largely as a result of the new PostgreSQL assertions,
many more tests have moved from legacy patterns into the newest.

An unfortunate discovery during the creation of this system is that
autouse fixtures (as well as if they are set up by
@pytest.mark.usefixtures) are not usable at our current scale with pytest
4.6.11 running under Python 2. It's unclear if this is due
to the older version of pytest or how it implements itself for
Python 2, as well as if the issue is CPU slowness or just large
memory use, but collecting the full span of tests takes over
a minute for a single process when any autouse fixtures are in
place and on CI the jobs just time out after ten minutes.
So at the moment this patch also reinvents a small version of
"autouse" fixtures when py2k is running, which skips generating
the real fixture and instead uses two global pytest fixtures
(which don't seem to impact performance) to invoke the
"autouse" fixtures ourselves outside of pytest.
This will limit our ability to do more with fixtures
until we can remove py2k support.

py.test is still observed to be much slower in collection in the
4.6.11 version compared to modern 6.2 versions, so add support for new
TOX_POSTGRESQL_PY2K and TOX_MYSQL_PY2K environment variables that
will run the suite for fewer backends under Python 2. For Python 3
pin pytest to modern 6.2 versions where performance for collection
has been improved greatly.

Includes the following improvements:

Fixed bug in asyncio connection pool where ``asyncio.TimeoutError`` would
be raised rather than :class:`.exc.TimeoutError`. Also repaired the
:paramref:`_sa.create_engine.pool_timeout` parameter set to zero when using
the async engine, which previously would ignore the timeout and block
rather than timing out immediately as is the behavior with regular
:class:`.QueuePool`.

For asyncio the connection pool will now also not interact
at all with an asyncio connection whose ConnectionFairy is
being garbage collected; a warning that the connection was
not properly closed is emitted and the connection is discarded.
Within the test suite the ConnectionKiller is now maintaining
strong references to all DBAPI connections and ensuring they
are released when tests end, including those whose ConnectionFairy
proxies are GCed.

Identified cx_Oracle.stmtcachesize as a major factor in Oracle
test scalability issues, this can be reset on a per-test basis
rather than setting it to zero across the board. the addition
of this flag has resolved the long-standing oracle "two task"
error problem.

For SQL Server, changed the temp table style used by the
"suite" tests to be the double-pound-sign, i.e. global,
variety, which is much easier to test generically. There
are already reflection tests that are more finely tuned
to both styles of temp table within the mssql test
suite. Additionally, added an extra step to the
"dropfirst" mechanism for SQL Server that will remove
all foreign key constraints first as some issues were
observed when using this flag when multiple schemas
had not been torn down.

Identified and fixed two subtle failure modes in the
engine, when commit/rollback fails in a begin()
context manager, the connection is explicitly closed,
and when "initialize()" fails on the first new connection
of a dialect, the transactional state on that connection
is still rolled back.

Fixes: #5826
Fixes: #5827
Change-Id: Ib1d05cb8c7cf84f9a4bfd23df397dc23c9329bfe
diff bcc17b1d Thu Oct 08 10:52:39 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> pin pytest lower bound at 4.6.11

we need 4.6 to support Python 2. however 4.6.0 is too low,
4.6.11 seems to work in both python 2 and 3, pin on that

Change-Id: I0a5995ca5e5bf08a06997130e4ffbaaceef8d942
diff 79a53645 Wed Feb 26 12:50:01 MST 2020 Gord Thompson <gord@gordthompson.com> Fix tests failing for SQLite file databases; repair provisioning

1. ensure provision.py loads dialect implementations when running
reap_dbs.py. Reapers haven't been working since
598f2f7e557073f29563d4d567f43931fc03013f .

2. add some exclusion rules to allow the sqlite_file target to work;
add to tox.

3. add reap dbs target for SQLite, repair SQLite drop_db routine
which also wasn't doing the right thing for memory databases
etc.

4. Fix logging in provision files, as the main provision logger
is the one that's enabled by reap_dbs and maybe others, have all
the provision files use the provision logger.

Fixes: #5180
Fixes: #5168

Change-Id: Ibc1b0106394d20f5bcf847f37b09d185f26ac9b5
diff 3ceb87da Tue Nov 19 10:38:58 MST 2019 Mike Bayer <mike_mp@zzzcomputing.com> Move max-worker-restart to pytest-xdist options in tox.ini

This way py.test can be run on the outside without a
command line error.

Change-Id: Ia3689806bffbabe176a827cca2f099e89a60ebf5
Closes: #4989
diff 664290ab Fri Nov 02 14:54:21 MDT 2018 Mike Bayer <mike_mp@zzzcomputing.com> - block py.test 3.9.1 and 3.9.2, 3.9.3 is OK

Change-Id: I35ecd78b92ea420ef7909de65f82e005b92bf986
/sqlalchemy/test/dialect/mysql/
H A Dtest_compiler.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff f1e96cb0 Sun Jan 10 11:44:14 MST 2021 Mike Bayer <mike_mp@zzzcomputing.com> reinvent xdist hooks in terms of pytest fixtures

To allow the "connection" pytest fixture and others work
correctly in conjunction with setup/teardown that expects
to be external to the transaction, remove and prevent any usage
of "xdist" style names that are hardcoded by pytest to run
inside of fixtures, even function level ones. Instead use
pytest autouse fixtures to implement our own
r"setup|teardown_test(?:_class)?" methods so that we can ensure
function-scoped fixtures are run within them. A new more
explicit flow is set up within plugin_base and pytestplugin
such that the order of setup/teardown steps, which there are now
many, is fully documented and controllable. New granularity
has been added to the test teardown phase to distinguish
between "end of the test" when lock-holding structures on
connections should be released to allow for table drops,
vs. "end of the test plus its teardown steps" when we can
perform final cleanup on connections and run assertions
that everything is closed out.

From there we can remove most of the defensive "tear down everything"
logic inside of engines which for many years would frequently dispose
of pools over and over again, creating for a broken and expensive
connection flow. A quick test shows that running test/sql/ against
a single Postgresql engine with the new approach uses 75% fewer new
connections, creating 42 new connections total, vs. 164 new
connections total with the previous system.

As part of this, the new fixtures metadata/connection/future_connection
have been integrated such that they can be combined together
effectively. The fixture_session(), provide_metadata() fixtures
have been improved, including that fixture_session() now strongly
references sessions which are explicitly torn down before
table drops occur afer a test.

Major changes have been made to the
ConnectionKiller such that it now features different "scopes" for
testing engines and will limit its cleanup to those testing
engines corresponding to end of test, end of test class, or
end of test session. The system by which it tracks DBAPI
connections has been reworked, is ultimately somewhat similar to
how it worked before but is organized more clearly along
with the proxy-tracking logic. A "testing_engine" fixture
is also added that works as a pytest fixture rather than a
standalone function. The connection cleanup logic should
now be very robust, as we now can use the same global
connection pools for the whole suite without ever disposing
them, while also running a query for PostgreSQL
locks remaining after every test and assert there are no open
transactions leaking between tests at all. Additional steps
are added that also accommodate for asyncio connections not
explicitly closed, as is the case for legacy sync-style
tests as well as the async tests themselves.

As always, hundreds of tests are further refined to use the
new fixtures where problems with loose connections were identified,
largely as a result of the new PostgreSQL assertions,
many more tests have moved from legacy patterns into the newest.

An unfortunate discovery during the creation of this system is that
autouse fixtures (as well as if they are set up by
@pytest.mark.usefixtures) are not usable at our current scale with pytest
4.6.11 running under Python 2. It's unclear if this is due
to the older version of pytest or how it implements itself for
Python 2, as well as if the issue is CPU slowness or just large
memory use, but collecting the full span of tests takes over
a minute for a single process when any autouse fixtures are in
place and on CI the jobs just time out after ten minutes.
So at the moment this patch also reinvents a small version of
"autouse" fixtures when py2k is running, which skips generating
the real fixture and instead uses two global pytest fixtures
(which don't seem to impact performance) to invoke the
"autouse" fixtures ourselves outside of pytest.
This will limit our ability to do more with fixtures
until we can remove py2k support.

py.test is still observed to be much slower in collection in the
4.6.11 version compared to modern 6.2 versions, so add support for new
TOX_POSTGRESQL_PY2K and TOX_MYSQL_PY2K environment variables that
will run the suite for fewer backends under Python 2. For Python 3
pin pytest to modern 6.2 versions where performance for collection
has been improved greatly.

Includes the following improvements:

Fixed bug in asyncio connection pool where ``asyncio.TimeoutError`` would
be raised rather than :class:`.exc.TimeoutError`. Also repaired the
:paramref:`_sa.create_engine.pool_timeout` parameter set to zero when using
the async engine, which previously would ignore the timeout and block
rather than timing out immediately as is the behavior with regular
:class:`.QueuePool`.

For asyncio the connection pool will now also not interact
at all with an asyncio connection whose ConnectionFairy is
being garbage collected; a warning that the connection was
not properly closed is emitted and the connection is discarded.
Within the test suite the ConnectionKiller is now maintaining
strong references to all DBAPI connections and ensuring they
are released when tests end, including those whose ConnectionFairy
proxies are GCed.

Identified cx_Oracle.stmtcachesize as a major factor in Oracle
test scalability issues, this can be reset on a per-test basis
rather than setting it to zero across the board. the addition
of this flag has resolved the long-standing oracle "two task"
error problem.

For SQL Server, changed the temp table style used by the
"suite" tests to be the double-pound-sign, i.e. global,
variety, which is much easier to test generically. There
are already reflection tests that are more finely tuned
to both styles of temp table within the mssql test
suite. Additionally, added an extra step to the
"dropfirst" mechanism for SQL Server that will remove
all foreign key constraints first as some issues were
observed when using this flag when multiple schemas
had not been torn down.

Identified and fixed two subtle failure modes in the
engine, when commit/rollback fails in a begin()
context manager, the connection is explicitly closed,
and when "initialize()" fails on the first new connection
of a dialect, the transactional state on that connection
is still rolled back.

Fixes: #5826
Fixes: #5827
Change-Id: Ib1d05cb8c7cf84f9a4bfd23df397dc23c9329bfe
diff fc97854f Thu Aug 13 22:58:56 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Bump minimum MySQL version to 5.0.2; use all-numeric server version

MySQL dialect's server_version_info tuple is now all numeric. String
tokens like "MariaDB" are no longer present so that numeric comparison
works in all cases. The .is_mariadb flag on the dialect should be
consulted for whether or not mariadb was detected. Additionally removed
structures meant to support extremely old MySQL versions 3.x and 4.x;
the minimum MySQL version supported is now version 5.0.2.

In addition, as the "MariaDB" name goes away from server version,
expand upon the change in I330815ebe572b6a9818377da56621397335fa702
to support the name "mariadb" throughout the dialect and test suite
when mariadb-only mode is used. This changes the "name" field
on the MariaDB dialect to "mariadb", which then implies a change
throughout the testing requirements system as well as all the
dialect-specific DDL argument names such as "mysql_engine" is
now specified as "mariadb_engine", etc. Make use of the
recent additions to test suite URL provisioning so that we can
force MariaDB databases to have a "mariadb-only" dialect which
allows us to test this name change fully.

Update documentation to refer to MySQL / MariaDB explicitly
as well as indicating the "mariadb_" prefix used for options.

It seems likely that MySQL and MariaDB version numbers are going to
start colliding at some point so having the "mariadb" name
be available as a totally separate dialect name should give us
some options in this regard.

Currently also includes a date related fix to a test for
the postgresql dialect that was implicitly assuming a
non-UTC timezone

Fixes: #4189
Change-Id: I00e76d00f62971e1f067bd61915fa6cc1cf64e5e
diff 3a0e0531 Mon Nov 04 15:11:21 MST 2019 CaselIT <cfederico87@gmail.com> Support for generated columns

Added DDL support for "computed columns"; these are DDL column
specifications for columns that have a server-computed value, either upon
SELECT (known as "virtual") or at the point of which they are INSERTed or
UPDATEd (known as "stored"). Support is established for Postgresql, MySQL,
Oracle SQL Server and Firebird. Thanks to Federico Caselli for lots of work
on this one.

ORM round trip tests included. The ORM makes use of existing
FetchedValue support and no additional ORM logic is present for
the basic feature.

It has been observed that Oracle RETURNING does not return the
new value of a computed column upon UPDATE; it returns the
prior value. As this is very dangerous, a warning is emitted
if a computed column is rendered into the RETURNING clause
of an UPDATE statement.

Fixes: #4894
Closes: #4928
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4928
Pull-request-sha: d39c521d5ac6ebfb4fb5b53846451de79752e64c

Change-Id: I2610b2999a5b1b127ed927dcdaeee98b769643ce
diff 1536bc46 Mon Jan 13 12:05:05 MST 2014 Mike Bayer <mike_mp@zzzcomputing.com> - The MySQL CAST compilation now takes into account aspects of a string
type such as "charset" and "collation". While MySQL wants all character-
based CAST calls to use the CHAR type, we now create a real CHAR
object at CAST time and copy over all the parameters it has, so that
an expression like ``cast(x, mysql.TEXT(charset='utf8'))`` will
render ``CAST(t.col AS CHAR CHARACTER SET utf8)``.

- Added new "unicode returns" detection to the MySQL dialect and
to the default dialect system overall, such that any dialect
can add extra "tests" to the on-first-connect "does this DBAPI
return unicode directly?" detection. In this case, we are
adding a check specifically against the "utf8" encoding with
an explicit "utf8_bin" collation type (after checking that
this collation is available) to test for some buggy unicode
behavior observed with MySQLdb version 1.2.3. While MySQLdb
has resolved this issue as of 1.2.4, the check here should
guard against regressions. The change also allows the "unicode"
checks to log in the engine logs, which was not previously
the case. [ticket:2906]
/sqlalchemy/lib/sqlalchemy/dialects/postgresql/
H A Dpsycopg2.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 20c0f774 Thu Apr 01 15:20:31 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> Default caching to opt-out for 3rd party dialects

Added a new flag to the :class:`_engine.Dialect` class called
:attr:`_engine.Dialect.supports_statement_cache`. This flag now needs to be present
directly on a dialect class in order for SQLAlchemy's
:ref:`query cache <sql_caching>` to take effect for that dialect. The
rationale is based on discovered issues such as :ticket:`6173` revealing
that dialects which hardcode literal values from the compiled statement,
often the numerical parameters used for LIMIT / OFFSET, will not be
compatible with caching until these dialects are revised to use the
parameters present in the statement only. For third party dialects where
this flag is not applied, the SQL logging will show the message "dialect
does not support caching", indicating the dialect should seek to apply this
flag once they have verified that no per-statement literal values are being
rendered within the compilation phase.

Fixes: #6184
Change-Id: I6fd5b5d94200458d4cb0e14f2f556dbc25e27e22
diff 442ca5c0 Tue Jan 12 14:14:38 MST 2021 Federico Caselli <cfederico87@gmail.com> Disallow non-native psycopg2 Unicode in Python 3; update docs

Fixed issue where the psycopg2 dialect would silently pass the
``use_native_unicode=False`` flag without actually having any effect under
Python 3, as the psycopg2 DBAPI uses Unicode unconditionally under Python
3. This usage now raises an :class:`_exc.ArgumentError` when used under
Python 3. Added test support for Python 2.

Additionally, added documentation for client_encoding parameter
that may be passed to libpq directly via psycopg2.

Change-Id: I40ddf6382c157fa9399c21f0e01064197ea100f8
diff 442ca5c0 Tue Jan 12 14:14:38 MST 2021 Federico Caselli <cfederico87@gmail.com> Disallow non-native psycopg2 Unicode in Python 3; update docs

Fixed issue where the psycopg2 dialect would silently pass the
``use_native_unicode=False`` flag without actually having any effect under
Python 3, as the psycopg2 DBAPI uses Unicode unconditionally under Python
3. This usage now raises an :class:`_exc.ArgumentError` when used under
Python 3. Added test support for Python 2.

Additionally, added documentation for client_encoding parameter
that may be passed to libpq directly via psycopg2.

Change-Id: I40ddf6382c157fa9399c21f0e01064197ea100f8
diff 442ca5c0 Tue Jan 12 14:14:38 MST 2021 Federico Caselli <cfederico87@gmail.com> Disallow non-native psycopg2 Unicode in Python 3; update docs

Fixed issue where the psycopg2 dialect would silently pass the
``use_native_unicode=False`` flag without actually having any effect under
Python 3, as the psycopg2 DBAPI uses Unicode unconditionally under Python
3. This usage now raises an :class:`_exc.ArgumentError` when used under
Python 3. Added test support for Python 2.

Additionally, added documentation for client_encoding parameter
that may be passed to libpq directly via psycopg2.

Change-Id: I40ddf6382c157fa9399c21f0e01064197ea100f8
diff 442ca5c0 Tue Jan 12 14:14:38 MST 2021 Federico Caselli <cfederico87@gmail.com> Disallow non-native psycopg2 Unicode in Python 3; update docs

Fixed issue where the psycopg2 dialect would silently pass the
``use_native_unicode=False`` flag without actually having any effect under
Python 3, as the psycopg2 DBAPI uses Unicode unconditionally under Python
3. This usage now raises an :class:`_exc.ArgumentError` when used under
Python 3. Added test support for Python 2.

Additionally, added documentation for client_encoding parameter
that may be passed to libpq directly via psycopg2.

Change-Id: I40ddf6382c157fa9399c21f0e01064197ea100f8
diff f9ef7fd3 Wed Sep 02 16:43:53 MDT 2020 RamonWill <ramonwilliams@hotmail.co.uk> Support for multiple hosts in PostgreSQL connection string

Provide support for multiple hosts in the PostgreSQL connection string.

A user requested for SQLAlchemy to support multiple hosts within a PostgreSQL URL string. The proposed fix allows this. In the event that the url contains multiple hosts the proposed code will convert the query["hosts"] tuple into a single string. This allows the hosts to then get converted into a valid dsn variable in the psycopg2 connect function.

This pull request is:

- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [X ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.

**Have a nice day!**
Fixes: #4392

Closes: #5554
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5554
Pull-request-sha: 3f7a0ab8df9f1411a9f1ac0e152583bc7bf0c365

Change-Id: I3f3768d51b8331de786ffdc025b7ecfc662eafe5
diff 4c2c2c40 Thu Dec 20 20:05:36 MST 2018 Mike Bayer <mike_mp@zzzcomputing.com> Add deprecation warnings to all deprecated APIs

A large change throughout the library has ensured that all objects, parameters,
and behaviors which have been noted as deprecated or legacy now emit
``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now
defaults to displaying deprecation warnings, as well as that modern test suites
based on tools like tox and pytest tend to display deprecation warnings,
this change should make it easier to note what API features are obsolete.

See the notes added to the changelog and migration notes for further
details.

Fixes: #4393
Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
diff 2f03ec08 Thu Feb 15 09:09:47 MST 2018 André Cruz <andre@cabine.org> Add ssl "operation timed out" message for psycopg2

Added "SSL SYSCALL error: Operation timed out" to the list
of messages that trigger a "disconnect" scenario for the
psycopg2 driver. Pull request courtesy André Cruz.

Change-Id: Ie1a8fc97e74b6906ccacf53dad70fed973c42b7f
Pull-request: https://bitbucket.org/zzzeek/sqlalchemy/pull-requests/3
/sqlalchemy/lib/sqlalchemy/dialects/oracle/
H A Dcx_oracle.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 20c0f774 Thu Apr 01 15:20:31 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> Default caching to opt-out for 3rd party dialects

Added a new flag to the :class:`_engine.Dialect` class called
:attr:`_engine.Dialect.supports_statement_cache`. This flag now needs to be present
directly on a dialect class in order for SQLAlchemy's
:ref:`query cache <sql_caching>` to take effect for that dialect. The
rationale is based on discovered issues such as :ticket:`6173` revealing
that dialects which hardcode literal values from the compiled statement,
often the numerical parameters used for LIMIT / OFFSET, will not be
compatible with caching until these dialects are revised to use the
parameters present in the statement only. For third party dialects where
this flag is not applied, the SQL logging will show the message "dialect
does not support caching", indicating the dialect should seek to apply this
flag once they have verified that no per-statement literal values are being
rendered within the compilation phase.

Fixes: #6184
Change-Id: I6fd5b5d94200458d4cb0e14f2f556dbc25e27e22
diff f1e96cb0 Sun Jan 10 11:44:14 MST 2021 Mike Bayer <mike_mp@zzzcomputing.com> reinvent xdist hooks in terms of pytest fixtures

To allow the "connection" pytest fixture and others work
correctly in conjunction with setup/teardown that expects
to be external to the transaction, remove and prevent any usage
of "xdist" style names that are hardcoded by pytest to run
inside of fixtures, even function level ones. Instead use
pytest autouse fixtures to implement our own
r"setup|teardown_test(?:_class)?" methods so that we can ensure
function-scoped fixtures are run within them. A new more
explicit flow is set up within plugin_base and pytestplugin
such that the order of setup/teardown steps, which there are now
many, is fully documented and controllable. New granularity
has been added to the test teardown phase to distinguish
between "end of the test" when lock-holding structures on
connections should be released to allow for table drops,
vs. "end of the test plus its teardown steps" when we can
perform final cleanup on connections and run assertions
that everything is closed out.

From there we can remove most of the defensive "tear down everything"
logic inside of engines which for many years would frequently dispose
of pools over and over again, creating for a broken and expensive
connection flow. A quick test shows that running test/sql/ against
a single Postgresql engine with the new approach uses 75% fewer new
connections, creating 42 new connections total, vs. 164 new
connections total with the previous system.

As part of this, the new fixtures metadata/connection/future_connection
have been integrated such that they can be combined together
effectively. The fixture_session(), provide_metadata() fixtures
have been improved, including that fixture_session() now strongly
references sessions which are explicitly torn down before
table drops occur afer a test.

Major changes have been made to the
ConnectionKiller such that it now features different "scopes" for
testing engines and will limit its cleanup to those testing
engines corresponding to end of test, end of test class, or
end of test session. The system by which it tracks DBAPI
connections has been reworked, is ultimately somewhat similar to
how it worked before but is organized more clearly along
with the proxy-tracking logic. A "testing_engine" fixture
is also added that works as a pytest fixture rather than a
standalone function. The connection cleanup logic should
now be very robust, as we now can use the same global
connection pools for the whole suite without ever disposing
them, while also running a query for PostgreSQL
locks remaining after every test and assert there are no open
transactions leaking between tests at all. Additional steps
are added that also accommodate for asyncio connections not
explicitly closed, as is the case for legacy sync-style
tests as well as the async tests themselves.

As always, hundreds of tests are further refined to use the
new fixtures where problems with loose connections were identified,
largely as a result of the new PostgreSQL assertions,
many more tests have moved from legacy patterns into the newest.

An unfortunate discovery during the creation of this system is that
autouse fixtures (as well as if they are set up by
@pytest.mark.usefixtures) are not usable at our current scale with pytest
4.6.11 running under Python 2. It's unclear if this is due
to the older version of pytest or how it implements itself for
Python 2, as well as if the issue is CPU slowness or just large
memory use, but collecting the full span of tests takes over
a minute for a single process when any autouse fixtures are in
place and on CI the jobs just time out after ten minutes.
So at the moment this patch also reinvents a small version of
"autouse" fixtures when py2k is running, which skips generating
the real fixture and instead uses two global pytest fixtures
(which don't seem to impact performance) to invoke the
"autouse" fixtures ourselves outside of pytest.
This will limit our ability to do more with fixtures
until we can remove py2k support.

py.test is still observed to be much slower in collection in the
4.6.11 version compared to modern 6.2 versions, so add support for new
TOX_POSTGRESQL_PY2K and TOX_MYSQL_PY2K environment variables that
will run the suite for fewer backends under Python 2. For Python 3
pin pytest to modern 6.2 versions where performance for collection
has been improved greatly.

Includes the following improvements:

Fixed bug in asyncio connection pool where ``asyncio.TimeoutError`` would
be raised rather than :class:`.exc.TimeoutError`. Also repaired the
:paramref:`_sa.create_engine.pool_timeout` parameter set to zero when using
the async engine, which previously would ignore the timeout and block
rather than timing out immediately as is the behavior with regular
:class:`.QueuePool`.

For asyncio the connection pool will now also not interact
at all with an asyncio connection whose ConnectionFairy is
being garbage collected; a warning that the connection was
not properly closed is emitted and the connection is discarded.
Within the test suite the ConnectionKiller is now maintaining
strong references to all DBAPI connections and ensuring they
are released when tests end, including those whose ConnectionFairy
proxies are GCed.

Identified cx_Oracle.stmtcachesize as a major factor in Oracle
test scalability issues, this can be reset on a per-test basis
rather than setting it to zero across the board. the addition
of this flag has resolved the long-standing oracle "two task"
error problem.

For SQL Server, changed the temp table style used by the
"suite" tests to be the double-pound-sign, i.e. global,
variety, which is much easier to test generically. There
are already reflection tests that are more finely tuned
to both styles of temp table within the mssql test
suite. Additionally, added an extra step to the
"dropfirst" mechanism for SQL Server that will remove
all foreign key constraints first as some issues were
observed when using this flag when multiple schemas
had not been torn down.

Identified and fixed two subtle failure modes in the
engine, when commit/rollback fails in a begin()
context manager, the connection is explicitly closed,
and when "initialize()" fails on the first new connection
of a dialect, the transactional state on that connection
is still rolled back.

Fixes: #5826
Fixes: #5827
Change-Id: Ib1d05cb8c7cf84f9a4bfd23df397dc23c9329bfe
diff 3ab5e378 Wed May 06 07:47:26 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Clarify create_engine encoding; update cx_Oracle

The create_engine()->encoding parameter is mostly irrelevant
under Python 3. make it clear this parameter is not generally
useful anymore and refer readers to the dialect documenation.

Update cx_Oracle documentation to feature many examples of
the encoding / nencoding parameters, remove extra detail that
is not generally useful and reorganize information into
more specific sections, de-emphasizing legacy / Python 2
specific sections.

Change-Id: I42dafb85de0a585c9e05e1e3d787d8d6bfa632c0
diff 3ab5e378 Wed May 06 07:47:26 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Clarify create_engine encoding; update cx_Oracle

The create_engine()->encoding parameter is mostly irrelevant
under Python 3. make it clear this parameter is not generally
useful anymore and refer readers to the dialect documenation.

Update cx_Oracle documentation to feature many examples of
the encoding / nencoding parameters, remove extra detail that
is not generally useful and reorganize information into
more specific sections, de-emphasizing legacy / Python 2
specific sections.

Change-Id: I42dafb85de0a585c9e05e1e3d787d8d6bfa632c0
diff d76cb721 Mon Aug 12 13:09:37 MDT 2019 Mike Bayer <mike_mp@zzzcomputing.com> Implment encoding_errors for cx_oracle

Added dialect-level flag ``encoding_errors`` to the cx_Oracle dialect,
which can be specified as part of :func:`.create_engine`. This is passed
to SQLAlchemy's unicode decoding converter under Python 2, and to
cx_Oracle's ``cursor.var()`` object as the ``encodingErrors`` parameter
under Python 3, for the very unusual case that broken encodings are present
in the target database which cannot be fetched unless error handling is
relaxed. The value is ultimately one of the Python "encoding errors"
parameters passed to ``decode()``.

Closes: #4801
Fixes: #4799
Change-Id: I1d542ba367bcd187347c54db1fee815f7890e71c
diff 4c2c2c40 Thu Dec 20 20:05:36 MST 2018 Mike Bayer <mike_mp@zzzcomputing.com> Add deprecation warnings to all deprecated APIs

A large change throughout the library has ensured that all objects, parameters,
and behaviors which have been noted as deprecated or legacy now emit
``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now
defaults to displaying deprecation warnings, as well as that modern test suites
based on tools like tox and pytest tend to display deprecation warnings,
this change should make it easier to note what API features are obsolete.

See the notes added to the changelog and migration notes for further
details.

Fixes: #4393
Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
diff d8efa225 Wed Jul 29 15:40:47 MDT 2015 Mike Bayer <mike_mp@zzzcomputing.com> - Fixed support for cx_Oracle version 5.2, which was tripping
up SQLAlchemy's version detection under Python 3 and inadvertently
not using the correct unicode mode for Python 3. This would cause
issues such as bound variables mis-interpreted as NULL and rows
silently not being returned.
fixes #3491
diff d8efa225 Wed Jul 29 15:40:47 MDT 2015 Mike Bayer <mike_mp@zzzcomputing.com> - Fixed support for cx_Oracle version 5.2, which was tripping
up SQLAlchemy's version detection under Python 3 and inadvertently
not using the correct unicode mode for Python 3. This would cause
issues such as bound variables mis-interpreted as NULL and rows
silently not being returned.
fixes #3491
/sqlalchemy/lib/sqlalchemy/testing/
H A Drequirements.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 4476dca0 Wed Mar 24 09:33:04 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> Repair pysqlcipher and use sqlcipher3

The ``pysqlcipher`` dialect now imports the ``sqlcipher3`` module
for Python 3 by default. Regressions have been repaired such that
the connection routine was not working.

To better support the post-connection steps of the pysqlcipher
dialect, a new hook Dialect.on_connect_url() is added, which
supersedes Dialect.on_connect() and is passed the URL object.
The dialect now pulls the passphrase and other cipher args
from the URL directly without including them in the
"connect" args. This will allow any user-defined extensibility
to connecting to work as it would for other dialects.

The commit also builds upon the extended routines in
sqlite/provisioning.py to better support running tests against
multiple simultaneous SQLite database files. Additionally enables
backend for test_sqlite which was skipping everything
for aiosqlite too, fortunately everything there is passing.

Fixes: #5848
Change-Id: I43f53ebc62298a84a4abe149e1eb699a027b7915
diff 3b8a1415 Sun Dec 20 10:20:10 MST 2020 Gord Thompson <gord@gordthompson.com> Fix issues with JSON and float/numeric

Decimal accuracy and behavior has been improved when extracting floating
point and/or decimal values from JSON strings using the
:meth:`_sql.sqltypes.JSON.Comparator.as_float` method, when the numeric
value inside of the JSON string has many significant digits; previously,
MySQL backends would truncate values with many significant digits and SQL
Server backends would raise an exception due to a DECIMAL cast with
insufficient significant digits. Both backends now use a FLOAT-compatible
approach that does not hardcode significant digits for floating point
values. For precision numerics, a new method
:meth:`_sql.sqltypes.JSON.Comparator.as_numeric` has been added which
accepts arguments for precision and scale, and will return values as Python
``Decimal`` objects with no floating point conversion assuming the DBAPI
supports it (all but pysqlite).

Fixes: #5788
Change-Id: I6eb51fe172a389548dd6e3c65efec9f1f538012e
diff 3ca8b453 Wed Oct 07 08:09:41 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Revert "Use monotonic time for pool age measurement"

This reverts commit 0220b58917b5a979891b5765f6ac5095e0368489.

I completely misread https://www.python.org/dev/peps/pep-0418/#rationale
and the accuracy of monotonic() is *worse* on windows than time.time(),
which is bizarre.

Change-Id: I2d571e268a2051bea68736507773d3904403af9e
diff 0220b589 Wed Oct 07 06:42:48 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Use monotonic time for pool age measurement

The internal clock used by the :class:`_pool.Pool` object is now
time.monotonic_time() under Python 3. Under Python 2, time.time() is still
used, which is legacy. This clock is used to measure the age of a
connection against its starttime, and used in comparisons against the
pool_timeout setting as well as the last time the pool was marked as
invalid to determine if the connection should be recycled. Previously,
time.time() was used which was subject to inaccuracies as a result of
system clock changes as well as poor time resolution on windows.

Change-Id: I94f90044c1809508e26a5a00134981c2a00d0405
diff fc97854f Thu Aug 13 22:58:56 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Bump minimum MySQL version to 5.0.2; use all-numeric server version

MySQL dialect's server_version_info tuple is now all numeric. String
tokens like "MariaDB" are no longer present so that numeric comparison
works in all cases. The .is_mariadb flag on the dialect should be
consulted for whether or not mariadb was detected. Additionally removed
structures meant to support extremely old MySQL versions 3.x and 4.x;
the minimum MySQL version supported is now version 5.0.2.

In addition, as the "MariaDB" name goes away from server version,
expand upon the change in I330815ebe572b6a9818377da56621397335fa702
to support the name "mariadb" throughout the dialect and test suite
when mariadb-only mode is used. This changes the "name" field
on the MariaDB dialect to "mariadb", which then implies a change
throughout the testing requirements system as well as all the
dialect-specific DDL argument names such as "mysql_engine" is
now specified as "mariadb_engine", etc. Make use of the
recent additions to test suite URL provisioning so that we can
force MariaDB databases to have a "mariadb-only" dialect which
allows us to test this name change fully.

Update documentation to refer to MySQL / MariaDB explicitly
as well as indicating the "mariadb_" prefix used for options.

It seems likely that MySQL and MariaDB version numbers are going to
start colliding at some point so having the "mariadb" name
be available as a totally separate dialect name should give us
some options in this regard.

Currently also includes a date related fix to a test for
the postgresql dialect that was implicitly assuming a
non-UTC timezone

Fixes: #4189
Change-Id: I00e76d00f62971e1f067bd61915fa6cc1cf64e5e
diff 43a111e3 Wed Jul 08 13:48:32 MDT 2020 Mike Bayer <mike_mp@zzzcomputing.com> Limit init_subclass test above python 3.6

The commit for I1654befe9eb1c8b8e7fc0784bdbe64284614f0ea #5357
runs the test on all Python 3 versions, however we need to limit
at least python 3.6 for this.

Change-Id: Ie86b78bbfd8c7bd013ff9aa7f8905328d792c1b3
diff 9e31fc74 Fri Jan 17 15:31:41 MST 2020 Mike Bayer <mike_mp@zzzcomputing.com> Remove jython code, remove all jython / pypy symbols

Removed all dialect code related to support for Jython and zxJDBC. Jython
has not been supported by SQLAlchemy for many years and it is not expected
that the current zxJDBC code is at all functional; for the moment it just
takes up space and adds confusion by showing up in documentation. At the
moment, it appears that Jython has achieved Python 2.7 support in its
releases but not Python 3. If Jython were to be supported again, the form
it should take is against the Python 3 version of Jython, and the various
zxJDBC stubs for various backends should be implemented as a third party
dialect.

Additionally modernized logic that distinguishes between "cpython"
and "pypy" to instead look at platform.python_distribution() which
reliably tells us if we are cPython or not; all booleans which
previously checked for pypy and sometimes jython are now converted
to be "not cpython", this impacts the test suite for tests that are
cPython centric.

Fixes: #5094
Change-Id: I226cb55827f997daf6b4f4a755c18e7f4eb8d9ad
diff 9e31fc74 Fri Jan 17 15:31:41 MST 2020 Mike Bayer <mike_mp@zzzcomputing.com> Remove jython code, remove all jython / pypy symbols

Removed all dialect code related to support for Jython and zxJDBC. Jython
has not been supported by SQLAlchemy for many years and it is not expected
that the current zxJDBC code is at all functional; for the moment it just
takes up space and adds confusion by showing up in documentation. At the
moment, it appears that Jython has achieved Python 2.7 support in its
releases but not Python 3. If Jython were to be supported again, the form
it should take is against the Python 3 version of Jython, and the various
zxJDBC stubs for various backends should be implemented as a third party
dialect.

Additionally modernized logic that distinguishes between "cpython"
and "pypy" to instead look at platform.python_distribution() which
reliably tells us if we are cPython or not; all booleans which
previously checked for pypy and sometimes jython are now converted
to be "not cpython", this impacts the test suite for tests that are
cPython centric.

Fixes: #5094
Change-Id: I226cb55827f997daf6b4f4a755c18e7f4eb8d9ad
/sqlalchemy/lib/sqlalchemy/dialects/mssql/
H A Dbase.pydiff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 3ea321ce Sat Sep 11 14:26:59 MDT 2021 Kevin Kirsche <kevin.kirsche@one.verizon.com> Fix various lib / test / examples typos (#7017)

* fix: lib/sqlalchemy/sql/lambdas.py

* fix: lib/sqlalchemy/sql/compiler.py

* fix: lib/sqlalchemy/sql/selectable.py

* fix: lib/sqlalchemy/orm/relationships.py

* fix: lib/sqlalchemy/dialects/mssql/base.py

* fix: lib/sql/test_compiler.py

* fix: lib/sqlalchemy/testing/requirements.py

* fix: lib/sqlalchemy/orm/path_registry.py

* fix: lib/sqlalchemy/dialects/postgresql/psycopg2.py

* fix: lib/sqlalchemy/cextension/immutabledict.c

* fix: lib/sqlalchemy/cextension/resultproxy.c

* fix: ./lib/sqlalchemy/dialects/oracle/cx_oracle.py

* fix: examples/versioned_rows/versioned_rows_w_versionid.py

* fix: examples/elementtree/optimized_al.py

* fix: test/orm/test_attribute.py

* fix: test/sql/test_compare.py

* fix: test/sql/test_type_expression.py

* fix: capitalization in test/dialect/mysql/test_compiler.py

* fix: typos in test/dialect/postgresql/test_reflection.py

* fix: typo in tox.ini comment

* fix: typo in /lib/sqlalchemy/orm/decl_api.py

* fix: typo in test/orm/test_update_delete.py

* fix: self-induced typo

* fix: typo in test/orm/test_query.py

* fix: typos in test/dialect/mssql/test_types.py

* fix: typo in test/sql/test_types.py
diff 707e5d70 Tue Jul 06 09:26:53 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> labeling refactor

To service #6718 and #6710, the system by which columns are
given labels in a SELECT statement as well as the system that
gives them keys in a .c or .selected_columns collection have
been refactored to provide a single source of truth for
both, in constrast to the previous approach that included
similar logic repeated in slightly different ways.

Main ideas:

1. ColumnElement attributes ._label, ._anon_label, ._key_label
are renamed to include the letters "tq", meaning
"table-qualified" - these labels are only used when rendering
a SELECT that has LABEL_STYLE_TABLENAME_PLUS_COL for its
label style; as this label style is primarily legacy, the
"tq" names should be isolated so that in a 2.0 style application
these aren't being used at all

2. The means by which the "labels" and "proxy keys" for the elements
of a SELECT has been centralized to a single source of truth;
previously, the three of _generate_columns_plus_names,
_generate_fromclause_column_proxies, and _column_naming_convention
all had duplicated rules between them, as well as that there
were a little bit of labeling rules in compiler._label_select_column
as well; by this we mean that the various "anon_label" "anon_key"
methods on ColumnElement were called by all four of these methods,
where there were many cases where it was necessary that one method
comes up with the same answer as another of the methods. This
has all been centralized into _generate_columns_plus_names
for all the names except the "proxy key", which is generated
by _column_naming_convention.

3. compiler._label_select_column has been rewritten to both not make
any naming decisions nor any "proxy key" decisions, only whether
to label or not to label; the _generate_columns_plus_names method
gives it the information, where the proxy keys come from
_column_naming_convention; previously, these proxy keys were matched
based on restatement of similar (but not really the same) logic in
two places. The heuristics of "whether to label or not to label"
are also reorganized to be much easier to read and understand.

4. a new method compiler._label_returning_column is added for dialects
to use in their "generate returning columns" methods. A
github search reveals a small number of third party dialects also
doing this using the prior _label_select_column method so we
try to make sure _label_select_column continues to work the
exact same way for that specific use case; for the "SELECT" use
case it now needs

5. After some attempts to do it different ways, for the case where
_proxy_key is giving us some kind of anon label, we are hard
changing it to "_no_label" right now, as there's not currently
a way to fully match anonymized labels from stmt.c or
stmt.selected_columns to what will be in the result map. The
idea of "_no_label" is to encourage the user to use label('name')
for columns they want to be able to target by string name that
don't have a natural name.

Change-Id: I7a92a66f3a7e459ccf32587ac0a3c306650daf11
diff 3c0a0a72 Wed Apr 14 14:14:36 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> don't base compilation off the int value of offset/limit part II

Fixed an additional regression in the same area as that of :ticket:`6184`,
where using a value of 0 for OFFSET in conjunction with LIMIT with SQL
Server would create a statement using "TOP", as was the behavior in 1.3,
however due to caching would then fail to respond accordingly to other
values of OFFSET. If the "0" wasn't first, then it would be fine. For the
fix, the "TOP" syntax is now only emitted if the OFFSET value is omitted
entirely, that is, :meth:`_sql.Select.offset` is not used. Note that this
change now requires that if the "with_ties" or "percent" modifiers are
used, the statement can't specify an OFFSET of zero, it now needs to be
omitted entirely.

Fixes: #6265
Change-Id: If30596b8dcd9f2ce4221cd87c5407fa81f5f9a90
diff 20c0f774 Thu Apr 01 15:20:31 MDT 2021 Mike Bayer <mike_mp@zzzcomputing.com> Default caching to opt-out for 3rd party dialects

Added a new flag to the :class:`_engine.Dialect` class called
:attr:`_engine.Dialect.supports_statement_cache`. This flag now needs to be present
directly on a dialect class in order for SQLAlchemy's
:ref:`query cache <sql_caching>` to take effect for that dialect. The
rationale is based on discovered issues such as :ticket:`6173` revealing
that dialects which hardcode literal values from the compiled statement,
often the numerical parameters used for LIMIT / OFFSET, will not be
compatible with caching until these dialects are revised to use the
parameters present in the statement only. For third party dialects where
this flag is not applied, the SQL logging will show the message "dialect
does not support caching", indicating the dialect should seek to apply this
flag once they have verified that no per-statement literal values are being
rendered within the compilation phase.

Fixes: #6184
Change-Id: I6fd5b5d94200458d4cb0e14f2f556dbc25e27e22
diff 3c53cfaa Tue Jan 12 12:58:22 MST 2021 Gord Thompson <gord@gordthompson.com> Add mssql+turbodbc to external dialect list

Fixes: #5834
Change-Id: I1f207b84751e7e3425aa9e8e393787eeb9b595b7
diff f1e96cb0 Sun Jan 10 11:44:14 MST 2021 Mike Bayer <mike_mp@zzzcomputing.com> reinvent xdist hooks in terms of pytest fixtures

To allow the "connection" pytest fixture and others work
correctly in conjunction with setup/teardown that expects
to be external to the transaction, remove and prevent any usage
of "xdist" style names that are hardcoded by pytest to run
inside of fixtures, even function level ones. Instead use
pytest autouse fixtures to implement our own
r"setup|teardown_test(?:_class)?" methods so that we can ensure
function-scoped fixtures are run within them. A new more
explicit flow is set up within plugin_base and pytestplugin
such that the order of setup/teardown steps, which there are now
many, is fully documented and controllable. New granularity
has been added to the test teardown phase to distinguish
between "end of the test" when lock-holding structures on
connections should be released to allow for table drops,
vs. "end of the test plus its teardown steps" when we can
perform final cleanup on connections and run assertions
that everything is closed out.

From there we can remove most of the defensive "tear down everything"
logic inside of engines which for many years would frequently dispose
of pools over and over again, creating for a broken and expensive
connection flow. A quick test shows that running test/sql/ against
a single Postgresql engine with the new approach uses 75% fewer new
connections, creating 42 new connections total, vs. 164 new
connections total with the previous system.

As part of this, the new fixtures metadata/connection/future_connection
have been integrated such that they can be combined together
effectively. The fixture_session(), provide_metadata() fixtures
have been improved, including that fixture_session() now strongly
references sessions which are explicitly torn down before
table drops occur afer a test.

Major changes have been made to the
ConnectionKiller such that it now features different "scopes" for
testing engines and will limit its cleanup to those testing
engines corresponding to end of test, end of test class, or
end of test session. The system by which it tracks DBAPI
connections has been reworked, is ultimately somewhat similar to
how it worked before but is organized more clearly along
with the proxy-tracking logic. A "testing_engine" fixture
is also added that works as a pytest fixture rather than a
standalone function. The connection cleanup logic should
now be very robust, as we now can use the same global
connection pools for the whole suite without ever disposing
them, while also running a query for PostgreSQL
locks remaining after every test and assert there are no open
transactions leaking between tests at all. Additional steps
are added that also accommodate for asyncio connections not
explicitly closed, as is the case for legacy sync-style
tests as well as the async tests themselves.

As always, hundreds of tests are further refined to use the
new fixtures where problems with loose connections were identified,
largely as a result of the new PostgreSQL assertions,
many more tests have moved from legacy patterns into the newest.

An unfortunate discovery during the creation of this system is that
autouse fixtures (as well as if they are set up by
@pytest.mark.usefixtures) are not usable at our current scale with pytest
4.6.11 running under Python 2. It's unclear if this is due
to the older version of pytest or how it implements itself for
Python 2, as well as if the issue is CPU slowness or just large
memory use, but collecting the full span of tests takes over
a minute for a single process when any autouse fixtures are in
place and on CI the jobs just time out after ten minutes.
So at the moment this patch also reinvents a small version of
"autouse" fixtures when py2k is running, which skips generating
the real fixture and instead uses two global pytest fixtures
(which don't seem to impact performance) to invoke the
"autouse" fixtures ourselves outside of pytest.
This will limit our ability to do more with fixtures
until we can remove py2k support.

py.test is still observed to be much slower in collection in the
4.6.11 version compared to modern 6.2 versions, so add support for new
TOX_POSTGRESQL_PY2K and TOX_MYSQL_PY2K environment variables that
will run the suite for fewer backends under Python 2. For Python 3
pin pytest to modern 6.2 versions where performance for collection
has been improved greatly.

Includes the following improvements:

Fixed bug in asyncio connection pool where ``asyncio.TimeoutError`` would
be raised rather than :class:`.exc.TimeoutError`. Also repaired the
:paramref:`_sa.create_engine.pool_timeout` parameter set to zero when using
the async engine, which previously would ignore the timeout and block
rather than timing out immediately as is the behavior with regular
:class:`.QueuePool`.

For asyncio the connection pool will now also not interact
at all with an asyncio connection whose ConnectionFairy is
being garbage collected; a warning that the connection was
not properly closed is emitted and the connection is discarded.
Within the test suite the ConnectionKiller is now maintaining
strong references to all DBAPI connections and ensuring they
are released when tests end, including those whose ConnectionFairy
proxies are GCed.

Identified cx_Oracle.stmtcachesize as a major factor in Oracle
test scalability issues, this can be reset on a per-test basis
rather than setting it to zero across the board. the addition
of this flag has resolved the long-standing oracle "two task"
error problem.

For SQL Server, changed the temp table style used by the
"suite" tests to be the double-pound-sign, i.e. global,
variety, which is much easier to test generically. There
are already reflection tests that are more finely tuned
to both styles of temp table within the mssql test
suite. Additionally, added an extra step to the
"dropfirst" mechanism for SQL Server that will remove
all foreign key constraints first as some issues were
observed when using this flag when multiple schemas
had not been torn down.

Identified and fixed two subtle failure modes in the
engine, when commit/rollback fails in a begin()
context manager, the connection is explicitly closed,
and when "initialize()" fails on the first new connection
of a dialect, the transactional state on that connection
is still rolled back.

Fixes: #5826
Fixes: #5827
Change-Id: Ib1d05cb8c7cf84f9a4bfd23df397dc23c9329bfe
diff 3b8a1415 Sun Dec 20 10:20:10 MST 2020 Gord Thompson <gord@gordthompson.com> Fix issues with JSON and float/numeric

Decimal accuracy and behavior has been improved when extracting floating
point and/or decimal values from JSON strings using the
:meth:`_sql.sqltypes.JSON.Comparator.as_float` method, when the numeric
value inside of the JSON string has many significant digits; previously,
MySQL backends would truncate values with many significant digits and SQL
Server backends would raise an exception due to a DECIMAL cast with
insufficient significant digits. Both backends now use a FLOAT-compatible
approach that does not hardcode significant digits for floating point
values. For precision numerics, a new method
:meth:`_sql.sqltypes.JSON.Comparator.as_numeric` has been added which
accepts arguments for precision and scale, and will return values as Python
``Decimal`` objects with no floating point conversion assuming the DBAPI
supports it (all but pysqlite).

Fixes: #5788
Change-Id: I6eb51fe172a389548dd6e3c65efec9f1f538012e
diff 3a0e0531 Mon Nov 04 15:11:21 MST 2019 CaselIT <cfederico87@gmail.com> Support for generated columns

Added DDL support for "computed columns"; these are DDL column
specifications for columns that have a server-computed value, either upon
SELECT (known as "virtual") or at the point of which they are INSERTed or
UPDATEd (known as "stored"). Support is established for Postgresql, MySQL,
Oracle SQL Server and Firebird. Thanks to Federico Caselli for lots of work
on this one.

ORM round trip tests included. The ORM makes use of existing
FetchedValue support and no additional ORM logic is present for
the basic feature.

It has been observed that Oracle RETURNING does not return the
new value of a computed column upon UPDATE; it returns the
prior value. As this is very dangerous, a warning is emitted
if a computed column is rendered into the RETURNING clause
of an UPDATE statement.

Fixes: #4894
Closes: #4928
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4928
Pull-request-sha: d39c521d5ac6ebfb4fb5b53846451de79752e64c

Change-Id: I2610b2999a5b1b127ed927dcdaeee98b769643ce
diff af3078e6 Thu Sep 05 18:10:08 MDT 2019 Gord Thompson <gord@gordthompson.com> add SQL Server 2017 to mssql/base.py - Fixes #4833

### Description
Add version 14 for SQL Server 2017 to mssql/base.py

Fixes #4833

### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)

-->

This pull request is:

- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.

Closes: #4832
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4832
Pull-request-sha: 3d72335284fd585c40b961bbc7013e53f3874cb5

Change-Id: Ib53a938a22386aab9e603048753cd2966c5d2b33

Completed in 1264 milliseconds