SQLAlchemy provides abstractions for most common database data types, and a mechanism for specifying your own custom data types.
The methods and attributes of type objects are rarely used directly. Type objects are supplied to Table definitions and can be supplied as type hints to functions for occasions where the database driver returns an incorrect type.
>>> users = Table('users', metadata,
... Column('id', Integer, primary_key=True)
... Column('login', String(32))
... )SQLAlchemy will use the Integer and String(32) type information when issuing a CREATE TABLE statement and will use it again when reading back rows SELECTed from the database. Functions that accept a type (such as Column()) will typically accept a type class or instance; Integer is equivalent to Integer() with no construction arguments in this case.
Generic types specify a column that can read, write and store a particular type of Python data. SQLAlchemy will choose the best database column type available on the target database when issuing a CREATE TABLE statement. For complete control over which column type is emitted in CREATE TABLE, such as VARCHAR see SQL Standard Types and the other sections of this chapter.
Bases: sqlalchemy.types.TypeEngine, sqlalchemy.types.SchemaType
A bool datatype.
Boolean typically uses BOOLEAN or SMALLINT on the DDL side, and on the Python side deals in True or False.
Construct a Boolean.
| Parameters: |
|
|---|
Bases: sqlalchemy.types._DateAffinity, sqlalchemy.types.TypeEngine
A type for datetime.date() objects.
Bases: sqlalchemy.types._DateAffinity, sqlalchemy.types.TypeEngine
A type for datetime.datetime() objects.
Date and time types return objects from the Python datetime module. Most DBAPIs have built in support for the datetime module, with the noted exception of SQLite. In the case of SQLite, date and time types are stored as strings which are then converted back to datetime objects when rows are returned.
Bases: sqlalchemy.types.String, sqlalchemy.types.SchemaType
Generic Enum Type.
The Enum type provides a set of possible string values which the column is constrained towards.
By default, uses the backend’s native ENUM type if available, else uses VARCHAR + a CHECK constraint.
Construct an enum.
Keyword arguments which don’t apply to a specific backend are ignored by that backend.
| Parameters: |
|
|---|
Bases: sqlalchemy.types.Numeric
A type for float numbers.
Returns Python float objects by default, applying conversion as needed.
Bases: sqlalchemy.types._DateAffinity, sqlalchemy.types.TypeEngine
A type for int integers.
Bases: sqlalchemy.types._DateAffinity, sqlalchemy.types.TypeDecorator
A type for datetime.timedelta() objects.
The Interval type deals with datetime.timedelta objects. In PostgreSQL, the native INTERVAL type is used; for others, the value is stored as a date which is relative to the “epoch” (Jan. 1, 1970).
Note that the Interval type does not currently provide date arithmetic operations on platforms which do not support interval types natively. Such operations usually require transformation of both sides of the expression (such as, conversion of both sides into integer epoch values first) which currently is a manual procedure (such as via func).
Construct an Interval object.
| Parameters: |
|
|---|
Bases: sqlalchemy.types._Binary
A type for large binary byte data.
The Binary type generates BLOB or BYTEA when tables are created, and also converts incoming values using the Binary callable provided by each DB-API.
Construct a LargeBinary type.
| Parameter: | length – optional, a length for the column for use in DDL statements, for those BLOB types that accept a length (i.e. MySQL). It does not produce a small BINARY/VARBINARY type - use the BINARY/VARBINARY types specifically for those. May be safely omitted if no CREATE TABLE will be issued. Certain databases may require a length for use in DDL, and will raise an exception when the CREATE TABLE DDL is issued. |
|---|
Bases: sqlalchemy.types._DateAffinity, sqlalchemy.types.TypeEngine
A type for fixed precision numbers.
Typically generates DECIMAL or NUMERIC. Returns decimal.Decimal objects by default, applying conversion as needed.
Construct a Numeric.
| Parameters: |
|
|---|
When using the Numeric type, care should be taken to ensure that the asdecimal setting is apppropriate for the DBAPI in use - when Numeric applies a conversion from Decimal->float or float-> Decimal, this conversion incurs an additional performance overhead for all result columns received.
DBAPIs that return Decimal natively (e.g. psycopg2) will have better accuracy and higher performance with a setting of True, as the native translation to Decimal reduces the amount of floating- point issues at play, and the Numeric type itself doesn’t need to apply any further conversions. However, another DBAPI which returns floats natively will incur an additional conversion overhead, and is still subject to floating point data loss - in which case asdecimal=False will at least remove the extra conversion overhead.
Bases: sqlalchemy.types.MutableType, sqlalchemy.types.TypeDecorator
Holds Python objects, which are serialized using pickle.
PickleType builds upon the Binary type to apply Python’s pickle.dumps() to incoming objects, and pickle.loads() on the way out, allowing any pickleable Python object to be stored as a serialized binary field.
Note: be sure to read the notes for MutableType regarding ORM performance implications.
Construct a PickleType.
| Parameters: |
|
|---|
Bases: object
Mark a type as possibly requiring schema-level DDL for usage.
Supports types that must be explicitly created/dropped (i.e. PG ENUM type) as well as types that are complimented by table or schema level constraints, triggers, and other rules.
Bases: sqlalchemy.types.Integer
A type for smaller int integers.
Typically generates a SMALLINT in DDL, and otherwise acts like a normal Integer on the Python side.
Bases: sqlalchemy.types.Concatenable, sqlalchemy.types.TypeEngine
The base for all string and character types.
In SQL, corresponds to VARCHAR. Can also take Python unicode objects and encode to the database’s encoding in bind params (and the reverse for result sets.)
The length field is usually required when the String type is used within a CREATE TABLE statement, as VARCHAR requires a length on most databases.
Create a string-holding type.
| Parameters: |
|
|---|
Bases: sqlalchemy.types.String
A variably sized string type.
In SQL, usually corresponds to CLOB or TEXT. Can also take Python unicode objects and encode to the database’s encoding in bind params (and the reverse for result sets.)
Bases: sqlalchemy.types._DateAffinity, sqlalchemy.types.TypeEngine
A type for datetime.time() objects.
Bases: sqlalchemy.types.String
A variable length Unicode string.
The Unicode type is a String which converts Python unicode objects (i.e., strings that are defined as u'somevalue') into encoded bytestrings when passing the value to the database driver, and similarly decodes values from the database back into Python unicode objects.
It’s roughly equivalent to using a String object with convert_unicode=True, however the type has other significances in that it implies the usage of a unicode-capable type being used on the backend, such as NVARCHAR. This may affect what type is emitted when issuing CREATE TABLE and also may effect some DBAPI-specific details, such as type information passed along to setinputsizes().
When using the Unicode type, it is only appropriate to pass Python unicode objects, and not plain str. If a bytestring (str) is passed, a runtime warning is issued. If you notice your application raising these warnings but you’re not sure where, the Python warnings filter can be used to turn these warnings into exceptions which will illustrate a stack trace:
import warnings
warnings.simplefilter('error')Bytestrings sent to and received from the database are encoded using the dialect’s encoding, which defaults to utf-8.
Create a Unicode-converting String type.
| Parameters: |
|
|---|
Bases: sqlalchemy.types.Text
An unbounded-length Unicode string.
See Unicode for details on the unicode behavior of this object.
Like Unicode, usage the UnicodeText type implies a unicode-capable type being used on the backend, such as NCLOB.
Create a Unicode-converting Text type.
| Parameter: | length – optional, a length for the column for use in DDL statements. May be safely omitted if no CREATE TABLE will be issued. Certain databases may require a length for use in DDL, and will raise an exception when the CREATE TABLE DDL is issued. Whether the value is interpreted as bytes or characters is database specific. |
|---|
The SQL standard types always create database column types of the same name when CREATE TABLE is issued. Some types may not be supported on all databases.
Bases: sqlalchemy.types._Binary
The SQL BINARY type.
Bases: sqlalchemy.types.LargeBinary
The SQL BLOB type.
Bases: sqlalchemy.types.Boolean
The SQL BOOLEAN type.
Bases: sqlalchemy.types.String
The SQL CHAR type.
Bases: sqlalchemy.types.Text
The CLOB type.
This type is found in Oracle and Informix.
Bases: sqlalchemy.types.Date
The SQL DATE type.
Bases: sqlalchemy.types.DateTime
The SQL DATETIME type.
Bases: sqlalchemy.types.Numeric
The SQL DECIMAL type.
Bases: sqlalchemy.types.Float
The SQL FLOAT type.
Bases: sqlalchemy.types.Integer
The SQL INT or INTEGER type.
Bases: sqlalchemy.types.Unicode
The SQL NCHAR type.
Bases: sqlalchemy.types.Unicode
The SQL NVARCHAR type.
Bases: sqlalchemy.types.Numeric
The SQL NUMERIC type.
Bases: sqlalchemy.types.SmallInteger
The SQL SMALLINT type.
Bases: sqlalchemy.types.Text
The SQL TEXT type.
Bases: sqlalchemy.types.Time
The SQL TIME type.
Bases: sqlalchemy.types.DateTime
The SQL TIMESTAMP type.
Bases: sqlalchemy.types._Binary
The SQL VARBINARY type.
Bases: sqlalchemy.types.String
The SQL VARCHAR type.
Database-specific types are also available for import from each database’s dialect module. See the sqlalchemy.dialects reference for the database you’re interested in.
For example, MySQL has a BIGINTEGER type and PostgreSQL has an INET type. To use these, import them from the module explicitly:
from sqlalchemy.dialects import mysql
table = Table('foo', meta,
Column('id', mysql.BIGINTEGER),
Column('enumerates', mysql.ENUM('a', 'b', 'c'))
)Or some PostgreSQL types:
from sqlalchemy.dialects import postgresql
table = Table('foo', meta,
Column('ipaddress', postgresql.INET),
Column('elements', postgresql.ARRAY(str))
)Each dialect provides the full set of typenames supported by that backend within its __all__ collection, so that a simple import * or similar will import all supported types as implemented for that backend:
from sqlalchemy.dialects.postgresql import *
t = Table('mytable', metadata,
Column('id', INTEGER, primary_key=True),
Column('name', VARCHAR(300)),
Column('inetaddr', INET)
)Where above, the INTEGER and VARCHAR types are ultimately from sqlalchemy.types, and INET is specific to the Postgresql dialect.
Some dialect level types have the same name as the SQL standard type, but also provide additional arguments. For example, MySQL implements the full range of character and string types including additional arguments such as collation and charset:
from sqlalchemy.dialects.mysql import VARCHAR, TEXT
table = Table('foo', meta,
Column('col1', VARCHAR(200, collation='binary')),
Column('col2', TEXT(charset='latin1'))
)User-defined types may be created to match special capabilities of a particular database or simply for implementing custom processing logic in Python.
The simplest method is implementing a TypeDecorator, a helper class that makes it easy to augment the bind parameter and result processing capabilities of one of the built in types.
To build a type object from scratch, subclass :class:UserDefinedType.
Bases: sqlalchemy.types.AbstractType
Allows the creation of types which add additional functionality to an existing type.
This method is preferred to direct subclassing of SQLAlchemy’s built-in types as it ensures that all required functionality of the underlying type is kept in place.
Typical usage:
import sqlalchemy.types as types
class MyType(types.TypeDecorator):
'''Prefixes Unicode values with "PREFIX:" on the way in and
strips it off on the way out.
'''
impl = types.Unicode
def process_bind_param(self, value, dialect):
return "PREFIX:" + value
def process_result_value(self, value, dialect):
return value[7:]
def copy(self):
return MyType(self.impl.length)The class-level “impl” variable is required, and can reference any TypeEngine class. Alternatively, the load_dialect_impl() method can be used to provide different type classes based on the dialect given; in this case, the “impl” variable can reference TypeEngine as a placeholder.
Types that receive a Python type that isn’t similar to the ultimate type used may want to define the TypeDecorator.coerce_compared_value() method. This is used to give the expression system a hint when coercing Python objects into bind parameters within expressions. Consider this expression:
mytable.c.somecol + datetime.date(2009, 5, 15)Above, if “somecol” is an Integer variant, it makes sense that we’re doing date arithmetic, where above is usually interpreted by databases as adding a number of days to the given date. The expression system does the right thing by not attempting to coerce the “date()” value into an integer-oriented bind parameter.
However, in the case of TypeDecorator, we are usually changing an incoming Python type to something new - TypeDecorator by default will “coerce” the non-typed side to be the same type as itself. Such as below, we define an “epoch” type that stores a date value as an integer:
class MyEpochType(types.TypeDecorator):
impl = types.Integer
epoch = datetime.date(1970, 1, 1)
def process_bind_param(self, value, dialect):
return (value - self.epoch).days
def process_result_value(self, value, dialect):
return self.epoch + timedelta(days=value)Our expression of somecol + date with the above type will coerce the “date” on the right side to also be treated as MyEpochType.
This behavior can be overridden via the coerce_compared_value() method, which returns a type that should be used for the value of the expression. Below we set it such that an integer value will be treated as an Integer, and any other value is assumed to be a date and will be treated as a MyEpochType:
def coerce_compared_value(self, op, value):
if isinstance(value, int):
return Integer()
else:
return selfSuggest a type for a ‘coerced’ Python value in an expression.
By default, returns self. This method is called by the expression system when an object using this type is on the left or right side of an expression against a plain Python object which does not yet have a SQLAlchemy type assigned:
expr = table.c.somecolumn + 35Where above, if somecolumn uses this type, this method will be called with the value operator.add and 35. The return value is whatever SQLAlchemy type should be used for 35 for this particular operation.
Loads the dialect-specific implementation of this type.
by default calls dialect.type_descriptor(self.impl), but can be overridden to provide different behavior.
Bases: sqlalchemy.types.TypeEngine
Base for user defined types.
This should be the base of new types. Note that for most cases, TypeDecorator is probably more appropriate:
import sqlalchemy.types as types
class MyType(types.UserDefinedType):
def __init__(self, precision = 8):
self.precision = precision
def get_col_spec(self):
return "MYTYPE(%s)" % self.precision
def bind_processor(self, dialect):
def process(value):
return value
return process
def result_processor(self, dialect, coltype):
def process(value):
return value
return processOnce the type is made, it’s immediately usable:
table = Table('foo', meta,
Column('id', Integer, primary_key=True),
Column('data', MyType(16))
)A hook which allows the given operator to be adapted to something new.
See also UserDefinedType._adapt_expression(), an as-yet- semi-public method with greater capability in this regard.
Return a conversion function for processing bind values.
Returns a callable which will receive a bind parameter value as the sole positional argument and will return a value to send to the DB-API.
If processing is not necessary, the method should return None.
Return the corresponding type object from the underlying DB-API, if any.
This can be useful for calling setinputsizes(), for example.
Return True if the target Python type is ‘mutable’.
This allows systems like the ORM to know if a column value can be considered ‘not changed’ by comparing the identity of objects alone.
Use the MutableType mixin or override this method to return True in custom types that hold mutable values such as dict, list and custom objects.
Return a conversion function for processing result row values.
Returns a callable which will receive a result row column value as the sole positional argument and will return a value to return to the user.
If processing is not necessary, the method should return None.
Bases: sqlalchemy.types.AbstractType
Base for built-in types.
Return a conversion function for processing bind values.
Returns a callable which will receive a bind parameter value as the sole positional argument and will return a value to send to the DB-API.
If processing is not necessary, the method should return None.
Return the corresponding type object from the underlying DB-API, if any.
This can be useful for calling setinputsizes(), for example.
Return True if the target Python type is ‘mutable’.
This allows systems like the ORM to know if a column value can be considered ‘not changed’ by comparing the identity of objects alone.
Use the MutableType mixin or override this method to return True in custom types that hold mutable values such as dict, list and custom objects.
Return a conversion function for processing result row values.
Returns a callable which will receive a result row column value as the sole positional argument and will return a value to return to the user.
If processing is not necessary, the method should return None.
Bases: sqlalchemy.sql.visitors.Visitable
Defines a bind parameter processing function.
| Parameter: | dialect – Dialect instance in use. |
|---|
Return the corresponding type object from the underlying DB-API, if any.
This can be useful for calling setinputsizes(), for example.
Return True if the target Python type is ‘mutable’.
This allows systems like the ORM to know if a column value can be considered ‘not changed’ by comparing the identity of objects alone.
Use the MutableType mixin or override this method to return True in custom types that hold mutable values such as dict, list and custom objects.
Defines a result-column processing function.
| Parameters: |
|
|---|
Bases: object
A mixin that marks a TypeEngine as representing a mutable Python object type.
“mutable” means that changes can occur in place to a value of this type. Examples includes Python lists, dictionaries, and sets, as well as user-defined objects. The primary need for identification of “mutable” types is by the ORM, which applies special rules to such values in order to guarantee that changes are detected. These rules may have a significant performance impact, described below.
A MutableType usually allows a flag called mutable=True to enable/disable the “mutability” flag, represented on this class by is_mutable(). Examples include PickleType and ARRAY. Setting this flag to False effectively disables any mutability- specific behavior by the ORM.
copy_value() and compare_values() represent a copy and compare function for values of this type - implementing subclasses should override these appropriately.
The usage of mutable types has significant performance implications when using the ORM. In order to detect changes, the ORM must create a copy of the value when it is first accessed, so that changes to the current value can be compared against the “clean” database-loaded value. Additionally, when the ORM checks to see if any data requires flushing, it must scan through all instances in the session which are known to have “mutable” attributes and compare the current value of each one to its “clean” value. So for example, if the Session contains 6000 objects (a fairly large amount) and autoflush is enabled, every individual execution of Query will require a full scan of that subset of the 6000 objects that have mutable attributes, possibly resulting in tens of thousands of additional method calls for every query.
Note that for small numbers (< 100 in the Session at a time) of objects with “mutable” values, the performance degradation is negligible. In most cases it’s likely that the convenience allowed by “mutable” change detection outweighs the performance penalty.
It is perfectly fine to represent “mutable” data types with the “mutable” flag set to False, which eliminates any performance issues. It means that the ORM will only reliably detect changes for values of this type if a newly modified value is of a different identity (i.e., id(value)) than what was present before - i.e., instead of operations like these:
myobject.somedict['foo'] = 'bar'
myobject.someset.add('bar')
myobject.somelist.append('bar')You’d instead say:
myobject.somevalue = {'foo':'bar'}
myobject.someset = myobject.someset.union(['bar'])
myobject.somelist = myobject.somelist + ['bar']A future release of SQLAlchemy will include instrumented collection support for mutable types, such that at least usage of plain Python datastructures will be able to emit events for in-place changes, removing the need for pessimistic scanning for changes.
Bases: object
A mixin that marks a type as supporting ‘concatenation’, typically strings.
Bases: sqlalchemy.types.TypeEngine
An unknown type.
NullTypes will stand in if Table reflection encounters a column data type unknown to SQLAlchemy. The resulting columns are nearly fully usable: the DB-API adapter will handle all translation to and from the database data type.
NullType does not have sufficient information to particpate in a CREATE TABLE statement and will raise an exception if encountered during a create() operation.