Executing SQL using an ArcSDE connection
Sometimes when working with tables that are not versioned, it may seem easier to query a table in a database using Structured Query Language (SQL) rather than using one of the geoprocessing tools. The ArcSDESQLExecute object supports the execution of most SQL statements and will return to the user the results of those statements. The object will return a list of lists in the case where the statement returns rows from a table; for statements that do not return rows, it will return an indication of the success or failure of the statement (True for success, None for failure).
- ArcSDE and GDB system tables should not be altered using anything other than ArcGIS software. Corruption can occur if these system tables are edited directly using SQL.
- Edits on versioned data performed using SQL should only be done through multiversioned views.
- For geodatabases implemented in a relational database management system (DBMS) using DBMS data types and table formats, the DBMS's own SQL may be used to work with the information stored in the database.
- Accessing the information in a geodatabase via SQL allows external applications to access the tabular data managed by the geodatabase. These external applications may be nonspatial database applications or custom spatial applications developed in an environment other than ArcObjects. Be aware, though, that SQL access to the geodatabase bypasses geodatabase functionality, such as topology, networks, terrains, or other class or workspace extensions.
- It may be possible to use DBMS features such as triggers and stored procedures to maintain the relationships between tables needed for certain geodatabase functionality. However, executing SQL commands against the database without taking this extra functionality into account—for example, issuing INSERT statements to add records to a business table—will circumvent geodatabase functionality and possibly corrupt the relationships between data in your geodatabase.
- Before attempting to access or modify any ArcSDE or GDB objects, read all ArcSDE and geodatabase documentation about using SQL against ArcSDE or GDB objects in the DBMS.
|
Property |
|
|---|---|
|
transactionAutoCommit |
The autocommit interval. This can be used to force intermediate commits after a specified number of features have been modified. |
|
Methods |
|
|---|---|
|
commitTransaction() |
No DML statements will be committed until the commitTransaction method is called.  Note:A commit may also occur when the connection to ArcSDE is terminated (check specific DBMS documentation to see how each DBMS deals with a disconnect while in a transaction). |
|
execute(sql_statement) |
Sends the SQL statement to the database via an ArcSDE connection. If execute is run outside of a transaction, a commit will automatically take place once the SQL DML (INSERT, UPDATE, DELETE) statement has been executed. |
|
rollbackTransaction() |
Rollback any DML operations to the previous commit. |
|
startTransaction() |
To control when your changes are committed to the database, call the startTransaction method before calling execute. This starts a transaction, and no DML statements will be committed until the commitTransaction method is called. |
The execute method sends the SQL statement to the database via an ArcSDE connection. If execute is run outside a transaction, a commit will automatically take place once the SQL DML (INSERT, UPDATE, DELETE) statement has been executed.
ArcSDESQLExecute supports the ArcSDE Transaction model. Transactions are a property of an ArcSDE connection and bind operations so that an entire set of changes is either recorded or rejected. For example, if a set of parcels is being updated in a particular order, you can use a transaction to define the beginning and end of the changes so that all changes are posted together. If a set of changes can't be successfully inserted, the entire transaction is rejected. All transactions end when a user disconnects. ArcSDESQLExecute uses the provided ArcSDE API functions to start, commit, and rollback transactions.
If you want to control when your changes are committed to the database, call the startTransaction method before calling execute. This starts a transaction, and no DML statements will be committed until the commitTransaction method is called. A commit may also occur when the connection to ArcSDE is terminated (check specific DBMS documentation to see how each DBMS deals with a disconnect while in a transaction). Within a transaction, it is also possible to rollback any DML operations to the previous commit.
An autocommit interval property, transactionAutoCommit, is available. This can be used to force intermediate commits after a specified number of features have been modified.
See your specific DBMS SQL Reference guide for help writing SQL statements.
Examples
import arcpy
from arcpy import env
import sys
try:
# Make data path relative
#
env.workspace = sys.path[0]
# Two ways to create the object, which also creates the connection to ArcSDE.
# Using the first method, pass a set of strings containing the connection properties:
# <serverName>,<portNumber>,<version>,<userName>,<password>
# sdeConn = arcpy.ArcSDESQLExecute("gpserver3","5151","#","toolbox","toolbox")
# Using the second method pass the path to a valid ArcSDE connection file
#
sdeConn = arcpy.ArcSDESQLExecute("data\Connection to GPSERVER3.sde")
# Get the SQL statements, separated by ; from a text string.
#
SQLStatement = arcpy.GetParameterAsText(0)
SQLStatementList = SQLStatement.split(";")
print "+++++++++++++++++++++++++++++++++++++++++++++\n"
# For each SQL statement passed in, execute it.
#
for sql in SQLStatementList:
print "Execute SQL Statement: " + sql
try:
# Pass the SQL statement to the database.
#
sdeReturn = sdeConn.execute(sql)
except Exception, ErrorDesc:
print ErrorDesc
sdeReturn = False
# If the return value is a list (a list of lists), display each list as a row from the
# table being queried.
if isinstance(sdeReturn, list):
print "Number of rows returned by query: " + len(sdeReturn), "rows"
for row in sdeReturn:
print row
print "+++++++++++++++++++++++++++++++++++++++++++++\n"
else:
# If the return value was not a list, the statement was most likely a DDL statment.
# Check its status.
if sdeReturn == True:
print "SQL statement: " + sql + " ran sucessfully."
print "+++++++++++++++++++++++++++++++++++++++++++++\n"
else:
print "SQL statement: " + sql + " FAILED."
print "+++++++++++++++++++++++++++++++++++++++++++++\n"
except Exception, ErrorDesc:
print Exception, ErrorDesc
except:
print "Problem executing SQL."
# WARNING - DO NOT USE ON VERSIONED TABLES OR FEATURE CLASSES.
# DO NOT USE ON ANY ArcSDE or GDB SYSTEM TABLES.
# DOING SO MAY RESULT IN DATA CORRUPTION.
import arcpy
from arcpy import env
import sys
try:
# Make data path relative (not relevant unless data is moved here and paths modified)
#
env.workspace = sys.path[0]
#Column name:value that should be in the record.
#
SQLvalues = {"STREET_NAM":"'EUREKA'"}
#Value that is incorrect if found in the above column.
#
badVal = "'EREKA'"
#List of tables to look in for the bad value.
#
tableList = ["streetaddresses_blkA","streetaddresses_blkB", "streetaddresses_blkC"]
# Two ways to create the object, which also creates the connection to ArcSDE.
# Using the first method, pass a set of strings containing the connection properties:
# <serverName>,<portNumber>,<version>,<userName>,<password>
#
sdeConn = arcpy.ArcSDESQLExecute("gpserver3","5151","#","toolbox","toolbox")
# Using the second method pass the path to a valid ArcSDE connection file
# sdeConn = arcpy.ArcSDESQLExecute("data\Connection to GPSERVER3.sde")
for tbl in tableList:
print "++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
for col, val in SQLvalues.items():
print "++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
#Check for the incorrect value in the column for the specific rows. If the table contains the
#incorrect value, correct it using the update SQL statement.
#
print "Analyzing table " + tbl + " for bad data: Column:" + col + " Value: " + badVal
try:
sql = "select OBJECTID," + col + " from " + tbl + " where " + col + " = " + badVal
print "Attempt to execute SQL Statement: " + sql
sdeReturn = sdeConn.execute(sql)
except Exception, ErrorDesc:
print ErrorDesc
sdeReturn = False
if isinstance(sdeReturn, list):
if len(sdeReturn) > 0:
print "Identified " + str(len(sdeReturn)) + " rows with incorrect data. Starting transaction for update."
# Start the transaction
#
sdeConn.startTransaction()
print "Transaction started....."
# Perform the update
#
try:
sql = "update " + tbl + " set " + col + "=" + str(val) + " where " + col + " = " + badVal
print "Changing bad value: " + badVal + " to the good value: " + val + " using update statement:\n " + sql
sdeReturn = sdeConn.execute(sql)
except Exception, ErrorDesc:
print ErrorDesc
sdeReturn = False
# If the update completed sucessfully, commit the changes. If not, rollback.
#
if sdeReturn == True:
print "Update statement: \n" + sql + " ran successfully."
# Commit the changes
#
sdeConn.commitTransaction()
print "Commited Transaction"
# List the changes.
#
try:
print "Displaying updated rows for visual inspection."
sql = "select OBJECTID," + col + " from " + tbl + " where " + col + " = " + val
print "Executing SQL Statement: \n" + sql
sdeReturn = sdeConn.execute(sql)
except Exception, ErrorDesc:
print ErrorDesc
sdeReturn = False
if isinstance(sdeReturn, list):
print len(sdeReturn), "rows"
for row in sdeReturn:
print row
print "+++++++++++++++++++++++++++++++++++++++++++++\n"
else:
if sdeReturn == True:
print "SQL statement: \n" + sql + "\nran successfully."
print "+++++++++++++++++++++++++++++++++++++++++++++\n"
else:
print "SQL statement: \n" + sql + "\nFAILED."
print "+++++++++++++++++++++++++++++++++++++++++++++\n"
print "+++++++++++++++++++++++++++++++++++++++++++++\n"
else:
print "SQL statement: \n" + sql + "\nFAILED. Rolling back all changes."
# Rollback changes
#
sdeConn.rollbackTransaction()
print "Rolled back any changes."
print "+++++++++++++++++++++++++++++++++++++++++++++\n"
else:
print "No records required updating."
# Disconnect and exit
del sdeConn
print "++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"
except Exception, ErrorDesc:
print Exception, ErrorDesc
except:
print "Problem executing SQL."
