Fix COPY to STDOUT/from STDIN leaving the connection stuck - #1636
anandghegde wants to merge 2 commits into
Conversation
Running COPY ... TO STDOUT or COPY ... FROM STDIN (instead of \copy) made psycopg raise "COPY cannot be used with this method" after the server had already started the COPY. The connection stayed in the COPY state, so every later query failed with "another command is already in progress" and quitting asked about an ongoing transaction. End the COPY on the connection (abort COPY FROM STDIN, drain COPY TO STDOUT) and report an error that suggests \copy instead. Fixes dbcli#1505
| cur.execute(split_sql) | ||
| try: | ||
| cur.execute(split_sql) | ||
| except psycopg.ProgrammingError as e: |
There was a problem hiding this comment.
Can psycopg.ProgrammingError be raised in situations other than copy, and if so, how do we handle that?
There was a problem hiding this comment.
Yes, and those pass through unchanged. psycopg raises ProgrammingError in two broad cases here:
- Server errors with SQLSTATE class 42, which psycopg maps to subclasses such as
SyntaxErrorandUndefinedTable. By the time these are raised the server has already ended the statement, so the transaction status isIDLEorINERROR, neverACTIVE. - Client-side misuse (parameter count mismatches and similar).
execute_normal_sqlpasses no parameters, and these are raised before anything is sent, so the status is notACTIVEeither.
The only result that leaves the connection ACTIVE is a COPY_IN/COPY_OUT status (_raise_for_result in psycopg's _cursor_base.py). That is why the handler checks transaction_status rather than the message, and re-raises the original exception otherwise.
To pin this down I added test_other_programming_errors_are_not_rewritten in ec70d50: a syntax error and an undefined table still raise their own exception classes without the \copy message, and the connection stays usable. With the ACTIVE check removed, both cases fail. The full suite passes against Postgres 16 (2785 passed).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MA8JGs7pCoXFL2W6jEJ8sD
Description
Running
copy ... to stdoutorcopy ... from stdin(instead of\copy) broke the session. psycopg'scursor.execute()raises "COPY cannot be used with this method; use copy() instead", but only after the server has already started the COPY. The connection stays in the COPY state, so:ACTIVE, soquitasks about an ongoing transaction, andc/rfail for the same reasonFix: in
execute_normal_sql, whenexecute()raises aProgrammingErrorand the connection is stillACTIVE, end the COPY on the connection. ACOPY FROM STDINis aborted withput_copy_end, and the rows of aCOPY TO STDOUTare read and thrown away. Then the error is re-raised with a message that points to\copy:If the COPY runs inside
BEGIN, aCOPY TO STDOUTleaves the transaction open and usable. An abortedCOPY FROM STDINputs it in the failed state, the same as any other error.How I tested it (Postgres 17 in Docker, psycopg 3.3.5):
dbtesttests intests/test_pgexecute.pycoverCOPY TO STDOUT(small and 10k rows),COPY FROM STDIN, andCOPY TO STDOUTinside an open transaction. They fail onmainand pass with this change.pytest tests: 2783 passed, 1 xfailed, 1 xpassed.behave tests/features: 33 scenarios passed, 0 failed.ruff checkandruff format --diffare clean.docutils --halt=warning changelog.rstis clean.main,copy (select 1) to stdout csv;followed byquithits the "A transaction is ongoing" prompt, and any query in between fails. With this change, later queries work,copy ... from stdinandcopyinsidebeginbehave the same way,\copystill works, andquitexits right away.Fixes #1505
Checklist
changelog.rst.AUTHORSfile (or it's already there).pip install pre-commit && pre-commit install).