E.24. Release 9.3.3

E.24.1. Migration to Version 9.3.3

A dump/restore is not required for those running 9.3.X.

However, several of the issues corrected in this release could have resulted in corruption of foreign-key constraints; that is, there might now be referencing rows for which there is no matching row in the referenced table. It may be worthwhile to recheck such constraints after installing this update. The simplest way to do that is to drop and recreate each suspect constraint; however, that will require taking an exclusive lock on both tables, so it is unlikely to be acceptable in production databases. Alternatively, you can do a manual join query between the two tables to look for unmatched rows.

Note also the requirement for replication standby servers to be upgraded before their master server is upgraded.

Also, if you are upgrading from a version earlier than 9.3.2, see Section E.25.

E.24.2. Changes

• Shore up GRANT ... WITH ADMIN OPTION restrictions (Noah Misch)

  Granting a role without ADMIN OPTION is supposed to prevent the grantee from adding or removing members from the granted role, but this restriction was easily bypassed by doing SET ROLE first. The security impact is mostly that a role member can revoke the access of others, contrary to the wishes of his grantor. Unapproved role member additions are a lesser concern, since an uncooperative role member could provide most of his rights to others anyway by creating views or SECURITY DEFINER functions. (CVE-2014-0060)

• Prevent privilege escalation via manual calls to PL validator functions (Andres Freund)

  The primary role of PL validator functions is to be called implicitly during CREATE FUNCTION, but they are also normal SQL functions that a user can call explicitly. Calling a validator on a function actually written in some other language was not checked for and could be exploited for privilege-escalation purposes. The fix involves adding a call to a privilege-checking function in each validator function. Non-core procedural languages will also need to make this change to their own validator functions, if any. (CVE-2014-0061)

• Avoid multiple name lookups during table and index DDL (Robert Haas, Andres Freund)

  If the name lookups come to different conclusions due to concurrent activity, we might perform some parts of the DDL on a different table than other parts. At least in the case of CREATE INDEX, this can be used to cause the permissions checks to be performed against a different table than the index creation, allowing for a privilege escalation attack. (CVE-2014-0062)

• Prevent buffer overrun with long datetime strings (Noah Misch)

  The MAXDATELEN constant was too small for the longest possible value of type interval, allowing a buffer overrun in interval_out(). Although the datetime input functions were more careful about avoiding buffer overrun, the limit was short enough to cause them to reject some valid inputs, such as input containing a very long timezone name. The ecpg library contained these vulnerabilities along with some of its own. (CVE-2014-0063)

• Prevent buffer overrun due to integer overflow in size calculations (Noah Misch, Heikki Linnakangas)

  Several functions, mostly type input functions, calculated an allocation size without checking for overflow. If overflow did occur, a too-small buffer would be allocated and then written past. (CVE-2014-0064)

• Prevent overruns of fixed-size buffers (Peter Eisentraut, Jozef Mlich)

  Use strlcpy() and related functions to provide a clear guarantee that fixed-size buffers are not overrun. Unlike the preceding items, it is unclear whether these cases really represent live issues, since in most cases there appear to be previous constraints on the size of the input string. Nonetheless it seems prudent to silence all Coverity warnings of this type. (CVE-2014-0065)

• Avoid crashing if crypt() returns NULL (Honza Horak, Bruce Momjian)

  There are relatively few scenarios in which crypt() could return NULL, but contrib/chkpass would crash if it did. One practical case in which this could be an issue is if libc is configured to refuse to execute unapproved hashing algorithms (e.g., "FIPS mode"). (CVE-2014-0066)

• Document risks of make check in the regression testing instructions (Noah Misch, Tom Lane)

  Since the temporary server started by make check uses "trust" authentication, another user on the same machine could connect to it as database superuser, and then potentially exploit the privileges of the operating-system user who started the tests. A future release will probably incorporate changes in the testing procedure to prevent this risk, but some public discussion is needed first. So for the moment, just warn people against using make check when there are untrusted users on the same machine. (CVE-2014-0067)

• Rework tuple freezing protocol (Álvaro Herrera, Andres Freund)

  The logic for tuple freezing was unable to handle some cases involving freezing of multixact IDs, with the practical effect that shared row-level locks might be forgotten once old enough.

  Fixing this required changing the WAL record format for tuple freezing. While this is no issue for standalone servers, when using replication it means that standby servers must be upgraded to 9.3.3 or later before their masters are. An older standby will be unable to interpret freeze records generated by a newer master, and will fail with a PANIC message. (In such a case, upgrading the standby should be sufficient to let it resume execution.)

• Create separate GUC parameters to control multixact freezing (Álvaro Herrera)

  9.3 requires multixact tuple labels to be frozen before they grow too old, in the same fashion as plain transaction ID labels have been frozen for some time. Previously, the transaction ID freezing parameters were used for multixact IDs too; but since the consumption rates of transaction IDs and multixact IDs can be quite different, this did not work very well. Introduce new settings vacuum_multixact_freeze_min_age, vacuum_multixact_freeze_table_age, and autovacuum_multixact_freeze_max_age to control when to freeze multixacts.

• Account for remote row locks propagated by local updates (Álvaro Herrera)

  If a row was locked by transaction A, and transaction B updated it, the new version of the row created by B would be locked by A, yet visible only to B. If transaction B then again updated the row, A's lock wouldn't get checked, thus possibly allowing B to complete when it shouldn't. This case is new in 9.3 since prior versions did not have any types of row locking that would permit another transaction to update the row at all.

  This oversight could allow referential integrity checks to give false positives (for instance, allow deletes that should have been rejected). Applications using the new commands SELECT FOR KEY SHARE and SELECT FOR NO KEY UPDATE might also have suffered locking failures of this kind.

• Prevent "forgetting" valid row locks when one of several holders of a row lock aborts (Álvaro Herrera)

  This was yet another mechanism by which a shared row lock could be lost, thus possibly allowing updates that should have been prevented by foreign-key constraints.

• Fix incorrect logic during update chain locking (Álvaro Herrera)

  This mistake could result in spurious "could not serialize access due to concurrent update" errors in REPEATABLE READ and SERIALIZABLE transaction isolation modes.

• Handle wraparound correctly during extension or truncation of pg_multixact/members (Andres Freund, Álvaro Herrera)

• Fix handling of 5-digit filenames in pg_multixact/members (Álvaro Herrera)

  As of 9.3, these names can be more than 4 digits, but the directory cleanup code ignored such files.

• Improve performance of multixact cache code (Álvaro Herrera)

• Optimize updating a row that's already locked by the same transaction (Andres Freund, Álvaro Herrera)

  This fixes a performance regression from pre-9.3 versions when doing SELECT FOR UPDATE followed by UPDATE/DELETE.

• During archive recovery, prefer highest timeline number when WAL segments with the same ID are present in both the archive and pg_xlog/ (Kyotaro Horiguchi)

  Previously, not-yet-archived segments could get ignored during recovery. This reverts an undesirable behavioral change in 9.3.0 back to the way things worked pre-9.3.

• Fix possible mis-replay of WAL records when some segments of a relation aren't full size (Greg Stark, Tom Lane)

  The WAL update could be applied to the wrong page, potentially many pages past where it should have been. Aside from corrupting data, this error has been observed to result in significant "bloat" of standby servers compared to their masters, due to updates being applied far beyond where the end-of-file should have been. This failure mode does not appear to be a significant risk during crash recovery, only when initially synchronizing a standby created from a base backup taken from a quickly-changing master.

• Fix bug in determining when recovery has reached consistency (Tomonari Katsumata, Heikki Linnakangas)

  In some cases WAL replay would mistakenly conclude that the database was already consistent at the start of replay, thus possibly allowing hot-standby queries before the database was really consistent. Other symptoms such as "PANIC: WAL contains references to invalid pages" were also possible.

• Fix WAL logging of visibility map changes (Heikki Linnakangas)

• Fix improper locking of btree index pages while replaying a VACUUM operation in hot-standby mode (Andres Freund, Heikki Linnakangas, Tom Lane)

  This error could result in "PANIC: WAL contains references to invalid pages" failures.

• Ensure that insertions into non-leaf GIN index pages write a full-page WAL record when appropriate (Heikki Linnakangas)

  The previous coding risked index corruption in the event of a partial-page write during a system crash.

• When pause_at_recovery_target and recovery_target_inclusive are both set, ensure the target record is applied before pausing, not after (Heikki Linnakangas)

• Ensure walreceiver sends hot-standby feedback messages on time even when there is a continuous stream of data (Andres Freund, Amit Kapila)

• Prevent timeout interrupts from taking control away from mainline code unless ImmediateInterruptOK is set (Andres Freund, Tom Lane)

  This is a serious issue for any application making use of statement timeouts, as it could cause all manner of strange failures after a timeout occurred. We have seen reports of "stuck" spinlocks, ERRORs being unexpectedly promoted to PANICs, unkillable backends, and other misbehaviors.

• Fix race conditions during server process exit (Robert Haas)

  Ensure that signal handlers don't attempt to use the process's MyProc pointer after it's no longer valid.

• Fix race conditions in walsender shutdown logic and walreceiver SIGHUP signal handler (Tom Lane)

• Fix unsafe references to errno within error reporting logic (Christian Kruse)

  This would typically lead to odd behaviors such as missing or inappropriate HINT fields.

• Fix possible crashes from using ereport() too early during server startup (Tom Lane)

  The principal case we've seen in the field is a crash if the server is started in a directory it doesn't have permission to read.

• Clear retry flags properly in OpenSSL socket write function (Alexander Kukushkin)

  This omission could result in a server lockup after unexpected loss of an SSL-encrypted connection.

• Fix length checking for Unicode identifiers (U&"..." syntax) containing escapes (Tom Lane)

  A spurious truncation warning would be printed for such identifiers if the escaped form of the identifier was too long, but the identifier actually didn't need truncation after de-escaping.

• Fix parsing of Unicode literals and identifiers just before the end of a command string or function body (Tom Lane)

• Allow keywords that are type names to be used in lists of roles (Stephen Frost)

  A previous patch allowed such keywords to be used without quoting in places such as role identifiers; but it missed cases where a list of role identifiers was permitted, such as DROP ROLE.

• Fix parser crash for EXISTS(SELECT * FROM zero_column_table) (Tom Lane)

• Fix possible crash due to invalid plan for nested sub-selects, such as WHERE (... x IN (SELECT ...) ...) IN (SELECT ...) (Tom Lane)

• Fix mishandling of WHERE conditions pulled up from a LATERAL subquery (Tom Lane)

  The typical symptom of this bug was a "JOIN qualification cannot refer to other relations" error, though subtle logic errors in created plans seem possible as well.

• Disallow LATERAL references to the target table of an UPDATE/DELETE (Tom Lane)

  While this might be allowed in some future release, it was unintentional in 9.3, and didn't work quite right anyway.

• Fix UPDATE/DELETE of an inherited target table that has UNION ALL subqueries (Tom Lane)

  Without this fix, UNION ALL subqueries aren't correctly inserted into the update plans for inheritance child tables after the first one, typically resulting in no update happening for those child table(s).

• Fix ANALYZE to not fail on a column that's a domain over a range type (Tom Lane)

• Ensure that ANALYZE creates statistics for a table column even when all the values in it are "too wide" (Tom Lane)

  ANALYZE intentionally omits very wide values from its histogram and most-common-values calculations, but it neglected to do something sane in the case that all the sampled entries are too wide.

• In ALTER TABLE ... SET TABLESPACE, allow the database's default tablespace to be used without a permissions check (Stephen Frost)

  CREATE TABLE has always allowed such usage, but ALTER TABLE didn't get the memo.

• Fix support for extensions containing event triggers (Tom Lane)

• Fix "cannot accept a set" error when some arms of a CASE return a set and others don't (Tom Lane)

• Fix memory leakage in JSON functions (Craig Ringer)

• Properly distinguish numbers from non-numbers when generating JSON output (Andrew Dunstan)

• Fix checks for all-zero client addresses in pgstat functions (Kevin Grittner)

• Fix possible misclassification of multibyte characters by the text search parser (Tom Lane)

  Non-ASCII characters could be misclassified when using C locale with a multibyte encoding. On Cygwin, non-C locales could fail as well.

• Fix possible misbehavior in plainto_tsquery() (Heikki Linnakangas)

  Use memmove() not memcpy() for copying overlapping memory regions. There have been no field reports of this actually causing trouble, but it's certainly risky.

• Fix placement of permissions checks in pg_start_backup() and pg_stop_backup() (Andres Freund, Magnus Hagander)

  The previous coding might attempt to do catalog access when it shouldn't.

• Accept SHIFT_JIS as an encoding name for locale checking purposes (Tatsuo Ishii)

• Fix *-qualification of named parameters in SQL-language functions (Tom Lane)

  Given a composite-type parameter named foo, $1.* worked fine, but foo.* not so much.

• Fix misbehavior of PQhost() on Windows (Fujii Masao)

  It should return localhost if no host has been specified.

• Improve error handling in libpq and psql for failures during COPY TO STDOUT/FROM STDIN (Tom Lane)

  In particular this fixes an infinite loop that could occur in 9.2 and up if the server connection was lost during COPY FROM STDIN. Variants of that scenario might be possible in older versions, or with other client applications.

• Fix incorrect translation handling in some psql \d commands (Peter Eisentraut, Tom Lane)

• Ensure pg_basebackup's background process is killed when exiting its foreground process (Magnus Hagander)

• Fix possible incorrect printing of filenames in pg_basebackup's verbose mode (Magnus Hagander)

• Avoid including tablespaces inside PGDATA twice in base backups (Dimitri Fontaine, Magnus Hagander)

• Fix misaligned descriptors in ecpg (MauMau)

• In ecpg, handle lack of a hostname in the connection parameters properly (Michael Meskes)

• Fix performance regression in contrib/dblink connection startup (Joe Conway)

  Avoid an unnecessary round trip when client and server encodings match.

• In contrib/isn, fix incorrect calculation of the check digit for ISMN values (Fabien Coelho)

• Fix contrib/pgbench's progress logging to avoid overflow when the scale factor is large (Tatsuo Ishii)

• Fix contrib/pg_stat_statement's handling of CURRENT_DATE and related constructs (Kyotaro Horiguchi)

• Improve lost-connection error handling in contrib/postgres_fdw (Tom Lane)

• Ensure client-code-only installation procedure works as documented (Peter Eisentraut)

• In Mingw and Cygwin builds, install the libpq DLL in the bin directory (Andrew Dunstan)

  This duplicates what the MSVC build has long done. It should fix problems with programs like psql failing to start because they can't find the DLL.

• Avoid using the deprecated dllwrap tool in Cygwin builds (Marco Atzeri)

• Enable building with Visual Studio 2013 (Brar Piening)

• Don't generate plain-text HISTORY and src/test/regress/README files anymore (Tom Lane)

  These text files duplicated the main HTML and PDF documentation formats. The trouble involved in maintaining them greatly outweighs the likely audience for plain-text format. Distribution tarballs will still contain files by these names, but they'll just be stubs directing the reader to consult the main documentation. The plain-text INSTALL file will still be maintained, as there is arguably a use-case for that.

• Update time zone data files to tzdata release 2013i for DST law changes in Jordan and historical changes in Cuba.

  In addition, the zones Asia/Riyadh87, Asia/Riyadh88, and Asia/Riyadh89 have been removed, as they are no longer maintained by IANA, and never represented actual civil timekeeping practice.