commit 8c7a1630516749a7eaa6aa4fc8f117962858dcea Author: David G. Johnston Date: Wed Oct 21 20:07:35 2020 +0000 Review of the Collaboration of Processes section diff --git a/doc/src/sgml/architecture.sgml b/doc/src/sgml/architecture.sgml index 2be9898d98..3fa896cefb 100644 --- a/doc/src/sgml/architecture.sgml +++ b/doc/src/sgml/architecture.sgml @@ -1,11 +1,11 @@ - Architectural and implementational Cornerstones + Architecture and Implementation - Every DBMS implements basic strategies for a fast and - robust system. This chapter provides an overview of what + Every DBMS implements basic strategies to ensure a fast + and robust system. This chapter provides an overview of the techniques PostgreSQL uses to achieve this. @@ -14,26 +14,35 @@ Collaboration of Processes, RAM, and Files In a client/server architecture - clients do not have direct access to the database. Instead, + clients do not have direct access to stored data. Instead, they send requests to the server and receive - the requested information. In the case of - PostgreSQL, at the server-side - there is one process per client, the so-called + the requested data in response. In the case of + PostgreSQL, the server launches a + single process for each connected client, referred to as a Backend process. + It acts in close cooperation with the instance which is a group of tightly coupled server-side processes plus a Shared Memory - area. + area located in RAM. + Notably, PostgreSQL does not utilize application threading within its + implementation. + + - At startup time, an instance is initiated by the - Postmaster. - The Postmaster process loads the + During instance startup time, the + Postmaster + process loads the configuration files, allocates Shared Memory, - and starts a network of processes: + and starts supporting background processes: Background Writer, Checkpointer, WAL Writer, @@ -62,9 +71,11 @@ - Whenever a client application tries to connect to a + + When a client application tries to connect to a database, - this request is handled in a first step by the + this request is handled initially by the Postmaster process. It checks authorization, starts a new Backend process, and instructs the client application to connect to it. All @@ -75,123 +86,64 @@ Client requests like SELECT or UPDATE usually lead to the - necessity to read or write some data. In a first attempt - the client's Backend process tries - to get the information out of Shared - Memory. This Shared - Memory is a mirror of parts of the - heap and - index files. - Because files are often larger than memory, it's likely that - the desired information is not (completely) available - in RAM. In this case the Backend process - must transfer additional file pages to - Shared Memory. Files are physically - organized in pages. Every transfer between files and - RAM is performed in units of complete pages; such transfers - do not change the size or layout of pages. - - - - Reading file pages is much slower than reading - RAM. This is the primary motivation for the usage of - Shared Memory. As soon as one - of the Backend processes has - read pages into memory, those pages become available for all - other Backend processes for direct - access in RAM. + necessity to read or write some data. + Reads involve a page-level cache housed in Shared Memory + for the benefit of all processes in the instance. + + Writes also involve this cache, in additional to a journal, + called a write-ahead-log or WAL, in PostgreSQL. Shared Memory is limited in size. - Sooner or later, it becomes necessary to overwrite old RAM - pages. As long as the content of such pages hasn't + thus it becomes necessary to evict pages. + As long as the content of such pages hasn't changed, this is not a problem. But in Shared Memory also write - actions take place - — performed by any of the Backend - processes (or an - autovacuum process, - or other processes). Such modified pages are called - dirty pages. - Before dirty pages can be overwritten, - they must be written back to disk. This is a two-step process. - - - - First, whenever the content of a page changes, a - WAL record - is created out - of the delta-information (difference between the old and - the new content) and stored in another area of - Shared Memory. These - WAL records are read by the - WAL Writer process, - which runs in parallel to the Backend - processes and other processes of - the Instance. It writes - the continuously arising WAL records to - the end of the current - WAL file. - Because this writing is sequential, it is much - faster than the more or less random access - to data files with heap - and index information. - As mentioned, this WAL-writing happens - in an independent process. All - WAL records created out of one - dirty page must be transferred - to disk before the dirty page - itself can be transferred to disk. - - - - Second, the transfer of dirty buffers - from Shared Memory to file must - take place. This is the primary task of the - Background Writer process. Because - I/O activities can block other processes significantly, - it starts periodically and acts only for a short period. - Doing so, its expensive I/O activities are spread over - time, avoiding debilitating I/O peaks. Also, the - Checkpointer process transfers - dirty buffers to file — - see next paragraph. - - - - The Checkpointer creates - Checkpoints. - A Checkpoint - is a point in time when all older dirty buffers, - all older WAL records, and - finally a special Checkpoint record - have been written and flushed to disk. - After a Checkpoint, we say - data files and WAL files are in sync. - In case of a recovery (after a crash of the instance) - it can be relied upon that the information of all - WAL records preceding - the last Checkpoint record - were already integrated into the data files. This - speeds up the recovery. - - - - As a result of data changes, - WAL records arise and get written - to WAL files. - Those WAL files — in combination with - a previously taken Base Backup — - are necessary to restore a database after a crash of the - disk on which data files have been stored. Therefore it is - recommended to transfer a copy of the - WAL files - to a second, independent place. The purpose of the - WAL Archiver process is to perform - this copy action. + actions take place. + Modified pages are called + dirty pages or + dirty buffers and + before dirty pages can be evicted + they must be written back to disk. This also happens regularly + by the Background Writer + process to ensure that the disk version of + the page is kept up-to-date. Writes are only performed against + the pages in Shared Memory. + + + + + + + The Background Writer process spreads + its expensive I/O activity over time while coordinating with + the overall system through the Checkpointer process - which + places checkpoint records into the WAL noting instances in + time when all dirty pages in Shared Memory corresponding to + previously written WAL have been written to file. At these + checkpoints previous WAL is no longer required so long as + the data files on hand are current. In other words, recovery + happens by replaying WAL from the last recorded checkpoint + on top of the current data files. + + + + While the Checkpointer ensures that a running system can crash + and restart itself in a valid state the administration needs to + handle the case where the data files themselves become corrupted + (and possibly the locally written WAL, though that is less common.) + The options and details are covered extensively in the backup + and restore section of the documentation. + + For our purposes here note just that the + WAL Archiver process can be enabled and configured + to run a script on a filled WAL file - usually to copy it to a remote location. + The Statistics Collector collects counters about accesses to SQL objects @@ -202,7 +154,7 @@ The Logger writes text lines about serious and less serious events which can happen - during database access, e.g. wrong password, no permission, + during database access, e.g., wrong password, no permission, long-running queries, etc.