Механизм копирования при записи (англ. Copy-On-Write, COW ) используется для оптимизации многих процессов, происходящих в операционной системе, таких как, например, работа с оперативной памятью или файлами на диске (пример — ext3cow).
Идея подхода copy-on-write заключается в том, что при чтении области данных используется общая копия, в случае изменения данных — создается новая копия.
Например, при работе UNIX-функции fork() вместо реального копирования, под которое надо выделять память, ядро меняет дескрипторы страниц памяти материнского процесса, запрещая какую-либо запись в страницы данных (страницы программного кода и так запрещены для записи; хотя здесь имеются свои тонкости — но на дальнейшие рассуждения они не влияют). Затем создаётся дочерний процесс, которому копируются дескрипторы страниц памяти материнского процесса. При этом ядро помечает эти страницы как совместно используемые.
Попытка записи в отображённые страницы (неважно, со стороны материнского или дочернего процесса) вызывает исключение (exception), которое передаёт управление в ядро. Ядро видит, что это обращение было законным, и создаёт копию изменяемой страницы. Таким образом удаётся снизить количество потребляемой программами физической памяти. Механизм COW достаточно сложен в реализации, особенно в многоядерных системах, а ошибки в нём могут приводить к уязвимостям, например, Уязвимость Dirty COW (Linux, 2007-октябрь 2016)
Механизм получил большое распространение при создании новейших файловых систем, таких как ZFS и Btrfs. Благодаря ему создание снимков в данных системах происходит практически мгновенно, не занимая при этом больших ресурсов носителя информации.
Содержание
- Contents
- Intent [ edit ]
- Also Known As [ edit ]
- Motivation [ edit ]
- Solution and Sample Code [ edit ]
- Contents
- In virtual memory management [ edit ]
- In software [ edit ]
- Examples [ edit ]
- In computer storage [ edit ]
- In high-reliability software [ edit ]
Contents
Intent [ edit ]
Achieve lazy copy optimization. Like lazy initialization, do the work just when you need because of efficiency.
Also Known As [ edit ]
- COW (copy-on-write)
- Lazy copy
Motivation [ edit ]
Copying an object can sometimes cause a performance penalty. If objects are frequently copied but infrequently modified later, copy-on-write can provide significant optimization. To implement copy-on-write, a smart pointer to the real content is used to encapsulate the object’s value, and on each modification an object reference count is checked; if the object is referenced more than once, a copy of the content is created before modification.
Solution and Sample Code [ edit ]
This implementation of copy-on-write is generic, but apart from the inconvenience of having to refer to the inner object through smart pointer dereferencing, it suffers from at least one drawback: classes that return references to their internal state, like
can lead to unexpected behaviour. [1]
Consider the following code snippet
The intention of the last line is to modify the original string s1 , not the copy, but as a side effect s2 is also accidentally modified.
A better approach is to write a custom copy-on-write implementation which is encapsulated in the class we want to lazy-copy, transparently to the user. In order to fix the problem above, one can flag objects that have given away references to inner state as «unshareable»—in other words, force copy operations to deep-copy the object. As an optimisation, one can revert the object to «shareable» after any non-const operations that do not give away references to inner state (for example, void string::clear() ), because client code expects such references to be inval >[1]
Copy-on-write (CoW or COW), sometimes referred to as implicit sharing [1] or shadowing, [2] is a resource-management technique used in computer programming to efficiently implement a «duplicate» or «copy» operation on modifiable resources. [3] If a resource is duplicated but not modified, it is not necessary to create a new resource; the resource can be shared between the copy and the original. Modifications must still create a copy, hence the technique: the copy operation is deferred to the first write. By sharing resources in this way, it is possible to significantly reduce the resource consumption of unmodified copies, while adding a small overhead to resource-modifying operations.
Contents
In virtual memory management [ edit ]
Copy-on-write finds its main use in sharing the virtual memory of operating system processes, in the implementation of the fork system call. Typically, the process does not modify any memory and immediately executes a new process, replacing the address space entirely. Thus, it would be wasteful to copy all of the process’s memory during a fork, and instead the copy-on-write technique is used.
Copy-on-write can be implemented efficiently using the page table by marking certain pages of memory as read-only and keeping a count of the number of references to the page. When data is written to these pages, the kernel intercepts the write attempt and allocates a new physical page, initialized with the copy-on-write data, although the allocation can be skipped if there is only one reference. The kernel then updates the page table with the new (writable) page, decrements the number of references, and performs the write. The new allocation ensures that a change in the memory of one process is not visible in another’s.
The copy-on-write technique can be extended to support efficient memory allocation by having a page of physical memory filled with zeros. When the memory is allocated, all the pages returned refer to the page of zeros and are all marked copy-on-write. This way, physical memory is not allocated for the process until data is written, allowing processes to reserve more virtual memory than physical memory and use memory sparsely, at the risk of running out of virtual address space. The combined algorithm is similar to demand paging. [3]
Copy-on-write pages are also used in the Linux kernel’s kernel same-page merging feature. [4]
Loading the libraries for an application is also a use of copy-on-write technique. The dynamic linker maps libraries as private like follows. Any writing action on the libraries will trigger a COW in virtual memory management.
openat(AT_FDCWD, «/lib64/libc.so.6», O_RDONLY|O_CLOEXEC) = 3
mmap(NULL, 3906144, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0)
mmap(0x7f8a3ced4000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b0000)
In software [ edit ]
CoW is also used in library, application and system code.
In multithreaded systems, COW can be implemented without the use of traditional locking and instead use compare-and-swap to increment or decrement the internal reference counter. Since the original resource will never be altered, it can safely be copied by multiple threads (after the reference count was increased) without the need of performance-expensive locking such as mutexes. If the reference counter turns 0, then by definition only 1 thread was holding a reference so the resource can safely be de-allocated from memory, again without the use of performance-expensive locking mechanisms. The benefit of not having to copy the resource (and the resulting performance gain over traditional deep-copying) will therefore be valid in both single- and multithreaded systems.
Examples [ edit ]
The string >[5] but not in the newer C++11 standard: [6]
In the PHP programming language, all types except references are implemented as copy-on-write. For example, strings and arrays are passed by reference, but when modified, they are duplicated if they have non-zero reference counts. This allows them to act as value types without the performance problems of copying on assignment or making them immutable. [7]
In the Qt framework, many types are copy-on-write («implicitly shared» in Qt’s terms). Qt uses atomic compare-and-swap operations to increment or decrement the internal reference counter. Since the copies are cheap, Qt types can often be safely used by multiple threads without the need of locking mechanisms such as mutexes. The benefits of CoW are thus val >[8]
In computer storage [ edit ]
CoW may also be used as the underlying mechanism for snapshots, such as those prov >[9] and database servers such as Microsoft SQL Server. Typically, the snapshots store only the modified data, and are stored close to the main array, so they are only a weak form of incremental backup and cannot substitute for a full backup. [10] Some systems also use a CoW technique to avoid the fuzzy backups, otherwise incurred when any file in the set of files being backed up changes during that backup.
When implementing snapshots, there are two techniques:
- The original storage is never modified. When a write request is made, it is redirected away from the original data into a new storage area. (called «Redirect-on-write» or ROW)
- When a write request is made, the data are copied into a new storage area, and then the original data are modified. (called «Copy-on-write» or CoW)
Despite their names, copy-on-write usually refers to the first technique. CoW does two data writes compared to ROW’s one; it is difficult to implement efficiently and thus used infrequently.
The copy-on-write technique can be used to emulate a read-write storage on media that require wear leveling or are physically write once read many.
The qcow2 (QEMU copy on write) disk image format uses the copy-on-write technique to reduce disk image size.
Some Live CDs (and Live USBs) use copy-on-write techniques to give the impression of being able to add and delete files in any directory, without actually making any changes to the CD (or USB flash drive).
In high-reliability software [ edit ]
Phantom OS uses CoW at all levels, not just a database or file system. At any time, a computer running this system can fail, and then, when it starts again, the software and operating system resume operation. Only small amounts of work can be lost.
The basic approach is that all program data are kept in virtual memory. On some schedule, a summary of all software data is written to virtual memory, forming a log that tracks the current value and location of each value.
When the computer fails, a recent copy of the log and other data remain safe on disk. When operation resumes, operating system software reads the log to restore consistent copies of all the programs and data.
This approach uses copy-on-write at all levels in all software, including in application software. This requires support within the application programming language. In practice, Phantom OS permits only languages that generate Java byte codes.
Источник: