line |
stmt |
bran |
cond |
sub |
pod |
time |
code |
1
|
|
|
|
|
|
|
=head1 NAME |
2
|
|
|
|
|
|
|
|
3
|
|
|
|
|
|
|
Hash::SharedMem - efficient shared mutable hash |
4
|
|
|
|
|
|
|
|
5
|
|
|
|
|
|
|
=head1 SYNOPSIS |
6
|
|
|
|
|
|
|
|
7
|
|
|
|
|
|
|
use Hash::SharedMem qw(shash_referential_handle); |
8
|
|
|
|
|
|
|
|
9
|
|
|
|
|
|
|
if(shash_referential_handle) { ... |
10
|
|
|
|
|
|
|
|
11
|
|
|
|
|
|
|
use Hash::SharedMem qw(is_shash check_shash); |
12
|
|
|
|
|
|
|
|
13
|
|
|
|
|
|
|
if(is_shash($arg)) { ... |
14
|
|
|
|
|
|
|
check_shash($arg); |
15
|
|
|
|
|
|
|
|
16
|
|
|
|
|
|
|
use Hash::SharedMem qw(shash_open); |
17
|
|
|
|
|
|
|
|
18
|
|
|
|
|
|
|
$shash = shash_open($filename, "rwc"); |
19
|
|
|
|
|
|
|
|
20
|
|
|
|
|
|
|
use Hash::SharedMem |
21
|
|
|
|
|
|
|
qw(shash_is_readable shash_is_writable shash_mode); |
22
|
|
|
|
|
|
|
|
23
|
|
|
|
|
|
|
if(shash_is_readable($shash)) { ... |
24
|
|
|
|
|
|
|
if(shash_is_writable($shash)) { ... |
25
|
|
|
|
|
|
|
$mode = shash_mode($shash); |
26
|
|
|
|
|
|
|
|
27
|
|
|
|
|
|
|
use Hash::SharedMem qw( |
28
|
|
|
|
|
|
|
shash_exists shash_length shash_get |
29
|
|
|
|
|
|
|
shash_set shash_gset shash_cset |
30
|
|
|
|
|
|
|
); |
31
|
|
|
|
|
|
|
|
32
|
|
|
|
|
|
|
if(shash_exists($shash, $key)) { ... |
33
|
|
|
|
|
|
|
$length = shash_length($shash, $key); |
34
|
|
|
|
|
|
|
$value = shash_get($shash, $key); |
35
|
|
|
|
|
|
|
shash_set($shash, $key, $newvalue); |
36
|
|
|
|
|
|
|
$oldvalue = shash_gset($shash, $key, $newvalue); |
37
|
|
|
|
|
|
|
if(shash_cset($shash, $key, $chkvalue, $newvalue)) { ... |
38
|
|
|
|
|
|
|
|
39
|
|
|
|
|
|
|
use Hash::SharedMem qw( |
40
|
|
|
|
|
|
|
shash_occupied shash_count shash_size |
41
|
|
|
|
|
|
|
shash_key_min shash_key_max |
42
|
|
|
|
|
|
|
shash_key_ge shash_key_gt shash_key_le shash_key_lt |
43
|
|
|
|
|
|
|
shash_keys_array shash_keys_hash |
44
|
|
|
|
|
|
|
shash_group_get_hash |
45
|
|
|
|
|
|
|
); |
46
|
|
|
|
|
|
|
|
47
|
|
|
|
|
|
|
if(shash_occupied($shash)) { ... |
48
|
|
|
|
|
|
|
$count = shash_count($shash); |
49
|
|
|
|
|
|
|
$size = shash_size($shash); |
50
|
|
|
|
|
|
|
$key = shash_key_min($shash); |
51
|
|
|
|
|
|
|
$key = shash_key_max($shash); |
52
|
|
|
|
|
|
|
$key = shash_key_ge($shash, $key); |
53
|
|
|
|
|
|
|
$key = shash_key_gt($shash, $key); |
54
|
|
|
|
|
|
|
$key = shash_key_le($shash, $key); |
55
|
|
|
|
|
|
|
$key = shash_key_lt($shash, $key); |
56
|
|
|
|
|
|
|
$keys = shash_keys_array($shash); |
57
|
|
|
|
|
|
|
$keys = shash_keys_hash($shash); |
58
|
|
|
|
|
|
|
$group = shash_group_get_hash($shash); |
59
|
|
|
|
|
|
|
|
60
|
|
|
|
|
|
|
use Hash::SharedMem qw(shash_snapshot shash_is_snapshot); |
61
|
|
|
|
|
|
|
|
62
|
|
|
|
|
|
|
$snap_shash = shash_snapshot($shash); |
63
|
|
|
|
|
|
|
if(shash_is_snapshot($shash)) { ... |
64
|
|
|
|
|
|
|
|
65
|
|
|
|
|
|
|
use Hash::SharedMem qw(shash_idle shash_tidy); |
66
|
|
|
|
|
|
|
|
67
|
|
|
|
|
|
|
shash_idle($shash); |
68
|
|
|
|
|
|
|
shash_tidy($shash); |
69
|
|
|
|
|
|
|
|
70
|
|
|
|
|
|
|
use Hash::SharedMem qw( |
71
|
|
|
|
|
|
|
shash_tally_get shash_tally_zero shash_tally_gzero |
72
|
|
|
|
|
|
|
); |
73
|
|
|
|
|
|
|
|
74
|
|
|
|
|
|
|
$tally = shash_tally_get($shash); |
75
|
|
|
|
|
|
|
shash_tally_zero($shash); |
76
|
|
|
|
|
|
|
$tally = shash_tally_gzero($shash); |
77
|
|
|
|
|
|
|
|
78
|
|
|
|
|
|
|
=head1 DESCRIPTION |
79
|
|
|
|
|
|
|
|
80
|
|
|
|
|
|
|
This module provides a facility for efficiently sharing mutable data |
81
|
|
|
|
|
|
|
between processes on one host. Data is organised as a key/value store, |
82
|
|
|
|
|
|
|
resembling a Perl hash. The keys and values are restricted to octet |
83
|
|
|
|
|
|
|
(Latin-1) strings. Structured objects may be stored by serialising them |
84
|
|
|
|
|
|
|
using a mechanism such as L. |
85
|
|
|
|
|
|
|
|
86
|
|
|
|
|
|
|
The data is represented in files that are mapped into each process's |
87
|
|
|
|
|
|
|
memory space, which for interprocess communication amounts to the |
88
|
|
|
|
|
|
|
processes sharing memory. Processes are never blocked waiting for each |
89
|
|
|
|
|
|
|
other. The use of files means that there is some persistence, with the |
90
|
|
|
|
|
|
|
data continuing to exist when there are no processes with it mapped. |
91
|
|
|
|
|
|
|
|
92
|
|
|
|
|
|
|
The data structure is optimised for the case where all the data fits |
93
|
|
|
|
|
|
|
into RAM. This happens either via buffering of a disk-based filesystem, |
94
|
|
|
|
|
|
|
or as the normal operation of a memory-backed filesystem, in either case |
95
|
|
|
|
|
|
|
as long as there isn't much swapping activity. If RAM isn't large enough, |
96
|
|
|
|
|
|
|
such that the data has to reside mainly on disk and parts of it have to |
97
|
|
|
|
|
|
|
be frequently reread from disk, speed will seriously suffer. The data |
98
|
|
|
|
|
|
|
structure exhibits poor locality of reference, and is not designed to |
99
|
|
|
|
|
|
|
play nicely with filesystem block sizes. |
100
|
|
|
|
|
|
|
|
101
|
|
|
|
|
|
|
=head2 Consistency and synchronisation |
102
|
|
|
|
|
|
|
|
103
|
|
|
|
|
|
|
A shared hash is held in regular files, grouped in a directory. At all |
104
|
|
|
|
|
|
|
times, the OS-visible state of the files provides a consistent view of the |
105
|
|
|
|
|
|
|
hash's content, from which read and write operations can proceed. It is |
106
|
|
|
|
|
|
|
no problem for a process using the shared hash to crash; other processes, |
107
|
|
|
|
|
|
|
running concurrently or later, will be unimpeded in using the shared hash. |
108
|
|
|
|
|
|
|
|
109
|
|
|
|
|
|
|
It is mainly intended that the shared hash should be held on a |
110
|
|
|
|
|
|
|
memory-backed filesystem, and will therefore only last as long as the |
111
|
|
|
|
|
|
|
machine stays up. However, it can use any filesystem that supports |
112
|
|
|
|
|
|
|
L, including conventional disk filesystems such as ext2. |
113
|
|
|
|
|
|
|
In this case, as long as the OS shuts down cleanly (synching all file |
114
|
|
|
|
|
|
|
writes to the underlying disk), a consistent state of the shared hash |
115
|
|
|
|
|
|
|
will persist between boots, and usage of the shared hash can continue |
116
|
|
|
|
|
|
|
after the OS boots again. Note that only the OS is required to shut |
117
|
|
|
|
|
|
|
down cleanly; it still doesn't matter if user processes crash. |
118
|
|
|
|
|
|
|
|
119
|
|
|
|
|
|
|
Because the OS is likely to reorder file writes when writing them to disk, |
120
|
|
|
|
|
|
|
the instantaneous state of the shared hash's files on disk is generally |
121
|
|
|
|
|
|
|
I guaranteed to be consistent. So if the OS crashes, upon reboot |
122
|
|
|
|
|
|
|
the files are liable to be in an inconsistent state (corrupted). |
123
|
|
|
|
|
|
|
|
124
|
|
|
|
|
|
|
Maintaining consistency across an OS crash is a feature likely to be |
125
|
|
|
|
|
|
|
added to this module in the future. Durability of writes, for which |
126
|
|
|
|
|
|
|
that is a prerequisite, is another likely future addition. |
127
|
|
|
|
|
|
|
|
128
|
|
|
|
|
|
|
=head2 File permissions |
129
|
|
|
|
|
|
|
|
130
|
|
|
|
|
|
|
To read normally from a shared hash requires read and search (execute) |
131
|
|
|
|
|
|
|
permission on the shared hash directory and read permission on all the |
132
|
|
|
|
|
|
|
regular files in the directory. To write normally requires read, write, |
133
|
|
|
|
|
|
|
and search permissions on the directory and read and write permissions |
134
|
|
|
|
|
|
|
on all the regular files. For security purposes, some information about |
135
|
|
|
|
|
|
|
shared hash content can be gleaned by anyone who has read or search |
136
|
|
|
|
|
|
|
permission on the directory, and content can be modified by anyone who |
137
|
|
|
|
|
|
|
has search permission on the directory and write permission on either |
138
|
|
|
|
|
|
|
the directory or any of the regular files. |
139
|
|
|
|
|
|
|
|
140
|
|
|
|
|
|
|
The file permission bits on a shared hash are determined by the |
141
|
|
|
|
|
|
|
circumstances in which it was created, specifically by the umask in |
142
|
|
|
|
|
|
|
effect at the point of creation. As shared hash creation is unavoidably |
143
|
|
|
|
|
|
|
non-atomic, competing creation attempts can cause trouble, and the |
144
|
|
|
|
|
|
|
resulting permissions are only guaranteed if all competing attempts at |
145
|
|
|
|
|
|
|
creation use the same umask. After the shared hash is fully created, |
146
|
|
|
|
|
|
|
subsequently opening it with the create flag set doesn't affect |
147
|
|
|
|
|
|
|
permissions. |
148
|
|
|
|
|
|
|
|
149
|
|
|
|
|
|
|
The directory gets permissions C modified by the creation |
150
|
|
|
|
|
|
|
umask, and the regular files in it get permissions C modified |
151
|
|
|
|
|
|
|
by the creation umask. All the regular files that contain any part of |
152
|
|
|
|
|
|
|
the shared hash content will get the same permission bits. This includes |
153
|
|
|
|
|
|
|
files created long after initial creation of the shared hash, which are |
154
|
|
|
|
|
|
|
created as part of shared hash write operations; the umask in effect at |
155
|
|
|
|
|
|
|
the point of those operations is insignificant. |
156
|
|
|
|
|
|
|
|
157
|
|
|
|
|
|
|
File ownership and group assignment are not so controlled. An attempt |
158
|
|
|
|
|
|
|
is made to give all files the same group assignment and ownership, |
159
|
|
|
|
|
|
|
determined by the creation of the shared hash, but the ability to do so |
160
|
|
|
|
|
|
|
is limited by OS semantics. Typically, users other than the superuser |
161
|
|
|
|
|
|
|
cannot affect ownership, and can only assign files to groups of which |
162
|
|
|
|
|
|
|
they are members. Also, as with permission bits, competing creation |
163
|
|
|
|
|
|
|
attempts can make ownerships and group assignments inconsistent, even |
164
|
|
|
|
|
|
|
if they are generally controllable. |
165
|
|
|
|
|
|
|
|
166
|
|
|
|
|
|
|
Where they can't be freely set, each regular file gets whatever ownership |
167
|
|
|
|
|
|
|
and group assignment arise naturally from the circumstances in which it |
168
|
|
|
|
|
|
|
was created. If multiple users write to a single shared hash, it is to be |
169
|
|
|
|
|
|
|
expected that the constituent files will end up having different owners. |
170
|
|
|
|
|
|
|
It is up to the user to ensure that the varying ownerships combined |
171
|
|
|
|
|
|
|
with the consistent permission bits yield compatible permissions for all |
172
|
|
|
|
|
|
|
intended users of the shared hash. Group-based permissions should work |
173
|
|
|
|
|
|
|
provided that all writers are members of the relevant group. |
174
|
|
|
|
|
|
|
|
175
|
|
|
|
|
|
|
=head2 Filesystem referential integrity |
176
|
|
|
|
|
|
|
|
177
|
|
|
|
|
|
|
If the system calls L et al are supported by the kernel |
178
|
|
|
|
|
|
|
and the C library, then an open shared hash handle contains an |
179
|
|
|
|
|
|
|
OS-supported first-class reference to the shared hash to which it refers. |
180
|
|
|
|
|
|
|
(Specifically, it has a file descriptor referring to the shared hash |
181
|
|
|
|
|
|
|
directory.) In this situation, the reference remains valid regardless of |
182
|
|
|
|
|
|
|
filename changes. The shared hash can be renamed or moved arbitrarily, |
183
|
|
|
|
|
|
|
within the filesystem, or the process can change its current directory |
184
|
|
|
|
|
|
|
or root directory, and the handle remains valid. |
185
|
|
|
|
|
|
|
|
186
|
|
|
|
|
|
|
If these modern system calls are not available, then an open shared |
187
|
|
|
|
|
|
|
hash handle cannot contain a first-class reference to the shared hash |
188
|
|
|
|
|
|
|
directory. Instead it must repeatedly refer to the directory by name. |
189
|
|
|
|
|
|
|
The name supplied to L is resolved to an absolute pathname, |
190
|
|
|
|
|
|
|
so the handle will continue to work if the process changes its current |
191
|
|
|
|
|
|
|
directory. But any renaming of the shared hash, or the process changing |
192
|
|
|
|
|
|
|
its root directory, will cause the handle to fail at the next operation |
193
|
|
|
|
|
|
|
that requires the use of filenames. (This is unlikely to be the very |
194
|
|
|
|
|
|
|
next operation after the pathname becomes invalid.) An attempt is made to |
195
|
|
|
|
|
|
|
ensure that the stored pathname is still correct each time it is used, but |
196
|
|
|
|
|
|
|
there is unavoidably a race condition, whereby some very unluckily timed |
197
|
|
|
|
|
|
|
renaming could cause an operation to be applied to the wrong directory. |
198
|
|
|
|
|
|
|
|
199
|
|
|
|
|
|
|
The means by which shared hash handles reference their directories is |
200
|
|
|
|
|
|
|
indicated by the constant L. |
201
|
|
|
|
|
|
|
|
202
|
|
|
|
|
|
|
When a shared hash is being opened, if it already exists then the name |
203
|
|
|
|
|
|
|
passed to L is resolved just once to determine to what shared |
204
|
|
|
|
|
|
|
hash it refers. If the modern system calls are supported, this yields |
205
|
|
|
|
|
|
|
perfectly clean name resolution semantics. However, if a shared hash does |
206
|
|
|
|
|
|
|
not already exist, its creation cannot currently be so perfectly clean. |
207
|
|
|
|
|
|
|
The name passed to L must be resolved at least twice, once |
208
|
|
|
|
|
|
|
to create the shared hash directory and once to acquire a reference to it |
209
|
|
|
|
|
|
|
(of whichever type). There is unavoidably a race condition here. |
210
|
|
|
|
|
|
|
|
211
|
|
|
|
|
|
|
=head2 File operations |
212
|
|
|
|
|
|
|
|
213
|
|
|
|
|
|
|
Because a shared hash is encapsulated in a directory, rather than being a |
214
|
|
|
|
|
|
|
single non-directory file, the ability to perform file operations on it is |
215
|
|
|
|
|
|
|
limited. Although it can be renamed or moved, under POSIX semantics such |
216
|
|
|
|
|
|
|
a rename can't atomically replace any file other than an empty directory. |
217
|
|
|
|
|
|
|
In particular, it can't atomically replace another shared hash. It also |
218
|
|
|
|
|
|
|
can't be hard-linked to have multiple names. (However, a major use |
219
|
|
|
|
|
|
|
case for L, non-overwriting renaming, can be achieved through |
220
|
|
|
|
|
|
|
L due to the latter's limitations for directories.) Finally, |
221
|
|
|
|
|
|
|
it can't be unlinked. (Linking and unlinking directories are possible for |
222
|
|
|
|
|
|
|
root on some systems, but cause undesirable filesystem irregularities.) |
223
|
|
|
|
|
|
|
|
224
|
|
|
|
|
|
|
A shared hash can be disposed of by applying C to its directory. |
225
|
|
|
|
|
|
|
This is not equivalent to L (C) on a regular file, |
226
|
|
|
|
|
|
|
because it not only removes the object's name but also disrupts its |
227
|
|
|
|
|
|
|
internal structure. If a process has an open handle referring to the |
228
|
|
|
|
|
|
|
shared hash at the time of C, the use of the shared hash through |
229
|
|
|
|
|
|
|
that handle is likely to fail, although probably not immediately. If a |
230
|
|
|
|
|
|
|
process is writing to the shared hash at the time of C, there is a |
231
|
|
|
|
|
|
|
race condition that could prevent the removal from completing. C |
232
|
|
|
|
|
|
|
should therefore only be applied after all processes have finished using |
233
|
|
|
|
|
|
|
the shared hash. |
234
|
|
|
|
|
|
|
|
235
|
|
|
|
|
|
|
A shared hash can be copied by means of C (not mere C), C, |
236
|
|
|
|
|
|
|
or similar means. It is safe to do this while processes have open handles |
237
|
|
|
|
|
|
|
referring to the shared hash, and while processes are reading from it. |
238
|
|
|
|
|
|
|
However, as with most forms of database file, if a process is writing to |
239
|
|
|
|
|
|
|
the shared hash then the file copier is liable to pick up an inconsistent |
240
|
|
|
|
|
|
|
(corrupted) view of the shared hash. Copying should therefore only |
241
|
|
|
|
|
|
|
be attempted at a time when no write operations are being performed. |
242
|
|
|
|
|
|
|
It is acceptable for processes to have the shared hash open in write |
243
|
|
|
|
|
|
|
mode, provided that they do not actually perform any write operation |
244
|
|
|
|
|
|
|
while the copy is being made. |
245
|
|
|
|
|
|
|
|
246
|
|
|
|
|
|
|
A file-level copying operation applied to a shared hash is likely to |
247
|
|
|
|
|
|
|
result in a copy that occupies much more filesystem space than the |
248
|
|
|
|
|
|
|
original. This occurs because most of the time a large part of the |
249
|
|
|
|
|
|
|
main data file is a filesystem hole, not occupying any actual storage. |
250
|
|
|
|
|
|
|
Some copying mechanisms (such as GNU C) can recognise this and avoid |
251
|
|
|
|
|
|
|
allocating unnecessary storage for the copy, but others (such as GNU |
252
|
|
|
|
|
|
|
C) will blindly fill space with a lot of zeroes. If the copy is |
253
|
|
|
|
|
|
|
subsequently used in shared hash write operations, ultimately it will |
254
|
|
|
|
|
|
|
recover from this inefficient block allocation. |
255
|
|
|
|
|
|
|
|
256
|
|
|
|
|
|
|
=head2 Forking |
257
|
|
|
|
|
|
|
|
258
|
|
|
|
|
|
|
If a process is duplicated by L while it holds a shared hash |
259
|
|
|
|
|
|
|
handle, the handle is duplicated with the rest of the process, so |
260
|
|
|
|
|
|
|
both resulting processes have handles referring to the same underlying |
261
|
|
|
|
|
|
|
shared hash. Provided that the duplication did not happen during a shared |
262
|
|
|
|
|
|
|
hash operation, both processes' handles will subsequently work normally, |
263
|
|
|
|
|
|
|
and can be used independently. |
264
|
|
|
|
|
|
|
|
265
|
|
|
|
|
|
|
Things are more difficult if a L happens while a shared hash |
266
|
|
|
|
|
|
|
operation is in progress. This should not normally be possible to |
267
|
|
|
|
|
|
|
achieve from Perl code: arbitrary Perl code should not run during the |
268
|
|
|
|
|
|
|
critical part of an operation. If a shared hash operator is given a |
269
|
|
|
|
|
|
|
tied variable as a parameter, the magic method call for access to that |
270
|
|
|
|
|
|
|
parameter occurs before the critical part, so a L |
271
|
|
|
|
|
|
|
in that method is safe. If a signal is received during a shared hash |
272
|
|
|
|
|
|
|
operation, any signal handler installed in L<%SIG|perlvar/%SIG> will |
273
|
|
|
|
|
|
|
be deferred until the operation is complete, so a L |
274
|
|
|
|
|
|
|
in such a signal handler is also safe. A problematic L should |
275
|
|
|
|
|
|
|
only be achievable by XS code. |
276
|
|
|
|
|
|
|
|
277
|
|
|
|
|
|
|
If a L does happen during the critical part of a shared hash |
278
|
|
|
|
|
|
|
operation, the two resulting handles are liable to interfere if the |
279
|
|
|
|
|
|
|
operation is resumed in both processes. In this case, it is safe for at |
280
|
|
|
|
|
|
|
most one process (which may be either of them) to resume the operation. |
281
|
|
|
|
|
|
|
The other process must neither resume the operation in progress nor make |
282
|
|
|
|
|
|
|
any further use of the handle. It is safe for the non-resuming process |
283
|
|
|
|
|
|
|
to chain a new program with L, to terminate with L<_exit(2)>, |
284
|
|
|
|
|
|
|
or generally to make use of the C library before doing either of those. |
285
|
|
|
|
|
|
|
Attempting to run Perl code would be unwise. |
286
|
|
|
|
|
|
|
|
287
|
|
|
|
|
|
|
On platforms lacking a native L, the Perl function |
288
|
|
|
|
|
|
|
L actually creates a Perl L. In that |
289
|
|
|
|
|
|
|
case the behaviour should be similar to that seen with a real L, |
290
|
|
|
|
|
|
|
as described in the next section. |
291
|
|
|
|
|
|
|
|
292
|
|
|
|
|
|
|
=head2 Threads |
293
|
|
|
|
|
|
|
|
294
|
|
|
|
|
|
|
This module can be used in multiple Perl L simultaneously. |
295
|
|
|
|
|
|
|
The module may be loaded by multiple threads separately, or from |
296
|
|
|
|
|
|
|
Perl 5.8.9 onwards may be loaded by a thread that spawns new threads. |
297
|
|
|
|
|
|
|
(Prior to Perl 5.8.9, limitations of the threading system mean that |
298
|
|
|
|
|
|
|
module data can't be correctly cloned upon thread spawning. Any but |
299
|
|
|
|
|
|
|
the most trivial cases of thread spawning with this module loaded will |
300
|
|
|
|
|
|
|
crash the interpreter. The rest of this section only applies to Perls |
301
|
|
|
|
|
|
|
that fully support cloning.) |
302
|
|
|
|
|
|
|
|
303
|
|
|
|
|
|
|
If a thread is spawned while the parent thread has an open shared hash |
304
|
|
|
|
|
|
|
handle, the handle is duplicated, so that both resulting threads have |
305
|
|
|
|
|
|
|
handles referring to the same underlying shared hash. Provided that the |
306
|
|
|
|
|
|
|
duplication did not happen during a shared hash operation, both threads' |
307
|
|
|
|
|
|
|
handles will subsequently work normally, and can be used independently. |
308
|
|
|
|
|
|
|
|
309
|
|
|
|
|
|
|
=head2 Tainting |
310
|
|
|
|
|
|
|
|
311
|
|
|
|
|
|
|
If L is enabled, taintedness is relevant |
312
|
|
|
|
|
|
|
to some operations on shared hashes. Shared hash handles mostly behave |
313
|
|
|
|
|
|
|
like regular file handles for tainting purposes. Where the following |
314
|
|
|
|
|
|
|
description says that a result is "not tainted", that means it does |
315
|
|
|
|
|
|
|
not get the taint flag set merely by virtue of the operation performed; |
316
|
|
|
|
|
|
|
it may still be marked as tainted if other tainted data is part of the |
317
|
|
|
|
|
|
|
same expression, due to Perl's conservative taint tracking. |
318
|
|
|
|
|
|
|
|
319
|
|
|
|
|
|
|
The classification functions are happy to operate on tainted arguments. |
320
|
|
|
|
|
|
|
Their results are not tainted. |
321
|
|
|
|
|
|
|
|
322
|
|
|
|
|
|
|
When opening a shared hash, if the shared hash filename or the mode |
323
|
|
|
|
|
|
|
string is tainted then it is not permitted to open for writing or with |
324
|
|
|
|
|
|
|
the possibility of creating. It is permitted to open non-creatingly for |
325
|
|
|
|
|
|
|
reading regardless of taint status. Of course, any kind of opening is |
326
|
|
|
|
|
|
|
permitted in an untainted expression. |
327
|
|
|
|
|
|
|
|
328
|
|
|
|
|
|
|
A shared hash handle per se is never tainted. |
329
|
|
|
|
|
|
|
|
330
|
|
|
|
|
|
|
The results of the mode checking functions are not tainted. |
331
|
|
|
|
|
|
|
|
332
|
|
|
|
|
|
|
The content of a shared hash is always treated as tainted. It is |
333
|
|
|
|
|
|
|
permitted to write tainted data to a shared hash. The data operations |
334
|
|
|
|
|
|
|
all accept tainted arguments. When reading from a shared hash, the keys |
335
|
|
|
|
|
|
|
existing in the hash and the values referenced by them are always tainted, |
336
|
|
|
|
|
|
|
but an absent item is treated as clean. So where a data operation returns |
337
|
|
|
|
|
|
|
a key or value from the shared hash, the result will be tainted if it is |
338
|
|
|
|
|
|
|
a string, but C representing an absent item will not be tainted. |
339
|
|
|
|
|
|
|
The count of keys existing in the hash, size of the hash, and the length |
340
|
|
|
|
|
|
|
of an existing value are also tainted, being derived from tainted content. |
341
|
|
|
|
|
|
|
However, the truth values returned by some operations are not tainted, |
342
|
|
|
|
|
|
|
even if they are derived entirely from tainted data. |
343
|
|
|
|
|
|
|
|
344
|
|
|
|
|
|
|
=cut |
345
|
|
|
|
|
|
|
|
346
|
|
|
|
|
|
|
package Hash::SharedMem; |
347
|
|
|
|
|
|
|
|
348
|
29
|
|
|
29
|
|
967246
|
{ use 5.006; } |
|
29
|
|
|
|
|
87
|
|
|
29
|
|
|
|
|
1097
|
|
349
|
29
|
|
|
29
|
|
133
|
use warnings; |
|
29
|
|
|
|
|
43
|
|
|
29
|
|
|
|
|
748
|
|
350
|
29
|
|
|
29
|
|
137
|
use strict; |
|
29
|
|
|
|
|
130
|
|
|
29
|
|
|
|
|
1284
|
|
351
|
|
|
|
|
|
|
|
352
|
|
|
|
|
|
|
our $VERSION = "0.004"; |
353
|
|
|
|
|
|
|
|
354
|
29
|
|
|
29
|
|
550
|
use parent "Exporter"; |
|
29
|
|
|
|
|
312
|
|
|
29
|
|
|
|
|
193
|
|
355
|
|
|
|
|
|
|
our @EXPORT_OK = qw( |
356
|
|
|
|
|
|
|
shash_referential_handle |
357
|
|
|
|
|
|
|
is_shash check_shash |
358
|
|
|
|
|
|
|
shash_open |
359
|
|
|
|
|
|
|
shash_is_readable shash_is_writable shash_mode |
360
|
|
|
|
|
|
|
shash_exists shash_getd shash_length shash_get |
361
|
|
|
|
|
|
|
shash_set shash_gset shash_cset |
362
|
|
|
|
|
|
|
shash_occupied shash_count shash_size |
363
|
|
|
|
|
|
|
shash_key_min shash_key_max |
364
|
|
|
|
|
|
|
shash_key_ge shash_key_gt shash_key_le shash_key_lt |
365
|
|
|
|
|
|
|
shash_keys_array shash_keys_hash |
366
|
|
|
|
|
|
|
shash_group_get_hash |
367
|
|
|
|
|
|
|
shash_snapshot shash_is_snapshot |
368
|
|
|
|
|
|
|
shash_idle shash_tidy |
369
|
|
|
|
|
|
|
shash_tally_get shash_tally_zero shash_tally_gzero |
370
|
|
|
|
|
|
|
); |
371
|
|
|
|
|
|
|
|
372
|
|
|
|
|
|
|
eval { local $SIG{__DIE__}; |
373
|
|
|
|
|
|
|
require Devel::CallChecker; |
374
|
|
|
|
|
|
|
Devel::CallChecker->VERSION(0.003); |
375
|
|
|
|
|
|
|
}; |
376
|
|
|
|
|
|
|
|
377
|
|
|
|
|
|
|
require XSLoader; |
378
|
|
|
|
|
|
|
XSLoader::load(__PACKAGE__, $VERSION); |
379
|
|
|
|
|
|
|
|
380
|
|
|
|
|
|
|
sub _deparse_shash_unop { |
381
|
32
|
|
|
32
|
|
43
|
my($self, $op, $name) = @_; |
382
|
32
|
|
|
|
|
26
|
my @k; |
383
|
32
|
|
|
|
|
210
|
for(my $k = $op->first; !$k->isa("B::NULL"); $k = $k->sibling) { |
384
|
48
|
|
|
|
|
224
|
push @k, $k; |
385
|
|
|
|
|
|
|
} |
386
|
48
|
|
|
|
|
2988
|
return __PACKAGE__."::".$name. |
387
|
32
|
|
|
|
|
62
|
"(".join(", ", map { $self->deparse($_, 6) } @k).")"; |
388
|
|
|
|
|
|
|
} |
389
|
|
|
|
|
|
|
|
390
|
|
|
|
|
|
|
if("$]" >= 5.013007) { |
391
|
|
|
|
|
|
|
foreach my $name (@EXPORT_OK) { |
392
|
|
|
|
|
|
|
next if $name eq "shash_referential_handle"; |
393
|
29
|
|
|
29
|
|
6791
|
no strict "refs"; |
|
29
|
|
|
|
|
44
|
|
|
29
|
|
|
|
|
4940
|
|
394
|
|
|
|
|
|
|
*{"B::Deparse::pp_$name"} = |
395
|
32
|
|
|
32
|
|
20790
|
sub { _deparse_shash_unop($_[0], $_[1], $name) }; |
396
|
|
|
|
|
|
|
} |
397
|
|
|
|
|
|
|
} |
398
|
|
|
|
|
|
|
|
399
|
|
|
|
|
|
|
=head1 CONSTANTS |
400
|
|
|
|
|
|
|
|
401
|
|
|
|
|
|
|
=over |
402
|
|
|
|
|
|
|
|
403
|
|
|
|
|
|
|
=item shash_referential_handle |
404
|
|
|
|
|
|
|
|
405
|
|
|
|
|
|
|
Truth value indicating whether each shared hash handle contains |
406
|
|
|
|
|
|
|
a first-class reference to the shared hash to which it refers. |
407
|
|
|
|
|
|
|
See L above for discussion of the |
408
|
|
|
|
|
|
|
significance of this. |
409
|
|
|
|
|
|
|
|
410
|
|
|
|
|
|
|
=back |
411
|
|
|
|
|
|
|
|
412
|
|
|
|
|
|
|
=head1 FUNCTIONS |
413
|
|
|
|
|
|
|
|
414
|
|
|
|
|
|
|
The I parameter that most of these functions take must be a handle |
415
|
|
|
|
|
|
|
referring to a shared hash object. |
416
|
|
|
|
|
|
|
|
417
|
|
|
|
|
|
|
=head2 Classification |
418
|
|
|
|
|
|
|
|
419
|
|
|
|
|
|
|
=over |
420
|
|
|
|
|
|
|
|
421
|
|
|
|
|
|
|
=item is_shash(ARG) |
422
|
|
|
|
|
|
|
|
423
|
|
|
|
|
|
|
Returns a truth value indicating whether I is a handle referring |
424
|
|
|
|
|
|
|
to a shared hash object. |
425
|
|
|
|
|
|
|
|
426
|
|
|
|
|
|
|
=item check_shash(ARG) |
427
|
|
|
|
|
|
|
|
428
|
|
|
|
|
|
|
Checks whether I is a handle referring to a shared hash object. |
429
|
|
|
|
|
|
|
Returns normally if it is, or Cs if it is not. |
430
|
|
|
|
|
|
|
|
431
|
|
|
|
|
|
|
=back |
432
|
|
|
|
|
|
|
|
433
|
|
|
|
|
|
|
=head2 Opening |
434
|
|
|
|
|
|
|
|
435
|
|
|
|
|
|
|
=over |
436
|
|
|
|
|
|
|
|
437
|
|
|
|
|
|
|
=item shash_open(FILENAME, MODE) |
438
|
|
|
|
|
|
|
|
439
|
|
|
|
|
|
|
Opens and returns a handle referring to a shared hash object, or Cs |
440
|
|
|
|
|
|
|
if the shared hash can't be opened as specified. I must |
441
|
|
|
|
|
|
|
refer to the directory that encapsulates the shared hash. |
442
|
|
|
|
|
|
|
|
443
|
|
|
|
|
|
|
If the filename string contains non-ASCII characters, then the filename |
444
|
|
|
|
|
|
|
actually used consists of the octets of the internal encoding of the |
445
|
|
|
|
|
|
|
string, which does not necessarily match the ostensible characters of the |
446
|
|
|
|
|
|
|
string. This gives inconsistent behaviour for the same character sequence |
447
|
|
|
|
|
|
|
represented in the two different ways that Perl uses internally. This is |
448
|
|
|
|
|
|
|
consistent with the treatment of filenames in Perl's built-in operators |
449
|
|
|
|
|
|
|
such as L; see L
|
450
|
|
|
|
|
|
|
Not Happen">. This may change in future versions of Perl (beyond 5.20). |
451
|
|
|
|
|
|
|
|
452
|
|
|
|
|
|
|
I is a string controlling how the shared hash will be used. |
453
|
|
|
|
|
|
|
It can contain these characters: |
454
|
|
|
|
|
|
|
|
455
|
|
|
|
|
|
|
=over |
456
|
|
|
|
|
|
|
|
457
|
|
|
|
|
|
|
=item B |
458
|
|
|
|
|
|
|
|
459
|
|
|
|
|
|
|
The shared hash is to be readable. If this is not specified then |
460
|
|
|
|
|
|
|
read operations through the handle will be denied. Beware that at the |
461
|
|
|
|
|
|
|
system-call level the files are necessarily opened readably. Thus read |
462
|
|
|
|
|
|
|
permission on the files is required even if one will only be writing. |
463
|
|
|
|
|
|
|
|
464
|
|
|
|
|
|
|
=item B |
465
|
|
|
|
|
|
|
|
466
|
|
|
|
|
|
|
The shared hash is to be writable. If this is not specified then write |
467
|
|
|
|
|
|
|
operations through the handle will be denied. This flag also determines |
468
|
|
|
|
|
|
|
in what mode the files are opened at the system-call level, so write |
469
|
|
|
|
|
|
|
permission on the files operates as expected. |
470
|
|
|
|
|
|
|
|
471
|
|
|
|
|
|
|
=item B |
472
|
|
|
|
|
|
|
|
473
|
|
|
|
|
|
|
The shared hash will be created if it does not already exist. The |
474
|
|
|
|
|
|
|
permission bits on the shared hash will be controlled by the creating |
475
|
|
|
|
|
|
|
process's umask. If this flag is not specified then the shared hash |
476
|
|
|
|
|
|
|
must already exist. |
477
|
|
|
|
|
|
|
|
478
|
|
|
|
|
|
|
=item B |
479
|
|
|
|
|
|
|
|
480
|
|
|
|
|
|
|
The shared hash must not already exist. If this is not specified and the |
481
|
|
|
|
|
|
|
shared hash already exists then it will be opened normally. This flag |
482
|
|
|
|
|
|
|
is meant to be used with B; it means that a successful open implies |
483
|
|
|
|
|
|
|
that this process, rather than any other, is the one that created the |
484
|
|
|
|
|
|
|
shared hash. |
485
|
|
|
|
|
|
|
|
486
|
|
|
|
|
|
|
=back |
487
|
|
|
|
|
|
|
|
488
|
|
|
|
|
|
|
When a shared hash is created, some of its constituent files will |
489
|
|
|
|
|
|
|
be opened in read/write mode even if read-only mode was requested. |
490
|
|
|
|
|
|
|
Shared hash creation is not an atomic process, so if a creation attempt |
491
|
|
|
|
|
|
|
is interrupted it may leave a half-created shared hash behind. This does |
492
|
|
|
|
|
|
|
not prevent a subsequent creation attempt on the same shared hash from |
493
|
|
|
|
|
|
|
succeeding: creation will continue from whatever stage it had reached. |
494
|
|
|
|
|
|
|
Likewise, multiple simultaneous creation attempts may each do part of the |
495
|
|
|
|
|
|
|
job. This can result in ownerships and permissions being inconsistent; |
496
|
|
|
|
|
|
|
see L above. |
497
|
|
|
|
|
|
|
|
498
|
|
|
|
|
|
|
Regardless of the combination of efforts leading to the creation of a |
499
|
|
|
|
|
|
|
shared hash, completion of the process is atomic. Non-creating open |
500
|
|
|
|
|
|
|
attempts will either report that there is no shared hash or open the |
501
|
|
|
|
|
|
|
created shared hash. Exactly one creation attempt will be judged to have |
502
|
|
|
|
|
|
|
created the shared hash, and this is detectable through the B flag. |
503
|
|
|
|
|
|
|
|
504
|
|
|
|
|
|
|
=back |
505
|
|
|
|
|
|
|
|
506
|
|
|
|
|
|
|
=head2 Mode checking |
507
|
|
|
|
|
|
|
|
508
|
|
|
|
|
|
|
=over |
509
|
|
|
|
|
|
|
|
510
|
|
|
|
|
|
|
=item shash_is_readable(SHASH) |
511
|
|
|
|
|
|
|
|
512
|
|
|
|
|
|
|
Returns a truth value indicating whether the shared hash can be read |
513
|
|
|
|
|
|
|
from through this handle. |
514
|
|
|
|
|
|
|
|
515
|
|
|
|
|
|
|
=item shash_is_writable(SHASH) |
516
|
|
|
|
|
|
|
|
517
|
|
|
|
|
|
|
Returns a truth value indicating whether the shared hash can be written |
518
|
|
|
|
|
|
|
to through this handle. |
519
|
|
|
|
|
|
|
|
520
|
|
|
|
|
|
|
=item shash_mode(SHASH) |
521
|
|
|
|
|
|
|
|
522
|
|
|
|
|
|
|
Returns a string in which characters indicate the mode of this handle. |
523
|
|
|
|
|
|
|
It matches the form of the I parameter to L, but |
524
|
|
|
|
|
|
|
mode flags that are only relevant during the opening process (B |
525
|
|
|
|
|
|
|
and B) are not included. The returned string can therefore contain |
526
|
|
|
|
|
|
|
these characters: |
527
|
|
|
|
|
|
|
|
528
|
|
|
|
|
|
|
=over |
529
|
|
|
|
|
|
|
|
530
|
|
|
|
|
|
|
=item B |
531
|
|
|
|
|
|
|
|
532
|
|
|
|
|
|
|
The shared hash can be read from through this handle. |
533
|
|
|
|
|
|
|
|
534
|
|
|
|
|
|
|
=item B |
535
|
|
|
|
|
|
|
|
536
|
|
|
|
|
|
|
The shared hash can be written to through this handle. |
537
|
|
|
|
|
|
|
|
538
|
|
|
|
|
|
|
=back |
539
|
|
|
|
|
|
|
|
540
|
|
|
|
|
|
|
=back |
541
|
|
|
|
|
|
|
|
542
|
|
|
|
|
|
|
=head2 Single-key data operations |
543
|
|
|
|
|
|
|
|
544
|
|
|
|
|
|
|
For all of these functions, the key of interest (I parameter) |
545
|
|
|
|
|
|
|
can be any octet (Latin-1) string, and values (I parameters and |
546
|
|
|
|
|
|
|
some return values) can be any octet string or C. The C |
547
|
|
|
|
|
|
|
value represents the absence of a key from the hash; there is no |
548
|
|
|
|
|
|
|
present-but-undefined state. Strings containing non-octets (Unicode |
549
|
|
|
|
|
|
|
characters above U+FF) and items other than strings cannot be used as |
550
|
|
|
|
|
|
|
keys or values. If a dualvar (scalar with independent string and numeric |
551
|
|
|
|
|
|
|
values) is supplied, only its string value will be used. |
552
|
|
|
|
|
|
|
|
553
|
|
|
|
|
|
|
=over |
554
|
|
|
|
|
|
|
|
555
|
|
|
|
|
|
|
=item shash_exists(SHASH, KEY) |
556
|
|
|
|
|
|
|
|
557
|
|
|
|
|
|
|
Returns a truth value indicating whether the specified key currently |
558
|
|
|
|
|
|
|
references a defined value in the shared hash. |
559
|
|
|
|
|
|
|
|
560
|
|
|
|
|
|
|
=item shash_getd(SHASH, KEY) |
561
|
|
|
|
|
|
|
|
562
|
|
|
|
|
|
|
Deprecated alias for L. |
563
|
|
|
|
|
|
|
|
564
|
|
|
|
|
|
|
=item shash_length(SHASH, KEY) |
565
|
|
|
|
|
|
|
|
566
|
|
|
|
|
|
|
Returns the length (in octets) of the value currently referenced by the |
567
|
|
|
|
|
|
|
specified key in the shared hash, or C if the key is absent. |
568
|
|
|
|
|
|
|
|
569
|
|
|
|
|
|
|
=item shash_get(SHASH, KEY) |
570
|
|
|
|
|
|
|
|
571
|
|
|
|
|
|
|
Returns the value currently referenced by the specified key in the |
572
|
|
|
|
|
|
|
shared hash. |
573
|
|
|
|
|
|
|
|
574
|
|
|
|
|
|
|
=item shash_set(SHASH, KEY, NEWVALUE) |
575
|
|
|
|
|
|
|
|
576
|
|
|
|
|
|
|
Modifies the shared hash so that the specified key henceforth references |
577
|
|
|
|
|
|
|
the specified value. |
578
|
|
|
|
|
|
|
|
579
|
|
|
|
|
|
|
=item shash_gset(SHASH, KEY, NEWVALUE) |
580
|
|
|
|
|
|
|
|
581
|
|
|
|
|
|
|
Modifies the shared hash so that the specified key henceforth references |
582
|
|
|
|
|
|
|
the value I, and returns the value that the key previously |
583
|
|
|
|
|
|
|
referenced. This swap is performed atomically. |
584
|
|
|
|
|
|
|
|
585
|
|
|
|
|
|
|
=item shash_cset(SHASH, KEY, CHKVALUE, NEWVALUE) |
586
|
|
|
|
|
|
|
|
587
|
|
|
|
|
|
|
Examines the value currently referenced by the specified key in the |
588
|
|
|
|
|
|
|
shared hash. If it is identical to I, the function modifies |
589
|
|
|
|
|
|
|
the shared hash so that the specified key henceforth references the value |
590
|
|
|
|
|
|
|
I, and returns true. If the current value is not identical |
591
|
|
|
|
|
|
|
to I, the function leaves it unmodified and returns false. |
592
|
|
|
|
|
|
|
This conditional modification is performed atomically. |
593
|
|
|
|
|
|
|
|
594
|
|
|
|
|
|
|
This function can be used as a core on which to build arbitrarily |
595
|
|
|
|
|
|
|
complex kinds of atomic operation (on a single key). For example, |
596
|
|
|
|
|
|
|
an atomic increment can be implemented as |
597
|
|
|
|
|
|
|
|
598
|
|
|
|
|
|
|
do { |
599
|
|
|
|
|
|
|
$ov = shash_get($shash, $key); |
600
|
|
|
|
|
|
|
$nv = $ov + 1; |
601
|
|
|
|
|
|
|
} until shash_cset($shash, $key, $ov, $nv); |
602
|
|
|
|
|
|
|
|
603
|
|
|
|
|
|
|
=back |
604
|
|
|
|
|
|
|
|
605
|
|
|
|
|
|
|
=head2 Whole-hash data operations |
606
|
|
|
|
|
|
|
|
607
|
|
|
|
|
|
|
=over |
608
|
|
|
|
|
|
|
|
609
|
|
|
|
|
|
|
=item shash_occupied(SHASH) |
610
|
|
|
|
|
|
|
|
611
|
|
|
|
|
|
|
Returns a truth value indicating whether there are currently any items |
612
|
|
|
|
|
|
|
in the shared hash. |
613
|
|
|
|
|
|
|
|
614
|
|
|
|
|
|
|
=item shash_count(SHASH) |
615
|
|
|
|
|
|
|
|
616
|
|
|
|
|
|
|
Returns the number of items that are currently in the shared hash. |
617
|
|
|
|
|
|
|
|
618
|
|
|
|
|
|
|
=item shash_size(SHASH) |
619
|
|
|
|
|
|
|
|
620
|
|
|
|
|
|
|
Returns the approximate size (in octets) of the entire content of the |
621
|
|
|
|
|
|
|
shared hash. The size of a hash is not a well-defined quantity, so the |
622
|
|
|
|
|
|
|
return value of this function should be interpreted with care. It aims |
623
|
|
|
|
|
|
|
specifically to indicate how much space is required in a shared hash |
624
|
|
|
|
|
|
|
data file to represent this content. It is affected by details of the |
625
|
|
|
|
|
|
|
file format (which may differ between shared hashes on one system) and |
626
|
|
|
|
|
|
|
by accidents of how the content is laid out in a particular shared hash. |
627
|
|
|
|
|
|
|
Calling this function twice on identical content will not necessarily |
628
|
|
|
|
|
|
|
produce identical results. The details of the size estimation may also |
629
|
|
|
|
|
|
|
change in the future. |
630
|
|
|
|
|
|
|
|
631
|
|
|
|
|
|
|
Although this function computes size specifically with respect to the |
632
|
|
|
|
|
|
|
file format used by this module, this function does not directly indicate |
633
|
|
|
|
|
|
|
the amount of space occupied by a shared hash. There is some non-content |
634
|
|
|
|
|
|
|
overhead, and, more importantly, the process by which content is modified |
635
|
|
|
|
|
|
|
requires space to store multiple versions of the content. It is normal |
636
|
|
|
|
|
|
|
for the amount of space actually occupied to fluctuate over the cycle |
637
|
|
|
|
|
|
|
of data file rewriting. If L is being used appropriately, |
638
|
|
|
|
|
|
|
the space occupied can be expected to vary up to a little over five |
639
|
|
|
|
|
|
|
times the size of the nominal content, and if L is not used |
640
|
|
|
|
|
|
|
then the normal maximum will be more than ten times the content size. |
641
|
|
|
|
|
|
|
Occasional spikes above these levels can be expected in any case, and |
642
|
|
|
|
|
|
|
fixed overheads make these multipliers inapplicable if the content is |
643
|
|
|
|
|
|
|
very small. |
644
|
|
|
|
|
|
|
|
645
|
|
|
|
|
|
|
=item shash_key_min(SHASH) |
646
|
|
|
|
|
|
|
|
647
|
|
|
|
|
|
|
Returns the lexicographically least of the keys that are currently in |
648
|
|
|
|
|
|
|
the shared hash, or C if there are none. |
649
|
|
|
|
|
|
|
|
650
|
|
|
|
|
|
|
=item shash_key_max(SHASH) |
651
|
|
|
|
|
|
|
|
652
|
|
|
|
|
|
|
Returns the lexicographically greatest of the keys that are currently |
653
|
|
|
|
|
|
|
in the shared hash, or C if there are none. |
654
|
|
|
|
|
|
|
|
655
|
|
|
|
|
|
|
=item shash_key_ge(SHASH, KEY) |
656
|
|
|
|
|
|
|
|
657
|
|
|
|
|
|
|
Returns the least of the keys currently in the shared hash that are |
658
|
|
|
|
|
|
|
lexicographically no less than the specified key, or C if there |
659
|
|
|
|
|
|
|
are none. |
660
|
|
|
|
|
|
|
|
661
|
|
|
|
|
|
|
=item shash_key_gt(SHASH, KEY) |
662
|
|
|
|
|
|
|
|
663
|
|
|
|
|
|
|
Returns the least of the keys currently in the shared hash that are |
664
|
|
|
|
|
|
|
lexicographically greater than the specified key, or C if there |
665
|
|
|
|
|
|
|
are none. |
666
|
|
|
|
|
|
|
|
667
|
|
|
|
|
|
|
=item shash_key_le(SHASH, KEY) |
668
|
|
|
|
|
|
|
|
669
|
|
|
|
|
|
|
Returns the greatest of the keys currently in the shared hash that |
670
|
|
|
|
|
|
|
are lexicographically no greater than the specified key, or C |
671
|
|
|
|
|
|
|
if there are none. |
672
|
|
|
|
|
|
|
|
673
|
|
|
|
|
|
|
=item shash_key_lt(SHASH, KEY) |
674
|
|
|
|
|
|
|
|
675
|
|
|
|
|
|
|
Returns the greatest of the keys currently in the shared hash that are |
676
|
|
|
|
|
|
|
lexicographically less than the specified key, or C if there |
677
|
|
|
|
|
|
|
are none. |
678
|
|
|
|
|
|
|
|
679
|
|
|
|
|
|
|
=item shash_keys_array(SHASH) |
680
|
|
|
|
|
|
|
|
681
|
|
|
|
|
|
|
Returns a reference to a plain array containing all the keys currently |
682
|
|
|
|
|
|
|
in the shared hash in lexicographical order. The array and the key |
683
|
|
|
|
|
|
|
scalars in it are unwritable. |
684
|
|
|
|
|
|
|
|
685
|
|
|
|
|
|
|
=item shash_keys_hash(SHASH) |
686
|
|
|
|
|
|
|
|
687
|
|
|
|
|
|
|
Returns a reference to a plain hash in which the keys are all the keys |
688
|
|
|
|
|
|
|
currently in the shared hash and all the values are C. The value |
689
|
|
|
|
|
|
|
scalars are unwritable. Writability of the hash is not guaranteed: |
690
|
|
|
|
|
|
|
currently in practice it is writable, but this may change in the future. |
691
|
|
|
|
|
|
|
|
692
|
|
|
|
|
|
|
=item shash_group_get_hash(SHASH) |
693
|
|
|
|
|
|
|
|
694
|
|
|
|
|
|
|
Returns a reference to a plain hash representing the entire current |
695
|
|
|
|
|
|
|
content of the shared hash. The value scalars are unwritable. |
696
|
|
|
|
|
|
|
Writability of the hash is not guaranteed: currently in practice it is |
697
|
|
|
|
|
|
|
writable, but this may change in the future. |
698
|
|
|
|
|
|
|
|
699
|
|
|
|
|
|
|
=back |
700
|
|
|
|
|
|
|
|
701
|
|
|
|
|
|
|
=head2 Snapshots |
702
|
|
|
|
|
|
|
|
703
|
|
|
|
|
|
|
=over |
704
|
|
|
|
|
|
|
|
705
|
|
|
|
|
|
|
=item shash_snapshot(SHASH) |
706
|
|
|
|
|
|
|
|
707
|
|
|
|
|
|
|
Returns a shared hash handle that encapsulates the current content of the |
708
|
|
|
|
|
|
|
shared hash. The entire state of the shared hash is captured atomically, |
709
|
|
|
|
|
|
|
and the returned handle can be used to perform arbitrarily many read |
710
|
|
|
|
|
|
|
operations on that state: it will never reflect later modifications to |
711
|
|
|
|
|
|
|
the shared hash. The snapshot handle cannot be used for writing. |
712
|
|
|
|
|
|
|
|
713
|
|
|
|
|
|
|
=item shash_is_snapshot(SHASH) |
714
|
|
|
|
|
|
|
|
715
|
|
|
|
|
|
|
Returns a truth value indicating whether this handle refers to a snapshot |
716
|
|
|
|
|
|
|
(as opposed to a live shared hash). |
717
|
|
|
|
|
|
|
|
718
|
|
|
|
|
|
|
=back |
719
|
|
|
|
|
|
|
|
720
|
|
|
|
|
|
|
=head2 Maintenance |
721
|
|
|
|
|
|
|
|
722
|
|
|
|
|
|
|
=over |
723
|
|
|
|
|
|
|
|
724
|
|
|
|
|
|
|
=item shash_idle(SHASH) |
725
|
|
|
|
|
|
|
|
726
|
|
|
|
|
|
|
Puts the shared hash handle into a state where it occupies less |
727
|
|
|
|
|
|
|
resources, at the expense of making the next operation through the |
728
|
|
|
|
|
|
|
handle more expensive. This doesn't change the visible behaviour of the |
729
|
|
|
|
|
|
|
handle, and doesn't affect the state of the shared hash itself at all. |
730
|
|
|
|
|
|
|
The invisible operations performed by this function may vary between |
731
|
|
|
|
|
|
|
versions of this module. |
732
|
|
|
|
|
|
|
|
733
|
|
|
|
|
|
|
This function should be called when the handle is going to be unused for |
734
|
|
|
|
|
|
|
a lengthy period. For example, if a long-running daemon uses a shared |
735
|
|
|
|
|
|
|
hash in brief bursts once an hour, it should idle its handle at the end |
736
|
|
|
|
|
|
|
of each burst of activity. |
737
|
|
|
|
|
|
|
|
738
|
|
|
|
|
|
|
Currently the effect of this operation is to discard the handle's |
739
|
|
|
|
|
|
|
memory mapping of the shared hash data file. The next operation has to |
740
|
|
|
|
|
|
|
reestablish the mapping. The benefit of discarding the mapping is that |
741
|
|
|
|
|
|
|
periodically the data file has to be replaced with a new one, but the |
742
|
|
|
|
|
|
|
old data file continues to exist as long as some process has it mapped. |
743
|
|
|
|
|
|
|
A process that is actively using the shared hash will quickly notice that |
744
|
|
|
|
|
|
|
the data file has been replaced and will unmap the old one. A process |
745
|
|
|
|
|
|
|
with a handle that it's not using, however, could keep the old data |
746
|
|
|
|
|
|
|
file in existence, occupying storage, long after it has no further use. |
747
|
|
|
|
|
|
|
A handle that has been put into the idle state won't perpetuate the |
748
|
|
|
|
|
|
|
existence of an obsolete data file. |
749
|
|
|
|
|
|
|
|
750
|
|
|
|
|
|
|
=item shash_tidy(SHASH) |
751
|
|
|
|
|
|
|
|
752
|
|
|
|
|
|
|
Rearranges the storage of the shared hash if it seems useful to do |
753
|
|
|
|
|
|
|
so, to avoid tidying work having to be performed by other processes. |
754
|
|
|
|
|
|
|
This doesn't change the visible content of the shared hash, but the |
755
|
|
|
|
|
|
|
handle must be open for writing, and this counts as a write operation for |
756
|
|
|
|
|
|
|
purposes concerned with the state of the underlying files. The invisible |
757
|
|
|
|
|
|
|
operations performed by this function may vary between versions of |
758
|
|
|
|
|
|
|
this module. |
759
|
|
|
|
|
|
|
|
760
|
|
|
|
|
|
|
This function should be called in circumstances where it is acceptable |
761
|
|
|
|
|
|
|
to incur some delay for this maintenance work to complete. For example, |
762
|
|
|
|
|
|
|
it could be called periodically by a cron job. Essentially, calling this |
763
|
|
|
|
|
|
|
function signals that this is a convenient time at which (and process |
764
|
|
|
|
|
|
|
in which) to perform maintenance. |
765
|
|
|
|
|
|
|
|
766
|
|
|
|
|
|
|
If this maintenance work is not carried out by means of this function, |
767
|
|
|
|
|
|
|
then ultimately it will be performed anyway, but less predictably and |
768
|
|
|
|
|
|
|
possibly less conveniently. Eventually it will become necessary to |
769
|
|
|
|
|
|
|
perform maintenance in order to continue using the shared hash, at which |
770
|
|
|
|
|
|
|
point the next process that attempts to write to it will carry out the |
771
|
|
|
|
|
|
|
work and incur the cost. The shared hash will still work properly in |
772
|
|
|
|
|
|
|
that case, but the unlucky writer will experience a disproportionately |
773
|
|
|
|
|
|
|
large delay in the completion of its write operation. This could well |
774
|
|
|
|
|
|
|
be a problem if the shared hash is large. |
775
|
|
|
|
|
|
|
|
776
|
|
|
|
|
|
|
=back |
777
|
|
|
|
|
|
|
|
778
|
|
|
|
|
|
|
=head2 Event counters |
779
|
|
|
|
|
|
|
|
780
|
|
|
|
|
|
|
=over |
781
|
|
|
|
|
|
|
|
782
|
|
|
|
|
|
|
=item shash_tally_get(SHASH) |
783
|
|
|
|
|
|
|
|
784
|
|
|
|
|
|
|
Returns a reference to a hash of counts of events that have occurred with |
785
|
|
|
|
|
|
|
this shared hash handle. These counts may be of interest for profiling |
786
|
|
|
|
|
|
|
and debugging purposes, but should not be relied upon for semantic |
787
|
|
|
|
|
|
|
purposes. The event types may vary between versions of this module. |
788
|
|
|
|
|
|
|
Few of the event types make sense in terms of the API supplied by this |
789
|
|
|
|
|
|
|
module: most of them are internal implementation details. |
790
|
|
|
|
|
|
|
|
791
|
|
|
|
|
|
|
Events are counted separately for each handle. The events counted are |
792
|
|
|
|
|
|
|
associated specifically with the handle, rather than with the shared hash |
793
|
|
|
|
|
|
|
as a whole. Generally, an idle handle will accumulate no events, even |
794
|
|
|
|
|
|
|
if the shared hash to which it refers is active. The event counters |
795
|
|
|
|
|
|
|
start at zero when a handle is opened, and can be reset to zero by |
796
|
|
|
|
|
|
|
L or L. |
797
|
|
|
|
|
|
|
|
798
|
|
|
|
|
|
|
In the returned hash, each key identifies a type of event, and the |
799
|
|
|
|
|
|
|
corresponding value is (unless wrapped) the number of times events of that |
800
|
|
|
|
|
|
|
type have occurred on the handle since the counters were last zeroed. |
801
|
|
|
|
|
|
|
Currently the event counters are held in fixed-size variables and can |
802
|
|
|
|
|
|
|
wrap, so if event counts might get as high as 2^64 then they can't be |
803
|
|
|
|
|
|
|
relied upon to be accurate. Wrapping will not occur at less than 2^64; |
804
|
|
|
|
|
|
|
in other respects, wrapping behaviour may change in the future. |
805
|
|
|
|
|
|
|
|
806
|
|
|
|
|
|
|
The event types that are currently counted are: |
807
|
|
|
|
|
|
|
|
808
|
|
|
|
|
|
|
=over |
809
|
|
|
|
|
|
|
|
810
|
|
|
|
|
|
|
=item B |
811
|
|
|
|
|
|
|
|
812
|
|
|
|
|
|
|
Parse an octet string representation in a shared hash data file. |
813
|
|
|
|
|
|
|
|
814
|
|
|
|
|
|
|
=item B |
815
|
|
|
|
|
|
|
|
816
|
|
|
|
|
|
|
Write an octet string representation into a shared hash data file. |
817
|
|
|
|
|
|
|
|
818
|
|
|
|
|
|
|
=item B |
819
|
|
|
|
|
|
|
|
820
|
|
|
|
|
|
|
Parse a B-tree node representation in a shared hash data file. |
821
|
|
|
|
|
|
|
|
822
|
|
|
|
|
|
|
=item B |
823
|
|
|
|
|
|
|
|
824
|
|
|
|
|
|
|
Write a B-tree node representation into a shared hash data file. |
825
|
|
|
|
|
|
|
|
826
|
|
|
|
|
|
|
=item B |
827
|
|
|
|
|
|
|
|
828
|
|
|
|
|
|
|
Compare two strings as shared hash keys. |
829
|
|
|
|
|
|
|
|
830
|
|
|
|
|
|
|
=item B |
831
|
|
|
|
|
|
|
|
832
|
|
|
|
|
|
|
Attempt to replace the root pointer in a shared hash data file. This may |
833
|
|
|
|
|
|
|
be done to change the content of the shared hash, or as part of the |
834
|
|
|
|
|
|
|
process of switching to a new data file. |
835
|
|
|
|
|
|
|
|
836
|
|
|
|
|
|
|
=item B |
837
|
|
|
|
|
|
|
|
838
|
|
|
|
|
|
|
Succeed in replacing the root pointer in a shared hash data file. |
839
|
|
|
|
|
|
|
An attempt will fail if another process changed the root pointer during |
840
|
|
|
|
|
|
|
the operation that required this process to change the root pointer. |
841
|
|
|
|
|
|
|
|
842
|
|
|
|
|
|
|
=item B |
843
|
|
|
|
|
|
|
|
844
|
|
|
|
|
|
|
Attempt to replace the data file in a shared hash. This is necessary |
845
|
|
|
|
|
|
|
from time to time as data files fill up. |
846
|
|
|
|
|
|
|
|
847
|
|
|
|
|
|
|
=item B |
848
|
|
|
|
|
|
|
|
849
|
|
|
|
|
|
|
Succeed in replacing the data file in a shared hash. An attempt will |
850
|
|
|
|
|
|
|
fail if another process replaced the data file while this process was |
851
|
|
|
|
|
|
|
initialising its new one. |
852
|
|
|
|
|
|
|
|
853
|
|
|
|
|
|
|
=item B |
854
|
|
|
|
|
|
|
|
855
|
|
|
|
|
|
|
Perform a high-level data operation that purely reads from the shared |
856
|
|
|
|
|
|
|
hash: L, L, L, |
857
|
|
|
|
|
|
|
L, L, L, |
858
|
|
|
|
|
|
|
L, L, |
859
|
|
|
|
|
|
|
L, L, L, L, |
860
|
|
|
|
|
|
|
L, L, |
861
|
|
|
|
|
|
|
or L. |
862
|
|
|
|
|
|
|
|
863
|
|
|
|
|
|
|
=item B |
864
|
|
|
|
|
|
|
|
865
|
|
|
|
|
|
|
Perform a high-level data operation that writes, or at least may write, |
866
|
|
|
|
|
|
|
to the shared hash: L, L, or L. |
867
|
|
|
|
|
|
|
|
868
|
|
|
|
|
|
|
=back |
869
|
|
|
|
|
|
|
|
870
|
|
|
|
|
|
|
The value scalars in the returned hash are unwritable. Writability of |
871
|
|
|
|
|
|
|
the hash is not guaranteed: currently in practice it is writable, but |
872
|
|
|
|
|
|
|
this may change in the future. |
873
|
|
|
|
|
|
|
|
874
|
|
|
|
|
|
|
=item shash_tally_zero(SHASH) |
875
|
|
|
|
|
|
|
|
876
|
|
|
|
|
|
|
Zero the event counters that can be read by L. |
877
|
|
|
|
|
|
|
|
878
|
|
|
|
|
|
|
=item shash_tally_gzero(SHASH) |
879
|
|
|
|
|
|
|
|
880
|
|
|
|
|
|
|
Zero the event counters that can be read by L, and |
881
|
|
|
|
|
|
|
return the values the event counters previously had, in the same form |
882
|
|
|
|
|
|
|
as L. This swap is performed atomically. |
883
|
|
|
|
|
|
|
|
884
|
|
|
|
|
|
|
=back |
885
|
|
|
|
|
|
|
|
886
|
|
|
|
|
|
|
=head1 BUGS |
887
|
|
|
|
|
|
|
|
888
|
|
|
|
|
|
|
As explained for L, creation of a shared hash is not atomic. |
889
|
|
|
|
|
|
|
This is an unavoidable consequence of the need for the shared hash to |
890
|
|
|
|
|
|
|
consist of multiple files in a directory. Multi-party creation can result |
891
|
|
|
|
|
|
|
in the files having different permission bits; to avoid this, all creators |
892
|
|
|
|
|
|
|
should use the same umask. Multiple users writing to a shared hash can |
893
|
|
|
|
|
|
|
result in the files having different ownerships, so the permission bits |
894
|
|
|
|
|
|
|
must be chosen to work appropriately with the chimeric ownership. |
895
|
|
|
|
|
|
|
|
896
|
|
|
|
|
|
|
When calls to the functions supplied by this module are compiled down to |
897
|
|
|
|
|
|
|
custom ops (which is attempted for performance reasons), the ability to |
898
|
|
|
|
|
|
|
deparse the resulting code with L is limited. Prior to Perl |
899
|
|
|
|
|
|
|
5.13.7, deparsing will generate very incorrect code. From Perl 5.13.7 |
900
|
|
|
|
|
|
|
onwards, deparsing should normally work, but will break if another module |
901
|
|
|
|
|
|
|
defines a separate type of custom op that happens to have the same short |
902
|
|
|
|
|
|
|
name (though these ops do not clash in other respects). |
903
|
|
|
|
|
|
|
|
904
|
|
|
|
|
|
|
=head1 SEE ALSO |
905
|
|
|
|
|
|
|
|
906
|
|
|
|
|
|
|
L, |
907
|
|
|
|
|
|
|
L |
908
|
|
|
|
|
|
|
|
909
|
|
|
|
|
|
|
=head1 AUTHOR |
910
|
|
|
|
|
|
|
|
911
|
|
|
|
|
|
|
Andrew Main (Zefram) |
912
|
|
|
|
|
|
|
|
913
|
|
|
|
|
|
|
=head1 COPYRIGHT |
914
|
|
|
|
|
|
|
|
915
|
|
|
|
|
|
|
Copyright (C) 2014, 2015 PhotoBox Ltd |
916
|
|
|
|
|
|
|
|
917
|
|
|
|
|
|
|
Copyright (C) 2014, 2015 Andrew Main (Zefram) |
918
|
|
|
|
|
|
|
|
919
|
|
|
|
|
|
|
=head1 LICENSE |
920
|
|
|
|
|
|
|
|
921
|
|
|
|
|
|
|
This module is free software; you can redistribute it and/or modify it |
922
|
|
|
|
|
|
|
under the same terms as Perl itself. |
923
|
|
|
|
|
|
|
|
924
|
|
|
|
|
|
|
=cut |
925
|
|
|
|
|
|
|
|
926
|
|
|
|
|
|
|
1; |