mirror/openldap
2
0
Fork 0
mirror of https://git.openldap.org/openldap/openldap.git synced 2024-12-27 03:20:22 +08:00
Code Issues Packages Projects Releases Wiki Activity
759 Commits 38 Branches 307 Tags 74 MiB
39ed8bef04
Commit Graph

757 Commits

Author SHA1 Message Date
Howard Chu
2c3488aeeb More for stale sub-cursor flags 
Same fix for cursor_first/last.
 2013-07-14 08:28:26 -07:00
Howard Chu
56a41d87d4 Fix stale sub-cursor C_INIT flag 
Whenever we enter cursor_set() the sub-cursor's flag must be
cleared. If the new cursor position has valid subdata it will
be initialized again, if not then the sub-cursor has nothing
to point to.
 2013-07-14 08:20:18 -07:00
Howard Chu
8ae56c34d5 Tweak comments, defaults should be OK already 2013-07-12 13:55:18 -07:00
Howard Chu
fa7228962b Bump version to 0.9.7 2013-07-12 13:36:05 -07:00
Hallvard Furuseth
f04dc0ebd2 Also set/clear P_KEEP in parent txn's cursors 2013-07-12 13:17:04 -07:00
Howard Chu
e9ed4d75f7 Spill pages, take 3 2013-07-12 13:17:04 -07:00
Howard Chu
d7bc4baf63 Delay touching pages until cursor is positioned. 
This avoids unnecessary rewrites of pages that do not change.
(Restructuring for upcoming mdb_page_spill work.)
 2013-07-12 13:17:04 -07:00
Hallvard Furuseth
c09db5757d Simplify: Always set C_UNTRACK for tracked cursors. 
TODO: Rename C_UNTRACK to C_TRACKED.  Omitted now for readability.
The current name is because it's lazy: not always set when tracked.
 2013-07-12 13:17:04 -07:00
Hallvard Furuseth
6741f9c0ef Save freelist using proper mdb_cursor_put(). 
(Restructuring for upcoming mdb_page_spill work.)
mdb_freelist_save() can't just Get() the destination, since
mdb_page_spill() may have put the destination in the read-only map.

TODO: Can this new put() modify the freelist, which would break it? The
final iteration's put() can shorten the node, the rest uses MDB_CURRENT.

We could set P_KEEP on dirty freeDB leaves and ovpages, since they are
all about to be modified.  But the code in this commit must stay anyway,
if mdb should support dropping a 256G DB. I.e. too big for dirty_list.
 2013-07-12 13:17:04 -07:00
Howard Chu
08373439a6 Move code out to mdb_page_dirty() 2013-07-12 13:17:04 -07:00
Howard Chu
87a7f06feb Factor out parent 2013-07-12 13:17:04 -07:00
Howard Chu
ba6dfe0bbb Fix env_read_header() on Windows 
Commit d6d2638acc broke read
on zero-length files.
 2013-07-12 12:55:08 -07:00
Hallvard Furuseth
3d46d5502a Do not follow uninited cursors' page pointers. 
Nor uninited cursors' subcursors' page pointers.
 2013-07-10 22:11:44 +02:00
Howard Chu
b711c07f34 Fix rebalance/cursor adjust 
When collapsing root, must also move cursor index down,
not just the page pointer.

Also in mtest, break from NEXT loops on error, otherwise it just
prints the previous key/data again, which looks confusing.
 2013-07-10 11:04:37 -07:00
Howard Chu
f81eb631ff Cursors: Clear C_EOF when clearing C_INITIALIZED 2013-07-10 08:49:29 -07:00
Howard Chu
64676da8d9 Fixup other cursors after delete op 2013-07-09 14:21:35 -07:00
Hallvard Furuseth
a77767013a ITS#7515 Reject conflicting page versions. 
If mdb_page_touch() sees a page in txn's dirty_list, that
is the page version txn's cursors should have.  Fail if
the user may be seeing and depending on another version.
 2013-07-07 17:14:38 +02:00
Hallvard Furuseth
be47ca7667 ITS#7515 Fix tracking of parent txn's cursors. 
Restore mc_flags and xcursors, they were tracked but not merged.
Simplify: Track parent txn's original cursors after backing them
up, instead of tracking copies and merging them back at commit.
 2013-07-07 17:13:27 +02:00
Hallvard Furuseth
9be6af0dcb Simplify MDB_cursor: Drop flags C_ALLOCD,C_SHADOW. 2013-07-07 17:13:27 +02:00
Hallvard Furuseth
3d1e709504 Silence more uninit warnings 2013-07-06 21:42:45 +02:00
Howard Chu
06a3ad08cd Silence uninit warning in prev commit 2013-07-02 02:23:49 -07:00
Howard Chu
589370d880 Tweaks for MDB_MULTIPLE 
Terminate loop on intermediate failures, return count of written items,
document usage.
 2013-07-02 02:19:17 -07:00
Howard Chu
fb674c1cf5 ITS#7635 simplify 9474c1a0b6 2013-07-01 14:53:29 -07:00
Howard Chu
9474c1a0b6 ITS#7635 fix read txn potential data race 2013-07-01 13:41:23 -07:00
Howard Chu
3347a02905 Fix uninit warnings, lseek usage 2013-06-30 07:40:02 -07:00
Hallvard Furuseth
7f67383552 Fix alloc/free issues. 
Page leak, mdb_page_alloc(). On error, don't shorten me_pghead.
Memleak, mdb_ovpage_free(). Free page or keep it in dirty_list.
Bad MIDL, mdb_midl_need(). Fix midl[-1] (allocated size).
 2013-06-26 18:02:52 +02:00
Hallvard Furuseth
12c558fe13 Factor out some vars, simplify. 2013-06-26 18:02:48 +02:00
Hallvard Furuseth
b7ce06f5c5 Makefile/user-macro comments. 2013-06-26 18:02:26 +02:00
Hallvard Furuseth
26a25df5fc Tweak I/O, fix last commit. 2013-06-26 18:02:17 +02:00
Hallvard Furuseth
d6d2638acc Improve MDB error handling, drop seek calls. 
Catch I/O errors. Do nothing between OS call failure and ErrCode().
Do not use errno after non-OS-errors like write() >= 0, which could
give a failure return of success (errno 0) or some irrelevant error
code.  Drop seek calls, use pwrite/pread/Windows OVERLAPPED offset.
 2013-06-22 23:15:10 +02:00
Hallvard Furuseth
a5701cf2fe Fix Windows I/O. 
Don't put a 64-bit filesize in a 32-bit int before shifting
down.  Always pass &sizehi to SetFilePointer->maxsize, so
sizelo not is treated a signed distance.  Hide unused vars
when _WIN32.  Reinitialize OVERLAPPED before reuse.
 2013-06-22 23:01:30 +02:00
Hallvard Furuseth
3d4ba01e8b Catch more MDB errors. DPRINTF in mdb_env_reset0. 2013-06-22 22:17:41 +02:00
Hallvard Furuseth
51ff20a4d6 Tweak MIDLs, catch errors. 
Grow midls earlier in order to catch errors earlier.  Use
mdb_midl_need() instead of mdb_midl_grow(), then mdb_midl_xappend()
needs no error checks. Factor out mdb_midl_append_range().
 2013-06-22 22:10:43 +02:00
Hallvard Furuseth
6b200e3beb Factor out MDB variables/expressions, cleanup. 
mdb_page_malloc(): Take a txn arg instead of a cursor.
 2013-06-22 12:30:04 +02:00
Hallvard Furuseth
9e6ef6c88f Rearrange MDB dirty page code. 
Split out mdb_dpage_free(), mdb_page_flush() and clean up.
 2013-06-22 11:56:04 +02:00
Hallvard Furuseth
c6f9323b3d Simplify mdb_page_alloc(). 
Merge if() branches. Restore retry=500 when MDB_PARANOID, for clarity.
 2013-06-20 07:41:35 +02:00
Hallvard Furuseth
4a9ee2cb72 ITS#7620: Keep empty IDLs. Tweak mdb_page_alloc(). 
MDB_env.me_pghead: Don't free it when empty. mdb_ovpage_free()
needs it, but cannot allocate it.
mdb_midl_alloc(): Fill in length=0.
mdb_page_alloc(): Also Skip freeDB if txnid<3, instead of <4,
and consistently DPRINTF consumed IDLs.
 2013-06-20 07:41:35 +02:00
Howard Chu
1d94ea5b55 ITS#7623 Clear P_SUBP on conversion from fake page 2013-06-17 22:26:11 +02:00
Hallvard Furuseth
60086c1bc7 ITS#7515 Nested MDB txns: Inherit txn flags. 
Committing a nested txn lost the MDB_TXN_DIRTY flag
in the parent, unless the child had set it too.
 2013-06-13 08:58:25 +02:00
Hallvard Furuseth
7030ad16e2 Clean up mdb_page_touch(), mdb_page_copy(). 
When copying, round up/down to aligned sizes. Skip the unused portion,
this was not done when touching a page dirty in the parent txn.
No other change in behavior.

Simplify mdb_page_touch(), including: Drop test m3==mc, the condition
is caught below. Don't "modify" the parent's pgno into the same pgno,
when a nested txn copies a parent's page into its freelist.
 2013-06-13 08:58:24 +02:00
Hallvard Furuseth
79844bd446 ITS#7594 Fix MDB cursor tracking with subDBs. 
The tracking code should not change the current cursor.
It did when that was a C_SUB cursor, which should not be
checked against the tracked cursors but their xcursors.

However, do not bother to skip the tracking code for the
current cursor when it would not change that cursor anyway.
 2013-06-13 08:58:24 +02:00
Hallvard Furuseth
2bacf6e59e ITS#7594 Invalidate a dropped MDB DB's cursors. 2013-06-13 08:58:24 +02:00
Hallvard Furuseth
62e4c4f9d0 Don't #define _GNU_SOURCE if already defined. 2013-06-13 08:58:24 +02:00
Hallvard Furuseth
c37a11a424 More for ITS#7620 Fix mdb_ovpage_free(). 
Do not binary-search dirty_list, it is unsorted when MDB_WRITEMAP.
Catch errors. In nested txns, put the page in mt_free_pgs after
all since pages dirty in a parent txn would add complexities.
 2013-06-13 08:25:25 +02:00
Howard Chu
2d0b362b6f Partial revert c2cac4588a 
MDB_NEXT was fine before, duh.
 2013-06-12 08:41:32 -07:00
Hallvard Furuseth
99427aa7de Drop me_pgfree, add mdb_freelist_save(). 
Split up saving me_pghead, to make me_pgfree unneeded. Also mf_pghead
is now a midl. Needed after e7f6767ea8
("Return fresh overflow pages to current pghead").
Tweak MDB_DEBUG freelist output, make it ascending.
 2013-06-12 17:20:42 +02:00
Howard Chu
c2cac4588a Fix CURSOR_NEXT/PREV on emptied DB 2013-06-11 17:13:08 -07:00
Howard Chu
bcb67dd22f Make sure mdb_stat() gets valid data 2013-06-08 14:10:08 -07:00
Howard Chu
e7f6767ea8 Return fresh overflow pages to current pghead 
And remove them from the current dirty list.
 2013-06-06 16:44:02 -07:00
Howard Chu
aff123ba11 ITS#7594 more for subDB cursor fix 2013-06-05 16:13:43 -07:00
Howard Chu
3b623d66e1 ITS#7594 better fix 
Update the subDB cursor, don't invalidate it
 2013-06-05 15:23:54 -07:00
Howard Chu
65faa5ed7e tweak mdb_copy, trap signals 2013-05-30 15:56:30 -07:00
Howard Chu
d29b9600e6 Windows portability fixes for prev commit 2013-05-30 15:33:59 -07:00
Howard Chu
f207c50b15 Add warning about interrupting copy 2013-05-30 13:13:33 -07:00
Howard Chu
ad573fe125 Fix prev commit 2013-05-30 13:09:28 -07:00
Howard Chu
c68e5ae9be Add mdb_env_copyfd() 
Allow writing backup to an already opened file handle, for piping
to tar/gzip/ssh/whatever.
 2013-05-30 13:06:12 -07:00
Howard Chu
4b49291653 Add _M_IX86 macro for MSVC 2013-05-25 10:16:55 -07:00
Howard Chu
e31c7d3b31 ITS#7594 De-init other subcursors in page_touch 2013-05-23 12:17:21 -07:00
Hallvard Furuseth
92fe958805 Drop unused liblmdb MIDL-range support. 2013-05-21 23:58:57 +02:00
Hallvard Furuseth
feaeab0c04 Factor out mdb_find_oldest,mdb_dlist_free,dirty_list. 
Do not rescan reader table (mdb_find_oldest) after "goto again".
Skip clearing dirty_list[nonzero].mid in mdb_dlist_free(); it
was not done in mdb_reset0() anyway.
 2013-05-21 23:55:13 +02:00
Hallvard Furuseth
5ea1cd8f6d mdb_stat cleanup. 
Exit with success when there was no failure.
Do not use data containing NUL as a DB name (which is a C string).
 2013-05-21 23:48:27 +02:00
Hallvard Furuseth
a1b16ce5f0 ITS#7598 Tweak MDB_<NEXT/PREV>_NODUP,fix mdb_stat. 
MDB_NEXT_NODUP, MDB_PREV_NODUP: Allow for non-MDB_DUPSORT databases.
No mdb.c code changes needed.

mdb_stat.c: Use MDB_NEXT_NODUP, to avoid a crash with a DUPSORT mainDB.
 2013-05-21 22:44:51 +02:00
Hallvard Furuseth
1b6d7ee7e1 ITS#7598 mdb_dbi_open(named DB): Check mainDB flags. 
Reject attempts to open named databases if the main
database has flag MDB_DUPSORT or MDB_INTEGERKEY.

DUPSORT would require an xcursor for the DB, INTEGERKEY
would expect the DB name to be a binary integer.
 2013-05-21 19:04:52 +02:00
Hallvard Furuseth
a2ce25482a ITS#7515 Fix nested txn touch of subpage/ovpage. 
mdb_page_touch(): Don't touch a subpage, replacing with non-subpage.
mdb_cursor_put(): Don't overwrite ancestor txn's dirty overflow page.
 2013-05-20 23:12:52 +02:00
Hallvard Furuseth
5bdf2aae6e Tweak mdb_page_malloc(),mdb_page_get() semantics. 
mdb_page_malloc(): Add "number of pages" parameter.
mdb_page_get(): Add output param for how page was found.
Do not set return params on error.
mdb_cursor_put(): Catch mdb_page_get() error.

Prepares for next commit, no change in caller behavior other
than on mdb_page_get error.
 2013-05-20 23:12:26 +02:00
Hallvard Furuseth
2eb50b1d2e More ITS#7589 followup: OVPAGES() -> mp_pages. 2013-05-19 19:08:07 +02:00
Hallvard Furuseth
e4ce404992 More ITS#7589: Fix prev commit. 
mp_pages gives #pages in the ovpage. OVPAGES() no longer does.
 2013-05-19 18:38:13 +02:00
Hallvard Furuseth
0cdd9dffdd ITS#7589 mdb_cursor_put(): Update ovpage nodesize. 
Update the nodesize when overwriting an overflow page.
As before, do not attempt to shrink the page.
 2013-05-17 19:31:17 +02:00
Howard Chu
833cd905b7 Include <netinet/in.h> for Solaris 8 
Needed for <resolv.h>, which is needed for BYTE_ORDER.
 2013-05-15 11:50:40 -07:00
Howard Chu
7233bc295b Fix mdb_page_split - nested split 
If updating a page's separator triggers a split in its parent,
we may have missed adjusting the parent's cursor position.
 2013-05-14 16:57:50 -07:00
Howard Chu
1105aa35ba Doxygen cleanup 2013-05-13 12:36:47 -07:00
Howard Chu
9ebd5992c9 Fix 66c839f029 
Forgot #include <errno.h>
 2013-05-10 05:56:16 -07:00
Howard Chu
66c839f029 Refactor mdb_midl_append, add mdb_midl_grow() 2013-05-09 17:27:29 -07:00
Hallvard Furuseth
2d6aed7537 ITS#7515 Fix nested transaction error handling. 
mdb_txn_begin(): Do not free(mt_free_pgs), it needs mdb_midl_free().
mdb_txn_commit(): Catch commit(child) error.
 2013-05-05 15:13:31 +02:00
Hallvard Furuseth
521fdb00cc Fix mdb_env_close(unopened MDB_env). 
Do not try to scan me_dbxs in a closed/never-opened MDB_env.
Broken by 7d643d3acb
and 151c416b46.
 2013-05-05 14:26:05 +02:00
Hallvard Furuseth
0b9a208530 Simplify/cleanup mdb_cursor_init, C_ALLOCD. 
No real change.
mdb_cursor_init() checks if it needs mx, so pass it unconditionally.
Set C_ALLOCD for shadow cursors, for clarity. (It was always set as
it should anyway from the origin cursor, which would have C_ALLOCD.)
 2013-05-04 12:53:43 +02:00
Hallvard Furuseth
f355de0298 MDB warning cleanup. 
Unused function when MDB_DEBUG. Unused 'excl' param.
 2013-05-04 12:41:28 +02:00
Hallvard Furuseth
b389341b4b mdb_dcmp(): Assume the database has MDB_DUPSORT. 
There was little point in returning EINVAL when not: Comparing (A,B)
and (B,A) would claim (A > B && B > A), which could confuse callers.
 2013-05-04 12:34:49 +02:00
Hallvard Furuseth
91a93004ce Update MDB doc: Cursors, DB handles, data lifetime 2013-05-04 12:34:20 +02:00
Hallvard Furuseth
385889b0be Allow mdb_cursor_close() after readonly txn ends. 
Catch mdb_cursor_renew(write txn's cursor).  Add flag C_UNTRACK, so
mdb_cursor_close need not peek inside a possibly-freed readonly txn.
 2013-05-04 12:34:02 +02:00
Hallvard Furuseth
17ffe9c992 Close cursors when commit(writer). Factor out code. 
Close remaining cursors when committing a write txn. The doc says this
happens, and it avoids cursor tracking when updating mainDB + freeDB.

Rename mdb_cursor_merge() -> mdb_cursors_close() for code reuse,
and add a merge option.  Simplify its loop a bit.

Factor out cleanup of DBIs.
 2013-05-04 12:33:06 +02:00
Hallvard Furuseth
dca0cef678 Plug txn leak and MDB_NOTLS slot leaks on error. 
On mdb_env_copy() error: Abort the txn.
On mdb_txn_renew0() error: Release new MDB_NOTLS reader slot.
 2013-05-04 12:31:31 +02:00
Hallvard Furuseth
f45d40a88b Cleaner "Support mdb_txn_abort(a reset txn)". 
Redo 8a562f560e so !mt_dbxs indicates
"txn was reset", so mt_numdbs gets one magic value instead of two.
 2013-05-04 10:03:19 +02:00
Hallvard Furuseth
151c416b46 Update fixes for dbi_open/close, ITS#7515. 
Reset me_dbflags[dbi] when closing DBI, to get rid of MDB_VALID flag.
mdb_env_close(): Re-fix DB-name memleak. DBIs > me_numdbs may exist.
 2013-05-04 10:03:19 +02:00
Howard Chu
21da623bf4 Allow reading freelist while working on it 
The circular dependency issues appear to have been resolved.
Still, need to watch closely, maybe revert this change if
problems arise.
 2013-05-01 22:45:15 -07:00
Howard Chu
568c22c4cc Avoid assert 
Due to underfilled branch page. We're in the process of merging/moving
nodes to it because we already know it's underfilled. Took this approach
rather than just removing the assert in mdb_page_search_root, because
that assert may yet catch other situations we don't know about.

(Although, it has been there since the original commit of mdb.c and
has never triggered any other times...)
 2013-04-30 21:09:09 -07:00
Hallvard Furuseth
37bd48a618 Plug mdb_cursor_renew() memleak 2013-04-30 21:06:36 -07:00
Howard Chu
c057582573 Fix MDB_LAST, reset cursor index 2013-04-20 19:30:06 -07:00
Howard Chu
da3e4d78d6 Fix typo from 7aba5f5ab9 2013-04-20 09:47:33 -07:00
Hallvard Furuseth
9dd61011fe Tweak comments/readability. 
Show MDB_PERSISTENT/MDB_VALID/DB_VALID relationship.
mdb_txn_renew0(): Remove obsolete "cannot fail" comment.
 2013-04-18 08:00:03 +02:00
Howard Chu
272e4e98ad Add MDB_NOTLS envflag. 2013-04-18 04:17:03 +02:00
Hallvard Furuseth
afe488d8a9 Catch MDB txn reuse/sync errors. 2013-04-18 04:16:07 +02:00
Hallvard Furuseth
8a562f560e Support mdb_txn_abort(a reset txn). 2013-04-18 04:15:45 +02:00
Hallvard Furuseth
ce6335b0d9 Cleanup for TLS key and read-only filesystem. 
Move key init into mdb_env_setup_locks().
Don't create unused TLS key when read-only filesystem.
Drop internal flag MDB_ROFS, we can instead test either
!me_txns, !mt_u.reader or me_lfd==INVALID_HANDLE_VALUE.
 2013-04-18 04:15:13 +02:00
Hallvard Furuseth
65a6542765 Clean up MDB_env setup. 
Malloc before I/O. Avoids possible malloc error after I/O.
Don't allocate dirty & free lists when MDB_RDONLY.
Factor out code.
 2013-04-18 04:13:43 +02:00
Howard Chu
dbb9ded2f8 Fix mdb_rebalance 
Don't do anything with (fake root) subpages
 2013-04-17 14:33:41 -07:00
Howard Chu
aca6c08897 ITS#7574 blind fix 
Fix 227329c8e1, don't persist
the MDB_VALID bit in db.md_flags.
 2013-04-16 12:05:03 -07:00
Howard Chu
92fc932f21 fix prev commit 2013-04-15 09:30:39 -07:00
Howard Chu
0cccf79a02 Fix mdb_rebalance 
Need to check NUMKEYS as well as fill threshold, when deciding
whether to change anything. Don't let the page drop below the
minimum number of keys.
 2013-04-14 23:59:54 -07:00
Howard Chu
14c5a5dff1 Revert "ITS#7570 fix MDB_GET_MULTIPLE and MDB_NEXT_MULTIPLE" 
This reverts commit 372b3ddb85.

Original code works as designed.
 2013-04-14 16:53:30 -07:00
Claude Brisson
372b3ddb85 ITS#7570 fix MDB_GET_MULTIPLE and MDB_NEXT_MULTIPLE 2013-04-11 15:10:23 -07:00
Howard Chu
fbe6a6051b Fix 227329c8e1 
mdb_dbi_close() must allow any dbi < maxdbs, since opens in an
uncommitted txn don't increment env->me_numdbs.
 2013-04-07 19:21:46 -07:00
Howard Chu
e43fa77eaa Revert "Fix 6beaad52129da5353fd40c0ec48e6a78c4f71a2e" 
This reverts commit 8eef7a4275.
Previous commit was correct, duplicate keys should not appear here.
 2013-04-05 03:44:56 -07:00
Howard Chu
227329c8e1 Fixes for dbi_open/close, ITS#7515 2013-04-05 02:58:08 -07:00
Howard Chu
6b46799379 ITS#7561 Fix mdb_drop bugs 
Check for overflow pages, reset cursor position properly.
 2013-04-04 16:52:55 -07:00
Howard Chu
8eef7a4275 Fix 6beaad5212 
In APPEND don't immediately reject matching key, since this
is valid for APPENDDUP.
 2013-04-04 07:25:21 -07:00
Howard Chu
6beaad5212 Fix d3990eb2f8 
Append mode should *reject* keys that are too small.
Also allow APPENDDUP in mdb_put().
 2013-04-03 18:26:41 -07:00
Howard Chu
e31d748dbe Fix rebalance when collapsing root page 
When fixing other cursors, must also fix their depth.
 2013-04-01 20:34:40 -07:00
Howard Chu
cb1cc64831 More fixes for MDB_LAST 
Make sure C_INITIALIZED gets set on successful call
 2013-04-01 19:49:40 -07:00
Howard Chu
3213fc4db7 ITS#7556 fix cursor_sibling for PREV 2013-04-01 19:06:05 -07:00
Hallvard Furuseth
7d643d3acb Plug MDB memory leak of DB names. 2013-03-31 23:50:02 +02:00
Hallvard Furuseth
ccb7b26916 Silence signedness warning 2013-03-31 23:48:10 +02:00
Hallvard Furuseth
a70b026f6b Fix MDB_DEBUG compile when no varargs macros. 
Variables mdb_debug, mdb_debug_start were used undefined.
 2013-03-31 23:47:45 +02:00
Howard Chu
30da15a18d Tweak mode_t def per Hallvard's suggestion 2013-03-31 20:56:08 +01:00
Howard Chu
0c568117ce MSVC doesn't define mode_t 2013-03-31 08:10:43 -08:00
Howard Chu
7aae15739b Fix CURSOR_LAST/CURSOR_PREV 2013-03-29 01:08:06 -07:00
Howard Chu
863fa77a08 ITS#7553 better fix 2013-03-26 11:24:59 -07:00
Howard Chu
8a99b00acc ITS#7553 deinit cursor after deleting last node 2013-03-25 12:15:53 -07:00
Howard Chu
bccbcd1938 Silence stupid warnings 2013-03-25 11:53:32 -07:00
Claude Brisson
bbb27cde4a bugfix for mdb_cursor_put with MDB_MULTIPLE 
If the variable dkey.mv_size is non-zero, then it means dkey
contains some original data which has to be put back in the
child db, typically when the child db has just been created.

But when using MDB_MULTIPLE, if this variable has not been
reset to zero, we may come back to this section and wrongly
think that there is some original data to be taken care of.
 2013-03-23 14:03:12 -07:00
Howard Chu
77001f549b ITS#7538 fallout from ITS#7536 fix. 
Allow leaf pages to have only 1 key.
 2013-03-12 17:29:34 -07:00
Howard Chu
3247d7cbd6 Precalculate max size of a node 2013-03-12 17:25:49 -07:00
Howard Chu
8c1d95a438 Fix f97552a83a 
Causes Bus Error on SPARC
 2013-03-11 10:28:25 -07:00
Howard Chu
f80171e079 Check for MADV_RANDOM 
Android supports madvise but not posix_madvise
 2013-03-11 06:04:54 -07:00
Howard Chu
9c4ee82297 ITS#7377 Fix c0f3d9b9a8 
Add missing parens
 2013-03-11 05:59:07 -07:00
Howard Chu
b1da555c4c Turn off readahead on main mmap 
It's harmful when the DB is larger than RAM.
 2013-03-11 05:49:14 -07:00
Howard Chu
d74330128a ITS#7536 fix mdb_rebalance 
A page must always have at least 2 keys (unless it's a root page)
 2013-03-06 15:10:22 -08:00
Howard Chu
6dab37f2ac Use CC/LDFLAGS to link the shared lib 2013-02-25 05:06:51 -08:00
Howard Chu
d8b407d9a2 Install manpages too 2013-02-25 02:02:15 -08:00
Howard Chu
26e4222ffb Delete obsolete @todo 
We do it now... (coalescing freelist entries in page_alloc)
 2013-02-24 23:43:36 -08:00
Howard Chu
58e27b6c20 Add sample apps for comparison/education purposes 2013-02-24 23:43:18 -08:00
Howard Chu
66c9f03055 Add $(DESTDIR) to make install target 2013-02-25 06:50:51 +00:00
Howard Chu
3394bac2c0 Update error code instances 2013-02-20 05:08:52 -08:00
Hallvard Furuseth
80cd881184 Tweak MDB_INCOMPATIBLE description 2013-02-20 12:19:45 +01:00
Hallvard Furuseth
fbd76c44e4 Tweak prev commit: Restore if-test as an assert 2013-02-20 09:08:41 +01:00
Hallvard Furuseth
208e5c614d ITS#7515 Fix mdb_txn_commit(nested txn). 
Don't modify the parent txn until the current txn cannot fail.
Don't assume new dirty child pgnos > dirty parent pgnos.

Page alloc/touch: Fail if child+parent dirty pages would exceed
dirty_list's maxsize. Avoids an error situation in commit.
 2013-02-19 18:44:10 -08:00
Hallvard Furuseth
f43ae20be7 ITS#7512 Plug mdb_txn_abort(nested txn) page leaks. 
Also catch mdb_cursor_shadow() errors.
 2013-02-19 22:03:04 +01:00
Hallvard Furuseth
f97552a83a Check DB flags when refreshing a stale MDB DBI. 
It's hairy to figure out when a DBI is valid. Catch destructive
user errors, and flags which another process changed under us.
 2013-02-19 22:02:37 +01:00
Hallvard Furuseth
2dbb8bb833 mdb_cursor_prev,mdb_cursor_next: Fix return value. 
Return mdb_node_read()'s return value if it fails, not 1.
(Can happen if mdb_page_get() fails and NDEBUG is #defined.)
 2013-02-19 22:02:15 +01:00
Hallvard Furuseth
e4af9ee5da ITS#7515 mdb_dbi_open(): Also open in parent txns. 
This makes aborting nested and non-nested txns more
similar: The new DBI is available to the surrounding
context (parent txn and MDB_env respectively).
 2013-02-19 22:01:29 +01:00
Hallvard Furuseth
4b67270374 mdb_page_alloc(): Handle freeDB txnid range holes. 
A txn writes no freeDB entry if previous txn dropped mainDB and a read
txn prevents freelist entry reuse. This surprised mdb_page_alloc (and
mdb_txn_commit too before 65c053a6e7).
 2013-02-19 21:17:33 +01:00
Hallvard Furuseth
f19655eabc ITS#7517 Document that dirty DBs may not be closed 2013-02-19 21:15:26 +01:00
Hallvard Furuseth
8ad25001ff ITS#7485 Document key/data size limits in lmdb.h. 
mdb.c already describes them. The user doc should too.
 2013-02-19 21:14:23 +01:00
Hallvard Furuseth
ef25056cfb Revert "ITS#7515 update parent's mt_next_pgno on child commit" 
This reverts commit fd4861bf00.
It duplicated earlier code.
 2013-02-17 08:42:14 +01:00
Howard Chu
fd4861bf00 ITS#7515 update parent's mt_next_pgno on child commit 2013-02-17 00:49:53 +00:00
Howard Chu
890f1da3ee Don't limit retries when coalescing freelist 
Try to use whatever's available.
 2013-02-17 00:48:43 +00:00
Hallvard Furuseth
00d7a96bd5 ITS#7515 Fix MDB parent/child txn interaction. 
mdb_txn_commit(child): Copy more state. Copy all of mt_dbs:
Include mainDB, and even freeDB since mdb_drop() can update
it. Don't skip DBs with unchanged root, this could break
when the new was newly opened and the old unused junk.

mdb_page_get(): Search parents' dirty lists.
 2013-02-16 19:11:20 +01:00
Hallvard Furuseth
d90581fa5a ITS#7377 Catch MDB failure updating root pointers. 
"cannot fail" was wrong, it fails at least when exceeding mapsize.
 2013-02-16 19:08:54 +01:00
Hallvard Furuseth
c7db955a94 ITS#7517 Don't save dropped dirty MDB databases. 
mdb_txn_commit's attempt to save the DB could corrupt another DB if
another thread had called mdb_dbi_open and reused the closed DBI.
 2013-02-16 19:08:37 +01:00
Hallvard Furuseth
8e1bbdf0dd mdb_stat -ff[f]: show contiguous page spans. 2013-02-16 19:07:16 +01:00
Hallvard Furuseth
7aba5f5ab9 Add error code MDB_MAP_RESIZED. 2013-02-16 19:06:28 +01:00
Howard Chu
5e59695b8d Don't memmove freelist entry when chunking it 
Just advance the pointer, now that it's no longer a complex struct
 2013-02-15 13:27:50 +00:00
Hallvard Furuseth
52ecd38e18 ITS#7455 Save freelist in single-page chunks 2013-02-15 13:27:50 +00:00
Hallvard Furuseth
1b8bfc5756 Freelist cleanup/streamlining 
Drop unneeded definitions, redundant code.
 2013-02-15 13:27:02 +00:00
Hallvard Furuseth
65c053a6e7 ITS#7512 Fix MDB page leak when malloc error. 
mdb_page_alloc(): Delay moving me_pgfirst,me_pglast
until malloc(MDB_oldpages to hold the IDs) succeeds.
 2013-02-07 08:17:30 +01:00
Howard Chu
e0eb495fb3 Revert "ITS#7515 fix mdb_dbi_open/close" 
This reverts commit 48dc782ea6.
 2013-02-04 10:53:17 -08:00
Howard Chu
744e311939 NULL key is allowed for MDB_CURRENT 2013-02-01 06:19:23 -08:00
Howard Chu
48dc782ea6 ITS#7515 fix mdb_dbi_open/close 
If a DBI handle is opened by a txn that aborts, the DBI handle
should no longer be valid.
 2013-01-31 20:05:48 -08:00
Howard Chu
27aaecc744 ITS#7511 make sure cursor is marked valid after successful put 2013-01-28 18:45:36 -08:00
Howard Chu
9fa0c3bf31 ITS#7505 fix prev commit, partial revert 2013-01-29 01:08:24 +00:00
Howard Chu
0b8ac92b7a ITS#7505 Fix mdb_update_key when key is too big 2013-01-27 18:02:18 +00:00
Howard Chu
cfedb365b4 Fix MDB_CURRENT updating for dupsort items 2013-01-26 18:55:35 +00:00
Hallvard Furuseth
7fdf672041 Update MDB comments: Caveats, flags, etc. 2013-01-16 18:42:57 +01:00
Howard Chu
8e7bb2042b ITS#7485 data sizes limited to 32 bits 
That's all we have space for in a node record.
 2013-01-15 04:28:47 -08:00
Howard Chu
1a0d02c4c4 ITS#7485 more size checks 2013-01-14 13:46:25 -08:00
Howard Chu
25a99d4024 ITS#7485 check maxkeysize in mdb_cursor_put 
Also MDB_MAXKEYSIZE is redefinable at compile time.
 2013-01-14 13:12:15 -08:00
Howard Chu
9e35127f1d One more rename 
Forgot this in 98fe339089
 2013-01-14 12:57:54 -08:00
Howard Chu
53cf2eed90 ITS#7491 check for filled dirty page list 
Very large single transactions will fail. It's not just a problem when
nested transactions are used. We could make this dynamically sized,
but I'm not sure what the point is.
 2013-01-11 11:45:25 -08:00
Howard Chu
fed573cb86 Happy New Year 2013-01-10 02:58:55 -08:00
Hallvard Furuseth
cd80a15474 Delete liblmdb/mfree. It has moved into mdb_stat. 2013-01-09 20:49:43 +01:00
Howard Chu
2a36346b50 ITS#7455 don't give up so soon 
If we're operating on the freelist, see if our current pghead
can satisfy the request before giving up and using new pages.
 2013-01-09 08:30:13 -08:00
Howard Chu
057e068630 Fix typo 2012-12-19 13:20:24 -08:00
Howard Chu
460a05bf78 Add initial mdb utility manpages 2012-12-19 13:09:56 -08:00
Howard Chu
82b97a8af0 init_meta should write mapsize 
Was leaving it 0 initially.
 2012-12-11 22:21:52 -08:00
Howard Chu
0702fbd688 Update version date 2012-12-11 16:01:25 -08:00
Howard Chu
9acf801cc1 Make sure init_meta is at beginning of file 2012-12-11 12:55:47 -08:00
Howard Chu
ed86a086f7 More for prev commit 
Use the latest meta page
 2012-12-11 12:46:17 -08:00
Howard Chu
e74d70e88e Partially revert 65d40eb5d2 
Allow both increasing and decreasing the environment size.
But don't allow decreasing below the currently occupied space.
 2012-12-11 12:03:19 -08:00
Howard Chu
282be11654 FIx prev commit 2012-12-10 17:21:24 -08:00
Howard Chu
0ce6bb4be0 Add error checks to txn_commit 
These ops could fail if the map is full.
 2012-12-10 12:21:36 -08:00
Howard Chu
aff2693fc0 ITS#7455 simplify 
Don't try to reclaim overflow pages while operating on
the freelist (for now). The circular dependencies are much like
the single-page case, but worse. Maybe look into this in the
future, but it's not absolutely necessary now.
 2012-12-10 12:16:50 -08:00
Howard Chu
e26fc73417 ITS#7467 add verbose freelist info 2012-12-07 13:19:36 -08:00
Howard Chu
784757faa9 ITS#7465 fail if both -a and -s specified 2012-12-07 12:06:30 -08:00
Howard Chu
9a5d4370be Yet more retries for overflow page scan 2012-12-06 17:39:49 -08:00
Howard Chu
5863d5cc61 Freelist fixes 
Keep list sorted if it grows during a write. Don't free pghead until
we're sure our writes are all finished.
 2012-12-06 09:05:30 -08:00
Howard Chu
7b10fdde00 Fix mdb_env_copy 
Just write in 2^31 sized chunks
 2012-12-04 06:30:22 -08:00
Howard Chu
04b7c42109 Rename mdb_open/close mdb_dbi_open/close 
Avoid possible symbol clashes with MDB-Tools libmdb
 2012-12-03 16:53:24 -08:00
Howard Chu
e545b742e5 Fix 5c1ee7f7ba 
mdb_cursor_sibling() no longer pops cursor before returning.
 2012-12-03 13:57:46 -08:00
Howard Chu
5af47adf6b Fix mdb_midl_shrink 2012-12-03 13:57:28 -08:00
Howard Chu
d3f51fdcf7 More retries for overflow page scan 2012-11-30 12:54:19 -08:00
Howard Chu
98fe339089 Rename libmdb to liblmdb 2012-11-30 12:45:34 -08:00
Howard Chu
2a94ebced6 Add install target 2012-11-30 12:39:53 -08:00
Howard Chu
13f3bcd59c Rename libmdb to liblmdb, lmdb.h 
Avoid naming conflicts with other mdb* packages. Bump version to 0.9.5
 2012-11-30 12:30:51 -08:00
Howard Chu
f165507199 ITS#7455 use freelist for overflow pages 2012-11-30 12:18:27 -08:00
Howard Chu
185136397d Drop unneeded #include <time.h> 2012-11-28 15:34:44 -08:00
Howard Chu
049b145ec7 Doc layout tweaks 2012-11-28 15:34:19 -08:00
Hallvard Furuseth
97ed96206d mdb_env_sync(,force=1): Override MDB_MAPASYNC. 
With MDB_MAPASYNC, the API provided no way to ensure full sync.
 2012-11-28 00:59:00 +01:00
Hallvard Furuseth
b9275c9991 mdb_cursor_dbi(): Assume valid args. 
It returned 0, which is the freelist DBI and is valid for a few uses.
 2012-11-28 00:59:00 +01:00
Hallvard Furuseth
7e620a34a2 API change: mdb_drop() only accepts del=0 or 1. 
This allows for other flag values in the future.
 2012-11-28 00:59:00 +01:00
Hallvard Furuseth
d903bed218 API change: mdb_env_set_maxdbs(, *named* DBs). 
Let the user specify the number of databases he will actually
use. Adding 2 for FREE_DBI and MAIN_DBI is an internal matter.
 2012-11-28 00:59:00 +01:00
Hallvard Furuseth
6b56e0fab1 ITS#7453 Fix reset of MDB_NOSYNC/MDB_NOMETASYNC. 
mdb_env_open() with these flags did not open a synchronizing
file descriptor.  Thus disabling them later did not work.
 2012-11-28 00:59:00 +01:00
Hallvard Furuseth
0201b9002c Tweak MDB_FIXEDMAP handling. 
Drop mmap()'s MAP_FIXED flag, so it returns another address instead
of unmapping existing overlapping pages. Verify the returned address.
 2012-11-28 00:59:00 +01:00
Hallvard Furuseth
52e3adbdec Renumber MDB_NOSUBDIR. Check mdb_env_open() flags. 
MDB_NOSUBDIR was == MDB_REVERSEKEY. That affected the freelist:
Env flags are stored in mm_flags alias mm_dbs[FREE_DBI].md_flags.

It stays a persistent flag, in case mdb_env_open someday wants
to pick/verify which lockfile to use with the datafile.

Catch bad flags so they will no longer make it into the data file.
 2012-11-28 00:59:00 +01:00
Howard Chu
bb36bdcd1c ITS#7432 fix typo in mdb_midl_sort 
Wasn't pushing the optimal half of the array onto the stack,
thus used more stack than expected -> overrun.
 2012-11-05 05:06:06 -08:00
Howard Chu
0108327c27 Better fix for MDB_DEBUG tweak -- from hbf 2012-10-25 07:03:25 -07:00
Howard Chu
b68aa3f86b Revert "MDB_DEBUG tweak for old compilers" 
This reverts commit ef2092eac9.
 2012-10-25 06:54:09 -07:00
Howard Chu
ef2092eac9 MDB_DEBUG tweak for old compilers 2012-10-25 05:28:55 -07:00
Howard Chu
c2f15990c7 Fix flag checks in mdb_open 2012-10-23 04:48:04 -07:00
Howard Chu
f5b8d017df fix Windows mdb_env_copy(), WriteFile overflow 2012-10-23 04:32:55 -07:00
Howard Chu
66f2b7b2ec Make sure mdb_open flags are committed on main DB 
If no data was modified in the txn, mdb_open flag changes were dropped.
 2012-10-22 17:03:15 -07:00
Howard Chu
5f2ddb5532 Minor cleanup 2012-10-22 16:13:42 -07:00
Howard Chu
5c1ee7f7ba Fix MDB_PREV 
Could return 1 garbage record before actual data, if starting from EOF
 2012-10-22 16:02:30 -07:00
Howard Chu
617769bce5 Tweak output, add free page info 2012-10-16 17:01:16 -07:00
Howard Chu
754cd88819 Add mdb_env_info() to retrieve interesting env params 2012-10-16 16:22:21 -07:00
Howard Chu
7f0771a3ed Merge mdb_stata into mdb_stat 2012-10-16 15:37:29 -07:00
Howard Chu
65d40eb5d2 Make sure increases of mapsize are persisted 2012-10-16 14:53:25 -07:00
Hallvard Furuseth
94965307c8 ITS#7377 Errorcheck Windows calls and thread keys. 2012-10-15 17:04:48 +02:00
Hallvard Furuseth
88b7bb4639 Explicitly unlock the MDB lockfile on Windows. 2012-10-15 17:04:43 +02:00
Hallvard Furuseth
e9d87866c4 mdb_env_open(): Keep mdb_env_set_flags() flags. 2012-10-15 17:04:11 +02:00
Hallvard Furuseth
092f9a5c66 More for mdb_env_copy(). 
Fix #define _GNU_SOURCE = 1 to match any command line -D_GNU_SOURCE.
.gitignore += mdb_copy.
 2012-10-15 17:03:46 +02:00
Hallvard Furuseth
462d9dfd10 Fix last commit. 
Add flag MDB_ENV_ACTIVE instead of using the state of me_lfd.
Require MDB_RDONLY for MDB_ROFS.  Swap reader/writer mutex lock.
 2012-10-15 17:03:33 +02:00
Howard Chu
e40713b631 Support read access on read-only media 2012-10-13 04:05:00 -07:00
Howard Chu
f037dd2d19 Silence warning in prev commit 2012-10-11 16:47:56 -07:00
Howard Chu
ebb6859ea5 Add mdb_copy for backing up a DB environment 2012-10-11 12:24:01 -07:00
Hallvard Furuseth
756ce8e10c Tweak MDB restrictions 2012-10-03 21:13:21 +02:00
Hallvard Furuseth
a3f33dd20e Fix mdb_open() off-by-one error in maxdbs check 2012-10-03 18:08:56 +02:00
Hallvard Furuseth
38fb8e6eb1 ITS#7377 Wrap fcntl F_SETLK in EINTR loop too. 
This can happen even on local filesystems.
 2012-10-03 18:06:29 +02:00
Hallvard Furuseth
ab3fea51dc ITS#7377 Catch ftruncate() error 2012-10-03 18:06:29 +02:00
Hallvard Furuseth
2b960613e0 ITS#7377 Document caveats/troubeshooting. 2012-10-03 18:04:07 +02:00
Howard Chu
7333b6bdc9 Fix prev commit 
It only worked if the freelist was already non-empty
 2012-09-30 10:01:57 -07:00
Howard Chu
cd228b4722 Fix mdb_drop 
if it was the only operation in a txn, it would be ignored.
 2012-09-30 07:49:30 -07:00
Howard Chu
0c4c6fe72a ITS#7385, check further for bad splits 
More for 48ef27b6f5
Our definition of "large" data item needs to be smaller to
handle more cases where a bad split is possible
 2012-09-27 05:06:23 -07:00
Howard Chu
ba6c3281a1 Add MDB_GET_CURRENT 
return whatever the cursor is currently pointing at
 2012-09-19 06:15:09 -07:00
Howard Chu
d13ca0de4f ITS#7394 ignore WRITEMAP if RDONLY 2012-09-18 12:36:54 -07:00
Howard Chu
0a359fb629 More for 48ef27b6f5 
page_split with newindex > split_indx
 2012-09-17 07:02:41 -07:00
Howard Chu
5ef56b437e More for ab04c50a32 
Use explicit MDB_RDONLY flag
 2012-09-17 07:01:28 -07:00
Hallvard Furuseth
c67ea9c060 ITS#7377 Catch MDB user errors. 2012-09-17 15:42:15 +02:00
Hallvard Furuseth
c0f3d9b9a8 ITS#7377 Catch MDB setup errors and clean up. 2012-09-17 15:42:15 +02:00
Hallvard Furuseth
fe1b3794de ITS#7364 Limit mdb lock upgrade before sem_unlink. 
Do not try shared lock when closing or after error.  Track file lock
state to decide.  Change meaning of *excl to reflect file lock state.
 2012-09-17 15:42:15 +02:00
Hallvard Furuseth
c760e536ec ITS#7364 Always sem_unlink() in mdb_env_open(). 
Drop the sem_unlink() error checks, which could prevent the 2nd
unlink.  Instead use O_EXCL in sem_open().

This makes "open+close the database" the API for trying to clean
away the old semaphores, if they were left behind by a previous run.
 2012-09-17 15:42:15 +02:00
Hallvard Furuseth
31be24896b ITS#7377 Wrap sem_wait & file locks in EINTR loops 2012-09-17 15:42:14 +02:00
Hallvard Furuseth
a35f9b2a53 Remove mdb data races. Use (txnid_t)-1 as "no ID". 
Avoid race between numreaders++ and reading numreaders at cleanup. Make
the un-mutexed reset of reader table entry, atomic: Reset mr_pid only.

Instead check mr_pid != 0 in mdb_page_alloc()'s scan for readers.
(txnid_t)-1 as "no ID"-mark avoids a check for mr_txnid != 0.
The scan can stop when seeing an old reader.
 2012-09-17 15:42:14 +02:00
Hallvard Furuseth
38cc1e96b4 Save pid in MDB_env instead of repeating getpid(). 
An open MDB environment does not survive or catch fork(),
so repeating getpid() was pointless.
 2012-09-17 15:42:14 +02:00
Hallvard Furuseth
20a216fcc9 MDB_WRITEMAP needs no DSYNC descriptor (me_mfd) 2012-09-17 15:42:14 +02:00
Hallvard Furuseth
588a84a5ae ITS#7363 Preprocessor namespace cleanup. 
Rename USE_POSIX_SEM to MDB_USE_POSIX_SEM.
Separate MDB_FDATASYNC from MDB_USE_POSIX_SEM.
 2012-09-17 15:42:14 +02:00
Howard Chu
8bb10add24 More for ab04c50a32 
Fix typos, error code ranges
 2012-09-17 06:35:03 -07:00
Howard Chu
acbff5b1ea Add mdb_cursor_renew() 
Allow cursors on read-only txns to be reused with later txns.
 2012-09-17 04:41:13 -07:00
Howard Chu
076b2b36a9 Shared lib should depend on pthread 2012-09-17 03:48:54 -07:00
Howard Chu
c542442c9f Add MDB_SET_KEY cursor op 
Overwrites the passed in key with the DB's key
 2012-09-17 02:17:25 -07:00
Howard Chu
ab04c50a32 ITS#7383 More explicit error codes 2012-09-14 06:51:31 -07:00