mirror/openldap
2
0
Fork 0
mirror of https://git.openldap.org/openldap/openldap.git synced 2024-12-27 03:20:22 +08:00
Code Issues Packages Projects Releases Wiki Activity
534 Commits 38 Branches 307 Tags 74 MiB
7f67383552
Commit Graph

532 Commits

Author SHA1 Message Date
Howard Chu
02abfb91ba Additional fix for prev commit 2011-09-18 18:26:54 -07:00
Howard Chu
4272e7de4f Make sure 2 subpages fit in one real page 2011-09-18 18:13:23 -07:00
Howard Chu
237fabda64 Fix last delete from a subpage 2011-09-18 17:07:47 -07:00
Howard Chu
0ab841598f Fix 09006ccec7 
Check for stale DBs was in the wrong place.
 2011-09-18 16:39:18 -07:00
Howard Chu
2c70d0430a Fix mdb_drop() if not deleting the DB record 2011-09-18 14:22:27 -07:00
Howard Chu
77b1017b85 Fix typo, minor cleanup 2011-09-18 14:12:49 -07:00
Howard Chu
57b5fbea49 Add mdb_drop() for deleting a DB 
Allow MIDLs to grow arbitrarily, to allow unlimited number of
pages to be freed in one txn.
 2011-09-18 12:37:02 -07:00
Howard Chu
09006ccec7 API change, consistency check 
mdb_close() takes an env, not a txn.
Also, add check for stale DB records (in case some other process
wrote to the DB last).
 2011-09-18 10:48:12 -07:00
Howard Chu
b7057eb61e New sorted-dup subpage support 
Instead of converting directly to a subDB when the first duplicate
item is seen for a key, convert to a subpage instead. Allow the
subpage to grow up to the overflow limit, then convert to a subDB.
This saves a significant amount of space in a typical slapd index
database.

Currently we don't convert back to the smaller form if items are
later deleted. Probably could do that with some hysteresis, e.g.,
convert back from subDB to subpage when the size drops below
(overflow limit/2). Maybe later.
 2011-09-17 03:33:07 -07:00
Howard Chu
0a28548131 Use FNV-1A hash to generate unique mutex names 
For Windows and MacOSX
 2011-09-15 10:36:15 -07:00
Howard Chu
2c033676ea Doc tweak - txn flags 2011-09-14 20:33:09 -07:00
Howard Chu
5b7c983a66 MacOSX semaphore names are limited to 31 chars 
Including the terminating NUL. Good job documenting that, guys.
 2011-09-14 20:30:26 -07:00
Howard Chu
0533f80364 Add MacOSX support 
mmap() with FIXEDMAP fails, otherwise things work.
 2011-09-14 11:31:27 -07:00
Howard Chu
e222506b88 Minor doc cleanup 2011-09-14 01:46:08 -07:00
Howard Chu
9e3e7cefd8 More docs 2011-09-14 01:08:01 -07:00
Howard Chu
71a665aa9d Namespace cleanup 2011-09-13 23:36:41 -07:00
Howard Chu
56fe0d4f1a Tweak search_node inner loop to avoid LEAF2 checks 2011-09-13 13:42:02 -07:00
Howard Chu
70c79df1df cursor_set/search_node tweaks, use intcmp when possible 2011-09-13 13:41:40 -07:00
Howard Chu
dc83777ab9 Minor tweaks, update relfunc behavior 2011-09-12 01:15:22 -07:00
Howard Chu
25529a4c36 More cursor_init cleanup 2011-09-11 23:57:17 -07:00
Howard Chu
8acb755034 One more sub-cursor fix 2011-09-11 23:13:36 -07:00
Howard Chu
92a63ff790 Merge branch 'mdb.master' of ssh://git-master.openldap.org/~git/git/openldap into mdb.master 2011-09-11 22:23:35 -07:00
Howard Chu
3501248548 Fix 2e3bc39fa9 
Various mistakes when converting from previous data structures.
Add a few more debug asserts/sanity checks.
Split all "if (foo) return" constructs to separate lines to allow
easier breakpoint setting.

Add mtest6 for checking mdb_split() behavior. This needs to be
expanded to check rebalance/merge cases too.
 2011-09-11 22:17:10 -07:00
Hallvard Furuseth
946b38f471 Fix mdb debug formats 2011-09-11 19:29:24 +02:00
Hallvard Furuseth
ed0fc55fd3 Restore missing LAZY_LOCKS semicolon. 
Lost in C90 compat commit c5dad7a6d0.
 2011-09-11 19:29:24 +02:00
Hallvard Furuseth
90ed553301 mdb integer type cleanup: Use MDB_dbi consistently. 
This changes the prototype of mdb_env_set_maxdbs().
 2011-09-11 19:29:24 +02:00
Hallvard Furuseth
b56be64d2e Drop ULONG in favor of size_t. 2011-09-10 22:49:34 +02:00
Hallvard Furuseth
b1cfff4aeb Add mdb.c:txnid_t for readability. 
Maybe later we'll test txnid wraparound with it defined to ushort.
 2011-09-10 22:49:34 +02:00
Hallvard Furuseth
bf9d722bec mdb type cleanup: Consistent reader count types. 
Use unsigned int for reader counts/max limits, that's the smallest
change.  Don't need uint32_t when mdb_env_set_maxreaders() takes less.
Change prototypes of mdb_env_set_maxreaders,mdb_env_get_maxreaders().
Check the mdb_env_set_maxreaders() argument.
 2011-09-10 22:49:34 +02:00
Hallvard Furuseth
5f68293475 explain mdl_midl_sort() istack size 2011-09-10 22:49:34 +02:00
Hallvard Furuseth
2d1f3b7ed3 Fix memn(r)cmp of key sizes > INT_MAX. Simplify. 2011-09-10 22:49:34 +02:00
Hallvard Furuseth
69e53cf700 Check some mdb integer type assumptions 2011-09-10 22:49:34 +02:00
Hallvard Furuseth
88da18cccf Put MDB_node.<mn_lo,mn_hi> in host byte order. 2011-09-10 22:49:20 +02:00
Hallvard Furuseth
c5dad7a6d0 C90 compatibility cleanup in mdb. 
Fix void* pointer arithmetic in cintcmp().
Fix invalid ';'s in declarations after possibly-empty macros.
 2011-09-10 15:39:02 +02:00
Hallvard Furuseth
3dc8fa491a Cleanup mdb macros. 
Parenthesize.  Rename GetPageSize -> GET_PAGESIZE since it does not
behave like a function (it has a non-pointer output argument).
 2011-09-10 10:11:55 +02:00
Hallvard Furuseth
ec2d82f79a libmdb/.gitignore += test[45], Doxygen output 2011-09-10 10:11:55 +02:00
Howard Chu
acdc248f57 If put(NOOVERWRITE) exists, return existing data 2011-09-09 22:28:07 -07:00
Howard Chu
a7edb95e64 Minor typos, additional clarification 2011-09-09 03:52:12 -07:00
Howard Chu
0c487538cf Partial revert, fix MDB_page definition 2011-09-08 20:18:29 -07:00
Howard Chu
9ac8638d9d More docs 2011-09-08 17:13:51 -07:00
Howard Chu
bed3123b82 More docs 2011-09-08 14:34:21 -07:00
Howard Chu
1b69295a48 Speedup cintcmp 
Now that we know we're always 2-byte aligned...
 2011-09-08 13:10:17 -07:00
Howard Chu
dac3fae3b5 Fix previous commit 2011-09-08 10:08:21 -07:00
Howard Chu
8678fef597 Forgot to commit this 2011-09-07 23:11:14 -07:00
Howard Chu
2e3bc39fa9 Further cursor-based restructuring 
Get rid of all of the redundant MDB_*page structs and other cruft.
 2011-09-07 23:09:45 -07:00
Howard Chu
976a150973 Cleanup unused code 2011-09-06 23:37:37 -07:00
Howard Chu
1f8c3369b9 Fix mdb_move_node again 2011-09-06 23:16:45 -07:00
Howard Chu
f9c8796d0b Fix search_node, add cintcmp 2011-09-06 15:22:53 -07:00
Howard Chu
5e6a864513 Guarantee nodes are 2-byte aligned 
Also fix xcursor_init1 md_dirty flag
Also another mdb_cmp refactoring fix
 2011-09-06 13:50:51 -07:00
Howard Chu
293df78b2b Change MDB_node to depend on 2-byte alignment 2011-09-06 05:17:24 -07:00
Howard Chu
807841b1aa Fix for mdb_cmp refactoring 2011-09-05 16:54:39 -07:00
Howard Chu
963c421a97 Minor cleanup 2011-09-05 14:22:46 -07:00
Howard Chu
2cde97ae94 Use mdb_midl_append/sort instead of _insert 2011-09-05 02:36:24 -07:00
Howard Chu
70a4f6f29d mdb_cmp refactoring 2011-09-05 00:46:32 -07:00
Howard Chu
7bb14a2e79 Cursor stuff working, no major perf gain 2011-09-04 21:33:57 -07:00
Howard Chu
1843fe0b53 Cursors in-progress, broken 2011-09-04 21:33:57 -07:00
Howard Chu
8d89a53165 Use cursor->mc_flags 2011-09-04 21:33:57 -07:00
Howard Chu
7f46832ec5 Begin reimplementing basic ops as cursor ops 2011-09-04 21:33:57 -07:00
Howard Chu
f6edbd7793 Add MDB_{FIRST,LAST}_DUP for cursor_get 2011-09-04 21:33:57 -07:00
Howard Chu
9d821c26c3 Fix mdb_split, fix MDB_GET_BOTH 2011-09-03 20:11:07 -07:00
Howard Chu
e91dae3ed5 Cleanup midl, start documenting internals 2011-09-03 14:27:07 -07:00
Howard Chu
b30d0266e7 Tweak docs 2011-09-02 01:09:11 -07:00
Howard Chu
30736a0ff5 Ugly-as-sin initial Windows port 2011-09-01 16:31:10 -07:00
Howard Chu
2510bea718 API cleanup, doxygen docs for public API 2011-09-01 16:31:10 -07:00
Howard Chu
0f459dd773 Cleanup, fix freelist alloc 
Don't allow new pages for free list to come from the free list.
Otherwise a nasty data self-reference occurs that is too much
trouble to unwind.
 2011-09-01 16:31:10 -07:00
Howard Chu
f7aa0d5e4a Tweak locks, fix race conditions 2011-09-01 16:31:10 -07:00
Howard Chu
d27a7754f3 Reduce excessive memcpy in mdb_split 2011-09-01 16:31:10 -07:00
Howard Chu
027e1daa10 Fix mdb_split, tweak split_indx if space is tight 2011-09-01 16:31:10 -07:00
Howard Chu
01b9fc59fb Fix update of big data items 2011-09-01 16:31:10 -07:00
Howard Chu
13c663f666 Fix free page reclaim and dbs table update 
Can re-use free pages 1 txn earlier
Must update all dbs tables on txn commit
 2011-09-01 16:31:10 -07:00
Howard Chu
94718fab00 More debug, fix MDB_DEL_DUP 2011-09-01 16:31:10 -07:00
Howard Chu
ee69051a7c More debug output 2011-09-01 16:31:10 -07:00
Howard Chu
d54eb7dd81 Add txn_reset / txn_renew 2011-09-01 16:31:10 -07:00
Howard Chu
590c728044 First cut at DUPFIXED support 
Also in cursor_set, check the current page (if already set from
before) before starting over again from the root.
 2011-09-01 16:31:10 -07:00
Howard Chu
720c4b0a3d Set MINKEYS to 2 
For better space utilization
 2011-09-01 16:31:10 -07:00
Howard Chu
66a647e55d Keep re-using dpages instead of freeing them 2011-09-01 16:31:10 -07:00
Hallvard B Furuseth
ca496bd357 Avoid cast-away-const for errstr[] 2011-09-01 16:31:10 -07:00
Hallvard B Furuseth
5b3b522313 Wrap O_DSYNC in MDB_DSYNC. 
If O_DSYNC is not defined, that may be due to poor compiler options.
We could fall back to the less efficient O_SYNC, but it seems better
to let the user learn of the problem and give better compiler options.
 2011-09-01 16:31:10 -07:00
Hallvard B Furuseth
cb7c955220 Simplify: Replace IDL*_CMP() with CMP() 2011-09-01 16:31:10 -07:00
Hallvard B Furuseth
3029bb694d Macro cleanup: Parenthesize, simplify, remove a ; 2011-09-01 16:31:10 -07:00
Hallvard B Furuseth
a1b4144b80 Drop gcc extension ,##__VA_ARGS__. Add DPUTS(). 
C99 says DPRINTF(fmt, ...) must get at least 2 args.  Using DPUTS for 1 arg.
 2011-09-01 16:31:10 -07:00
Howard Chu
8f77043372 Fix cursor_set for dups 
Also make sure dups are counted in md_entries
 2011-09-01 16:31:10 -07:00
Howard Chu
edf9d8e51b Sorted dups fix 
Write the first datum normally. Convert to a sub-db if
additional items for the same key are written.
 2011-09-01 16:31:10 -07:00
Howard Chu
e15d4d16b6 Don't alloc in cursor_push/pop 2011-09-01 16:31:10 -07:00
Howard Chu
1ceae5c2c0 Fix multi-page commits 2011-09-01 16:31:10 -07:00
Howard Chu
6d924c1dda Fix overflow page read 2011-09-01 16:31:10 -07:00
Howard Chu
3dd815162c Sub-DB init/dirty fixes 2011-09-01 16:31:10 -07:00
Howard Chu
11308a190d Fix dupsort fetch 2011-09-01 16:31:10 -07:00
Howard Chu
697e00a8e2 More txn/cursor cleanup 2011-09-01 16:31:10 -07:00
Howard Chu
e29660566c Update DB tables on commit even if no dirty pages 2011-09-01 16:31:10 -07:00
Howard Chu
bab6be8077 Add mdb_strerror(), mdb_env_set_flags() 2011-09-01 16:31:10 -07:00
Howard Chu
9902a7764e allow silent commit of read-only TXNs 2011-09-01 16:31:10 -07:00
Howard Chu
9860d5a43f Protect err codes 2011-09-01 16:31:10 -07:00
Howard Chu
5e3a3db1f2 Error cleanup 
Replace MDB_FAIL with more specific codes
Return MDB_PANIC after failure to write meta page
 2011-09-01 16:31:09 -07:00
Howard Chu
5ef6c43acf Get rid of AC_MEMCPY 2011-09-01 16:31:09 -07:00
Howard Chu
1aa5105b67 Use IDL2 for dirty page list 2011-09-01 16:31:09 -07:00
Howard Chu
88a5f35c43 Use O_DSYNC on metafd. 
Use sync writes in mtest.
 2011-09-01 16:31:09 -07:00
Howard Chu
3e1c72a08e Minor cleanup 2011-09-01 16:31:09 -07:00
Howard Chu
19f5eebff8 Better defs for locker structures 2011-09-01 16:31:09 -07:00
Howard Chu
af0b49aa60 Silence stupid warning 2011-09-01 16:31:09 -07:00
Howard Chu
f9f0ec92a9 Add mdb_version() 2011-09-01 16:31:09 -07:00
Howard Chu
76d7ca8ce0 Rationalize mdbenv namespace 2011-09-01 16:31:09 -07:00
Howard Chu
c378b2525c Move MDB_VERSION to mdb.h, use major/minor/patch 2011-09-01 16:17:08 -07:00
Howard Chu
af70c0d9ce Tweak MDB_val to match struct berval 2011-09-01 16:17:08 -07:00
Howard Chu
985f765dc5 Rename idl to midl 2011-09-01 16:17:08 -07:00
Howard Chu
f5297dfc25 Add another FIXME comment 2011-09-01 16:17:07 -07:00
Howard Chu
35fca0f6af Add mtest2 and 3 2011-09-01 16:17:07 -07:00
Howard Chu
b070f7bff9 Portability tweak 2011-09-01 16:17:07 -07:00
Howard Chu
170703b0e4 Make a shared library too 2011-09-01 16:17:07 -07:00
Howard Chu
785d8e184c Start using as a library 2011-09-01 16:17:07 -07:00
Howard Chu
920115f747 Sorted dup fixes 2011-09-01 16:17:07 -07:00
Howard Chu
ce9456d95c Add MDB_INTEGERKEY db flag 2011-09-01 16:17:07 -07:00
Howard Chu
d499c244cd Allow setting DB flags on main DB 2011-09-01 16:17:07 -07:00
Howard Chu
2f7ddb4103 Cleanup, add mdb_cursor_count() 2011-09-01 16:17:07 -07:00
Howard Chu
2e776bd36e More DB checks 2011-09-01 16:17:07 -07:00
Howard Chu
b834dc05b3 Consolidate stat functions 2011-09-01 16:17:07 -07:00
Howard Chu
11204d176a Header cleanup 2011-09-01 16:17:07 -07:00
Howard Chu
40df306d52 More subDB fixes 2011-09-01 16:17:07 -07:00
Howard Chu
39817f6e85 Parameter cleanup 2011-09-01 16:17:07 -07:00
Howard Chu
819d98169e Valgrind complaints 2011-09-01 16:17:07 -07:00
Howard Chu
01c2f7816f More subDB stuff 2011-09-01 16:17:07 -07:00
Howard Chu
2fe9fbd1a3 Make sure subDB info is saved after writes 2011-09-01 16:17:07 -07:00
Howard Chu
314ac28938 Sorted dups basically completed, needs testing 2011-09-01 16:17:07 -07:00
Howard Chu
f586e57742 Added cursor_get for sorted dups 2011-09-01 16:17:07 -07:00
Howard Chu
f48f085ed8 Add mdb_get for sorted dups 2011-09-01 16:17:07 -07:00
Howard Chu
e3703ab58c Add mdb_del for sorted dups 2011-09-01 16:17:07 -07:00
Howard Chu
bb6a034e28 Add mdb_put for sorted dups 2011-09-01 16:17:07 -07:00
Howard Chu
51e210c604 Fix reader txn locking 2011-09-01 16:17:07 -07:00
Howard Chu
e3b668e84a Don't malloc the free_pgs list every time 2011-09-01 16:17:07 -07:00
Howard Chu
9f0b00b467 Setup context for sorted dups 2011-09-01 16:17:07 -07:00
Howard Chu
e5cf84495b Add explicit md_dirty flag to dbxs 2011-09-01 16:17:07 -07:00
Howard Chu
edf7869466 Add cursor_prev 2011-09-01 16:17:07 -07:00
Howard Chu
cc56ba04fb Use BDB cursor terminology 2011-09-01 16:17:07 -07:00
Howard Chu
60550e14ee Add some legalese 
Note derived from Martin Hedenfalk's btree.c; the code
is significantly reworked but owes a lot to the original.
 2011-09-01 16:17:07 -07:00
Howard Chu
36b0883564 Additional debug, fixes 
Fixed order of mdb_del0 operations
Drop in-memory free list from env on txn_abort
 2011-09-01 16:17:07 -07:00
Howard Chu
4ba37eed5e Save free-list in DB 2011-09-01 16:17:07 -07:00
Howard Chu
852ac0e3bc More sub-db stuff 2011-09-01 16:17:07 -07:00
Howard Chu
a687f2dc3c Use STAILQ instead of SIMPLEQ 
For compat with ancient BSD. Will probably just copy <ldap_queue.h> instead.
 2011-09-01 16:17:07 -07:00
Howard Chu
1f3fdc811e More subdb stuff, add cursor_last() 2011-09-01 16:17:07 -07:00
Howard Chu
9b4c689601 Tweak mt_dbxs array 2011-09-01 16:17:07 -07:00
Howard Chu
8211dd3960 Fix free page re-use 2011-09-01 16:17:07 -07:00
Howard Chu
415d98755d Fixup metapage writes 2011-09-01 16:17:06 -07:00
Howard Chu
4fd0f278d2 Tweaks to txn handling 2011-09-01 16:17:06 -07:00
Howard Chu
f89d0cc4b7 More subDBs, compiles now 2011-09-01 16:17:06 -07:00
Howard Chu
cb45191793 Start DB restructuring 2011-09-01 16:17:06 -07:00
Howard Chu
194334b2f7 Replace ulong with ULONG macro 2011-09-01 16:17:06 -07:00
Howard Chu
2766e88eeb More prep for subdatabases 2011-09-01 16:17:06 -07:00
Howard Chu
86e78566e3 Tweak comment about overflow allocations 2011-09-01 16:17:06 -07:00
Howard Chu
bc0ca49d16 Use an environment directory 
Go back to using an mmap'd lock file
 2011-09-01 16:17:06 -07:00
Howard Chu
5a72c6a294 No longer need -lssl 2011-09-01 15:53:33 -07:00
Howard Chu
5e6f32243c Fix db_stat stuff 2011-09-01 15:53:33 -07:00
Howard Chu
3f025d1d4c More fixes for seeking 2011-09-01 15:53:33 -07:00
Howard Chu
27ab75ef36 Re-use old pages 2011-09-01 15:53:33 -07:00
Howard Chu
661d4f80e3 Drop header page 
it's useless, just use meta pages instead
 2011-07-01 02:31:36 -07:00
Hallvard B Furuseth
c98c14fef5 Private: make.sh 2011-06-29 22:29:24 +02:00
Hallvard B Furuseth
63370215dd #define _XOPEN_SOURCE for random() 2011-06-29 22:29:24 +02:00
Hallvard B Furuseth
f88d626ff1 Drop unused vars 2011-06-29 22:29:24 +02:00
Hallvard B Furuseth
95c88a57b1 Warning cleanup: Make parent index unsigned 2011-06-29 22:29:24 +02:00
Hallvard B Furuseth
7c1e5e9540 Warning cleanup: printf(%p, struct*) 2011-06-29 22:29:24 +02:00
Hallvard B Furuseth
5fdbc54d76 Warning cleanup: if(rc = mdb_<func>()) & co 2011-06-29 22:29:24 +02:00
Hallvard B Furuseth
160585b1bc Warning cleanup: Drop unneeded 'unsigned >= 0' 2011-06-29 22:29:24 +02:00
Hallvard B Furuseth
bd2ceeb74e Fix P_INVALID signedness 2011-06-29 22:29:24 +02:00
Hallvard B Furuseth
f42f7f3442 Fix formats, reduce PAGEHDRSZ size_t->unsigned. 
NUMKEYS was size_t due to PAGEHDRSZ; printing unsigned int is simpler.

Instead increase PAGEFILL to long, since it involves a multiply.
Possibly uint32_t or something would be sufficient.
 2011-06-29 22:29:24 +02:00
Hallvard B Furuseth
aed36f5780 Compact memnrcmp, just for fun 2011-06-29 22:26:32 +02:00
Hallvard B Furuseth
ab0d6fa05e Fix memncmp(): Do not memcmp twice 2011-06-29 22:26:32 +02:00
Hallvard B Furuseth
86b40907d4 DPRINTF(): Fix format, handle non-gcc, simplify. 
Add C99-required 1st parameter.  Disable for for non-gcc.  Simplify.
GNU extension #define DPRINTF(<no 1st arg>...) -> GNU ext ,##__VA_ARGS__.
Fix __LINE__ format %ld -> %d.
 2011-06-29 22:26:32 +02:00
Hallvard B Furuseth
84659a5bb6 Makefile 2011-06-29 22:26:32 +02:00
Hallvard B Furuseth
f73a992ac9 .gitignore 2011-06-29 22:26:32 +02:00
Howard Chu
2d4f522a73 More cursor tests, fail... 2011-06-29 01:55:53 -07:00
Howard Chu
bddc86c0ee Save a byte in nodesz 2011-06-29 01:55:07 -07:00
Howard Chu
3a62d59c45 meta init 2011-06-28 22:34:53 -07:00
Howard Chu
fe703b0ee0 Log txnID in msgs 2011-06-28 22:05:54 -07:00
Howard Chu
421ee6bb1d Use toggling meta page instead of always appending 2011-06-28 18:40:52 -07:00
Howard Chu
a37195f686 Use shmat for lock region instead of mmap 2011-06-28 14:22:22 -07:00
Howard Chu
f367441b69 Add locking support 2011-06-28 13:46:48 -07:00
Howard Chu
a4c3626267 Fix P_INVALID def 2011-06-28 02:57:02 -07:00
Howard Chu
7d678832a7 Cursor testing 2011-06-28 02:43:49 -07:00
Howard Chu
cd57e9741f Fix splits, cursor navigation, newpage flags 2011-06-28 02:40:46 -07:00
Howard Chu
11e80dae63 Return statistics for a DB 
Eventually this will have to grow up to be like BDB db_stat...
 2011-06-28 00:16:05 -07:00
Howard Chu
a2de260fb3 Use NOSYNC flag 2011-06-28 00:15:40 -07:00
Howard Chu
0c2610ebdd Use header when some fields are unspecified 2011-06-28 00:15:15 -07:00
Howard Chu
d620d4368a Checkpoint 2011-06-27 22:39:24 -07:00