Commit Graph

459 Commits

Author SHA1 Message Date
Hallvard Furuseth
c7db955a94 ITS#7517 Don't save dropped dirty MDB databases.
mdb_txn_commit's attempt to save the DB could corrupt another DB if
another thread had called mdb_dbi_open and reused the closed DBI.
2013-02-16 19:08:37 +01:00
Hallvard Furuseth
8e1bbdf0dd mdb_stat -ff[f]: show contiguous page spans. 2013-02-16 19:07:16 +01:00
Hallvard Furuseth
7aba5f5ab9 Add error code MDB_MAP_RESIZED. 2013-02-16 19:06:28 +01:00
Howard Chu
5e59695b8d Don't memmove freelist entry when chunking it
Just advance the pointer, now that it's no longer a complex struct
2013-02-15 13:27:50 +00:00
Hallvard Furuseth
52ecd38e18 ITS#7455 Save freelist in single-page chunks 2013-02-15 13:27:50 +00:00
Hallvard Furuseth
1b8bfc5756 Freelist cleanup/streamlining
Drop unneeded definitions, redundant code.
2013-02-15 13:27:02 +00:00
Hallvard Furuseth
65c053a6e7 ITS#7512 Fix MDB page leak when malloc error.
mdb_page_alloc(): Delay moving me_pgfirst,me_pglast
until malloc(MDB_oldpages to hold the IDs) succeeds.
2013-02-07 08:17:30 +01:00
Howard Chu
e0eb495fb3 Revert "ITS#7515 fix mdb_dbi_open/close"
This reverts commit 48dc782ea6.
2013-02-04 10:53:17 -08:00
Howard Chu
744e311939 NULL key is allowed for MDB_CURRENT 2013-02-01 06:19:23 -08:00
Howard Chu
48dc782ea6 ITS#7515 fix mdb_dbi_open/close
If a DBI handle is opened by a txn that aborts, the DBI handle
should no longer be valid.
2013-01-31 20:05:48 -08:00
Howard Chu
27aaecc744 ITS#7511 make sure cursor is marked valid after successful put 2013-01-28 18:45:36 -08:00
Howard Chu
9fa0c3bf31 ITS#7505 fix prev commit, partial revert 2013-01-29 01:08:24 +00:00
Howard Chu
0b8ac92b7a ITS#7505 Fix mdb_update_key when key is too big 2013-01-27 18:02:18 +00:00
Howard Chu
cfedb365b4 Fix MDB_CURRENT updating for dupsort items 2013-01-26 18:55:35 +00:00
Hallvard Furuseth
7fdf672041 Update MDB comments: Caveats, flags, etc. 2013-01-16 18:42:57 +01:00
Howard Chu
8e7bb2042b ITS#7485 data sizes limited to 32 bits
That's all we have space for in a node record.
2013-01-15 04:28:47 -08:00
Howard Chu
1a0d02c4c4 ITS#7485 more size checks 2013-01-14 13:46:25 -08:00
Howard Chu
25a99d4024 ITS#7485 check maxkeysize in mdb_cursor_put
Also MDB_MAXKEYSIZE is redefinable at compile time.
2013-01-14 13:12:15 -08:00
Howard Chu
9e35127f1d One more rename
Forgot this in 98fe339089
2013-01-14 12:57:54 -08:00
Howard Chu
53cf2eed90 ITS#7491 check for filled dirty page list
Very large single transactions will fail. It's not just a problem when
nested transactions are used. We could make this dynamically sized,
but I'm not sure what the point is.
2013-01-11 11:45:25 -08:00
Howard Chu
fed573cb86 Happy New Year 2013-01-10 02:58:55 -08:00
Hallvard Furuseth
cd80a15474 Delete liblmdb/mfree. It has moved into mdb_stat. 2013-01-09 20:49:43 +01:00
Howard Chu
2a36346b50 ITS#7455 don't give up so soon
If we're operating on the freelist, see if our current pghead
can satisfy the request before giving up and using new pages.
2013-01-09 08:30:13 -08:00
Howard Chu
057e068630 Fix typo 2012-12-19 13:20:24 -08:00
Howard Chu
460a05bf78 Add initial mdb utility manpages 2012-12-19 13:09:56 -08:00
Howard Chu
82b97a8af0 init_meta should write mapsize
Was leaving it 0 initially.
2012-12-11 22:21:52 -08:00
Howard Chu
0702fbd688 Update version date 2012-12-11 16:01:25 -08:00
Howard Chu
9acf801cc1 Make sure init_meta is at beginning of file 2012-12-11 12:55:47 -08:00
Howard Chu
ed86a086f7 More for prev commit
Use the latest meta page
2012-12-11 12:46:17 -08:00
Howard Chu
e74d70e88e Partially revert 65d40eb5d2
Allow both increasing and decreasing the environment size.
But don't allow decreasing below the currently occupied space.
2012-12-11 12:03:19 -08:00
Howard Chu
282be11654 FIx prev commit 2012-12-10 17:21:24 -08:00
Howard Chu
0ce6bb4be0 Add error checks to txn_commit
These ops could fail if the map is full.
2012-12-10 12:21:36 -08:00
Howard Chu
aff2693fc0 ITS#7455 simplify
Don't try to reclaim overflow pages while operating on
the freelist (for now). The circular dependencies are much like
the single-page case, but worse. Maybe look into this in the
future, but it's not absolutely necessary now.
2012-12-10 12:16:50 -08:00
Howard Chu
e26fc73417 ITS#7467 add verbose freelist info 2012-12-07 13:19:36 -08:00
Howard Chu
784757faa9 ITS#7465 fail if both -a and -s specified 2012-12-07 12:06:30 -08:00
Howard Chu
9a5d4370be Yet more retries for overflow page scan 2012-12-06 17:39:49 -08:00
Howard Chu
5863d5cc61 Freelist fixes
Keep list sorted if it grows during a write. Don't free pghead until
we're sure our writes are all finished.
2012-12-06 09:05:30 -08:00
Howard Chu
7b10fdde00 Fix mdb_env_copy
Just write in 2^31 sized chunks
2012-12-04 06:30:22 -08:00
Howard Chu
04b7c42109 Rename mdb_open/close mdb_dbi_open/close
Avoid possible symbol clashes with MDB-Tools libmdb
2012-12-03 16:53:24 -08:00
Howard Chu
e545b742e5 Fix 5c1ee7f7ba
mdb_cursor_sibling() no longer pops cursor before returning.
2012-12-03 13:57:46 -08:00
Howard Chu
5af47adf6b Fix mdb_midl_shrink 2012-12-03 13:57:28 -08:00
Howard Chu
d3f51fdcf7 More retries for overflow page scan 2012-11-30 12:54:19 -08:00
Howard Chu
98fe339089 Rename libmdb to liblmdb 2012-11-30 12:45:34 -08:00
Howard Chu
2a94ebced6 Add install target 2012-11-30 12:39:53 -08:00
Howard Chu
13f3bcd59c Rename libmdb to liblmdb, lmdb.h
Avoid naming conflicts with other mdb* packages. Bump version to 0.9.5
2012-11-30 12:30:51 -08:00
Howard Chu
f165507199 ITS#7455 use freelist for overflow pages 2012-11-30 12:18:27 -08:00
Howard Chu
185136397d Drop unneeded #include <time.h> 2012-11-28 15:34:44 -08:00
Howard Chu
049b145ec7 Doc layout tweaks 2012-11-28 15:34:19 -08:00
Hallvard Furuseth
97ed96206d mdb_env_sync(,force=1): Override MDB_MAPASYNC.
With MDB_MAPASYNC, the API provided no way to ensure full sync.
2012-11-28 00:59:00 +01:00
Hallvard Furuseth
b9275c9991 mdb_cursor_dbi(): Assume valid args.
It returned 0, which is the freelist DBI and is valid for a few uses.
2012-11-28 00:59:00 +01:00
Hallvard Furuseth
7e620a34a2 API change: mdb_drop() only accepts del=0 or 1.
This allows for other flag values in the future.
2012-11-28 00:59:00 +01:00
Hallvard Furuseth
d903bed218 API change: mdb_env_set_maxdbs(, *named* DBs).
Let the user specify the number of databases he will actually
use. Adding 2 for FREE_DBI and MAIN_DBI is an internal matter.
2012-11-28 00:59:00 +01:00
Hallvard Furuseth
6b56e0fab1 ITS#7453 Fix reset of MDB_NOSYNC/MDB_NOMETASYNC.
mdb_env_open() with these flags did not open a synchronizing
file descriptor.  Thus disabling them later did not work.
2012-11-28 00:59:00 +01:00
Hallvard Furuseth
0201b9002c Tweak MDB_FIXEDMAP handling.
Drop mmap()'s MAP_FIXED flag, so it returns another address instead
of unmapping existing overlapping pages. Verify the returned address.
2012-11-28 00:59:00 +01:00
Hallvard Furuseth
52e3adbdec Renumber MDB_NOSUBDIR. Check mdb_env_open() flags.
MDB_NOSUBDIR was == MDB_REVERSEKEY. That affected the freelist:
Env flags are stored in mm_flags alias mm_dbs[FREE_DBI].md_flags.

It stays a persistent flag, in case mdb_env_open someday wants
to pick/verify which lockfile to use with the datafile.

Catch bad flags so they will no longer make it into the data file.
2012-11-28 00:59:00 +01:00
Howard Chu
bb36bdcd1c ITS#7432 fix typo in mdb_midl_sort
Wasn't pushing the optimal half of the array onto the stack,
thus used more stack than expected -> overrun.
2012-11-05 05:06:06 -08:00
Howard Chu
0108327c27 Better fix for MDB_DEBUG tweak -- from hbf 2012-10-25 07:03:25 -07:00
Howard Chu
b68aa3f86b Revert "MDB_DEBUG tweak for old compilers"
This reverts commit ef2092eac9.
2012-10-25 06:54:09 -07:00
Howard Chu
ef2092eac9 MDB_DEBUG tweak for old compilers 2012-10-25 05:28:55 -07:00
Howard Chu
c2f15990c7 Fix flag checks in mdb_open 2012-10-23 04:48:04 -07:00
Howard Chu
f5b8d017df fix Windows mdb_env_copy(), WriteFile overflow 2012-10-23 04:32:55 -07:00
Howard Chu
66f2b7b2ec Make sure mdb_open flags are committed on main DB
If no data was modified in the txn, mdb_open flag changes were dropped.
2012-10-22 17:03:15 -07:00
Howard Chu
5f2ddb5532 Minor cleanup 2012-10-22 16:13:42 -07:00
Howard Chu
5c1ee7f7ba Fix MDB_PREV
Could return 1 garbage record before actual data, if starting from EOF
2012-10-22 16:02:30 -07:00
Howard Chu
617769bce5 Tweak output, add free page info 2012-10-16 17:01:16 -07:00
Howard Chu
754cd88819 Add mdb_env_info() to retrieve interesting env params 2012-10-16 16:22:21 -07:00
Howard Chu
7f0771a3ed Merge mdb_stata into mdb_stat 2012-10-16 15:37:29 -07:00
Howard Chu
65d40eb5d2 Make sure increases of mapsize are persisted 2012-10-16 14:53:25 -07:00
Hallvard Furuseth
94965307c8 ITS#7377 Errorcheck Windows calls and thread keys. 2012-10-15 17:04:48 +02:00
Hallvard Furuseth
88b7bb4639 Explicitly unlock the MDB lockfile on Windows. 2012-10-15 17:04:43 +02:00
Hallvard Furuseth
e9d87866c4 mdb_env_open(): Keep mdb_env_set_flags() flags. 2012-10-15 17:04:11 +02:00
Hallvard Furuseth
092f9a5c66 More for mdb_env_copy().
Fix #define _GNU_SOURCE = 1 to match any command line -D_GNU_SOURCE.
.gitignore += mdb_copy.
2012-10-15 17:03:46 +02:00
Hallvard Furuseth
462d9dfd10 Fix last commit.
Add flag MDB_ENV_ACTIVE instead of using the state of me_lfd.
Require MDB_RDONLY for MDB_ROFS.  Swap reader/writer mutex lock.
2012-10-15 17:03:33 +02:00
Howard Chu
e40713b631 Support read access on read-only media 2012-10-13 04:05:00 -07:00
Howard Chu
f037dd2d19 Silence warning in prev commit 2012-10-11 16:47:56 -07:00
Howard Chu
ebb6859ea5 Add mdb_copy for backing up a DB environment 2012-10-11 12:24:01 -07:00
Hallvard Furuseth
756ce8e10c Tweak MDB restrictions 2012-10-03 21:13:21 +02:00
Hallvard Furuseth
a3f33dd20e Fix mdb_open() off-by-one error in maxdbs check 2012-10-03 18:08:56 +02:00
Hallvard Furuseth
38fb8e6eb1 ITS#7377 Wrap fcntl F_SETLK in EINTR loop too.
This can happen even on local filesystems.
2012-10-03 18:06:29 +02:00
Hallvard Furuseth
ab3fea51dc ITS#7377 Catch ftruncate() error 2012-10-03 18:06:29 +02:00
Hallvard Furuseth
2b960613e0 ITS#7377 Document caveats/troubeshooting. 2012-10-03 18:04:07 +02:00
Howard Chu
7333b6bdc9 Fix prev commit
It only worked if the freelist was already non-empty
2012-09-30 10:01:57 -07:00
Howard Chu
cd228b4722 Fix mdb_drop
if it was the only operation in a txn, it would be ignored.
2012-09-30 07:49:30 -07:00
Howard Chu
0c4c6fe72a ITS#7385, check further for bad splits
More for 48ef27b6f5
Our definition of "large" data item needs to be smaller to
handle more cases where a bad split is possible
2012-09-27 05:06:23 -07:00
Howard Chu
ba6c3281a1 Add MDB_GET_CURRENT
return whatever the cursor is currently pointing at
2012-09-19 06:15:09 -07:00
Howard Chu
d13ca0de4f ITS#7394 ignore WRITEMAP if RDONLY 2012-09-18 12:36:54 -07:00
Howard Chu
0a359fb629 More for 48ef27b6f5
page_split with newindex > split_indx
2012-09-17 07:02:41 -07:00
Howard Chu
5ef56b437e More for ab04c50a32
Use explicit MDB_RDONLY flag
2012-09-17 07:01:28 -07:00
Hallvard Furuseth
c67ea9c060 ITS#7377 Catch MDB user errors. 2012-09-17 15:42:15 +02:00
Hallvard Furuseth
c0f3d9b9a8 ITS#7377 Catch MDB setup errors and clean up. 2012-09-17 15:42:15 +02:00
Hallvard Furuseth
fe1b3794de ITS#7364 Limit mdb lock upgrade before sem_unlink.
Do not try shared lock when closing or after error.  Track file lock
state to decide.  Change meaning of *excl to reflect file lock state.
2012-09-17 15:42:15 +02:00
Hallvard Furuseth
c760e536ec ITS#7364 Always sem_unlink() in mdb_env_open().
Drop the sem_unlink() error checks, which could prevent the 2nd
unlink.  Instead use O_EXCL in sem_open().

This makes "open+close the database" the API for trying to clean
away the old semaphores, if they were left behind by a previous run.
2012-09-17 15:42:15 +02:00
Hallvard Furuseth
31be24896b ITS#7377 Wrap sem_wait & file locks in EINTR loops 2012-09-17 15:42:14 +02:00
Hallvard Furuseth
a35f9b2a53 Remove mdb data races. Use (txnid_t)-1 as "no ID".
Avoid race between numreaders++ and reading numreaders at cleanup. Make
the un-mutexed reset of reader table entry, atomic: Reset mr_pid only.

Instead check mr_pid != 0 in mdb_page_alloc()'s scan for readers.
(txnid_t)-1 as "no ID"-mark avoids a check for mr_txnid != 0.
The scan can stop when seeing an old reader.
2012-09-17 15:42:14 +02:00
Hallvard Furuseth
38cc1e96b4 Save pid in MDB_env instead of repeating getpid().
An open MDB environment does not survive or catch fork(),
so repeating getpid() was pointless.
2012-09-17 15:42:14 +02:00
Hallvard Furuseth
20a216fcc9 MDB_WRITEMAP needs no DSYNC descriptor (me_mfd) 2012-09-17 15:42:14 +02:00
Hallvard Furuseth
588a84a5ae ITS#7363 Preprocessor namespace cleanup.
Rename USE_POSIX_SEM to MDB_USE_POSIX_SEM.
Separate MDB_FDATASYNC from MDB_USE_POSIX_SEM.
2012-09-17 15:42:14 +02:00
Howard Chu
8bb10add24 More for ab04c50a32
Fix typos, error code ranges
2012-09-17 06:35:03 -07:00
Howard Chu
acbff5b1ea Add mdb_cursor_renew()
Allow cursors on read-only txns to be reused with later txns.
2012-09-17 04:41:13 -07:00
Howard Chu
076b2b36a9 Shared lib should depend on pthread 2012-09-17 03:48:54 -07:00