Metadata-Version: 2.1
Name: python-tkvdb
Version: 0.2.3
Summary: Cython wrapper for tkvdb radix trie key-value database
Home-page: https://github.com/vladimir-g/python-tkvdb/
Author: Vladimir Gorbunov
Author-email: vsg@suburban.me
License: ISC
Classifier: Development Status :: 3 - Alpha
Classifier: Topic :: Database
Classifier: Topic :: Database :: Database Engines/Servers
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: ISC License (ISCL)
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: C
Classifier: Programming Language :: Cython
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.5
Description-Content-Type: text/markdown
License-File: LICENSE

# python-tkvdb

Python-tkvdb is a Cython wrapper for
[tkvdb](https://github.com/vmxdev/tkvdb) trie key-value
database. Python 3 is required.

Code isn't well tested in production environments.

## Installation

This is a typical python/cython package that uses setuptools build
system.

### From PyPi

The most simple way of installing is using `pip`:

    pip install python-tkvdb

Considering that package is using Cython, C compiler may be required
for building if suitable wheel for current platform isn't found.

### Downloading

Original tkvdb sources are included as submodule, so when installing
from git please use `git clone --recursive` for cloning, or init
submodules after with `git submodule update --init
--recursive`. Custom sources may also be provided in `tkvdb/`
subdirectory before build.

Source archives that were made with `python setup.py sdist` contain
generated C-code without `.pyx` files as Cython official documentation
recommends. Cython header files (`.pxd`) still provided though.

Package also can be distributed as python wheels. With wheels no
additional post-installation actions are required.

Initialization and usage of virtualenv or alternatives aren't
properly described in this manual, so use them by you own discretion.

### Building

Project uses Cython to generate C extension for Python. To build
package, you need to have C compiler and build tools.

For installation from the source archive, Cython isn't required, but
git versions require it. Source directory also includes
`pyproject.toml` (PEP 518), so if your build tool uses it, Cython
would be installed anyway.

To make common tasks easier, project contains simple Makefile that may
be used instad of pip/python commands. It isn't a requirement, so `pip
install .` also works. For additional reference, look into
[Makefile](Makefile).

Install example with virtualenv:

    cd python-tkvdb/
    python -m venv env
    source ./env/bin/activate
    make

Both Makefile and setup.py uses ``USE_CYTHON`` env variable (int, 0/1)
to determine if Cython (cythonize) would be started before extension
building. Cython needs to be installed in local environment for
this. Default value is 1 for make.

Makefile also has ``PYTHON`` env var that allows overriding python
interpreter path. Default value is just ``python``.

Example usage:

    USE_CYTHON=1 PYTHON=./env/bin/python make build

Make commands:

- `build` -- just build extension with `python setup.py build_ext`.
- `install` -- run `pip install`. Extension would be built if needed.
- no arguments (just `make`) -- alias for `install`.
- `dist` -- create wheel and sdist archive.
- `test` -- run unit tests
- `clean` -- remove generated code, compiled objects and distribution
archives.
- `uninstall` -- remove previously installed package (through `pip`)

After installing module `tkvdb` must be importable in the Python environment.

## Usage

Original `tkvdb` uses pretty specific terminology for some actions
(like transaction), so it is recommended to first consult with
original documentation anyway. Thread-safety notes and some caveats
are also described in the original README file.

Python-tkvdb provides pythonic-style wrapper for most parts of the original
library.

Most object have destructors that call `free` internally and do memory
management on garbage collection, but don't forget that this is
wrapper for C library.

Some code examples may also be found in [tests code](tests/).

### Modules

Project is splitted into multiple python modules:

- `ctkvdb` -- Cython wrapper with C definitions from `tkvdb.h`.
- `tkvdb.db` -- database object and initialization. Also imported in
  `__init__.py` (i.e. main `tkvdb` module).
- `tkvdb.transaction` -- transaction (actually main input/output.
  interface). Wrapper around ``tkvdb_tr``.
- `tkvdb.cursor` -- transaction cursors for iteration. Wrapper around `tkvdb_cursor`.
- `tkvdb.iterators` -- pythonic iterators for `tkvdb.cursor`.
- `tkvdb.errors` -- all db-related exceptions that code may throw.
- `tkvdb.params` -- database and transaction params. Wrapper around `tkvdb_params`.

### Database initialization

Database is wrapped into the `Tkvdb` object from `tkvdb` or `tkvdb.db`
modules. At this time only path to database file is
supported. Parameters (`tkvdb.params.Params`) optionally may be passed
to constructor.

```python
from tkvdb import Tkvdb

db = Tkvdb(path_to_db_file)
# some code
db.close()
```

Context manager (`with` statment) that includes auto closing is also
available:

```python
with Tkvdb(path_to_db_file) as db:
    # some code
    with db.transaction() as tr:
        # more code
```

Attributes (readonly):
- `path: str` -- path to database file.
- `is_opened: bool` -- shows that database is initialized properly.

Methods (may raise exceptions):
- `Tkvdb(path: str, params: tkvdb.params.Params = None)` (constructor)
  -- create database instance.
- `close()` -- close database.
- `transaction(params: tkvdb.params.Params = None) ->
  tkvdb.transaction.Transaction` -- create transaction.

There is also Cython method `get_db` that returns `tkvdb_db *`
pointer.

### Transactions

Transactions are basic way to do any operation with database. Consult
with original documentation about transaction term, because it doesn't
mean same thing as in other database systems.

**Input and ouput uses `bytes` type for everything**. Encode and
decode strings if needed.

Parameters (`tkvdb.params.Params`) optionally may be passed to
constructor.

Transaction must be created from database instance (described in
previous part):

```python
transaction = db.transaction()
transaction.begin()
transaction.put(b'key', b'value')
transaction.commit() # or transaction.rollback()
print(transaction.getvalue(b'key'))
transaction.free()
```

Pythonic `with` statment also available:

```python
with db.transaction() as tr:
    tr.put(b'key', b'value')
    tr.commit()
    print(tr.getvalue(b'key'))
```

Note that `with` statement *does not do commit, but rollbacks on
exception*. Do `commit` or `rollback` with your own code, or don't do
anything (implies rollback-like behavior). Transaction is started
(`begin`) automatically and will be freed (`free`) at exit from `with`
block though.

Transaction also has Python dict-like interface:

- `__getitem__` and `__setitem__`
- `get(key, default=None)`
- `keys()`, `values()` and `items()` iterators

```python
with db.transaction() as tr:
    tr[b'key'] = b'value'
    print(tr.get(b'other-key', b'default')) # prints b'default'
    tr.commit()
    print(tr[b'key']) # prints b'value'

    # Iterators
    for key in tr: # or tr.keys()
        print(key)
    for key, value in tr.items():
        print(key, value)
    for value in tr.values():
        print(value)
```

Multiple keys may be deleted by `delete` method using optional
`prefix` argument (default `False`). Passing `True` allows deleting
keys starting from prefix. Dict-like `del` operator always deletes
only exact key.

Attributes (readonly):
- `is_initialized: bool` -- shows that transaction underlying
  structures are initialized properly.
- `is_started: bool` -- shows that `begin()` method was called.
- `is_changed: bool` -- shows that transaction had any uncommited
  changes (i.e. `put()` was used).
- `ram_only: bool` -- indicates that transaction is RAM-only.

Transaction methods. Most of them may raise an exception:

- `Transaction(ram_only=True, params: tkvdb.params.Params = None)`
  (constructor) -- create transaction instance. Must be called
  manually only for RAM-only usage, otherwise `db.transaction()` must
  be used instead.
- `begin()` -- starts transaction, calls underlying `tkvdb_tr->begin()`.
- `getvalue(key: bytes) -> bytes` -- get value by key.
- `put(key: bytes)` -- insert value into db by key.
- `get(key: bytes, default: bytes = None) -> bytes` -- dict-like get with
  default value.
- `delete(key: bytes, prefix=False)` -- delete value by key. With
  `prefix=True` all keys staring with `key` will be deleted.
- `__getitem__`, `__setitem__`, `__delitem__` -- dict-like methods.
- `__contains__` -- allows usage of `in` operator.
- `free()` -- free transaction (called in `with` statement
  automatically).
- `keys()`, `values()`, `items()` -- return dict-like iterators.
- `cursor(seek_key=None, seek_type=Seek.EQ)` -- return transaction
  cursor (see `Cursors`), with optional seek.


#### RAM-only transactions

Transactions also may be used in RAM-only mode. These transactions don't
require database file and use less memory. They are cleared on
`commit()` or `rollback()`. See more about RAM-only transactions in
original documentation.

Use `tkvdb.Transaction()` constructor to create RAM-only transaction
without database. This transaction may also be used with `with`
statement, same auto-begin rules apply. Example:

```python
with Transaction() as tr:
    tr[b'key'] = b'value'
    print(tr[b'key'])  # prints b'value'
    tr.commit()  # clears transaction
```

### Iterators

Transaction can be traversed using iterators. It is also the main way
for iterating through database contents.

Module `tkvdb.iterators` provides three dict-like iterators that use
`tkvdb.cursor.Cursor` inside:

- `tkvdb.iterators.KeysIterator` -- iterating over keys.
- `tkvdb.iterators.ValuesIterator` -- iterating over values.
- `tkvdb.iterators.ItemsIterator` -- iterating over key-value pair.

They can be used with transaction:

```python
with db.transaction() as tr:
    for key in tr:  # or tr.keys()
        print(tr[key])
    for value in tr.values():
        print(value)
    for key, value in tr.items():
        print(key, value)
```

In all loops new instance of `Cursor` is used.

They also can be used with cursor:

```python
with db.transaction() as tr:
    with tr.cursor() as c:
        for key in c:
            print(c.key(), c.keysize())
```

Notice that cursor iterators use same underlying `Cursor` object, so
they would iterate from same place where cursor stopped before:

```python
with db.transaction() as tr:
    with tr.cursor() as c:
        c.first()
        # do some iteration with c.next()
        for key in c:
            print(key)  # it wouldn't be first key
            if something:
                break
        for value in c.values():  # starts from last iterated key
            print(value)
```

Reverse iteration is available through the standard `reversed`
function. Iterators and transaction have required methods for this.
For new cursors iteration will start from the end using
`tkvdb.cursor.Cursor.last`.

```python
with db.transaction() as tr:
    for key in reversed(tr):
        print(key)
    # All iterator types allow this
    for key, value in reversed(tr.items()):
        print(key, value)

with db.transaction() as tr:
    with tr.cursor() as c:
        for key in reversed(c):
            print(key)
    with tr.cursor() as c:
        for value in reversed(c.values()):
            print(value)
```

Cursors created from transacton can used for searching (as shorthand
for `tkvdb.cursor.Cursor.seek()`). More information in next section.

```python
with db.transaction() as tr:
    with tr.cursor(seek_key=b'seek-tr-31', seek_type=Seek.GE) as c:
        print(c.key())
```

### Cursors

Cursors are used to iterate through database contents. They are
defined in `tkvdb.cursor` module, C implementation is wrapped in
`tkvdb.cursor.Cursor` class.

Cursors are attached to transaction and created by
`Transacion.cursor()` method. They also may be created directly.

Although cursors are sole way to iterate and seek in tkvdb, it is
better and easier to use python-style iterators for
such purposes.

Example usage:

```python
with db.transaction() as tr:
    with tr.cursor() as c:
        c.first()
        while True:
            print(c.key(), c.value())
            try:
                c.next()
            except tkvdb.errors.NotFoundError:
                break
```

Cursor also may be used without `with` statement, it would be freed
anyway on garbage collection:

```python
with db.transaction() as tr:
    c = tr.cursor():
    c.first()
    # ...
```

Notice: `first` and `next` methods throw `tkvdb.errors.EmptyError` on
empty database, not `NotFoundError`. Cursors may be iterated by using
iterators (see previous section).

Cursor also may be iterated in reverse order using `prev()`
method. Another method called `last()` moves cursor to last record and
often useful for the reverse iteration.

Cursor can be used for search with `seek()` method. This allows
searching k-v pair by prefix using seek criteria. Criterias are
defined in `tkvdb.cursor.Seek` enum:

- `Seek.EQ` -- search for the exact key match.
- `Seek.LE` -- search for less (in terms of memcmp()) or equal key.
- `Seek.GE` -- search for greater (in terms of memcmp()) or equal key.

Seeking is also may be initiated using `Transaction.cursor()` method
(see more in `Transactions` section).

```python
from tkvdb.cursor import Seek

with db.transaction() as tr:
    with tr.cursor() as c:
        c.seek(b'key', Seek.EQ)
        key = c.key()
        # ...
        c.next()

    with tr.cursor(seek_key=b'key') as c:
        # ...
```

Attributes (readonly):
- `is_initialized: bool` -- shows that cursor underlying
  structures are initialized properly.
- `is_started: bool` -- shows that `first()` method was called.

Cursor methods.

- `first()` -- move cursor to first item in database.
- `last()` -- move cursor to last item in database.
- `next()` -- move cursor to next item in database.
- `prev()` -- move cursor to previous item in database.
- `key() -> bytes` -- get current key.
- `val() -> bytes` -- get current value.
- `keysize() -> int` -- get current key size.
- `valsize() -> int` -- get current value size.
- `free()` -- free cursor.
- `__iter__()` -- returns `tkvdb.iterators.KeysIterator`.
- `seek(key: bytes, seek: tkvdb.cursor.Seek)` -- search key by
  criteria.
- `keys()`, `values()`, `items()` -- return dict-like iterators.

### Params

Params are used to specify different options for database and/or
transactions. They are defined in `tkvdb.params` module: the C
implementation (`tkvdb_params` struct) is wrapped by
`tkvdb.params.Params` class, and param values are wrapped by the
`tkvdb.params.Param` enum.

Parameter names transfomed using CamelCase, example: 

```
TKVDB_PARAM_TR_DYNALLOC => TrDynalloc
TKVDB_PARAM_CURSOR_STACK_DYNALLOC => CursorStackDynalloc
```

Available parameters in `tkvdb.params.Param` enum:

- `TrDynalloc` -- `TKVDB_PARAM_TR_DYNALLOC`
- `TrLimit` -- `TKVDB_PARAM_TR_LIMIT`
- `Alignval` -- `TKVDB_PARAM_ALIGNVAL`
- `Autobegin` -- `TKVDB_PARAM_AUTOBEGIN`
- `CursorStackDynalloc` -- `TKVDB_PARAM_CURSOR_STACK_DYNALLOC`
- `CursorStackLimit` -- `TKVDB_PARAM_CURSOR_STACK_LIMIT`
- `CursorKeyDynalloc` -- `TKVDB_PARAM_CURSOR_KEY_DYNALLOC`
- `CursorKeyLimit` -- `TKVDB_PARAM_CURSOR_KEY_LIMIT`
- `DbfileOpenFlags` -- `TKVDB_PARAM_DBFILE_OPEN_FLAGS`

Consult with original `tkvdb` documentation for params meaning and
possible values.

Params usage:

```python
from tkvdb.params import Params, Param

# Passing params to database
params = Params({Param.Autobegin: 1})  # set params at init
with Tkvdb(path, params) as db:
    # Params also will be passed to db transactions
    with db.transaction() as tr:
        # ...
    # Transaction may use own params
    params = Params()
    params.set(Param.TrLimit, 100)
    with db.transaction(params) as tr:
        # ...

# Setting params after init
params = Params()
params.set(Param.Autobegin, 1)
tr = Transaction(params=params)  # RAM-only transaction
```

As in original `tkvdb` code, params from database are passed to
db-bound transaction, if they aren't overrided directly by passing
another `Params` instance to transaction.

Python implementation also stores all set params and stores them in
internal `values` dict. Notice that it tracks only params that were
set directly, so default values aren't known to `Params` wrapper.

Params attributes:
- `is_initialized: bool` -- shows that params underlying structures
  are initialized properly.

Params methods:
- `Params(params=None)` (constructor) -- create params
  instance. Argument `params` may be dict with param names and values.
- `get_values()` -- return all set params.
- `get(param: tkvdb.params.Param)` -- return single param value.
- `set(param: tkvdb.params.Param, value: int)` -- set param value.
- `free()` -- free params object.

There is also Cython method `get_params` that returns `tkvdb_params *`
pointer.

### Errors

Error classes are defined in `tkvdb.errors` module. Every non-ok
return value from the underlying C code is converted to python
Exception. Only `TKVDB_RES.TKVDB_OK` considered as success.

Consult with original documentation for error codes meaning.

One exception from this rule is
`tkvdb.transaction.Transaction.__getitem__()` (dict-like access) that
raises `KeyError` for python compatibility.

Examples:

```python
from tkvdb.errors import EmptyError, NotFoundError, NotStartedError

# ...
tr = db.transaction()
try:
    print(tr.getvalue(b'key'))
except (NotFoundError, EmptyError):
    print('key not found')
except NotStartedError:
    print('transaction not started')

with db.transaction() as tr:
    try:
        print(tr[b'key'])
    except KeyError:
        print('key not found')
```

Note that tkvdb raises `EmptyError` (`TKVDB_RES.TKVDB_EMPTY` return
code), not `NotFoundError` when key is not found in empty database,

Errors:

- `Error` -- base class for all tkvdb-related errors.
- `IoError` -- `TKVDB_RES.TKVDB_IO_ERROR` code.
- `LockedError` -- `TKVDB_RES.TKVDB_LOCKED` code.
- `EmptyError` -- `TKVDB_RES.TKVDB_EMPTY` code.
- `NotFoundError` -- `TKVDB_RES.TKVDB_NOT_FOUND` code.
- `EnomemError` -- `TKVDB_RES.TKVDB_ENOMEM` code.
- `CorruptedError` -- `TKVDB_RES.TKVDB_CORRUPTED` code.
- `NotStartedError` -- `TKVDB_RES.TKVDB_NOT_STARTED` code.
- `ModifiedError` -- `TKVDB_RES.TKVDB_MODIFIED` code.


## License

Python-tkvdb is licensed under ISC license as original tkvdb project.
