mcetin

AmberDB is an embedded, schema-driven NoSQL database engine for Perl, designed for applications that need structured records, fast indexed queries, full-text search, transactions, and a simple deployment model.

It is built on top of Berkeley DB through DB_File, but AmberDB provides a much higher-level database API rather than exposing the underlying key-value store directly.

Why AmberDB?

AmberDB brings several capabilities together in a single Perl database engine:

• a simple Perl-native CRUD API
• schemaless and schema-driven data structures
• indexed matching and filtering
• full-text search
• sorting and faceted queries
• relational and repeated data blocks
• multi-table transactions
• concurrency control
• high-throughput batch operations
• portable database archives and restore tools

The result is a database where operations that often require additional layers, services, or complex query logic can be handled directly by the database itself.

For example, a Perl application can insert, read, modify, search, and delete records using a small and consistent API, while the schema controls validation and indexing behind the scenes.

AmberDB is built around a straightforward principle:

The database should make common application data operations simple, fast, and predictable.

A database operation can be this simple

After creating an AmberDB instance, inserting a record is straightforward:

use AmberDB;



my $db = AmberDB->new(
    cfg  => { user => 'admin', language => 'en' },
    path => { dbase_dir => './dbstore' }
);

my @product = (
    0,
    "Wireless Headphones",
    149.99,
    "Electronics",
    1
);

my $id = $db->insert_id(
    "products",   # table
    @product      # record
);

Reading the record back:

my @product = $db->read_id("products", $id);

Modify it:

$product[2] = 129.99;

$db->modify_id(
    "products",
    @product
);

And delete it:

$db->delete_id("products", $id);

That's the basic CRUD API.

There is no SQL query to construct, no ORM required, and no separate query language between the Perl application and the database.

For many applications, database access can therefore look very similar to ordinary Perl data manipulation.

Using schemas for advanced applications

AmberDB works even without a schema definition. However, schemas can significantly increase the engine's capabilities by defining the structure of a table and how its data should be indexed and searched.

There is no separate indexing system to configure. Once the schema is defined, AmberDB automatically builds and maintains the required indexes.

For example:

{
    name         => "Table Name",
    record_index => 1,              # readall
    search_block => [2,4,5,8],      # blocks to search
    match_block  => [1,3,8],        # blocks to match
    sort_block   => [1,3,5,7,10],   # blocks to sort
}

AmberDB automatically creates inverted indexes for the blocks defined by the table schema. These indexes provide very fast search and filtering operations, even with large datasets.

Indexed reads instead of scanning records

AmberDB is designed to perform much of the work when data is inserted or indexes are rebuilt, rather than repeatedly doing expensive work during queries.

This makes operations such as filtering, sorting, pagination, and searching large datasets possible without requiring an external search server.

For example:

my ($total, @products) = $db->search_table(
    "products",
    "wireless headphones",
    start => 0,
    limit => 20
);

The same database can therefore provide both ordinary record access and full-text search.

AmberDB does not join

A record can contain or be linked to relational data from other tables. For example, an order record might contain customer information, payment information, shipping and delivery information, various dates, and a collection of order items containing the product number, name, price at the time of sale, applied discount, and other information.

AmberDB processes these structures as part of the record model and uses custom indexes to make them easy to query.

This can be useful when the application's natural model is something like:

Order

 ├── Customer
 ├── Address
 ├── Dates [ as array ]
 ├── Payment
 └── Items
      ├── Product
      ├── Quantity
      └── Price

Instead of reconstructing the object from multiple tables through joins, the application can work with the structure directly, in a way that is much closer to Perl's own data model.

ACID Transactions

Although AmberDB is a performance-oriented NoSQL engine, it provides multi-table ACID transactions using an undo journal and Strict Two-Phase Locking (Strict 2PL).

AmberDB also supports concurrent access using OS-level file locking, making it suitable for multi-process Perl applications.

The transact API is simple to use, while the mechanisms behind it provide atomicity, consistency, isolation, and durability across related operations.

$db->transact_start();

# update order
# update customer account
# update stock
# insert payment record

$db->transact_end();

If an error occurs, the transaction can be rolled back, including the corresponding index changes.

The undo journal also provides recovery after an unexpected process or system failure.

Batch operations for large imports

Transactional record operations and bulk data loading are intentionally treated as different workloads.

For ETL, CSV imports, migrations, and large datasets, AmberDB provides batch methods such as:

$db->insert_list("products", @records);
$db->modify_list("products", @records);
$db->delete_list("products", @ids);

For example, when you insert a list of a thousand records using insert_list, AmberDB creates the records and their indexes as a batch operation.

Similarly, a large price list can be modified in a single batch operation.

This makes batch methods particularly useful for imports, migrations, synchronization jobs, and other high-volume data operations.

Portable backups

AmberDB also includes native database tools for backup and restore.

A database can be exported to a portable .amberdb archive containing the authoritative database state and schema. Integrity can be verified using SHA-256, and derived indexes can be rebuilt during restore.

This makes it possible to move or archive a complete database without requiring a separate database server.

Getting started

AmberDB 5.22.0 is available on CPAN:

cpanm AmberDB

The project is released under the Artistic License 2.0.

The source code, documentation, and examples are available on MetaCPAN and GitHub.

If you are a Perl developer working with a significant amount of structured data, give AmberDB a try.