> For the complete documentation index, see [llms.txt](https://buildsystem.eintosti.de/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://buildsystem.eintosti.de/getting-started/storage.md).

# Data Storage

Keep worlds, folders and players in YAML, SQLite, MySQL, MariaDB, PostgreSQL or MongoDB.

BuildSystem keeps three kinds of data in its storage: the world registry, folders, and player settings. Since 4.1.0 that storage can be YAML files or a database.

Some things never move, whatever you pick:

* World folders stay on disk.
* `statuses.yml`, `categories.yml`, `setup.yml` and `spawn.yml` stay YAML files, because they are server configuration.

***

## Storage Types

Set the type with `storage.type` in `config.yml`.

| Type         | Where the data is kept                                                     |
| ------------ | -------------------------------------------------------------------------- |
| `yaml`       | `worlds.yml`, `folders.yml` and `players.yml` in the plugin folder.        |
| `sqlite`     | `storage.db` in the plugin folder. The default on new installs.            |
| `mysql`      | A MySQL server, set in `storage.mysql`. Needs MySQL 5.7.8 or newer.        |
| `mariadb`    | A MariaDB server, set in `storage.mariadb`. Needs MariaDB 10.2.7 or newer. |
| `postgresql` | A PostgreSQL server, set in `storage.postgresql`.                          |
| `mongodb`    | A MongoDB server, set in `storage.mongodb`.                                |

A server upgraded from 4.0 keeps `yaml`. The config migration to version 7 picks `yaml` when `worlds.yml` or `players.yml` already exists, and `sqlite` otherwise.

The PostgreSQL and MongoDB drivers are part of the plugin jar, so nothing is downloaded at startup. SQLite and the MySQL driver, which is also used for MariaDB, come from the server.

***

## Configuration

```yaml
storage:
  type: sqlite
  table-prefix: buildsystem_
  mysql:
    host: localhost
    port: 3306
    database: buildsystem
    username: ''
    password: ''
    tls: prefer
  mariadb:
    host: localhost
    port: 3306
    database: buildsystem
    username: ''
    password: ''
    tls: prefer
  postgresql:
    host: localhost
    port: 5432
    database: buildsystem
    username: ''
    password: ''
    tls: prefer
  mongodb:
    host: localhost
    port: 27017
    database: buildsystem
    username: ''
    password: ''
    tls: prefer
```

The `s3` and `sftp` blocks under `storage` are for backups and downloads and have nothing to do with this. See [External Storage](/reference/config.md#external-storage-storage).

* **`table-prefix`**: Put in front of every table, view and collection name, so the tables are `buildsystem_worlds`, `buildsystem_folders` and `buildsystem_players`, plus `buildsystem_meta`, which records the format the data is in. Lowercase letters, digits and underscores, at most 32, not starting with a digit.
* **`host`**: A host name, an IPv4 address, or an IPv6 address in brackets.
* **`database`**: Letters, digits, `-` and `_`, at most 64.
* **`username`**: Required for MySQL, MariaDB and PostgreSQL. For MongoDB, leave it empty for a server without authentication. MongoDB users are authenticated against the `admin` database.
* **`password`**: Can also be set as an environment variable named after the block, which takes precedence and keeps the secret out of `config.yml`: `BUILDSYSTEM_MYSQL_PASSWORD`, `BUILDSYSTEM_MARIADB_PASSWORD`, `BUILDSYSTEM_POSTGRESQL_PASSWORD` or `BUILDSYSTEM_MONGODB_PASSWORD`.

Only the block of the type in use is checked at startup. A mistake in another block shows up when that block is used, for example as a migration target.

{% hint style="danger" %}
**Every server needs its own prefix or its own database.** Each server loads the data once on startup and keeps it in memory, so two servers sharing a prefix overwrite each other's worlds, folders and players.
{% endhint %}

### Encryption (`tls`)

Each database block has a `tls` setting:

| Value     | Meaning                                                      |
| --------- | ------------------------------------------------------------ |
| `prefer`  | Encrypt when the server offers it. The default.              |
| `off`     | Never encrypt.                                               |
| `require` | Always encrypt, without checking the certificate.            |
| `verify`  | Always encrypt, and check the certificate and the host name. |

`verify` trusts what the Java trust store trusts, so a private CA has to be added to it. For PostgreSQL this means `~/.postgresql/root.crt` is not used.

MongoDB behaves a little differently. It cannot fall back to plain text, so `prefer` does what `off` does. `require` still checks the certificate, only not the host name. With `verify`, give MongoDB a host name rather than an IP address.

***

## Switching to a Database

There are two ways to switch.

### With a command, while the server runs

```
/buildsystem storage migrate <type>
/buildsystem storage migrate <type> confirm
```

* **Permission**: `buildsystem.storage.migrate` (Default: `OP`). Also works from the console.

Fill in the target's block in `config.yml` first. The command reads it from the file, so there is no need to reload.

Without `confirm`, the command only checks. It connects to the target and shows how many worlds, folders and players would be copied. It also warns when the target already holds data for this server, which the copy replaces. It creates nothing.

With `confirm`, it copies everything, reads the target back to check the copy, and switches. It then writes the new `storage.type` to `config.yml`. No restart is needed. Saves made during the copy end up in the target. If anything fails, nothing is switched, `config.yml` is left alone, any tables the migration created are dropped again, and the reason is shown (`storage_migrate_failed`).

Entries that are stored but not loaded, such as a world whose folder failed to load, are copied as they are. The check says how many there are (`storage_migrate_not_loaded`).

The command refuses to go ahead when:

* **Some stored entries cannot be read**, for example a row with broken data. They would be lost in the copy, so the check and the migration both stop with `storage_migrate_unreadable`. The server log names them. Fix or delete them first.
* **The target is the storage in use**: the same type (`storage_already_in_use`), or a block that points at the same host, port, database and prefix. MySQL and MariaDB have the same default block, so migrating between them needs a different database or prefix. `localhost`, `127.0.0.1` and `[::1]` count as the same host.
* **The target's block is missing or invalid**, or the server cannot be reached. For example, MySQL, MariaDB and PostgreSQL refuse an empty `username`.
* **Another migration is running** (`storage_migrate_running`).

Migrating to `yaml` writes the current data into `worlds.yml`, `folders.yml` and `players.yml`.

### By editing `config.yml`

Stop the server, set `storage.type`, and start it again. On that start, each table or collection that does not exist yet is created and filled from `worlds.yml`, `folders.yml` and `players.yml`. The console says how many entries came from each file. A 3.x file is converted first, as a YAML load would.

The import only happens once. A table or collection that already exists is never imported into again, even when it is empty, so deleted entries stay deleted.

{% hint style="warning" %}
Editing `storage.type` only brings data along from the YAML files. To move from one database to another, for example from the default SQLite to PostgreSQL, use `/buildsystem storage migrate`. Editing the type and restarting would start with empty tables, because the data in `storage.db` is not copied.
{% endhint %}

***

## Switching Back to YAML

The YAML files are never deleted. Once a database is in use they are no longer updated, so they keep the data as it was at the switch.

* Setting `storage.type: yaml` and restarting loads those files as they are, without anything added since the switch.
* `/buildsystem storage migrate yaml` writes the current data into them first.

***

## Reading the Data

Each table has an `id` column and a `data` column holding the entry as JSON. That JSON is BuildSystem's own format and can change between versions, so for queries use the views below instead. BuildSystem creates them on SQLite, MySQL, MariaDB and PostgreSQL and rebuilds them on every start to match the current format.

| View                         | Columns                                                                                                                          |
| ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `buildsystem_worlds_view`    | `id`, `name`, `type`, `status`, `visibility`, `project`, `permission`, `creator_id`, `creator_name`, `created_at`, `last_edited` |
| `buildsystem_world_builders` | `world_id`, `builder_id`, `builder_name`                                                                                         |
| `buildsystem_folders_view`   | `id`, `name`, `category`, `parent_id`, `project`, `permission`, `creator_id`, `creator_name`, `created_at`                       |
| `buildsystem_folder_worlds`  | `folder_id`, `world_id`                                                                                                          |

Times are milliseconds since 1970. For example, the worlds Sam can build in:

```sql
SELECT w.name
FROM buildsystem_worlds_view w
JOIN buildsystem_world_builders b ON b.world_id = w.id
WHERE b.builder_name = 'Sam';
```

{% hint style="warning" %}
**Read only.** BuildSystem reads the data once at startup and keeps it in memory, so anything written into these tables is ignored and then overwritten by the next save.
{% endhint %}

New worlds and folders are saved at once, other changes to them within five seconds, and player data within a minute. A world or folder last saved before 4.1 has no creator or builders in the views until it is saved again, which happens a few seconds into the first start on 4.1 for every entry that loads.

Creating a view needs the `CREATE VIEW` privilege, and the builder and folder views need MySQL 8.0 or MariaDB 10.6 or newer. A view that cannot be created is logged as a warning and BuildSystem runs as normal. MongoDB has no views: its documents have the same fields as the JSON.

***

## When Something Goes Wrong

BuildSystem never falls back to YAML on its own. It stops at startup, with the reason in the console, when:

* `storage.type` is unknown;
* the block for the type in use is invalid;
* the database cannot be reached;
* a YAML file that is being imported cannot be read;
* the database was written by a newer version of BuildSystem, whose format this version may not understand.

If SQLite cannot load its native library, the error says what to do: set `storage.type` to `yaml`, or start the server with `-Djava.io.tmpdir=<folder>` or `-Dorg.sqlite.tmpdir=<folder>`, pointing at a folder that allows native code.

A remote database gets 30 seconds to answer a read. On shutdown BuildSystem waits at most 90 seconds for the last saves and logs any that did not finish, so a database that stops answering cannot hang `/stop`.

If an entry cannot be read when it is loaded, it is skipped and logged, and later saves do not overwrite it. If a whole table, collection or YAML file fails to load, BuildSystem refuses every save to it until it loads again. That way a player who joins in the meantime cannot replace their stored settings with the defaults.