Feature Request: Allow custom base data dir for SQLite and NATS Server, via CLI or ENV

[gedw99]

Hey @panthyy

Really cool project .. Everything works for me.

$(MAKE) clean install build

Would you mind enabling the default base data dir to be changed via the CLI or Env ?

At the moment the binary creates and uses the "flat_data" directory and stores the Sqlite DB and NATS Server there in a sub directory of where the binary is started from.

My use cases is to manage data volumes with a bit more control.

Also a dev may already have a NATS Server somewhere, and so they might want to use that ? Or maybe you're thinking that each foot instance is a Leaf Node ?

Anyway, for now, could we get some control of the placement of the DB and NATS Server data

Looks like the activation is here, https://github.com/FlotHQ/flot/blob/main/flot.go#L49, but does not seem to be exposed to the CLI or ENV ?

flot serve [domain(s)] [flags]

Flags:
  -h, --help              help for serve
      --http string       TCP address to listen for the HTTP server
                          (if domain args are specified - default to 0.0.0.0:80, otherwise - default to 127.0.0.1:8090)
      --https string      TCP address to listen for the HTTPS server
                          (if domain args are specified - default to 0.0.0.0:443, otherwise - default to empty string, aka. no TLS)
                          The incoming HTTP traffic also will be auto redirected to the HTTPS version
      --origins strings   CORS allowed domain origins list (default [*])

[bytekai]

Hello @gedw99
Those are very good points, we definitively want to expose some way of deciding which directory the data is stored in.

Im imagining something like this

flot serve --data-dir /var/lib/flot-data
flot serve --db-data-dir /var/lib/flot-db
flot serve --nats-data-dir /var/lib/flot-nats
flot serve --nats-data-dir /var/lib/flot-nats --db-data-dir /var/lib/flot-db

the default location would be $CWD/flot-data/x

I also want to keep it flexible and allow an external NATS server to be provided and used. As you also mentioned we could also treat every instance as a leaf node with the embedded NATS server. the only issue I see with this option is, that we also have an SQLite driver running, to solve this we could make sqlite distributed but I would rather design the queries and schema to not use db-specific features giving us the ability to just switch the connection and use something like PostgreSQL.

also introducing another set of flags:

flot serve --nats-url
flot serve --db-url

All these configurations would also be available as enviroment variables under

FLOT_DATA_DIR=/var/lib/flot-data
FLOT_DB_DATA_DIR=/var/lib/flot-db
FLOT_NATS_DATA_DIR=/var/lib/flot-nats
FLOT_NATS_URL=nats://127.0.0.1:1222, nats://127.0.0.1:1223, nats://127.0.0.1:1224
FLOT_DB_URL=postgresql://localhost:5432

the whole philosophy I want to adopt for this project is to keep everything within one executable with escape hatches.

what do you think?

[gedw99]

That’s exactly how I would do it too !