Feed a database from an external database
Connect a Filarr database to D1, PostgreSQL, MySQL, Supabase, Airtable, Google Sheets, Notion or a CSV: the four directions, who runs it, what you see.
En esta página
Secciones de la ayuda
A Filarr database can be fed by an external database (your shop's orders table, a Google sheet, a Notion database), feed it, or both. The external database's key stays with you. From the desktop app or your gate, the key and the rows go straight from the source to you, without going through Filarr's servers. From the web, they go through Filarr's relay, which keeps nothing: that leg is not end-to-end encrypted.
On the database, click ··· then Feed from an external database…. Once the sync is set, the same item becomes Edit the sync with the external database…. It does not exist for a database backed by a folder, nor in a read-only note.
Plans
- A one-off import from the desktop app is offered on every plan.
- A scheduled sync (mirror, publishing, both ways) is offered from Solo.
- On the web, everything goes through Filarr's relay, which keeps nothing: the item only shows from Solo, one-off import included.
A scheduled sync needs a Filarr account signed in on this profile: the database then moves to Filarr's large database storage, where the sync's definition lives. The current mobile app does not open a database kept in that storage yet: it shows a newer-version notice in its place. Edit it from the desktop app or the web.
The four directions
| direction | what happens |
|---|---|
| Once | the rows come in once; the database is then yours, with no link to the source |
| Incoming mirror | the source is the authority: each run brings its changes; the columns it fills are locked in Filarr |
| Publish to the source | Filarr is the authority: your changes go to the source. It needs a key that can write |
| Both ways | either side can change a cell; a conflict follows the rule you choose (Sync a database both ways) |
A source read through an SQL query only works with Once and Incoming mirror. So does a published CSV or JSON.
The three steps
1. Connector and key
Give a Name of the sync, choose where the rows come from, fill in the connector's fields and its key (see External database connectors to create a limited key). The key joins this profile's encrypted keychain (Settings → Integrations → Keychain), bound to the source's host: it is never sent to another address.
Test the connection lists the tables the key can read. From the desktop app, the test runs from your device; from the web, it goes through Filarr's relay. If the source cannot be read from here, Describe the columns by hand lets you go on: the runner checks them on its first run.
2. Columns
- Row key: the column that identifies a row, such as
id. It finds the same row from one run to the next; two devices importing the same source make no duplicates. Without a row key, only a one-off import is possible. - Change marker: a column updated at each change, such as
updated_at(a date, or an increasing number). It lets a run read only what changed. Without a marker, each run reads everything. - Each source column goes to a Property in Filarr, existing or new, or stays ignored: an ignored column is not even asked from the source. The Filarr properties you do not map stay yours: never overwritten, never sent.
A relation column of the source (Notion, Airtable linked records) cannot be imported yet: leave it ignored.
3. Direction and frequency
- Which direction?: one of the four directions above.
- Who does the work?:
- This device: only while Filarr is open on it; a missed run is caught up at the next opening.
- My gate "…": always, even at night. It also reaches a database on your local network, which Filarr cannot. You first need to open this database to an API in Read and write, then give the source's key to the gate, on your side.
- The A box hosted by Filarr choice is shown, but not available yet: the hosted gate is open, but it does not run external database syncs yet.
- When?: every 15 min, every hour, every day, or on demand.
- A row disappears from the source: Mark it "gone" (struck through, your columns kept), Delete it in Filarr, or Keep it as is. When the sync follows a marker, a deletion in the source only shows at the full read, done about once a day (at the first run after 24 hours, or every 96 runs).
- A column appears in the source: Suggest it to me, or Add it, held by the source.
- For publishing and both ways, A row deleted in Filarr: Leave it in the source, or Delete it in the source too.
Click Import now (once) or Start the first sync.
The sync's definition lives in the database, encrypted like it, and never holds a key. It is signed by your account: if another member changes it, it stops running until you click Approve the change.
What you see in the database
- At the top: Fed by the connector, by the runner, then the state: "5 min ago · next run at 14:30", or what blocks it. Run now asks for a run right away.
- A column filled by the source carries a padlock: ""Status" comes from D1: change it in D1, or make it a column of your own." To take it back, edit the sync and leave its source column ignored.
- A mirrored database says "The rows come from D1" instead of New row.
- A row gone from the source is struck through, with "Gone from D1".
- Both ways, "Ask me" conflicts wait: "3 conflicts to settle. These cells keep their Filarr value until you decide; everything else keeps syncing."
Every member of the database sees the state, but never the key. Settings → API access → Syncs lists all your fed databases: the source, the direction, the frequency, who runs it, the last run and the state.
Safety stops
- If more than 20% of the rows disappear at once, or if the source suddenly returns zero rows, the run stops before writing anything: "Safety stop: 1,102 rows out of 1,284 disappeared from D1 at once. The sync is stopped. Nothing was marked or deleted in Filarr." Check the source: once its rows are back, the next run goes on normally.
- Too many conflicts at once also stop the run: check the chosen direction and the row key.
- A gate's writes into Filarr count against your plan's write limit; when it is used up, the changes wait until midnight UTC, nothing is lost.
Detach
Detach the database… stops the sync: the values already in Filarr stay and the columns become yours again. The key stays in the keychain.
Learn more
- Sync a database both ways
- External database connectors
- The gate's D1 and PostgreSQL tutorials: github.com/filarr-work/filarr-gate.