File Connectors
      Back to home
      1. Seek Connect Help Center
      2. File Connectors

      Local CSV

      Local CSV

      DANGER

      This destination is meant to be used on a local workstation and won't work on Kubernetes

      Overview​

      This destination writes data to a directory on the local filesystem on the host running Airbyte. By default, data is written to /tmp/airbyte_local. To change this location, modify the LOCAL_ROOT environment variable for Airbyte.

      CAUTION

      Please make sure that Docker Desktop has access to /tmp (and /private on a MacOS, as /tmp has a symlink that points to /private. It will not work otherwise). You allow it with "File sharing" in Settings -> Resources -> File sharing -> add the one or two above folder and hit the "Apply & restart" button.

      Sync Overview​

      Output schema​

      Each stream will be output into its own file. Each file will contain 3 columns:

      • _airbyte_ab_id: a uuid assigned by Airbyte to each event that is processed.
      • _airbyte_emitted_at: a timestamp representing when the event was pulled from the data source.
      • _airbyte_data: a json blob representing with the event data.

      Features​

      Performance considerations​

      This integration will be constrained by the speed at which your filesystem accepts writes.

      Getting Started​

      The destination_path will always start with /local whether it is specified by the user or not. Any directory nesting within local will be mapped onto the local mount.

      By default, the LOCAL_ROOT env variable in the .env file is set /tmp/airbyte_local.

      The local mount is mounted by Docker onto LOCAL_ROOT. This means the /local is substituted by /tmp/airbyte_local by default.

      Example:​

      • If destination_path is set to /local/cars/models
      • the local mount is using the /tmp/airbyte_local default
      • then all data will be written to /tmp/airbyte_local/cars/models directory.

      Access Replicated Data Files​

      If your Airbyte instance is running on the same computer that you are navigating with, you can open your browser and enter file:///tmp/airbyte_local to look at the replicated data locally. If the first approach fails or if your Airbyte instance is running on a remote server, follow the following steps to access the replicated files:

      1. Access the scheduler container using docker exec -it airbyte-server bash
      2. Navigate to the default local mount using cd /tmp/airbyte_local
      3. Navigate to the replicated file directory you specified when you created the destination, using cd /{destination_path}
      4. List files containing the replicated data using ls
      5. Execute cat {filename} to display the data in a particular file

      You can also copy the output file to your host machine, the following command will copy the file to the current working directory you are using: 

      docker cp airbyte-server:/tmp/airbyte_local/{destination_path}/{filename}.csv .

      Note: If you are running Airbyte on Windows with Docker backed by WSL2, you have to use a similar step as above or refer to this link for an alternative approach.