FTP Data Ingestion¶
Improvado can import your data from a FTP server. We can use your FTP server instance or provide one for free if you prefer.
Requirements and preconditions¶
- Data should be in one of the following formats: tsv, csv, xls(x) files or its compressed versions (zip, gzip, tar.gz archives)- Data set should be presented as a flat table (no tabulation, no data hierarchy, no images, column names in the first row, one table per file)- Data structure (field names, field order and data types) should NOT be changed without warning- Your data provider or your own software is able to send reports via FTP- It must happen automatically and regularly
How to setup a FTP data ingestion¶
Step 1. Contact our technical support and tell us what you want:- Which data source do you need to connect?- An example of your data frame is very helpful (for uncommon report types or data formats it is necessary)- How often should this data be refreshed (typically - on a daily basis)?- Update policy: should we insert, update or replace data- Explain FTP server’s folder structure- Specify type of FTP connection needed: FTP, SFTP, FTPSStep 2. We will confirm whether we can ingest your data or what needs to be changed in order to make it possible.Step 3. Setup access:- We can provide you access to our FTP-server with two possible options: SFTP or FTPS- To setup SFTP we need your public SSH-key- In case of FTPS we will create a login and password for you- We can load data from you FTP-server- For SFTP connections we will provide our public SSH-key after your request- In case of FTP/FTPS we will require a login/password from youStep 4. Send test data with credentials given.Step 5. We will confirm whether we have successfully loaded the data.Step 6. Schedule reports from your data platform.
In addition to the above we can offer the following additional options:
- One-time historical data download or reference data download if required for custom reports or data transformations- For unstructured or atypical data formats (examples of data set needed), we offer a client data analysis in order to select the best data loading method.- Data update frequency changes (if necessary)- Report format or data structure update (if data source changes the set of fields available)