The CSV Upload Adaptor allows data to be processed whenever a file is uploaded from the Data Stream page.
Below you will find details on how to configure this adaptor.
CSV Upload Adaptor
1. Select the CSV Upload adaptor and add sample data
Once you select the CSV Upload adaptor, the first step is to enter a sample of your data so that K3 can assist you in configuring the route. Click Add Sample Data to begin.
2. Upload or paste sample data
Sample data can be input by either uploading a file or pasting data into the box on the left.
3. Review uploaded data and determine separator
Once sample data is uploaded, K3 will attempt to determine the separator of your data. In the below picture, the right panel will shows how many fields were parsed for the following separators: comma (24 fields below) and tab (1 field below). K3 will highlight Configure Fields for the separator it believes should be used based on the data provided. If your data uses a custom separator, such as a | , click the Custom Separator link to input the custom separator.
At this stage, you can also tell K3 whether your data has headers by using the Header Row toggle at the top.
Click the CONFIGURE FIELDS button for the appropriate separator to move on to the next step.
4. Review fields names and field types
The Configure Fields step allows you to review the field names and field types that K3 has determined from your sample data. The first five rows of your data are shown to the right of each field to help guide you in making these decisions.
Note that K3 makes its best guess at the field types, so you might need to make some modifications based on your knowledge of the data.
The field name is only editable if you specified that your file does not contain a header row.
Once you are satisfied, click OK in the bottom right to move on.
| Configuration Parameter | Description |
|---|---|
| Field Name | This is the K3 field name corresponding to a column in your CSV. It will only be editable if you specify that your file does not contain a header. |
| Field Type | This is the K3 field type corresponding to a column in your CSV. K3 will recommend a field type, which you can modify. |
5. Date Formats and Finalizing Configuration
The final step is to determine Date/DateTime formats, which K3 uses to parse strings representing Date/DateTime values. K3 will attempt to suggest formats that are appropriate for your data, but you might need to provide custom formats by clicking + Add below the Date/Date Time sections. Keep in mind that multiple formats are allowed and K3 will attempt to parse any value in the order that the formats are specified.
If you find that Date/DateTime field values are missing in Data Stream, you might need to return to this page to review the formats specified below.
| Configuration Field | Value |
|---|---|
| Date | Enter the format of Date fields in the CSV so that K3 can parse it properly. Examples: yyyy-MM-dd -> 1978-05-28 yyyyMMdd -> 19780528 MM/dd/yyy -> 05/28/1978 |
| Date Time | Enter the format of DateTime fields in the CSV so that K3 can parse it properly. Examples: yyyy-MM-dd'T'HH:mm:ss.SSSZ -> 1978-05-28T09:07:21.201-07:00 yyyy-MM-dd'T'HH:mm:ss.SSS -> 1978-05-28T09:07:21.201 yyyy-MM-dd'T'HH:mm:ssZ -> 1978-05-28T09:07:21-07:00 |
6. Save the route
Click Create on the top right side to save the route configuration.
7. Automatic source field import
Once you create the route, you will be navigated to the "Source Fields" dialog with your fields ready to be created. Click Save to create the fields.
8. Apply and Restart
Once you make a change to the route, whether it is adding fields or mappings, you must click Apply and Restart on the top right side of K3. This process tells K3 that you are ready to use the new data, mappings and rules.
9. Uploading Data for Processing
Once an upload route is configured, you can upload data to that route in the Data Stream. Select Data Stream for the route you have created, then click the ellipsis on the right-hand side of the data table to upload your file.
How to Explicitly Declare .CSV headers
K3 may need to ingest a file which has no headers, or you may want to override the headers that are in a file.
The below property will allow you to explicitly set the headers in the route advanced config screen:
csv-headers [“A” “B” “C” … “N”]
This can also be useful when files have repeating headers, such as “A” “A” A”.
Such a scenario in K3 would require disambiguation of each “A” by declaring them as: csv-headers [“A1” “A2” “A3”]