Upload entries from CSV files.
On Epicollect5 It is possible to add entries to a project by uploading a
csvfile where each row is a valid entry for that project. This is particularly handy for those projects where the collection of data could be done by filling in a spreadsheet, for example.
Other use cases might be performing a bulk edit of existing entries or copying some entries from a master project to a cloned one.
To avoid any abuse of the bulk upload feature,
csvfiles are limited to 1MB in size, and bulk uploads are cut to 150 rows per file. This is done to avoid the creation of any redundant data on our systems.
Bulk uploads are available only for some question types, check the table below.
Media files cannot be uploaded in bulk.
By default, the bulk upload feature is disabled for a project. To enable it, go to your project details page and from there, Manage Entries > Bulk Uploads.
Three options are available:
- NOBODY: basically the bulk upload feature is disabled for all the users.
- MEMBERS: only project members (aside from VIEWER roles) can bulk upload entries.
- EVERYBODY: anyone logged in to Epicollect5.
For private projects, choosing either MEMBERS or EVERYBODY does not make any difference since only project members can access the project.
Moreover, VIEWER roles are never allowed to perform bulk uploads.
If bulk uploads are enabled and the user has permissions, a bulk upload button is shown at the bottom right when viewing the entries.
The bulk upload button can be disabled.
That means the user does not have the permissions to perform a bulk upload.
The button is also disabled when the currently selected view cannot accept bulk uploads directly, for example, a child form without any parent entry selected.
See Child form bulk upload for more info.
When viewing a parent form, press the upload button to open the bulk upload drawer.
Picking a valid file will trigger the upload, one row at a time. A valid
csvfile must have the proper columns to match the selected mapping. It is possible to download a blank template by clicking the download button of the "Current View Template" section in the upload drawer.
The csv file MUST have an extra
ec5_uuidcolumn, which is used as an internal control flag for entries.
EC5 Demo Project csv file example
Feedback is given on each row like so:
You can expand a row to see the details about any errors by clicking the down arrow on each row that contains errors.
It is possible to filter out uploaded entries to view only the failed one and download a
csvfile of only failed entries.
Complex projects with more than one single form consist of a hierarchy structure with a parent > child relationship across forms.
To upload entries for a child form, a parent entry must be chosen first.
When viewing UNIVERSITY entries, click on one of the numbers of the COURSE column, (the total number of child entries acts as a button) to view all the child entries for that UNIVERSITY entry.
All the entries uploaded will be linked to the parent "UCL" University entry.
All COURSE entries uploaded successfully
There are now 3 COURSE entries for "UCL"
The procedure to upload entries to a branch is very similar to uploading entries to a child form. The user needs to select what entry the uploaded branch entries will be linked to.
The csv file MUST have an extra
ec5_branch_uuidcolumn, which is used as an internal control flag for branch entries.
Add "Player" branch entries to "Juventus"
Open drawer and pick file
Csv file for branch entries
All branch entries uploaded
"Juventus" has now 4 "Player" branch entries
By using the bulk upload feature it is possible to perform bulk edits on the entries already uploaded. This can be done by downloading the entries which need to be amended, perform the edits with your favorite text editor (like Excel or Google Sheets), then re-upload the entries.
Each entry created on the Epicollect5 platform is given a universal identifier, a uuid, like
87dc71ea-1323-47ae-8acb-6002c66f08fd. When entries are downloaded, all the identifiers are added to the exported dataset along with other metadata (See metadata). When uploading entries in bulk, by providing these identifiers in the
csvfile, the system will look for a match. If a match is found and the user has the permissions to edit that entry, the existing entry will be updated.
For example, we have a project called People, with a single form PERSON asking just for the person's name. We have just a few entries and we would like to replace all the "Johnny" with "John".
Both "Johnny" must become "John"
We open the bulk upload drawer and we click on the "Current View Subset" download button to download all the entries in view.
Get all the entries in view
The downloaded file will look like below. Please note the ec5_uuid column has all system identifiers of those entries.
After the edits, it will look like below. We have not touched any of the metadata columns.
Changed only data in the "1_name" column
Uploading the edited file will update the existing entries. Please note the double tick icon to indicate an edit. (A new entry would get only a single tick).
Entries to be edited in bulk can be filtered beforehand by using the dataviewer filter controls. For example, we can filter entries by name, like "Richard".
We have only one entry matching the title "Richard"
If we get the "Current View Subset"
csvfile, it will contain only one entry, "Richard".
We can change "Richard" to "Rick" and upload the same file to perform the edit. We leave the metadata columns as they are.
The procedure to bulk edit entries of a child form is almost identical to the one used to edit entries of a parent form. The only extra step is selecting the correct parent entry.
Select "Kingston University" child entries
Download entries in view
Entries before the edits
Entries after the edits, changed all "2_Code" values
COURSE child entries updated
Selecting the correct parent entry is crucial.
If we try to upload the same file to another UNIVERSITY entry, like "Oxford University", an error will be shown because the child entries are linked to the "Kingston University" entry, see below.
Selecting the wrong parent entry, the hierarchy relationship is incorrect
The procedure to bulk edit entries of a branch is almost identical to the one used to edit entries of a parent or child form. The only extra step is selecting the correct entry which contains the branch entries that need to be amended.
Get "Juventus" branch entries
Download branch entries in view
Branch entries before the edits
Branch entries after the edit
Selecting the correct entry is crucial.
If we try to upload the same file to another TEAM entry, like "Barcelona", an error will be shown because the branch entries are linked to the "Juventus" entry, see below.
Selecting the wrong entry, the branch relationship is incorrect
The Current View Template file is an empty csv file with the correct headers for the currently selected form or branch. Useful for collecting new entries from scratch.
The Current View Subset file is a csv file that contains all the entries and their metadata (like
uuidsystem identifiers) for the currently selected form, child form, or branch. Useful for editing entries in bulk.
It is possible to use the dataviewer to filter out entries by title and creation date. The downloaded subset file will contain only the matching entries.
Answers to LOCATION questions can be uploaded in bulk only in signed degrees format. Epicollect5 splits answers to LOCATION question on both the mobile and the web app in three separate columns:
- altitude (prefixed
- longitude (prefixed
- and accuracy (prefixed
Using the bulk upload feature it is possible to copy some entries from an original project to its clone.
It is not possible to copy media files, only text-based data
Moreover, entries can be copied up to 150 at a time given the file limitations of bulk uploads. These limitations are in place to avoid any abuse of our systems and the creation of redundant data.
When entries get downloaded from Epicollect5, the
csvfile will contain the
uuididentifiers of those entries,
ec5_branch_uuidfor branch entries
Entries downloaded have uuid identifiers
To upload those entries to a cloned project as new entries, the identifiers need to be cleared otherwise they will clash with the existing entries already in the system.
Empty ec5_uuid column for new entries
Empty ec5_branch_uuid column for new branch entries
Once the identifiers are cleared the entries can be uploaded to the cloned project.
Trying to bulk upload entries with identifiers (
uuid) already on the system will give an error.
Sometimes, depending on the mapping you are using, you might find issues when importing entries with valid values.
We highly recommend the use of a simple mapping for exporting and importing. Values that can cause troubles usually contains commas, double quotes or special symbols.