In this tutorial I am going to show you how to read a local CSV file using JavaScript and parse it with the Papa Parse library. In case you are interested in a working example then have a look at heatmap.joyofdata.de for which you will find detailed description here.
For whatever reason JavaScript developed into an awesome tool for breathing life into data. But before you can visualize data, you have to read and parse it. JSON is with JavaScript a cinch anyway – and CSVs are now a cinch, too, thanks to Papa Parse – yes, I kind of don’t like the name as well, but let’s be pragmatic. Thanks to HTML5’s rather new File API it became possible to read locally stored files with JavaScript from a browser. Due to security concerns this file access is bound to files that have been opened manually by the user though. That means it is not (yet) possible to simply read arbitrary files from a folder.
Few lines of code say more than a thousand words
To get started we need a button to open a file:
|
1 |
<input type="file" id="csv-file" name="files"/> |
Of course the Papa Parse JavaScript library …
|
1 2 |
<script src="./papaparse.min.js"></script> <input type="file" id="csv-file" name="files"/> |
… and jQuery for the event handling:
|
1 2 3 4 5 6 7 8 9 10 11 12 |
<script src="./papaparse.min.js"></script> <script src="./jquery-2.1.1.min.js"></script> <script> function handleFileSelect(evt) { // ... } $(document).ready(function(){ $("#csv-file").change(handleFileSelect); }); </script> <input type="file" id="csv-file" name="files"/> |
A line to give us the File object:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
<script src="./papaparse.min.js"></script> <script src="./jquery-2.1.1.min.js"></script> <script> function handleFileSelect(evt) { var file = evt.target.files[0]; // ... } $(document).ready(function(){ $("#csv-file").change(handleFileSelect); }); </script> <input type="file" id="csv-file" name="files"/> |
And finally Papa Parse to – you guessed it – parse the selected file:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
<script src="./papaparse.min.js"></script> <script src="./jquery-2.1.1.min.js"></script> <script> var data; function handleFileSelect(evt) { var file = evt.target.files[0]; Papa.parse(file, { header: true, dynamicTyping: true, complete: function(results) { data = results; } }); } $(document).ready(function(){ $("#csv-file").change(handleFileSelect); }); </script> <input type="file" id="csv-file" name="files"/> |
Papa Parse will deduce itself what separator was used – in case its heuristics fail you can help out with delimiter:";" . header: true makes sure the first line of the file is interpreted as the header and dynamicTyping: true will have “true” and “false” be typed as booleans and numeric values like “3.1415” be typed as float for example. Otherwise all fields would be typed as strings.
And here’s what it looks like in action
Let’s read the following CSV and inspect data :
|
1 2 3 4 5 6 |
city;lat;lon;population;isCapital "Tel Aviv";32.091225;34.781453;1318300;false "Jerusalem";31.762542;35.220906;780517;true "Gaza City";31.5225606;34.453593;515556;false "Haifa"; 32.769212; 35.004862;272181;false "Hebron";31.5325686;35.099826;250000;false |
Thsi is what data looks like from the developer tools (F12):
(Pretty) Big Data is no problem for Papa Parse
Papa Parse won’t have any problems reading CSVs containing several million records – yes, I tried it! It even allows you to have a file read within a separate thread to keep your web application from freezing for a while. And in case your file is SO big it won’t fit into memory you can stream its content – that means reading it in chunks. Ain’t that something? A succinct documentation on further features – like error handling f.x. – you’re going to find here.
(original article published on www.joyofdata.de)
Raffael,
This is an awesome tutorial! thank you. the only thing I was wondering is how you went about calling these functions. Do you have a submit button somewhere that calls these functions?
Thanks
Devin
What a nice clean tutorial! Many thanks.
I experienced a similar issue in Chrome, and I think what may have happened is that the script was run, and then the user opened up the console. When I loaded the page, then opened the console, then ran the script using a different user, I did not experience the issue.
Hi,
Thanks for the nice tutorial!
I tried this off and it’s really cool.
The only problem is that I have to load the file twice to get a proper data through parsing, since if I want to log out the contents of the “data” object, the console says it’s undefined.
Any ideas?
Thank you once more for this great example!
Greetings,
Kamfor
Hi Kamfor,
thanks, you’re most welcome.
To be honest, I don’t understand your problem really. There should be no reason why you would have to load a file twice – either it works the first time or never.
Please post a reproducible question featuring simplified and commented code on stackoverflow.com. I will have a look at it then.
Greetings
Raffael