How can I parse a csv in low memory, using some parser in Java?

59
September 13, 2017, at 10:06 PM

I used InputStream, and on parsing, if there is a "," in one column then it considers it as a separate column. ex - abc, xyz, "m,n" then the parsed output is abc , xyz, m, n Here m and n are considered as separate columns.

Answer 1

I really like the Apache Commons CSVParser. This is almost verbatim from their user guide:

Reader reader = new FileReader("input.csv");
final CSVParser parser = new CSVParser(reader, CSVFormat.DEFAULT);
try {
    for (final CSVRecord record : parser) {
        final String string = record.get("SomeColumn");
        ...
    }
} finally {
    parser.close();
    reader.close();
}

This is simple, configurable and line-oriented.

You could configure it like this:

final CSVParser parser = new CSVParser(reader, CSVFormat.DEFAULT.withHeader().withDelimiter(';'));

For the record, this configuration is unnecessary, as the CSVFormat.DEFAULT works exactly the way you want it to.

This would be my first attempt to see whether it fits into the memory. If it doesn't, can you be a little more specific about low memory footprint?

Answer 2

There are many thirdParty Csv parsing library like

  1. UniVocity Parser

  2. CommonsCsv Parser

  3. OpenCsv Parser

  4. SuperCsv Parser

I am using UniVocity csv parser which is very fast and automatically detect separator in rows. You can go through above given csv libraries.

READ ALSO
Spring JdbcTemplate “insert into.. select from..” not working

Spring JdbcTemplate “insert into.. select from..” not working

I am trying to execute following SQL with Spring JdbcTemplate:

62
@RolesAllowed has no effect on my RESTful Service

@RolesAllowed has no effect on my RESTful Service

I'm having some trouble with setting up my secure REST-ServiceI wanted to create a simple login/logout service and mess around with it

53
How can I examine external source code for incidence of dead code? [on hold]

How can I examine external source code for incidence of dead code? [on hold]

Recently I received JAVA application to write for my university classesThis is the task:

72
Add object to ArrayList at specified index

Add object to ArrayList at specified index

I think it's a fairly simple question, but I can't figure out how to do this properly

63