Add an header line assertion/check #151

Activity

1. osiegmar commented on Jul 31, 2025

   

   osiegmar

   on Jul 31, 2025

   Owner

   More actions

   There are several ways to assert the header line of a CSV file without having to check all records.

   Depending on your needs, you could use an enhanced for-each loop:

   boolean first = true;
   try (var csvReader = CsvReader.builder().ofNamedCsvRecord(file)) {
       for (NamedCsvRecord rec : csvReader) {
           if (first) {
               first = false;
               validateHeader(rec.getHeader());
           }
   
           // process the record ...
       }
   }

   Or, when preferring a stream-based approach, you can use a Gatherer (in Java >= 24) to validate the header only once before processing the records:

   Gatherer<NamedCsvRecord, AtomicBoolean, Object> validator = Gatherer.ofSequential(
       () -> new AtomicBoolean(true),
       (first, rec, downstream) -> {
           if (first.getAndSet(false)) {
               validateHeader(rec.getHeader());
           }
           return downstream.push(rec);
       }
   );
   
   try (var csvStream = CsvReader.builder().ofNamedCsvRecord(file).stream()) {
       csvStream
           .gather(validator)
           .forEach(rec ->
               // process the record ...
           );
   }

   Alternatively, you can access the header directly without managing a "special" first state.
   An iterator-based approach works well for this:

   var handler = NamedCsvRecordHandler.of(builder ->
       builder.returnHeader(true)
   );
   
   try (var iterator = CsvReader.builder().build(handler, file).iterator()) {
       if (iterator.hasNext()) {
           validateHeader(iterator.next().getFields());
       }
   
       iterator.forEachRemaining(rec ->
           // process the record ...
       );
   }

   Combinations are also possible, as you could start with an iterator to validate the header and then switch to a stream.

   All approaches are sharing this validation method:

   static void validateHeader(final List<String> header) {
       var expectedHeader = List.of("header A", "header B");
   
       if (!expectedHeader.equals(header)) {
           throw new IllegalStateException("Header mismatch: expected %s but found %s"
               .formatted(expectedHeader, header));
       }
   }

   ---

   Adding an expectedHeader method to NamedCsvRecordHandler would certainly make header validation easier, but it is narrowly focused on that specific task. Some users may want to ignore the order of header fields or validate only certain required fields. Therefore, a more flexible approach, such as a validateHeader method, could be more useful in the long term.

   var handler = NamedCsvRecordHandler.of(builder ->
       builder.validateHeader(header -> {
           if (!header.contains("header A")) {
               throw new IllegalStateException("Header must contain at least 'header A'");
           }
       })
   );

   A few predefined Validators could be provided to allow most common use cases.

   If you want, you can create a PR for further discussion and implementation of this feature.
2. fbf-prog64 commented on Jul 6, 2026

   

   fbf-prog64

   on Jul 6, 2026

   More actions

   Hello @osiegmar! 👋

   I gave this issue a try and have some code I would like you to review, but after I completed everything, I saw that this project does not accept PRs. 😞

   How can I share with you the modifications done so that you can take a look at them?

   You can find them here if you are interested: https://github.com/fbf-prog64/FastCSV/commit/88b7d347012e18721b76b3e974e866583ab5b998

   Thank you! 👍
3. osiegmar commented on Jul 6, 2026

   

   osiegmar

   on Jul 6, 2026

   Owner

   More actions

   Thanks for the implementation and the link!

   Just a heads-up: I've since disabled PRs on the repo. Most PRs touch trade-offs that are central to FastCSV's design goals (performance, small surface area, RFC compliance), so I've found it works better to discuss the need here in the issue and do the implementation myself. Nothing against your code — just how I prefer to steer these decisions.

   A few thoughts on the commit:

   • Moving checkForDuplicates into Util isn't needed — Util is internal (not exported via JPMS), so there's nothing to reuse it from, and it just widens the diff.
   • The feature itself doesn't require the surrounding refactor (swapping the boolean for a composed Function, rewiring the constructor). The existing private validateHeader(...) is already the single choke point both call sites go through, so a hook could be added there with a very small change.

   The bigger open question for me is the shape of the API. A generic validateHeader(Consumer<...>) where you signal failure by throwing is quite low-level and pushes the comparison and error message onto every user. The declarative expectedHeader("A", "B") from the original request reads more in line with the library's simplicity goal. What's the concrete use case — a strict "must match exactly" assertion, or "these columns must be present"? That helps decide whether this belongs in the library at all (it's currently a one-liner against record.getHeader()) or what form it should take.
4. fbf-prog64 commented on Jul 6, 2026

   

   fbf-prog64

   on Jul 6, 2026

   Last edited by fbf-prog64

   More actions

   Thank you for checking the work out! 😊

   I needed to refactor because it wouldn't let me compile with 8 parameters in the NamedCsvRecordHandler constructor (linter error as if it was SonarLint).

   So then I thought about simplifying the constructor by replacing the duplicate columns boolean with a more generic validation method; after all, as long as the Builder is used to create the objects, that logic can be extracted out and passed to the handler without worrying about nullabilty. Furthermore, this allows the main validation method to be "extended" with the user provided one, if given. Sadly, this also required to move out the duplicate columns method, and so I put it in the Util class.

   As for the validation method itself, I also followed the convention with the existing methods inside the handler (before extracting them out), which throw an exception if the headers are null or if, optionally, they contain duplicate columns. I also thought about using a Predicate to avoid expecting the user throw exceptions, but this would have required aligning both the existing and the new validation methods, so I preferred sticking to the convention.

   Finally, about the actual scope of the validation, I'm fine with whichever method ends up being implemented. I honestly wanted to dabble at the issue and produce some code that would be useful for both the project (to get the functionality) and myself (to continue learning and sharpen my skills), so if the decision is to rather limit changes to keep them controlled and minimal, there is no problem with that.

   Personally, though, I find your previous idea of validateHeader with Validator presets as the perfect balance between extensibility and impact in refactoring, and I don't think it makes the project any more complex than it actually is. Moreover, it's backwards-compatible with existing code as it's optional.

   Thank you again for taking the time to check the code! 👍
5. osiegmar

   added a commit that references this issue on Jul 8, 2026

   feat: add header validation support to NamedCsvRecordHandler (#151)

   ...

   e50ad43
6. osiegmar commented on Jul 8, 2026

   

   osiegmar

   on Jul 8, 2026

   Owner

   More actions

   Thanks for raising this and for the discussion – it shaped the API quite a bit.

   Implemented in e50ad43 for the upcoming 4.4.0 release.

   NamedCsvRecordHandler now accepts a HeaderValidator that is called once the header is determined – either from the first record or from a predefined header:

   NamedCsvRecordHandler handler = NamedCsvRecordHandler.builder()
       .headerValidator(HeaderValidator.containsExactly("col1", "col2"))
       .build();

   Two standard validators are provided:

   • HeaderValidator.containsExactly(...) – the header must consist of exactly the given fields, in the given order
   • HeaderValidator.containsAtLeast(...) – the header must contain at least the given fields, in any order; additional fields are allowed

   Anything beyond that (case-insensitive comparison, pattern matching, …) can be implemented as a lambda:

   .headerValidator(header -> {
       if (!header.contains("id")) {
           throw new CsvParseException("Missing id column: " + header);
       }
   })

   Validation failures throw a CsvParseException – fail fast, before any records are processed. The built-in duplicate header field check is now implemented as a HeaderValidator as well and consequently also throws CsvParseException (previously IllegalArgumentException).

   Documentation: https://fastcsv.org/guides/examples/header-validation/

   Feedback on the API is very welcome before this ships with 4.4.0 – if the naming or behavior doesn't fit your use case, now is the time to say so.
7. osiegmar commented on Oct 3, 2026

   

   osiegmar

   on Oct 3, 2026

   Owner

   More actions

   Released in 4.4.0.
8. osiegmar

   closed this as completedon Oct 3, 2026