Skip to content

create higher level base64 functions #377

Description

@lemire

We should also provide base64_to_binary(const char* input) -> std::vector<uint8_t>, that does calculation of safe size and allocate memory internally. Or maybe something like base_to_binary(const char* input, cont: &Container) and static_assert that the Container has method resize. (credit: @WojciechMula)

Activity

WojciechMula commented on Mar 29, 2024

@WojciechMula
Collaborator

I was thinking a little about API. My generic proposal is providing a convenient wrapper that would work incrementally. I mean: user provides partial data (like input buffer when reading from file) and output buffer of fixed size. Using for decoding would be something like:

auto decoder = Base64Decoder::new();

std::string input;
input.resize(32 * 1024);

std::string output;
output.resize(16 * 1024);
while (/**/) {
      // read a few kilobytes data from into `input`
  
      const size_t bytes_stored = decoder.decode(input.data(), input.size(), output.data(), output.size());
      // bytes_stored will never be greater than output.size()

      write (output.data(), bytes_stored)

      if input file reached EOF {
          while (decoder.pending_output()) {
              const size_t bytes_stored = decoder.flush(output.data(), output.size());
              write (output.data(), bytes)
         }
      }
}

Of course this flexibility is at cost of performance, but my gut feeling is that if somebody want to process data in chunks, than problem is likely I/O bound.

WojciechMula commented on Mar 29, 2024

@WojciechMula
Collaborator

Another thing for base64 encoding - it would be practical if we allowed wrapping output, for instance:

const size_t max_line_length = 72;
const char* separator = "\n";
encode(input, output, max_line_length, separator);
```

Again, nobody would expect that this variant will be as fast as the plain encoding.

lemire commented on Mar 29, 2024

@lemire
MemberAuthor

@WojciechMula I'm pinging you later today as I have a major upgrade to the base64 support, with a slightly improved API.

lemire commented on Mar 30, 2024

@lemire
MemberAuthor

Please see #382 where the base64 API was slightly extended (i.e., we have _safe functions).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions