Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • A ArduinoJson
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 24
    • Issues 24
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Benoît Blanchon
  • ArduinoJson
  • Issues
  • #1804
Closed
Open
Issue created Sep 28, 2022 by Miroslav Talasek@mirecta

Lazy value serialization

Hi for example i want send jpeg image from ESP32 camera (through for example MQTT) using Json, so i must encode jpeg data as base64. And its vaste of memory store jpeg in memory again even bigger because encoded as base64. So i patched ArduinoJson library to do it incrementaly i created Abstract class DynamicData

class DynamicData {
 public:
  typedef std::function<void(const char* s, size_t n)> WriteJsonFunc;
  typedef std::function<void(const uint8_t* s, size_t n)> WriteRawFunc;

  virtual void writeJsonTo(WriteJsonFunc writeFunc) = 0;
  virtual void writeRawTo(WriteRawFunc writeFunc) {}
  virtual size_t sizeJson() {
    return 0;
  }
  virtual size_t sizeRaw() {
    return 0;
  }
};

writeJsonTo write data for Json serializer using pointer to write func writeRawTo write data for MessagePack serializer using pointer to write func

and then i for example create Base64 encoder inherited from Dynamic data and i use it

#include <ArduinoJson.h>
#include <stdio.h>

class Base64 : public DynamicData {
 public:
  Base64(const char* data, size_t size) : data(data), dataSize(size) {}
  Base64() : data(0), dataSize(0) {}

  void encodeTriplet(const char (&input)[3], char (&output)[4]) {
    output[0] = base64_chars[(input[0] & 0xfc) >> 2];
    output[1] =
        base64_chars[((input[0] & 0x03) << 4) + ((input[1] & 0xf0) >> 4)];
    output[2] =
        base64_chars[((input[1] & 0x0f) << 2) + ((input[2] & 0xc0) >> 6)];
    output[3] = base64_chars[input[2] & 0x3f];
  }

  virtual size_t sizeRaw() {
    return dataSize;
  }

  virtual size_t sizeJson() {
    return 4 * (1 + ((sizeRaw() - 1) / 3));
  }

  virtual void writeJsonTo(DynamicData::WriteJsonFunc writer) {
    uint j = 0;
    char input[3];
    char output[4];

    for (uint i = 0; i < sizeRaw(); ++i) {
      input[j++] = data[i];
      if (j == 3) {
        encodeTriplet(input, output);
        writer(output, 4);
        j = 0;
      }
    }
    if (j != 0) {
      for (uint i = j; i < 3; ++i) {
        input[i] = '\0';
      }
      encodeTriplet(input, output);

      for (uint i = j + 1; i < 4; ++i) {
        output[i] = '=';
      }
      writer(output, 4);
    }
  }

 private:
  const char* data;
  size_t dataSize;
  static const char* base64_chars;
};
const char* Base64::base64_chars =
    "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";


int main() {
  char output[1280];
  const char binaryData[] =
      "test big data in memory which cand be stored encoded as base64 again "
      "because we must save memory!!";
  size_t size =
      strlen(binaryData);  // only for example (i must know size of binary data)

  DynamicJsonDocument doc(1024);
  doc["sensor"] = "gps";
  doc["binary"] = Base64(binaryData, size);

  serializeJson(doc, output);

  printf("%s\n ", output);
  return 0;
}

and after compile and run i got

{"sensor":"gps","binary":"dGVzdCBiaWcgZGF0YSBpbiBtZW1vcnkgd2hpY2ggY2FuZCBiZSBzdG9yZWQgZW5jb2RlZCBhcyBiYXNlNjQgYWdhaW4gYmVjYXVzZSB3ZSBtdXN0IHNhdmUgbWVtb3J5ISE="}

so it works perfectelly.

Do you want patch ? It is usefull for others ? it is in my fork ...

Assignee
Assign to
Time tracking