Skip to content

parsing object mangles the input buffer and fails to parse, but parse returns true!  #342

Closed
@sticilface

Description

@sticilface

I've written a class that parses a SPIFFS file into a buffer, parses it into an abject and returns it for the program to use. the buffer is a unique_ptr kept until the jsonpackage is out of scope... but for some reason parsing the buffer mangles it, the parse fails, but it returns true!

here it the code

int JSONpackage::parseSPIFS(const char * file, FS & fs) {

        //using namespace ESPMAN;
        File f = fs.open(file, "r");
        int totalBytes = f.size();
        if (!f) {
                return -1;
        }
        if (totalBytes > MELVANA_MAX_BUFFER_SIZE) {
                f.close();
                return -2;
        }

        _data = std::unique_ptr<char[]>(new char[totalBytes]);

        if (!_data) {
                f.close();
                return -3;
        }
        int position = 0;
        int bytesleft = totalBytes;
        while ((f.available() > -1) && (bytesleft > 0)) {
                // get available data size
                int sizeAvailable = f.available();

                if (sizeAvailable) {
                        int readBytes = sizeAvailable;

                        // read only the asked bytes
                        if (readBytes > bytesleft) {
                                readBytes = bytesleft;
                        }

                        // get new position in buffer
                        char * buf = &_data.get()[position];
                        // read data
                        int bytesread = f.readBytes(buf, readBytes);
                        if (readBytes && bytesread == 0) { break; } //  this fixes a corrupt file that has size but can't be read.
                        bytesleft -= bytesread;
                        position += bytesread;
                        Serial.printf("totalbytes = %u, sizeAvailable = %u, readBytes = %u, bytesleft = %u, position = %u\n", totalBytes, sizeAvailable, readBytes, bytesleft, position);
                        Serial.println("chunk = ");
                        Serial.write(buf, bytesread);
                        Serial.println();

                }
                // time for network streams
                delay(0);
        }
        f.close();

        if (bytesleft) {
          Serial.printf("parseERROR: bytesleft = %u\n", bytesleft);
          return -5;
        }

        Serial.println("BUFFER before parse:");
        Serial.write(_data.get(), totalBytes);
        Serial.println();

        if (_isArray) {
                _root = _jsonBuffer.parseArray(_data.get(),totalBytes);
        } else {
                _root = _jsonBuffer.parseObject(_data.get(),totalBytes);
        }
        if (!_root.success()) {
                return -4;
        }

        Serial.println("BUFFER after parse:");
        Serial.write(_data.get(), totalBytes);
        Serial.println();

        Serial.println("_root =");
        _root.prettyPrintTo(Serial);
        Serial.println();

        return 0;

}

here is the debug output

totalbytes = 256, sizeAvailable = 256, readBytes = 256, bytesleft = 0, position = 256
chunk = 
{
  "globals": {
    "MQTT": {
      "enabled": true,
      "ip": [
        192,
        168,
        1,
        10
      ],
      "port": 1883,
      "topics": [
        "home/downstairs/lights/+/set"
      ]
    },
    "pixels": 20
  }
}
BUFFER before parse:
{
  "globals": {
    "MQTT": {
      "enabled": true,
      "ip": [
        192,
        168,
        1,
        10
      ],
      "port": 1883,
      "topics": [
        "home/downstairs/lights/+/set"
      ]
    },
    "pixels": 20
  }
}
BUFFER after parse:
globals.obals": {
    "MQTT": {
      "enabled": true,
      "ip": [
        192,
        168,
        1,
        10
      ],
      "port": 1883,
      "topics": [
        "home/downstairs/lights/+/set"
      ]
    },
    "pixels": 20
  }
}
_root =
{}

From the output you can see that the file is read correctly into the buffer... you can see the buffer before and after the parse... where before it is fine but after globals.obals": { is problematic... I've not got a clue about why this is happening... can you spot anything from my code?

here are the full files...

JsonPackage.h.txt
JsonPackage.cpp.txt

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions