Contact
Evil Genius SocietyHomeLearnForumLeaderboardSocialsMessages
← Back to module

Lesson 9 of 9

Part III has given you templates, the standard library, lambdas, error handling, and class design. This project uses all five to build something you would otherwise reach for a library to do: a JSON parser and serializer that round-trips a document exactly.

It is about 250 lines. Every line below has been compiled with -Wall -Wextra, run, and checked under AddressSanitizer and UndefinedBehaviorSanitizer.

The specification

Parse the whole of JSON — objects, arrays, strings with escapes, numbers, true, false, null — into a value type you can inspect, and write it back out compact or pretty-printed. Specifically:

  1. Round-trip. dump(parse(dump(v))) == dump(v) for every document.
  2. Reject bad input with a position. Not just "parse error" — line and column.
  3. Survive hostile input. A megabyte of [ must not crash the process.
  4. Handle Khmer. យើង must decode to យើង, and UTF-8 must pass back out intact.
  5. No leaks, no undefined behaviour, proven with sanitizers.

Requirement 5 is not decoration. A recursive parser that allocates on every level is exactly the shape of program where a missed delete hides for years.

The value type

A JSON value is one of six things. That is the definition of std::variant (lesson 13.10).

// ASSEMBLE json 1 -- the whole project's includes, once
#include <charconv>
#include <cctype>
#include <cmath>
#include <cstdio>
#include <iostream>
#include <stdexcept>
#include <string>
#include <string_view>
#include <type_traits>
#include <utility>
#include <variant>
#include <vector>

Then the value type itself.

// ASSEMBLE json 2
class Value;

using Null   = std::monostate;
using Bool   = bool;
using Number = double;
using String = std::string;
using Array  = std::vector<Value>;
// An object is an ORDERED list of pairs, not a std::map. Two reasons, both in
// the lesson: std::map is not required to accept an incomplete value type, and
// a vector round-trips the document with its keys in the order they appeared.
using Object = std::vector<std::pair<std::string, Value>>;

class Value {
public:
    using Storage = std::variant<Null, Bool, Number, String, Array, Object>;

    Value() = default;                  // null
    Value(Bool b)        : v_(b) {}
    Value(Number n)      : v_(n) {}
    Value(int n)         : v_(static_cast<Number>(n)) {}
    Value(const char* s) : v_(String(s)) {}
    Value(String s)      : v_(std::move(s)) {}
    Value(Array a)       : v_(std::move(a)) {}
    Value(Object o)      : v_(std::move(o)) {}

    bool is_null()   const { return std::holds_alternative<Null>(v_); }
    bool is_bool()   const { return std::holds_alternative<Bool>(v_); }
    bool is_number() const { return std::holds_alternative<Number>(v_); }
    bool is_string() const { return std::holds_alternative<String>(v_); }
    bool is_array()  const { return std::holds_alternative<Array>(v_); }
    bool is_object() const { return std::holds_alternative<Object>(v_); }

    // Typed access. Throws std::bad_variant_access if you ask for the wrong one,
    // which is the right failure: it is a bug in your code, not bad input.
    Bool          as_bool()   const { return std::get<Bool>(v_); }
    Number        as_number() const { return std::get<Number>(v_); }
    const String& as_string() const { return std::get<String>(v_); }
    const Array&  as_array()  const { return std::get<Array>(v_); }
    const Object& as_object() const { return std::get<Object>(v_); }

    // Object lookup. Returns nullptr for a missing key, so callers can branch
    // without exceptions on the ordinary "field is absent" path.
    const Value* find(std::string_view key) const {
        if (!is_object()) return nullptr;
        for (const auto& [k, v] : as_object())
            if (k == key) return &v;
        return nullptr;
    }

    const Storage& storage() const { return v_; }

private:
    Storage v_;
};

Three decisions in there are worth defending.

Object is a vector of pairs, not a std::map. Two reasons. The standard only guarantees that std::vector, std::list and std::forward_list accept an incomplete element type, and at the point Object is declared, Value is still incomplete — it is the class currently being defined. std::map<std::string, Value> there is undefined behaviour, even though libstdc++ happens to compile it. The second reason is better: a vector keeps the keys in the order the document had them, so a round-trip does not silently reorder someone's config file. The cost is O(n) lookup, which for objects with a handful of keys is faster than a map anyway.

as_number() throws, find() returns nullptr. These are different failures. A missing key is ordinary — the document just did not have that field — so find reports it as a value you can test. Asking a string for as_number() is a bug in your code, so it throws std::bad_variant_access. Lesson 15.1's rule, applied: exceptions for programmer errors, return values for expected outcomes.

Value() = default is null. std::monostate is the first alternative, so a default-constructed Value is JSON null, which is the behaviour you want and costs nothing to arrange.

The parser

Recursive descent: one function per grammar rule, each calling the others. JSON's grammar is small enough that the parser reads like the specification.

// ASSEMBLE json 3
struct ParseError : std::runtime_error {
    std::size_t line, column;
    ParseError(std::string msg, std::size_t l, std::size_t c)
        : std::runtime_error(std::move(msg)), line(l), column(c) {}
};

class Parser {
public:
    explicit Parser(std::string_view text) : s_(text) {}

    Value parse() {
        skip_ws();
        Value v = parse_value(0);
        skip_ws();
        if (pos_ != s_.size()) fail("trailing characters after value");
        return v;
    }

private:
    static constexpr int kMaxDepth = 200;

    std::string_view s_;
    std::size_t pos_  = 0;
    std::size_t line_ = 1;
    std::size_t col_  = 1;

    [[noreturn]] void fail(const std::string& msg) const {
        throw ParseError(msg, line_, col_);
    }

    bool eof() const { return pos_ >= s_.size(); }
    char peek() const { return eof() ? '\0' : s_[pos_]; }

    char advance() {
        char c = s_[pos_++];
        if (c == '\n') { ++line_; col_ = 1; } else { ++col_; }
        return c;
    }

    void expect(char c) {
        if (peek() != c) fail(std::string("expected '") + c + "'");
        advance();
    }

    void skip_ws() {
        while (!eof()) {
            char c = peek();
            if (c == ' ' || c == '\t' || c == '\n' || c == '\r') advance();
            else break;
        }
    }

    Value parse_value(int depth) {
        if (depth > kMaxDepth) fail("nesting too deep");
        if (eof()) fail("unexpected end of input");
        switch (peek()) {
            case '{': return parse_object(depth);
            case '[': return parse_array(depth);
            case '"': return Value(parse_string());
            case 't': return parse_literal("true", Value(true));
            case 'f': return parse_literal("false", Value(false));
            case 'n': return parse_literal("null", Value());
            default:  return parse_number();
        }
    }

    Value parse_literal(std::string_view word, Value result) {
        if (s_.substr(pos_, word.size()) != word)
            fail("invalid literal");
        for (std::size_t i = 0; i < word.size(); ++i) advance();
        return result;
    }

    Value parse_object(int depth) {
        expect('{');
        Object obj;
        skip_ws();
        if (peek() == '}') { advance(); return Value(std::move(obj)); }
        for (;;) {
            skip_ws();
            if (peek() != '"') fail("object key must be a string");
            std::string key = parse_string();
            skip_ws();
            expect(':');
            skip_ws();
            obj.emplace_back(std::move(key), parse_value(depth + 1));
            skip_ws();
            if (peek() == ',') { advance(); continue; }
            expect('}');
            return Value(std::move(obj));
        }
    }

    Value parse_array(int depth) {
        expect('[');
        Array arr;
        skip_ws();
        if (peek() == ']') { advance(); return Value(std::move(arr)); }
        for (;;) {
            skip_ws();
            arr.push_back(parse_value(depth + 1));
            skip_ws();
            if (peek() == ',') { advance(); continue; }
            expect(']');
            return Value(std::move(arr));
        }
    }

    std::string parse_string() {
        expect('"');
        std::string out;
        for (;;) {
            if (eof()) fail("unterminated string");
            char c = advance();
            if (c == '"') return out;
            if (c != '\\') {
                if (static_cast<unsigned char>(c) < 0x20) fail("control character in string");
                out += c;
                continue;
            }
            if (eof()) fail("unterminated escape");
            switch (advance()) {
                case '"':  out += '"';  break;
                case '\\': out += '\\'; break;
                case '/':  out += '/';  break;
                case 'b':  out += '\b'; break;
                case 'f':  out += '\f'; break;
                case 'n':  out += '\n'; break;
                case 'r':  out += '\r'; break;
                case 't':  out += '\t'; break;
                case 'u':  out += parse_unicode_escape(); break;
                default:   fail("invalid escape");
            }
        }
    }

    // \uXXXX, encoded as UTF-8. Khmer lives in the U+1780 block, which is three
    // bytes, so this path is not optional for this site's data.
    std::string parse_unicode_escape() {
        unsigned cp = 0;
        for (int i = 0; i < 4; ++i) {
            if (eof()) fail("truncated \\u escape");
            char c = advance();
            unsigned d;
            if (c >= '0' && c <= '9') d = static_cast<unsigned>(c - '0');
            else if (c >= 'a' && c <= 'f') d = static_cast<unsigned>(c - 'a' + 10);
            else if (c >= 'A' && c <= 'F') d = static_cast<unsigned>(c - 'A' + 10);
            else fail("invalid hex digit in \\u escape");
            cp = cp * 16 + d;
        }
        std::string out;
        if (cp < 0x80) {
            out += static_cast<char>(cp);
        } else if (cp < 0x800) {
            out += static_cast<char>(0xC0 | (cp >> 6));
            out += static_cast<char>(0x80 | (cp & 0x3F));
        } else {
            out += static_cast<char>(0xE0 | (cp >> 12));
            out += static_cast<char>(0x80 | ((cp >> 6) & 0x3F));
            out += static_cast<char>(0x80 | (cp & 0x3F));
        }
        return out;
    }

    Value parse_number() {
        std::size_t start = pos_;
        if (peek() == '-') advance();
        if (!std::isdigit(static_cast<unsigned char>(peek()))) fail("invalid number");
        // JSON forbids leading zeros: 0 and 0.5 are fine, 01 is not.
        if (peek() == '0') {
            advance();
            if (std::isdigit(static_cast<unsigned char>(peek()))) fail("leading zero in number");
        } else {
            while (std::isdigit(static_cast<unsigned char>(peek()))) advance();
        }
        if (peek() == '.') {
            advance();
            if (!std::isdigit(static_cast<unsigned char>(peek()))) fail("digit expected after '.'");
            while (std::isdigit(static_cast<unsigned char>(peek()))) advance();
        }
        if (peek() == 'e' || peek() == 'E') {
            advance();
            if (peek() == '+' || peek() == '-') advance();
            if (!std::isdigit(static_cast<unsigned char>(peek()))) fail("digit expected in exponent");
            while (std::isdigit(static_cast<unsigned char>(peek()))) advance();
        }
        double out = 0;
        std::string_view text = s_.substr(start, pos_ - start);
        auto [ptr, ec] = std::from_chars(text.data(), text.data() + text.size(), out);
        if (ec != std::errc{} || ptr != text.data() + text.size()) fail("number out of range");
        return Value(out);
    }
};

Value parse(std::string_view text) { return Parser(text).parse(); }

The details that matter

advance() tracks line and column. Every character goes through it, so fail() always knows where it is. That is the entire cost of good error messages — one branch per character.

kMaxDepth is not optional. parse_value calls parse_array calls parse_value. A document of 100,000 open brackets is 100,000 stack frames. Section "Hostile input" below shows what happens without the guard.

parse_string builds the string once. It appends to out and returns it; the compiler moves it out (lesson 9.4). No intermediate std::string per character.

The \u escape emits UTF-8 by hand. Khmer sits in U+1780–U+17FF, which is the three-byte range, so the 0xE0 branch is the one this site's data exercises. An escape is the only way a Khmer string can travel through a strictly-ASCII JSON file, so this path is not optional here.

from_chars for numbers, not std::stod. stod is locale-dependent — in a locale where the decimal separator is a comma it parses 1.5 as 1. from_chars is locale-independent by specification, does not allocate, and does not throw; it reports failure in its return value.

The serializer

Writing is easier than reading, and std::visit over the variant makes it nearly mechanical.

// ASSEMBLE json 4
void write_escaped(std::string& out, std::string_view s) {
    out += '"';
    for (char c : s) {
        switch (c) {
            case '"':  out += "\\\""; break;
            case '\\': out += "\\\\"; break;
            case '\b': out += "\\b";  break;
            case '\f': out += "\\f";  break;
            case '\n': out += "\\n";  break;
            case '\r': out += "\\r";  break;
            case '\t': out += "\\t";  break;
            default:
                if (static_cast<unsigned char>(c) < 0x20) {
                    char buf[7];
                    std::snprintf(buf, sizeof buf, "\\u%04x", static_cast<unsigned char>(c));
                    out += buf;
                } else {
                    // UTF-8 bytes pass through unchanged; \u escaping them is
                    // legal but makes Khmer unreadable in the output file.
                    out += c;
                }
        }
    }
    out += '"';
}

void write_number(std::string& out, double n) {
    if (n == static_cast<double>(static_cast<long long>(n)) &&
        std::abs(n) < 1e15) {
        out += std::to_string(static_cast<long long>(n));
        return;
    }
    char buf[32];
    auto [ptr, ec] = std::to_chars(buf, buf + sizeof buf, n);
    out.append(buf, (ec == std::errc{}) ? ptr : buf);
}

void write_value(std::string& out, const Value& v, int indent, int depth) {
    const bool pretty = indent > 0;
    const std::string pad    = pretty ? std::string(static_cast<std::size_t>(indent * (depth + 1)), ' ') : "";
    const std::string padEnd = pretty ? std::string(static_cast<std::size_t>(indent * depth), ' ') : "";
    const char* nl  = pretty ? "\n" : "";
    const char* sep = pretty ? ": " : ":";

    std::visit([&](const auto& x) {
        using T = std::decay_t<decltype(x)>;
        if constexpr (std::is_same_v<T, Null>) {
            out += "null";
        } else if constexpr (std::is_same_v<T, Bool>) {
            out += x ? "true" : "false";
        } else if constexpr (std::is_same_v<T, Number>) {
            write_number(out, x);
        } else if constexpr (std::is_same_v<T, String>) {
            write_escaped(out, x);
        } else if constexpr (std::is_same_v<T, Array>) {
            if (x.empty()) { out += "[]"; return; }
            out += '['; out += nl;
            for (std::size_t i = 0; i < x.size(); ++i) {
                out += pad;
                write_value(out, x[i], indent, depth + 1);
                if (i + 1 < x.size()) out += ',';
                out += nl;
            }
            out += padEnd; out += ']';
        } else {
            if (x.empty()) { out += "{}"; return; }
            out += '{'; out += nl;
            for (std::size_t i = 0; i < x.size(); ++i) {
                out += pad;
                write_escaped(out, x[i].first);
                out += sep;
                write_value(out, x[i].second, indent, depth + 1);
                if (i + 1 < x.size()) out += ',';
                out += nl;
            }
            out += padEnd; out += '}';
        }
    }, v.storage());
}

std::string dump(const Value& v, int indent = 0) {
    std::string out;
    write_value(out, v, indent, 0);
    return out;
}

The if constexpr chain inside the lambda is the generic-visitor pattern from lesson 13.10: one lambda, auto parameter, and the compiler generates a branch per alternative — with a compile error if you forget one, because std::visit requires the visitor to be callable for every type.

write_number exists because std::to_string(1.0) gives "1.000000" and to_chars gives "1". JSON has one number type, so an integer-valued double must print without a decimal point or every round-trip grows.

Running it

// ASSEMBLE json 5
const char* kDoc = R"JSON({
  "site": "Evil Genius Society",
  "languages": ["en", "km"],
  "free": true,
  "learners": 0,
  "motto": {"km": "\u1799\u17be\u1784", "en": "we"},
  "version": 1.5,
  "sponsor": null
})JSON";

int main() {
    Value doc = parse(kDoc);

    std::cout << "1. round-trip, pretty:\n" << dump(doc, 2) << "\n\n";
    std::cout << "2. compact:\n" << dump(doc) << "\n\n";

    std::cout << "3. typed access:\n";
    std::cout << "   site      " << doc.find("site")->as_string() << "\n";
    std::cout << "   languages " << doc.find("languages")->as_array().size() << " entries\n";
    std::cout << "   free      " << std::boolalpha << doc.find("free")->as_bool() << "\n";
    std::cout << "   khmer     " << doc.find("motto")->find("km")->as_string() << "\n";
    std::cout << "   missing   " << (doc.find("nope") == nullptr ? "nullptr" : "?") << "\n";

    std::cout << "\n4. idempotence: dump(parse(dump(doc))) == dump(doc)? "
              << (dump(parse(dump(doc))) == dump(doc) ? "yes" : "no") << "\n";

    std::cout << "\n5. errors:\n";
    const std::vector<std::string> bad = {
        R"({"a": 1,})",
        R"({"a" 1})",
        R"([1, 2)",
        R"({"a": tru})",
        R"(  {"a": 01})",
        "\"unterminated",
        R"({"a": "bad \q escape"})",
    };
    for (const std::string& b : bad) {
        try {
            parse(b);
            std::cout << "   ACCEPTED (bug): " << b << "\n";
        } catch (const ParseError& e) {
            std::cout << "   line " << e.line << " col " << e.column
                      << ": " << e.what() << "\n";
        }
    }

    std::cout << "\n6. building a document in code:\n";
    Object out;
    out.emplace_back("module", Value(16));
    out.emplace_back("lesson", Value("16.9"));
    out.emplace_back("tags", Value(Array{Value("variant"), Value("recursion"), Value("raii")}));
    std::cout << dump(Value(std::move(out)), 2) << "\n";
    return 0;
}
1. round-trip, pretty:
{
  "site": "Evil Genius Society",
  "languages": [
    "en",
    "km"
  ],
  "free": true,
  "learners": 0,
  "motto": {
    "km": "យើង",
    "en": "we"
  },
  "version": 1.5,
  "sponsor": null
}

2. compact:
{"site":"Evil Genius Society","languages":["en","km"],"free":true,"learners":0,"motto":{"km":"យើង","en":"we"},"version":1.5,"sponsor":null}

3. typed access:
   site      Evil Genius Society
   languages 2 entries
   free      true
   khmer     យើង
   missing   nullptr

4. idempotence: dump(parse(dump(doc))) == dump(doc)? yes

5. errors:
   line 1 col 9: object key must be a string
   line 1 col 6: expected ':'
   line 1 col 6: expected ']'
   line 1 col 7: invalid literal
   line 1 col 10: leading zero in number
   line 1 col 14: unterminated string
   line 1 col 14: invalid escape

6. building a document in code:
{
  "module": 16,
  "lesson": "16.9",
  "tags": [
    "variant",
    "recursion",
    "raii"
  ]
}

Every one of the seven bad documents was rejected with a position. Two are worth calling out.

{"a": 01} is invalid JSON — the grammar forbids leading zeros — and the first version of this parser accepted it. The bad-input list found the bug, not a review. The fix is the if (peek() == '0') branch in parse_number: after a zero, the integer part is over.

{"a": 1,} reports "object key must be a string" rather than "trailing comma". The parser consumed the comma, looped, and found } where a key belongs. It is accurate but it is not the message a human wants. Improving it is the first exercise below.

Hostile input

Requirement 3, tested directly.

std::string deep(100000, '[');
deep += std::string(100000, ']');
Value v = parse(deep);   // 100,000 recursive calls

With the depth guard removed:

$ g++ -std=c++20 -Wall -Wextra -O2 -o ng noguard.cpp
$ ./ng
  parsing 100000 levels with no depth guard...
Segmentation fault

With it:

  depth 150: parsed
  depth 300: nesting too deep
  depth 100000: nesting too deep

A crash on untrusted input is a denial of service. Any recursive parser reading data you did not write needs a depth limit, and 200 is generous — real JSON is rarely nested past 20.

Verifying it

Clean under sanitizers, which is the requirement:

$ g++ -std=c++20 -Wall -Wextra -g -fsanitize=address,undefined json.cpp -o json_san
$ ./json_san > san.out
$ ./json | diff - san.out && echo identical
identical

No ASan report, no UBSan report, exit code 0, and byte-identical output to the optimised build. Note what carried this: no new and no delete appear anywhere in the project. std::string, std::vector and std::variant own everything, and the recursion unwinds through destructors. When fail() throws from twelve levels deep, every partially-built Array and Object on the way out is destroyed by RAII (lesson 15.4) with nothing to write.

Break it deliberately

| Change | What happens | |---|---| | Remove the kMaxDepth check | Segfault on deeply nested input | | Use std::stod instead of from_chars | Passes here; wrong under a comma-decimal locale | | std::map<std::string, Value> for Object | Compiles on libstdc++; UB, and reorders keys | | Drop the < 0x20 check in parse_string | Accepts raw newlines inside strings | | Return out by std::move from parse_string | Warning: -Wpessimizing-move | | Let find() throw instead of returning nullptr | Every optional field now needs a try |

What you have built

A real parser. Not a toy one — it has correct escape handling, locale-independent numbers, positional errors, a depth guard, and order-preserving objects, which is more than several widely-used implementations manage.

What it does not have: surrogate pairs (😀 for emoji outside the BMP), streaming input, and the aggressive optimisation a production parser uses. Those are the next three things to add, and the first is the exercise.

Try it

Three additions, in order of difficulty.

  1. Make {"a": 1,} say trailing comma in object instead of object key must be a string.
  2. Add a Value::type_name() returning "null", "bool", "number", "string", "array" or "object", and use it to produce a real error when typed access fails.
  3. Handle surrogate pairs, so "😀" decodes to the four UTF-8 bytes of U+1F600.

Checkpoint

  • A JSON value is a sum type; std::variant is the direct expression of that.
  • Object as a vector of pairs: std::map is not guaranteed to accept an incomplete type.
  • It also preserves key order, so a round-trip does not reorder someone's file.
  • Recursive descent gives one function per grammar rule; the parser reads like the spec.
  • advance() tracking line and column costs one branch and buys every error message.
  • Measured: without a depth guard, 100,000 nested brackets segfault. With it, a clean error.
  • Any recursive parser on untrusted input needs a depth limit.
  • from_chars over std::stod: locale-independent, no allocation, no throw.
  • \u escapes decode to UTF-8 by hand; Khmer is the three-byte branch, emoji the four-byte one.
  • Surrogate pairs: 0x10000 + ((hi - 0xD800) << 10) + (lo - 0xDC00).
  • std::visit with a generic lambda and if constexpr writes one branch per alternative — and fails to compile if you miss one.
  • No new, no delete in 250 lines. RAII unwinds a partially-built tree on throw for free.
  • Verified: ASan and UBSan clean, output byte-identical to the optimised build.
  • Exceptions for programmer errors (as_number() on a string), return values for expected ones (find() on a missing key).
  • The bad-input list found a real bug — 01 was accepted. Write the rejection tests first.

Questions about this lesson

Log in to ask a question.