C++ implementation for friendly handling of Json data

Background

C/C++ client needs to receive and send JSON format data to the backend to achieve communication and data interaction, but there is no ready-made interface for handling JSON format data in C++, so we can’t avoid splitting and splicing by directly referring to third-party libraries. Considering that there will be a lot of JSON data to be processed in this project, we can’t avoid the repetitive splitting and splicing. So I intend to encapsulate a set of C++ structure object to JSON data, JSON data directly mounted on the interface of C++ structure object, similar to the common serialization and deserialization in data transfer, in order to facilitate the subsequent processing of data and improve development efficiency.

Design

Objectives

To be able to convert a C++ structure object instance to JSON string data, or to load and assign a string of JSON string data to a C++ structure object instance through a simple interface. Ideal interface: Json2Object(inJsonString, outStructObject) , or Object2Json(inStructObject, outJsonString). 2.
Support Json conversion of built-in basic types such as bool, int, double, Json conversion of custom structures, Json conversion of the above types as arrays of elements, and Json conversion of nested structures.

Effect

first post the unit test code:

TEST_CASE("解析结构体数组到JSON串", "[json]")
{
    struct DemoChildrenObject
    {
        bool boolValue;
        int intValue;
        std::string strValue;
        /*JSON相互转换成员变量声明（必需）*/
        JSONCONVERT2OBJECT_MEMEBER_REGISTER(boolValue, intValue, strValue)
    };

    struct DemoObjct
    {
        bool boolValue;
        int intValue;
        std::string strValue;
        /*嵌套的支持JSON转换的结构体成员变量，数组形式*/
        std::vector< DemoChildrenObject> children;
        
        /*JSON相互转换成员变量声明（必需）*/
        JSONCONVERT2OBJECT_MEMEBER_REGISTER(boolValue, intValue, strValue, children)
    };

    DemoObjct demoObj;
    /*开始对demoObj对象的成员变量进行赋值*/
    demoObj.boolValue = true;
    demoObj.intValue = 321;
    demoObj.strValue = "hello worLd";

    DemoChildrenObject child1;
    child1.boolValue = true;
    child1.intValue = 1000;
    child1.strValue = "hello worLd child1";

    DemoChildrenObject child2;
    child2.boolValue = true;
    child2.intValue = 30005;
    child2.strValue = "hello worLd child2";

    demoObj.children.push_back(child1);
    demoObj.children.push_back(child2);
    /*结束对demoObj对象的成员变量的赋值*/

    std::string jsonStr;
    /*关键转换函数*/
    REQUIRE(Object2Json(jsonStr, demoObj)); 
    std::cout << "returned json format: " << jsonStr << std::endl;

    /*打印的内容如下：
    returned json format: {
       "boolValue" : true,
       "children" : [
          {
             "boolValue" : true,
             "intValue" : 1000,
             "strValue" : "hello worLd child1"
          },
          {
             "boolValue" : true,
             "intValue" : 30005,
             "strValue" : "hello worLd child2"
          }
       ],
       "intValue" : 321,
       "strValue" : "hello worLd"
    }
    */
    
    DemoObjct demoObj2;
    /*关键转换函数*/
    REQUIRE(Json2Object(demoObj2, jsonStr));

    /*校验转换后的结构体变量中各成员变量的内容是否如预期*/
    REQUIRE(demoObj2.boolValue == true);
    REQUIRE(demoObj2.intValue == 321);
    REQUIRE(demoObj2.strValue == "hello worLd");

    REQUIRE(demoObj2.children.size() == 2);

    REQUIRE(demoObj.children[0].boolValue == true);
    REQUIRE(demoObj.children[0].intValue == 1000);
    REQUIRE(demoObj.children[0].strValue == "hello worLd child1");

    REQUIRE(demoObj.children[1].boolValue == true);
    REQUIRE(demoObj.children[1].intValue == 30005);
    REQUIRE(demoObj.children[1].strValue == "hello worLd child2");
}

Implementation

This time we will only focus on how to convert between structures and Json strings in a friendly way, without focusing in depth on how exactly JSon strings are converted with basic data types. There are already quite a few third-party libraries to help us with this, such as cJSON, Jsoncpp, rapidjson, without having to create tools repeatedly.

This time we chose JsonCPP as the underlying JSON parsing support, if you want to replace it with other three-party libraries is also relatively simple, modify the corresponding embedded content can be.

Our goal is to implement two interfaces.

Json2Object(inJsonString, outStructObject)
Object2Json(inStructObject, outJsonString)

Combined with the types defined by JsonCPP itself, we further need to implement

Json2Object(const Json::Value& jsonTypeValue, outStructObject)
Object2Json(inStructObject, const std::string& key, Json::Value& jsonTypeValue)

Basic data type conversion

For basic data types such as bool, int, double, string, etc., the implementation is relatively simple.

/*int 类型支持*/
static bool Json2Object(int& aimObj, const Json::Value& jsonTypeValue)
{
    if (jsonTypeValue.isNull() || !jsonTypeValue.isInt()) {
        return false;
    } else {
        aimObj = jsonTypeValue.asInt();
        return true;
    }
}

static bool Object2Json(Json::Value& jsonTypeValue, const std::string& key, const int& value)
{
    jsonTypeValue[key] = value;
    return true;
}

/*std::string 字符串类型支持*/
static bool Json2Object(std::string& aimObj, const Json::Value& jsonTypeValue)
{
    if (jsonTypeValue.isNull() || !jsonTypeValue.isString()) {
        return false;
    } else {
        aimObj = jsonTypeValue.asString();
        return true;
    }
}

static bool Object2Json(Json::Value& jsonTypeValue, const std::string& key, const std::string& value)
{
    jsonTypeValue[key] = value;
    return true;
}

Custom data structure type

For the custom structure type, all we have to do is to make sure that its member variables can correspond to the JSON nodes one by one and can match for data filling.

   /*Json字符串：
   {
   "boolValue" : true,
   "intValue" : 1234,
   "strValue" : "demo object!"
   }*/

   struct DemoObjct
    {
        bool boolValue;
        int intValue;
        std::string strValue;
    };

As the above example, in the process of mutual conversion, “boolValue” can correspond to the member variable named boolValue in the DemoObjct object, and “strValue” corresponds to the member variable named strValue in the DemoObjct object.

Under normal circumstances, for this scenario, we can only implement the processing function for data conversion for DemoObjct structure extra, because the member variables defined by different structure declarations are different, so for each structure need to be implemented separately, the work is tedious and not universal.

From here, what we need to do is “hide “ the conversion functions implemented for class structures and use the language’s own features (function templates, etc.) to let them do this for us.

declare the conversion member functions, and in this member function implementation, enable each member variable to read or write values from the JSON native data.
register member variables so that the conversion member function knows which member variables it needs to handle and which node field in the JSON native data each member variable corresponds to in order to match reads and writes.
trigger calls to this conversion member function when the Json2Object and Object2Json functions are called externally in order to populate or output the contents of the member variables.

Fitting the elephant into the fridge takes only three steps, let’s see how these three steps work.

Member Variable Handling

Considering the uncontrollable type and number of member variables per structure and the need to treat each member variable as a left value (when Json2Object), you can’t simply use array enumeration to handle them, you can use C++11’s feature, variable parameter template, to iterate through each member variable from inside to outside

template <typename T>
static bool JsonParse(const std::vector<std::string>& names, int index, const Json::Value& jsonTypeValue, T& arg)
{
    const auto key = names[index];
    if (!jsonTypeValue.isMember(key) || Json2Object(arg, jsonTypeValue[key])) {
        return true;
    } else {
        return false;
    }
}

template <typename T, typename... Args>
static bool JsonParse(const std::vector<std::string>& names, int index, const Json::Value& jsonTypeValue, T& arg, Args&... args)
{
    if (!JsonParse(names, index, jsonTypeValue, arg)) {
        return false;
    } else {
        return JsonParse(names, index + 1, jsonTypeValue, args...);
    }
}

Member variables have correspondence with the JSON native node key field. Initially, we will first simply consider the member variable name as the key name of the corresponding node in JSON first. This can be achieved by the macro definition feature, which splits the list of key names by declaring the content of the registered member variable as a string.

#define JSONCONVERT2OBJECT_MEMEBER_REGISTER(...)  \
bool ParseHelpImpl(const Json::Value& jsonTypeValue)
{
    std::vector<std::string> names = Member2KeyParseWithStr(#__VA_ARGS__);
    return JsonParse(names, 0, jsonTypeValue, __VA_ARGS__);
}

Member Variable Registration

For example, for the class structure DemoObjct, add JSONCONVERT2OBJECT_MEMEBER_REGISTER with a registration statement for the member variable.

   struct DemoObjct
    {
        bool boolValue;
        int intValue;
        std::string strValue;
       
       JSONCONVERT2OBJECT_MEMEBER_REGISTER(boolValue, intValue, strValue)
           
    };

Equivalent to.

   struct DemoObjct
    {
        bool boolValue;
        int intValue;
        std::string strValue;
       
       bool ParseHelpImpl(const Json::Value& jsonTypeValue, 
                          std::vector<std::string> &names)
        {
            names = Member2KeyParseWithStr("boolValue, intValue, strValue");
            //names 得到 ["boolValue","intValue", "strValue"]
            //然后带着这些key逐一从Json中取值赋值到成员变量中
            return JsonParse(names, 0, jsonTypeValue, boolValue, intValue, strValue);
        }      
    };

Template matching to prevent compilation errors

So far, the core functionality has been implemented. If the target struct class does not add the JSON conversion declaration registration, external use of the Json2Object interface will result in a compilation error indicating that the declaration definition of the member function ParseHelpImpl cannot be found. …… We can use enable_if to provide a default function for to provide default functions for structs that do not declare registered macros.

template <typename TClass, typename enable_if<HasConverFunction<TClass>::has, int>::type = 0>
static inline bool Json2Object(TClass& aimObj, const Json::Value& jsonTypeValue)
{
    std::vector<std::string> names = PreGetCustomMemberNameIfExists(aimObj);
    return aimObj.JSONCONVERT2OBJECT_MEMEBER_REGISTER_RESERVERD_IMPLE(jsonTypeValue, names);
}

template <typename TClass, typename  enable_if<!HasConverFunction<TClass>::has, int>::type = 0>
static inline bool Json2Object(TClass& aimObj, const Json::Value& jsonTypeValue)
{
    return false;
}

Member variable matching Key renaming

The current implementations all use the name of the member variable as the name of the Key in the JSON string. For flexible handling, another macro is added for redeclaring the key corresponding to the JSON string in the member variable of the structure, for example.

struct DemoObjct
{
    bool boolValue;
    int intValue;
    std::string strValue;

    JSONCONVERT2OBJECT_MEMEBER_REGISTER(boolValue, intValue, strValue)
    /*重新声明成员变量对应到JSON串的key，注意顺序一致*/
    JSONCONVERT2OBJECT_MEMEBER_RENAME_REGISTER("bValue", "iValue", "sValue")
};

DemoObjct demoObj;
/*boolValue <--> bValue; intValue <--> iValue; ...*/
REQUIRE(Json2Object(demoObj, std::string("{\"bValue\":true, \"iValue\":1234, \"sValue\":\"demo object!\"}")));
REQUIRE(demoObj.boolValue == true);
REQUIRE(demoObj.intValue == 1234);
REQUIRE(demoObj.strValue == "demo object!");

Object2Json implementation

The above mentioned is mostly for the implementation of the Json2Object interface provided by the operation, from the structure object to Json is also a similar operation, here will not elaborate, detailed can refer to the source code.

Highlights

Simplify the processing of JSON data in C++, shield attention to the operation of splitting and processing JSON data.
Provide easy interface to switch from structure to JSON string and JSON string to structure freely.

Source code

#include "json/json.h"
#include <string>
#include <vector>
#include <initializer_list>

#define JSONCONVERT2OBJECT_MEMEBER_REGISTER(...)  \
bool JSONCONVERT2OBJECT_MEMEBER_REGISTER_RESERVERD_IMPLE(const Json::Value& jsonTypeValue, std::vector<std::string> &names) \
{     \
    if(names.size() <= 0)  { \
        names = Member2KeyParseWithStr(#__VA_ARGS__); \
    }   \
    return JsonParse(names, 0, jsonTypeValue, __VA_ARGS__); \
}   \
bool OBJECTCONVERT2JSON_MEMEBER_REGISTER_RESERVERD_IMPLE(Json::Value& jsonTypeValue, std::vector<std::string> &names) const \
{     \
    if(names.size() <= 0)  { \
        names = Member2KeyParseWithStr(#__VA_ARGS__); \
    }   \
    return ParseJson(names, 0, jsonTypeValue, __VA_ARGS__); \
}   \


#define JSONCONVERT2OBJECT_MEMEBER_RENAME_REGISTER(...)  \
std::vector<std::string> JSONCONVERT2OBJECT_MEMEBER_RENAME_REGISTER_RESERVERD_IMPLE() const \
{     \
    return Member2KeyParseWithMultiParam({ __VA_ARGS__ }); \
}

namespace JSON
{
template <bool, class TYPE = void>
struct enable_if
{
};

template <class TYPE>
struct enable_if<true, TYPE>
{
    typedef TYPE type;
};
} //JSON

template <typename T>
struct HasConverFunction
{
    template <typename TT>
    static char func(decltype(&TT::JSONCONVERT2OBJECT_MEMEBER_REGISTER_RESERVERD_IMPLE)); //@1
    
    template <typename TT>
    static int func(...); //@2

    const static bool has = (sizeof(func<T>(NULL)) == sizeof(char));

    template <typename TT>
    static char func2(decltype(&TT::JSONCONVERT2OBJECT_MEMEBER_RENAME_REGISTER_RESERVERD_IMPLE)); //@1
    template <typename TT>
    static int func2(...); //@2
    const static bool has2 = (sizeof(func2<T>(NULL)) == sizeof(char));
};

static std::vector<std::string> Member2KeyParseWithMultiParam(std::initializer_list<std::string> il)
{
    std::vector<std::string> result;
    for (auto it = il.begin(); it != il.end(); it++) {
        result.push_back(*it);
    }
    return result;
}

inline static std::string NormalStringTrim(std::string const& str)
{
    static char const* whitespaceChars = "\n\r\t ";
    std::string::size_type start = str.find_first_not_of(whitespaceChars);
    std::string::size_type end = str.find_last_not_of(whitespaceChars);
    return start != std::string::npos ? str.substr(start, 1 + end - start) : std::string();
}

inline static std::vector<std::string> NormalStringSplit(std::string str, char splitElem)
{
    std::vector<std::string> strs;
    std::string::size_type pos1, pos2;
    pos2 = str.find(splitElem);
    pos1 = 0;
    while (std::string::npos != pos2) {
        strs.push_back(str.substr(pos1, pos2 - pos1));
        pos1 = pos2 + 1;
        pos2 = str.find(splitElem, pos1);
    }
    strs.push_back(str.substr(pos1));
    return strs;
}

static std::vector<std::string> Member2KeyParseWithStr(const std::string& values)
{
    std::vector<std::string> result;
    auto enumValues = NormalStringSplit(values, ',');
    result.reserve(enumValues.size());
    for (auto const& enumValue : enumValues) {
        result.push_back(NormalStringTrim(enumValue));
    }
    return result;
}

//////////////////////////////////////////////////////////////////////////////

static bool Json2Object(bool& aimObj, const Json::Value& jsonTypeValue)
{
    if (jsonTypeValue.isNull() || !jsonTypeValue.isBool()) {
        return false;
    } else {
        aimObj = jsonTypeValue.asBool();
        return true;
    }
}
static bool Object2Json(Json::Value& jsonTypeValue, const std::string& key, bool value)
{
    jsonTypeValue[key] = value;
    return true;
}

static bool Json2Object(int& aimObj, const Json::Value& jsonTypeValue)
{
    if (jsonTypeValue.isNull() || !jsonTypeValue.isInt()) {
        return false;
    } else {
        aimObj = jsonTypeValue.asInt();
        return true;
    }
}
static bool Object2Json(Json::Value& jsonTypeValue, const std::string& key, const int& value)
{
    jsonTypeValue[key] = value;
    return true;
}

static bool Json2Object(unsigned int& aimObj, const Json::Value& jsonTypeValue)
{
    if (jsonTypeValue.isNull() || !jsonTypeValue.isUInt()) {
        return false;
    } else {
        aimObj = jsonTypeValue.asUInt();
        return true;
    }
}
static bool Object2Json(Json::Value& jsonTypeValue, const std::string& key, const unsigned int& value)
{
    jsonTypeValue[key] = value;
    return true;
}

static bool Json2Object(double& aimObj, const Json::Value& jsonTypeValue)
{
    if (jsonTypeValue.isNull() || !jsonTypeValue.isDouble()) {
        return false;
    } else {
        aimObj = jsonTypeValue.asDouble();
        return true;
    }
}
static bool Object2Json(Json::Value& jsonTypeValue, const std::string& key, const double& value)
{
    jsonTypeValue[key] = value;
    return true;
}

static bool Json2Object(std::string& aimObj, const Json::Value& jsonTypeValue)
{
    if (jsonTypeValue.isNull() || !jsonTypeValue.isString()) {
        return false;
    } else {
        aimObj = jsonTypeValue.asString();
        return true;
    }
}
static bool Object2Json(Json::Value& jsonTypeValue, const std::string& key, const std::string& value)
{
    jsonTypeValue[key] = value;
    return true;
}

template <typename TClass, typename JSON::enable_if<HasConverFunction<TClass>::has2, int>::type = 0>
static inline std::vector<std::string> PreGetCustomMemberNameIfExists(const TClass& aimObj)
{
    return aimObj.JSONCONVERT2OBJECT_MEMEBER_RENAME_REGISTER_RESERVERD_IMPLE();
}

template <typename TClass, typename JSON::enable_if<!HasConverFunction<TClass>::has2, int>::type = 0>
static inline std::vector<std::string> PreGetCustomMemberNameIfExists(const TClass& aimObj)
{
    return std::vector<std::string>();
}

template <typename TClass, typename JSON::enable_if<HasConverFunction<TClass>::has, int>::type = 0>
static inline bool Json2Object(TClass& aimObj, const Json::Value& jsonTypeValue)
{
    std::vector<std::string> names = PreGetCustomMemberNameIfExists(aimObj);
    return aimObj.JSONCONVERT2OBJECT_MEMEBER_REGISTER_RESERVERD_IMPLE(jsonTypeValue, names);
}

template <typename TClass, typename JSON::enable_if<!HasConverFunction<TClass>::has, int>::type = 0>
static inline bool Json2Object(TClass& aimObj, const Json::Value& jsonTypeValue)
{
    return false;
}

template <typename T>
static bool Json2Object(std::vector<T>& aimObj, const Json::Value& jsonTypeValue)
{
    if (jsonTypeValue.isNull() || !jsonTypeValue.isArray()) {
        return false;
    } else {
        aimObj.clear();
        bool result(true);
        for (int i = 0; i < jsonTypeValue.size(); ++i) {
            T item;
            if (!Json2Object(item, jsonTypeValue[i])) {
                result = false;
            }
            aimObj.push_back(item);
        }
        return result;
    }
}

template <typename T>
static bool JsonParse(const std::vector<std::string>& names, int index, const Json::Value& jsonTypeValue, T& arg)
{
    const auto key = names[index];
    if (!jsonTypeValue.isMember(key) || Json2Object(arg, jsonTypeValue[key])) {
        return true;
    } else {
        return false;
    }
}

template <typename T, typename... Args>
static bool JsonParse(const std::vector<std::string>& names, int index, const Json::Value& jsonTypeValue, T& arg, Args&... args)
{
    if (!JsonParse(names, index, jsonTypeValue, arg)) {
        return false;
    } else {
        return JsonParse(names, index + 1, jsonTypeValue, args...);
    }
}

/** Provider interface*/
template<typename TClass>
bool Json2Object(TClass& aimObj, const std::string& jsonTypeStr)
{
    Json::Reader reader;
    Json::Value root;

    if (!reader.parse(jsonTypeStr, root) || root.isNull()) {
        return false;
    }
    return Json2Object(aimObj, root);
}

static bool GetJsonRootObject(Json::Value& root, const std::string& jsonTypeStr)
{
    Json::Reader reader;
    if (!reader.parse(jsonTypeStr, root)) {
        return false;
    }
    return true;
}

////////////////////////////////////////////////////////////////////////

template <typename TClass, typename JSON::enable_if<HasConverFunction<TClass>::has, int>::type = 0>
static inline bool Object2Json(Json::Value& jsonTypeOutValue, const std::string& key, const TClass& objValue)
{
    std::vector<std::string> names = PreGetCustomMemberNameIfExists(objValue);
    if (key.empty()) {
        return objValue.OBJECTCONVERT2JSON_MEMEBER_REGISTER_RESERVERD_IMPLE(jsonTypeOutValue, names);
    } else {
        Json::Value jsonTypeNewValue;
        const bool result = objValue.OBJECTCONVERT2JSON_MEMEBER_REGISTER_RESERVERD_IMPLE(jsonTypeNewValue, names);
        if (result) {
            jsonTypeOutValue[key] = jsonTypeNewValue;
        }
        return result;
    }
}

template <typename TClass, typename JSON::enable_if<!HasConverFunction<TClass>::has, int>::type = 0>
static inline bool Object2Json(Json::Value& jsonTypeOutValue, const std::string& key, const TClass& objValue)
{
    return false;
}

template <typename T>
static bool Object2Json(Json::Value& jsonTypeOutValue, const std::string& key, const std::vector<T>& objValue)
{
    bool result(true);
    for (int i = 0; i < objValue.size(); ++i) {
        Json::Value item;
        if (!Object2Json(item, "", objValue[i])) {
            result = false;
        } else {
            if (key.empty()) jsonTypeOutValue.append(item);
            else jsonTypeOutValue[key].append(item);
        }
     }
    return result;
}

template <typename T>
static bool ParseJson(const std::vector<std::string>& names, int index, Json::Value& jsonTypeValue, const T& arg)
{
    if (names.size() > index) {
        const std::string key = names[index];
        return Object2Json(jsonTypeValue, key, arg);
    } else {
        return false;
    }
}

template <typename T, typename... Args>
static bool ParseJson(const std::vector<std::string>& names, int index, Json::Value& jsonTypeValue, T& arg, Args&... args)
{
    if (names.size() - (index + 0) != 1 + sizeof...(Args)) {
        return false;
    }
    const std::string key = names[index];
    Object2Json(jsonTypeValue, key, arg);

    return ParseJson(names, index + 1, jsonTypeValue, args...);
}

/** Provider interface*/
template<typename T>
bool Object2Json(std::string& jsonTypeStr, const T& obj)
{
    //std::function<Json::Value()>placehoder = [&]()->Json::Value { return Json::Value(); };
    //auto func = [&](std::function<Json::Value()>f) { return f(); };
    //Json::Value val = func(placehoder);

    Json::StyledWriter writer;
    Json::Value root;
    const bool result = Object2Json(root, "", obj);
    if (result) {
        jsonTypeStr = writer.write(root);
    }
    return result;
}

Table of Contents