Skip to content

Add helpers to URL-encode user input with GET requests #2082

Open
@ungive

Description

@ungive

I noticed that this library offers no way to URL encode certain characters in the URL path, like &. My request path is /v2/searchresults/Piri Tommy Villiers - on & on/relationships/tracks which contains an ampersand, yet a client's Get method does not automatically encode it (which it can't anyway, since the parameter to Get includes query parameters as well), nor is there a public function/method to URL encode part of the path, when there is data that may contain characters like these (e.g. when the path contains user input).

I need to include another library or manually write a method to URL encode my user input here: Piri Tommy Villiers - on & on.

The path above is used with the TIDAL API and I get 0 API results when I don't encode the ampersand:

for (auto ampersand : { "%26", "&" }) {
    std::cerr << ampersand << std::endl;
    httplib::Client cli("https://openapi.tidal.com");
    cli.set_bearer_token_auth("...");
    std::string path = "/v2/searchresults/Piri Tommy Villiers - on " + std::string(ampersand) + " on/relationships/tracks";
    auto result = cli.Get(httplib::append_query_params(path,
        {
            { "include", "tracks,tracks.artists,tracks.albums" },
            { "countryCode", "DE" },
        }));
    ASSERT_EQ(httplib::Error::Success, result.error());
    auto body = nlohmann::json::parse(result->body);
    EXPECT_GT(body["data"].size(), 0);
}

This fails with & for obvious reasons.

IMO this library should offer a public encode_url_component method of some kind or a method to sanitize user input for use in the request path (or a query parameter), such that it works well with httplib::append_query_params and httplib::Client::Get. I can't naively replace & with %26 and ? with %3F because httplib::detail::encode_url does not actually encode all characters it should be encoding, if I put a tabstop in the URL (\t) then my request fails as well (it's not replaced with %09):

cpp-httplib/httplib.h

Lines 2850 to 2880 in 71ba7e7

inline std::string encode_url(const std::string &s) {
std::string result;
result.reserve(s.size());
for (size_t i = 0; s[i]; i++) {
switch (s[i]) {
case ' ': result += "%20"; break;
case '+': result += "%2B"; break;
case '\r': result += "%0D"; break;
case '\n': result += "%0A"; break;
case '\'': result += "%27"; break;
case ',': result += "%2C"; break;
// case ':': result += "%3A"; break; // ok? probably...
case ';': result += "%3B"; break;
default:
auto c = static_cast<uint8_t>(s[i]);
if (c >= 0x80) {
result += '%';
char hex[4];
auto len = snprintf(hex, sizeof(hex) - 1, "%02X", c);
assert(len == 2);
result.append(hex, static_cast<size_t>(len));
} else {
result += s[i];
}
break;
}
}
return result;
}

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions