Repository navigation
Why slugs returned in URI encoded format? #2243
Description
Activity
- changed the title
[-]utf-8 post slugs returned in URL encoded mode[/-][+]utf-8 post slugs returned in unclear format[/+]on Feb 8, 2016 oops, the reason is the slug was URI encoded and I have to use
decodeURIto make the slug string as what I expect.
So the question is why slug should be returned in URI encoded format while the slugs saved in URL friendly format in backend?- changed the title
[-]utf-8 post slugs returned in unclear format[/-][+]Why slugs returned in URI encoded format?[/+]on Feb 10, 2016 Good catch. This is how it's stored in the database, which is why we're returning it that way. The dashboard is actually decoding this when it displays it to you, which is why you see it as the original text.
I'm not sure whether we should decode this before returning it, since it is stored in the DB this way, and there could conceivably be cases where you lose data because of hidden encoding/decoding.
@WP-API/amigos Thoughts here?
This is how it's stored in the database, which is why we're returning it that way.
Why is it stored in the database this way?
I'm not sure whether we should decode this before returning it, since it is stored in the DB this way, and there could conceivably be cases where you lose data because of hidden encoding/decoding.
I'm not familiar enough with character encoding to be able to make a determination one way or the other.
@nylen How do you handle this on WPcom? Do you have any prior art or conversations you can pull up?
Related #1227 (comment)
IMO if it's stored in the DB this way, we should return that in the API. If web client also shows that link as the dashboard does, it will similarly be decoded.
For reference,
get_terms()permits querying by either representation:salty-wordpress ➜ wordpress-develop.dev wp term list post_tag +---------+------------------+----------+-----------------------------------+-------------+--------+-------+ | term_id | term_taxonomy_id | name | slug | description | parent | count | +---------+------------------+----------+-----------------------------------+-------------+--------+-------+ | 3 | 3 | Foo | foo | | 0 | 0 | | 2 | 2 | سلام دنی | %d8%b3%d9%84%d8%a7%d9%85-%d8%af%d9%86%db%8c | | 0 | 0 | +---------+------------------+----------+-----------------------------------+-------------+--------+-------+ salty-wordpress ➜ wordpress-develop.dev wp shell wp> get_terms( 'post_tag', array( 'slug' => 'foo', 'hide_empty' => 0 ) ); array(1) { [0]=> object(WP_Term)#767 (10) { ["term_id"]=> int(3) ["name"]=> string(3) "Foo" ["slug"]=> string(3) "foo" ["term_group"]=> int(0) ["term_taxonomy_id"]=> int(3) ["taxonomy"]=> string(8) "post_tag" ["description"]=> string(0) "" ["parent"]=> int(0) ["count"]=> int(0) ["filter"]=> string(3) "raw" } } wp> get_terms( 'post_tag', array( 'slug' => '%d8%b3%d9%84%d8%a7%d9%85-%d8%af%d9%86%db%8c', 'hide_empty' => 0 ) ); array(1) { [0]=> object(WP_Term)#824 (10) { ["term_id"]=> int(2) ["name"]=> string(15) "سلام دنی" ["slug"]=> string(43) "%d8%b3%d9%84%d8%a7%d9%85-%d8%af%d9%86%db%8c" ["term_group"]=> int(0) ["term_taxonomy_id"]=> int(2) ["taxonomy"]=> string(8) "post_tag" ["description"]=> string(0) "" ["parent"]=> int(0) ["count"]=> int(0) ["filter"]=> string(3) "raw" } } wp> get_terms( 'post_tag', array( 'slug' => 'سلام دنی', 'hide_empty' => 0 ) ); array(1) { [0]=> object(WP_Term)#826 (10) { ["term_id"]=> int(2) ["name"]=> string(15) "سلام دنی" ["slug"]=> string(43) "%d8%b3%d9%84%d8%a7%d9%85-%d8%af%d9%86%db%8c" ["term_group"]=> int(0) ["term_taxonomy_id"]=> int(2) ["taxonomy"]=> string(8) "post_tag" ["description"]=> string(0) "" ["parent"]=> int(0) ["count"]=> int(0) ["filter"]=> string(3) "raw" } }fantastic.
@joehoyle I'm using wp-api with my angular app and the links created with the encoded slug, will not decoded in browser automatically, else I decode theme manually as this way:
$rootScope.decodeFa = function(str) { return decodeURI(str); }
So there is two ways:
- Release the encoded slugs AS IS and notice users this problem to let theme make their own approaches to decode or not or something else.
- Return the encoded slug.
Why is it stored in the database this way?
So, I've been thinking about this, and I think it actually makes sense. The
slugis intended as the slug (duh) for the URL, and the primary (and basically only) use case is in URLs. It makes sense then that this value needs to be safe for embedding in URLs, and that invalid characters are encoded.I'd say this is a wontfix, but we should document this in the schema as being specifically URL-safe data, and that invalid characters need to be encoded.
My understanding is that storing content encoded in the DB was done before the best practice of encoding immediately before display was firmly established and recognized. From https://codex.wordpress.org/Data_Validation:
Tip: It's best to do the output validation as late as possible, ideally as it's being outputted, as opposed to further up in your script. This way you can always be sure that your data is properly validated/escaped and you don't need to remember if the variable has been previously validated.
Having to deal with already-encoded data is more tricky because you are forced to violate this rule, and there can be bugs associated with double-encoding.
I couldn't find any discussions on slugs in particular on the WPCOM side, probably because as you've mentioned they are only used in a URL context which isn't so bad to deal with. My experience with these issues has been more related to HTML entities (#1227 (comment)).
IMO it would be a win for application developers to have the API decode this data. But a pretty minor issue in the scheme of things.
I disagree that this is necessarily a legacy thing, I think it's intentional. If the field was a URL value, we'd expect the URL encoded data, as we're saying "this is a valid URL that we are returning". The value here isn't a full URL, but it is URL data, so it makes sense to only return valid URL characters.
For example, if I had a slug like
foo\bar, I'd expect a URL field to havehttp://example.com/foo%5Cbar, so it makes sense for the slug field to havefoo%5Cbar. The only time I think it may be confusing is when saving data.No further discussion has happened here in a while, so going to close this out.
What will happen if we send the non-escaped version to WP in a request, e.g. we PUT the unescaped version when we got the escaped version out? Will WP re-escape when it inserts into the DB?
Update from my question: Yes, if you PUT the Arabic text as the slug, it will be inserted into the DB as escaped, while it will be untransformed in the content, excerpt and title field. This matches the URL-oriented purpose of the slugs.
Hi there,
I have a post which it's slug is in Persian language, for example
سلام دنیا. When I try to retrieve the post's slug, it's returned in unclear format like this:While English slugs returned as is, And other fields like
titleandcontentthat have Persian content, shows as well. Is it a true behavior? How can I get the true slug in Persian language?