utf 8 - how to support UTF8 (japanese, arabic, spanish, ...) URL's in PHP

Question

For a web application, we need to link to some user generated content. A users types in a title for e.g. a product and we generate an SEO friendly url for that product:

like this

title: a nice product

www.user.com/product/a-nice-product

title: ???? ???

www.user.com/product/???? ???

The problem is that those foreign language url's aren't supported and a browser refuses to open those links. I've seen wordpress setups support that kind of url's so I guess it's possible to do this.

Does anyone know how we should support this in php?

wikipedia handles this just fine: http://ar.wikipedia.org

Answer 1

Although the URL itself only allows US-ASCII characters, you can use Unicode characters in the URI path if you encode them with UTF-8 and then convert them in US-ASCII characters by using the percent-encoding:

A system that internally provides identifiers in the form of a different character encoding, such as EBCDIC, will generally perform character translation of textual identifiers to UTF-8 [STD63] (or some other superset of the US-ASCII character encoding) at an internal interface, thereby providing more meaningful identifiers than those resulting from simply percent-encoding the original octets.

So you can do something like this (assuming UTF-8):

$title = '???? ???';
$path = '/product/'.rawurlencode($title);
echo $path;  // "/product/%D8%A3%D8%A8%D8%AC%D8%AF%20%D9%87%D9%88%D8%B2"

Although the URI path is actually encoded with the percent-encoding, most modern browsers will display the characters this sequence represents in Unicode when UTF-8 is used.