charset / encoding issues (utf-8 � iso-8859-2)

[MiśUszatek]

...any idea how to fix problem with encoding for extra fields?

[Plamen +]

Hi,
your script have a problem with utf
I have your script and have the same problem but in the RSS. Look here - the problem is &
After symbol &, in the name of listing!
There must be a empty space not amp;
Plamen

[Plamen +]

Any suggestions?

[darek +]

I may have one

so far I used windows-1250, I know it is not very professional but finally worked well

with v4, I desided to do something diffrent, my page will have full translation (including categories) and both langages will be available to users and guest - this made me think to change encoding


ok, I read these posts, did what have been advised and also had problems with UTF-8

solved after all

1)
MiśUszatku, when you work on your langage pages you use a program ( I use dreamweaver), most of them have page encoding that can be set, my previous page encoding was windows, when I changed the encoding in that program special characters changed to funny things, I changes them to normal letters and all started to display properly in the page - so - write files using the same encoding UTF-8

2)
still some things displayed not the way they should - the reason was that they have been written in another encoding, so I opened them as user or admin and wrote them again (page was encoded UTF-8) and they became to display properly- so - write data using the same encoding UTF-8

68c uses UTF-8 through out the page, also in files you wouldn't expect, it is wise to use UTF-8 or I think the only choice is to use windows-1250 as I run a page with this encoding happily for over a year

[ombre +]

Originally Posted by Eric Barnes

Great and thanks for posting that. I have updated the docs here:
UTF-8 [68 Classifieds]

Why i have not found this stuff earlier. Now i have lost 1,5 hours to found the fix on my own . That is something that should be added in the next version, please.

[seymourjames +]

I am still not getting this. ISO character sets for example can be defined in the layout.tpl file for the user of the system and things will display properly. A small change is also required in external.php to get that to display correctly as well. There is very little issue here that I have found.

The problem that I am battling with is in the admin control panel. If ISO characters are defined in the administration template, certain things work and certain things do not. For example ; extrafields cannot be defined if they include special characters and the same goes for displaying the titles of adverts when entered. The body text of an advert does display correctly if special characters are used. I do not understand why.

From my point of view, I just would like to reach a point where whatever is defined, all characters which come off an English, U.S. or French keyboard will display correctly. If UTF-8 is the answer then fine but when I have tried to keep the whole system in UTF-8, it does not work and spurious characters start to appear everywhere when I use so called latin characters. The other problem is most people in continental Europe will also set their browsers up to be iso-latin 1.

We need a conclusion and the solution to be documented clearly, step by step.

[darek +]

seymourjames, you are right, however you know that diffrent chairset show the same charcter diffrently, now, if you look at the code you will notice that when you submit something (not all though) it goes via UTF-8

shure - it is not good, I think the same chairest should be used everywhere, but it is not, we think that if we set up chaiset to iso or whatever on layout - all should be fine, but inside php files there is still UTF-8 used, I don't think it likes iso and, actually, I don't think it is necessary

here is the example

page content when edited using tinyMCE was with strange characters, I solved it by changing

PHP Code:

$class_tpl->assign('pPageTitle',safeStripSlashes(htmlentities($rs['pPageTitle'], ENT_QUOTES, 'UTF-8')));
                    $class_tpl->assign('pPageContent',safeStripSlashes(htmlentities($rs['pPageContent'], ENT_QUOTES, 'UTF-8'))); 


to

PHP Code:

$class_tpl->assign('pPageTitle',safeStripSlashes($rs['pPageTitle']));
                    $class_tpl->assign('pPageContent',safeStripSlashes($rs['pPageContent'])); 


a lot of people complaint that extra fields don't have proper characters, I bet it is because saved data goes through format.php where you can find 2 functions:

PHP Code:

    function special_chars($string)
    {
         return htmlspecialchars($string, ENT_QUOTES, 'UTF-8'); 
    }
    function safeHTML($value)
    {
        return mysql_real_escape_string(htmlentities(trim($value), ENT_QUOTES, 'UTF-8'));
    } 


I am not sure if I can call it that UTF-8 is hard php coded and it is there all the time, if you switch to any other chairset - you may have problems

I found almost no problems when using windows-1250

finally when I switched to UTF-8 and got rid of some code like above I have all characters like using iso

seymourjames, forget the iso, go to UTF-8
(by the way) when corel writes a pdf file I mark Unicode instead of ASCII (default) - then I have all proper characters

[seymourjames +]

If everything is set to UTF-8 as it is with a default installation then it needs to be documented clearly and precisely what steps need to be taken so characters coming from foreign keyboards are rendered correctly. It needs to become part of the readme documentation.

There is also the issue of what encoding you save your files in. For example, if you modify a file with a program like Coda or Dreamweaver and it is set to iso then problems may occur and visa-versa. There is then the issue of encoding on your browser. Many people in Europe will set the encoding on their browser to iso as the default.

The solution to the latter is difficult but to the former something does need to be written down in my opinion which people can follow.

So from a practical point of view - what is the solution. My sellers are French for example but my visitors are predominantly English.

[darek +]

Problems do occur!
I don't know..
I just set it to UTF-8
many webpages work with this chairset (not only english), and I think most browsers switch encoding to the one that is set on a webpage - I wouldn't worry about this

[hel68c +]

I had set to UTF-8 in my DB and in classified.php but I had to changed all accent character with the special code like � with à and � with â in french.php. In the DB everything display ok.

Now I can see all characters with accent in French except in the javascript alert windows. I only see special characters like à or � not � or � characters.

In userfavorites.tpl I change this line

cmsg = "{/literal}{$smarty.const.LANG_JS_DELETE_CONFIRM}{lit eral}"

to

cmsg = "{/literal}�tes-vous s�r de vouloir continuer{literal}"

And It display correctly.
Do I have to change something in Javascript language?