Non-English characters encoding troubles

[SecondShoe +]

Just bought the developer edition and found two problems. I am going to use it in Bulgaria. We use our own Cyrillic alphabet.
The language files come encoded in Windows-1251 format but the 68 script works with UTF-8. All of the files should be UTF-8 encoded. If not, all of the non-English characters will not be displayed and the users will see only "monkeys" or "?????". I converted english.php to UTF-8 and solved the problem for me though but it should come UTF-8 by default.

The second problem is somewhere in the core files, I guess. I changed the currency symbol from "$" to "лв." but it shows "лв" instead. Also, when the ad contain "ш" it shows "�?" .

Example: http://shumen.biz/68/viewlisting.php?view=2

[SecondShoe +]

Looks like I just fixed the currency symbol problem. It was DB related. I changed the DB collation to utf8_unicode_ci and now the symbol is corectly displayed. When I change the currency symbol one more time it`s back to the "лв". Strange.

OK, after more experimenting I figured out that all of the encoding problems are caused by the DB. At "class_listings" -> field "Description" is type "Text". When I change it to "varchar(255)" I`ve got "Ш" corectly displayed. The question is why?

[SecondShoe +]

My fellow web developer fixed all of the encoding problems. I don`t know what he did but he changed 4 files and everything is running nice and smooth now.
I will attach those files. Hopefully some of the developers will look through them and will get the stuff fixed for the future 68 versions.

P.S. I will try to translate the instructions from my fellow developer:

"We found some bugs in 68 with the cyrillic chars.

First - Validate::isLessThan

"You perform check for the length with strlen but with utf-8 encoding cyrillic chars are count like * 2
We changed it to mb_strlen function ( can use also iconv_strlen ) but you shoud check with function_exits before using our code. It works fine on our server but if this extensions is not installed will cause an error.

Second - Mysql::connectToDb

Some problem with the encoding. You should say to Mysql Server to transfer the data with utf-8 encoding otherwise it brakes the cyrillic chars.
We put 1 query at the end of function

SET NAMES utf8

Good luck :P"

I will send the fixed files through the ticket support system.
Sorry for the bad grammar! We are not a native English speakers.

[SecondShoe +]

Just found and fixed another bug causing the activation and other system mail messages to contain "Ш�ƒмен��ки�‚е ма" instead of a real text.

1. Open: includes\classes\mailer\class.phpmailer.php
2. Find: "iso-8859-1"
3. Replace with: "utf-8""
4. Save and upload.

[canariasdata +]

Originally Posted by SecondShoe

Second - Mysql::connectToDb

Some problem with the encoding. You should say to Mysql Server to transfer the data with utf-8 encoding otherwise it brakes the cyrillic chars.
We put 1 query at the end of function

SET NAMES utf8

Could you explain this more detailed please, as I still have similar problems with the spanish language!

On the frontend all is fine now, only problem I still have is that when editing texts in admin (homepage text for example), after re-opening the page in the editor all the special characters show funny again

Where would I place "SET NAMES utf8" ????

Thanks,
Gregor.

[canariasdata +]

In one of the other threads I was reading about the developers changed the collation of the database in version 4 to UTF-8 because of other languages and that it was "latin1_swedish_ci" before - which worked perfect for spanish C68 running sites.

Could please somebody confirm this, as I need to have the website running (in spanish) stable on monday. I would than change all the collations in the database back to latin1_swedish_ci and see if this solves my problem.

And if so, is there a way to change the collations with just one SQL query and how would that be?

Thanks and kind regards,
Gregor

[ombre +]

Originally Posted by SecondShoe

Just found and fixed another bug causing the activation and other system mail messages to contain "Шумен��ките ма" instead of a real text.

1. Open: includes\classes\mailer\class.phpmailer.php
2. Find: "iso-8859-1"
3. Replace with: "utf-8""
4. Save and upload.

Yes, this works fine. Thanks for other posts will save me few hours

[Jon]

This may be of use to some of you: here is our fix for smart text with a western character set:

Updated Meta Content-Type

[canariasdata +]

Its getting worse

I just noticed that the seo plugin does not work with special characters (�, �, �,...) also.
and links get rewritten like clientsdomain.net/listing/5/moto_bonito_�_�_�_�.html which leads to 404 Page not found.

Thank you to all forum gurus with all your good and friendly advices but now I really would appreciate some input from the DEVELOPERS as THIS IS NOT A FREE SOFTWARE!!!! (where I would expect some bugs at the begining ,but not even major bugs like these 2 I am having). Now it is time for a real CLIENT SUPPORT because I am a PAYING CUSTUMER with a script that does not what was written on "The Box"

[canariasdata +]

And another error I noticed regading the language problem.

All JavaSript error messages show wrong and I couldnt find a way to make them show right.

If I put in the language file (spanish.php) "í" it shows like "í"
If I put "�" it shows like "�"