Please login or register.

Login with username, password and session length
Advanced search  

News:

You need/want an older version of sNews ? Download an older/unsupported version here.

Pages: [1] 2

Author Topic: Bloody aliens chars  (Read 8692 times)

Sven

  • ULTIMATE member
  • ******
  • Karma: 88
  • Posts: 2029
  • Chasing MY bugs!
    • hiseo.fr - rédacteur Web
Bloody aliens chars
« on: September 23, 2008, 03:50:28 PM »

Howdee all
This happens locally or in the distant Website:
Quote
créé en prévision
Chars are displayed correctly on pages but this is what I get in the DB.
UTF-8 is used (as well in collate)
Where should I look to fix the issue?

EDIT: sorry to have posted in the wrong section... :-[
« Last Edit: September 23, 2008, 03:53:59 PM by Sven »
Logged

Sven

  • ULTIMATE member
  • ******
  • Karma: 88
  • Posts: 2029
  • Chasing MY bugs!
    • hiseo.fr - rédacteur Web
Re: Bloody aliens chars
« Reply #1 on: September 23, 2008, 05:39:13 PM »

I guess 1.7 RC can't handle foreign chars.
Not sure either there will an stable version.
Back to my "old" 1.6. ;D

Keyrocks

  • Doug
  • ULTIMATE member
  • ******
  • Karma: 449
  • Posts: 6019
  • Semantically Challenged
    • snews.ca
Re: Bloody aliens chars
« Reply #2 on: September 23, 2008, 05:55:36 PM »

I guess 1.7 RC can't handle foreign chars.
Not sure either there will an stable version.
Back to my "old" 1.6. ;D

I haven't looked into your foreign characters issue Sven... (it's never an issue for me since I only use english) but I can assure you that 1.7RC is very close to being released as a "stable" version. Philmoz has continued fine tuning for some time and currently has a well-populated demo online where some of the Dudes have been testing it further and providing further tune-up input.

Until such time as the stable version is released... and this is the same advice we always provide with any RC version... I suggest that your use of 1.7RC be limited to localhost and online testing only in its default format. I would not advise anyone to use it in a production environment until the stable release has been issued.
Logged
Do it now... later may not come.
-------------------------------------------------------------------------------------------------
sNews 1.6 MESU | sNews 1.6 MEMU

Sven

  • ULTIMATE member
  • ******
  • Karma: 88
  • Posts: 2029
  • Chasing MY bugs!
    • hiseo.fr - rédacteur Web
Re: Bloody aliens chars
« Reply #3 on: September 23, 2008, 06:29:25 PM »

Hi Doug, it's no big deal.
I don't create websites, you know, just a snoozer using sNews for my services to companies.
Just wanted to use it for a light blog.
But this issue might be a bug in the next version too.

Joost

  • Guest
Re: Bloody aliens chars
« Reply #4 on: September 23, 2008, 06:48:20 PM »

I guess 1.7 RC can't handle foreign chars.
Not sure either there will an stable version.
Back to my "old" 1.6. ;D

Are you telling me that characters like é, made in 1.6 look alright in the database (probably phpmyadmin)?  ???
 For almost 2 years, I've been wandering whether:
- It is a phpmyadmin rendering issues,
- A server issue
- Or an sNews issue

In my branch, you want to have the word café spelled right.  ;D
So far I've always managed doing so.
Logged

Sven

  • ULTIMATE member
  • ******
  • Karma: 88
  • Posts: 2029
  • Chasing MY bugs!
    • hiseo.fr - rédacteur Web
Re: Bloody aliens chars
« Reply #5 on: September 23, 2008, 07:37:15 PM »

Good evening Mr Joost.
Are you telling me that characters like é, made in 1.6 look alright in the database (probably phpmyadmin)?  ???
In 1.6 version those  kinds of chars are displayed correctly everywhere. But I had to applied a fix.
For almost 2 years, I've been wandering whether:
- It is a phpmyadmin rendering issues,
- A server issue
I've installed some other english CMS for testing (I had to write down an article about some) and never seen such a problem elsewhere.
- Or an sNews issue
Maybe the bug is repeated in the 1.7 RC version I've downloaded..
In my branch, you want to have the word café spelled right.  ;D
So far I've always managed doing so.
;D

Keyrocks

  • Doug
  • ULTIMATE member
  • ******
  • Karma: 449
  • Posts: 6019
  • Semantically Challenged
    • snews.ca
Re: Bloody aliens chars
« Reply #6 on: September 23, 2008, 07:41:13 PM »

@ Sven...

In a default install of 1.6 (no patches) ... this line displays properly in an article using utf8 (taken from your website):
<p>La rédaction pour un site obéit à des règles spécifiques qui sont différentes de la rédaction d'autres documents.</p>
and in the database... it shows as:
<p>La rédaction pour un site obéit à  des règles spécifiques qui sont différentes de la rédaction d'autres documents.</p>

I've never questioned why it stores in the dbase that way. I don't think sNews itself does this. I assumed that it was being converted by mysql (probably a wrong assumption on my part) and it really doesn't matter to me anyways... as long as it renders properly when retrieved from the dbase and displayed in the article.

However... I am sure there is an answer out there on the WWW. There is a karma waiting for you if you can find it.  ;)
Logged
Do it now... later may not come.
-------------------------------------------------------------------------------------------------
sNews 1.6 MESU | sNews 1.6 MEMU

Joost

  • Guest
Re: Bloody aliens chars
« Reply #7 on: September 23, 2008, 07:54:26 PM »

In 1.6 version those  kinds of chars are displayed correctly everywhere. But I had to applied a fix.

What's the secret?

To late pal, no karma for you today. :P

Code: [Select]
<?php
mysql_query
("SET CHARACTER SET 'UTF8'");
mysql_query("SET NAMES 'utf8'");
?>

Only one of those queries is needed. Both mean the same.
« Last Edit: September 24, 2008, 04:44:32 AM by Joost »
Logged

Keyrocks

  • Doug
  • ULTIMATE member
  • ******
  • Karma: 449
  • Posts: 6019
  • Semantically Challenged
    • snews.ca
Re: Bloody aliens chars
« Reply #8 on: September 23, 2008, 09:35:24 PM »

According to this post at ONLamp.com "Turning MySQL data in latin1 to utf8"...
... how data gets stored in the dbase tables has to do with server character settings. James Ryan (the author) notes that to run "purely utf8..."
1. The server's apache/httpd.conf file must have:
         AddCharset UTF-8 .utf8
         AddDefaultCharset UTF-8
   * this file in my xampp installation does not have this at all.

2. The server's php.ini file must have:
         default_charset = "utf-8"
   * by default, this file in my xampp installation has default_charset = "iso-8859-1" but it is disabled
     with default_mimetype = "text/html" enabled.

3. The server's mysql/bin/my.cnf file must have:
         character-set-server=utf8
         default-collation=utf8_unicode_ci
   * this file in my xampp installation has this enabled:
         character-set-server = latin1
         collation-server = latin1_general_ci

I am assuming that if I were to set these three as described by the author... that the data would be stored differently.
I just do not want to alter these as I do not want to screw up any of the 80 or so databases I have on my local machine right now and do not want to back them all up before doing such a test.

So... there is likely a chance that all these are not as recommended on your host server?
Codetwist should know what this is about... he seems quite knowledgeable when it comes to MySQL stuff.

« Last Edit: September 24, 2008, 02:22:08 PM by Keyrocks »
Logged
Do it now... later may not come.
-------------------------------------------------------------------------------------------------
sNews 1.6 MESU | sNews 1.6 MEMU

Joost

  • Guest
Re: Bloody aliens chars
« Reply #9 on: September 24, 2008, 07:13:30 AM »

Philippe,

Could it be a Windows only issue? It is suggested here.

This works for me on localhost

Add to my.ini in section [mysqld]

skip-character-set-client-handshake

Then restart the mysql server.
« Last Edit: September 24, 2008, 07:56:46 AM by Joost »
Logged

Sven

  • ULTIMATE member
  • ******
  • Karma: 88
  • Posts: 2029
  • Chasing MY bugs!
    • hiseo.fr - rédacteur Web
Re: Bloody aliens chars
« Reply #10 on: September 24, 2008, 08:03:14 AM »

Hi pals
@ Sven...
I've never questioned why it stores in the dbase that way. I don't think sNews itself does this. I assumed that it was being converted by mysql (probably a wrong assumption on my part) and it really doesn't matter to me anyways... as long as it renders properly when retrieved from the dbase and displayed in the article.

However... I am sure there is an answer out there on the WWW. There is a karma waiting for you if you can find it.  ;)
Doug, it's a bit ennoying to have those alien chars in the backups. Restoring will screw the whole site.

Finally yesterday after I left the forum I found the previous fix I applied on the 1.6 version.

What's the secret?

To late pal, no karma for you today. :P

Code: [Select]
<?php
mysql_query
("SET CHARACTER SET 'UTF8'");
mysql_query("SET NAMES 'utf8'");
?>

Only one of those queries is needed. Both mean the same.
That was it!
Quote
// CONNECT TO DATABASE
function connect_to_db() {
   $db = mysql_connect(db('dbhost'), db('dbuname'), db('dbpass'));
   mysql_query("SET CHARACTER SET 'UTF8'");
   mysql_query("SET NAMES 'utf8'");
   mysql_select_db(db('dbname')) or die(db('dberror'));
}
It was brother Mo who gave it since he had the same issue with (I suppose hebrew) chars.
So now it's working out.
This function connect_to_db() might be in the default core?

Joost

  • Guest
Re: Bloody aliens chars
« Reply #11 on: September 24, 2008, 08:37:46 AM »

I think we have to look into that Philippe,
It seems that local issues can be solved through fixes within my.ini (see previous post). I have to explore whether server (webhosting) issues can be solved through explicit database settingss (utf-8) instead of default settings.

It is actually quite surprising that the database setup script doesn't define character sets. I wonder what the reason might be.
« Last Edit: September 24, 2008, 08:52:15 AM by Joost »
Logged

Sven

  • ULTIMATE member
  • ******
  • Karma: 88
  • Posts: 2029
  • Chasing MY bugs!
    • hiseo.fr - rédacteur Web
Re: Bloody aliens chars
« Reply #12 on: September 24, 2008, 09:16:45 AM »

I think we have to look into that Philippe,
It seems that local issues can be solved through fixes within my.ini (see previous post). I have to explore whether server (webhosting) issues can be solved through explicit database settingss (utf-8) instead of default settings.
I'm trying to fix it locally but don't kno where to look for and where I should paste it.
In a "my.ini" file? ???
It is actually quite surprising that the database setup script doesn't define character sets. I wonder what the reason might be.
Yeap I noticed that too.
(Also I noticed there were some CSS classes added in the Core. ???
and function get_id doesn't work so most of the Mods won't work)
EDIT: okey, it works by pasting get_id() on steroids somewhere (better in separate file)
« Last Edit: September 24, 2008, 09:28:47 AM by Sven »
Logged

Keyrocks

  • Doug
  • ULTIMATE member
  • ******
  • Karma: 449
  • Posts: 6019
  • Semantically Challenged
    • snews.ca
Re: Bloody aliens chars
« Reply #13 on: September 24, 2008, 02:24:17 PM »


Doug, it's a bit ennoying to have those alien chars in the backups. Restoring will screw the whole site.


I know what you mean... and I think this post (as noted above) provides some solutions for converting the characters for database backups.
Logged
Do it now... later may not come.
-------------------------------------------------------------------------------------------------
sNews 1.6 MESU | sNews 1.6 MEMU

Joost

  • Guest
Re: Bloody aliens chars
« Reply #14 on: September 24, 2008, 05:01:56 PM »


I'm trying to fix it locally but don't kno where to look for and where I should paste it.
In a "my.ini" file? ???


my.ini is located in your mysql folder (old name: my.cnf ). It is a CoNFiguration file or INIitialization file.
Like Doug noted, the following ini settings must be applied as well:

httpd.conf:
AddCharset UTF-8 .utf8
AddDefaultCharset UTF-8

php.ini
default_charset = "utf-8"

my.cnf  (= my.ini Add to my.ini in section [mysqld] skip-character-set-client-handshake)
character-set-server=utf8
default-collation=utf8_unicode_ci

php.ini is located somewhere in.... /PHP/
httpd.conf is located somewhere in /Apache/.
Change the values (after the = sign) accordingly

Reason for adding skip-character-set-client-handshake, is the fact that the php binary for windows, requests communication in latin1. This we don't want

Any issues server side, is caused by the fact that the database setup file does not explicitly set the characters to utf-8. mysql_query("SET NAMES 'utf8'"); is a workaround to invoke the use of utf-8.... . . at least, that's what I think is happening. Any feedback on this would be appreciated.

Logged
Pages: [1] 2