- Posts: 15
COMMUNITY FORUM
K2 `extra_fields` character encoding problem
- Jerzy Z Gierzy
-
Topic Author
- Offline
- New Member
Less
More
8 years 8 months ago - 8 years 8 months ago #157249
by Jerzy Z Gierzy
K2 `extra_fields` character encoding problem was created by Jerzy Z Gierzy
Hello.
I encoutered a strange behavior while performing queries on Joomla! database.
Our site is in Polish, containing regional characters (ą ę ó ł ż ź ś ć ).
I performed a query on '#__k2_items' table for `extra_fields' column and results seem to fail with character encoding: I got ""Krak\u00f3w"" instead of "Kraków".
For curiosity sake i performed queries on '#__content' and results came correct (Polish characters shown ok). Then i queried '#__k2_items' for the 'title' column and again Polish characters were queried correctly.
I must add, that K2 items display OK on the page and when i look at appropiate fields in mysql the values show Polish signs.
Troubleshooting so far:
-i tried both JFactory::getDbo() and custom queries to the same result
-i enforced utf8 encoding in my queries
-i ensured that the scripts are encoded with utf8
-Polish characters echoed to page by script show OK
-I exported table in question to my localhost mysql server and reproduced the error. Same queries ran on a fresh 'k2_items' table came OK. Tables in question differ in storage engine (faulty table is a InnoDB table, healthy table is MyISAM). Alas, changing the engine didn't changed behaviour.
Im running out of ideas here. Am i missing something?
On a related note: are the extra fields stored as JSON or not? they seem to be JSON but PHP complains it cant decode them into array.
EDIT
I just installed a fresh Joomla on my localhost with fresh k2 installation, fresh categories, extra fields groups, extra fields. Again k2 item show ok on front and backend, but `extra_fields` database storage seems broken.
EDIT2
I used php preg_replace_callbackto change the query output back to special characters letters.
Still i don't see the point in escaping unicode in `extra_fields` and not in other columns.
I encoutered a strange behavior while performing queries on Joomla! database.
Our site is in Polish, containing regional characters (ą ę ó ł ż ź ś ć ).
I performed a query on '#__k2_items' table for `extra_fields' column and results seem to fail with character encoding: I got ""Krak\u00f3w"" instead of "Kraków".
For curiosity sake i performed queries on '#__content' and results came correct (Polish characters shown ok). Then i queried '#__k2_items' for the 'title' column and again Polish characters were queried correctly.
I must add, that K2 items display OK on the page and when i look at appropiate fields in mysql the values show Polish signs.
Troubleshooting so far:
-i tried both JFactory::getDbo() and custom queries to the same result
-i enforced utf8 encoding in my queries
-i ensured that the scripts are encoded with utf8
-Polish characters echoed to page by script show OK
-I exported table in question to my localhost mysql server and reproduced the error. Same queries ran on a fresh 'k2_items' table came OK. Tables in question differ in storage engine (faulty table is a InnoDB table, healthy table is MyISAM). Alas, changing the engine didn't changed behaviour.
Im running out of ideas here. Am i missing something?
On a related note: are the extra fields stored as JSON or not? they seem to be JSON but PHP complains it cant decode them into array.
EDIT
I just installed a fresh Joomla on my localhost with fresh k2 installation, fresh categories, extra fields groups, extra fields. Again k2 item show ok on front and backend, but `extra_fields` database storage seems broken.
EDIT2
I used php preg_replace_callback
$data = preg_replace_callback('/\\\\u([0-9a-fA-F]{4})/', function ($match) {
return mb_convert_encoding(pack('H*', $match[1]), 'UTF-8', 'UCS-2BE');
}, $data);Still i don't see the point in escaping unicode in `extra_fields` and not in other columns.
Last edit: 8 years 8 months ago by Jerzy Z Gierzy.
Please Log in or Create an account to join the conversation.
- Krikor Boghossian
-
- Offline
- Platinum Member
Less
More
- Posts: 15920
8 years 8 months ago #157265
by Krikor Boghossian
Replied by Krikor Boghossian on topic K2 `extra_fields` character encoding problem
Hello,
Which version of Joomla! and K2 are you using?
Which version of Joomla! and K2 are you using?
Please Log in or Create an account to join the conversation.
- Jerzy Z Gierzy
-
- Offline
- New Member
Less
More
- Posts: 15
8 years 8 months ago #157334
by Jerzy Z Gierzy
Replied by Jerzy Z Gierzy on topic K2 `extra_fields` character encoding problem
HI!
My Joomla version is 3.6.2 Stable
My k2 version is 2.7.1
My Php version is 5.6.24
My database connection collation is utf8mb4_general_ci
My database is 10.1.13-MariaDB-cll-lve
Running on debian 8 OS
My Joomla version is 3.6.2 Stable
My k2 version is 2.7.1
My Php version is 5.6.24
My database connection collation is utf8mb4_general_ci
My database is 10.1.13-MariaDB-cll-lve
Running on debian 8 OS
Please Log in or Create an account to join the conversation.
- Krikor Boghossian
-
- Offline
- Platinum Member
Less
More
- Posts: 15920
8 years 8 months ago #157345
by Krikor Boghossian
Replied by Krikor Boghossian on topic K2 `extra_fields` character encoding problem
Thanks :)
We will look into it Jerzy :)
We will look into it Jerzy :)
Please Log in or Create an account to join the conversation.