Hello to all the community

Since sometime (about 2 years ago) i built a joomla website based mainly on K2 for all the articles. The site is based mainly in news and political content and it has high activity, since there are posted at about 10-15 articles a day. At about 2-3 months ago some problems appeared in the stability of the site, since at frequent times the server was unable to handle the load and we was facing "503 temporary service unavailable" problems. In the beginning we thought these problems was due to high visitor count, since we had at about 4-5.000 unique visitors a day. But at about 2-3 weeks ago the problem worsen so much that the web host forced to disable the database since it was making huge database queries and it was affecting the nearby servers and almost the whole network. Initially we was at a shared hosting.

They told us that due to very high activity we should move better to a VPS in order to handle the high load of visitors (about 7-8000 unique visitors a day) and that the database (almost 2gigabytes) was very huge to work properly. So at last we moved to a VPS. But after the migration nothing changed, the same problem appeared and in the new server. Initially we thought that the resources wasn't enough, so we upgraded temporarily, actually they doubled the RAM of the server, but again the same problem was persisting. From the host they told me that the problem was the SQL server and the database was making large queries that was unable to complete so it was overloading, the same time HDD activity was at about 50-60Mb/s and the server was unable to make anything else so the website was always down and unavailable. Here you can see the activity from the server:



I told them to send me the large query that made the problem and it is the following:

SELECT a.*, cc.name AS cattitle, cc.id AS categoryid, cc.alias AS categoryalias, cc.params AS categoryparams, u.username as username, u.name as realname FROM xxkw9_k2_items a LEFT JOIN xxkw9_k2_categories cc ON cc.id=a.catid
LEFT JOIN xxkw9_users AS u ON u.id = a.created_by WHERE a.published =1 AND a.trash = 0 AND cc.trash = 0 AND cc.published = 1 AND ( a.publish_up = '0000-00-00 00:00:00' OR a.publish_up <= '2015-10-10 19:53:56' ) AND ( a.publish_down = '0000-00-00 00:00:00' OR a.publish_down >= '2015-10-10 19:53:56' ) AND cc.id IN (8,9,10,11,12,13,14,17,15,21)

I forgot to mention that we have about 10.000 items in K2 articles and almost 4 to 5.000 tags. After those facts i forced to move the site in a subfolder, and magically there it was functioning properly and fairly fast. When i was moving the site in the root folder, we had the same problems with the overload. I even disabled all the plugins and the modules in the frontpage and i left only the articles. But nothing changed.
From other similar cases i read that maybe the tags was the problem, because i saw a large query with tags, you can see it here:

codepaste.net/r55o6r

So i read here that with a minor change in the code, we can disable the IDs from querying the database. Here is the post:

www.joomlaworks.net/forum/k2-en/42833-k2-tags-causing-slow-mysql-queries

So i disabled the IDs but again nothing changed, i saw some small improvements but the initial problem persisted. By the way here are the queries at a whole, through the debug system of joomla:

codepaste.net/q4ffiw
codepaste.net/r55o6r
codepaste.net/5rojbi

You can spot that there are a large number of queries that they are repeated 3,4 or 5 times and i don't know the reason.

As a temporary measure, as i told you early i moved the website in a subfolder, and through htaccess i redirected all the traffic there with this code:

RewriteEngine On
RewriteRule ^(/)?$ subfolder [L]

In the beginning it seems it was working, but again we had the same problem. So i made a single php page with a redirection to the subfolder:

<?php
Header( "HTTP/1.1 301 Moved Permanently" );
Header( "Location: www.example.com/subfolder" );
?>

This time things was better, the website was working, but again at frequent manner we had again the overload and the site was unavailable. Here is the past 12 hours from today:



In the blue peaks the whole server is unavailable and no one can see the website.
At the peak load the CPU was also very high, here are some numbers

Last 1 minute 20.95
Last 5 minutes 31.52
Last 15 minutes 28.09

In order for the server to work seamlessly these numbers must be under 5.00
From the technical stuff of the host they did some changes, i.e they changed the server from Apache to Nginx, but they couldn't do much more, they told me that the problem is the large database, only one table is about 1 Gigabyte and the database as a whole nearly 2, and the problem was in K2 so i must look there.

Please guys help me, for about a week or so i am dealing with this problem and i can't find a permanent solution, only temporary fixes but the problem is here, and you can imagine the visitors that see all these problems, all the past weeks. When there is an overload i am forcing to restart the VPS in order to calm down from the queries. Some people told me to delete the tags and see what is happening and if the database shrinks. But this isn't a good solution since the guy that manage the webpage, needs them.

If you can propose anything i would be glad.
Thanks in advance...

@John Kahrhoff

Things are simple regarding performance:

1. Enable Joomla! caching. Set it's interval to a logical value.
2. Enable the core system cache plugin.
3. Update K2 and Joomla! to their latest versions.
4. Finally in any case, you can always enable the system debugger. It will give you information regarding the page load and which extensions take a lot of time to load.

Hello - this thread is old now, but I agree with Scott regarding the ever increasing load times of K2 content. I am using a JoomlArt template that came with K2 system pre-loaded. K2 is Free, but I have purchased several other items from the company.

I have no custom code on my site - none - and I stay away from non-native extensions. In other words - I used the native components on the template and the Joomla core.

I am using the latest stable version of Joomla and K2 and it is having problems. The sad part it that I built my site and articles around it and it just started slowing down...

I stumbled on this thread because I am searching for a solution.

I will continue to search!


Thanks!

Well, I waited long enough. I have switched over to using Ohanah for my events and tweaked the output a little to make it the same as what I was doing with K2. And it's rediculously faster. Was taking 20+ seconds to load a page with a list of K2 items on it and now it's <2 seconds.

I had an announcement go out that encouraged a bunch of visitors to the site. It got about 49 on at once at peak. This is basically nothing in the world of web, but because the site was loading so slowly, every one of those visitors was repeatedly hitting refresh - at least this is my best guess - and the entire VPS was completely overloaded. All sites on the box slowed to a crawl and this site never even came up for them. Embarrassing.

There is no question that it was K2 that was killing it. As soon as I disabled the extensino, the whole site was instantly usable and the whole server load dropped from 90+ (this is the WHM reading) to <1.

So now I am running with no caches other than the template cache and the site is SO responsive. I still have to figure out one other part, but I have some ideas.

I didn't get any useful help here and I'm disappointed. Out of principal, I'm going to be using something else in those places where I WAS using K2 on new sites. I simply can't be embarrassed like this again because of the performance of 1 extension.

Jack,

While I appreciate your perspective here, I provided a lot of detail about the issue I was having, but you're right that I didn't supply the exact K2 version. The code that I presented was based on the K2 Blog post showing off this new possibility in using the code to display individual "extra fields" - that was v2.6.2. I have upgraded that now to 2.6.5 in hopes that the new "speed improvements" that were advertised in 2.6.2+ would have some effect here. They didn't. But, the demo version of the site is now on K2 2.6.5, nonetheless.

Please note that I didn't start with a negative tone, and I don't feel that it got negative until after there was pretty much no response. I understand that K2 is free. I also understand that it is one of the most popular extensions to the Joomla system, has a pretty big user community, and is used in tons of templates from many different vendors. As such, I was really hoping for some community or developer support.

I understand that you can't debug my problems, but honestly, neither can I and this is why I came here for ideas. If you take a look at the demo site that I've posted and go to any of the non-K2 pages (basically anything that is not the home page and is not under one of the men's or women's sports menus), you will find that the pages load just fine. I just tested this now - 19seconds for the home page - 2 seconds for several different pages without K2 on them. Even the "coaches directory", which loads up about 25 articles with articles anywhere, loads in about 3 seconds. One sports page with a schedule of 15 games take 10+ seconds to load.

This is not a problem with the site in general. When I take the two K2 content modules off the home page, the speed problems are gone. If I remove the one K2-filled tab from the sports pages, they load just fine. This is almost definitely a K2 issue. I've tested this about as many ways as I could. I use the same T3 framework from Joomlart for all of the sites I build and none of them have this issue except this one - and MANY of them use K2, just not where they load a pile of extra fields on the page where they are having to pull many k2 items with their associated extra fields per page - usually they pull k2 items for Blog purpose or like our web design portfolio does, more or less one k2 item at a time or just a list of them without extra fields.

Hey Scott,

I appreciate your frustration, however I wanted to provide you with some feedback that might shed some light on lack of response to your problems.

#1. You did not provide any meaningful specs on your software. Was it K2 2.6.5?? Are you running any K2 related plugins?? LAMP stack?? What template did you build off of??

#2. Overall negative tone of your posts.

K2 is free. People who help out usually do so out of the kindness of their heart. When I see a negative person posting about problems that I usually don't experience then I just move on.

I can't debug your problems but I can say this: When there is a 15-20 delay from your website when a page loads, it's something more than K2.

I would suggest you look at the performance and functionality using a simpler template that's not loading as much javascript. Strip the site back, test and then build up.

Also, if you are thinking of using Zoo then try a YooTheme as well. I find that templates can cause lots of problems.

Hope this helps and good luck.
Jack

Well, this post has now been out here for a month and I am nearly the only one who has posted anything to it: feel like I'm talking to myself...

I guess I have to give up and find another way to accomplish this. 20 second load times are totally unacceptable, especially for what amounts to about 25 records actually being loaded out of a table with only a few hundred in total. Ridiculous.

An not one single developer that I can tell! Really? One of the most popular extensions for Joomla and not one bit of help. One of only a handful of CCK options and not one bit of help?

Grrr.

One quick note: doing the above seems to be a supported way of accomplishing this, but it obviously has killed my site performance. I don't know why, obviously, but take the advice with a grain of salt! :(

Sawan,

I used the information from the K2 blog about how to get the values of specific fields rather than the whole list of extra fields. So I used this code in place of what's there. The only real differences are that 1. It creates an html table rather than an unordered list and 2. It specifies the fields individually rather than iterating over the whole list of fields (even though in this case I actually use all of the values, which allows the individual placement. I'm sure you could probably use the iterative loop to load the html table too if you wanted to use all the fields - this was just clearer to me. Here's the code:

Hope that helps.

Tell me one thing scott how u r converted the list form to menu tabs......



Means that, if i enter the new entry in the extra field it seems on web page like a list form
example
i select 4 field that it show like -
Award
Publication
Conference
Research

but i want to show this in menu tabs form like -
Award Publication Conference Reserch



how can it is possible give me some help

Sorely disappointed in the lack of response. I understand that K2 is a free extension, but with its popularity, I would have expected better support from the community and/or developers. I have really tried to make extensive use of the extension in this project, but it will be the last time I do so since it has rendered the site a support dilemma! Now I basically MUST go back to the drawing board to re-create what I have already done with K2 because of what appear to be pretty limiting issues.

Please someone, prove me wrong. Tell me what is going in with the slowness. Anything - really!

Really? Nothing? I guess I will have to start using Zoo for this stuff instead...like I want to learn the idiosyncrasies of yet another system. :(

:S

Weeping. Is there any support out there for this component? Pleas help - just rying to use the application the way that I believe it was designed to be used.

Now getting less patient and disappointed in the lack of feedback. I see a lot of complaints about K2 being slow for one reason or another and thought that folks got their answers about "why" and went on their way without updating the thread. Bu tnow I'm starting to get the impression that people just give up and either just accept it the way it is or they find a different way to handle the situation.

I'm sure getting to the bottom of this would help others in the community to speed up their sites as well as help the developers of the extension to make things work better.

Please help.

Waiting patiently for anyone with any sort of comments or solutions to this issue. Is this not a problem in general for folks out there? I only have about 350 K2 Items with 6 Extra Fields - I would expect this sort of behavior is I had many thousands, but this small number, only about 40 of which appear on any given, full, page, seems like it would be a no-brainer for such an industry standard Joomla extension.

Do I need to switch this over to Zoo or something else to get it to perform correctly? I'd like to be able to continue to use K2 for future sites as well as "fix" this one, but with this performance, I would need to look at other CCK options.

Any help is greatly appreciated by both me and my clients.

Anyone have any ideas on this?

So - to level the playing field a little:

I have setup a test site based on the production one and have done the following:

• Removed my custom bit of code from the mod_k2_content php file
• Turned off caching so you can see how badly it performs
• Turned on debugging so you can see the queries that are being generated
• Turned off all js and css compression options

The home page will not display the "recent results" because they depend on the query modification I put in, so this is not the best example. Use a sports page, like Mens Basketball, then the tab on that page called "schedule", which is just a list of K2 Items, specifically calling their extra fields into the page. All I did was modify the output template for this according to a recent K2 blog post about how to use extraa fields anywhere on a site. Here's the code I used to do the table output. This doesn't change anything about how many queries are run and doesn't add any additional logic to the page. It simply changes the code from iterating over "all extra fields for the item", instead directing it to use specific ones to build the table.

What seems to be the issue is that the "normal" way to use K2 would be to show a single k2 item on a page with its associated extra fields. In this case, I'm loading all of the k2 items in a category and the extra fields that go with each. Still doesn't seem like it should be killing the site.

Here's the link to the test site. I sure would like some feedback about what can be done to speed this up!

chstest.technicalrs.net

Thanks,
Scott

I have modified the k2_content routine with a conditional to check if a setting in the backend was chosen and modify the query accordingly. To validate that it wasn't my code cause the issue, I upgraded to the newest version of K2 in a test, which removed any modifications I made to the code. This didn't help. So the version you are seeing on the live site is with my modifications. When running on the new version of K2, my edits are not there and there is no notable difference in the performance.

Basically, what the code I added does is check to see if the k2 item has been marked in the backend as "within the last 45 days". If it is, then I add a bit to the query to say to only pull certain events that have a created date within the last 45 days.

In the case of the items that are on the "sports pages", the two switches that are checked are both "negative" so the query remains unmodified, so the code is identical to the original K2_content code.

So, in short, my code edits only apply on the home page examples, not the sports pages. And replacing my edited version with the original K2_content php doesn't improve things.

Good observation, but I don't think it is what's causing the issues here...

Also, I'm using the JaT3 framework and the css/js compression built in there. I have to at least combine the css files or I have issues with the Internet Explorer css limit on the Inside Sports...coaches directory page, but even with things in no compression, no minification, no combining, no cache, it still has the problem.

Sorry..

I'Ve see your code... and... your problem is your "mod_k2_content" just be sure that you don't have any error code... i see that you have many comment... and i suppose that you have a loop...

Best regards,

Hi Scott Lavelle,

I see that you have many error on your website.. i see that you use (I suppose) A third-party app to compress your Javascript.. Your "K2" javascript load before the jquery.js.

maybe just try your "page load" without your third-party app compressing files ??


Best Regards,