- Posts: 140
COMMUNITY FORUM
K2 and it's image handling
- Uzi
-
Topic Author
- Offline
- Premium Member
I have a very large installation with K2 and a lot of pictures. The cache folder in the media folder is around 100.000 images large. I had some trouble with downloading all these pictures via FTP because most servers have a limit on the amount of items you can list. So I did some digging and found out a way to tackle this problem.
I have a comment regarding the image handling. Isn't it an idea to split up the image saving into folders with for example the year and the month (media/k2/items/src/2018/11/image.jpg , and , media/k2/items/cache/2018/11/image_S.jpg etc)
Because this way the server won't get slowed (Linux has a hard time with folders that are extremely large, filled with individual files) and you can access the files a lot easier with most configurations of the servers out there. I think also that K2 gets a speed boost if you go this route. Maybe something to think about?
Also I think it's an idea to built in a external server for spreading of the load of the website. For example use the Amazon S3 servers for your images. With a large website, images that gets pulled from the same server tend do dramatically slow the website down. A native solution where you directly upload the images to an external server would be nice.
Also regarding SIGPro, I really would like to have an option to use a external server to directly upload the images to, and also let the plugin directly generate the thumbnails (this gets done every 120 minutes standard). I think this could also drastically make the plugin more valuable and useful for websites with lots of images.
Is this something that is on the list of to do? Maybe I could help in getting this to work? I really like the K2 component and the Sigpro gallery plugin, but these 2 things are one of the most significant problems I encounter.
Thanks for reading.
Please Log in or Create an account to join the conversation.
- JoomlaWorks
-
- Offline
- Admin
- Posts: 6227
If you have space issues, you can safely clean up your "/media/k2/items/src/" folder. This is probably larger than your cache folder there and (as the name implies) it keeps the source folders uploaded. If you don't wanna do that, you can always consider "mounting" (locally or via NFS, CIFS) an external drive in /media/k2/items/ so you can offload these files to another server or storage system.
All this was common knowledge when I designed K2 back in 2008 and I don't plan on changing that behaviour. We operate some pretty big sites around the globe with more than 100,000 items and storage ranging from a few dozen GBs (with careful image planning) to a few hundred GBs (large sites that have been migrated to K2 with over 10 years of history). Never had a single problem. Using rsync we could always migrate servers without hassle (with server to server direct transfers).
As for SIGPro, I've already mentioned that the plan is to use a universal "driver" for any cloud storage to be utilized for remote (gallery) file hosting. I'll do my best for v4 of SIGPro.
Please Log in or Create an account to join the conversation.
- Uzi
-
Topic Author
- Offline
- Premium Member
- Posts: 140
Ok, looking forward to the v4 of Sigpro.
Please Log in or Create an account to join the conversation.
- JoomlaWorks
-
- Offline
- Admin
- Posts: 6227
Please Log in or Create an account to join the conversation.
- william white
-
- Offline
- Platinum Member
- Posts: 3722
The way i see it K2 is a mousetrap that comes with the cheese! Keep up the good work, looking forward to the coming advances.
Please Log in or Create an account to join the conversation.
- JoomlaWorks
-
- Offline
- Admin
- Posts: 6227
Please Log in or Create an account to join the conversation.
- Ad
-
- Offline
- New Member
- Posts: 8
I have a news portal with K2. The cache folder is the only thing that worries me in K2. I publish ~5-6 items per day (~2.000 per year). My cache folder has ~15.000 files but in a few years folder will have ~100.000 files or more. I worry about performance and hosting provider's reaction.
In my opinion, separating files in many folders (for example '/2018/12/' like in Wordpress) would be a good idea. Is it possible to implement this solution in the future releases of K2?
At the end, thank you for really great component. I am waiting for K2 Pro :)
Please Log in or Create an account to join the conversation.
- JoomlaWorks
-
- Offline
- Admin
- Posts: 6227
From a practical perspective, even in a few years, the size of the cache folder will be smaller than any regular Joomla folder where you'd allow users to upload images. That's because images in K2 are compressed and resized automatically, but users usually don't care when uploading images elsewhere. And you can safely delete the "src" folder there, which keeps the source images being uploaded into K2. We did that in case a redesign in the future requires larger images, so with a plugin, you could utilize the source folder to re-generate the cached ones. Our experience has also shown that people never really care to browse for these images. Most of the times, e.g. in large publications, there is an archive kept in /images which is used by content authors to re-publish images in K2.
Having run K2 in sites since 2008 (a year before it was publicly released), I can confirm that in around 10 years, K2 has proven more optimal in terms of disk occupied, compared to plain ol' Joomla and of course WordPress. The latter can prove a PITA especially if you change image dimensions after a redesign. In K2, it's a piece of cake. We even got a plugin for that: github.com/getk2/rebuildK2imageCache
With Joomla/K2, we've had sites with more than half a million items occupy less than 70-80GBs in total. And we've seen WordPress sites with ~20,000 posts occupy 250GBs!
Please Log in or Create an account to join the conversation.
- Ad
-
- Offline
- New Member
- Posts: 8
Please Log in or Create an account to join the conversation.
- JoomlaWorks
-
- Offline
- Admin
- Posts: 6227
Please Log in or Create an account to join the conversation.
- Iosif Chatzimichail
-
- Offline
- New Member
- Posts: 6
If you are a developer, check out this plugin I made: github.com/afonic/rimage-k2
It offers much better image handling, multiple image sets, and it splits the generated images in folders for each item so it's nice and tidy.
Plus, you can now reorder galleries!
Please Log in or Create an account to join the conversation.
- Steven Trooster
-
- Offline
- Premium Member
- Posts: 130
Please Log in or Create an account to join the conversation.
- JoomlaWorks
-
- Offline
- Admin
- Posts: 6227
If you already have a large size for your XL images, then you can safely delete the "src" folder.
Please Log in or Create an account to join the conversation.