Allowing true caching by DPL dev page

Hello, my name is EmuWikiAdmin1, and I realised lately that my website will need DPL to cache its output if I want to have high traffic and not overload the server with too many DPL requests. I thus decided to start this page here and start modifying the DPL code so we have a caching system for DPL. Eventually this code could be included in the official DPL releases if that's what Gero wants, or we can just keep it separated and used only by those who really need it, it will be the decision of Gero.

What's been done

• A table named dplcache is added to the mediawiki database following DPL installation. (if not already created)

• A new parameter is added to DPL : |CacheID=. User can either specify a 6 number (ex : 924924) cache ID, or he can just put CacheID=true. In the case where the user specifies a cache ID, DPL tries to find the cached content associated with this ID. The advantage is, for example, if you have multiple pages with the same DPL invocations, you could just give them the same cacheID, thus preventing from having to maintain multiple caches that would actually contain the same thing. In case the paramater is CacheID=true, then DPL chooses a random available cache ID and actually replaces the CacheID=true string with CacheID=xxxxxx in the wikitext.

• The render function of the DPL parser function is modified : before rendering, check if the CacheID parameter is used. If it is used, then don't refresh the DPL content : just output the cache content from the database. In case the CacheID parameter is present but the database does not have cache content for this cacheID, then just refresh the DPL, store it in the cache, and display the refreshed content to the user. So basically, how the cache works is that it always look for the content in the database, if it's not there, it creates it. So our expiration system will be very simple : just delete the field values associated with a cacheID in the database if we want it to expire. The next time a user goes to a page with the DPL invocation, it will get refreshed.

What's left

• Special page that would allow the administrators to empty all the DPL cache

• Rules for expiration of contents (category-related changes in articles, changes of templates used in the DPL invocation, changes in articles called by the DPL invocation). I implemented when people save or edit articles, when people delete articles, and I also made a special function to compare last edit with the new edit so we can detect when people remove categories from the article.
• What's left would be Article Move. Unfortunately the hooks of articles moves in mediawiki don't get passed the &$article object and we just have a title, we don't have access to the content. so unless mediawiki decides to pass this parameter, we would be stuck to keep huge article title lists in the tables and scan for these titles when an article is moved.
• Unfortunately, for now the way I implemented it, if people edit or save articles with Category:{{{VariableCategory}}}, category detection will work but category removal won't work. We need to find a way to get &$article object information at the InternalParseBeforeLinks hook level if we want to extract last revision content and compare it to new revision content. It will not work either for deletion of variable category for the same reason.
• Maybe a user-controllable bouton that would make the content of the cache expire.

Discussion

Add your input here.

Code

This is the current state of the code. Improvments are welcome. If you want to see only the differences from the original DPL code, search for //EmuWikiAdmin. This is alpha-quality code. Do not use on a production server. :

cachecode.txt

Note

Gresat that you started this!

1. check if the MW12 hack ("parser must be coaxed ..") is really needed. I think you can just return the cached content. Yep, really needed on 1.12. Actually the output seems ok, except if your DPL invocation you call a parser function, then the parser functions refuses to output.
   • I was thinking about downward compatibility. We must makje sure, that your solution also runs with MW1.9, 1.10, 1.11. Gero 00:22, 27 July 2008 (CEST)
2. Please use a new revision number, I recommend 1.8.0 Done
3. I don´t think the ArticleSave hook will do the job. Try ParserAfterTidy or something similar. You must get access to the parser and you need access to data structures which tell you to which categories the article being edited belongs. Great, finally realised that ParserAfterTidy do not have category information. I used InternalParseBeforeLinks instead.
4. you must also catch the "delete article" operation. Done. Article save, article edit, and change between last revision and current revision have been implemented too.

Good luck!

Gero 18:44, 26 July 2008 (CEST)