Difference between revisions of "Talk:Allowing true caching by DPL dev page"

From FollowTheScore
Jump to: navigation, search
(status update?)
 
Line 8: Line 8:
  
 
--[[User:Jb|Jb]] 03:38, 18 February 2009 (UTC)
 
--[[User:Jb|Jb]] 03:38, 18 February 2009 (UTC)
 +
 +
=== Reply ===
 +
 +
As mentioned several times the current lack of caching is the most prevailing blocking point for DPL to be used on really big sites. I would love to see a caching feature and I will support you with advice - but I cannot afford the time to create that feature myself.
 +
 +
DPL is technically very stable but of course it can slow down your response times significantly if used without care. I think in many cases it would be sufficient to have a nightly rebuild of pages containing DPL. A more sophisticated solution might want to analyse dependencies and work asynchronously in the background - but I think these two things can be separated.
 +
 +
My basic idea is as follows:
 +
# we could add a time pattern to DPL syntax which specifies the point in time where DPL must recalculate its output in the normal way (this could be done in absolute time or relative time compared to the last processing)
 +
# if a DPL statement contains such a pattern, DPL will check the time stamp of another page (which contains the result of the last DPL execution)
 +
# if the result is still considered to be acceptable (according to the time pattern) DPL will just include the content of that page instead of its normal processing.
 +
# if not, DPL will behave normally (i.e. calculate its output) AND in addition it will save its output (as wiki text) to the cache page.
 +
# Ideally the cache page would have versioning disabled (otherwise its history would create useless amounts of data in the wiki database).
 +
# I think it would be acceptable to require that a page using that cach mechanism could only have ONE DPL statement in its code - so the name of the original page plus some generated suffix could serve as a page name for the cache.
 +
#deleting the cache file manually could be a simple way to enforce a rebuild if you explicitly want the rebuild to be done right after you made changes to dependant pages(of course this requires the special right to delete a page)
 +
 +
Could you put some effort in making further investigations?
 +
 +
[[User:Gero|Gero]] 23:23, 22 February 2009 (UTC)
 +
 +
 +
 +
I would offer to integrate some syntax into the DPL code which allows DPL to decide whether it should to specify a time pattern for rebuilding.
 +
Maybe the rest of the task could even be solved as a separate extension which takes an order to rebuild a certain page at a certain time.
 +
 +
Whenever you edit/update a page containing a DPL statement with that syntax element, DPL would pass an according update order to that extension. And DPL would have to learn to include the output of that other etension instead of recalculating its output

Revision as of 00:23, 23 February 2009

status update?

Am quite interested in this concept -- as we have a high-traffic wiki with over 70k pages, and would like to use DPL on every page (several times over) to display news, images, etc. Caching and db load are a significant issue.

Any chance there is a status update? Anything I can do to help?

Maybe there is a simpler (temporary) solution?

--Jb 03:38, 18 February 2009 (UTC)

Reply

As mentioned several times the current lack of caching is the most prevailing blocking point for DPL to be used on really big sites. I would love to see a caching feature and I will support you with advice - but I cannot afford the time to create that feature myself.

DPL is technically very stable but of course it can slow down your response times significantly if used without care. I think in many cases it would be sufficient to have a nightly rebuild of pages containing DPL. A more sophisticated solution might want to analyse dependencies and work asynchronously in the background - but I think these two things can be separated.

My basic idea is as follows:

  1. we could add a time pattern to DPL syntax which specifies the point in time where DPL must recalculate its output in the normal way (this could be done in absolute time or relative time compared to the last processing)
  2. if a DPL statement contains such a pattern, DPL will check the time stamp of another page (which contains the result of the last DPL execution)
  3. if the result is still considered to be acceptable (according to the time pattern) DPL will just include the content of that page instead of its normal processing.
  4. if not, DPL will behave normally (i.e. calculate its output) AND in addition it will save its output (as wiki text) to the cache page.
  5. Ideally the cache page would have versioning disabled (otherwise its history would create useless amounts of data in the wiki database).
  6. I think it would be acceptable to require that a page using that cach mechanism could only have ONE DPL statement in its code - so the name of the original page plus some generated suffix could serve as a page name for the cache.
  7. deleting the cache file manually could be a simple way to enforce a rebuild if you explicitly want the rebuild to be done right after you made changes to dependant pages(of course this requires the special right to delete a page)

Could you put some effort in making further investigations?

Gero 23:23, 22 February 2009 (UTC)


I would offer to integrate some syntax into the DPL code which allows DPL to decide whether it should to specify a time pattern for rebuilding. Maybe the rest of the task could even be solved as a separate extension which takes an order to rebuild a certain page at a certain time.

Whenever you edit/update a page containing a DPL statement with that syntax element, DPL would pass an according update order to that extension. And DPL would have to learn to include the output of that other etension instead of recalculating its output

Retrieved from "http://followthescore.org/dpldemo/index.php?title=Talk:Allowing_true_caching_by_DPL_dev_page&oldid=11394"