VWR-9509 and the Texture Cache [updated]

Musing on the texture cache… This is largely  in reference to VWR-9509, from a comment I was going to add but ended up using below.

So here are my big issues:

  • Something about the cache simple seems broken. With a gig of dusk space, you shouldn’t need to go back to the network si much. While it is true that on a great network connection you might be able to download pretty quickly, this isn’t a scalable approach for the SL system as a whole as it places extra load on fixed resources (LLs servers and network).
  • The current cache scheme doesn’t scale. A replacement ought to support caches of more than 100gb, IMHO. Especially in regions where Internet traffic is pay as you go, it is much more cost effective to buy a big disk (1T drives are around US$100) than pay bandwidth overage.
  • Use of j2k is a fine choice if you’ve got plenty of CPU power AND very little disk space. It is a poor choice if you want to maximize frame rate and you’ve got disk to burn. I’d like to see an option to store decoded (or lightly recompressed) files in large caches.
  • Textures aren’t necessarily ever fully downloaded, instead the viewer (at least in snowglobe) makes a guess as the level of detail required to render each texture and discards the rest.  This is great for large textures that are stuck on small objects, but results in excess network traffic and cache churn when textures end up getting rendered at multiple resolutions over time.  I suspect that in the long run it is better to download the whole texture and decode only the resolution you need (if you keep the j2k variation). [Added 1/27]

If I were going to redo this, I’d implement something like the following.  Yes, this is quite consciously like a generational garbage collector!

  • each cache segment would be a single file for data and a single index file.
  • Each index has a hard-fixed size using a bucket hash algorithm to avoid needing a linear search.
  • The index files are copies of memory indexes with the same data (key, offset, size, usetime, fmt). This is rather smaller than the current primary cache index file, which can directly store small textures.
  • Use updates should only be done on cache Hits.
  • Each data file would max at (say) 1GB.
  • Segment data would hold textures data without indexes.
  • Writes to data file are only appends, writes to index can be random access.
  • Textures are discarded only when segments are recached or compressed.
  • Textures that survive many segment compresses will be reconsidered less and less often by automatic processes, assuming near static use
  • Merge handling of static textures into the main texture cache – any key-based delivered textures should be copied into the cache and indexed that way rather than the old mechanism of trying the static skins directory for multiple filesnames on ever texture lookup from the cache.

The in-memory hash maps might be implemented with Google sparsehash (also see VWR-16363) or Boost’s unordered maps. The former has some nice features, including a nice sparse variation which might make it tractable to keep a single index structure in RAM and has the ability to save the map to disk.  The latter is simpler, possibly faster, and Boost is already included in LL’s library list (for Snowglobe, at any rate).

FWIW, since this is a viewer issue, it ought to be approachable for non lindens to debug and fix. There has been lots of discussion on the dev list as well that is applicable. Another related issue is being able to effectively cache larger amounts of data (I’d love to have something that would scale to 100g or more on reasonable FSs). KirstenLee’s viewer supports a larger cache, but I’ve no idea if it actually works better. Yet another part is the storage of textures on disk with a cheaper compression scheme without making it trivial to rip textures straight from disk.

It would help to know what the status of this is wrt viewer 2.0 and if someone in LL is actively looking at the issue. Nobody is going to want to spend a lot of time on such a serious change set if it is likely to be wasted effort.

More information on the texture cache path: http://blogs.modularsystems.sl/zwagoth/texture-system-madness/