My rough take is managing how to fill the context windows is the new "cache management", think lots of ways to do it and very specific to use cases but powerful when done with right abstraction.
I don't think we've got the abstractions right yet that make this all easy, interested to follow the more popular attempts at abstractions to help (like above). Some of these abstractions will end up being really useful, once they catch on.