@hanstutschku @tremblap @james.bradbury @o.green @Rodrigo
Thanks for joining - I have been working on some new features, and I am now at the stage where I probably need to get them out to people and make some decisions about how things should work (and what is possible).
The new features are:
- automatic multithreading of realtime processing
- the ability to run framelib in non-realtime from a max patch
Both these features have drawn attention to the need to think about what context objects are in which create some issues of interface
A bit of explanation:
- A context is a frame lib concept - a self-contained area in which objects can be connected, but objects cannot be connected between contexts
- Currently in max that is related to the patch you are in - I search the hierarchy upwards until I hit a poly~ or similar - the context is the uppermost patch I can reach before I reach a poly~ or something like that.
- Named storage is also context specific (for fl.sotre and fl.recall objects).
Why does this now matter?
- Multithreading needs switching and this happens at the context level
- Non-realtime needs to be a different context to realtime processing
For a variety of technical reasons the ideal thing (and only currently possible thing) is that objects are created in a context that does not change. So at the moment you have to put “nrt” as the first item in every single object box to make a non-realtime network - that is a real pain, but it keeps realtime and non-realtime totally separate.
That’s probably more than enough for now - sorry if this is a bit abstract - later I’ll post a version for test and start figuring out how contexts should now work with these new features - ideally you wouldn’t have to care, about any of this, but there are some technical and conceptual issues that come into play - I’ll outline them once everyone has asked for the impossible…
2 Likes
i’m glad you are using framelib to deal with “interrace” issues.
2 Likes
I’ve fixed the title. In very general terms I would like issues between races to be addressed, but I’m not sure that my programming abilities have much traction in that complex area.
1 Like
OK - there is a link to a build - this is slightly outside of the normal alpha chain, as I’m wanting to fix the interface with users who’ve spent a bit more time with frame lib before I send to a wider group
There are some very cursory demos of new features just inside the top level
Welcome @francesco.cameli - you may wish to take a peek at this…
1 Like
Could the nrt arg be something which is applied across a whole context rather than having to specify every object? or is it possible to mix and match inside of a single context?
There are several complexities here:
1 - For now objects need to know if they are realtime or not when you make them - having objects that can change their context after creation is currently unsupported by framelib itself (the bit that doesn’t know about max), and changing that is a major design change and quite likely near impossible - framelib objects have access to a context’s resources from when they construct (such as memory allocation) and following that through places a burden on the development of every single object. There may be other solutions, but they probably involve recreating objects internally, so that there is only one max object, but what is inside it changes. I’ve also considered every object having two things in it at all times, but that is very wasteful of memory.
2 - How does one specify what a “context” is, and therefore how would you specify that the whole context was non-realtime (and how would you do that so that when each object was created it knew that)?
3 - schedulers/audio inputs/audio outputs currently have a different number of audio inlets (none) when non-realtime - it seems to me likely that these will have to be dealt with explicitly and that doesn’t feel too bad, but the other objects are tedious to have to alter manually.
4 - Unfortunately, a lot of this is stuff the user shouldn’t have to worry about - it’d be nice just to patch away - and to copy a realtime network and quickly change it to a non-realtime one/vice versa, but doing that safely and under stably for the user is a problem.
Okay this makes it clearer why you have to specify. It is not so bad if you set out to make something nrt, but as you said, it would be nice to quickly convert.
That’s gnarly. Having to change every object would make this pretty intractable for any network of a reasonable size. Presumably you wouldn’t even be able to copy paste and start editing without invalidating connections between objects.
I had two thoughts on my first read of this, neither of which really work. Both founder on the problem of specifying nrt (buffer) inputs and outputs, and on only being to able to partially support the range of possible nrt usage patterns one could imagine.
- A message similar to
export which would take buffer arguments, clone a network to an nrt context, and run. (problems: how to specify i/o; large upfront cost)
- A special wrapper object that represents a nrt context, into which one could load existing patches (a la
poly~, pfft~, dynamicdsp~ etc). Still unclear how i/o could be specified.
Bonus 3rd thought
- An
fl.context object, more similar in spirit to block~ in PD, that defines the context for a whole graph if it’s found anywhere in the hierarchy for that graph. Many problems, incl. the order in which objects are created / discovered (you’d have to defer making any actual framelib objects until the whole patcher hierarchy was walked), what to do with conflicting instances.
Thanks Owen.
1 - the issue here is that the network should persist so that you can interact with max messages - it is limiting to simply have a network that is created and run once all in one go.
2 - a wrapper object would be viable, but I had tried to avoid that - it would make an odd situation where the patch was realtime when coding (not in the wrapper) but NRT when loaded.
3 - indeed specifying via a present object is problematic - not only due to object creation, but also due to the possibility for conflicts.
Any further ideas welcome!
Just as an update. I think I’ve figured out how to get (max) objects that live in dynamic contexts (the framelib objects only ever exist in a single context), and get that up and running. The code I have right now is a bit proof of concept, and it involves continuously resolving graphs until they stop changing, which is a bit inefficient, so I will look to reduce redundant calls, and speed that up for instances where a patch is being changed from realtime to non-realtime or vice versa.
I’ll update when I have something viable that doesn’t require you to explicitly set the contexts of all the processor (non scheduler/audio IO) objects
1 Like
owen’s #2 and #3 are the exact options i would have imagined myself.
the #3 could be only in top level hierarchy, ignored elsewhere. the pd model is indeed great (in some ways) and i always wish we had this model in max.
the #2 is my favourite. cycling have been toying with the idea for years of introducing a nrt subpatch context. it is the best option to me, as it translates very well to when we have framelib eventually not in a graphiccal patching context, but a more useful (!!) scripting language context.
2 rubbish petey cents
Something new for this is coming soon. For now that will just be better NRT/realtime selection. However, I’m toying with the idea of named contexts. In both cases it will/would be mandatory to state the context in every audio-related object (scheduler or audio IO) but nowhere else. This solves various things about the other solutions. to do with multiple objects/unnecessary patchers or mixing different contexts in a patch.
Thoughts?
I personally like this approach more than all the others proposed.
I was also in fact thinking of suggesting to have the nrt selection specified only at the top-most fl object (be it a scheduler or something else), with the NRT/RT mode cascading on the whole chain.
Thanks Francesco for your input. Audio objects change their number of audio inputs/outputs, hence requiring that you are explicit, but cascading is otherwise possible. I’m just ironing out some crashes/edge cases and I’ll get a build up for people to try and then I’ll see how I can improve it further.
OK - here is the new “Slovenian Mountain” version of framelib with dynamic contexts. Still a few things to look at but it hopefully shouldn’t crash.
What you should see:
- Only audio objects/schedulers can now take the “nrt” argument
- “nrt” must be specified (if required) or omitted (if not) on all relevant objects
- patching with “nrt” objects shouldn’t (mostly - soon hopefully always) interrupt audio
- for now you will see a post in the max window when the graph is updated
Please let me know of any crashes, and any times when you thing the contexts/graphs are behaving unexpectedly.
extremely impressed with alex’s apposite title for the great roglic vuelta result.
oh, and nice update.
1 Like
The internet informs me this is a sports thing. However, I have just been in the Slovenian mountains so my reasoning was somewhat different.
it is always a sports thing alex.
i hope the slovenian mountains were as fantastic as they romantically sound to me.
2 Likes
I have just updated the link above with some bug fixes for potential crashes, and to make the fl.context~ refer to the correct context again. If you downloaded yesterday then please re-download and test on the new version.