onchange variable #23
Labels
No Label
bugfix
cleanup
discussion
documentation
doing
done
feature
improvement
packaging
Stale
testing
TODO
No Milestone
No project
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: ungleich-public/cdist#23
Loading…
Reference in New Issue
No description provided.
Delete Branch "%!s(<nil>)"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
let's imagine following:
basically same as
require=''
, but following type will be run only if dependency generated code.@poljakowski can probably comment if this is even possible with current dependency resolver.
with this we also need non-idempotent
__execute
or__exec
or whatever... which effectively adds--onchange
to every type.I guess this is rejected so I am closing both, the issue and the MR.
Just to make it clear, I have implemented it to see if it is even possible, and to see how and how easy/hard it can be done. And it is not about "executing a type", but about "processing a type" which includes all: explorers, manifest, gencode-*.
b-b-but @poljakowski just did that. what am I missing here? if type generates code, then there is a change, some actions. sorry, but I get this feeling that we are going in totally separate paths here and we are not talking about same thing.
and please stop, for moment, thinking that this is only good for restarting some service.
With the above being said, and as all mentioned examples are related to restarting/reloading services on config changes: I do think cdist could use a proper __service type, because this is just something that everybody uses all the time.
IMHO the by far most common pattern in cdist (or any other config management system) is:
I wrote such a __service type long time ago. I'm sure some will not like it and will want to reinvent it. No problem.
What it does is:
e.g.
I once had ideas to grow this into a formal API for types to interact.
Here are some related notes from a different life.
Some other, somewhat related concept we had talked about in the past is what puppet calls run stages.
Not sure any of this is relevant for any of you. Just dumping it here for reference.
I guess my problem with this is that you are basically all saying that you want to
execute a type if something changed
.Yet cdist just does not have any concept to do this. There is no such thing as
executing a type
. Using a type in a manifest creates an object in a database. If this leads to generation of some code is entirely up to the type itself. You also don't get this in a generic way by adding this onchange feature to cdist.The following means nothing:
If the foobar service is already in it's desired state, the __service type will just do nothing, because it is not aware that the onchange'ed __file has anything to do with it. And cdist core can not do anything about it.
So you can only use this onchange feature with types that unconditionally generate code. The only type that I can imagine to be useful in this context is an __exec type. Or some other purpose built custom type.
Then there's the ad-hoc / non-reusable nature of it.
e.g. consider the given examples:
I use Ubuntu. Any types using the above constructs will not work for me.
Should such types be part of cdist?
If the above problem is solved using a custom type, it can be done in a way (or changed over time) to also work on e.g. Ubuntu.
If you really absolutely want such a 'onchange' feature for some ad-hoc solution, which I'm sure there are valid reasons for, you could just solve it with a custom type like the following.
Or not?
Yes, I know how messaging works and I have read the manual multiple times and this doesn't change anything for me. If I could get 1 € every time I have to create new type just to get some single use specific situation resolved, I wouldn't propose
onchange=""
.@nico Well, my point is not, that there is another set of problems, that this would solve. I just wanted to highlight, that this would be an easier, OOTB interface to the messaging system: any type could be messaging-aware without the said type actually reading messages.
Hence it would make the usage and writing of types easier with more concise manifests.
@fancsali Can you specify a problem that onchange would solve that messaging doesn't solve at the moment?
Thanks for the example, @ander. How this would typically be solved
From the internals:
__unpack
to have some kind of notion to find out whether it did already it's job. Maybe by checking for a specific file existence. It's important to remember that__unpack
itself should be idempotent.__download
to emit a message, when it actually downloaded something. This is good behaviour and allows loose coupling.__unpack_downloaded
that only unpacks, if__download
emitted a messageSo for nothing of above is onchange required and given the current examples I'd say that it is even confusing to support or implement it, because we do have a straight forward, easy to use solution builtin.
I strongly recommend to read https://www.cdi.st/manual/latest/cdist-messaging.html, which also references exactly the case @matze was talking about.
@nico Noting that most of them just send messages that the messaging system can be supported, there are 7 types searching through the messaging system (
grep -ri ".*grep.*\$__messages" .
should display them).I rather think the general cdist user isn't really aware about he can use the messaging system. He don't know about that it's capable or usable of doing so. Also, it can only be used in type manifests where the type that emits messages must be in the dependency, while the
onchange=
can be used in initial manifests, too. Also, did I mentioned recursion?@matze I am not sure where the impression comes from that the messaging system is not used. It is an integral part of cdist and "just not knowing about it" and trying to re-implement an existing feature does not sound very promising to me.
I count 150 occurrences with a simple check only in the upstream repo:
So it's not that nobody is using it, it's more the opposite.
I promised not to add anything, but...
imho, messaging system is unintuitive and feels like a hack. messaging support hasn't been a requirment to get type into core, but idempotency is required and that's something we always can rely on in very simple way, just like
require=''
.having conditional execution for type is not only good for "just restarting something", but for whatever actions one must do, like I mentioned in older comment. I have to run many types in order, but some types are only relevant when something else before them actually does something. with that we can also win in execution time.
idea for
onchange=''
came to me when I started to write__download
and__unpack
.something I wrote yesterday:
these two types can be chained together, because
__unpack
makes sense only when__download
happens. I'm pretty sure that once we open this door foronchange=""
we can have many such examples.so, instead of writing gencode script or coming up clever ways using messaging or writing some state information to target host, I can write (type or init) manifest, use battle-tested existing types, chain them together and have much more readable and maintainable manifest versus many times longer gencode. also don't have to reinvent the wheel all the time.
and yes, restarting some service after series of actions, is main use, because instead of having
--onchange
for every type, we have that in core, out of the box.So if "messaging solve that problem", why many struggle with it? Why there are not really a type out there utilizing the messaging system? If the messaging system would solve it, why it isn't solved in upstream types? The most common used solution are
--onchange
and executing the__service
type all times (at least this is my impression). Why there are no types that reloading services on demand upstream?I don't think we're bored and think how we can annoy you with pointless cdist proposals. It's rather a try to solve our problems. By now, we commonly have many types that are required by the service-reload type. As there isn't just one init-system, each type must develop it's own code to recursively check for messages. Also, service reloading might not the only usecase for messages. This means there will be many code out there doing the same. If you don't want copy'n'paste disasters, we need a library or an option to call a cdist-helper for it inside the scripts.
The feature we propose is to provide a cdist-native generic way of doing so. Reasons why this should be better are in other posts of this issue - I don't want to repeat me again. If you wanna refuse this feature for cdist, there should be clarification and documentation/education about how to do such things properly (as nobody except Steven came to this idea). Also, cdist should provide such types to do so.
@matze Now we are talking.
What is the problem to solve: reloading services on demand.
What is the solution: the messaging system already solves this problem
So if that's the main problem we want to solve, then we don't need onchange. If there is a different problem we want to solve, let's discuss that.
What is tired to solve? To provide a generic way of only executing a type if something changed based on current run information. Common example: service reloading on demand.
Again, to share some POV from someone, who's mainly a user...
Having
systemctl ... --if-required
and/or__check_messages
seems to me to be a bit off from the more declarativecdist
way of doing things. It's a bit like scripting in an imperative manner, within the manifest, while those should really go into the output of thegencode-*
scripts.Also, that would mean, potentially all the types I might need to apply or not apply depending on the outcome of other things need to be amended to be
--if-required
-aware. Or, I have bit of a boilerplate with__check_messages
.So, having a
onchange=
variable similar to therequire=
would be moving that logic one level higher, and allowing the users to have simpler, and yet smarter types. In other words, in my view the potential in that is that it comes out of the box...It might not make sense, but that's how I see it... ;)
I'm still standing on my original question of the problem that is tried to be solved. If we are not solving a problem, we should not add code.
I agree, we should not prolong this.
@nico @steven what is your final decision?
Something we haven't talked about is idempotency. As
onchange=
and other methods based on the messaging system only use runtime information. Given that, they "fail" to refresh the service if changes are done outside the current cdist run. That may be an abort in a cdist run or a manual change. But it doesn't did it's job in reloading the service.To be sure, the service has to be reloaded every run, or everything is checked which could be changed since the last reload. I think for most people, it's ok if the accuracy isn't 100% or if it's just based on the current run, but since types should check the current state and change it to the should-state - which is to update the service that it runs with the freshest configuration - it's something if there should come types dedicated to reload services if required.
@poljakowski thanks for your hard work for implementing and additionally explaining this feature.
yes, init and type manifests.
it's totally okay and there's no hard feelings if @nico and @steven decide not to have this feature in cdist, because I can maintain my own patches for cdist just fine. my only ask is not to delay with final decision, so I can go on with my stuff and stop waiting.
other than that, I have nothing else to add to this conversation.