Compare commits

...

17 commits

Author SHA1 Message Date
b4f381ed4c __ini_value: fix use of removed constant
_param was removed in preference of the param functions.
2021-09-29 20:24:22 +02:00
122354a0dc __ini_value: draft of comment handling
work in progress for a long time and I didn't touched it again till just
to commit it ... I have to work on it.
2021-09-29 20:23:01 +02:00
9d9577d891 Merge remote-tracking branch 'origin/master' into new/__ini_value 2021-09-14 18:03:42 +02:00
0d4868dcd5 __ini_value: new parameter --quote
This parameter ensures the value is surrounded by double quotes. As it
directly edits the value variable, the quoting will be added/removed
automaticly.
2021-03-20 15:39:38 +01:00
19c701e95d __ini_value: make type nonparallel
Because of the temporary files and because multiple types can change a
file at the same time, it can't be run in parallel.
2021-03-20 15:10:11 +01:00
46638f0839 __ini_value: change empty value check
Change the short-circuit check for an empty value to a more structual
approach. This might be better for the code or not ..
2021-03-17 18:17:21 +01:00
ad736a66d0 __ini_value: remove is_state 'nosuchfile'
Because the state 'nosuchfile' is exactly the same as 'absent', it was
removed with the code connected to it. It's not important if the file
exists or not - because in both cases, it contains no key-value.

Also, the code if the explorer returned 'nosuchfile' was wrong:
Completly overwrite the file with the assumption the file does not exist
is not correct, as it can return for multiple objects of the same file
and therefore, they overwrite themself.
2021-03-16 21:38:08 +01:00
8f687e5ce2 __ini_value: make shellcheck happy 2021-03-14 14:50:21 +01:00
e3e4a91abe __ini_value: moved awk script generation to gencode-remote
The script generation of the awk script was moved from `files/gen-awk.sh`
to `gencode-remote` because the size not really matters. If the file
does not exist, it will be created by a predefined template via an
here-doc to avoid a big awk-script where it is not needed.
2021-03-14 12:28:55 +01:00
7380fcaaf9 __ini_value: add missing bracket
(oops)
2021-03-13 18:46:04 +01:00
b76d848540 __ini_value: add delimiter space detection
This adds spaces around the delimiter to provide some flexibility than
just do it in the delimiter string directly. It can be set via the
parameter `--delimiter-space`.
2021-03-11 21:50:55 +01:00
9006994023 __ini_value: fix explorer detect value as commented
Because the value was not reseted on a fase positive.
2021-03-09 18:42:10 +01:00
ea7ac72f9a __ini_value: unify present + commented state awk scripts
Because the difference between the both states is just the comment sign
before it, it was now merged into one folder + symlink to avoid
redundancy. To acomplish the difference, a further parser step was
introduced to execute different print function based on the state (it is
a bit of hard-coded but ok).
2021-03-06 23:12:25 +01:00
3e67cdbf9b __ini_value: add space detection on insert
This detection tires to somehow inteligent places the new entry at the
end of the section, but tires to not infer with existing comments for
the section etc.

This detection is not perfect but will work in some cases. Hope it helps
(if someone whats to look at these files after they got edited).

This currently only applies to the `present` state, but not to the
`commented` state. This will be changed somehow when both scripts will
be unifed cause of there great similarities.
2021-03-06 22:09:59 +01:00
72c6306ba2 __ini_value: add man.rst 2021-03-06 17:46:40 +01:00
10427fde84 __ini_value: implement multi-line buffer
This buffer gives the ability to look a bit longer into the past than
just a single line. This is helpfull to get a bigger context when
fiddling around comments. This also erases the need of
`lastlinepopulated` as there is something in the pipe or nothing.
2021-03-06 17:06:32 +01:00
8c7a6906de __ini_value: starting base parts
Initial body of the __ini_value. It contains two awk-scripts: to detect
the state that is important to trigger a code generation, and to change
that line to the correct line. It might have problems which is not
matured enoght.

The most difficult points will be the comment detection, as this is a
critical point as you don't know if the user want to have this touched.
2021-03-05 20:38:19 +01:00
18 changed files with 735 additions and 0 deletions

View file

@ -0,0 +1,165 @@
#!/bin/sh -e
# __ini_value/explorer/state
# Check the state of the key-value pair in the ini file
#
# There are following states:
# - present
# - wrongvalue
# - wrongformat
# - commented
# - absent
# - nosuchfile
# Using ' \t' for matching spaces as char classes not implemented in mawk
# see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=65617#40
# Parameters
# (maybe multi-variable object id for this ..)
#state_should="$(cat "$__object/parameter/state")"
file="$(cat "$__object/parameter/file")"
# abort if no file exist
if ! [ -f "$file" ]; then
echo absent
exit
fi
# run awk
awk -f - "$file" <<'AWK'
function trim(var) {
sub(/^[ \t]*/, "", var)
sub(/[ \t]*$/, "", var)
return var
}
function check_spaces(var) {
return match(var, /^[ \t]*$/) == 1
}
function state(val) {
print val
exit
}
BEGIN {
_param = (ENVIRON["__object"] "/parameter/")
getline state_should < (_param "state")
getline section < (_param "section")
getline key < (_param "key")
getline delimiter < (_param "delimiter")
getline value < (_param "value")
getline indentation < (_param "indentation")
getline delimiter_space < (_param "delimiter-space")
do_normalization = (system("test -f " (_param "normalize")) == 0)
i=0; _comm_param = (_param "comment-sign");
while((getline tmp < _comm_param) > 0) {
comment_signs[i++] = tmp
}
if(system("test -f " (_param "quote")) == 0) {
# quote it now that it only wins checks against quoted values
value = ("\"" value "\"")
}
found=0
curr_section=""
if(section == "")
found_section=1
else
found_section=0
}
# catch sections
/^[ \t]*\[.*\][ \t]*$/ {
curr_section = trim($0)
if(found_section)
exit # game over, section ends
if(section == curr_section)
found_section=1
next
}
# only interesting if a delimiter was found
found_section {
line = $0
# index 1 cause of trimmed string
if((idel = index(line, delimiter)) && (ikey = index(line, key))) {
is_com=0
if(ikey > 1) {
# maybe comment character or only spaces
start_string = substr(line, 1, ikey - 1)
# something inside rather than a space -> comment
if((icom = match(start_string, /[^ \t]+/)) > 0) {
# icom = RSTART
# only one free-standing char or directly before the key
if(RLENGTH == 1 || icom == ikey - 1) {
start_sign = substr(line, RSTART, 1)
for(i in comment_signs) {
if(start_sign == comment_signs[i]) {
is_com = 1; break;
}
}
if(!is_com) next
else {
aftercom_length = ikey - icom - 1
if(!check_spaces(substr(line, icom + 1, aftercom_length))) next
start_spaces = (icom - 1) + aftercom_length
}
}
else next
}
# must only contain spaces
else start_spaces = ikey - 1
}
idelspace_start = ikey + length(key)
idelspace_length = idel - idelspace_start
# check for delimiter is only preceded with spaces
if(idelspace_length == 0 || check_spaces(substr(line, idelspace_start, idelspace_length))) {
found = 1
# short-circuit on state absent to just delete
if(state_should == "absent") state("present");
# extract the value
found_value = substr(line, idel + length(delimiter))
is_value = trim(found_value)
# check if value is incorrect
if(value != is_value) state("wrongvalue")
else {
# check if the format is important
if(do_normalization) {
if(match(found_value, /^[ \t]+/) == 1) {
found_value = substr(found_value, 1 + RLENGTH)
del_val_spacelen = RLENGTH
}
else
del_val_spacelen = 0
# the format must exactly match, else it is incorrect
if(start_spaces != indentation || found_value != is_value ||
idelspace_length != delimiter_space || del_val_spacelen != delimiter_space)
state("wrongformat")
}
if(is_com)
state("commented")
else
state("present")
}
# this will never be reached
}
}
}
# in the end, check if it is absent
END {
if(!found)
state("absent")
}
AWK

View file

@ -0,0 +1,138 @@
BEGIN {
bufindex = -1
buflen = 0
maxbuflen = 10
# no section means the start to the first section
if(section == "") {
is_curr_section = 1
found_section = 1
}
}
# controls the line buffer
function flush_buffer() {
while(buflen > 0)
_pop_line()
}
function flush_lines(n) {
while(buflen > 0 && n-- > 0)
_pop_line()
}
function push_line() {
linebuf[++bufindex] = $0
buflen++
while(buflen > maxbuflen) _pop_line()
}
function revert_line() {
# no delete, because it will be overwritten by the next line if any ..
bufindex--
buflen--
}
function lastline() {
if(buflen > 0) return linebuf[bufindex]
}
function pop_line() {
if(buflen > 0) _pop_line()
}
function _pop_line() {
_index = bufindex - (--buflen)
print linebuf[_index]
delete linebuf[_index]
}
# excepts the first character is the sign to check (string is trimmed)
function is_comment(line) {
# get character and check
line_sign = substr(line, 1, 1)
for(c in comment_signs)
if(line_sign == comment_signs[c])
return 1
# nothing found
return 0
}
function was_comment(line, comment) {
line = trim(line)
if(is_comment(line)) {
return trim(substr(line, 2)) == comment
}
}
# print everything if line found instead of processing it
# maybe just a function to loop through getline for lightest overhead
found {print; next}
# main loop (til the line was found)
!found {
line = trim($0)
# process if the line is not empty (or only contains spaces)
if(line != "") {
# check for a ini section
if(substr(line, 1, 1) == "[" && substr(line, length(line), 1) == "]") {
is_section = 1
curr_section = line
if(curr_section == section) {
found_section = 1
is_curr_section = 1
}
else {
# if nothing found, print it in the valid section before the next one
if(is_curr_section) {
if(!found) {
# set found as it is there now
found=1
# %codeblock_insert%
# print line as it would else only be populated below
print
next
}
is_curr_section = 0
}
}
}
else {
# only current session is interessting
if(is_curr_section) {
# check for a comment
is_com = is_comment(line)
if(is_com) {
line = trim(substr(line, 2))
}
# check for a delimiter and a key (must be at first position due to trimming)
if((idel = index(line, delimiter)) && (ikey = index(line, key)) == 1) {
# check there are only spaces between the key and delimiter
if(check_spaces(substr(line, ikey + length(key), idel - (length(key) + 1)))) {
found = 1
# %codeblock_found%
next
}
}
}
}
}
# works cause no next statement from above *structual programming*
push_line()
}
END {
# if not found, it's not already printed
if(!found) {
flush_buffer()
# print with section if not found
if(!found_section) {
# TODO check via buffer if a empty line is necessary
print section
# %codeblock_insert%
}
}
}

View file

@ -0,0 +1,57 @@
# We try to find a comment block -- how?
# check how much paragraphs it has
# check if
#
# this code is crap - at least not well written
# Check the buffer if the comment was found
function check_comments() {
_lastline = bufindex - (buflen - 1)
_comm_size = length(comments)
lastfreeline = 0
lastfreecommline = 0
comm_index = 0
# go through all lines
for(i = bufindex; i < _lastline; i++) {
_line = trim(linebuf[i])
# empty line?
if(_line == "") {
lastfreeline = i
continue
}
# line start matched
if(_line == trim(comments[comm_index])) {
# end? else continue
if(comm_index < _comm_size) {
continue
}
else {
}
}
# reset again cause not matched
else comm_index = 0
# empty comment line
if(is_comment(_line)) {
_comment = trim(substr(_line, 2))
# check if empty comment
if(_comment == "") {
lastfreecommline = i
}
}
# check if comments fit in or is too big
if((_lastline - bufindex) < _comm_size) {
# too short
}
else {
#if()
}
}
}

View file

@ -0,0 +1,68 @@
BEGIN {
# parameter variables
section = get_param_string("section")
key = get_param_string("key")
delimiter = get_param_string("delimiter")
value = get_param_string("value")
comment = get_param_string("comment")
indentation = get_param_string("indentation")
delimiter_space = get_param_string("delimiter-space")
get_param_array("comment-sign", comment_signs)
comment_sign = comment_signs[0]
if(system("test -f " (ENVIRON["__object"] "/parameter/quote")) == 0) {
# quote it now that it only wins checks against quoted values
value = ("\"" value "\"")
}
base_spaces = spaces(indentation)
delimiter_spaces = spaces(delimiter_space)
delimiter_w_spaces = (delimiter_spaces delimiter delimiter_spaces)
}
function trim(var) {
sub(/^[ \t]*/, "", var)
sub(/[ \t]*$/, "", var)
return var
}
function spaces(a) {
rspaces = ""
for(b = 0; b < a; b++)
rspaces = (rspaces " ")
return rspaces
}
function check_spaces(part) {
return match(part, /^[ \t]*$/) == 1
}
function get_param_string(name) {
_paramfile = (ENVIRON["__object"] "/parameter/" name)
if((getline tmp < _paramfile) > 0) {
close(_paramfile)
return tmp
}
else return ""
}
function get_param_array(name, arr) {
_paramfile = (ENVIRON["__object"] "/parameter/" name)
i=0
split("", arr) # portable clear, like `delete arr`
while((getline tmp < _paramfile) > 0) {
arr[i++] = tmp
}
close(_paramfile)
}
# print value
function v_print() {
printf "%s%s%s%s%s", base_spaces, key, delimiter_w_spaces, value, ORS
}
# print commented value
function v_print_commented() {
printf "%s%s%s%s%s%s", base_spaces, comment_sign, key, delimiter_w_spaces, value, ORS
}
# print comment
function c_print() {
printf "%s%s %s%s", base_spaces, comment_sign, comment, ORS
}

View file

@ -0,0 +1,5 @@
# revert line if it was a comment
if(was_comment(lastline, comment)) revert_line()
# value line was not pushed to the buffer yet
flush_buffer()

View file

@ -0,0 +1 @@
present

View file

@ -0,0 +1,8 @@
# check if last line was the comment
was_com_there = was_comment(lastline(), comment)
# print + comment if not there
flush_buffer()
if(comment && !was_com_there) c_print()
# %code_print%

View file

@ -0,0 +1,56 @@
# check if there is a comment block before the section
firstline_index = bufindex - (buflen - 1)
insertpoint = -1 # the insertpoint marks the point before the insert
lastfreespace = -1
for(i = bufindex; i >= firstline_index; i--) {
_line = trim(linebuf[i])
if(_line == "") {
lastfreespace = i
continue
}
if(comment && was_comment(_line, comment)) {
insertpoint = i + 1
no_insert_comment = 1
if(lastfreespace != insertpoint)
insert_line_after = 1
break
}
if(!is_comment(_line) || index(_line, delimiter) > 0) {
insertpoint = i + 1
# only insert a line before if we do not have a space around
if(lastfreespace == insertpoint)
insertpoint++
else
insert_line_before = 1
# check for empty line after the insert point
# use absolute boundary cause the insertpoint can be changed
if(trim(linebuf[i + 2]) != "")
insert_line_after = 1
break
}
}
# insert into the last free space
if(insertpoint == -1) {
if(lastfreespace != -1) {
insertpoint = lastfreespace
insert_line_before = 1
}
else {
insertpoint = firstline_index
insert_line_after = 1
}
}
# print lines before
flush_lines(insertpoint - firstline_index)
# print before and comment
if(insert_line_before) print ""
if(comment && !no_insert_comment) c_print()
# %code_print%
if(insert_line_after) print ""
flush_buffer()

View file

@ -0,0 +1,82 @@
#!/bin/sh -e
# __ini_value/gencode-remote
#
# Generates the code. It will generate an AWK script to add, modify or remove
# the line. The script differ in some points depend on the state. If the file
# does not exist, it will only generate the script without the awk overhead.
# strip comments and newlines for a tighter script
strip_comments() {
grep -v '^[[:space:]]*\($\|#\)'
}
state_is="$(cat "$__object/explorer/state")"
state_should="$(cat "$__object/parameter/state")"
# short-circuit if nothing to do
if [ "$state_is" = "$state_should" ]; then exit; fi
# file to change
file="$(cat "$__object/parameter/file")"
# validation check
case "$state_should" in
present|commented|absent)
# Generate the basic awk struct if a file already exists
cat <<SHELL
tmpfile="\$(mktemp '${file}.cdist.XXXXXXXX')"
if [ -f '$file' ]; then
cp -p '$file' "\$tmpfile"
fi
awk -f - '$file' > "\$tmpfile" <<'AWK'
SHELL
# generate the awk script and strip unnecessary things
{
# basic functions which everyone needs
cat "$__type/files/common.awk"
# generate the script
awk -v state="$state_should" '
function parse(line) {
if(match(line, /^[ \t]*# %code_print%$/) > 0) {
if(state == "present")
print "v_print()"
else if(state == "commented")
print "v_print_commented()"
else
print "print \"script compile error! cdist state " state " unkown!\" > /dev/stderr"
}
else print line
}
{
if(match($0, /^[ \t]*# %codeblock_([^%]+)%$/) > 0) {
split($2, result, "_"); type = substr(result[2], 1, length(result[2]) - 1)
file = (ENVIRON["__type"] "/files/parts/" state "/" type ".awk")
while((getline line < file) > 0)
parse(line)
close(file)
}
else print
}' "$__type/files/base.awk"
} | strip_comments
# end of here-doc
cat <<SHELL
AWK
mv -f "\$tmpfile" '$file'
SHELL
# Do not threat it differently if the file does not exist. It's just
# absent. Because multiple explorers can say the file does not exist,
# so the file should not be completly overwritten all times.
;;
*)
echo "not done yet!" >&2
exit 1
;;
esac

View file

@ -0,0 +1,139 @@
cdist-type__ini_value(7)
========================
NAME
----
cdist-type__ini_value - Handles ini- and conf-style configuration options
DESCRIPTION
-----------
This cdist type allow changes to more advanced key-value based configurations.
Most commonly this would be ini- or conf-style configurations.
The type can have following states:
present
The line exists with the correct value.
commented
The key-value is outcommented.
absent
The key-value line does not exist in the given section.
REQUIRED PARAMETERS
-------------------
file
The file to modify.
delimiter
The delimiter which seperates each key-value pair.
OPTIONAL PARAMETERS
-------------------
state
One of the states defined in the above section. Defaults to `present`.
section
The section where the value is located at. It always need to be surrounded
by square brackets as common for ini files. If not, the section will not be
found. If no section is specified, the block before any section is meant.
key
The key to identify the key-value pair. Must be set if the state is not
absent.
value
The value assigned to the key. Must be set if the state is not absent.
Else, an empty value is assigned to the given key.
comment
The comment which should be placed above the configuration line.
indentation
The indentation the key-value pair should have. Will be applied on inserts,
but also be enforced if ``--normalize`` is set.
comment-sign
This declares the comment signs that are valid to use in the configuration
file. Each parameter must declare a single character only; multiple
parameters are possible. It uses the first specified sign as comment
character if this type needs to insert comments.
delimiter-space
The number of spaces before and after the delimiter which should be free.
This number applies to each site of the delimiter separately, so one space
means one space to the left and right side of the delimiter.
The delimiter will be matched independendtly of this parameter and will
only be corrected if ``--normalize`` is set.
BOOLEAN PARAMETERS
------------------
normalize
This parameter enforces that the parameter is always pretty in the
configuration file. Even if a key-value pair is correct as-is, it will
correct the line to be pretty and perfect.
quote
Wrap double quotes (``"``) around the value. If the value is previously
unquoted, the file will be modified to quote the value.
MESSAGES
--------
The type currently fails to give a correct information of what he did cause of
the following construct. It has two `awk` scripts which do the job:
1. The explorer script which will outputs a single state of the given
key-value. Because the current state can contain much more states than the
state that should be, one state is returned like `wrongvalue` even if
`commented` is correct, too. Therefor, it vanishes the information that the
line is commented, too, even this could be a nice information that the
messaging system could emit.
2. The `code-remote` script also goes through the whole file and print out the
same file except the line line that should be changed. This is done because
it can not be garanteed that an other type already modifed the file, which
may moved the key-value to an other position. Then, the script replaces the
line which a pretty-printed key-value pair.
So the detected state is not important for the remote script, as it only needs
to know that it must be run cause of differences and what the state should be.
So if there are a state like `wrongvalue`, it triggers to correction of the
line, but it do not care if it was `wrongvalue`, `wrongformat` or `commented`
which trigged the run. Because of this need, the explorer retuns only an
easy-to-use value to detect if something needs to be changed.
Therefor, it is unable to correctly emit messages with the current base.
EXAMPLES
--------
.. code-block:: sh
# set a value in a configuration
__ini_value fancy-id --file /etc/foo/bar.ini --section '[welcome]' \
--key hi --value baz --delimiter ' = '
# outcomment a value
__ini_value foo --file /etc/bar/foo.conf --state commented \
--key noop --value true --delimiter ' = ' --comment 'not this time!'
AUTHORS
-------
Matthias Stecher <matthiasstecher at gmx.de>
COPYING
-------
Copyright \(C) 2021 Matthias Stecher. You can redistribute it
and/or modify it under the terms of the GNU General Public License as
published by the Free Software Foundation, either version 3 of the
License, or (at your option) any later version.

View file

View file

@ -0,0 +1,2 @@
normalize
quote

View file

@ -0,0 +1,2 @@
;
#

View file

@ -0,0 +1 @@
0

View file

@ -0,0 +1 @@
present

View file

@ -0,0 +1,7 @@
section
key
state
value
indentation
comment
delimiter-space

View file

@ -0,0 +1 @@
comment-sign

View file

@ -0,0 +1,2 @@
file
delimiter