From b3012b99114f586da045e37a2ac74985c33bcc82 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Fri, 29 Nov 2019 13:56:32 +0100 Subject: [PATCH 01/51] ++changelog --- docs/changelog | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/changelog b/docs/changelog index f6e9357c..175d9b61 100644 --- a/docs/changelog +++ b/docs/changelog @@ -1,6 +1,9 @@ Changelog --------- +next: + * Core: Redefine/reimplement/fix CDIST_ORDER_DEPENDENCY (Darko Poljak) + 6.1.1: 2019-11-26 * Type __xymon_config: Add parameters for ownership and generic rsync options (Thomas Eckert) * Type __xymon_client: Add msgcache parameter to support passive clients (Thomas Eckert) From a1f33ca8ebbeb6faae1b7f14af7fb83700a329e4 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Sat, 30 Nov 2019 23:11:36 +0100 Subject: [PATCH 02/51] Release 6.2.0 --- docs/changelog | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/changelog b/docs/changelog index 175d9b61..09e30e79 100644 --- a/docs/changelog +++ b/docs/changelog @@ -1,7 +1,7 @@ Changelog --------- -next: +6.2.0: 2019-11-30 * Core: Redefine/reimplement/fix CDIST_ORDER_DEPENDENCY (Darko Poljak) 6.1.1: 2019-11-26 From 648cdf8e29c886652c0ea20599e7b17370ed81de Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Sun, 1 Dec 2019 14:23:02 +0100 Subject: [PATCH 03/51] Add LICENSE --- LICENSE | 674 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 674 insertions(+) create mode 100644 LICENSE diff --git a/LICENSE b/LICENSE new file mode 100644 index 00000000..14682ad6 --- /dev/null +++ b/LICENSE @@ -0,0 +1,674 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + cdist + Copyright (C) 2019 ungleich-public + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + cdist Copyright (C) 2019 ungleich-public + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. From 8c0ee28b9940b765e48e839008b8f3765567da8e Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Mon, 2 Dec 2019 12:35:43 +0100 Subject: [PATCH 04/51] gitlab CI runner should have necessary tools --- .gitlab-ci.yml | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 44de6b0b..1cc17995 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -3,26 +3,16 @@ stages: unit_tests: stage: test - before_script: - - 'apk update' - - 'apk add python3' script: - ./bin/build-helper version - ./bin/build-helper test pycodestyle: stage: test - before_script: - - 'apk update' - - 'apk add python3 py3-pycodestyle' - - 'ln -f -s /usr/bin/pycodestyle-3 /usr/bin/pycodestyle' script: - ./bin/build-helper pycodestyle shellcheck: stage: test - before_script: - - 'wget https://storage.googleapis.com/shellcheck/shellcheck-stable.linux.x86_64.tar.xz' - - 'tar xf shellcheck-stable.linux.x86_64.tar.xz && mv shellcheck-stable/shellcheck /usr/bin/' script: - ./bin/build-helper shellcheck From d25c72e678f8d4fb311a9800a3097e1bf6e9bf16 Mon Sep 17 00:00:00 2001 From: Dominique Roux Date: Tue, 3 Dec 2019 10:13:46 +0100 Subject: [PATCH 05/51] [PACKAGE-UPDATE-INDEX]: Bugfix of the alpine part of the __package_update_index type --- cdist/conf/type/__package_update_index/gencode-remote | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/cdist/conf/type/__package_update_index/gencode-remote b/cdist/conf/type/__package_update_index/gencode-remote index 9b2ecba2..6c51cbed 100755 --- a/cdist/conf/type/__package_update_index/gencode-remote +++ b/cdist/conf/type/__package_update_index/gencode-remote @@ -47,9 +47,9 @@ case "$type" in echo "pacman --noprogressbar --sync --refresh" echo "pacman package database synced (age was: $currage)" >> "$__messages_out" ;; - alpine) + apk) echo "apk update" - echo "apk package database updated." + echo "apk package database updated." >>"$__messages_out" ;; *) echo "Don't know how to manage packages for type: $type" >&2 From 320f962e1df7f27909e6157e8dfc6033706c8ddd Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Wed, 4 Dec 2019 22:27:58 +0100 Subject: [PATCH 06/51] ++changelog --- docs/changelog | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/changelog b/docs/changelog index 09e30e79..8bdee928 100644 --- a/docs/changelog +++ b/docs/changelog @@ -1,6 +1,9 @@ Changelog --------- +next: + * Type __package_update_index: Fix Alpine part (Dominique Roux) + 6.2.0: 2019-11-30 * Core: Redefine/reimplement/fix CDIST_ORDER_DEPENDENCY (Darko Poljak) From 4f21bf534dd91c7395845beb08715ad6c84bafef Mon Sep 17 00:00:00 2001 From: Evilham Date: Thu, 5 Dec 2019 23:14:27 +0100 Subject: [PATCH 07/51] [docs] PreOS: English nitpicking. --- cdist/argparse.py | 2 +- docs/src/cdist-preos.rst | 70 ++++++++++++++++++++-------------------- 2 files changed, 36 insertions(+), 36 deletions(-) diff --git a/cdist/argparse.py b/cdist/argparse.py index ca69cdae..ed94d266 100644 --- a/cdist/argparse.py +++ b/cdist/argparse.py @@ -424,7 +424,7 @@ def get_parsers(): parser['inventory'].set_defaults( func=cdist.inventory.Inventory.commandline) - # PreOs + # PreOS parser['preos'] = parser['sub'].add_parser('preos', add_help=False) # Shell diff --git a/docs/src/cdist-preos.rst b/docs/src/cdist-preos.rst index e85af2de..2f102fef 100644 --- a/docs/src/cdist-preos.rst +++ b/docs/src/cdist-preos.rst @@ -4,9 +4,9 @@ PreOS Description ----------- With cdist you can install and configure new machines. You can use cdist to -create PreOS, minimal OS which purpose is to boot new machine. -After PreOS is booted machine is ready for installing desired OS and -then it is ready for configuration. +create PreOS, minimal OS whose purpose is to boot a new machine. +After PreOS is booted, the machine is ready for installing the desired OS and +afterwards it is ready for configuration. PreOS creation -------------- @@ -15,62 +15,62 @@ Currently supported PreOS-es include: * debian * ubuntu -* devuan. +* devuan -PreOS is created using cdist preos command. preos command has subcommands that -create the desired PreOS. +PreOS is created using the ``cdist preos`` command. +This command has subcommands that determine the desired PreOS. -For example, to create ubuntu PreOS: +For example, to create an ubuntu PreOS: .. code-block:: sh $ cdist preos ubuntu /preos/preos-ubuntu -B -C \ -k ~/.ssh/id_rsa.pub -p /preos/pxe-ubuntu -For more info about available options see cdist manual page. +For more info about the available options see the cdist manual page. -This will bootstrap (``-B``) ubuntu PreOS in ``/preos/preos-ubuntu`` directory, it -will be configured (``-C``) using default built-in initial manifest and with -specified ssh authorized key (``-k``). -After bootstrapping and configuration PXE -boot directory will be created (``-p``) in ``/preos/pxe-ubuntu``. +This will bootstrap (``-B``) ubuntu PreOS in the ``/preos/preos-ubuntu`` +directory, it will be configured (``-C``) using default the built-in initial +manifest and with specified ssh authorized key (``-k``). +After bootstrapping and configuration, the PXE boot directory will be +created (``-p``) in ``/preos/pxe-ubuntu``. -After PreOS is created new machines can be booted using created PXE (after -proper dhcp, tftp setting). +After PreOS is created, new machines can be booted using the created PXE +(after proper dhcp and tftp settings). -Since PreOS is configured with ssh authorized key it can be accessed throguh +Since PreOS is configured with ssh authorized key it can be accessed through ssh, i.e. it can be further installed and configured with cdist. -Implementing new PreOS sub-command ----------------------------------- +Implementing a new PreOS sub-command +------------------------------------ preos command is implemented as a plugin system. This plugin system scans for -preos subcommands in ``cdist/preos/`` distribution directory and also in +preos subcommands in the ``cdist/preos/`` distribution directory and also in ``~/.cdist/preos/`` directory if it exists. preos subcommand is a module or a class that satisfies the following: -* it has attribute ``_cdist_preos`` set to ``True`` -* it has function/method ``commandline``. +* it has the attribute ``_cdist_preos`` set to ``True`` +* it defines a function/method ``commandline``. -For a module based preos subcommand ``commandline`` function accepts a module -object as its first argument and the list of command line +For a module-based preos subcommand, the ``commandline`` function accepts a +module object as its first argument and the list of command line arguments (``sys.argv[2:]``). -For a class preos subcommand ``commandline`` method should be staticmethod and -it accepts a class object as its first argument and the list of command line -arguments(``sys.argv[2:]``). +For a class-based preos subcommand ``commandline`` method should be +static-method and must accept a class as its first argument and the +list of command line arguments (``sys.argv[2:]``). If preos scanning finds a module/class that has ``_cdist_preos`` set -to ``True`` and it has function/method ``commandline`` then this module/class is +to ``True`` and a function/method ``commandline`` then this module/class is registered to preos subcommands. The name of the command is set to ``_preos_name`` -attribute if it exists, otherwise it is set to the module/class name, lowercase. -When registered preos subcommand is specified as preos command then ``commandline`` -will be called with first argument set to module/class object and second argument -set to ``sys.argv[2:]``. +attribute if defined in the module/class, defaulting to the module/class name in lowercase. +When a registered preos subcommand is specified, ``commandline`` +will be called with the first argument set to module/class and the second +argument set to ``sys.argv[2:]``. -Example writing new dummy preos sub-command -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Module based preos: +Example of writing new dummy preos sub-command +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Module-based preos: ^^^^^^^^^^^^^^^^^^^ #. Create directory ``~/.cdist/preos/`` if it does not exist @@ -134,4 +134,4 @@ When you try to run this new preos you will get: FreeBSD dummy preos: [] In the ``commandline`` function/method you have all the freedom to actually create -PreOS. +a PreOS. From f22349ce8a6571601478a76bfe3b68bb574154ba Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Fri, 6 Dec 2019 08:56:57 +0100 Subject: [PATCH 08/51] Fix documentation for install types Fixes gitlab #790. --- cdist/conf/type/__install_directory/man.rst | 102 +++++++++++++++++++- cdist/conf/type/__install_file/man.rst | 14 ++- docs/changelog | 1 + 3 files changed, 115 insertions(+), 2 deletions(-) mode change 120000 => 100644 cdist/conf/type/__install_directory/man.rst diff --git a/cdist/conf/type/__install_directory/man.rst b/cdist/conf/type/__install_directory/man.rst deleted file mode 120000 index 1ad7fa84..00000000 --- a/cdist/conf/type/__install_directory/man.rst +++ /dev/null @@ -1 +0,0 @@ -../__directory/man.rst \ No newline at end of file diff --git a/cdist/conf/type/__install_directory/man.rst b/cdist/conf/type/__install_directory/man.rst new file mode 100644 index 00000000..c402cbad --- /dev/null +++ b/cdist/conf/type/__install_directory/man.rst @@ -0,0 +1,101 @@ +cdist-type__install_directory(7) +================================ + +NAME +---- +cdist-type__install_directory - Manage a directory with install command + + +DESCRIPTION +----------- +This cdist type allows you to create or remove directories on the target. + + +REQUIRED PARAMETERS +------------------- +None. + + +OPTIONAL PARAMETERS +------------------- +state + 'present' or 'absent', defaults to 'present' + +group + Group to chgrp to. + +mode + Unix permissions, suitable for chmod. + +owner + User to chown to. + + +BOOLEAN PARAMETERS +------------------ +parents + Whether to create parents as well (mkdir -p behaviour). + Warning: all intermediate directory permissions default + to whatever mkdir -p does. + + Usually this means root:root, 0700. + +recursive + If supplied the chgrp and chown call will run recursively. + This does *not* influence the behaviour of chmod. + +MESSAGES +-------- +chgrp + Changed group membership +chown + Changed owner +chmod + Changed mode +create + Empty directory was created +remove + Directory exists, but state is absent, directory will be removed by generated code. +remove non directory + Something other than a directory with the same name exists and was removed prior to create. + + +EXAMPLES +-------- + +.. code-block:: sh + + # A silly example + __install_directory /tmp/foobar + + # Remove a directory + __install_directory /tmp/foobar --state absent + + # Ensure /etc exists correctly + __install_directory /etc --owner root --group root --mode 0755 + + # Create nfs service directory, including parents + __install_directory /home/services/nfs --parents + + # Change permissions recursively + __install_directory /home/services --recursive --owner root --group root + + # Setup a temp directory + __install_directory /local --mode 1777 + + # Take it all + __install_directory /home/services/kvm --recursive --parents \ + --owner root --group root --mode 0755 --state present + + +AUTHORS +------- +Nico Schottelius + + +COPYING +------- +Copyright \(C) 2011 Nico Schottelius. You can redistribute it +and/or modify it under the terms of the GNU General Public License as +published by the Free Software Foundation, either version 3 of the +License, or (at your option) any later version. diff --git a/cdist/conf/type/__install_file/man.rst b/cdist/conf/type/__install_file/man.rst index c5409167..977ed77c 100644 --- a/cdist/conf/type/__install_file/man.rst +++ b/cdist/conf/type/__install_file/man.rst @@ -23,6 +23,10 @@ symlink directory replace it with the source file +One exception is that when state is pre-exists, an error is raised if +the file would have been created otherwise (e.g. it is not present or +not a regular file). + In any case, make sure that the file attributes are as specified. @@ -33,7 +37,7 @@ None. OPTIONAL PARAMETERS ------------------- state - 'present', 'absent' or 'exists', defaults to 'present' where: + 'present', 'absent', 'exists' or 'pre-exists', defaults to 'present' where: present the file is exactly the one from source @@ -41,6 +45,9 @@ state the file does not exist exists the file from source but only if it doesn't already exist + pre-exists + check that the file exists and is a regular file, but do not + create or modify it group Group to chgrp to. @@ -56,6 +63,9 @@ source If not supplied, an empty file or directory will be created. If source is '-' (dash), take what was written to stdin as the file content. +onchange + The code to run if file is modified. + MESSAGES -------- chgrp @@ -93,6 +103,8 @@ EXAMPLES __install_file /home/frodo/.bashrc --source "/etc/skel/.bashrc" \ --state exists \ --owner frodo --mode 0600 + # Check that the file is present, show an error when it is not + __install_file /etc/somefile --state pre-exists # Take file content from stdin __install_file /tmp/whatever --owner root --group root --mode 644 --source - << DONE Here goes the content for /tmp/whatever diff --git a/docs/changelog b/docs/changelog index 8bdee928..ed993609 100644 --- a/docs/changelog +++ b/docs/changelog @@ -3,6 +3,7 @@ Changelog next: * Type __package_update_index: Fix Alpine part (Dominique Roux) + * Documentation: Fix man pages for install types (Darko Poljak) 6.2.0: 2019-11-30 * Core: Redefine/reimplement/fix CDIST_ORDER_DEPENDENCY (Darko Poljak) From 90e85d81285a1ed60085b4df0a1618e4aeaa8c6f Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Fri, 6 Dec 2019 09:37:21 +0100 Subject: [PATCH 09/51] Embed config skeleton instead of rewriting it Resolves #796. --- .gitignore | 1 + Makefile | 8 +++- docs/changelog | 1 + docs/src/cdist-configuration.rst | 70 ++------------------------------ 4 files changed, 13 insertions(+), 67 deletions(-) diff --git a/.gitignore b/.gitignore index ed8b453a..fb831051 100644 --- a/.gitignore +++ b/.gitignore @@ -24,6 +24,7 @@ docs/src/man1/*.1 docs/src/man7/*.7 docs/src/man7/cdist-type__*.rst docs/src/cdist-reference.rst +docs/src/cdist.cfg.skeleton # Ignore cdist cache for version control /cache/ diff --git a/Makefile b/Makefile index fa3327d1..e6a2a28e 100644 --- a/Makefile +++ b/Makefile @@ -63,6 +63,11 @@ DOCSREFSH=$(DOCS_SRC_DIR)/cdist-reference.rst.sh $(DOCSREF): $(DOCSREFSH) $(DOCSREFSH) +DOCSCFGSKEL=./configuration/cdist.cfg.skeleton + +configskel: $(DOCSCFGSKEL) + cp -f "$(DOCSCFGSKEL)" "$(DOCS_SRC_DIR)/" + version: @[ -f "cdist/version.py" ] || { \ printf "Missing 'cdist/version.py', please generate it first.\n" && exit 1; \ @@ -72,7 +77,7 @@ version: man: version $(MANTYPES) $(DOCSREF) $(SPHINXM) -html: version $(MANTYPES) $(DOCSREF) +html: version configskel $(MANTYPES) $(DOCSREF) $(SPHINXH) docs: man html @@ -114,6 +119,7 @@ speeches: $(SPEECHES) # clean: docs-clean rm -f $(DOCS_SRC_DIR)/cdist-reference.rst + rm -f $(DOCS_SRC_DIR)/cdist.cfg.skeleton find "$(DOCS_SRC_DIR)" -mindepth 2 -type l \ | xargs rm -f diff --git a/docs/changelog b/docs/changelog index ed993609..ca33d7bd 100644 --- a/docs/changelog +++ b/docs/changelog @@ -4,6 +4,7 @@ Changelog next: * Type __package_update_index: Fix Alpine part (Dominique Roux) * Documentation: Fix man pages for install types (Darko Poljak) + * Documentation: Embed config skeleton instead of rewriting it (Darko Poljak) 6.2.0: 2019-11-30 * Core: Redefine/reimplement/fix CDIST_ORDER_DEPENDENCY (Darko Poljak) diff --git a/docs/src/cdist-configuration.rst b/docs/src/cdist-configuration.rst index 4c9b4d33..706ed761 100644 --- a/docs/src/cdist-configuration.rst +++ b/docs/src/cdist-configuration.rst @@ -31,73 +31,11 @@ can be used. Config file format ------------------ + cdist configuration file is in the INI file format. Currently it supports only [GLOBAL] section. -The possible keywords and their meanings are as follows: -:strong:`archiving` - Use specified archiving. Valid values include: - 'none', 'tar', 'tgz', 'tbz2' and 'txz'. +Here you can find configuration file skeleton: -:strong:`beta` - Enable beta functionality. It recognizes boolean values from - 'yes'/'no', 'on'/'off', 'true'/'false' and '1'/'0'. - -:strong:`cache_path_pattern` - Specify cache path pattern. - -:strong:`conf_dir` - List of configuration directories separated with the character conventionally - used by the operating system to separate search path components (as in PATH), - such as ':' for POSIX or ';' for Windows. - If also specified at command line then values from command line are - appended to this value. - -:strong:`init_manifest` - Specify default initial manifest. - -:strong:`inventory_dir` - Specify inventory directory. - -:strong:`jobs` - Specify number of jobs for parallel processing. If -1 then the default, - number of CPU's in the system is used. If 0 then parallel processing in - jobs is disabled. If set to positive number then specified maximum - number of processes will be used. - -:strong:`local_shell` - Shell command used for local execution. - -:strong:`out_path` - Directory to save cdist output in. - -:strong:`parallel` - Process hosts in parallel. If -1 then the default, number of CPU's in - the system is used. If 0 then parallel processing of hosts is disabled. - If set to positive number then specified maximum number of processes - will be used. - -:strong:`remote_copy` - Command to use for remote copy (should behave like scp). - -:strong:`remote_exec` - Command to use for remote execution (should behave like ssh). - -:strong:`remote_out_path` - Directory to save cdist output in on the target host. - -:strong:`remote_shell` - Shell command at remote host used for remote execution. - -:strong:`save_output_streams` - Enable/disable saving output streams (enabled by default). - It recognizes boolean values from 'yes'/'no', 'on'/'off', 'true'/'false' - and '1'/'0'. - -:strong:`timestamp` - Timestamp log messages with the current local date and time - in the format: YYYYMMDDHHMMSS.us. - -:strong:`verbosity` - Set verbosity level. Valid values are: - 'ERROR', 'WARNING', 'INFO', 'VERBOSE', 'DEBUG', 'TRACE' and 'OFF'. +.. literalinclude:: cdist.cfg.skeleton + :language: ini From 71b6646b38799f7316d791a4d9b511098d879ab0 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Fri, 6 Dec 2019 10:00:32 +0100 Subject: [PATCH 10/51] Rm cdist-type prefix and man page ref from docs E.g. 16.10. __block 16.11. __ccollect_source 16.12. __cdist 16.13. __cdistmarker 16.14. __check_messages insted of 16.10. cdist-type__block(7) 16.11. cdist-type__ccollect_source(7) 16.12. cdist-type__cdist(7) 16.13. cdist-type__cdistmarker(7) 16.14. cdist-type__check_messages(7) --- .gitignore | 1 + Makefile | 10 ++++++- docs/src/cdist-types.rst | 8 ------ docs/src/cdist-types.rst.sh | 55 +++++++++++++++++++++++++++++++++++++ 4 files changed, 65 insertions(+), 9 deletions(-) delete mode 100644 docs/src/cdist-types.rst create mode 100755 docs/src/cdist-types.rst.sh diff --git a/.gitignore b/.gitignore index fb831051..85a8ccc7 100644 --- a/.gitignore +++ b/.gitignore @@ -24,6 +24,7 @@ docs/src/man1/*.1 docs/src/man7/*.7 docs/src/man7/cdist-type__*.rst docs/src/cdist-reference.rst +docs/src/cdist-types.rst docs/src/cdist.cfg.skeleton # Ignore cdist cache for version control diff --git a/Makefile b/Makefile index e6a2a28e..f89ac1e7 100644 --- a/Makefile +++ b/Makefile @@ -63,6 +63,13 @@ DOCSREFSH=$(DOCS_SRC_DIR)/cdist-reference.rst.sh $(DOCSREF): $(DOCSREFSH) $(DOCSREFSH) +# Html types list with references +DOCSTYPESREF=$(MAN7DSTDIR)/cdist-types.rst +DOCSTYPESREFSH=$(DOCS_SRC_DIR)/cdist-types.rst.sh + +$(DOCSTYPESREF): $(DOCSTYPESREFSH) + $(DOCSTYPESREFSH) + DOCSCFGSKEL=./configuration/cdist.cfg.skeleton configskel: $(DOCSCFGSKEL) @@ -77,7 +84,7 @@ version: man: version $(MANTYPES) $(DOCSREF) $(SPHINXM) -html: version configskel $(MANTYPES) $(DOCSREF) +html: version configskel $(MANTYPES) $(DOCSREF) $(DOCSTYPESREF) $(SPHINXH) docs: man html @@ -119,6 +126,7 @@ speeches: $(SPEECHES) # clean: docs-clean rm -f $(DOCS_SRC_DIR)/cdist-reference.rst + rm -f $(DOCS_SRC_DIR)/cdist-types.rst rm -f $(DOCS_SRC_DIR)/cdist.cfg.skeleton find "$(DOCS_SRC_DIR)" -mindepth 2 -type l \ diff --git a/docs/src/cdist-types.rst b/docs/src/cdist-types.rst deleted file mode 100644 index d5104667..00000000 --- a/docs/src/cdist-types.rst +++ /dev/null @@ -1,8 +0,0 @@ -cdist types -=========== - -.. toctree:: - :titlesonly: - :glob: - - man7/* diff --git a/docs/src/cdist-types.rst.sh b/docs/src/cdist-types.rst.sh new file mode 100755 index 00000000..2eb66009 --- /dev/null +++ b/docs/src/cdist-types.rst.sh @@ -0,0 +1,55 @@ +#!/bin/sh +# +# 2019 Darko Poljak (darko.poljak at gmail.com) +# +# This file is part of cdist. +# +# cdist is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# cdist is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with cdist. If not, see . +# +# +# Generate cdist-types.rst that lists available types. +# + +__cdist_pwd="$(pwd -P)" +__cdist_mydir="${0%/*}"; +__cdist_abs_mydir="$(cd "$__cdist_mydir" && pwd -P)" +__cdist_myname=${0##*/}; +__cdist_abs_myname="$__cdist_abs_mydir/$__cdist_myname" + +filename="${__cdist_myname%.sh}" +dest="$__cdist_abs_mydir/$filename" + +cd "$__cdist_abs_mydir" + +exec > "$dest" +cat << eof +cdist types +=========== + +.. toctree:: + :titlesonly: + +eof + +# If there is no such file then ls prints error to stderr, +# so redirect stderr to /dev/null. +for type in $(ls man7/cdist-type__*.rst 2>/dev/null | LC_ALL=C sort); do + no_dir="${type#man7/}"; + no_type="${no_dir#cdist-type}"; + name="${no_type%.rst}"; + manref="${no_dir%.rst}" + man="${manref}(7)" + + echo " $name" "" +done From f2cbc01e3f885c41962a1e474922127c49099203 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Fri, 6 Dec 2019 10:04:06 +0100 Subject: [PATCH 11/51] ++changelog --- docs/changelog | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/changelog b/docs/changelog index ca33d7bd..bf0398a9 100644 --- a/docs/changelog +++ b/docs/changelog @@ -5,6 +5,7 @@ next: * Type __package_update_index: Fix Alpine part (Dominique Roux) * Documentation: Fix man pages for install types (Darko Poljak) * Documentation: Embed config skeleton instead of rewriting it (Darko Poljak) + * Documentation: Remove cdist-type prefix and man page reference from type list in html (Darko Poljak) 6.2.0: 2019-11-30 * Core: Redefine/reimplement/fix CDIST_ORDER_DEPENDENCY (Darko Poljak) From 839e7a408e199cd5a0884c25e283c9bebe13e26d Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Fri, 6 Dec 2019 19:46:17 +0100 Subject: [PATCH 12/51] ++changelog --- docs/changelog | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/changelog b/docs/changelog index bf0398a9..f4e38578 100644 --- a/docs/changelog +++ b/docs/changelog @@ -6,6 +6,7 @@ next: * Documentation: Fix man pages for install types (Darko Poljak) * Documentation: Embed config skeleton instead of rewriting it (Darko Poljak) * Documentation: Remove cdist-type prefix and man page reference from type list in html (Darko Poljak) + * Documentation: PreOS english nitpicking (Evil Ham) 6.2.0: 2019-11-30 * Core: Redefine/reimplement/fix CDIST_ORDER_DEPENDENCY (Darko Poljak) From d5ac9ea348aa0ed640941232fbc2c783d95b8750 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Sun, 8 Dec 2019 16:34:38 +0100 Subject: [PATCH 13/51] Docs: install from source, verify signature Add chapter for installing from source with signature verification. Resolves #795. --- docs/changelog | 1 + docs/src/cdist-install.rst | 27 +++++++++++++++++++++++++-- 2 files changed, 26 insertions(+), 2 deletions(-) diff --git a/docs/changelog b/docs/changelog index f4e38578..33db10da 100644 --- a/docs/changelog +++ b/docs/changelog @@ -7,6 +7,7 @@ next: * Documentation: Embed config skeleton instead of rewriting it (Darko Poljak) * Documentation: Remove cdist-type prefix and man page reference from type list in html (Darko Poljak) * Documentation: PreOS english nitpicking (Evil Ham) + * Documentation: Add installing from source with signature verification (Darko Poljak) 6.2.0: 2019-11-30 * Core: Redefine/reimplement/fix CDIST_ORDER_DEPENDENCY (Darko Poljak) diff --git a/docs/src/cdist-install.rst b/docs/src/cdist-install.rst index a9b7d6b5..880b9f8e 100644 --- a/docs/src/cdist-install.rst +++ b/docs/src/cdist-install.rst @@ -23,8 +23,6 @@ Target Hosts Install cdist ------------- -You can install cdist either from git or as a python package. - From git ~~~~~~~~ @@ -42,6 +40,7 @@ To install cdist, execute the following commands: From version 4.2.0 cdist tags and releases are signed. You can get GPG public key used for signing `here <_static/pgp-key-EFD2AE4EC36B6901.asc>`_. +It is assumed that you are familiar with *git* ways of signing and verification. You can also get cdist from `github mirror `_. @@ -157,3 +156,27 @@ Cdist is available as a python package at .. code-block:: sh pip install cdist + +Installing from source with signature verification +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you want to install cdist from signed source and verify it, first you need to +download cdist archive and its detached signature. + +Get both, *cdist-x.y.z.tar.gz* and *cdist-x.y.z.tar.gz.asc* from release +notes of the desired tag *x.y.z* at +`cdist git repository `_. + +Get GPG public key used for signing `here <_static/pgp-key-EFD2AE4EC36B6901.asc>`_ +and import it into GPG. + +Now cdist source archive can be verified using `gpg`, e.g. to verify `cdist-6.2.0`: + +.. code-block:: sh + + $ gpg --verify cdist-6.2.0.tar.gz.asc cdist-6.2.0.targ.gz + gpg: Signature made Sat Nov 30 23:14:19 2019 CET + gpg: using RSA key 69767822F3ECC3C349C1EFFFEFD2AE4EC36B6901 + gpg: Good signature from "ungleich GmbH (ungleich FOSS) " [ultimate] + +Further steps are the same as for `installing from git `_. From 4735df1bed2b23db8c16f568280ef1c5a8ed4c03 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Fri, 6 Dec 2019 19:40:05 +0100 Subject: [PATCH 14/51] Add plugins-dir preos option --- cdist/argparse.py | 2 +- cdist/preos.py | 72 +++++++++++++++++++++++++++++----------- docs/src/cdist-preos.rst | 30 ++++++----------- docs/src/man1/cdist.rst | 52 +++++++++++++++++------------ 4 files changed, 95 insertions(+), 61 deletions(-) diff --git a/cdist/argparse.py b/cdist/argparse.py index ed94d266..7dc683f3 100644 --- a/cdist/argparse.py +++ b/cdist/argparse.py @@ -103,7 +103,7 @@ def get_parsers(): name="log level"), help=('Set the specified verbosity level. ' 'The levels, in order from the lowest to the highest, are: ' - 'ERROR (-1), WARNING (0), INFO (1), VERBOSE (2), DEBUG (3) ' + 'ERROR (-1), WARNING (0), INFO (1), VERBOSE (2), DEBUG (3), ' 'TRACE (4 or higher). If used along with -v then -v ' 'increases last set value and -l overwrites last set ' 'value.'), diff --git a/cdist/preos.py b/cdist/preos.py index 46b45554..378071db 100644 --- a/cdist/preos.py +++ b/cdist/preos.py @@ -5,6 +5,8 @@ import inspect import argparse import cdist import logging +import re +import cdist.argparse _PREOS_CALL = "commandline" @@ -12,15 +14,24 @@ _PREOS_NAME = "_preos_name" _PREOS_MARKER = "_cdist_preos" _PLUGINS_DIR = "preos" _PLUGINS_PATH = [os.path.join(os.path.dirname(__file__), _PLUGINS_DIR), ] +log = logging.getLogger("PreOS") + + +def extend_plugins_path(dirs): + for dir in dirs: + preos_dir = os.path.expanduser(os.path.join(dir, "preos")) + if os.path.isdir(preos_dir): + _PLUGINS_PATH.append(preos_dir) + + cdist_home = cdist.home_dir() if cdist_home: - cdist_home_preos = os.path.join(cdist_home, "preos") - if os.path.isdir(cdist_home_preos): - _PLUGINS_PATH.append(cdist_home_preos) -sys.path.extend(_PLUGINS_PATH) - - -log = logging.getLogger("PreOS") + extend_plugins_path((cdist_home, )) +x = 'CDIST_PATH' +if x in os.environ: + vals = re.split(r'(? Date: Mon, 9 Dec 2019 09:57:54 +0100 Subject: [PATCH 15/51] ++changelog --- docs/changelog | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/changelog b/docs/changelog index 33db10da..04ffac62 100644 --- a/docs/changelog +++ b/docs/changelog @@ -8,6 +8,7 @@ next: * Documentation: Remove cdist-type prefix and man page reference from type list in html (Darko Poljak) * Documentation: PreOS english nitpicking (Evil Ham) * Documentation: Add installing from source with signature verification (Darko Poljak) + * Core: preos: Support top command logging options, custom conf-dir option and CDIST_PATH env var (Darko Poljak) 6.2.0: 2019-11-30 * Core: Redefine/reimplement/fix CDIST_ORDER_DEPENDENCY (Darko Poljak) From bee95efa4317eacbde98237f7952d020cdfb5e61 Mon Sep 17 00:00:00 2001 From: Evilham Date: Mon, 9 Dec 2019 19:13:10 +0100 Subject: [PATCH 16/51] [new-type] __openldap_server Originally developed at ungleich and improved for Open Sourcing by Evilham. --- cdist/conf/type/__openldap_server/man.rst | 148 +++++++++++ cdist/conf/type/__openldap_server/manifest | 235 ++++++++++++++++++ .../type/__openldap_server/parameter/boolean | 2 + .../type/__openldap_server/parameter/optional | 4 + .../parameter/optional_multiple | 2 + .../type/__openldap_server/parameter/required | 4 + 6 files changed, 395 insertions(+) create mode 100644 cdist/conf/type/__openldap_server/man.rst create mode 100644 cdist/conf/type/__openldap_server/manifest create mode 100644 cdist/conf/type/__openldap_server/parameter/boolean create mode 100644 cdist/conf/type/__openldap_server/parameter/optional create mode 100644 cdist/conf/type/__openldap_server/parameter/optional_multiple create mode 100644 cdist/conf/type/__openldap_server/parameter/required diff --git a/cdist/conf/type/__openldap_server/man.rst b/cdist/conf/type/__openldap_server/man.rst new file mode 100644 index 00000000..29bbc231 --- /dev/null +++ b/cdist/conf/type/__openldap_server/man.rst @@ -0,0 +1,148 @@ +cdist-type__openldap_server(7) +============================== + +NAME +---- +cdist-type__openldap_server - Setup an openldap(4) server instance + + +DESCRIPTION +----------- +This type can be used to bootstrap an LDAP environment using openldap as slapd. + + +REQUIRED PARAMETERS +------------------- +manager-dn + The rootdn to set up in the directory. + E.g. `cn=manager,dc=ungleich,dc=ch`. See `slapd.conf(5)`. + +manager-password-hash + The password for `manager-dn` in the directory. + This should be valid for `slapd.conf` like `{SSHA}qV+mCs3u8Q2sCmUXT4Ybw7MebHTASMyr`. + Generate e.g. with: `slappasswd -s weneedgoodsecurity`. + See `slappasswd(8C)`, `slapd.conf(5)`. + TODO: implement this: http://blog.adamsbros.org/2015/06/09/openldap-ssha-salted-hashes-by-hand/ + to allow for a manager-password parameter and ensure idempotency (care with salts). + Such manager-password parameter should be mutually exclusive with this one. + +serverid + The server for the directory. + E.g. `dc=ungleich,dc=ch`. See `slapd.conf(5)`. + +suffix + The suffix for the directory. + E.g. `dc=ungleich,dc=ch`. See `slapd.conf(5)`. + + +OPTIONAL PARAMETERS +------------------- +syncrepl-credentials + Only has an effect if `replicate` is set; required in that case. + This secret is shared amongst the hosts that will replicate the directory. + Note that each replication server needs this secret and it is saved in + plain text in the directory. + +syncrepl-searchbase + Only has an effect if `replicate` is set; required in that case. + The searchbase to use for replication. + E.g. `dc=ungleich,dc=ch`. See `slapd.conf(5)`. + +tls-cert + If defined, `__letsencrypt_cert` is not used and this must be the path in + the remote hosts to the PEM-encoded TLS certificate. + Requires: `tls-privkey` and `tls-ca`. + Permissions, existence and renewal of these files are left up to the + type's user. + +tls-privkey + Required if `tls-cert` is defined. + Path in the remote hosts to the PEM-encoded private key file. + +tls-ca + Required if `tls-cert` is defined. + Path in the remote hosts to the PEM-encoded CA certificate file. + + +OPTIONAL MULTIPLE PARAMETERS +---------------------------- +syncrepl-host + Only has an effect if `replicate` is set; required in that case. + Set once per host that will replicate the directory. + +module + LDAP module to load. See `slapd.conf(5)`. + Default value is OS-dependent, see manifest. + + +BOOLEAN PARAMETERS +------------------ +staging + Passed to `cdist-type__letsencrypt_cert`; has otherwise no use. + Obtain a test certificate from a staging server. + +replicate + Whether to setup replication or not. + If present `syncrepl-credentials` and `syncrepl-host` are also required. + +EXAMPLES +-------- + +.. code-block:: sh + + # Modify the ruleset on $__target_host: + __pf_ruleset --state present --source /my/pf/ruleset.conf + require="__pf_ruleset" \ + __pf_apply + + # Remove the ruleset on $__target_host (implies disabling pf(4): + __pf_ruleset --state absent + require="__pf_ruleset" \ + __pf_apply + + root@ldap-for-guacamole:~# cat ldapbase.ldif + dn: dc=guaca-test,dc=ungleich,dc=ch + objectClass: top + objectClass: dcObject + objectClass: organization + o: Some description + dc: guaca-test + + + # Sample usage: + # + # id=1 + # for host in ldap-test1.ungleich.ch ldap-test2.ungleich.ch; do + # echo "__ungleich_ldap ${host} \ + # --manager-dn 'cn=manager,dc=ungleich,dc=ch' \ + # --manager-password '{SSHA}fooo' \ + # --serverid '${id}' \ + # --staging \ + # --suffix 'dc=ungleich,dc=ch' \ + # --searchbase 'dc=ungleich,dc=ch' \ + # --syncrepl-credentials 'fooo' \ + # --syncrepl-host 'ldap-test1.ungleich.ch' \ + # --syncrepl-host 'ldap-test2.ungleich.ch' \ + # --descriptiont 'Ungleich LDAP server'" \ + # | cdist config -i - -v ${host} + # id=$((id + 1)) + # done + + +SEE ALSO +-------- +:strong:`cdist-type__letsencrypt_cert`\ (7) + + +AUTHORS +------- +ungleich +Evilham + + +COPYING +------- +Copyright \(C) 2020 ungleich glarus ag. You can redistribute it +and/or modify it under the terms of the GNU General Public License as +published by the Free Software Foundation, either version 3 of the +License, or (at your option) any later version. diff --git a/cdist/conf/type/__openldap_server/manifest b/cdist/conf/type/__openldap_server/manifest new file mode 100644 index 00000000..2acaaed5 --- /dev/null +++ b/cdist/conf/type/__openldap_server/manifest @@ -0,0 +1,235 @@ +#!/bin/sh + +name="${__target_host}" +manager_dn=$(cat "${__object}/parameter/manager-dn") +manager_password_hash=$(cat "${__object}/parameter/manager-password-hash") +serverid=$(cat "${__object}/parameter/serverid") +suffix=$(cat "${__object}/parameter/suffix") +slapd_modules=$(cat "${__object}/parameter/module" || true) + + +OS="$(cat "${__global}/explorer/os")" + +# Setup OS-dependent vars +# TODO: treat other OS better, defaulting to Debian-like +case "${OS}" in + freebsd) + PKGS="openldap-server" + ETC="/usr/local/etc" + SLAPD_DIR="/usr/local/etc/openldap" + SLAPD_DATA_DIR="/var/db/openldap-data" + SLAPD_RUN_DIR="/var/run/openldap" + SLAPD_MODULE_PATH="/usr/local/libexec/openldap" + if [ -z "${slapd_modules}" ]; then + # It looks like ppolicy and syncprov must be compiled + slapd_modules="back_mdb back_monitor" + fi + ;; + *) + PKGS="slapd ldap-utils" + ETC="/etc" + SLAPD_DIR="/etc/ldap" + SLAPD_DATA_DIR="/var/lib/ldap" + SLAPD_RUN_DIR="/var/run/slapd" + SLAPD_MODULE_PATH="/usr/lib/ldap" + if [ -z "${slapd_modules}" ]; then + slapd_modules="back_mdb ppolicy syncprov back_monitor" + fi + ;; +esac + + + +# Determine if __letsencrypt_cert is to be used and setup vars accordingly +if [ -f "${__object}/parameter/tls-cert" ]; then + tls_cert=$(cat "${__object}/parameter/tls-cert") + + if [ ! -f "${__object}/parameter/tls-privkey" ]; then + echo "When tls-cert is defined, tls-privkey is also required." >&2 + exit 1 + fi + tls_privkey=$(cat "${__object}/parameter/tls-privkey") + + if [ ! -f "${__object}/parameter/tls-ca" ]; then + echo "When tls-cert is defined, tls-ca is also required." >&2 + exit 1 + fi + tls_ca=$(cat "${__object}/parameter/tls-ca") + + _skip_letsencrypt_cert="YES" +else + tls_cert="${SLAPD_DIR}/sasl2/cert.pem" + tls_privkey="${SLAPD_DIR}/sasl2/privkey.pem" + tls_ca="${SLAPD_DIR}/sasl2/chain.pem" +fi + +mkdir "${__object}/files" +ldapconf="${__object}/files/ldapconf" + +replication="" +if [ -f "${__object}/parameter/replicate" ]; then + replication=yes + + if [ ! -f "${__object}/parameter/syncrepl-searchbase" ]; then + echo "Requiring the searchbase for replication" >&2 + exit 1 + fi + syncrepl_searchbase=$(cat "${__object}/parameter/syncrepl-searchbase") + + if [ ! -f "${__object}/parameter/syncrepl-credentials" ]; then + echo "Requiring credentials for replication" >&2 + exit 1 + fi + + syncrepl_credentials=$(cat "${__object}/parameter/syncrepl-credentials") + + if [ ! -f "${__object}/parameter/syncrepl-host" ]; then + echo "Requiring host(s) for replication" >&2 + exit 1 + fi + syncrepl_hosts=$(cat "${__object}/parameter/syncrepl-host") + +fi + +# Install required packages +for pkg in ${PKGS}; do + __package ${pkg} +done + + +# TODO: Implement __start_on_boot for BSD +require="__package/slapd" __start_on_boot slapd + +# TODO: treat other OS better. Defaulting to Debian-like. +if [ "${OS}" != "freebsd" ]; then + require="__package/slapd" __line rm_slapd_conf \ + --file ${ETC}/default/slapd \ + --regex 'SLAPD_CONF=.*' \ + --state absent + + require="__package/slapd" __line rm_slapd_services \ + --file ${ETC}/default/slapd \ + --regex 'SLAPD_SERVICES=.*' \ + --state absent + + require="__line/rm_slapd_conf" __line add_slapd_conf \ + --file ${ETC}/default/slapd \ + --line 'SLAPD_CONF=${SLAPD_DIR}/slapd.conf' \ + --state present + + require="__line/rm_slapd_services" __line add_slapd_services \ + --file ${ETC}/default/slapd \ + --line "SLAPD_SERVICES=\"ldap://localhost/ ldap://${name}/\"" \ + --state present +fi + + +if [ -z "${_skip_letsencrypt_cert}" ]; then + if [ -f "${__object}/parameter/staging" ]; then + staging="--staging" + else + staging="" + fi + + __letsencrypt_cert "${name}" --admin-email technik@ungleich.ch \ + --renew-hook "cp ${ETC}/letsencrypt/live/${name}/*.pem ${SLAPD_DIR}/sasl2 && chown -R openldap:openldap ${SLAPD_DIR}/sasl2 && service slapd restart" \ + --automatic-renewal ${staging} +fi + +require="__package/slapd" __directory ${SLAPD_DIR}/slapd.d --state absent + +if [ -z "${_skip_letsencrypt_cert}" ]; then + require="__package/slapd __letsencrypt_cert/${name}" \ + __file ${SLAPD_DIR}/slapd.conf --owner root --group root --mode 644 \ + --source "${ldapconf}" +else + require="__package/slapd" \ + __file ${SLAPD_DIR}/slapd.conf --owner root --group root --mode 644 \ + --source "${ldapconf}" +fi + +# Start slapd.conf +cat << EOF > "${ldapconf}" +pidfile ${SLAPD_RUN_DIR}/slapd.pid +argsfile ${SLAPD_RUN_DIR}/slapd.args + +TLSCipherSuite NORMAL +TLSCertificateFile ${tls_cert} +TLSCertificateKeyFile ${tls_privkey} +TLSCACertificateFile ${tls_ca} + +disallow bind_anon +require bind +security tls=1 + +include ${SLAPD_DIR}/schema/corba.schema +include ${SLAPD_DIR}/schema/core.schema +include ${SLAPD_DIR}/schema/cosine.schema +include ${SLAPD_DIR}/schema/duaconf.schema +include ${SLAPD_DIR}/schema/dyngroup.schema +include ${SLAPD_DIR}/schema/inetorgperson.schema +include ${SLAPD_DIR}/schema/java.schema +include ${SLAPD_DIR}/schema/misc.schema +include ${SLAPD_DIR}/schema/nis.schema +include ${SLAPD_DIR}/schema/openldap.schema +include ${SLAPD_DIR}/schema/ppolicy.schema +include ${SLAPD_DIR}/schema/collective.schema + +modulepath ${SLAPD_MODULE_PATH} +EOF + +# Add specified modules +for module in ${slapd_modules}; do + echo "moduleload ${module}.la" >> "${ldapconf}" +done + +# Rest of the config +cat << EOF >> "${ldapconf}" +loglevel 1024 + +database mdb +maxsize 1073741824 + +suffix "${suffix}" +directory ${SLAPD_DATA_DIR} +rootdn "${manager_dn}" +rootpw "${manager_password_hash}" + +index objectClass eq,pres +index ou,cn,mail,surname,givenname eq,pres,sub +index uidNumber,gidNumber,loginShell eq,pres +index uid,memberUid eq,pres,sub +index nisMapName,nisMapEntry eq,pres,sub +index entryCSN,entryUUID eq + +serverid ${serverid} +EOF + +# Setup replication +if [ "${replication}" ]; then + rid=1; + for syncrepl in ${syncrepl_hosts}; do + cat <> "${ldapconf}" +syncrepl rid=${rid} + provider=ldap://${syncrepl} + bindmethod=simple + starttls=yes + binddn="${manager_dn}" + credentials=${syncrepl_credentials} + searchbase="${syncrepl_searchbase}" + type=refreshAndPersist + retry="5 + 5 +" + interval=00:00:00:05 +EOF + rid=$((rid + 1)) + done + cat <> "${ldapconf}" +mirrormode true +overlay syncprov +syncprov-checkpoint 100 5 +syncprov-sessionlog 100 + +database monitor +limits dn.exact="${manager_dn}" time=unlimited size=unlimited +EOF +fi diff --git a/cdist/conf/type/__openldap_server/parameter/boolean b/cdist/conf/type/__openldap_server/parameter/boolean new file mode 100644 index 00000000..45056fe9 --- /dev/null +++ b/cdist/conf/type/__openldap_server/parameter/boolean @@ -0,0 +1,2 @@ +staging +replicate diff --git a/cdist/conf/type/__openldap_server/parameter/optional b/cdist/conf/type/__openldap_server/parameter/optional new file mode 100644 index 00000000..a9a8ab2c --- /dev/null +++ b/cdist/conf/type/__openldap_server/parameter/optional @@ -0,0 +1,4 @@ +description +syncrepl-credentials +syncrepl-searchbase +tls-cert diff --git a/cdist/conf/type/__openldap_server/parameter/optional_multiple b/cdist/conf/type/__openldap_server/parameter/optional_multiple new file mode 100644 index 00000000..107c03d9 --- /dev/null +++ b/cdist/conf/type/__openldap_server/parameter/optional_multiple @@ -0,0 +1,2 @@ +syncrepl-host +module diff --git a/cdist/conf/type/__openldap_server/parameter/required b/cdist/conf/type/__openldap_server/parameter/required new file mode 100644 index 00000000..1ee6f219 --- /dev/null +++ b/cdist/conf/type/__openldap_server/parameter/required @@ -0,0 +1,4 @@ +manager-dn +manager-password-hash +serverid +suffix From fd430eab622206b9a04780ca5fa9d7c807f16c93 Mon Sep 17 00:00:00 2001 From: Evilham Date: Mon, 9 Dec 2019 19:39:43 +0100 Subject: [PATCH 17/51] [new-type] __openldap_server: Add a "schema" optional parameter. --- cdist/conf/type/__openldap_server/man.rst | 7 +++++ cdist/conf/type/__openldap_server/manifest | 29 +++++++------------ .../parameter/default/schema | 12 ++++++++ .../type/__openldap_server/parameter/optional | 3 +- 4 files changed, 31 insertions(+), 20 deletions(-) create mode 100644 cdist/conf/type/__openldap_server/parameter/default/schema diff --git a/cdist/conf/type/__openldap_server/man.rst b/cdist/conf/type/__openldap_server/man.rst index 29bbc231..1fc24eaa 100644 --- a/cdist/conf/type/__openldap_server/man.rst +++ b/cdist/conf/type/__openldap_server/man.rst @@ -74,6 +74,13 @@ module LDAP module to load. See `slapd.conf(5)`. Default value is OS-dependent, see manifest. +schema + Name of LDAP schema to load. Must be the name without extension of a + `.schema` file in slapd's schema directory (usually `/etc/slapd/schema` or + `/usr/local/etc/openldap/schema`). + Example value: `inetorgperson` + The type user must ensure that the schema file is deployed. + This defaults to a sensible subset, for details see the type definition. BOOLEAN PARAMETERS ------------------ diff --git a/cdist/conf/type/__openldap_server/manifest b/cdist/conf/type/__openldap_server/manifest index 2acaaed5..518edd86 100644 --- a/cdist/conf/type/__openldap_server/manifest +++ b/cdist/conf/type/__openldap_server/manifest @@ -6,6 +6,7 @@ manager_password_hash=$(cat "${__object}/parameter/manager-password-hash") serverid=$(cat "${__object}/parameter/serverid") suffix=$(cat "${__object}/parameter/suffix") slapd_modules=$(cat "${__object}/parameter/module" || true) +schemas=$(cat "${__object}/parameter/schema") OS="$(cat "${__global}/explorer/os")" @@ -14,8 +15,8 @@ OS="$(cat "${__global}/explorer/os")" # TODO: treat other OS better, defaulting to Debian-like case "${OS}" in freebsd) - PKGS="openldap-server" - ETC="/usr/local/etc" + PKGS="openldap-server" + ETC="/usr/local/etc" SLAPD_DIR="/usr/local/etc/openldap" SLAPD_DATA_DIR="/var/db/openldap-data" SLAPD_RUN_DIR="/var/run/openldap" @@ -27,7 +28,7 @@ case "${OS}" in ;; *) PKGS="slapd ldap-utils" - ETC="/etc" + ETC="/etc" SLAPD_DIR="/etc/ldap" SLAPD_DATA_DIR="/var/lib/ldap" SLAPD_RUN_DIR="/var/run/slapd" @@ -39,7 +40,6 @@ case "${OS}" in esac - # Determine if __letsencrypt_cert is to be used and setup vars accordingly if [ -f "${__object}/parameter/tls-cert" ]; then tls_cert=$(cat "${__object}/parameter/tls-cert") @@ -161,24 +161,15 @@ TLSCACertificateFile ${tls_ca} disallow bind_anon require bind security tls=1 - -include ${SLAPD_DIR}/schema/corba.schema -include ${SLAPD_DIR}/schema/core.schema -include ${SLAPD_DIR}/schema/cosine.schema -include ${SLAPD_DIR}/schema/duaconf.schema -include ${SLAPD_DIR}/schema/dyngroup.schema -include ${SLAPD_DIR}/schema/inetorgperson.schema -include ${SLAPD_DIR}/schema/java.schema -include ${SLAPD_DIR}/schema/misc.schema -include ${SLAPD_DIR}/schema/nis.schema -include ${SLAPD_DIR}/schema/openldap.schema -include ${SLAPD_DIR}/schema/ppolicy.schema -include ${SLAPD_DIR}/schema/collective.schema - -modulepath ${SLAPD_MODULE_PATH} EOF +# Add specified schemas +for schema in ${schemas}; do + echo "include ${SLAPD_DIR}/schema/${schema}.schema" >> "${ldapconf}" +done + # Add specified modules +echo "modulepath ${SLAPD_MODULE_PATH}" >> "${ldapconf}" for module in ${slapd_modules}; do echo "moduleload ${module}.la" >> "${ldapconf}" done diff --git a/cdist/conf/type/__openldap_server/parameter/default/schema b/cdist/conf/type/__openldap_server/parameter/default/schema new file mode 100644 index 00000000..825bdb15 --- /dev/null +++ b/cdist/conf/type/__openldap_server/parameter/default/schema @@ -0,0 +1,12 @@ +corba +core +cosine +duaconf +dyngroup +inetorgperson +java +misc +nis +openldap +ppolicy +collective diff --git a/cdist/conf/type/__openldap_server/parameter/optional b/cdist/conf/type/__openldap_server/parameter/optional index a9a8ab2c..53587c4e 100644 --- a/cdist/conf/type/__openldap_server/parameter/optional +++ b/cdist/conf/type/__openldap_server/parameter/optional @@ -1,4 +1,5 @@ -description syncrepl-credentials syncrepl-searchbase tls-cert +tls-privkey +tls-ca From 1ad605a509cfd534334111c1d6c1ea8682446b10 Mon Sep 17 00:00:00 2001 From: Evilham Date: Mon, 9 Dec 2019 19:49:05 +0100 Subject: [PATCH 18/51] [new-type] __openldap_server: Add admin-email parameter. --- cdist/conf/type/__openldap_server/man.rst | 5 +++++ cdist/conf/type/__openldap_server/manifest | 8 +++++++- cdist/conf/type/__openldap_server/parameter/optional | 1 + 3 files changed, 13 insertions(+), 1 deletion(-) diff --git a/cdist/conf/type/__openldap_server/man.rst b/cdist/conf/type/__openldap_server/man.rst index 1fc24eaa..4447971a 100644 --- a/cdist/conf/type/__openldap_server/man.rst +++ b/cdist/conf/type/__openldap_server/man.rst @@ -48,6 +48,11 @@ syncrepl-searchbase The searchbase to use for replication. E.g. `dc=ungleich,dc=ch`. See `slapd.conf(5)`. +admin-email + Passed to `cdist-type__letsencrypt_cert`; has otherwise no use. + Required if using `__letsencrypt_cert`. + Where to send Let's Encrypt emails like "certificate needs renewal". + tls-cert If defined, `__letsencrypt_cert` is not used and this must be the path in the remote hosts to the PEM-encoded TLS certificate. diff --git a/cdist/conf/type/__openldap_server/manifest b/cdist/conf/type/__openldap_server/manifest index 518edd86..e10432d6 100644 --- a/cdist/conf/type/__openldap_server/manifest +++ b/cdist/conf/type/__openldap_server/manifest @@ -58,6 +58,12 @@ if [ -f "${__object}/parameter/tls-cert" ]; then _skip_letsencrypt_cert="YES" else + if [ ! -f "${__object}/parameter/admin-email" ]; then + echo "When using __letsencrypt_cert, admin-email is also required." >&2 + exit 1 + fi + admin_email=$(cat "${__object}/parameter/admin-email") + tls_cert="${SLAPD_DIR}/sasl2/cert.pem" tls_privkey="${SLAPD_DIR}/sasl2/privkey.pem" tls_ca="${SLAPD_DIR}/sasl2/chain.pem" @@ -131,7 +137,7 @@ if [ -z "${_skip_letsencrypt_cert}" ]; then staging="" fi - __letsencrypt_cert "${name}" --admin-email technik@ungleich.ch \ + __letsencrypt_cert "${name}" --admin-email "${admin_email}" \ --renew-hook "cp ${ETC}/letsencrypt/live/${name}/*.pem ${SLAPD_DIR}/sasl2 && chown -R openldap:openldap ${SLAPD_DIR}/sasl2 && service slapd restart" \ --automatic-renewal ${staging} fi diff --git a/cdist/conf/type/__openldap_server/parameter/optional b/cdist/conf/type/__openldap_server/parameter/optional index 53587c4e..f4254cb6 100644 --- a/cdist/conf/type/__openldap_server/parameter/optional +++ b/cdist/conf/type/__openldap_server/parameter/optional @@ -1,5 +1,6 @@ syncrepl-credentials syncrepl-searchbase +admin-email tls-cert tls-privkey tls-ca From 42914d26c5addae027cc382ec035a687cd3aaf6d Mon Sep 17 00:00:00 2001 From: Evilham Date: Mon, 9 Dec 2019 19:59:15 +0100 Subject: [PATCH 19/51] [new-type] __openldap_server: sudo make nico.happy. --- cdist/conf/type/__openldap_server/manifest | 53 ++++++++++++---------- 1 file changed, 30 insertions(+), 23 deletions(-) diff --git a/cdist/conf/type/__openldap_server/manifest b/cdist/conf/type/__openldap_server/manifest index e10432d6..ffd92626 100644 --- a/cdist/conf/type/__openldap_server/manifest +++ b/cdist/conf/type/__openldap_server/manifest @@ -9,11 +9,10 @@ slapd_modules=$(cat "${__object}/parameter/module" || true) schemas=$(cat "${__object}/parameter/schema") -OS="$(cat "${__global}/explorer/os")" +os="$(cat "${__global}/explorer/os")" # Setup OS-dependent vars -# TODO: treat other OS better, defaulting to Debian-like -case "${OS}" in +case "${os}" in freebsd) PKGS="openldap-server" ETC="/usr/local/etc" @@ -26,7 +25,7 @@ case "${OS}" in slapd_modules="back_mdb back_monitor" fi ;; - *) + debian|ubuntu|devuan) PKGS="slapd ldap-utils" ETC="/etc" SLAPD_DIR="/etc/ldap" @@ -37,6 +36,10 @@ case "${OS}" in slapd_modules="back_mdb ppolicy syncprov back_monitor" fi ;; + *) + echo "Don't know the openldap defaults for: $os" >&2 + exit 1 + ;; esac @@ -106,28 +109,32 @@ done # TODO: Implement __start_on_boot for BSD require="__package/slapd" __start_on_boot slapd -# TODO: treat other OS better. Defaulting to Debian-like. -if [ "${OS}" != "freebsd" ]; then - require="__package/slapd" __line rm_slapd_conf \ - --file ${ETC}/default/slapd \ - --regex 'SLAPD_CONF=.*' \ - --state absent +case "${os}" in + debian|ubuntu|devuan) + require="__package/slapd" __line rm_slapd_conf \ + --file ${ETC}/default/slapd \ + --regex 'SLAPD_CONF=.*' \ + --state absent - require="__package/slapd" __line rm_slapd_services \ - --file ${ETC}/default/slapd \ - --regex 'SLAPD_SERVICES=.*' \ - --state absent + require="__package/slapd" __line rm_slapd_services \ + --file ${ETC}/default/slapd \ + --regex 'SLAPD_SERVICES=.*' \ + --state absent - require="__line/rm_slapd_conf" __line add_slapd_conf \ - --file ${ETC}/default/slapd \ - --line 'SLAPD_CONF=${SLAPD_DIR}/slapd.conf' \ - --state present + require="__line/rm_slapd_conf" __line add_slapd_conf \ + --file ${ETC}/default/slapd \ + --line 'SLAPD_CONF=${SLAPD_DIR}/slapd.conf' \ + --state present - require="__line/rm_slapd_services" __line add_slapd_services \ - --file ${ETC}/default/slapd \ - --line "SLAPD_SERVICES=\"ldap://localhost/ ldap://${name}/\"" \ - --state present -fi + require="__line/rm_slapd_services" __line add_slapd_services \ + --file ${ETC}/default/slapd \ + --line "SLAPD_SERVICES=\"ldap://localhost/ ldap://${name}/\"" \ + --state present + ;; + *) + # Nothing to do here, move on. + ;; +esac if [ -z "${_skip_letsencrypt_cert}" ]; then From 22c5cd550bb1069c2bb8106bbe907f016b9b821a Mon Sep 17 00:00:00 2001 From: Evilham Date: Tue, 10 Dec 2019 12:49:07 +0100 Subject: [PATCH 20/51] [new-type] __openldap_server: first public version. This already takes care of setting up the base DN and managing it as well as allowing for settings for the listener URLS. The type was also made a singleton as it doesn't make much sense to setup multiple LDAP servers on the same machine. --- .../type/__openldap_server/gencode-remote | 44 ++++++ cdist/conf/type/__openldap_server/man.rst | 126 +++++++++++++----- cdist/conf/type/__openldap_server/manifest | 48 +++++-- .../parameter/default/description | 1 + .../type/__openldap_server/parameter/optional | 4 +- .../parameter/optional_multiple | 1 + .../type/__openldap_server/parameter/required | 1 + .../parameter/required_multiple | 1 + cdist/conf/type/__openldap_server/singleton | 0 9 files changed, 176 insertions(+), 50 deletions(-) create mode 100644 cdist/conf/type/__openldap_server/gencode-remote create mode 100644 cdist/conf/type/__openldap_server/parameter/default/description create mode 100644 cdist/conf/type/__openldap_server/parameter/required_multiple create mode 100644 cdist/conf/type/__openldap_server/singleton diff --git a/cdist/conf/type/__openldap_server/gencode-remote b/cdist/conf/type/__openldap_server/gencode-remote new file mode 100644 index 00000000..0ac434af --- /dev/null +++ b/cdist/conf/type/__openldap_server/gencode-remote @@ -0,0 +1,44 @@ +#!/bin/sh + +manager_dn=$(cat "${__object}/parameter/manager-dn") +manager_password=$(cat "${__object}/parameter/manager-password") +description=$(cat "${__object}/parameter/description") +suffix=$(cat "${__object}/parameter/suffix") +suffix_dc=$(echo -n ${suffix} | awk -F',' '{print $1}' | awk -F'=' '{print $2}') + +SLAPD_IPC=$(cat "${__object}/parameter/slapd-url" | tr '\n' ' ' | awk '{ print $1}') + +cat <&1 > /dev/null; then + # Already exists, use ldapmodify + ldapmodify -xZ -D "${manager_dn}" -w "${manager_password}" -H '${SLAPD_IPC}' </dev/null || true) schemas=$(cat "${__object}/parameter/schema") +slapd_urls=$(cat "${__object}/parameter/slapd-url" | tr '\n' ' ') +tls_cipher_suite=$(cat "${__object}/parameter/tls-cipher-suite" 2>/dev/null || true) os="$(cat "${__global}/explorer/os")" # Setup OS-dependent vars +CONF_OWNER="root" +CONF_GROUP="root" case "${os}" in freebsd) PKGS="openldap-server" @@ -24,6 +28,12 @@ case "${os}" in # It looks like ppolicy and syncprov must be compiled slapd_modules="back_mdb back_monitor" fi + CONF_OWNER="ldap" + CONF_GROUP="ldap" + if [ -z "${tls_cipher_suite}" ]; then + # TODO: research default for FreeBSD. 'NORMAL' appears to not work + tls_cipher_suite="HIGH:MEDIUM:+SSLv2" + fi ;; debian|ubuntu|devuan) PKGS="slapd ldap-utils" @@ -35,6 +45,9 @@ case "${os}" in if [ -z "${slapd_modules}" ]; then slapd_modules="back_mdb ppolicy syncprov back_monitor" fi + if [ -z "${tls_cipher_suite}" ]; then + tls_cipher_suite="NORMAL" + fi ;; *) echo "Don't know the openldap defaults for: $os" >&2 @@ -42,6 +55,8 @@ case "${os}" in ;; esac +PKG_MAIN=$(echo ${PKGS} | awk '{print $1;}') + # Determine if __letsencrypt_cert is to be used and setup vars accordingly if [ -f "${__object}/parameter/tls-cert" ]; then @@ -106,17 +121,26 @@ for pkg in ${PKGS}; do done -# TODO: Implement __start_on_boot for BSD -require="__package/slapd" __start_on_boot slapd +require="__package/${PKG_MAIN}" __start_on_boot slapd +# Setup -h flag for the listeners. See man slapd (-h flag). case "${os}" in + freebsd) + require="__package/${PKG_MAIN}" __key_value \ + --file "/etc/rc.conf" \ + --key "slapd_flags" \ + --value "\"-h '${slapd_urls}'\"" \ + --delimiter "=" \ + --comment "# LDAP Listener URLs" \ + "${__target_host}__slapd_flags" + ;; debian|ubuntu|devuan) - require="__package/slapd" __line rm_slapd_conf \ + require="__package/${PKG_MAIN}" __line rm_slapd_conf \ --file ${ETC}/default/slapd \ --regex 'SLAPD_CONF=.*' \ --state absent - require="__package/slapd" __line rm_slapd_services \ + require="__package/${PKG_MAIN}" __line rm_slapd_services \ --file ${ETC}/default/slapd \ --regex 'SLAPD_SERVICES=.*' \ --state absent @@ -128,7 +152,7 @@ case "${os}" in require="__line/rm_slapd_services" __line add_slapd_services \ --file ${ETC}/default/slapd \ - --line "SLAPD_SERVICES=\"ldap://localhost/ ldap://${name}/\"" \ + --line "SLAPD_SERVICES=\"${slapd_urls}\"" \ --state present ;; *) @@ -149,15 +173,15 @@ if [ -z "${_skip_letsencrypt_cert}" ]; then --automatic-renewal ${staging} fi -require="__package/slapd" __directory ${SLAPD_DIR}/slapd.d --state absent +require="__package/${PKG_MAIN}" __directory ${SLAPD_DIR}/slapd.d --state absent if [ -z "${_skip_letsencrypt_cert}" ]; then - require="__package/slapd __letsencrypt_cert/${name}" \ - __file ${SLAPD_DIR}/slapd.conf --owner root --group root --mode 644 \ + require="__package/${PKG_MAIN} __letsencrypt_cert/${name}" \ + __file ${SLAPD_DIR}/slapd.conf --owner ${CONF_OWNER} --group ${CONF_GROUP} --mode 644 \ --source "${ldapconf}" else - require="__package/slapd" \ - __file ${SLAPD_DIR}/slapd.conf --owner root --group root --mode 644 \ + require="__package/${PKG_MAIN}" \ + __file ${SLAPD_DIR}/slapd.conf --owner ${CONF_OWNER} --group ${CONF_GROUP} --mode 644 \ --source "${ldapconf}" fi @@ -166,7 +190,7 @@ cat << EOF > "${ldapconf}" pidfile ${SLAPD_RUN_DIR}/slapd.pid argsfile ${SLAPD_RUN_DIR}/slapd.args -TLSCipherSuite NORMAL +TLSCipherSuite ${tls_cipher_suite} TLSCertificateFile ${tls_cert} TLSCertificateKeyFile ${tls_privkey} TLSCACertificateFile ${tls_ca} diff --git a/cdist/conf/type/__openldap_server/parameter/default/description b/cdist/conf/type/__openldap_server/parameter/default/description new file mode 100644 index 00000000..6d8e37e1 --- /dev/null +++ b/cdist/conf/type/__openldap_server/parameter/default/description @@ -0,0 +1 @@ +Managed by cdist, do not edit manually. diff --git a/cdist/conf/type/__openldap_server/parameter/optional b/cdist/conf/type/__openldap_server/parameter/optional index f4254cb6..a92b9c6e 100644 --- a/cdist/conf/type/__openldap_server/parameter/optional +++ b/cdist/conf/type/__openldap_server/parameter/optional @@ -1,6 +1,8 @@ +description syncrepl-credentials syncrepl-searchbase admin-email +tls-cipher-suite tls-cert tls-privkey -tls-ca +tls-ca \ No newline at end of file diff --git a/cdist/conf/type/__openldap_server/parameter/optional_multiple b/cdist/conf/type/__openldap_server/parameter/optional_multiple index 107c03d9..52a83d5c 100644 --- a/cdist/conf/type/__openldap_server/parameter/optional_multiple +++ b/cdist/conf/type/__openldap_server/parameter/optional_multiple @@ -1,2 +1,3 @@ syncrepl-host module +schema diff --git a/cdist/conf/type/__openldap_server/parameter/required b/cdist/conf/type/__openldap_server/parameter/required index 1ee6f219..ff58158d 100644 --- a/cdist/conf/type/__openldap_server/parameter/required +++ b/cdist/conf/type/__openldap_server/parameter/required @@ -1,4 +1,5 @@ manager-dn +manager-password manager-password-hash serverid suffix diff --git a/cdist/conf/type/__openldap_server/parameter/required_multiple b/cdist/conf/type/__openldap_server/parameter/required_multiple new file mode 100644 index 00000000..848b8dc2 --- /dev/null +++ b/cdist/conf/type/__openldap_server/parameter/required_multiple @@ -0,0 +1 @@ +slapd-url \ No newline at end of file diff --git a/cdist/conf/type/__openldap_server/singleton b/cdist/conf/type/__openldap_server/singleton new file mode 100644 index 00000000..e69de29b From 79d58f0813fd08c77d826ac552dfdb9e0f40dbbd Mon Sep 17 00:00:00 2001 From: Evilham Date: Tue, 10 Dec 2019 13:04:24 +0100 Subject: [PATCH 21/51] [new-type] __openldap_server: fix docs / dependency. It's nicer to have slapd_flags after slapd_enable in rc.conf on freebsd. --- cdist/conf/type/__openldap_server/man.rst | 6 +++--- cdist/conf/type/__openldap_server/manifest | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/cdist/conf/type/__openldap_server/man.rst b/cdist/conf/type/__openldap_server/man.rst index 84b534bd..1c2c9b70 100644 --- a/cdist/conf/type/__openldap_server/man.rst +++ b/cdist/conf/type/__openldap_server/man.rst @@ -141,9 +141,9 @@ EXAMPLES --serverid 0 \ --suffix 'dc=camilion,dc=cloud' \ --slapd-url 'ldaps://ldap.camilion.cloud' \ - --tls-cert '${pki_prefix}/default.crt' \ - --tls-privkey '${pki_prefix}/default.key' \ - --tls-ca '${pki_prefix}/CA.crt' + --tls-cert "${pki_prefix}/default.crt" \ + --tls-privkey "${pki_prefix}/default.key" \ + --tls-ca "${pki_prefix}/CA.crt" # The created basedn looks as follows: # diff --git a/cdist/conf/type/__openldap_server/manifest b/cdist/conf/type/__openldap_server/manifest index 2d1df32f..070a31d5 100644 --- a/cdist/conf/type/__openldap_server/manifest +++ b/cdist/conf/type/__openldap_server/manifest @@ -126,7 +126,7 @@ require="__package/${PKG_MAIN}" __start_on_boot slapd # Setup -h flag for the listeners. See man slapd (-h flag). case "${os}" in freebsd) - require="__package/${PKG_MAIN}" __key_value \ + require="__start_on_boot/slapd" __key_value \ --file "/etc/rc.conf" \ --key "slapd_flags" \ --value "\"-h '${slapd_urls}'\"" \ From 35f0d4dbfa9307923fa013ff806d30087a8cf291 Mon Sep 17 00:00:00 2001 From: Evilham Date: Tue, 10 Dec 2019 13:10:19 +0100 Subject: [PATCH 22/51] [new-type] __openldap_server: another documentation typo. --- cdist/conf/type/__openldap_server/man.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/cdist/conf/type/__openldap_server/man.rst b/cdist/conf/type/__openldap_server/man.rst index 1c2c9b70..d20101d1 100644 --- a/cdist/conf/type/__openldap_server/man.rst +++ b/cdist/conf/type/__openldap_server/man.rst @@ -133,8 +133,8 @@ EXAMPLES .. code-block:: sh # Example of a simple server with manual certificate management. - pki_prefix="/usr/local/etc/pki/realms/ldap.camilion.cloud" \ - __openldap_server \ + pki_prefix="/usr/local/etc/pki/realms/ldap.camilion.cloud" + __openldap_server \ --manager-dn 'cn=manager,dc=camilion,dc=cloud' \ --manager-password "foo" \ --manager-password-hash '{SSHA}foo' \ From 3ba230c10db98cf2cddb0aeeb256392223fb2558 Mon Sep 17 00:00:00 2001 From: Evilham Date: Tue, 10 Dec 2019 13:12:54 +0100 Subject: [PATCH 23/51] [type-docs] __start_on_boot: remove unspported *BSD claim. The type appears to support {Open,Free}BSD properly. --- cdist/conf/type/__start_on_boot/man.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cdist/conf/type/__start_on_boot/man.rst b/cdist/conf/type/__start_on_boot/man.rst index b7c73ab1..f8afe94b 100644 --- a/cdist/conf/type/__start_on_boot/man.rst +++ b/cdist/conf/type/__start_on_boot/man.rst @@ -12,7 +12,7 @@ This cdist type allows you to enable or disable stuff to be started at boot of your operating system. Warning: This type has not been tested intensively and is not fully -supported (i.e. \*BSD are not implemented). +supported. REQUIRED PARAMETERS From 3c62a88ca7907b15dc501ddb3395c66b6196cb13 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Tue, 10 Dec 2019 19:33:29 +0100 Subject: [PATCH 24/51] ++changelog --- docs/changelog | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/changelog b/docs/changelog index 04ffac62..186dce7a 100644 --- a/docs/changelog +++ b/docs/changelog @@ -9,6 +9,8 @@ next: * Documentation: PreOS english nitpicking (Evil Ham) * Documentation: Add installing from source with signature verification (Darko Poljak) * Core: preos: Support top command logging options, custom conf-dir option and CDIST_PATH env var (Darko Poljak) + * Type __start_on_boot: Docs: remove unsupported *BSD claim (Evil Ham) + * New type: __openldap_server (Evil Ham) 6.2.0: 2019-11-30 * Core: Redefine/reimplement/fix CDIST_ORDER_DEPENDENCY (Darko Poljak) From 2c7b4ddc55a9a2f76ae5b51a6020c0cd14900705 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Thu, 12 Dec 2019 06:18:04 +0100 Subject: [PATCH 25/51] Update cdist man page --- docs/src/man1/cdist.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/man1/cdist.rst b/docs/src/man1/cdist.rst index d6937272..55db82ed 100644 --- a/docs/src/man1/cdist.rst +++ b/docs/src/man1/cdist.rst @@ -915,5 +915,5 @@ such case and display a warning message. An example of such a case: COPYING ------- -Copyright \(C) 2011-2017 Nico Schottelius. Free use of this software is +Copyright \(C) 2011-2019 Nico Schottelius. Free use of this software is granted under the terms of the GNU General Public License v3 or later (GPLv3+). From bd4eee7925356c8e557a75688694feff94cca297 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Thu, 12 Dec 2019 07:00:23 +0100 Subject: [PATCH 26/51] Fix shellcheck reported issues --- cdist/conf/type/__openldap_server/gencode-remote | 4 ++-- cdist/conf/type/__openldap_server/manifest | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/cdist/conf/type/__openldap_server/gencode-remote b/cdist/conf/type/__openldap_server/gencode-remote index 0ac434af..b1e98f8c 100644 --- a/cdist/conf/type/__openldap_server/gencode-remote +++ b/cdist/conf/type/__openldap_server/gencode-remote @@ -4,9 +4,9 @@ manager_dn=$(cat "${__object}/parameter/manager-dn") manager_password=$(cat "${__object}/parameter/manager-password") description=$(cat "${__object}/parameter/description") suffix=$(cat "${__object}/parameter/suffix") -suffix_dc=$(echo -n ${suffix} | awk -F',' '{print $1}' | awk -F'=' '{print $2}') +suffix_dc=$(printf "%s" "${suffix}" | awk -F',' '{print $1}' | awk -F'=' '{print $2}') -SLAPD_IPC=$(cat "${__object}/parameter/slapd-url" | tr '\n' ' ' | awk '{ print $1}') +SLAPD_IPC=$(tr '\n' ' ' < "${__object}/parameter/slapd-url" | awk '{ print $1}') cat </dev/null || true) schemas=$(cat "${__object}/parameter/schema") -slapd_urls=$(cat "${__object}/parameter/slapd-url" | tr '\n' ' ') +slapd_urls=$(tr '\n' ' ' < "${__object}/parameter/slapd-url") tls_cipher_suite=$(cat "${__object}/parameter/tls-cipher-suite" 2>/dev/null || true) @@ -55,7 +55,7 @@ case "${os}" in ;; esac -PKG_MAIN=$(echo ${PKGS} | awk '{print $1;}') +PKG_MAIN=$(echo "${PKGS}" | awk '{print $1;}') # Determine if __letsencrypt_cert is to be used and setup vars accordingly @@ -117,7 +117,7 @@ fi # Install required packages for pkg in ${PKGS}; do - __package ${pkg} + __package "${pkg}" done @@ -147,7 +147,7 @@ case "${os}" in require="__line/rm_slapd_conf" __line add_slapd_conf \ --file ${ETC}/default/slapd \ - --line 'SLAPD_CONF=${SLAPD_DIR}/slapd.conf' \ + --line "SLAPD_CONF=${SLAPD_DIR}/slapd.conf" \ --state present require="__line/rm_slapd_services" __line add_slapd_services \ From 8562871da9a51752fea47308e496e139210edf75 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Thu, 12 Dec 2019 07:35:53 +0100 Subject: [PATCH 27/51] Fix shellcheck exit status shellcheck* targets were always reporting exit status 0. With this fix, if shellcheck fails, then build-helper script exits with 1. --- bin/build-helper | 50 ++++++++++++++++++++++++++++++++---------------- 1 file changed, 33 insertions(+), 17 deletions(-) diff --git a/bin/build-helper b/bin/build-helper index 7cfc4d55..69dee4c7 100755 --- a/bin/build-helper +++ b/bin/build-helper @@ -74,6 +74,7 @@ SHELLCHECKCMD="shellcheck -s sh -f gcc -x" # Skip SC2154 for variables starting with __ since such variables are cdist # environment variables. SHELLCHECK_SKIP=': __.*is referenced but not assigned.*\[SC2154\]' +SHELLCHECKTMP=".shellcheck.tmp" # Change to checkout directory basedir="${0%/*}/../" @@ -431,53 +432,67 @@ eof ;; shellcheck-global-explorers) - find cdist/conf/explorer -type f -exec ${SHELLCHECKCMD} {} + | grep -v "${SHELLCHECK_SKIP}" || exit 0 + # shellcheck disable=SC2086 + find cdist/conf/explorer -type f -exec ${SHELLCHECKCMD} {} + | grep -v "${SHELLCHECK_SKIP}" > "${SHELLCHECKTMP}" + test ! -s "${SHELLCHECKTMP}" || { cat "${SHELLCHECKTMP}"; exit 1; } ;; shellcheck-type-explorers) - find cdist/conf/type -type f -path "*/explorer/*" -exec ${SHELLCHECKCMD} {} + | grep -v "${SHELLCHECK_SKIP}" || exit 0 + # shellcheck disable=SC2086 + find cdist/conf/type -type f -path "*/explorer/*" -exec ${SHELLCHECKCMD} {} + | grep -v "${SHELLCHECK_SKIP}" > "${SHELLCHECKTMP}" + test ! -s "${SHELLCHECKTMP}" || { cat "${SHELLCHECKTMP}"; exit 1; } ;; shellcheck-manifests) - find cdist/conf/type -type f -name manifest -exec ${SHELLCHECKCMD} {} + | grep -v "${SHELLCHECK_SKIP}" || exit 0 + # shellcheck disable=SC2086 + find cdist/conf/type -type f -name manifest -exec ${SHELLCHECKCMD} {} + | grep -v "${SHELLCHECK_SKIP}" > "${SHELLCHECKTMP}" + test ! -s "${SHELLCHECKTMP}" || { cat "${SHELLCHECKTMP}"; exit 1; } ;; shellcheck-local-gencodes) - find cdist/conf/type -type f -name gencode-local -exec ${SHELLCHECKCMD} {} + | grep -v "${SHELLCHECK_SKIP}" || exit 0 + # shellcheck disable=SC2086 + find cdist/conf/type -type f -name gencode-local -exec ${SHELLCHECKCMD} {} + | grep -v "${SHELLCHECK_SKIP}" > "${SHELLCHECKTMP}" + test ! -s "${SHELLCHECKTMP}" || { cat "${SHELLCHECKTMP}"; exit 1; } ;; shellcheck-remote-gencodes) - find cdist/conf/type -type f -name gencode-remote -exec ${SHELLCHECKCMD} {} + | grep -v "${SHELLCHECK_SKIP}" || exit 0 + # shellcheck disable=SC2086 + find cdist/conf/type -type f -name gencode-remote -exec ${SHELLCHECKCMD} {} + | grep -v "${SHELLCHECK_SKIP}" > "${SHELLCHECKTMP}" + test ! -s "${SHELLCHECKTMP}" || { cat "${SHELLCHECKTMP}"; exit 1; } ;; shellcheck-scripts) - ${SHELLCHECKCMD} scripts/cdist-dump scripts/cdist-new-type || exit 0 + # shellcheck disable=SC2086 + ${SHELLCHECKCMD} scripts/cdist-dump scripts/cdist-new-type > "${SHELLCHECKTMP}" + test ! -s "${SHELLCHECKTMP}" || { cat "${SHELLCHECKTMP}"; exit 1; } ;; shellcheck-gencodes) - "$0" shellcheck-local-gencodes - "$0" shellcheck-remote-gencodes + "$0" shellcheck-local-gencodes || exit 1 + "$0" shellcheck-remote-gencodes || exit 1 ;; shellcheck-types) - "$0" shellcheck-type-explorers - "$0" shellcheck-manifests - "$0" shellcheck-gencodes + "$0" shellcheck-type-explorers || exit 1 + "$0" shellcheck-manifests || exit 1 + "$0" shellcheck-gencodes || exit 1 ;; shellcheck) - "$0" shellcheck-global-explorers - "$0" shellcheck-types - "$0" shellcheck-scripts + "$0" shellcheck-global-explorers || exit 1 + "$0" shellcheck-types || exit 1 + "$0" shellcheck-scripts || exit 1 ;; shellcheck-type-files) - find cdist/conf/type -type f -path "*/files/*" -exec ${SHELLCHECKCMD} {} + | grep -v "${SHELLCHECK_SKIP}" || exit 0 + # shellcheck disable=SC2086 + find cdist/conf/type -type f -path "*/files/*" -exec ${SHELLCHECKCMD} {} + | grep -v "${SHELLCHECK_SKIP}" > "${SHELLCHECKTMP}" + test ! -s "${SHELLCHECKTMP}" || { cat "${SHELLCHECKTMP}"; exit 1; } ;; shellcheck-with-files) - "$0" shellcheck - "$0" shellcheck-type-files + "$0" shellcheck || exit 1 + "$0" shellcheck-type-files || exit 1 ;; shellcheck-build-helper) @@ -535,6 +550,7 @@ eof # Temp files rm -f ./*.tmp + rm -f ./.*.tmp ;; distclean) From 9859080217a9dc067250a0fc0f686a2242ef1083 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Thu, 12 Dec 2019 07:43:07 +0100 Subject: [PATCH 28/51] Release 6.3.0 --- docs/changelog | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/changelog b/docs/changelog index 186dce7a..674e14c7 100644 --- a/docs/changelog +++ b/docs/changelog @@ -1,7 +1,7 @@ Changelog --------- -next: +6.3.0: 2019-12-12 * Type __package_update_index: Fix Alpine part (Dominique Roux) * Documentation: Fix man pages for install types (Darko Poljak) * Documentation: Embed config skeleton instead of rewriting it (Darko Poljak) From f407e8825fc2eb70fa489dc218b46f1f2d92d747 Mon Sep 17 00:00:00 2001 From: Nico Schottelius Date: Sun, 15 Dec 2019 22:56:16 +0100 Subject: [PATCH 29/51] [consul/alpine] no init script required to be deployed anymore --- cdist/conf/type/__consul_agent/manifest | 2 +- docs/changelog | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/cdist/conf/type/__consul_agent/manifest b/cdist/conf/type/__consul_agent/manifest index a88d26ed..ee682d72 100755 --- a/cdist/conf/type/__consul_agent/manifest +++ b/cdist/conf/type/__consul_agent/manifest @@ -181,7 +181,7 @@ init_upstart() # Install init script to start on boot case "$os" in - alpine|devuan) + devuan) init_sysvinit debian ;; centos|redhat) diff --git a/docs/changelog b/docs/changelog index 674e14c7..415af0af 100644 --- a/docs/changelog +++ b/docs/changelog @@ -1,6 +1,10 @@ Changelog --------- +next: + * Type __consul_agent: don't deploy init script on Alpine anymore, + it ships with one itself (Nico Schottelius) + 6.3.0: 2019-12-12 * Type __package_update_index: Fix Alpine part (Dominique Roux) * Documentation: Fix man pages for install types (Darko Poljak) From df05abd15b6676d6594b3e8aebff112c2f5c0ce7 Mon Sep 17 00:00:00 2001 From: Steven Armstrong Date: Thu, 19 Dec 2019 12:33:47 +0100 Subject: [PATCH 30/51] bugfix: __install_chroot_umount was not using __chroot_umount/manifest Signed-off-by: Steven Armstrong --- cdist/conf/type/__install_chroot_umount/manifest | 1 + 1 file changed, 1 insertion(+) create mode 120000 cdist/conf/type/__install_chroot_umount/manifest diff --git a/cdist/conf/type/__install_chroot_umount/manifest b/cdist/conf/type/__install_chroot_umount/manifest new file mode 120000 index 00000000..f17af67a --- /dev/null +++ b/cdist/conf/type/__install_chroot_umount/manifest @@ -0,0 +1 @@ +../__chroot_umount/manifest \ No newline at end of file From beb930c0dc6e80d0718029fa8fc5539f373154a2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Timoth=C3=A9e=20Floure?= Date: Tue, 31 Dec 2019 11:05:40 +0100 Subject: [PATCH 31/51] __postgres_*: use delimited identifiers (double quoted) in generated SQL --- cdist/conf/type/__postgres_database/gencode-remote | 6 +++--- cdist/conf/type/__postgres_role/gencode-remote | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/cdist/conf/type/__postgres_database/gencode-remote b/cdist/conf/type/__postgres_database/gencode-remote index 61cfa50d..9f12d215 100755 --- a/cdist/conf/type/__postgres_database/gencode-remote +++ b/cdist/conf/type/__postgres_database/gencode-remote @@ -41,12 +41,12 @@ if [ "$state_should" != "$state_is" ]; then present) owner="" if [ -f "$__object/parameter/owner" ]; then - owner="-O '$(cat "$__object/parameter/owner")'" + owner="-O \"$(cat "$__object/parameter/owner")\"" fi - echo "su - '$postgres_user' -c \"createdb $owner '$name'\"" + echo "su - '$postgres_user' -c \"createdb $owner \"$name\"\"" ;; absent) - echo "su - '$postgres_user' -c \"dropdb '$name'\"" + echo "su - '$postgres_user' -c \"dropdb \"$name\"\"" ;; esac fi diff --git a/cdist/conf/type/__postgres_role/gencode-remote b/cdist/conf/type/__postgres_role/gencode-remote index fd56e85d..0b8852f4 100755 --- a/cdist/conf/type/__postgres_role/gencode-remote +++ b/cdist/conf/type/__postgres_role/gencode-remote @@ -54,7 +54,7 @@ case "$state_should" in [ -n "$password" ] && password="PASSWORD '$password'" - cmd="CREATE ROLE $name WITH $password $booleans" + cmd="CREATE ROLE \"$name\" WITH $password $booleans" echo "su - '$postgres_user' -c \"psql postgres -wc \\\"$cmd\\\"\"" ;; absent) From c32e4040b1b17f1e4c8173f23f1bdb2d5112f227 Mon Sep 17 00:00:00 2001 From: Ander Punnar Date: Tue, 31 Dec 2019 19:16:49 +0200 Subject: [PATCH 32/51] __mysql_*: initial rewrite --- .../conf/type/__mysql_database/explorer/state | 15 ++++ .../conf/type/__mysql_database/gencode-remote | 72 ++++++------------- cdist/conf/type/__mysql_database/man.rst | 49 ------------- cdist/conf/type/__mysql_database/manifest | 26 +++++++ .../__mysql_database/parameter/default/state | 1 + .../type/__mysql_database/parameter/optional | 1 + .../type/__mysql_privileges/explorer/state | 22 ++++++ .../type/__mysql_privileges/gencode-remote | 31 ++++++++ .../__mysql_privileges/parameter/default/host | 1 + .../parameter/default/privileges | 1 + .../parameter/default/state | 1 + .../parameter/default/table | 1 + .../__mysql_privileges/parameter/optional | 4 ++ .../__mysql_privileges/parameter/required | 2 + cdist/conf/type/__mysql_user/explorer/state | 36 ++++++++++ cdist/conf/type/__mysql_user/gencode-remote | 50 +++++++++++++ .../type/__mysql_user/parameter/default/host | 1 + .../type/__mysql_user/parameter/default/state | 1 + .../conf/type/__mysql_user/parameter/optional | 4 ++ 19 files changed, 221 insertions(+), 98 deletions(-) create mode 100755 cdist/conf/type/__mysql_database/explorer/state delete mode 100644 cdist/conf/type/__mysql_database/man.rst create mode 100755 cdist/conf/type/__mysql_database/manifest create mode 100644 cdist/conf/type/__mysql_database/parameter/default/state create mode 100755 cdist/conf/type/__mysql_privileges/explorer/state create mode 100755 cdist/conf/type/__mysql_privileges/gencode-remote create mode 100644 cdist/conf/type/__mysql_privileges/parameter/default/host create mode 100644 cdist/conf/type/__mysql_privileges/parameter/default/privileges create mode 100644 cdist/conf/type/__mysql_privileges/parameter/default/state create mode 100644 cdist/conf/type/__mysql_privileges/parameter/default/table create mode 100644 cdist/conf/type/__mysql_privileges/parameter/optional create mode 100644 cdist/conf/type/__mysql_privileges/parameter/required create mode 100755 cdist/conf/type/__mysql_user/explorer/state create mode 100755 cdist/conf/type/__mysql_user/gencode-remote create mode 100644 cdist/conf/type/__mysql_user/parameter/default/host create mode 100644 cdist/conf/type/__mysql_user/parameter/default/state create mode 100644 cdist/conf/type/__mysql_user/parameter/optional diff --git a/cdist/conf/type/__mysql_database/explorer/state b/cdist/conf/type/__mysql_database/explorer/state new file mode 100755 index 00000000..16cc9ce5 --- /dev/null +++ b/cdist/conf/type/__mysql_database/explorer/state @@ -0,0 +1,15 @@ +#!/bin/sh -e + +if [ -f "$__object/parameter/name" ] +then + name="$( cat "$__object/parameter/name" )" +else + name="$__object_id" +fi + +if [ -n "$( mysql -B -N -e "show databases like '$name'" )" ] +then + echo 'present' +else + echo 'absent' +fi diff --git a/cdist/conf/type/__mysql_database/gencode-remote b/cdist/conf/type/__mysql_database/gencode-remote index 23e51b05..d3692572 100755 --- a/cdist/conf/type/__mysql_database/gencode-remote +++ b/cdist/conf/type/__mysql_database/gencode-remote @@ -1,54 +1,28 @@ #!/bin/sh -e -# -# 2012 Benedikt Koeppel (code@benediktkoeppel.ch) -# -# This file is part of cdist. -# -# cdist is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# cdist is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with cdist. If not, see . -# -# -# if --database was specified -if [ -f "$__object/parameter/name" ]; then - database="$(cat "$__object/parameter/name")" -else # otherwise use the object id as database name - database="$__object_id" +state_is="$( cat "$__object/explorer/state" )" + +state_should="$( cat "$__object/parameter/state" )" + +if [ "$state_is" = "$state_should" ] +then + exit 0 fi -cat <<-EOFF -mysql -u root <<-EOF - CREATE DATABASE IF NOT EXISTS $database -EOF -EOFF - -# if --user was specified -if [ -f "$__object/parameter/user" ]; then - user="$(cat "$__object/parameter/user")" - - # if --password was specified - if [ -f "$__object/parameter/password" ]; then - password="$(cat "$__object/parameter/password")" - cat <<-EOFF - mysql -u root <<-EOF - GRANT ALL PRIVILEGES ON $database.* to '$user'@'localhost' IDENTIFIED BY '$password'; -EOF -EOFF - else - cat <<-EOFF - mysql -u root <<-EOF - GRANT ALL PRIVILEGES ON $database.* to '$user'@'localhost'; -EOF -EOFF - fi +if [ -f "$__object/parameter/name" ] +then + name="$( cat "$__object/parameter/name" )" +else + name="$__object_id" fi + +case "$state_should" in + present) + echo "mysql -e 'create database \`$name\`'" + echo "create database $name" >> "$__messages_out" + ;; + absent) + echo "mysql -e 'drop database \`$name\`'" + echo "drop database $name" >> "$__messages_out" + ;; +esac diff --git a/cdist/conf/type/__mysql_database/man.rst b/cdist/conf/type/__mysql_database/man.rst deleted file mode 100644 index 1e245a08..00000000 --- a/cdist/conf/type/__mysql_database/man.rst +++ /dev/null @@ -1,49 +0,0 @@ -cdist-type__mysql_database(7) -============================= - -NAME ----- -cdist-type__mysql_database - Manage a MySQL database - - -DESCRIPTION ------------ -This cdist type allows you to install a MySQL database. - - -REQUIRED PARAMETERS -------------------- -None. - -OPTIONAL PARAMETERS -------------------- -name - The name of the database to install - defaults to the object id - -user - A user that should have access to the database - -password - The password for the user who manages the database - - -EXAMPLES --------- - -.. code-block:: sh - - __mysql_database "cdist" --name "cdist" --user "myuser" --password "mypwd" - - -AUTHORS -------- -Benedikt Koeppel - - -COPYING -------- -Copyright \(C) 2012 Benedikt Koeppel. You can redistribute it -and/or modify it under the terms of the GNU General Public License as -published by the Free Software Foundation, either version 3 of the -License, or (at your option) any later version. diff --git a/cdist/conf/type/__mysql_database/manifest b/cdist/conf/type/__mysql_database/manifest new file mode 100755 index 00000000..a57c31ce --- /dev/null +++ b/cdist/conf/type/__mysql_database/manifest @@ -0,0 +1,26 @@ +#!/bin/sh -e + +if [ -f "$__object/parameter/user" ] +then + user="$( cat "$__object/parameter/user" )" +fi + +if [ -f "$__object/parameter/password" ] +then + password="$( cat "$__object/parameter/password" )" +fi + +if [ -n "$user" ] && [ -n "$password" ] +then + if [ -f "$__object/parameter/name" ] + then + database="$( cat "$__object/parameter/name" )" + else + database="$__object_id" + fi + + __mysql_user "$user" --password "$password" + + require="__mysql_user/$user" \ + __mysql_privileges "$database/$user" --database "$database" --user "$user" +fi diff --git a/cdist/conf/type/__mysql_database/parameter/default/state b/cdist/conf/type/__mysql_database/parameter/default/state new file mode 100644 index 00000000..e7f6134f --- /dev/null +++ b/cdist/conf/type/__mysql_database/parameter/default/state @@ -0,0 +1 @@ +present diff --git a/cdist/conf/type/__mysql_database/parameter/optional b/cdist/conf/type/__mysql_database/parameter/optional index 756afee7..6c0b1e85 100644 --- a/cdist/conf/type/__mysql_database/parameter/optional +++ b/cdist/conf/type/__mysql_database/parameter/optional @@ -1,3 +1,4 @@ name user password +state diff --git a/cdist/conf/type/__mysql_privileges/explorer/state b/cdist/conf/type/__mysql_privileges/explorer/state new file mode 100755 index 00000000..97674479 --- /dev/null +++ b/cdist/conf/type/__mysql_privileges/explorer/state @@ -0,0 +1,22 @@ +#!/bin/sh -e + +privileges="$( cat "$__object/parameter/privileges" )" + +database="$( cat "$__object/parameter/database" )" + +table="$( cat "$__object/parameter/table" )" + +user="$( cat "$__object/parameter/user" )" + +host="$( cat "$__object/parameter/host" )" + +check_privileges="$( + mysql -B -N -e "show grants for '$user'@'$host'" \ + | grep -Ei "^grant $privileges on .$database.\..$table. to " || true )" + +if [ -n "$check_privileges" ] +then + echo 'present' +else + echo 'absent' +fi diff --git a/cdist/conf/type/__mysql_privileges/gencode-remote b/cdist/conf/type/__mysql_privileges/gencode-remote new file mode 100755 index 00000000..6b2e0fc1 --- /dev/null +++ b/cdist/conf/type/__mysql_privileges/gencode-remote @@ -0,0 +1,31 @@ +#!/bin/sh -e + +state_is="$( cat "$__object/explorer/state" )" + +state_should="$( cat "$__object/parameter/state" )" + +if [ "$state_is" = "$state_should" ] +then + exit 0 +fi + +privileges="$( cat "$__object/parameter/privileges" )" + +database="$( cat "$__object/parameter/database" )" + +table="$( cat "$__object/parameter/table" )" + +user="$( cat "$__object/parameter/user" )" + +host="$( cat "$__object/parameter/host" )" + +case "$state_should" in + present) + echo "mysql -e 'grant $privileges on \`$database\`.\`$table\` to \`$user\`@\`$host\`'" + echo "grant $privileges on $database.$table to $user@$host" >> "$__messages_out" + ;; + absent) + echo "mysql -e 'revoke $privileges on \`$database\`.\`$table\` from \`$user\`@\`$host\`'" + echo "revoke $privileges on $database.$table from $user@$host" >> "$__messages_out" + ;; +esac diff --git a/cdist/conf/type/__mysql_privileges/parameter/default/host b/cdist/conf/type/__mysql_privileges/parameter/default/host new file mode 100644 index 00000000..2fbb50c4 --- /dev/null +++ b/cdist/conf/type/__mysql_privileges/parameter/default/host @@ -0,0 +1 @@ +localhost diff --git a/cdist/conf/type/__mysql_privileges/parameter/default/privileges b/cdist/conf/type/__mysql_privileges/parameter/default/privileges new file mode 100644 index 00000000..5472efad --- /dev/null +++ b/cdist/conf/type/__mysql_privileges/parameter/default/privileges @@ -0,0 +1 @@ +all privileges diff --git a/cdist/conf/type/__mysql_privileges/parameter/default/state b/cdist/conf/type/__mysql_privileges/parameter/default/state new file mode 100644 index 00000000..e7f6134f --- /dev/null +++ b/cdist/conf/type/__mysql_privileges/parameter/default/state @@ -0,0 +1 @@ +present diff --git a/cdist/conf/type/__mysql_privileges/parameter/default/table b/cdist/conf/type/__mysql_privileges/parameter/default/table new file mode 100644 index 00000000..72e8ffc0 --- /dev/null +++ b/cdist/conf/type/__mysql_privileges/parameter/default/table @@ -0,0 +1 @@ +* diff --git a/cdist/conf/type/__mysql_privileges/parameter/optional b/cdist/conf/type/__mysql_privileges/parameter/optional new file mode 100644 index 00000000..d4ed5bc5 --- /dev/null +++ b/cdist/conf/type/__mysql_privileges/parameter/optional @@ -0,0 +1,4 @@ +privileges +table +host +state diff --git a/cdist/conf/type/__mysql_privileges/parameter/required b/cdist/conf/type/__mysql_privileges/parameter/required new file mode 100644 index 00000000..152b4a1e --- /dev/null +++ b/cdist/conf/type/__mysql_privileges/parameter/required @@ -0,0 +1,2 @@ +database +user diff --git a/cdist/conf/type/__mysql_user/explorer/state b/cdist/conf/type/__mysql_user/explorer/state new file mode 100755 index 00000000..c91bb36a --- /dev/null +++ b/cdist/conf/type/__mysql_user/explorer/state @@ -0,0 +1,36 @@ +#!/bin/sh -e + +if [ -f "$__object/parameter/name" ] +then + name="$( cat "$__object/parameter/name" )" +else + name="$__object_id" +fi + +if [ -f "$__object/parameter/password" ] +then + password="$( cat "$__object/parameter/password" )" +else + password='' +fi + +host="$( cat "$__object/parameter/host" )" + +check_user="$( mysql -B -N -e "select user from mysql.user where user = '$name' and host = '$host'" )" + +if [ -n "$check_user" ] +then + if [ -n "$password" ] + then + check_password="$( mysql -B -N -e "select user from mysql.user where user = '$name' and host = '$host' and password = password( '$password' )" )" + fi + + if [ -n "$password" ] && [ -z "$check_password" ] + then + echo 'change-password' + else + echo 'present' + fi +else + echo 'absent' +fi diff --git a/cdist/conf/type/__mysql_user/gencode-remote b/cdist/conf/type/__mysql_user/gencode-remote new file mode 100755 index 00000000..67500716 --- /dev/null +++ b/cdist/conf/type/__mysql_user/gencode-remote @@ -0,0 +1,50 @@ +#!/bin/sh -e + +state_is="$( cat "$__object/explorer/state" )" + +state_should="$( cat "$__object/parameter/state" )" + +if [ "$state_is" = "$state_should" ] +then + exit 0 +fi + +if [ -f "$__object/parameter/name" ] +then + name="$( cat "$__object/parameter/name" )" +else + name="$__object_id" +fi + +host="$( cat "$__object/parameter/host" )" + +if [ -f "$__object/parameter/password" ] +then + password="$( cat "$__object/parameter/password" )" +else + if [ "$state_should" = 'present' ] + then + echo '--password needed' >&2 + exit 1 + else + password='' + fi +fi + +if [ "$state_is" = 'absent' ] && [ "$state_should" = 'present' ] +then + echo "mysql -e 'create user \`$name\`@\`$host\` identified by \"$password\"'" + echo "create user $name@$host" >> "$__messages_out" + +elif [ "$state_is" != 'absent' ] && [ "$state_should" = 'absent' ] +then + echo "mysql -e 'drop user \`$name\`@\`$host\`'" + echo "drop user $name@$host" >> "$__messages_out" + +elif [ "$state_is" = 'change-password' ] +then + # this only works with MySQL 5.7.6 and later or MariaDB 10.1.20 and later + echo "mysql -e 'alter user \`$name\`@\`$host\` identified by \"$password\"'" + echo "mysql -e 'flush privileges'" + echo "change password $name@$host" >> "$__messages_out" +fi diff --git a/cdist/conf/type/__mysql_user/parameter/default/host b/cdist/conf/type/__mysql_user/parameter/default/host new file mode 100644 index 00000000..2fbb50c4 --- /dev/null +++ b/cdist/conf/type/__mysql_user/parameter/default/host @@ -0,0 +1 @@ +localhost diff --git a/cdist/conf/type/__mysql_user/parameter/default/state b/cdist/conf/type/__mysql_user/parameter/default/state new file mode 100644 index 00000000..e7f6134f --- /dev/null +++ b/cdist/conf/type/__mysql_user/parameter/default/state @@ -0,0 +1 @@ +present diff --git a/cdist/conf/type/__mysql_user/parameter/optional b/cdist/conf/type/__mysql_user/parameter/optional new file mode 100644 index 00000000..a286266c --- /dev/null +++ b/cdist/conf/type/__mysql_user/parameter/optional @@ -0,0 +1,4 @@ +name +host +password +state From 9a693537f4192e3cd133e14cd31ab1bdcc792608 Mon Sep 17 00:00:00 2001 From: Ander Punnar Date: Wed, 1 Jan 2020 12:38:12 +0200 Subject: [PATCH 33/51] __mysql_*: add license headers --- .../conf/type/__mysql_database/explorer/state | 18 ++++++++++++++++++ .../conf/type/__mysql_database/gencode-remote | 18 ++++++++++++++++++ cdist/conf/type/__mysql_database/manifest | 18 ++++++++++++++++++ .../type/__mysql_privileges/explorer/state | 18 ++++++++++++++++++ .../type/__mysql_privileges/gencode-remote | 18 ++++++++++++++++++ cdist/conf/type/__mysql_user/explorer/state | 18 ++++++++++++++++++ cdist/conf/type/__mysql_user/gencode-remote | 18 ++++++++++++++++++ 7 files changed, 126 insertions(+) diff --git a/cdist/conf/type/__mysql_database/explorer/state b/cdist/conf/type/__mysql_database/explorer/state index 16cc9ce5..79858695 100755 --- a/cdist/conf/type/__mysql_database/explorer/state +++ b/cdist/conf/type/__mysql_database/explorer/state @@ -1,4 +1,22 @@ #!/bin/sh -e +# +# 2020 Ander Punnar (ander-at-kvlt-dot-ee) +# +# This file is part of cdist. +# +# cdist is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# cdist is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with cdist. If not, see . +# if [ -f "$__object/parameter/name" ] then diff --git a/cdist/conf/type/__mysql_database/gencode-remote b/cdist/conf/type/__mysql_database/gencode-remote index d3692572..1bdb2b11 100755 --- a/cdist/conf/type/__mysql_database/gencode-remote +++ b/cdist/conf/type/__mysql_database/gencode-remote @@ -1,4 +1,22 @@ #!/bin/sh -e +# +# 2020 Ander Punnar (ander-at-kvlt-dot-ee) +# +# This file is part of cdist. +# +# cdist is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# cdist is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with cdist. If not, see . +# state_is="$( cat "$__object/explorer/state" )" diff --git a/cdist/conf/type/__mysql_database/manifest b/cdist/conf/type/__mysql_database/manifest index a57c31ce..628b543c 100755 --- a/cdist/conf/type/__mysql_database/manifest +++ b/cdist/conf/type/__mysql_database/manifest @@ -1,4 +1,22 @@ #!/bin/sh -e +# +# 2020 Ander Punnar (ander-at-kvlt-dot-ee) +# +# This file is part of cdist. +# +# cdist is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# cdist is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with cdist. If not, see . +# if [ -f "$__object/parameter/user" ] then diff --git a/cdist/conf/type/__mysql_privileges/explorer/state b/cdist/conf/type/__mysql_privileges/explorer/state index 97674479..0cfbaacd 100755 --- a/cdist/conf/type/__mysql_privileges/explorer/state +++ b/cdist/conf/type/__mysql_privileges/explorer/state @@ -1,4 +1,22 @@ #!/bin/sh -e +# +# 2020 Ander Punnar (ander-at-kvlt-dot-ee) +# +# This file is part of cdist. +# +# cdist is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# cdist is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with cdist. If not, see . +# privileges="$( cat "$__object/parameter/privileges" )" diff --git a/cdist/conf/type/__mysql_privileges/gencode-remote b/cdist/conf/type/__mysql_privileges/gencode-remote index 6b2e0fc1..bcd362e6 100755 --- a/cdist/conf/type/__mysql_privileges/gencode-remote +++ b/cdist/conf/type/__mysql_privileges/gencode-remote @@ -1,4 +1,22 @@ #!/bin/sh -e +# +# 2020 Ander Punnar (ander-at-kvlt-dot-ee) +# +# This file is part of cdist. +# +# cdist is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# cdist is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with cdist. If not, see . +# state_is="$( cat "$__object/explorer/state" )" diff --git a/cdist/conf/type/__mysql_user/explorer/state b/cdist/conf/type/__mysql_user/explorer/state index c91bb36a..6817ee9d 100755 --- a/cdist/conf/type/__mysql_user/explorer/state +++ b/cdist/conf/type/__mysql_user/explorer/state @@ -1,4 +1,22 @@ #!/bin/sh -e +# +# 2020 Ander Punnar (ander-at-kvlt-dot-ee) +# +# This file is part of cdist. +# +# cdist is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# cdist is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with cdist. If not, see . +# if [ -f "$__object/parameter/name" ] then diff --git a/cdist/conf/type/__mysql_user/gencode-remote b/cdist/conf/type/__mysql_user/gencode-remote index 67500716..5f13bc87 100755 --- a/cdist/conf/type/__mysql_user/gencode-remote +++ b/cdist/conf/type/__mysql_user/gencode-remote @@ -1,4 +1,22 @@ #!/bin/sh -e +# +# 2020 Ander Punnar (ander-at-kvlt-dot-ee) +# +# This file is part of cdist. +# +# cdist is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# cdist is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with cdist. If not, see . +# state_is="$( cat "$__object/explorer/state" )" From 24862e0208705cd9081f5963d1f07e0fcceb23f2 Mon Sep 17 00:00:00 2001 From: Ander Punnar Date: Fri, 3 Jan 2020 18:26:11 +0200 Subject: [PATCH 34/51] __mysql_database: carry over state --- cdist/conf/type/__mysql_database/manifest | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/cdist/conf/type/__mysql_database/manifest b/cdist/conf/type/__mysql_database/manifest index 628b543c..a3c9ed5d 100755 --- a/cdist/conf/type/__mysql_database/manifest +++ b/cdist/conf/type/__mysql_database/manifest @@ -37,8 +37,16 @@ then database="$__object_id" fi - __mysql_user "$user" --password "$password" + state_should="$( cat "$__object/parameter/state" )" + __mysql_user "$user" \ + --password "$password" \ + --state "$state_should" + + # removing user should remove all user's privileges require="__mysql_user/$user" \ - __mysql_privileges "$database/$user" --database "$database" --user "$user" + __mysql_privileges "$database/$user" \ + --database "$database" \ + --user "$user" \ + --state "$state_should" fi From fcc774cb7b2b3f2128dc77d622d7801397d906b7 Mon Sep 17 00:00:00 2001 From: Ander Punnar Date: Fri, 3 Jan 2020 18:33:23 +0200 Subject: [PATCH 35/51] __mysql_database: add manual --- cdist/conf/type/__mysql_database/man.rst | 55 ++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 cdist/conf/type/__mysql_database/man.rst diff --git a/cdist/conf/type/__mysql_database/man.rst b/cdist/conf/type/__mysql_database/man.rst new file mode 100644 index 00000000..b3b56b5f --- /dev/null +++ b/cdist/conf/type/__mysql_database/man.rst @@ -0,0 +1,55 @@ +cdist-type__mysql_database(7) +============================= + +NAME +---- +cdist-type__mysql_database - Manage a MySQL database + + +DESCRIPTION +----------- + +Create MySQL database and optionally user with all privileges. + + +OPTIONAL PARAMETERS +------------------- +name + Name of database. Defaults to object id. + +user + Create user and give all privileges to database. + +password + Password for user. + +state + Defaults to present. + If absent and user is also set, both will be removed (with privileges). + + +EXAMPLES +-------- + +.. code-block:: sh + + # just create database + __mysql_database foo + + # create database with respective user with all privileges to database + __mysql_database bar \ + --user name \ + --password secret + + +AUTHORS +------- +Ander Punnar + + +COPYING +------- +Copyright \(C) 2020 Ander Punnar. You can redistribute it and/or modify it +under the terms of the GNU General Public License as published by the Free +Software Foundation, either version 3 of the License, or (at your option) any +later version. From 5e8dc7122d764896247258ae4c9049e222f4d7ff Mon Sep 17 00:00:00 2001 From: Ander Punnar Date: Fri, 3 Jan 2020 18:48:11 +0200 Subject: [PATCH 36/51] __mysql_user: add manual --- cdist/conf/type/__mysql_user/man.rst | 48 ++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100644 cdist/conf/type/__mysql_user/man.rst diff --git a/cdist/conf/type/__mysql_user/man.rst b/cdist/conf/type/__mysql_user/man.rst new file mode 100644 index 00000000..c2b222d5 --- /dev/null +++ b/cdist/conf/type/__mysql_user/man.rst @@ -0,0 +1,48 @@ +cdist-type__mysql_user(7) +========================= + +NAME +---- +cdist-type__mysql_user - Manage a MySQL user + + +DESCRIPTION +----------- + +Create MySQL user or change password for the user. + + +OPTIONAL PARAMETERS +------------------- +name + Name of user. Defaults to object id. + +host + Host of user. Defaults to localhost. + +password + Password of user. + +state + Defaults to present. + + +EXAMPLES +-------- + +.. code-block:: sh + + __mysql_user user --password secret + + +AUTHORS +------- +Ander Punnar + + +COPYING +------- +Copyright \(C) 2020 Ander Punnar. You can redistribute it and/or modify it +under the terms of the GNU General Public License as published by the Free +Software Foundation, either version 3 of the License, or (at your option) any +later version. From 4329cced82930a336378765b294b9a3be9433991 Mon Sep 17 00:00:00 2001 From: Ander Punnar Date: Fri, 3 Jan 2020 18:55:55 +0200 Subject: [PATCH 37/51] __mysql_privileges: add manual --- cdist/conf/type/__mysql_privileges/man.rst | 57 ++++++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 cdist/conf/type/__mysql_privileges/man.rst diff --git a/cdist/conf/type/__mysql_privileges/man.rst b/cdist/conf/type/__mysql_privileges/man.rst new file mode 100644 index 00000000..8208d7d4 --- /dev/null +++ b/cdist/conf/type/__mysql_privileges/man.rst @@ -0,0 +1,57 @@ +cdist-type__mysql_privileges(7) +=============================== + +NAME +---- +cdist-type__mysql_privileges - Manage MySQL privileges + + +DESCRIPTION +----------- + +Grant and revoke privileges of MySQL user. + + +REQUIRED PARAMETERS +------------------- +database + Name of database. + +User + Name of user. + + +OPTIONAL PARAMETERS +------------------- +privileges + Defaults to "all". + +table + Defaults to "*". + +host + Defaults to localhost. + +state + "present" grants and "absent" revokes. Defaults to present. + + +EXAMPLES +-------- + +.. code-block:: sh + + __mysql_privileges user-to-db --database db --user user + + +AUTHORS +------- +Ander Punnar + + +COPYING +------- +Copyright \(C) 2020 Ander Punnar. You can redistribute it and/or modify it +under the terms of the GNU General Public License as published by the Free +Software Foundation, either version 3 of the License, or (at your option) any +later version. From 6369bc1ae5e17dd9c0a13d57419107cbdb166245 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Sat, 4 Jan 2020 10:07:53 +0100 Subject: [PATCH 38/51] ++changelog --- docs/changelog | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/changelog b/docs/changelog index 415af0af..8feb0b83 100644 --- a/docs/changelog +++ b/docs/changelog @@ -2,8 +2,9 @@ Changelog --------- next: - * Type __consul_agent: don't deploy init script on Alpine anymore, - it ships with one itself (Nico Schottelius) + * Type __consul_agent: Don't deploy init script on Alpine anymore, it ships with one itself (Nico Schottelius) + * Type __install_chroot_umount: Bugfix: type was not using __chroot_umount/manifest (Steven Armstrong) + * Types __postgres_*: Use double quoted identifiers in generated SQL (fnux) 6.3.0: 2019-12-12 * Type __package_update_index: Fix Alpine part (Dominique Roux) From 72935e0a797461d39f64b9f85bac02ad06347b1d Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Sat, 4 Jan 2020 11:08:48 +0100 Subject: [PATCH 39/51] ++changelog --- docs/changelog | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/changelog b/docs/changelog index 8feb0b83..d0a3a2de 100644 --- a/docs/changelog +++ b/docs/changelog @@ -4,7 +4,7 @@ Changelog next: * Type __consul_agent: Don't deploy init script on Alpine anymore, it ships with one itself (Nico Schottelius) * Type __install_chroot_umount: Bugfix: type was not using __chroot_umount/manifest (Steven Armstrong) - * Types __postgres_*: Use double quoted identifiers in generated SQL (fnux) + * Types __postgres_*: Use double quoted identifiers in generated SQL (Timothée Floure) 6.3.0: 2019-12-12 * Type __package_update_index: Fix Alpine part (Dominique Roux) From e4596593c08f43284e1fa85d1e353b733bebdc57 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Mon, 30 Dec 2019 09:32:15 +0100 Subject: [PATCH 40/51] Add cdist info command --- cdist/argparse.py | 32 +++++++ cdist/exec/local.py | 42 ++------- cdist/exec/util.py | 25 ++++++ cdist/info.py | 183 ++++++++++++++++++++++++++++++++++++++++ docs/src/man1/cdist.rst | 37 +++++++- 5 files changed, 283 insertions(+), 36 deletions(-) create mode 100644 cdist/info.py diff --git a/cdist/argparse.py b/cdist/argparse.py index 7dc683f3..611c484a 100644 --- a/cdist/argparse.py +++ b/cdist/argparse.py @@ -6,6 +6,7 @@ import collections import functools import cdist.configuration import cdist.preos +import cdist.info # set of beta sub-commands @@ -436,6 +437,37 @@ def get_parsers(): ' should be POSIX compatible shell.')) parser['shell'].set_defaults(func=cdist.shell.Shell.commandline) + # Info + parser['info'] = parser['sub'].add_parser('info') + parser['info'].add_argument( + '-a', '--all', help='Display all info. This is the default.', + action='store_true', default=False) + parser['info'].add_argument( + '-c', '--conf-dir', + help='Add configuration directory (can be repeated).', + action='append') + parser['info'].add_argument( + '-e', '--global-explorers', + help='Display info for global explorers.', action='store_true', + default=False) + parser['info'].add_argument( + '-F', '--fixed-string', + help='Interpret pattern as a fixed string.', action='store_true', + default=False) + parser['info'].add_argument( + '-f', '--full', help='Display full details.', + action='store_true', default=False) + parser['info'].add_argument( + '-g', '--config-file', + help='Use specified custom configuration file.', + dest="config_file", required=False) + parser['info'].add_argument( + '-t', '--types', help='Display info for types.', + action='store_true', default=False) + parser['info'].add_argument( + 'pattern', nargs='?', help='Glob pattern.') + parser['info'].set_defaults(func=cdist.info.Info.commandline) + for p in parser: parser[p].epilog = EPILOG diff --git a/cdist/exec/local.py b/cdist/exec/local.py index f83c85df..ad6c6e36 100644 --- a/cdist/exec/local.py +++ b/cdist/exec/local.py @@ -69,7 +69,6 @@ class Local(object): self.exec_path = exec_path self.custom_initial_manifest = initial_manifest - self._add_conf_dirs = add_conf_dirs self.cache_path_pattern = cache_path_pattern self.quiet_mode = quiet_mode if configuration: @@ -84,16 +83,7 @@ class Local(object): self._init_cache_dir(None) self._init_paths() self._init_object_marker() - self._init_conf_dirs() - - @property - def dist_conf_dir(self): - return os.path.abspath(os.path.join(os.path.dirname(cdist.__file__), - "conf")) - - @property - def home_dir(self): - return cdist.home_dir() + self._init_conf_dirs(add_conf_dirs) def _init_log(self): self.log = logging.getLogger(self.target_host[0]) @@ -140,28 +130,9 @@ class Local(object): # Does not need to be secure - just randomly different from .cdist self.object_marker_name = tempfile.mktemp(prefix='.cdist-', dir='') - def _init_conf_dirs(self): - self.conf_dirs = [] - - self.conf_dirs.append(self.dist_conf_dir) - - # Is the default place for user created explorer, type and manifest - if self.home_dir: - self.conf_dirs.append(self.home_dir) - - # Add directories defined in the CDIST_PATH environment variable - # if 'CDIST_PATH' in os.environ: - # cdist_path_dirs = re.split(r'(?. +# +# + +import cdist +import cdist.configuration +import cdist.core +import cdist.exec.util as util +import os +import glob +import fnmatch + + +class Info(object): + + def __init__(self, conf_dirs, args): + self.conf_dirs = conf_dirs + self.all = args.all + self.display_global_explorers = args.global_explorers + self.display_types = args.types + if not self.display_global_explorers and not self.display_types: + self.all = True + self.fixed_string = args.fixed_string + self._setup_glob_pattern(args.pattern) + self.full = args.full + + def _setup_glob_pattern(self, pattern): + if pattern is None: + self.glob_pattern = '*' + elif ('?' in pattern or '*' in pattern or '[' in pattern or + self.fixed_string): + self.glob_pattern = pattern + else: + self.glob_pattern = '*' + pattern + '*' + + @classmethod + def commandline(cls, args): + cfg = cdist.configuration.Configuration(args) + configuration = cfg.get_config(section='GLOBAL') + conf_dirs = util.resolve_conf_dirs(configuration, + args.conf_dir) + c = cls(conf_dirs, args) + c.run() + + def _get_global_explorers(self, conf_path): + rv = [] + global_explorer_path = os.path.join(conf_path, "explorer", + self.glob_pattern) + if self.fixed_string: + if os.path.exists(global_explorer_path): + rv.append(global_explorer_path) + else: + for explorer in glob.glob(global_explorer_path): + rv.append(explorer) + return rv + + def _should_display_type(self, dir_entry): + if not dir_entry.is_dir(): + return False + if self.glob_pattern is None: + return True + if self.fixed_string: + return dir_entry.name == self.glob_pattern + else: + return fnmatch.fnmatch(dir_entry.name, self.glob_pattern) + + def _get_types(self, conf_path): + rv = [] + types_path = os.path.join(conf_path, "type") + if not os.path.exists(types_path): + return rv + with os.scandir(types_path) as it: + for entry in it: + if self._should_display_type(entry): + rv.append(entry.path) + return rv + + def _display_details(self, title, details, default_values=None, + deprecated=None): + if not details: + return + if isinstance(details, bool): + print("\t{}: {}".format(title, 'yes' if details else 'no')) + elif isinstance(details, str): + print("\t{}: {}".format(title, details)) + elif isinstance(details, list): + dv = dict(default_values) if default_values else {} + dp = dict(deprecated) if deprecated else {} + + print("\t{}:".format(title)) + for x in sorted(details): + print("\t\t{}".format(x), end='') + has_default = x in dv + is_deprecated = x in dp + need_comma = False + if has_default or is_deprecated: + print(" (", end='') + if has_default: + print("default: {}".format(dv[x]), end='') + need_comma = True + if is_deprecated: + print("{}deprecated".format(', ' if need_comma else ''), + end='') + if has_default or is_deprecated: + print(")", end='') + print() + + def _display_type_parameters(self, cdist_type): + self._display_details("required parameters", + cdist_type.required_parameters, + default_values=cdist_type.parameter_defaults, + deprecated=cdist_type.deprecated_parameters) + self._display_details("required multiple parameters", + cdist_type.required_multiple_parameters, + default_values=cdist_type.parameter_defaults, + deprecated=cdist_type.deprecated_parameters) + self._display_details("optional parameters", + cdist_type.optional_parameters, + default_values=cdist_type.parameter_defaults, + deprecated=cdist_type.deprecated_parameters) + self._display_details("optional multiple parameters", + cdist_type.optional_multiple_parameters, + default_values=cdist_type.parameter_defaults, + deprecated=cdist_type.deprecated_parameters) + self._display_details("boolean parameters", + cdist_type.boolean_parameters, + default_values=cdist_type.parameter_defaults, + deprecated=cdist_type.deprecated_parameters) + + def _display_type_characteristics(self, cdist_type): + characteristics = [] + if cdist_type.is_install: + characteristics.append('install') + else: + characteristics.append('config') + if cdist_type.is_singleton: + characteristics.append('singleton') + if cdist_type.is_nonparallel: + characteristics.append('nonparallel') + else: + characteristics.append('parallel') + if cdist_type.deprecated is not None: + characteristics.append('deprecated') + print("\t{}".format(', '.join(characteristics))) + + def _display_type_details(self, type_path): + dirname, basename = os.path.split(type_path) + cdist_type = cdist.core.CdistType(dirname, basename) + + self._display_type_characteristics(cdist_type) + self._display_type_parameters(cdist_type) + + def run(self): + rv = [] + for conf_path in self.conf_dirs: + if self.all or self.display_global_explorers: + rv.extend((x, 'E', ) for x in self._get_global_explorers( + conf_path)) + if self.all or self.display_types: + rv.extend((x, 'T', ) for x in self._get_types(conf_path)) + rv = sorted(rv, key=lambda x: x[0]) + for x, t in rv: + print(x) + if self.full and t == 'T': + self._display_type_details(x) diff --git a/docs/src/man1/cdist.rst b/docs/src/man1/cdist.rst index 55db82ed..66c356ec 100644 --- a/docs/src/man1/cdist.rst +++ b/docs/src/man1/cdist.rst @@ -11,7 +11,7 @@ SYNOPSIS :: - cdist [-h] [-V] {banner,config,install,inventory,preos,shell} ... + cdist [-h] [-V] {banner,config,install,inventory,preos,shell,info} ... cdist banner [-h] [-l LOGLEVEL] [-q] [-v] @@ -84,6 +84,8 @@ SYNOPSIS cdist shell [-h] [-l LOGLEVEL] [-q] [-v] [-s SHELL] + cdist info [-h] [-a] [-c CONF_DIR] [-e] [-F] [-f] [-t] [pattern] + DESCRIPTION ----------- @@ -604,6 +606,39 @@ usage. Its primary use is for debugging type parameters. be POSIX compatible shell. +INFO +---- +Display information for cdist (global explorers, types). + +**pattern** + Glob pattern. If it contains special characters('?', '*', '[') then it is + used as specified, otherwise it is translated to `*pattern*`. + +**-h, --help** + Show help message and exit. + +**-a, --all** + Display all info. This is the default. + +**-c CONF_DIR, --conf-dir CONF_DIR** + Add configuration directory (can be repeated). + +**-e, --global-explorers** + Display info for global explorers. + +**-F, --fixed-string** + Interpret pattern as a fixed string. + +**-f, --full** + Display full details. + +**-g CONFIG_FILE, --config-file CONFIG_FILE** + Use specified custom configuration file. + +**-t, --types** + Display info for types. + + CONFIGURATION ------------- cdist obtains configuration data from the following sources in the following From 7b1192257d8517cc838f84a8c57ac0964d104588 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Sat, 4 Jan 2020 13:17:54 +0100 Subject: [PATCH 41/51] Fix incomplete cdist info synopsis --- docs/src/man1/cdist.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/src/man1/cdist.rst b/docs/src/man1/cdist.rst index 66c356ec..bc73a0b8 100644 --- a/docs/src/man1/cdist.rst +++ b/docs/src/man1/cdist.rst @@ -84,7 +84,7 @@ SYNOPSIS cdist shell [-h] [-l LOGLEVEL] [-q] [-v] [-s SHELL] - cdist info [-h] [-a] [-c CONF_DIR] [-e] [-F] [-f] [-t] [pattern] + cdist info [-h] [-a] [-c CONF_DIR] [-e] [-F] [-f] [-g CONFIG_FILE] [-t] [pattern] DESCRIPTION From e2015367925c8a2716e6a79f6f2609f2877cd134 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Sat, 4 Jan 2020 13:18:13 +0100 Subject: [PATCH 42/51] ++changelog --- docs/changelog | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/changelog b/docs/changelog index d0a3a2de..7489489e 100644 --- a/docs/changelog +++ b/docs/changelog @@ -5,6 +5,7 @@ next: * Type __consul_agent: Don't deploy init script on Alpine anymore, it ships with one itself (Nico Schottelius) * Type __install_chroot_umount: Bugfix: type was not using __chroot_umount/manifest (Steven Armstrong) * Types __postgres_*: Use double quoted identifiers in generated SQL (Timothée Floure) + * Core: Add cdist info command (Darko Poljak) 6.3.0: 2019-12-12 * Type __package_update_index: Fix Alpine part (Dominique Roux) From bc1990c7c8244b2d00a91bc13dbf796d91c21041 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Sat, 4 Jan 2020 13:44:04 +0100 Subject: [PATCH 43/51] ++changelog --- docs/changelog | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/changelog b/docs/changelog index 7489489e..fc4d69a8 100644 --- a/docs/changelog +++ b/docs/changelog @@ -6,6 +6,8 @@ next: * Type __install_chroot_umount: Bugfix: type was not using __chroot_umount/manifest (Steven Armstrong) * Types __postgres_*: Use double quoted identifiers in generated SQL (Timothée Floure) * Core: Add cdist info command (Darko Poljak) + * New types: __mysql_user, __mysql_privileges (Ander Punnar) + * Type __mysql_database: Rewrite (Ander Punnar) 6.3.0: 2019-12-12 * Type __package_update_index: Fix Alpine part (Dominique Roux) From 7c9dd3b03e31f39cbe758510e2aa1f542eae4825 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Sat, 4 Jan 2020 14:36:14 +0100 Subject: [PATCH 44/51] Release 6.4.0 --- docs/changelog | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/changelog b/docs/changelog index fc4d69a8..706d76af 100644 --- a/docs/changelog +++ b/docs/changelog @@ -1,7 +1,7 @@ Changelog --------- -next: +6.4.0: 2020-01-04 * Type __consul_agent: Don't deploy init script on Alpine anymore, it ships with one itself (Nico Schottelius) * Type __install_chroot_umount: Bugfix: type was not using __chroot_umount/manifest (Steven Armstrong) * Types __postgres_*: Use double quoted identifiers in generated SQL (Timothée Floure) From d1a64596fe73697d3e6a8e514991ffc173d04772 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Sat, 4 Jan 2020 14:56:05 +0100 Subject: [PATCH 45/51] Update build-helper --- bin/build-helper | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/build-helper b/bin/build-helper index 69dee4c7..ed41e438 100755 --- a/bin/build-helper +++ b/bin/build-helper @@ -370,7 +370,7 @@ eof cat << eof Manual steps post release: - cdist-web - - send mail body generated in mailinglist.tmp and inform Dmitry for deb + - send generated mailinglist.tmp mail - twitter eof ;; From d4bd49bbb598dfd4e4a510a2bf9035dcb4686e4a Mon Sep 17 00:00:00 2001 From: Ander Punnar Date: Sat, 4 Jan 2020 17:43:57 +0200 Subject: [PATCH 46/51] __acl: rename --acl to --entry for the sake of consistency, add compatibility --- cdist/conf/type/__acl/gencode-remote | 5 +++- cdist/conf/type/__acl/man.rst | 28 +++++++++---------- .../conf/type/__acl/parameter/deprecated/acl | 1 + .../type/__acl/parameter/optional_multiple | 1 + 4 files changed, 20 insertions(+), 15 deletions(-) create mode 100644 cdist/conf/type/__acl/parameter/deprecated/acl diff --git a/cdist/conf/type/__acl/gencode-remote b/cdist/conf/type/__acl/gencode-remote index 6dab4d09..f4f0d1e2 100755 --- a/cdist/conf/type/__acl/gencode-remote +++ b/cdist/conf/type/__acl/gencode-remote @@ -28,7 +28,10 @@ acl_path="/$__object_id" acl_is="$( cat "$__object/explorer/acl_is" )" -if [ -f "$__object/parameter/acl" ] +if [ -f "$__object/parameter/entry" ] +then + acl_should="$( cat "$__object/parameter/entry" )" +elif [ -f "$__object/parameter/acl" ] then acl_should="$( cat "$__object/parameter/acl" )" elif diff --git a/cdist/conf/type/__acl/man.rst b/cdist/conf/type/__acl/man.rst index 85e946ce..c3493e49 100644 --- a/cdist/conf/type/__acl/man.rst +++ b/cdist/conf/type/__acl/man.rst @@ -15,7 +15,7 @@ See ``setfacl`` and ``acl`` manpages for more details. REQUIRED MULTIPLE PARAMETERS ---------------------------- -acl +entry Set ACL entry following ``getfacl`` output syntax. @@ -36,8 +36,8 @@ remove DEPRECATED PARAMETERS --------------------- -Parameters ``user``, ``group``, ``mask`` and ``other`` are deprecated and they -will be removed in future versions. Please use ``acl`` parameter instead. +Parameters ``acl``, ``user``, ``group``, ``mask`` and ``other`` are deprecated and they +will be removed in future versions. Please use ``entry`` parameter instead. EXAMPLES @@ -49,27 +49,27 @@ EXAMPLES --default \ --recursive \ --remove \ - --acl user:alice:rwx \ - --acl user:bob:r-x \ - --acl group:project-group:rwx \ - --acl group:some-other-group:r-x \ - --acl mask::r-x \ - --acl other::r-x + --entry user:alice:rwx \ + --entry user:bob:r-x \ + --entry group:project-group:rwx \ + --entry group:some-other-group:r-x \ + --entry mask::r-x \ + --entry other::r-x # give Alice read-only access to subdir, # but don't allow her to see parent content. __acl /srv/project2 \ --remove \ - --acl default:group:secret-project:rwx \ - --acl group:secret-project:rwx \ - --acl user:alice:--x + --entry default:group:secret-project:rwx \ + --entry group:secret-project:rwx \ + --entry user:alice:--x __acl /srv/project2/subdir \ --default \ --remove \ - --acl group:secret-project:rwx \ - --acl user:alice:r-x + --entry group:secret-project:rwx \ + --entry user:alice:r-x AUTHORS diff --git a/cdist/conf/type/__acl/parameter/deprecated/acl b/cdist/conf/type/__acl/parameter/deprecated/acl new file mode 100644 index 00000000..94e14159 --- /dev/null +++ b/cdist/conf/type/__acl/parameter/deprecated/acl @@ -0,0 +1 @@ +see manual for details diff --git a/cdist/conf/type/__acl/parameter/optional_multiple b/cdist/conf/type/__acl/parameter/optional_multiple index 95c25d55..c615d507 100644 --- a/cdist/conf/type/__acl/parameter/optional_multiple +++ b/cdist/conf/type/__acl/parameter/optional_multiple @@ -1,3 +1,4 @@ +entry acl user group From 51ba4a49d8ec79968f79563f994489c619f10bac Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Sat, 4 Jan 2020 18:21:23 +0100 Subject: [PATCH 47/51] ++changelog --- docs/changelog | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/changelog b/docs/changelog index 706d76af..a7bcf9b1 100644 --- a/docs/changelog +++ b/docs/changelog @@ -1,6 +1,9 @@ Changelog --------- +next: + * Type __acl: Add --entry parameter to replace --acl, deprecate --acl (Ander Punnar) + 6.4.0: 2020-01-04 * Type __consul_agent: Don't deploy init script on Alpine anymore, it ships with one itself (Nico Schottelius) * Type __install_chroot_umount: Bugfix: type was not using __chroot_umount/manifest (Steven Armstrong) From 11f569959d6e331d4d5052ca73fb5d83bf9df8e7 Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Sat, 11 Jan 2020 14:16:33 +0100 Subject: [PATCH 48/51] Fix missing configuration file usage, support -g PreOS code did not use configuration support. This fix adds support for using cdist configuration, which takes into account cdist configuration file, environment variables and command line options, especially conf_dir. It also adds support for -g, --config-file option, for specifying custom configuration file. --- cdist/preos.py | 24 +++++++++++------------- docs/changelog | 1 + docs/src/man1/cdist.rst | 7 ++++++- 3 files changed, 18 insertions(+), 14 deletions(-) diff --git a/cdist/preos.py b/cdist/preos.py index 378071db..491338d2 100644 --- a/cdist/preos.py +++ b/cdist/preos.py @@ -5,8 +5,9 @@ import inspect import argparse import cdist import logging -import re import cdist.argparse +import cdist.configuration +import cdist.exec.util as util _PREOS_CALL = "commandline" @@ -24,16 +25,6 @@ def extend_plugins_path(dirs): _PLUGINS_PATH.append(preos_dir) -cdist_home = cdist.home_dir() -if cdist_home: - extend_plugins_path((cdist_home, )) -x = 'CDIST_PATH' -if x in os.environ: - vals = re.split(r'(? Date: Sat, 11 Jan 2020 15:26:46 +0100 Subject: [PATCH 49/51] Info command: support tilde expansion --- cdist/exec/util.py | 7 +++++++ cdist/info.py | 8 +++----- cdist/preos.py | 7 ++----- docs/changelog | 1 + 4 files changed, 13 insertions(+), 10 deletions(-) diff --git a/cdist/exec/util.py b/cdist/exec/util.py index 5513f01d..9787f431 100644 --- a/cdist/exec/util.py +++ b/cdist/exec/util.py @@ -24,6 +24,7 @@ import os from tempfile import TemporaryFile import cdist +import cdist.configuration # IMPORTANT: @@ -200,3 +201,9 @@ def resolve_conf_dirs(configuration, add_conf_dirs): conf_dirs.extend(add_conf_dirs) conf_dirs = set(conf_dirs) return conf_dirs + + +def resolve_conf_dirs_from_config_and_args(args): + cfg = cdist.configuration.Configuration(args) + configuration = cfg.get_config(section='GLOBAL') + return resolve_conf_dirs(configuration, args.conf_dir) diff --git a/cdist/info.py b/cdist/info.py index 4c1d3560..b896a3d1 100644 --- a/cdist/info.py +++ b/cdist/info.py @@ -53,10 +53,7 @@ class Info(object): @classmethod def commandline(cls, args): - cfg = cdist.configuration.Configuration(args) - configuration = cfg.get_config(section='GLOBAL') - conf_dirs = util.resolve_conf_dirs(configuration, - args.conf_dir) + conf_dirs = util.resolve_conf_dirs_from_config_and_args(args) c = cls(conf_dirs, args) c.run() @@ -170,7 +167,8 @@ class Info(object): def run(self): rv = [] - for conf_path in self.conf_dirs: + for cp in self.conf_dirs: + conf_path = os.path.expanduser(cp) if self.all or self.display_global_explorers: rv.extend((x, 'E', ) for x in self._get_global_explorers( conf_path)) diff --git a/cdist/preos.py b/cdist/preos.py index 491338d2..e353fe3b 100644 --- a/cdist/preos.py +++ b/cdist/preos.py @@ -101,13 +101,10 @@ class PreOS(object): action='store_true', default=False) parser.add_argument('remainder_args', nargs=argparse.REMAINDER) args = parser.parse_args(argv[1:]) - cdist.argparse.handle_loglevel(args) + st.argparse.handle_loglevel(args) log.debug("preos args : {}".format(args)) - cfg = cdist.configuration.Configuration(args) - configuration = cfg.get_config(section='GLOBAL') - conf_dirs = util.resolve_conf_dirs(configuration, - args.conf_dir) + conf_dirs = util.resolve_conf_dirs_from_config_and_args(args) extend_plugins_path(conf_dirs) sys.path.extend(_PLUGINS_PATH) diff --git a/docs/changelog b/docs/changelog index 1b1a909e..526fc320 100644 --- a/docs/changelog +++ b/docs/changelog @@ -4,6 +4,7 @@ Changelog next: * Type __acl: Add --entry parameter to replace --acl, deprecate --acl (Ander Punnar) * Core: preos: Fix missing configuration file usage, support -g, --config-file option (Darko Poljak) + * Core info command: Support tilde expansion of conf directories (Darko Poljak) 6.4.0: 2020-01-04 * Type __consul_agent: Don't deploy init script on Alpine anymore, it ships with one itself (Nico Schottelius) From 3258fc98e15fedbd98e17f7d0b568a38b8da139c Mon Sep 17 00:00:00 2001 From: Darko Poljak Date: Sun, 12 Jan 2020 12:19:49 +0100 Subject: [PATCH 50/51] Fix typo --- cdist/preos.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cdist/preos.py b/cdist/preos.py index e353fe3b..bf2a8e60 100644 --- a/cdist/preos.py +++ b/cdist/preos.py @@ -101,7 +101,7 @@ class PreOS(object): action='store_true', default=False) parser.add_argument('remainder_args', nargs=argparse.REMAINDER) args = parser.parse_args(argv[1:]) - st.argparse.handle_loglevel(args) + cdist.argparse.handle_loglevel(args) log.debug("preos args : {}".format(args)) conf_dirs = util.resolve_conf_dirs_from_config_and_args(args) From ef2f4b9a004369cc390f1271721bc22d6a44f402 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Timoth=C3=A9e=20Floure?= Date: Fri, 17 Jan 2020 11:21:28 +0100 Subject: [PATCH 51/51] __postgres_*: fix forgotten edge cases in delimited identifier escape --- cdist/conf/type/__postgres_database/gencode-remote | 8 ++++++-- cdist/conf/type/__postgres_role/gencode-remote | 10 ++++++---- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/cdist/conf/type/__postgres_database/gencode-remote b/cdist/conf/type/__postgres_database/gencode-remote index 9f12d215..47e6b97c 100755 --- a/cdist/conf/type/__postgres_database/gencode-remote +++ b/cdist/conf/type/__postgres_database/gencode-remote @@ -43,10 +43,14 @@ if [ "$state_should" != "$state_is" ]; then if [ -f "$__object/parameter/owner" ]; then owner="-O \"$(cat "$__object/parameter/owner")\"" fi - echo "su - '$postgres_user' -c \"createdb $owner \"$name\"\"" + cat << EOF +su - '$postgres_user' -c "createdb $owner \"$name\"" +EOF ;; absent) - echo "su - '$postgres_user' -c \"dropdb \"$name\"\"" + cat << EOF +su - '$postgres_user' -c "dropdb \"$name\"" +EOF ;; esac fi diff --git a/cdist/conf/type/__postgres_role/gencode-remote b/cdist/conf/type/__postgres_role/gencode-remote index 0b8852f4..977832c9 100755 --- a/cdist/conf/type/__postgres_role/gencode-remote +++ b/cdist/conf/type/__postgres_role/gencode-remote @@ -53,11 +53,13 @@ case "$state_should" in done [ -n "$password" ] && password="PASSWORD '$password'" - - cmd="CREATE ROLE \"$name\" WITH $password $booleans" - echo "su - '$postgres_user' -c \"psql postgres -wc \\\"$cmd\\\"\"" + cat << EOF +su - '$postgres_user' -c "psql postgres -wc 'CREATE ROLE \"$name\" WITH $password $booleans;'" +EOF ;; absent) - echo "su - '$postgres_user' -c \"dropuser \\\"$name\\\"\"" + cat << EOF +su - '$postgres_user' -c "dropuser \"$name\"" +EOF ;; esac