Merge remote-tracking branch 'telmich/master' into oo-restructure

This commit is contained in:
Steven Armstrong 2011-10-06 22:32:03 +02:00
commit b4fc05ba09
24 changed files with 491 additions and 570 deletions

View file

@ -21,6 +21,7 @@
# #
import argparse import argparse
import datetime
import logging import logging
import os import os
import re import re
@ -105,6 +106,8 @@ if __name__ == "__main__":
try: try:
logging.basicConfig(format='%(levelname)s: %(message)s') logging.basicConfig(format='%(levelname)s: %(message)s')
time_start = datetime.datetime.now()
if re.match(TYPE_PREFIX, os.path.basename(sys.argv[0])): if re.match(TYPE_PREFIX, os.path.basename(sys.argv[0])):
import cdist.emulator import cdist.emulator
cdist.emulator.run(sys.argv) cdist.emulator.run(sys.argv)
@ -117,6 +120,14 @@ if __name__ == "__main__":
import cdist.path import cdist.path
commandline() commandline()
time_end = datetime.datetime.now()
duration = time_end - time_start
# FIXME: move into runner
# log.info("Finished run of %s in %s seconds", self.target_host,
# duration.total_seconds())
log.info("Finished run in %s seconds", duration.total_seconds())
except KeyboardInterrupt: except KeyboardInterrupt:
sys.exit(0) sys.exit(0)
except cdist.Error as e: except cdist.Error as e:

View file

@ -1,6 +1,7 @@
2.0.3: 2.0.3:
* Improved logging, added --verbose, by more quiet by default * Improved logging, added --verbose, by more quiet by default
* Bugfix __user: Correct quoting (Steven Armstrong) * Bugfix __user: Correct quoting (Steven Armstrong)
* FIXME: Support for __remote_exec and __remote_copy
2.0.2: 2011-09-27 2.0.2: 2011-09-27
* Add support for detection of OpenWall Linux (Matthias Teege) * Add support for detection of OpenWall Linux (Matthias Teege)

30
doc/dev/logs/2011-10-06 Normal file
View file

@ -0,0 +1,30 @@
GlobalExplorer
list_explorers()
list_explorers_names()
base_dir
__init__(name)
out_dir
env
name = id
path
return_code
return_value
--------------------------------------------------------------------------------
Exec:
normal:
scp /from/where $USER@$HOST:REMOTE_BASE/cdist-internal
ssh $USER@$HOST MY_CMD_THAT_NEEDS_TO_RUN_IN_BIN_SH (including ENV)
sudo:
scp $USER@$HOST:REMOTE_BASE/cdist-internal
ssh $USER@$HOST sudo MY_CMD_THAT_NEEDS_TO_RUN_IN_BIN_SH (including ENV)
chroot:
[sudo] cp file /chroot/THE_HOST_BASE/REMOTE_BASE/cdist-internal
[sudo] chroot /chroot MY_CMD_THAT_NEEDS_TO_RUN_IN_BIN_SH (including ENV)

View file

@ -0,0 +1,87 @@
Commands needed:
conf/cmd/remote_exec
conf/cmd/copy
If ! conf/cmd/remote_exec:
use builtin
If ! conf/cmd/copy:
use builtin
--------------------------------------------------------------------------------
--cmd-dir?
$__cdist_cmd_dir
--------------------------------------------------------------------------------
-> Depend on session!
Builtin:
cdist.exec.run_or_fail(["scp", "-qr", source,
self.remote_user + "@" +
self.target_host + ":" +
destination])
self.remote_prefix = ["ssh", self.remote_user + "@" + self.target_host]
self.remote_user = remote_user
self.remote_prefix = remote_prefix
--------------------------------------------------------------------------------
What is in a session?
base_dir
target_host
--------------------------------------------------------------------------------
remote_user
pseudo-static, can be hardcoded again
--------------------------------------------------------------------------------
Result:
os.environ['__remote_exec'] = ["ssh", "-l", "root" ]
os.environ['__remote_exec'] = ["ssh", "-o", "User=root" ]
os.environ['__remote_copy'] = ["scp", "-o", "User=root" ]
__remote_exec=~/sudossh __remote_copy=... cdist config localhost
~/sudossh hostname $@...
~/sudocopy a hostname:b
~/chrootssh
~/chrootcopy
a)
3 cmd verzeichnnise: cdist, sudo, chroot
pro aufruf variable ändern
b)
1 dir, mit zeug
pro aufruf variablen ändern
conf/cmd/remote_exec
args for __remote_exec
$1 = hostname
$2 - ... = stuff to be executed in /bin/sh on remote side
$2 - $7 = env
$7 - 12 = cmd
args for __remote_copy
$1 = file here
$2 = hostname:destination
--------------------------------------------------------------------------------
There needs to be an easy way to change those cmds!
--------------------------------------------------------------------------------
Env-Passing:
_a=b test -> test can access $_a
_a=b test $_a -> $1 = "", because _a is *not* set within the shell
_a=b; test -> can access $_a
_a=b; test $_a -> $1 == "b"

View file

@ -1,3 +1,18 @@
2.0.3:
- fix emulator
- introduce tests:
- does $require work?
- $whatever should fail if there is no global explorer directory
- emulator may only be called with __ as prefix - fail otherwise!
- Create GlobalExplorer
--------------------------------------------------------------------------------
- insert prefix into logger to distinguish between modules
- in debug/info only?
- Fix / rewrite cdist-quickstart - Fix / rewrite cdist-quickstart
- write tutorial!!!!!!!!! - write tutorial!!!!!!!!!

View file

@ -1,48 +0,0 @@
cdist-config(1)
===============
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-config - Read basic cdist configuration
DESCRIPTION
-----------
Cdist-config is sourced by cdist programs and provides hints on where to find
types, manifests, etc. Generally speaking, it's just usable from within the
core and is only of interest for cdist-developers.
ENVIRONMENT VARIABLES
---------------------
The following list contains environment variables that are known
to be changed by users in various situations. To change the variable,
use your current shell and export it, so all cdist-binaries know about it.
__cdist_tmp_base_dir::
Normally this points to /tmp. In case /tmp is not suitable for
cdist (i.e. has noexec flag setup) you can change this variable
to point to a better location.
EXAMPLES
--------
If /tmp has the noexec flag, you can use $HOME/.tmp for instance:
--------------------------------------------------------------------------------
export __cdist_tmp_base_dir=$HOME/.tmp
--------------------------------------------------------------------------------
SEE ALSO
--------
cdist(7)
COPYING
-------
Copyright \(C) 2010-2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,49 +0,0 @@
cdist-env(1)
============
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-env - Setup environment for using cdist
SYNOPSIS
--------
cdist-env
DESCRIPTION
-----------
cdist-env outputs two strings suitable for usage in your current shell,
so you can use cdist from the checkout. cdist-env essentially helps you
to easily setup PATH and MANPATH.
If you've multiple checkouts of cdist and run cdist-env from the various
checkouts, a new run will prepend the last directory, thus ensures you
can run it multiple times and does what one expects.
EXAMPLES
--------
For use in bourne shell variants (like dash, bash, ksh) as well as
in csh variants (csh, tcsh):
--------------------------------------------------------------------------------
eval `./bin/cdist-env`
--------------------------------------------------------------------------------
For bourne shell, there is also a shorter version:
--------------------------------------------------------------------------------
. ./bin/cdist-env
--------------------------------------------------------------------------------
SEE ALSO
--------
cdist(7)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,47 +0,0 @@
cdist-quickstart(1)
===================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-quickstart - Make use of cinit in 5 minutes
SYNOPSIS
--------
cdist-quickstart
DESCRIPTION
-----------
cdist-quickstart is an interactive guide to cdist. It should be one
of the first tools you use when you begin with cdist.
EXAMPLES
--------
To use cdist-quickstart, add the bin directory to your PATH, execute
cdist-quickstart and enjoy cdist:
--------------------------------------------------------------------------------
# Bourne shell example
export PATH=$(pwd -P)/bin:$PATH
# Alternatively, usable for csh and bsh, set's up PATH and MANPATH
eval `./bin/cdist-env`
# Let's go!
cdist-quickstart
--------------------------------------------------------------------------------
SEE ALSO
--------
- cdist(7)
- cdist-env(1)
COPYING
-------
Copyright \(C) 2010-2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -1,56 +0,0 @@
cdist-type-emulator(1)
======================
Nico Schottelius <nico-cdist--@--schottelius.org>
NAME
----
cdist-type-emulator - Emulate type and record parameters and dependencies
SYNOPSIS
--------
cdist-type-emulator [TYPE ARGS]
DESCRIPTION
-----------
cdist-type-emulator is normally called through a link to it of the
name of a specifc type. It saves the given parameters into
a parameters directory and the requirements into a require file.
It checks whether the parameters are valid:
- are required parameter given?
- are all other required parameters specified as optional?
EXAMPLES
--------
Your manifest may contain stuff like this:
--------------------------------------------------------------------------------
__addifnosuchline /tmp/linetest --line "test"
__motd
--------------------------------------------------------------------------------
In both cases, cdist-type-emulator is called instead of a real type.
In the first case, the object id "/tmp/linetest" is recorded and the
parameter "line" stored with the content "test".
In the second case, __motd must be decleared as a singleton, as the
object id is missing.
SEE ALSO
--------
- cdist(7)
- cdist-type-build-emulation(1)
COPYING
-------
Copyright \(C) 2011 Nico Schottelius. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -33,4 +33,4 @@ class MissingEnvironmentVariableError(Error):
self.name = name self.name = name
def __str__(self): def __str__(self):
return 'Missing required environment variable: {0.name}'.format(o) return 'Missing required environment variable: ' + str(self.name)

View file

@ -22,10 +22,13 @@
import datetime import datetime
import logging import logging
log = logging.getLogger(__name__) import os
import sys
import cdist.config_install import cdist.config_install
log = logging.getLogger(__name__)
class Config(cdist.config_install.ConfigInstall): class Config(cdist.config_install.ConfigInstall):
pass pass
@ -35,6 +38,9 @@ def config(args):
time_start = datetime.datetime.now() time_start = datetime.datetime.now()
os.environ['__remote_exec'] = "ssh -o User=root -q"
os.environ['__remote_copy'] = "scp -o User=root -q"
for host in args.host: for host in args.host:
c = Config(host, initial_manifest=args.manifest, home=args.cdist_home, debug=args.debug) c = Config(host, initial_manifest=args.manifest, home=args.cdist_home, debug=args.debug)
if args.parallel: if args.parallel:

View file

@ -20,7 +20,6 @@
# #
# #
import datetime
import logging import logging
import os import os
import stat import stat
@ -29,37 +28,32 @@ import sys
import cdist.emulator import cdist.emulator
import cdist.path import cdist.path
import cdist.core
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
CODE_HEADER = "#!/bin/sh -e\n" CODE_HEADER = "#!/bin/sh -e\n"
class ConfigInstall: class ConfigInstall:
"""Class to hold install and config methods""" """Cdist main class to hold arbitrary data"""
def __init__(self, target_host, def __init__(self, target_host,
initial_manifest=False, initial_manifest=False,
remote_user="root",
home=None, home=None,
exec_path=sys.argv[0], exec_path=sys.argv[0],
debug=False): debug=False):
self.target_host = target_host self.target_host = target_host
self.debug = debug os.environ['target_host'] = target_host
self.remote_user = remote_user
self.exec_path = exec_path
# FIXME: broken - construct elsewhere! self.debug = debug
self.remote_prefix = ["ssh", self.remote_user + "@" + self.target_host] self.exec_path = exec_path
self.path = cdist.path.Path(self.target_host, self.path = cdist.path.Path(self.target_host,
initial_manifest=initial_manifest, initial_manifest=initial_manifest,
remote_user=self.remote_user,
remote_prefix=self.remote_prefix,
base_dir=home, base_dir=home,
debug=debug) debug=debug)
self.objects_prepared = []
def cleanup(self): def cleanup(self):
self.path.cleanup() self.path.cleanup()
@ -68,7 +62,7 @@ class ConfigInstall:
log.info("Running global explorers") log.info("Running global explorers")
explorers = self.path.list_global_explorers() explorers = self.path.list_global_explorers()
if(len(explorers) == 0): if(len(explorers) == 0):
raise CdistError("No explorers found in", self.path.global_explorer_dir) raise cdist.Error("No explorers found in " + self.path.global_explorer_dir)
self.path.transfer_global_explorers() self.path.transfer_global_explorers()
for explorer in explorers: for explorer in explorers:
@ -78,14 +72,13 @@ class ConfigInstall:
cmd.append("__explorer=" + cdist.path.REMOTE_GLOBAL_EXPLORER_DIR) cmd.append("__explorer=" + cdist.path.REMOTE_GLOBAL_EXPLORER_DIR)
cmd.append(self.path.remote_global_explorer_path(explorer)) cmd.append(self.path.remote_global_explorer_path(explorer))
cdist.exec.run_or_fail(cmd, stdout=output_fd, remote_prefix=self.remote_prefix) cdist.exec.run_or_fail(cmd, stdout=output_fd, remote_prefix=True)
output_fd.close() output_fd.close()
# FIXME: where to call this from?
def run_type_explorer(self, cdist_object): def run_type_explorer(self, cdist_object):
"""Run type specific explorers for objects""" """Run type specific explorers for objects"""
type = self.path.get_type_from_object(cdist_object) type = cdist_object.type
self.path.transfer_type_explorers(type) self.path.transfer_type_explorers(type)
cmd = [] cmd = []
@ -98,7 +91,6 @@ class ConfigInstall:
# Need to transfer at least the parameters for objects to be useful # Need to transfer at least the parameters for objects to be useful
self.path.transfer_object_parameter(cdist_object) self.path.transfer_object_parameter(cdist_object)
# FIXME: Broken due to refactoring into type.py
explorers = self.path.list_type_explorers(type) explorers = self.path.list_type_explorers(type)
for explorer in explorers: for explorer in explorers:
remote_cmd = cmd + [os.path.join(self.path.remote_type_explorer_dir(type), explorer)] remote_cmd = cmd + [os.path.join(self.path.remote_type_explorer_dir(type), explorer)]
@ -107,7 +99,7 @@ class ConfigInstall:
log.debug("%s exploring %s using %s storing to %s", log.debug("%s exploring %s using %s storing to %s",
cdist_object, explorer, remote_cmd, output) cdist_object, explorer, remote_cmd, output)
cdist.exec.run_or_fail(remote_cmd, stdout=output_fd, remote_prefix=self.remote_prefix) cdist.exec.run_or_fail(remote_cmd, stdout=output_fd, remote_prefix=True)
output_fd.close() output_fd.close()
def link_emulator(self): def link_emulator(self):
@ -115,14 +107,6 @@ class ConfigInstall:
cdist.emulator.link(self.exec_path, cdist.emulator.link(self.exec_path,
self.path.bin_dir, self.path.list_types()) self.path.bin_dir, self.path.list_types())
def init_deploy(self):
"""Ensure the base directories are cleaned up"""
log.debug("Creating clean directory structure")
self.path.remove_remote_dir(cdist.path.REMOTE_BASE_DIR)
self.path.remote_mkdir(cdist.path.REMOTE_BASE_DIR)
self.link_emulator()
def run_initial_manifest(self): def run_initial_manifest(self):
"""Run the initial manifest""" """Run the initial manifest"""
log.info("Running initial manifest %s", self.path.initial_manifest) log.info("Running initial manifest %s", self.path.initial_manifest)
@ -172,6 +156,8 @@ class ConfigInstall:
def object_run(self, cdist_object, mode): def object_run(self, cdist_object, mode):
"""Run gencode or code for an object""" """Run gencode or code for an object"""
log.debug("Running %s from %s", mode, cdist_object) log.debug("Running %s from %s", mode, cdist_object)
# FIXME: replace with new object interface
file=os.path.join(self.path.object_dir(cdist_object), "require") file=os.path.join(self.path.object_dir(cdist_object), "require")
requirements = cdist.path.file_to_list(file) requirements = cdist.path.file_to_list(file)
type = self.path.get_type_from_object(cdist_object) type = self.path.get_type_from_object(cdist_object)
@ -205,7 +191,6 @@ class ConfigInstall:
outfile_fd = open(outfile, "w") outfile_fd = open(outfile, "w")
# Need to flush to ensure our write is done before stdout write # Need to flush to ensure our write is done before stdout write
# FIXME: CODE_HEADER needed in our sh -e scenario?
outfile_fd.write(CODE_HEADER) outfile_fd.write(CODE_HEADER)
outfile_fd.flush() outfile_fd.flush()
@ -238,8 +223,36 @@ class ConfigInstall:
remote_remote_code = os.path.join(remote_dir, "code-remote") remote_remote_code = os.path.join(remote_dir, "code-remote")
if os.path.isfile(local_remote_code): if os.path.isfile(local_remote_code):
self.path.transfer_file(local_remote_code, remote_remote_code) self.path.transfer_file(local_remote_code, remote_remote_code)
# FIXME: remote_prefix cdist.exec.run_or_fail([remote_remote_code], remote_prefix=True)
cdist.exec.run_or_fail([remote_remote_code], remote_prefix=self.remote_prefix)
### Cleaned / check functions: Round 1 :-) #################################
def stage_run(self):
"""The final (and real) step of deployment"""
log.info("Generating and executing code")
# Now do the final steps over the existing objects
for cdist_object in cdist.core.Object.list_objects():
log.debug("Run object: %s", cdist_object)
self.object_run(cdist_object, mode="gencode")
self.object_run(cdist_object, mode="code")
def deploy_to(self):
"""Mimic the old deploy to: Deploy to one host"""
log.info("Deploying to " + self.target_host)
self.stage_prepare()
self.stage_run()
def deploy_and_cleanup(self):
"""Do what is most often done: deploy & cleanup"""
self.deploy_to()
self.cleanup()
def init_deploy(self):
"""Ensure the base directories are cleaned up"""
log.debug("Creating clean directory structure")
self.path.remove_remote_dir(cdist.path.REMOTE_BASE_DIR)
self.path.remote_mkdir(cdist.path.REMOTE_BASE_DIR)
self.link_emulator()
def stage_prepare(self): def stage_prepare(self):
"""Do everything for a deploy, minus the actual code stage""" """Do everything for a deploy, minus the actual code stage"""
@ -249,58 +262,19 @@ class ConfigInstall:
log.info("Running object manifests and type explorers") log.info("Running object manifests and type explorers")
old_objects = [] log.debug("Searching for objects in " + cdist.core.Object.base_dir())
objects = self.path.list_objects()
# Continue process until no new objects are created anymore # Continue process until no new objects are created anymore
while old_objects != objects: new_objects_created = True
old_objects = list(objects) while new_objects_created:
for cdist_object in objects: new_objects_created = False
if cdist_object in self.objects_prepared: for cdist_object in cdist.core.Object.list_objects():
if cdist_object.prepared:
log.debug("Skipping rerun of object %s", cdist_object) log.debug("Skipping rerun of object %s", cdist_object)
continue continue
else: else:
# FIXME: run_type_explorer: log.debug("Preparing object: " + cdist_object.name)
# object can return type
# type has explorers
# path knows about where to save explorer output
# type = self.path.objects[object].type()
# self.path.types['type'].explorers()
# for explorer in explorers:
# output = cdist.exec.run_debug_or_fail_shell(explorer)
# if output:
# write_output_to(output, os.path.join(self.path.objects[object].explorer_dir(),explorer) )
#
self.run_type_explorer(cdist_object) self.run_type_explorer(cdist_object)
self.run_type_manifest(cdist_object) self.run_type_manifest(cdist_object)
self.objects_prepared.append(cdist_object) cdist_object.prepared = True
new_objects_created = True
objects = self.path.list_objects()
def stage_run(self):
"""The final (and real) step of deployment"""
log.info("Generating and executing code")
# Now do the final steps over the existing objects
for cdist_object in self.path.list_objects():
log.debug("Run object: %s", cdist_object)
self.object_run(cdist_object, mode="gencode")
self.object_run(cdist_object, mode="code")
def deploy_to(self):
"""Mimic the old deploy to: Deploy to one host"""
log.info("Deploying to " + self.target_host)
time_start = datetime.datetime.now()
self.stage_prepare()
self.stage_run()
time_end = datetime.datetime.now()
duration = time_end - time_start
log.info("Finished run of %s in %s seconds",
self.target_host,
duration.total_seconds())
def deploy_and_cleanup(self):
"""Do what is most often done: deploy & cleanup"""
self.deploy_to()
self.cleanup()

View file

@ -1,6 +1,7 @@
# -*- coding: utf-8 -*- # -*- coding: utf-8 -*-
# #
# 2010-2011 Steven Armstrong (steven-cdist at armstrong.cc) # 2011 Steven Armstrong (steven-cdist at armstrong.cc)
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
# #
# This file is part of cdist. # This file is part of cdist.
# #
@ -19,12 +20,14 @@
# #
# #
import logging
import os import os
import collections import collections
import cdist import cdist
import cdist.core.property import cdist.core.property
log = logging.getLogger(__name__)
DOT_CDIST = '.cdist' DOT_CDIST = '.cdist'
@ -65,7 +68,7 @@ class Object(object):
for object_name in cls.list_object_names(): for object_name in cls.list_object_names():
type_name = object_name.split(os.sep)[0] type_name = object_name.split(os.sep)[0]
object_id = os.sep.join(object_name.split(os.sep)[1:]) object_id = os.sep.join(object_name.split(os.sep)[1:])
yield cls(Type(type_name), object_id=object_id) yield cls(cdist.core.Type(type_name), object_id=object_id)
@classmethod @classmethod
def list_type_names(cls): def list_type_names(cls):
@ -90,6 +93,10 @@ class Object(object):
self.__parameters = None self.__parameters = None
self.__requirements = None self.__requirements = None
# Whether this object was prepared/ran
self.prepared = False
self.ran = False
def __repr__(self): def __repr__(self):
return '<Object %s>' % self.name return '<Object %s>' % self.name

View file

@ -77,6 +77,8 @@ def run(argv):
object_dir = os.path.join(global_dir, "object", type, object_dir = os.path.join(global_dir, "object", type,
object_id, cdist.path.DOT_CDIST) object_id, cdist.path.DOT_CDIST)
log.debug("Object output dir = " + object_dir)
param_out_dir = os.path.join(object_dir, "parameter") param_out_dir = os.path.join(object_dir, "parameter")
object_source_file = os.path.join(object_dir, "source") object_source_file = os.path.join(object_dir, "source")
@ -127,7 +129,7 @@ def run(argv):
param_fd.close() param_fd.close()
# Record requirements # Record requirements
if "__require" in os.environ: if "require" in os.environ:
requirements = os.environ['__require'] requirements = os.environ['__require']
log.debug(object_id + ":Writing requirements: " + requirements) log.debug(object_id + ":Writing requirements: " + requirements)
require_fd = open(os.path.join(object_dir, "require"), "a") require_fd = open(os.path.join(object_dir, "require"), "a")

View file

@ -20,6 +20,7 @@
# #
import logging import logging
import os
import subprocess import subprocess
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
@ -32,6 +33,8 @@ def shell_run_or_debug_fail(script, *args, remote_prefix=False, **kargs):
args[0][:0] = [ "/bin/sh", "-e" ] args[0][:0] = [ "/bin/sh", "-e" ]
if remote_prefix: if remote_prefix:
remote_prefix = os.environ['__remote_exec'].split()
remote_prefix.append(os.environ['target_host'])
args[0][:0] = remote_prefix args[0][:0] = remote_prefix
log.debug("Shell exec cmd: %s", args) log.debug("Shell exec cmd: %s", args)
@ -43,6 +46,7 @@ def shell_run_or_debug_fail(script, *args, remote_prefix=False, **kargs):
subprocess.check_call(*args, **kargs) subprocess.check_call(*args, **kargs)
except subprocess.CalledProcessError: except subprocess.CalledProcessError:
log.error("Code that raised the error:\n") log.error("Code that raised the error:\n")
if remote_prefix: if remote_prefix:
run_or_fail(["cat", script], remote_prefix=remote_prefix) run_or_fail(["cat", script], remote_prefix=remote_prefix)
@ -60,6 +64,8 @@ def shell_run_or_debug_fail(script, *args, remote_prefix=False, **kargs):
def run_or_fail(*args, remote_prefix=False, **kargs): def run_or_fail(*args, remote_prefix=False, **kargs):
if remote_prefix: if remote_prefix:
remote_prefix = os.environ['__remote_exec'].split()
remote_prefix.append(os.environ['target_host'])
args[0][:0] = remote_prefix args[0][:0] = remote_prefix
log.debug("Exec: " + " ".join(*args)) log.debug("Exec: " + " ".join(*args))

View file

@ -27,9 +27,6 @@ import cdist.config_install
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
Class Install(cdist.config_install.ConfigInstall):
pass
def install(args): def install(args):
"""Install remote system""" """Install remote system"""
process = {} process = {}

View file

@ -1,51 +0,0 @@
# -*- coding: utf-8 -*-
#
# 2010-2011 Steven Armstrong (steven-cdist at armstrong.cc)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
import os
import logging
log = logging.getLogger(__name__)
class Object(object):
def __init__(self, path, remote_path, object_fq):
self.path = path
self.remote_path = remote_path
self.object_fq = object_fq
self.type = self.object_fq.split(os.sep)[0]
self.object_id = self.object_fq.split(os.sep)[1:]
self.parameter_dir = os.path.join(self.path, "parameter")
self.remote_object_parameter_dir = os.path.join(self.remote_path, "parameter")
self.object_code_paths = [
os.path.join(self.path, "code-local"),
os.path.join(self.path, "code-remote")]
@property
def type_explorer_output_dir(self):
"""Returns and creates dir of the output for a type explorer"""
if not self.__type_explorer_output_dir:
dir = os.path.join(self.path, "explorer")
if not os.path.isdir(dir):
os.mkdir(dir)
self.__type_explorer_output_dir = dir
return self.__type_explorer_output_dir

View file

@ -57,8 +57,6 @@ class Path:
def __init__(self, def __init__(self,
target_host, target_host,
remote_user,
remote_prefix,
initial_manifest=False, initial_manifest=False,
base_dir=None, base_dir=None,
debug=False): debug=False):
@ -70,10 +68,8 @@ class Path:
self.base_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir, os.pardir)) self.base_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir, os.pardir))
self.temp_dir = tempfile.mkdtemp() self.temp_dir = tempfile.mkdtemp()
self.target_host = target_host
self.remote_user = remote_user self.target_host = target_host
self.remote_prefix = remote_prefix
# Input directories # Input directories
self.conf_dir = os.path.join(self.base_dir, "conf") self.conf_dir = os.path.join(self.base_dir, "conf")
@ -96,6 +92,8 @@ class Path:
self.object_base_dir = os.path.join(self.out_dir, "object") self.object_base_dir = os.path.join(self.out_dir, "object")
self.bin_dir = os.path.join(self.out_dir, "bin") self.bin_dir = os.path.join(self.out_dir, "bin")
os.environ['__cdist_out_dir'] = self.out_dir
# List of type explorers transferred # List of type explorers transferred
self.type_explorers_transferred = {} self.type_explorers_transferred = {}
@ -135,28 +133,24 @@ class Path:
# FIXME: belongs to here - clearify remote* # FIXME: belongs to here - clearify remote*
def remote_mkdir(self, directory): def remote_mkdir(self, directory):
"""Create directory on remote side""" """Create directory on remote side"""
cdist.exec.run_or_fail(["mkdir", "-p", directory], remote_prefix=self.remote_prefix) cdist.exec.run_or_fail(["mkdir", "-p", directory], remote_prefix=True)
# FIXME: belongs to here - clearify remote* # FIXME: belongs to here - clearify remote*
def remove_remote_dir(self, destination): def remove_remote_dir(self, destination):
cdist.exec.run_or_fail(["rm", "-rf", destination], remote_prefix=self.remote_prefix) cdist.exec.run_or_fail(["rm", "-rf", destination], remote_prefix=True)
# FIXME: belongs to here - clearify remote* # FIXME: belongs to here - clearify remote*
def transfer_dir(self, source, destination): def transfer_dir(self, source, destination):
"""Transfer directory and previously delete the remote destination""" """Transfer directory and previously delete the remote destination"""
self.remove_remote_dir(destination) self.remove_remote_dir(destination)
cdist.exec.run_or_fail(["scp", "-qr", source, cdist.exec.run_or_fail(os.environ['__remote_copy'].split() +
self.remote_user + "@" + ["-r", source, self.target_host + ":" + destination])
self.target_host + ":" +
destination])
# FIXME: belongs to here - clearify remote* # FIXME: belongs to here - clearify remote*
def transfer_file(self, source, destination): def transfer_file(self, source, destination):
"""Transfer file""" """Transfer file"""
cdist.exec.run_or_fail(["scp", "-q", source, cdist.exec.run_or_fail(os.environ['__remote_copy'].split() +
self.remote_user + "@" + [source, self.target_host + ":" + destination])
self.target_host + ":" +
destination])
# FIXME: Explorer or stays # FIXME: Explorer or stays
def global_explorer_output_path(self, explorer): def global_explorer_output_path(self, explorer):
@ -198,11 +192,6 @@ class Path:
return object_paths return object_paths
# FIXME: Object
def get_type_from_object(self, cdist_object):
"""Returns the first part (i.e. type) of an object"""
return cdist_object.split(os.sep)[0]
# FIXME: Object # FIXME: Object
def get_object_id_from_object(self, cdist_object): def get_object_id_from_object(self, cdist_object):
"""Returns everything but the first part (i.e. object_id) of an object""" """Returns everything but the first part (i.e. object_id) of an object"""
@ -266,19 +255,19 @@ class Path:
# Stays here - FIXME: adjust to type code, loop over types! # Stays here - FIXME: adjust to type code, loop over types!
def transfer_type_explorers(self, type): def transfer_type_explorers(self, type):
"""Transfer explorers of a type, but only once""" """Transfer explorers of a type, but only once"""
if type in self.type_explorers_transferred: if type.transferred:
log.debug("Skipping retransfer for explorers of %s", type) log.debug("Skipping retransfer for explorers of %s", type)
return return
else: else:
# Do not retransfer # Do not retransfer
self.type_explorers_transferred[type] = 1 type.transferred = True
src = self.type_dir(type, "explorer") # FIXME: Can be explorer_path or explorer_dir, I don't care.
remote_base = os.path.join(REMOTE_TYPE_DIR, type) src = type.explorer_path()
dst = self.remote_type_explorer_dir(type) dst = type.remote_explorer_path()
# Only continue, if there is at least the directory # Transfer if there is at least one explorer
if os.path.isdir(src): if len(type.explorers) > 0:
# Ensure that the path exists # Ensure that the path exists
self.remote_mkdir(remote_base) self.remote_mkdir(dst)
self.transfer_dir(src, dst) self.transfer_dir(src, dst)

View file

@ -1,51 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
#
# 2011 Nico Schottelius (nico-cdist at schottelius.org)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
import logging
import os
log = logging.getLogger(__name__)
class Type(object):
def __init__(self, path, remote_path):
self.path = path
self.remote_path = remote_path
def list_explorers(self):
"""Return list of available explorers"""
dir = os.path.join(self.path, "explorer")
if os.path.isdir(dir):
list = os.listdir(dir)
else:
list = []
log.debug("Explorers for %s in %s: %s", type, dir, list)
return list
def is_install(self):
"""Check whether a type is used for installation (if not: for configuration)"""
return os.path.isfile(os.path.join(self.path, "install"))
def remote_explorer_dir(self):
"""Return remote directory that holds the explorers of a type"""
return os.path.join(self.remote_path, "explorer")

View file

@ -0,0 +1,4 @@
Moved out of conf/type/ to think about whether this type makes sense or not.
Cdist describes the state and using an init_script may be useful, but
should only be used conditionally.

View file

@ -0,0 +1,40 @@
#!/bin/sh
#
# 2010-2011 Daniel Roth (dani-cdist@d-roth.li)
#
# This file is part of cdist.
#
# cdist is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# cdist is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with cdist. If not, see <http://www.gnu.org/licenses/>.
#
#
if [ -f "$__object/parameter/script" ]; then
script=$(cat "$__object/parameter/script")
else
script="/$__object_id"
fi
if [ -f "$__object/parameter/base_dir" ]; then
base_dir=$(cat "$__object/parameter/base_dir")
else
os="$(cat "$__global/explorer/os")"
case "$os" in
archlinux|netbsd|macosx|freebsd|openbsd) base_dir="/etc/rc.d" ;;
*) base_dir="/etc/init.d"
esac
fi
mode=$(cat "$__object/parameter/mode")
echo "${base_dir}/${script} ${mode}"

View file

@ -0,0 +1,51 @@
cdist-type__init_script(7)
==========================
Daniel Roth <dani-cdist--@--d-roth.li>
NAME
----
cdist-type__init_script - Use the init scripts
DESCRIPTION
-----------
This type can be used to control your init scripts.
REQUIRED PARAMETERS
-------------------
mode::
Specifies what shall be done with the init script (usually one of 'start'|'stop'|'restart'|'reload' or 'force-reload')
OPTIONAL PARAMETERS
-------------------
script::
If supplied, use this as the init-script.
Otherwise the object_id is used.
base_dir::
If supplied, this type uses this directory instead of '/etc/init.d'. The parameter will not need an ending slash.
EXAMPLES
--------
--------------------------------------------------------------------------------
# Reloads the configuration for lighttpd
__init_script lighttpd --mode force-reload
# Reloads the configuration for lighttpd
__init_script lighty --script lighttpd --mode force-reload
--------------------------------------------------------------------------------
SEE ALSO
--------
- cdist-type(7)
COPYING
-------
Copyright \(C) 2011 Daniel Roth. Free use of this software is
granted under the terms of the GNU General Public License version 3 (GPLv3).

View file

@ -0,0 +1,2 @@
script
base_dir

View file

@ -0,0 +1 @@
mode