File System Tasks

Filesystem Related Tasks

Tasks to manipulate the filesystem: create/delete files and directories, manage permissions, etc...

There are individual tasks that are named using UNIX conventions. There are also higher-level tasks: builder() and fs(), which are controlled by the action argument.

fs.builder -- The file-system power-house¶

fs.builder() is a power-house for setting up your file-system, as it takes a list of core.Items to streamline the creation large swaths of your file-system.

For example, you could:

- Write an apt.sources.list file, and trigger an apt update if written.
- Create a var directory.
- Template a systemd file and trigger a daemon-reload if written.
- Create a /usr/local/src directory.
- Template a docker-compose file into it.
- Trigger a `docker compose` run if updated.

Here is an example:

def semaphore_compose():
    with fs.cd(path="/usr/local/src/ansible-semaphore"):
        core.run(cmd="docker compose up -d")

fs.builder([
    core.Item(path="/usr/local/src/ansible-semaphore", state="directory"),
    core.item(path="/usr/local/src/ansible-semaphore/docker-compose.yml",
              notify=semaphore_compose),
    ])

fs.fs -- the builder() back-end¶

fs.fs() is just like fs.builder(), except it does not take a list of Items, it takes the exploded arguments. Mostly, it exists as a back-end for fs.builder(), though there may be situations where it is useful, so it is left as exposed.

Tasks¶

`builder(items, defaults=Item())` ¶

All-in-one filesystem builder. This is the swiss-army knife of the fs module, given a list of all the filesystem changes that need to be accomplished, it will carry them out.

This is targeted for use with Items() loops, for easily populating or modifying many filesystem objects in compact declarations.

Parameters:

Name	Type	Description	Default
`items`	`List[Item]`	A list of the file-system actions to take, `Item()` objects with attributes as below.	required
`defaults`	`Item`	An `Item()` specifying defaults that can be overridden by the specific items, for example if most owners should be "root", but a few need to be "nobody", specify `owner="root"` in the defaults. (Optional)	`Item()`

Item Attributes

Items can have these attributes (from fs.fs):

path: Name of destination filesystem object. (templateable).
action: Type of `path` to build, can be: (optional, templatable, default="template")
    - "directory"
    - "template"
    - "exists"
    - "copy"
    - "absent"
    - "link"
    - "symlink"
src: Name of template to use as source (optional, templateable).
    Defaults to the basename of `path` + ".j2".
mode: Permissions of file (optional, templatable string or int).
owner: Ownership to set on `path`. (optional, templatable).
group: Group to set on `path`. (optional, templatable).
notify:  Handler to notify of changes.
    (optional, Callable)

Examples:

fs.builder(items=[
    #  creates "/tmp/foo" from "foo.j2" as a template
    Item(path="/tmp/foo",

    #  makes the "/tmp/bar" directory
    Item(path="/tmp/bar", action="directory"),
    ]))


fs.builder(defaults=Item(owner="root", group="root", mode="a=rX,u+w",
           items=[
               Item(path="/tmp/{{ modname }}", action="directory"),
               Item(path="/tmp/{{ modname }}/__init__.py"),
               Item(path="/tmp/should-not-exist", action="absent"),
               Item(path="/tmp/{{ modname }}/nobody-file", owner="nobody", group="nobody"),
               Item(path="/tmp/{{ modname }}/site.conf", notify=restart_apache),
           ])

fs.builder(defaults=Item(owner="headscale", group="headscale", mode="a=-,ug+rwX"),
           items=[
               Item(path="/etc/headscale", state="directory"),
               Item(path="/etc/headscale/config.yaml", notify=restart_headscale),
               Item(path="/etc/headscale/acls.yaml", notify=restart_headscale),
               Item(path="/etc/headscale/derp.yaml", notify=restart_headscale),
               ])

`cd(path)` ¶

Change working directory to path.

Sets "extra.old_dir" on the return object to the directory before the cd is done. Can also be used as a context manager and when the context is exited you are returned to the previous directory.

Parameters:

Name	Type	Description	Default
`path`	`TemplateStr`	Directory to change into (templateable).	required

Examples:

fs.cd(path="/tmp")

#  As context manager:
with fs.cd(path="/tmp"):
    #  creates /tmp/tempfile
    fs.mkfile("tempfile")
#  now are back in previous directory

`chmod(path, mode=None, is_directory=None)` ¶

Change permissions of path.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to change (templateable).	required
`mode`	`Optional[Union[str, int]]`	Permissions of `path` (optional, templatable string or int).	`None`
`is_directory`	`Optional[bool]`	Treat `path` as a directory, impacts "X". If not specified `path` is examined to determine if it is a directory. (optional, bool).	`None`

Examples:

fs.chmod(path="/tmp/foo", mode="a=rX,u+w")
fs.chmod(path="/tmp/foo", mode=0o755)

`chown(path, user=None, group=None)` ¶

Change ownership/group of path.

Parameters:

Name	Type	Description	Default
`path`	`str`	Path to change (templateable).	required
`user`	`Optional[TemplateStr]`	User to set on `path`. (optional, templatable).	`None`
`group`	`Optional[TemplateStr]`	Group to set on `path`. (optional, templatable).	`None`

Examples:

fs.chown(path="/tmp", owner="root")
fs.chown(path="/tmp", group="wheel")
fs.chown(path="/tmp", owner="nobody", group="nobody")

`cp(path, src=None, mode=None, encrypt_password=None, decrypt_password=None, template=True, template_filenames=True, recursive=True)` ¶

Copy the src file(s) to path.

If src is a relative path, the src is searched for in the playbook (using UP_FILES_PATH). A fully qualified src will NOT be relative to the playbook.

Relative src is typically used for templates or other files/data that are provided as a part of the playbook to write to the destination system (for example: configurations, scaffolding, etc).

Optionally templating the contents in src. It can also template file names when doing recursive operations.

Parameters:

Name	Type	Description	Default
`path`	`TemplateStr`	Name of destination file. (templateable).	required
`src`	`Optional[TemplateStr]`	Name of template to use as source (optional, templateable). Defaults to the basename of `path` + ".j2".	`None`
`mode`	`Optional[Union[TemplateStr, int]]`	Permissions of directory (optional, templatable string or int). Sets mode on creation.	`None`
`template`	`bool`	If True, apply Jinja2 templating to the contents of `src`, otherwise copy verbatim. (default: True)	`True`
`template_filenames`	`bool`	If True, filenames found during recursive copy are jinja2 template expanded. (default: True)	`True`
`recursive`	`bool`	If True and `src` is a directory, recursively copy it and everything below it to the `path`. If `path` ends in a "/", the last component of `src` is created under `path`, otherwise the contents of `src` are written into `path`. (default: True)	`True`

Examples:

fs.cp(path="/tmp/foo")
fs.cp(src="bar-{{ fqdn }}.j2", path="/tmp/bar", template=False)

`exists(path, ignore_failure=True)` ¶

Does path exist?

Parameters:

Name	Type	Description	Default
`path`	`TemplateStr`	File location to see if it exists. (templateable).	required
`ignore_failure`	`bool`	If True, do not treat file absence as a fatal failure. (optional, bool, default=True)	`True`

Examples:

fs.exists(path="/tmp/foo")
if fs.exists(path="/tmp/foo"):
    #  code for when file exists

`fs(path, action='template', src=None, mode=None, owner=None, group=None, notify=None)` ¶

This is a unified entry point for the fs module that can take paths and actions and call the other functions in the module based on the action.

This is targeted for use with Items() loops, for easily populating or modifying many filesystem objects in compact declarations.

NOTE: Arguments are inserted into templating context so that arguments can reference other arguments as templates.

Parameters:

Name	Type	Description	Default
`path`	`TemplateStr`	Name of destination filesystem object. (templateable).	required
`action`	`Union[TemplateStr, str]`	Type of `path` to build, can be: (optional, templatable, default="template") - "directory" - "template" - "exists" - "copy" - "absent" - "link" - "symlink"	`'template'`
`src`	`Optional[TemplateStr]`	Name of template to use as source (optional, templateable). Defaults to the basename of `path` + ".j2".	`None`
`mode`	`Optional[Union[TemplateStr, int]]`	Permissions of file (optional, templatable string or int).	`None`
`owner`	`Optional[TemplateStr]`	Ownership to set on `path`. (optional, templatable).	`None`
`group`	`Optional[TemplateStr]`	Group to set on `path`. (optional, templatable).	`None`
`notify`	`Optional[Callable]`	Handler to notify of changes. (optional, Callable)	`None`

Examples:

fs.fs(path="/tmp/foo")
fs.fs(path="/tmp/bar", action="directory")
for _ in [
        core.Item(path="/tmp/{{ modname }}", action="directory"),
        core.Item(path="/tmp/{{ modname }}/__init__.py"),
        ]:
    fs.fs()

`globitems(path, recursive=True, use_cwd=False, **kwargs)` ¶

Walk the filesystem looking for matching filenames and produce Item()s for each.

This is a uPlaybook wrapper around the Python glob.iglob() function. If recursive is true, "**" in the path will match (possibly 0) intermediate directories. Other arguments given will be passed directly to the returned Item() arguments.

Parameters:

Name	Type	Description	Default
`path`	`Union[TemplateStr, str]`	Glob to expand. (templateable)	required
`recursive`	`bool`	If True (default), "" in the path will expand to 0 or more intermediate directories. For example "/*.pb" will look for all playbook files in and below the current directory.	`True`
`use_cwd`	`bool`	If True, the process current working directory is used, if False (default) the playbook directory is used. False causes files to be found next to the playbook, for example template files.	`False`

Examples:

fs.builder(
        defaults=Item(mode="a=rX"),
        items=[
            Item(path="config_dir", state="directory")
            ] +
            list(fs.globitems(path="**.j2", src="{{path}}", mode="a=r")) +
            list(fs.globitems(path="bin/*", src="{{path}}"))
        )

`ln(path, src, symbolic=False)` ¶

Create a link from src to path.

Parameters:

Name	Type	Description	Default
`path`	`TemplateStr`	Name of destination of link. (templateable).	required
`src`	`TemplateStr`	Name of location of source to create link from. (templateable).	required
`symbolic`	`bool`	If True, makes a symbolic link. (bool, default: False)	`False`

Examples:

fs.ln(path="/tmp/foo", src="/tmp/bar")
fs.ln(path="/tmp/foo", src="/tmp/bar", symbolic=True)

`mkdir(path, mode=None, parents=True)` ¶

Create a directory. Defaults to creating necessary parent directories.

Parameters:

Name	Type	Description	Default
`path`	`TemplateStr`	Name of file to create (templateable).	required
`mode`	`Optional[Union[TemplateStr, int]]`	Permissions of directory (optional, templatable string or int). Sets mode on creation.	`None`
`parents`	`Optional[bool]`	Make parent directories if needed. (optional, default=True)	`True`

Examples:

fs.mkdir(path="/tmp/foo")
fs.mkdir(path="/tmp/bar", mode="a=rX,u+w")
fs.mkdir(path="/tmp/baz/qux", mode=0o755, parents=True)

`mkfile(path, mode=None, contents=None)` ¶

Create an empty file if it does not already exist.

Parameters:

Name	Type	Description	Default
`path`	`TemplateStr`	Name of file to create (templateable).	required
`mode`	`Optional[Union[TemplateStr, int]]`	Permissions of file (optional, templatable string or int). Atomically sets mode on creation.	`None`
`contents`	`Optional[TemplateStr]`	If specified, ensure the contents of the file are `contents`.	`None`

Examples:

fs.mkfile(path="/tmp/foo")
fs.mkfile(path="/tmp/bar", mode="a=rX,u+w")
fs.mkfile(path="/tmp/baz", mode=0o755)

`mv(path, src)` ¶

Rename src to path. If src does not exist but path does, it is considered successful without change. If neither exists, it is failed.

Parameters:

Name	Type	Description	Default
`path`	`TemplateStr`	New name. (templateable).	required
`src`	`TemplateStr`	Old name. (templateable).	required

Examples:

fs.mv(path="/tmp/foo", src="/tmp/bar")

`newer_than(src, path)` ¶

Is src newer than path? If path does not exist, it is treated as older than src, Like [fs.cp()](tasks/fs.md#uplaybook.fs.cp), src is found using the UP_FILES_PATH. Can be used in an if, and can trigger a handler.

Parameters:

Name	Type	Description	Default
`src`	`TemplateStr`	File to check if is newer than `path`. (templateable).	required
`path`	`TemplateStr`	Destination location to check age against. (templateable).	required

Examples:

src_file = "foo.c"
if fs.newer_than(src=src_file, path="{{src_file.rsplit('.', 1)[0]}}.o"):
    core.run(cmd="cc -c {{src_file}}")

def compile():
    core.run(cmd="cc -c {{src_file}}")
fs.newer_than(src=src_file, path="{{src_file.rsplit('.', 1)[0]}}.o").notify(compile)

`rm(path, recursive=False)` ¶

Remove a file or recursively remove a directory.

Parameters:

Name	Type	Description	Default
`path`	`TemplateStr`	Name of file/directory to remove. (templateable).	required
`recursive`	`bool`	If True, recursively remove directory and all contents of `path`. Otherwise only remove if `path` is a file. (default: False)	`False`

Examples:

fs.rm(path="/tmp/foo")
fs.rm(path="/tmp/foo-dir", recursive=True)

`stat(path, follow_symlinks=True)` ¶

Get information about path.

Parameters:

Name	Type	Description	Default
`path`	`TemplateStr`	Path to stat. (templateable).	required
`follow_symlinks`	`bool`	If True (default), the result will be on the destination of a symlink, if False the result will be about the symlink itself. (bool, default: True)	`True`

Extra Data:

perms: The permissions of `path` (st_mode & 0o777).
st_mode: Full mode of `path` (permissions, object type).  You probably want the
    "perms" field if you just want the permissions of `path`.
st_ino: Inode number.
st_dev: ID of the device containing `path`.
st_nlink: Number of hard links.
st_uid: User ID of owner.
st_gid: Group ID of owner.
st_size: Total size in bytes.
st_atime: The time of the last access of file data.
st_mtime: The time of last modification of file data.
st_ctime: The time of the last change of status/inode.
S_ISBLK: Is `path` a block special device file?
S_ISCHR: Is `path` a character special device file?
S_ISDIR: Is `path` a directory?
S_ISDOOR: Is `path` a door?
S_ISFIFO: Is `path` a named pipe?
S_ISLNK: Is `path` a symbolic link?
S_ISPORT: Is `path` an event port?
S_ISREG: Is `path` a regular file?
S_ISSOCK: Is `path` a socket?
S_ISWHT: Is `path` a whiteout?

Examples:

stat = fs.stat(path="/tmp/foo")
print(f"UID: {{stat.extra.st_uid}}")
fs.stat(path="/tmp/foo", follow_symlinks=False)

File System Tasks

fs.builder -- The file-system power-house¶

fs.fs -- the builder() back-end¶

Tasks¶

builder(items, defaults=Item()) ¶

cd(path) ¶

chmod(path, mode=None, is_directory=None) ¶

chown(path, user=None, group=None) ¶

cp(path, src=None, mode=None, encrypt_password=None, decrypt_password=None, template=True, template_filenames=True, recursive=True) ¶

exists(path, ignore_failure=True) ¶

fs(path, action='template', src=None, mode=None, owner=None, group=None, notify=None) ¶

globitems(path, recursive=True, use_cwd=False, **kwargs) ¶

ln(path, src, symbolic=False) ¶

mkdir(path, mode=None, parents=True) ¶

mkfile(path, mode=None, contents=None) ¶

mv(path, src) ¶

newer_than(src, path) ¶

rm(path, recursive=False) ¶

stat(path, follow_symlinks=True) ¶

`builder(items, defaults=Item())` ¶

`cd(path)` ¶

`chmod(path, mode=None, is_directory=None)` ¶

`chown(path, user=None, group=None)` ¶

`cp(path, src=None, mode=None, encrypt_password=None, decrypt_password=None, template=True, template_filenames=True, recursive=True)` ¶

`exists(path, ignore_failure=True)` ¶

`fs(path, action='template', src=None, mode=None, owner=None, group=None, notify=None)` ¶

`globitems(path, recursive=True, use_cwd=False, **kwargs)` ¶

`ln(path, src, symbolic=False)` ¶

`mkdir(path, mode=None, parents=True)` ¶

`mkfile(path, mode=None, contents=None)` ¶

`mv(path, src)` ¶

`newer_than(src, path)` ¶

`rm(path, recursive=False)` ¶

`stat(path, follow_symlinks=True)` ¶