Timeline

20 Nov 2020 - Pentti

In this task I did three separate things: Firstly I booted Windows 10 OS VM using Virtualbox and installed 10 arbitary programs using Salt. Second task was to install Visual Studio Code to a Linux system using a Microsoft package repository. In the third task I applied ufw firewall rules and searched from the timeline which files were modified and by using these files, I could copy and apply the same rules to Salt-minions.

Set up Windows as a slave

I focused only on installation and not preconfigurations of the programs. I mainly followed this artice when I did this task.

After succesfully booting the Windows VM, the first task was to configure it to act as a minion. I downloaded the latest stable build from Salt and ran the installer. At first the installer asked for master ip address and id for the minion and asked if I wanted to use custom configuration. I typed in my master ip, id as windows_minion and selected to use default configuration and launched the program.

Now I was able to see the minion in the salt-key list as a unaccepted key:

$ sudo salt-key
Accepted Keys:
Denied Keys:
Unaccepted Keys:
windows_minion

and accept the key using

$ sudo salt-key -A

test that the connection works

$ sudo salt '*' test.ping
windows_minion:
    True

Windows packages are not distributed with Salt by default, so I had to run command to initialize the repository on master

$ sudo salt-run winrepo.update_git_repos

and then synchronize the package repository with my minion:

$ sudo salt-G 'os:windows' pkg:refresh_db
windows_minion:
    ----------
    failed:
        0
    success:
        300
    total:
        300

Now I was ready to create a new state for my windows packages

$ sudo mkdir /srv/salt/windows/

and I started by creating a state for firefox

$ sudo mkdir /srv/salt/windows/firefox/
$ vim init.sls /srv/salt/windows/firefox/init.sls
firefox:
  pkg.installed

However, when testing the state, I got error

windows_minion:
    ----------
    firefox:
        Unable to locate package firefox

I did little bit of searching and found out that the problem might be in different Salt versions between master and minion. So I checked the versions and noticed that my salt master was older version than the windows minion. I updated packages and checked that the version match.

$ sudo apt update
$ sudo apt upgrade -y

master

$ salt --version
salt 3002.2

minion

$ sudo salt 'win*' pkg.list_pkgs salt
...
    salt-minion-py3:
        3002.2

I tried to apply the state again, but still got the same error. I listed all the available packages from the minion

$ sudo salt 'win*' pkg.get_repo_data | less

and found out that actually there are no package for name firefox but for example 64bit system I should use name firefox_x64

So, I modified the firefox state file accordingly

firefox:
  pkg.installed
    - name: firefox_x64

and applied the state

$ sudo salt 'win*' state.apply windows.firefox
windows_minion:
----------
          ID: firefox
    Function: pkg.installed
        Name: firefox_x64
      Result: True
     Comment: The following packages were installed/updated: firefox_x64
...
              firefox_x64:
                  ----------
                  new:
                      83.0
                  old:

Summary for windows_minion
------------
Succeeded: 1 (changed=1)
Failed:    0
------------
Total states run:     1

Next up I installed the VLC mediaplayer the same way.

I wanted to be sure that the package exists

$ sudo salt 'win*' pkg.list_available vlc
...
    - 3.0.10

after that I could create a state

/salt/windows/vlc/init.sls

vlc:
  pkg.installed

and apply the state

windows_minion:
----------
          ID: vlc
    Function: pkg.installed
      Result: True
     Comment: The following packages were installed/updated: vlc
...
              vlc:
                  ----------
                  new:
                      3.0.10
...
------------
Succeeded: 1 (changed=1)
Failed:    0
------------

From now on I just kept installing more packages following the same principals. You can see the list of installed packages from the highstate below.

Every package except winscp installed flawlessly. For some reason winscp installation hung forever at a running state so I decided to debug it locally on minion’s end. When installing locally, the installer asks Install to all users or just me question and I think this is where the master installation also hangs. I wasn’t be able to find workaround for this but I think Salt should handle this automatically.

< EDIT 23.11.2020

I was able to get WinSCP installed by adding /ALLUSERS install flag to the file win/repo-ng/salt-winrepo-ng/winscp.sls. I got the idea from here to use /? to see what install flag options were available. I tested the command locally using the file path I saw while debugging the winscp installation

PS> c:\salt\var\cache\salt\minion\extrn_files\base\downloads.sourceforge.net\projects\winscp\WinSCP\5.17\WinSCP-5.17-Setup.exe /?

and I could see that there is /ALLUSERS install flag available. With using it I was able to install it locally. Now I had to find a way to pass those install flags from my master to minion. I included the flag into /srv/salt/win/repo-ng/salt-winrepo-ng/winscp.sls file

...
install_flags: '/ALLUSERS /SP- /verysilent /norestart'
...

synchronized repositories

$ sudo salt -G 'os:windows' pkg.refresh_db

and tested result

$ sudo salt 'win*' state.apply windows.winscp
windows_slave:
----------
          ID: winscp
    Function: pkg.installed
      Result: True
     Comment: The following packages were installed/updated: winscp
...
              winscp:
                  ----------
                  new:
                      5.17
                  old:

Summary for windows_slave
------------
Succeeded: 1 (changed=1)
Failed:    0
------------
Total states run:     1
Total run time:   4.538 s

EDIT >

Now that every package is tested separately, I tested them together in a hightstate

base:
  'windows_minion':
    - windows.vlc
    - windows.steam
    - windows.vscode
    - windows.nodejs
    - windows.git
    - windows.firefox
    - windows.putty
    - windows.mongodb
    - windows.openvpn
    - windows.winscp

$ sudo salt 'win*' state.highstate
Summary for windows_minion
------------
Succeeded: 9 (changed=10)
Failed:    1

The failure was from mongodb, but interestingly state change still implies mongodb install status: success


          ID: mongodb
    Function: pkg.installed
      Result: False
     Comment: The following packages failed to install/update: mongodb
     Started: 22:46:45.825811
    Duration: 134899.29 ms
     Changes:
              ----------
              MongoDB 4.4.1 2008R2Plus SSL (64 bit):
                  ----------
                  new:
                      4.4.1
                  old:
              mongodb:
                  ----------
                  install status:
                      success

To check whether the packages were actually installed or not

$ sudo salt 'win*' pkg.list_pkgs
...
    MongoDB 4.4.1 2008R2Plus SSL (64 bit):
        4.4.1
    MongoDB Compass:
        1.23.0
...

Local install on the minion gave me the same result without any errors:

PS> salt-call pkg.install mongodb --local -l debug
local:
    ----------
    MongoDB 4.4.1 2008R2Plus SSL (64 bit):
        ----------
        new:
            4.4.1
        old:
     mongodb:
        ----------
        install status:
            success
PS>

Installing Visual Studio Code with Salt

In this task I wanted to use pkgrepo.managed state to add a repository to my system and then use it as source for Visual Studio Code.

Firstly I created state for the repository using this information.

vscoderepo:
  pkgrepo.managed:
    - name: deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main
    - key_url: https://packages.microsoft.com/keys/microsoft.asc

test the state:

pena@webserver:/srv/salt/linux/vscode$ sudo salt-call --local state.apply linux.vscode
local:
----------
          ID: vscode
    Function: pkgrepo.managed
        Name: deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main
      Result: True
...
              repo:
                  deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main

Summary for local
------------
Succeeded: 1 (changed=1)
Failed:    0
------------

Then I could just add pkg.installed state for the vscode and also add require in property for the repo to ensure when running the vscode state also the vscode repo state will be ran.

vscoderepo:
  pkgrepo.managed:
    - name: deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main
    - key_url: https://packages.microsoft.com/keys/microsoft.asc
    - require_in:
      - pkg: vscode

vscode:
  pkg.installed:
    - refresh: True
    - name: code

To test:

pena@webserver:/srv/salt/linux/vscode$ sudo salt-call --local state.apply linux.vscode
...
          ID: vscode
    Function: pkg.installed
        Name: code
      Result: True
     Comment: The following packages were installed/updated: code
...
Summary for local
------------
Succeeded: 2 (changed=1)
Failed:    0
------------

Setting up UFW with Salt

I started by allowing couple rules for ufw

$ sudo ufw allow ssh
$ sudo ufw allow salt

Then I checked which files were actually modified using find.

With find it is possible to list contents of any folder like I did for the /etc folder. With option -printf I could modify the standard input format to a form %T+ %p\n where T stands for time, p stands for path for a file and \n for new line. Lastly I wanted to sort the result so I was able to see which files were modified lastly.

Here I could see that these two files were modified at the same time:

$ sudo find /etc -printf '%T+ %p\n' | sort | tail
...
2020-11-21+06:25:42.0927447280 ./user.rules
2020-11-21+06:25:42.1007445480 ./user6.rules

I created a state for ufw and first I wanted to ensure that ufw is installed

ufwinstalled:
  pkg.installed:
    - name: ufw
sudo salt-call state.apply linux.ufw --local -l debug
local:
----------
          ID: ufwinstalled
    Function: pkg.installed
        Name: ufw
      Result: True
     Comment: All specified packages are already installed

Now that I could tell that ufw is installed, I could use file.managed state and move the previously modified user*.rules files to the minion. Before handling the files I wanted to add service.running state to watch everything inside /etc/ufw/ and restart the service if anything at all gets modified in the folder. At the end of the file I’m forcing the ufw to be enabled.

...
ufwrunning:
  service.running:
    - name: ufw
    - watch:
      - file: /etc/ufw/*

/etc/ufw/user.rules:
  file.managed:
    - source: salt://linux/ufw/user.rules

/etc/ufw/user6.rules:
  file.managed:
    - source: salt://linux/ufw/user6.rules

ufwenable:
  cmd.run:
    - name: "ufw --force enable"

Testing that everything works:

$ sudo salt-call state.apply linux.ufw
$ sudo ufw status verbose
Status: active

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere
Salt                       ALLOW       Anywhere
22/tcp (v6)                ALLOW       Anywhere (v6)
Salt (v6)                  ALLOW       Anywhere (v6)

Test with a real minion

I booted a vagrant instance the same way I have done in the previous assigment.

This is a fresh install so there are no rules applied and status is inactived by default:

$ sudo ufw status
Status: inactive

I applied the state, but got error:

$ sudo salt 'slave*' state.highstate
slavea57f2184:
    Minion did not return. [Not connected]
ERROR: Minions returned with non-zero exit code

I was still able to establish ssh connection to the minion and check the ufw status:

$ sudo ufw status
Status: active

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere
Salt                       ALLOW       Anywhere
80/tcp                     ALLOW       Anywhere
22/tcp (v6)                ALLOW       Anywhere (v6)
Salt (v6)                  ALLOW       Anywhere (v6)

Everything seemed to be like it should, so I ran test state on master to test the connection again

$ sudo salt 'slave*' cmd.run 'pwd'
slavea57f2184:
    /root

and it worked! Enabling the ufw must’ve been resulted a small connection failure and for that reason the minion couldn’t return.