Ansible Powered Web Cluster Part 2: Creating Websites

Introduction

In part 1, we went over the architecture and layout of the servers that host websites for me. As the servers come and go as needs require, creating new websites can be challenging. It also becomes very time consuming as you add more servers or roles to the cluster.

The entire Github repo for this cluster is now public! You can view it or grab a copy here.

Requirements

An understanding of the cluster and its parts, available in Part 1 of this series, available Here.
An understanding of PHP website hosting.

The Playbook

We will be going over the NewWordpressSite.yml playbook, available here.

Getting Started

1
2
3
4
5
6


#########################################################
# Create New Wordpress Site
#########################################################
---
- hosts: database_master[0]:wpadmin[0]
  become: true

This section is really easy, it just tells Ansible that the playbook is going to use the first database master server and the first Wordpress admin server.¹ The database syncs in real time, so as data is created on the master, the information will be on the other DB servers before the playbook finishes. The playbook will be placing system files and become makes that easier.

Create the Database

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33


    # DB Config
- hosts: database_master[0]
  vars_prompt:
    - name: sqluser
      prompt: Please provide an SQL username
      private: no

    - name: sqlpass
      prompt: Please Provide an SQL Password   

    - name: sqlname
      prompt: Please provide an SQL database name
      private: no
  tasks: 
        - name: Create a new database for website'
          community.mysql.mysql_db:
            name: "{{ sqlname }}"
            state: present 

        - name: Create database user and great access to database
          community.mysql.mysql_user:
            name: "{{ sqluser }}"
            password: "{{ sqlpass }}"
            priv: '{{ sqlname }}.*:ALL'
            host: 10.123.0.% 
            state: present       

        - name: "Add SQL info to dummy host for use later in this playbook"
          add_host:
            name:   "SQLINFOHOLDER"
            sqluser:  "{{ sqluser }}"
            sqlpass:   "{{ sqlpass }}"
            sqlname:   "{{ sqlname }}"            

All of these tasks will happen on the database master server. The playbook first asks for database information: Username, Password, and Database name. If you are using a non interactive session (AWX, Semaphore, cron) you can pass these with --extra-vars.²

Once the playbook has that information it creates the database and then grants the correct user permissions. The web server will need this information later but we run into an issue. Ansible does not persist variables between hosts in a playbook. I got around this by storing this information as an extra host, it will only exist while the playbook is running.

Web Server Configuration

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32


    # Nginx Config
- hosts: wpadmin[0]  
  vars_prompt:
    - name: domain
      prompt: What is the domain of the website
      private: no

    - name: domain2
      prompt: Is there a second domain for the website, like www.domain.com | Leave blank if no
      private: no   
  tasks: 
          - name: Create site directory if missing
            become: yes
            become_user: root  
            ansible.builtin.file:
              path: /mnt/Web/Websites/Wordpress/{{ domain }}
              state: directory

          - name: Add Nginx vhost for website
            become: yes
            become_user: root             
            template: src=templates/wpnginxvhost 
                        dest=/mnt/Web/VHosts/{{ domain }} 
                        backup=no
                        owner=www-data
                        group=www-data
                        mode=0644

          - name: "Add domain info to dummy host for use later in this playbook"
            add_host:
              name:   "WEBINFOHOLDER"
              domain:  "{{ domain }}"

For a standard “normal” website, 2 subdomains point to the website, WWW and @ or blank. The playbook asks for 2 domains to account for this.

If this is the case you put domain.com for the first question, and www.domain.com for the second question. If the website exists on a subdomain, like blog, you answer the first question with blog.domain.com and leave domain 2 blank. I always have WWW set as a CNAME of @ at the DNS level, but leave this as a backup for flexibility.

The playbook is configuring the web servers , but uses the Wordpress Admin server to do it. This ensures that the public facing web nodes ONLY SERVE WEBSITES. The domain is used as the folder name in our web directory and Nginx virtual host name. The Nginx virtual host uses a template file for domain information.

Remember that this is our shared storage so its already available for all of the web servers to use. We use the domain as that makes maintenance and updates easier(More info in a future blog post).

Just like the Database step, the playbook adds more info to the “dummy” host to use later.

Wordpress Install and Configuration

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43


    # Wordpress Config
          - name: Download and extract wordpress files
            become: yes
            become_user: root          
            ansible.builtin.unarchive:
              src: https://wordpress.org/latest.tar.gz
              dest: /mnt/Web/Websites/Wordpress/{{ domain }}/
              remote_src: yes    

          - name: Move wordpress files to public directory
            become: yes
            become_user: root   
            command: mv /mnt/Web/Websites/Wordpress/{{ domain }}/wordpress /mnt/Web/Websites/Wordpress/{{ domain }}/public_html

          - name: Add custom WP-Config
            become: yes
            become_user: root             
            template: src=templates/wp-config.php 
                        dest=/mnt/Web/Websites/Wordpress/{{ domain }}/public_html/wp-config.php 
                        backup=no                

          - name: Add HyperDB DB file to wp-content
            become: yes
            become_user: root    
            copy:
             src: files/wordpress/db.php
             dest: /mnt/Web/Websites/Wordpress/{{ domain }}/public_html/wp-content/db.php
             remote_src: no


          - name: Add HyperDB config to public directory
            become: yes
            become_user: root          
            template: src=templates/db-config.php 
                        dest=/mnt/Web/Websites/Wordpress/{{ domain }}/public_html/db-config.php
                        backup=no                

          - name: Create robots.txt file
            become: yes
            become_user: root
            template: src=templates/robots.txt
                          dest=/mnt/Web/Websites/Wordpress/{{ domain }}/public_html/robots.txt
                          backup=no

This section will use all of the information that was added to the extra host. The playbook downloads the latest version of Wordpress and places it in the websites web root using the domain variable.

A site specific WP-Config file is generated and placed in the correct location. Wordpress sites need site specific specific Auth Keys and Salts for security. The template generates those as the file is created.

The wordpress sites use Hyper-DB to fully use the highly available database set. The playbook places a db.php file , which is Hyper DB. Then a db-config.php file is placed, containing connection info for the second database. Username, Password, and Name information is the same as the master db host, so we reference that in this file.

Next a Wordpress specific robots.txt file is generated and placed at the web root.

Cleanup

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


    # Cleanup
          - name: Set permissions on web directory
            become: yes
            become_user: root            
            ansible.builtin.file:
              path: /mnt/Web/Websites/Wordpress/{{ domain }}
              state: directory
              owner: www-data
              group: www-data
              recurse: yes

  #Have to generate a random number to be used for cron to prevent them all happening at once 
          - name: Genrate Random Cron Time 1
            set_fact:
              num1: "{{30 | random(start=1) }}"

          - name: Genrate Random Cron Time 1
            set_fact:
              num2: "{{59 | random(start=31) }}"

          - name: Add cron entry for new Website
            become: true
            ansible.builtin.lineinfile:
              path=/mnt/Web/Ansible/wpcron
              state=present
              insertafter="\# * \#"
              line="{{num1}},{{num2}} * * * * www-data wget https://{{ domain }}/wp-cron.php?doing_cron &> /dev/null"

Now that all the files are in place, the playbook sets permissions on all of them to ensure they are correct. Or you could have issues.

Next a wordpress cron entry is created. Triggering it manually is much faster than having Wordpress check on every page load. To do that a random number is generated, this will be the minute the cron runs every hour.

Spreading the entries out prevents them all happening at once on the WP-Admin server. An entry is created in a master cron file on the shared storage. This file is used by the Wordpress Admin server to keep its cron list up to date.

Finishing up the Web Servers

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


- hosts: web
  become: true
  tasks: 
          - name: Create cache directory for new site
            become: yes
            become_user: root  
            ansible.builtin.file:
              path: /opt/Cache/{{ hostvars['WEBINFOHOLDER']['domain'] }}
              state: directory

          - name: Set permissions on cache directory
            become: yes
            become_user: root            
            ansible.builtin.file:
              path: /opt/Cache/{{ hostvars['WEBINFOHOLDER']['domain'] }}
              state: directory
              owner: www-data
              group: www-data
              recurse: yes
              
          - name: Reload Nginx to pick up the new website
            ansible.builtin.systemd: 
              name: nginx
              state: reloaded

Every Wordpress site on the cluster uses a FastCGI based cache to speed up page loads considerably. A directory is needed for this cache. Putting it on the local NVNME storage of the web nodes maximizes its speed. This also reduces strain on the shared storage, keeping it fast. The playbook creates that needed directory and sets correct permissions.

Nginx does not actively watch config directories for files. It checks when it is Started, Restarted, or Reloaded. The playbook created all of the needed files but the web servers do not see them yet, and they are not being used. The playbook reloads Nginx so it picks up the new website without dropping existing traffic.

Finishing up the Wordpress Admin Servers

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


- hosts: wpadmin[0]
  become: true
  tasks: 
          - name: Update cron file for new website
            become: yes
            become_user: root            
            copy: 
              src: /mnt/Web/Ansible/wpcron
              dest: /etc/cron.d/wpcron
              mode: '0644'
              backup: no
              
          - name: Reload Nginx to pick up the new website
            ansible.builtin.systemd: 
              name: nginx
              state: reloaded

Caching is disabled for any page the Wordpress Admin servers would serve, so creating a cache directory is not needed. The cron file the playbook added to is pushed into the Wordpress Admin server’s cron list. Like the Web servers, Nginx has to be reloaded to see the new website.

Next Steps

Once the playbook finishes, open the primary domain in your browser of choice. The domain was already pointed at the cluster’s floating IP, so the Wordpress install page loads. Follow the prompts and log into /wp-admin for your new site.

Final Notes

When I just used a single Web node and single Database node, creating a new Wordpress site took 30 minutes to 1 hour. Remembering comments, switching back and forth, copy/pasting, it ate up my time.

On average this playbook takes 3 minutes to run and does a better job than I did manually, on a much more complex cluster. This single playbook alone has saved me over 80 hours, in 3 years.

Computers start counting at 0 ↩︎
More information here. ↩︎