Some time ago I moved my blog back to be a static site after a couple of years of using WordPress. In doing so, I needed a more convenient and robust way to publish the site than I had done with my previous static site.
Why I use UnoEuro
I use UnoEuro to host my site and have done so for many years. I do not have any special attachment to UnoEuro I use them because they are cheap and work well enough for a blog like this. The last time I used a static site I was publishing via FTP. My ISP, however, does not seem to like FTP traffic and is throttling it making transfer speed painfully slow. The generated site is, at the time of this post, about 6MB in size and would take anything from 5 minutes to an hour or more to upload. Not practical. Luckily, UnoEuro makes it possible to connect via SSH which my ISP does not throttle.
SSH
Since SSH was an option I wanted to try to publish via SFTP. So I generated an SSH key and added it to the list of SSH keys under Website/_SSH access in the UnoEuro control panel. I then connected using the key and was somewhat puzzled by the fact that I was prompted to enter a password. At first, I thought that the connection did not use the SSH key so I tried to connect without and could not connect at all meaning it was using the SSH key. So why the extra password prompt? I am not sure.
However, I am guessing it is because it is a shared hosting environment and the SSH key gets me into the server and the password gets me to my account and the specific website folder. Whatever the reason this extra password entry was a problem. I was planning to use FreeFileSync to publish the site. Mostly because I know and use FreeFileSync for other things and it would get me going quick. However, FreeFileSync does not work with this extra password prompt. At least I could not figure out how to make it work.
So, I connected via SSH again to see what I had to work with. I was pleasantly surprised to discover that Git was available. I don’t know why but, I did not expect to find git. Maybe because it is a shared host I do not know. But discovering git change the game completely. Git makes it possible to create a “proper” deployment process and automate it.
I have setup git deploy via git hooks a couple of times before. However, it has been a while and I can not remember the details and had to look in up. This time I used a very simple how-to from medium.com called Deploy a website to a remote server with Git push
Automation
The difficult part was automating it. I use fabric to automate a lot of tasks such as new post creation, build, test, deployment, image optimization, etc. So, I of cause wanted to automate this git deployment I just set up. Now that very annoying extra password prompt was again a problem so it took a little longer to write the automation than I had expected.
Because the ssh server prompted for a password to be entered manually I needed something that could execute the git command, monitor the output, and react when the command reached the password prompt.
It took some googling and after trying a couple approaches. I found a python module called pexpect. The Pexpect documentation describes the module as such.
Pexpect is a pure Python module for spawning child applications; controlling them; and responding to expected patterns in their output.
Pexpect can be used for automating interactive applications such as ssh, ftp, passwd, telnet, etc.
Since the module description described exactly what I want to achieve I tried it out. Below is the function that publish this blog.
def publish(context):
result = check_output('pelican -o %s -v -s publishconf.py' % DEPLOY_PATH, shell=True).decode()
print(result)
cmd = f'git add -A && git commit -m "Release" && git push -f origin +master:refs/heads/master'
os.chdir(DEPLOY_PATH)
child = pexpect.spawn('git add -A', cwd=DEPLOY_PATH)
print(str(child.read().decode("utf-8")), end="")
child.close()
child = pexpect.spawn('git commit -m "Release"', cwd=DEPLOY_PATH)
print(str(child.read().decode("utf-8")), end="")
child.close()
child = pexpect.spawn('git push -f origin +master:refs/heads/master', cwd=DEPLOY_PATH)
child.expect('password:')
child.sendline(PASSWORD)
print(str(child.read().decode("utf-8")), end="")
It is the last part that is interesting. This is where pexpect executes the git push command that when successful til trigger the server to publish the new content.
child = pexpect.spawn('git push -f origin +master:refs/heads/master',
cwd=DEPLOY_PATH)
I then tell pexpect to watch for a line with the content “password:”
child.expect('password:')
Then I tell pexpect to type the password when it sees the line.
child.sendline(PASSWORD)
When I first found the right tool for the job it did not take me long to get it working. Now deploying my static site is as simple as executing a single command and the site is deployed in seconds. Awesome!