The Sailors
Back to 2015, we were doing freelances at that time and trying to adapt automations for our Drupal projects. It was tough and tedious to maintain a CMS on a shared hosting using FTP (yes… FTP!). The juice is not worth the squeeze. To get rid of the pain, we would like to take back the hosting control and create a proper project setup with the following capabilities:
Our wish list
Sounds great! But how could we get it done?
Docker
We were getting tired of the old ways about server and application setup. We wanted a neat solution. Docker was the first tool came to our minds. Environment discrepancy could be solved by Infrastructure as Code (IaC) and we could keep the application runtime in version control. Moreover, human errors and the time spent on provisioning could be greatly reduced.
In addition, we could make use of any Docker images available on Docker Hub or we could build our custom image for different purposes such as development and housekeeping. For example:
Create a Drush container and clear the Drupal cache, dump database or even archive the whole website.
Create a NodeJS container with compass installed in order to compile the .sass files.
As a developer, all I need is a machine with Docker installed ( Ansible is also installed in our case). All other tools could be run as containers which are already well defined in the project repository.
Furthermore, Docker makes provisioning and deployment much more easier than before. With tools like Docker Compose and the Ansible docker_container module, just a single command, we can sit back and relax. The Drupal CMS with all the modules we needed would get up and running in a few minutes.
Our wish list
Other tools that we used
We love the container approach and Docker is a good starting point for us compared to other container runtime because of its big community where we could find a lot of useful references and support. On the other hand, there are two software which are worth to be mentioned.
Ansible
Unlike Puppet, Ansible is not just a configuration management tool. Instead, it is more like a powerful version of Apache Ant with many modules and provides a variety of functions. In contrast to Ant, the Playbook concept helps managing the configurations, deployment and orchestration in an organized manner. In our case, we defined playbooks for developers such that they could execute those pre-defined tasks during the development.
- Start and stop the application
- Watch and compile the sass files
- Clear application cache
- Export assets and DB
- Import assets and DB
- Provision a new instance which are ready to deploy on hosting providers
- Deployment to remote machines
- Execute a full backup to Amazon S3
- Retrieve the latest backup from Amazon S3
Besides, the Ansible Vault allows keeping sensitive data by means of encryption so that we could check in those files into version control without worrying.
Our wish list
Duply as Docker container for data backup
Duply is a great tool for managing backups and it works well with different cloud services like Amazon S3 and Google Drive. You could encrypt the backup to safeguard the data.
Our wish list
The remaining challenge…
So now we have fulfilled the 6 out of 7 items on the wish list which frames our project workflow: from development to production launch. But the whole workflow isn’t complete after launch. Change requests, service monitoring, performance tuning etc., are these follow up tasks are crucial to keep a project healthy for long term. We believe that without monitoring, it can’t be called a production environment. To accomplish this, Monitoring and log management is the key to success. On the order hand, log could give you insight in many aspects by using analytic tool.
Gather the logs
Here are some generic ways to read logs from containers:
- Using the docker logs command
- Output the container logs to physical files
- Enable the syslog logging driver
The above methods were useful for development, but we wanted a much better solution for long term. So we created a syslog container to collect all other containers’ log. It works well but this approach somehow intrusive to the project because we had to add extra configurations to all containers. Also, reading logs from terminal is not user friendly and that’s why we came up with the idea of Boatswain.
The Boatswain
As you might have read, Boatswain is a service which provides basic monitoring for your servers running Docker as well as log management. In short, we hope to offer you following quality features.
- Host and Docker container monitoring
- Log management
- Easy troubleshooting
- Real time notifications
- Simple installation
- Dashboard
Currently we are still in closed beta and looking for feedbacks and improvements. Please feel free to let us know if you would like to register an account. We are in the progress of moving our infra to Kubernetes on Google Cloud Platform and hopefully we could open the registration for the beta launch after the migration.
Hope to see you on our journey !
