Open Source Databases for the Enterprise
31 October 2016
Today many organizations need to deploy applications that use a variety of open source data sources like PostgressSQL, MongoDB and Reddis, but frequently don’t have the skills to manage them to enterprise standards. However, to keep up with the demands of business to rapidly develop new applications these data sources get deployed anyway and are put on machines under peoples’ desks or in closets. This can easily lead to a nightmare scenario because the applications they support become critical to the business and the data stores frequently have sensitive data, but the systems do not get backed up, don’t have support if they fail and don’t get security patches applied. Fortunately, the IBM Compose system can solve this problem for you by providing several open source databases for you that we manage, backup, monitor 24X7 and make it easy for you to deploy and use.
For some perspectives on different databases based for different trade-offs between highly consistent and available systems vs. the need for fast development see my Different DB Platforms for Different Jobs article. Lines of business are hiring people with great skills who know these open source databases and let them use them because they can get their applications faster that way. No one wants an insecure data store without backups, but good database practices can become casualties when speed is of the essence. With Compose Enterprise you can be the hero because your central IT team can provide these data stores quickly using Compose Enterprise. These data stores are in high availability clusters, configured for good performance, backed up and monitored 24x7 and fixed if there are problems. So if your organization can provide these data stores quickly that meet your enterprise standards, then everyone wins.
IBM Compose allows your organization to deploy several different open source databases very easily. These databases are managed by IBM which means that they are monitored, and if needed, fixed or restored by IBM if there is a failure. They are backed up daily and can be restored simply by requesting a restore for any reason. When deploying, you specify an increment of storage and a database is deployed with RAM and processor resources based on predefined ratios to the storage. This means that you do not need an expert to configure the database system. It is done for you based on configurations that have worked well for others. IBM also updates the system with patches as they become available and are tested, which means that you will not fall behind on security and other fixes. You can also elastically scale your data stores up or down through the web interface. When you go to the compose.io website, and deploy individual data stores you are using what we call Compose Public. Each will be hosted on a virtual machine in the cloud.
Another option that is good for many types of organizations is the ability to reserve a virtual or physical server and deploy all of your data stores on one cluster of machines. This is called Compose Enterprise. One technical benefit of Compose Enterprise is that it encrypts your data on disk (data at rest). This is quite important for many organizations. You can choose from three different cluster sizes when you purchase a Compose Enterprise system. From an organizational perspective it allows you to have one bill for all of your services, but deploy many different databases on one cluster. This also has a performance advantage when data stores are used together such as MongoDB and Elasticsearch because now these databases are communicating on individual machines instead of across a network as will be frequently be done when deploying through Compose Public.
With a Compose Enterprise cluster you start with a cluster of servers and deploy data stores using the same interface as Compose Public. You can add new data stores and scale up existing data stores as you wish until 80% of the capacity of the cluster is consumed. At that point you can only add capacity to existing data stores. When running a Compose Enterprise on a cluster of physical servers (bare metal) you actually get a set of physical machines in the cloud that are shared with no one else. This can prevent problems with “noisy neighbors” and shared resources and could have some security benefits.
Compose Enterprise can be deployed on Amazon AWS or IBM Softlayer clouds. With AWS you get virtual private nodes and with IBM Softlayer you can get the bare metal nodes. The databases that you can get on all Compose environments at the date this article are shown in Figure 1.
Figure 1. Available Compose Data Sources as of Oct. 2016
More are being added all of the time so please check the compose.io web site if you are looking for something different.
Even though the major configuration and management tasks are taken care of for you, it is easy to deploy your Compose data stores. I will walk you through an example of deploying a PostgreSQL database and then connecting to it remotely. I’m doing this on Compose Public, but the steps and pages are identical for Compose Enterprise. As of the writing of this article anyone who signs up can get a 30 day free trial, so you can follow along with this example if you like.
1. Open your browser, go to www.compose.io and sign in or sign up for your free trial.
2. This will generally land you on the Deployments page. If not click the deployments button in the upper left:
3. Click the Create Deployment button in the upper right of the deployments page.
4. This brings you to the page where you can select the data store you want to deploy. Figure 1 is a screen grab of the choices on this page.
5. Select the data source you want. In this case I selected PostgreSQL and got this page. As you can see I merely name my deployment, select the Location, the database version and the size of the database that I want and then click the “Create Deployment” button
You can name the deployment anything that you like. Using the Location drop-drop down you select whose cloud you want ASW, IBM Softlayer or Google, and the data center available from that cloud provider.
6. The deployment takes a few minutes and when it is done you land on the over view tab of the deployment. Notice the other menu options on the left pane. Here you can click on the Browser tab to see your database (“compose” is the default), drop and create databases, create and drop tables and execute SQL. You can also do other things on the main deploy menu like look at the settings of your database, check on your backups and several other things including creating users. Back on the Overview tab you can scroll down to see the credentials of your deployment so that you can get the information you need to connect to it from your application.
7. When you scroll down to the credentials section of the Overview tab you will see something like this:
8. For this deployment, notice that the important connection attributes are:
HOST NAME: sl-us-dal-9-portal.3.dblayer.com
DATABASE NAME: compose
9. You are now ready to go to your client and connect, assuming you have the needed drivers. In this case I’m going to go to the IBM Bluemix cloud and deploy an instance of Bluemix DataConnect (formerly known as DataWorks) to show how I would use these credentials to connect to this database.
So as you can see from this example, it was quite easy to deploy a data source on Compose and connect to it. While it was quite easy, this data was properly configured for the amount of data I selected, it is monitored around the clock and restarted if needed, it is backed up daily and patched as needed, keeping it secure. It is also in a highly available cluster and if it were deployed on Compose Enterprise the data at rest would also be encrypted. This allows your team to focus on the critical core systems, while providing various enterprise grade data sources to the fast moving development teams that need them.