Databases in Google Cloud

 We all know about different types of Databases. Just highlighting them to start off the topic.

Relational DB:

• Fixed table schema and relations (primary and foreign key)
• Strong transactional capability
• Update all table data in one transaction and commit
• Failure of one, implies rollback in all.
• Since its strong in transactions, Relational DB is preferred for OLTP (Online transaction processing) systems like banking.
• Lots of transactions (large number of users and small transactions per user)
• Data is stored in row storage
• In Google Cloud we have
• Cloud SQL
• Supports MySQL, SQL server and PostgreSQL
• Regional Database
• Multi zone for High availability
• Can't create Global Cloud SQL DB
• Data upto a few TB
• Fully managed (replication, patch management, DB management etc)
• Public IP provided to connect to the DB
• Cloud shell option also provided in console (gcloud sql command)
• Enable Cloud SQL Admin API as a pre-requisite
• Cloud Spanner
• Unlimited scale
• Horizontal scaling
• High availability
• Global applications (Globally consistent)
• Fully managed
• Multi region
• Option to add compute capacity when being configured
• 1 node = 1000 processing units
• Compute cost is the hourly cost charge for nodes or processing units.
• Storage cost is separate and billed per GB per month
• Min 100 processing units or 1 node

• Can be used for Analytics as well (OLAP - Online Analytics Processing)
• Used to analyze huge amount of data
• Reporting, data warehouse
• Uses columnar storage (not row) [High compression]
• Since data is stored via columns, it can be distributed
• Queries can run over multiple nodes (efficient execution for complex queries)
• GCP Managed service is BigQuery

NO SQL DB: (Not only SQL)

• Flexible schema
• Scales horizontally
• Highly scalable
• High performance
• GCP managed No SQL service:
• Cloud Firestore (Data store)
• Serverless
• Document DB
• Can run SQL like queries
• Used for high transactions
• Mobile and web applications
• Small to medium DB (few TB)
• BigTable
• Scalable
• Wide column DB
• Not serverless
• Data size > 10TB
• Does not support multi row transaction. Supports only single row.
• Not to be used for transactional applications

In Memory Database

• Faster to retrieve data since data is not on the disk
• Low latency (microseconds)
• Persistent data stored in memory
• Use for caching or session management
• GCP service is Managed Store

Cassandra (High Level details)

What is Cassandra?

• Open source
• Distributed Database
• NoSQL Database
• Can add lots of nodes in a cluster hence highly scalable
• High availability since the nodes can be spread across DC's.
• Fault tolerant, durable with no single point of failure.
• Note that in Cassandra distributed architecture, there are multiple nodes. These nodes can be replicated across multiple data centers to avoid a single point of failure.

All the nodes have the same functionality and no one node is a master node. We do not have a master slave node architecture. All nodes have the same functionality.

If all nodes have the same functionality, how do the nodes know about the other nodes in the cluster?
Ans. Snitch.

What is a Snitch?

As per official documentation: https://cassandra.apache.org/doc/latest/operating/snitch.html?highlight=snitch

In Cassandra, a snitch has two functions:

1. Teaches Cassandra enough about your network topology to route requests efficiently
2. Allows Cassandra to spread replicas around your cluster to avoid correlated failures. It does this by grouping machines into “data centers” and “racks.” Cassandra will do its best not to have more than one replica on the same “rack” (which may not actually be a physical location).

There are various snitch classes. Read more about them in the above link.
SimpleSnitch is suitable for only a single DC.

In PropertyFileSnitch, one will put the IP address of all the nodes in a cluster and also specify the rack and DC. Data is entered in the cassandra-topology.properties file.
This is fine if we had lesser # of nodes. But if we had 1000 nodes, it would be very tedious to keep creating and modifying entries for the nodes.

What is Gossip?
Gossip is how the nodes in a cluster communicate with each other.
Every second, a node sends information about itself and the other nodes information (that it has) to at-least 3 other nodes.
Thus, as mentioned before, there is no master slave configuration. Eventually all nodes have information of all the other nodes and they keep sending/sharing information every 1 sec.
Do note that this is internal communication between nodes in a cluster.

Data is distributed across nodes (so rows in a table can be in different nodes).

But if the rows in a table are in different nodes and if a node goes down, would we lose data?
We have an option to replicate data by setting a replication factor. If the replication factor is 1, then data is not replicated and you may lose data. But if the replication factor is increased to 2 or 3 etc, the data is replicated across nodes and we have high availability of data even if nodes go down.

Oracle NOSQL Architecture (Basics)

Oracle NOSQL services network requests to store and retrieve data which is organized into key-value pairs.

The typical application is a web application which is servicing requests across the traditional three-tier architecture: web server, application server and NOSQL DB.

An application makes use of Oracle NoSQL Database by performing network requests against Oracle NoSQL Database's key-value store, which is referred to as the KVStore.

The requests are made using the Oracle NoSQL Database Driver, which is linked into your application as a Java library (.jar file), and then accessed using a series of Java APIs.

By using the Oracle NoSQL Database APIs, the developer is able to perform create, read, update and delete operations on the data contained in the KVStore



 Oracle NoSQL Database is tested using Java 7.

Key Value Store (KV Store):

The KVStore is a collection of Storage Nodes which host a set of Replication Nodes. Data is spread across the Replication Nodes. The store contains multiple Storage Nodes.

A Storage Node is a physical (or virtual) machine with its own local storage. The machine is intended to be commodity hardware. It should be, but is not required to be, identical to all other Storage Nodes within the store.




Replication Nodes and Shards
A Replication Node can be thought of as a single database which contains key-value pairs.

Replication Nodes are organized into shards. A shard contains a single Replication Node, called the master node, which is responsible for performing database writes.


 




The master node copies those writes to the other Replication Nodes in the shard, called the replicas. 
These replicas obtain a full copy of the data from the corresponding master node and are used to service read-only operations. Although there can be only one master node at any given time, any of the members of the shard are capable of becoming a master node.


Note:
The more shards that your store contains, the better your write performance is because the store contains more nodes that are responsible for servicing write requests.

Replication Factor
The number of nodes belonging to a shard is called its Replication Factor.

The larger a shard's Replication Factor, the faster its read throughput (because there are more machines to service the read requests) but the slower its write performance (because there are more machines to which writes must be copied).