Google Cloud - Miscellaneous

 Google Cloud - Miscellaneous

Just categorizing some miscellaneous stuff under one post.

BILLING ACCOUNT:

• Billing account contains the payment details.
• Every project is associated to one billing account.
• A billing account can have multiple projects.
• An organization can have multiple billing accounts.

Types of Billing accounts:

• Self Served - Billed directly to credit card or Bank account
• Invoiced - Invoice generated

Hierarchy:

Please don't get confused with projects and organization etc mentioned above.

The Hierarchy in Google Cloud is

Organization > Folder > Projects

• Recommended to create different projects per environment (one for Dev and one for Prod)
• Recommended to create different Folder for different departments in an organization

Budget and Alerts

• We looked at creating a billing service.
• How do we get alerts to avoid surprises?
• Setup a Billing Account Budget
• Configure Alerts (set up thresholds - 50% 90% and 100%)
• Emails are sent to admins
• Export the data to BigQuery or Cloud Storage

Types of Cloud Configurations:

• Public
• Hosted in cloud
• No CAPEX, pay as per needs/usage
• Upgrades/installations owned by Google
• Shared with multiple enterprises (tenants)
• Private
• Host in your data center
• High CAPEX
• Quick scale is an issue (unless bought and kept leading to low utilization)
• Upgrades to infrastructure leads to more CAPEX
• Advantage 
• Complete control
• High level of security
• Hybrid
• Mix of public and private cloud
• On Premise application interacting with DB or another application on the cloud
• Cloud VPN
• Use Cloud VPN to connect on premise network to GCP.
• Uses IPSec VPN Tunnel
• Traffic goes through Internet (public) and hence encryption is needed
• Encryption using IEX (Internet Exchange) protocol.
• Two Types of Cloud VPN:
• HA VPN
• High availability
• 99.99% service availability
• Two external IPs
• Static routing not supported. Only dynamic routing.
• Classic VPN
• Static and dynamic routing supported
• One external IP
• 99.99% service availability
• Suitable for low Bandwidth needs.
• Cloud interconnect
• High Speed
• Physical connection between On premise and GCP
• High availability
• High Throughput
• Types
• Dedicated Interconnect
• 10 Gbps or 100 Gbps.
• Partner Interconnect
• 50 Mbps to 10 Gbps
•  
• Traffic goes through a private network.
• DIRECT PEERING
• Using network peering, connect to google network.
• This is NOT a GCP service and hence NOT RECOMMENDED.

Some things to remember:

• Cloud Data Flow:
• Provides unified streaming and batch data processing thats server less, fast and cost effective.
• Helps to create a streaming pipeline
• e.g. storage > database  (using data flow batch data load)
• Based on an open source framework called "Apache Beam" 
• Server less
• Auto scales
• For a CI/CD pipeline
• Store code in a private github called "Cloud source repositories"
• Store Docker images in "Container Registry"
• Jenkins for CI
• Cloud Build to build jars/docker images etc
• Spinnaker is a multi cloud continuous delivery.
• Cloud Monitoring for alerts and metrics
• Cloud Debugger for real time debugging
• Cloud Logging is for centralized logging
• Error Reporting provides real time exception monitoring
• Cloud Deployment Manager is Infrastructure as Code service 
• Cloud Audit Logs for Audit Logging
• To trace requests across various micro services, use Cloud Trace.
• After tracing if we want to run profiler on a specific micro service to debug slowness etc we can use Cloud Profiler.

What is Pub/Sub?

• Pub/Sub stands for Publisher and Subscriber.
• Imagine service A calls service B which does some action
• e.g Service A calls a logging service B which inserts logs into a DB
• Direct calls to service B could be an issue if load is high or service B fails
• Use a pub/sub
• Service A inserts requests into a topic
• Service A is the publisher 
• Service B picks from Topic
• Service B is subscriber
• No impact if Service B goes down
• Scale service B if too many requests into a Topic
• No loss of requests if service B goes down (can pick up when up)
• Pub/Sub is
• Fully managed asynchronous service
• Helps to make applications highly available and scalable
• Low cost (pay per use) - # of messages
• Both push and pull message deliveries is supported
• Creating a topic is a pre-requisite.
• Make requests to pubsub.googleapis.com
• Subscribers to provide a web hook endpoint if push notifications needed