Google Cloud - Q&A

 

1. What is critical outcome of API Management? - Measuring and tracking business performance.
2. Who provides highest level of security? Titan Security Keys 
3. 4 key benefits to manage cloud costs? Visibility, Accountability, Control and Intelligent recommendations.
4. What is Chronicle? Its is a service built on top of Google Cloud Infrastructure, to ingest data (logs etc) and scan for threats.
5. Types of support? Basic, Standard, Enhanced and Premium. 
6. What is
1. DataProc - Hadoop/Spark
2. DataFlow - Streaming Data
3. DataPrep - wrangle data based on tabular/interactive or visual structure
4. DataPlex - Unified Data Management
7. Three components of Google Clouds defence-in-depth data security design? Sharding, encryption key, key encryption key
8. What is
1. Cloud Profiler - Analyze application performance (CPU)
2. Cloud Debugger - Monitor Performance
3. Cloud trace - Optimize code
4. Cloud Monitoring - monitor the performance of the entire cloud infra.
5. Cloud Vision API - identify images/text etc in a document
9. What is BYOIP? - Bring your own IP.
10. Build a new application on cloud while keeping old application On-Premise. What is this pattern called? - Invent in Brownfield. [Greenfield implies something completely new]
11. Minimize payment for traffic from Google cloud to Internet? use Cloud VPN.
12. Your org uses Active Directory to authenticate users. Google account access must be removed when their AD account is terminated. ---- Use single sign on in the Google domain
13. Migrating on Premise to Google Cloud. Functions owned by the cloud provider? - Infra arch and Hardware Maintenance
14. Which product provides consistent platform for multi-cloud application deployments and extends other Google Cloud services to your environment? - Anthos

15. Your organization needs to restrict access to a Cloud Storage bucket. Only employees who are based in Canada should be allowed to view the contents.What is the most effective and efficient way to satisfy this requirement? - Configure Armor to allow access to only IP from Canada

16. Google Cloud managed solutions to automate your build, testing, and deployment process? - Cloud Build

17. Google Cloud to privately and securely access your large volume of on-premises data, and you also want to minimize latency? - Google Edge network

18. 2 hour SLA - Enhanced support model

19. Plug-and-play AI components which can easily build ML services -AI Hub

20. Recommendations AI delivers highly personalized product recommendations at scale.

21. Document AI uses AI to unlock insights from documents.

22. Cloud Talent Solution uses AI with job search and talent acquisition capabilities.

23. Preview, Early Access, Alpha, and Beta do not have any SLA commitments.

24. Which of the following NIST Cloud characteristics uses the business model of shared resources in a cloud environment? - Multi-Tenancy

25. What are the network requirements for Private Google Access?

1. Because Private Google Access is enabled on a per-subnet basis, you must use a VPC network. Legacy networks are not supported because they don't support subnets.
2. - Private Google Access does not automatically enable any API. You must separately enable the Google APIs you need to use via the APIs & services page in the Google Cloud Console.
3. If you use the private.googleapis.com or the restricted.googleapis.com domain names, you'll need to create DNS records to direct traffic to the IP addresses associated with those domains.
4. Your network must have appropriate routes for the destination IP ranges used by Google APIs and services. These routes must use the default internet gateway next hop. If you use the private.googleapis.com or the restricted.googleapis.com domain names, you only need one route (per domain). Otherwise, you'll need to create multiple routes.
5. Egress firewalls must permit traffic to the IP address ranges used by Google APIs and services. The implied allow egress firewall rule satisfies this requirement. For other ways to meet the firewall requirement.
26. manage a bunch of API keys for external services that are accessed by different applications, which are used by a few teams - Store the information in Secret Manager is a secure and convenient storage system for API keys, passwords, certificates, and other sensitive data. Secret Manager provides a central place and single source of truth to manage access, and audit secrets across Google Cloud.
27. Which Google Cloud product gives you a consistent platform for multi-cloud application deployments and extends other Google Cloud services to your environment? - Anthos
28. Bigtable is the best suited for time series data. It also has high read-write throughput and ability to scale globally.
29. VM instances that only have internal IP addresses (no external IP addresses) can use Private Google Access. They can reach the external IP addresses of Google APIs and services. 
30. Google offers Firebase, In terms of Firebase Console, any particular message that has to be delivered to a customer at a certain degree of change in behavior can be managed through _________________ >> notification composer
31. Google Clouds WebApp and API Protection (WAAP) protects the application from BOTS.
32. You are working with a user to set up an application in a new VPC behind a firewall and it is noticed that the user is concerned about data egress. Therefore, to provide assistance you want to configure the fewest open egress ports >>> Setup a low priority rule (65534) that blocks all egress. Create a high priority rule (1000) that allows only specific port.
33. Container Registry is only multi-regional but Artifact Registry supports multi regional or regional repositories

Google Cloud - Costs

One of the most important factor to decide on moving to cloud is Cost savings apart from the flexibility provided by Cloud.

How are costs broken down in Google Cloud?

Fixed Price Models and Consumption based Models.

Consumption based Models implies billing for the resources being used/consumed.

e.g. Cloud storage Billed for the amount of storage

Fixed Price Models implies billing fixed irrespective of the usage.

e.g. VM Instance, GKE Cluster (till you delete)

Costs are based on:

• Data transfer (ingress and egress)
• Ingress is mostly free
• Egress to same zone with Google cloud using internal IP is free
• Egress from one region to another region in Google Cloud is not free
• Capacity (memory/CPU etc)
• Invocations 
• Time (VM running)
• Region wise price difference

Billing account gives you an overview of the total costs (forecasted as well).

On the left hand side menu, you will see multiple options

Reports will give you reports (as expected) and trends per project or services (timeline as well).

Cost Table More details and can download an invoice for a specific month.

Cost Breakdown gives details of the base usage cost and adjustments, credits and taxes.

Commitment is the summary of the committed use discounts by commitment type.

e.g put a commitment of a VM for 3 months etc.

Budget and alerts helps you to avoid surprises by creating a budget and an alert mechanism when the costs are going over board than the set budget (alert levels 50%, 90% or 100% via email or message).

Google Cloud - Miscellaneous - Part 3

Building apps for Android/iOS?

Use FireBase

• Google Cloud mobile platform
• Server less
• Backend DB is Firestore (No SQL DB)
• Authentication
• Monitoring
• Check out firebase.google.com

Container Registry and Artifact Registry

• To store docker images
• Container Registry uses Google Cloud Storage buckets to store images
• Cannot store jar/zip etc. Only container images only similar to Docker Hub
• Artifact Registry is an evolution of Container Registry
• Can store jar, zip, container images etc
• Create repositories for different formats like docker, npm, python etc
• Does not store in Google Cloud Storage buckets but in repositories
• Repositories can be multi-region
• Automatically encrypted

Security related services?

• KMS
• Key management service
• Create and manage cryptographic keys
• For encrypting and decrypting data
• Secret Manager
• To manage DB passwords
• Manage API Keys
• Cloud Data Loss Prevention
• Mask data like credit card numbers, passwords, credentials
• Provides API
• Cloud Armor
• SQL Injection prevention
• DDos Attack prevention
• Cross site scripting (XSS) prevention
• Web Security Scanner
• Identifies vulnerability by running security tests
• Binary Authorization
• Ensures only trusted containers are deployed
• Container Threat Detection
• Checks for threats at runtime to containers
• Cloud DLP
• Find sensitive data in your cloud storage buckets

Google Cloud - Machine Learning (ML) intro

Pre Built models are provided in Google Cloud as APIs.

• Speech to Text API
• Text to Speech API
• Translation API
• Natural Language API - Derive insights from unstructured texts
• Image based insights - Cloud vision API (Detect faces/objects etc)

Custom Models

• AutoML
• Build custom models easily
• AutoML Vision (Images)
• AutoML Video (Streaming data) 
• AutoML tables (Model from tables)
• Vertex AI
• Build and Deploy (MLOps)
• Tensor Processing Units (TPUs) for running faster ML workloads

Google Cloud - Miscellaneous (Part 2) - BigData related

 Cloud DataProc:

• Managed Spark and Hadoop service used for batch processing for AI or ML.
• Spark, HIVE, Hadoop, Pig etc are all supported
• Uses VMs
• Multi cluster mode where we can have multiple masters (upto 3)
• For simple data pipelines without clusters one can use DataFlow.
• Server less hence no clusters management
• For ETL (Extract/Transform/Load) we can use
• Data Prep for simple clean and load (intelligent service)
• Data Flow - Little more complex pipelines
• Data Proc - For very complex processing
• To visualize data in BigQuery - use data studio or Looker
• Visualize your data pipelines - Cloud Data Fusion

For Streaming data?

• Cloud Pub/Sub > Data Flow > BigQuery or BigTable

For IOT?

• Cloud IOT Core > Cloud Pub/Sub > Data Flow > BigQuery or BigTable or Data Store

For Complex Big Data solutions (Data Lake)?

• Data Ingestion
• Cloud Pub/Sub + Data Flow
• Processing and Analytics
• BigQuery (SQL) or Data Proc (Hadoop cluster)
• Data Mining
• Data Prep

REST API Management

• APIGEE
• API Management Platform
• For Cloud/On-Premise or Hybrid
• Provides Cloud Endpoints as well
• API Gateway
• Simpler than APIGEE and newer
• Relatively simple to setup than APIGEE

Google Cloud - Miscellaneous

 Google Cloud - Miscellaneous

Just categorizing some miscellaneous stuff under one post.

BILLING ACCOUNT:

• Billing account contains the payment details.
• Every project is associated to one billing account.
• A billing account can have multiple projects.
• An organization can have multiple billing accounts.

Types of Billing accounts:

• Self Served - Billed directly to credit card or Bank account
• Invoiced - Invoice generated

Hierarchy:

Please don't get confused with projects and organization etc mentioned above.

The Hierarchy in Google Cloud is

Organization > Folder > Projects

• Recommended to create different projects per environment (one for Dev and one for Prod)
• Recommended to create different Folder for different departments in an organization

Budget and Alerts

• We looked at creating a billing service.
• How do we get alerts to avoid surprises?
• Setup a Billing Account Budget
• Configure Alerts (set up thresholds - 50% 90% and 100%)
• Emails are sent to admins
• Export the data to BigQuery or Cloud Storage

Types of Cloud Configurations:

• Public
• Hosted in cloud
• No CAPEX, pay as per needs/usage
• Upgrades/installations owned by Google
• Shared with multiple enterprises (tenants)
• Private
• Host in your data center
• High CAPEX
• Quick scale is an issue (unless bought and kept leading to low utilization)
• Upgrades to infrastructure leads to more CAPEX
• Advantage 
• Complete control
• High level of security
• Hybrid
• Mix of public and private cloud
• On Premise application interacting with DB or another application on the cloud
• Cloud VPN
• Use Cloud VPN to connect on premise network to GCP.
• Uses IPSec VPN Tunnel
• Traffic goes through Internet (public) and hence encryption is needed
• Encryption using IEX (Internet Exchange) protocol.
• Two Types of Cloud VPN:
• HA VPN
• High availability
• 99.99% service availability
• Two external IPs
• Static routing not supported. Only dynamic routing.
• Classic VPN
• Static and dynamic routing supported
• One external IP
• 99.99% service availability
• Suitable for low Bandwidth needs.
• Cloud interconnect
• High Speed
• Physical connection between On premise and GCP
• High availability
• High Throughput
• Types
• Dedicated Interconnect
• 10 Gbps or 100 Gbps.
• Partner Interconnect
• 50 Mbps to 10 Gbps
•  
• Traffic goes through a private network.
• DIRECT PEERING
• Using network peering, connect to google network.
• This is NOT a GCP service and hence NOT RECOMMENDED.

Some things to remember:

• Cloud Data Flow:
• Provides unified streaming and batch data processing thats server less, fast and cost effective.
• Helps to create a streaming pipeline
• e.g. storage > database  (using data flow batch data load)
• Based on an open source framework called "Apache Beam" 
• Server less
• Auto scales
• For a CI/CD pipeline
• Store code in a private github called "Cloud source repositories"
• Store Docker images in "Container Registry"
• Jenkins for CI
• Cloud Build to build jars/docker images etc
• Spinnaker is a multi cloud continuous delivery.
• Cloud Monitoring for alerts and metrics
• Cloud Debugger for real time debugging
• Cloud Logging is for centralized logging
• Error Reporting provides real time exception monitoring
• Cloud Deployment Manager is Infrastructure as Code service 
• Cloud Audit Logs for Audit Logging
• To trace requests across various micro services, use Cloud Trace.
• After tracing if we want to run profiler on a specific micro service to debug slowness etc we can use Cloud Profiler.

What is Pub/Sub?

• Pub/Sub stands for Publisher and Subscriber.
• Imagine service A calls service B which does some action
• e.g Service A calls a logging service B which inserts logs into a DB
• Direct calls to service B could be an issue if load is high or service B fails
• Use a pub/sub
• Service A inserts requests into a topic
• Service A is the publisher 
• Service B picks from Topic
• Service B is subscriber
• No impact if Service B goes down
• Scale service B if too many requests into a Topic
• No loss of requests if service B goes down (can pick up when up)
• Pub/Sub is
• Fully managed asynchronous service
• Helps to make applications highly available and scalable
• Low cost (pay per use) - # of messages
• Both push and pull message deliveries is supported
• Creating a topic is a pre-requisite.
• Make requests to pubsub.googleapis.com
• Subscribers to provide a web hook endpoint if push notifications needed

Google Cloud - Identity and Access Management

 Google Cloud - Identity and Access Management (IAM)

• We have resources in the cloud
• VMs
• Databases
• These resources are accessed by
• Applications
• Services
• People logging in via console
• These are known as identities
• IAM allows
• Identities access to resources
• Configure actions permission (e.g. start a VM, delete a VM etc)
• There are two main parts
• Authentication
• Is this the right user/identity 
• Authorization
• Does this user/identity have correct access to a resource and
• Does this user/identity have the correct access to perform an action on a resource.
• In Google Cloud IAM, we create
• ROLES which are a set of permissions
• Basic/Primitive role - Permissions like owner, editor, viewer
• Owner - Able to edit, manage roles/permissions and billing
• Editor - View and Edit
• Viewer - Read only view
• NOT RECOMMENDED for production.
• PreDefined Roles
• Pre defined and managed by Google Cloud
• Different roles for different purposes
• Storage Object Admin, Storage Object Viewer etc
• CUSTOM ROLES
• Ability to create custom roles
• POLICY to bind the role to a member/user
• What if we want to an application to access cloud storage?
• SERVICE ACCOUNTS
• We can create service accounts which can be used by applications
• These accounts do not have any password but have a public/private RSA key
• Can't use them to login via browsers
• Types of Service accounts?
• Default
• Created by default when a service is created.
• Editor role by default
• Not recommended since it has Editor role by default.
• User Managed
• Create your own service account
• Grant it access via Role
• Google Managed
• These are Google's internal accounts.

Let's talk a bit about best practices.

For IAM Roles, look at the principle of least privileges, implies give the least possible privilege for a role.

e.g do not give admin access where only edit+view is sufficient. Similarly, give only view access if that suffices the role.

Don't use a common service account between different applications. Create separate service accounts with different roles assigned.