Devops tools and useful stacks

Here is a list of tools and stacks that will be useful in DevOps project.

terraform

https://www.terraform.io/ 
Infrastructure defined as text files. Product by HashiCorp.
Terraform is an open-source infrastructure as code software tool created by HashiCorp. It enables users to define and provision a data center infrastructure using a high-level configuration language known as Hashicorp Configuration Language, or optionally JSON.

sonarqube

https://www.sonarqube.org/
Continuous code quality checks tool.

kibana

https://www.elastic.co/products/kibana
A data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on an Elasticsearch cluster.

mesos

http://mesos.apache.org
Distributed OS. Uses for orchestrating clusters.

zipkin

https://zipkin.io/
Zipkin is a distributed tracing system

helm

https://helm.sh/
Helm helps you manage Kubernetes applications — Helm Charts helps you define, install, and upgrade even the most complex Kubernetes application.

quarkus

https://quarkus.io/
A Kubernetes Native Java stack tailored for GraalVM & OpenJDK HotSpot.

gradle

https://gradle.org/
Gradle is an open-source build automation system that builds upon the concepts of Apache Ant and Apache Maven and introduces a Groovy-based domain-specific language instead of the XML form used by Apache Maven for declaring the project configuration.

groovy

Apache Groovy is a Java-syntax-compatible object-oriented programming language for the Java platform. It is both a static and dynamic language with features similar to those of Python, Ruby, Perl, and Smalltalk.

jenkins

Jenkins is an free and open source automation server written in Java. Jenkins helps to automate the non-human part of the software development process, with continuous integration and facilitating technical aspects of continuous delivery.

gitlab CI

GitLab CI (Continuous Integration) service is a part of GitLab that build and test the software whenever developer pushes code to application.

prometheus

An open-source monitoring system with a dimensional data model, flexible query language, efficient time series database and modern alerting approach.

ansible 

https://www.ansible.com/

Ansible is the simplest way to automate apps and IT infrastructure. Application Deployment + Configuration Management + Continuous Delivery. Ansible is an open-source software provisioning, configuration management, and application-deployment tool. It runs on many Unix-like systems, and can configure both Unix-like systems as well as Microsoft Windows. It includes its own declarative language to describe system configuration.

kubernetes

Container orchestration tool.

docker

Enterprise container platform. Docker is a set of platform-as-a-service products that use operating-system-level virtualization to deliver software in packages called containers

NiFi

Designed to automate the flow of data between software systems.

docker-compose

Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application's services.

cypress.io

E2E testing framework. Fast, easy and reliable testing for anything that runs in a browser.

salt

https://www.saltstack.com/
Salt is Python-based, open-source software for event-driven IT automation, remote task execution, and configuration management.

vault

https://www.vaultproject.io/
Secure, store and tightly control access to tokens, passwords, certificates, encryption keys for protecting secrets and other sensitive data using a UI, CLI, or HTTP API.

Bazel

https://bazel.build/

In software development, Bazel is a free software tool that allows for the automation of building and testing of software. The company Google uses the build tool Blaze internally and released and open-sourced part of the Blaze tool as Bazel, named as an anagram of Blaze.

Jaegar

open-source, end-to-end distributed tracing

Monitor and troubleshoot transactions in complex distributed systems

Dream keyboards

Few of best keyboards I found on google.

1.

Microsoft 5KV-00001 Sculpt Ergonomic Wireless Keyboard for Business

 

https://www.amazon.in/Microsoft-Ergonomic-Keyboard-Business-5KV-00001/dp/B00CYX26BC/ref=sr_1_2?tag=guru99-21&ie=UTF8&qid=1540935383&sr=8-2&keywords=sculpt+keyboard

2. Kinesis Advantage2 Keyboard KB600

https://www.amazon.in/Kinesis-Advantage2-Ergonomic-Keyboard-KB600/dp/B01KR1C5PY/ref=as_li_ss_tl?tag=guru99-21&ie=UTF8&qid=1479099603&sr=8-1-fkmr0&keywords=kinesis+advantage+2+keyboard&linkId=ce855438d358e648afa10f77b8f91195

AWS components

EC2 

https://aws.amazon.com/ec2/ -eg : t2.micro (burstable CPU) -There are four ways to pay for Amazon EC2 instances: On-Demand, Reserved Instances, and Spot Instances. You can also pay for Dedicated Hosts which provide you with EC2 instance capacity on physical servers dedicated for your use.

ECS 

https://aws.amazon.com/containers/services/ -elastic container service -amazon provided orchestration for docker

EKS 

Elastic K8s service -amazon managed orchestration for docker using K8s

ECR 

Container registry

AWS Fargate 

Serverless compute engine built to run containers in production at any scale.

LightSail

(https://aws.amazon.com/websites/?nc2=h_m2) -Amazon Lightsail is the easiest way to launch and manage a Web server using AWS. Lightsail includes everything you need to jumpstart your Website – a virtual machine, SSD-based storage, data transfer, DNS management, and a static IP – for a low, predictable price. You can get started using Lightsail for your website with just a few clicks. Choose the operating system or application template that's best for your website, and your virtual private server is ready in less than a minute. You can easily manage your web server, DNS, and IP addresses directly from the Lightsail console

Sails JS action2 snippet

action2:
------------

module.exports = {

   friendlyName: 'Welcome user',

   description: 'Look up the specified user and welcome them, or redirect to a signup page if no user was found.',

   inputs: {
      userId: {
        description: 'The ID of the user to look up.',
        // By declaring a numeric example, Sails will automatically respond with `res.badRequest`
        // if the `userId` parameter is not a number.
        type: 'number',
        // By making the `userId` parameter required, Sails will automatically respond with
        // `res.badRequest` if it's left out.
        required: true
      }
   },

   exits: {
      success: {
        responseType: 'view',
        viewTemplatePath: 'pages/welcome'
      },
      notFound: {
        description: 'No user with the specified ID was found in the database.',
        responseType: 'notFound'
      }
   },

   fn: async function (inputs, exits) {

      // Look up the user whose ID was specified in the request.
      // Note that we don't have to validate that `userId` is a number;
      // the machine runner does this for us and returns `badRequest`
      // if validation fails.
      var user = await User.findOne({ id: inputs.userId });

      // If no user was found, respond "notFound" (like calling `res.notFound()`)
      if (!user) { return exits.notFound(); }

      // Display the welcome view.
      return exits.success({name: user.name});
   }
};

Sails JS Policies

Applying policies to a controller
==================================
config/policies.js

module.exports.policies = {
  UserController: {
    // By default, require requests to come from a logged-in user
    // (runs the policy in api/policies/isLoggedIn.js)
    '*': 'isLoggedIn',

    // Only allow admin users to delete other users
    // (runs the policy in api/policies/isAdmin.js)
    'delete': 'isAdmin',

    // Allow anyone to access the login action, even if they're not logged in.
    'login': true
  }
};

Applying policies to actions
----------------------------

module.exports.policies = {
  'user/*': 'isLoggedIn',
  'user/delete': 'isAdmin',
  'user/login': true
}

Sample policy
-------------
// policies/isLoggedIn.js
module.exports = async function (req, res, proceed) {

  // If `req.me` is set, then we know that this request originated
  // from a logged-in user.  So we can safely proceed to the next policy--
  // or, if this is the last policy, the relevant action.
  // > For more about where `req.me` comes from, check out this app's
  // > custom hook (`api/hooks/custom/index.js`).
  if (req.me) {
    return proceed();
  }

  //--•
  // Otherwise, this request did not come from a logged-in user.
  return res.forbidden();

};

Sails JS sample helper file

sample helper file
-------------------
// api/helpers/format-welcome-message.js
module.exports = {

  friendlyName: 'Format welcome message',


  description: 'Return a personalized greeting based on the provided name.',


  inputs: {

    name: {
      type: 'string',
      example: 'Ami',
      description: 'The name of the person to greet.',
      required: true
    }

  },


  fn: async function (inputs, exits) {
    var result = `Hello, ${inputs.name}!`;
    return exits.success(result);
  }

};

Sails JS sample classic action

sample classic action:
----------------------

module.exports = async function welcomeUser (req, res) {

  // Get the `userId` parameter from the request.
  // This could have been set on the querystring, in
  // the request body, or as part of the URL used to
  // make the request.
  var userId = req.param('userId');

   // If no `userId` was specified, or it wasn't a number, return an error.
  if (!_.isNumeric(userId)) {
    return res.badRequest(new Error('No user ID specified!'));
  }

  // Look up the user whose ID was specified in the request.
  var user = await User.findOne({ id: userId });

  // If no user was found, redirect to signup.
  if (!user) { return res.redirect('/signup' );

  // Display the welcome view, setting the view variable
  // named "name" to the value of the user's name.
  return res.view('welcome', {name: user.name});

}

Sails js Basic Commands

sails cmnds
-------------


#sails new test-project
#sails lift
#sails generate api cats // generates api/controllers/CatsController.js along with a matching model
#sails generate controller user//to quickly create a controller file. api/controllers/UserController.js
#sails generate action user/signup //generate action2 file
#sails generate action user/signup --no-actions2 //generate classic actions 
#sails generate helper tickle-user //generate helper files. api/helpers/tickle-user.js

Bash terminal navigations shortcuts

Ctrl-A : go to the beginning of line.
Ctrl-E : go to the end of line.
Alt-B : skip one word backward.
Alt-F : skip one word forward.
Ctrl-U : delete to the beginning of line.
Ctrl-K : delete to the end of line.
Alt-D : delete to the end of word.


Ctrl + a – go to the start of the command line
Ctrl + e – go to the end of the command line
Ctrl + k – delete from cursor to the end of the command line
Ctrl + u – delete from cursor to the start of the command line
Ctrl + w – delete from cursor to start of word (i.e. delete backwards one word)
Ctrl + y – paste word or text that was cut using one of the deletion shortcuts (such as the one above) after the cursor
Ctrl + xx – move between start of command line and current cursor position (and back again)
Alt + b – move backward one word (or go to start of word the cursor is currently on)
Alt + f – move forward one word (or go to end of word the cursor is currently on)
Alt + d – delete to end of word starting at cursor (whole word if cursor is at the beginning of word)
Alt + c – capitalize to end of word starting at cursor (whole word if cursor is at the beginning of word)
Alt + u – make uppercase from cursor to end of word
Alt + l – make lowercase from cursor to end of word
Alt + t – swap current word with previous
Ctrl + f – move forward one character
Ctrl + b – move backward one character
Ctrl + d – delete character under the cursor
Ctrl + h – delete character before the cursor
Ctrl + t – swap character under cursor with the previous one

X11 virtual frame buffer

# Xvfb :99 -screen 0 1600x900x16 -nolisten tcp 2>&1 >/dev/null &

This will start a new X virtual frame buffer at display :99 on screen 0 .

We can connect vnc to this vfb, using x11vnc.

First add Extra Packages for Enterprise Linux repo rpm
# yum install  https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

Then install x11vnc
#yum install x11vnc

start the x11vnc
#x11vnc -display :99

Now you can connect vnc to the default port 5900

Ansible

https://www.ansible.com/

In computing, Ansible is an open-source software provisioning, configuration management, and application-deployment tool. It runs on many Unix-like systems, and can configure both Unix-like systems as well as Microsoft Windows. It includes its own declarative language to describe system configuration.

Helm.sh

https://helm.sh/

Helm helps you manage Kubernetes applications — Helm Charts help you define, install, and upgrade even the most complex Kubernetes application.

Zipkin

https://zipkin.apache.org/

Zipkin is a distributed tracing system. It helps gather timing data needed to troubleshoot latency problems in microservice architectures. It manages both the collection and lookup of this data. Zipkin’s design is based on the Google Dapper paper.

Apache Mesos

http://mesos.apache.org/ Distributed OS

Apache Mesos abstracts CPU, memory, storage, and other compute resources away from machines (physical or virtual), enabling fault-tolerant and elastic distributed systems to easily be built and run effectively.

What is Mesos?

A distributed systems kernel Mesos is built using the same principles as the Linux kernel, only at a different level of abstraction. The Mesos kernel runs on every machine and provides applications (e.g., Hadoop, Spark, Kafka, Elasticsearch) with API’s for resource management and scheduling across entire datacenter and cloud environments. 

Kibana

Kibana lets you visualize your Elasticsearch data and navigate the Elastic Stack

https://www.elastic.co/products/kibana

Sonarqube

SonarQube Continous code quality checks tool

https://www.sonarqube.org/

SonarQube is an open-source platform developed by SonarSource for continuous inspection of code quality to perform automatic reviews with static analysis of code to detect bugs, code smells, and security vulnerabilities on 20+ programming languages. 

Conan

Conan : C++ package manager

https://conan.io/

The open source, decentralized and multi-platform package manager to create and share all your native binaries.