styk.tv

Cloud Developer's Technology Encounters

helm-charts

by polfilm on June 8, 2017

=== INSTALL
in /tmp
curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get > get_helm.sh
sudo bash get_helm.sh

=== INITIALIZE
peter@ubuntu:~$ helm init
Creating /home/peter/.helm 
Creating /home/peter/.helm/repository 
Creating /home/peter/.helm/repository/cache 
Creating /home/peter/.helm/repository/local 
Creating /home/peter/.helm/plugins 
Creating /home/peter/.helm/starters 
Creating /home/peter/.helm/repository/repositories.yaml 
$HELM_HOME has been configured at /home/peter/.helm.

helm search
helm test
helm repo list
helm serve (--repo-path - but this is setup below already by init)

=== REPO LIST
peter@ubuntu:~/.helm/repository$ cat repositories.yaml 
apiVersion: v1
generated: 2017-06-08T00:44:36.366334051+01:00
repositories:
- caFile: ""
  cache: /home/peter/.helm/repository/cache/stable-index.yaml
  certFile: ""
  keyFile: ""
  name: stable
  url: https://kubernetes-charts.storage.googleapis.com
- caFile: ""
  cache: /home/peter/.helm/repository/cache/local-index.yaml
  certFile: ""
  keyFile: ""
  name: local
  url: http://127.0.0.1:8879/charts

mac pip issues

by polfilm on February 7, 2017

sudo -H pip install --ignore-installed -U numpy
pip install --ignore-installed six
pip install --user (last one installs everything into /User//Library/Python/2.7/ path)

Ref: http://apple.stackexchange.com/questions/209572/how-to-use-pip-after-the-os-x-el-capitan-upgrade comment by Yuri

AWS SSH Fingerprint

by polfilm on January 14, 2017

IF SHORT

root@ubuntu:~/.ssh/mac$ openssl pkey -in id_rsa -pubout -outform DER | openssl md5 -c
(stdin)= 6a:96:26:bd:fd:41:4f:53:77:67:78:13:01:c9:9c:12

IF LONG

openssl pkcs8 -in aws_private.pem -nocrypt -topk8 -outform DER | openssl sha1 -c

Git big file cleaner. Really cool, really easy.

by polfilm on December 12, 2016

https://rtyley.github.io/bfg-repo-cleaner/

Example of use:

polfilm@petersmacbook2:~/git_madcore$ java -jar ../Downloads/bfg-1.12.14.jar  --strip-blobs-bigger-than 100M controlbox.git

Using repo : /Users/polfilm/git_madcore/controlbox.git

Scanning packfile for large blobs: 1749
Scanning packfile for large blobs completed in 61 ms.
Found 1 blob ids for large blobs - biggest=118360126 smallest=118360126
Total size (unpacked)=118360126
Found 149 objects to protect
Found 10 commit-pointing refs : HEAD, refs/heads/Denis-fix-chain, refs/heads/Denis-fix-ssl, ...

Protected commits
-----------------

These are your protected commits, and so their contents will NOT be altered:

 * commit 1e3682a9 (protected by 'HEAD')

Cleaning
--------

Found 432 commits
Cleaning commits:       100% (432/432)
Cleaning commits completed in 598 ms.

Updating 2 Refs
---------------

    Ref                      Before     After
    --------------------------------------------
    refs/heads/development | 3c08c456 | 38f9abd0
    refs/heads/master      | 1e3682a9 | 30931052

Updating references:    100% (2/2)
...Ref update completed in 59 ms.

Commit Tree-Dirt History
------------------------

    Earliest                                              Latest
    |                                                          |
    .......................................................DDDDD

    D = dirty commits (file tree fixed)
    m = modified commits (commit message or parents changed)
    . = clean commits (no changes to file tree)

                            Before     After
    -------------------------------------------
    First modified commit | ad02cdfe | 327d6762
    Last dirty commit     | b6cf4f0b | 549a928f

Deleted files
-------------

    Filename                               Git id
    ----------------------------------------------------------
    spark-examples-1.5.2-hadoop2.6.0.jar | 4d798227 (112.9 MB)


In total, 50 object ids were changed. Full details are logged here:

    /Users/polfilm/git_madcore/controlbox.git.bfg-report/2016-12-12/13-57-10

BFG run is complete! When ready, run: git reflog expire --expire=now --all && git gc --prune=now --aggressive


Has the BFG saved you time?  Support the BFG on BountySource:  https://j.mp/fund-bfg

Mac Uninstall Java & Switch Java Versions

by polfilm on August 17, 2015

# Uninstall JDK
sudo rm -rf /Library/Java/JavaVirtualMachines/jdk<version>.jdk
sudo rm -rf /Library/PreferencePanes/JavaControlPanel.prefPane
sudo rm -rf /Library/Internet\ Plug-Ins/JavaAppletPlugin.plugin

#Add Ability to switch between 7 & 8
export JAVA_8_HOME=$(/usr/libexec/java_home -v1.8)
export JAVA_7_HOME=$(/usr/libexec/java_home -v1.7)

alias java7='export JAVA_HOME=$JAVA_7_HOME'
alias java8='export JAVA_HOME=$JAVA_8_HOME'

#default java8
export JAVA_HOME=$JAVA_8_HOME

Encrypt files using your public key and OpenSSL

by polfilm on July 24, 2015

==== GET PUBLIC KEY FROM CERT - CONVERT P12 to PEM
openssl pkcs12 -in my.client.cert.p12 -out my.client.cert.pem
Enter Import Password:
MAC verified OK
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:

=== GET PUBLIC KEY FROM CERT - EXTRACT PUBLIC KEY
openssl rsa -in my.client.cert.pem -pubout -out my.client.cert.pub
Enter pass phrase for my.client.cert.pem:
writing RSA key

=== ENCRYPT FILE - GENERATE RANDOM PASS
openssl rand -base64 40 -out pass.txt

=== ENCRYPT FILE - USE PASSWORD TO ENCRYPT FILE
openssl enc -e -aes-256-cbc -salt -pass file:pass.txt -in <filename> -out <filename>
e.g.
openssl enc -e -aes-256-cbc -salt -pass file:pass.txt -in somefile -out somefile.encrypted

=== ENCRYPT FILE - ENCRYPT PASS.TXT WITH PUBLIC KEY
openssl rsautl -encrypt -pubin -inkey <public key> -in <original file> -out <encrypted file>
e.g.
openssl rsautl -encrypt -pubin -inkey my.client.cert.pub -in pass.txt -out pass.encrypted

>> file is now encrypted
>> pass.txt can be deleted

=== DECRYPT FILE - DECRYPT PASSWORD
openssl rsautl -decrypt -inkey <pem file> -in <encrypted file> -out <original file>
e.g.
openssl rsautl -decrypt -inkey my.client.cert.pem -in pass.encrypted -out pass.txt

=== DECRYPT FILE - USE PASSWORD TO DECRYPT FILE
openssl enc -d -aes-256-cbc -pass file:pass.txt -in <filename>  -out <filename>
e.g.
openssl enc -d -aes-256-cbc -pass file:pass.txt -in somefile.encrypted -out somefile

>> delete pass.txt (no longer needed)

Screen

by polfilm on July 11, 2015

screen -list to list defatched
screen -r <screenID> to attach back
screen -d -r attach all

screen command Task
Ctrl+a c Create new window
Ctrl+a k Kill the current window / session
Ctrl+a w List all windows
Ctrl+a 0-9 Go to a window numbered 0 9, use Ctrl+a w to see number
Ctrl+a Ctrl+a Toggle / switch between the current and previous window
Ctrl+a S Split terminal horizontally into regions and press Ctrl+a c to create new window there
Ctrl+a :resize Resize region
Ctrl+a :fit Fit screen size to new terminal size. You can also hit Ctrl+a F for the the same task
Ctrl+a :remove Remove / delete region. You can also hit Ctrl+a X for the same taks
Ctrl+a tab Move to next region
Ctrl+a D (Shift-d) Power detach and logout
Ctrl+a d Detach but keep shell window open
Ctrl-a Ctrl-\ Quit screen
Ctrl-a ? Display help screen i.e. display a list of commands

Docker Bench for Security

by polfilm on June 10, 2015

The Docker Bench for Security is a script that checks for all the automatable tests included in the CIS Docker 1.6 Benchmark. We are releasing this as a follow-up to our Understanding Docker Security and Best Practices blog post.

We are making this available as an open-source utility so the Docker community can have an easy way to self-assess their hosts and docker containers against this benchmark.

https://github.com/docker/docker-bench-security

Docker Hadoop Spark Setup

by polfilm on March 12, 2015

CentOS 7 x86_64 Devel AtomicHost EBS HVM 20150306_01 – ami-a522b0d2
CentOS 7 (x86_64) with Updates HVM – ami-e4ff5c93