Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save scarolan/e0d9b91e189cf80ab4fb3550ba6c4734 to your computer and use it in GitHub Desktop.
Save scarolan/e0d9b91e189cf80ab4fb3550ba6c4734 to your computer and use it in GitHub Desktop.

Revisions

  1. @apolloclark apolloclark revised this gist Jun 12, 2020. 1 changed file with 1 addition and 1 deletion.
    2 changes: 1 addition & 1 deletion system testing.md
    Original file line number Diff line number Diff line change
    @@ -1,4 +1,4 @@
    # Build Tools
    # Build Tools and Test

    ## Build Manager

  2. @apolloclark apolloclark revised this gist Sep 8, 2019. 1 changed file with 1 addition and 0 deletions.
    1 change: 1 addition & 0 deletions system testing.md
    Original file line number Diff line number Diff line change
    @@ -157,6 +157,7 @@ Ruby -&nbsp;<a href="https://www.chef.io/inspec/">inspec </a>(Chef)
    <h3>BDD Testing, Infrastructure (ops devs)</h3>
    Goal: environment should behave consistently.<br />
    Python - <a href="https://github.com/ryotarai/infrataster">infrataster </a>(HTTP, pgsql, redis)<br />
    Terraform - <a href="https://github.com/gruntwork-io/terragrunt">Terragrunt</a><br/>
    <br />
    &nbsp;
    <h2>Dynamic Analysis</h2>
  3. @apolloclark apolloclark revised this gist Sep 8, 2019. 1 changed file with 2 additions and 0 deletions.
    2 changes: 2 additions & 0 deletions system testing.md
    Original file line number Diff line number Diff line change
    @@ -192,6 +192,8 @@ Python - <a href="http://lettuce.it/tutorial/simple.html">Lettuce</a>
    <a href="https://jmeter.apache.org/">JMeter</a><br />
    <a href="https://www.blazemeter.com/">Blazemeter</a> - Hosted JMeter<br />
    <a href="https://gatling.io/">Gatling</a>
    <a href="https://github.com/locustio/locust">Locust</a>
    <a href="http://grinder.sourceforge.net/">The Grinder</a>

    <h3>BDD Testing, Security (security)</h3>
    <strong>Goal</strong> - systems should behave securely, end-to-end<br />
  4. @apolloclark apolloclark revised this gist Sep 8, 2019. 1 changed file with 2 additions and 2 deletions.
    4 changes: 2 additions & 2 deletions system testing.md
    Original file line number Diff line number Diff line change
    @@ -34,7 +34,7 @@
    <a href="https://docs.gitlab.com/ee/user/project/import/gemnasium.html">Gemnasium</a> (java, python, ruby, node.js)<br />
    <a href="https://ossindex.sonatype.org/">OSSIndex</a> (java, python, ruby, node.js)<br />
    Java - <a href="https://jeremylong.github.io/DependencyCheck/dependency-check-maven/index.html">dependency-check-maven</a><br />
    Python - <a href="https://pypi.org/project/dependency-check/">dependency-check</a><br />
    Python - <a href="https://pypi.org/project/dependency-check/">dependency-check</a>, <a href="https://pyup.io/">pyup</a><br />
    Ruby - <a href="https://github.com/rubysec/bundler-audit">bundle-audit</a>, <a href="https://hakiri.io/">Hakiri</a><br />
    Javascript - <a href="https://retirejs.github.io/retire.js/">RetireJS</a>

    @@ -89,7 +89,7 @@ React - <a href="https://github.com/pzavolinsky/react-unit">React-unit</a>, <a h
    <a href="https://www.sonarqube.org/">SonarCube</a> (Java, Python, Ruby, Node.js, Javascript)<br />
    <a href="https://github.com/pmd/pmd">PMD</a> (Java, Python, Ruby, Javascript)<br />
    Java - <a href="https://github.com/spotbugs/spotbugs">spotbugs</a><br />
    Python - <a href="https://bandit.readthedocs.io/en/latest/">bandit</a>, <a href="https://pyup.io/">pyup</a><br />
    Python - <a href="https://bandit.readthedocs.io/en/latest/">bandit</a><br />
    Ruby - <a href="https://brakemanscanner.org/">Brakeman</a><br />
    Javascript - <a href="https://code.google.com/archive/p/mustache-security/">mustache-security</a><br />
    <br />
  5. @apolloclark apolloclark created this gist Apr 18, 2019.
    234 changes: 234 additions & 0 deletions system testing.md
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,234 @@
    # Build Tools

    ## Build Manager

    *Goal*- centralized UI, and scheduler, for managing automated builds<br />
    -&nbsp;<a href="https://www.hashicorp.com/products/terraform#features">Terraform Enterprise</a> - HashiCorp<br />
    - <a href="https://jenkins.io/">Jenkins</a>&nbsp;- Cloudbees<br />
    - <a href="https://travis-ci.org/">TravisCI</a><br />
    - <a href="https://circleci.com/">CircleCI</a><br />
    - <a href="https://www.atlassian.com/software/bamboo">Bamboo</a>&nbsp;- Atlassian<br />
    - <a href="https://kitchen.ci/">KitchenCI</a>&nbsp;- Chef

    <h3>Build Runner</h3>
    <strong>Goal</strong>&nbsp;- automated build runner<br />
    - Linux: <a href="https://www.tldp.org/LDP/Bash-Beginners-Guide/html/">Bash</a>, <a href="https://www.gnu.org/software/make/manual/html_node/Introduction.html">Makefile</a><br />
    - Java: <a href="https://docs.gradle.org/current/userguide/building_java_projects.html">Gradle</a>, <a href="https://maven.apache.org/what-is-maven.html">Maven</a>, <a href="https://ant.apache.org/">Ant</a><br />
    - Python: <a href="https://tox.readthedocs.io/en/latest/">tox</a>, <a href="http://www.fabfile.org/">fabric</a>, <a href="https://molecule.readthedocs.io/en/latest/">molecule</a><br />
    - Ruby: <a href="https://ruby.github.io/rake/">rake</a><br />
    - Node.js: <a href="https://gulpjs.com/">Gulp</a>, <a href="https://gruntjs.com/getting-started">Grunt</a>

    <h3>Build Tasks</h3>
    <strong>Goal</strong>&nbsp;- automatically call build runners<br />
    <a href="https://githooks.com/">https://githooks.com/</a><br />
    <a href="https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks">https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks</a><br />
    <br />
    <br />
    &nbsp;
    <h2>Static Analysis</h2>

    <h3>Dependency Security</h3>
    <strong>Goal</strong>&nbsp;- everyone&#39;s code should use secure components.<br />
    <a href="https://www.owasp.org/index.php/OWASP_Dependency_Check">OWASP Dependency Check</a> (java, python, ruby, php, node.js, Swift)<br />
    <a href="https://snyk.io/">Snyk</a> (java, python, ruby, node.js)<br />
    <a href="https://docs.gitlab.com/ee/user/project/import/gemnasium.html">Gemnasium</a> (java, python, ruby, node.js)<br />
    <a href="https://ossindex.sonatype.org/">OSSIndex</a> (java, python, ruby, node.js)<br />
    Java - <a href="https://jeremylong.github.io/DependencyCheck/dependency-check-maven/index.html">dependency-check-maven</a><br />
    Python - <a href="https://pypi.org/project/dependency-check/">dependency-check</a><br />
    Ruby - <a href="https://github.com/rubysec/bundler-audit">bundle-audit</a>, <a href="https://hakiri.io/">Hakiri</a><br />
    Javascript - <a href="https://retirejs.github.io/retire.js/">RetireJS</a>

    <h3>Code Format</h3>
    <strong>Goal</strong> -&nbsp;everyone&#39;s code should look the same, automatically.<br />
    Java - <a href="https://github.com/google/google-java-format">Google Java Format</a><br />
    Python - <a href="https://github.com/hhatto/autopep8">autopep8</a>, <a href="https://github.com/ambv/black">black</a><br />
    Ruby - <a href="https://rubocop.readthedocs.io/en/latest/">rubocop</a>, <a href="https://github.com/ruby-formatter/rufo">rufo</a><br />
    Javascript - <a href="https://github.com/prettier/prettier">prettier</a>

    <h3>Code Linting</h3>
    <strong>Goal</strong> - everyone&#39;s code should look the same, and be easy to maintain.<br />
    Java - <a href="http://checkstyle.sourceforge.net/">checkstyle</a>, <a href="http://findbugs.sourceforge.net/">findbugs</a>, <a href="https://github.com/google/error-prone">Google error-prone</a><br />
    Python - <a href="https://www.pylint.org/">pylint</a>, <a href="https://pypi.org/project/flake8/">flake8</a><br />
    Ruby - <a href="https://rubocop.readthedocs.io/en/latest/">rubocop</a><br />
    Javascript - <a href="https://eslint.org/">eslint</a>, <a href="https://github.com/douglascrockford/JSLint">jslint</a>, <a href="https://jshint.com/docs/">jshint</a>

    <h3>Code Complexity</h3>
    <strong>Goal</strong> - everyone&#39;s code should not be complicated, no conjugations in descriptions, and easy to maintain<br />
    <a href="https://en.wikipedia.org/wiki/Cyclomatic_complexity">McCabe Cyclomatic Complexity</a><br />
    <a href="https://www.sonarqube.org/">SonarCube</a> (Java, Python, Ruby, Node.js, Javascript)<br />
    <a href="https://github.com/pmd/pmd">PMD</a> (Java, Python, Ruby, Javascript)<br />
    Java - ???<br />
    Python - <a href="https://radon.readthedocs.io/en/latest/">radon</a><br />
    Ruby - <a href="https://github.com/whitesmith/rubycritic">Ruby Critic</a><br />
    Javascript - <a href="https://eslint.org/">eslint</a>, <a href="https://jshint.com/docs/">jshint</a>

    <h3>Code Documentation</h3>
    Goal: everyone&#39;s code should be easy to understand, and easy to maintain<br />
    Java - <a href="https://www.oracle.com/technetwork/java/javase/documentation/index-137868.html">javadoc</a>, <a href="http://checkstyle.sourceforge.net/">checkstyle</a><br />
    Python - <a href="http://www.pydocstyle.org/en/2.1.1/usage.html">pydocstyle</a><br />
    Ruby - <a href="https://rubocop.readthedocs.io/en/latest/">rubocop</a><br />
    Javascript - <a href="http://usejsdoc.org/">jsdoc</a>

    <h3>Unit Testing w/ Mocks</h3>
    <strong>Goal</strong> - code should function as expected, in isolation.

    <h3>Integration Testings, Server-side, w/ Mocks</h3>
    <strong>Goal</strong> - code should function as expected, with immediate dependencies

    <h3>Integration Testing, Client-side, w/ mocks</h3>
    <strong>Goal</strong> - code should behave consistently, only on the browser / mobile app, offline<br />
    Javascript - <a href="https://mochajs.org/">Mocha</a><br />
    jQuery - <a href="https://qunitjs.com/">QUnit</a><br />
    Angularjs - <a href="https://jasmine.github.io/">Jasmine </a>+ <a href="https://docs.angularjs.org/guide/unit-testing#angular-mocks">angularjs-mocks<br />
    https://docs.angularjs.org/guide/unit-testing</a><br />
    React - <a href="https://github.com/pzavolinsky/react-unit">React-unit</a>, <a href="https://jestjs.io/">Jest</a>, <a href="https://github.com/airbnb/enzyme/">Enzyme</a><br />
    <a href="https://reactjs.org/community/testing.html">https://reactjs.org/community/testing.html</a>

    <h3>Code Security</h3>
    <strong>Goal</strong> - everyone&#39;s code should be written securely<br />
    <a href="https://www.sonarqube.org/">SonarCube</a> (Java, Python, Ruby, Node.js, Javascript)<br />
    <a href="https://github.com/pmd/pmd">PMD</a> (Java, Python, Ruby, Javascript)<br />
    Java - <a href="https://github.com/spotbugs/spotbugs">spotbugs</a><br />
    Python - <a href="https://bandit.readthedocs.io/en/latest/">bandit</a>, <a href="https://pyup.io/">pyup</a><br />
    Ruby - <a href="https://brakemanscanner.org/">Brakeman</a><br />
    Javascript - <a href="https://code.google.com/archive/p/mustache-security/">mustache-security</a><br />
    <br />
    <br />
    &nbsp;
    <h2>Deploy</h2>

    <h3>Build</h3>
    <strong>Goal</strong> - build local VMs, Linux Containers, or Cloud VMs.<br />
    <a href="https://www.packer.io/">Packer</a> - Virtualbox, VMWare, Hyper-V, AWS AMI, Azure VM, Google Image

    <h3>Local VM</h3>
    <strong>Goal</strong> - automatically deploy local VMs.<br />
    <a href="https://www.vagrantup.com/">Vagrant</a> - Virtualbox, VMWare, Hyper-V

    <h3>Local Linux Container</h3>
    <strong>Goal</strong> - deploy a linux container.<br />
    <a href="https://www.docker.com/">Docker</a><br />
    <a href="https://linuxcontainers.org/">LXC</a>

    <h3>Local / Remote Containers</h3>
    <strong>Goal</strong> - orchestrate container deployment.<br />
    <a href="https://kubernetes.io/">Kubernetes</a><br />
    <a href="https://www.openshift.com/">OpenShift</a><br />
    <a href="https://linuxcontainers.org/lxd/introduction/">LXD</a>

    <h3>Container Security</h3>

    <p><strong>Goal</strong> - ensure containers have secure components.<br />
    <a href="https://jfrog.com/xray/">Artifactory JFrog XRay</a><br />
    <a href="https://github.com/anchore/anchore-cli">anchore-cli</a><br />
    <a href="https://github.com/coreos/clair">coreos claire</a><br />
    <a href="https://github.com/eliasgranderubio/dagda">dagda</a><br />
    <a href="https://github.com/cilium/cilium">cilium</a> - container network security<br />
    <a href="https://github.com/falcosecurity/falco">sysdig falco</a> - Kubernetes<br />
    <a href="https://docs.docker.com/v17.12/docker-cloud/builds/image-scan/">https://docs.docker.com/v17.12/docker-cloud/builds/image-scan/</a><br />
    <a href="https://techbeacon.com/security/10-top-open-source-tools-docker-security">https://techbeacon.com/security/10-top-open-source-tools-docker-security</a><br />
    <a href="https://sysdig.com/blog/20-docker-security-tools/">https://sysdig.com/blog/20-docker-security-tools/</a></p>

    <h3>Cloud</h3>
    <strong>Goal</strong> - deploy remote cloud resources.<br />
    <a href="https://aws.amazon.com/cloudformation/">AWS Cloudformation</a><br />
    <a href="https://azure.microsoft.com/en-us/features/resource-manager/">Azure Resource Manager</a><br />
    <a href="https://cloud.google.com/deployment-manager/">Google Deployment Manager</a><br />
    <a href="https://www.terraform.io/">Terraform</a> (AWS, Azure, GCE)

    <h3>Provision Runner</h3>
    <strong>Goal</strong> - automate running provision in multiple environments<br />
    Ansible - <a href="https://molecule.readthedocs.io/en/latest/">Molecule</a><br />
    Chef - <a href="https://kitchen.ci/">Kitchen</a>

    <h3>Provision</h3>
    <strong>Goal</strong> - install and configure services.<br />
    <a href="https://www.ansible.com/">Ansible</a><br />
    <a href="https://puppet.com/">Puppet</a><br />
    <a href="https://www.chef.io/chef/">Chef</a><br />
    <a href="https://www.saltstack.com/">Saltstack</a>

    <h3>Infrastructure Testing</h3>
    <strong>Goal</strong> - environments should be installed consistently.<br />
    Ruby - <a href="https://serverspec.org/">serverspec</a><br />
    Python - <a href="https://testinfra.readthedocs.io/en/latest/">testinfra</a>, <a href="https://github.com/aelsabbahy/goss">goss</a><br />
    Ruby -&nbsp;<a href="https://www.chef.io/inspec/">inspec </a>(Chef)

    <h3>BDD Testing, Infrastructure (ops devs)</h3>
    Goal: environment should behave consistently.<br />
    Python - <a href="https://github.com/ryotarai/infrataster">infrataster </a>(HTTP, pgsql, redis)<br />
    <br />
    &nbsp;
    <h2>Dynamic Analysis</h2>

    <h3>Integration Testing, Server-side</h3>
    <strong>Goal</strong> - code should behave consistently, only on the server.

    <h3>BDD Testing, Integration, Server-side (server devs)</h3>
    Ruby - <a href="https://github.com/cucumber/cucumber-ruby">cucumber</a>, <a href="http://rspec.info/">rspec</a><br />
    <a href="https://www.martinfowler.com/articles/rake.html">https://www.martinfowler.com/articles/rake.html</a><br />
    Python - <a href="https://behave.readthedocs.io/en/latest/">behave</a><br />
    Java - <a href="https://jbehave.org/">jbehave</a>

    <h3>Integration Testing, Client-side, headless</h3>
    <strong>Goal</strong> - code should behave consistently, only in the browser / mobile app, quickly.<br />
    Javascript - <a href="http://casperjs.org/">CasperJS </a>+ <a href="http://phantomjs.org/">PhantomJS</a><br />
    AngularJS - <a href="https://karma-runner.github.io/latest/index.html">Karma</a>, <a href="https://www.protractortest.org/#/">Protractor</a>

    <h3>BDD Testing, Integration, Client-side (ui devs)</h3>
    <strong>Goal</strong> - systems should behave consistently, on the client-side<br />
    Javascript - <a href="https://github.com/cucumber/cucumber-js">cucumber.js</a><br />
    jQuery - ???<br />
    Angularjs - <a href="https://docs.angularjs.org/guide/unit-testing#jasmine">Jasmine</a><br />
    React - <a href="https://github.com/pzavolinsky/react-cucumber">react-cucumber</a>

    <h3>BDD Testing, e2e in a Browser (project managers)</h3>
    <strong>Goal</strong> - systems should behave consistently, end-to-end, w/ Selenium<br />
    Java - <a href="https://github.com/cucumber/cucumber-jvm">Cucumber-jvm</a> + <a href="https://github.com/cucumber/cucumber-jvm/tree/master/junit">Cucumber-junit</a><br />
    Python - <a href="http://lettuce.it/tutorial/simple.html">Lettuce</a>

    <h3>Testing, Performance</h3>
    <strong>Goal</strong> - systems should be performant, and reliable.<br />
    <a href="https://jmeter.apache.org/">JMeter</a><br />
    <a href="https://www.blazemeter.com/">Blazemeter</a> - Hosted JMeter<br />
    <a href="https://gatling.io/">Gatling</a>

    <h3>BDD Testing, Security (security)</h3>
    <strong>Goal</strong> - systems should behave securely, end-to-end<br />
    Ruby - <a href="http://gauntlt.org/">Gauntlt</a><br />
    Python - <a href="https://github.com/F-Secure/mittn">Mittn</a><br />
    Java - <a href="https://continuumsecurity.net/bdd-security/">BDD-Security</a><br />
    <br />
    <br />
    &nbsp;
    <h2>References</h2>
    https://erik.doernenburg.com/2008/11/how-toxic-is-your-code/<br />
    <br />
    https://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis<br />
    <br />
    https://github.com/collections/clean-code-linters<br />
    <br />
    https://realpython.com/python-code-quality/<br />
    <br />
    https://www.aspectsecurity.com/uploads/downloads/2012/03/aspect-security-the-unfortunate-reality-of-insecure-libraries.pdf<br />
    <br />
    http://softwaretestingfundamentals.com/integration-testing/<br />
    <br />
    https://martinfowler.com/bliki/IntegrationTest.html<br />
    <br />
    https://www.youtube.com/watch?v=hQyXgKENDtg&amp;t=2321s<br />
    <br />
    https://developer.ibm.com/tutorials/d-bbd-guide-iac/<br />
    <br />
    https://www.softwaretestinghelp.com/behavior-driven-development-bdd-tools/<br />
    <br />
    https://docs.angularjs.org/guide/unit-testing<br />
    <br />
    https://scotch.io/tutorials/testing-angularjs-with-jasmine-and-karma-part-1<br />
    <br />
    https://reactjs.org/community/testing.html<br />
    <br />
    https://www.toolsqa.com/selenium-cucumber-framework/convert-selenium-test-into-cucumber-bdd-style-test/<br />
    <br />
    https://www.guru99.com/what-is-security-testing.html<br />
    &nbsp;