Update github actions documentation (#606)

* Update github actions documentation * Update tips * Addressed comments * Address comments * Added file widget * Update affected-stacks.mdx
cloudposse · Jun 6, 2024 · 4ea538d · 4ea538d
1 parent f58a540
commit 4ea538d
Show file tree

Hide file tree

Showing 8 changed files with 370 additions and 294 deletions.
diff --git a/...rations/github-actions/affected-stacks.md → ...ations/github-actions/affected-stacks.mdx b/...rations/github-actions/affected-stacks.md → ...ations/github-actions/affected-stacks.mdx
@@ -4,6 +4,8 @@ sidebar_position: 30
 sidebar_label: Affected Stacks
 ---
 
+import File from '@site/src/components/File'
+
 **Streamline Your Change Management Process**
 
 The [Atmos Affected Stacks GitHub Action](https://github.com/cloudposse/github-action-atmos-affected-stacks) empowers you to easily identify the affected [atmos stacks](/core-concepts/stacks/) for a pull request so you can gain better insights into the impact of your pull requests. It works by installing Atmos and running [`atmos describe affected`](/cli/commands/describe/affected), and outputs a comprehensive list of affected stacks, both as raw output and as a matrix to be used in subsequent GitHub action jobs.
@@ -23,9 +25,9 @@ Atmos checks component folders for changes first, marking all related components
 
 ## Usage Example
 
+<File title=".github/workflows/atmos-terraform-plan.yaml">
 ```yaml
 name: Pull Request
-
 on:
   pull_request:
     branches: [ 'main' ]
@@ -35,11 +37,42 @@ jobs:
   context:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
       - id: affected
-        uses: cloudposse/[email protected]
+        uses: cloudposse/github-action-atmos-affected-stacks@v3
+        with:
+          atmos-config-path: ./rootfs/usr/local/etc/atmos/
+          atmos-version: 1.63.0
+          nested-matrices-count: 1
 
     outputs:
       affected: ${{ steps.affected.outputs.affected }}
       matrix: ${{ steps.affected.outputs.matrix }}
+
+  # This job is an example how to use the affected stacks with the matrix strategy
+  atmos-plan:
+    needs: ["atmos-affected"]
+    if: ${{ needs.atmos-affected.outputs.has-affected-stacks == 'true' }}
+    name: ${{ matrix.stack_slug }}
+    runs-on: ['self-hosted']
+    strategy:
+      max-parallel: 10
+      fail-fast: false # Don't fail fast to avoid locking TF State
+      matrix: ${{ fromJson(needs.atmos-affected.outputs.matrix) }}
+    ## Avoid running the same stack in parallel mode (from different workflows)
+    concurrency:
+      group: ${{ matrix.stack_slug }}
+      cancel-in-progress: false
+    steps:
+      - name: Plan Atmos Component
+        uses: cloudposse/github-action-atmos-terraform-plan@v2
+        with:
+          component: ${{ matrix.component }}
+          stack: ${{ matrix.stack }}
+          atmos-config-path: ./rootfs/usr/local/etc/atmos/
+          atmos-version: 1.63.0
 ```
+</File>
+
+## Requirements
+
+This action has the requirements as [Github Actions](/integrations/github-actions/). Use the same config described there.
diff --git a/...s/github-actions/atmos-terraform-apply.md → .../github-actions/atmos-terraform-apply.mdx b/...s/github-actions/atmos-terraform-apply.md → .../github-actions/atmos-terraform-apply.mdx
@@ -4,6 +4,8 @@ sidebar_position: 50
 sidebar_label: Terraform Apply
 ---
 
+import File from '@site/src/components/File'
+
 The Cloud Posse GitHub Action for "Atmos Terraform Apply" simplifies provisioning Terraform entirely within GitHub Action workflows. It makes it very easy to understand exactly what happened directly within the GitHub UI.
 
 Given any component and stack in an Atmos supported infrastructure environment, [`github-action-atmos-terraform-apply`](https://github.com/cloudposse/github-action-atmos-terraform-apply) will retrieve an existing Terraform [planfile](https://developer.hashicorp.com/terraform/tutorials/automation/automate-terraform) from a given S3 bucket using metadata stored inside a DynamoDB table, run `atmos terraform apply` with that planfile, and format the Terraform Apply result as part of a [GitHub Workflow Job Summary](https://github.blog/2022-05-09-supercharging-github-actions-with-job-summaries/).
@@ -27,7 +29,7 @@ In the following screenshot, we see a successful "apply" Job Summary report. The
 
 ![Example Image](/img/github-actions/apply.png)
 
-## Usage
+## Usage Example
 
 In this example, the action is triggered when certain events occur, such as a manual workflow dispatch or the opening, synchronization, or reopening of a pull request, specifically on the main branch. It specifies specific permissions related to assuming roles in AWS. Within the "apply" job, the "component" and "stack" are hardcoded (`foobar` and `plat-ue2-sandbox`). In practice, these are usually derived from another action. 
 
@@ -37,53 +39,38 @@ We recommend combining this action with the [`affected-stacks`](/integrations/gi
 
 :::
 
-
-```yaml
-  # .github/workflows/atmos-terraform-apply.yaml
-  name: "atmos-terraform-apply"
-
-  on:
-    workflow_dispatch:
-    pull_request:
-      types:
-        - closed
-      branches:
-        - main
-
-  # These permissions are required for GitHub to assume roles in AWS
-  permissions:
-    id-token: write
-    contents: read
-
-  jobs:
-    apply:
-      runs-on: ubuntu-latest
-      steps:
-        - name: Terraform Apply
-          uses: cloudposse/github-action-atmos-terraform-apply@v2
-          with:
-            component: "foobar"
-            stack: "plat-ue2-sandbox"
-```
-
-with the following configuration as an example:
-
+<File title=".github/workflows/atmos-terraform-apply.yaml">
 ```yaml
-  # .github/config/atmos-gitops.yaml
-  atmos-config-path: ./rootfs/usr/local/etc/atmos/
-  atmos-version: 1.65.0
-  aws-region: us-east-2
-  enable-infracost: false
-  group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-")
-  sort-by: .stack_slug
-  terraform-apply-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops
-  terraform-plan-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops
-  terraform-state-bucket: cptest-core-ue2-auto-gitops
-  terraform-state-role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha
-  terraform-state-table: cptest-core-ue2-auto-gitops
-  terraform-version: 1.65.0
+name: "atmos-terraform-apply"
+
+on:
+  workflow_dispatch:
+  pull_request:
+    types:
+      - closed
+    branches:
+      - main
+
+# These permissions are required for GitHub to assume roles in AWS
+permissions:
+  id-token: write
+  contents: read
+
+jobs:
+  apply:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Terraform Apply
+        uses: cloudposse/github-action-atmos-terraform-apply@v2
+        with:
+          component: "foobar"
+          stack: "plat-ue2-sandbox"
+          sha: ${{ github.sha }}          
+          atmos-config-path: ./rootfs/usr/local/etc/atmos/
+          atmos-version: 1.63.0
 ```
+</File>
 
 ## Requirements
 
-This action has the same requirements as [Atmos Terraform Plan](/integrations/github-actions/atmos-terraform-plan). Use the same S3 Bucket, DynamoDB table, and IAM Roles created with the requirements described there.
+This action has the requirements as [Github Actions](/integrations/github-actions/). Use the same S3 Bucket, DynamoDB table, IAM Roles and config described there.
diff --git a/...ctions/atmos-terraform-drift-detection.md → ...tions/atmos-terraform-drift-detection.mdx b/...ctions/atmos-terraform-drift-detection.md → ...tions/atmos-terraform-drift-detection.mdx
@@ -4,6 +4,8 @@ sidebar_position: 60
 sidebar_label: Terraform Drift Detection
 ---
 
+import File from '@site/src/components/File'
+
 The Cloud Posse GitHub Action for "Atmos Terraform Drift Detection" and "Atmos Terraform Drift Remediation" define a scalable pattern for detecting and remediating Terraform drift from within GitHub using workflows and Issues. "Atmos Terraform Drift Detection" will determine drifted Terraform state by running [Atmos Terraform Plan](/integrations/github-actions/atmos-terraform-plan) and creating GitHub Issues for any drifted component and stack. Furthermore, "Atmos Terraform Drift Remediation" will run [Atmos Terraform Apply](/integrations/github-actions/atmos-terraform-apply) for any open Issue if called and close the given Issue. With these two actions, we can fully support drift detection for Terraform directly within the GitHub UI.
 
 This action is intended to be used together with [Atmos Terraform Plan](/integrations/github-actions/atmos-terraform-plan) and [Atmos Terraform Apply](/integrations/github-actions/atmos-terraform-apply).
@@ -20,7 +22,7 @@ This GitHub Action incorporates superior GitOps support for Terraform by utilizi
 * **Free Tier GitHub** Use GitHub Issues to track drifted resources.
 * **100% Open Source with Permissive APACHE2 License** means you have no expensive subscriptions or long-term commitments.
 
-## Usage
+## Usage Example
 
 ```mermaid
 ---
@@ -82,70 +84,69 @@ We can quickly see a complete list of all drift components in the "Issues" tab i
 
 ![Example Issue List](/img/github-actions/drift-issue-list.png)
 
-#### Example Usage
+#### Usage Example
 
+<File title=".github/workflows/atmos-terraform-drfit-detection.yaml">
 ```yaml
-  name: 👽 Atmos Terraform Drift Detection
-
-  on:
-    schedule:
-      - cron: "0 * * * *"
-
-  permissions:
-    id-token: write
-    contents: write
-    issues: write
-
-  jobs:
-    select-components:
-      runs-on: ubuntu-latest
-      name: Select Components
-      outputs:
-        matrix: ${{ steps.components.outputs.matrix }}
-      steps:
-        - name: Selected Components
-          id: components
-          uses: cloudposse/github-action-atmos-terraform-select-components@v0
-          with:
-            jq-query: 'to_entries[] | .key as $parent | .value.components.terraform | to_entries[] | select(.value.settings.github.actions_enabled // false) | [$parent, .key] | join(",")'
-            debug: ${{ env.DEBUG_ENABLED }}
-
-    plan-atmos-components:
-      needs:
-        - select-components
-      runs-on: ubuntu-latest
-      if: ${{ needs.select-components.outputs.matrix != '{"include":[]}' }}
-      strategy:
-        fail-fast: false # Don't fail fast to avoid locking TF State
-        matrix: ${{ fromJson(needs.select-components.outputs.matrix) }}
-      name: ${{ matrix.stack_slug }}
-      env:
-        GITHUB_TOKEN: "${{ github.token }}"
-      steps:
-        - name: Plan Atmos Component
-          id: atmos-plan
-          uses: cloudposse/github-action-atmos-terraform-plan@v0
-          with:
-            component: ${{ matrix.component }}
-            stack: ${{ matrix.stack }}
-            component-path: ${{ matrix.component_path }}
-            drift-detection-mode-enabled: "true"
-            terraform-plan-role: "arn:aws:iam::111111111111:role/acme-core-gbl-identity-gitops"
-            terraform-state-bucket: "acme-core-ue2-auto-gitops"
-            terraform-state-role: "arn:aws:iam::999999999999:role/acme-core-ue2-auto-gitops-gha"
-            terraform-state-table: "acme-core-ue2-auto-gitops"
-            aws-region: "us-east-2"
-
-    drift-detection:
-      needs:
-        - plan-atmos-components
-      runs-on: ubuntu-latest
-      steps:
-        - name: Drift Detection
-          uses: cloudposse/github-action-atmos-terraform-drift-detection@v0
-          with:
-            max-opened-issues: '3'
+name: 👽 Atmos Terraform Drift Detection
+
+on:
+  schedule:
+    - cron: "0 * * * *"
+
+permissions:
+  id-token: write
+  contents: write
+  issues: write
+
+jobs:
+  select-components:
+    name: Select Components
+    runs-on: ubuntu-latest
+    steps:
+      - name: Selected Components
+        id: components
+        uses: cloudposse/github-action-atmos-terraform-select-components@v2
+        with:
+          select-filter: '.settings.github.actions_enabled and .metadata.type != "abstract"'
+          atmos-config-path: ./rootfs/usr/local/etc/atmos/
+          atmos-version: 1.63.0
+    outputs:
+      stacks: ${{ steps.components.outputs.matrix }}
+      has-selected-components: ${{ steps.components.outputs.has-selected-components }}
+
+  plan-atmos-components:
+    needs: ["select-components"]
+    runs-on: ubuntu-latest
+    if: ${{ needs.select-components.outputs.has-selected-components == 'true' }}
+    name: Detect Drift (${{ matrix.name }})
+    uses: ./.github/workflows/atmos-terraform-plan-matrix.yaml
+    strategy:
+      max-parallel: 1 # This is important to avoid ddos GHA API
+      fail-fast: false # Don't fail fast to avoid locking TF State
+      matrix: ${{ fromJson(needs.select-components.outputs.stacks) }}
+    with:
+      stacks: ${{ matrix.items }}
+      sha: ${{ github.sha }}
+      drift-detection-mode-enabled: "true"
+      continue-on-error: true
+      atmos-config-path: ./rootfs/usr/local/etc/atmos/
+      atmos-version: 1.63.0
+    secrets: inherit
+
+  drift-detection:
+    needs: ["plan-atmos-components"]
+    if: always()
+    name: Reconcile issues
+    runs-on: ubuntu-latest
+    steps:
+      - name: Drift Detection
+        uses: cloudposse/github-action-atmos-terraform-drift-detection@v1
+        with:
+          max-opened-issues: '4'
+          process-all: 'true'
 ```
+</File>
 
 #### 256 Matrix Limitation
 
@@ -199,8 +200,10 @@ stateDiagram-v2
 
 Once we have an open Issue for a drifted component, we can trigger another workflow to remediate the drifted Terraform resources. When an Issue is labeled with `apply`, the "Atmos Terraform Drift Remediation" workflow will take the component and stack in the given Issue and run `atmos terraform apply <component> --stack <stack>` using the latest Terraform Planfile. If the apply is successful, the workflow will close the given Issue as resolved.
 
-#### Example Usage
+#### Usage Example
+
 
+<File title=".github/workflows/atmos-terraform-drfit-remediation.yaml">
 ```yaml
 name: 👽 Atmos Terraform Drift Remediation
 run-name: 👽 Atmos Terraform Drift Remediation
@@ -224,11 +227,12 @@ jobs:
       contains(join(github.event.issue.labels.*.name, ','), 'apply')
     steps:
       - name: Remediate Drift
-        uses: cloudposse/github-action-atmos-terraform-drift-remediation@v1
+        uses: cloudposse/github-action-atmos-terraform-drift-remediation@v2
         with:
           issue-number: ${{ github.event.issue.number }}
           action: remediate
-          atmos-gitops-config-path: ./.github/config/atmos-gitops.yaml
+          atmos-config-path: ./rootfs/usr/local/etc/atmos/
+          atmos-version: 1.63.0
 
   discard-drift:
     runs-on: ubuntu-latest
@@ -238,13 +242,15 @@ jobs:
       !contains(join(github.event.issue.labels.*.name, ','), 'remediated')
     steps:
       - name: Discard Drift
-        uses: cloudposse/github-action-atmos-terraform-drift-remediation@v1
+        uses: cloudposse/github-action-atmos-terraform-drift-remediation@v2
         with:
           issue-number: ${{ github.event.issue.number }}
           action: discard
-          atmos-gitops-config-path: ./.github/config/atmos-gitops.yaml
+          atmos-config-path: ./rootfs/usr/local/etc/atmos/
+          atmos-version: 1.63.0
 ```
+</File>
 
 ## Requirements
 
-This action has the same requirements as [Atmos Terraform Plan](/integrations/github-actions/atmos-terraform-plan). Use the same S3 Bucket, DynamoDB table, and IAM Roles created with the requirements described there.
+This action has the requirements as [Github Actions](/integrations/github-actions/). Use the same S3 Bucket, DynamoDB table, IAM Roles and config described there.