Skip to content

Commit

Permalink
Update workflows for 0.7.0 & new release workflow
Browse files Browse the repository at this point in the history
Includes template update for now optional version & modified date
  • Loading branch information
dalito committed Aug 17, 2023
1 parent 0390115 commit 879d9d9
Show file tree
Hide file tree
Showing 6 changed files with 156 additions and 24 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/ci-pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ jobs:
python -m pip install --upgrade pip setuptools wheel
# install tagged version
python -m pip install git+https://github.com/nfdi4cat/voc4cat-tool.git@v0.6.2
python -m pip install git+https://github.com/nfdi4cat/voc4cat-tool.git@v0.7.0
# python -m pip install git+https://github.com/nfdi4cat/voc4cat-tool.git@main
# Example for installing from private repo:
Expand Down Expand Up @@ -98,7 +98,7 @@ jobs:
voc4cat transform --split --inplace --logfile outbox/voc4cat.log --outdir outbox_new_voc outbox/
merge_vocab --logfile outbox/voc4cat.log outbox_new_voc/ vocabularies/
# copy xlsx to outbox so that the xlsx file is part of the artifact
cp $(find inbox-excel-vocabs -name "*.xlsx") outbox/
find inbox-excel-vocabs -name '*.xlsx' -exec cp {} -t outbox/ \;
git status
- name: Store artifacts
Expand Down
101 changes: 101 additions & 0 deletions .github/workflows/merge.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# This action runs on merges to main.
#
# Workflow steps:
# - checkout gh-pages for updating
# - create joined vocabulary file in addition to split version
# - create excel-file from turtle
# - build docs
# - publish docs, vocabulary-turtle files and excel-file to gh-pages

name: Build & Publish
on:
push:
branches:
- main
workflow_dispatch:

env:
FORCE_COLOR: "1" # Make tool output pretty.
PIP_DISABLE_PIP_VERSION_CHECK: "1"
PIP_PROGRESS_BAR: "off"
LOGLEVEL: "DEBUG"

jobs:
build:
name: Development build of vocabulary & documentation
permissions:
# Required for peaceiris/actions-gh-pages below
contents: write
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v3

- name: Checkout gh-pages branch to dir publish/
uses: actions/checkout@v3
with:
ref: gh-pages
path: publish
fetch-depth: 1

- uses: actions/setup-python@v4
with:
python-version: '3.11'

- name: Install dependencies
run: |
python -VV
python -m pip install --upgrade pip setuptools wheel
# install tagged version
python -m pip install git+https://github.com/nfdi4cat/[email protected]
# python -m pip install git+https://github.com/nfdi4cat/voc4cat-tool.git@main
- name: Set dynamic environment variables.
run: |
echo "VOC4CAT_MODIFIED=$(date +'%Y-%m-%d')" >> $GITHUB_ENV
# set first 8 chars of commit hash as version for "dev"
echo "VOC4CAT_VERSION=v_${GITHUB_SHA:0:8}" >> $GITHUB_ENV
- name: Run voc4cat (publish joined & split SKOS/turtle)
run: |
voc4cat --version
mkdir -p publish/dev/
# delete files xlsx and ttl produced in previous workflow runs
find publish/dev/ -type f \( -name "*.xlsx" -o -name "*.ttl" \) -delete
cp -r vocabularies/. publish/dev/
# Build joined turtle
voc4cat transform --logfile publish/dev/voc4cat.log --join publish/dev/
- name: Run voc4cat (build HTML documentation)
run: |
voc4cat docs --force --logfile publish/dev/voc4cat.log publish/dev/
- name: Run voc4cat (build current Excel file)
run: |
voc4cat convert --logfile publish/dev/voc4cat.log --template templates/voc4cat_template_043.xlsx publish/dev/
- name: Deploy updated gh-pages content
# This replaces all prior content in gh-pages branch. But we have
# checked out the gh-pages branch above so that we keep all prior
# content and just re-publish the extended version.
# Pin third party action (v3.9.3)
uses: peaceiris/actions-gh-pages@373f7f263a76c20808c831209c920827a82a2847
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
with:
publish_branch: gh-pages
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./publish
force_orphan: true

- name: Store publish dir (=pages) as artifact
# This step is not required and may be removed.
# It may be helpful for trouble shooting.
if: ${{ always() }}
uses: actions/upload-artifact@v3
with:
name: voc4cat-pages-dev-content
path: publish/

# Lit:
# https://github.com/peaceiris/actions-gh-pages
51 changes: 36 additions & 15 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# This action runs when a release is created and on merges to main.
# This action runs when a release is created via tagging or via GitHub UI.
#
# Workflow steps:
# - checkout gh-pages for updating
# - create joined vocabulary file in addition to split version
# - create excel-file from turtle
# - build docs
Expand All @@ -9,21 +10,21 @@
name: Build & Publish
on:
push:
branches:
- main
tags:
- v[0-9]+.[0-9]+.[0-9]+
workflow_dispatch:
- 'v[0-9]+-[0-9]+-[0-9]+'
release:
types: [published]

env:
FORCE_COLOR: "1" # Make tool output pretty.
PIP_DISABLE_PIP_VERSION_CHECK: "1"
PIP_PROGRESS_BAR: "off"
LOGLEVEL: "DEBUG"
VOC4CAT_VERSION: ${{github.ref_name}} # use release tag as version

jobs:
build:
name: Build vocabulary & documentation
name: Release vocabulary & documentation
permissions:
# Required for peaceiris/actions-gh-pages below
contents: write
Expand All @@ -32,6 +33,13 @@ jobs:
steps:
- uses: actions/checkout@v3

- name: Checkout gh-pages branch to dir publish/
uses: actions/checkout@v3
with:
ref: gh-pages
path: publish
fetch-depth: 1

- uses: actions/setup-python@v4
with:
python-version: '3.11'
Expand All @@ -42,29 +50,42 @@ jobs:
python -m pip install --upgrade pip setuptools wheel
# install tagged version
python -m pip install git+https://github.com/nfdi4cat/voc4cat-tool.git@v0.6.2
python -m pip install git+https://github.com/nfdi4cat/voc4cat-tool.git@v0.7.0
# python -m pip install git+https://github.com/nfdi4cat/voc4cat-tool.git@main
- name: Set dynamic environment variables.
run: |
echo "VOC4CAT_MODIFIED=$(date +'%Y-%m-%d')" >> $GITHUB_ENV
- name: Run voc4cat (publish joined & split SKOS/turtle)
run: |
voc4cat --version
cp -r vocabularies/ publish/
mkdir -p publish/latest/
# delete files xlsx and ttl produced in previous workflow runs
find publish/latest/ -type f \( -name "*.xlsx" -o -name "*.ttl" \) -delete
cp -r vocabularies/. publish/latest/
# Build joined turtle
voc4cat transform --logfile publish/voc4cat.log --join publish/
voc4cat transform --logfile publish/latest/voc4cat.log --join publish/latest/
- name: Run voc4cat (build HTML documentation)
run: |
voc4cat docs --force --logfile publish/voc4cat.log publish/
voc4cat docs --force --logfile publish/latest/voc4cat.log publish/latest/
- name: Run voc4cat (build current Excel file)
run: |
voc4cat convert --logfile publish/voc4cat.log --template templates/voc4cat_template_043.xlsx publish/
ls -l publish/ || echo "publish directory does not exist."
voc4cat convert --logfile publish/latest/voc4cat.log --template templates/voc4cat_template_043.xlsx publish/latest/
- name: Copy release to release-name directory
run: |
mkdir -p publish/${{ github.ref_name }}
cp -r publish/latest/. publish/${{ github.ref_name }}/
- name: Deploy documentation & SKOS/turtle & xlsx
- name: Deploy updated gh-pages content
# This replaces all prior content in gh-pages branch. But we have
# checked out the gh-pages branch above so that we keep all prior
# content and just re-publish the extended version.
# Pin third party action (v3.9.3)
uses: peaceiris/actions-gh-pages@373f7f263a76c20808c831209c920827a82a2847
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
with:
publish_branch: gh-pages
github_token: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -77,7 +98,7 @@ jobs:
if: ${{ always() }}
uses: actions/upload-artifact@v3
with:
name: voc4cat-pages-content
name: voc4cat-pages-release-${{ github.ref_name }}
path: publish/

# Lit:
Expand Down
20 changes: 13 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,17 @@ To validate IRIs the configuration supports ID-ranges (similar to [OBO idrange](
The idea is that every author gets their own range of IDs to consume.
This allows independent work and avoids using the same ID repeatedly.

In addition, HTML-documentation is created automatically and published on GitHub-pages.
The voc4cat-template implements automatic storage of different versions of the vocabularies in gh-pages:

- `dev` - Directory with artifacts built from the most recent commit to the main branch.
- `latest` - Directory with all files built for the latest release.
- `vYYYY-MM-DD` (for example `v2023-08-16`) - Directory with all files built for the release with this tag.

For all versions, multiple files are stored (see https://github.com/nfdi4cat/voc4cat-template/issues/11#issuecomment-1680592185 for details). The correct version string is automatically inserted to all build artifacts. For `dev`, the first eight characters of the commit hash are used as version (for example `v_fadfa5f9`).

- Taking into account the above scheme, the url for the artifacts for the `dev` version in gh-pages is `https://{gh-org-name}.github.io/{repository-name}/dev/{vocabulary-name}/`
- For example, in repository `nfdi4cat/voc4cat-template` the vocabulary `vocab_example` is documented at [https://nfdi4cat.github.io/voc4cat-template/dev/vocab_example/](https://nfdi4cat.github.io/voc4cat-template/dev/vocab_example/)

- The general url is `https://{gh-org-name}.github.io/{repository-name}/{vocabulary-name}/`
- In repository `nfdi4cat/voc4cat-template` the vocabulary `vocab_example` is documented at [https://nfdi4cat.github.io/voc4cat-template/vocab_example/](https://nfdi4cat.github.io/voc4cat-template/vocab_example/)

### Creating vocabularies for catalysis or catalytic reaction engineering

Please strongly consider contributing to [voc4cat](https://github.com/nfdi4cat/voc4cat) instead of creating your own.
Expand All @@ -43,9 +49,9 @@ To contribute new concepts or collections or change existing ones, you may eithe
Here are the steps for submitting updates in Excel.

- Get the Excel/xlsx-vocabulary file
- The latest version of the vocabulary is always available via github-pages.
- The general url is `https://{gh-org-name}.github.io/{repository-name}/{vocabulary-name}.xlsx`
- For example in nfdi4cat/voc4cat-template the vocabulary `vocab_example` can be downloaded from [https://nfdi4cat.github.io/voc4cat-template/vocab_example.xlsx](https://nfdi4cat.github.io/voc4cat-template/vocab_example.xlsx)
- The most recent version of the vocabulary is always available via github-pages.
- The general url is `https://{gh-org-name}.github.io/{repository-name}/dev/{vocabulary-name}.xlsx`
- For example in nfdi4cat/voc4cat-template the most recent vocabulary `vocab_example` can be downloaded from [https://nfdi4cat.github.io/voc4cat-template/dev/vocab_example.xlsx](https://nfdi4cat.github.io/voc4cat-template/dev/vocab_example.xlsx)
- For setting up a new vocabulary, use the xlsx-file from the templates-folder.
- Make changes to the Excel file
- Add the xlsx file to your clone of the repository into the folder `inbox-excel-vocabs`
Expand Down
4 changes: 4 additions & 0 deletions templates/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ The templates here match structure-wise with the templates in [Voc4Cat-tool](htt

### Version 0.4.3

#### Revision 2023-08b

- Cells version and modified date in concept scheme sheet are now optional. Updated notes accordingly.

#### Revision 2023-08a

- Changed included example data to be consistent with improved validation.
Expand Down
Binary file modified templates/voc4cat_template_043.xlsx
Binary file not shown.

0 comments on commit 879d9d9

Please sign in to comment.