ci: write down breaking change policy (#1294)

* Enforce conventional commit PR titles
* Add automatic labelling of PRs
* Write down breaking change policy.

Left for another PR:
* Validation of breaking change version bumps. (This is complicated due
to separate releases for Python and other package.)

.github/labeler.yml

@@ -0,0 +1,33 @@
+version: 1
+appendOnly: true
+# Labels are applied based on conventional commits standard
+# https://www.conventionalcommits.org/en/v1.0.0/
+# These labels are later used in release notes. See .github/release.yml
+labels:
+# If the PR title has an ! before the : it will be considered a breaking change
+# For example, `feat!: add new feature` will be considered a breaking change
+- label: breaking-change
+  title: "^[^:]+!:.*"
+- label: breaking-change
+  body: "BREAKING CHANGE"
+- label: enhancement
+  title: "^feat(\\(.+\\))?!?:.*"
+- label: bug
+  title: "^fix(\\(.+\\))?!?:.*"
+- label: documentation
+  title: "^docs(\\(.+\\))?!?:.*"
+- label: performance
+  title: "^perf(\\(.+\\))?!?:.*"
+- label: ci
+  title: "^ci(\\(.+\\))?!?:.*"
+- label: chore
+  title: "^(chore|test|build|style)(\\(.+\\))?!?:.*"
+- label: Python
+  files:
+    - "^python\\/.*"
+- label: Rust
+  files:
+    - "^rust\\/.*"
+- label: typescript
+  files:
+    - "^node\\/.*"

.github/release.yml

@@ -0,0 +1,25 @@
+# TODO: create separate templates for Python and other releases.
+changelog:
+  exclude:
+    labels:
+      - ci
+      - chore
+  categories:
+    - title: Breaking Changes 🛠
+      labels:
+        - breaking-change
+    - title: New Features 🎉
+      labels:
+        - enhancement
+    - title: Bug Fixes 🐛
+      labels:
+        - bug
+    - title: Documentation 📚
+      labels:
+        - documentation
+    - title: Performance Improvements 🚀
+      labels:
+        - performance
+    - title: Other Changes
+      labels:
+        - "*"

.github/workflows/dev.yml

@@ -0,0 +1,81 @@
+name: PR Checks
+
+on:
+  pull_request_target:
+    types: [opened, edited, synchronize, reopened]
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  labeler:
+    permissions:
+      pull-requests: write
+    name: Label PR
+    runs-on: ubuntu-latest
+    steps:
+      - uses: srvaroa/labeler@master
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+  commitlint:
+    permissions:
+      pull-requests: write
+    name: Verify PR title / description conforms to semantic-release
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/setup-node@v3
+        with:
+          node-version: "18"
+      # These rules are disabled because Github will always ensure there
+      # is a blank line between the title and the body and Github will
+      # word wrap the description field to ensure a reasonable max line
+      # length.
+      - run: npm install @commitlint/config-conventional
+      - run: >
+          echo 'module.exports = {
+            "rules": {
+              "body-max-line-length": [0, "always", Infinity],
+              "footer-max-line-length": [0, "always", Infinity],
+              "body-leading-blank": [0, "always"]
+            }
+          }' > .commitlintrc.js
+      - run: npx commitlint --extends @commitlint/config-conventional --verbose <<< $COMMIT_MSG
+        env:
+          COMMIT_MSG: >
+            ${{ github.event.pull_request.title }}
+
+            ${{ github.event.pull_request.body }}
+      - if: failure()
+        uses: actions/github-script@v6
+        with:
+          script: |
+            const message = `**ACTION NEEDED**
+              
+              Lance follows the [Conventional Commits specification](https://www.conventionalcommits.org/en/v1.0.0/) for release automation.
+
+              The PR title and description are used as the merge commit message.\
+              Please update your PR title and description to match the specification.
+
+              For details on the error please inspect the "PR Title Check" action.
+              `
+            // Get list of current comments
+            const comments = await github.paginate(github.rest.issues.listComments, {
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              issue_number: context.issue.number
+            });
+            // Check if this job already commented
+            for (const comment of comments) {
+              if (comment.body === message) {
+                return // Already commented
+              }
+            }
+            // Post the comment about Conventional Commits
+            github.rest.issues.createComment({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              issue_number: context.issue.number,
+              body: message
+            })
+            core.setFailed(message)

release_process.md

@@ -0,0 +1,44 @@
+# Release process
+
+There are five total packages we release. Three are the `lancedb` packages
+for Python, Rust, and Node.js. The other two are the legacy `vectordb`
+packages for Rust and node.js.
+
+The Python package is versioned and released separately from the Rust and Node.js
+ones. For Rust and Node.js, the release process is shared between `lancedb` and
+`vectordb` for now.
+
+## Breaking changes
+
+We try to avoid breaking changes, but sometimes they are necessary. When there
+are breaking changes, we will increment the minor version. (This is valid 
+semantic versioning because we are still in `0.x` versions.)
+
+When a PR makes a breaking change, the PR author should mark the PR using the 
+conventional commit markers: either exclamation mark after the type
+(such as `feat!: change signature of func`) or have `BREAKING CHANGE` in the
+body of the PR. A CI job will add a `breaking-change` label to the PR, which is
+what will ultimately be used to CI to determine if the minor version should be
+incremented.
+
+A CI job will validate that if a `breaking-change` label is added, the minor
+version is incremented in the `Cargo.toml` and `pyproject.toml` files. The only
+exception is if it has already been incremented since the last stable release.
+
+**It is the responsibility of the PR author to increment the minor version when
+appropriate.**
+
+Some things that are considered breaking changes:
+
+* Upgrading `lance` to a new minor version. Minor version bumps in Lance are
+  considered breaking changes during `0.x` releases. This can change behavior
+  in LanceDB.
+* Upgrading a dependency pin that is in the Rust API. In particular, upgrading
+  `DataFusion` and `Arrow` are breaking changes. Changing dependencies that are
+  not exposed in our public API are not considered breaking changes.
+* Changing the signature of a public function or method.
+* Removing a public function or method.
+
+We do make exceptions for APIs that are marked as experimental. These are APIs
+that are under active development and not in major use. These changes should not
+receive the `breaking-change` label.