Best approach to customize changelog templates (e.g. ordering)

Question

I used a custom changelog template and want to switch to the much improved default ones, but customize them at the same time.

I would like to know what is the best approach to customize the default templates? Should I copy all templates and macros into my project and customize just the sections I want to differ from default? Or is there a way to just change one macro and keep the rest of the defaults? Sadly, I doubt it.

I would like to change:

the order of commit types (from alphabetic to Features, Bug Fixes, ...)
add a compare url to the heading of each tagged release
only display the first line of the commit, not the whole description

These changes need to be applied in

changes.md.j2
versioned_changes.md.j2

Can you give me a hint of how to achieve this?

I would like to know what is the best approach to customize the default templates? Should I copy all templates and macros into my project and customize just the sections I want to differ from default? Or is there a way to just change one macro and keep the rest of the defaults? Sadly, I doubt it.

Yes, a recursive copy from the defaults folder. I have considered the individual override but I feel it is more error prone than it seems upon outset. Will still think about it for the future.

When you go to modify the macros it may be better to make your own macros file my-macros.j2 and then just change the imports on the associated templates because as you want to consume the default templates in the future this would make it easier to drop-in and replace.

VIRTUAL_ENV=".venv"

cp -r "$VIRTUAL_ENV/lib/python3*/site-packages/semantic-release/data/templates/angular/md/ ./templates/

I would like to change:

the order of commit types (from alphabetic to Features, Bug Fixes, ...)

add a compare url to the heading of each tagged release

only display the first line of the commit, not the whole description

Note that 1 & 3 are planned for the future in the default template so I do have solutions for those already which I will provide below.

Can you give me a hint of how to achieve this?

First change to porting the default templates to user defined templates is to hardcode the changelog file name into the changelog template if you want to support an updating changelog.

diff a/templates/CHANGELOG.md.j2 b/templates/CHANGELOG.md.j2

  #}{%  set insertion_flag = ctx.changelog_insertion_flag
+ %}{%. set this_file = "CHANGELOG.md"
  %}{%  set unreleased_commits = ctx.history.unreleased | dictsort
  %}{%  set releases = ctx.history.released.values() | list
  %}{#
  #}{%  if ctx.changelog_mode == "init"
  %}{%    include ".components/changelog_init.md.j2"
  %}{#
  #}{%  elif ctx.changelog_mode == "update"
+ %}{%    set prev_changelog_file = this_file
- %}{%    set prev_changelog_file = ctx.prev_changelog_file
  %}{%    include ".components/changelog_update.md.j2"
  %}{#
  #}{%  endif

To accomplish goal 3, yes you should modify changes.md.j2, like so:

diff a/templates/.components/changes.md.j2 b/templates/.components/changes.md.j2

  #}{%    set commit_descriptions = []
  %}{#
  #}{%    for commit in ns.commits
+ %}{#      # Add reference links to the commit summary line
- %}{#      # Update the first line with reference links and if commit description
-           # has more than one line, add the rest of the lines
-           # NOTE: This is specifically to make sure to not hide contents
-           # of squash commits (until parse support is added)
  #}{%      set description = "- %s" | format(format_commit_summary_line(commit))
- %}{%      if commit.descriptions | length > 1
- %}{%        set description = "%s\n\n%s" | format(
-               description, commit.descriptions[1:] | join("\n\n")
-             )
- %}{%      endif
  %}{%      set description = description | autofit_text_width(max_line_width, hanging_indent)
  %}{{      commit_descriptions.append(description) | default("", true)
  }}{%    endfor

To accomplish goal 1, I accomplished this with a separate import but you wouldn't have to. Define the order in your custom macros file my-macros.j2. Then import and use it to lay out the version's changes in the order you desire in changes.md.j2.

diff a/templates/.components/my-macros.j2 b/templates/.components/my-macros.j2

+ {% set section_heading_translations = {
+     'feat': 'Features',
+     'fix': 'Bug Fixes',
+     'perf': 'Performance Improvements',
+     'docs': 'Documentation',
+     'build': 'Build System',
+     'refactor': 'Refactoring',
+     'test': 'Testing',
+     'ci': 'Continuous Integration',
+     'chore': 'Chores',
+     'style': 'Code Style',
+ } %}
+ 
+ {% set section_heading_order = section_heading_translations.values() %}

diff a/templates/.components/changes.md.j2 b/templates/.components/changes.md.j2

  {%    from 'macros.md.j2' import apply_alphabetical_ordering_by_brk_descriptions
  %}{%  from 'macros.md.j2' import apply_alphabetical_ordering_by_descriptions
  %}{%  from 'macros.md.j2' import format_breaking_changes_description, format_commit_summary_line
+ %}{%  from 'my-macros.j2' import section_heading_order
  %}{#
  #}{%  set max_line_width = max_line_width | default(100)
  %}{%  set hanging_indent = hanging_indent | default(2)
  %}{#
+ #}{%  for type_ in section_heading_order if type_ in commit_objects
- #}{%  for type_, commits in commit_objects if type_ != "unknown"
  %}{#    PREPROCESS COMMITS (order by description & format description line)
+ #}{%    set ns = namespace(commits=commit_objects[type_])
- #}{%    set ns = namespace(commits=commits)
  %}{{    apply_alphabetical_ordering_by_descriptions(ns) | default("", true)

For Goal 2, I would reference the code I use in the .release_notes.md.j2 template to generate the compare url prior to including the versioned_changes.md.j2 template in the changelog_init.md.j2 and the changelog_update.md.j2 files. The version_comparison_url value will exist in the environment once you use the set directive and then when the .versioned_changes.md.j2 template is rendered, you would reference the version_comparison_url for your formatting.

{# relevant exerpt from '.release_notes.md.j2' with detailed_changes_link changed to version_comparison_url #}
%}{%  set releases = context.history.released.values() | list
%}{%  set curr_release_index = releases.index(release)
%}{%  set prev_release_index = curr_release_index + 1
%}{#
#}{%  if 'compare_url' is filter and prev_release_index < releases | length
%}{%    set prev_version_tag = releases[prev_release_index].version.as_tag()
%}{%    set new_version_tag = release.version.as_tag()
%}{%    set version_compare_url = prev_version_tag | compare_url(new_version_tag)
%}{%    set version_comparison_url = '[{}...{}]({})'.format(
          prev_version_tag, new_version_tag, version_compare_url
        )
%}{%  endif

Note that releases is already defined in CHANGELOG.md.j2. compare_url filter is not available on all VCS platforms. The conditional here is to make sure to only provide a comparison url when there is more than 1 release (ie release 2 and greater). The use of Version.as_tag() is to ensure the version is formatted properly to the tag format configurable by the user. I leave the rest to you to figure out how to integrate this excerpt (maybe move the excerpt into a macro) into the version print loop of the changelog init template and the update template.

You may not need to worry about the conditional I use in the release notes template if you are maintaining the first release mechanism I have in the default template's update/init code. That is the edge case that does not work for a comparison url.

diff a/templates/.components/versioned_changes.md.j2 b/templates/.components/versioned_changes.md.j2

  #}{{
  
-       "## %s (%s)\n" | format(
+       "## %s (%s) %s\n" | format(
          release.version.as_semver_tag(),
-         release.tagged_date.strftime("%Y-%m-%d")
+         release.tagged_date.strftime("%Y-%m-%d"),
+         version_comparison_url,
        )
  
  }}{%  set commit_objects = release["elements"] | dictsort
  %}{%  include "changes.md.j2"
  -%}

Oh my god, thank you very much for this detailed answer. This is an early christmas present.
I will try the suggestions in the coming days and come back to you.

I will copy the default templates then, as suggested, and reference your project as the origin of the code.

As you pointed out, some of the changes are planned for a future release of the default templates. I will switch then, once those are available.

One more question: After version 9.10, is it mandatory to select update mode? I need to adapt to that as well.

You're welcome! Glad I could help.

I will copy the default templates then, as suggested, and reference your project as the origin of the code.

That's not necessary in this case but thanks!

As you pointed out, some of the changes are planned for a future release of the default templates. I will switch then, once those are available.

I wouldn't wait on me to make those available. I just wanted you to be aware of changes to the default templates that are coming. It takes me much longer to update default changelog templates because I have to make them compatible for all parser variants and the change type ordering is not parser agnostic. I did give you the intended changes for angular so it shouldn't require any changes on your end when they are released.

One more question: After version 9.10, is it mandatory to select update mode? I need to adapt to that as well.

It's definitely NOT mandatory. I consider a change to the default implementation a major breaking change. I added the functionality to enable an update mode as it was a highly desired feature that was dropped from v7 to v8 during the conversion to jinja based templates. You can choose to switch or say in init mode. There's a significant write up on the changelog templates page of the documentation. It is planned for v10 that the default mode will be switched to update so if you don't have it defined in your config as init then you could have unintended results. I believe update mode makes the most sense because then people can modify and adjust the changelog manually if the automated results didn't include all details desired to provide the end user. If PSR is in init mode then any changes are overwritten upon each new release. The update mode is also better for late adoption to an existing project where a dev team might not of used commit conventions or changed their convention along its history. Init mode assumes the latest standard and configuration applies across the entire project's history all the way back to the initial commit which is just not really feasible.

I think I am almost there.

Regarding your suggestion:

for type_ in section_heading_order if type_ in commit_objects

This does not seem to work. commit_objects is a list of tuples, right? So I cannot easily test if a string is part of commit_objects using in.

I am struggling to wrap my head around this. Any ideas?

I am struggling to wrap my head around this. Any ideas?

Oh yeah, I forgot to mention you need to remove the dictsort filter in versioned_changes.md.j2. That maintains it as a dictionary as opposed to converting it to a list of tuples. You no longer need dictsort because you are no longer alphabetically sorting the change categories.

You must also remove the dictsort from .release_notes.md.j2 as well.

And with the change from a list of tuples to a dictionary, that will change the processing of the breaking changes section. This should solve it:

diff a/templates/.components/changes.md.j2 b/templates/.components/changes.md.j2

  }}{{    "%s\n" | format(commit_descriptions | unique | join("\n\n"))
  }}{%  endfor
  %}{#
-       # Determine if there are any breaking change commits by filtering the list by breaking descriptions
-       # commit_objects is a list of tuples [("Features", [ParsedCommit(), ...]), ("Bug Fixes", [ParsedCommit(), ...])]
-       # HOW: Filter out breaking change commits that have no breaking descriptions
-       #  1. Re-map the list to only the list of commits under the breaking category from the list of tuples
-       #  2. Peel off the outer list to get a list of ParsedCommit objects
-       #  3. Filter the list of ParsedCommits to only those with a breaking description
- #}{%  set breaking_commits = commit_objects | map(attribute="1.0")
- %}{%  set breaking_commits = breaking_commits | rejectattr("error", "defined") | selectattr("breaking_descriptions.0") | list
+    #  # Determine if any commits have a breaking change
+    #  # commit_objects is a dictionary of strings to a list of commits { "Features", [ParsedCommit(), ...] }
+ #}{%  set breaking_commits = []
+ %}{%  for commits in commit_objects.values()
+ %}{#    # Filter out breaking change commits that have no breaking descriptions
+ #}{{    breaking_commits.extend(
+           commits | rejectattr("error", "defined") | selectattr("breaking_descriptions.0")
+         ) | default("", true)
+ }}{%  endfor
  %}{#
  #}{%  if breaking_commits | length > 0
  %}{#    PREPROCESS COMMITS

Thank you very much, I will try that. I ran exactly into the "now it does not work with a dictionary, because it expects a list of tuples" scenario. I did not want to ask again for help so soon. But now that you provided it anyway. Thanks a lot.

@afuetterer

@afuetterer, there was a lot of little changes and it all works together which is why I forgot in the first place. Not a problem. Hopefully it works now?

I will let you know, once I got it working

This issue has not received a response in 14 days. If no response is received in 7 days, it will be closed. We look forward to hearing from you.

This issue was closed because no response was received.

@codejedi365

@codejedi365, it took a while, but now I got it working. Thanks for all your suggestions. You can close this issue as completed, if you like. But the stalebot closed it as not planned in the meantime. Thanks again.

For future reference: My templates are located here: https://github.com/afuetterer/python-re3data/tree/main/.github/templates

@afuetterer

@afuetterer, glad to hear! Thanks for reporting back, templates look good and I like how you added installation instructions and such.

I know you just finished adapting the templates but just a heads up that in the next release there are some upgrades to the default templates you might want to include.

a new Additional Release Information section that is populated from parsing a NOTICE: git footer from commits (similar to Breaking Changes)
a license statement added to the release notes
a new create_pypi_url jinja filter
general error avoidance & performance within templates

Thanks a lot.

I will try the new features, once released. create_pypi_url seems pretty useful.

I ran into a couple of issues with the new default templates though. I will search the issue tracker for related findings or open new ones. For example I ran into trouble with "feat!: add new breaking feature".

For example I ran into trouble with "feat!: add new breaking feature".

With that example, if you are suggesting there was no breaking changes section that would be expected as the commit does not include any breaking change description in the body BREAKING CHANGE: .... The point is to actually describe what the user now has to do differently. The initial subject line will just be placed under the features section as it should be the subject line of what feature was added.

A proper breaking change commit looks like:

feat!: added support for element

BREAKING CHANGE: the API no longer  allows raw string values but instead requires JSON serialized objects. Follow the migration docs to convert your client to provide the expected JSON values

In my opinion, the ! character is only a visual reference cue if quickly evaluating a set of commits rather than looking for which has the commit body element.

Navigation Menu

Repository navigation

Best approach to customize changelog templates (e.g. ordering) #1122

Question

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Best approach to customize changelog templates (e.g. ordering) #1122

Description

Question

Activity

codejedi365 commented on Dec 13, 2024

afuetterer commented on Dec 14, 2024

codejedi365 commented on Dec 14, 2024

afuetterer commented on Dec 16, 2024

codejedi365 commented on Dec 16, 2024

codejedi365 commented on Dec 17, 2024

afuetterer commented on Dec 17, 2024

codejedi365 commented on Dec 17, 2024

afuetterer commented on Dec 22, 2024

github-actions commented on Jan 6, 2025

github-actions commented on Jan 13, 2025

afuetterer commented on Feb 2, 2025

codejedi365 commented on Feb 2, 2025

afuetterer commented on Feb 5, 2025

codejedi365 commented on Feb 5, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions