'true a-z' makes 'Name ABC' comes earlier than 'Name'

[cmasfo]

This happens because space( ) comes earlier than dot(.) of the file extension part(.md).
Same things happen in vs code.
how do i solve this?

[SebastianMC]

Hi @cmasfo

a good catch!

It looks for me as a retro-bug which slipped in as the sorting was getting more and more complex.
Indeed the fullname (basename + extension) is being used for sorting of files unconditionally in all cases, which results in the reported problem. It applies for both true alphabetical (true a-z) and plain alphanumeric (a-z) sorting.
The original intention was to use the basename of notes (files) for sorting, until explicitly indicated to use full filename in sorting specification.

No simple workarounds exist now in the plugin - I created a bug #123 to address the issue.
Complex workarounds exist, but I prefer to invest my time in resolving the bug instead of creating non-trivial documentation for workarounds.

At a glance the fix seems to be easy and quick. At the same time if I want to keep backward compatibility (e.g. sorting of different types of files in the same folder) and also allow for conscious selection of file basename vs. full filename, I need to give some more thought to the solution.

Please monitor the status of #123 to get notified when the fix reaches the plugin release.

[cmasfo]

sorry, i've just checked that the order of file and folder of same name was just matter of creation time. i hope it have file-first rule, or some way to sort them. currently i'm using all files first and all folders later rule as an alternative.

suggestion:

< true a-z /:// # files then folders
< true a-z ///: # folders then files
# or
< true a-z /: // # files then folders
< true a-z // /: # folders then files

also when i give rule with name, how do i specify if the rule is folder or file? should i do like this?

//rulename
/:rulename
# or
rulename/
rulename.md

[cmasfo]

this might be a stupid question, but when we use obsidian-show-hidden-files plugin, the rule .... is quite ambiguous. it could both mean starting with dot, or ending with dot. is there anyway to remove the ambiguity? i ask this because i wanted to make dot folders first(e.g. .vscode, .obsidian) and underscore folders later(e.g. _asset, _media) just like vscode when we sort by unicode.

suggestion:

\.... # starting with dot
...\. # ending with dot

sorry for keep asking, but apologize me. it's because i like this plugin since i like sorting things.

[SebastianMC]

Hi @cmasfo

what a series of excellent findings, valuable remarks and suggestions!
Thank you, I appreciate your contribution!
It helps in identifying bugs, gaps, areas of potential improvement, new features, etc.

Github discussions UI doesn't allow me to respond to subsequent messages separately, so here we go one-by-one, respectively:

Unicode sorting

As you wrote - the unicode support is partial in some areas, true. It has to stay so for now because the area is complex and multilayered. For various reasons:

• the JS unicode support in collator has its own limitations, it is doing a 'best match' internally
• the JS unicode collator engine reads the unicode sorting setting from the system or app language / locale setting. Which means that if your system doesn't allow you to configure and finetune some unicode settings, the JS collator will behave accordingly
• there is a series of finetuning configurations, like the Order of Case. For me there is no reason to hardcode any of available options and arbitrarily give preference to lowercase first or uppercase first - let the JS collator read their settings from system / app language and locale setting
• the specific example of unicode order for different language: being a fundamental feature of JS collator, this works correctly, driven by the system (or app) language and locale settings (which is then read by Obsidian).

To recap, let me emphasize: the custom sort plugin (and Obsidian standard UI sorting as well) already by default use unicode sorting to the extend supported by JS collator engine. The configuration of the unicode sorting is read from system or app locale and language settings. If you don't observe some specific unicode behavior (e.g. lowercase first of uppercase first) it means that the system/app unicode setting specify it so (and hence JS collator reproduces it), or that the system/app setting leaves it unspecifed or that the JS collator engine doesn't support a specific unicode sorting capability.

Perhaps when you mention sort by unicode you think about some specific custom-crafted finetuned unicode sorting configuration (e.g. some employed by / embedded in VS Code). If that's the case, then the potential solution for the plugin would be to expose some configuration mechanism (e.g. in plugin settings window or via new non-trivial syntax). So for now, when you don't observe some unicode-specific sorting behavior, please assume (for simplicity) that your system/app locale setting configures it differently, or the JS collator engine doesn't support that specific behavior, or that some defaults apply (which are different from what you expect).

To see the variety and complexity of unicode sorting options these are good tech references:

• https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/Collator/Collator
• https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl

Relative order of file with basename equal to folder name

Another good catch!
Due to support for multi-level sorting in the custom-sort plugin this was not trivial to address.
Eventually, the following behavior was implemented in the plugin release 2.1.4:

• when file basename and folder name are the same and their relative order is undetermined by other sorting criteria at any level (e.g. by modified / created dates), by default the last-resort last-level implicit sorting gives priority to a file in the described case
• new syntax introduced files-first and folders-first to allow explicit controlling of the order, e.g. < a-z, files-first

Syntax for consciously applying grouping to files and/or folders selectively

The documentation is partial or not up to date in this area, so please take a look at the relevant code snippet, it should be readable:

• lines 246 to 256

For example, to apply a rule for files only, you can:

• /:files, e.g. /:files Note..., or
• /:, e.g. /: Meeting notes...
  If matching against file name is relevant, the above syntax matches against file basename. To enforce matching agains full file name with ext, use:
• /:files. Note..., or
• /:. Meeting notes...

The equivalent tokens (prefixes) to match against folders are:

• / e.g. / Daily notes
• /folders e.g. /folders Person...

When there is no prefix, both folders and files are in scope. At the same time you can be very verbose and specify it explicitly as:

• /files:folders ...123, or equivalent concise syntax
• % ...blabla suffix

To evaluate grouping rule against folders and file full names, use:

• /folders:files. xyz...abc

Some more syntax variants available, as visible in the code snippet.

Ambiguity of ....

To address this, the recent plugin release 2.1.4 includes an explicit syntax of ./....
I'd agree that the standard approach of escaping with backslash, which you suggested \.... would be more standard. At the same time, because of construction of the parser and my limited time, the low-risk and low-hanging-fruit solution is the one which I implemented. Long term the standard escaping sequence is the goal.

Summary

Don't be scared with the long detailed info above :-)
I'm in the process of preparing a new incarnation of up-to-date documentation for the plugin and I treat our discussions also as preparations for that (so that later on I can copy & paste).
Maybe someone will search / read this discussion thread seeking for instructions?

Again, thank you for your valuable input and insights resulting in many improvements and items added to the roadmap

[cmasfo]

Updated instantly.

It's quite sad to hear about limitations of unicode sort, but I think you already improved the vanilla Obsidian much. Thank you, especially for file-folder sorting. Now organizing notes seems to be more fun. < true a-z, files-first is now my favorite.

---

By the way, I think I've found another weird behavior(didn't intend tho).

When the new note location setting is 'Same folder as current file', new notes be created inside the folder which has same name with note.

• a.md
• a/

In this situation, when a.md note is opened, making new note be like following:

• a.md
• a/
  • Untitled.md

[cmasfo]

oh sorry, it was matter of folder note core plugin. i was using the core plugin only since it supported auto renaming note and folder at same time. i'll just leave this here for people might wonder like me.

anyway the latest series of updates just felt like magical. thanks for fast updates, and ill make new thread if i come up with something another.

[SebastianMC]

Hi @cmasfo

having some free coffee-time I looked at VS Source code to see, what hides behind their 'unicode sort' label. It turned out to be a trivial sorting method, and the name 'unicode' being arbitrary, potentially confusing.

Being so trivial, I spent few minutes to expose it as an experimental feature, together with two additional uppercase first and lowercase first options.

The release 2.1.5 exposes these experimental features (clear annotation in release notes that they are subject to change or removal):

• aaa-z and a-zzz sort orders (uppercase first, alphanumeric order and lowercase first, alphanumeric)
• vsc-unicode sort order, which is equivalent to what VS Code refers to as 'unicode' sorting (which name is arbitrary and confusing, yet familiar to VS Code users)

Take a look when you have an opportunity!
I'm curious of your feedback, if you find it valuable and worth retaining in the plugin.
And first and foremost - if they work as you'd expect.

[cmasfo]

vscode's unicode sort was in fact arbitrary? what a plot twist. but yes, as you said I feel like home to see vsc unicode sort. it's quite surprising to see it in obsidian. I just wonder what made you thought that vscode unicode sort is arbitrary and the name unicode is misleading. because when i compared the order of unicode the vscode looked like sorting quite well.

---

Also, I can see that vsc-unicode is working, but for some reason aaa-z and a-zzz are not working. maybe because of locale? I don't know.

• vsc-unicode: work as i expected. i've also checked some cases like non-english letters such as cjk ideographs, and they also look fine to me as far as i've seen.

• aaa-z: i've seen parsing ok message but it looks like default obsidian sorting.

[SebastianMC]

Hi @cmasfo

let me be precise and clarify:

• I referred only to the name 'unicode' as being arbitrary and potentially confusing for that specific sorting method
• I described the actual sorting method (named in VS as 'unicode') as 'trivial sorting method'.

In particular:

• I did not name (nor thought) that the specific sorting method is arbitrary. That specific sorting method (regardless how it is named) is pretty standard and a basic one.

The name 'unicode' being so short is very generic and - in context of sorting - it is typically understood as 'unicode collation sorting algorithm' (as documented on wikipedia here).
Then, what hides under the name 'unicode' in VS Code is the simple ordering 'by unicode code points (aka values aka codes aka numbers aka byte representation) of subsequent characters in the string'

Now you can probably see why the confusion here: you came from VS Code world and used the name 'unicode' sorting, which in VS Code is not ambiguous and refers to a specific sorting method, with that specific UI label. At the same time I - not a VS Code user - understood the sorting name in a wider sense.

If they had used a longer and more precise label in VS Code, it would have saved some of our time comparing oranges to apples.

I tried to find the rationale behind the why the name 'unicode' was used in VS, I scanned some of discussions like this for more background. It looks like an arbitrary decision motivated by an intent to keep the label short. Even the UI hint which VSC displays 'Names are sorted in Unicode order' is ambiguous, if one is aware of the complexity behind unicode ('Is this by Unicode character codes order or by Unicode collation order?')

Knowing all of the above I should be able to find a better name for the sorting method in the custom-sort plugin syntax. I will keep the vsc-unicode name, it sounds familiar to VSC users. On top of that I will probably expose an alias name like unicode-codepoints or unicode character codes (trying to find something unambiguous and short)

---

Regarding the two additional sort methods exposed under aaa-z and a-zzz they don't work correctly, as you noticed. To be investigated why.
They are expected to sort by unicode collation with priority to lower or upper case. In other words:

• for aaa-z the order of characters would be 'AaBbCcDdEeFf........'
• for a-zzz the order of characters would be 'aAbBcCdDeEfF......'

BTW as part of examination of VS Code source code and scanning discussions on github there I realised that the 'upper' and 'lower' sorting methods (in VSC context) refer to a different concepts: name starting with a lowercase character vs name starting with a capital letter. This is not implemented in custom-sort plugin.

Again, thanks for your valuable input!