Skip to content

csvtk sort inconsistency #337

Description

@LeonHafner

Hi,
First of all, thanks for developing this tool! It's extremely useful in bioinformatics workflows.
I noticed an inconsistency in the sort subcommand.

Example 1 (test_1.tsv)

Input:

seq,start,stop,strand
chr1,1872,1873,+
chr1,1872,1873,-

Command:

csvtk sort test_1.csv

Output:

seq,start,stop,strand
chr1,1872,1873,-
chr1,1872,1873,+

Here, the rows are reordered because + and - swapped positions.


Example 2 (test_2.csv)

Input:

seq,start,stop,strand
chr1,1872,1873,-
chr1,1872,1873,+

Command:

csvtk sort test_2.csv

Output:

seq,start,stop,strand
chr1,1872,1873,+
chr1,1872,1873,-

Again, the rows are reordered, but in the opposite way compared to the first example.


Problem

This is confusing because:

  • There should be a consistent ordering of + and -.
  • The outcome changes depending on the input order.
  • I expect to tool to sort on all columns, if no column is specified.

If I explicitly include the strand column in the sort keys, the output is as expected. However, my assumption was that if no columns are specified, the tool should sort using all columns. Why then does the behavior differ between these cases? Is there any situation where this inconsistency is intentional, or is it a bug?

I tested on version 0.31.0 which is the most recent version available on bioconda.

Thanks for looking into it!

Activity

  1. shenwei356 commented on Sep 28, 2025

    @shenwei356
    Owner

    You're right, it is a bug! Just fixed it.

    I expect to tool to sort on all columns, if no column is specified.

    Good suggestion!
    It supports field ranges now, and the default value 1- means all columns.

    Please have a try:

    The help message is also updated with more details.

    $ csvtk sort  -h
    sort by selected fields
    
    Field types:
      - Column name                  : -k name
      - Nth field                    : -k 1
      - field range:
         - All fields                : -k 1-
         - From 3rd to 5th column    : -k 3-5
         - Fields except for the 4th : -k -4
         - Fields except for 3rd-5th : -k -3--5
    
    Sort types:
      - Default: alphabetical order  : -k 1-
      - n      : numeric order       : -k 1:n
      - N      : natrual order       : -k 1:N   (e.g., "a9" should be in front of "a10")
      - d      : sort by date        : -k 1:d   (support multiple formats of date and time)
      - u      : custom levels       : -k 1:u -L levels.txt
    
    Combinations:
      - All sort types can be used with "r" for reversing the order, e.g., -k 1:nr
      - Multiple fields can be used, e.g., -k year:n -k name
    
    Usage:
      csvtk sort [flags] 
    
    Flags:
      -h, --help             help for sort
      -i, --ignore-case      ignore-case
      -k, --keys strings     keys (multiple values supported). sort type supported, "N" for natural order,
                             "n" for number, "d" for date/time, "u" for user-defined order and "r" for
                             reverse. e.g., "-k 1", "-k 2-", "-k 3-5:nr", "-k A:r", "-k 1:nr -k 2" (default [1-])
      -L, --levels strings   user-defined level file (one level per line, multiple values supported).
                             format: <field>:<level-file>.  e.g., "-k name:u -L name:level.txt"
    
  2. LeonHafner commented on Sep 28, 2025

    @LeonHafner
    Author

    Thank you, it works perfectly now!
    Also many thanks for the improved help text and the implementation of the field ranges, they are really helpful.
    The only thing I noticed is the typo "natrual" in the help text, apart from that it looks great!

  3. added a commit that references this issue on Sep 28, 2025
  4. shenwei356 commented on Oct 16, 2025

    @shenwei356
    Owner

    fixed and available in the new release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions