1. Overview

It is often necessary to use bash to batch process file formats, and it is troublesome to query some basic syntax every time.

So this article summarizes some common syntax when using Bash to traverse a folder directory to process images as an example.

2. Iteration

Bash traversal is relatively simple and not significantly different from the for loops of most programming languages.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

#/bin/sh
#author: chancel.yang

for file in ./*
do
    if test -f "$file"
    then
        echo -e "$file"
    fi
done

The output is as follows.

1
2
3
4
5

➜ bash demo.sh  
./a.png  
./b.png  
./c.jpg  
./demo.sh

This traversal is practical in most cases.

3. Path handling

The above output shows that the execution script demo.sh is also in it, and it can only be processed for the current path.

Then try to add a custom path and put the demo script in another location.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

#/bin/sh  
#author: chancel.yang  

path="/mnt/SDA/TMP/test"  

for file in $path/*  
do  
if test -f "$file"
then  
    echo -e "$file"
fi  
done

The output is as follows.

1
2
3
4

➜ bash /mnt/SDA/TMP/demo.sh  
/mnt/SDA/TMP/test/a.png  
/mnt/SDA/TMP/test/b.png  
/mnt/SDA/TMP/test/c.jpg

In practice, you can adjust $path as an incoming parameter with the following script.

1
2
3
4
5
6
7
8
9

#/bin/sh  
#author: chancel.yang  
for file in $1/*  
do  
if test -f "$file"
then  
    echo -e "$file"
fi  
done

Execution.

1
2
3
4

➜ bash demo.sh "/mnt/SDA/TMP/test"
/mnt/SDA/TMP/test/a.png
/mnt/SDA/TMP/test/b.png
/mnt/SDA/TMP/test/c.jpg

$0 is the name of the script file, e.g. demo.sh

In addition to $[number] getting the passed-in parameter, there are some common special parameter values that can be used directly.

parameter description
$# Total number of parameters
$$ Current PID number

4. Extracting file names

After adding a custom path above, the output $file also carries the path information. In practice, you will also use filenames, extensions, etc.

The following shows how to extract file path, file name (with extension), file name, extension.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

#/bin/sh  
#author: chancel.yang  

path="/mnt/SDA/TMP/test"  

for file in $path/*  
do  
if test -f "$file"
then
    echo "dirname: $(dirname $file)"
    echo "filename: $(basename $file)"  
    filename=$(basename $file)  
    echo "filename(suffix): ${filename%%.*}"
    echo "suffix: ${filename#*.}"
fi  
done

The output after execution is as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

➜ bash demo.sh  
dirname: /mnt/SDA/TMP/test  
filename: a.png  
filename(suffix): a  
suffix: png  
dirname: /mnt/SDA/TMP/test  
filename: b.png  
filename(suffix): b  
suffix: png  
dirname: /mnt/SDA/TMP/test  
filename: c.jpg  
filename(suffix): c  
suffix: jpg

If there are multiple suffixes, such as a.tar.gz, other less common ones are extracted as follows.

1
2
3
4
5

filename='a.tar.gz'
# a.tar
${file%.*}
# gz
${file##*.}

5. conditional judgments

5.1. test and []

Bash provides the test keyword for conditional judgments, and occasionally you will see if [ string1 ! = string2 ]

The above two write-ups are equivalent, i.e. test and [ are equivalent, both are keywords in Bash, the difference is that [ requires the last argument to be ]

Since [ is a keyword and ] is an argument, there must be spaces, [string1 != string2], which is written without spaces, is wrong.

Since the two are equivalent, the following illustration for test, replacing test condition in the script with [ condition ], is also valid.

5.2. Judgment Operators

The following is a sample script for common conditional judgments.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

#/bin/sh
#author: chancel.yang

a=100
b=200
c=100

echo -e "a="$a", b="$b", c="$c

if test "$a" -eq "$c"
then
    echo -e "a = c"
fi

if test "$a" -ne "$b"
then
    echo -e "a != b"
fi

if test "$b" -gt "$a"
then
    echo -e "b > a"
fi


dirname=`pwd`
filename=$dirname/demo.sh

echo -e "dirname: "$dirname", filename: "$filename

if test -d "$dirname"
then
    echo -e $dirname" is a folder"
fi
if test -f "$filename"
then
    echo -e $filename" is a file"
fi

The output after execution is as follows.

1
2
3
4
5
6
7
8

➜ bash demo.sh
a=100, b=200, c=100
a = c
a != b
b > a
dirname: /mnt/SDA/TMP, filename: /mnt/SDA/TMP/demo.sh
/mnt/SDA/TMP is a folder
/mnt/SDA/TMP/demo.sh is a folder

The list of common operator parameters for conditional judgment numeric type is as follows.

parameter description
-eq equal
-ne not equal
-gt >
-ge >=
-lt <
-le <=

When judging the condition document, the list of common operator parameters is as follows

parameter description
-e file exists
-r file exists and is readable
-w file exists and is writable
-x file exists and is executable
-s file exists and is not empty
-d is a directory
-f is a file

The determination of file and numeric content is relatively simple, and the next step is to look at the more complex string determination.

In test, when determining if a string is empty, -z means True when the string is empty, and -n means True when the string is not empty.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

#/bin/sh
#author: chancel.yang

string_1=""
string_2="hello"

if test -z "$string_1"
then
    echo -e "string_1 is null"
fi

if test -n "$string_2"
then
    echo -e "string_2 is not null"
fi

The output after execution is as follows.

1
2
3

➜ bash demo.sh  
string_1 is null  
string_2 is not null

When using test to make judgments, it is recommended to use double quotes to enclose variables, otherwise the expected execution will not be achieved because of spaces in the variable content.

In addition to double quotes to avoid empty cells in the variable path, you can also use [[]] to avoid splitting the content, considering that the variable path may be entered by the user.

5.3. Special [[]]

Both [ and test split the content when judged, e.g. test="go to" is recognized as go and to arguments.

Unlike [, both [[ and ]] are commands, and ]] is not an argument.

The following code demonstrates the difference.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

#/bin/sh
#author: chancel.yang

a="go to"

# This will be an exception
if test $a == "go to"
then
    echo -e "$a == 'go to'"
fi

# Adding double quotation marks here will correctly determine
if test "$a" == "go to"
then
    echo -e "$a == 'go to'"
fi

# This can be correctly determined without double quotes
if [[ $a == "go to" ]]
then
    echo -e "$a == 'go to'"
fi

The output is as follows.

1
2
3
4

➜  bash demo.sh
demo.sh: line 7: test: too many arguments
go to == 'go to'
go to == 'go to'

6. Image conversion example

By this point, the basic syntax for writing a traversal script is complete, and for reference, here is a traversal script that converts all files in a given directory to webp images.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69

#/bin/sh
#author: chancel.yang
#date: 2022-06-14
show_help() {
    echo "$0 [-h|-?|--help] [--who me] [--why hhh]"
    echo "-h|-?|--help    Show Help"
    echo "--path          Folder path"
}

while [[ $# -gt 0 ]]; do
    case $1 in
    -h | -\? | --help)
        show_help
        exit 0
        ;;
    --in)
        in="${2}"
        shift
        ;;
    --out)
        out="${2}"
        shift
        ;;
    *)
        echo -e "Error: $0 invalid option '$1'\nTry '$0 --help' for more information.\n" >&2
        exit -1
        ;;
    esac
    shift
done

if !(test "$in")
then
    echo "Please run --help for usage"
    exit
fi
if !(test "$out")
then
    echo "Please run --help for usage"
    exit
fi

if !(test -d "$in")
then
    echo "Error: $in is not a folder"
    exit
fi

if test -f "$out"
then
    echo "Error: $out is a file"
    exit
fi

if [ ! -d "$out" ]; then
  mkdir $out
fi

for file in $in/*
do
    filename=$(basename $file)
    if test -f "$file"
    then
        /usr/bin/ffmpeg -i $file -c:v libwebp $out/${filename%%.*}.webp
        echo "$file convert to $out/$filename.webp"
    fi
done

echo -e "Convert success"