Shell scripts | Common commands related to data processing

Write file path to lst

• The absolute path to write to the file in the current folder:ls | sed.

// Write to the absolute path of all files in the current folderls | sed "s:^:`pwd`/:" >all_file.lst
 // Write to the absolute path of the first file under the current folderls | sed "s:^:`pwd`/:" head -n 1 > first_file.lst

• 1
• 2
• 3
• 4
• 5

AI writing code

• The absolute path to write to a specific suffix file under the specified folder:ls.

ls -R $dir/*.txt > 

• 1

AI writing code

• Write to the absolute path of all txt files with names containing string strings under the specified folder (dir):find | grep.

find $dir/ -name "*.txt" | grep string >> 

• 1

AI writing code

Extract filename/file business fragment in absolute path

• Extract the last fragment (file name) in the path:awk.

// Each line is formatted as follows, dir and text are separated by /t
 // /.../.../filename	text
file=`echo $line | awk -F '/t' '{print $1}' | awk -F '/' '{print $NF}'`

• 1
• 2
• 3

AI writing code

• Remove the suffix of the file name

filename=`echo ${file%.*}`

• 1

AI writing code

Two cycles

for loop

• Read content from the file + append write to lst:cat.

for line in `cat `
{
	name=`echo $line | awk -F '/' '{print $NF}'`
	echo $name >> 
}

• 1
• 2
• 3
• 4
• 5

AI writing code

• Use i to generate the folder name in the current directory (0001-0022) for traversal + find the name containing a certain string + append to write to lst

for ((f=1;f<=22;f++))
do
	foldernum=`printf %04d $f`
	echo $foldernum
	find $PWD/$foldernum/ -name "*.txt" | grep string >> 
done

• 1
• 2
• 3
• 4
• 5
• 6

AI writing code

While loop

• Read in from file + append write to lst

while read line;
do
	text=`head -n 1 $line`
	file=`echo $line | awk -F '/' '{print $NF}'`
	newline="$file $text"
	echo $newline >> 
done < 

• 1
• 2
• 3
• 4
• 5
• 6
• 7

AI writing code