1. Home
  2. Tips and Tricks
  3. Split a file but keep the headers

Split a file but keep the headers

Option 1

cat bigFile.csv | parallel --header : --pipe -N999 'cat >file_{#}.csv'

 

References

 

Option 2

Part-1

In order to split your CSV file but keep the first header line, you need to create
a Shell script first.

  1. Open your code editor.
  2. Type in the below script and change the number of lines (currently set to 1000)
    and file name (currently set to mycsvfile.csv)
    accordingly.
  3. Save the file in the same directory as your CSV file as split.sh.
#!/bin/bash
FILENAME=mycsvfile.csv
HDR=$(head -1 $FILENAME)
split -l 1000 $FILENAME xyz
n=1
for f in xyz*
do
if [ $n -gt 1 ]; then 
echo $HDR > Part${n}.csv
fi
cat $f >> Part${n}.csv
rm $f
((n++))
done

 

Part-2

Once you’ve done this, open up Terminal on your Mac and navigate to your split.sh script
accordingly and run the sh command
(see below).

Note, it’s important to have a backup of your CSV file before running the below command:

sh split.sh

References

Updated on July 16, 2022
Was this article helpful?

Related Articles