$CSVfilesFolder = "C:\CSVFiles\"
$CompressFolder = "C:\CSVFiles\Compress"
$ArchiveFolder = "C:\CSVFiles\Archive"
$filesForZip = Get-ChildItem -Path $CSVfilesFolder -Filter *.csv | Sort-Object LastWriteTime | Select-Object -First $CountofFilesToZip | select fullname | ForEach-Object { $_.FullName }
$newFileName = "$($CompressFolder)\GeneratedZipFile.zip"
#Encoding to UTF8
foreach($file in $filesForZip)
{
$UTF8Only = New-Object System.Text.UTF8Encoding($false)
$FileContent = Get-Content -Path $file
[System.IO.File]::WriteAllLines($file, $FileContent, $UTF8Only)
}
#Generate Zip file
$compress = @{
Path = $filesForZip
CompressionLevel = "Fastest"
DestinationPath = $newFileName
}
Compress-Archive @compress -Force
#Archive the files
foreach($file in $filesForZip)
{
Move-Item $file -Destination $ArchiveFolder -Force
}
Monday, 8 February 2021
Saving CSV file in UTF8, Compress and Archive them
We had a requirement, where the CSV files generated by SSIS were not in UTF8. We had to resave in UTF8 format. We also had to compress the group of files and archive the original set of files. We achieved that using Powershell.
Maintain timestamp of CSV File
We were having a scenario, where we had to fix few CSV field issues. But, we also wanted to retain timestamp. We used below script to fix the CSV and maintain the timestamp.
$BulkLoadFolder = "C:\Projects\Sntrax\Bulkload\FilesToFix"
$Folder = "C:\FilesToFix\"
$filesList = Get-ChildItem -Path $Folder -Filter *.csv | select fullname | ForEach-Object { $_.FullName }
foreach($file in $filesList)
{
$newfile = $file.Replace("FilestoFix","FilestoFix\fixed")
#Get lastwritetime for the filename
$LastWriteDateTime = Get-ItemPropertyValue -Path $file -Name LastWriteTime
#Import file and Export
Import-Csv $file | ForEach-Object {
if ($_.Field1.Trim() -eq '') {
$_.Field1 = '--'
}
if ($_.Field2.Trim() -eq '') {
$_.Field2 = '--'
}
$_
} | Export-Csv $newfile -NoTypeInformation -Force
Set-ItemProperty -Path $newfile -Name LastWriteTime -Value $LastWriteDateTime
}
Subscribe to:
Posts (Atom)
How to Handle SSIS Database movement from one environment to another
Below are the steps to follow the movement of SSISDB from one environment to another: -- opening the existing Database master key in S...
-
We are using Azure Data Factory to load data from Azure storage blobs to SQL Server on-premises. During data loading, we faced SQL Error: ...
-
Hello World, Through this blog, I am going to ruminate over lots of technical stuff around databases. Hoping to provide quality conten...
-
Below are the steps to follow the movement of SSISDB from one environment to another: -- opening the existing Database master key in S...