Monday 8 February 2021

Saving CSV file in UTF8, Compress and Archive them

We had a requirement, where the CSV files generated by SSIS were not in UTF8. We had to resave in UTF8 format. We also had to compress the group of files and archive the original set of files. We achieved that using Powershell.
 $CSVfilesFolder = "C:\CSVFiles\"  
 $CompressFolder = "C:\CSVFiles\Compress"  
 $ArchiveFolder = "C:\CSVFiles\Archive"  
 $filesForZip = Get-ChildItem -Path $CSVfilesFolder -Filter *.csv | Sort-Object LastWriteTime | Select-Object -First $CountofFilesToZip | select fullname | ForEach-Object { $_.FullName }  
 $newFileName = "$($CompressFolder)\GeneratedZipFile.zip"  
 #Encoding to UTF8   
 foreach($file in $filesForZip)  
 {  
   $UTF8Only = New-Object System.Text.UTF8Encoding($false)  
   $FileContent = Get-Content -Path $file  
   [System.IO.File]::WriteAllLines($file, $FileContent, $UTF8Only)     
 }  
 #Generate Zip file  
 $compress = @{  
  Path = $filesForZip  
  CompressionLevel = "Fastest"  
  DestinationPath = $newFileName  
 }  
 Compress-Archive @compress -Force  
 #Archive the files  
 foreach($file in $filesForZip)  
 {  
   Move-Item $file -Destination $ArchiveFolder -Force  
 }  

Maintain timestamp of CSV File

We were having a scenario, where we had to fix few CSV field issues. But, we also wanted to retain timestamp. We used below script to fix the CSV and maintain the timestamp.
 $BulkLoadFolder = "C:\Projects\Sntrax\Bulkload\FilesToFix"  
 $Folder = "C:\FilesToFix\"  
 $filesList = Get-ChildItem -Path $Folder -Filter *.csv | select fullname | ForEach-Object { $_.FullName }  
 foreach($file in $filesList)  
 {   
     $newfile = $file.Replace("FilestoFix","FilestoFix\fixed")  
     #Get lastwritetime for the filename   
     $LastWriteDateTime = Get-ItemPropertyValue -Path $file -Name LastWriteTime  
     #Import file and Export   
     Import-Csv $file | ForEach-Object {  
  if ($_.Field1.Trim() -eq '') {  
         $_.Field1 = '--'  
       }  
       if ($_.Field2.Trim() -eq '') {  
         $_.Field2 = '--'  
       }  
       $_  
  } | Export-Csv $newfile -NoTypeInformation -Force  
     Set-ItemProperty -Path $newfile -Name LastWriteTime -Value $LastWriteDateTime  
 }  

How to Handle SSIS Database movement from one environment to another

Below are the steps to follow the movement of SSISDB from one environment to another: -- opening the existing Database master key in S...