Data Movement Automation via PowerShell Script: Part-1

 

PowerShell is a task automation and configuration management framework developed by Microsoft. It is a scripting language and an interactive command-line shell designed to simplify system management tasks.

PowerShell is used extensively in various IT domains, including system administration, network configuration, and cyber security. Its benefits include automation, simplified scripting, and the ability to manage both Windows and non-Windows systems.

While PowerShell shares some similarities with other scripting languages like Python and Bash, it’s specifically designed for Windows environments. It offers seamless integration with Windows features and applications.

1. Upload Files on ftp portal:

    a. To copy the latest files from a local folder to an FTP server using PowerShell with credentials, you can follow the steps below. This script assumes that you want to upload only the files modified within a certain time range.

# FTP server details
$ftpServer = "ftp://example.com"
$ftpUsername = "yourUsername"
$ftpPassword = 'yourPassword'
# Local folder path
$localFolderPath = "C:\Path\To\Your\Folder"
# Remote folder path on the FTP server
$remoteFolderPath = "/Remote/Path/On/FTP/"
# Create a WebClient object with credentials
$webClient = New-Object System.Net.WebClient
$webClient.Credentials = New-Object System.Net.NetworkCredential($ftpUsername, $ftpPassword)
# Get the latest files modified within the last 24 hours (adjust as needed)
$latestFiles = Get-ChildItem -Path $localFolderPath | Where-Object { $_.LastWriteTime -gt (Get-Date).AddDays(-1) }
# Iterate through each file and upload to FTP
foreach ($file in $latestFiles) {
# Build the URI for the FTP upload
$uri = New-Object System.Uri("$ftpServer$remoteFolderPath$file")

# Upload the file
$webClient.UploadFile($uri, $file.FullName)
Write-Host "File $($file.Name) uploaded successfully to $uri"
}
# Dispose of the WebClient object
$webClient.Dispose()




1. 1.Sets FTP server details, local folder path, and remote folder path.
2. 2. Creates a WebClient object with the provided credentials.
3. 3. Retrieves the latest files modified within the last 24 hours (you can adjust the timeframe as needed).
4. 4. Iterates through each of the latest files and uploads them to the FTP server.

Please replace the placeholder values (example.com, yourUsername, yourPassword, C:\Path\To\Your\Folder, /Remote/Path/On/FTP/) with your actual FTP server details and local folder paths. Also, adjust the time range for file modification as per your requirements.

b. If you want to check for the existence of a file on the FTP server based on its name rather than its size, you can use the FTP NLST command to get a list of files in the remote directory and then check if the file you're trying to upload is in that list. Here's an updated script:


# FTP server details
$ftpServer = "ftp://example.com"
$ftpUsername = "yourUsername"
$ftpPassword = "yourPassword"

# Local folder path
$localFolderPath = "C:\Path\To\Your\Folder"
# Remote folder path on the FTP server
$remoteFolderPath = "/Remote/Path/On/FTP/"
# Create a WebClient object with credentials
$webClient = New-Object System.Net.WebClient
$webClient.Credentials = New-Object System.Net.NetworkCredential($ftpUsername, $ftpPassword)
# Get a list of files in the remote folder
$remoteFiles = $webClient.DownloadString("$ftpServer$remoteFolderPath")
# Get all files in the local folder
$files = Get-ChildItem -Path $localFolderPath
# Iterate through each file and upload to FTP
foreach ($file in $files) {
# Build the URI for the FTP upload
$uri = New-Object System.Uri("$ftpServer$remoteFolderPath$file")
# Check if the file already exists on the FTP server
$fileExists = $remoteFiles -split [Environment]::NewLine -contains $file.Name

if (-not $fileExists) {
# Upload the file only if it does not exist on the server
$webClient.UploadFile($uri, $file.FullName)
Write-Host "File $($file.Name) uploaded successfully to $uri"
} else {
Write-Host "File $($file.Name) already exists on $uri. Skipping."
}
}

# Dispose of the WebClient object
$webClient.Dispose()




In this script:

• 1. The DownloadString method is used to get a list of files in the remote folder ($remoteFiles).
• 2. The list of remote files is split into an array of filenames based on the newline character.
• 3. The script then checks if the current local file's name is in the list of remote files.

To be Continued.......