I am creating an application that will
process a CSV file,
create JObject for each record in CSV file, and finally
upload all these files to SFTP server
After looking around for a free library for the 3rd point, I decided to use SSH.NET.
I have created the following class to perform the upload operation asynchronously.
public class JsonFtpClient : IJsonFtpClient
{
private string _sfptServerIp, _sfptUsername, _sfptPassword;
public JsonFtpClient(string sfptServerIp, string sfptUsername, string sfptPassword)
{
_sfptServerIp = sfptServerIp;
_sfptUsername = sfptUsername;
_sfptPassword = sfptPassword;
}
public Task<string> UploadDocumentAsync(string sourceFilePath, string destinationFilePath)
{
return Task.Run(() =>
{
using (var client = new SftpClient(_sfptServerIp, _sfptUsername, _sfptPassword))
{
//ALL ERRORS ARE THROWN WHEN EXECUTING THIS LINE
client.Connect();
using (Stream stream = File.OpenRead(sourceFilePath))
{
client.UploadFile(stream, destinationFilePath);
}
client.Disconnect();
}
return (destinationFilePath);
});
}
}
The UploadDocumentAsync method returns a TPL Task so that I can call it to upload multiple files asynchronously.
I call this UploadDocumentAsync method from the following method which is in a different class:
private async Task<int> ProcessJsonObjects(List<JObject> jsons)
{
var uploadTasks = new List<Task>();
foreach (JObject jsonObj in jsons)
{
var fileName = string.Format("{0}{1}", Guid.NewGuid(), ".txt");
//save the file to a temp location
FileHelper.SaveTextIntoFile(AppSettings.ProcessedJsonMainFolder, fileName, jsonObj.ToString());
//call the FTP client class and store the Task in a collection
var uploadTask = _ftpClient.UploadDocumentAsync(
Path.Combine(AppSettings.ProcessedJsonMainFolder, fileName),
string.Format("/Files/{0}", fileName));
uploadTasks.Add(uploadTask);
}
//wait for all files to be uploaded
await Task.WhenAll(uploadTasks);
return jsons.Count();
}
Although the CSV file results in thousands of JSON records, but I want to upload these in batches of at least 50. This ProcessJsonObjects always receives a list of 50 JObjects at a time which I want to upload asynchronously to the SFTP server. But I receive the following error on client.Connect(); line of the UploadDocumentAsync method:
Session operation has timed out
Decreasing the batch size to 20 results in the following error:
Client not connected.
But this also uploads 3 files to the SFTP server as well, but not all the 20.
What am I doing wrong? Your help is much appreciated.