Thanks for the response.
If i use sftpclient.downfile which return the memorystream it take around 10 to 15 mins for the same file. But i am running into system.outofmemory in multiple threads of bigger files.
WinSCP also takes around 10 to 15 mins and works fine for bigger files in multiple threads. But i don't want to deploy the winscp.exe in my servers.
Tried manually using filezilla in the same environment download time is around 10 mins.
sftpfilestream looks like the perfect option for the multi thread environment. Because of the transfer speed i am stuck.
Hi drieseng,
I'm pleased to see that one is going to analyze the bug again.
Please let me know if you need any further information or support.
I still need a solution
Thanks
Ulrich
I did some tests myself, and SftpFileStream (SftpClient.OpenRead) is about twice as slow as DownloadFile:
DownloadFile (BufferSize=14000): 2100 Kb/s
DownloadFile (BufferSize=16384): 1500 Kb/s
OpenRead (Buffer=14000): 900 Kb/s
DownloadFile (Buffer=16384): 950 Kb/s
I'm gonna try to look into this later today.
I'll also be reducing logging, as that increases throughput by 100%.
For now, you may want to consider using DownloadFile with a FileStream, and lowering the BufferSize to 14000.
Hello!
I have the same issue in my win service:
ERROR WCFService.Server.WinService Unhandled exception.
System.NullReferenceException: Object reference not set to an instance of an object.
at Renci.SshNet.Sftp.SubsystemSession.RaiseError(Exception error)
at System.EventHandler`1.Invoke(Object sender, TEventArgs e)
at Renci.SshNet.Session.RaiseError(Exception exp)
at Renci.SshNet.Session.MessageListener()
at Renci.SshNet.Session.<Connect>b__4()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
I recently fixed this issue (changeset 34793).
Please build from SVN, or wait for the upcoming (beta) release.
I've made quite a few changes that improve throughput significantly, but I now also found the reason why SftpFileStream is much slower: we do not take into account the buffer size for SftpFileStream. We always use a buffer size of 4 KB!
I'll fix and test the impact tomorrow.
at Renci.SshNet.Session.WaitHandle(WaitHandle waitHandle)
at Renci.SshNet.Channels.Channel.WaitHandle(WaitHandle waitHandle)
at Renci.SshNet.Channels.ChannelSession.SendExecRequest(String command)
at Renci.SshNet.SshCommand.BeginExecute(AsyncCallback callback, Object state)
at LogFileArchive.LogFileArchive.runningcommand() in C:\Utilities\LogFileArchive\Program.cs:line 381
at LogFileArchive.LogFileArchive.Main(String[] args) in C:\Utilities\LogFileArchive\Program.cs:line 475
at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart() sshClient.Connect();
var cmds = sshClient.RunCommand("echo 1");
var output = cmds.Result;Ulrich,
Please provide a simple repro (code only), and any files necessary to reproduce this issue.
Thanks!
Gert
Please provide more details. Do you know of a public server that allows us to reproduce this issue?
using (var ssh = new Renci.SshNet.SshClient(connectionInfo))
{
ssh.Connect();
var cmd = ssh.RunCommand("telnet host -l user");
}
using (var ssh = new Renci.SshNet.SshClient(connectionInfo))
{
ssh.Connect();
Renci.SshNet.ShellStream stream = ssh.CreateShellStream("xterm", 80, 50, 1024, 1024, 1024);
StreamReader Reader = new StreamReader(stream);
StreamWriter Writer = new StreamWriter(stream);
Writer.AutoFlush = true;
Thread.Sleep(500);
string Result = Reader.ReadToEnd();
Writer.WriteLine("telnet host -l user");
Thread.Sleep(500);
Result = Reader.ReadToEnd();
Writer.WriteLine("password");
Thread.Sleep(500);
Result = Reader.ReadToEnd();
}
var coninfo = new ConnectionInfo("1.2.3.4", "peter", new PasswordAuthenticationMethod("peter", "peter"),
new PrivateKeyAuthenticationMethod("peter", new PrivateKeyFile("foo.key")));
var client = new SshClient(coninfo); A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected
host has failed to respond ...//Expect1.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using Renci.SshNet;
//C:\Windows\Microsoft.NET\Framework\v3.5\csc.exe /t:exe /r:C:\Users\ande\Desktop\CSharp\Renci.SshNet.dll Expect1.cs
namespace ExpectTest
{
class Expect
{
static void Main(string[] args)
{
string prompt = @".+#$";
SshClient shell = new SshClient("IOS Router Device or Catalyst IOS", "user", "pass");
shell.Connect();
if (shell.IsConnected)
{
Console.WriteLine("Connected");
}
var stream = shell.CreateShellStream("dumb",80, 24, 800, 600, 1024);
var a = stream.Expect(new Regex(prompt), new TimeSpan(0, 0, 5));
Console.WriteLine(a);
stream.WriteLine("show ip arp");
var r = stream.Expect(new Regex(prompt), new TimeSpan(0, 0, 5));
Console.WriteLine(r);
}
}
}
/*OUTPUT
C:\Users\ande\Desktop\CSharp>.\Expect1.exe
Connected
NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE
Unauthorized access and/or use prohibited. All access and/or use subject to monitoring.
NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE
device-name line 645
3945-Router1#
show ip arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.5.224.1 162 0000.0c07.acfa ARPA GigabitEthernet0/1
Internet 10.5.224.2 3 4403.a7a5.2281 ARPA GigabitEthernet0/1
Internet 10.5.224.3 0 0006.f619.e8bc ARPA GigabitEthernet0/1
Internet 10.5.224.116 - c067.af42.5f01 ARPA GigabitEthernet0/1
3945-Router1#
*/
//Expect2.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using Renci.SshNet;
//C:\Windows\Microsoft.NET\Framework\v3.5\csc.exe /t:exe /r:C:\Users\ande\Desktop\CSharp\Renci.SshNet.dll Expect2.cs
namespace ExpectTest
{
class Expect
{
static void Main(string[] args)
{
//string prompt = @".+#$";
SshClient shell = new SshClient("NXOS Device", "user", "pass");
shell.Connect();
if (shell.IsConnected)
{
Console.WriteLine("Connected");
}
var stream = shell.CreateShellStream("dumb",80, 24, 800, 600, 1024);
//var a = stream.Expect(new Regex(prompt), new TimeSpan(0, 0, 5));
var a = stream.Expect("NX3064_01#", new TimeSpan(0, 0, 5));
Console.WriteLine(a);
stream.WriteLine("show ip arp");
//var r = stream.Expect(new Regex(prompt), new TimeSpan(0, 0, 5));
var r = stream.Expect("NX3064_01#", new TimeSpan(0, 0, 5));
Console.WriteLine(r);
}
}
//If regular expressions are used for expect, both a and r are empty
//If a regular expression for a is used and a name or regex is used for the second, r stays empty.
/*OUTPUT
C:\Users\ande\Desktop\CSharp>.\Expect2.exe
Connected
Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002-2013, Cisco Systems, Inc. All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under
license. Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or the GNU
Lesser General Public License (LGPL) Version 2.1. A copy of each
such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php
NX3064_01#*/
}