I wanted to build an awesome place for people to discuss module specific issues, but I don't have any more time for this, and there are much better places to discuss Perl-related issues. I'd recommend asking your question on Stack Overflow or on Perl Monks.
If you are looking for a Perl tutorial or Perl-related news, I hope these links will serve you well.
Posted on 2006-03-31 20:17:00-08 by mp
put() command hanging on files larger than 2kb or so - changing COPY_SIZE fixes
We're running version 0.10 of Net::SFTP and version 1.29 of Net::SSH::Perl. The server we're connecting to is a "5.0.1.79 SSH Tectia Server", with protocol version 2.0.

When small files are transferred Net::SFTP works perfectly. When they get larger than a few kilobytes they hang at this point:

sftp: Sending SSH2_FXP_INIT
sftp: Remote version: 3
sftp: Sent SSH2_FXP_OPEN I:0 P: "filename"
sftp: Sent message T:17 I:1
sftp: Received stat reply T:105 I:1
sftp: Sent message SSH2_FXP_WRITE I:2 O:0

That's in the "do_write" procedure. So we've been trying different things to figure out what's wrong, and one thing that worked is to change the value of COPY_SIZE in SFTP.pm from 8192 to something lower (4063 and lower seem to make it work).

Does anyone have an idea why this is happening and why changing COPY_SIZE fixed it? Below is the test script we're using:



use strict; use warnings; use Net::SFTP; my %args = (ssh_args => []); $args{debug} = 1; $args{user} = <user>; push @{ $args{ssh_args} }, identity_files => [ <identity_filename> ]; push @{ $args{ssh_args} }, options => [ "BatchMode yes", "Port 22" ]; push @{ $args{ssh_args} }, options => [ "PreferredAuthentications publickey", "HostKeyAlgorithms ss +h-dss,ssh-rsa", "BatchMode yes", "Port 22" ]; my $rhost = <rhost>; my $source_file = <source_file>; my $dest_file = <dest_file>; eval { my $sftp = new Net::SFTP($rhost, %args); $sftp->put($source_file, $dest_file); }; if ($@) { print "doh: $@\n"; }
Direct Responses: 2326 | Write a response
Posted on 2006-05-19 14:19:51-07 by franky in response to 2092
Re:put() command hanging on files larger than 2kb
Hi, I am having the same problem with pretty much the same environment. We're running version 0.10 of Net::SFTP and version 1.30 of Net::SSH::Perl. The server we're connecting to is a "5.0.0.844 SSH Tectia Server", with protocol version 2.0. Have you figured out why changing the COPY_SIZE made it work? Also, before I made the change, Put was hanging but Get was working fine. Strange isn't it? Franky
Direct Responses: 2327 | Write a response
Posted on 2006-05-19 15:39:22-07 by mp in response to 2326
Re:put() command hanging on files larger than 2kb
We dug a little deeper and the hang seems to be in the SSH library, not SFTP. It occurs when the amount of data sent exceeds the packet size defined by the remote server (it was roughly 4kb for Tectia). This triggers some buffering in the Perl SSH library. Changing the COPY_SIZE means you're basically doing the buffering yourself. Maybe some sample cases will make it clearer:

* COPY_SIZE=8kb, file size=3kb:
  3kb string put with Net::SFTP
  Net::SFTP examines COPY_SIZE: 8kb
  Net::SFTP sends 3kb to Net::SSH
  Net::SSH examines Tectia packet size: 4kb
  Net::SSH sends 3kb
  *success*

* COPY_SIZE=8kb, file size=10kb:
  10kb string put with Net::SFTP
  Net::SFTP examines COPY_SIZE: 8kb
  Net::SFTP sends first 8kb of 10kb to Net::SSH
  Net::SSH examines Tectia packet size: 4kb
  Net::SSH buffering triggered
  Net::SSH sends first 4kb of 8kb
  *Net::SSH hangs waiting for Tectia response*

* COPY_SIZE=4kb, file size=3kb
  ...same as first
  *success*

* COPY_SIZE=4kb, file size=10kb
  10kb string put with Net::SFTP
  Net::SFTP examines COPY_SIZE: 4kb
  Net::SFTP sends first 4kb of 10kb to Net::SSH
  Net::SSH examines Tectia packet size: 4kb
  Net::SSH sends 4kb
  Net::SFTP sends next 4kb of 10kb to Net::SSH
  Net::SSH examines Tectia packet size: 4kb
  Net::SSH sends 4kb
  Net::SFTP sends final 2kb of 10kb to Net::SSH
  Net::SSH examines Tectia packet size: 4kb
  Net::SSH sends 2kb
  *success*

It appears there's an issue with the interaction between Perl SSH and Tectia 5.0.x when multiple data packets need to be sent. We've only encountered this problem when using this specific SFTP/SSH library and this specific Tectia server (4.0.x seems fine), so we don't know what's up. We'll probably end up using Net::SFTP::Foreign.
Direct Responses: 3244 | Write a response
Posted on 2006-10-11 19:39:05-07 by odglark in response to 2327
Re:put() command hanging on files larger than 2kb
Hi ! I do not know where to change COPY_SIZE on our installation. We do not have an SFTP.pm module. Perl 5.8.7. We have an /etc/sshd/sshd_config. $OpenBSD: sshd_config,v 1.59 2002/09/25 11:17:16 markus Exp $ I cannot find COPY_SIZE anywhere on this box. Can anyone help ? Thanks.
Direct Responses: 6315 | Write a response
Posted on 2007-10-23 13:05:19-07 by csgonan in response to 3244
Re:put() command hanging on files larger than 2kb
I am experiencing a very similar thing. I have net::SFTP installed on a solaris 8 server, connecting to a linux server. The upload starts and I get the error below on the second packet. The file size is exactly the total size of those 2 numbers in the SSH2_FXP_WRITE statements (8192 + 5821). This is my ftp variable that is used through-out the script. Where do you change the COPY_SIZE parameter? Any ideas or suggestions would help. Just a note: I am using Net::xFTP to be able to use the ftp variables in the sftp module.
my %args = (user => $ftpuser, password => $ftppass, debug => 1); $ftp = Net::xFTP->new(Net::SFTP,$ftpip, %args) print "Uploading $_[0] to Server .......... " if (!$silent); $ftp->put($_[0]) or emailerr("Error: $0 could not upload $_[0] to Server."); print "Done\n\n" if (!$silent);
server: sftp: Sent message T:16 I:9 server: sftp: Sent message T:16 I:10 server: sftp: Sent message T:16 I:11 server: sftp: Sent SSH2_FXP_OPEN I:12 P:/opt/websites/ register.company.org/hccs/WC_file.dat server: sftp: Sent message SSH2_FXP_WRITE I:13 O:0 server: sftp: In write loop, got 8192 offset 0 server: sftp: Sent message SSH2_FXP_WRITE I:14 O:8192 server: sftp: In write loop, got 5821 offset 8192 server: sftp: Sent message T:10 I:15 Uploading WC_file.dat to Server .......... Error: ./ uploadSSH_company2.pl could not upload WC_file.dat to Server.
Direct Responses: Write a response