IBM Sterling SFTP Server Adapter 2.0 vs. SFTP Server Adapter 1.0: Performance Comparison

9 min readApr 6, 2022

--

The blog gives a detailed comparison of SFTP Server Adapter v2.0 and SFTP Server Adapter v1.0 with respect to the performance. The performance metrics at the end will give the end-users a clearer picture of deciding between 1.0 and 2.0.

In addition, the blog also details the supported RFC standards, Key Exchange Algorithms, Ciphers, and Macs supported by SFTP Server Adapter v2.0

Note: This blog is applicable for Sterling Integrator version 6.1.1 and onwards

What is SFTP?

SFTP (SSH File Transfer Protocol) is a secure file transfer protocol. It runs over the SSH protocol. It supports the full security and authentication functionality of SSH.

SFTP has pretty much-replaced legacy FTP as a file transfer protocol. It provides all the functionality offered by these protocols, but more securely and more reliably, with easier configuration.

SFTP also protects against password sniffing and man-in-the-middle attacks. It protects the integrity of the data using encryption and cryptographic hash functions and authenticates both the server and the user.

SFTP made better in IBM Sterling!!

The SFTP Server Adapter 2.0 is an out-of-the-box feature in IBM Sterling that is based on the Apache SSHD library.

The product was developed by us, keeping the below points in mind:

Faster and better performing than the commercial SFTP 1.0 legacy server.
Not dependent on external support teams for any production-related issues, Apache SSHD being an open-source library the development team can contribute to any issues or newer implementations

GitHub - apache/mina-sshd: Mirror of Apache MINA SSHD

Apache SSHD is a 100% pure java library to support the SSH protocols on both the client and server side. This library…

github.com

Follows RFC standards making it a more secure implementation for end users.
Supports compliant additional ciphers, MAC, and Key Exchange Algorithms.
Supports both inbound SSH/SFTP and SSH/SCP protocols.
Apache SSHD is an open-source library unlike SFTP 1.0 commercial server, so we were able to cut down additional licensing and support cost

SFTP 2.0 is faster and how!!

SFTP Server Adapter 2.0 is faster than the commercial SFTP 1.0 legacy server, and there are multiple reasons, which is being cited below

SFTP 2.0 is implemented using Apache SSHD, a 100% pure java library to support the SSH protocols on both the client and server-side, we leveraged Apache MINA, a scalable and high-performant asynchronous IO library (mostly NIO2) based on the Java NIO2 package which follows an asynchronous approach to non-blocking IO operations compared to Java NIO package for commercial SFTP 1.0 legacy server
Easy integration with IBM Perimeter Server and NIO2 IO channels.

The Apache SSHD code easily interacts with an abstraction of the network layer also known as IoServiceFactory. Currently, we have provided 2 such built-in implementations:

NIO2 (the default and recommended one) — Uses NIO2 asynchronous sockets, it’s the best performing compared to Perimeter and IBM docs also state that
Perimeter —Uses IBM’s perimeter server and NIO sockets for data transfer

SFTP Server Configuration Parameters

We have newly introduced a few configuration parameters for SFTP 2.0 server adapter, and each of them has a role to play in improving the overall throughput during the file transfer

IO Provider: This is a dropdown providing 2 options: NIO2 and Perimeter (by default NIO2 is selected)
IO Worker Threads: IO Worker Threads are the NIO2 worker threads that are responsible for handling raw incoming packets from NIO2 asynchronous sockets. Once a packet has been partially decoded and found to be relevant to the SFTP subsystem (in this case), it is passed along to an sshd-SftpSubsystem-thread-xx for further handling. This way, the NIO2 worker becomes available to handle the next incoming packet (which may be relevant for some other channel — e.g., SCP) and thus not depend on the time it takes to process the previous packet.

Does increasing the IO Worker Threads count to 9,10 will increase the throughput and why out of the box do we set the value to 8?

Depends on the traffic actually — given my previous explanation. Furthermore, at some time you may run into context switching overhead issues the more SFTP connections you have the more sshd-SftpSubsystem-thread-xx threads are running and thus increases memory, CPU, I/O, and context switch overhead. Therefore it is hard to suggest an “optimum” — as it depends on the kind of traffic — just good old trial and error, we set the value to 8, as internally in our software labs we did performance testing and found to get the maximum throughput hence we came up with this value.

Copy Buffer Size: Once a successful handshake is established between client and server and copying of data packets starts from point A to point B, it’s broken into chunks of data of size equal to the buffer size

Does increasing the Copy Buffer Size to multiples of 32678 increase the throughput?

Again it depends on the traffic, therefore it is hard to suggest an “optimum” value, we set the value to 32678 (in bytes), as we found to get the maximum throughput with this combination.

Supported RFC standards

Note: — The server side supports these signatures by default. The client-side requires specific initialization — see section 3.3 and also the above-mentioned hooks for RFC 8308.

Note: the server-side support these keys by default. The client-side requires specific initialization

OpenSSH public-key certificate authentication system for use by SSH
SFTP version 3–6 + extensions
supported - DRAFT 05 - section 4.4
supported2 - DRAFT 13 section 5.4
versions - DRAFT 09 Section 4.6
vendor-id - DRAFT 09 - section 4.4
acl-supported - DRAFT 11 - section 5.4
newline - DRAFT 09 Section 4.3
md5-hash, md5-hash-handle - DRAFT 09 - section 9.1.1
check-file-handle, check-file-name - DRAFT 09 - section 9.1.2
copy-file, copy-data - DRAFT 00 - sections 6, 7
space-available - DRAFT 09 - section 9.2
filename-charset, filename-translation-control - DRAFT 13 - section 6 - only client side
Several OpenSSH SFTP extensions

Supported Ciphers, Macs & Key Exchange Algorithms

Ciphers: aes128cbc, aes128ctr, aes192cbc, aes192ctr, aes256cbc, aes256ctr, arcfour128, arcfour256, blowfishcbc, tripledescbc, aes128-gcm@openssh.com, aes256-gcm@openssh.com, chacha20-poly1305@openssh.com
Digests: md5, sha1, sha224, sha256, sha384, sha512
Macs: hmacmd5, hmacmd596, hmacsha1, hmacsha196, hmacsha256, hmacsha512, hmac-sha2–256-etm@openssh.com , hmac-sha2–512-etm@openssh.com, hmac-sha1-etm@openssh.com
Key exchange: diffie-hellman-group1-sha1, diffie-hellman-group-exchange-sha256, diffie-hellman-group14-sha1, diffie-hellman-group14-sha256 , diffie-hellman-group15-sha512, diffie-hellman-group16-sha512, diffie-hellman-group17-sha512, diffie-hellman-group18-sha512 , ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, curve25519-sha256, curve25519-sha256@libssh.org, curve448-sha512
Compressions: none, ZLIB, zlib@openssh.com
Signatures/Keys: ssh-dss, ssh-rsa, rsa-sha2–256, rsa-sha2–512, nistp256, nistp384, nistp521 , ssh-ed25519 (requires eddsa optional module), sk-ecdsa-sha2-nistp256@openssh.com, sk-ssh-ed25519@openssh.com , ssh-rsa-cert-v01@openssh.com, ssh-dss-cert-v01@openssh.com, ssh-ed25519-cert-v01@openssh.com , ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521-cert-v01@openssh.com

The following settings have been deprecated and are no longer included in the default setup:

While it refers to Kerberos, it mentions weaknesses in DES as well.

SHA-1 based key exchanges and signatures
MD5-based and truncated HMAC algorithms

Note: Per the latest recommendations the default client/server setup includes only the security settings that are currently considered safe to use. Users who wish to include the unsafe settings must do so explicitly from the customer.overrides as mentioned in the IBM knowledge center docs.

Special notice: ssh-RSA was left in as part of the default setup since there are still a lot of systems/users using it. However, in a future version, it will be removed from the default. We therefore strongly encourage users to migrate to other keys (e.g. ECDSA, ED25519) as soon as possible.

How fast is “fast” SFTP Server Adapter 2.0?

Load Performance Test Setup

Single Node Sterling Integrator server with dedicated 4 cores CPU, and 8 Gb of RAM, Linux RHEL 7.5
DB2 Server 11.1.1 dedicated 8 cores CPU, and 16 Gb of RAM, Linux RHEL 7.5
Performance Test Data Generator VM with dedicated 4 cores CPU, and 8 Gb of RAM, Linux RHEL 7.5
External Perimeter Server with dedicated 4 cores CPU, and 8 Gb of RAM, Linux RHEL 7.5

SFTP 1.0 vs SFTP 2.0 (Payload Repo: Mailbox, Doc-Storage: Data Base)

Performance Metrics for file sizes 1.5kb, 100MB, 2GB (SFTP PUT)

SFTP 1.0 vs SFTP 2.0 (Payload Repo: Mailbox, Doc-Storage: File System)

SFTP 1.0 vs SFTP 2.0 (Payload Repo: File System, Doc-Storage: File System)

SFTP 1.0 vs SFTP 2.0 (High Availability Global Mailbox)

We carried out performance testing with different combinations as shown in the bars above, and the benchmarks we got for SFTP 2.0 excelled in every aspect compared to SFTP 1.0 commercial server.

The average Sterling Integrator VM’s CPU utilization and the DB2 VM’s CPU utilization were well within the threshold range for all combinations of tests performed for both SFTP Server Adapter 1.0 & 2.0

Note: Above results are from internal performance tests in a lab environment, the results may vary with the hardware/network configuration and on different factors.

Conclusion

End-users should slowly start migrating from SFTP 1.0 commercial server to SFTP 2.0 Apache SSHD based implementation, considering the performance throughput and wide range of supported RFC standards combined with compliant supported Ciphers, Macs & Key Exchange Algorithms

If you found this blog helpful please leave a clap and share with a larger community

Cheers & Happy Reading!