Using External FTP for ESS File Transfers

Overview

In a default EchoSystem installation, devices exchange capture data with the EchoSystem Server (ESS) via an internal secure file transfer protocol (SFTP) server that listens on port 8022. While the standard method may be comfortable for most circumstances, SFTP carries a substantial amount of compression and encryption overhead that can cause transfer bottlenecks on large-scale installations.

The ESS provides the option to use an external FTP server for handling these file transfers rather than its internal SFTP facility. This has two advantages:

• The transfer speeds available are much higher
• A dedicated service can be used for back-end content transfers, consolidating back-end file transfer resources away from the ESS

Configure the FTP Server

Although any FTP server can be used, this article will describe the procedure for setting up the FTP server built in to the Internet Information Services (IIS) suite for Windows Server 2003, which is recommended for use alongside the ESS especially in situations where simple transfer speed is desired.

1. If the IIS FTP server is not installed, from the Windows, Control Panel, select Add or Remove Programs > Windows Components Wizard. You may need a Windows installation CD if these components are not available. At a minimum, check the following tree of components:
   • Application Server
   • Internet Information Services (IIS)
   • File Transfer Protocol (FTP) Service; Internet Information Services Manager
2. In the Windows Control Panel, select Administrative Tools > Internet Information Services (IIS) Manager. Using the navigation pane on the left, expand the local computer entry and FTP Sites folder to expose the list of sites. You may use the default FTP site, or create a new one from scratch.
3. Right-click your FTP site and click Properties. The following tabs contain configuration options of interest:
   • FTP Site: This contains basic connection parameters for the FTP server.
     If your server has multiple network interfaces, you can use the "IP address" field to specify which one the FTP will listen on, allowing you to segregate internal and external network traffic.
     As a security measure, you may wish to force the FTP server to use a non-default TCP port as well, or limit the number of concurrent connections.
   • Security Accounts: Anonymous access is enabled or disabled here. By default, it is enabled; we strongly recommend against this. Clear the option, read the warning that Windows produces, and select Yes to continue. We will set up security in the next step.
   • Home Directory: Best practice standards recommend that the path given here should be part of the ESS content upload file system hierarchy. An ideal selection would be something like D:\echo360\upload\ext (as opposed to "int," which is the default). If the FTP server is remote to the ESS, you can also specify a shared location on the ESS server where you would like the FTP server to look for its files. In either case, be sure to enable both read and write permissions.
   • Directory Security: At a Network Administrator's discretion, you can restrict access to this FTP service to specific IP address ranges. Be sure, however, that you do not inadvertently lock out actual devices on your EchoSystem network from reaching the service to upload their content.
4. In the Windows Control Panel, select Administrative Tools > Computer Management. Using the navigation pane on the left, select System Tools\Local Users and Groups\Users to see a list of all users on the local system.
   • You will need to either create or repurpose a user that will control access to the FTP service from your appliances. Make a note of the user name and password that you choose.
   • In the user properties, be sure that "User must change password at next logon" is not enabled, and enable at least "Password never expires."
5. In Windows Explorer, navigate to the folder that you specified for the home directory of the FTP server in step 3 (e.g., D:\echo360\upload\ext).
   • Right click on the folder itself and click Properties.
   • Click the Security tab.
   • Click Add under the name list.
   • Enter the name of the user you selected in Step 4, and click OK.
   • In the "Group or user names" list, click the user you just added.
   • In the permissions list, give Allow permissions for at least the Modify action. We recommend giving full control.
   • Click the Advanced button.
   • Under the Permissions tab, enable both check-boxes for inheriting and replacing permissions to all child objects.
   • Click OK to both dialog boxes to save your changes and close.
6. As an optional test of the new configuration, open a command prompt window.
   • Execute the following command: ftp localhost (or the IP address you configured).
   • When prompted, enter the user name and password for the account you selected in step 4.
   • Execute the following commands: ls, mkdir test, and rmdir test.
   • If all of the above succeeded, you have confirmed that the FTP server is online and that the user account you selected has all the necessary permissions.

A Word on Firewalls

In testing, we have found that when using IIS as an FTP server, a firewall may not always automatically open the ports necessary for successful active-mode FTP connections; even when manually configuring exceptions for the FTP control and data ports (21 and 20, respectively), FTP in active mode uses other random ports for its data connections. However, if you must use a firewall, adding an application-based exception (the software must be able to support this; Windows Firewall does, for instance) rather than opening port ranges is the most effective method of allowing access. If you are using IIS, the application to clear with the firewall is called inetinfo.exe for Windows Internet Information Services. It may also be necessary under such circumstances to set a long session timeout value to prevent the control port from being closed during long file transfers.

Configure the ESS

1. In the ESS administration interface, select System > System Settings.
2. Click the Edit button at the bottom of the page.
3. Under the Intake Settings heading, update the following configuration items:
   
   • FTP Server: External
   • FTP Protocol: FTP
   • FTP User Name: As configured on the FTP server. See Configure the FTP Server.
   • FTP Password: As configured on the FTP server. See Configure the FTP Server.
   • FTP Folder: The FTP home folder, as configured on the FTP server. See Configure the FTP Server.
   • FTP Host: The FQDN of the FTP server for the address on which it is listening.
   • FTP Port: The TCP port, as configured on the FTP server. See Configure the FTP Server.
   • FTP Path: empty (unless you are using one common FTP server with multiple subdirectories for multiple applications)
   • Default Processor Path to FTP Folder: empty (unless you have followed the procedure in the article "Using UNC Paths for Processor Data Transfer")

4. Click the Save button at the bottom of the page.

   This restarts all of your capture appliances.

5. As an optional test of the new configuration, click the green Test Settings button, which is under the Intake Settings heading. You should see a success message.